In 2003, I got my first Mac and I heard that MacOS X ran on top of Mach, which is a microkernel. This started my whole “What’s a microkernel” thing, which has lived rent-free in my brain ever since. In 2016, I saw that QEMU had made an advent calendar with different computer architectures, which included a microkernel on day 11 called seL4. There’s a project called Genode, which is an operating system framework that can be run on top of various microkernels. I played around with Genode for a bit, but it didn’t spark joy, as it isn’t similar to other desktop experiences I’m used to. I wondered if someone could build a more familiar system on seL4, but I didn’t have the skillset to be that person. Later, a PH.D sort of person did just that.

What is NeptuneOS?

An operating system that I’ve been following for a while is NeptuneOS. The goal of this operating system is to implement a Windows NT ntdll.dll on top of the seL4 microkernel. MacOS and iOS both run on Mach, which was originally a pure microkernel, but traditional system calls were added later for performance reasons. The L4 microkernel is a next-gen successor to the Mach microkernel which solves a lot of the performance problems inherent in the previous generation. I don’t completely understand the differences, but Wikipedia has some explanations. SeL4 is a variant of the L4 microkernel which has been proven correct, which means that it won’t crash or deadlock on non-faulty hardware, assuming the specification of the OS is correct.

When NeptuneOS is launched, the seL4 kernel is loaded. Next, a file called ntos is loaded, which contains the “root task”, which owns and manages the rest of the tasks. Once the system is booted, applications are started. Each application links to a library called ntdll.dll. On Linux, applications communicate with the kernel via system calls, which tell the operating system what to do. On Windows, ntdll.dll exposes an API which doesn’t change from version to version of Windows. This is nicer, IMO, because Microsoft can redesign the system calls without breaking the API of the operating system. Neptune currently shows a command prompt interface, but at some point a Windows-like desktop interface will be possible.

It occurs to me that you could build something like DOS on seL4, not taking advantage of the microkernel’s affordances for subtasks. It’s not clear to me when you are meant to spawn a subtask and communicate via messaging or shared memory and when you should use a function call in the same task. In iOS 7, the software keyboard transitioned from an-process library to a separate tasks, which caused stuttering during typing on some older phones. I’m sure there are dozens of examples with similar tradeoffs inside the implementation of ntdll.dll. In the next few weeks, I plan to investigate this. Maybe I’ll find code which has been made into separate tasks which could have been function calls.

How do we build it?

The remainder of this post will cover how to set up a Debian 12 (bookworm) system to build NeptuneOS from the sources on GitHub.

First, we install all the packages we’ll need.

apt install make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev curl git \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev \
libffi-dev liblzma-dev cmake jq ninja-build clang libxml2-utils \
cpio grub-common xorriso

Install pyenv, which will help manage multiple versions of Python.

curl -fsSL https://pyenv.run | bash

List the versions of Python which are available.

pyenv install --list

Install Python 3.10.0. You probably don’t need to use exactly the same version if you don’t want to.

pyenv install 3.10.0

Set Python 3.10.0 for the local session. This will take a long time, as it’s compiling Python from source. You can also use global to affect all logins.

pyenv local 3.10.0

Create a virtual environment for the NeptuneOS build and activate it.

pyenv virtualenv 3.10.0 neptune
pyenv activate neptune

Install all the Python dependencies.

pip install jinja2
pip install future
pip install ply
pip install setuptools
pip install six
pip install lxml
pip install pyyaml

Check out the repository.

git clone --recurse-submodules https://github.com/cl91/NeptuneOS.git
cd NeptuneOS

Check out the latest release.

git checkout tags/v0.2.0002

On Debian, a lot of the LLVM utilities are suffixed with their major version, 14. I decided to fix this by making symlinks in /opt/local/bin for each tool.

mkdir -p /opt/local/bin
cd /opt/local/bin
ln -s `which llvm-ar-14` llvm-ar
ln -s `which lld-link-14` lld-link
ln -s `which llvm-objcopy-14` llvm-objcopy
ln -s `which ld.lld-14` ld.lld
ln -s `which llvm-strip-14` llvm-strip
cd
cd NeptuneOS

Once everything is done, the build should succeed.

RC=`which llvm-rc-14` ./build.sh amd64 release

You’ll see this message when it’s finished building.

####################################
		 Build successful!
####################################

Once that’s done, you need to build a CD image with the following command

./mkiso.sh amd64 release