Introduction to XNU compiling
From version to version, I always love to play around with the kernel. And it has always been a great lack in guides and documentation on how to build Mac OSX's kernel, XNU. For those of you that already have tried compiling XNU for Mac OSX 10.12 (Sierra), you probably noticed that earlier build guides like ssen's blog - Building xnu for OS X 10.11 El Capitan don't work anymore. However, many thanks to ssen to put in time to write a guide.
The problem is that Apple introduced something named Circular dependency with the libdispatch library and the kernel headers. So the order of the build process just got really important.
If you tried compiling yourself, the header
firehose_buffer_private.h might be quite familiar with some errors you hit on the road to success. This is a header from the libdispatch library which forces you to build libdispatch first. Copying the header won't really help, since you need a static library as well (
libfirehose_kernel.a). Just to make it a little more hassle, the libdispatch source won't even be close to compiling before the XNU kernel headers are in place.
Get the sources
You need to download in total five sources, including XNU itself. This is version numbers for Mac OSX 10.12.4, which you find at Apple downloads
curl -O https://opensource.apple.com/tarballs/dtrace/dtrace-209.50.12.tar.gz && \ curl -O https://opensource.apple.com/tarballs/AvailabilityVersions/AvailabilityVersions-26.50.4.tar.gz && \ curl -O https://opensource.apple.com/tarballs/libdispatch/libdispatch-703.50.37.tar.gz && \ curl -O https://opensource.apple.com/tarballs/libplatform/libplatform-126.50.8.tar.gz # To use the XNU package do curl -O https://opensource.apple.com/tarballs/xnu/xnu-3789.51.2.tar.gz export XNUPATH=$(pwd)/xnu-3789.51.2 # (Note that using github is experimental, if you break anything, it's not my fault.) # To use github do git clone https://github.com/opensource-apple/xnu.git export XNUPATH=$(pwd)/xnu
To extract all files at once, you can run
for file in *.tar.gz; do tar -zxf $file; done && rm -f *.tar.gz. The guide will assume you managed to extract the tar files from now on.
First of all, make sure you got Xcode 8, try to have the last version to encrease the chance it will build without too many errors. Next up you need to make sure you have build tools from command line, you can check this by running
clang -v. If the output is something like
zsh: command not found: clang then you need to install the command line tools. If you followed my instructions and got Xcode 8, this can be easily installed with the command
Next up is to ensure the SDK is correctly set. A good way to check if your SDK path is set correctly is to run
xcrun -sdk macosx -show-sdk-path and check the output.
By the way, if you strugle with this, kernel compiling might be dangerous for your data, since this can break your OS install. At least, test it in a virtual machine.
Dtrace is a fantastic debugging tool which you should really lookup if you haven't yet. It's worth your time. I also wondered why FreeBSD, Solaris and Darwin (OSX) had dtrace in official source but not Linux, and it seems to be because of some stupid licensing issues.
cd dtrace-209.50.12 mkdir -p obj sym dst xcodebuild install -target ctfconvert -target ctfdump -target ctfmerge ARCHS="x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst sudo ditto $PWD/dst/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
This is just a perl script that's required to build the kernel.
cd ../AvailabilityVersions-26.50.4/ mkdir -p dst make install SRCROOT=$PWD DSTROOT=$PWD/dst sudo ditto $PWD/dst/usr/local `xcrun -sdk macosx -show-sdk-path`/usr/local
XNU - Header install
You will need the XNU headers installed to the SDK directory to be able to build the libdispatch library.
cd $XNUPATH mkdir -p BUILD.hdrs/obj BUILD.hdrs/sym BUILD.hdrs/dst make installhdrs SDKROOT=macosx ARCH_CONFIGS=X86_64 SRCROOT=$PWD OBJROOT=$PWD/BUILD.hdrs/obj SYMROOT=$PWD/BUILD.hdrs/sym DSTROOT=$PWD/BUILD.hdrs/dst sudo xcodebuild installhdrs -project libsyscall/Libsyscall.xcodeproj -sdk macosx ARCHS='x86_64 i386' SRCROOT=$PWD/libsyscall OBJROOT=$PWD/BUILD.hdrs/obj SYMROOT=$PWD/BUILD.hdrs/sym DSTROOT=$PWD/BUILD.hdrs/dst sudo ditto BUILD.hdrs/dst `xcrun -sdk macosx -show-sdk-path`
This is also required to build libdispatch. It just need it's source to be available, so you don't need to build anything here.
cd ../libplatform-126.50.8 sudo ditto $PWD/include `xcrun -sdk macosx -show-sdk-path`/usr/local/include
At last you can build
libfirehose_kernel.a from the libdispatch library. You won't build the whole library since you're only out after
libfirehose_kernel.a and headers.
cd ../libdispatch-703.50.37 mkdir -p BUILD.hdrs/obj BUILD.hdrs/sym BUILD.hdrs/dst sudo xcodebuild install -project libdispatch.xcodeproj -target libfirehose_kernel -sdk macosx ARCHS='x86_64 i386' SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst sudo ditto $PWD/dst/usr/local `xcrun -sdk macosx -show-sdk-path`/usr/local
Finally we can build the XNU kernel itself.
cd $XNUPATH # You can choose between RELEASE, DEVELOPMENT or DEBUG, or all. make SDKROOT=macosx ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="RELEASE DEVELOPMENT DEBUG"
Depending on which kernel config you choose, your kernel should be found at either locations.
# If you built DEBUG file BUILD/obj/DEBUG_X86_64/kernel.debug BUILD/obj/DEBUG_X86_64/kernel.debug: Mach-O 64-bit executable x86_64 # If you built DEVELOPMENT file BUILD/obj/DEVELOPMENT_X86_64/kernel.development BUILD/obj/DEVELOPMENT_X86_64/kernel.development: Mach-O 64-bit executable x86_64 # If you built RELEASE file BUILD/obj/RELEASE_X86_64/kernel BUILD/obj/RELEASE_X86_64/kernel: Mach-O 64-bit executable x86_64
And there you go, fresh XNU builds. However I choose to end it here, if you wonder how to actually switch it with the pre shipped one, you'll have to do something like this.
And just to say it one more time, if you do this and you experience data loss, crashes, or your mac turns into a badger and tries to kill you, or any other unexpected events, it's not my fault.
sudo cp BUILD/obj/RELEASE_X86_64/kernel /System/Library/Kernels/ sudo kextcache -invalidate /
Enjoy your custom XNU!