April 6, 2018

NetBSD cross-compiling tutorial

Lets build NetBSD starting from nothing!

Thanks the to the great build.sh system I’ve already gushed about, cross-compiling NetBSD from any other POSIX’y host is cake.

However, the docs are slightly out of date, and everything needed is scattered across a few pages which may not be obvious for new-comers. For example, grabbing sets’ which is parlance for gzips of source code, is on a different page, and the kernel compile page is missing some new flags.

Similarly, man pages are great if you already have an idea of what you’re looking for. But I’ll save that diatribe for another day.

There is not much absolute-beginner orientated task-orientated documentation, too. Certainly the BSDs are niche and demand expertise but there is no reason to unnecessarily burden the learning curve for passerby’s. This increases adoption!

I’ve also learned a few other things since writing my previous compiling notes. Particularly, the drivers in use when NetBSD is a virtualbox guest cause noticeably slower disk performance at the moment. I highly recommend to build sources from anything but a VM

So with that said, this will be a tutorial for building NetBSD from a GNU/Linux host. The instructions should be very similar for Mac and other BSDs.

Also, this will cover the 7.1.2 release, not current (WIP) sources To recap:

  • We will build NetBSD 7.1.2 from sources.
  • This is written from a GNU/Linux host
  • the target we will cross compile for will be amd64

Step 1: grab source sets

Navigate your big nasty web browser to the source sets directory here:

https://ftp.netbsd.org/pub/NetBSD/NetBSD-7.1.2/source/sets/

Grab all the tgz, you can skip xsrc.tgz unless you want to compile X windows.

I recommend saving all these in a tidy folder like ~/netbsd_build/7.1.2

Next, untar em all, eg manually:

$ pwd
~/netbsd_build/7.1.2
$ tar xvzf gnusrc.tgz
$ tar xvzf sharesrc.tgz
$ tar xvzf src.tgz
$ tar xvzf syssrc.tgz
$ tar xvzf xsrc.tgz  #(again, only if you want to compile X)

if this pissed you off: write a shell for with wildcard expansion to do it in a oneliner!

Once that’s all done you should end up with something like this:

dexter@slag:~/netbsd_build/7.1.2$ tree -L 2
.
├── gnusrc.tgz
├── sharesrc.tgz
├── src.tgz
├── syssrc.tgz
└── usr
    └── src

2 directories, 4 files
dexter@slag:~/netbsd_build/7.1.2$

the funky sub usr/src dir is ok. Navigate into that directory for the rest of the instructions $ cd usr/src

Next, we need to make a directory to output objects during build, if we dont do this we will get errors. It doesnt really matter where, just make the directory in the usr/src subdir we are already in:

$ mkdir obj

We’re ready to start compiling stuff!

Building toolchain

The first step to build NetBSD is to build the compiler it expects to use. Long story short, dont expect, or try to use your host compiler. It’s much easier (albeit it can be slow on older machines) to use the provided gcc sources.

To do so, fire off build.sh like so:

$ ./build.sh -U -m amd64 -O obj -j17 tools

A quick explaination of the flags:

  • -U: we are not building as root. this is needed
  • -m amd64: build for amd64 machine
  • -O obj: stick our build objs in this obj’ folder we made above
  • -j17: use 17 threads. A fast-dirty value is number of cores/threads machine has. Its complex. Personally I add a few +/- for I/O overhead.
  • tools: lastly, what we want to make. Tools is a prerequisite to building the kernel and userspace

It will grind a while, especially if you’re on older hardware but you should eventually get an output like this

===> Tools built to /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
===> build.sh ended:      Sun Apr  8 00:15:25 MDT 2018
===> Summary of results:
     build.sh command:    ./build.sh -U -m amd64 -O obj -j17 tools
     build.sh started:    Sun Apr  8 00:12:01 MDT 2018
     NetBSD version:      7.1.2
     MACHINE:             amd64
     MACHINE_ARCH:        x86_64
     Build platform:      Linux 4.13.0-38-generic x86_64
     HOST_SH:             /bin/sh
     MAKECONF file:       /etc/mk.conf (File not found)
     TOOLDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
     DESTDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/destdir.amd64
     RELEASEDIR path:     /home/dexter/netbsd_build/7.1.2/usr/src/obj/releasedir
     Updated makewrapper: /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64/bin/nbmake-amd64
     Tools built to /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
     build.sh ended:      Sun Apr  8 00:15:25 MDT 2018
===> .

Now we’re ready to move on

Building a kernel redux

To kick out the kernel, we follow suite with a slightly different command:

$ ./build.sh -U -m amd64 -O obj -j17 kernel=GENERIC

The new option kernel=GENERIC denotes which amd64 kernel config file we want to build. These configs live in the subdir sys/arch/<arch>/conf/ so in our case sys/arch/amd64/conf/GENERIC is the file. Ideally, you’d copy and modify this file to your desire. But for this example lets compile the stock GENERIC. While its out of scope for this scenic tour, pique your interest with the makeoptions COPTS` line.

Same looking output:

--- netbsd ---
#      link  GENERIC/netbsd
/home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64/bin/x86_64--netbsd-ld -Map netbsd.map --cref -T /home/dexter/netbsd_build/7.1.2/usr/src/sys/arch/amd64/conf/kern.ldscript -Ttext 0xffffffff80100000 -e start -z max-page-size=0x100000 -X -o netbsd ${SYSTEM_OBJ} ${EXTRA_OBJ} vers.o
NetBSD 7.1.2 (GENERIC) #1: Sun Apr  8 00:33:55 MDT 2018
   text    data     bss     dec     hex filename
14227197     654556  593920 15475673     ec23d9 netbsd
===> Kernels built from GENERIC:
  /home/dexter/netbsd_build/7.1.2/usr/src/obj/sys/arch/amd64/compile/GENERIC/netbsd
===> build.sh ended:      Sun Apr  8 00:33:56 MDT 2018
===> Summary of results:
     build.sh command:    ./build.sh -U -m amd64 -O obj -j17 kernel=GENERIC
     build.sh started:    Sun Apr  8 00:33:12 MDT 2018
     NetBSD version:      7.1.2
     MACHINE:             amd64
     MACHINE_ARCH:        x86_64
     Build platform:      Linux 4.13.0-38-generic x86_64
     HOST_SH:             /bin/sh
     MAKECONF file:       /etc/mk.conf (File not found)
     TOOLDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
     DESTDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/destdir.amd64
     RELEASEDIR path:     /home/dexter/netbsd_build/7.1.2/usr/src/obj/releasedir
     Updated makewrapper: /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64/bin/nbmake-amd64
     Building kernel without building new tools
     Building kernel:     GENERIC
     Build directory:     /home/dexter/netbsd_build/7.1.2/usr/src/obj/sys/arch/amd64/compile/GENERIC
     Kernels built from GENERIC:
      /home/dexter/netbsd_build/7.1.2/usr/src/obj/sys/arch/amd64/compile/GENERIC/netbsd
     build.sh ended:      Sun Apr  8 00:33:56 MDT 2018
==> .

The actual netbsd kernel binary will be sitting in our obj/sys/arch/amd64/compile/GENERIC/ dir. To actually install it you’d just copy it over your existing netbsd kernel.

dexter@slag:~/netbsd_build/7.1.2/usr/src$ file obj/sys/arch/amd64/compile/GENERIC/netbsd
obj/sys/arch/amd64/compile/GENERIC/netbsd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for NetBSD 7.1.2, not stripped

Userland

Next is userland. no frills here, you will see the pattern

$ ./build.sh -U -m amd64 -O obj -j17 release

The only new option is just release. If you want to build X also pass in -x

Further reading

At this point I’ve just overlapped the official guide on building, which has all the details. Check it out once you are ready to make modifications and actually install your new bins!

https://www.netbsd.org/docs/guide/en/chap-build.html


Previous post
Reading linux mouse input with golang I think mouse input in linux is generally awful. It always feels terrible in any window manager and I have begun digging into the whys. It seems
Next post
Windows 10 Wifi tweaks Wifi has never been great on windows 10, especially after the latest ‘fall creator update’. One of my wifi cards no longer works at all, despite