April 12, 2018

Windows 10 Wifi tweaks

Wifi has never been great on windows 10, especially after the latest fall creator update’. One of my wifi cards no longer works at all, despite working on this same exact version of windows before a clean install!

Anyway, I’ve found a few ways that seem to make the wifi slightly more consistent, so here they are

Step zero: Wifi adapter name

We will need the short name of the wifi adapter and the easiest way is with **netsh*.

To list wlan settings run the following in a cmd or powershell (Win+X):

netsh wlan show settings

PS C:\Users\Dexter> netsh wlan show settings

Wireless LAN settings
---------------------
    Show blocked networks in visible network list: No

    Only use GP profiles on GP-configured networks: No

    Hosted network mode allowed in WLAN service: Yes

    Allow shared user credentials for network authentication: Yes

    Block period: Not Configured.

    Auto configuration logic is disabled on interface "Wi-Fi 3"
    MAC randomization not available on interface Wi-Fi 3

Look for the quoted part after interface, in this case my adapter name is Wi-Fi 3. Alternatively, you can go to

Control Panel -> Network Connections

And that will have the same name. We’ll need it for the rest of the steps.

First: Wifi autoconfig on/off bats

Save these two in batch files so you can easily run them as needed:

WIFI_AUTO_ON.bat netsh wlan set autoconfig enabled=yes interface="Wi-Fi 3"

WIFI_AUTO_OFF.bat netsh wlan set autoconfig enabled=no interface="Wi-Fi 3"

Replace Wi-Fi 3” with the adapter name from the first step!

Now you can right click on WIFI_AUTO_OFF.BAT and run as admin before playing an online game. For me, this seems to help with the persistent, roughly 1 minute apart ping spikes.

Second: disable tuning heuristics

This one is a bit harder to quantify but in my highly non-scientific testing, my ping appeared to be more stable overall after turning off auto tuning. Note this is machine wide and seems to be related to TCP window sizes. I’ve encounted no side effects but..

netsh int tcp set heuristics disabled
netsh int tcp set global autotuninglevel=disabled
netsh int tcp set global rss=enabled

Disable dat heuristic tuning

Hopefully this helps others eek out a remotely usable wifi experience on windows.

April 6, 2018

NetBSD cross-compiling tutorial

Lets build NetBSD starting from nothing!

Thanks the to the great build.sh system I’ve already gushed about, cross-compiling NetBSD from any other POSIX’y host is cake.

However, the docs are slightly out of date, and everything needed is scattered across a few pages which may not be obvious for new-comers. For example, grabbing sets’ which is parlance for gzips of source code, is on a different page, and the kernel compile page is missing some new flags.

Similarly, man pages are great if you already have an idea of what you’re looking for. But I’ll save that diatribe for another day.

There is not much absolute-beginner orientated task-orientated documentation, too. Certainly the BSDs are niche and demand expertise but there is no reason to unnecessarily burden the learning curve for passerby’s. This increases adoption!

I’ve also learned a few other things since writing my previous compiling notes. Particularly, the drivers in use when NetBSD is a virtualbox guest cause noticeably slower disk performance at the moment. I highly recommend to build sources from anything but a VM

So with that said, this will be a tutorial for building NetBSD from a GNU/Linux host. The instructions should be very similar for Mac and other BSDs.

Also, this will cover the 7.1.2 release, not current (WIP) sources To recap:

  • We will build NetBSD 7.1.2 from sources.
  • This is written from a GNU/Linux host
  • the target we will cross compile for will be amd64

Step 1: grab source sets

Navigate your big nasty web browser to the source sets directory here:

https://ftp.netbsd.org/pub/NetBSD/NetBSD-7.1.2/source/sets/

Grab all the tgz, you can skip xsrc.tgz unless you want to compile X windows.

I recommend saving all these in a tidy folder like ~/netbsd_build/7.1.2

Next, untar em all, eg manually:

$ pwd
~/netbsd_build/7.1.2
$ tar xvzf gnusrc.tgz
$ tar xvzf sharesrc.tgz
$ tar xvzf src.tgz
$ tar xvzf syssrc.tgz
$ tar xvzf xsrc.tgz  #(again, only if you want to compile X)

if this pissed you off: write a shell for with wildcard expansion to do it in a oneliner!

Once that’s all done you should end up with something like this:

dexter@slag:~/netbsd_build/7.1.2$ tree -L 2
.
├── gnusrc.tgz
├── sharesrc.tgz
├── src.tgz
├── syssrc.tgz
└── usr
    └── src

2 directories, 4 files
dexter@slag:~/netbsd_build/7.1.2$

the funky sub usr/src dir is ok. Navigate into that directory for the rest of the instructions $ cd usr/src

Next, we need to make a directory to output objects during build, if we dont do this we will get errors. It doesnt really matter where, just make the directory in the usr/src subdir we are already in:

$ mkdir obj

We’re ready to start compiling stuff!

Building toolchain

The first step to build NetBSD is to build the compiler it expects to use. Long story short, dont expect, or try to use your host compiler. It’s much easier (albeit it can be slow on older machines) to use the provided gcc sources.

To do so, fire off build.sh like so:

$ ./build.sh -U -m amd64 -O obj -j17 tools

A quick explaination of the flags:

  • -U: we are not building as root. this is needed
  • -m amd64: build for amd64 machine
  • -O obj: stick our build objs in this obj’ folder we made above
  • -j17: use 17 threads. A fast-dirty value is number of cores/threads machine has. Its complex. Personally I add a few +/- for I/O overhead.
  • tools: lastly, what we want to make. Tools is a prerequisite to building the kernel and userspace

It will grind a while, especially if you’re on older hardware but you should eventually get an output like this

===> Tools built to /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
===> build.sh ended:      Sun Apr  8 00:15:25 MDT 2018
===> Summary of results:
     build.sh command:    ./build.sh -U -m amd64 -O obj -j17 tools
     build.sh started:    Sun Apr  8 00:12:01 MDT 2018
     NetBSD version:      7.1.2
     MACHINE:             amd64
     MACHINE_ARCH:        x86_64
     Build platform:      Linux 4.13.0-38-generic x86_64
     HOST_SH:             /bin/sh
     MAKECONF file:       /etc/mk.conf (File not found)
     TOOLDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
     DESTDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/destdir.amd64
     RELEASEDIR path:     /home/dexter/netbsd_build/7.1.2/usr/src/obj/releasedir
     Updated makewrapper: /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64/bin/nbmake-amd64
     Tools built to /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
     build.sh ended:      Sun Apr  8 00:15:25 MDT 2018
===> .

Now we’re ready to move on

Building a kernel redux

To kick out the kernel, we follow suite with a slightly different command:

$ ./build.sh -U -m amd64 -O obj -j17 kernel=GENERIC

The new option kernel=GENERIC denotes which amd64 kernel config file we want to build. These configs live in the subdir sys/arch/<arch>/conf/ so in our case sys/arch/amd64/conf/GENERIC is the file. Ideally, you’d copy and modify this file to your desire. But for this example lets compile the stock GENERIC. While its out of scope for this scenic tour, pique your interest with the makeoptions COPTS` line.

Same looking output:

--- netbsd ---
#      link  GENERIC/netbsd
/home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64/bin/x86_64--netbsd-ld -Map netbsd.map --cref -T /home/dexter/netbsd_build/7.1.2/usr/src/sys/arch/amd64/conf/kern.ldscript -Ttext 0xffffffff80100000 -e start -z max-page-size=0x100000 -X -o netbsd ${SYSTEM_OBJ} ${EXTRA_OBJ} vers.o
NetBSD 7.1.2 (GENERIC) #1: Sun Apr  8 00:33:55 MDT 2018
   text    data     bss     dec     hex filename
14227197     654556  593920 15475673     ec23d9 netbsd
===> Kernels built from GENERIC:
  /home/dexter/netbsd_build/7.1.2/usr/src/obj/sys/arch/amd64/compile/GENERIC/netbsd
===> build.sh ended:      Sun Apr  8 00:33:56 MDT 2018
===> Summary of results:
     build.sh command:    ./build.sh -U -m amd64 -O obj -j17 kernel=GENERIC
     build.sh started:    Sun Apr  8 00:33:12 MDT 2018
     NetBSD version:      7.1.2
     MACHINE:             amd64
     MACHINE_ARCH:        x86_64
     Build platform:      Linux 4.13.0-38-generic x86_64
     HOST_SH:             /bin/sh
     MAKECONF file:       /etc/mk.conf (File not found)
     TOOLDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64
     DESTDIR path:        /home/dexter/netbsd_build/7.1.2/usr/src/obj/destdir.amd64
     RELEASEDIR path:     /home/dexter/netbsd_build/7.1.2/usr/src/obj/releasedir
     Updated makewrapper: /home/dexter/netbsd_build/7.1.2/usr/src/obj/tooldir.Linux-4.13.0-38-generic-x86_64/bin/nbmake-amd64
     Building kernel without building new tools
     Building kernel:     GENERIC
     Build directory:     /home/dexter/netbsd_build/7.1.2/usr/src/obj/sys/arch/amd64/compile/GENERIC
     Kernels built from GENERIC:
      /home/dexter/netbsd_build/7.1.2/usr/src/obj/sys/arch/amd64/compile/GENERIC/netbsd
     build.sh ended:      Sun Apr  8 00:33:56 MDT 2018
==> .

The actual netbsd kernel binary will be sitting in our obj/sys/arch/amd64/compile/GENERIC/ dir. To actually install it you’d just copy it over your existing netbsd kernel.

dexter@slag:~/netbsd_build/7.1.2/usr/src$ file obj/sys/arch/amd64/compile/GENERIC/netbsd
obj/sys/arch/amd64/compile/GENERIC/netbsd: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for NetBSD 7.1.2, not stripped

Userland

Next is userland. no frills here, you will see the pattern

$ ./build.sh -U -m amd64 -O obj -j17 release

The only new option is just release. If you want to build X also pass in -x

Further reading

At this point I’ve just overlapped the official guide on building, which has all the details. Check it out once you are ready to make modifications and actually install your new bins!

https://www.netbsd.org/docs/guide/en/chap-build.html

April 3, 2018

Reading linux mouse input with golang

I think mouse input in linux is generally awful. It always feels terrible in any window manager and I have begun digging into the whys. It seems mostly related with X adding acceleration. In my ventures I learned how to read from a mouse device without X. Lets read the mouse input from linux!

In UNIX’y operating systems everything is a file which many people seem to enjoy shouting off at irrelevant times with no context. For our purposes, this means the mouse will be represented as a file somewhere.

If you only have one mouse (why would you have two? freak) you can check /dev/input/mouse0

$ file /dev/input/mouse0

/dev/input/mouse0: character special (13/32)

With that we can see the mouse is a so called special’ which is short for a special file’, aka device file

A quick and dirty example in go to get the relative x/y update and whether or not left/right/middle is down would look something like this. Note this is blocking and could be drastically better

package main

import (
    "os"
    "log"
    "fmt"
)

// TODO https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/input.h#n28

func main() {
    f, err := os.OpenFile("/dev/input/mouse0", os.O_RDONLY, 0600)
    if err != nil {
        log.Fatal(err)
    }

    data := make([]byte,  3)

    for {
        nr, err := f.Read(data)
        if err != nil {
            log.Fatal(err)
        }

        if nr == 0 {
            break
        }
        
        left := data[0] &amp; 0x1 != 0;
        right := data[0] &amp; 0x2 != 0;
        middle := data[0] &amp; 0x4 != 0;
        x := int8(data[1])
        y := int8(data[2])
        fmt.Printf("\r x=%d y=%d left=%v right=%v middle=%v ",
            x, y, left, right, middle)
    }
}

Assuming you have your GOPATH setup correctly (or use the default of ~/go), save this in a package called eg, linux-mouse-events.

Then you can test it like so (CTRL-C to exit):

$ go install linux-mouse-events
$ sudo ~/go/bin/linux-mouse-events
x=1 y=0 left=false right=false middle=false ^C

Amazing! Next we will move to timing evdev events to see if anything mucks with raw mouse report rate

March 26, 2018

Le Potato’ boot serial port

The AML-S905X-CC (Le Potato) is a strange beast. It doesn’t have reams of friendly documentations and is fairly barebones.

I decided to start looking into where u-boot lives, where early boot output goes and things of that nature.

So off I went and plugged in a USB serial cable to the usual spot for a raspberry pi style header. From top right, its VCC VCC Ground UART_TX UART_RX.

attempt 1attempt 1

Cracked open putty, serial 115200 baud lets go. Not my first rodeo..

dohdoh

Never said I was able to ride the bull 8 seconds. Wrong baud rate? That would be a first for me. Maybe it has different headers. I go to check pin-out, but they do not provide a pinout!

They do however provide a schematic. I’ve mirrored it HERE because they served it from drive so who knows how long that will be around.

Huh, it mostly matches the raspberry pi layout..

header schematicheader schematic

I decided to quickly flip around to the CPU(s) section of the schematic to make sure UART_A_TX and UART_A_RX are the right thing. In the process I discovered they have broken out the IO pins for UART in a separate spot named UART DEBUG (promising!) and they are labelled differently as linux rx/tx’:

uart debuguart debug

physical location next to hdmiphysical location next to hdmi

Instead of wasting time guessing UART baud rate incase its something weird I decided to crack out a saleae logic analyzer. By the way, I cannot recommend these things enough, they are amazing.

I plugin the UART pins and add async serial analyzers with autobaud’ on, so they will scan smallest pulses as they are received and adjust baud.

hookedhooked

serial analyzerserial analyzer

This works without issue and the reported baud setting is ~116279 after a capture, so 115200 is probably correct.

Plugging the usual USB serial cable into the debug UART at 115200 shows us glorious U-boot output and works.

serial analyzerserial analyzer

This is a custom u-boot that I am unsure is completely upstreamed. I’ve never seen so much stuff printed out before uboot header.

Takeaways

As expected, the le potato will require digging. aside from one (active) forum, you’re basically on you’re own. On the plus side the developers seem engaged with upstream, have their sources on github and provided schematics.

March 25, 2018

NetBSD features you may have missed

It’s not secret NetBSD is the smallest of the three contemporary modern BSDs (FreeBSD, OpenBSD, NetBSD, intentionally ignoring PC-BSD/TrueOS). Many don’t give NetBSD any more thought than oh its the one that can run on a toaster’ (which is true as long as it has a MMU :)

the netbsd poster-child toaster-childthe netbsd poster-child toaster-child (Full article here )

There are some interesting features that are useful on desktops as well as embedded that deserve being mentioned. I list some of the features that caught my eye in no particular order. Also in no way are these unique to NetBSD (well mostly), but some may surprise you.

Lua kernel / ring0 code

I’m not sure how popular it is or if its really used, but it is a possibility to write kernel modules in Lua. This is pretty darn cool and lua support in the kernel is the fruits of a previous Google Summer of Code (GSoC) project.

For Lua to work, you need to load the lua and lua sytem modules:

# modload lua
# modload luasystm

Then you can create a Lua state and load a script like so:

# luactl create state1
# luactl require state1 systm
# luactl load state1 ./luahello.lua

You can find full examples here Def checkout intro(9lua) too.

CGD disk/partition encryption

CGD (Cryptographic device driver) lets you encrypt disks at an entire disk or partition level. Of course, this is nothing new, but it has more flexibility for key sizes and available ciphers versus, eg, OpenBSD. This may no longer be true

rump kernel drivers in userspace

juicy rumpjuicy rump

NetBSD Rump kernels offer many benefits, of particular interest is being able to compile kernel drivers and test it in seconds without using a VM. In simplified terms, it is basically the various drivers that usually live in kernel space, but running as separate entities in the rump kernel userspace.

Rump kernels are not an operating system specific concept, for background see What is a rump kernel? NetBSD anykernel’ is (was?) the first implementation of a usable rump kernel though. It has a funny name to boot

pkgsrc can do some cool stuff

I have to admit I’m just starting to dip my toes in pkgsrc. It’s a solid package manager, downloading and compiling sources for you and managing deps, as you would expect of a package manager in 2018. It has some lesser known auxiliary features:

  • Portable, you can use it on other BSDs as well as Linux!
  • Built in tools for auditing vulnerable software with pkg_admin audit
  • It can check licenses if you care about that. Eg warn/disable installing XYZ license software.

Veriexec - Ring0 binary/file/data verification

In layman terms, veriexec compares binaries and files to a pre-configured whitelist of hashes and refuses to run any that diverge. While this would require a bit of setup (and would be annoying for casual daily desktop use), you could really lock down a machine with this setup that would be resilient to even root privilege escalations.

With the state of MD5 and SHA1 hashes being worthless for integrity and having vulnerabilities and proof of collision respectively, it supports SHA256, SHA384, SHA512, and RMD160 hashing. There are a few levels of strictness which can make the machine dynamically learn or report in an IDS style of operation as well.

Sane config/directory hierarchies and other non-technical things

This is not so much a feature as my personal preferences, but on NetBSD you get things like a sane, centralized rc.d startup system. No 6 runlevels, no systemd, no having to rifle around to find defaults. /etc/rc.d after looking at /etc/defaults/rc.conf and be done with it.

Additionally, third party stuff always ends up in /usr/pkg and their configs go in /usr/pkg/etc. While GNU/Linux isnt awful, I am annoyed switching distros and having no idea where things go, install to, overwrite, etc.

Being able to install an entire system (with X) using an installer under 350mb is unheard of in this day and age too. I’m pretty sure the Nvidia drivers are bigger than that now.

March 22, 2018

NetBSD docs tarpits and kernel compiling

While my VM setup is the way to go I eventually got NetBSD-current going on the spare machines I previously mentioned. It would only be fair to mention that when NetBSD 8 comes out, it will work much better on amd64 systems.

With that said, I’ve been trucking along to basic usage and compiling the system. For package management, I’ve been just grabbing binary packages as needed with pkgin, not really interested in trying to compile big stuff with pkgsource yet.

As a beginner, I found the package management scenario presented in a burdened way. The flexibility of (having to) picking your package trees, binary versus sources, pkgsource config, pulling in CVS etc increases the cognitive load. I just needed tmux! So I only do basic pkgin usage for now.

Scan mailing lists

It’s worth mentioning, even if you have no intention of running NetBSD-current or getting into the code, if you run into issues, visually scan or search the mailing list archives, someone else likely has already griped about your issue if its even remotely common.

There are actually two approaches to this, you can search GNATS, but its kinda slow to query, and you kind of have to know what you’re looking for in issues. Or just check netbsd-bugs and netbsd-users mailing lists for a decent litmus-test.

netbsd-bugsnetbsd-bugs

NetBSD-bugs will be slightly exaggerated noise with GNATS auto replies, but if you’re having an issue it will likely be beaten to death by faithful testers here.

Moving on to grabbing tarballs of sources (sets as it were) and compiling the kernel, I googled and ended up at the NetBSD wikis for various things, eg:

I plodded through and things immediately went very badly and in hindsight I should have checked sooner..

oopsoops

long story short: DO NOT USE THE WIKI (for stuff in the guide). It’s basically redundant compared to the guide, and has largely fallen to the wayside. I think the guide is in tree next to sources and was mostly up to date.

I still ran into minor issues with the current release and reported them. But overall there was no show stoppers for the kernel compiling section of the guide.

System and kernel compiling

One minor issue I had was the build.sh instructions not working as prescribed. But looking at the error output, it was easy to figure out. For people less weathered it might not have been trivial to know to create an obj’ folder or specify it and run as -U for non-root users. But then again NetBSD demands a bit of knowledge up front. For example (this is after you have dong config <cfgfile>, GENERIC on amd64 here), here’s how I got going. Change -j9 to number of cores in your machine +/- 1 for best results. There is a useful file, BUILDING sitting in /usr/src once you have unzipped sets that has full explanations and useful examples. Alternatively the building the kernel manually’ guide works fine still. I’m interested in using the excellent cross compiling capabilities of build.sh though.

$ cd /usr/src
$ mkdir obj
$ ./build.sh -U -u -O obj -j9 tools

(this will take a while, gcc is a bear of a compile)

$ ./build.sh -U -u -O obj -j9 kernel=GENERIC

--- netbsd ---
#      link  GENERIC/netbsd
/usr/src/obj/tooldir.NetBSD-7.1.2-amd64/bin/x86_64--netbsd-ld -Map netbsd.map --cref -T /usr/src/sys/arch/amd64/conf/kern.ldscript -Ttext 0xffffffff80100000 -e start -z max-page-size=0x100000 -X -o netbsd ${SYSTEM_OBJ} ${EXTRA_OBJ} vers.o
NetBSD 7.1.2 (GENERIC) #1: Fri Mar 23 13:51:54 MDT 2018
   text    data     bss     dec     hex filename
14223597         654556  593920 15472073         ec15c9 netbsd
===> Kernels built from GENERIC:
  /usr/src/obj/sys/arch/amd64/compile/GENERIC/netbsd
===> build.sh ended:      Fri Mar 23 13:51:57 MDT 2018
===> Summary of results:
         build.sh command:    ./build.sh -U -O obj -j9 kernel=GENERIC
         build.sh started:    Fri Mar 23 13:49:31 MDT 2018
         NetBSD version:      7.1.2
         MACHINE:             amd64
         MACHINE_ARCH:        x86_64
         Build platform:      NetBSD 7.1.2 amd64
         HOST_SH:             /bin/sh
         MAKECONF file:       /etc/mk.conf (File not found)
         TOOLDIR path:        /usr/src/obj/tooldir.NetBSD-7.1.2-amd64
         DESTDIR path:        /usr/src/obj/destdir.amd64
         RELEASEDIR path:     /usr/src/obj/releasedir
         Updated makewrapper: /usr/src/obj/tooldir.NetBSD-7.1.2-amd64/bin/nbmake-amd64
         Building kernel without building new tools
         Building kernel:     GENERIC
         Build directory:     /usr/src/obj/sys/arch/amd64/compile/GENERIC
         Kernels built from GENERIC:
          /usr/src/obj/sys/arch/amd64/compile/GENERIC/netbsd
         build.sh ended:      Fri Mar 23 13:51:57 MDT 2018
===> .

Finally, I can start digging in. This compile time certainly helps! Modern computing rules! This VM is not even on a SSD. I’m hopeful if I moved it and gave it a few more cores it could build in 60 seconds. A full release build of tools, userland and X across 16 cores probably still takes 10s of minutes though.