How to Make a Minimum Viable Software Package with NetBSD, ARM, and QEMU
I began writing this article with the grand ambition of publishing an API for Flying Car.
The plan was to rely on pkgsrc (package source), the primary software package management system for NetBSD.
Package source proved to be formidable to say the least, and the extra complication of targeting ARM soon made me check my ambitions.
Learning how to prepare a software package for offline distribution from start to finish was a deeply humbling experience, and I wish to share some of what I learned along the way.
Hello World
In this tutorial you can follow my steps to create a minimum viable software package from local source code for offline distribution on NetBSD running on ARM using pkgsrc and QEMU.
The following text assumes that the reader is running Linux and is familiar with the C programming language. If you are new to C, then I highly recommend picking up “The C Programming Language” by Brian W. Kernighan and Dennis M. Ritchie.
My first challenge was to compile the package for NetBSD running on ARM. I could not figure out how to cross-compile for this environment, so I used qemu to virtualize it.
The second challenge was to rely on pkgsrc as little as possible and to resist the temptation to invent anything new.
Platform dependence would slow me down in the future.
Any additions I make today would inevitably have to be rewritten tomorrow by someone who actually knows what they’re doing.
My final challenge was to figure out how to distribute the package offline. As almost everything assumes an Internet connection nowadays, finding instructions for installing packages from a USB flash drive or even a CD-ROM may be difficult.
Setting Up the Development Environment
Start by installing qemu version 4.1.0 or higher as well as qemu-system-arm on the host machine.
Then, download NetBSD-9-earmv7hf--generic.img.gz from here and QEMU_EFI.fd from here.
The first file is NetBSD 9.3 release disk image for ARM (armv7). The second is Tianocore EDK2 firmware for ARM (armv7).
If you experience bad performance, try the NetBSD 9.X daily disk image armv7.img.gz instead from here. You can browse all available options here.
Prepare two disk images armv7.img and armv7-2.img.
$ gunzip NetBSD-9-earmv7hf--generic.img.gz
$ mv NetBSD-9-earmv7hf--generic.img armv7.img
$ cp armv7.img armv7-2.img
$ qemu-img resize armv7.img 20g
$ qemu-img resize armv7-2.img 20g
Use qemu-system-arm to boot the virtual machine from the first disk image armv7.img.
$ qemu-system-arm -M virt -cpu cortex-a15 -smp 4 -m 2g \
-drive if=none,file=armv7.img,id=hd0 -device virtio-blk-device,drive=hd0 \
-netdev type=user,id=net0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=net0,mac=00:11:22:33:44:55 \
-bios QEMU_EFI.fd -nographic
Note that port 2222 can now be used to ssh to the virtual machine from the host machine.
After NetBSD boots, login with account root. There is no password on a fresh installation.
Run /usr/bin/passwd to set a password. Write it down as you’ll need it later.
$ /usr/bin/passwd
Edit the file /etc/ssh/sshd_config and add the following line to permit SSH login with root.
PermitRootLogin yes
Restart the sshd service.
$ service sshd restart
You can now copy files back and forth between your host machine and virtual machine.
Update software packages on the virtual machine. This may take a while. Note the preference of ipv4 over ipv6.
$ ftp -4 ftp://ftp.NetBSD.org/pub/pkgsrc/pkgsrc-2024Q2/pkgsrc.tar.gz
$ tar -xzf pkgsrc.tar.gz -C /usr
There are issues with QEMU and ipv6 on NetBSD. The recommended workaround is to use ip6addrctrl.
In /etc/rc.conf add the following lines.
ip6addrctl=YES
ip6addrctl_policy="ipv4_prefer"
Reboot the virtual machine safely with reboot for changes to take effect. This should fix poor network performance.
$ reboot
Login with account root and the password which you set earlier.
Install the pkgin tool for managing pkgsrc binary packages.
$ pkg_add http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/earmv7hf/9.3/All/pkgin-23.8.1nb4.tgz
Fix a known issue with SSL certificates.
$ pkg_add http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/earmv7hf/9.3/All/mozilla-rootcerts-openssl-2.15.tgz
Finally, update all the software packages using pkgin.
$ pkgin update
Install the package developer toolkit pkg_developer.
$ pkgin install pkg_developer
Install bmake.
$ pkgin install bmake
Your virtual machine now has a directory /usr/pkgsrc/ where the source code for all packages lives.
Build and Package Your App
At this point, I would like to give credit to jperkin on Libera Chat #pkgsrc who kindly recommended that I look at /usr/pkgsrc/pkgtools/digest/ to figure out what to do next.
Create a directory at /usr/pkgsrc/misc/app where you will put the source code for your application.
$ mkdir /usr/pkgsrc/misc/app
Create a DESCR file at /usr/pkgsrc/misc/app/DESCR with the following contents. This file normally contains a paragraph describing the application.
You call this archaeology?
Create a Makefile file at /usr/pkgsrc/misc/app/Makefile with the following contents.
# $NetBSD $
MAINTAINER= yourname@example.com
CATEGORIES= misc
VERSION= 1.0
PKGNAME?= app-1
CHECK_PERMS= no
.include "../../mk/bsd.prefs.mk"
.include "../../mk/bsd.pkg.mk"
Create a PLIST file at /usr/pkgsrc/misc/app/PLIST with the following contents. This file normally contains the package’s “packing list”.
@comment $NetBSD $
bin/app
Create a work directory at /usr/pkgsrc/misc/app/work. In it, create a folder called app-1 at /usr/pkgsrc/misc/app/work/app-1.
$ mkdir -p /usr/pkgsrc/misc/app/work/app-1
Inside the directory app-1 create a second Makefile file at /usr/pkgsrc/misc/app/work/app-1/Makefile with the following contents.
# Define the C compiler to use
CC=gcc
# Define any compile-time flags
CFLAGS=-Wall -g
# Define the C source files
SRCS=app.c
# Define the C object files
OBJS=$(SRCS:.c=.o)
# Define the executable file
MAIN=app
.PHONY: clean
all: $(MAIN)
$(MAIN): $(OBJS)
$(CC) $(CFLAGS) -o $(MAIN) $(OBJS)
.c.o:
$(CC) $(CFLAGS) -c $< -o $@
clean:
rm -f $(OBJS) $(MAIN)
install:
echo "This install command does nothing."
Create a file app.c at /usr/pkgsrc/misc/app/work/app-1/app.c with the following contents:
#include <stdio.h>
int main() {
printf("Hello World\n");
return 1;
}
Navigate to /usr/pkgsrc/misc/app and run bmake.
$ cd /usr/pkgsrc/misc/app
$ bmake
Move the binary file around to appease the pkgsrc gods.
$ mkdir -p /usr/pkgsrc/misc/app/work/.destdir/usr/pkg/bin
$ cp /usr/pkgsrc/misc/app/work/app-1/app /usr/pkgsrc/misc/app/work/.destdir/usr/pkg/bin/
Then, run bmake install.
$ cd /usr/pkgsrc/misc/app/
$ bmake install
You should see the following message appear.
=> Creating binary package /usr/pkgsrc/packages/All/app-1.tgz
Test that the app was compiled and installed correctly by running app.
$ app
Hello World
Return to the host machine.
Use scp to copy app-1.tgz from the virtual machine to the home directory on the host machine.
scp -P 2222 root@localhost:/usr/pkgsrc/packages/All/app-1.tgz ~
The software package app-1.tgz is now ready for distribution. You can copy it to any portable medium such as a USB flash drive.
Installing and Updating Your App on Another Machine
Remember that the binary you compiled can only run on NetBSD running on ARMv7.
Let’s make sure that our app can install and run on a fresh virtual machine.
Shut down NetBSD on the virtual machine safely with shutdown.
$ shutdown -h now
Boot a new virtual machine from the second disk image armv7-2.img.
$ qemu-system-arm -M virt -cpu cortex-a15 -smp 4 -m 2g \
-drive if=none,file=armv7-2.img,id=hd0 -device virtio-blk-device,drive=hd0 \
-netdev type=user,id=net0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=net0,mac=00:11:22:33:44:55 \
-bios QEMU_EFI.fd -nographic
Return to the host machine.
Use scp to copy app-1.tgz from the host machine to the home directory on the virtual machine.
scp -P 2222 app-1.tgz root@localhost:~
Return to the virtual machine.
Install the application with pkg_add.
pkg_add -u ~/app-1.tgz
See the description of the pkg_add -u option below.
If the package that’s being installed is already installed, an update is performed. Installed dependent packages are updated recursively, if they are too old to fulfill the dependencies of the to-be-installed version.
What “update” does in reality is replace any package files that are already installed.
An alternative, albeit more confusing, option is pkg_add -U described below.
Replace an already installed version from a package. Implies -u.
I assume that -U is just more forceful than the humble -u.
The operation will fail if the package is already installed, and neither option is provided.
Until next time! Stay tuned and subscribe to the mailing list!