Development Workflow in OPNsense

It’s pretty hard for me to approach a larger repository I have never worked with. The biggest issue is that few projects have defined development (as in actual coding) workflow laid out for new contributors, so one is just going to be stabbing in the dark for a few days or weeks until things start making sense. Speaking of sense, let’s explain how we’ve designed the development experience for OPNsense and how you can start contributing code in no time.

Structure: Source, Ports, Core, Tools

The structure is pretty much FreeBSD: we have a source code repository and a ports tree. Historically, we also have a core code and tools repository. The tools repository is project shell code gluing all repositories together, producing final images, while the core is the important GUI and system configuration bits.

This is the first thing that’s interesting: why is the core not part of the source repo? Because it depends on third party software found in the ports so we can’t stick it into source, because ports are things that don’t fit into the base system. It also helps to keep source repo changes to the minimum to make major FreeBSD upgrades easier in the future.

So then why is the core repo not in the ports? Well, we have a couple of custom ports in the ports tree, but these are small. The core repo as well as the ports tree itself are so big that it made sense to keep them separate. Another reason is that the core repo only contains scripts in Shell, Python and PHP. So the tools repo actually treats the core repository as a package that depends on all the ports it needs. This way, on the images, it looks like the core code is just another package. That makes upgrading the core code very easy and fast without modifying base. We can even upgrade to newer ports pages and add and remove them as we go forward.

Building: Not Clobbering the Build System

The tools repo is designed to run on a stock FreeBSD using chroot mechanics to keep the build contained and consistent. There’s nothing worse than a build system that modifies the build system and at some point starts to dash out working images–only to stop working some time in the future. No, no, no.

You can also “cross-build” between FreeBSD versions. We’ve successfully built images on FreeBSD 10.1 although OPNsense runs on FreeBSD 10.0. That’s not a huge gap and the ABI is the same, but we expect this to work with FreeBSD 11 and beyond as well so that if you have a FreeBSD box you will always be able to produce your own images if you desire–without spinning up extra machines, jails or virtual machines.

Here are the build instructions for OPNsense.

Running: Use a Virtual Machine for Development

It’s just easier and less tedious if the kernel crashes, to revert to a previous state, or isolate and test different features. A VM can be set up easily using the ISO images. Also very good for testing installation and upgrades. Of course, at some point you will want to bring OPNsense to your actual target device—just make sure you know the system well enough before you attempt this.

VirtualBox is a solid tool for the job, but be sure to check out FreeBSD’s Bhyve as well (it was added in FreeBSD 10). However, If you are only interested in GUI coding, you can skip all the build parts and directly download an image and spin it up. Because…

Packages: It Gets Even Better

Once you have a running instance, how to produce and push code? Well, there’s a couple optional of packages that help you to be productive:

# pkg install vim-lite joe nano gnu-watch git tmux screen

The most important one is git for obtaining the code, the rest is optional if you need it—mostly editors and terminal wrappers. It is also possible to mount sshfs or do sftp to sync files from the repo if you would like to use a graphical editor from your own system. As the root user do the following:

# cd
# git clone

Once you have the repository, you switch it live using:

# cd core
# make mount

Yes, the changes that you make in the repository will show up directly in the GUI now! Unfortunately, this will mount the repo over /usr/local so if you modify packages the changes will light up in git. Be careful. To prevent that from happening you can temporarily unmount using:

# make umount

To finish that off the boot sequence will mount the core repository set up in /root/core as early as possible (if it’s available) and will use its modifications for booting up (with the exception of /usr/local/etc/rc itself). This makes it possible to work on the backend configuration and boot sequence improvements without having out of sync system files and repositories.

Summary: Easy Access is Key to Collaboration

Although this is just a peek into OPNsense development workflow, I hope that this brings to attention a key aspect: moving barriers out of the way to enable as many people as possible to produce quick results. Yes, there’s barriers like git and GitHub to deal with, maybe even learning FreeBSD intricacies, but once you have your code in the GUI and working fine, you’ll feel proud enough to endure the hardships of making sure your patch will have a place in our upstream repositories so the community as a whole can benefit from your dedication.

We look forward to your feedback; we are seeking for more improvements in the build system and eagerly await your pull requests. That’s all for now, take care and code responsibly. :)

2 thoughts on “Development Workflow in OPNsense”

  1. A small point to note.

    When you come to “make umount” ensure that you cd /root/core or the command will fail.

    After mounting I cd’d a few times, then tried to make umount and couldn’t figure why it failed.

    Another point is to ensure that you have umounted BEFORE you try and rm -rf /root/core :-) A hard reset fixed that fortunately.

  2. Hi John,

    yeah, the mounting is tricky and “make” only works with the appropriate Makefile, which is under /root/core. I’ve recently moved the directory to /root/core_safe for my production systems–the auto-mount is neat, but serves mostly the purpose of testing rc/startup scripts, which have stabilised considerably over the last months.

    The mount is nasty in conjunctions with firmware upgrades as well as pkgng will install files into the git directory… Generally, the live-mount should be used with great care or only for brief periods of time.

    Two more solutions for more permanent solutions:

    (a) apply patches to the system files under /usr/local/ (which corresponds with core.git/src).

    (b) I am investigating methods to produce a system update package on the running system that installs cleanly, which is easier to handle than (a) and less fragile than the live-mount.

    Thank you for your comments! :)


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.