EuroBSDcon 2018


Last year for EuroBSDcon 2017 in Paris, I caught a horrendous cold on the first day and spent the week in a fragile state, this year I was well prepared and caught the terrible cold in advance. Things kicked off on Monday when I had to go and see a man about a goat at the train station as I was the chosen mule for this leg of the conference. No curried goat for dinner that night.

This was to be my first time in Bucharest and also the first time flying from London Luton Airport. I packed the goat and cough medicine and headed for the train station again on Wednesday morning. By lunch time I had met up with sborrill from the NetBSD release engineering team, we discussed the changes happening in NetBSD-HEAD and the tooling built on top of NetBSD at Precedence before setting off.

Thursday was about the FreeBSD devsummit for me or at least it was meant to be. For the flexibility of being able to use any computer without having to spread keys everywhere and the safety of travelling lightly without important keys, I use a Yubikey which contains a key that provides access to a few non-critical systems. At the devsummit, when it came time for our first break, I unloaded my key from the system and ejected the Yubikey, I don’t know what happened next in my head but when I came back I tried to load my key again from the Yubikey and entered an incorrect pin consecutively until the device was locked. This was a total disaster as I was not carrying the admin pin to unlock the Yubikey and I wasn’t sure what I had done at the time. I spent most of the devsummit trying to understand why I was unable to load my key and how to tell if the device was actually locked or not, documentation is pretty sparse and somewhat poor. It doesn’t help that there are multiple components required to use and manage a security token, all of which are independently developed with separate sets of documentation.

My favourite piece of documentation was from GnuPG.

PIN retry counter

This field saves how many tries still are left to enter the right PIN. They are decremented whenever a wrong PIN is entered. They are reset whenever a correct AdminPIN is entered. The first and second PIN are for the standard PIN. gpg makes sure that the two numbers are synchronized. The second PIN is only required due to peculiarities of the ISO-7816 standard; gpg tries to keep this PIN in sync with the first PIN. The third PIN represents the retry counter for the AdminPIN.

Hopefully, it might be edited at some point in the future.

I eventually gave up, conceding that I am locked out until I get home.

On the Friday, it was time for the NetBSD devsummit. We first covered hypervisors and support of different types of virtualisation in Xen, joerg gave a status update on his ongoing repository conversion work and what avenues it could potentially enable as well as a core and TNF board status update which segued into more technical details such as the DRM update. It was a fun day as we had the opportunity to ask questions and cover the answer in detail, something which is not possible in the conference talk setting. The discussion continued over dinner and late into the night.

For the first day of the conference, I spent the day in room 2. I heard the first of a series of talks on sanitisers, David Carlier and kamil co-presented on the state of sanitiser support. Sanitisers featured heavily at the conference this year which shows the importance of such tooling. Kristaps Dzonsons gave a talk about trying to utilise an open source stack for use with diving, from photo and video management to GPS and the rough edges with some options. Andrew von Dollen gave a talk about utilising the NPF Lua binding to provide a simple high-level interface to the firewall and in the spirit of the Scriptable Operating Systems with Lua paper, using the interface to explore different filtering scenarios with ease. NPF scripting with Lua was previously presented at EuroBSDcon 2014.

I was the last speaker of the day in room 2, I had hoped to present off NetBSD/macppc 8.0 on my 12″ G4 iBook but was unable to get it working with the projector due to genfb(4) not recognising it as connected. Instead I presented using maya‘s Dell XPS 15″ using the latest DRM update in NetBSD-HEAD and everything just worked. In hindsight I should have booted a kernel from HEAD to get radeonfb(4) and try again, but at the time I was actually thinking about recompiling my kernel! 🙂

The day wrapped up with the second keynote by Ron Broersma, he came equipped with a lot of historical memorabilia, It was cool to see a first edition copy of Computer Lib (you can order a copy from Ted Nelson here).

Ron spoke about the evolution of ARPANET & The Internet with an emphasis on the use of Berkeley UNIX. It was amusing to hear how  UNIX tape images had been provisioned out to sites from government agencies, Bob Morris also mentioned the subject in An Oral History of UNIX and getting the initial image to the agencies.

The actually, pieces of the government, peddled the idea of using UNIX to
national security agencies. I kind of laughed at the people there. Because, are
they aware of the fact that the UNIX that they are now running actually got to
NSA in the trunk of my car.

As with Ted Nelson, Ron mentioned priesthoods which I believe is still prevalent in tech communities especially networking. Me and khorben discussed Patterns in Network Architecture: A Return to Fundamentals by John Day which provides insights about how things came to be and an alternative approach to address technical issues in modern day networking.

My favourite historical tidbit from the keynote was that in the pre DNS era when a hosts file was circulated, unofficial revisions were a thing and Berkeley hosts also appeared as Berserkeley on some.  🙂

Day two of the conference began with a heavy dose of sanitisers, kamil this time speaking about finding kernel bugs through the use of sanitisers followed by Yang Zheng on integrating libfuzzer with the NetBSD userland for Google Summer of Code and some of the bug he’d found in NetBSD as a result.

After lunch, khorben spoke about how he got into operating system development, his DeforaOS project and the approach in simplicity it takes with reference to John Day’s Pattern in Network Architecture book.

With a quick change over, maya spoke of the various bugs she ran into on NetBSD and her approach to debug them, covering everything from bugs on DEC Alpha to MIPS to drivers.

For the last stretch of the conference, I headed down to Being a BSD user by Roller Angel. It was a talk about growing within a community and the personal challenges one goes through as they develop. Roller also provided support for users in a technical role and covered the tooling he used to help users learn, such as screen-casting. Afterwards, I headed back up to room 2 for agc‘s talk on source code tracking and the experience with various tools used over the years at Netflix. Things starting life as a bunch of scripts which were added to subversion, migrated to mercurial and now stored in git. Of the dingbats in the talk, my favourite was Jose which the intro for the section on OCA.  When the conference finished, we headed out for dinner. Having heard Scott Long’s talk about Netflix at NYCBSDCon 2014, I asked agc about his experiences in the early days of the appliance project, over dinner we heard about suffering disk firmware issues and building a strategy to re-flash appliances, extending tooling like camcontrol(8) and improving build performance.

Snapped in London, when I returned on Monday:

maya & kamil also wrote up about their experiences here and here.

Slides for my talk “What’s in store for NetBSD 9.0” are available here.

A blast from the past – Amit Singh

I spent most of today tied up with working on getting an upspin instance operational again. Instead of running things as a standalone daemon, I proxy through a web server so that the upspinserver can run as a unprivileged user, on a unprivileged port and everything should be good and happy. Except it’s not, on the client side is macOS with osxfuse and the upspin space expose to the OS via upspinfs, browsing works great, manipulation through finder, not so much. You can copy a file in to place and when the transfer is finished, the file disappears from finder however the upspin tool is able to show things correctly as they should appear. I spent some time looking in to the osxfuse source repositories and installer package as there is a separate repo for the kext which I wondered where it fits in, in a world with SIP.

While going through the osxfuse source, Amit Singh’s name popped up in the licenses. I was a big fan of his blog kernelthread.com when it was active and would look forward to new posts. I fondly remember trying out the test app he wrote to demonstrate the sudden motion sensor working on Mac OS X with my 17″ PowerBook G4 and the posts to complement the Mac OS X Internals book.

Turns out he gave a talk at Google back in the day on MacFuse which is the predecessor to osxfuse.

It’s a nice talk which provides some history on how the project came about and why he decided to work on it, there were moments which made me smile especially when discussing conditioning and deeply held (mis?)beliefs.

I also found that Apple still runs their mailman instance with a small number of list being served from it, most importantly Darwin related ones. 🙂

Back to more wrestling tomorrow, gnite!

Something blogged (on pkgsrcCon 2018)

pkgsrcCon 2018 badge

For this years pkgsrcCon, the baton was passed on to Pierre Pronchery & Thomas Merkel, location Berlin. It wasn’t clear whether I would be able to attend this year until the very last minute, booking plane tickets and accommodation a couple of days before. The day before I flew out was really hectic and I did not get any sleep. I left home around 1:30am and made my way to St Pancras for the train to the  airport as It was an early morning flight. Once we’d boarded the flight, I passed out without any recollection and came to just before we were going to descend.  With less than two hours sleep and plenty of time to spare until the evening social I slowly made my way from the airport to the city centre. I roamed the city centre looking at the street art and jumping on and off metro stations between Jannowitzbruke and Westkreuz.

In the evening I headed to the social event and met up with others.  I heard about the state of The Unleashed Operating System, the latest buzzword soup to add to current projects for instant success (block chain, AI and something else I forgot the to note down), debug work flow and lots of other things over food and drinks. I had taken my ThinkPad X60s alongside a 12″ iBook G4 to Berlin, with the plan to see if jdolecek with the machine in person could shed any light on a deadlock issue I experience when hammering the CPU with a kernel build while 2 or more CVS operations are in progress and to also get setup with Yubikey on NetBSD so I could commit from any machine with a USB port without having to copy my keys around to different machines.

On the first day of talks, between preparing slides, spz pointed out what I needed to get the Yubikey working with SSH on NetBSD. The current Yubikeys support multiple features (U2F, OTP, CCID). In order to use a Yubikey as a CCID device with pcscd, pcscd expects a ugen(4) device which doesn’t work on NetBSD because the USB keyboard driver ukbd(4) attaches to the Yubikey and prevents other modes of access. The workaround is to roll a new kernel with the device explicitly hard coded to use ugen(4). With that in place, I was able to checkout the developers source repo on the iBook. For the X60s, jdolecek rolled a fresh kernel with with the DIAGNOSTIC and LOCKDEBUG which we booted and reproduced the deadlock, unfortunately, there was no new information provided by these option, possibly indicating that it’s not a locking issue. A new kernel was rolled with an extra printf() which I tested with, however I couldn’t reproduce the issue with this change. Instead the CVS update just got slower and slower while the system operated normally (without deadlock). Unsure if it was the lack of bandwidth or a new behaviour, I gave up after several hours of the cvs update running. To rule out connectivity, my next plan is to test locally using a mirror of the repository using rsync and perform the CVS operations from that source instead.

I gave a talk titled “Something Old, Something New, Something Borrowed”. Something Old was about NetBSD/macppc, a homage to my first pkgsrcCon back in 2014 where I gave a short talk about pkgsrc on Tiger PowerPC with my 12″ PowerBook, Something New was about Upspin, Something Borrowed was about Minix3. Slides.

For coverage of the different talks and leot‘s experience in Berlin, see this blog post.

After a day of presentations we headed over to a restaurant where we chatted some more over dinner. While we waited for our food to arrive, we were given a demo of  the J programming language by Martin, he described J as a pocket calculator on drugs. I’d previously heard of the K programming language and its cryptic syntax but had not seen anything with such a minimal and cryptic syntax in action.

For the following morning, uwe gave a talk about the Forth programming language and experiences with it, especially with OpenFirmware which originates from Sun Microsystems and was also used by Apple on the PowerPC based Macs. The cool thing about OpenFirmware is that hardware can itself contain a driver for OpenFirmware which gets loaded when the device is initialised, making OpenFirmware aware of how to interact with the device without any manual loading of software by the operator.uwe mentioned NetBSD/ofppc which builds on that by having a kernel without drivers for on-board hardware, instead relying on OpenFirmware to communicate with devices instead. He also described how it is possible to learn about how a system works by using the ccommand in OpenFirmware to disassemble modules, though he warned that unfortunately this didn’t work so well on Macs as symbols had been stripped from OpenFirmware environment. There was lots of references to follow up material to read up on, from The Evolution of Forth paper to uwe‘s notes.

After lunch time, we packed up and headed to a computer museum. On the way we spoke about emacs and workflows like using M-x make-frame-on-display.

On the Monday, I met up with sborrillfor breakfast before heading to the spy museum and doing some sightseeing. I headed back to c-base where I met Pierre, Youri & Sebastian. As it approached the evening, It was time to make way to the airport again. I was back home by around 1am, Tuesday morning, so glad I went 🙂

21/07/18 10:20BST – There has been backlash from residents to stop Google from setting up a new campus in Kreuzberg, Berlin.

A week of NetBSD #1

I wanted to resume writing up notes about what I’d been working on as with the “A week of pkgsrc” series, this actually spans the last few weeks. 🙂

Things basically came together around the event of a late 2009 13″ MacBook entering my life. First port of call was to see state of NetBSD support since I last visited it.
I’m still running FreeBSD on my 11″ Air and so was keen to see if there had been any progress in NetBSD with regards to supporting the Intel based Macs. Unfortunately the same issue is still present, the system panics very early in the boot process kern/52229 when using the UEFI image and fails when enumerating CPUs present. I was able to get the system to boot using the conventional (non-UEFI) image by opting to boot without SMP & ACPI enabled. I used the MacBook install wiki article as a rough guide on gpt partitioning and got a daily image installed. The wiki article needed some attention as the syntax for commands did not apply but through this process I discovered a crash in gpt where a missing check in the source means that a null pointer is passed to one of the gpt commands.
e.g gpt show -a

Running the system without ACPI was not a good idea, turns out the thermal management does not get initialised and the system eventually switches off as a failsafe measure. I found this out through leaving the machine bulk building some of my packages, only to return to a machine that’s powered off.

GCC 6 has landed in NetBSD-HEAD and work is in progress to bring things into shape as the next revision of GCC shipped in NetBSD. Unfortunately cross compilation from macOS does not work at the moment (toolchain/53013) which limited my ability to experiment with different NetBSD kernel configuration while I was trying to look into the MacBook issue.

The one frustration with crash was that when the system panicked, the system would drop to DDB (the kernel debugger) but the keyboard would not be functioning. I realised that when running into panics on NetBSD/macppc in the past I’d always have a backtrace to include in my bug report. This was because the macppc kernel was built with the option to execute a backtrace command when entering the debugger and the amd64 one wasn’t. Asking on the tech-kern list to enable this option across the board I learnt of the shortfall of this approach and received suggestion on extending ddb, thus the dumpstack option was born and enabled by default. With this, setting ddb.onpanic sysctl to 2 for backtraces went away as well as setting the DDB_COMMANDONENTER option to run backtrace explicitly in kernel configuration files. The next step now is to extend DDB to show the panic message after the backtrace so that the panic message and the tail end of potentially lengthy trace is visible.

Up until very recently there was only support for PCIe based G5 PowerMacs using the POWERMAC_G5_11_2 kernel configuration in NetBSD, I am lacking such a system but do have a first gen G5 iMac which is PCI-X based. The initial work to bring up NetBSD on the G5 was actually done on a PCI-X based system long ago so I was curious what had diverged since then. Previous GSoC project participants used a repo in the NetBSD-gsoc sourecforge project to share their work, the two G5 related GSoC projects are there in a single repo. Some further work also took place on the port-macppc mailing list in 2013. It was a fun weekend albeit no further progress to seeing even a copyright notice on my part. However I learnt lots about the kernel build process, poked at lowcore.S and the boot process. I also learnt of an emulator called Mambo. Mambo was a full PowerPC system simulator, produced by IBM research. There are kernel configuration files to support Mambo (theoretically) in NetBSD & FreeBSD but unfortunately, I couldn’t find any binaries to try Mambo, along with that I also found the links for the PowerPC 970FX (G5 CPU) documentation all dead and the documentation removed from the IBM site. 🙁
To rule out the kernel working on a G5 but console being an issue, I tried experimenting with different frame buffers and found macofcons(4) broke the build (port-macppc/53004). After discussing with macallan@, macofcons(4) and the OFB_ENABLE_CACHE option have now been removed [1] [2], on the basis that though they may have worked at one point they cause more problems that solve.

macallan@ has been working on extending support for the G5 based systems  and it is now possible to boot NetBSD on both the early PCI-X based systems as well as the PCIe models thanks to his work. I was finally able to netboot NetBSD on my first gen G5 iMac, this should come in very handy for pkgsrc work.

The NetBSD/macppc website has a supported system page which was due a review. After a call for feedback on any discrepancies , the awacs(4) soundcard driver is now enabled and I’m looking for a source to find out which systems shipped with which chipsets. Along the way NetBSD gained modem drivers for some of these systems but the page still states they are not supported.

In NetBSD, there is support for different buffer queue strategies for disk I/O, on the tier 1 ports such as i386 & amd64, the per-priority cyclical scan strategy is enabled by default, to bring macppc on par, it is now also enabled there by default too. Now to document the option so that it’s somewhat like the description of another strategy for read priority. See BUFQ_READPRIO and BUFQ_PRIOCSCAN in options(4).

This weekend I’ve been testing the HEAD-llvm builds on i386 & macppc as well as ATF testing, but I’ll write about that in another time.

Thanks to jmcneil, martin, mrg, pgoyette, uwe for the help and suggestions.

Introducing RunBSD.info

RunBSD.info is an introductory site covering the features & benefits across the different members of the BSD family of operating systems. To get things started, the content for the site was taken from an earlier post originally put together with a specific audience in mind. That audience was a group of hardware enthusiasts who are not necessarily interested in software but may require an operating system to run. This is no longer a constraint in this scenario and so will now move to cover things in a generic sense,  allowing wider coverage of highlights in each operating system. There is a lot of room for improvement.

The site is hosted on Github and the content is composed of markdown text files. Contributions welcome by pull request or If you prefer not to use Github, patches against repo files or just plain text content via email gladly accepted. You can reach me on venture37 at the domain of this blog post.

Thanks to FiLiS for the logo.

FreeBSD on 11″ MacBook Air 5,1 (mid-2012)

This tiny machine has been with me for a few years now, It has mostly run OS X though I have tried OpenBSD on it. Besides the screen resolution I’m still really happy with it, hardware wise. Software wise, not so much. I use an external disk containing a zpool with my data on it. Among this data are several source trees. CVS on a ZFS filesystem on OS X is painfully slow. I dislike that builds running inside Terminal.app are slow at the expense of a responsive UI. The system seems fragile, at the slightest push the machine will either hang or become unresponsive. Buggy serial drivers which do not implement the break signal and cause instability are frustrating.
Last week whilst working on Rump kernel builds I introduced some new build issues in the process of fixing others, I needed to pick up new changes from CVS by updating my copy of the source tree and run builds to test if issues were still present.
I was let down on both counts, it took ages to update source and in the process of cross compiling a NetBSD/evbmips64-el release, the system locked hard. That was it, time to look what was possible elsewhere. While I have been using OS X for many years, I’m not tied to anything exclusive on it, maybe tweetbot, perhaps, but that’s it.
On the BSDnow podcast they’ve been covering changes coming in to TrueOS (formerly PC-BSD – a desktop focused distro based on FreeBSD), their experiments seemed interesting, the project now tracks FreeBSD-CURRENT, they’ve replaced rcng with OpenRC as the init system and it comes with a pre-configured desktop environment, using their own window manager (Lumina). Booting the USB flash image it made it to X11 without any issue. The dock has a widget which states the detected features, no wifi (Broadcom), sound card detected and screen resolution set to 1366×768. I planned to give it a try on the weekend. Friday, I made backups and wiped the system. TrueOS installed without issue, after a short while I had a working desktop, resuming from sleep worked out of the box. I didn’t spend long testing TrueOS, switching out NetBSD-HEAD only to realise that I really need ZFS so while I was testing things out, might as well give stock FreeBSD 11-STABLE a try (TrueOS was based on -CURRENT). Turns out sleep doesn’t work yet but sound does work out of the box and with a few invocations of pkg(8) I had xorg, dwm, firefox, CVS and virtuabox-ose installed from binary packages. VirtualBox seems to cause the system to panic (bug 219276) but I should be able to survive without my virtual machines over the next few days as I settle in. I’m considering ditching VirtualBox and converting the vdi files to raw images so that they can be written to a new zvol for use with bhyve. As my default keyboard layout is Dvorak, OS X set the EFI settings to this layout. The first time I installed FreeBSD 11-STABLE, I opted for full disk encryption but ran into this odd issue where on boot the keyboard layout was Dvorak and password was accepted, the system would boot and as it went to mount the various filesystems it would switch back to QWERTY. I tried entering my password with both layout but wasn’t able to progress any further, no bug report yet as I haven’t ruled myself out as the problem.
Thunderbolt gigabit adapter – bge(4) and DVI adapter both worked on FreeBSD though the gigabit adapter needs to be plugged in at boot to be detected. The trackpad bind to wsp(4), left, right and middle clicks are available through single, double and tripple finger tap. Sound card binds to snd_hda(4) and works out of the box.
For wifi I’m using a urtw(4) Alfa adapter which is a bit on the large side but works very reliably.
A copy of the dmesg is here.

Lessons learnt from adding OpenBSD/x86_64 support to pkgsrc

Before even getting into the internals of operating systems to learn about differences among a group of operating systems, It’s fairly evident that something as simple as naming is different between operating systems.

For example, the generations of trusty 32bit x86 PC is commonly named i386 in most operating systems, FreeBSD may also refer to it as just pc, Solaris & derivatives refer to it as i86pc, Mac OS X refers to it as i486 (NeXTSTEP never ran on a 386, it needed a minimum of a 486 and up until Sierra, machine(1) would report i486 despite being on a Core i7 system), this is one of the many architectures which needed to hadled within pkgsrc. To simplify things and reduce lengthy statements, all variants for an arch are translated to a common name whiche is then used for reference in pkgsrc. This means that all the examples above are grouped together under the MACHINE_ARCH i386. In the case of 64bit x86 or commonly referred to as amd64, we group under x86_64 or at least tried to. The exception to this grouping was OpenBSD/amd64, this resulted in the breakage of many packages because any special attention required was generally handled under the context of MACHINE_ARCH=x86_64. In some packages, developers had added a new exception for MACHINE_ARCH=amd64 when OPSYS=OPENBSD but it was not a sustainable strategy because to be affective, the entire tree would need to be handled. I covered the issue at the time in A week of pkgsrc #11 but to summarise, $machine_arch may be set at the start in the bootstrap script but as the process works through the list of tasks, the value of this variable is overriden despite being passed down the chain at the begining of a step. After some experimentation and the help of Jonathan Perkin, the hurdles were removed and thus OpenBSD/x86_64 was born in pkgsrc 😉

The value of this exercise for me was that I learnt the number of places within the internals of pkgsrc I could set something (by the nature of coupling components which share the same conventions (pkgtools, bsd make)) and really the only place I should be seeking to set something is at the start of the process and have that carry through, rather than trying to short circuit the process and repeat myself.

Thanks to John Klos, I was given control of a IBM Power 8+ S822LC running Ubuntu, which started setting up for pkgsrc bulk builds.
First issue I hit was pkgsrc not being able to find libc.so, this turned out to be the lack of handling for the multilib paths found on Debian & derivates for PowerPC based systems.
This system is a little endian 64bit PowerPC machine which is a new speciality in itself and so I set out to make my first mistake. Adding a new check for the wrong MACHINE_ARCH, long forgotten about the previous battle with OpenBSD/x86_64 I added a new statment to resolve the relevant paths for ppc64le systems. Bootstrap was happy with that & things moved forward. At this point I was pointed to lang/python27 most likely being borken by Maya Rashish, John had previously reported the issue and we started to poke at things. As we started rummaging through the internals of pkgsrc (pkgsrc/mk) I started to realise we’re heading down the wrong path of marking things up in multiple places again, rather than setting things once & propogating through.

It turned out that I only need to make 3 changes to add support for Linux running on little endian 64bit PowerPC to pkgsrc (2 additions & 1 correction 😉 )
First, add a case in the pkgsrc/bootstrap/bootstrap script to set $machine_arch to what we want to group under when the relevant machine type is detected. In this case it was when Linux running on a ppc64le host, set $machine_arch to powerpc64le. As this is a new machine arch, also ensure it’s listed in the correct endianness category of pkgsrc/mk/bsd.prefs.mk, in this case add powerpc64le to _LITTLEENDIANCPUS.
Then correct the first change to replace the reference to ppc64le for handling the multilib paths in pkgsrc/mk/platform/Linux.mk.

The bulkbuild is still in progress as I write this post but 5708/18148 packages in an the only fall out so far appears to be the ruby interepreters.

12″ PowerBook G4 PT5 – Electronic Battle Weapon

Preparation for a trip started off a little earlier this christmas. I planned to take my PowerBook on the road with me to Hamburg for 33c3. Previous attempts to use this machine as my primary system on the road in the past had been thwarted by leaving too little time to build & prepare before departure.
The system has been dual booting NetBSD & Mac OS X Tiger for some time now, recently I’ve been doing almost daily upgrades to NetBSD-HEAD on the system using the generated iso images from NYFTP.
My plan was to get the machine installed with a current build FireFox on NetBSD & bring the existing installed packages up to date. I managed to update the existing packages without any problems but it didn’t look like FireFox was going to build successfully. The package as-is currently in pkgsrc does not build on NetBSD/macppc. I was pointed to a patch in pkg/48595 which was pending commit and required testing. It cleared up the initial issue I ran into but the build still failed (see previous link on updates about the failure), though it took a little longer to fail in the day. After several days of failed build attempts I made sure I had an up to date copy of TenFourFox installed on Tiger and settled for Dillo on NetBSD instead.

My usage of Dillo stayed somewhat basic during the trip, despite having the Mozilla certificate bundle installed, I could see any obvious way to point Dillo to it & have it use it. Hence, any site using SSL I visited generated a certificate warning. Perhaps the config should’ve been done in wget?

www/dillo pkgdepgraph

Alexander Nasonov created packages for Dillo & Links-gui targeted for running under a minimal chroot but I did not get around to trying them out. There are chrooted browser packages for other browsers in his pkgsrc github repo. The screenshot above shows the www/dillo package’s dependencies, generated using pkgtools/pkgdepgraph

Moving on, the AirPort Extreme card in the laptop is based on a Broadcom chipset which has a flaw, it’s incapable of addressing memory above 1GB (30 bits) which means the driver needs to care for that or else the card doesn’t work. This is not unique to this Broadcom chipset, the BCM4401 10/100 ethernet interfaces which use the bce(4) driver also suffer from the same problem (unable to address memory allocated above 30 bits), the BCM580x ethernet interfaces which use the bge(4) driver suffer from not being able to address more than 40 bits. Going back to the wireless chipset, the bwi(4) driver which is used in the BSDs, originated from DragonFly BSD. This driver was put together by Sepherosa Ziehau using the documentation from a reversing effort in the Linux community. The bwi driver was then imported in to Free/Open/NetBSD and was eventually removed from DragonFly BSD. A new wireless subsystem was introduced in DragonFly which required change to drivers to work again and the bwi driver was never adapted. It now lives on in the other BSDs.

The version of bwi(4) driver came to NetBSD from OpenBSD, ported by Taylor R. Campbell back in 2009. At the time neither version of drivers could handle the 30 bit bug so you either ran with less than 1GB of RAM or used another card. In 2014 Stefan Sperling committed a workaround for this in OpenBSD. I wanted this fix in NetBSD so my wifi could also work & asked the NetBSD developers if such a change was appropriate in NetBSD. I was introduced to bus_dma(9) and the bus_dmatag_subregion() function, the bce(4) driver was my reference on how to use the function. Looked fairly straight forward, a single call this function and off you go, wasn’t too sure how it would fit into the bwi driver but I thought I’d have a go.

This was one of the things I was hoping to work on during my trip but It turned out to be the only thing I attempt. I happened to meet Stefan at 33c3 and we discussed the driver, the work around and the mighty days of the past when Damien Bergamini was hacking on the OpenBSD WiFi stack. In the OpenBSD driver Stefan had opted to deal with the issue of allocating memory in a specific region directly in the driver rather than adding a new interface to the kernel for  such a task so with a bit of thought about the past and a review of the driver, I was given a diff of the changes and suggestions about where I could start making changes.

I still don’t know yet if it’s possible to lift the changes from OpenBSD and apply them to the NetBSD version of the driver, because the DMA framework is different between the systems.
Partially implementing the change Stefan made without all the bounce buffers he’d added in the OpenBSD driver didn’t work and using the bus_dmatag_subregion() function didn’t work either. I pursued the bus_dmatag_subregion() path during 33c3 and didn’t get anywhere. At this point I started looking deeper in the system by looking at the implementation. It was at this point that I discovered this function was defined to EOPNOTSUPP on PowerPC based systems. No matter what I had tried with this function it was a waste of time^W^W^Wvaluable learning experience about keeping documentation up to date & consistent.

At this point I started looking into adding support for tagged subregions so I could make use of the function. The implementation is fairly simple, a public function for a developer to use which performs various tests and a private function which is called to deal with the memory allocation. Unfortunately there were some missing members from the data structure on the powerpc side of NetBSD which needed further investigation and I stopped there for the time being.

For the trip I relied on a tiny Realtek RTL8188CUS based wifi adapter to get me network access. The card worked on the 802.1x enabled SSID at 33c3 using wpa_supplicant(8) on NetBSD/maccppc.

urtwn0 at uhub4 port 5
urtwn0: Planex Communications Inc. GW-USNANO2, rev 2.00/2.00, addr 2
urtwn0: MAC/BB RTL8188CUS, RF 6052 1T1R, address 00:22:cf:xx:xx:xx
urtwn0: 1 rx pipe, 2 tx pipes
urtwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
urtwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps

The driver for this card is now enabled by default in the GENERIC config file for NetBSD/macppc along with a group of other drivers for USB peripherals.

Thanks to Stefan for his help and advice with the bwi driver and Alex for the chrooted browser packages! 🙂

pkgsrcCon 2016 videos

The videos from some of the talks that took place at pkgsrcCon in Kraków, Poland during the Summer are now available on the Internet Archive.
This years conference drew speakers from many different project, not necessarily BSD related though Net/Free/OpenBSD were represented. Talks were on a pretty diverse range of topics around software but unfortunately not all talks were recorded. I had the opportunity to tag on Mateusz Kocielski’s slot on security & give a brief talk on the work of the pkgsrc security team (slides), whilst he covered the work of the NetBSD security team (see files security-team & flash-die).

The schedule for the conf is available here

Videos from OSHUG #46

Earlier this year, I gave a short talk at OSHUG about the BSD family of operating systems to a group of open hardware enthusiasts. I previously wrote about the event here
The videos from the event are now available on youtube


A new addition to FreeBSD.org

Adding Sevan Janiyan as a documentation committer
After many years of tinkering with FreeBSD, I received an invite to join the FreeBSD project earlier last month. When I first started out with FreeBSD (back in v5.0), the handbook was what lead me through the start and made me realise how empowering decent documentation is. My previous experience with $LICENSEPREFIX/$SOMEKERNEL distros had mainly consisted of marathon searches on instruction how to accomplish $thing, finding instructions for another distro which I wasn’t running & going down another rabbit hole from there. I’ll be working with my mentor Benedict Reuschling as a member of the documentation team to continue the maintenance and improvement of the documentation & manual pages in FreeBSD and also cross-polinating necessary changes to the other BSDs in the family, where applicable.

As a starting point, the committers guide instructs a new committer on some preliminary commits to the doc, base and ports repositories to add necessary information such as name / email address, PGP keys and ICBM co-ordinates.

Pretty stoked to reach this mile stone as a part of a journey that started some years back and took me travelling around the world because of work to attending conferences and other events such as the doc sprints at BSDCan.
Now begins the next milestone to make the documentation even greater, again!
to the kernel source code!

FreeBSD Latest News

FreeBSD News Flash

A slow / low-end system capable of running most modern BSDs

I was looking to test a change related to buffering in cat(1) and wondered what was the slowest system I could use which was capable of running the current versions of NetBSD, FreeBSD, OpenBSD. An old PC and the ARM based BeagleBone Black sprang to mind immediately, then a PowerPC Mac? SPARC64?

Apart from a Sun Fire T1000, I do not have any SPARC hardware, sun4v is only supported on NetBSD & OpenBSD at present, FreeBSD/sun4v was only a pre-alpha rough cut from before the days of version 7 and sparc64 support may be going away in FreeBSD moving forward.

Considered the BeagleBone Black but currently NetBSD-HEAD does not boot on it port-arm/51380 and FreeBSD has issues with running DTrace bug/211389. So that was off the list.

A G4 based PowerPC Mac is supported between my choice of BSDs, unfortunately I couldn’t get a working disk burnt from the FreeBSD iso files to try it out on a 12″ PowerBook. bug/211488.

I settled on running i386 builds on a Alix 2c3 I have, it has 256MB RAM and a 500Mhz Geode CPU, currently running FreeBSD/i386 11-BETA3 without issue and has no problems with any of the other BSDs. It’s a little too “modern” and high spec though in my test.

Running FreeBSD / OpenBSD / NetBSD as a virtualised guest on Online.net

I’ve been running a mixture of FreeBSD / OpenBSD & NetBSD as guests on a dedicated server at Online.net. While getting the operating systems installed was fairly seamless, getting networking going was not.

  1. Client are not isolated in a layer 2 domain
  2. DHCPv6 config is broken

Clients not being isolated is not so much a problem itself and is typically what you’d expect if you plugged a bunch of computers into a switch with a single VLAN or unmanaged switched for example; but in a shared environment with untrusted tenants it can cause problems. Broadcast & IPv6 multicast floods aside, one is open to most of the attacks in something like THC-IPv6 due to lack of MLD snooping which would prevent a rogue IPv6 router.

Attacks via IPv6 are not so much of a problem as their use of non-RFC complaint timers settings in their DHCPv6 make it unfeasible to use the offered native IPv6 connectivity as clients will fail to renew leases. Depending on the DHCPv6 client used, the amount of time it takes fail to renew a lease will vary. dhcpcd for example now warns if detects a lease is not compliant with RFC 3315 section 22.4 “Identity Association for Non-temporary Addresses Option”.

Despite having a vast address range in IPv6 and a /48 subnet is allotted free of charge, you’ll need the equal amount of v4 address addresses as the v6 addresses you intend to use at Online.net. There is a way of using a /48 and allocating addresses yourself but it’s only possible using a version of Proxmox which they provide.

You can save yourself a lot of hassle both with configuration & trying to deal with their support  regarding IPv6 by using a Hurricane Electric tunnel. I actually found connectivity was also faster from Hurricane Electric than using the native connectivity.

For IPv4 connectivity on a guest (assuming you’re renting individual IP addresses & not a /27 prefix), you’ll need to use the default gateway IP address assigned to your host alongside the allotted IP address and a /32 prefix.

Assuming the network details are as follows
Default gateway on host: 192.0.2.1
Failover IP #1: 198.51.100.10, assigned to MAC address 00:50:56:00:01:AA
Failover IP #2: 203.0.113.11, assigned to MAC address 00:50:56:00:02:BB
Failover IP #3: 203.0.113.100, assigned to MAC address 00:50:56:00:03:CC

The MAC addresses need to be assigned to the tap(4) interface on the host.
If you’re using bhyve and your guest is using the interface tap0, this would be performed using the -s flag to configure the virtual PCI ethernet card, eg -s 1:0,virtio-net,tap0,mac=00:50:56:00:01:AA

It’s then onto configuring each OS to handle a gateway which is in a another subnet for IPv4 connectivity.

FreeBSD

In FreeBSD you need to construct a route to reach the default IP address first, before you specify the default IP address, otherwise things will not work. So assuming we’re going to use Failover IP #1, your configuration in /etc/rc.conf would be as follows

ifconfig_vtnet0="inet 198.51.100.10/32"
gateway_if="vtnet0"
gateway_ip="192.0.2.1"
static_routes="gateway default"
route_gateway="-host $gateway_ip -interface $gateway_if"
route_default="default $gateway_ip"

Note, the installer at present prevents network installs, you should use a iso image containing the distfiles, bug 206355 has more details.

NetBSD

On NetBSD, configure networking using /etc/netstart.local, entering the commands you’d enter at the console inside the file. Assuming failover IP #2 is going to be used for the NetBSD VM, the following would configure the guest to reach the outside world using 192.0.2.1, as discussed in the NetBSD Network FAQ

ifconfig vioif0 203.0.113.11/32
route add -net 192.0.2.1 -link -cloning -iface vioif0
route add default -ifa 203.0.113.11 192.0.2.1

OpenBSD

On OpenBSD, configure the networking from the ethernet interfaces configuration file hostname.if(5).

Assuming failover IP #3 is going to be used for the OpenBSD VM, the following will setup networking.

/etc/hostname.vio0

inet 203.0.113.100 255.255.255.255 NONE
!/sbin/route add -net 192.0.2.1 -netmask 255.255.255.255 -link -cloning -iface vio0
!/sbin/route add default -ifa 203.0.113.100 192.0.2.1

It’s also possible to not specify the -cloning flag but a patch is required if you’re running 5.9 release.

The BSD family of operating systems

At OSHUG #46 I was given the opportunity to present the BSD’s to a group of open source hardware enthusiast & speak about why this family of operating systems would benefit the person running a flavour on their hardware. There was a recording made of the talk but it may be some time before it is made available online, so I thought I’d take the time to write something up to share in the meantime.

My slides.

BSD
This line of operating systems started out life as a series of patches to AT&T UNIX which was introduced to the University of Berkeley by Ken Thompson whilst on sabbatical in 1977.
From the 1BSD TAPE file included in the CSRG archive CD set

Berkeley UNIX Software Tape
Jan 16, 1978 TP 800BPI

The first release came with things such as the ex editor, ashell and Pascal compiler as an add-on for UNIX v7, running on a PDP-11. Over the life time of the CSRG they produced releases which included vi, csh, the IPv4 TCP/IP network stack, the virtual memory subsystem (the kernel being named vmunix, parodied by Linux as vmlinuz) and UFS.
The distribution tapes were only available to AT&T licensees; over time the code base of the distribution grew increasingly independent from AT&T UNIX. At the same time the cost of the AT&T license continued to increase as well. Starting out at a cost of $10000 and reaching north of $250000 in the late 80’s. According to Kirk McKusick there was pressure to release the independently developed components of the CSRG so the community could benefit from the use of things such as the network stack without purchasing a costly license. This resulted in several release, comprised mostly of the code developed outside of AT&T such as 4.3BSD-Net/1, Net/2, 4.4BSD-Lite & Lite2. “Mostly” in that with the release of Net/2 AT&T file a lawsuit against the University of California for alleged code copying and theft of trade secrets.
During its lifetime, BSD saw itself being run on several CPU architectures from the DEC PDP-11, VAX to the MIPS, HP 9000 and Motorola 68000 to name a few. These ports along with the  Power 6/32 helped to improve the portability of the code base. The code base was deemed to be 90% platform independent, the remaining 10% being mostly related to the VM subsystem which was platform specific. As with AT&T UNIX, portability & migration between different systems was part of the nature of the code base, from the beginning.

The 4.3BSD-Net/2 code base was used as the basis for a port to the Intel 386, resulting in 386BSD (free) & BSD386 (commercial) releases.

The Modern BSD variants
At the time of writing there are many BSD variants in existence, each with its own area of focus. Everything still leads back to 2 major variants.

NetBSD
NetBSD was the first of the modern variants that is still actively developed. It started out life as a fork of 386BSD. The focus of NetBSD is portability which not only makes porting to new hardware easier (currently supporting over 60 different ports across many CPU architectures).
Everything from a VAX, ARM & MIPS Windows CE based PDAs to a Sega Dreamcast and many other systems are supported and able to run the latest version of NetBSD. There’s even a toaster which runs NetBSD
The focus on portability also makes reusing components on other operating systems easy. For example the packaging system (forked from FreeBSD (which we’ll talk about next)) supports over 20 operating systems.
This enables a consistent toolset to be used regardless of operating system.

Some of the highlights of NetBSD include ATF, unprivileged builds and portable build infrastructure using build.sh.

ATF, as the name suggests is used for automated tests of the source code to discover regression in the code base in an automated manner. Results can be found on the NetBSD release engineering page.

Unprivileged builds allow a user to not only build a copy of the operating systems without elevated privileges, but they can also build and install software from pkgsrc in a location they have write access to (by default, in a prefix under their home directory).

build.sh, the build framework, allows NetBSD to be built on any modern POSIX compliant operating system. Freeing the person to use a operating system of their choice to build releases.

FreeBSD
04/05/2016- Note Ollivier’s comment, I made a mistake when I was gathering info and looked at the source for head and checked the history for the COPYRIGHT file there, not noticing the repository started with v2.0.

Forked from the 4.4BSD Lite code base, 6 months after NetBSD was started. The focus of FreeBSD was performance on i386 systems. Over time support was added for the DEC Alpha as this meant porting the code base to a 64bit systems and addressing any bugs which would prevent the code base from running on a 64bit system. Many years later the project branched out and introduced support for additional platforms. Today the project boasts support for CPUs such as ARMv8, RISC-V and BERI.

OpenBSD
Forked from NetBSD, the focus of OpenBSD is security. The project is home to many components which see wider use outside of OpenBSD, such as OpenSSH, PF (firewall), LibreSSL and others.

DragonFlyBSD
Forked from FreeBSD, the focus of DragonFly BSD is scalability & performance. Taking the operating system in a new direction with regards to how SMP is implemented and from there, developing a new files system called HAMMER.

Documentation
No matter the flavour, documentation is a key part of the development process for the BSD’s.
Whether it is the Design & Implementation series which started with covering 4.3BSD in 1989 and more recently FreeBSD 10 in the fourth instalment of the series, or each projects own set of documentation. Documentation is important as it distinguishes intent & implementation as well as save a lot of question and answers emails.
FreeBSD has handbooks, NetBSD has guides, OpenBSD has FAQs and all projects make their man pages available online as web pages. There is even  a teaching course based around the  The Design and Implementation of the FreeBSD Operating System, 2nd edition.

Frameworks for building embedded images
Each operating system release is a complete, self contained bundle, containing the documentation and necessary toolchain required for building a copy of the operating system from source. release(7) on FreeBSD & NetBSD, release(8) on OpenBSD, nerelease(7) on DragonFlyBSD

For the purpose of embedding the operating system it may not be desirable to build a full blown release. Depending on the choice of variant, either the functionality is built in as standard or a project exists to assist with generating customised images with ease.

FreeBSD had PicoBSD which is now superseded by NanoBSD.
OpenBSD has flashrd and resflash.
NetBSD has a target for generating an image in build.sh, customisations controlled by variables set in mk.conf.
DragonFlyBSD has nrelase.

RetroBSD / LiteBSD
RetroBSD is a port of 2.11BSD (originally targeted for the PDP-11) to the MIPS M4K core found on the PIC32 micro-controllers. LiteBSD is a port of 4.4BSD to the PIC32MZ micro-controllers with a MIPS32 core. Due to the limited resources available, RetroBSD does not offer a network stack, Of the 128KB of RAM, 96KB are available for user space applications. A compiler, editor & various utilities come bundled with the OS so software could be developed on the PIC itself. Variants of common software titles are available to extend the system, such as an Emacs like editor.
LiteBSD is based on a more recent version of BSD, taking advantage of the availability of more RAM (512KB) and MMU on the targeted micro controller. It features a network stack.

Projects such as these take advantage of prior effort and offer the user a consistent environment from the microcontroller to desktop to server. With the extensive documentation and availability of source history, it is possible to realise at which stage in the evolution of the code base the currently running system is and if a desired feature is implemented.

Conclusion
The development of BSD is closely tied with that of the internet. BSD’s modern variants are some of the oldest communities who have collaborated over the internet to develop a software project. The workflow of the projects has transpired to become the standard way of developing open source software on the internet, whether it’s adhering to a style guide or developing with a publicly accessible source repository or holding a hackathon.

For a newcomer interested in an operating system to run on your hardware, it is a great opportunity to be a part of a tech savvy community working to evolve an idea started almost 40 years ago.

As a business, each project produces a mature and robust operating system that has seen many applications from running on devices such as game consoles, mobile phones, cars, satellites and the international space station. Nearly all projects are backed by a non-profit foundation which can act as a liaison for businesses and assist with enquiries regarding development.

A NeXT workstation in Brighton

NeXTstationFor many years I’ve wondered if there were any NeXT systems in my home town. Search results certainly brought up mirrors at University of Brighton many years ago or promotional announcements of classic UNIX workstation vendors such as Solborne for University of Sussex. End of last summer I received a pleasant message from Luke on twitter to say one had been found sitting around in a disused room that had been locked for many years .
It Turns out such systems are still around and finally one is in my hands. Internet friends are the best! 🙂
grazed NeXT logo
Despite the grazed logo, system is in great condition. The status of the system is unknown due to the lack of peripherals. It’s just the slab. By the logo it appears to be a stock mono NeXTstation. I’ll leave it connected to the power before I try and power it up NeXT week!

NeXTstation label

Building a l2tp/IPsec VPN based around a OpenBSD head-end – Part 1

This is the first in a series of posts to cover building a l2tp/IPsec VPN service which remote users (road warriors) connect to.
In this post I will begin with getting OpenBSD setup as the head-end & follow up with subsequent posts to cover configuration of various platforms as clients which compose the road warriors.
Undeadly featured an article on configuring OpenBSD in 2012, things have improved since this article was posted and some of the steps are no longer required, hence I will go over the process again here.

It’s assumed you have an install of OpenBSD running that’s setup as a gateway and communicating on the network, we will continue from there.

The following snippet of config needs to be added to your PF config (/etc/pf.conf by default). It unconditionally permits the IPsec ESP & AH protocols intended for the OpenBSD host, as well as any UDP traffic for ISAKMP and to support NAT traversal.
pass quick proto { esp, ah } from any to self
pass quick proto udp from any to self port {isakmp, ipsec-nat-t} keep state
pass on enc0 from any to self keep state (if-bound)

A minimal PF config which just permits the establishment of a VPN tunnel might look like the following

set skip lo
block return
pass quick proto { esp, ah } from any to self
pass quick proto udp from any to self port {isakmp, ipsec-nat-t} keep state
pass on enc0 from any to self keep state (if-bound)

By only permitting isakmp, it enforces having a working IPsec config before anything else happens whereas permitting UDP port 1701 would permit the establishment of a l2tp tunnel without IPsec which in this scenario would likely be undesired.

A basic IPsec config to use a pre-shared key.The default ciphers used for main & quick mode are documented in ipsec.conf(5). The IP address 1.2.3.4 is configured on the OpenBSD host which connections will be accepted on.

ike passive esp transport proto udp from 1.2.3.4 to any port 1701 psk "password"

Note, the OpenBSD defaults are too high for establishing a connection using the networking preferences on Apple devices and so would need to be restricted down to auth "hmac-sha1" enc "3des" group modp1024 which is not recommended, configuring Apple systems will be covered as a separate article.

The default npppd config (/etc/npppd/nppd.conf) works as-is, without any further changes required. That is unless you prefer to use RADIUS for accounting, instead of local user accounts.

myuser:\
    :password=mypass:\
    :framed-ip-address=10.0.0.111:

npppd is set to use pppx(4) interfaces for established sessions, in order for these interfaces to work correctly, pipex(4) needs to be enabled.

sysctl net.pipex.enable=1

and adding net.pipex.enable=1 to /etc/sysctl.conf so it’s set on boot.

Note, hosts missing this commit (5.8-RELEASE and snapshots from today & prior) will suffer a panic on the OpenBSD host upon establishment of a session by clients, if pipex(4) is not enabled.

Start isakmpd & npppd with

isakmpd -K
npppd

Load your ipsec.conf with
ipsecctl -f /etc/ipsec.conf

Your host should be ready to accept VPN connections, set this services to be started on boot by adding the following to /etc/rc.conf.local
isakmpd_flags="-K"
ipsec=YES
npppd_flags=""

Adventures in Open Source Software: Dealing with Security

I had the opportunity to give this talk at the London chapter of DefCon, DC4420 and censecutivily at London Perl Mongers technical meeting last week.
The subject of the talk was all the factors outside of doing security resceach which can make the process of dealing with advisories a daunting or a seamless process. As observed during the last year, while working as part of a security team.

Slides

Intro:
pkgsrc is a crossplatform packaging system by the NetBSD project, forked from the FreeBSD ports in the late 90’s, initially the primary target was NetBSD but with the portability focus of the project, the list of supported flatforms has grown to a list of 23 operating systems (16 out of those 23 are currently actively worked on). Within the pkgsrc project, there is a dedicated security team whose responsibility is to audit published vulnerabilities and ensure that those which apply to software we offer packages for are listed in a file. Users download this file & use it to check their installed packages.
Other open source projects have teams who are more involved and participate in the security research process and publish their own advisories, such as Debian or Redhat but that is not  main focus of our team.
There may be several reasons for this, during my talk I refered to the aquisition of a security company by Redhat, but looking up Redhat on Wikipedia, there doesn’t appear to be anything to suggest that.
I can say that for the pkgsrc-security team, the role is focused on filtering information and ensuring that items are listed in the pkg-vulnerabilities file, maintainers are notified (if there is one) and co-ordinating with the release engineering team so that necessary commits are pulled into the relevant branches. This is because we try to avoid dealing with development within our tree and opt to co-ordinate with upstream to submit fixes. Majority of our changes focus on removing assumptions to ensure things are built in a consistent manner and allow the software to be packaged how we like.

Dealing with advisories:
The advisories which we receive range in quality / detail.
A personal favourite are the drupal advisories. We offer the drupal core as a package but not any of the 3rd party modules. Their advisories clearly indicate if they apply to core or any published 3rd party modules and which scenario is needed for the vulnerability to be exploitable eg the user must be able to upload content.

The opposite of that is independently published advisories without any co-ordination with affected parties or independently published advisories which are disputed or not acknowledge from upstream. In these situation the role becomes more involved in order to work out if there is clearly an issue or not.

Then there’s Oracle advisories, we can confirm there’s one or more problems in the following versions of software, no more details than that. Upgrade to this version at a minimum to fix said issue(s). Here’s a chart so you can evaluate the risk.

It can be that upstream has actually made an announcement with the details of an issue in public but the mitre website will still lists the CVE as reserved. Ideally you’d like to list the mitre site in pkg-vulnerabilities as it’s where IDs are assigned and it’s self referencing (url will contain the CVE id). But it’s a terrible thing to do to a user. “You have a package installed which is vulnerable to the following type of issue follow this link to not find out any more information about it. Go fish”. Or maybe you have no choice.

Project Websites:
If the published advisory come via a Linux distribution it can be common that the fix references a binary package for users to install or perhaps further information required. In the 2000’s Soureceforge was a popular host for open source projects usually complimented with a separate web page of some kind, it’s now common to have projects which solely exists as an authoritative repo on github. In either scenario, a dedicated section for publishing security information is usually not found. This trend is also prevelant in large commercially backed projects, which play an extremely critical role. Projcts such as ICU (International Components for Unicode), a project by IBM which deals with unicode, an issue in ICU can mean an issue in chrome/chromium, java.

There are also projects like Qemu which have a security page for submitting vulnerability information but never publish advisories themselves. It is common for advisories to reference a git commit email. KVM completely lacks any links related to security. Qemu has a strong link with Xen & KVM which rely on Qemu in one way or another.
While we do not offer KVM as a package, we do at present have four different versions of Xen in our tree and Qemu. This becomes a bit of a timesink when there are multiple advisories to address.

Commercial Repositories:
There are opensource projects with no publicly accessible source code repo. This makes the evaluation of the range of effected verisons difficult if the project only chooses to cover their supported versions.
ISC up until recently (past two years?) required paid membership to access BIND’s repo.
In the talk I refered to the ICU project here, this was incorrect. ICU advisories are either reserved or the bug report access blocked from public view.

OpenSSL:
Relationships with projects are important and they play a critical role in not only sharing information but code as well. Changes for 3rd party software really needs to be passed to the 3rd party to take care of. If relationships have tourned sour, it makes sharing somewhat difficult and has further implications when developing a project with the support for other projects in mind.
The LibreSSL project published a patches page which covered the changes needed to get affected software built but also co-ordinated with upstream projects to get the fixes integrated. Some projects needed more pressure^Wpersuasion than others to accept the patches.

Key components & deadware:
As mentioned previously, you want changes to go back up stream and not to carry changes in your own tree. But there are scenarios where this is possible, for example, the project is no longer developed. This is a huge problem if the project is widely used because you end up carrying local patches which hinders progress when auditing for vulnerabilites by consumers of the software downstream. It’s no longer a case of ensuring you have a specific version number but which patches are also applied to that baseline version, that then opens further questions about the patches, have you created a new issue that didn’t exist previously??
The widely used unzip utility is such an example,  are you patched for CVE-2015-7696?

An example fragmentation of fixes being carried locally is libwmf, with the announcement of some CVEs earlier this year, Jason Unovitch from the FreeBSD project discovered that there were unpatched vulnerabilities in this library going back to 2004, with patches spread across different Linux distributions, none carrying fixes for all advisories, in one case a hunk of the patch didn’t even apply. Development for libwmf stopped in the early 2000’s but it still exists as a project on sourceforge.

Jasper is another commonly used graphics library, this time for jpeg-2000, again development ceased long ago. In this case Slackware put out an advisory for their package to cover vulnerabilities from the past, going back to 2008, at which point we realised that we didn’t have the fixes either. The version in OpenBSD ports was vulnerable to the issues listed from 2014 but the vulnerabilities from prior (2008) were fixed because they’d been flagged up by the compiler in OpenBSD.

Widely Deployed:
Popular projects which have a large install base can greatly increase impact of a mistake, hence local changes should be kept to a minimum to ease maintanence and auditability.
Projects can see a very fast release cycle, especially ones which have advisories published about them regularly.
Keeping local changes to a minimum reduces the necessary effort to update. With projects which rely on downstream consumers to publish information it makes the process more difficult. Both KVM & QEMU projects do not publish any advisories themselves, at best you may have a git commit email which may be the patch you carry locally. Thankfully the Xen project publish advisories on Qemu as it can be a dependency. They are able to flesh out the details of the issue a little better than a vague commit message.
I’m unsure what happens if you’re not a Linux distro and utilise KVM.

Co-ordinating with upstream:
As I mentioned, relationships are important. An understanding and tolerrance for difference is absolutely essential in the world of software just as it is in day to day life. A common topic of disagreement is licensing, the terms expressed by said licenses and the strong opinions expressed by the participants in the disagreement. Whatever ones belief, the need to co-ordinate with people from different groups is absolutely necessary.
Of the fixes upstreamed from LibreSSL, the author of stunnel rejected a fix initially but eventually changed his mind. The change in question was a 2 line addition to add an ifdef statement so that RAND_egd function was only used if the SSL library being linked to offered such a function (detected by autoconf already). The author rejected the change based on terms of licensing of his project when the change was submitted.

Taking bigger leaps in a software project by trying to clean up a popular target can amass a large collection of local patches which need to make their way upstream. As observed by the Alpine Linux project, a Linux distribution with a new libc called musl libc. While it’s possible to build over 13000 packages with Debian 8 on pkgsrc, the package count is less than 9000 on Alpine, despite both being Linux distributions.

The submission process can be quite daunting depending on the project, to filter out submissions which may not be sound and reduce the workload of developers working on a project, some opt to requiring certain things such as results from a test suite or alike. It doesn’t help matters if project has multiple branches developed in parallel without changes being in sync.
Dealing with GNU toochain such as GCC can be very much like this. Again, local changes amass, slow transforming the local version of the toolchain to an extended version of upstream. While the toolchain may offer security features such as SSP (stack smashing protection), it’s not just the simple case of being able to switch it on, in some cases it either doesn’t work or worse, it results in broken binaries. Work to enable some of these features in pkgsrc began in the summer.

While not specifically security related, I was reminded of an incident on the OpenBSD mailing lists with the author of the once popular ion window manager. The ion developer was frustrated with operating system projects packaging older versions of his software with local changes as users where following up with him for support (in this case he was referring to RedHat). So he came on to the OpenBSD mailing list to ask about updating the software to the latest version available but that was not possible due to incompatible license changes. As usual, the combination of making demanding + licensing drama didn’t work out to well.

Conclusion

  • Organise the information on your site, for an Open Source software project your source control repo should be a single click away from your websites main page.
  • Provide a dedicated security page or section where you post advisories.
  • Write descriptive commit messages.
  • Participate in the community and help your neighbours
  • Even if you stop developing software, the code may live on longer than envisioned, think about what happens if/when you decide to stop (who become the authoritative repo).
  • Don’t make changes locally which do not go upstream by default, for it’ll surely bite you or a member of the project later down the line.
  • Publish actual advisories for your project, don’t pass the buck.
  • Technical problems are best solved with technical solutions eg a bug can still continue to exist despite adhering to a license.
  • Make the submission process to your project effortless for both parties, not just one or the other.

Unable to mount or open disk images generated with Nero (.nrg file)

It appears that VirtualBox & OS X are unable to open .nrg files, despite them essentially being a ISO 9660 format file.

VirtualBox reports:
Result Code:
VBOX_E_IPRT_ERROR (0x80BB0005)
Component:
MediumWrap
Interface:
IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda}
Callee:
IVirtualBox {0169423f-46b4-cde9-91af-1e9d5b6cd945}
Callee RC:
VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)

Finder reports:
image not recognised

This turns out to be due to a footer added by Nero which may make the file size something which in not a the sum of a multiple of 2K.

Editing the file in a hex editor and removing the footer (of 72 bytes) should result in the file being usable

28633000 45 54 4e 32 00 00 00 20 00 00 00 00 00 00 00 00 |ETN2... ........|
28633010 00 00 00 00 28 63 30 00 00 00 00 00 00 00 00 00 |....(c0.........|
28633020 00 00 00 00 00 00 00 00 4d 54 59 50 00 00 00 04 |........MTYP....|
28633030 00 00 00 01 45 4e 44 21 00 00 00 00 4e 45 52 35 |....END!....NER5|
28633040 00 00 00 00 28 63 30 00 |....(c0.|
28633048