Tag: OpenBSD

Posted on

Introducing RunBSD.info

Update ~2018 – This project is now maintained by Mischa Peters and Roman Zolotarev on a new Github org, hosted on OpenBSD.Amsterdam.

RunBSD.info is an introductory site covering the features & benefits across the different members of the BSD family of operating systems. To get things started, the content for the site was taken from an earlier post originally put together with a specific audience in mind. That audience was a group of hardware enthusiasts who are not necessarily interested in software but may require an operating system to run. This is no longer a constraint in this scenario and so will now move to cover things in a generic sense,  allowing wider coverage of highlights in each operating system. There is a lot of room for improvement.

The site is hosted on Github and the content is composed of markdown text files. Contributions welcome by pull request or If you prefer not to use Github, patches against repo files or just plain text content via email gladly accepted. You can reach me on venture37 at the domain of this blog post.

Thanks to FiLiS for the logo.

Posted on

Lessons learnt from adding OpenBSD/x86_64 support to pkgsrc

Before even getting into the internals of operating systems to learn about differences among a group of operating systems, It’s fairly evident that something as simple as naming is different between operating systems.

For example, the generations of trusty 32bit x86 PC is commonly named i386 in most operating systems, FreeBSD may also refer to it as just pc, Solaris & derivatives refer to it as i86pc, Mac OS X refers to it as i486 (NeXTSTEP never ran on a 386, it needed a minimum of a 486 and up until Sierra, machine(1) would report i486 despite being on a Core i7 system), this is one of the many architectures which needed to hadled within pkgsrc. To simplify things and reduce lengthy statements, all variants for an arch are translated to a common name whiche is then used for reference in pkgsrc. This means that all the examples above are grouped together under the MACHINE_ARCH i386. In the case of 64bit x86 or commonly referred to as amd64, we group under x86_64 or at least tried to. The exception to this grouping was OpenBSD/amd64, this resulted in the breakage of many packages because any special attention required was generally handled under the context of MACHINE_ARCH=x86_64. In some packages, developers had added a new exception for MACHINE_ARCH=amd64 when OPSYS=OPENBSD but it was not a sustainable strategy because to be affective, the entire tree would need to be handled. I covered the issue at the time in A week of pkgsrc #11 but to summarise, $machine_arch may be set at the start in the bootstrap script but as the process works through the list of tasks, the value of this variable is overriden despite being passed down the chain at the begining of a step. After some experimentation and the help of Jonathan Perkin, the hurdles were removed and thus OpenBSD/x86_64 was born in pkgsrc 😉

The value of this exercise for me was that I learnt the number of places within the internals of pkgsrc I could set something (by the nature of coupling components which share the same conventions (pkgtools, bsd make)) and really the only place I should be seeking to set something is at the start of the process and have that carry through, rather than trying to short circuit the process and repeat myself.

Thanks to John Klos, I was given control of a IBM Power 8+ S822LC running Ubuntu, which started setting up for pkgsrc bulk builds.
First issue I hit was pkgsrc not being able to find libc.so, this turned out to be the lack of handling for the multilib paths found on Debian & derivates for PowerPC based systems.
This system is a little endian 64bit PowerPC machine which is a new speciality in itself and so I set out to make my first mistake. Adding a new check for the wrong MACHINE_ARCH, long forgotten about the previous battle with OpenBSD/x86_64 I added a new statment to resolve the relevant paths for ppc64le systems. Bootstrap was happy with that & things moved forward. At this point I was pointed to lang/python27 most likely being borken by Maya Rashish, John had previously reported the issue and we started to poke at things. As we started rummaging through the internals of pkgsrc (pkgsrc/mk) I started to realise we’re heading down the wrong path of marking things up in multiple places again, rather than setting things once & propogating through.

It turned out that I only need to make 3 changes to add support for Linux running on little endian 64bit PowerPC to pkgsrc (2 additions & 1 correction 😉 )
First, add a case in the pkgsrc/bootstrap/bootstrap script to set $machine_arch to what we want to group under when the relevant machine type is detected. In this case it was when Linux running on a ppc64le host, set $machine_arch to powerpc64le. As this is a new machine arch, also ensure it’s listed in the correct endianness category of pkgsrc/mk/bsd.prefs.mk, in this case add powerpc64le to _LITTLEENDIANCPUS.
Then correct the first change to replace the reference to ppc64le for handling the multilib paths in pkgsrc/mk/platform/Linux.mk.

The bulkbuild is still in progress as I write this post but 5708/18148 packages in an the only fall out so far appears to be the ruby interepreters.

Posted on

12″ PowerBook G4 PT5 – Electronic Battle Weapon

Preparation for a trip started off a little earlier this christmas. I planned to take my PowerBook on the road with me to Hamburg for 33c3. Previous attempts to use this machine as my primary system on the road in the past had been thwarted by leaving too little time to build & prepare before departure.
The system has been dual booting NetBSD & Mac OS X Tiger for some time now, recently I’ve been doing almost daily upgrades to NetBSD-HEAD on the system using the generated iso images from NYFTP.
My plan was to get the machine installed with a current build FireFox on NetBSD & bring the existing installed packages up to date. I managed to update the existing packages without any problems but it didn’t look like FireFox was going to build successfully. The package as-is currently in pkgsrc does not build on NetBSD/macppc. I was pointed to a patch in pkg/48595 which was pending commit and required testing. It cleared up the initial issue I ran into but the build still failed (see previous link on updates about the failure), though it took a little longer to fail in the day. After several days of failed build attempts I made sure I had an up to date copy of TenFourFox installed on Tiger and settled for Dillo on NetBSD instead.

My usage of Dillo stayed somewhat basic during the trip, despite having the Mozilla certificate bundle installed, I could see any obvious way to point Dillo to it & have it use it. Hence, any site using SSL I visited generated a certificate warning. Perhaps the config should’ve been done in wget?

www/dillo pkgdepgraph

Alexander Nasonov created packages for Dillo & Links-gui targeted for running under a minimal chroot but I did not get around to trying them out. There are chrooted browser packages for other browsers in his pkgsrc github repo. The screenshot above shows the www/dillo package’s dependencies, generated using pkgtools/pkgdepgraph

Moving on, the AirPort Extreme card in the laptop is based on a Broadcom chipset which has a flaw, it’s incapable of addressing memory above 1GB (30 bits) which means the driver needs to care for that or else the card doesn’t work. This is not unique to this Broadcom chipset, the BCM4401 10/100 ethernet interfaces which use the bce(4) driver also suffer from the same problem (unable to address memory allocated above 30 bits), the BCM580x ethernet interfaces which use the bge(4) driver suffer from not being able to address more than 40 bits. Going back to the wireless chipset, the bwi(4) driver which is used in the BSDs, originated from DragonFly BSD. This driver was put together by Sepherosa Ziehau using the documentation from a reversing effort in the Linux community. The bwi driver was then imported in to Free/Open/NetBSD and was eventually removed from DragonFly BSD. A new wireless subsystem was introduced in DragonFly which required change to drivers to work again and the bwi driver was never adapted. It now lives on in the other BSDs.

The version of bwi(4) driver came to NetBSD from OpenBSD, ported by Taylor R. Campbell back in 2009. At the time neither version of drivers could handle the 30 bit bug so you either ran with less than 1GB of RAM or used another card. In 2014 Stefan Sperling committed a workaround for this in OpenBSD. I wanted this fix in NetBSD so my wifi could also work & asked the NetBSD developers if such a change was appropriate in NetBSD. I was introduced to bus_dma(9) and the bus_dmatag_subregion() function, the bce(4) driver was my reference on how to use the function. Looked fairly straight forward, a single call this function and off you go, wasn’t too sure how it would fit into the bwi driver but I thought I’d have a go.

This was one of the things I was hoping to work on during my trip but It turned out to be the only thing I attempt. I happened to meet Stefan at 33c3 and we discussed the driver, the work around and the mighty days of the past when Damien Bergamini was hacking on the OpenBSD WiFi stack. In the OpenBSD driver Stefan had opted to deal with the issue of allocating memory in a specific region directly in the driver rather than adding a new interface to the kernel for  such a task so with a bit of thought about the past and a review of the driver, I was given a diff of the changes and suggestions about where I could start making changes.

I still don’t know yet if it’s possible to lift the changes from OpenBSD and apply them to the NetBSD version of the driver, because the DMA framework is different between the systems.
Partially implementing the change Stefan made without all the bounce buffers he’d added in the OpenBSD driver didn’t work and using the bus_dmatag_subregion() function didn’t work either. I pursued the bus_dmatag_subregion() path during 33c3 and didn’t get anywhere. At this point I started looking deeper in the system by looking at the implementation. It was at this point that I discovered this function was defined to EOPNOTSUPP on PowerPC based systems. No matter what I had tried with this function it was a waste of time^W^W^Wvaluable learning experience about keeping documentation up to date & consistent.

At this point I started looking into adding support for tagged subregions so I could make use of the function. The implementation is fairly simple, a public function for a developer to use which performs various tests and a private function which is called to deal with the memory allocation. Unfortunately there were some missing members from the data structure on the powerpc side of NetBSD which needed further investigation and I stopped there for the time being.

For the trip I relied on a tiny Realtek RTL8188CUS based wifi adapter to get me network access. The card worked on the 802.1x enabled SSID at 33c3 using wpa_supplicant(8) on NetBSD/maccppc.

urtwn0 at uhub4 port 5
urtwn0: Planex Communications Inc. GW-USNANO2, rev 2.00/2.00, addr 2
urtwn0: MAC/BB RTL8188CUS, RF 6052 1T1R, address 00:22:cf:xx:xx:xx
urtwn0: 1 rx pipe, 2 tx pipes
urtwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
urtwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps

The driver for this card is now enabled by default in the GENERIC config file for NetBSD/macppc along with a group of other drivers for USB peripherals.

Thanks to Stefan for his help and advice with the bwi driver and Alex for the chrooted browser packages! 🙂

Posted on

Videos from OSHUG #46

Earlier this year, I gave a short talk at OSHUG about the BSD family of operating systems to a group of open hardware enthusiasts. I previously wrote about the event here
The videos from the event are now available on youtube


Posted on

A slow / low-end system capable of running most modern BSDs

I was looking to test a change related to buffering in cat(1) and wondered what was the slowest system I could use which was capable of running the current versions of NetBSD, FreeBSD, OpenBSD. An old PC and the ARM based BeagleBone Black sprang to mind immediately, then a PowerPC Mac? SPARC64?

Apart from a Sun Fire T1000, I do not have any SPARC hardware, sun4v is only supported on NetBSD & OpenBSD at present, FreeBSD/sun4v was only a pre-alpha rough cut from before the days of version 7 and sparc64 support may be going away in FreeBSD moving forward.

Considered the BeagleBone Black but currently NetBSD-HEAD does not boot on it port-arm/51380 and FreeBSD has issues with running DTrace bug/211389. So that was off the list.

A G4 based PowerPC Mac is supported between my choice of BSDs, unfortunately I couldn’t get a working disk burnt from the FreeBSD iso files to try it out on a 12″ PowerBook. bug/211488.

I settled on running i386 builds on a Alix 2c3 I have, it has 256MB RAM and a 500Mhz Geode CPU, currently running FreeBSD/i386 11-BETA3 without issue and has no problems with any of the other BSDs. It’s a little too “modern” and high spec though in my test.

Posted on

Running FreeBSD / OpenBSD / NetBSD as a virtualised guest on Online.net

I’ve been running a mixture of FreeBSD / OpenBSD & NetBSD as guests on a dedicated server at Online.net. While getting the operating systems installed was fairly seamless, getting networking going was not.

  1. Client are not isolated in a layer 2 domain
  2. DHCPv6 config is broken

Clients not being isolated is not so much a problem itself and is typically what you’d expect if you plugged a bunch of computers into a switch with a single VLAN or unmanaged switched for example; but in a shared environment with untrusted tenants it can cause problems. Broadcast & IPv6 multicast floods aside, one is open to most of the attacks in something like THC-IPv6 due to lack of MLD snooping which would prevent a rogue IPv6 router.

Attacks via IPv6 are not so much of a problem as their use of non-RFC complaint timers settings in their DHCPv6 make it unfeasible to use the offered native IPv6 connectivity as clients will fail to renew leases. Depending on the DHCPv6 client used, the amount of time it takes fail to renew a lease will vary. dhcpcd for example now warns if detects a lease is not compliant with RFC 3315 section 22.4 “Identity Association for Non-temporary Addresses Option”.

Despite having a vast address range in IPv6 and a /48 subnet is allotted free of charge, you’ll need the equal amount of v4 address addresses as the v6 addresses you intend to use at Online.net. There is a way of using a /48 and allocating addresses yourself but it’s only possible using a version of Proxmox which they provide.

You can save yourself a lot of hassle both with configuration & trying to deal with their support  regarding IPv6 by using a Hurricane Electric tunnel. I actually found connectivity was also faster from Hurricane Electric than using the native connectivity.

For IPv4 connectivity on a guest (assuming you’re renting individual IP addresses & not a /27 prefix), you’ll need to use the default gateway IP address assigned to your host alongside the allotted IP address and a /32 prefix.

Assuming the network details are as follows
Default gateway on host: 192.0.2.1
Failover IP #1: 198.51.100.10, assigned to MAC address 00:50:56:00:01:AA
Failover IP #2: 203.0.113.11, assigned to MAC address 00:50:56:00:02:BB
Failover IP #3: 203.0.113.100, assigned to MAC address 00:50:56:00:03:CC

The MAC addresses need to be assigned to the tap(4) interface on the host.
If you’re using bhyve and your guest is using the interface tap0, this would be performed using the -s flag to configure the virtual PCI ethernet card, eg -s 1:0,virtio-net,tap0,mac=00:50:56:00:01:AA

It’s then onto configuring each OS to handle a gateway which is in a another subnet for IPv4 connectivity.

FreeBSD

In FreeBSD you need to construct a route to reach the default IP address first, before you specify the default IP address, otherwise things will not work. So assuming we’re going to use Failover IP #1, your configuration in /etc/rc.conf would be as follows

ifconfig_vtnet0="inet 198.51.100.10/32"
gateway_if="vtnet0"
gateway_ip="192.0.2.1"
static_routes="gateway default"
route_gateway="-host $gateway_ip -interface $gateway_if"
route_default="default $gateway_ip"

Note, the installer at present prevents network installs, you should use a iso image containing the distfiles, bug 206355 has more details.

NetBSD

On NetBSD, configure networking using /etc/netstart.local, entering the commands you’d enter at the console inside the file. Assuming failover IP #2 is going to be used for the NetBSD VM, the following would configure the guest to reach the outside world using 192.0.2.1, as discussed in the NetBSD Network FAQ

ifconfig vioif0 203.0.113.11/32
route add -net 192.0.2.1 -link -cloning -iface vioif0
route add default -ifa 203.0.113.11 192.0.2.1

OpenBSD

On OpenBSD, configure the networking from the ethernet interfaces configuration file hostname.if(5).

Assuming failover IP #3 is going to be used for the OpenBSD VM, the following will setup networking.

/etc/hostname.vio0

inet 203.0.113.100 255.255.255.255 NONE
!/sbin/route add -net 192.0.2.1 -netmask 255.255.255.255 -link -cloning -iface vio0
!/sbin/route add default -ifa 203.0.113.100 192.0.2.1

It’s also possible to not specify the -cloning flag but a patch is required if you’re running 5.9 release.

Posted on

The BSD family of operating systems

At OSHUG #46 I was given the opportunity to present the BSD’s to a group of open source hardware enthusiast & speak about why this family of operating systems would benefit the person running a flavour on their hardware. There was a recording made of the talk but it may be some time before it is made available online, so I thought I’d take the time to write something up to share in the meantime.

My slides.

BSD
This line of operating systems started out life as a series of patches to AT&T UNIX which was introduced to the University of Berkeley by Ken Thompson whilst on sabbatical in 1977.
From the 1BSD TAPE file included in the CSRG archive CD set

Berkeley UNIX Software Tape
Jan 16, 1978 TP 800BPI

The first release came with things such as the ex editor, ashell and Pascal compiler as an add-on for UNIX v7, running on a PDP-11. Over the life time of the CSRG they produced releases which included vi, csh, the IPv4 TCP/IP network stack, the virtual memory subsystem (the kernel being named vmunix, parodied by Linux as vmlinuz) and UFS.
The distribution tapes were only available to AT&T licensees; over time the code base of the distribution grew increasingly independent from AT&T UNIX. At the same time the cost of the AT&T license continued to increase as well. Starting out at a cost of $10000 and reaching north of $250000 in the late 80’s. According to Kirk McKusick there was pressure to release the independently developed components of the CSRG so the community could benefit from the use of things such as the network stack without purchasing a costly license. This resulted in several release, comprised mostly of the code developed outside of AT&T such as 4.3BSD-Net/1, Net/2, 4.4BSD-Lite & Lite2. “Mostly” in that with the release of Net/2 AT&T file a lawsuit against the University of California for alleged code copying and theft of trade secrets.
During its lifetime, BSD saw itself being run on several CPU architectures from the DEC PDP-11, VAX to the MIPS, HP 9000 and Motorola 68000 to name a few. These ports along with the  Power 6/32 helped to improve the portability of the code base. The code base was deemed to be 90% platform independent, the remaining 10% being mostly related to the VM subsystem which was platform specific. As with AT&T UNIX, portability & migration between different systems was part of the nature of the code base, from the beginning.

The 4.3BSD-Net/2 code base was used as the basis for a port to the Intel 386, resulting in 386BSD (free) & BSD386 (commercial) releases.

The Modern BSD variants
At the time of writing there are many BSD variants in existence, each with its own area of focus. Everything still leads back to 2 major variants.

NetBSD
NetBSD was the first of the modern variants that is still actively developed. It started out life as a fork of 386BSD. The focus of NetBSD is portability which not only makes porting to new hardware easier (currently supporting over 60 different ports across many CPU architectures).
Everything from a VAX, ARM & MIPS Windows CE based PDAs to a Sega Dreamcast and many other systems are supported and able to run the latest version of NetBSD. There’s even a toaster which runs NetBSD
The focus on portability also makes reusing components on other operating systems easy. For example the packaging system (forked from FreeBSD (which we’ll talk about next)) supports over 20 operating systems.
This enables a consistent toolset to be used regardless of operating system.

Some of the highlights of NetBSD include ATF, unprivileged builds and portable build infrastructure using build.sh.

ATF, as the name suggests is used for automated tests of the source code to discover regression in the code base in an automated manner. Results can be found on the NetBSD release engineering page.

Unprivileged builds allow a user to not only build a copy of the operating systems without elevated privileges, but they can also build and install software from pkgsrc in a location they have write access to (by default, in a prefix under their home directory).

build.sh, the build framework, allows NetBSD to be built on any modern POSIX compliant operating system. Freeing the person to use a operating system of their choice to build releases.

FreeBSD
04/05/2016- Note Ollivier’s comment, I made a mistake when I was gathering info and looked at the source for head and checked the history for the COPYRIGHT file there, not noticing the repository started with v2.0.

Forked from the 4.4BSD Lite code base, 6 months after NetBSD was started. The focus of FreeBSD was performance on i386 systems. Over time support was added for the DEC Alpha as this meant porting the code base to a 64bit systems and addressing any bugs which would prevent the code base from running on a 64bit system. Many years later the project branched out and introduced support for additional platforms. Today the project boasts support for CPUs such as ARMv8, RISC-V and BERI.

OpenBSD
Forked from NetBSD, the focus of OpenBSD is security. The project is home to many components which see wider use outside of OpenBSD, such as OpenSSH, PF (firewall), LibreSSL and others.

DragonFlyBSD
Forked from FreeBSD, the focus of DragonFly BSD is scalability & performance. Taking the operating system in a new direction with regards to how SMP is implemented and from there, developing a new files system called HAMMER.

Documentation
No matter the flavour, documentation is a key part of the development process for the BSD’s.
Whether it is the Design & Implementation series which started with covering 4.3BSD in 1989 and more recently FreeBSD 10 in the fourth instalment of the series, or each projects own set of documentation. Documentation is important as it distinguishes intent & implementation as well as save a lot of question and answers emails.
FreeBSD has handbooks, NetBSD has guides, OpenBSD has FAQs and all projects make their man pages available online as web pages. There is even  a teaching course based around the  The Design and Implementation of the FreeBSD Operating System, 2nd edition.

Frameworks for building embedded images
Each operating system release is a complete, self contained bundle, containing the documentation and necessary toolchain required for building a copy of the operating system from source. release(7) on FreeBSD & NetBSD, release(8) on OpenBSD, nerelease(7) on DragonFlyBSD

For the purpose of embedding the operating system it may not be desirable to build a full blown release. Depending on the choice of variant, either the functionality is built in as standard or a project exists to assist with generating customised images with ease.

FreeBSD had PicoBSD which is now superseded by NanoBSD.
OpenBSD has flashrd and resflash.
NetBSD has a target for generating an image in build.sh, customisations controlled by variables set in mk.conf.
DragonFlyBSD has nrelase.

RetroBSD / LiteBSD
RetroBSD is a port of 2.11BSD (originally targeted for the PDP-11) to the MIPS M4K core found on the PIC32 micro-controllers. LiteBSD is a port of 4.4BSD to the PIC32MZ micro-controllers with a MIPS32 core. Due to the limited resources available, RetroBSD does not offer a network stack, Of the 128KB of RAM, 96KB are available for user space applications. A compiler, editor & various utilities come bundled with the OS so software could be developed on the PIC itself. Variants of common software titles are available to extend the system, such as an Emacs like editor.
LiteBSD is based on a more recent version of BSD, taking advantage of the availability of more RAM (512KB) and MMU on the targeted micro controller. It features a network stack.

Projects such as these take advantage of prior effort and offer the user a consistent environment from the microcontroller to desktop to server. With the extensive documentation and availability of source history, it is possible to realise at which stage in the evolution of the code base the currently running system is and if a desired feature is implemented.

Conclusion
The development of BSD is closely tied with that of the internet. BSD’s modern variants are some of the oldest communities who have collaborated over the internet to develop a software project. The workflow of the projects has transpired to become the standard way of developing open source software on the internet, whether it’s adhering to a style guide or developing with a publicly accessible source repository or holding a hackathon.

For a newcomer interested in an operating system to run on your hardware, it is a great opportunity to be a part of a tech savvy community working to evolve an idea started almost 40 years ago.

As a business, each project produces a mature and robust operating system that has seen many applications from running on devices such as game consoles, mobile phones, cars, satellites and the international space station. Nearly all projects are backed by a non-profit foundation which can act as a liaison for businesses and assist with enquiries regarding development.

Posted on

Building a l2tp/IPsec VPN based around a OpenBSD head-end – Part 1

This is the first in a series of posts to cover building a l2tp/IPsec VPN service which remote users (road warriors) connect to.
In this post I will begin with getting OpenBSD setup as the head-end & follow up with subsequent posts to cover configuration of various platforms as clients which compose the road warriors.
Undeadly featured an article on configuring OpenBSD in 2012, things have improved since this article was posted and some of the steps are no longer required, hence I will go over the process again here.

It’s assumed you have an install of OpenBSD running that’s setup as a gateway and communicating on the network, we will continue from there.

The following snippet of config needs to be added to your PF config (/etc/pf.conf by default). It unconditionally permits the IPsec ESP & AH protocols intended for the OpenBSD host, as well as any UDP traffic for ISAKMP and to support NAT traversal.
pass quick proto { esp, ah } from any to self
pass quick proto udp from any to self port {isakmp, ipsec-nat-t} keep state
pass on enc0 from any to self keep state (if-bound)

A minimal PF config which just permits the establishment of a VPN tunnel might look like the following

set skip lo
block return
pass quick proto { esp, ah } from any to self
pass quick proto udp from any to self port {isakmp, ipsec-nat-t} keep state
pass on enc0 from any to self keep state (if-bound)

By only permitting isakmp, it enforces having a working IPsec config before anything else happens whereas permitting UDP port 1701 would permit the establishment of a l2tp tunnel without IPsec which in this scenario would likely be undesired.

A basic IPsec config to use a pre-shared key.The default ciphers used for main & quick mode are documented in ipsec.conf(5). The IP address 1.2.3.4 is configured on the OpenBSD host which connections will be accepted on.

ike passive esp transport proto udp from 1.2.3.4 to any port 1701 psk "password"

Note, the OpenBSD defaults are too high for establishing a connection using the networking preferences on Apple devices and so would need to be restricted down to auth "hmac-sha1" enc "3des" group modp1024 which is not recommended, configuring Apple systems will be covered as a separate article.

The default npppd config (/etc/npppd/nppd.conf) works as-is, without any further changes required. That is unless you prefer to use RADIUS for accounting, instead of local user accounts.

myuser:\
    :password=mypass:\
    :framed-ip-address=10.0.0.111:

npppd is set to use pppx(4) interfaces for established sessions, in order for these interfaces to work correctly, pipex(4) needs to be enabled.

sysctl net.pipex.enable=1

and adding net.pipex.enable=1 to /etc/sysctl.conf so it’s set on boot.

Note, hosts missing this commit (5.8-RELEASE and snapshots from today & prior) will suffer a panic on the OpenBSD host upon establishment of a session by clients, if pipex(4) is not enabled.

Start isakmpd & npppd with

isakmpd -K
npppd

Load your ipsec.conf with
ipsecctl -f /etc/ipsec.conf

Your host should be ready to accept VPN connections, set this services to be started on boot by adding the following to /etc/rc.conf.local
isakmpd_flags="-K"
ipsec=YES
npppd_flags=""

Posted on

A week of pkgsrc #12

To fill in the gap since the last post, I thought I’d get the notes which had been collecting up, posted here. pkgsrc got a mention in the Quarterly FreeBSD status report. My bulkbuild effort started on FreeBSD/amd64 10.1-RELEASE but thanks to my friend James O’Gorman, I was able to expand to FreeBSD 11-CURRENT and recently switched over from 10.1-RELEASE to 10.2-RELEASE.
I got the idea to try to pkgsrc on Android after someone posted a screenshot of their Nexus 7 tablet with the bootstrap process completed.

There are several projects on the google play store for running the user land built from a Linux/arm distro in a chroot on Android.
The first project I tried was Debian noroot (based on the tweet that inspired me), it spawned a full X11 desktop to run & so the process was painfully slow.

Switching to GNUroot Debian which just ran a shell in the chroot was much faster at extracting the pkgsrc archive though bootstrap still took long. The best result was with Linux deploy using an Arch Linux user land, everything was very snappy.

On Mac OS X Tiger PowerPC, GCC 5 appears to no longer require switching off multilib support when building on a 32-bit PowerPC CPU, my hardware has changed but the CPU is still a G4. The same changes to force dwarf2 and removing the space in-between flags and paths fed to the linker were otherwise required, as with previous versions of GCC.

I spent a little time with OmniOS and “addressed” the outstanding issues which prevented it from working out of the box. shells/standalone-tcsh was excluded on OmniOS which prevented the version of tcsh shipped with the OS from being clobbered during bulkbuilds. The other issue was what appeared to be a problem with gettext but turned to be an issue with the compiler shipped with OmniOS. This became a topic of discussion on what the correct solution to the problem is. The GCC provided with OmniOS is built with Fortran support and includes the OpenMP libraries (I’m guessing this is the reason for the libraries) in its private lib directory inside /opt/gcc-4.8.1/lib, it turns out that gettext will make use of OpenMP libraries if it detects them during configure stage which I’ve not been able to find a concrete answer for why, the GCC documentation don’t say more than a paragraph about the OpenMP libraries themselves (libgomp) either. The problem was that GCC was exposing its private library in the link path but not in the run path, this meant you could produce binaries which would compile fine but would not run without having to play around with the runtime linker. In my case I’d previously added the private library locate to the runtime linkers search path as a workaround, I disabled the OpenMP support in devel/gettext-tools and that’s where the discussion began. Basically, it’s not possible to expose the private library location to the linker because that would cause issues with upgrades. The location should not be exposed by the compiler in the first place (I guess this was for the convenience of building the actual release of OS?). Richard Palo pursued the issue further and I’m informed that future releases of OmniOS will move libgomp out from this private location to /usr/lib so that it’s in the default library search path.

With the introduction of the GPLv3 license, GNU projects have been switching to the new license. This causes problems for projects outside the GNU eco-system which utilise them if the terms of the new license are unacceptable for them. Each project has dealt with it differently, for OpenBSD they maintain the last version which was available under GPLv2 & extend the functionality it provides. Bitrig has inherited some of this through the fork. Through the bulkbuilds it was revealed that the upstream version of binutils has no support for OpenBSD/amd64 or Bitrig at all. Adding rudimentary support was easily achieved by lifting some of the changes from the OpenBSD CVS repo. While at present I’m running bulkbuilds against a patched devel/binutils which I’ve not upstreamed or committed for both OpenBSD & Bitrig, I am thinking that for OpenBSD we should actually just use the native version and not attempt to build the package. For Bitrig, there is already a separate package in their ports tree for a newer version of binutils, it’s pulled in alongside other modern versions of tools under the meta/bitrig-syscomp package so it makes sense to mimic that behaviour.

Coming to the realisation that stock freedesktop components were not going to build on OpenBSD, I switched to using X11_TYPE=native to utilise what’s provided by Xenocara. Despite the switch, pkgsrc still attempted to ignore the native version of MesaLib and try to build its own, the build would fail and prevent a couple of thousand packages from building.
This turned out to be because of a test to detect the presence of X11 in mk/defaults/mk.conf, it was testing for the presence of an old path which no longer exists. As this test would fail, the native components would be ignored & pkgsrc components would be preferred. The tests for OpenBSD & Bitrig were removed & now default to a default of an empty PREFER_PKGSRC variable. The remaining platforms need to be switched over after testing now.

As Mac OS X on PowerPC gets older and older with time, the requirement for defining MACOSX_DEPLOYMENT_TARGET grows ever more redundant, Ruby now ships with it & unless it’s defined, you will find that it’s not possible to build the ruby interpreter any more. I am considering setting MACOSX_DEPLOYMENT_TARGET="10.4" for PowerPC systems running Tiger or Leopard so that packages could be shared between the two but have not had a chance to test on Leopard yet to commit it. I somehow ended up on a reply list for a ticket in the Perl RT for dealing with this exact issue there. They opted to cater for both legacy & modern version of OS X by setting the necessary variables where necessary.

Getting through backlogged notes to be continued

Posted on

A week of pkgsrc #11

It’s been a while since the last post in the series, the details of what was covered in these posts was the partial basis of my talk at BSDCan and I got to repeat the talk again in Berlin, I was much less nervous the second time, not having a fire alarm going off during the talk may have helped. I will cover briefly some things that were mentioned in the talks which I hadn’t written up here, for the sake of completeness.
Thanks to the DragonFlyBSD folks, I have access to a build server for doing regular bulkbuilds on. As I’m running these as a unprivileged user, there’s not much parallelism in the package builds, it’s one package at a time. The system aptly named Monster is a 48 Opteron CPU server with a 128GB of RAM so I can at least run with MAKE_JOBS set to 96. At the start of the bulkbuilds some deadlock issues in DragonFlyBSD were revealed by pkgsrc which Mat Dillon addressed promptly.

On the Bitrig front, I managed to add support for the OS to lang/python27 which was the package causing the biggest breakage and now in the process of trying to get the support added upstream, there appears to be a bug report from 2013 in the Python bug tracker to add support ubut it was marked as won’t fix, I’m hoping the decision will be changed but will have to wait and see.
With Python 2.7 built successfully it was onto the next set of breakages, gettext!
I had taken a patch from OpenBSD ports for getting devel/gettext-tools building but was asked to back it out as it was not the correct solution to the problem. I decided to reapply the fix in my build just to progress to the next hurdle. The next major breakage was with devel/p5-gettext which needed to be told to include libiconv, I’m now stuck at getting converters/help2man building.
During this process I found that we were missing some necessary flags for creating shared libraries which were highlighted by clang:
relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC

This turned out to be bug in the platform support, the necessary fPIC flags were defined but under a if statement for version of OS running with a.out binaries still. mk/platform/Bitrig.mk was stripped of anything related to a.out and everything was rebuilt again from scratch.

OpenBSD and Bitrig probably have many more breakages due to the fact that their architecture is detected as amd64 and not under the x86_64 banner by the build system. One example is x11/libdrm which is set to add sysutils/libpciaccess as a dependency if the host is a i386 or x86_64.
At present libdrm fails at the configure stage with
checking for PCIACCESS... no
configure: error: Package requirements (pciaccess >= 0.10) were not met:

No package 'pciaccess' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables PCIACCESS_CFLAGS
and PCIACCESS_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
Trying to add OpenBSD to the x86_64 arch list revealed a problem in pkgsrc, the culprit being devel/bmake.
The problem is that there are three separate points where the architecture is defined. In the bootstrap script, in BSDMake’s on source and the settings it passes onto pkgtools/pkg_install. Unfortunately the settings defined at the start in bootstrap are ignored at the bootstrap stage & are not necessarily what pkg_install is built with. To add to this, it’s possible that BSDMake may need to work out what the system is for itself rather than to be expected to have settings passed to itself. That is they should build with settings passed down in succession or independently.
With severe bludgeoning of code between devel/bmake and pkgtools/pkg_install, I managed to get it to
pkg_add: OpenBSD/x86_64 5.7 (pkg) vs. OpenBSD/amd64 5.7 (this host)
pkg_install performs a check of the OS it’s running on against the settings it was built with (the settings bmake passed it during bootstrap), removing the check revealed there was nothing else preventing things from working but the check needs to be there.

For OmniOS, a major components components in the OS which caused many packages to break was the bundled gettext, failing during builds as it could not find the libgomp from (the also bundled) GCC. As a temporary work around to see how the build would progress if libgomp could be found, I added the lib directory to the search path of ld using crle(1).

Configuration file [version 4]: /var/ld/ld.config
Platform: 32-bit LSB 80386
Default Library Path (ELF): /lib:/usr/lib:/opt/gcc-4.8.1/lib
Trusted Directories (ELF): /lib/secure:/usr/lib/secure (system default)

Command line:
crle -c /var/ld/ld.config -l /lib:/usr/lib:/opt/gcc-4.8.1/lib

It was possible to build 13398 packages out of 16536 possible packages with this workaround in place.

With the help of Joerg Sonnenberger, at pkgsrcCon I added support for fetching the OS version info in OmniOS & SmartOS for use in build build reports, this should mean that these operating systems will be reported correctly rather than as SunOS 5.11.

sevan.mit.edu is back online as a G4 Mac Mini with 128GB SSD. It’s yet to complete its first bulkbuild since the rebuild but it’s nearly finished as I type this.

It’s now possible to build more than 14100 packages on FreeBSD 10.1-RELEASE with pkgsrc.

Posted on

A week of pkgsrc #10

Following on from last weeks post, I forgot to mention building on OpenBSD/sparc64 via a LDOM running on a Sun T5210, this was even more painful than the Solaris counterpart and took the best part of a month, some of this delay was initially caused by problematic packages which held up the build, not parallelising the builds and again issues with FTP mirrors.
devel/electric-fence was another of packages which was responsible for holding up the build that I didn’t mention in the previous post. During the build it runs a binary called eftest and that’s it, it’s stuck there until killed.
The LDOM I was running in was allocated 4 vCPUs but the build was running as a single threaded build. Defining MAKE_JOBS=4 in pkg/etc/mk.conf and recompressing the bootstrap kit (bootstrap.tar.gz) helped this situation. To work around FTP issues, bulkbuilds were switched to HTTP only thanks to a pointer from Joerg Sonnenberger. As defined in pkgsrc/mk/defaults/mk.conf
#MASTER_SORT_REGEX= ftp://.*/
# Same as MASTER_SORT, but takes a regular expression for more
# flexibility in matching. Regexps defined here have higher priority
# than MASTER_SORT. This example would prefer ftp transfers over
# anything else.
# Possible: Regexps as in awk(1)
# Default: none

Setting MASTER_SORT_REGEX= http://.*/ in pkg/etc/mk.conf and recompressing the bootstrap kit ensured builds use HTTP from there on.

The bulkbuild report showed lots of fallout from packages which hadn’t been updated yet to support LibreSSL e.g. net/wget expects DES support.
Rodent@ fixed lang/python27 with the changes due for subsequent releases of Python.
Bernard Spil fixed Heimdal which failed due to the lack of RAND_EGD in LibreSSL, these fixes will be in the next release of Heimdal (1.6.0?), back porting the changes to 1.5.3 which is the current release available resolved the issue with lack of RAND_EGD but then failed at building a kerberised telnet due to changes in the OpenBSD IPv6 stack which removed functionality telnet was expecting to be there. There is no fix for the issue in the OpenBSD ports as Heimdal is set to build without legacy and insecure protocols such as telnet and rsh.

Due to the connectivity issues on the OpenCSW build cluster, I erased the error report and restarted the bulkbuild on Solaris 10 SPARC and 11 x86 to re-attempt everything that had failed during the previous run for whichever reason. The Solaris 10 SPARC bulkbuild has now finished with a total of 7389 packages built, previously 5701. I discovered a particularly nasty bug with lang/gcc3-c++ which cost 3 days as the configure stage ran over and over again before being killed manually.

My access to the AIX LPAR expired, taking with it what I had previously tried, I requested access again but this time also requested access to a LPAR running SUSE 12 on Power8 as well.
Still no further with the AIX LPAR but managed to getting bulkbuild going on the SUSE 12 one with a little bit of assistance.
The first thing which needed to be done was to specify the ABI and the suffix applied to the library search path. This is because the system is 64 bit without any 32bit libraries installed and by default pkgsrc opts for 32bit unless set otherwise. When attempting to bootstrap initially, it failed with ERROR: bin/digest: missing library: libc.so.6. I initially set about the wrong path of trying to locate the glibc-32bit rpm for SUSE on Power before realising what was actually required. This may have been a knee-jerk reaction from the past before the days of yum and such on Linux. With the necessary change to pkgsrc/mk/platform/Linux.mk the bulkbuild environment setup continued before hanging on the installation of pkgtools/pkg_install. pkg_add would hang and CPU utilisation would spike to 100%.

A backtrace of the running process in gdb revealed it was stuck on mpool_get().
(gdb) bt
#0 0x0000000010096650 in mpool_get ()
#1 0x0000000010093658 in __bt_search ()
#2 0x000000001009318c in __bt_put ()
#3 0x000000001000b614 in pkgdb_store ()
#4 0x000000001000430c in extract_files ()
#5 0x0000000010006fd0 in pkg_do ()
#6 0x00000000100075a4 in pkg_perform ()
#7 0x0000000010005650 in main ()
Turns out the issue also affects pkgsrc on Linux/ARM and was previously reported in a bug report from 2013 with a workaround. Setting the GCC optimisation level to 0 for pkgtools/libnbcompat and pkgtools/pkg_install allowed mk/pbulk/pbulk.sh to setup a buklbuild environment and a bulkbuild is currently in progress. The bulkbuild was initially aborted to added some critical missing components which caused major breakage.

zypper install libxshmfence-devel gettext-tools gcc-c++.

With Suse Linux on Power8, that bumps my operating system count to 9 across 5 architectures. Just need to get AIX going to round off the OS count. 🙂

Posted on

A week of pkgsrc #9

The past few weeks have been pretty hectic, as the time for BSDcan gets shorter and shorter, I’m thinking about my talk and testing more and more in pkgsrc. Rodent@ added support for Bitrig to pkgsrc-current last month, his patches highlighted an issue with the autoconf scripts (which should be shared across core components) not being pulled in automatically. Joerg Sonnenberger resolved this issue and I regenerated the patch set again. With the system bootstrapped the next thing which was broken was Perl, applying the changes needed for OpenBSD resolved any remaining issues and the bulk build environment was ready. After three days, the first bulkbuild attempt on Bitrig was complete and a report was published. There is now a bulkbuild in progress with devel/gettext-tools and archivers/unzip fixed, that should free over 8400 packages to be attempted to be built.
For Solaris, my first bulkbuild on Solaris 10 completed after 22 days. Mid-April I also started off bulkbuilds on Solaris 11 (x86 and SPARC) using the SunStudio compilers (It’s not possible to use GCC at the moment due to removed functionality that was previously deprecated). The Solaris 11 SPARC bulkbuild is still in progress and the x86 bulkbuild is running. Unfortunately the build cluster had some connectivity issues and needed rebooting during the bulkbuild but not until lots of packages had failed to fetch distfiles, hence the figures look a lot worse than they could be. Solaris 10 SPARC report, Solaris 11 x86 report.

Through bulk building on multiple operating systems another issue that’s surfaced is problematic packages that hold the build up. On Bitrig mail/fml4 is an issue, on OpenBSD www/wml, FTP mirror issues for ruby extension on Solaris, Xorg FTP mirror issues on OmniOS. Things need regular kicking, a brief glance into pkgsrc/mk didn’t reveal any knobs which would allow the preference of HTTP for fetching distfiles. On Bitrig & OpenBSD I’ve excluded these packages from being attempted via NOT_FOR_PLATFORM statement in their Makefile until I have a look into the issue.

sevan.mit.edu completed another bulkbuild, pkgsrc-current now ships with MesaLib 10.5.3 as graphics/MesaLib, version 7 has now been re-imported as graphics/MesaLib7 by tnn@, the new MesaLib needed a patch for FreeBSD, similar to NetBSD to build successfully, due to ERESTART not being defined. At present, it’s still broken on Tiger as I’ve not looked into yet.

I revisited AIX again to test out pkgsrc once again, this has turned into a massive yak shaving session. I’ve yet to run a bulkbuild successfully as the scan stage ends with a coredump.
I originally started off with using the stock system shell, bootstrap completed successfully but scan stage of a bulkbuild would just stop without anything being logged. Manually changing the shell used to shells/pdksh in pkg/etc/mk.conf and pbulk/etc/mk.conf resulted in the following error message:
bmake: don't know how to make pbulk-index. Stop
pbulk-scan: realloc failed:

This turned to be a lack of RAM, my shell account was to a AIX 7.1 LPAR running on a Power8 host with 2 CPUs and 2GB of RAM committed, unfortunately the OS image IBM provided came with Tivoli support enabled and a bug in the resource management controller which meant RMC was consuming way more resource than it needed to. I was running with less than 128MB of RAM.
Stopping Tivoli & RMC freed up about 500MB of RAM, attempting to bulkbuild again, caused the process to fail once again at the same stage. With a heads up from David Brownlee & Joerg Sonnenberger, I bumped the memory and data area resource limits to 256MB.
This allowed the scan to finish with a segfault.
/usr/pkgsrc/pbulk/libexec/pbulk/scan[54]: 11272416 Segmentation fault(coredump).
pscan.stderr logged multiple instances of
bmake: don't know how to make pbulk-index. Stop.
The segfault generated a coredump but it turned out that dbx, the debugger in AIX was not installed. IBMPDP on twitter helped by pointing to the path where some components are available for installation, unfortunately, while the dbx package was available there, some of its dependencies were not. Waiting on IBMPDP to get back to me, I fetched a new pkgsrc-current snapshot (I couldn’t update via CVS because it wouldn’t build) and re-setup my pbulk environment via mk/pbulk/pbulk.sh.
I should mention that initially when I setup, I’d explicitly set CC=/usr/bin/gcc last time, then while trying to get various things to build subsequently, I’d symlink /usr/bin/cc to /usr/bin/gcc. When I came to set thing up with the new snapshot, I did not pass CC=/usr/bin/gcc this time round and found that I was unable to link Perl, not sure if this was the Perl build files assuming if on AIX & /usr/bin/cc exists, it’s XLC or if ld(1) takes on different behaviour but I had to remove this symlink.
Once everything was setup, the bulkbuild failed agin at the same place, except this time I had a different message logged.
/bin/sh: There is no process to read data written to a pipe..
I edited the bootstrap/bootstrap script & devel/bmake/Makefile to set shells/pdksh as a dependency & rerun bulkbuild.
The scan stage again completed with a coredump with this time pscan.stderr just contained Memory fault (core dumped).
I’ve committed these changes so pkgsrc-current now defaults to using shells/pdksh as its shell but have not been able to try anything else as this weekend the system is unaccessible due to maintenance.

At present, I’m attempting to bulkbuild pkgsrc-current on 8 Operating systems
OpenBSD (5.6-RELEASE & -current), FreeBSD, Bitrig (current), Mac OS X (Tiger), Solaris (10 & 11), OmniOS on 4 architectures (i386, AMD64, SPARC, PowerPC).
If I could get AIX going that would bump the OS & arch could up by 1. Maybe by the next post perhaps. 🙂

Thanks to Patrick Wildt for access to host running Bitrig and Rodent@ for adding support to pkgsrc.

Posted on

Captive Portals & Brighton

Yesterday I gave a talk at SANE user group on my history with wireless networks as part of the PierToPier.net project in Brighton (now defunct) and my experimentation with captive portal software which I began revisiting this time last year. I thought it would be a good opportunity to develop my programming skills by tidying up and modernising parts of the codebase which caused problems, such as things preventing builds on a modern system with clang which is now the default compiler for FreeBSD on i386/AMD64 architectures and OS X. The slides from my talk can be found here.

IMG_3248

Back in the early to mid 2000’s there were 2 initiatives to provide public access wifi in Brighton. Loose Connection and PierToPier.net, each had a different focus & approach.
Loose Connection was a commercial venture which could be deemed a VAR, they resold a ADSL connection along with a draytek router & that was it. Individual wireless networks with the loose connection SSID dotted around drinking holes in Brighton, the founding(?) company Metranet lives on as a WISP today.
PierToPier.net was a community driven effort with a technical team of volunteers, predominantly from a service provider / telecoms / networking background. Each node on the network was sponsored by a host who’d buy and run the equipment while the project members managed it.

The project started off based around the fanless VIA mini-itx boards, Prism2 chipset wireless cards, booting linux with hostapd and nocatauth/nocatsplash off a CF card in a IDE to CF adapter.
This was a very flexible platform, if there was no package for it you could build it with ease, problem was that it had a high cost for entry, £250 to £300? so from the start we were looking to reduce costs.
The other issue was though we’d eliminated moving parts, the casing was not suitable for outdoor use.

With the availability of 3rd party firmware and promotional sales of the WRT54G, PierToPier switched hardware platforms as the low cost solution for new nodes.
Tom Grifiths discovered Chillispot around the same time frame and we adopted it due to enhanced functionality it provided, such as RADIUS accounting and working captive portal. We’d previously ran into issues with browser support running nocatauth which by that point was no longer maintained and stability issues with nocatsplash.

Glastonbury 2005
VIA motherboard again this time with CF adapter & mini-pci slot onboard
2x Atheros A/B/G mini-pci cards, one on mini-pci to PCI bridges and second onboard
Stuck to a pelican case with epoxy
Two holes drilled in the side of the case for external antennas

It was early times for support of the cards and the wireless standards. In this era OpenBSD was leading the way in terms of support of hardware and development of their ieee80211 wifi stack, they were the first to reverse engineer the Atheros binary blob HAL (years before anyone?) but late to the game for the 802.11g, 11a was enabled from the start but didn’t appear to work – bringing the interface up in .11a hostap mode wouldn’t necessarily work.
Looking at alternatives there was a short lived live environment named WifiBSD which was based around FreeBSD but later moved to NetBSD before development ceased. The support for the Atheros cards was not as good as OpenBSD, hence not wasn’t much use.

The hardware for the Glastonbury nodes were truly terrible, all functionality had been wired to a single bus which caused the system to lock hard in most configurations. e.g. you booted from a CF card and tried to bring up a wireless interface. The only way to use the system was to disable everything that wasn’t needed in the BIOS including VGA, if there was an issue, you’d have to factory reset the BIOS before diagnosing.

For the wireless network at Glastonbury, the 11a 5GHz network was used as the backhaul while the 11b/g interface was used for connecting wireless clients. No captive portal, connect to the AP & off you go.

We arrived onsite on Tuesday, starting getting things running, Thursday morning the rain and lightning started things went downhill from there. loss of connectivity between the backhaul links meant things fell apart.

By the time we’d discovered Chillipot the project had a 1.0 release out which had preliminary support for FreeBSD. The website claimed only FreeBSD 5 and up were supported, I created a port and submitted it for inclusion in the tree, net-mgmt/chillispot was born, Edwin@ from the ports team fixed the code so that it’d work on previous releases. I then moved onto creating a OpenBSD port, this was slightly harder and the final peace was actually resolved by a Steve Davies. I got the code to build on OpenBSD but networking wouldn’t work. This turned out to be because an additional 4 bytes needed to be allocated which Steve fixed. It never made it into the OpenBSD tree (only tested it on i386 and SPARC, it used strcpy() everywhere and didn’t run on SPARC) but it can be found in ports-wip. I then moved onto creating a live CD environment based on FreeBSD 6 using freesbie for advocacy purposes named BrightonChilli. The idea was to remove the hurdle of going through the installation process and provided an environment that just needed the configuration of network interfaces and chillispot. A person with previous experience of running chillispot would be familiar and a new user would not be too out of place.
This was in the days of X configuration being a part of sysinstall which could hang if Xconfigure was run and you’d have to start the install process again as the install was incomplete (for a newcomer). I was interviewed on BSDtalk #73 regarding BrightonChilli.

PierToPier also produced its own Linux image named Muddy Linux targeted for x86 hardware that ran the necessary stack to serve as a node on the network.
After Chillispot 1.1.0, the project went quiet, there was no answer from the founding developer for quite a while and eventually the web hosting stopped and the domain expired.
The community rehomed to coova.org and development continued in Coovachilli which was founded by David Bird, a contributor to Chillispot.
Coovachilli initially lacked support for FreeBSD but it was eventually added in by David and net-mgmt/coovachilli was born in ports. Not much else was done after that until a year ago. With FreeBSD 10 and the switch to clang, the codebase needed attention, first step was to get it to build correctly with GCC. The use of error_t from glibc caused the build to fail as it’s not available in FreeBSD, ensuring this was declared allowed the build to complete successfully. To resolve build issues with clang, nested functions were separated out. Any function with missing prototypes & parameter lists were addressed next.
struct ifreq had been marked as deprecated since 2000 and was finally removed in FreeBSD 10. The *BSD specific sections of Coova were switched out to the new struct ifaliasreq & Linux was left to use the pre-existing method. There was extensive use of macros for the logging functionality, these were dropped in favour of using the existing standard syslog(3) with the correct log level defined. This had the benefit of revealing issues which were not detected previously such as incorrect format specifiers.
There are still many things that need to be cleared up, the 3rd party functions added in are particularly problematic and will probably be my next task to replace with standard components.
CoovaChilli & Chillisport have seen large scale deployments thanks to use by Fon, o2 and Google which now owns Coova.

IMG_3250

Posted on

A week of pkgsrc #8

Or should that be a month of pkgsrc!
Since I wrote my last post I received the good news that I’ll be presenting at BSDCan this year in Ottawa. I’m really stoked about this and looking forward to presenting there. In preparation for the big day, I’ll be speaking about different aspects of what I’ve been working on at various events/meetings. My first speaking opportunities will be at Oracle’s SolarisSIG (LOSUG in a past life) tomorrow on cross platform packaging with pkgsrc.
I didn’t have any hosts running Solaris to attempt bulkbuilds on for this, so I asked for shell access at various places, this landed me with access to Solaris 9 to 11 on x86 & SPARC and OmniOS.
Shell access to a Solaris 10u9 host came first and I quickly found that mandoc would not build due to the use of mkdtemp(), dirfd() and vasprintf() whilst attempting to setup a bulkbuild environment using the mk/pbulk/pbulk.sh script in pkgsrc.
Following up with the mandoc developers regarding the issue meant the issue was fixed by Ingo Schwarze within a couple of days with access to Solaris via OpenCSW, the next release of mandoc due Summer time will contain these fixes. I’m undecided whether to back port the fixes to the current release or not. If you have a Solaris 10 host with a “recent” patch cluster (unsure how recent) you should be unaffected by this issue.

On OmniOS, the shells/standalone-tcsh package reared its head again. OmniOS ships with tcsh as standard which shells/standalone-tcsh clobbers again during bulkbuild. The solution to this may be that pkgsrc becomes aware of the various Illumos distros instead of treating everything as SunOS, but nothing has been decided for definite and needs further discussion.
The bulkbuilds on OmniOS using the toolchain published in IPS for that release show issues in pkgsrc where libgomp bundled with GCC is not being found by gettext, this causes lots of packages to fail.
The first bulkbuild which completed showed it was possible to build 9547 out 15245 packages without issue but the inability to build databases/shared-mime-info due to the libgomp issue caused 2274 packages to fail. This was the large impact of the issue mentioned but there are more packages which suffer from the same problem.

With the resumption of bulk building pkgsrc on FreeBSD, I found lots of packages which would fail to package due to use of the -o flag with pax(1) which is not present in the FreeBSD version. I’ve not yet addresses the issue with those packages but I did raise a patch for FreeBSD-current so that it will be available in future versions of FreeBSD. It is now committed and will treacle down to 10-STABLE at some point for inclusion in FreeBSD 10.2-RELEASE.

Bulk builds have also resumed on OpenBSD/amd64 and Sparc64, there is a third bulkbuild in progress on amd64 while the sparc64 host is still running the first attempt. The use of unprivileged user account and the lack of nullfs support prevents the setup of parallel runs to speed up the process. Building on OpenBSD was extremely useful for indicating what software in pkgsrc is effected by LibreSSL without having to put effort into integrating LibreSSL into pkgsrc first. It also raised an instance of copying overlapping buffers destructively (using memcpy() vs memmove()), a fix for this issue is yet to be committed pending further investigation. MAKE_JOBS was raised to 8 on OpenBSD/sparc64 to improve performance whilst compiling, the build environment is hosted on a Sun T5210 with a dedicated disk mounted with soft writes & the kern.bufcachepercent value bumped from 20 to 50.

With the pkgsrc-2015Q1 release announced, the figures for packages indicate that Darwin/PowerPC exceeded Darwin/x86.

11224 binary packages built with gcc for Darwin 8.11.0/powerpc
10019 binary packages built with gcc for Darwin 10.8.0/i386

My first bulkbuild report sent to to the pkgsrc-bulk mailing list back in September 2014 shows a total of 8501 packages could be built. There is more work required to keep this position in the future, as a new version of MesaLib is to be committed at some point soon. There is a bulkbuild in progress on sevan.mit.edu, upon completion I’ll be placing a request for the host to be wiped and setup again, the bulk builds are getting slower and slower and I suspect the IDE disk may be failing.

Thanks to Phil Stracchino, Sascha Curth and the OpenCSW Project for access to hosts running Solaris. Ian Kremlin, Brandon Mercer & Rodent @ NetBSD for access to hosts running OpenBSD.

Posted on

Using ifstated to monitor links and dynamically adjust PF config on event

It’s possible to misuse NAT to load balance outbound traffic across multiple internet connections from different service providers,see the Load Balance Outgoing Traffic section of PF FAQ.
The shortfall with this configuration is when implemented alongside unstable links, forwarding will continue to be attempted over the links which are down, this will cause issues such as long hangs for users behind the NAT while connections time out. To mitigate this, ifstated can be used to smooth things over.
ifstated can be used to run tests & on event perform tasks, if you’re familiar with Cisco IOS, this is similar to some of what is available in EEM. In this scenario, ifstated will be set to ping each gateway at the service provider end of each link every 10 seconds & upon failure, adapt the configuration so traffic is not forwarded down that link. ifstated will continue to perform the tests & when tests start passing because link has re-established successfully, ifstated will reconfigure the system again so links are utilised.

For this post we’ll use the example ruleset from the PF FAQ and adapt it so it can be manipulated by ifstated.

Original pf.conf

lan_net = "192.168.0.0/24"
int_if = "dc0"
ext_if1 = "fxp0"
ext_if2 = "fxp1"
ext_gw1 = "198.51.100.100"
ext_gw2 = "203.0.113.200"

# nat outgoing connections on each internet interface
match out on $ext_if1 from $lan_net nat-to ($ext_if1)
match out on $ext_if2 from $lan_net nat-to ($ext_if2)

# default deny
block in
block out

# pass all outgoing packets on internal interface
pass out on $int_if to $lan_net
# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if
# load balance outgoing traffic from internal network.
pass in on $int_if from $lan_net \
route-to { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } \
round-robin
# keep https traffic on a single connection; some web applications,
# especially "secure" ones, don't allow it to change mid-session
pass in on $int_if proto tcp from $lan_net to port https \
route-to ($ext_if1 $ext_gw1)

# general "pass out" rules for external interfaces
pass out on $ext_if1
pass out on $ext_if2

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
pass out on $ext_if1 from $ext_if2 route-to ($ext_if2 $ext_gw2)
pass out on $ext_if2 from $ext_if1 route-to ($ext_if1 $ext_gw1)

Modified pf.conf

lan_net = "192.168.0.0/24"
int_if = "dc0"
ext_if1 = "fxp0"
ext_if2 = "fxp1"
ext_gw1 = "198.51.100.100"
ext_gw2 = "203.0.113.200"

# nat outgoing connections on each internet interface
anchor nat-isp1
anchor nat-isp2

set skip on lo

# default deny
block in
block out

anchor "ftp-proxy/*"

# pass all outgoing packets on internal interface
pass out on $int_if to $lan_net
# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if
# load balance outgoing traffic from internal network.
anchor loadbalance

# keep https traffic on a single connection; some web applications,
# especially "secure" ones, don't allow it to change mid-session
anchor applications

# general "pass out" rules for external interfaces
pass out on $ext_if1
pass out on $ext_if2

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
anchor pass-isp1
anchor pass-isp2

The rules for NAT, load balancing & routing are replaced with anchors, ifstated will use these anchors to add & manipulate rules.

ifstated.conf

isp1 = '( "ping -q -c 1 -w 1 -S 198.51.100.199 198.51.100.100 >/dev/null" every 10)'

#If inteface is configured dynamically via dhcp use this instead
#isp2 = '( "ping -q -c 1 -w 1 -S `ifconfig vr2 inet |awk \'/inet/ { print $2 }\'` `awk \'/routers/ { print $3 }\' /var/db/dhclient.leases.vr2 |tail -1 |sed \'s/;//\'`>/dev/null" every 10)'

isp2 = '( "ping -q -c 1 -w 1 -S 203.0.113.220 203.0.113.200 >/dev/null" every 10)'

state allworking {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'
run 'pfctl -a pass-isp2 -F rules'

run 'route change default 203.0.113.200'

run 'echo "pass in on vr1 from 192.168.1.0/24 \
route-to { (vr0 198.51.100.100), (vr2 203.0.113.200) } round-robin" | pfctl -a loadbalance -f -'

run 'echo "pass in on vr1 proto tcp from 192.168.1.0/24 to port https route-to (vr2 203.0.113.200)" | pfctl -a applications -f -'

run 'echo "match out on vr0 from 192.168.1.0/24 nat-to (vr0)" | pfctl -a nat-isp1 -f -'

run 'echo "match out on vr2 from 192.168.1.0/24 nat-to (vr2)" | pfctl -a nat-isp2 -f -'

run 'echo "pass out on vr0 from vr2 route-to (vr2 203.0.113.200)" | pfctl -a pass-isp2 -f -'

run 'echo "pass out on vr2 from vr0 route-to (vr0 198.51.100.100)" | pfctl -a pass-isp1 -f -'
}
if ! $isp1
set-state noisp1
if ! $isp2
set-state noisp2
}

state noisp1 {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'

run 'route change default 203.0.113.200'

run 'echo "pass in on vr1 from 192.168.1.0/24 route-to { (vr2 203.0.113.200) }" | pfctl -a loadbalance -f -'

run 'echo "pass in on vr1 proto tcp from 192.168.1.0/24 to port https route-to (vr2 203.0.113.200)" | pfctl -a applications -f -'

run 'echo "match out on vr2 from 192.168.1.0/24 nat-to (vr2)" | pfctl -a nat-isp2 -f -'

run 'echo "pass out on vr2 route-to (vr2 203.0.113.200)" | pfctl -a pass-isp2 -f -'
}
if $isp1
set-state allworking
if ! $isp2
set-state alldown
}

state noisp2 {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'

run 'route change default 198.51.100.100'

run 'echo "pass in on vr1 from 192.168.1.0/24 route-to { (vr0 198.51.100.100) }" | pfctl -a loadbalance -f -'

run 'echo "pass in on vr1 proto tcp from 192.168.1.0/24 to port https route-to (vr0 198.51.100.100)" | pfctl -a applications -f -'

run 'echo "match out on vr0 from 192.168.1.0/24 nat-to (vr0)" | pfctl -a nat-isp1 -f -'

run 'echo "pass out on vr0 route-to (vr0 198.51.100.100)" | pfctl -a pass-isp1 -f -'
}
if ! $isp1
set-state alldown
if $isp2
set-state allworking
}

state alldown {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'
}
if $isp1 && ! $isp2
set-state noisp2
if $isp2 && ! $isp1
set-state noisp1
if $isp1 && $isp2
set-state all working
}

As ifstated is initialised & when it switches states, it flushes the anchors in the pf.conf, sets the default gateway so the host itself can be reachable remotely on the WAN and then injects rules into the PF anchors.

Posted on

OpenBSD on 11″ MacBook Air 5,1 (mid-2012)

While my MacBook was away at the service centre to have the SSD replaced I noticed the NetBSD wiki had marked the install guide as obsolete as it’s no longer required to build a custom kernel because the necessary changes have been integrated so that the generic kernel works out of the box.
The last time I tried to run OpenBSD on a MacBook Air was over the christmas holiday on a mid-2012 13″ model & while I managed to boot a multiuser system, USB support was very unstable & eDP support was missing from Xenocara.
Having received my MacBook back I decided to revisit Net/OpenBSD, I tried booting NetBSD/AMD64 6.1.1 & the 29/08/2013 AMD64 OpenBSD snapshot.
Both exhibited the same behavior, as soon as the kernel loaded into memory the screen would go blank, I attached a thunderbolt display which made displayed some output before going to a blank screen as the kernel probes for devices, I believe this is when OpenBSD now changes font.
I switched from the OpenBSD snapshot to what I (half asleep) thought was 5.4-RELEASE but it turned out to be a mislabelled iso of a snapshot from July. This time it worked fine using a Thunderbolt display, screen still goes blank otherwise.
Onboard wireless doesn’t work, instead I’m using a tiny urtwn(4) wireless adapter.

20130905-021145.jpg

OpenBSD 5.4-current (GENERIC.MP) #50: Mon Sep 2 13:43:54 MDT 2013
deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error b1
real mem = 8475713536 (8083MB)
avail mem = 8242003968 (7860MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe0000 (54 entries)
bios0: vendor Apple Inc. version "MBA51.88Z.00EF.B02.1211271028" date 11/27/2012
bios0: Apple Inc. MacBookAir5,1
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP HPET APIC SBST ECDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT DMAR MCFG
acpi0: wakeup devices P0P2(S4) PEG2(S4) EC__(S4) HDEF(S4) RP02(S4) ARPT(S4) RP05(S4) EHC1(S4) EHC2(S4) XHC1(S4) ADP1(S4) LID0(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-3667U CPU @ 2.00GHz, 1896.01 MHz
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
cpu0: apic clock running at 99MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i7-3667U CPU @ 2.00GHz, 1895.70 MHz
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Core(TM) i7-3667U CPU @ 2.00GHz, 1895.70 MHz
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 1, core 0, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Core(TM) i7-3667U CPU @ 2.00GHz, 1895.70 MHz
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 2
acpiec0 at acpi0
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-153
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P2)
acpiprt2 at acpi0: bus -1 (PEG2)
acpiprt3 at acpi0: bus 2 (RP02)
acpiprt4 at acpi0: bus 3 (RP05)
acpicpu0 at acpi0: C3, C1, PSS
acpicpu1 at acpi0: C3, C1, PSS
acpicpu2 at acpi0: C3, C1, PSS
acpicpu3 at acpi0: C3, C1, PSS
acpibat0 at acpi0: BAT0 model "3545797981023400290" type 3545797981528607052 oem "3545797981528673619"
acpiac0 at acpi0: AC unit offline
acpibtn0 at acpi0: LID0
acpibtn1 at acpi0: PWRB
acpibtn2 at acpi0: SLPB
acpivideo0 at acpi0: IGPU
acpivout0 at acpivideo0: DD02
cpu0: Enhanced SpeedStep 1896 MHz: speeds: 2001, 2000, 1900, 1800, 1700, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800 MHz
memory map conflict 0xe00f8000/0x1000
memory map conflict 0xfed1c000/0x4000
memory map conflict 0xffe70000/0x30000
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 3G Host" rev 0x09
vga1 at pci0 dev 2 function 0 "Intel HD Graphics 4000" rev 0x09
intagp0 at vga1
agp0 at intagp0: aperture at 0x90000000, size 0x10000000
inteldrm0 at vga1
drm0 at inteldrm0
inteldrm0: 1366x768
wsdisplay0 at vga1 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
"Intel 7 Series xHCI" rev 0x04 at pci0 dev 20 function 0 not configured
"Intel 7 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
ehci0 at pci0 dev 26 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 7 Series HD Audio" rev 0x04: msi
azalia0: codecs: Cirrus Logic CS4206, Intel/0x2806, using Cirrus Logic CS4206
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 7 Series PCIE" rev 0xc4: msi
pci1 at ppb0 bus 1
ppb1 at pci0 dev 28 function 1 "Intel 7 Series PCIE" rev 0xc4: msi
pci2 at ppb1 bus 2
"Broadcom BCM43224" rev 0x01 at pci2 dev 0 function 0 not configured
ppb2 at pci0 dev 28 function 4 "Intel 7 Series PCIE" rev 0xc4: msi
pci3 at ppb2 bus 3
ppb3 at pci3 dev 0 function 0 vendor "Intel", unknown product 0x1547 rev 0x03
pci4 at ppb3 bus 4
ppb4 at pci4 dev 0 function 0 vendor "Intel", unknown product 0x1547 rev 0x03: msi
pci5 at ppb4 bus 5
vendor "Intel", unknown product 0x1547 (class system subclass miscellaneous, rev 0x03) at pci5 dev 0 function 0 not configured
ppb5 at pci4 dev 3 function 0 vendor "Intel", unknown product 0x1547 rev 0x03: msi
pci6 at ppb5 bus 6
ppb6 at pci4 dev 4 function 0 vendor "Intel", unknown product 0x1547 rev 0x03: msi
pci7 at ppb6 bus 55
ppb7 at pci4 dev 5 function 0 vendor "Intel", unknown product 0x1547 rev 0x03: msi
pci8 at ppb7 bus 104
ppb8 at pci4 dev 6 function 0 vendor "Intel", unknown product 0x1547 rev 0x03: msi
pci9 at ppb8 bus 105
ehci1 at pci0 dev 29 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 22
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 vendor "Intel", unknown product 0x1e56 rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 7 Series AHCI" rev 0x04: msi, AHCI 1.3
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: SCSI3 0/direct fixed naa.0000000000000000
sd0: 115712MB, 512 bytes/sector, 236978176 sectors, thin
ichiic0 at pci0 dev 31 function 3 "Intel 7 Series SMBus" rev 0x04: apic 2 int 18
iic0 at ichiic0
iic0: addr 0x2c 03=fc 05=66 06=40 71=06 72=80 86=70 90=37 91=1c 92=35 93=3f 94=62 95=8c 96=63 97=85 98=24 99=04 9a=88 9f=7c a0=7f a1=b5 a2=bf a3=7b a4=28 a5=cf a6=64 a7=2d words 00=0000 01=0000 02=00fc 03=fc00 04=0066 05=6640 06=4000 07=0000
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns8250, no fifo
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
nvram: invalid checksum
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
uhub2 at uhub0 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uvideo0 at uhub2 port 1 configuration 1 interface 0 "Apple Inc. FaceTime HD Camera (Built-in)" rev 2.00/80.25 addr 3
video0 at uvideo0
ugen0 at uhub2 port 1 configuration 1 "Apple Inc. FaceTime HD Camera (Built-in)" rev 2.00/80.25 addr 3
uhub3 at uhub1 port 1 "Intel Rate Matching Hub" rev 2.00/0.00 addr 2
uhub4 at uhub3 port 8 "Standard Microsystems product 0x2512" rev 2.00/b.b3 addr 3
uhub5 at uhub4 port 1 "Apple Inc. BRCM20702 Hub" rev 2.00/1.00 addr 4
uhub5: device problem, disabling port 1
uhidev0 at uhub5 port 2 configuration 1 interface 0 "Apple Computer product 0x820b" rev 2.00/1.00 addr 5
uhidev0: iclass 3/1, 2 report ids
ums0 at uhidev0 reportid 2: 3 buttons
wsmouse0 at ums0 mux 0
ugen1 at uhub5 port 3 "Apple Inc. Bluetooth USB Host Controller" rev 2.00/1.00 addr 6
uhidev1 at uhub4 port 2 configuration 1 interface 0 "Apple Inc. Apple Internal Keyboard / Trackpad" rev 2.00/2.19 addr 7
uhidev1: iclass 3/1, 9 report ids
ukbd0 at uhidev1 reportid 1: 8 variable keys, 6 key codes, country code 15
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhid0 at uhidev1 reportid 9: input=0, output=0, feature=3
uhidev2 at uhub4 port 2 configuration 1 interface 1 "Apple Inc. Apple Internal Keyboard / Trackpad" rev 2.00/2.19 addr 7
uhidev2: iclass 3/0, 68 report ids
uhid1 at uhidev2 reportid 68: input=511, output=0, feature=0
uhidev3 at uhub4 port 2 configuration 1 interface 2 "Apple Inc. Apple Internal Keyboard / Trackpad" rev 2.00/2.19 addr 7
uhidev3: iclass 3/1, 2 report ids
ums1 at uhidev3 reportid 2: 3 buttons
wsmouse1 at ums1 mux 0
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on sd0a (0f625884de9eca57.a) swap on sd0b dump on sd0b
clock: unknown CMOS layout
hw.sensors.cpu0.temp0=36.00 degC
hw.sensors.cpu1.temp0=36.00 degC
hw.sensors.cpu2.temp0=36.00 degC
hw.sensors.cpu3.temp0=36.00 degC
hw.sensors.acpibat0.volt0=7.50 VDC (voltage)
hw.sensors.acpibat0.volt1=7.23 VDC (current voltage)
hw.sensors.acpibat0.power0=12.26 W (rate)
hw.sensors.acpibat0.watthour0=42.41 Wh (last full capacity)
hw.sensors.acpibat0.watthour1=0.25 Wh (warning capacity)
hw.sensors.acpibat0.watthour2=0.10 Wh (low capacity)
hw.sensors.acpibat0.watthour3=6.90 Wh (remaining capacity), OK
hw.sensors.acpibat0.raw0=1 (battery discharging), OK
hw.sensors.acpiac0.indicator0=Off (power supply)
hw.sensors.acpibtn0.indicator0=On (lid open)

Posted on

FOSDEM 2013

Last week I attended a conference on open source software called FOSDEM in Brussels, the two day event has lots of tracks, based on either specific projects or topics such as Java or securiy.

I attended the following talks
On Saturday
XMPP 101
The Open Observatory of Network Interference
Practical Security for developers, using OWASP ZAP
The future of X.org on non-Linux systems
Declarative style GUI programming
How to build an Identity Management System on Linux

On Sunday
The Lua Scripting Language in the NetBSD Kernel
Supporting the new C and C++ standards in FreeBSD
Improvements in the OpenBSD IPsec stack

My favourite talk of the event was the OWASP talk on Saturday by Simon Bennetts who did a great job of clearly explaining what ZAP can do & how it is of use for testing the security of your web application.
The XMPP 101 talk gave an overview of what the protocol can do, the OONI talk had a very late start & laptop issues, didn’t get much from the talk but it does seem like an interesting project from the info on the website. Matthieu Herrb  talked about the progress of running X.org on UNIX, conclusion “Tough times for non-linux systems”. Marc Balmer gave two talks on using Lua, first in GUI programming & the second on the lua(4) subsystem in the NetBSD kernel, allowing users to explore the system easily & doing rapid prototype without the initial steep learning curve of learning C & kernel internal, making the system internals easily accessible. The last talk on the Security track was on FreeIPA, luckily the slides were quiet detailed as it was impossible to hear the speaker because the mic was hanging too low off  his shirt collar.

The BSD track on Sunday was where I spent most of the day. David Chisnall spoke about the C & C++ standards & the mistakes made by the standards groups which we have to live with. I spent the lunch break talking with David about FreeBSD, how I struggle with doing buildworld on my X61s, what can be done to speed up buildworld, why the buildworld process takes so long & the tools Juniper has developed which allow you to track the dependency path for building each component in FreeBSD base.
Mike Belopuhov spoke about the IPsec stack & NAT64 support in OpenBSD, I had an opportunity to ask Mike about dead peer detection, in my previous site to site VPN deployment I had issues where if the connection dropped at either site, the tunnel with not be re-established, needing manual intervention, It was good to hear that this was a problem with the isakmpd & not necessarily a configuration issue.

There were a lots of projects & businesses with stands, Oreilly had a stand selling books, Google were in the recruitment section, Oracle with three big banners for java, mysql & something else, the lady on the stand was very friendly, telling me about how Oracle participates in open source software such as Java, the penny then dropped about the update 13 release.
It was good to see CAcert had a stand and were looking very busy with assurances. I visited the mozilla stand & had the opportunity to try out the firefoxOS on a nexus s?
I’m strongly considering moving to it as I’d rather go with firefoxOS than android, the lock down of iOS is very painful for sharing data between my own devices & makes it frustrating for getting content from several devices to a single place.
I visited the google stand to talk to the recruiters there, I was curious to learn about their recruitment process, since 2007 I have been approached by Google on 3 different occasion, the most recent being back in July last year. I always assumed they had drives every so many years & I’d just been lucky to have been listed on three separate occasions, it turns out actually that once you’re on their radar, they will make contact every once in a while to see if your situation has changed & if have developed sufficiently since last time to be able to pass the interview tests.
I spoke with others regarding this, with those now employed by them & those who have also been approached in the past, discussing why systems folks are sought after & what options you have should you wish to no longer be contacted (supposedly under Californian law, if a person requests a company to never be contacted again, the company has to comply?).

Over the weekend I spotted a few OpenBSD tops (more hoodies than t-shirts) & met my first MirBSD user/developer, Benny Siegert who was the organiser of the BSD track at FOSDEM.
I also had the opportunity to meet up with/bump into folks from communities such as MetaBUG, OSHUG, LOSUG, Brighton 2600, London *BSD, it was good to catch up.

Posted on

Goodbye Alphastation

My second cool legacy UNIX workstation which got me started on FreeBSD & OpenBSD, I obtained this workstation back in the summer of 2002, I first tried Redhat Linux 7.2 which was available as a free download as a promotion to demonstrate the optimisation ability of the Compaq compiler suite for the Alpha. It was a terrible experience consistent with my previous attempts at running Linux up to that point ( I’d started off on Slackware in 96, moved onto Redhat 5.2 followed by Suse 6.2 ), I soon dropped it & moved onto Debian 3.0 (Woody) which was ok but the 7 cd set was a bit too much hassle for doing package installs, the performance wasn’t much better with the compared to the “optimised” Redhat so I moved onto NT 4.0 workstation & FX32! & ran that for a bit before getting bored. In the new year FreeBSD 5.0 release was announced & Alpha was a supported platform so I gave it try on this machine, armed with a copy of the handbook & the help of IRC I made a lot of progress, first by dropping 5.0 & going back to version 4.7 after being told either x was broken in 5 or y was a bug in 5 too many times. I was blown away by how much faster it was compared to the so-called “optimised” edition of Redhat.
Towards the end of 2003 I started thinking about trying OpenBSD as a firewall after hearing about PF & deployed it when 3.4 was released, the Alphastation served as my gateway connected to a 512k/128k cable modem connection but ended up dropping it & moving to i386 when 3.5 was released because php mysql extension was broken on alpha & I wanted to launch this blog.
After that the Alphastation was used less & less over the years so I passed it onto a fellow techie who would appreciate it.

Posted on

Beginning LaTeX – Typesetting the OpenBSD FAQ

I attended a one day training course held by the UK TUG back in July of last year which introduced beginners to LaTeX.
It was relatively simple to get up & running & we were able to put together basic documents with ease after a little practice.
Slides from the course
Handout from the course
To apply what I’d learnt on the course I decided to typeset the OpenBSD FAQ to get me on my way with LaTeX as the official PDF available for download appears to be generated using a pdf printer from the website which is great (links & chapters are there & working) but I don’t think it look that great.
A beautiful OS deserves beautiful documentation! 🙂
So I had a brief attempt at it the days proceeding the course & got side tracked after doing the very basics on the first 4 chapters. Nearly a year on, I thought I’d have another stab at it.
The tex files are in a Mercurial repo & there’s a PDF too 🙂
I’ve managed to get 10 of the 15 sections from the faq into tex files so far, with basic formatting applied to text, but there are lots to do yet e.g links, tidying up formatting, setting a typographical convention & applying it consistently.

Posted on

ThinkPad X61s

I couldn’t justify spending £1400+ on a built to order MacBook Air with 4GB of RAM, I settled on what has turned out to be a mint condition X61s with 8 cell battery & still under warranty for £172.98.
I’m really pleased with it so far but it’s still no MacBook Air (I went into the Apple store to double check) 🙂
Though they’re both “ultraportable laptops” they both scratch a totally different itch for me, the macbook is a sleek, tightly integrated with Mac OS, the ThinkPad is an extendible machine which is far more accommodating to various operating systems.
The reason I was looking to move to an ultraportable was so I have something I can with me at all times (obviously) which was to replace my back breaking 17″ MacBook Pro with the ability to run multiple operating systems with ease.
I was able to successfully multi-boot MacOS, FreeBSD & OpenBSD on my MacBook Pro using the gtpsync tool from rEFIT but support for the hardware wasn’t great e.g as It had an nVidia graphics card there was no resume support on OpenBSD, power management didn’t really work under FreeBSD either If I remember right & having a single mouse button meant it was a pain to use X, having to use workarounds with the eject button on the keyboard to emulate right clicks.
The 11″ MacBook Air seemed like the perfect machine for me, but the whole sealed unit really grinds my gears, it’s not that I wanted to take a screw driver to it but I’d like to have the option to extend the system at a later date instead of having to decide on the system configuration which would be set in stone, requiring a new system if I wanted to expand, the ram being the most important thing, buy it with 4GB of ram or be stuck with 2GB. Though 2GB is fine for OS X alone, it really doesn’t cut it when you’re multi tasking with iChat, Terminal.app, iTunes, Thunderbird, Safari, Omniweb or Opera. These are the apps which are usually always open on my system & my 2007 Mac Mini really struggled with this work load with 2GB or RAM, grinding to a halt regularly as the system swapped furiously, moving to 3GB gave the machine a new lease of life & stopped this behaviour, I would hate to be in the same position with a new system so the BTO Air was the only option for me.
I would also be stuck with another nVidia based system if I went for the Macbook Air which means I would still have problems with sleep & X acceleration so the second hand Thinkpad X61s with the intel chipset was the way to go.
The machine is currently multi-booting OpenBSD-CURRENT, FreeBSD-CURRENT & OpenIndiana 148a development build quiet happily.
The system works a treat under OpenBSD, sleep support is still not there in FreeBSD 9.0-CURRENT but I suspect that may just be a bug in acpi_ibm(4). I was hoping to be running Schillix on this system but was unable to get the system to boot after install, I suspect a change in device paths between booting from the optical drive in the ultrabase & hard disk is the cause but didn’t look into it in-depth settling for OpenIndiana after trying Solaris 11 express (which freaked out after the rwn driver was installed) while I work through the DTrace book , though I’ve compiled in dtrace support for FreeBSD & it’s there out of the box on MacOS X most of the examples in the book don’t work as covered in the book on these platforms.

Hardware wise I ditched the supplied intel wireless card & installed a AzureWave AW-NE766 Ralink chipset wireless card. The ThinkPads check minipci-e & wireless usb devices against a device id whitelist in the bios & if not listed the system presents a “1802: Unauthorized network card is plugged in – Power off and remove the miniPCI card” error & refuses to boot. Reflashing the bios with a modified bios image by someone called Zender turns this off & allows the system to boot without any problems.

All in all a great system which is cheaper than a netbook, far superior in build & spec but inferior to a MacBook Air in some ways 🙂

OpenBSD 4.8 dmesg
FreeBSD 9.0-CURRENT dmesg