Tag Archives: BSD

Running BSDi BSD/OS on VirtualBox

By default, the BSD/OS kernel recognises the CPU of a VirtualBox guest VM as a Pentium II. The kernel is able to boot correctly but performing any I/O results in failure due to memory errors. Adjusting the CPU mask of the VM from the host resolves this issue.

Note the name or GUID of the VM to be modified
% VBoxManage list vms
"BSDi BSD/OS" {36772f8c-ec06-4f37-a995-662fc38ad103}

Adjust the CPU of the VM
VBoxManage modifyvm "BSDi BSD/OS" --cpuidset 1 0x4a7 0x7100800 0x17bae3ff 0xbfebfbff

Obtained from OS2 Museum.

Book review: The Design and Implementation of the 4.3BSD UNIX Operating System

The Design and Implementation of 4.3BSD UNIX Operating System
According to my photographs, I picked up this book in February of this year. With a 105 sections spread over 13 chapters I’ve been working through the book slowly at a section a day. Despite being a technical subject, the book does a very good job of explaining the operation system at a high level without becoming a study of the source code. There are snippets of source code & pseudo code to compliment the text and an extensive list of papers for reference at end of each chapter for those that wish to dig deeper.

I had previously attempted to complete the Minix book, Operating Systems: Design And Implementation but struggled with the extensive source reference. switching back and fourth between chapters or the requirement for a computer to view the source code was not a viable option. I took a chance on this book as used copies are available on Amazon for the cost of a postage which is less than a couple of pounds. The book is well written and enjoyable to read, while implementation details may not be completely applicable to modern BSD variants The fundamental details may still hold true in most cases if not providing a historical background around the technical challenges they faced at the time. What I liked with the Minix was that it provided lots of background to accommodated a beginner and get a reader up to speed though I much preferred the ability to read this book by itself without requiring access to the source code.

I found some of the details in the interprocess communication part a little unclear at times but enjoyed the filesystem and memory management chapters the most and the terminal handling chapter the least though I did learn of Berknet there, aswell as many other historical artefacts throughout the book, some of which I tweeted under the hashtag di43bsd.

Berknet is an obsolete batch-oriented network that was used to connect PDP-11 and VAX UNIX systems using 9600-baud serial lines. Due to the overhead of input processing in the standard line discipline, a special reduced-function network discipline was devised.

The 4.3BSD kernel is not partitioned into multiple processes. This was a basic design decision in the earliest versions of UNIX. The first two implementations by Ken Thompson had no memory mapping at all, and thus made no hardware-enforced distinction between user and kernel space. A message-passing system could have been implemented as readily as the actually implemented model of kernel and user processes. The latter was chosen for simplicity. And the early kernels were small. It has been largely the introduction of more and larger facilities (such as networking) into the kernel that has made their separation into user processes an attractive prospect — one that is being pursued in, for example, Mach.

The book breaks down the percentage of components in each category (such as headers) which are platform independent and platform specific. With a total of 48270 lines of platform independent code versus 68200 lines of platform specific code, the 4.3BSD kernel was largely targeted at the VAX.

From the details on the implementation of mmap() in the BSD memory management design decisions section it was interesting to read about virtual memory subsystems of old

The original virtual memory design was based on the assumption that computer memories were small and expensive, whereas disk were locally connected, fast, large, and inexpensive. Thus, the virtual-memory system was designed to be frugal with its use of memory at the expense of generating extra disk traffic.

It made me think of Mac OS X 10.4 (Tiger) as that still struggled with the same issue many years on which I have to suffer when building from pkgsrc. Despite having a system with 2GB of RAM, memory utilisation rarely goes above 512MB.

The idea of having to compile the system timezone in the kernel amused me though it was stated that with 4.3BSD Tahoe, support for the Olson timezone database that we are now familiar with was first added, allowing individual processes to select a set of rules.

I enjoyed the filesystem chapter as I learnt about the old berkley filesystem and the “new” which evolved into what we use today, the performance issues with the old filesystem due to the free list becoming scrambled with the age of the filesystem (in weeks), resulting in longer seek times and the amount of space wasted as a function of block size.

Although the old filesystem provided transfer rates of up to 175 Kbyte per second when it was first created, the scrambling of the free list caused this rate to deteriorate to an average of 30 Kbyte per second after a few weeks of moderate use.

The idea of being rotationally optimal to reduce seek times and implementing mechanisms to account for that was very interesting to read about.

To simplify the task of locating rotationally optimal blocks, the summary information for each cylinder group includes a count of the available blocks at different rotational positions. Eight rotational positions are distinguished, so the resolution of the summary information is 2 milliseconds for a 3600 revolution-per-minute-drive.

Though this is not so valid today with traditional spindle disks as there is not a 1:1 mapping between the physical location & logical representation of the blocks on disk.

The book is a bargain second hand and worth it for the BSD archeology.

Two months after the beginning of the first implementation of the UNIX operating system, there were two processes, one for each of the terminals of the PDP-7. At age 10 months, and still on the PDP-7, UNIX had many processes, the fork operation, and something like the wait system call. A process executed a new program by reading a new program in on top of itself. The PDP-11 system (first edition UNIX) saw the introduction of exec. All these systems allowed only one process in memory at a time. When PDP-11 with memory management (a KS-11) was obtained, the system was modified to permit several processes to remain in memory simultaneously, in order to reduce swapping. But this modification did not apply to multiprogramming, because disk I/O was synchronous. This state of affairs persisted into 1972 and the first PDP-11/45 system. True multiprogramming was finally introduced when the system was rewritten in C. Disk I/O for one process could then proceed while another process ran. The basic structure of process management in UNIX has not changed since that time.

Building RetroBSD on Mac OS X

Earlier this week it was announced on the RetroBSD forum that the official repository had moved to GitHub. Along with the move came changes which fixed previous issues with the build so now it is possible to build a RetroBSD image on Mac OS X.

As covered in the RetroBSD installation document, I installed the required dependencies from MacPorts & used the prebuilt PIC32 toolchain to build RetroBSD.

port install bison byacc flex libelf

The compiler was uncompressed to /usr/local to keep it inline with the code base & reduce the number of changes need to get thing to build.

I’d previously installed the command line tools from within Xcode, while this had installed clang & the rest of the toolchain into /usr, I was missing /usr/include, installing the command line tools manually with xcode-select --install fixed this.

Kernel built from the SVN code base on Google Code:
2.11 BSD Unix for PIC32, revision 892M build 2:
Compiled 2014-03-29 by xxx@xxx:
/xxx/retrobsd/sys/pic32/max32-eth
cpu: 795F512L 80 MHz, bus 80 MHz
oscillator: XT crystal, PLL div 1:2 mult x20
console: tty0 (5,0)
sd0: port SPI2, select pin C14
sd0: type SDHC, size 15339520 kbytes, speed 13 Mbit/sec
phys mem = 128 kbytes
user mem = 96 kbytes
root dev = rd0a (0,1)
root size = 102400 kbytes
swap dev = rd0b (0,2)
swap size = 2048 kbytes
temp0: allocated 30 blocks
/dev/rd0a: 588 files, 8752 used, 93247 free
temp0: released allocation
Starting daemons: update cron

2.11 BSD UNIX (pic32) (console)

Kernel built from the git code base on GitHub:
2.11 BSD Unix for PIC32, revision G38 build 1:
Compiled 2014-04-19 by xxx@xxx:
/xxx/retrobsd/sys/pic32/max32-eth
cpu: 795F512L 80 MHz, bus 80 MHz
oscillator: XT crystal, PLL div 1:2 mult x20
console: tty0 (5,0)
sd0: port SPI2, select pin C14
sd0: type SDHC, size 15339520 kbytes, speed 13 Mbit/sec
phys mem = 128 kbytes
user mem = 96 kbytes
root dev = rd0a (0,1)
root size = 102400 kbytes
swap dev = rd0b (0,2)
swap size = 2048 kbytes
/dev/rd0a: 588 files, 8752 used, 93247 free
Starting daemons: update cron

2.11 BSD UNIX (pic32) (console)

RetroBSD on chipKIT Max32 with Arduino Ethernet shield

I bought myself one of these boards to play with RetroBSD a few months back but with build issues & other distractions in real life I didn’t get very far because the stock MAX32 kernel configuration is for SD card slots wired to the board whereas I was trying to use the SD card slot on an ethernet shield. This resulted in the kernel not being able to detect the SD card, hence no root file system. I’d use prebuilt binaries without trying to build my own but for some reason there is no prebuilt kernel image for the configuration I require. Finally, tonight I managed to get a MAX32-ETH kernel built & flashed onto the board, still unable to build the userland but I was able to get an older prebuilt image from the RetroBSD Autobuild Server which currently has images for build 860 & 778.

Will follow up once I have everything building from scratch.
Building RetroBSD on Mac OS X

2.11 BSD Unix for PIC32, revision 892 build 1:
Compiled 2014-03-29 by xxx@xxx.xxx:
/usr/local/retrobsd/sys/pic32/max32-eth
cpu: 795F512L 80 MHz, bus 80 MHz
oscillator: XT crystal, PLL div 1:2 mult x20
console: tty0 (5,0)
sd0: port SPI2, select pin C14
sd0: type SDHC, size 15339520 kbytes, speed 13 Mbit/sec
phys mem = 128 kbytes
user mem = 96 kbytes
root dev = rd0a (0,1)
root size = 163840 kbytes
swap dev = rd0b (0,2)
swap size = 2048 kbytes
temp0: allocated 47 blocks
/dev/rd0a: 576 files, 8644 used, 154555 free
temp0: released allocation
Starting daemons: update cron

2.11 BSD UNIX (pic32) (console)

login:

Building tcsh on 4.3BSD-Quasijarus

So after build installing 4.3BSD on simh I was left with a somewhat bare environment.
My first need was a better shell than stock csh with history & tab completion. I proceeded to download the latest version of tcsh to attempt a build.
Things fell into place quickly as I realised this was not going to be as straight forward.
At some point in the development path of tcsh, support for the GNU tool chain was added which would allow one to use autotools to build the software.
Bootstrapping the baseline is always a lengthy pain which prepackaged binaries ease. I was reminded of the period I avoided using OpenBSD-CURRENT because I was unaware packages were available so I would resort to painfully long builds as I started to tried to compile firefox on a freshly installed system.
I decided to see how far I can get in building software without the introduction GNU components, not using up to date versions is acceptable just so I can achieve the task. This also falls in line with the philosophy of the 4.3BSD-Quasijarus project.

I struggled to get tcsh 6.18.01 to build so thought I’d try the earliest version I could find on the official mirror to see if that would work any better.
Version 6.00 is the earliest one available on the US FTP site, the README file references 4.2 & 4.3BSD among a whole bunch of other legacy OS’s & no sign of GNU autoconf.
With a bit of experimentation I had tcsh compiled, essentially just needed to undefine functionality that was not present in my stock install (NLS) or available in the version I was running (better POSIX compliance).
There is lots of code which is spaced out so as to disable its functionality requiring the deletion of the spaces to re-enable depending on the intended target.
This approach assumes knowledge of the OS & its capabilities along with the functionality enabled in the version installed as apposed to the autotools approach where the developer is responsible for putting together something which in theory will reduced the requirement from the end user installing the software to running configure & make.
In my opinion it’s an idea which is poorly executed while not actually solving the problem, instead pushing the responsiblity from one end to the other with added complexity.
I discovered through an interview with Larry Wall that he was responsible for the patch utility we commonly use, I was curious to see how it would function on 4.3BSD & to my surprise there was no patch utility in base. There is the diff utility for generating differential patches but no support for unified context just copied context. You can generate scripts for ed which can be used to apply the changes for subsequent users.

To build tcsh on 4.3BSD Qausijarus
Fetch the tcsh-6.00.00.tar.gz from FTP
Uncompress with
gzcompat tcsh-6.00.00.tar.gz | uncompress | tar xf -

Copy tcsh-6.00/config/config.bsdreno to tcsh-6.00/config.h

Apply the following changes in this diff to your source & build by running make

By default, the install process puts files into /usr/local so you’ll need to create /usr/local/bin & /usr/local/man

> echo $version
tcsh 6.00.00 (Cornell) 07/04/91 options 8b,dl,al,dir

Last week I came across a project to develop 4.5BSD, a continuation from where 4.4BSD stopped. Comparing the current state of the modern BSD derivatives, we’ve come a long long way, with effort going in to reduce the need to bring the system to a useable state post install & shipping with sane defaults we are in a much better place. 20 years worth of changes to back port is a serious effort.

Running 4.3BSD Quasijarus with simh VAX

I’d previously installed simh via macports & though it was supposedly built with networking support, the guests didn’t actually work with networking on mavericks.
I gave the v4.0 beta a try to see if the problem existed, following the simh Ethernet document I built simh using
make USE_NETWORK=1 LPATH=/usr/lib:/usr/local/lib:/opt/local/lib with libpcap installed via macports & tuntap installed from sourceforge (it is available from macports if you prefer as tuntaposx).
Networking support turned out to work fine with the stock MicroVAX 3900 simulator, failing on the other VAX guests (11/{730,750,780} and 8600) with an error about a non-existent device.

To install 4.3BSD Quasijarus on simh I followed this guide fetching the Mkdisttap.pl script & gzcompat.c from the Computer History Wiki.

I used the following ini file I used for the guest once everything was installed

load -r /opt/local/share/simh/VAX/ka655x.bin
set cr dis
set lpt dis
set rl dis
set ry dis
set ts dis
set rq0 ra82
att rq0 quas.dsk
set rq1 dis
set rq2 dis
set rq3 dis
att tq0 quas.tap
set dz lines=8
att dz 8888
set dz 7b
att xq tap:tap0
set cpu idle=ALL
att nvr nvram.dat
dep bdr 1
! ifconfig tap0 192.168.6.1 netmask 255.255.255.0

With the ini file above, the host is assigned the IP address 192.168.6.1, serial console listening on TCP port 8888 and IDLE loop detection works with set cpu idle=ALL, this means that a guest sits at around 3% CPU & only goes up to a maximum of 70% CPU on my macbook air when compiling software as opposed to 100% at all time. As the application needs to bind to a tap interface & set an IP address it needs to be run as root.

20131209-031911.jpg

Running IPv6 on 4.4BSD – Summer 2012 Retrochallenge entry

I’m entering the summer Retrochallenge which runs throughout July, the plan is to get an emulated host running with 4.4BSD-lite connected via IPv6 using the NRL patches & if possible serving a basic webpage over HTTP or fail at it & either way document the journey in this post.
It will be a good opportunity to visit the early implementation of IPv6 & learn about the origin of this stack, IPv6 is sexy right? or is that DNS??
I was unaware of the existence of these patches & assumed KAME was the original implementation until I was pointed to the USENIX paper by fellow tweep Dan McDonald.
Looking forward to doing some digging for information & patches 🙂
To fit the challenge rules, the host the emulation will run on will be a Cobalt Qube2 with NetBSD/Cobalt.

3/7/12
Before I started to attempt building things I did some research on what was required & if the patches and any documentation was still around.
Sun Microsystems IPng Implmentations page is still available on archive.org though the NRL IPv6+IPsec is long gone, I was able to obtain the patch sets from the RIPE FTP server
The installing & operating 4.4BSD UNIX section of UNIX System Manager’s Manual (SMM) lists the supported platforms by 4.4BSD, I hadn’t realised that VAX was not a officially supported platform in this release by CSRG, the 3 official supported platforms which binaries are shipped for being the HP, DECstation & SPARC platforms, this means that simh is off the list & it’s a toss up between tme & GXemul, I’ve not played with GXemul before but tried the Sun3 emulation on tme a few years back in an attempt to get SunOS 4.x but failed after not getting the disk setup right.
I found a copy of the 4.4BSD-Lite CD ISO files on Poul-Henning Kamps website.
Next step is to get a virtual machine up & running with 4.4BSD-Lite, I’m probably going to start with the DECstation (MIPS) emulation on GXemul & move onto SPARC on tme if things don’t work out. The README with patches states the patches were built on SPARC but I’m hoping this isn’t an issue (endianness?).

21/7/12
My CSRG archives cd set is here, this week will be a cram as I play catch up.
Front cover for CSRG Archives cd set

Back cover for CSRG Archives CD set