GeekLAN

19/08/2014

A week of pkgsrc #3

Filed under: OS X — Tags: , , — Venture37 @ 5:25 pm

Didn’t uncover anything new in pkgsrc last week as my attention was more on coreboot, I had previously been building different parts of the tree on a couple of Mac’s which where disconnected from each other & copying packages to sevan.mit.edu manually for serving, as a first off this was a good idea but bad as an ongoing thing. What ends up happening is stale packages become left behind as they are unaccounted for, luckily there aren’t too many duplicates currently but it’s something which needs to be addressed in the set of packages currently available.

There is now a page on the NetBSD wiki to keep note of issues & ideas.

To test the status of AIX support in pkgsrc I joined the IBM Power Developer Platform which provides access to Power7/7+/8 systems running AIX 6.1 & 7.1 to build software on. This’ll be my first time on a Power system & AIX, looking forward to seeing the OS is like.

System reservation on IBM PDP

With the addition of a G5 iMac to the effort kindly donated again by Thomas Brand, I started testing builds of lang/gcc48 on sevang5.mit.edu. Next step will be to get the two systems at MIT working together to build packages once I’ve been able to get GCC 4.8 to build successfully.

11/08/2014

A week of pkgsrc #2

Filed under: OS X — Tags: , , — Venture37 @ 4:19 pm

Following on from last week, I worked on components which caused large numbers of packages not to build.
textproc/icu failed to build due to localtime_r() not being used if either _ANSI_SOURCE or _POSIX_C_SOURCE is defined & using an opcode that the shipped version of assembler didn’t understand. Ticket #9367 provided fixes for both issues spanning over 2 years, pkg/49077 covers this but has not been committed.
databases/sqlite3 failed to link with ld: Undefined symbols: _OSAtomicCompareAndSwapPtrBarrier error, this is due to the lack of zone memory allocator, PR #49081 fixed this issue by defining -DSQLITE_WITHOUT_ZONEMALLOC for OS X releases prior to Leopard. This is PR was committed. A subsequent PR (pkg/49082) was raised to do the same for lang/tcl which also bundles its own copy of sqlite3 for its sqlite module, but has not been committed.

devel/pango was broken on OS X releases prior to Leopard as the package enabled the CoreText option by default but failed due to packing errors  (CoreText is not available hence the .la file not existing when build has completed). pkg/49090 resolved the issue & was committed.

Packages for GCC 4.4 to 4.6 are now available, lang/gcc47 failed to build successfully with sh consumed all resources on a CPU before being terminated manually.

sh(22232) malloc: *** error for object 0x34e340: incorrect checksum for freed object - object was probably modified after being freed, break at szone_error to debug
sh(22232) malloc: *** set a breakpoint in szone_error to debug
sh(22232) malloc: *** Deallocation of a pointer not malloced: 0x34d7ab; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug
sh(22232) malloc: *** Deallocation of a pointer not malloced: 0x34e340; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug
checking sys/time.h usability... Makefile:16170: recipe for target 'configure-stage2-target-libgomp' failed
gmake[2]: *** [configure-stage2-target-libgomp] Error 137

This behaviour has previously been observed when attempting to build GCC on the PowerBook

The stability of sevan.mit.edu was improved by re-applying the 10.4.11 combo update.

Currently in the process of fixing devel/cmake, cmake now get through most of the build (it was previously failed at 3%) but fails at the linking stage due to path issues. It picks up the pkgsrc version of CURL as /usr/pkg/bin/curl but tries to link against libraries in /Developer/SDKs/MacOSX10.4u.sdk/usr/pkg/lib which doesn’t exist.

The TenFourFox blog mentioned the effort thanks to Cameron Kaiser of Floodgap.

10/08/2014

coreboot on PC Engines Alix 2c3

Filed under: General — Tags: , — Venture37 @ 12:29 am

IMG_2883.JPG
I’ve been following coreboot (formally LinuxBIOS) for quite a while but my last attempt to run it ended disastrously a couple of years back on the defunct coreboot 3 branch, requiring an RMA for a new Alix 2c3 board & the purchase of additional tools.
A ThinkPad X60s was offered up by my friendly local “old computer” pusher which I chose to try coreboot on but before I bricked that, I thought I’d try the Alix again as I was in possession of the everything needed to debug or recover the board if things went wrong.
A lot of time was wasted trying to get things built on OS X & diverged into briefly trying CentOS 7 (systemd, run away) before settling on Debian 7.6 to build on.

Running apt-get install gcc make libncurses-dev doxygen iasl gdb flex bison will install the necessary dependencies to build coreboot on Debian following the build guide.

Selecting a board from the menu should set the ROM size as well but I was advised to double check, on the Alix 2c3 the flash chip is located under the board, my board had a AMIC A49LF040ATY-33F.
IMG_2891

Once the image was built, coreboot.rom was copied to the Alix running FreeBSD/i386 10-RELEASE & flashed with sysutils/flashrom built from ports.
Flashrom currently assumes that there’s a device node named /dev/cpu0 and sysutils/devcpu-data offers the necessary device but on FreeBSD it’s named /dev/cpuctl0, a symlink allows you to work around the hardcoded assumption once the micro_code service has been started.

service micro_code onestart
ln -s /dev/cpuctl0 /dev/cpu0

coreboot can then be programmed with flashrom -p internal:laptop=this_is_not_a_laptop -w ~/coreboot.rom

flashrom v0.9.7-r1711 on FreeBSD 10.0-RELEASE-p7 (i386)
flashrom is free software, get the source code at http://www.flashrom.org

Calibrating delay loop… delay loop is unreliable, trying to continue OK.
========================================================================
WARNING! You may be running flashrom on an unsupported laptop. We could
not detect this for sure because your vendor has not setup the SMBIOS
tables correctly. You can enforce execution by adding
‘-p internal:laptop=this_is_not_a_laptop’ to the command line, but
please read the following warning if you are not sure.

Laptops, notebooks and netbooks are difficult to support and we
recommend to use the vendor flashing utility. The embedded controller
(EC) in these machines often interacts badly with flashing.
See the manpage and http://www.flashrom.org/Laptops for details.

If flash is shared with the EC, erase is guaranteed to brick your laptop
and write may brick your laptop.
Read and probe may irritate your EC and cause fan failure, backlight
failure and sudden poweroff.
You have been warned.
========================================================================
Proceeding anyway because user forced us to.
Found chipset “AMD CS5536″. Enabling flash write… OK.
Warning: unexpected second chipset match: “AMD CS5536″
ignoring, please report lspci and board URL to flashrom@flashrom.org
with ‘CHIPSET: your board name’ in the subject line.
Found AMIC flash chip “A49LF040A” (512 kB, LPC) at physical address 0xfff80000.
Reading old flash chip contents… done.
Erasing and writing flash chip… Erase/write done.
Verifying flash… VERIFIED.

Power cycling the box will result in text output or garbage depending on if your console speed settings have changed or not, coreboot defaults to the speed of 115200bps, the factory default speed of Alix is 38400bps & the default console speed of FreeBSD is 9600bps.

When attempting to generate a new image with different settings be sure to make clean before starting.
In my image I removed the “PS/2 keyboard init” option from the “Generic Drivers” menu. Still todo is building an image with PXE support, test booting other BSD’s, try different payloads, in particular Open Firmware & generate status data for submission to change board support status on wiki.

IMG_2886.JPG

Through the previously failed attempt to run coreboot 3 I ended up with a LPC.1a & a POST.5a mini-pci board, the LPC.1a is absolutely essential for testing if you do not have more advanced equipment which would allow you to re-program the flash chip. The LPC.1a is a secondary BIOS chip which can override the onboard the flash, allowing you to boot the system & reprogram the onboard chip again. J2 jumper allows you to select which chip to boot from. When reflashing the chip on the motherboard with flashrom with the LPC.1a inserted (and set to read only), flashrom trashed the image on the onboard chip. Rebooting the system & carefully removing the LPC.1a before re-flashing allowed the process to complete successfully.

05/08/2014

Issuing secure erase ATA command using camcontrol(8)

Filed under: FreeBSD — Tags: , , — Venture37 @ 1:28 am

The ATA command set has a command to instruct a device to secure erase itself.
Depending on the application & level of sensitivity of the data on disk, it can be a convenient way to decommission a disk or reset an SSD to regain performance. On FreeBSD this can be issued using camcontrol(8).

The command below performs an enhanced erase with a timeout of 60 seconds for the command to be accepted by the disk, this is needed if you get timeout errors when you do not specify it.
camcontrol security ada0 -U user -s Erase -h Erase -T 60

04/08/2014

A week of pkgsrc #1

Filed under: OS X — Tags: , , — Venture37 @ 6:50 pm

This is summary of the things I worked on along with the help of others over the last week on pkgsrc.
With the donation of sevan.mit.edu along with a G4 Mac Mini at pksrcCon 2014 I setup bulk package builds as per chapter 7 of the pkgsrc guide to generate packages for OS X.
The bootstrap process is now able to differentiate between gcc & clang, as clang tries to be GCC compatible it tries to pass itself as GCC in tests, this would cause an issue where the bootstrap would use /usr/bin/clang for some parts of the build & /usr/bin/gcc for others, on top of that, the bootstrap process was hardcoded to use gcc on Darwin. The bootstrap process now defaults to using cc & correctly detects if that is clang or gcc.

By default git attempts to use the Apple CommonCrypto framework which meant it would only build successfully on Leopard or newer, devel/git-base now links against openssl instead which means it’s consistent with other platforms using pkgsrc as well as being able to build on older releases of Mac OS X. Unprivileged builds of this are still currently broken on Tiger as tar tries to set the group ownership of files to wheel, a patch to fix the issue is awaiting to be committed.

security/sudo now builds on Darwin (confirmed on Tiger PowerPC & Mavericks), the no_exec module doesn’t build on Darwin & is switched off in the Apple supplied build of sudo, this wasn’t switched off in pkgsrc & caused the build to fail. There are more options set in the Apple build to improve posture which are not set in pkgsrc version, that needs looking into further & is on the TODO list.

The new release of help2man committed last week broke on Tiger due to NLS being switched off & the new version introducing additional translations of info pages. The patch in pkg/49059 fixes things so shared libraries are taken care of as with Leopard & the package is built with NLS support.

Currently working on trying to get graphics/MesaLib building with XQuartz, the version shipped with Tiger is based on XFree86 & MesaLib fails to link libraries, macports seem to have some fixes related to building on Tiger which I’m hoping may fix some of the issues.

Will also be looking at devel/cmake as it’s currently broken on Tiger which means things such as mysql server cannot be built at the moment.

Through the existence of a directory called devel in /tmp which was owned by a user other than the the one pbulk runs under, some critical components such as autoconf & tradcpp did not build on the Mac Mini, this caused many builds to fail, that aside, the Mini has managed to build 1064 out of a queue of 2083 packages over the last week.
sevan.mit.edu is currently down (due to possible hardware issues) & awaiting a reboot.

02/08/2014

Packages for PowerPC Mac OS X with pkgsrc

Filed under: OS X — Tags: , , — Venture37 @ 5:33 pm

In pkgsrc there’s a facility which allows you to perform bulk builds of packages called pbulk.
Using this facility on a couple of donated systems I have started to generate packages for PowerPC OS X. Currently builds are performed on 32bit PowerPC Macs running OS X with pkgsrc-current. The binaries should in theory work on 64bit PowerPC systems and on Leopard but have not tested to confirm.
The packages are made available at sevan.mit.edu.
To utilise the packages on your system, fetch & uncompress the bootstrap archive which contains the pkgsrc tools.
curl -s http://sevan.mit.edu/packages/bootstrap.tar.gz | sudo tar -zxpf - -C /

Update your PATH & MANPATH variables
PATH=/usr/pkg/sbin:/usr/pkg/bin:$PATH
On Tiger
MANPATH=/usr/pkg/man can be declared in /usr/share/misc/man.conf
On Leopard use path_helper(8) and create a file in /etc/manpaths.d which just contains /usr/pkg/man.
This can also be extended to PATH by creating a file in /etc/paths.d/ containing one path element per line. This requires testing however as the impact is system wide.

Set PKG_PATH to http://sevan.mit.edu/packages/All/

Packages can then be installed using the pkg_add command, for example to install wget
pkg_add wget

This service is very much in its infancy & not stable yet, the current offering of packages is small but more packages are building on a daily basis albeit very slowly due to the age of the hardware.

If you’re interested in pkgsrc on Intel Macs try the Save OS X blog and Joyent packages which offer packages for Ilumos derivatives, Linux as well as OS X on Intel hardware.

Thanks to the generosity of David Brownlee, Thomas Brand & Justin Cormack for their generous donation of hardware.

31/07/2014

Using ifstated to monitor links and dynamically adjust PF config on event

Filed under: OpenBSD — Tags: , , — Venture37 @ 11:30 am

It’s possible to misuse NAT to load balance outbound traffic across multiple internet connections from different service providers,see the Load Balance Outgoing Traffic section of PF FAQ.
The shortfall with this configuration is when implemented alongside unstable links, forwarding will continue to be attempted over the links which are down, this will cause issues such as long hangs for users behind the NAT while connections time out. To mitigate this, ifstated can be used to smooth things over.
ifstated can be used to run tests & on event perform tasks, if you’re familiar with Cisco IOS, this is similar to some of what is available in EEM. In this scenario, ifstated will be set to ping each gateway at the service provider end of each link every 10 seconds & upon failure, adapt the configuration so traffic is not forwarded down that link. ifstated will continue to perform the tests & when tests start passing because link has re-established successfully, ifstated will reconfigure the system again so links are utilised.

For this post we’ll use the example ruleset from the PF FAQ and adapt it so it can be manipulated by ifstated.

Original pf.conf

lan_net = "192.168.0.0/24"
int_if = "dc0"
ext_if1 = "fxp0"
ext_if2 = "fxp1"
ext_gw1 = "198.51.100.100"
ext_gw2 = "203.0.113.200"

# nat outgoing connections on each internet interface
match out on $ext_if1 from $lan_net nat-to ($ext_if1)
match out on $ext_if2 from $lan_net nat-to ($ext_if2)

# default deny
block in
block out

# pass all outgoing packets on internal interface
pass out on $int_if to $lan_net
# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if
# load balance outgoing traffic from internal network.
pass in on $int_if from $lan_net \
route-to { ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } \
round-robin
# keep https traffic on a single connection; some web applications,
# especially "secure" ones, don't allow it to change mid-session
pass in on $int_if proto tcp from $lan_net to port https \
route-to ($ext_if1 $ext_gw1)

# general "pass out" rules for external interfaces
pass out on $ext_if1
pass out on $ext_if2

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
pass out on $ext_if1 from $ext_if2 route-to ($ext_if2 $ext_gw2)
pass out on $ext_if2 from $ext_if1 route-to ($ext_if1 $ext_gw1)

Modified pf.conf

lan_net = "192.168.0.0/24"
int_if = "dc0"
ext_if1 = "fxp0"
ext_if2 = "fxp1"
ext_gw1 = "198.51.100.100"
ext_gw2 = "203.0.113.200"

# nat outgoing connections on each internet interface
anchor nat-isp1
anchor nat-isp2

set skip on lo

# default deny
block in
block out

anchor "ftp-proxy/*"

# pass all outgoing packets on internal interface
pass out on $int_if to $lan_net
# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if
# load balance outgoing traffic from internal network.
anchor loadbalance

# keep https traffic on a single connection; some web applications,
# especially "secure" ones, don't allow it to change mid-session
anchor applications

# general "pass out" rules for external interfaces
pass out on $ext_if1
pass out on $ext_if2

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
anchor pass-isp1
anchor pass-isp2

The rules for NAT, load balancing & routing are replaced with anchors, ifstated will use these anchors to add & manipulate rules.

ifstated.conf

isp1 = '( "ping -q -c 1 -w 1 -S 198.51.100.199 198.51.100.100 >/dev/null" every 10)'

#If inteface is configured dynamically via dhcp use this instead
#isp2 = '( "ping -q -c 1 -w 1 -S `ifconfig vr2 inet |awk \'/inet/ { print $2 }\'` `awk \'/routers/ { print $3 }\' /var/db/dhclient.leases.vr2 |tail -1 |sed \'s/;//\'`>/dev/null" every 10)'

isp2 = '( "ping -q -c 1 -w 1 -S 203.0.113.220 203.0.113.200 >/dev/null" every 10)'

state allworking {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'
run 'pfctl -a pass-isp2 -F rules'

run 'route change default 203.0.113.200'

run 'echo "pass in on vr1 from 192.168.1.0/24 \
route-to { (vr0 198.51.100.100), (vr2 203.0.113.200) } round-robin" | pfctl -a loadbalance -f -'

run 'echo "pass in on vr1 proto tcp from 192.168.1.0/24 to port https route-to (vr2 203.0.113.200)" | pfctl -a applications -f -'

run 'echo "match out on vr0 from 192.168.1.0/24 nat-to (vr0)" | pfctl -a nat-isp1 -f -'

run 'echo "match out on vr2 from 192.168.1.0/24 nat-to (vr2)" | pfctl -a nat-isp2 -f -'

run 'echo "pass out on vr0 from vr2 route-to (vr2 203.0.113.200)" | pfctl -a pass-isp2 -f -'

run 'echo "pass out on vr2 from vr0 route-to (vr0 198.51.100.100)" | pfctl -a pass-isp1 -f -'
}
if ! $isp1
set-state noisp1
if ! $isp2
set-state noisp2
}

state noisp1 {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'

run 'route change default 203.0.113.200'

run 'echo "pass in on vr1 from 192.168.1.0/24 route-to { (vr2 203.0.113.200) }" | pfctl -a loadbalance -f -'

run 'echo "pass in on vr1 proto tcp from 192.168.1.0/24 to port https route-to (vr2 203.0.113.200)" | pfctl -a applications -f -'

run 'echo "match out on vr2 from 192.168.1.0/24 nat-to (vr2)" | pfctl -a nat-isp2 -f -'

run 'echo "pass out on vr2 route-to (vr2 203.0.113.200)" | pfctl -a pass-isp2 -f -'
}
if $isp1
set-state allworking
if ! $isp2
set-state alldown
}

state noisp2 {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'

run 'route change default 198.51.100.100'

run 'echo "pass in on vr1 from 192.168.1.0/24 route-to { (vr0 198.51.100.100) }" | pfctl -a loadbalance -f -'

run 'echo "pass in on vr1 proto tcp from 192.168.1.0/24 to port https route-to (vr0 198.51.100.100)" | pfctl -a applications -f -'

run 'echo "match out on vr0 from 192.168.1.0/24 nat-to (vr0)" | pfctl -a nat-isp1 -f -'

run 'echo "pass out on vr0 route-to (vr0 198.51.100.100)" | pfctl -a pass-isp1 -f -'
}
if ! $isp1
set-state alldown
if $isp2
set-state allworking
}

state alldown {
init {
run 'pfctl -a loadbalance -F rules'
run 'pfctl -a applications -F rules'
run 'pfctl -a nat-isp1 -F rules'
run 'pfctl -a nat-isp2 -F rules'
run 'pfctl -a pass-isp2 -F rules'
run 'pfctl -a pass-isp1 -F rules'
}
if $isp1 && ! $isp2
set-state noisp2
if $isp2 && ! $isp1
set-state noisp1
if $isp1 && $isp2
set-state all working
}

As ifstated is initialised & when it switches states, it flushes the anchors in the pf.conf, sets the default gateway so the host itself can be reachable remotely on the WAN and then injects rules into the PF anchors.

26/07/2014

Switching from Zevo to OpenZFS on OS X

Filed under: OS X — Tags: , — Venture37 @ 7:54 pm

I recently moved my last Mac from Greenbytes Zevo to OpenZFS on OS X, the reason for both sticking with Zevo & switching to OpenZFS were one and the same, CPU usage.
Prior to the development of OpenZFS on OS X, the two choices for using ZFS on OS X where Zevo or MacZFS, Zevo originally started out as a commercial product but switched to a freebie after Greenbytes picked it up. Zevo had much better integration with OS X e.g disk would be automatically mounted when connected to system just like any other disk with a supported file system and it supported a v28 of the filesystem whereas MacZFS supported a much older version.

When the OpenZFS on OS X development began just over a year ago, I ran the test builds that where made available, though these supported new features through feature flags it was very early days, attempting to scrub a zpool on a i7 MacBook Air with a USB 3 disk would spike the CPU for the duration and again the integration was still missing, you manually had to import & export pools. I continued to try newer builds on my MacBook Air but stuck with Zevo on my 2007 MacBook Pro.

The two things which where annoying about Zevo was that it was a dead end, development had stopped, the last version available wasn’t compatible with Mavericks available and its conservative memory setting meant that disk performance wasn’t that great, during playing audio files it would break to buffer audio in iTunes for example (luckily not in Serato as mid set would’ve been embarrassing).

As the MacBook Pro was running low on disk space I tried to move around 40GB of files in several chunks in parallel to my external USB3 disk & noticed the CPU pegged and fans started up with Zevo too. OpenZFS on OS X is fairly robust now (though still rough around the edges) so I decided to switch over.

The OpenZFS on OS X disk image comes with uninstall scripts for Zevo & though the main script was unable to detect the installed copy of Zevo, I was able to run the subsequent scripts individually to remove Zevo from my system and reboot (eject the disk containing the filesystem beforehand (export the zpool)).

The integration with OS X is still missing though it seems that on boot zpools are imported, I’ve not worked out if that’s because the system caches the state from previous boot or this is the preliminary support for auto mounting???

If you want to eject a disk, you still have to export the pool manually from terminal, pressing the eject button in finder will remove the disk icon but the filesystem is still mounted. That aside, OpenZFS on OS X performed well, scrubbing the zpool on the 2007 MacBook Pro did not cause the CPU to spike at all, there is now a shorter delay in iTunes when starting to play a track but haven’t noticed any drops in audio yet, so things are looking positive.

Scrubbing the zpool on a 2007 17″ MacBook Pro with 4GB RAM

pool: tank
state: ONLINE
scan: scrub in progress since Fri Jul 25 18:58:48 2014
28.2G scanned out of 579G at 30.1M/s, 5h11m to go
0 repaired, 4.87% done
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
disk1s2 ONLINE 0 0 0

errors: No known data errors

All properties on the zpool I was using:
NAME PROPERTY VALUE SOURCE
tank type filesystem -
tank creation Mon Jul 29 5:00 2013 -
tank used 579G -
tank available 1.22T -
tank referenced 579G -
tank compressratio 1.00x -
tank mounted yes -
tank quota none default
tank reservation none default
tank recordsize 128K default
tank mountpoint /tank default
tank sharenfs off default
tank checksum on default
tank compression off default
tank atime on default
tank devices on default
tank exec on default
tank setuid on default
tank readonly off default
tank zoned off default
tank snapdir hidden default
tank aclmode discard default
tank aclinherit restricted default
tank canmount on default
tank xattr on default
tank copies 1 default
tank version 5 -
tank utf8only on -
tank normalization formD -
tank casesensitivity sensitive -
tank vscan off default
tank nbmand off default
tank sharesmb off default
tank refquota none default
tank refreservation none default
tank primarycache all default
tank secondarycache all default
tank usedbysnapshots 0 -
tank usedbydataset 579G -
tank usedbychildren 4.48M -
tank usedbyrefreservation 0 -
tank logbias latency default
tank dedup off default
tank mlslabel none default
tank sync standard default
tank refcompressratio 1.00x -
tank written 579G -
tank logicalused 578G -
tank logicalreferenced 578G -
tank snapdev hidden default
tank com.apple.browse on default
tank com.apple.ignoreowner off default

Upgrading the zpool with OpenZFS on OS X
This system supports ZFS pool feature flags.

Successfully upgraded 'tank' from version 28 to feature flags.
pool_set_props
Enabled the following features on 'tank':
async_destroy
pool_set_props
empty_bpobj
pool_set_props
lz4_compress

After Dark flying toasters clone

Filed under: OS X — Tags: , , — Venture37 @ 6:04 pm

Many years ago, the Uneasy Silence website had a clone of the After Dark flying toasters screensaver. Uneasy Silence is now defunct but archive.org has many snapshots of the site where you can download the Windows & OS X version of the screensaver & source code still.

 

Modern flying toasters

20/07/2014

GeekLAN is 10 years old

Filed under: General — Venture37 @ 8:54 pm

This blog started life with a zero dot release of wordpress on the end of a 512kb down/128k up cable modem connection 10 years ago. Originally I had intended to host it on my AlphaStation which at the time was acting as my gateway, running OpenBSD. Unfortunately gettext was broken on Alpha at the time which meant though php was available I couldn’t build extensions such as the mysql one, I had a slot 1 PIII which was my previous gateway using RRAS on Windows 2000 Server, it replaced the AlphaStation and assumed the role of gateway again, this time on OpenBSD.

Up until 2009 this blog was served from my bedroom by then on a VIA C7 mini-its board with a ADSL connection. At some point it gained an SSL certificate from CAcert & IPv6 connectivity. Through this domain I discovered that NTL overrode the TTL values for records in the early days, caching DNS records for a week by default. Blocked several IP addresses from Thailand for excessive hits to the site. Of all of the computers which I collected over the years, most are now gone. I still have the Cobalt Qube2, some Macs and the ThinkPad X61s, the rest found new homes or where thrown away. The most popular posts so far have been on Apple products, the post that’s still holds true is the Solaris installer misreporting disk failure if it finds a disk label other than the one it was expecting.

 

30/06/2014

Book review: The Art of Unix Programming

Filed under: General — Tags: , — Venture37 @ 10:59 pm

I picked this book by mistake, assuming that it was going to be a technically detailed book in line with the Advanced Programming in the Unix Environment book written by the late Richard Stevens, it turned out to be much more high level than that but I was not disappointed, It’s been a pleasure to read whilst travelling over the last month.
The book is 20 chapters split across four parts (context, design, implementation, community) with commentary from some big names of the UNIX world. There are lots of great advice in the book but I would look at what’s now available in regards to software today if I was looking to implement something. It does explain why lots of software relies on some common (and heavy weight?) components. Let me explain, long ago I was unaware that packages for the -current branch of OpenBSD were being built, whenever I grudgingly tried a new snapshot I went through & built my packages from the ports tree after a fresh install, then something would depend on XML related components & then pull in a bunch of things which would involve building ghostscript, on a Sun Blade 100, between Firefox & ghostscript, 24 hours would easily be wasted, I now understand that all that wasted time was thanks to someone taking the advice of ESR on how to prepare documentation for a software project.
Besides the dubious software recommendation (11-year-old book?) everything is explained in a clear manner that’s very easy to read.

Rule of Robustness: Robustness is the child of transparency and simplicity.
Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.
Rule of Optimization: Prototype before polishing. Get it working before you optimize it.
Rule of Diversity: Distrust all claims for “one true way”.
Rule of Extensibility: Design for the future, because it will be here sooner than you think.

The Pragmatic Programmer articulates a rule for one particular kind of orthogonality that is especially important. Their “Don’t Repeat Yourself” rule is: every piece of knowledge must have a single, unambiguous, authoritative representation within a system. In this book we prefer, following a suggestion by Brian Kernighan, to call this the Single Point Of Truth or SPOT rule.

The book is critical of Microsoft & their approach to software, explaining some of the design decisions (some inherited from the world of VMS).

From a complexity-control point of view, threads are a bad substitute for lightweight processes with their own address spaces; the idea of threads is native to operating systems with expensive process-spawning and weak IPC facilities.

the Microsoft version of CSV is a textbook example of how not to design a textual file format.

Criticisms of MacOS are of version 9 and prior which don’t really apply to OS X e.g. single shared address space. There are explanations of why things are such in the world of Unix and lots of great advice.

The ’rc’ suffix goes back to Unix’s grandparent, CTSS. It had a command-script feature called “runcom”. Early Unixes used ’rc’ for the name of the operating system’s boot script, as a tribute to CTSS runcom.

most Unix programs first check VISUAL, and only if that’s not set will they consult EDITOR. That’s a relic from the days when people had different preferences for line-oriented editors and visual editors

When you feel the urge to design a complex binary file format, or a complex binary application protocol, it is generally wise to lie down until the feeling passes.

One of the main lessons of Zen is that we ordinarily see the world through a haze of preconceptions and fixed ideas that proceed from our desires.

Doug McIlroy provides some great commentary too

As, in a different way, was old-school Unix. Bell Labs had enough resources so that Ken was not confined by demands to have a product yesterday. Recall Pascal’s apology for writing a long letter because he didn’t have enough time to write a short one. —Doug McIlroy

I’d recommend the book for anyone involved with computers and not necessarily involved with Unix or open source variants/likes. The author does a great job of explaining the theory of an approach to developing software and the operating it typically runs on, It’s accessible, easy to read and doesn’t require a computer to work through. You may need one however if you want to read it online for free.

My ideal for the future is to develop a filesystem remote interface (a la Plan 9) and then have it implemented across the Internet as the standard rather than HTML. That would be ultimate cool. —Ken Thompson

28/06/2014

Zvol backed bhyve guest

Filed under: FreeBSD — Tags: , , , — Venture37 @ 5:11 pm

Things have moved forward in the world of bhyve since I last looked at it over a year ago, support for zvol backed guests where fixed long ago among other things such as the birth of vmrc by Michael Dexter.
To run a guest with a ZFS zvol as its disk is no different to running with a disk image, only thing is that my version of /usr/share/examples/bhyve/vmrun.sh (11.0-CURRENT r267869 at the time of writing) fails to start from the disk once the OS has been installed.

A typical deployment scenario would be

Create a zvol of some size, e.g. 10GB

zfs create -V 10g zroot/guest0

Start a guest which’ll boot from the FreeBSD install CD iso & install onto the zvol

# sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d /dev/zvol/zroot/guest0 -i -I FreeBSD-10.0-RELEASE-amd64-disc1.iso guest0

Use the “ZFS – Automatic Root-on-ZFS” option from the Partitioning menu

As instructed in the bhyve section of the handbook, before rebooting, drop to the shell & edit /etc/ttys & replace the console line with

console "/usr/libexec/getty std.9600" xterm on secure

Shutdown the operating system
halt -p

Kill the guest
/usr/sbin/bhyvectl --destroy --vm=guest0

Create a new guest
bhyveload -m 4G -d /dev/zvol/zroot/guest0 guest0

Boot the new guest from the zvol
bhyve -c 1 -m 4G -A -H -P -s0:0,hostbridge -s 1:0,virtio-net,tap0 -s 2:0,ahci-hd,/dev/zvol/zroot/guest0 -s 31,lpc -l com1,stdio guest0

These instruction skip the creation of networking which is covered in the FreeBSD handbook as linked to above.

Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014
root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3399.54-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x306a9 Family = 0x6 Model = 0x3a Stepping = 9
Features=0x8f83ab7f
Features2=0xfe9a6257
AMD Features=0x20100800
AMD Features2=0x1
Standard Extended Features=0x200
TSC: P-state invariant
real memory = 5368709120 (5120 MB)
avail memory = 4103143424 (3913 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table:
ioapic0 irqs 0-23 on motherboard
random: initialized
module_register_init: MOD_LOAD (vesa, 0xffffffff80cfa5e0, 0) error 19
kbd1 at kbdmux0
acpi0: on motherboard
acpi0: Power Button (fixed)
atrtc0: port 0x70-0x71 irq 8 on acpi0
Event timer "HPET" frequency 10000000 Hz quality 550
Event timer "HPET1" frequency 10000000 Hz quality 450
Event timer "HPET2" frequency 10000000 Hz quality 450
Event timer "HPET3" frequency 10000000 Hz quality 450
Event timer "HPET4" frequency 10000000 Hz quality 450
Event timer "HPET5" frequency 10000000 Hz quality 450
Event timer "HPET6" frequency 10000000 Hz quality 450
Event timer "HPET7" frequency 10000000 Hz quality 450
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: port 0x408-0x40b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
virtio_pci0: port 0x2000-0x201f mem 0xc0000000-0xc0001fff irq 16 at device 1.0 on pci0
vtnet0: on virtio_pci0
virtio_pci0: host features: 0x1018020
virtio_pci0: negotiated features: 0x18020
vtnet0: Ethernet address: 00:a0:98:7f:5a:41
virtio_pci1: port 0x2040-0x207f mem 0xc0002000-0xc0003fff irq 17 at device 2.0 on pci0
vtblk0: on virtio_pci1
virtio_pci1: host features: 0x10000044
virtio_pci1: negotiated features: 0x10000044
vtblk0: 40960MB (83886080 512 byte sectors)
isab0: at device 31.0 on pci0
isa0: on isab0
uart0: port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (9600,n,8,1)
uart1: port 0x2f8-0x2ff irq 3 on acpi0
sc0: at flags 0x100 on isa0
sc0: MDA
vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff on isa0
atkbdc0: at port 0x60,0x64 on isa0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 10.000 msec
random: unblocking device.
Netvsc initializing... Timecounter "TSC-low" frequency 1699769676 Hz quality 1000
Trying to mount root from zfs:zroot/ROOT/default []...

23/06/2014

Verifying myself: I am 0x4ECB7B42 on pgp.mit.edu

Filed under: General — Tags: , , , — Venture37 @ 10:47 am

I’ve posted a PGP key for use under this domain to the key servers.

Email address: the word ending with the numbers three and seven at this domain (see field after Filed under & Tags on this blog post)
Key ID: 4ECB7B42
Fingerprint: 7BD6 2BFC 00E2 FAA1 5322 B9E4 D13F F837 4ECB 7B42

15/06/2014

Bricking & Unbricking a Dell Inspiron Mini 9

Filed under: General — Tags: , , — Venture37 @ 9:04 pm

I dusted off an old Dell Mini 9 netbook I had lying around, I’d stopped using it as the netbook refused to charge its battery once it’d run down completely (easily do-able if the machine is used lightly & could go up to a month without use), after the second battery It happend to, I gave up on it.
A side effect of the battery being run down was that it was not possible to flash the BIOS through windows, Phoenix WinPhlash refuses to write to the flash if there is no battery detected & exits with error code: -144.
This is not an issue on DOS using the phlash16 utility. Here lies a different kind of madness, as it’s attempting to write to the flash it stops using the mains & switches to battery power source, which if you have an unchargable battery results in bricking the netbook.
At this point, you can perform a flash recovery using a boot floppy containing a special boot sector, the phlash16 utility, some library and a copy of the image to flash named as BIOS.WPH generated with the BIOS recovery tool.
To initiate the recovery mode, the power and battery need to be removed, a USB floppy drive connected (of the left hand side ports only the USB port closest to the SD card reader worked on mine). With the Fn & B keys held down on the keyboard, connect the power whilst continuing to hold the keys down. At this point the power light should switch on. Press the power button & when you hear a beep, let go of the keyboard keys.
After a moment the system will begin reading the floppy & once reflashing commences, the system will begin beeping, once it has finished the system will reboot & startup normally.
I was unable to find working links for prepared images which I could write using dd so instead had to resort to finding another machine running Windows XP & a USB floppy drive but I’ve imaged the floppies I created so hopefully It wont be a repeated exercise.

The images can be written to a USB flash drive & used to recover a Mini 9

A04 Bios recovery image
A07 Bios recovery image

If you have a faulty or uncharged battery & you intend to use the phlash16 utility, remove the battery before attempting to flash the BIOS.

10/06/2014

Giving up on creating a port of OpenNMS

Filed under: FreeBSD — Tags: , , — Venture37 @ 1:18 am

After 5 years of going back & forth, I’ve decided to give up on trying to complete the OpenNMS ports for FreeBSD and dropped maintainership of the Java dependencies in the ports tree (net/jicmp, net/jicmp6, databases/jrrd, databases/iplike)

There are some issues which are show stoppers that need addressing upstream

  1. Separation of configuration & user data from the location of application binaries, Initially I began patching the source to look for files in a different location to the default so that things would integrate with the user land correctly but it soon became apparent that the patching would be a nightmare to maintain on an ongoing basis due to the number of patches required per configuration file. It was clear that things would need to be dealt with at the source rather than patched post release, a long running discussion with developers, bug reports raised, some (minor) patches submitted, 4+ years on, still ignored due to a lack of interest.
  2. Dynamically generated filenames, inherited from Google Web Toolkit, every build attempt generates new filename which make packing impossible.
    Update OpenNMS developer Benjamin Reed points to a possible fix
  3. Unreliable build process, maven fails between 2 to 3 times minimum which would cause lots of false alarms in an automated build environment e.g. the freebsd build cluster.
    This is somewhat of an improvement from a few years back where it would not be possible to build because repositories were not available for a day or two.

As it stands, the port is a shell which make it easy to install OpenNMS on FreeBSD but has major issues when it comes to upgrades or uninstallation. It’s best install dependencies from ports & install OpenNMS manually.

08/06/2014

System fails to boot with root on ZFS

Filed under: FreeBSD — Tags: , , — Venture37 @ 4:24 pm

I’d installed FreeBSD on my ThinkPad X61s when the head branch of the source tree was at 9-CURRENT, multi-booting it with Windows 7 & OpenBSD.
At the time I was not aware that it was possible to boot FreeBSD from a root file system on a ZFS volume from a disk partitioned using a MBR scheme. Instead, I opted to store /boot on a UFS filesystem.
This install existed for a couple of years, the ThinkPad got a roasting every once in a while to build a new release to install for updates. At some point support for 4K sectors in ZFS was improved, zpool status began to report degraded performance as the disk had been using 512byte sectors where in fact it could support 4K sized sectors.

Eventually, I deleted the existing slices in the FreeBSD partition & attempted to reinstall but found this time the system would not boot.
Booting from the install CD & issuing zpool import reported the new pool & old pool from the previous install.
Destroying a pool before deleting slices stopped this problem from re-occurring but the system still wouldn’t boot from a ZFS volume on a MBR partition.
The next step was to see if things would work if the whole disk was dedicated to FreeBSD, with a GPT partition scheme, things worked but switching to MBR, again, it failed to boot, hanging at a flashing cursor.
Over the next four months, many installs were attempted. On a MBR partitioned disk
FreeBSD failed to boot but PCBSD could by using GRUB.

I stopped trying any further at this point & took a break from it, one thing that had been raised at BSDCan was the possibility it could be lingering metadata, I’d thought as zpool(8) wasn’t reporting any existing pools when asked to import that this wasn’t the case. To give the benefit of a doubt, I ran dd on the disk with no difference in result.
This approach to clearing old pools seemed a little rough so over the weekend I looked into what other options are available.

The zpool(8) man page documents the labelclear option as

zpool labelclear [-f] device

Removes ZFS label information from the specified device. The device
must not be part of an active pool configuration.

-v Treat exported or foreign devices as inactive.

I still had the FreeBSD snapshot from the last attempt which I booted the X61s with, headed to the shell, deleted the existing partitions & issued
zpool labelclear -f /dev/ada0

Everything worked as intended after that.

Thanks to Allan Jude & everyone who chipped in at BSDCan.
Through the trial of getting this working Allan added the option to use a BSD partition scheme to the FreeBSD installer as well as MBR & GPT, which was previously unavailable.

03/06/2014

Inconsolata-g font

Filed under: General — Tags: , , — Venture37 @ 12:31 am

Looking through The Setup website, I found a reference to the Inconsolata-g font in the interview with Gary Bernhardt. I’ve been using Inconsolata as my terminal font for a while now and thought it’d make a nice change. The g variant is bigger than the stock font, the increase in size is from the dz variant according to the change list on the website. There is a 2pt difference in size between the two version of fonts, Inconsolata-g at 12pt is equivalent to Inconsolata at 14pt. On my MacBook Air, font smoothing doesn’t look right with Inconsolata-g lower than 14pt using the OpenType version.

 

Inconsolata
Inconsolata-g

29/05/2014

Switching from MySQL to MariaDB

Filed under: General — Tags: — Venture37 @ 11:44 am

This blog started life on MySQL 4.x & continued to live on 5.0 until today. Whilst performing maintenance, all the packages came up to date apart from two, the MySQL 5.0 client & server which had been long removed.
I was about to commence with installing version 5.5 when I remembered a conversation I had a couple of weeks back about MariaDB, after a quick check to see what the switchover entailed, I decided to install to MariaDB instead.
It’s intended to act as a drop in replacement for MySQL, my instance has been for serving blogs & other fairly common 3rd party open source software so I didn’t have to do much apart from run mysql_upgrade after install.
In /var/db/mysql/server.example.com.err MariaDB logged
Column count of mysql.db is wrong. Expected 22, found 20. Created with MySQL 50084, now running 50312. Please use mysql_upgrade to fix this error. to highlight the fact.

mysql_upgrade output:
Phase 1/3: Fixing table and database names
Phase 2/3: Checking and upgrading tables
Processing databases
information_schema
mydb
mydb.wp_commentmeta Needs upgrade
mydb.wp_comments OK
mydb.wp_links OK
mydb.wp_options OK
mydb.wp_postmeta OK
mydb.wp_posts OK
mydb.wp_term_relationships OK
mydb.wp_term_taxonomy OK
mydb.wp_terms OK
mydb.wp_usermeta OK
mydb.wp_users OK
mysql
mysql.columns_priv OK
mysql.db OK
mysql.func OK
mysql.help_category Needs upgrade
mysql.help_keyword Needs upgrade
mysql.help_relation OK
mysql.help_topic Needs upgrade
mysql.host OK
mysql.proc Needs upgrade
mysql.procs_priv OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name Needs upgrade
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_topic OK
mysql.proc OK
mysql.time_zone_name OK
Phase 3/3: Running 'mysql_fix_privilege_tables'...
OK

I decided to recreate my.cnf using the files shipped with MariaDB due to the introduction of new settings and a difference in values for existing settings.

27/05/2014

12″ PowerBook G4 PT 4

Filed under: OS X — Tags: , , , , — Venture37 @ 3:17 am

Due to various factors, I’ve not had much of a chance to play with the PowerBook much this month, earlier this moth a follow up to PR/48740 happened, requesting feedback on new changes which had been committed that I’ve not had a chance to test yet.
One thing I did do tonight was to re-flash the SuperDrive with a RPC-1 firmware image which turns the DVD drive region-free.
The firmware images are hosted on MacBook.fr and cover Macs all the way back to G3′s.
Flashing was straightforward though I could only re-flash with the version currently on the drive. It was not possible to flash a newer stock or region-free image on the drive.
Aside from the firmware on the DVD drive, Mac OS also tries to enforce region locking, the Region X utility can reset the Mac OS related setting regarding content region.

24/05/2014

FPGA on a Sun Fire T1000

Filed under: SPARC / Solaris / OpenSolaris — Tags: , , — Venture37 @ 6:49 pm

There’s a Xilinix SPARTAN FPGA in the top left hand side of the motherboard, I’m guessing this is for the ILOM?

20140524-184816-67696696.jpg

Older Posts »

Powered by WordPress