GeekLAN

When I wrote up the previous post, there was a couple of points which I forgot to cover and since then a few more things have come up on the same subject that I want to go over here as a follow up.

I’ve been carrying on with the package building since the previous article and keeping on the same theme of old computer, old OS, old compiler, what can I build? (PowerPC mac, running OS X 10.4, with GCC 4.0.1). I resort to GCC 5 if I need C11 support & can go up to GCC 7 if needed. Have not yet attempted to package a newer version of GCC yet but after a certain point beyond GCC 7, the currently supported versions of GCC (11 and above?) require interim toolchain builds to go from a system with only GCC 4.x to 10+. Taking around 24 hours to build GCC 5 alone on the hardware I have at hand, I’m putting off attempting toolchain related changes as much as possible.

One thing I seem to spend a lot of time doing is trying to dig into finding requirements for software since the current trend is at most to list the names of components, without any version info. There are a couple of scenarios where the requirements are skewed indirectly.
The software I want to build has modest requirements, but they require a critical 3rd party component which depends on newer tooling, thus setting a higher requirement and sometimes it’s just the build infrastructure which has a far higher system requirement.
Of course if I was working on a modern systems with current hardware it wouldn’t be a problem since either binaries would be readily available or building would not take much time, but perhaps if you’re going to strive for minimalism, avoid requiring build tools which need the latest and greatest language support and beyond.
No matter how much of a time the shiny new build infra will save, it would take many days for me to get up to having the tooling in place in order to attempt building your project.

I find myself in the situatio where I now appreciate autoconf in these scenarios and think that projects should carry the infra for both autotools & more fancy tooling.
Autoconf may be old, odd, and slow, but it has been dragged through many scenarios and systems and has built up knowledge of edge cases, and in the world of old machines and uniprocessors, something that reduces down to a shell script and make files is just fine for such systems. A PowerBook would appreciate it, an iBook definitely would.
Hence, I’ve been holding back versions in some cases where the projects have moved on to different infrastructure, pinning things to the last version to support autotools, for now. Through doing this, I discovered that sometimes there were no mirrors kept for the source files, the source archives are published on their repo but it’s an archive generated from the source tag, not the bootstrapped release archive which would’ve been published originally, resulting in a detour to bootstrapping before attempting to build and additional build dependencies to pull in autotools. I guess I could patch that infrastructure in, having bootstrapped separately, but that would be a hefty diff.

Whilst making progress with packages, I made a discovery as to the source of my troubles with pthread support detection, that I wrote about in the previous post, the common failure is to just check for the presence of the header file and not checking for the implementation details. It turns out, this is largely due to gnulib, the GNU portability library that is vastly used in the GNU ecosystem. The issue there is the detection of pthread support is split across several modules which cover different aspects such as “is there a pthread.h?”, “is there support for rwlock?” and in this fragmented way, it is possible to misdetect functionality, since projects are selective in what they import from the library. One issue, now fixed, was the miss detection of support for rwlock, which OS X Tiger & prior lack. If projects import all of the pthread components from gnulib, detection of pthread support works correctly and gnulib build process makes the correct substitutions and then its tests suite passes ok. If only some parts have been imported things fail when running the test suite since for example the test for pthread support tries to make use of rwlock support in the testcase. There are many other issues with gnulib and its test suite broadly, completely separate from pthread support that for now I’ve skipped on running the gnulib tests. I am unaware of the scope of breakage in the library since attempting to generate a library comprised of all components in gnulib take some time and disk space to generate (documentation states 512MB disk space but I crossed a couple of gigabytes before aborting). I have been working my way through the GNU software projects to see what’s broken in the current versions related to gnulib. I’m sad to say GNU Hello is a casualty and currently unbuildable on OS X Tiger. There is a lot more work related to gnulib that I haven’t even started on.

Technical aspects of building software aside, one thing that is re-occurring is peoples response to legacy systems. On the plus side folks work on restoring functionality on old systems and others question the age of the system and whether it is time to rip out support for them. I can’t help to wonder if this is conditioning from the Apple side, since there is often support for other long dead operating systems from extinct companies/products that remain in a projects codebase. I was once asked if I try new software and technology as soon as it becomes available for a vacancy with them. As a user of an iPod Classic, PowerBook G4 running OS X Tiger, a 2007 17″ MacBook Pro, and a resume mentioning packaging for legacy OS X, I answered yes & awaited to hear back from them.

Something I’ve been thinking over on and off for a while, is the meaning of portability claimed by folks regarding software.
Fish out an old computer with a matching vintage closed source operating system, how does the portable software fair in such a situation?
Supporting an ancient operating systems when building recent software is enlightening, there’s so much room for dependencies to creep in, only to realise it when the dependencies are not there. Ideally, at the least, documentation would cover when said functionality was introduced so you have a vague idea in advance if something will work, but an exact idea of dependencies is needed to catalogue what is required.

I’ve been wading through building things on OS X Tiger again and looking for low hanging fruit to tick off. Armed with the stock version of GCC 4.0.1 from XCode 2.5, I’ve been avoiding going down the newer toolchain route and attempting to build anything which lists needing nothing more than C99/C++98 as I want to get as much done with the stock toolchain as possible since it reduces time starting with a fresh setup.

GCC 4.0.1 supports C99 and C++98, though according to the GCC wiki, C99 support was not completed in GCC until v4.5. GCC 4.0.1 defaults to C89, unless a standard via -std is specified. I can define __DARWIN_UNIX03 if I want to kick it up a notch for modernity, but that’s at least one version of SUS behind, implementing changes to existing functionality from IEEE Std 1003.1-2001 (defaults being older behaviour in some cases to accommodate migration to the then new behaviour, which has now become the expected behaviour).
Still, I have a compiler which recognises -std=gnu99 and a system which was on the way to UNIX03 certification but not (first reached in Leopard). See compat(5) from Tiger, Leopard, Catalina.
On my quest for low hanging fruit for things, I keep on getting caught out by software which claim to require just language support but really needing functionality beyond language support and 3rd party libraries from the the system.
Some examples of things which have tripped my up like

• Stating -std=c89 and requiring a C11 supported compiler.
• Claiming just C99 support but requiring POSIX functionality from IEEE Std 1003.1-2008.
• With a C11 compiler installed (GCC 5), missing POSIX functionality from IEEE Std 1003.1-2008.
• The assumption that PowerPC hardware means host is running Linux.
• Python modules with Rust dependency (luckily that is a quick, hard failure on PowerPC OS X 🙂 )

More than just language support aside, another common issue is the misdetection of functionality through the test being too broad. Usually see this with pthread support, where more recent pthread support is required but all that happens from the build side is a check for the presence of pthread.h header file rather than checking for required functions like pthread_setname_np(3) / pthread_getname_np(3). Luckily, from the lacking user-space aspect there is help at hand, either via libraries which provide an implementation of missing functionality, or re-implement the functionality as a wrapper around existing functionality.
Sometimes there’s advice on how to use existing functionality for things which haven’t already been provided by compatibility libraries from the Windows community since they still have portability related issues on current OS versions.
Just stuck with things which need hardware/kernel support – memory atomic operations?
Going lower level, since the tooling is now so ancient, functionality which allows things to be treated generically across platforms hadn’t cross-pollinated. Non-platform specific mnemonics for assembler are lacking, and flags for linker to generate shared objects are missing.
Luckily the linker fixes is easy to patch and forward compatible with newer OS versions since they have retained backwards compatibility for the old way, specifying -dynamiclib instead of -shared which is what is now commonly used.

The Practice of Programming, written in the late 1990s, before C99 was standardised, has a chapter on portability.

It’s hard to write software that runs correctly and efficiently. So once a program works in one environment, you don’t want to repeat much of the effort if you move it to a different compiler or processor or operating system. Ideally, it should need no changes whatsoever. This ideal is called portability. In practice, “portability” more often stands for the weaker concept that it will be easier to modify the program as it moves than to rewrite it from scratch. The less revision it needs, the more portable it is.

Of course the degree of portability must be tempered by reality. There is no such thing as an absolutely portable program, only a program that hasn’t yet been tried in enough environments. But we can keep portability as our goal by aiming towards software that runs without change almost everywhere. Even if the goal is not met completely, time spent on portability as the program is created will pay off when the software must be updated. Our message is this: try to write software that works within the intersection of the various standards, interfaces and environments it must accommodate. Don’t fix every portability problem by adding special code; instead, adapt the software to work within the new constraints. Use abstraction and encapsulation to restrict and control unavoidable non-portable code. By staying within the intersection of constraints and by localizing system dependencies, your code will become cleaner and more
general as it is ported.

The Practice of Programming, Introduction to Chapter 8, Portability

When building Monocypher, I needed to drop -march=native from CFLAGS, s/-shared/-dynamiclib, and remove -Wl,-soname,$(SONAME) from the makefile in order for it to build on OS X Tiger. As stated on their homepage, there were no external dependencies.

Portable code is an ideal that is well worth striving for, since so much time is wasted making changes to move a program from one system to another or to keep it running as it evolves and the systems it runs on changes.
Portability doesn’t come for free, however. It requires care in implementation and knowledge of portability issues in all the potential target systems. We have dubbed the two approaches to portability union and intersection. The union approach amounts to writing versions that work on each target, merging the code as much as possible with mechanisms like conditional compilation. The drawbacks are many: it takes more code and often more complicated code, it’s hard to keep up to date, and it’s hard to test.
The intersection approach is to write as much of the code as possible in a form that will work without change on each system. Inescapable system dependencies are encapsulated in single source files that act as an interface between the program and the underlying system. The intersection approach has drawbacks too, including potential loss of efficiency and even of features, but in the long run, the benefits out­ weigh the costs

The Practice of Programming, Summary of Chapter 8, Portability

I guess the union approach is what you would call the autoconf workflow: test for the required functionality and generate definitions, based on the test results. Those definitions can then be checked for in the codebase to steer the build process.

It seems today portable software in most cases means with the listed requirements, you have a chance to build given an up to date, current, operating system, rather than building on the earliest version of where requirements are available.
For portability, in terms of future proofing, being explicit in expectations helps prevent breakage caused by riding defaults when in time defaults change.
As proof of portability, test, test, test, beyond current version of mainstream systems.
If not for extending support, to bring clarity to expectations of where the software will be able to run.