The Perils of Porting

Recently, the pointyhat package building cluster got a new machine to help out with i386 builds. The new machine, installed by Kris Kennaway as gohan18.freebsd.org, sports two sockets populated by quad-core Intel Xeon 5320 chips clocked at 1.86 GHz. Machine have 4 GB of RAM, but it’s promised to get upgraded to 8 GB soon. That’s eight cores in a single chassis, yes. Pretty shiny! The machine was donated by Yahoo!

Existing i386 cluster on pointyhat consists of 18 blades, each fitted with Pentium III at 700 MHz processor and 1 GB of memory. If we assume that the Woodcrest/Clovertown generation of Intel chips is two and a half times as effective as Pentium III on the same clock, this single new machine pretty much doubles the throughput of the i386 cluster.

I have collected build times of 15,935 packages built on the old gohan blades, and put them into the graph. Click for the full size.

I have marked build times of some of the popular ports in the graph. The only conclusion which can be drawn from this graph is that we got way too much p5 module ports. The longest port to successfully build (that rules out OpenOffice) is koffice, which takes 30 hours to complete.

I have compared build times of some random ports on the old gohans and gohan18. On small ports the speedup is up to 8x, perhaps because of much faster hard drives. The typical p5 module went from 12 minutes to a minute and 30 seconds. Larger ports exhibited a more modest speedup in the range of 2x – 3x.

The future of package building is bright!

Today an experimental run on pointyhat finished, and the following user visible changes were committed:

• Default LIBTOOLFILES values was changed from literal configure to value of CONFIGURE_SCRIPT.


• bsd.efl.mk was moved to Mk directory. From now on, all ports are free to define USE_EFL without explicitly including bsd.elf.mk. In case you wonder what EFL is, it stands for Enlightenment Foundation Libraries. Stanislav Sedov promised to write a nice section on it for The Porter’s Handbook.


• Bored of figuring out how to construct a tar line to install the hierarchy of files? Here’s the solution – COPYTREE_BIN and COPYTREE_SHARE macros. Details.


• WWWDIR is a new variable predefined for all ports. It’s value is PREFIX/www/PORTNAME. I have swept the tree of the ports that used to define this on their own.


• And finally, a batch of related predefined variables – DATADIR_REL, DOCSDIR_REL, EXAMPLESDIR_REL and WWWDIR_REL was added. They carry the same value as their REL-less cousins, but relative to the installation prefix.

Three patches haven’t worked out on the exprun as we hoped, and will be refined and included in the next exprun. They were – Mikhail Teterin’s optimizations, which had some problems with distfiles in nested subdirectories; perky’s patch to switch default version of Python to 2.5, perky promised me an updated patch; and finally a switch of default GhostScript implementation from GNU to AFPL, where we hit some ports defining direct dependency on the GhostScript port, instead of using USE_GHOSTSCRIPT.