Author Archives: Bernhard Fröhlich

Personal story of a ports committer

You all know that sometimes you end up dealing with all the ugly stuff instead of doing useful work. Over the last few months I was kept busy at $dayjob got assimilated by portmgr and had to look after redports. All of those new challenges are nice on it's own and I really enjoyed being part of the FreeBSD community and ecosystem but then 11/11 happened.

At that day quite a lot has changed for me since redports was isolated as a precaution and all ports building clusters of portmgr were effectively shut down. That situation was quite a mess since all automated systems and clusters were gone. No INDEX builds, no QAT, no pointyhat so also no exp-runs anymore. Whenever someone broke the ports tree we didn't even knew. It proved to be quite hard to get back on track again after that incident. INDEX checks and a very very limited QAT are already running again but pointyhat and redports are still dead. :(

The daily frustration and dealing with all that strange decisions that are taken because of the need to get stuff done is hard sometimes. But It's almost Christmas and without redports I have much more spare time so I try to calm down and focus on stuff that I can hack on my own. And that worked out quite nice so far ...

I've noticed in the XBMC 12.0 release notes that they have included the PVR branch and thus support DVB-S2/C/T in XBMC. Well actually they only provide some backend configuration interfaces and rely on a backend like mythtv or tvheadend to handle the DVB stuff. Mythtv would be okay for that task but It's huge for such a small job. Tvheadend is a nice and small TV streaming server that suites perfectly and only does the bare minimum without a lot of dependencies. Configuration is done in a web based GUI or can be done in XBMC. So I started working on a tvheadend port. A few weeks later I'm at the point now where tvheadend compiles fine and also starts. I've just ripped out all that epoll stuff and linuxisms that I stumbled accross so it doesn't run properly yet. Adding kqueue support is the next step now.

Due to redports being unavailable the vbox work has also frozen. I tried to collect all that patches and complains in my inbox so that they don't get lost. Since the situation did not improve I temporary created a github repository for the virtualbox ports and committed all the accumulated patches there:

This includes almost all patches that were flying around on mailinglists and updates vbox to 4.2.6 / 4.1.24 but testing was very limited so take care if you give them a try. Testing will show us if we can commit it to the portstree by New Year's Eve.

Ports QAT functionality integrated into redports

We used to have a FreeBSD Ports QAT machine that did automatically build all affected ports after a commit. Well that machine is down since quite some time now because of an hardware defect I think. In my plans for I started quite early to think about integrating the QAT service so I talked to itetcu at BSDDay in 2011 about the current implementation of the QAT system. It works by parsing the ports CVS mails to find out which ports are affected by the commit. Then it updates the CVS tree from one of the tier1 CVS mirrors and hopes to have a consistent portstree. After that it schedules new jobs in the Ports Tinderbox and sends out mails to the committer if building failed. That worked fine most of the time but it had quite some weak spots which required to constantly look after the machine to keep it going.

The most important thing that I learned from that was that we need to migrate our ports repository from CVS to something that allows a consistent checkout. Now that beat is working on the cvs to svn migration and has a testing repository I used that to implement QAT functionality into the redports infrastructure. Instead of parsing CVS mails I can use svn info to find new commits and consistent repository checkout is also guaranteed by subversion. After all it took me about one working day to fully integrate the QAT functionality and test the new stuff.

There are a few benefits for the upcoming QAT system now that it is a part of the regular redports infrastructure:
  • access to all redports building machines (more power!)
  • parallel builds on multiple boxes
  • archived buildlogs
  • run QAT jobs for multiple FreeBSD versions/architectures
  • nice web frontend with RSS feeds and the usual modern stuff
  • you still get mails of course