Category Archives: Weblog

Playing around with PFSense

In the last period I became rather familiar with the PFSense project. I decided to migrate some of my firewalling devices to PFSense, first starting at 1.2.3-RELEASE, and finally I upgraded them to 2.0-BETA1. Doing the latter thing is possible since the locations only use the internet from the LAN, and have some minor settings applied locally. Playing around makes it much easier because of that.

Currently I am checking the GRE and GIF interfaces, I am using them to create an OSPF network, and there are some oddities in them :-)

So perhaps I can see why the oddities are there and if needed correct them (or myself when I am misbehaving :) )

You should test PFSense, it runs FreeBSD 8, and is awesome !

Welcome to FreeBSD 8!

Welcome to FreeBSD 8!

In this article I will write about the latest release from FreeBSD, 8.0. This is a major version that offers new functionality and much improved parts of the code.

Why are major releases so special?

Major releases offer the possibility to include changes in the system that are not allowed in stable branches. This could be new applications or API/ABI interfaces, or serious changes to datastructures and things like that.

FreeBSD brings out major releases every 18 months. Ofcourse the 18 months may vary, depending on the amount of new features and problems found while preparing for the first .0 release. The schedule should guide as a reference on when to expect something new.

FreeBSD 8?

FreeBSD 8 will be the latest version of FreeBSD that is currently on the market, it was preceded by FreeBSD 6 and 7, where we found features like ZFS, Improved Jail support, the widely known Danish Axe was used to further take out the GIANT lock, and many more features that had been in the previous release.

What will FreeBSD 8 offer us?

FreeBSD 8 will offer us the following:

USB Storage hotplug removal

Previously it was not possible to take out lets say an USB storage device while the operating system was running, without crashing it. The system code is updated to make it possible to take out an USB storage device without the system crashing. This will make many people happy!

Multi-IP/IPv6 jails

With FreeBSD 8 it will be possible to assign multiple IPv4 addresses to a jail, as well as assigning IPv6 addresses. People that offer hosting services can greatly enjoy these features, a jailed webserver for example can now use many more addresses and do proper SSL virtualhosting.

NFSv4 support

NFSv4 support is added to the FreeBSD Operating System, while this is still experimental, it gives you a NFSv4 client, as well as a server that can service NFSv4 nodes. NFSv4 gives you better security access control’s, a stateful protocol, performance improvements, etc.

Enhanced ULE scheduler

The ULE scheduler is much improved for FreeBSD 8, this allows even better scheduling and responses to systems running FreeBSD 8, especially when used with SMP systems. Systems running the FreeBSD 8 operating system, have this scheduler enabled by default.


DTrace is a suite of applications imported from Solaris. DTrace will assist you in profiling your system and applications, data which you can use to make the system perform even better under your specific configuration. Every key developer needs a tool like this.

Wireless Mesh Networking

The Wireless Mesh Networking (802.11s) is an experimental feature and early adoption framework that gives the ability to setup a Meshed Wireless network. Currently most Wireless networks are build around one central access points, with leaf nodes / bridges / repeaters attached to them. This experimental feature offers the ability to ‘perfectly’ roam accross the network, without being tied to one specific Access Point.


VIMAGE, is an network virtualisation project that aims in delivering multiple networking related instances. So for example you can give your jails, their own vimage, enabling them to run IPSEC, Packet filter(s), setting their own routing tables etc. So this actually makes your jails, more or less independant servers.

Multiple routing tables

Multiple routing tables, enable you to select different routing tables for different services. For example you can setup different routes for your webserver(s) and mailserver(s). The “setfib” utility is added to provide a management layer for these FIB’s (Routing tables).

Equal Cost multipath routing

Equal-Cost Multi-Path Routing (ECMP) is a new feature under FreeBSD 8, that enables the administrator to loadbalance traffic by equally balancing traffic over various routes. In theory this could increase the bandwidth that you can use, so be sure to check this out!

FreeBSD ports Parallel building

The FreeBSD ports framework is adopted to that multiple ports can be build at once. People with multi core systems will definitly see significant improvements in this region. While this update is not limited to the 8.0 release itself, it is the first release that will have this feature onboard. (after fetching the ports tree ofcourse).


The TTY subsystem is one of the last sections of the entire operating system that had not been rewritten or massively updated in the last decade or two. FreeBSD 8 will have a replaced version that no longer uses the GIANT lock. That reduces the impact of the lock even more and will assist with fine grained multiprocessor capabilities. It is also more modular and will make it easier to hook into the new layer. This will help in lagging console and sessions.


Procstat is a new application, eliminating the need for procfs(4), it gives the user information about processes, command line arguments, kernel thread stacks, and many other functions that could assist people in troubleshooting and debugging.


Textdumps had been written to make it much easier to get coredump information back from the Operating System after fatal events occured.

The application had been setup to generate a tar archive that includes several text files with valuable troubleshooting information. Users that send this tar file to one of the appropriate lists, make a much better chance in getting their problem fixed. Before this tool one needed to do this manually, often making this an impossible step for the average user. Not only will this help the user that is facing problems, but also the bugbusters that will have all required information available without needing to explain on how to get the information there.

Experimental AHCI driver

An experimental AHCI driver had been added to the tree, which gives direct and native AHCI commands through the CAM layer. This includes the NCQ feature (Native Command Queueing).

Gvinum v2

Gvinum had been updated and reworked, making it much more production ready and usable then before.

Disk utility upgrades

Several disk utility applications had been updated, or adjusted to become the default. For example the GEOM_PART utility becomes the defacto standard for disk slicing. One should pay attention when this comes in to play though, since the labels might be read differently with GEOM_PART your devices might be enumerated in a different way then before. Make sure you are actively involved with the upgrade and make sure you have serial access to the machine in case of problems!

Beyond that GPT partitions are now capable of being boot from, and the bsdlabel utility supports up to 26 partitions now.

ProPolice Stack Protector

The ProPolice SSP framework is added to the builds, protecting the kernel and userland (where configured) from being stack-smashed. This will make it harder to exploit stack based buffer overflows.


ZFS is updated to version 13, and sees several improvements. The experimental warning is removed, making it a production ready filesystem which you can use with very large datapools. Make sure you have enough memory when playing with it though!

Wireless Virtual AP

The Wireless code is updated to support Virtual AP’s. Within the /etc/rc.conf file you are now able to clone the ‘direct hardware interface’ like ath0 into a wlan device. If you clone this more often you will be able to support more SSID’s and things like that. For small companies or freaky home users, you can setup your own protected SSID, and a guest SID which visitors can use and is protected differently.

VirtualBox Host support

The FreeBSD ports tree saw VirtualBox added to the tree, making FreeBSD 8 one of the possible candidates for Virtual Machines delivered through VirtualBox. This makes it a good alternative for VMWare.

NULL-pointer dereferences made harder

The FreeBSD Security Team included a little rewrite of the NULL pointer handling within FreeBSD. Programs are no longer able to make use of the NULL pointer by default and will be stacked on a random address location. This feature makes FreeBSD a bit safer against NULL pointer dereference vulnerabilities.

lukemftpd build disabled by default

Personally I axed the build of lukemftpd. This is one of the two ftp daemons that we have in the base system, and was opted for removal a long time ago. That had happened and the daemon is no longer available by default on the latest version of FreeBSD. Further removal of the lukemftpd software will be investigated to see what consequences will popup because of that.

How to upgrade?

Now that we spoiled you with all the new features in FreeBSD 8, there is probably interest in how one can upgrade to the FreeBSD 8 release.

Before explaining the process of upgrading the system, I expect that you already have a FreeBSD system running on 7.2 or 7-STABLE, if you are unsure on how to get that going, please refer to my previous article in BSDMag which explains this in great detail (and it includes pictures too!).

Ofcourse it is also be possible to install FreeBSD 8 from CD/DVD, please browse to
and fetch the proper media. All you have to do is follow the instructions that are printed on screen. If you need more details about this, there is an excellent handbook that will guide you through it, or again my previous article will assist you. The FreeBSD handbook can be found here:

In some cases there are localised versions, like the Dutch Handbook:

replace ‘/nl’ by ‘/yourlangcode’ to see whether your version also exists.

As mentioned we will be upgrading a running 7.2 / 7-STABLE system to FreeBSD 8.0-RELEASE.

We use “csup” for this, which will fetch our source code. This will lead up to a new system and new kernel.

For the instructions that follow, you will need “root” access, if you do not have this access, please consult your system administrator and ask him for help on how to upgrade FreeBSD as “root”.

Copy over the example csup file to /root and begin editing:

# cp /usr/share/examples/cvsup/stable-supfile /root/8-release
# vi /root/8-release

Navigate to the line which mentions ‘default host’, and change it to a server near you, so for example in the Netherlands you would use:


Now find the part that has ‘default release’ mentioned. We need to update that as well. Change the contents of the tag parameter to RELENG_8_0. Thus the line would look like something as:

*default release=cvs tag=RELENG_8_0

Note that there is another tag that will give you the actual release bits (RELENG_8_0_0_RELEASE), but this will not get updated for Errata Notice’s and Security Advisories, making it an unwanted tag to use.

After making sure the above updates were done, we will need to update our tree. If you are a survivor of my previous article you will have an old tree and old objects tree which you might want to save for future reference.

# mv /usr/src /usr/src.releng7
# mv /usr/obj /usr/obj.releng7

Now we can download / update the new tree.

# csup /root/8-release

Wait for it to finish, and after that we can build a new kernel and world. We will not do modifications to the kernel and world this will be very straightforward.

# cd /usr/src
# make buildworld && make buildkernel

This will generate both a new world, and a new kernel, which we will use later on.

When this had been done, we will update the configuration files by issueing:

# mergemaster -p

This will update the configuration files that are needed before the new release can be started at all. If no output is given then the current configuration is sufficient to start the 8.0 version later on.

Configuration had been updated, so we can start by installing the kernel, after which we need to reboot the system:

# make installkernel
# reboot

The system will restart, when the bootloader starts, press a key and/or start the system in single user mode. In case you do not have the menu; you should issue “boot -s” on the command line so that the system starts in single user mode.

Assuming that nothing went wrong and the system starts normally into the single user mode, we can continue by installing the new world.

# cd /usr/src
# make installworld

Next we need to run a new mergemaster, but this time without any additional flags, so that “optional” configuration files will get updated.

# mergemaster

Walk through all differences and make updates where you think they are appropriate. Be carefull though, if you update passwd and group, you might no longer be able to login remotely. I have been there, done it and got the t-shirt. Again I’d assume you made all the changes you wanted to, without messing up with the system itself. We need to reboot one more time before we can be a happy 8.0 user.

# reboot

And the system will reboot and restart all services under the new world order.

Updating the jails:

In my previous article we also installed jails, that will do our hosting needs, without compromising the host machine. They are not updated through the way described above. Though a fairly simple way of doing that is by installing the new world over the old jail-world. For this you need to stop the jails, because otherwise the system might not be able to update essential files because they are in use.

Once again navigate to the usr/src tree so that we can start by updating the jails. I’d assume you have stopped the jails already so we are safe to go.

# cd /usr/src

Next, update the jails master infrastructure:

# make installworld DESTDIR=/home/j/mroot

However, please do note that it might be possible that certain ports need to be rebuild. Carefully inventorize what kind of applications you currently use, and check whether they might be using libraries that are no longer provided. After you did this you are free to cleanup old libraries:

# cd /usr/src
# make delete-old-libs
# make delete-old

Again this is at your own risk. If you fail to update bash for example to use the new libraries, you are no longer able to restart it if you relogin. Again I have been there, done it and, got a nice second t-shirt!


So now you are running a fresh and modern 8 system, which you can use to do various kind of (new) things. Not only did we update your base system, we also updated your jail infrastructure and you also updated several ports so survive this cleanup. You also got an impression about what new features you can use under FreeBSD, so I’d suggest you start playing with them now. In case you find problematic things, please consider sending an email to the various mailing lists of FreeBSD, located at

In case of a more serious problem, which cannot be resolved by the various mailing lists, please consider sending in a bug report so that the developer crew can act accordingly and if there is enough information available resolve the problem you are facing.

Do realise though: Most people working on FreeBSD is a volunteer, who all work for bosses or are being hired for different projects. They also have families, kids, and a social life that demand attention. Please take care when sending in a problem report. Make the report as detailed as possible, send in the textdump if that is available, and give people time to respond. Most often a gentle prod works better then bashing around.


I used Ivan Voras’ list of “Whats cooking for FreeBSD 8″ as my main source for new and improved updates. You can find the list at the following URL:

About the Author:

Remko Lodder is a 26 year old FreeBSD enthusiast, in his spare time he likes being with his son and girlfriend, playing with FreeBSD systems, and wearing various FreeBSD hats to help the community. In his professional life Remko is an Unix Engineer for Snow B.V. in the Netherlands mostly focussing on Firewalls and Security (Checkpoint/Juniper etc.). You can contact him by sending an email to: [email protected], or if you want to hire him send an email to [email protected] or check out their website on:

Setting up Calendarserver on FreeBSD 7.2

The last few days I am trying to setup CalendarServer on FreeBSD 7.2. I failed so far :-)

Together with Tom Scholten I got rid of a lot of errors (oh well a lot, just getting forward); but I am stuck with the following error;

Building Zope Interface…

Building select26…

Building PyDirector…

Using python as Python

Starting server…
exec python /usr/local/src/Twisted/bin/twistd -n caldav -f /usr/local/src/CalendarServer/conf/caldavd-dev.plist -o ProcessType=Combined
/usr/local/src/CalendarServer/calendarserver/tap/ DeprecationWarning: mktap and related support modules are deprecated as of Twisted 8.0. Use Twisted Application Plugins with the ‘twistd’ command directly, as described in ‘Writing a Twisted Application Plugin for twistd’ chapter of the Developer Guide.
from twisted.scripts.mktap import getid
Traceback (most recent call last):
File “/usr/local/src/Twisted/bin/twistd”, line 19, in
File “/usr/local/src/Twisted/twisted/scripts/”, line 27, in run, ServerOptions)
File “/usr/local/src/Twisted/twisted/application/”, line 689, in run
File “/usr/local/src/Twisted/twisted/application/”, line 669, in parseOptions
usage.Options.parseOptions(self, options)
File “/usr/local/src/Twisted/twisted/python/”, line 231, in parseOptions
File “/usr/local/src/Twisted/twisted/python/”, line 241, in parseOptions
File “/usr/local/src/CalendarServer/calendarserver/tap/”, line 218, in postOptions
File “/usr/local/src/CalendarServer/calendarserver/tap/”, line 230, in loadConfiguration
File “/usr/local/src/CalendarServer/twistedcaldav/”, line 199, in load
configDict = ConfigDict(self._provider.loadConfig())
File “/usr/local/src/CalendarServer/twistedcaldav/”, line 401, in loadConfig
configDict = readPlist(self._configFileName)
File “/usr/local/src/CalendarServer/twext/python/”, line 80, in readPlist
rootObject = p.parse(pathOrFile)
File “/usr/local/src/CalendarServer/twext/python/”, line 403, in parse
File “/usr/local/src/CalendarServer/twext/python/”, line 415, in handleEndElement
File “/usr/local/src/CalendarServer/twext/python/”, line 467, in end_string
File “/usr/local/src/CalendarServer/twext/python/”, line 428, in addObject
File “/usr/local/src/CalendarServer/twext/python/”, line 298, in __getattr__
raise AttributeError, attr
AttributeError: append

If someone has a suggestion on how to fix this I would be very interested. For now I had a lot of time in this today, and I am not willing to spend more time on it.

FreeBSD: Lukemftpd disconnected from the build

I did it.

Lukemftpd is disconnected from the build, I will MFC this after three days, and start by looking into removing the code entirely. For the build disconnect see the commit log here:

or below:

Do the first step in removing lukemftpd from the base system. Disconnect
it from the build.

If you are using the FTP daemon, please consider using the port ftp/tnftpd
which is the same FTP server, but newer and might have more/better

This results in us providing only one ftp daemon by default.

Reviewed by: bz
Approved by: imp (mentor, implicit)
MFC after: 3 days
Silence from: obrien

FreeBSD: Removal of lukemftpd imminent

A long long long discussion had taken place around the two ftp daemons that we serve in our FreeBSD operating system. We had discussed that we would want to only serve on, and the security track record isn’t great either.

For this reasons I will be removing lukemftpd real soon now! If you use it from base, please consider using ftp/tnftpd instead, this port is more recent then the code in our base system, and can be updated when needed. It’s too late to rediscuss this.

I am waiting for a “make universe“ before I will commit this to head, and I will commit it to the 8-stable branch 3 days after the commit.

edit: 20090903: The category is ftp instead of net. Suggested by Thomas Abthorpe.
20090903: The universe build is almost complete, the most important architectures build fine.

FreeBSD – Current freeze over

Last night the FreeBSD team had unfrozen the development branch for -HEAD (What will eventually lead up to 9.0-RELEASE). Although people need to be conservative about commits they make, they can continue development again. RELENG_8 is still frozen but people are allowed to merge after consulting the Release Engineering Team. Good Progress is being made now to get 8.0 out of the door.

Releng_8 ” opened for commits ”

Today RELENG_8 had been opened up again. Reason for stalled commits and such is that the SVN to CVS exporter was unable to cope with the new branch and didn’t export anymore. To prevent massive surgeries needed to get the system backup again, we stopped the exporter and fixed it. Simon Nielsen and Peter Wemm did a lot of work to get this in the proper position again.

Unfortunatly this stalled development and working towards RELENG_8 a bit. But headsup, things are now getting back to normal and the queue by re@ is being flushed.

Stay tuned!

Open Source fans? This you must have!

You should order yourself a copy of the BSD Magazine _FAST_! You can find it here make sure you do that as soon as possible to get the magazine in your possession. Why? Because I wrote in it? Because they need it? Because the community needs these kind of things! The magazine only costs a few bucks, it’s worth spending that few bucks!

A BSD Magazine article

At the beginning of this year I was asked by the BSD Magazine people to write an article. I asked what the idea was and I was told that I could emphasize on security, or the installation of FreeBSD and things like that. But then in a way to that everyone can understand it. Also I knew that I was going to move out of the house within limited time, so I agreed writing it if I could have some help.

Jeremy Reed, Michael Lucas and Murray Stokely helped where possible and it resulted in a very nice article. The issue should be out soon, so you are invited to order the issue of and read the article I wrote. It goes about installing FreeBSD 7.1 (yeah it applies to 7.2 as well for whats it worth :) ). So stay tuned!

20090612: In addition you can download/order it from here:

If you read it, I am ofcourse interested in seeing what you think about it, so do not hesitate to let me know!

FreeBSD 8.0 Slush

The FreeBSD 8.0 code slush had been announced. This means that large projects are no longer allowed to do “drive by commits” to the head branch, but that there is an organisation behind it that checks everything and makes sure there are enough people to cover the project and make sure it’s in the best possible shape before the release. The release will take a little to get going, but the process had been started. From here on the team will have to manouver through a pipe that keeps getting smaller and smaller. If your favorite new feature is not in yet, don’t hold your breath because this might mean that it will take a little longer to get it in a first -RELEASE installation. Stay tuned!