Category Archives: Open Source

X11 in a jail with NVidia hardware

Just before christmas I decided I will spend the “immense� amount of 40 EUR for a graphic card for a system which was without one. The system is supposed to replace my dying home-server. I already moved everything, except my Desktop-in-a-Jail (actually it is my home-cinema-jail).

The old system had a Radeon 9200SE, and it was enough for what I used it for. Now… for a few bucks you can get a lot more horsepower today. After looking around a little bit I decided to buy a NVidia card. I made this decision because it looks like I can get better driver support for it. So I got a GeForce GT 520 with 1 GB of RAM (I doubt I will be able to use that much RAM) and without a fan.

With the Radeon 9200SE I was not able to get the 3D stuff activated (at least in the jail, I did not try without), Xorg complains about a missing agpgart module but I have AGP in the kernel (no /dev/agpgart outside the jail). I did not spend time to investigate this, as the main purpose — playing movies — worked. Now with the NVidia card I decided to give the 3D part a try again.

After adding the NVidia device entries to the jail, and a little bit of fighting with the Xorg-HAL interaction, I got a working desktop. The biggest problem to verify that 3D is working was, that I did not had xdriinfo installed. After installing it, I noticed that it does not work with the NVidia driver.  :-(   Next stop nvidia-settings: runs great, displays a nice FreeBSD+NVidia logo, and … tells me that OpenGL is configured. Hmmm… OK, but I want to see it!

As I decided to switch from Gnome to KDE 4 at  the same time (I was using KDE when it was at V 0.x, switched to Gnome as it looked nicer to me, and now I switch back after reading all the stuff in the net that KDE 4 is “better� than Gnome 3), I was a little bit out of knowledge how to see the 3D stuff in action. So I quickly went to the settings and searched for something which looks like it may use 3D. To my surprise, it was already using 3D stuff. Nice. I fully realized how nice, when playing a video and using Alt-Tab to switch windows: the video was playing full speed scaled down in the window-switcher-thumbnail-view.

That was too easy. I am happy about it.

Now that I have a working setup of X11-in-a-jail for Radeon and GeForce cards, I want to cleanup my changes to the kernel and the config files (devfs.rules) and have a look to get this committed. A big part of this work is probably writing documentation (most probably in the wiki).

I still want to see some fancy 3D stuff now. I tried to install x11-clocks/glclock, but the build fails with an undefined reference to ‘glPolygonOffsetEXT’. :-( Any recommendation for a fancy 3D display? My priority is on “fancy/nice� with as less violence as possible. Most probably I will look at it once and then deinstall it again, so it should be available in the Ports Collection (or included in KDE 4).

Share

Tuning guide in the wiki

In the light of the recent benchmark discussion, a volunteer imported the tuning man-page into the wiki. Some comments at some places for possible improvements are already made. Please go over there, have a look, and participate please (testing/verification/discussion/improvements/…).

As always, feel free to register with FirstnameLastname and tell a FreeBSD committer to add you to the contributors group for write access (you also get the benefit to be able to register for an email notification for specific pages).

Share

A phoronix benchmark creates a huge benchmarking discussion

The recent Phoronix benchmark which compared a release candidate of FreeBSD 9 with Oracle Linux Server 6.1 created a huge discussion in the FreeBSD mailinglists. The reason was that some people think the numbers presented there give a wrong picture of FreeBSD. Partly because not all benchmark numbers are presented in the most prominent page (as linked above), but only at a different place. This gives the impression that FreeBSD is inferior in this benchmark while it just puts the focus (for a reason, according to some people) on a different part of the benchmark (to be more specific, blogbench is doing disk reads and writes in parallel, FreeBSD gives higher priority to writes than to reads, FreeBSD 9 outperforms OLS 6.1 in the writes while OLS 6.1 shines with the reads, and only the reads are presented on the first page). Other complaints are that it is told that the default install was used (in this case UFS as the FS), when it was not (ZFS as the FS).

The author of the Phoronix article participated in parts of the discussion and asked for specific improvement suggestions. A FreeBSD committer seems to be already working to get some issues resolved. What I do not like personally, is that the article is not updated with a remark that some things presented do not reflect the reality and a retest is necessary.

As there was much talk in the thread but not much obvious activity from our side to resolve some issues, I started to improve the FreeBSD wiki page about benchmarking so that we are able to point to it in case someone wants to benchmark FreeBSD. Others already chimed in and improved some things too. It is far from perfect, some more eyes — and more importantly some more fingers which add content — are needed. Please go to the wiki page and try to help out (if you are afraid to write something in the wiki, please at least tell your suggestions on a FreeBSD mailinglist so that others can improve the wiki page).

What we need too, is a wiki page about FreeBSD tuning (a first step would be to take the man-page and convert it into a wiki page, then to improve it, and then to feed back the changes to the man-page while keeping the wiki page to be able to cross reference parts from the benchmarking page).

I already told about this in the thread about the Phoronix benchmark: everyone is welcome to improve the situation. Do not talk, write something. No matter if it is an improvement to the benchmarking page, tuning advise, or a tool which inspects the system and suggests some tuning. If you want to help in the wiki, create a FirstnameLastname account and ask a FreeBSD comitter for write access.

A while ago (IIRC we have to think in months or even years) there was some framework for automatic FreeBSD benchmarking. Unfortunately the author run out of time. The framework was able to install a FreeBSD system on a machine, run some specified benchmark (not much benchmarks where integrated), and then install another FreeBSD version to run the same benchmark, or to reinstall the same version to run another benchmark. IIRC there was also some DB behind which collected the results and maybe there was even some way to compare them. It would be nice if someone could get some time to talk with the author to get the framework and set it up somewhere, so that we have a controlled environment where we can do our own benchmarks in an automatic and repeatable fashion with several FreeBSD versions.

Share

(Free)BSD inside Android

Today I was looking into the OpenSource licenses which are displayed for Android (2.3.4). There are several files which come with a BSD license.

During looking at it, I noticed that the libm has the copyright of several FreeBSD people. I did not had an in-deep look if this is because they took the FreeBSD libm, or if this is because parts of the FreeBSD libm where adopted by other BSD projects.

What I noticed is, that some special characters are not displayed correctly. For example the name Dag-Erling Smørgrav looks mangled in the display of the license inside the phone (I hope it is displayed better in my blog). His name is not the only problem case, there are also other characters which are not rendered as expected.

This does not really look professional.

Share

Linuxulator explained (for developers): adding ioctls directly to the kernel

After giving an overview of the in-kernel basics of the Linuxulator, I want now to describe how to add support for new ioctls to the Linuxulator.

Where are the files to modify?

The platform independent code for the ioctls is in SRC/sys/compat/linux/linux_ioctl.c. The defines to have names for the ioctl values are in SRC/sys/compat/linux/linux_ioctl.h.

How to modify them?

First of all create a new header which will contain all the structures, named values and macros for those new ioctls. As written above, the ioctl values (e.g. #define LINUX_VIDIOC_ENCODER_CMD 0x564d /* 0xc028564d */) do not belong there, they shall be added to linux_ioctl.h. During the course of adding support for ioctls, you will need this new header. Add it in the SRC/sys/compat/linux/ directory, and prefix the name with a linux_. It would be good to decide on a common tag here (referenced as yourtag in the following), and stay with it. Use it wherever you need to have some specific name for the ioctl-set you want to add. In this case it would result in linux_yourtag.h (or even linux_ioctl_yourtag.h, depending if this is used for something very specific to the ioctls, or some generic linux feature) as the name of the header file. This was not done in the past, so do not expect that the names inside the linux_ioctl.c file will be consistent to this naming scheme, but it is never too late to correct mistakes of the past (at least in Open Source software development).

Now add this header to linux_ioctl.c (you need to include compat/linux/linux_yourtag.h). After that add the ioctl values to linux_ioctl.h. As can be seen above, the defines should be named the same as on linux, but with a LINUX_ prefix (make sure they where not defined before somewhere else). The ioctl values need to be the same hex values as in Linux, off course. Sort them according to their hex value. When you added all, you need to add two more defines. The LINUX_IOCTL_yourtag_MIN and LINUX_IOCTL_yourtag_MAX ones. The MIN-one needs to be an alias for the first (sorted according to the hex value) ioctl you added, and MAX needs to be an alias for the last (again, sorted according to the hex value) ioctl you added.

The next step is to let the Linuxulator know that it is able to handle the ioctls in the LINUX_IOCTL_yourtag_MIN to LINUX_IOCTL_yourtag_MAX range. Search the static linux_ioctl_function_t section of linux_ioctl.c and add such a variable for your ioctl set. The name of the variable should be something like linux_ioctl_yourtag.

Similar for the handler-definition for this. Search the static struct linux_ioctl_handler section and add a yourtag_handler. Set it to { linux_ioctl_yourtag, LINUX_IOCTL_yourtag_MIN, LINUX_IOCTL_yourtag_MAX }. To make this handler known to the Linuxulator, you need to add it to the DATA_SET section. Add DATA_SET(linux_ioctl_handler_set, yourtag_handler) there.

Now the meat, the function which handles the ioctls. You already defined it as linux_ioctl_function_t, but now you need to write it. The outline of it looks like this:

static int
linux_ioctl_yourtag(struct thread *td, struct linux_ioctl_args *args)
{
        struct file *fp;
        int error;
        switch (args->cmd & 0xffff) {
        case LINUX_an_easy_ioctl:
                break;
        case LINUX_a_not_so_easy_ioctl:
                /* your handling of the ioctl */
                fdrop(fp, td);
                return (error);
        /* some more handling of your ioctls */
        default:
       return (ENOIOCTL);
        }
        error = ioctl(td, (struct ioctl_args *)args);
        return (error);
}

An easy ioctl in the switch above is an ioctl where you do not have to do something but can pass the ioctl through to FreeBSD itself. The not so easy ioctl case is an ioctl where you need to do e.g. a fget(td, args->fd, &fp). This is just an example, there are also other possibilities where you need to do additional stuff before the return, or where you do not pass the ioctl to FreeBSD. A typical example of what needs to be done here is to copy values from linux structures to FreeBSD structures (and the other way too), or to translate between 64bit and 32bit. Linux programs on amd64 are 32bit executables and 32bit structures/pointers. To make this work on amd64, you need to find a way to map between the two. There are examples in the kernel where this is already the case. The more prominent examples in the 64bit<->32bit regard are the v4l and v4l2 ioctls.

The tedious part is to research if a translation has to be done and if yes what needs to be translated how. When this is done, most of the work is not so hard. The linux_yourtag.h should contain the structures you need for this translation work.

It is also possible to add ioctls in a kernel module, but this is not subject to this description (I will update this posting with a link to it when I get time to write about it).

Share

(Free)BSD inside Android

Today I was looking into the OpenSource licenses which are displayed for Android (2.3.4). There are several files which come with a BSD license.

During looking at it, I noticed that the libm has the copyright of several FreeBSD people. I did not had an in-deep look if this is because they took the FreeBSD libm, or if this is because parts of the FreeBSD libm where adopted by other BSD projects.

What I noticed is, that some special characters are not displayed correctly. For example the name Dag-Erling Smørgrav looks mangled in the display of the license inside the phone (I hope it is displayed better in my blog). His name is not the only problem case, there are also other characters which are not rendered as expected.

This does not really look professional.

Share

HOWTO add linux-infrastructure ports for a new linux_base port

In my last blog-post I described how to create a new linux_base port. This blog-post is about the other Linux–ports which make up the Linux–infrastructure in the FreeBSD Ports Collection for a given Linux-release.

What are linux-infrastructure ports?

A linux_base port contains as much as possible and at the same time as little as possible to make up a useful Linux-compatibility-experience in FreeBSD. I know, this is not a descriptive explanation. And it is not on purpose. There are no fixed rules what has to be inside or what not. It “matured� into the current shape. A practical example is, that there is no GUI-stuff in the linux_base. While you need the GUI parts like GTK or QT for software like Skype and acroread, you do not need them for headless game servers. While you may need various libraries for game servers, you may not need those for Skype or acroread. As such some standard parts are in separate ports which are named linux–LINUX_DIST_SUFFIX-NAME. For GTK and the Fedora 10 release this results in linux-f10-gtk2. Such generic ports which depend upon a specific Linux-release make up the Linux-infrastructure in the FreeBSD Ports Collection. Those ports are referenced in port-Makefiles via the USE_LINUX_APPS variable, e.g. USE_LINUX_APPS=gtk2.

If you created a new linux_base port, you need most standard infrastructure ports in a version for the Linux-release used in the linux_base port, to have the Linux-application ports in the FreeBSD Ports Collection working (if you are unlucky, some ports do not play well with the Linux-release you have chosen, but this is out of the scope of this HOWTO).

Updating Mk/bsd.linux-apps.mk

 First we need to set the LINUX_DIST_SUFFIX variable to a value suitable to the new Linux-release. This is done in the conditional which checks the OVERRIDE_LINUX_NONBASE_PORTS variable for valid values. Add an appropriate conditional, and do not forget to add the new valid value to the IGNORE line in the last else branch of the conditional.

The next step is to check the _LINUX_APPS_ALL and _LINUX_26_APPS variables. If there are some infrastructure ports which are not available for the new Linux-release, the conditional which checks the availability of a given infrastructure port for a given Linux-release needs to be modified. If at a later step you notice that there are some additional infrastructure ports necessary for the new Linux-release, _LINUX_APPS_ALL and the check-logic needs to be modified too (e.g. add a new variable for your Linux-release, add the content of the variable to _LINUX_APPS_ALL, and change the check to do the right thing).

After that two tedious parts need to be done.

For each infrastructure port there is a set of variables. The name_PORT variable contains the location of the port in the Ports Collection. Typically you do not have to change it (if you really want to change it, do not do it, fix the naming of the infrastructure port instead), because we use a naming convention here which includes the LINUX_DIST_SUFFIX. The name_DETECT variable is an internal variable, do not change it (if you create a new infrastructure port, copy it from somewhere else and make sure the name in value of the variable matches the port name in the name of the variable). Then there are several name_suffix_FILE variables. Leave the existing ones alone, and add a new one with the correct suffix for your new Linux-release. The value of the variable needs to be an important file which is installed by the infrastructure port in question. FYI: The content of the name_suffix_FILE variables are used to set the name_DETECT variables, depending on the Linux-relase the name_DETECT variables are used to check if the port is already installed. Ideally the name_suffix_FILE variable points to a library in the port. The name_DEPENDS variable lists dependencies of this infrastructure port. If the dependencies changed in your Linux-release, you need to add a conditional to change the dependency if LINUX_DIST_SUFFIX is set to your Linux-release.

Normally this is all what needs to be done in PORTSDIR/Mk/bsd.linux-apps.mk, the rest of the file is code to check dependencies and some correctness checks.

The second tedious part is to actually create all those infrastructure ports. Normally you can copy an existing infrastructure port, rename it, adjust the PORTNAME, PORTVERSION, PORTREVISION, MASTER_SITES, PKGNAMEPREFIX, DISTFILES, CONFLICTS (also in all other Linux-release versions of this infrastructure port), LINUX_DIST_VER, RPMVERSION (if set/neccesary) and SRC_DISTFILE variables, generate the distfile checksums (make makesum), and fix the plist. I suggest to script parts of this work (as of this writing Freshports counts 68 ports where the portname starts with linux-f10-).

Adding new infrastructure ports, or removing infrastructure ports for a given Linux-release

If your Linux-release does not come with a package for an existing infrastructure port, just do not create a corresponding name_suffix_FILE line. You still need to do the right thing regarding dependencies of ports which depend upon this non-existing infrastructure port (if your Linux-release comes with packages for them).

To add a new infrastructure port, copy an existing block, rename the variables, set them correctly, add a new variable for your Linux-release in the first _LINUX_APPS_ALL section, add the content of this variable to _LINUX_APPS_ALL, and change the check-logic as described above.

Final words

If you have something which installs and deinstalls correctly, feel free to provide it on [email protected] for review/testing. If you have questions during the porting, feel also free to send a mail there.

Share

HOWTO create a new linux_base port

FreeBSD is in need of a new linux_base port. It is on my TODO list since a long time, but I do not get the time to create one. I still do not have the time to work on a new one, but when you read this, I managed to get the time to create a HOWTO which describes what needs to be done to create a new linux_base port.

I will not describe how to create a new linux_base port from scratch, I will just describe how you can copy the last one and update it to something newer based upon the existing infrastructure for RPM packages.

Specific questions which come up during porting a new Linux release should be asked on [email protected],  there are more people which can answer questions than here in my blog. I will add useful information to this HOWTO if necessary.

In the easy case most of the work is searching the right RPMs and their dependencies to use, and to create the plist.

Why do we need a new linux_base port?

The current linux_base port is based upon Fedora 10, which is end of life since December 2009. Even Fedora 13 is already end of life. Fedora 16 is supposed to be released this year. From a support point of view, Fedora 15 or maybe even Fedora 16 would be a good target for the next linux_base port. Other alternatives would be to use an extended lifetime release of another RPM based distribution, like for example CentOS 6 (which seems to be based upon Fedora 12 with backports from Fedora 13 and 14). Using a Linux release which is told to be supported for at least 10 years, sounds nice from a FreeBSD point of view (only minor changes to the linux ports in such a case, instead of creating a complete new linux_base each N+2 releases like with Fedora), but it also means additional work if you want to create the first linux_base port for it.

The mysteries you have to conquer if you want to create a new linux_base port

What we do not know is, if Fedora 15/16, CentOS 6, or any other Linux release will work in a supported FreeBSD release. There are two ways to find this out.

The first one is to take an existing Linux system, chroot into it (either via NFS or after making a copy into a directory of a FreeBSD system), and to run a lot of programs (acroread, skype, shells, scripts, …). The LTP testsuite is not that much useful here, as it will test mostly kernel features, but we do not know which kernel features are mandatory for a given userland of a Linux release.

The second way of testing if a given Linux release works on FreeBSD is to actually create a new linux_base port for it and test it without chrooting.

The first way is faster, if you are only interested in testing if something works. The second way provides an easy to setup testbed for FreeBSD kernel developers to fix the Linuxulator so that it works with the new linux_base port. Both ways have their merits, but it is up to the person doing the work to decide which way to go.

The meat: HOWTO create a new linux_base port

First off, you need a system (or a jail) without any linux_base port installed. After that you can create a new linux_base port (= lbN), by just making a copy of the latest one (= lbO). In lbN you need to add lbO as a CONFLICT, and in all other existing linux_base ports, you need to add lbN as a conflict.

Change the PORTNAME, PORTVERSION, reset the PORTREVISION in lbN, and set LINUX_DIST_VER  to the new Linux-release version in the lbN Makefile (this is used in PORTSDIR/Mk/bsd.linux-rpm.mk and PORTSDIR/Mk/bsd.linux-apps.mk).

If you do not stay with Fedora, there is some more work to do before you can have a look at chosing RPMs for installation. You need to have a look at PORTSDIR/Mk/bsd.linux-rpm.mk and add some cases for the new LINUX_DIST you want to use. Do not forget to set LINUX_DIST in the lbN Makefile to the name of the distribution you use. You also need to augment the LINUX_DIST_VER check in PORTSDIR/Mk/bsd.linux-rpm.mk with some LINUX_DIST conditionals. If you are lucky, the directory structure for downloads is similar to the Fedora structure, and there is not a lot to do here.

When this is done, you can have a look at the BIN_DISTFILES variable in the lbN Makefile. Try to find similar RPMs for the new Linux release you want to port. Some may not be available, and it may also be the case that different ones are needed instead. I suggest to first work with the ones which are available (make makesum, test install and create plist). After that you need to find out what the replacement RPMs for non-existing ones are. You are on your own here. Search around the net, and/or have a look at the dependencies in the RPMs of lbO to determine if something was added as a dependency of something else or not (if not, forget about it ATM). When you managed to find replacement RPMs, you can now have a look at the dependencies of the RPMs in lbN. Do not add blindly all dependencies, not all are needed in FreeBSD (the linux_base ports are not supposed to create an environment which you can chroot into, they are supposed to augment the FreeBSD system to be able to run Linux programs in ports like they where FreeBSD native programs). What you need in the linux_base ports are libraries, config and data files which do not exist in FreeBSD or have a different syntax than in FreeBSD (those config or data files which are just in a different place, can be symlinked), and basic shell commands (which commands are needed or not… well… good question, in the past we made decisions what to include based upon problem reports from users). Now for the things which are not available and where not added as a dependency. Those are things which are either used during install, or where useful to have in the past. Find out by what it was replaced and have a look if this replacement can easily be used instead. If it can be used, add it. If not, well… bad luck, we (the FreeBSD community) will see how to handle this somehow.

If you think that you have all you need in BIN_DISTFILES, please update SRC_DISTFILES accordingly and generate the distfile via  make –DPACKAGE_BUILDING makesum to have the checksums of the sources (for legal reasons we need them on our mirrors).

The next step is to have a look at REMOVE_DIRS, REMOVE_FILES and ADD_DIRS if something needs to be modified. Most of them are there to fall back to the corresponding FreeBSD directories/files, or because they are not needed at all (REMOVE_*). Do not remove directories from ADD_DIRS, they are created here to fix some edge conditions (I do not remember exactly why we had to add them, and I do not take the time ATM to search in the CVS history).

If you are lucky, this is all (make sure the plist is correct). If you are not lucky and you need to make some modifications to files, have a look at the do-build target in the Makefile, this is the place where some changes are done to create a nice user experience.

If you arrive here while creating a new linux_base port, lean back and feel a bit proud. You managed to create a new linux_base port. It is not very well tested at this moment, and it is far from everything which needs to be done to have the complete Linux infrastructure for a given Linux release, but the most important part is done. Please notify [email protected] and call for testers.

What is missing?

The full Linuxulator infrastructure for the FreeBSD Ports Collection has some more ports around a linux_base port. Most of the infrastructure for this is handled in Mk/bsd.linux-apps.mk.

UPDATE: I got some time to write how to update the Linux-infrastructure ports.

Share

v4l2 support in the linuxulator now in 8-stable

I MFCed the v4l2 support in the linuxulator to 8–stable. This allows now to use v4l2–webcams in skype/flash on 8-stable too.

Share

What keramida said… » FreeBSD 2011-05-20 18:11:15

I found out how to concatenate the parts of a video to a single file with MPlayer. It’s relatively easy, so this is just a mini-post to save it for posterity:

$ cat video_part1.avi video_part2.avi ... > temp.avi
$ mencoder -forceidx -oac copy -ovc copy \
    temp.avi -o final.avi && \
    rm temp.avi

Filed under: Computers, Free software, FreeBSD, Linux, Open source, Software Tagged: Computers, Free software, FreeBSD, hellug, Linux, Open source, Software

How I setup a Jail-Host

Everyone has his own way of setting up a machine to serve as a host of multiple jails. Here is my way, YMMV.

Initial FreeBSD install

I use several harddisks in a Software–RAID setup. It does not matter much if you set them up with one big partition or with several partitions, feel free to follow your preferences here. My way of partitioning the harddisks is described in a previous post. That post only shows the commands to split the harddisks into two partitions and use ZFS for the rootfs. The commands to initialize the ZFS data partition are not described, but you should be able to figure it out yourself (and you can decide on your own what kind of RAID level you want to use). For this FS I set atime, exec and setuid to off in the ZFS options.

On the ZFS data partition I create a new dataset for the system. For this dataset I set atime, exec and setuid to off in the ZFS options. Inside this dataset I create datasets for /home, /usr/compat, /usr/local, /usr/obj, /usr/ports/, /usr/src, /usr/sup and /var/ports. There are two ways of doing this. One way is to set the ZFS mountpoint. The way I prefer is to set relative symlinks to it, e.g. “cd /usr; ln –s ../data/system/usr_obj obj�. I do this because this way I can temporary import the pool on another machine (e.g. my desktop, if the need arises) without fear to interfere with the system. The ZFS options are set as follows:

ZFS options for data/system/*

Dataset

Option

Value
data/system/homeexecon
data/system/usr_compatexecon
data/system/usr_compatsetuidon
data/system/usr_localexecon
data/system/usr_localsetuidon
data/system/usr_objexecon
data/system/usr_portsexecon
data/system/usr_portssetuidon
data/system/usr_srcexecon
data/system/usr_supsecondarycachenone
data/system/var_portsexecon

The exec option for home is not necessary if you keep separate datasets for each user. Normally I keep separate datasets for home directories, but Jail-Hosts should not have users (except the admins, but they should not keep data in their homes), so I just create a single home dataset. The setuid option for the usr_ports should not be necessary if you redirect the build directory of the ports to a different place (WRKDIRPREFIX in /etc/make.conf).

Installing ports

The ports I install by default are net/rsync, ports-mgmt/portaudit, ports-mgmt/portmaster, shells/zsh, sysutils/bsdstats, sysutils/ezjail, sysutils/smartmontools and sysutils/tmux.

Basic setup

In the crontab of root I setup a job to do a portsnap update once a day (I pick a random number between 0 and 59 for the minute, but keep a fixed hour). I also have http_proxy specified in /etc/profile, so that all machines in this network do not download everything from far away again and again, but can get the data from the local caching proxy. As a little watchdog I have a little @reboot rule in the crontab, which notifies me when a machine reboots:

@reboot grep "kernel boot file is" /var/log/messages | mail -s "`hostname` rebooted" root >/dev/null 2>&1

This does not replace a real monitoring solution, but in cases where real monitoring is overkill it provides a nice HEADS-UP (and shows you directly which kernel is loaded in case a non-default one is used).

Some default aliases I use everywhere are:

alias portmlist="portmaster -L | egrep -B1 '(ew|ort) version|Aborting|installed|dependencies|IGNORE|marked|Reason:|MOVED|deleted|exist|update' | grep -v '^--'"
alias portmclean="portmaster -t --clean-distfiles --clean-packages"
alias portmcheck="portmaster -y --check-depends"

Additional devfs rules for Jails

I have the need to give access to some specific devices in some jails. For this I need to setup a custom /etc/devfs.rules file. The files contains some ID numbers which need to be unique in the system. On a 9-current system the numbers one to four are already used (see /etc/defaults/devfs.rules). The next available number is obviously five then. First I present my devfs.rules entries, then I explain them:

[devfsrules_unhide_audio=5]
add path 'audio*' unhide
add path 'dsp*' unhide
add path midistat unhide
add path 'mixer*' unhide
add path 'music*' unhide
add path 'sequencer*' unhide
add path sndstat unhide
add path speaker unhide
[devfsrules_unhide_printers=6]
add path 'lpt*' unhide
add path 'ulpt*' unhide user 193 group 193
add path 'unlpt*' unhide user 193 group 193
[devfsrules_unhide_zfs=7]
add path zfs unhide
[devfsrules_jail_printserver=8]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_unhide_printers
add include $devfsrules_unhide_zfs
[devfsrules_jail_withzfs=9]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_unhide_zfs

The devfs_rules_unhide_XXX ones give access to specific devices, e.g. all the sound related devices or to local printers. The devfsrules_jail_XXX ones combine all the unhide rules for specific jail setups. Unfortunately the include directive is not recursive, so that we can not include the default devfsrules_jail profile and need to replicate its contents. The first three includes of each devfsrules_jail_XXX accomplish this. The unhide_zfs rule gives access to /dev/zfs, which is needed if you attach one or more ZFS datasets to a jail. I will explain how to use those profiles with ezjail in a follow-up post.

Jails setup

I use ezjail to manage jails, it is more comfortable than doing it by hand while at the same time allows me to do something by hand. My jails normally reside inside ZFS datasets, for this reason I have setup a special area (ZFS dataset data/jails) which is handled by ezjail.The corresponding ezjail.conf settings are:

ezjail_jaildir=/data/jails
ezjail_use_zfs="YES"
ezjail_jailzfs="data/jails"

I also disabled procfs and fdescfs in jails (but they can be enabled later for specific jails if necessary).

Unfortunately ezjail (as of v3.1) sets the mountpoint of a newly created dataset even if it is not necessary. For this reason I always issue a “zfs inherit mountpoint � after creating a jail. This simplifies the case where you want to move/rename a dataset and want to have the mountpoint automcatically follow the change.

The access flags of  /data/jails directory are 700, this prevents local users (there should be none, but better safe than sorry) to get access to files from users in jails with the same UID.

After the first create/update of the ezjail basejail the ZFS options of basejail (data/jails/basejail) and newjail (data/jails/newjail) need to be changed. For both exec and setuid should be changed to “on� The same needs to be done after creating a new jail for the new jail (before starting it).

The default ezjail flavour

In my default ezjail flavour I create some default user(s) with a basesystem-shell (via /data/jails/flavours/mydef/ezjail.flavour) before the package install, and change the shell to my preferred zsh afterwards (this is only valid if the jails are used only by in-house people, if you want to offer lightweight virtual machines to (unknown) customers, the default user(s) and shell(s) are obviously up to discussion). At the end I also run a “/usr/local/sbin/portmaster –y –check-depends� to make sure everything is in a sane state.

For the packages (/data/jails/flavours/mydef/pkg/) I add symlinks to the unversioned packages I want to install. I have the packages in a common (think about setting PACKAGES in make.conf and using PACKAGES/Latest/XYZ.tbz) directory (if they can be shared over various flavours), and they are unversioned so that I do not have to update the version number each time there is an update. The packages I install by default are bsdstats, portaudit, portmaster, zsh, tmux and all their dependencies.

In case you use jails to virtualize services and consolidate servers (e.g. DNS, HTTP, MySQL each in a separate jail) instead of providing lightweight virtual machines to (unknown) customers, there is also a benefit of sharing the distfiles and packages between jails on the same machine. To do this I create /data/jails/flavours/mydef/shared/ports/{distfiles,packages} which are then mounted via nullfs or NFS into all the jails from a common directory. This requires the following variables in /data/jails/flavours/mydef/etc/make.conf (I also keep the packages for different CPU types and compilers in the same subtree, if you do not care, just remove the “/${CC}/${CPUTYPE}� from the PACAKGES line):

DISTDIR=  /shared/ports/distfiles
PACKAGES= /shared/ports/packages/${CC}/${CPUTYPE}

New jails

A future post will cover how I setup new jails in such a setup and how I customize the start order of jails or use some non-default settings for the jail-startup.

Share

Video4Linux2 sup­port in FreeBSD (linuxulator)

I com­mit­ted the v4l2 sup­port into the lin­ux­u­la­tor (in 9-current). Part of this was the import of the v4l2 header from linux. We have the per­mis­sion to use it (like the v4l one), it is not licensed via GPL. This means we can use it in FreeBSD native dri­vers, and they are even allowed to be com­piled into GENERIC (but I doubt we have a dri­ver which could pro­vide the v4l2 inter­face in GENERIC).

The code I com­mit­ted is “just� the glue-code which allows to use FreeBSD native devices which pro­vide a v4l2 inter­face (e.g. multimedia/pwcbsd or multimedia/webcamd) from linux programs.

Thanks to nox@ for writing the glue code.

Share

Another root-on-zfs HOWTO (optimized for 4k-sector drives)

After 9 years with my current home-server (one jail for each service like MySQL, Squid, IMAP, Webmail, …) I decided that it is time to get something more recent (specially as I want to install some more jails but can not add more memory to this i386 system).

With my old system I had an UFS2-root on a 3-way-gmirror, swap on a 2-way-gmirror and my data in a 3-partition raidz (all in different slices of the same 3 harddisks, the 3rd slice which would correspond to the swap was used as a crashdump area).

For the new system I wanted to go all-ZFS, but I like to have my boot area separated from my data area (two pools instead of one big pool). As the machine has 12 GB RAM I also do not configure swap areas (at least by default, if I really need some swap I can add some later, see below). The system has five 1 TB harddisks and a 60 GB SSD. The harddisks do not have 4k-sectors, but I expect that there will be more and more 4k-sector drives in the future. As I prefer to plan ahead I installed the ZFS pools in a way that they are “4k-ready�. For those which have 4k-sector drives which do not tell the truth but announce they have 512 byte sectors (I will call them pseudo-4k-sector drives here) I include a description how to properly align the (GPT-)partitions.

A major requirement to boot 4k-sector-size ZFS pools is ZFS v28 (to be correct here, just the boot-code needs to support this, so if you take the pmbr and gptzfsboot from a ZFS v28 system, this should work… but I have not tested this). As I am running 9-current, this is not an issue for me.

A quick description of the task is to align the partition/slices properly for pseudo-4k-sector drives, and then use gnop temporary during pool creation time to have ZFS use 4k-sectors during the lifetime of the pool. The long description follows.

The layout of the drives

The five equal drives are partitioned with a GUID partition table (GPT). Each drive is divided into three partitions, one for the boot code, one for the root pool, and one for the data pool. The root pool is a 3-way mirror and the data pool is a raidz2 pool over all 5 disks. The remaining space on the two harddisks which do not take part in the mirroring of the root pool get swap partitions of the same size as the root partitions. One of them is used as a dumpdevice (this is –current, after all), and the other one stays unused as a cold-standby. The 60 GB SSD will be used as a ZFS cache device, but as of this writing I have not decided yet if I will use it for both pools or only for the data pool.

Calculating the offsets

The first sector after the GPT (created with standard settings) which can be used as the first sector for a partition is sector 34 on a 512 bytes-per-sector drive. On a pseudo-4k-sector drive this would be somewhere in the sector 4 of a real 4k-sector, so this is not a good starting point. The next 4k-aligned sector on a pseudo-4k-sector drive is sector 40 (sector 5 on a real 4k-sector drive).

The first partition is the partition for the FreeBSD boot code. It needs to have enough space for gptzfsboot. Only allocating the space needed for gptzfsboot looks a little bit dangerous regarding future updates, so my harddisks are configured to allocate half a megabyte for it. Additionally I leave some unused sectors as a safety margin after this first partition.

The second partition is the root pool (respectively the swap partitions). I let it start at sector 2048, which would be sector 256 on a real 4k-sector drive (if you do not want to waste less than half a megabyte just calculate a lower start sector which is divisible by 8 (-> start % 8 = 0)). It is a 4 GB partition, this is enough for the basesystem with some debug kernels. Everything else (/usr/{src,ports,obj,local}) will be in the data partition.

The last partition is directly after the second and uses the rest of the harddisk rounded down to a full GB (if the disk needs to be replaced with a similar sized disk there is some safety margin left, as the number of sectors in harddisks fluctuates a little bit even in the same models from the same manufacturing charge). For my harddisks this means a little bit more than half a gigabyte of wasted storage space.

The commands to partition the disks

In the following I use ada0 as the device of the disk, but it also works with daX or adX or similar. I installed one disk from an existing 9-current system instead of using some kind of installation media (beware, the pool is linked to the system which creates it, I booted a life-USB image to import it on the new system and copied the zpool.cache to /boot/zfs/ after importing on the new system).

Create the GPT:

gpart create -s gpt ada0

Create the boot partition:

gpart add -b 40 -s 1024 -t freebsd-boot ada0

Create the root/swap partitions and name them with a GPT label:

gpart add -b 2048 -s 4G -t freebsd-zfs -l rpool0 ada0

or for the swap

gpart add -b 2048 -s 4G -t freebsd-swap -l swap0 ada0

Create the data partition and name them with a GPT label:

gpart add -s 927G -t freebsd-zfs -l data0 ada0

Install the boot code in partition 1:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

The result looks like this:

# gpart show ada0
=>        34  1953525101  ada0  GPT  (931G)
          34           6        - free -  (3.0k)
          40        1024     1  freebsd-boot  (512k)
        1064         984        - free -  (492k)
        2048     8388608     2  freebsd-zfs  (4.0G)
     8390656  1944059904     3  freebsd-zfs  (927G)
  1952450560     1074575        - free -  (524M)

Create the pools with 4k-ready internal structures

Creating a ZFS pool on one of the ZFS partitions without preparation will not create a 4k-ready pool on a pseudo-4k-drive. I used gnop (the settings do not survive a reboot) to make the partition temporary a 4k-sector partition (only the command for the root pool is shown, for the data partition gnop has to be used in the same way):

gnop create -S 4096 ada0p2
zpool create -O utf8only=on -o failmode=panic rpool ada0p2.nop
zpool export rpool
gnop destroy ada0p2.nop
zpool import rpool

After the pool is created, it will keep the 4k-sectors setting, even when accessed without gnop. You can ignore the options I used to create the pool, they are just my preferences (and the utf8only setting can only be done at pool creation time). If you prepare this on a system which already has a zpool on its own, you can maybe specify “-o cachefile=/boot/zfs/zpool2.cache� and copy it to the new pool as zpool.cache to make it bootable without the need of a life-image for the new system (I did not test this).

Verifying if a pool is pseudo-4k-ready

To verify that the pool will use 4k-sectors, you can have a look at the ashift values of the pool (the ashift is per vdev, so if you e.g. concattenate several mirrors, the ashift needs to be verified for each mirror, and if you concattenate just a bunch of disks, the ashift needs to be verified for all disks). It needs to be 12. To get the ashift value you can use zdb:

zdb rpool | grep ashift

Setting up the root pool

One of the benefits of root-on-zfs is that I can have multiple FreeBSD boot environments (BE). This means that I not only can have several different kernels, but also several different userland versions. To handle them comfortably, I use manageBE from Philipp Wuensche. This requires a specific setup of the root pool:

zfs create rpool/ROOT
zfs create rpool/ROOT/r220832M
zpool set bootfs=rpool/ROOT/r220832M rpool
zfs set freebsd:boot-environment=1 rpool/ROOT/r220832M   # manageBE setting

The r220832M is my initial BE. I use the SVN revision of the source tree which was used during install of this BE as the name of the BE here. You also need to add the following line to /boot/loader.conf:

vfs.root.mountfrom="zfs:rpool/ROOT/r220832M"

As I want to have a shared /var and /tmp for all my BEs, I create them separately:

zfs create -o exec=off -o setuid=off -o mountpoint=/rpool/ROOT/r220832M/var rpool/var
zfs create -o setuid=off -o mountpoint=/rpool/ROOT/r220832M/tmp rpool/tmp

As I did this on the old system, I did not set the mountpoints to /var and /tmp, but this has to be done later.

Now the userland can be installed (e.g. buildworld/installworld/buildkernel/buildkernel/mergemaster with DESTDIR=/rpool/ROOT/r220832M/, do not forget to put a good master.passwd/passwd/group in the root pool).

When the root pool is ready make sure an empty /etc/fstab is inside, and configure the root as follows (only showing what is necessary for root-on-zfs):

loader.conf:
---snip---
vfs.root.mountfrom="zfs:rpool/ROOT/r220832M"
zfs_load="yes"
opensolaris_load="yes"
---snip---
rc.conf
---snip---
zfs_enable="YES"
---snip---

At this point of the setup I unmounted all zfs on rpool, set the mountpoint of rpool/var to /var and of rpool/tmp to /tmp, exported the pool and installed the harddisk in the new system. After booting a life-USB-image, importing the pool, putting the resulting zpool.cache into the pool (rpool/ROOT/r220832M/boot/zfs/), I rebooted into the rpool and attached the other harddisks to the pool (“zpool attach rpool ada0p2 ada1p2�, “zpool attach rpool ada0p2 ada2p2�):

After updating to a more recent version of 9-current, the BE looks like this now:

# ./bin/manageBE list
Poolname: rpool
BE                Active Active Mountpoint           Space
Name              Now    Reboot -                    Used
----              ------ ------ ----------           -----
r221295M          yes    yes    /                    2.66G
cannot open '-': dataset does not exist
r221295M@r221295M no     no     -
r220832M          no     no     /rpool/ROOT/r220832M  561M
Used by BE snapshots: 561M

The little bug above (the error message which is probably caused by the snapshot which shows up here probably because I use listsnapshots=on) is already reported to the author of manageBE.

Share

DTrace probes for the Linuxulator updated

If someone had a look at the earlier post about DTrace probes for the Linuxulator: I updated the patch at the same place. The difference between the previous one is that some D-scripts are fixed now to do what I meant, specially the ones which provide statistics output.

Share

New DTrace probes for the linuxulator

I forward ported my DTrace probes for the FreeBSD linuxulator from a 2008-current to a recent –current. I have not the complete FreeBSD linuxulator covered, but a big part is already done. I can check the major locks in the linuxulator, trace futexes, and I have a D-script which yells at a lot of errors which could happen but should not.

Some of my D-scripts need some changes, as real-world testing showed that they are not really working as expected. They can get overwhelmed by the amount of speculation and dynamic variables (error message: dynamic variable drops with non-empty dirty list). For the dynamic variables problem I found a discussion on the net with some suggestions. For the speculation part I expect similar tuning-possibilities.

Unfortunately the D-script which checks the internal locks fails to compile. Seems there is a little misunderstanding on my side how the D-language is supposed to work.

I try to get some time later to have a look at those problems.

During my development I stumbled over some generic DTrace problems with the SDT provider I use for my probes:

  • If you load the Linux module after the SDT module, your system will panic as soon as you want to access some probes, e.g. “dtrace –lâ€? will panic the system. Loading the Linux module before the SDT module prevents the panic.
  • Unloading the SDT module while the Linux module with the SDT probes is still loaded panics the system too. Do not unload the Linux module if you run with my patch.

According to avg@ those are known problems, but I think nobody is working on this. This is bad, because this means I can not commit my current patchset.

If someone wants to try the new DTrace probes for the linuxulator, feel free to go to http://www.Leidinger.net/FreeBSD/current-patches/ and download linuxulator-dtrace.diff. I do not offer a working hyperlink here on purpose, the SDT bugs can hurt if you are not careful, and I want to make the use of this patch a strong opt-in because of this. If the patch hurts you, it is your fault, you have been warned.

Share

VDR ports docs

After a quick discussion with nox@ I made a copy&paste of his “VDR is committed now�-mail into the FreeBSD wiki. I also re-styled some small parts of it to fit better into the wiki. It is not perfect, but already usable. Now interested people can go and improve the docs there.

Thanks to Juergen for all his work in this area!

Share

Non-default linux base ports deprecated

Yesterday I deprecated the non-default Fedora based Linux base ports. This means fc6, f7, f8 and f9 will vanish soon (I decided for one month of expiry time). This is because all of them are End of Life upstream since a long time (= no security updates).

The fc4 and f10 ones are still available — even if they are End of Life too — because FreeBSD 7.x can not use something newer than the fc4 one, and we have not tested yet a more recent Linux distribution.

Probably the most easy way to update the Linux base ports to something newer is to stay with Fedora (we have a lot of ports-infrastructure for it already). Unfortunately it is not known if something newer works without problems (missing epoll/inotify support could be a roadblock here in case it is extensively used in a more recent version).

I want to get some time to have a look if a more recent Fedora version is suitable for the use as a Linux base in FreeBSD 8.x+, but I do not have an estimate when I can start and how long it may take. In case someone already tested a more recent Fedora version feel free to share your experience.

Share

HOWTO: creating your own updated linux RPM for the FreeBSD linuxulator

Background info

The FreeBSD linux compatibility environment currently uses RPMs from Fedora 10. Unfortunately Fedora 10 is end of life since a while. For one of the RPMs (the pango one) we where aware of a security vulnerability. As we do not know if it is feasible to update the linuxulator ports to something more recent, I decided to setup a VM with Fedora 10 and generate a new RPM for the linux-f10-pango port. Thanks to Luchesar V. ILIEV for explaining me how to do this.

Setup of the VM

I used VirtualBox 4.0.4 on a Solaris 10 x86 machine. I configured a fixed size disk of 16 GB and kept the default network setup (after installing the guest tools / kernel modules I switched to virtio, as I was not able to do anything useful besides a ping) and RAM size. The CD/DVD drive was configured to use the image of the full Fedora 10 DVD for i386 systems.

Setup of Fedora 10

Booting the VM from the DVD leads to the graphical Fedora 10 install software (after chosing to install a new system on the console). There I accepted all the defaults, except for the software to install. I deselected the Office and Productivity group and selected the Software Development group. When I was asked if I want to install some additional RPMs I had a look at the complete list and installed some I thought are necessary. I do not remember anymore which ones I chose, but everything which looks related to RPM building is a good candidate.

After a while the install will be finished and you can boot into the new system (eject the DVD from the drive before reboot). After reboot chose to install the Guest Additions in the menu of the VM. This should mount the ISO image in the VM. As root execute the file for Linux. This will build some kernel modules for better integration (e.g. seamless integration of the mouse between your desktop and the VM). At this point I rebooted and configured virtio as the NIC. I also had to configure the network settings by hand, as the GUI tool did not safe all the settings correctly.

Update and install of required RPMs

After the VM was up and the network configured, I updated the entire system (chose System Update in the menu). To update the pango port, I had to install the libthai-devel RPM. I had the RPM for it (and all the files I need to build a new pango RPM) already downloaded, so I did a “yum install /path/to/rpm�. At this point I was ready to create the RPM build environment.

The RPM build environment

As a normal user I executed the command rpmdev-setuptree which creates the directory rpmbuild and populates it with some directories. Now you just need to find a suitable .spec file and put it into rpmbuild/SPECS, put the sources (and maybe patches referenced in the .spec file) into rpmbuild/SOURCES, and you are ready to go (I patched pango.spec for a more recent pango version, basically just changing the version numbers). If you want to have a custom packager and vendor attribute in the RPM, you can add a line for each to ~/.rpmmacros, e.g. %packager yournamehere and %vendor whateverisappropriate. I used my @FreeBSD.org EMail address as the packager, and FreeBSD as the vendor.

Building a RPM

I used rpmbuild –ba –target i386-redhat-linux-gnu –clean rpmbuild/SPECS/pango.spec to build the new pango RPM. If everything is OK, the resulting RPMs (a source RPM, a devel RPM, a debuginfo RPM and the RPM for the binaries) are in rpmbuild/RPMS and rpmbuild/SRPMS. For a FreeBSD port we just need the source RPM (to comply to the (L)GPL) and the RPM for the binaries.

Additional info

The i386-redhat-linux-gnu string which is used for the –target option of the rpmbuild command is what seems to be used to build the Fedora 10 RPMs. After building pango, the RPM has i686-pc-linux-gnu in some filenames instead (the default value for this setup). The binaries seem to be compiled for i386, so there should be no problem even for old systems.

Share

A new linux-f10-pango port is ready

In the last days I took (and even had) the time to install a VM with Fedora 10, updated all the packages after installation, and created a new linux-f10-pango port (v 1.28.3). I did this because the port has a security vulnerability according to our VuXML DB and there where more and more reports in the last months from users which had a problem with this.

During the update of the port I noticed that the port does not contain a FORBIDDEN entry, just portaudit complains about it because there is an entry in the VuXML. That is not nice. I was told that the ports slush will be lifted soon (I need to bump some PORTREVISIONs), this means that I can commit the update probably tomorrow, just in time when the new RPM should hit the FreeBSD distribution infrastructure (MASTER_SITE_LOCAL is updated once a day from a specific folder in our home directories).

Thanks to Luchesar V. ILIEV for the nice writeup of what to install in Fedora 10 to be able to build RPMs, and the description of how to build your own RPM.

Share

Additional FEATURE macros on the way

It seems I have a bit of free time now to take care about some FreeBSD related things.

As part of this I already committed the UFS/FFS related FEATURE macros which where developed by kibab@ during the Google Summer of Code 2010. The network/ALTQ related FEATURE macros are in the hands of bz@, he already reviewed them and wants to commit them (with some changes) as part of his improvements of parts of the network related code.

The GEOM related FEATURE macros I just send some minutes ago to geom@ for review. All the rest went out to hackers@ for review. The rest in this case is related to AUDIT, CAM, IPC, KTR, MAC, NFS, NTP, PMC, SYSV and a few other things.

If everything is committed, it should look a bit like this if queried from userland (not all features are shown, those are just the ones which are enabled in the kernel in one of my machines):

kern.features.scbus: 1
kern.features.geom_vol: 1
kern.features.geom_part_bsd: 1
kern.features.geom_part_ebr_compat: 1
kern.features.geom_part_ebr: 1
kern.features.geom_part_mbr: 1
kern.features.kposix_priority_scheduling: 1
kern.features.kdtrace_hooks: 1
kern.features.ktrace: 1
kern.features.invariant_support: 1
kern.features.compat_freebsd7: 1
kern.features.compat_freebsd6: 1
kern.features.pps_sync: 1
kern.features.stack: 1
kern.features.sysv_msg: 1
kern.features.sysv_sem: 1
kern.features.sysv_shm: 1
kern.features.posix_shm: 1
kern.features.ffs_snapshot: 1
kern.features.softupdates: 1
kern.features.ufs_acl: 1

Share