Monthly Archives: September 2011

Wireless update – 802.11n development

A few of us have been working towards 802.11n support in FreeBSD. Bernhard Schmidt and I were hoping to get 802.11n support up to scratch before 9.0-RELEASE but a combination of timing constraints and work constraints has hampered things somewhat.

But that said, we the net80211 support for 802.11n is in a lot better shape than it was in previous releases. Bernhard has been working out the kinks in the intel driver (if_iwn) and has 802.11n support mostly stable for those NICs. Someone else has been working on if_ral 802.11n support and has had quite a bit of success there. And I've been working on if_ath/ath_hal support.

(As a side-note, I've ordered some Marvell if_mwl compatible NICs to do some testing with.)

The 802.11n support in the FreeBSD atheros driver is already mostly usable for testing - 802.11n TX and RX works, but 802.11n TX aggregation doesn't work. So if you'd like to test it out in 9.0-RELEASE, you can do this:

* build a kernel with "options ATH_ENABLE_11N";
* ifconfig wlan0 -ampdutx
* associate as per normal to an 802.11n network.

Your download speeds will be good (as RX aggregation works) but as TX aggregation doesn't, you won't be getting full speed.

Now, as for the current work I'm doing. I've been porting over (and reimplementing in places) the 802.11n TX aggregation support, based on a combination of the atheros reference/carrier codebases and what's filtered through to the Linux ath9k driver.

In short - so far, so good. There's a lot to do, but basic TX aggregation is working in both station and hostap mode. My current test setup is:
  • A routerstation pro with an AR9160 NIC, running locally built firmware (FreeBSD) in hostap mode;
  • An EEEPC 701 retrofitted with an AR9280 NIC, running FreeBSD, as a station;
  • My macbook pro (broadcom 802.11n) as a station.
So far it's performing well - 200mbit TX/RX UDP in 5ghz HT/40 2x2 stream mode; and 130-140mbit TX/RX TCP.

I'm currently doing the work in a FreeBSD user svn branch. The details are in the Wiki - . I currently have exactly one tester (AR5416 STA, AR9280 hostap) who is reporting excellent success. I'd obviously like a few more. :)

There's still a lot of work to be done before this can be merged into HEAD. I'm sorry to say it won't happen before 9.0-RELEASE as there are some upcoming net80211 changes which will be rather intrusive and a bit risky to throw into the release at this late stage. It may also be difficult to backport as some changes will break kernel ABI.

But don't let anyone tell you FreeBSD doesn't support 802.11n. 9.0-RELEASE may not have much support, but all the key pieces are there. Once the release has been cut, I'll do some chasing up to get the Intel, Marvell and Ralink 802.11n support updated. (If someone would like to take charge of the broadcom NIC drivers, please drop freebsd-wireless@ a line.)

The next 6 months will be very interesting in the FreeBSD 802.11 world. :)

Finally, none of this current work would've been possible without the support of the sponsor paying me to get FreeBSD's net80211 and ath support updated - Hobnob, Inc. They contacted me a few months ago to help work out kinks in the Atheros NIC they're using in FreeBSD and seem impressed enough by my work to sponsor general net80211/ath improvements which they'll likely roll into future products. So when you're using Atheros 802.11n hardware in FreeBSD and getting that nice 140-150mbit TCP throughput, please give them thanks. :)

And finally finally, Qualcomm Atheros and the Linux ath9k developers have been instrumental in this work. The documentation, reference code and general interactive discussions have allowed me to get all of this work done in such a short period of time. (Yes, I did say "Atheros", "Documentation" and "Source Code" there, in the context of open source. Honest.)

Wireless update – DFS wrapup

Now that the DFS work has been completed, tested and paid for, I guess I can now write about it.

The DFS work was sponsored by KBC Networks, a wireless product company based in the USA. The eventual aim is to integrate this DFS work into their mesh product. I can't talk any further about what's going on there, except to say that they very graciously allowed the FreeBSD work to be released back to the community.

So FreeBSD now has:

  • Updated DFS master (hostap) and DFS slave (station) support in net80211;
  • Updated regulatory domain entries for the USS ("FCC3");
  • Changes to the ath(4) driver to fix corner cases in DFS master and DFS slave modes.
All of this work is now in FreeBSD-HEAD and will be available in FreeBSD-9.0.

Future work will hopefully include DFS IBSS and DFS mesh functionality. This work didn't include 802.11n DFS - but that's not too difficult to do (hint: if someone would like the mini-project, please contact me!)

Along with that was a port of the Atheros radar detection code (from Fusion, their earlier carrier codebase) which is currently proprietary. The Linux wireless developers have been working towards DFS compliance for a while, but haven't yet integrated working radar detection into any drivers. So this was a good exercise in uncharted territory to see how difficult it would be.

It turns out that it wasn't terribly difficult at all. After spending a bit of time chasing down missing bits in the FreeBSD atheros HAL driver, the third party doing said radar code porting managed to get FreeBSD doing radar detection at the same level as the commercial firmware.

During this particular exercise we discovered a few things about the software radar detection, which I think the community at large should know about.

The process of getting a device FCC/ETSI certified is just that - the whole device. This includes the NIC, the internal cabling, the enclosure, the antenna(s) and any external cabling. Maybe even whether it's indoors or outdoors, I wasn't involved in that. This likely has repercussions for any open source DFS implementation - although the DFS machinery in a specific FreeBSD release could be certified, the software based radar matching code may need tuning for the specific hardware used. So it's likely that Linux/FreeBSD can't just publish a single "ath_dfs" source module that will work for all Atheros NICs - or even a single NIC. It's likely that things will need tuning based on how a given NIC is used, the environment it's intending to be used in, and a variety of other things I haven't even thought about yet.

So if/when this does occur, the open source wireless community (FreeBSD, Linux, NetBSD, OpenBSD, DragonflyBSD, whatever..) will have to be responsible and not just simply run an "ath_dfs" radar module, assuming that it works. For complete, correct, certified behaviour, it's going to likely need the community to work with vendors and users in order to create "working, certified" combinations of hardware and software. So, if a vendor (call them U) wishes to release a new "SR-M" NIC, they could work with the open source wireless community to create a certified NIC+board+enclosure+antenna combination and publish a set of radar patterns and settings. They could leverage the proposed existing open source radar pattern matching code and simply add whatever configuration is needed. Users should then be responsible - they should only use the given radar dfs code with the correct setup. If they wish to run a modified setup, they should work with the vendor and upstream community to get things re-certified.

This way the DFS and radar pattern matching code is open source - what then changes is the radar patterns and configuration, which depends upon the NIC, cabling, antenna and environment.

For home users/developers, having a set of certified hardware combinations will make things easier for them.

For commercial vendors, if they adopt open-source FreeBSD or Linux on their platforms then (in an ideal world) they could get their equipment certified once (which as I note, they have to do anyway) and then publish the relevant radar patterns and configuration. The rest of the driver (including the radar pattern matching source and the general DFS support) would already be open source. Users can then use this when running their own kernel/distribution/OS on that platform.

I know this is all very fluffy and assumes that vendors/users act in a responsible, open manner. But I think this is what needs to occur if the open source community wishes to see stable, supported DFS regulatory compliance in their open source project.

To wrap this up, if you're a company wishing to leverage open source on their (Atheros) wireless platform and are interested in regulatory compliance, please drop me a line. I'll put you in contact with the relevant people inside Qualcomm Atheros to discuss how to best achieve this.

FreeBSD 9.0-BETA2 Available

The second BETA build for the FreeBSD-9.0 release cycle is now available. ISO images for the architectures amd64, i386, powerpc, powerpc64, and sparc64 are available on most of our FreeBSD mirror sites. One of the many new features in 9.0 we would like to be tested is the new installer, so we encourage our users to do fresh installation on test systems.

(Free)BSD inside Android

Today I was looking into the OpenSource licenses which are displayed for Android (2.3.4). There are several files which come with a BSD license.

During looking at it, I noticed that the libm has the copyright of several FreeBSD people. I did not had an in-deep look if this is because they took the FreeBSD libm, or if this is because parts of the FreeBSD libm where adopted by other BSD projects.

What I noticed is, that some special characters are not displayed correctly. For example the name Dag-Erling Smørgrav looks mangled in the display of the license inside the phone (I hope it is displayed better in my blog). His name is not the only problem case, there are also other characters which are not rendered as expected.

This does not really look professional.


Linuxulator explained (for developers): adding ioctls directly to the kernel

After giving an overview of the in-kernel basics of the Linuxulator, I want now to describe how to add support for new ioctls to the Linuxulator.

Where are the files to modify?

The platform independent code for the ioctls is in SRC/sys/compat/linux/linux_ioctl.c. The defines to have names for the ioctl values are in SRC/sys/compat/linux/linux_ioctl.h.

How to modify them?

First of all create a new header which will contain all the structures, named values and macros for those new ioctls. As written above, the ioctl values (e.g. #define LINUX_VIDIOC_ENCODER_CMD 0x564d /* 0xc028564d */) do not belong there, they shall be added to linux_ioctl.h. During the course of adding support for ioctls, you will need this new header. Add it in the SRC/sys/compat/linux/ directory, and prefix the name with a linux_. It would be good to decide on a common tag here (referenced as yourtag in the following), and stay with it. Use it wherever you need to have some specific name for the ioctl-set you want to add. In this case it would result in linux_yourtag.h (or even linux_ioctl_yourtag.h, depending if this is used for something very specific to the ioctls, or some generic linux feature) as the name of the header file. This was not done in the past, so do not expect that the names inside the linux_ioctl.c file will be consistent to this naming scheme, but it is never too late to correct mistakes of the past (at least in Open Source software development).

Now add this header to linux_ioctl.c (you need to include compat/linux/linux_yourtag.h). After that add the ioctl values to linux_ioctl.h. As can be seen above, the defines should be named the same as on linux, but with a LINUX_ prefix (make sure they where not defined before somewhere else). The ioctl values need to be the same hex values as in Linux, off course. Sort them according to their hex value. When you added all, you need to add two more defines. The LINUX_IOCTL_yourtag_MIN and LINUX_IOCTL_yourtag_MAX ones. The MIN-one needs to be an alias for the first (sorted according to the hex value) ioctl you added, and MAX needs to be an alias for the last (again, sorted according to the hex value) ioctl you added.

The next step is to let the Linuxulator know that it is able to handle the ioctls in the LINUX_IOCTL_yourtag_MIN to LINUX_IOCTL_yourtag_MAX range. Search the static linux_ioctl_function_t section of linux_ioctl.c and add such a variable for your ioctl set. The name of the variable should be something like linux_ioctl_yourtag.

Similar for the handler-definition for this. Search the static struct linux_ioctl_handler section and add a yourtag_handler. Set it to { linux_ioctl_yourtag, LINUX_IOCTL_yourtag_MIN, LINUX_IOCTL_yourtag_MAX }. To make this handler known to the Linuxulator, you need to add it to the DATA_SET section. Add DATA_SET(linux_ioctl_handler_set, yourtag_handler) there.

Now the meat, the function which handles the ioctls. You already defined it as linux_ioctl_function_t, but now you need to write it. The outline of it looks like this:

static int
linux_ioctl_yourtag(struct thread *td, struct linux_ioctl_args *args)
        struct file *fp;
        int error;
        switch (args->cmd & 0xffff) {
        case LINUX_an_easy_ioctl:
        case LINUX_a_not_so_easy_ioctl:
                /* your handling of the ioctl */
                fdrop(fp, td);
                return (error);
        /* some more handling of your ioctls */
       return (ENOIOCTL);
        error = ioctl(td, (struct ioctl_args *)args);
        return (error);

An easy ioctl in the switch above is an ioctl where you do not have to do something but can pass the ioctl through to FreeBSD itself. The not so easy ioctl case is an ioctl where you need to do e.g. a fget(td, args->fd, &fp). This is just an example, there are also other possibilities where you need to do additional stuff before the return, or where you do not pass the ioctl to FreeBSD. A typical example of what needs to be done here is to copy values from linux structures to FreeBSD structures (and the other way too), or to translate between 64bit and 32bit. Linux programs on amd64 are 32bit executables and 32bit structures/pointers. To make this work on amd64, you need to find a way to map between the two. There are examples in the kernel where this is already the case. The more prominent examples in the 64bit<->32bit regard are the v4l and v4l2 ioctls.

The tedious part is to research if a translation has to be done and if yes what needs to be translated how. When this is done, most of the work is not so hard. The linux_yourtag.h should contain the structures you need for this translation work.

It is also possible to add ioctls in a kernel module, but this is not subject to this description (I will update this posting with a link to it when I get time to write about it).


(Free)BSD inside Android

Today I was looking into the OpenSource licenses which are displayed for Android (2.3.4). There are several files which come with a BSD license.

During looking at it, I noticed that the libm has the copyright of several FreeBSD people. I did not had an in-deep look if this is because they took the FreeBSD libm, or if this is because parts of the FreeBSD libm where adopted by other BSD projects.

What I noticed is, that some special characters are not displayed correctly. For example the name Dag-Erling Smørgrav looks mangled in the display of the license inside the phone (I hope it is displayed better in my blog). His name is not the only problem case, there are also other characters which are not rendered as expected.

This does not really look professional.


HOWTO add linux-infrastructure ports for a new linux_base port

In my last blog-post I described how to create a new linux_base port. This blog-post is about the other Linux–ports which make up the Linux–infrastructure in the FreeBSD Ports Collection for a given Linux-release.

What are linux-infrastructure ports?

A linux_base port contains as much as possible and at the same time as little as possible to make up a useful Linux-compatibility-experience in FreeBSD. I know, this is not a descriptive explanation. And it is not on purpose. There are no fixed rules what has to be inside or what not. It “matured