Monthly Archives: March 2011

Nano jails

A la demande populaire voici un micro howto sur comment créer des nanos jails(8).

Pour cela je vais vous monter comment tourne mon blog (à base de cblog).

Quelques petites commandes pour bien montrer de quoi on parle

$ jls -j cblog
JID  IP Address      Hostname                      Path
1  10.50.0.1       cblog                         /zdata/jails/cblog/
$ du -hs /zdata/jails/cblog/
2,3M    /zdata/jails/cblog

c'est tout rien de plus.

Il y a même des choses en trop par exemple cette jail n'a pas besoin de réseau pour fonctionner normalement.

En fait une jail n'a pas besoin de grand chose pour fonctionner et depuis que l'on dispose des jails persistantes elles ont même besoin de rien du tout.

Examinons un peu le contenu de cette jail :

/zdata/jails/cblog
/zdata/jails/cblog/bin
/zdata/jails/cblog/bin/cblog.fcgi
/zdata/jails/cblog/data
/zdata/jails/cblog/data/*
/zdata/jails/cblog/usr/local/etc
/zdata/jails/cblog/usr/local/etc/cblog.conf
/zdata/jails/cblog/libexec
/zdata/jails/cblog/libexec/ld-elf.so.1
/zdata/jails/cblog/lib
/zdata/jails/cblog/lib/libfcgi.so.0
/zdata/jails/cblog/lib/libz.so.5
/zdata/jails/cblog/lib/libc.so.7

Nous y retrouvons donc le strict nécessaire pour faire tourner cblog.

pour en arriver là il faut préparer le fs. (je suis utilisateur de zfs :))

$ zfs create /zdata/jails/cblog
$ mkdir /zdata/jails/cblog/bin
$ mkdir /zdata/jails/cblog/lib
$ mkdir /zdata/jails/cblog/libexec

Je dépose ensuite le binaire cblog.fcgi (je l'ai pris dans le paquet officiel)

$ cp /tmp/cblog-0.1.15/libexec/cblog.fcgi /zdata/jails/cblog/bin

Je regarde ce que demande ce binaire:

$ ldd /zdata/jails/cblog/bin/cblog.fcgi
/zdata/jails/cblog/bin/cblog.fcgi:
libfcgi.so.0 => /usr/local/lib/libfcgi.so.0 (0x80066b000)
libz.so.5 => /lib/libz.so.5 (0x800775000)
libc.so.7 => /lib/libc.so.7 (0x80088a000)

Je rajoute donc ce qui va bien dans ma future jail:

$ cp /lib/libz.so.5 /zdata/jails/cblog/lib
$ cp /lib/libc.so.7 /zdata/jails/cblog/lib
$ cp /usr/local/lib/libfcgi.so.0 /zdata/jails/cblog/lib

J'ai vérifié les dépendances de la libfcgi et elle ne ramène rien d'autre.

Le paquet cblog est compilé avec comme prefix /usr/local du coup il cherche son fichier de conf dans /usr/local/etc/cblog.conf (vu de la jail)

$ mkdir /zdata/jails/cblog/usr/local/etc

Je prévoie de mettre les données que le binaire va manipuler dans /data (vu de la jail)

$ mkdir /zdata/jails/cblog/data

Je prépare le fichier de conf de mon cblog pour lui dire de trouver ses données dans /data (je zappe cette partie ce n'est pas le but de ce post). Le fichier de conf est déposé dans /zdata/jails/cblog/usr/local/etc sous le nom cblog.conf comme attendu.

C'est là que ça devient intéressant. Il est impossible de faire tourner une jail sans son runtime link-editor.

$ cp /libexec/lib-elf.so.1 /zdata/jails/cblog/libexec/ld-elf.so.1

Ici j'ai pris celui du système hôte car les binaire sont ceux du même format que ceux de l'hôte. Dans le cas d'un jail linux par exemple, il faudra prendre l'équivalent au format linux.

Nous pouvons maintenant démarrer la jail en question.

Ne surtout pas passer par /etc/rc.d/jail qui est complètement dépassé maintenant.

$ jail -c persist name=cblog path=/zdata/jails/cblog/ host.hostname=cblog ip4.addr=10.50.0.1

Si vous ne voulez pas d'adresse ip ni de hostname

$ jail -c persist name=cblog path=/zdata/jails/cblog/

jls vous montrera la jail qui tourne.

Démarrons le cblog pour qu'il écoute sur une socket unix.

$ jexec cblog /sbin/cblog.fcgi unix:/cblog.sock

Il suffit de donner le chemin /zdata/jails/cblog/cblog.sock à nginx par exemple dans sa config fastcgi pour que votre cblog soit accessible au plus grand nombre.

Pour stopper la jail:

$ jail -r cblog

New DTrace probes for the linuxulator

I forward ported my DTrace probes for the FreeBSD linuxulator from a 2008-current to a recent –current. I have not the complete FreeBSD linuxulator covered, but a big part is already done. I can check the major locks in the linuxulator, trace futexes, and I have a D-script which yells at a lot of errors which could happen but should not.

Some of my D-scripts need some changes, as real-world testing showed that they are not really working as expected. They can get overwhelmed by the amount of speculation and dynamic variables (error message: dynamic variable drops with non-empty dirty list). For the dynamic variables problem I found a discussion on the net with some suggestions. For the speculation part I expect similar tuning-possibilities.

Unfortunately the D-script which checks the internal locks fails to compile. Seems there is a little misunderstanding on my side how the D-language is supposed to work.

I try to get some time later to have a look at those problems.

During my development I stumbled over some generic DTrace problems with the SDT provider I use for my probes:

  • If you load the Linux module after the SDT module, your system will panic as soon as you want to access some probes, e.g. “dtrace –l

11n TX and RX experiments..

My experiments so far are as follows (all in 5GHz mode - 2GHz is just too busy in my apartment complex):
  • FreeBSD <-> FreeBSD 11n is stable on the AR9160 and AR9220. TX aggregation is disabled, so it reaches around 35-40mbit in HT/20 and 45mbit in HT/40 mode. I'm getting MCS13 between an AP and STA which are in different rooms in my apartment. MCS15 is achievable if the units are close to each other.
  • Short-GI in 40MHz works fine.
  • When RX aggregation is enabled, and IEEE80211_AMPDU_AGE is enabled with net.wlan.ampdu_age set to 5 (it defaults to 500), I get ~ 55mbit RX in HT/20 and ~ 75mbit RX in HT/40 mode. There's problems with out of order packets that are appearing -before- the calculated block-ack window and net80211 is dropping those. I'll investigate why at some point.
  • FreeBSD works fine as an 11n AP, obviously with A-MPDU TX disabled.
There's lots of stuff that needs to be tested at this point, including:
  • 2GHz operation, obviously!
  • 11n and legacy co-operation.
  • Other chipsets - AR5416, AR9280, AR9285.
  • Behaviour in the presence of noise, busy channel, etc.
  • Make sure that HT/40 mode is -correctly- protecting frames on both channels and is correctly waiting to make sure both channels are free before transmitting!
And stuff that needs to be implemented before we enable it by default:
  • A-MPDU TX aggregation, for STA, AP, Mesh and adhoc modes - this one is really needed and is going to take some time to do.
  • HT RTS frame protection - I have code for this which I'll commit soon.

11n TX and RX experiments..

My experiments so far are as follows (all in 5GHz mode - 2GHz is just too busy in my apartment complex):
  • FreeBSD <-> FreeBSD 11n is stable on the AR9160 and AR9220. TX aggregation is disabled, so it reaches around 35-40mbit in HT/20 and 45mbit in HT/40 mode. I'm getting MCS13 between an AP and STA which are in different rooms in my apartment. MCS15 is achievable if the units are close to each other.
  • Short-GI in 40MHz works fine.
  • When RX aggregation is enabled, and IEEE80211_AMPDU_AGE is enabled with net.wlan.ampdu_age set to 5 (it defaults to 500), I get ~ 55mbit RX in HT/20 and ~ 75mbit RX in HT/40 mode. There's problems with out of order packets that are appearing -before- the calculated block-ack window and net80211 is dropping those. I'll investigate why at some point.
  • FreeBSD works fine as an 11n AP, obviously with A-MPDU TX disabled.
There's lots of stuff that needs to be tested at this point, including:
  • 2GHz operation, obviously!
  • 11n and legacy co-operation.
  • Other chipsets - AR5416, AR9280, AR9285.
  • Behaviour in the presence of noise, busy channel, etc.
  • Make sure that HT/40 mode is -correctly- protecting frames on both channels and is correctly waiting to make sure both channels are free before transmitting!
And stuff that needs to be implemented before we enable it by default:
  • A-MPDU TX aggregation, for STA, AP, Mesh and adhoc modes - this one is really needed and is going to take some time to do.
  • HT RTS frame protection - I have code for this which I'll commit soon.

FreeBSD Project to participate in Google Summer of Code 2011

The FreeBSD Project is pleased to announce its participation In Google's 2011 Summer of Code program, which funds summer students to participate in open source projects. This will be the FreeBSD Project's seventh year in the program, having mentored over 100 successful students through summer-long coding projects between 2005 and 2010.

kde@ updating… well, everything

What a big set of updates the one committed today by kde@! While a bit late (but problems should now be solved: thank you PC-BSD!), we are pleased to announce: Qt 4.7.2; PyQt4 4.8.3; KDE SC 4.6.1, featuring an improved ksysguardd on its own (sysutils/ksysguardd, with no dependencies) that can be used for remote monitoring [...]

Grazer Linuxtag 2011

Grazer Linuxtag 2011 (http://www.linuxtage.at/), FH Joanneum Graz, Graz, Austria 09 April, 2011. The Grazer Linuxtag is a one day event on Linux and free software in general. Besides a FreeBSD booth and the possibility to take the BSDA certification exam there will also be a BSD Bootcamp with live workshops covering different FreeBSD topics. More information can be found here.

Converting from Courier IMAP to dovecot is easy

I have used Courier IMAP at home since a long time. As I want to update a dovecot 1.2 setup to dovecot 2.x, I decided to first have a look at dovecot 2.x at home.

Switching from Courier IMAP to dovecot is really easy. I just configured the correct path to the maildir, setup a passdb/userdb, and it was working.

The important part was the correct transfer of the passwords. I used already an userdb in Courier IMAP with MD5 passwords. For each user it has imappw=XXX with XXX similar to $1$abc.

This can be converted into a dovecot passdb/userdb line very easily:

username:{MD5-CRYPT}$1$abc::UID:GID::HOMEDIR::userdb_mail=maildir:~/path/to/maildir

The corresponding passdb/userdb settings for dovecot are:

passdb {
   args = scheme=MD5-CRYPT username_format=%u /usr/local/etc/dovecot/dovecot.pws
   driver = passwd-file
}
userdb {
   args = username_format=%u /usr/local/etc/dovecot/dovecot.pws
   driver = passwd-file
}

Compared to when I had a look the last time, dovecot is also able to use OTP as an authentication mechanism now. Unfortunately I did not find any documentation how to configure/use it.

Share

Foundation at ILF and FlourishConf

I'll be representing the Foundation at the BSD booth at Indiana LinuxFest this upcoming weekend and at FlourishConf on the following weekend. If you're near Indianapolis or Chicago, drop by and say hi as registration is free for both events. Dropping by the booth provides an opportunity to discuss what the Foundation has been up to, check out the Foundation swag, and/or make a donation to the Foundation.

Foundation at ILF and FlourishConf

I'll be representing the Foundation at the BSD booth at Indiana LinuxFest this upcoming weekend and at FlourishConf on the following weekend. If you're near Indianapolis or Chicago, drop by and say hi as registration is free for both events. Dropping by the booth provides an opportunity to discuss what the Foundation has been up to, check out the Foundation swag, and/or make a donation to the Foundation.

(More) wireless hackery

So instead of doing what I should be doing, I did some more wifi hackery.
The short summary version (all 802.11g 2.4ghz testing):
  • AR5416 TX is still unhappy and unpredictable. It's also like that with ath9k. AR5416 RX is perfectly fine.
  • AR9160 TX/RX is fine.
  • AR9220 (Ubiquiti SR71-12 and Ubiquiti-SR71-15) TX/RX is now fine.
  • AR9280 closed and open-loop TX is fine, RX is also fine.
  • AR9285 TX/RX is fine and stable now.
I'll now go and test out 5GHz AP/STA modes with the above (where appropriate) and see how they behave. I hope they'll be just as stable now.
Bernard has been making inroads into fleshing out the missing parts of the net80211 802.11n support. Many thanks to him for taking the time to dissect the 802.11n spec, the Linux 802.11 stack and spend time with some APs determining what they're doing.

(More) wireless hackery

So instead of doing what I should be doing, I did some more wifi hackery.

The short summary version (all 802.11g 2.4ghz testing):
  • AR5416 TX is still unhappy and unpredictable. It's also like that with ath9k. AR5416 RX is perfectly fine.
  • AR9160 TX/RX is fine.
  • AR9220 (Ubiquiti SR71-12 and Ubiquiti-SR71-15) TX/RX is now fine.
  • AR9280 closed and open-loop TX is fine, RX is also fine.
  • AR9285 TX/RX is fine and stable now.
I'll now go and test out 5GHz AP/STA modes with the above (where appropriate) and see how they behave. I hope they'll be just as stable now.

Bernard has been making inroads into fleshing out the missing parts of the net80211 802.11n support. Many thanks to him for taking the time to dissect the 802.11n spec, the Linux 802.11 stack and spend time with some APs determining what they're doing.

Accepting Travel Grant Applications for BSDCan 2011

Calling all FreeBSD developers needing assistance with travel expenses to BSDCan 2011.

The FreeBSD Foundation will be providing a limited number of travel grants to individuals requesting assistance. Please fill out and submit the Travel Grant Request Application by April 15, 2011 to apply for this grant.

This program is open to FreeBSD developers of all sorts (kernel hackers, documentation authors, bugbusters, system administrators, etc). In some cases we are also able to fund non-developers, such as active community members and FreeBSD advocates.

You request funding based on a realistic and economical estimate of travel costs (economy airfare, trainfare, ...), accommodations (conference hotel and sharing a room), and registration or tutorial fees. If there are other sponsors willing to cover costs, such as your employer or the conference, we prefer you talk to them first, as our budget is limited. We are happy to split costs with you or another sponsor, such as just covering airfare or board. If you are a speaker at the conference, we expect the conference to cover your travel costs, and will most likely not approve your direct request to us.

We review your application and if approved, authorize you to seek reimbursement up to a limit. We consider several factors, including our overall and per-event budgets, and (quite importantly) the benefit to the community by funding your travel. Most rejected applications are rejected because of an over-all limit on travel budget for the event or year, due to unrealistic or uneconomical costing, or because there is an unclear or unconvincing argument thatfunding the applicant will directly benefit the FreeBSD Project. Please take these points into consideration when writing your application.

We reimburse costs based on actuals (receipts), and by check or bank transfer. And, we do not cover your costs if you end up having to cancel your trip. We require you to submit a report on your trip, which we may show to current or potential sponsors, and may include in our semi-annual newsletter and on this blog.

There's some flexibility in the mechanism, so talk to us if something about the model doesn't quite work for you or if you have any questions. The travel grant program is one of the most effective ways we can spend money to help support the FreeBSD Project, as it helps developers get together in the same place at the same time, and helps advertise and advocate FreeBSD in the larger community.

Accepting Travel Grant Applications for BSDCan 2011

Calling all FreeBSD developers needing assistance with travel expenses to BSDCan 2011.

The FreeBSD Foundation will be providing a limited number of travel grants to individuals requesting assistance. Please fill out and submit the Travel Grant Request Application by April 15, 2011 to apply for this grant.

This program is open to FreeBSD developers of all sorts (kernel hackers, documentation authors, bugbusters, system administrators, etc). In some cases we are also able to fund non-developers, such as active community members and FreeBSD advocates.

You request funding based on a realistic and economical estimate of travel costs (economy airfare, trainfare, ...), accommodations (conference hotel and sharing a room), and registration or tutorial fees. If there are other sponsors willing to cover costs, such as your employer or the conference, we prefer you talk to them first, as our budget is limited. We are happy to split costs with you or another sponsor, such as just covering airfare or board. If you are a speaker at the conference, we expect the conference to cover your travel costs, and will most likely not approve your direct request to us.

We review your application and if approved, authorize you to seek reimbursement up to a limit. We consider several factors, including our overall and per-event budgets, and (quite importantly) the benefit to the community by funding your travel. Most rejected applications are rejected because of an over-all limit on travel budget for the event or year, due to unrealistic or uneconomical costing, or because there is an unclear or unconvincing argument thatfunding the applicant will directly benefit the FreeBSD Project. Please take these points into consideration when writing your application.

We reimburse costs based on actuals (receipts), and by check or bank transfer. And, we do not cover your costs if you end up having to cancel your trip. We require you to submit a report on your trip, which we may show to current or potential sponsors, and may include in our semi-annual newsletter and on this blog.

There's some flexibility in the mechanism, so talk to us if something about the model doesn't quite work for you or if you have any questions. The travel grant program is one of the most effective ways we can spend money to help support the FreeBSD Project, as it helps developers get together in the same place at the same time, and helps advertise and advocate FreeBSD in the larger community.