Planet FreeBSD

I just commited the translation of Jails chapter. The translation should appear in few hours here.

About a week ago i came across this post:

http://lists.freebsd.org/pipermail/freebsd-mobile/2007-April/009666.html

This evning i decided to give it a try.

For a starter, my laptop is a Lenovo 3000 N100 (<LENOVO TP-61> according to acpi).

#pciconf -lv

[SNIP]

none2@pci5:6:1: class=0×080500 card=0×207717aa chip=0×08221180 rev=0×19 hdr=0×00
vendor     = ‘Ricoh Company, Ltd.’
device     = ‘SD Bus Host Adapter’
class      = base peripheral
none3@pci5:6:2: class=0×088000 card=0×207817aa chip=0×08431180 rev=0×01 hdr=0×00
vendor     = ‘Ricoh Company, Ltd.’
device     = ‘unknown Ricoh MMC Host Controller’
class      = base peripheral
none4@pci5:6:3: class=0×088000 card=0×207917aa chip=0×05921180 rev=0×0a hdr=0×00
vendor     = ‘Ricoh Company, Ltd.’
device     = ‘unknown Ricoh Memory Stick Host Controller’
class      = base peripheral
none5@pci5:6:4: class=0×088000 card=0×207a17aa chip=0×08521180 rev=0×05 hdr=0×00
vendor     = ‘Ricoh Company, Ltd.’
device     = ‘unknown Ricoh xD-Picture Card Host Controller’
class      = base peripheral
This are some of the hard facts.

Getting src/sys patched up was somewhat easy. There is one small problem though.

The function ‘ bus_setup_intr()’ has lately changed the number of arguments.

after looking at some other drivers i decided to try and just add NULL,.

It compiled then, even loaded as a module.

sdh0: <SD Host Controller> mem 0xb0300400-0xb03004ff irq 23 at device 6.1 on pci5

sdh0: Found 1 slots

sdh0: [ITHREAD]

Everything looks fine.. atleast till i decide to hotplug a sd card .. whoops reboot no panic.

Debugging will continue later..

Dru posted this on IRC today:

http://blogs.ittoolbox.com/unix/bsd/archives/unix-humour-15908

I specially like the ‘make love’ one.

Béranger, the author of the long rant on which I have commented twice before, seems deeply hurt by my comments. Deeply enough, at least, to spend most of his after game report lambasting me, and to post a complaint on freebsd-advocacy.

Read it if you like. He deliberately misunderstands me, twists my words (including some from private conversation), pounces on strawmen, and still can't understand that the FreeBSD Foundation is a different entity from the FreeBSD Project, because apparently if the Foundation licenses and distributes software that runs on FreeBSD but isn't included in FreeBSD, then the Foundation is FreeBSD.

And he still can't get my name right.

I won't bother rebutting.

One further comment on The sorry state of open source today, which I did not want to include in my previous entry as I felt it would distract from my main point, which was the inaccuracies in the author's discussion of FreeBSD.

On page 19, Béranger discusses problems with the disk drivers in Linux 2.6.20. These problems are real (though hopefully transient), and I have myself been bitten by them, as on one machine, Ubuntu's linux-image-2.6.20-14-386 would not recognize the disks at all; I could boot an older kernel, but then of course nvidia-glx, which had been updated to match the newer non-working kernel, would not load.

Where Béranger stumbles is where he asserts—or implies—that there are fundamental differences between PATA, SATA and SCSI, and that it therefore does not make sense to use similar names (/dev/sdX) for them all.

The reality is far more fluid. Not only is the line between these technologies shifting, it is also gradually disappearing. With the arrival of Serial Attached SCSI, for instance, SATA and SCSI drives share the same cables, connectors and electrical specs, and can be attached to the same controller (although the protocol remains different). Going further back, ATAPI (the standard for attaching removable-media drives to the ATA bus) uses SCSI commands for everything except actual data transfer.

The questions Béranger should be asking are not why PATA, SATA and SCSI hard disks need to share a name, but rather:

• why they haven't done so since the start,
• why the common name should refer explicitly to only one of these standards (sd is short for "SCSI disk"), and
• why removable-media and fixed-media drives shared a name (/dev/hdX) for so long, especially when that name refers explicitly to fixed-media disks (hd being short for "hard disk").

Compare with FreeBSD, which also uses different names for ATA and SCSI devices, but at least differentiates between the various types of media: ad ("ATA disk") and da ("direct access") for PATA/SATA and SCSI fixed-media disks respectively, acd and cd for CD and DVD drives, ast and st for tape streamers. There is work under way to bring the ATA driver into the CAM framework (which was introduced in 3.0 to clean up SCSI device handling), erasing these differences.

Jem Matzan's The Jem Report is running a so-called editorial by Radu-Cristian Fotescu (aka. Béranger) titled The sorry state of open source today. I say so-called, because it is more of a rant than an editorial: 26 pages long and not entirely coherent.

I won't waste your time with a point-by-point rebuttal of this piece, not least because most of what he writes is pure opinion and interpretation. I don't necessarily agree with it—I find him a little too radical and a little too confrontational—but he's entitled to it.

(I do agree with his views on the differences between the GPL and the BSD license, but that's neither here nor there)

What I take exception to are factual errors in his discussion of *BSD, and specifically of FreeBSD.

First, on page 7, he writes:

[members of] the *BSD family [are] either backed by 501(c)(3) non-profit organizations like The FreeBSD Foundation or the NetBSD Project, or the task of individuals like Theo de Raadt for OpenBSD and Matt Dillon for DragonFly BSD

This is inaccurate as far as FreeBSD goes. The FreeBSD Foundation was established because a recognized legal entity (which the FreeBSD Project itself is not) was required for three purposes:

• to hold the FreeBSD trademarks,
• to obtain 501(c)(3) status so financial contributions are tax-deductible,
• to enter license agreements and contracts, e.g. for the certification and distribution of the Java JDK and JRE.

The Foundation contributes to the Project in the sense that it owns some of the hardware that the Project uses, organizes and funds events such as the Developer Summits, which are held twice a year at BSDCan and EuroBSDCon, and occasionally funds development work.

This does not mean that the FreeBSD Project is backed by the FreeBSD Foundation. The Foundation does not control the Project in any way; it does not control the Project's servers and repositories, it does not approve new committers, write roadmaps or schedule releases. The democratically elected Core Team does that.

This also invalidates the following quote from page 3:

even the FreeBSD foundation [sic] is an American subject

The implication is that since the FreeBSD Foundation is subject to American law, the FreeBSD Project is subject to American software patents.

Once again, the Foundation does not control FreeBSD. Moreover, the bulk of FreeBSD development today—at least the cutting-edge work—happens in Europe (especially Eastern Europe, but there are also strong contingents in Denmark, Italy and the UK) and Asia.

The NetBSD situation is different, as The NetBSD Foundation, Inc. appoints the NetBSD Core Group and various executive committees, and handles applications from new developers, who must sign a membership agreement with the Foundation before obtaining write access to the repositories.

The next objection I have relates to the following quotes from pages 10 and 22:

Maybe the first sign was when FreeBSD tried to mimic Linux and to be "more user-friendly". It was rather wrong, and I can only hope they realized the mistake. The gradual recovery of the once legendary FreeBSD stability can be seen with 6.1 and 6.2. Hopefully the trend will not be reversed again.

Unfortunately, starting with 5.0, the quality and stability of FreeBSD have deteriorated, or maybe it was my hardware that was less supported. Compromises have been made to match the popularity of Linux, and the price to pay was rather high.

These two paragraphs are based on a completely incorrect understanding of what happened between FreeBSD 4 and FreeBSD 6. User friendliness, popularity, Linux envy: none of this ever played any part. I will attempt to briefly explain what actually happened.

FreeBSD first gained support for symmetric multiprocessing in 3.0. The approach chosen then was a simplistic one: the entire kernel was serialized under a so-called "giant lock". This saved a lot of work, as there was no need to make the various kernel subsystems reentrant. It also meant that FreeBSD's SMP performance was not very good, except for computationally intensive tasks (i.e. tasks which did not involve the kernel much), but it was better than nothing.

After FreeBSD 4 was released, work was started on a project called SMPng, which would replace the giant lock with a number of fine-grained locks, allowing separate kernel threads to run simultaneously on separate processors, performing separate tasks.

This is where two important mistakes were made.

Firstly, the release schedule was feature-bound rather then time-bound. This meant that FreeBSD 5 would be released "when it was done", not "when it was time". Consequently, there was no deadline for developers to focus on, and little effort was made to partition work into manageable pieces to avoid having the source tree in an unreleasable state for long periods of time.

Secondly, and most importantly, it was decided very early on to go for a complicated M:N scheduling model, based on a variant of scheduler activations called KSE. With KSE, a userland process with N threads (1 <= N) would receive M kernel threads (1 <= M <= N) on which to run. The userland threads library would be responsible for scheduling application threads on top of the kernel threads. Within the kernel, threads would be scheduled using a two-tier system involving KSE groups (groups of threads belonging to the same process) and individual threads.

Scheduler activations had academic backing and the M:N model had proved workable in Solaris, and NetBSD was moving in the same direction, but this decision was made before there was even proof-of-concept code to demonstrate its viability in FreeBSD.

(Ironically, around that time, Sun decided to abandon the M:N model in Solaris, and NetBSD is now moving to 1:1)

To cut a long story short, KSE was a disaster. It took years to implement, delaying other work which depended on it, and it never worked properly. FreeBSD 5 was going nowhere fast until a developer who until then had not been involved in either SMPng or KSE lost patience and implemented a 1:1 thread library on top of KSE. In the end, KSE was never even fully implemented, and it is now being slowly replaced with a pure 1:1 model.

So there you have it. FreeBSD 5 was the result of a failed experiment in M:N threading. FreeBSD 6 was an attempt to correct those mistakes, and FreeBSD 7 will dispose of KSE entirely and complete the fine-grained locking work that started with FreeBSD 5.

Finally, the FreeBSD Project learned an expensive lesson with FreeBSD 5. We now have a separate repository where experimental code can mature and high-risk changes can be tried out before entering CVS. We have also switched our release model to time-based releases rather than feature-based releases. If, as a release date approaches, we realize that a particular feature won't be ready in time, we simply leave that feature out of the release rather than change our schedule.

Again, the fun has begun. We have a lot of interesting projects. Mine is accepted again. Here’s what I’m going to work on with the mentorship of Andrew Pantyukhin(sat@):

http://www.kovesdan.org/soc/article.html

GNOME 2.18.1 has been http://mail.gnome.org/archives/gnome-announce-list/2007-April/msg00008.html and ports and ../gnome/docs/faq2.html#q21 are available for everyone's faorite operating system. This release is a polishing of 2.18.0, so expect a more stable, nicer looking desktop experience. On top of that, some of our users have also submitted ../gnome/screenshots.html !

Well, a lot of interesting things happened in the recent days, thus it’s time to write a bit of summary.

1. The ZFS filesystem support was committed to HEAD by Pawel Jakub Dawidek (pjd@). Pawel has done a great work on porting this great functionality from OpenSolaris, and yesterday the code hit the tree. It lacks from ACL and extended attributes support and can’t be booted off. There’s no iSCSI target daemon support yet. it is only for i386 and can be used only as a module. These are the current limitations, but apart from those it is reported to work well. As Pawel said, support for amd64 is going very soon and other architectures will be supported later due to the missing atomic operations.
2. There’s a nice utility in the graphviz packag, called dot. It can be used to render nice graphs from text files. Some time ago we already talked about making some graphs to display the mentor-mentee relationships between the project committers, and recently Florent Thoumie added a sample dotfiles to src/share/misc. The files are still being completed, I hope we can add them to our website later.
3. The Google Summer of Code 2007 has begun. Applications are no longer accepted, thus applicants had to submit their proposals by now. I submitted an application about the Ports Collection infrastructure again. I’m looking forward to the results, which we will get to know by 5:00 PM at April 11 Pacific time.

Got a new patches to the ports infrastructure committed, after they’ve been tested on Pointyhat over the past week.

• USE_GHOSTSCRIPT default dependency was flipped from ghostscript-gnu to ghostscript-gpl. According to the vendor website, GPL is the license of choice for the Ghostscript project now.
• bsd.tcl.mk got a total overhaul by Martin MatuÅ¡ka, a promising new guy from neighbouring country Slovakia, who, if I can spill the secret, recently got approved for his own commit bit. The most visible change is that the USE_TCL* and USE_TK* macros are now modelled after the USE_PERL5* ones.
• There’s been some optimizations by Mikhail Teterin, but the patch is so esotheric I have no idea what he did changed. The previous revision of the patch was in the previous exp-run, but only the current one tested without any failures.
• OPTIONS got two important fixes. First, courtesy of Rong-En Fan from Taiwan, the default variables for not-yet set options are not correctly created all the time. This should allow people to test just any WITH_FOO / WITHOUT_FOO combinations they please to. Second change, done by me, makes the blue dialog pop up every time the new version of port adds a new variable that wasn’t around when user saved his options last time.

And that’s about it. There’s ongoing work on the python25 patch, so that will get another exp-run soon.