Monthly Archives: January 2011

[CFT] KDE SC 4.6.0 for FreeBSD.

Hello Internet,

The FreeBSD KDE Team is happy to let you know that KDE SC 4.6.0 has been
released a few Days ago, and the Release is ready for a public test. Before
you ask, no, we do not want to put KDE 4.6.0 in the ports tree before
FreeBSD 8.2/7.4 is released.

What’s new:
KDE SC 4.6.0 provides major updates to the KDE Plasma workspaces, KDE
Applications and KDE Platform. Theses releases, version 4.6, provide many
new features in each of KDE’s three product lines. The official
release notes for these releases can be found at
http://kde.org/announcements/4.6/

Now you can get KDE SC 4.6 via svn checkout:

svn co http://area51.pcbsd.org/trunk/area51/PYQT/
svn co http://area51.pcbsd.org/trunk/area51/PORTS
svn co http://area51.pcbsd.org/trunk/area51/KDE
svn co http://area51.pcbsd.org/trunk/area51/Tools/

Then try:

sh Tools/scripts/portsmerge
sh Tools/scripts/kdemerge

Happy updating!!

REST service on top of Marcuscom Tinderbox

Like many people who work on FreeBSD Ports I've been using great Marcuscom Tinderbox to verify my changes. Some time ago I started to realise that it's quite a boring task to schedule a lot of builds using web interface again and again and finally I've managed to make myself start making the process a little bit easier.

It seems that building a REST service on top on the existing is a nice way to go, so that's what I've been doing last few days and finally I have a prototype now.

As I didn't want to patch original tinderbox's webui sources, I just created a sub-directory in it called 'api' and started there. To my surprise, I wasn't able to find any ready to use solution to implement REST service in PHP (maybe I've missed something though), so ended up with a few php scripts and rewrite rules to make URLs feel RESTy. It was a quite hard part of the task since I haven't been coding in PHP for years and don't feel comfortable with it even after few days of practice.

Simultaneously I've started coding a client for which I choose Python.

Workflow

The goal was to support minimum but most important (for me at least) scenario -- testing an update for a port. Let's assume the port being tested is security/gnutls-devel. The process looks like that:

1. Produce a patch and apply it to the tree (can be done via ssh, or nfs, whatever, and could be easily scripted up)
2. Check builds we have:


(21:33) novel@fsol:~ %> tbc build
id name status current port updated
1 8.x-FreeBSD IDLE None 2011-01-30 14:05:25
(21:40) novel@fsol:~ %>


So, we see an idle build here with id 1. Not so much choices, so let's use it, but first let's see what's in the queue:


(21:40) novel@fsol:~ %> tbc queue
id username port build pri status enqueued completed
10 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 13:54:43 2011-01-30 14:05:27
(21:45) novel@fsol:~ %>


Now adding a port to the queue:


(21:46) novel@fsol:~ %> tbc queue add -b 1 security/gnutls-devel
(21:46) novel@fsol:~ %>


Let's check if new port build added successfully:


(21:46) novel@fsol:~ %> tbc queue
id username port build pri status enqueued completed
10 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 13:54:43 2011-01-30 14:05:27
11 novel security/gnutls-devel 8.x-FreeBSD 10 ENQUEUED 2011-01-30 21:46:20 None
(21:46) novel@fsol:~ %>


Cool, it's here! Now we can see that our build turned into PREPARE state:


(21:48) novel@fsol:~ %> tbc build
id name status current port updated
1 8.x-FreeBSD PREPARE None 2011-01-30 21:48:15
(21:48) novel@fsol:~ %>


All we have to do now is wait and poll for changes. Now we can see it's building:


(21:59) novel@fsol:~ %> tbc build
id name status current port updated
1 8.x-FreeBSD PORTBUILD gnutls-devel-2.11.5 2011-01-30 22:00:25
(22:00) novel@fsol:~ %>


And finally we see it's done:

(22:07) novel@fsol:~ %> tbc queue
id username port build pri status enqueued completed
11 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 21:46:20 2011-01-30 21:57:18
12 novel security/gnutls-devel 8.x-FreeBSD 10 SUCCESS 2011-01-30 21:50:54 2011-01-30 22:09:22
(22:15) novel@fsol:~ %> tbc queue 11


For everyone who's interested I've uploaded sources on github:



NOTICE: it's an early work in progress! The code is unstable and lacks a lot of features. Anyway, any feedback about an idea and implementation is welcome.

REST service on top of Marcuscom Tinderbox

Like many people who work on FreeBSD Ports I've been using great Marcuscom Tinderbox to verify my changes. Some time ago I started to realise that it's quite a boring task to schedule a lot of builds using web interface again and again and finally I've managed to make myself start making the process a little bit easier. It seems that building a REST service on top on the existing is a nice way to go, so that's what I've been doing last few days and finally I have a prototype now.As I didn't want to patch original tinderbox's webui sources, I just created a sub-directory in it called 'api' and started there. To my surprise, I wasn't able to find any ready to use solution to implement REST service in PHP (maybe I've missed something though), so ended up with a few php scripts and rewrite rules to make URLs feel RESTy. It was a quite hard part of the task since I haven't been coding in PHP for years and don't feel comfortable with it even after few days of practice.Simultaneously I've started coding a client for which I choose Python.WorkflowThe goal was to support minimum but most important (for me at least) scenario -- testing an update for a port. Let's assume the port being tested is security/gnutls-devel. The process looks like that:1. Produce a patch and apply it to the tree (can be done via ssh, or nfs, whatever, and could be easily scripted up)2. Check builds we have:(21:33) novel@fsol:~ %> tbc build id name status current port updated 1 8.x-FreeBSD IDLE None 2011-01-30 14:05:25(21:40) novel@fsol:~ %>So, we see an idle build here with id 1. Not so much choices, so let's use it, but first let's see what's in the queue:(21:40) novel@fsol:~ %> tbc queue id username port build pri status enqueued completed 10 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 13:54:43 2011-01-30 14:05:27(21:45) novel@fsol:~ %>Now adding a port to the queue:(21:46) novel@fsol:~ %> tbc queue add -b 1 security/gnutls-devel(21:46) novel@fsol:~ %>Let's check if new port build added successfully:(21:46) novel@fsol:~ %> tbc queue id username port build pri status enqueued completed 10 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 13:54:43 2011-01-30 14:05:27 11 novel security/gnutls-devel 8.x-FreeBSD 10 ENQUEUED 2011-01-30 21:46:20 None(21:46) novel@fsol:~ %>Cool, it's here! Now we can see that our build turned into PREPARE state:(21:48) novel@fsol:~ %> tbc build id name status current port updated 1 8.x-FreeBSD PREPARE None 2011-01-30 21:48:15(21:48) novel@fsol:~ %>All we have to do now is wait and poll for changes. Now we can see it's building:(21:59) novel@fsol:~ %> tbc build id name status current port updated 1 8.x-FreeBSD PORTBUILD gnutls-devel-2.11.5 2011-01-30 22:00:25(22:00) novel@fsol:~ %>And finally we see it's done:(22:07) novel@fsol:~ %> tbc queue id username port build pri status enqueued completed 11 novel security/gnutls 8.x-FreeBSD 10 SUCCESS 2011-01-30 21:46:20 2011-01-30 21:57:18 12 novel security/gnutls-devel 8.x-FreeBSD 10 SUCCESS 2011-01-30 21:50:54 2011-01-30 22:09:22(22:15) novel@fsol:~ %> tbc queue 11For everyone who's interested I've uploaded sources on github:NOTICE: it's an early work in progress! The code is unstable and lacks a lot of features. Anyway, any feedback about an idea and implementation is welcome.

Who uses FreeBSD?

One reason why I maintain this blog is to do some advocacy for FreeBSD - attempt to spread the word about new developments and generally promote it.

It's not an easy job as the system constantly struggles for popularity aside the much more popular Linux (and on the other side - older Unixes and Windows). I've accidentally come across this page on www.freebsd.org listing some popular FreeBSD users but I think it badly needs updating.

Are you a big FreeBSD user? Working in an organization which uses FreeBSD on a larger scale? Write about it here (in comments)! No need to mention details if you think they are confidential!

Read more...

Who uses FreeBSD?

One reason why I maintain this blog is to do some advocacy for FreeBSD - attempt to spread the word about new developments and generally promote it.

It's not an easy job as the system constantly struggles for popularity aside the much more popular Linux (and on the other side - older Unixes and Windows). I've accidentally come across this page on www.freebsd.org listing some popular FreeBSD users but I think it badly needs updating.

Are you a big FreeBSD user? Working in an organization which uses FreeBSD on a larger scale? Write about it here (in comments)! No need to mention details if you think they are confidential!

Read more...

Who uses FreeBSD?

One reason why I maintain this blog is to do some advocacy for FreeBSD - attempt to spread the word about new developments and generally promote it.

It's not an easy job as the system constantly struggles for popularity aside the much more popular Linux (and on the other side - older Unixes and Windows). I've accidentally come across this page on www.freebsd.org listing some popular FreeBSD users but I think it badly needs updating.

Are you a big FreeBSD user? Working in an organization which uses FreeBSD on a larger scale? Write about it here (in comments)! No need to mention details if you think they are confidential!

Read more...

Donations for 2010

Since a few years I have a tradition to donate a small percentage of my yearly income to Open Source projects. Over the years I have refined my thoughts about it and what I want to archive with it. To keep the overhead low and to reach the correct people with that donation I prefer to directly donate to the people. What is also important to mention is that it is always meant as a "Thank you for your great work" so it is not connected to any requirements. I usually take a few moments at the end of the year to think about what was archieved in the last year and had a big impact on my work or to what I think is important for FreeBSD.

So without further ado my donation for 2010 goes to Hans Petter Selasky for his extraordinary work on webcamd which enables FreeBSD to use modern DVB-S/C/T USB devices and webcams. This makes FreeBSD a viable platform for Home Theater PCs and improves our multimedia support.

Thanks a lot Hans for your great work!

Last years recipient:

DatePersonProject
01.2011Hans Petter Selaskywebcamd 
12.2009Alexander Eichner & Knut St. OsmundsenVirtualBox
01.2009Martin WilkeFreeBSD
09.2008Jannis PohlmannXfce
11.2006OpenBSDOpenBSD
05.2006FreeBSD Foundation and OpenBSDFreeBSD and OpenBSD

AR9280/AR9285 support now working

I managed to find and fix the AR9280 and AR9285 support.Things that needed repairing:
  • Updating the AR9280 initvals to the Linux ath9k ones resolved the radio initialisation issues, making everything magically stable!
  • The v4k EEPROM code (for the AR9285/AR2427) needed some surgery to reflect what was really going on. In particular, the 8 bit radio bias values in the AR9280 (v14) EEPROM are actually lots of 4 bit values; so the bias values being written to the radio were woefully incorrect. This restored AR9285 stability and made the AR2427 function.
  • The AR9280 RF registers need an extra delay before being written to. I guess since the earlier radios are externally connected via single-bit IO (and thus shift registers are involved), the later radios are too. Without this delay, the AR9220 panics on my MIPS board and I would get occasional unexplained resets when I configured an AR9280 on my eeepc. This fix has resolved both those issues.
I have one more fix to integrate for the AR9220 init path so the Ubiquiti SR71-12 and SR71-15 work; then the AR9220 is supported.The AR2427 baseband seems to hang after a few hours of use, requiring a complete cold (power off) restart. There's some more initialisation code I need to port from ath9k and there's a lot of AR9280/AR9285 calibration code which I need to port over. I hope porting these two over will fix the stability issues that I was seeing. There's also some noise floor calibration fixes from ath9k which I need to integrate into the FreeBSD-HEAD tree.So, there's been quite a bit of progress! I've had reports from users that the AR9280, AR9285 and AR9220 support is now very stable; much more stable than before.

AR9280/AR9285 support now working

I managed to find and fix the AR9280 and AR9285 support.

Things that needed repairing:

  • Updating the AR9280 initvals to the Linux ath9k ones resolved the radio initialisation issues, making everything magically stable!
  • The v4k EEPROM code (for the AR9285/AR2427) needed some surgery to reflect what was really going on. In particular, the 8 bit radio bias values in the AR9280 (v14) EEPROM are actually lots of 4 bit values; so the bias values being written to the radio were woefully incorrect. This restored AR9285 stability and made the AR2427 function.
  • The AR9280 RF registers need an extra delay before being written to. I guess since the earlier radios are externally connected via single-bit IO (and thus shift registers are involved), the later radios are too. Without this delay, the AR9220 panics on my MIPS board and I would get occasional unexplained resets when I configured an AR9280 on my eeepc. This fix has resolved both those issues.
I have one more fix to integrate for the AR9220 init path so the Ubiquiti SR71-12 and SR71-15 work; then the AR9220 is supported.

The AR2427 baseband seems to hang after a few hours of use, requiring a complete cold (power off) restart. There's some more initialisation code I need to port from ath9k and there's a lot of AR9280/AR9285 calibration code which I need to port over. I hope porting these two over will fix the stability issues that I was seeing. There's also some noise floor calibration fixes from ath9k which I need to integrate into the FreeBSD-HEAD tree.

So, there's been quite a bit of progress! I've had reports from users that the AR9280, AR9285 and AR9220 support is now very stable; much more stable than before.

Quarterly Status Report

The FreeBSD Quarterly Status Report, covering the period of October to December 2010, has been published. The FreeBSD Foundation's status was reported as follows:

We raised $325,000 towards our goal of $350,000 for 2010! This will allow us to increase our project development and equipment spending for 2011.

We were proud to be a sponsor for EuroBSDCon 2010, BSDDay Argentina 2010, MeetBSD California 2010, and NYBSDCon 2010.

Completed the Foundation funded projects: DAHDI Project by Max Khon and BSNMP Improvements by Shteryana Sotirova.

We kicked off a new project by the University of Melbourne called Feed-Forward Clock Synchronization Algorithms Project. The Five New TCP Congestion Control Algorithms for FreeBSD Project by Swinburne University also officially started.

We continued our work on infrastructure projects to beef up hardware for package-building, network-testing, etc. This includes purchasing equipment as well as managing equipment donations.

Stop by and visit with us at FOSDEM (Feb 5-6), SCALE (Feb 26), AsiaBSDCon (March 17-20), and Indiana Linuxfest (March 26).

Read more about how we supported the project and community by reading our end-of-year newsletter.

We are fund-raising for 2011 now!

Quarterly Status Report

The FreeBSD Quarterly Status Report, covering the period of October to December 2010, has been published. The FreeBSD Foundation's status was reported as follows:

We raised $325,000 towards our goal of $350,000 for 2010! This will allow us to increase our project development and equipment spending for 2011.

We were proud to be a sponsor for EuroBSDCon 2010, BSDDay Argentina 2010, MeetBSD California 2010, and NYBSDCon 2010.

Completed the Foundation funded projects: DAHDI Project by Max Khon and BSNMP Improvements by Shteryana Sotirova.

We kicked off a new project by the University of Melbourne called Feed-Forward Clock Synchronization Algorithms Project. The Five New TCP Congestion Control Algorithms for FreeBSD Project by Swinburne University also officially started.

We continued our work on infrastructure projects to beef up hardware for package-building, network-testing, etc. This includes purchasing equipment as well as managing equipment donations.

Stop by and visit with us at FOSDEM (Feb 5-6), SCALE (Feb 26), AsiaBSDCon (March 17-20), and Indiana Linuxfest (March 26).

Read more about how we supported the project and community by reading our end-of-year newsletter.

We are fund-raising for 2011 now!

OpenBSD IPSec backdoor allegations: update

I'm sure I don't need to remind anyone what this is about...

The latest news: Theo now says that it is probable that NetSec was indeed contracted to insert backdoor code into OpenBSD, but after a month of review and changelog archeology, there is still no sign that they succeeded or even attempted to push tainted code into the tree.

The audit (which is still ongoing) did uncover one serious bug, but there is no reason to believe that it was planted deliberately. This relates to CBC mode, an encryption protocol in which each block of plaintext is combined with the ciphertext of the previous block before encryption to make it harder to attack ciphertext blocks individually.

If I understand Theo's message correctly,

  • It used to be common practice to use the last ciphertext block from one message as IV for the next message. This seemed like a good idea at the time, because the alternative is to generate a random IV for each new message, which requires a strong, fast PRNG, and strong, fast PRNGs didn't grow on trees back when this scheme was devised. By reusing the last ciphertext block from the previous message, a costly random IV was only required for the very first message.
  • This practice was discovered to be a bad idea because in n - 1 out of n cases (where n is the block size in bytes), the last plaintext block of any message encrypted with a block cipher contains somewhat predictable padding.
  • The flawed IV logic was replicated in several parts of the OpenBSD source tree, and the fix was implemented in some of them, but not all.
  • The person who implemented this flawed logic was at that time a NetSec employee, but he had been involved in the development of OpenBSD's IPSec stack for years before he was hired, and, as previously mentioned, he was only following common practice.
  • The same person implemented the obvious fix (generating a new, random IV for every message) once the attack was discovered.
  • The person responsible for those parts of the tree in which the fix was not implemented is one of the people fingered by Perry, but his tenure started after Perry had left and ended before the attack was discovered.
  • Anyone with any amount of experience in a large F/OSS project, or any large software development effort for that matter, can tell you that this kind of oversight is the rule rather than the exception. Although there is no evidence that he did not intentionally “forget” to fix his code, it is far more likely that he simply did not realize that the fix that had already been committed did not extend to his own code, or that he wasn't paying attention, and nobody else noticed.

My bounty still stands, and I will even relax the requirements a bit: you are not required to show that OpenBSD is still exploitable, only that it was exploitable on December 11, 2010 (the date of Perry's email to Theo).

OpenBSD IPSec backdoor allegations: update

I'm sure I don't need to remind anyone what this is about...

The latest news: Theo now says that it is probable that NetSec was indeed contracted to insert backdoor code into OpenBSD, but after a month of review and changelog archeology, there is still no sign that they succeeded or even attempted to push tainted code into the tree.

The audit (which is still ongoing) did uncover one serious bug, but there is no reason to believe that it was planted deliberately. This relates to CBC mode, an encryption protocol in which each block of plaintext is combined with the ciphertext of the previous block before encryption to make it harder to attack ciphertext blocks individually.

If I understand Theo's message correctly,

  • It used to be common practice to use the last ciphertext block from one message as IV for the next message. This seemed like a good idea at the time, because the alternative is to generate a random IV for each new message, which requires a strong, fast PRNG, and strong, fast PRNGs didn't grow on trees back when this scheme was devised. By reusing the last ciphertext block from the previous message, a costly random IV was only required for the very first message.
  • This practice was discovered to be a bad idea because in n - 1 out of n cases (where n is the block size in bytes), the last plaintext block of any message encrypted with a block cipher contains somewhat predictable padding.
  • The flawed IV logic was replicated in several parts of the OpenBSD source tree, and the fix was implemented in some of them, but not all.
  • The person who implemented this flawed logic was at that time a NetSec employee, but he had been involved in the development of OpenBSD's IPSec stack for years before he was hired, and, as previously mentioned, he was only following common practice.
  • The same person implemented the obvious fix (generating a new, random IV for every message) once the attack was discovered.
  • The person responsible for those parts of the tree in which the fix was not implemented is one of the people fingered by Perry, but his tenure started after Perry had left and ended before the attack was discovered.
  • Anyone with any amount of experience in a large F/OSS project, or any large software development effort for that matter, can tell you that this kind of oversight is the rule rather than the exception. Although there is no evidence that he did not intentionally “forget

FreeBSD 7.4-RC2 Available

The second Release Candidate build for the FreeBSD-7.4 release cycle is now available. ISO images for Tier-1 architectures can be downloaded from most of the FreeBSD mirror sites. Please see the official announcement for further details about this release.

AR9220 support; why AR9280 is broken

I acquired a pair of shiny AR9220 minipci NICs last week - Ubiquiti SR71-12 and SR71-15. They're supposed to be like AR9280 but on PCI, not PCIe. Unfortunately they didn't work under FreeBSD. After some sleuthing (read: guessing), I stumbled across an ath9k commit which reworked some workaround for some AR9220's, which include the Ubiquiti SR71 versions.

https://patchwork.kernel.org/patch/90926/

There's also a bus error when reading a random register which is only referenced in a debug statement. That's a whole other bug to try and deal with.

But now I've verified that with both the AR9280 and the AR9220, the FreeBSD code is definitely tickling the radio wrong. Only one of the two radio chains is even active for the most part, and that chain seems to go deaf quite often, resulting in a baseband hang and chip reset. It also explains why I was getting absolutely shocking 11n performance out of it in my 11n branch.

AR9220 support; why AR9280 is broken

I acquired a pair of shiny AR9220 minipci NICs last week - Ubiquiti SR71-12 and SR71-15. They're supposed to be like AR9280 but on PCI, not PCIe. Unfortunately they didn't work under FreeBSD. After some sleuthing (read: guessing), I stumbled across an ath9k commit which reworked some workaround for some AR9220's, which include the Ubiquiti SR71 versions.https://patchwork.kernel.org/patch/90926/There's also a bus error when reading a random register which is only referenced in a debug statement. That's a whole other bug to try and deal with.But now I've verified that with both the AR9280 and the AR9220, the FreeBSD code is definitely tickling the radio wrong. Only one of the two radio chains is even active for the most part, and that chain seems to go deaf quite often, resulting in a baseband hang and chip reset. It also explains why I was getting absolutely shocking 11n performance out of it in my 11n branch.