Monthly Archives: September 2011

More Advanced Format drives: Samsung SpinPoint F4 EcoGreen and Seagate Barracuda Green

I've acquired a couple more 2 TB Advanced Format drives: a Seagate Barracuda Green (ST2000DL003) and a Samsung SpinPoint F4 EcoGreen (HD204UI, no data sheet available online).

I was extremely impressed with the Samsung HD204UI. It's the first AF drive I've seen with decent performance. In fact, it's the fastest disk I've tested so far—its unaligned writes are faster than the non-AF Hitachi I used as a reference last time, and its aligned writes are twice as fast.

   count    size  offset    step        msec     tps    kBps

  131072    1024       0    4096       43984    2979    2979
  131072    1024     512    4096      127047    1031    1031

   65536    2048       0    8192       14764    4438    8877
   65536    2048     512    8192       12453    5262   10524
   65536    2048    1024    8192       12460    5259   10518

   32768    4096       0   16384        4609    7109   28436
   32768    4096     512   16384        7829    4185   16740
   32768    4096    1024   16384        8413    3894   15579
   32768    4096    2048   16384        8211    3990   15961

   16384    8192       0   32768        3952    4145   33165
   16384    8192     512   32768        9050    1810   14481
   16384    8192    1024   32768        9317    1758   14067
   16384    8192    2048   32768        9315    1758   14069
   16384    8192    4096   32768        3996    4099   32793

The Seagate ST2000DL003, on the other hand, is so slow it's not even funny. It's actually the slowest of all the drives I've tested: its performance on aligned random writes is half that of the Western Digital WD20EARS. It's three times as fast on unaligned writes, but three times nothing (100 kBps) is still nothing (300 kBps) compared to the Samsung HD204UI (15 MBps). Here are the numbers:

   count    size  offset    step        msec     tps    kBps

  131072    1024       0    4096     2419280      54      54
  131072    1024     512    4096     2199286      59      59

   65536    2048       0    8192     1283667      51     102
   65536    2048     512    8192      985184      66     133
   65536    2048    1024    8192      995423      65     131

   32768    4096       0   16384       45980     712    2850
   32768    4096     512   16384      345291      94     379
   32768    4096    1024   16384      432533      75     303
   32768    4096    2048   16384      429781      76     304

   16384    8192       0   32768       34192     479    3833
   16384    8192     512   32768      166440      98     787
   16384    8192    1024   32768      210147      77     623
   16384    8192    2048   32768      207356      79     632
   16384    8192    4096   32768       34221     478    3830

This time, I also ran sequential write tests—basically, dding eight gigabytes' worth of zeroes to the disk in 128 kB blocks, which is the optimal I/O size for FreeBSD. This time, the results are pretty close: the Samsung HD204UI gets slightly less than 90 MBps, and the Seagate ST2000DL003 gets slightly less than 80 MBps.

FreeBSD 9.0-BETA3 Available

The third BETA build for the FreeBSD-9.0 release cycle is now available. ISO images for the architectures amd64, i386, ia64, powerpc, powerpc64, and sparc64 are available on most of our FreeBSD mirror sites. One of the many new features in 9.0 we would like to be tested is the new installer, so we encourage our users to do fresh installation on test systems.

Implementing xlocale APIs Project Update

The following project update was written by David Chisnall who received a grant from us to implement xlocale APIs to enable porting libc++. We're pleased that the project is almost completed!It's traditional to start this sort of thing by telling you who I am. I started using FreeBSD around 2001. At the time, I'd used Linux but switched to FreeBSD because it sounded like it worked correctly - I could have xmms playing music, my IM and email clients notifying me of new messages, and BZFLag making gunshot noises all at the same time. Apparently, ten years later, this still doesn't work reliably on Linux...
I got involved with clang via a somewhat indirect route. I'm a member of the core team of the Étoilé project, which aims to build a (BSD licensed) desktop environment on top of GNUstep. I grew increasingly frustrated with the level of Objective-C support in GCC, which included shipping one release with Objective-C completely broken and displaying no progress towards supporting the Objective-C 2 extensions that were about 5 years old at the time. I looked at the code, but it was an incomprehensible mess of spaghetti.

Apple had just released a new compiler front end (clang) that had Objective-C parsing mostly finished, but code generation missing. I started poking the clang code to try to support the GCC Objective-C runtime, and a few weeks later had a working Objective-C 2 compiler. I then grew frustrated with the limitations (including the license) of the GCC Objective-C runtime and wrote a more modern (MIT licensed) replacement. Clang now supports both and with the new runtime is at feature parity with Apple's implementation of the language.

While hacking on clang - which I do on FreeBSD - I fixed various FreeBSD-related bugs. This put me in contact with Roman Divacky, who had been working on importing clang into the base system. This is an important task, because FreeBSD currently uses the last GPLv2 version of GCC as the system compiler. Although this release seems less buggy than subsequent ones, it is now over 4 years old and is no longer supported upstream. It won't, for example, get any of the features of C1x or C++11.

The compiler is only half of the problem. The other half is the standard library. For C, this isn't an issue: FreeBSD has had its own C standard library implementation since before it was FreeBSD. For C++, it's a bigger problem. FreeBSD currently ships with GNU libstdc++, which has undergone the same sort of license change as GCC, leaving FreeBSD stuck with an old version.

A good candidate for replacement is libc++, developed as part of the LLVM project and available under UIUC and MIT licenses. This has a few dependencies. One is a low-level C++ ABI library, which implements the dynamic parts of C++ such as exception handling and run-time type information. I'd written an implementation of this for PathScale, and the FreeBSD and NetBSD Foundations jointly paid for it to be open sourced. I've since extended it with some additional features required to support C++11, which includes an std::exception_ptr object that allows exceptions to be passed between threads.

The other dependency is the C standard library. Libc++ was written by Apple developers (Apple is in the same situation as FreeBSD with regards to the GPLv3) and so uses some features of Darwin libc that are not portable. Specifically, Darwin libc has a convenient set of extensions to localisation: xlocale. This extends the POSIX 2008 per-thread locale APIs (missing in FreeBSD) to provide a set of _l variants of locale-aware libc functions that use a specific locale, rather than the global one.

My recent work, sponsored by the FreeBSD Foundation, has been to implement the missing xlocale APIs. This is now mostly done and pending code review. With this and the new tweaks to libcxxrt, it's now possible to build libc++ on FreeBSD and most of the tests pass.

Most of the remaining test failures are in the header. This defines a lot of complex atomic operations and requires a lot of compiler support. Eli Friedman has been working on adding this support in clang, and with his latest patch applied 25 of the 52 atomic tests pass. There are still a few remaining failures:

- 27 caused by clang not fully supporting the atomic operations yet
- 3 caused by clang not fully supporting the C++11 type-trait intrinsics
- 20 that I don't think are real failures - they're caused by the VM where I'm running the tests not having sufficiently fine-grained time reporting for the thread operation timeout tests to work properly
- 1 is caused by FreeBSD lacking the C1x quick_exit() APIs.
- 2 caused by FreeBSD lacking the uchar.h header

For comparison, Howard Hinnant, the libc++ lead developer, just sent me a list of the failures on OS X. On FreeBSD, 4271 tests pass, 53 fail. On OS X 4253 pass, 71 fail. This is looking very promising for an entirely GNU-free C++ stack.

Alberto Villa, FreeBSD committer

At last, we started shipping packages – amd64 only – to help with the CFT announced last week. If you’re one of those who want to test the incoming KDE Software Compilation, but don’t want to build it all from source, follow these steps: make sure you have a working X.Org installed (with a graphic […]

I know this is old..

From if_wi.c in FreeBSD:

/*
* Lucent WaveLAN/IEEE 802.11 PCMCIA driver.
*
* Original FreeBSD driver written by Bill Paul <[email protected]>
* Electrical Engineering Department
* Columbia University, New York City
*/

From if_wi.c in OpenBSD:

/*
* Lucent WaveLAN/IEEE 802.11 driver for OpenBSD.
*
* Originally written by Bill Paul <[email protected]>
* Electrical Engineering Department
* Columbia University, New York City
*/

DIFFUSE for FreeBSD

The FreeBSD Foundation is pleased to announce that Swinburne Universityof Technology's Centre for Advanced Internet Architectures hasbeen awarded a grant to implement DIFFUSE for FreeBSD.

DIFFUSE (Distributed Firewall and Flow-shaper Using StatisticalEvidence) is an extension to the FreeBSD IPFW firewall subsystemdeveloped by CAIA. It allowsIPFW to classify traffic based on statistical properties of flows beingobserved in realtime, and instantiate network actions across adistributed set of "action nodes" for particular flows if required.

This project will tidy up and integrate theexisting DIFFUSEprototype into FreeBSD, and incorporate a number of important newfeatures. Integration of DIFFUSE into FreeBSD will increase FreeBSD'sutility to designers and implementers of FreeBSD-based networkinginfrastructure.

Network architects frequently require the ability to classify differenttraffic types flowing across a network, typically using packetinspection capabilities of base system tools such as ipfw and pf.Traffic classification then enables the provision of customized servicelevels to different traffic types (such as priority packet queuing andforwarding, or allocation of specific bandwidth guarantees).

DIFFUSE uses machine learning techniques to enable robust and efficientclassification of IP traffic flows based on their unique statisticalproperties in addition to traditional inspection of packet header orpayload contents. DIFFUSE also allows traffic classification to occur inone place (e.g. in the core of a network) and trigger traffic shapingand differentiation elsewhere (e.g. at the edges of a network). DIFFUSEhas applications in ISP, residential broadband and large corporatenetwork scenarios to name a few.

The project will conclude the end of October 2011.

Implementing xlocale APIs

The FreeBSD Foundation is pleased to announce that David Chisnall has been awarded a grant to implement xlocale APIs to enable porting libc++.

The C standard library (libc) is one of the most important parts of a UNIX system as most programs interact with the kernel through interfaces written in C. Porting code between platforms with similar libc implementations is trivial and if something is supported by libc, higher-level languages can use it without being reimplemented.

Over time, the C language has slowly evolved to modern multicore systems, but there are still some places that are problematic. One of these is localization as C began originally had no localization support. FreeBSD libc and Darwin libc (used by Mac OS X) are similar, making it much easier to port code from OS X to FreeBSD than from OS X to Linux. The libc used by OS X supports a set of extended locale functions (xlocale) that allow locale to be set on a per-thread basis.

Additionally, libc++, from the LLVM project, was originally developed on Darwin, so it uses xlocale for most of the C++ locale support. The lack of this support is the primary obstacle to porting it to FreeBSD.

Once xlocale is supported in FreeBSD libc, we can port libc++ to FreeBSD, giving us an MIT-licensed C++11 standard library implementation. This, in conjunction with Clang and libcxxrt, means that the entire C++ stack in FreeBSD will be free of any GNU code. This leaves the linker as the only significant obstacle to a GPL-free FreeBSD 10.

The project will conclude the end of September 2011.

Alberto Villa, FreeBSD committer

The moment has finally arrived. We’re ready to share our work on the latest KDE Software Compilation to all the brave testers who wish to give a hand. The ports should be quite stable, and if we receive good feedback, I hope to be able to commit it to /usr/ports before the release slush, and […]

Participate in Software Freedom Day

This Saturday, September 17, is Software Freedom Day (SFD). SFD is an annual global event that encourages open source software users to reach out to their local community to educate others about the benefits of using open source.

Frederic Muller, President of SFI, the non-profit organization behind Software Freedom Day, has been very helpful in encouraging FreeBSD users to participate in SFD. FreeBSD is listed as a partner on the SFD website. In addition, the FreeBSD logo is included on the cover letter and a copy of PC-BSD was included with the 210 packages that were shipped to the pre-registered teams. He also added the FreeBSD news RSS feed to planet SFD so that other SFD participants will get FreeBSD updates.

Julian H. Stacey has also been helpful in spreading the word and will be participating with the Munich team.

BSD user groups are encouraged to put together a team and to register it on the SFD wiki. If you have not participated in SFD before, take a moment to read through the Start Guide for instructions on how to promote your team. If you do register a team, leave a comment so that we can mention it on the @bsdevents twitter feed.


FreeBSD vendor summit – November 2011; Sunnyvale, California

Since this doesn't seem to have gotten much notice - there's a FreeBSD vendor summit being held in Sunnyvale, California on the 3rd and 4th of November 2011. This is an opportunity for developers and vendors to share project direction and goals, collaborate on various projects, and likely hit each other with a big "why can't you do X" stick :-)

NetApp is sponsoring the event.

I'll be there with someone from Qualcomm Atheros, who will be presenting on Qualcomm/Atheros' open source direction and strategy. There'll likely be some (informal) discussion about how developers and vendors can sign up to their open source developer programme and obtain documentation/reference code for the Atheros wireless and embedded chipsets.

(This is what I've signed up to and it's been a huge help in getting Atheros 802.11n support up and going in FreeBSD.)

More details can be found here:

http://wiki.freebsd.org/201111VendorSummit

If you're a vendor using FreeBSD, or you're a vendor thinking about using FreeBSD in a project (wireless or otherwise) this mini-conference is just for you. Personally I'm interested in talking with the nvidia representative about trying to get some CUDA stuff going on FreeBSD.