Todays wow, this really works! moment brought to you by: NFS, sshfs (from FUSE) and Samba.
There’s a NFS file server in my office, from which I mount stuff into my home directory on my workstation. There’s a small FreeBSD server in my living room which, among other things, serves Samba to my Windows desktop machine. Using sshfs on the home server, I mounted my home directory from my workplace as a subdirectory of my local user, and I’m accessing it from my Windows desktop over Samba.
Before the bytes hit the drives on the server, here’s the path they must take:
[Home desktop, Windows] -- Samba -- [Home server, FreeBSD] -- sshfs -- [Work desktop, Linux] -- NFS -- [Work server, FreeBSD]
And it works. Really. I’m editing OpenOffice files on my Work server right now.
Of course it should work – all of these individual components in the chain are tested and known to work so there’s practically no real concern, but seeing all this in operation made me think how many standards, interoperability specs and engineering went into making this possible, especially since the actual connections between the components are very varying: ADSL, Ethernet of various speeds and I’m sure there’s still ATM somewhere in the telco’s infrastructure. The number of different operating systems the bytes go through (if “embedded” ones on routers and similar equipment is also counted) is probably huge.
We live in great times.
(Of course, I won’t try anything that depends on file locking 
)
The only problem is that sshfs basically hangs the system when the IP changes on the ADSL side (file system lookups hang).
The answer, of course, is “two or more”. And it’s not nice when it happens.
Two of the drives on a nice shiny FC array failed at approximately the same time (possibly within about two minutes of each other), and both were in the same RAID5 array. Definitely not good. On the other hand I confirmed that PostgreSQL can run off NFS (both server and client on FreeBSD) without problems so far (this is the temporary setup until we get a new array).
I created a small program to help me synchronize files in sort-of real-time between two directories (the idea is that one of them is on a NFS server). There are no replicated file systems for FreeBSD and the canonical way to do this is usually to use rsync or something like it. The problem here is that rsync always traverses both directory structures, compares files and then copies them (via a variety of smart algorithms but it’s still very resource-intensive).
So I created a daemon that uses kqueue(2) to monitor which files changed and feed only those files to rsync (it’s not exactly a new idea, I’m sure somebody has also mentioned it somewhere on the FreeBSD lists). This is in many ways a suboptimal solution since it needs to keep an open file descriptor for all the monitored files (which ties in kernel resources and memory) so it won’t scale for really large directories, which could actually benefit the most from this approach. It will work reasonably well for a small number of files (up to several tens of thousands), with modifying kern.maxfiles and kern.maxfilesperproc sysctls and login session limits (if applicable).
Anyone who’s interested can download the adfs daemon and try it. This was hacked together over the weekend so it probably has some problems. I’ll fix those problems that prevent me from using it, but I’ll update the online archive only if there are interested users.
I’m writing a small project in C (will talk about it later) and I really miss the expressiveness that dynamic languages like Python offer. There’s one more thing in addition to elusive “elegance” and similar nontangible properties: the ability to easily use and implement better algorithms. Yes, since Turing it was obvious that the actual programming language in use is more-or-less syntactic sugar, but you wouldn’t exactly like to spend your days programming infinite tapes of symbols, would you?
In this (again, emphasis on “small”) project, there were a couple opportunities where I could make use of a fast data-access structure like a hash table (since I need to store and retrieve a lot of data entries) or dynamic memory allocation (since I don’t want to artificially limit the number of these entries) but I just didn’t feel like writing all that code to implement a hash table in C (or use a heavy external library) and deal with memory reallocation and track all those pointers. Yes, I’m lazy. In a more abstract language I could just instantiate a dictionary and say d[i] = something and this would actually be very efficient and take care of memory allocation automagically. Since I limited myself to basic C, I chose simpler algorithms like linked lists and evil static arrays on stack. Ironically, these structures would be comparatively significantly more inefficient in Python.
Of course, at its roots this can be stripped down to be simply a choice between using pre-packaged routines instead of writing your own (aka the NIH problem), but in this case it would actually make my simple program faster and more efficient – despite the overhead of an interpreted, dynamic language.
There are many more similar cases – programmers write bubblesorts in C because they are easy to implement, while going to a higher level of abstraction they could just write mylist.sort() and would get QuickSort or some other efficient algorithm for free, etc. etc.
Does anyone know of a library / collection of algorithms for C similar to glib only BSD-licensed? (Yes, I know about C++ algorithms, I don’t want to use C++).
The day has finally come – FreeBSD is using Subversion instead of CVS for the base source tree! Congratulations to everyone involved, especially Peter Wemm
FreeBSD’s source CVS is one of the oldest and biggest in existence; it’s approximately 12 years old and has apparently had something like 180,000 commits over the years, or on average slightly more than 41 commits daily. A checkout of RELENG_7 branch holds more than 42,000 files (in 482 MB as du sees it).
This move was discussed extensively during the DevSummit at BSDCan 2008; there have been many issues with CVS over the years, most of which are minor enough to be overlooked, but some of which are just nasty (the inability of CVS to move/rename files, bad handling of branching in the event of constant new development and additions to the directory tree, non-atomic commits) and have frequently required manual interventions in the CVS repository (“repocopy” is one of the relatively frequently requested operations to the CVS admins).
Old infrastructure, of which cvsup / csup is probably the most important part, will continue to work as code will continually be mirrored from SVN to CVS, until suitable replacements or upgrades to the above tools are created. This is also the reason the name “CVS” will be present for some time in the system infrastructure, until all of it is updated. Ports will continue to use CVS for the foreseeable future.
To make a source base this large work efficiently on SVN, version 1.5 had to be used, since it creates its database files in a hash-tree of directories on the file system instead of one huge directory with all the files in it.
Also see the official announcement of Subversion and Peter Wemm’s notes about Subversion (very useful to developers).
It was a really great conference! I have so many good impressions it’s just hard to sort them all out and write them down. Instead of that, here’s a treat for the geeky-minded
In accordance with the specification expressed here:
... I’ve created a device driver that implements the functionality in kernel. In honour of the operating system by which this work was inspired, I name the driver random.debian. The kernel module creates a device entry (/dev/random.debian) which is a infinite source of random data with entropy compatible with the above specification. The source code tarball for the Debian-like random data source is of course available under the BSD license. This will work on any recent version of FreeBSD.
As much as I would like to have though of this first, the idea was actually put out by PHK or Robert Watson while we were waiting for dinner, so that part of the credit goes to them.
This DevSummit+BSDCan was very fun and educational and will definitely try to be here in the next years also.
It was a nice day in Ottawa, Canada today, though judging by the clouds tomorrow might not be. Not that we noticed, spending all the time within the conference room. Various talks kept developers interested, and the lack of Internet connectivity (web is not the Internet…) kept them focused on the issues at hand. I gave my talk about finstall among the first in the morning and gathered very positive feedback and many new ideas. It’s unfortunate that the project isn’t sponsored any more (Google SoC for it was not extended to this year) so this might encourage people or organizations to support the project financially. Among other interesting talks (my personal, unobjective choice) were presentations on VImage (network virtualization) by my colleague from University, Marko Zec, the DTrace talk by John Birrell, and various talks about the ongoing network stack optimizations (many people here). Unfortunately the Release packaging BoF didn’t happen due to lack of interest apparently, so there’s one missed chance to discuss finstall.
All together, it was a very nice spent day with many opportunities to learn new things and speak to like-minded people working on interesting things.
Here’s another build of finstall, alpha4. The most important change this time is native support for remote / headless installs via systoold network daemon.
This enables headless installs of FreeBSD in the following fashion:
- Insert the CD with the finstall in the server (obviously, the server needs to have a CD/DVD reader; USB ones are mostly fine). The server can be headless, i.e. without a monitor, a keyboard or a mouse.
- Connect the server to a LAN. No remote-network hops (routing) are allowed since UDP broadcasts are used to locate the headless nodes. Boot off the finstall CD.
- On another machine (one that has monitor+keyboard+mouse or equivalent X11 devices) start the installer front-end.
- In the front-end, opt to connect to a remote finstall node, choose the one you want. At this point you can see boot-time dmesg data from the nodes so you can locate the right one in case there are many of them.
- Proceed to use the front-end GUI just like it was a local install.
- Reboot, configure, use the server, etc.
As described, the primary usage for this is to setup headless servers.
PXE is supported in theory, but not tried. The idea is that, since the whole finstall setup is actually a live FreeBSD system, PXE can be configured manually once the CD is booted somewhere (possibly on a virtual machine), and remote systems can be booted from this CD-based file system, then installed as if they are booted locally. This is experimental and untried.
This mode of installation has many side-effect uses, such as scripting the remote install, etc.
More about this and other features of finstall will be presented on BSDCan 2008.
Update: The original ISO image posted above had a trivial but unfortunate bug. Download the new ISO image with MD5 fingerprint a9eebbdc546565a9eb9c6622bb948d75.
Background: I’m developing something that should eventually become a high performance network server, with high transactions/s rate (basically a database cache). Currently I’m experimenting with various modes of using SMP facilities for the server (thread usage, binding, etc.). A big problem is that, while I temporarily have a server on which to test it, I don’t have a client machine which could push the server to the limits. I currently have a “dumb” multithreaded benchmark client, spawning N threads (N is 40 in these tests), each of which is a blocking network client (i.e. one thread per connection). This setup, when run via local Unix sockets on the server, can achieve 125,000 trans/s, but I believe the result should be much better if the client doesn’t task the CPU of the server.
Marko Zec helped me with that, temporarily providing me a machine which dual boots 7.x and 4.x with his VIMAGE patches, as well as without the patches. Originally I just used the 7.0 system, and achieved something like 62,000 trans/s, which is too low for me. On his insistence I booted 4.11 and ported the client-side benchmark on it. Without any significant modifications except those needed for the difference between gcc 2.9x and gcc 4, the same client code rocketed to 81,000 trans/s! This is using libc_r, meaning the whole 40-threaded thing is visible to the kernel as one process (4.x doesn’t have kernel support for multithreading)! This number is still too low and I’ll probably need to find several machines that could work at the same time to overtax the server (which will be very hard) but just the raw difference between 4.x and 7.x is staggering. Network card is bge, gigabit, directly connected to the server via crossover cable.
On the bright side, VIMAGE patches don’t influence the performance noticeably.
If you’re wondering how long it takes to propagate a commit made on ports to portsnap servers, the answer is: about two hours