Archive
Again this year, the FOSDEM organization had reserved a DevRoom for the BSDs. I hadn’t been to FOSDEM for several years and was pleasantly surprised to see how many BSD developers and users had turned up.
Unfortunately, I did miss the first talk as the Sunday bus schedule clearly didn’t scale to the huge numbers of conference goers. The second talk was Ed Schouten on his Newcons project for FreeBSD. Of course, I was already familiar with the utmpx part of the project with ~100 ports failing on the cluster after those changes and we’re working together on fixing those. Ed showed some very promising performance improvements and much better UTF-8 non-ASCII support, although some fonds do need more work.
Benny Siegert introduced some of the nitty-gritty of autotools and libtool to ease software portability over multiple platforms. While some of the most hated parts in the ports world, they are by far an improvement over previous tools and, especially, manual development.
Next up was Shteryana Shopova showing how to debug the FreeBSD kernel with the large number of tools provided by the operating system. With generous amounts of examples and demos, she gave a number of tips on which information to include when sending a problem report to the FreeBSD bug tracking database to get the best support from the FreeBSD developers, and even more important, how to collect that data out of a crashed system.
A face seen at most european BSD-related conferences over the last many years, Marc Balmer presented a case study of using BSD Unix and BSD licensed software in a commercial setting, talking both of the advantages of the BSD license over other licenses (illustrated by the number of words in the license), the BSD development process and contributing code back to the project, and about the point of sale (POS) software his company makes on top of a BSD operating system.
By far the most popular talk with well over 80 attendees, Axel Beckert talked about the Debian/kFreeBSD project, building a Debian GNU userland on top of a FreeBSD kernel. While he spend some time to answer the biggest question of all: “Why?”, I’m not sure everybody was convinced by the answer: “Because we can”. Most people will probably still install Debian when they want Debian and FreeBSD if they want FreeBSD, there are some that consider this combination the best of both worlds. It will be interesting to follow how the project will develop in the future.
Treading carefully not the restart the version control system wars of old, Giorgos Keramidas showed how he tracked the FreeBSD subversion changes in his local Mercurial setup. Some of the speedups of having the changes locally in a Mercurial repository over a remote subversion system were quite impressive, and the combination does provide some advantages for people wanting to develop proprietary changes locally while still easily being able to import upstream changes.
Last but not least was Brooks Davis with a short presentation on his current work to increase the number of groups a process (and thus user) can be a member of. The historic lessons on how FreeBSD and other Unices handled this was both hilarious and sad at the same time, but with the number already increased from ~15 to 1023 in 8.0 and going forward to not having any limit at all, the future looks bright.
In all, a very interesting and well-attended BSD track at FOSDEM this year. Many thanks to the FOSDEM organizers for providing the room and to Marius Nünnerich for inviting the speakers. I hope to be there again next year.
No related posts.
Related posts brought to you by Yet Another Related Posts Plugin.
Currently I play a little bit around with my ZFS setup. I want to make it faster, but I do not want to spend a lot of money.
The disks are connected to an ICH 5 controller, so an obvious improvement would be to either buy a controller for the PCI slot which is able to do NCQ with the SATA disks (a siis(4) based one is not cheap), or to buy a new system which comes with a chipset which knows how to do NCQ (this would mean new RAM, new CPU, new MB and maybe even a new PSU). A new controller is a little bit expensive for the old system which I want to tune. A new system would be nice, and reading about the specs of new systems lets me want to get a Core i5 system. The problem is that I think the current offers of mainboards for this are far from good. The system should be a little bit future proof, as I would like to use it for about 5 years or more (the current system is somewhere between 5–6 years old). This means it should have SATA-3 and USB 3, but when I look at what is offered currently it looks like there are only beta-versions of hardware with SATA-3 and USB 3 support available on the marked (according to tests there is a lot of variance of the max speed the controllers are able to achieve, bugs in the BIOS, or the controllers are attached to a slow bus which prevents to use the full bandwidth). So it will not be a new system soon.
As I had a 1GB USB-stick around, I decided to attach it to the one of the EHCI USB ports and use it as a cache device for ZFS. If someone wants to try this too, be careful with the USB ports. My mainboard has only 2 USB ports connected to an EHCI, the rest are UHCI ones. This means that only 2 USB ports are fast (sort of… 40 MBit/s), the rest is only usable for slow things like a mouse, keyboard or a serial line.
Be warned, this will not give you a lot of bandwidth (if you have a fast USB stick, the 40MBit/s of the EHCI are the limit which prevent a big streaming bandwidth), but the latency of the cache device is great when doing small random IO. When I do a gstat and have a look how long a read operation takes for each involved device, I see something between 3 msec and 20 msec for the harddisks (depending if they are reading something at the current head position, or if the harddisk needs to seek around a lot). For the cache device (the USB stick) I see something between around 1 mssec and 5 msec. That is 1/3th to 1/4th of the latency of the harddisks.
With a “zfs send” I see about 300 IOops per harddisk (3 disks in a RAIDZ). Obviously this is an optimum streaming case where the disks do not need to seek around a lot. You see this in the low latency, it is about 2 msec in this case. In the random-read case, like for example when you run a find, the disks can not keep this amount of IOops, as they need to seek around. And here the USB-stick shines. I’ve seen upto 1600 IOops on it during running a find (if the corresponding data is in the cache, off course). This was with something between 0.5 and 0.8 msec of latency.
This is the machine at home which is taking care about my mails (incoming and outgoing SMTP, IMAP and Webmail), has a squid proxy and acts as a file server. There are not many users (just me and my wife) and there is no regular usage pattern for all those services. Because of this I did not do any benchmark to see how much time I can gain with various workloads (and I am not interested in some artificial performance numbers of my webmail session, as the browsing experience is highly subjective in this case). For this system a 1 GB USB stick (which was just collecting dust before) seems to be a cheap way to improve the response time for often used small data. When I use the webmail interface now, my subjective impression is, that it is faster. I am talking about listing emails (subject, date, sender, size) and displaying the content of some emails. FYI, my maildir storage has 849 MB with 35000 files in 91 folders.
Bottom line is: do not expect a lot of bandwidth increase with this, but if you have a workload which generates random read requests and you want to decrease the read latency, it could be a cheap solution to add a (big) USB stick as a cache device.
I've been promising great 2D performance from open source graphics for years. It was reaching the point where I was feeling awfully bad about being wrong so frequently. So this summer I started playing in my free time with making a GL backend for cairo. There was a previous sort of GL backend in the form of glitz, but it made a big mistake in trying to abstract GL through a Render-like API. The problem with accelerating 2D is that Render is a bad match for hardware!
A native GL backend turned out to be shockingly easy, now that we have support for EXT_framebuffer_objects all over, non-power-of-two textures, and GLSL. Here's a comparison of 3 backends, normalized to the image backend. Bigger bars means faster.
This shows an accelerated backend beating the CPU rasterization backend on 3 tests. Note that things for the image backend are a little unfair in its favor -- we can't scan out from cached system memory buffers, so if you want to actually see the results you have to do an upload at some point, which isn't reflected in the cairo-perf-trace results. Being able to beat that with GPU rendering to something that could be scanned out is pretty awesome. But that's only 3 tests -- for most of them image is winning. I've got some ideas for hacks on the 965 driver that may fix up a bunch of those bars (it's hard to estimate, since it's all about cache effects, and fixing those has a tendency to improve by more than the amount of time spent according to sysprof).
Since comparing to image isn't too fair, and we're not using image today, I did a comparison to xlib. This looks awesome:
By replacing Xlib usage with GL, we get a speedup on almost all the testcases, and a huge speed up on one that Xlib is pathologically slow on (I haven't figured out why for xlib yet). We've got a good pass rate on the cairo test suite, so I think this stuff is ready for people to start experimenting with in apps.
There's much more to do for performance still. I've got a plan to work on the 965 driver to improve glyphs-heavy tests like firefox-talos-gfx (and ETQW and WoW as well). For firefox-talos-svg, right now we're hitting aperture full because of all the spans data we're sending out before the GPU gets done with things. If we speed up the GPU rendering just a little, for example by tuning the inefficient shaders we're using right now, we can probably avoid hitting aperture full and cut CPU further. I think we're missing throttling for non-swapbuffers apps in DRI2, and we might actually do better and avoid aperture full if we do some appropriate throttling. And there's a lot of room for people who'd like to experiment with GL shader and state optimizations to jump in and tear this code apart.
I'd say that the Linux 2D acceleration story is starting to finally look good after all these years.
I’ve recently run into a problem with 7-STABLE on VMware ESXi 3.5u4. With a recent change my VM shuts off shortly after probing the LSI (mpt) disk controller. The same behavior started occurring over the summer in HEAD and the quick workaround is to change the VM’s disk controller type from LSI to BusLogic. Lately I have some time to poke people about this issue so I figured I would. The problem is getting as much as I can while booting and having some usable boot messages for someone to look at. This would usually be accomplished by redirecting console output to a serial port on the problem machine and hooking up a cross over cable between it and another box. I haven’t done this on VMware before though so I had to do a little googling and it’s pretty simple.
On the FreeBSD side the following needs to be added to /boot/loader.conf:
console=”vidconsole,comconsole”
This will redirect the console to both the video display and a serial port. Once that is done shutdown the VM so the serial port can be added and configured.
With the crashing VM turned off go to “Edit Settings”:
Click the “Add” button to add a serial port to the VM:
For the serial port output select “Connect to named pipe”:
Finally configure the pipe:
The name of the pipe should be a file location on the VM host machine, not the guest. The near end is “Server” since I want to see the output from this VM and the far end will be another virtual machine. For the VM I’ll be connecting from to view the console output I would do the same but near end would be “Client”.
Once all this is done, from the second working VM launch cu(1).
# cu -l cuad0
After that boot the crashing VM and kernel messages should appear on the second VM. That’s all it takes to setup a serial connection between two FreeBSD VMs on VMware.
Update: Images fixed.
In the last period I became rather familiar with the PFSense project. I decided to migrate some of my firewalling devices to PFSense, first starting at 1.2.3-RELEASE, and finally I upgraded them to 2.0-BETA1. Doing the latter thing is possible since the locations only use the internet from the LAN, and have some minor settings applied locally. Playing around makes it much easier because of that.
Currently I am checking the GRE and GIF interfaces, I am using them to create an OSPF network, and there are some oddities in them 
So perhaps I can see why the oddities are there and if needed correct them (or myself when I am misbehaving )
You should test PFSense, it runs FreeBSD 8, and is awesome !
In preparation for 7.3-RELEASE, the ports tree will be in feature freeze after release candidate 1 (RC1 )is released, currently planned for February 8.
If you have any commits with high impact planned, get them in the tree before then and if they require an experimental build, have a request for one in portmgr hands within the next few days.
Note that this again will be a feature freeze and not a full freeze. Normal upgrade, new ports, and changes that only affect other branches will be allowed without prior approval but with the extra Feature safe: yes tag in the commit message. Any commit that is sweeping, i.e. touches a large number of ports, infrastructural changes, commits to ports with unusually high number of dependencies, and any other commit that requires the rebuilding of many packages will not be allowed without prior explicit approval from portmgr after that date.
Related posts:
- ports feature freeze now in effect In preparation for 7.3-RELEASE, the ports tree is now in...
- Ports feature freeze now enforced As an experiment, there will not be a complete ports...
- Ports tree frozen in preparation for 7.2-RELEASE The ports tree is now frozen for the 7.2 release...
Related posts brought to you by Yet Another Related Posts Plugin.
Today I had again some energy to look at why mono fails to build on FreeBSD–current.
I decided to do a debug-build of mono. This did not work initially, I had to produce some patches. 
Does this mean nobody is doing debug builds of mono on FreeBSD?
I have to say, this experience with lang/mono is completely unsatisfying.
Ok, bottom line, either the debug build seems to prevent a race condition in most cases (I had a lot less lockups for each of the two builds I did).
Whatever it is, I do not care ATM (if the configure stuff is looking at the architecture of the system, it may be the case that the i386-portbld-freebsdX does not enable some important stuff which would be enabled when run with i486-portbld-freebsdX or better). Here are the patches I used in case someone is interested (warning, copy&paste converted tabs to spaces, you also have to apply the map.c (a generated file… maybe a touch of the right file would allow to apply this patch in the normal patch stage) related stuff when the build fails, else there is some parser error in mono):
--- mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs.orig 2010-01-29 11:34:00.592323482 +0100
+++ mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs 2010-01-29 11:34:18.540607357 +0100
@@ -57,7 +57,7 @@ namespace Mono.Unix {
int r = Native.Syscall.waitpid (pid, out status,
Native.WaitOptions.WNOHANG | Native.WaitOptions.WUNTRACED);
UnixMarshal.ThrowExceptionForLastErrorIf (r);
- return r;
+ return status;
}
public int ExitCode {
--- mono/io-layer/processes.c.orig 2010-01-29 11:36:08.904331535 +0100
+++ mono/io-layer/processes.c 2010-01-29 11:42:21.819159544 +0100
@@ -160,7 +160,7 @@ static gboolean waitfor_pid (gpointer te
ret = waitpid (process->id, &status, WNOHANG);
} while (errno == EINTR);
- if (ret <= 0) {
+ if (ret == 0 || (ret < 0 && errno != ECHILD)) {
/* Process not ready for wait */
#ifdef DEBUG
g_message ("%s: Process %d not ready for waiting for: %s",
@@ -169,6 +169,17 @@ static gboolean waitfor_pid (gpointer te
return (FALSE);
}
+
+ if (ret < 0 && errno == ECHILD) {
+#ifdef DEBUG
+ g_message ("%s: Process %d does not exist (anymore)", __func__,
+ process->id);
+#endif
+ /* Faking the return status. I do not know if it is correct
+ * to assume a successful exit.
+ */
+ status = 0;
+ }
#ifdef DEBUG
g_message ("%s: Process %d finished", __func__, ret);
--- mono/metadata/mempool.c.orig 2010-01-29 11:58:16.871052861 +0100
+++ mono/metadata/mempool.c 2010-01-29 12:30:45.143367454 +0100
@@ -212,12 +212,14 @@ mono_backtrace (int size)
EnterCriticalSection (&mempool_tracing_lock);
g_print ("Allocating %d bytesn", size);
+#if defined(HAVE_BACKTRACE_SYMBOLS)
symbols = backtrace (array, BACKTRACE_DEPTH);
names = backtrace_symbols (array, symbols);
for (i = 1; i < symbols; ++i) {
g_print ("t%sn", names [i]);
}
free (names);
+#endif
LeaveCriticalSection (&mempool_tracing_lock);
}
--- mono/metadata/metadata.c.orig 2010-01-29 11:59:38.552316989 +0100
+++ mono/metadata/metadata.c 2010-01-29 12:00:43.957337476 +0100
@@ -3673,12 +3673,16 @@ mono_backtrace (int limit)
void *array[limit];
char **names;
int i;
+#if defined(HAVE_BACKTRACE_SYMBOLS)
backtrace (array, limit);
names = backtrace_symbols (array, limit);
for (i =0; i < limit; ++i) {
g_print ("t%sn", names [i]);
}
g_free (names);
+#else
+ g_print ("No backtrace available.n");
+#endif
}
#endif
--- support/map.c.orig 2010-01-29 12:05:22.374653708 +0100
+++ support/map.c 2010-01-29 12:10:29.024412452 +0100
@@ -216,7 +216,7 @@
#define _cnm_dump(to_t, from) do {} while (0)
#endif /* def _CNM_DUMP */
-#ifdef DEBUG
+#if defined(DEBUG) && !defined(__FreeBSD__)
#define _cnm_return_val_if_overflow(to_t,from,val) G_STMT_START {
int uns = _cnm_integral_type_is_unsigned (to_t);
gint64 min = (gint64) _cnm_integral_type_min (to_t);
I try to build mono on FreeBSD–current (it is a dependency of some GNOME program). Unfortunately this does not work correctly.
What I see are hangs of the build. If I stop the build when it hangs and restart it, it will continue and succeed to process the build steps a little bit further, but then it hangs again.
If I ktrace the hanging process, I see that there is a call to wait returning with the error message that the child does not exist. Then there is a call to nanosleep.
It looks to me like this process missed some SIGCLD (or is waiting for something which did not exist at all), and a loop is waiting for a child to exit. This loop probably has no proper condition for the fact that there is no such child (anymore). As such it will stay forever in this loop.
So I grepped a litte bit around in mono and found the following code in <mono-src-dir>/mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs:
public void WaitForExit ()
{
int status;
int r;
do {
r = Native.Syscall.waitpid (pid, out status, (Native.WaitOptions) 0);
} while (UnixMarshal.ShouldRetrySyscall (r));
UnixMarshal.ThrowExceptionForLastErrorIf (r);
}
This does look a little bit as it could be related to the problem I see, but ShouldRetrySyscall only returns true if the errno is EINTR. So this looks correct.
I looked a little bit more at this file and it looks like either I do not understand the semantic of this language, or GetProcessStatus does return the returnvalue of the waitpid call instead of the status (which is not what it shall return to my understanding). If I am correct, it can not really detect the status of a process. It would be very bad if such a fundamental thing went unnoticed in mono… which does not put a good light on the unit-tests (if any) or the general testing of mono. For this reason I hope I am wrong.
I did not stop there, as this part does not look like it is the problem. I found the following in mono/io-layer/processes.c:
static gboolean waitfor_pid (gpointer test, gpointer user_data)
{
...
do {
ret = waitpid (process->id, &status, WNOHANG);
} while (errno == EINTR);
if (ret <= 0) {
/* Process not ready for wait */
#ifdef DEBUG
g_message ("%s: Process %d not ready for waiting for: %s",
__func__, process->id, g_strerror (errno));
#endif
return (FALSE);
}
#ifdef DEBUG
g_message ("%s: Process %d finished", __func__, ret);
#endif
process->waited = TRUE;
...
}
And here we have the problem, I think. I changed the (ret <= 0) to (ret == 0 || (ret < 0 && errno != ECHILD)). This will not really give the correct status, but at least it should not block anymore and I should be able to see the difference during the build.
And now after testing, I see a difference, but the problem is still there. The wait with ECHILD is gone in the loop, but there is still some loop with a semaphore operation:
62960 mono CALL clock_gettime(0xd,0xbf9feef8)
62960 mono RET clock_gettime 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL nanosleep(0xbf9fef84,0)
62960 mono RET nanosleep 0
62960 mono CALL clock_gettime(0xd,0xbf9feef8)
62960 mono RET clock_gettime 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL semop(0x20c0000,0xbf9feef6,0x1)
62960 mono RET semop 0
62960 mono CALL nanosleep(0xbf9fef84,0)
OK, there is more going on. I think someone with more knowledge about mono should have a look at this (do not only look at this semop thing, but also look why it loses a child).
