msdos@freebsd

So I finally got through all the relevant sections of McKusick’s book and completed a first pass through MS’ FAT specification (v1.03).  Modeling UFS userspace fsck and UFS geom label detection, I’ve decided that the headers should live in sys/fs/msdosfs, but I’ll be following the naming conventions in structs defined in sys/geom/label/g_label_msdosfs.h.  The reasoning is simple: it’s what the specification uses.

To setup an appropriate branch that lets me edit sys/ and things in sbin/, I made a new directory within the perforce project/soc2007 directory because there didn’t appear to be a straightforward way of respecifying the branch specification (using a g4 branch and a g4 integrate didn’t seem to produce the desired result).

The only problem of decoding all the copies of different structures is that the specification doesn’t cover every format in sys/fs/msdosfs/*.h, but everything is pretty similar to begin with anyway.  We’ll see how I do.

Turns out there’s a design and implementation book for FreeBSD, The Design and Implementation of FreeBSD (amazon link). This is so cool! Thanks to those who helped me find it.

Over the weekend, many suggested that I use 7.0 as the host system so I wouldn’t have to worry about toolchains and such. So I tried doing that today, and, unfortunately for me, qemu page faults to a non-existent page upon startup. I guess this won’t work about this way.

I also realized that before I start making changes I’d better read up on FreeBSD VFS layers. A similar question for VFS information was posted on the freebsd-hacker list a few years back, and the replies came back with several papers on the interface and a book – Design and Implementation of the 4.4 BSD Operating System – which I’ll see if I can borrow from somewhere.

An oddity I stumbled upon: while trying to scope how much it would take to get fsck to call fsck_msdosfs when appropriate (a recommendation from someone that I didn’t document), I found that fsck seems to defer the decision of which filesystem is being read to a couple of libc calls: getfsfile() and getfsspec(). There must be a reason for wanting to implement this in userspace instead of kernelspace, and I wonder what that is.

I also took a look at where the msdosfs structures were declared (a recommendation from Poul-Henning Kamp), and found that his intuitions were correct. Three places where the msdosfs structures are independently declared are:

• sys/geom/label/g_label_msdosfs.h
• sys/fs/msdosfs/*.h
• sbin/fsck_msdosfs/dosfs.h

Back to reading!

So the 7.0 latest Perforce compiles with the GENERIC config and boots in bochs!  That was my goal for today so I’m going to stop and sleep.

Next I’ll try to figure out why my config made it break (I mean, I’ve compiled several 6.2 kernels using similar methods for stripping out features and I didn’t have any problems at all).

Not much as happened in the last 12+ hours. (Except maybe where I hosed my guest’s /usr directory *tree*, which was very fun.) Most of it (some 8 hours) were spent on the following hurdle: I kept getting config(8) version mismatches. I had hoped to use 6.2 as the host and emulating 7.0 as the guest, while using 6.2′s toolchain to compile 7.0 code (if it worked). Turns out there’s a trick step: either just use buildworld to build the toolchain (which I originally avoided because I thought I had to install the binaries into my 6.2 instance) or just build src/usr.sbin/config and install it. Thanks to all those on IRC for your help (cpet, whack-, constant, Darius). I should’ve asked this at 3pm today instead of 11pm.

Now I’m getting a weird error after installing the kernel into the guest’s image:

CPU: Pentium/P55C (Unknown-class CPU)
Origin = "GenuineIntel"  Id = 0x543  Stepping = 3
Features = 0x80013b
panic: CPU class not configured
cpuid = 0
KDB: enter: panic
[thread pid 0 tid 0]
Stopped at   kdb_enter+0x32: leave

I’ve decided to recompile using the GENERIC kernel config in case it’s because I ripped something out I wasn’t supposed to, and also added “machine i386″ to the top of everything. Kind of shooting blind-eye, but whatever; I’m trying everything I can think of to get this working.

Hi! I’m Brian, and I was chosen by the FreeBSD team to do msdosfs cleanup. Although the official start date for coding was three weeks ago, I’ve had other obligations to tend to, all of which took way longer than I thought. I acknowledge I’m several weeks behind, but giving up is the worst thing I can possibly do now.

In preparation for this SoC project, I purchased a VT/Pacifica-capable machine to do my development on, in hopes of being able to parallelize my host compilation instance and my guest testing instance. Unfortunately I jumped the gun and overlooked the fact that FreeBSD on Xen doesn’t work that well with Ubuntu’s packages that have PAE enabled. Either way, I’d have to make a new dom0 kernel every single time I want to boot, and that’s overhead that I don’t have time to pursue.

All the other difficulties I encounterd included fighting what seemed to be a kernel problem (solved by a BIOS flash), insane amounts of unfamiliar configuration (just on the 6.2 side), and much, much more that I can’t remember now. I blame myself for not attempting to use FreeBSD at a earlier time; I was brought up into the Unix world with Linux. I had hoped that I would have familiarized myself with FreeBSD administration and use as soon as I knew I was accepted, but I had to take time out of my precious coding period to do so.

After worrying my mentor (which I’ll learn not to do know) and finally getting in touch with him, I almost ready to really start the project. The things I have left to do include properly setting up cross-compilation environment and installing the latest Perforce into the emulated version. Unfortunately I forgot my Perforce password (I had reformatted that password during my numerous tries to get the host system up and running), and have send a password reset to the perforce admins.

My dev solution, for now, uses the bochs emulator (qemu would never boot any FreeBSD ISO and I fear that it’s a configuration problem on the host installation) and md to mount the hard drive images. Using Konstantin’s environment script and http://sig9.com/articles/freebsd-on-bochs, I’ve gotten the root filesystems to mount through /dev/md*’s.

What I’ll be working on for the rest of today will be to compile stuff correctly into the mounted /dev/md*’s.