Users coming from other systems don't usually know it (and probably don't care, in today's environment of multi-TB drives), but UFS has a rather interesting approach to space allocation which is different from the "usual suspects" like ext2/3, NTFS and FAT(32). Though it sometimes feels "interesting" in the sense of the Chinese proverb, it's time-tested to work.
I'm talking about clustering, blocks and fragments.
At a recent Linux users' gathering I temporarily saved the day when a WRT router was practically bricked, by setting up my netbook (Acer Aspire One) running 8-CURRENT as a wireless access point. It had wired connectivity to the Internet from one side and offered WiFi via its Atheros card on the other side. In between it did NAT and protected the LAN side from the Linux hackers, both with ipfw. Here is how I configured it.
I committed my patch for tools/kerneldoc/subsys. Except for not generating the PDF part, this is now the same config which I use to generate the online version. While writing the commit log I noticed that I did more changes than I thought…
So anyone who wants to generate the Doxygen docs of some FreeBSD kernel subsystems on his own, can do it now. Adding more subsystems is easy, just make a copy of one the the existing Doxyfile-* files — keep the same naming scheme — and change the contents. Everything else is handled automatically.
I also added a link to the FreeBSD wiki. It is not at a prominent place (near the end of the main page), but at least someone can find the link to the my FreeBSD–doxygen page there.
Earlier today, Ken Smith bumped the version of the HEAD sources to 9.0. I have just modified the scripts to make INDEX-8 based on RELENG_8 and add support for INDEX-9 based on HEAD. They are available for ‘make fetchindex’ now. Mark Linimon is working on adding 9 to the pointyhat scripts, so expect the first 9-current packages to be available later this week.
Packages for the release of 8.0 have been built for i386 and amd64, with a few days to go for sparc64. They are currently not available on the mirrors yet, but will be included on the BETA3 cd sets. Please give them as much testing as you can. With the first iterations of the package sets already finished, we are currently in a soft feature freeze, and if no major changes appear, we do expect to keep the freeze very short. Although the exact date is dependent on how the release progress in general progresses, it is currently expected to start at September 14 and last for two weeks. Please don’t hesitate to contact portmgr@ if you have any questions or doubts whether a given change would be too big for the feature freeze.
Related posts:
Related posts brought to you by Yet Another Related Posts Plugin.
Last night the FreeBSD team had unfrozen the development branch for -HEAD (What will eventually lead up to 9.0-RELEASE). Although people need to be conservative about commits they make, they can continue development again. RELENG_8 is still frozen but people are allowed to merge after consulting the Release Engineering Team. Good Progress is being made now to get 8.0 out of the door.
Renato Botelho, aka garga, and I picked up the gauntlet and put together the compat7x port for the upcoming release of FreeBSD 8.
We initially targeted a small audience, and gradually extended our scope. We are now looking for more testers. Those inclined to give it a try are encouraged to download the shar from http://people.freebsd.org/~tabthorpe/patches/misc-compat7x-7.2.702000.200908.shar
Please note that the standard disclaimers apply.
Any feedback is appreciated to garga@ and/or tabthorpe@
In this post I write how to install FreeBSD on a remote linux system by creating a root image prepared for mirroring on a local system. There is no (free) access to the remote console, only access via ssh.
While I was at the university, I worked remotely for an ISP (administration of several FreeBSD systems). During this time I started to use my own domain, and I was allowed to host it directly at the ISP for free. I tried to not use too much space on the harddisk (at the end about 400 MB) and to not provide something which attracted too much people to keep the bandwith on a sane level. After the university I was still available to answer questions, and I was still allowed to host my website there for free. As the number of questions can be counted with IIRC one hand since then, I decided at some point (recently, to be exact — better late than never) to move my website to a different place (I am still available if they have some questions — for free — but I do not expect to get much questions from them).
At the same time my brother decided to move to a new server hoster, as his old one decided to increase the amount of money customers have to pay for the service. So we searched together a new ISP where he either could host his server, or get a rootserver for a good price (the idea was to have my domain in a jail on a server of the company of my brother, and we share the costs for it). We found Manitu (the owner has even a blog about his business), which is even not far away from my place.
Unfortunately they do not provide FreeBSD preinstalled on their rootservers, but they offer a remote rescue system (read: booting a linux image… I assume via PXE or similar) and we knew someone who has some servers there, so I was able to get a rough idea what kind of hardware is used there (the hard facts like PCI IDs and such). The idea was to build a very small disk image, put it on the harddisk over the network via the remote rescue system, and then to configure the remainder of the harddisk(s) to use it. And here is how I did it (my brother thought “who is better suited to install a FreeBSD system remotely without access to the console of the machine (the ISP offers to hook up a KVM switch, but only during business hours and you have to pay for it) than one of the developers of FreeBSD…”).
In the title of this post I wrote “via a tiny disk image”. This is true for a suitable definition of tiny.
What we have in the rootserver are two 160 GB harddisks. They shall be used in a software mirror (via gmirror). The root-FS shall have about 5 GB. This is more than needed, but as this is only 3% of one harddisk, I prefer to keep it a little bit bigger than too small after an remote update or two. The machine has 2 GB of RAM. We do not expect much kernel panics (= crash dumps) there, so we do not really need >2 GB of swap (forget the rule of having twice as much swap than RAM, with the current amount of RAM in a machine you are in “trouble” when you need even the same amount of swap than RAM). I decided to go with 1 GB of swap (mirrored too, to prevent a harddisk failure to take down the machine), this is more than enough. The rest of the harddisk will be used for jails, the distfiles/packages for/of the ports, and as WRKDIRPREFIX when building ports.
Now, pushing/pulling a 160 GB image over the network to install a system is not really something I want to do. I would prefer to transfer less than 500 MB (that is 0.3% of the entire disk) to get this job done, and this is feasible. Due to an error or two I had to transfer the image several times until everything was working, so it was more in the area of maybe 2 GB (~1% of the entire disk).
First let us define some variables in the shell, this way you just need to change the values in one place and the rest is copy&paste:
ROOTFS_SIZE=5G
ROOTFS_NAME=root0
FILENAME=rootfs
Then change your current directory to a place where you have enough space for the image. There we will create a container for the image, and make it ready for partitioning:
truncate -s ${ROOTFS_SIZE} ${FILENAME}
mdconfig -a -t vnode -f ${FILENAME}
Create the rootfs:
# create one active FreeBSD slice with $ROOTFS_SIZE GB fdisk -e /dev/mdX gmirror label ${ROOTFS_NAME} /dev/mdXsY bsdlabel -w /dev/mirror/${ROOTFS_NAME} # create an "a"-partition for everything bsdlabel -e/dev/mirror/${ROOTFS_NAME} newfs -U /dev/mirror/${ROOTFS_NAME}a
Mount the new rootfs to /mnt and install FreeBSD:
mount /dev/mirror/${ROOTFS_NAME}a /mnt
cd /usr/src
make buildworld >&! buildworld.log
make buildkernel >&! build_generic.log
make installworld DESTDIR=/mnt
make distribution DESTDIR=/mnt
make installkernel DESTDIR=/mnt
Now you need to create /mnt/etc/rc.conf (set the defaultrouter, the IP address via ifconfig_IF (and do not forget to use the right IF for it), the hostname, set sshd_enable to yes, add an user (I used “vipw –d /mnt/etc”) which is in the wheel group so you can login remotely, and maybe other things you are interested in), /mnt/etc/resolv.conf, /mnt/etc/hosts. Finally, do not forget to load the gmirror module, it will safe a lot of head-scratching (yes, an echo would be shorter, but WP converts the single-quotes to double-quotes), and add the rootfs to the fstab:
cat > /mnt/boot/loader.conf <<EOT
geom_mirror_load="YES"
EOT
echo "/dev/mirror/root0a / ufs rw,noatime 1 1" >/mnt/etc/fstab
Now we are ready to install, the image can be unmounted:
umount /mnt
The final steps are to login into the rescue console (theoretically you should be able to overwrite even a running system, but then you need to make sure there is a way to power-cycle the system remotely to force a reboot) of the new system and to install via a compressed ssh connection (my remote rescue system is a linux system, so linux-syntax has to be used). The login to the rescue console is not shown here, but the install from the remote system is simple (this assumes the image resides on a system which is accessible from the new system):
ssh -C -o CompressionLevel=9 user@myhost cat /path/to/${FILENAME} | dd of=/dev/hda bs=1m
Alternatively you can compress it with bzip2 on your system and add an bunzip2 into the pipe above. This way you could even use an HTTP server to fetch the file from, but then the command to fetch the file needs to have a way to output the file it downloads on stdout. Both ways of transferring the image depend upon the stability of the transfer. If the connection is cut, the system can not boot anymore. So if you do not have a rescue console which is independent from the content of the harddisk, you better have a plan B.
If I did not make an error here (I did this some months ago, I hope I did not forget to write down some important step and also corrected all steps which where not written down correctly), you did everything correctly too, and the remote system does not need other kernel modules loaded, you can now reboot the new system, cross your fingers for some moments, and then login to the new system.
Now you should have a basic system up and running, and you can start to configure it.
I added two more FreeBSD partitions, 1 GB for swap, and the rest of the harddisk (I took care to not have the last partition cover also the last sector of the harddisk, else gmirror will think it has to mirror the entire harddisk and not only the last partition) as one big partition. For the second harddisk I made the same partitioning as for the first harddisk.
Then I created two more gmirrors, one mirror for the swap, and one for the rest of the space. The mirror for the swap I created with the option “-F”, to not synchronize the partition after a power failure (not necessary for swap). All two I also created with the “-n” option, to not sync the contents (there is nothing yet, so it does not matter what is written there).
Now just a quick “bsdlabel” on the two mirrors to create a “b”-partition for the swap and a “d”-partition for the large partition. To add the swap it is just an “echo /dev/mirror/swap0b none swap sw 0 0 >/etc/fstab”. Do not forget to add the big partition to the fstab (after doing a “newfs” off course).
To be sure it will survive a reboot, do a quick test of manually mounting the big partition (use the easy manual way via “mount /place/where/it/is/mounted” after adding it to the fstab, not the complete manual way, this is meant to test the fstab entry too), and a “swapon –a” to test the swap. I also made a reboot test, just to be sure everything is ok.
The final step was to add the partition on the second harddisk to the rootfs (“gmirror insert ${ROOTFS_NAME} /dev/…”).
Believe it or not, FreeBSD as a whole is a really big project. The practice of keeping kernel and userland together in an operating system results in about 45,000 files that comprise the system's source. As with other big projects, some degree of control is needed.
(This article is intended for end-users, not developers; developers have a better inside view so they will have to ovelook omitted details)
I managed to get some time to setup an automated generation of the doxygen docs for kernel subsystems of FreeBSD on my webserver.
Every night/morning (German timezone) the sources will be updated, and the docs get regenerated (this takes some time). Currently this depends upon some patches to the makefile and doxygen config files in tools/kerneldoc/subsys. Everything is generated directly in the place where the webserver will look for to deliver the pages, so if you browse this in the middle of the generation, the content may not be consistent (yet).
Please be nice to the webserver and do not mirror this. You can generate this yourself very easy. Assuming you have the FreeBSD source on a local hard disk, you just need to download the patch from http://www.Leidinger.net/FreeBSD/current-patches/ (if you do not find dox.diff, update your FreeBSD sources and everything will be OK), apply the patch, cd into tools/kerneldoc/subsys and run “make all” (or “make vm” or whatever you are interested in). You need doxygen installed, off course.
If you want to setup something like this yourself, just download the script which is doing all the work, change some variables in the beginning, and create your own local version of the complete docs.
In case this is using significant traffic, I will ask core/admins if there is the possibility to host it on FreeBSD.org resources.
I don't know exactly when it happened but there is currently an OpenJDK 6 binary port available for download from the usual place. The reason this is important is because the "official" Diablo/Caffe binaries distributed by the Foundation cannot be legally redistributed by third parties, precluding it for being included in FreeBSD-based products, LiveCD-s, etc.
Today RELENG_8 had been opened up again. Reason for stalled commits and such is that the SVN to CVS exporter was unable to cope with the new branch and didn’t export anymore. To prevent massive surgeries needed to get the system backup again, we stopped the exporter and fixed it. Simon Nielsen and Peter Wemm did a lot of work to get this in the proper position again.
Unfortunatly this stalled development and working towards RELENG_8 a bit. But headsup, things are now getting back to normal and the queue by re@ is being flushed.
Stay tuned!
I will be mentoring Dima along with Martin Wilke. Dima was instrumental working behind the scenes in preparing the FreeBSD port of KDE 4.3 and as a result we are punishing him with a commit bit.
Dima’s new handle is fluffy@FreeBSD.org.
Just remember Dima, you are not a real committer until you break the INDEX, then fix it!
The prebuilt FreeBSD Documentation (such as the FreeBSD Handbook etc) is now again available via FTP at ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.
The system will resume regular builds with weekly Sunday uploads within a couple of days, but I uploaded a new documentation set today. It probably hasn’t hit the FTP mirrors yet, but it should be available on most mirrors within 24-48 hours.
While updating the build system I also disabled the Palm book versions – I simply don’t believe they are used enough anymore to justify mirroring them every week to all the mirrors. If anybody misses the Palm precompiled versions, let me know and I will reconsider if they should be built.
