Category Archives: Sysadmin

Official Vagrant FreeBSD Images

I am very proud to announce that FreeBSD Vagrant images are now available.

Usage:
For VMWare, create a Vagrantfile like so:

Vagrant.configure("2") do |config|
  config.vm.synced_folder ".", "/vagrant", id: "vagrant-root", disabled: true
  config.vm.box = "freebsd/FreeBSD-11.0-CURRENT"
  config.ssh.shell = "sh"
end

For VirtualBox, create a Vagrantfile like:

Vagrant.configure("2") do |config|
  config.vm.synced_folder ".", "/vagrant", id: "vagrant-root", disabled: true
  config.vm.box = "freebsd/FreeBSD-11.0-CURRENT"
  config.ssh.shell = "sh"
  config.vm.base_mac = "080027D14C66"
end

Then run:
vagrant up

On first boot the machine will come up and install missing pkgs and run freebsd-update if needed. Note that this can take a few minutes. If it fails to boot try using: vagrant up --no-destroy-on-error. On my 2004 iMac with a spinning disk it takes just over 3 minutes. On my mid 2014 MBP with a SSD it takes about 1 minute and 45 seconds. In the future we will reevaluate installing the missing packages on boot vs when the VM is built.

Note that you can replace `FreeBSD-11.0-CURRENT’ with `FreeBSD-10.0-RC2′ or others. To see a full list of versions available, check the Hashicorp Atlas website here: https://atlas.hashicorp.com/FreeBSD/

Going forward:

  • All snapshots will include Vagrant images, so weekly updates of FreeBSD -STABLE branches and -CURRENT.
  • All future releases will including Vagrant images.

Fan Speed monitoring

Recently I moved a server into a proper cabinet with doors. After a few days I noticed the fans were spinning up and down. So I started investigating ways to monitor the fan speed. I figured having a graph of them long term would give me a nice way to show changes in the environment, beyond the temperature monitoring I am already doing.

I was not having much luck searching the Internet. Luckily, Darius on IRC pointed me to a project called bsdhwmon by Jeremy Chadwick, a fellow FreeBSD Developer. The server is running an older Supermicro X7SBi motherboard with a Winbond 83627HG chip which is listed on the supported page of bsdhwmon.

It was easy to setup:

  • Install bsdhwmon: pkg install bsdhwmon
  • Load the SMBus Controller driver for my motherboard: kldload ichsmb
  • Load the Generic SMB I/O Device driver: kldload smb

All I had to do from that point was run bsdhwmon:
# bsdhwmon
CPU1 Temperature 46 C
System Temperature 29 C
FAN1 10975 RPM
FAN2 11344 RPM
FAN3 7219 RPM
FAN4 7068 RPM
FAN5 0 RPM
FAN6 11065 RPM
VcoreA 1.122 V
MCH Core 1.508 V
-12V -12.672 V
V_DIMM 1.808 V
+3.3V 3.296 V
+12V 11.904 V
5Vsb 5.046 V
5VDD 4.998 V
P_VTT 1.228 V
Vbat 3.312 V

It is important to remember to add the kernel modules to be loaded at boot. Adding the following to /boot/loader.conf will take care of that:
smb_load="YES"
ichsmb_load="YES"

Note that ichsmb will load smbus, but not the smb kernel driver.

Now that I have the tools, I can monitor it at will.

FreeBSD + Packer = Vagrant

So I recently discovered a tool to build Vagrant images called Packer. It allows you to script the install via key presses over VNC to automate the install of any OS. I am running on a rather fast machine (Core i7, 16GB of RAM, SSD), so I suspect there might be some lurking problems for people on slower machines due to timing of the commands.

Everything is available from my Github repo: https://github.com/so14k/packer-freebsd

To get started:

  • Install Vagrant and Packer
  • Clone the repo onto your machine
  • Build the Vagrant box: packer build template.json
  • Wait while it builds..
  • Start the Vagrant box: vagrant up
  • Start hacking: vagrant ssh

Give it a spin and let me know what you think!

Puppet + pkgng/poudriere

First thing we will need a clone of https://github.com/xaque208/puppet-pkgng into /usr/local/etc/puppet/modules/.

This will be pushed out to the clients as long as: pluginsync = true

For me the next step is to create a manifests/init.pp in the new module directory. This is important to me because I want to sync out a /usr/local/etc/pkg.conf to all my machines so that they point to my internal poudriere repos. So I end up with something like this:

file { "/usr/local/etc/pkg.conf":
        mode => 755,
        owner => root,
        content => "packagesite: http://pkg/91-web/
",
}

Once that is done it is easy to use pkgng packages via:

package { "www/apache22":
        ensure => installed,
        provider => pkgng,
        require => File['/usr/local/etc/pkg.conf'],
}

Patching screens EDID information

Quite a long time ago (September 2012), the x11/nvidia-driver port was updated to 304.43, which brought-in some major changes. The most noticeable one from my point of view was the inability to use my two Samsung SyncMaster BX2240 monitors at a resolution better than 800x600 (they have a native resolution of 1920x1080), providing a stunning 1600x600 desktop experience. With no time to dig-in this at that time, I reverted to the previous nvidia-driver and planned to have a look at it later.

I finally took the time to search for a solution. I post it here for two reasons: a) I might need this information at some point in the future and am more likely to find it that way and b) I might not be the only one here which faced this situation.

The first step was to modify my Xorg configuration to be a bit more verbose about the process of choosing a screen resolution when X starts to see if something was different with both drivers. This can be achieved by editing the Monitor section:

Section "Monitor"
    ...
    Option "ModeDebug" "TRUE"
EndSection

With the new driver, no section for Validating Mode "1920x1080", but when trying to validate other modes, the driver reported that it would be happier with EDID information from the screens:

217 (II) Feb 07 09:52:30 NVIDIA(GPU-0):   Validating Mode "640x350":
218 (II) Feb 07 09:52:30 NVIDIA(GPU-0):     640 x 350 @ 85 Hz
219 (II) Feb 07 09:52:30 NVIDIA(GPU-0):     Mode Source: X Server
220 (II) Feb 07 09:52:30 NVIDIA(GPU-0):       Pixel Clock      : 31.50 MHz
221 (II) Feb 07 09:52:30 NVIDIA(GPU-0):       HRes, HSyncStart :  640,  672
222 (II) Feb 07 09:52:30 NVIDIA(GPU-0):       HSyncEnd, HTotal :  736,  832
223 (II) Feb 07 09:52:30 NVIDIA(GPU-0):       VRes, VSyncStart :  350,  382
224 (II) Feb 07 09:52:30 NVIDIA(GPU-0):       VSyncEnd, VTotal :  385,  445
225 (II) Feb 07 09:52:30 NVIDIA(GPU-0):       H/V Polarity     : +/-
226 (WW) Feb 07 09:52:30 NVIDIA(GPU-0):     Mode is rejected: Only EDID-provided modes are allowed on
227 (WW) Feb 07 09:52:30 NVIDIA(GPU-0):     DFP-0.

I enabled EDID which was previously disabled because the screen reported invalid information:

Section "Device"
    ...
    #Option "IgnoreEDIDChecksum" "DFP-0, DFP-1"
EndSection

With the new driver, a Validating Mode "1920x1080" section appeared but with the very same Mode is rejected: Only EDID-provided modes are allowed on DFP-0. The old driver provided a bit more of information actually:

1822 (II) Feb 07 09:54:10 NVIDIA(GPU-0):   Validating Mode "1920x1080":
1823 (II) Feb 07 09:54:10 NVIDIA(GPU-0):     1920 x 1080 @ 50 Hz
1824 (II) Feb 07 09:54:10 NVIDIA(GPU-0):     Mode Source: EDID
1825 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Pixel Clock      : 148.50 MHz
1826 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       HRes, HSyncStart : 1920, 2448
1827 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       HSyncEnd, HTotal : 2492, 2640
1828 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       VRes, VSyncStart : 1080, 1084
1829 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       VSyncEnd, VTotal : 1089, 1125
1830 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       H/V Polarity     : +/+
1831 (WW) Feb 07 09:54:10 NVIDIA(GPU-0): The EDID for Samsung SMBX2240 (DFP-1) contradicts itself: mode
1832 (WW) Feb 07 09:54:10 NVIDIA(GPU-0):     "1920x1080" is specified in the EDID; however, the EDID's
1833 (WW) Feb 07 09:54:10 NVIDIA(GPU-0):     valid VertRefresh range (56.000-75.000 Hz) would exclude
1834 (WW) Feb 07 09:54:10 NVIDIA(GPU-0):     this mode's VertRefresh (50.0 Hz); ignoring VertRefresh
1835 (WW) Feb 07 09:54:10 NVIDIA(GPU-0):     check for mode "1920x1080".
1836 (II) Feb 07 09:54:10 NVIDIA(GPU-0):     Viewport                 1920x1080+360+22
1837 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Horizontal Taps        0
1838 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Vertical Taps          0
1839 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Base SuperSample       x4
1840 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Base Depth             32
1841 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Distributed Rendering  1
1842 (II) Feb 07 09:54:10 NVIDIA(GPU-0):       Overlay Depth          32
1843 (II) Feb 07 09:54:10 NVIDIA(GPU-0):     Mode is valid.

Okay, since all this seems to be related to EDID, and because some major changes where introduced in version 302.xx of the NVidia driver, let's try to patch it! I could not find a tool to dump the EDID directly from FreeBSD, however when Xorg starts, it prints a hex dump of it so it is actually quite trivial to get it from the log file (don't want to code? Clone!).

sysutils/edid-decode can be used to easily locate relevant information from the EDID dump (monitor range), and spot inconsistencies:

Extracted contents:
header:          00 ff ff ff ff ff ff 00
serial number:   4c 2d 84 06 32 32 42 43 0e 15
version:         01 03
basic params:    80 30 1b 78 2a
chroma info:     78 f1 a6 55 48 9b 26 12 50 54
established:     bf ef 80
standard:        71 4f 81 00 81 40 81 80 95 00 b3 00 a9 40 95 0f
descriptor 1:    02 3a 80 18 71 38 2d 40 58 2c 45 00 dd 0c 11 00 00 1e
descriptor 2:    00 00 00 fd 00 38 4b 1e 51 11 00 0a 20 20 20 20 20 20
descriptor 3:    00 00 00 fc 00 53 4d 42 58 32 32 34 30 0a 20 20 20 20
descriptor 4:    00 00 00 ff 00 48 39 58 42 34 30 31 30 32 34 0a 20 20
extensions:      01
checksum:        b0

Manufacturer: SAM Model 684 Serial Number 1128411698
Made week 14 of 2011
EDID version: 1.3
Digital display
Maximum image size: 48 cm x 27 cm
Gamma: 2.20
DPMS levels: Off
Supported color formats: RGB 4:4:4, YCrCb 4:2:2
First detailed timing is preferred timing
Established timings supported:
720x400@70Hz
640x480@60Hz
640x480@67Hz
640x480@72Hz
640x480@75Hz
800x600@56Hz
800x600@60Hz
800x600@72Hz
800x600@75Hz
832x624@75Hz
1024x768@60Hz
1024x768@70Hz
1024x768@75Hz
1280x1024@75Hz
1152x870@75Hz
Standard timings supported:
1152x864@75Hz
1280x800@60Hz
1280x960@60Hz
1280x1024@60Hz
1440x900@60Hz
1680x1050@60Hz
1600x1200@60Hz
1440x900@75Hz
Detailed mode: Clock 148.500 MHz, 477 mm x 268 mm
1920 2008 2052 2200 hborder 0
1080 1084 1089 1125 vborder 0
+hsync +vsync
Monitor ranges (GTF): 56-75Hz V, 30-81kHz H, max dotclock 170MHz
Monitor name: SMBX2240
Serial number: H9XB401024
Has 1 extension blocks
Checksum: 0xb0 (valid)

CEA extension block
Extension version: 1
0 8-byte timing descriptors
Detailed mode: Clock 148.500 MHz, 477 mm x 268 mm
1920 2448 2492 2640 hborder 0
1080 1084 1089 1125 vborder 0
+hsync +vsync
Detailed mode: Clock 74.250 MHz, 477 mm x 268 mm
1280 1390 1430 1650 hborder 0
720  725  730  750 vborder 0
+hsync +vsync
Detailed mode: Clock 74.250 MHz, 277 mm x 286 mm
1280 1720 1760 1980 hborder 0
720  725  730  750 vborder 0
+hsync +vsync
Detailed mode: Clock 27.000 MHz, 477 mm x 286 mm
720  732  796  864 hborder 0
576  581  586  625 vborder 0
-hsync -vsync
Detailed mode: Clock 27.000 MHz, 477 mm x 286 mm
720  736  798  858 hborder 0
480  489  495  525 vborder 0
-hsync -vsync
Checksum: 0x0 (should be 0x99)

EDID block does not conform at all!
Block has broken checksum

After changing the lower limit of the vertical refresh rate from 56 Hz (0x38) to 50 Hz (0x32) and adjusting the checksums, it's time to see if the situation is improved. Once more we have to tweak xorg.conf to tell Xorg to use our dumps instead of querying the screens for EDID information:

Section "Device"
    ...
    Option "CustomEDID" "DFP-0:/usr/local/etc/X11/EDID-BX2240-FIXED; DFP-1:/usr/local/etc/X11/EDID-BX2240-FIXED"
EndSection

And suddenly, it worked!

February 11th, 2013 edit

I wrote an e-mail to the screen manufacturer (Samsung) asking for assistance. I will publish their response on this blog as soon as I get one. The sent message is basically:

Hi,

I own two BX2240 screens. These screens provide invalid EDID information (I wrote an article [1] about that) that makes them unusable with the prorietary NVidia driver version 302.xx and up. It is possible to workaround the problem at the OS level (I provide some instructions in my article) however it would be better to put valid information back into the screen so that they work out of the box.

However, you don't provide any information about this on your website. Can you please send me a technical notice providing information on flashing EDID information if it's possible, and if not, the location of the EEPROM containing the wrong EDID information so that I can replace it and continue to use my screens in proper conditions?

Please find attached to this message the fixed EDID information I use at the moment and want to flash back into my screens.

Kind regards,
Romain Tartière

References:

  1. http://romain.blogreen.org/blog/2013/02/patching-screens-edid-information/

The Importance of Serial Console

I have long been a huge fan of having serial console on my servers–it can really save the day when a mistake is made. Yesterday, one of my coworkers botched the sshd_config in an upgrade of a server, so the server came up fine, but without sshd. As a result, the system was not accessible for remote login via the network.

Over the years, I have done serial console in many ways. I began with a single null modem cable between the back of two servers. Next, I utilized a RocketPort multi-port serial card with 8 serial ports on it. These days, I have moved on to employing big serial console servers such as those made by OpenGear, providing up to 48 ports. They also have ancillary features such as providing a Nagios platform and Environmental monitoring.

No matter your physical connectivity, I recommend using Conserver. This helps by logging what is happening on the console, which can be very handy if you need to see what happened in the past whether it be a function of the system, or to see who did what. It also provides multi-user access, so you can watch while someone else is working and both of you can collaborate on fixing a problem.

In order for the previous technologies to be useful, the servers require configuration as well. The first step is to configure the BIOS for serial console redirection. Once this has been performed, the OS will need to be configured to present a console login via the serial port. The FreeBSD Handbook explains how to do this Here.

Book Review: FreeBSD Device Drivers

Joseph Kong strikes again! When I saw Designing BSD Rootkits I was quite excited because:

  1. it was on some topics I had interests in but had not as much time as I would have liked to to dig in;
  2. it was focussed on my main operating system.

I bought the book and although it was rather small (~140 pages) enjoyed learning quite a large amount of things about the FreeBSD kernel.

So, when I was contacted by No Stash Press to know if I would be interested in a review copy of their upcoming book FreeBSD Device Drivers by the same author in exchange of writing what I though about it, I had no hesitation and accepted the deal.

Book cover

Just like the preceding book, this one is based on examples, each chapter focussing on a different aspect of device drivers, and going progressively further and further in the details of the kernel, explaining how things work and why they do work that way so that the reader can better understand the involved mechanisms, even if he is not an Operating Systems guru.

Precious information about the FreeBSD kernel (like what you have in The Design and Implementation of the FreeBSD Operating-System but in a more up-to-date and friendlier fashion) are spread all along the book, and — apart from the long C listings — the style is quite pleasant to read.

To sum-up, this book is definitively a must have for anybody interested in how are designed FreeBSD device drivers, not mentioning those who are interested in writing their very own ones for the FreeBSD operating system!

Many thanks Joseph for this awesome book!

Updating a ZFS Mirror

A few days ago, while I was on the phone, my machine experienced a kernel panic. The backtrace pointed a problem somewhere in the swap management code. I was on a hurry at that time and rebooted the machine without taking the time dig in the problem deeper.

On the next day, I eventually realised that an hard disk was logically missing on the system and the ZFS mirror it was belonging to was working in a degraded mode. This disk holding a swap partition, the panic quite makes sense: some data was stored there and could not be paged-out anymore.

# zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
data   294G  68,2G   226G    23%  1.25x  DEGRADED  -
tank  1,81T   300G  1,52T    16%  1.06x  ONLINE  -
# zpool status data
  pool: data
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://www.sun.com/msg/ZFS-8000-2Q
  scan: none requested
config:

NAME                                            STATE     READ WRITE CKSUM
data                                            DEGRADED     0     0     0
  mirror-0                                      DEGRADED     0     0     0
    gptid/36711e52-a69e-11de-8adf-0018f38af467  ONLINE       0     0     0
    15152536002702365387                        UNAVAIL      0     0     0  was /dev/gptid/602da1ae-c474-11de-960d-0008a14dbca1

errors: No known data errors

This disk already had problems in the past and I even had to improve sysutils/smartmontools periodic script to take into account SMART attributes that have been failing at some point in the past but have recovered since that time on that disk. This time, the disk is really dead, so no other choice than changing it. Hopefully, I have a brunch of spare heard disks on the shelve, so I took two 500 GB disks to replace the two 320 GB of the degraded ZFS pool.

I first replaced the broken disk with a new one, identified it using geom(8) and partitioned using basically the same settings I used when installing FreeBSD on full-ZFS:

# geom disk list ada1
Geom name: ada1
Providers:
1. Name: ada1
   Mediasize: 500107862016 (465G)
   Sectorsize: 512
   Mode: r2w2e3
   descr: ST3500418AS
   ident: (null)
   fwsectors: 63
   fwheads: 16

# geom part create -s GPT ada1
ada1 created
# geom part add -s 128 -t freebsd-boot ada1
ada1p1 added
# geom part add -s 4G -t freebsd-swap ada1
ada1p2 added
# geom part add -t freebsd-zfs ada1
ada1p3 added
# geom part show ada1
=>       34  976773101  ada1  GPT  (465G)
         34        128     1  freebsd-boot  (64k)
        162    8388608     2  freebsd-swap  (4.0G)
    8388770  968384365     3  freebsd-zfs  (461G)

# geom part bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
bootcode written to ada1

I then replaced the unavailable ZFS partition with the new one:

# zpool replace data 15152536002702365387 ada1p3
Make sure to wait until resilver is done before rebooting.

If you boot from pool 'data', you may need to update
boot code on newly attached disk 'ada1p3'.

Assuming you use GPT partitioning and 'da0' is your new boot disk
you may use the following command:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

… waited a few moments for resilvering to finish:

% zpool status data
  pool: data
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Mon Jan  2 19:26:44 2012
    330M scanned out of 68,2G at 4,07M/s, 4h44m to go
    329M resilvered, 0,47% done
config:

	NAME                                            STATE     READ WRITE CKSUM
	data                                            DEGRADED     0     0     0
	  mirror-0                                      DEGRADED     0     0     0
	    gptid/36711e52-a69e-11de-8adf-0018f38af467  ONLINE       0     0     0
	    replacing-1                                 UNAVAIL      0     0     0
	      15152536002702365387                      UNAVAIL      0     0     0  was /dev/gptid/602da1ae-c474-11de-960d-0008a14dbca1
	      ada1p3                                    ONLINE       0     0     0  (resilvering)

errors: No known data errors

… then shut the system down, replaced the working 320 GB disk by a new 500 GB one and booted into FreeBSD again:

% zpool status data
  pool: data
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
	the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://www.sun.com/msg/ZFS-8000-2Q
  scan: resilvered 68,2G in 1h58m with 0 errors on Mon Jan  2 21:25:42 2012
config:

	NAME                     STATE     READ WRITE CKSUM
	data                     DEGRADED     0     0     0
	  mirror-0               DEGRADED     0     0     0
	  3635039039460500206    UNAVAIL      0     0     0  was /dev/gptid/36711e52-a69e-11de-8adf-0018f38af467
	    ada1p3               ONLINE       0     0     0

errors: No known data errors

Same story with the other disk:

geom part create -s GPT ada0
ada0 created
# geom part add -s 128 -t freebsd-boot ada0
ada0p1 added
# geom part add -s 4G -t freebsd-swap ada0
ada0p2 added
# geom part add -t freebsd-zfs ada0
ada0p3 added
# geom part show ada0
=>       34  976773101  ada0  GPT  (465G)
         34        128     1  freebsd-boot  (64k)
        162    8388608     2  freebsd-swap  (4.0G)
    8388770  968384365     3  freebsd-zfs  (461G)

# geom part bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
bootcode written to ada0
# zpool replace data 3635039039460500206 ada0p3
Make sure to wait until resilver is done before rebooting.

If you boot from pool 'data', you may need to update
boot code on newly attached disk 'ada0p3'.

Assuming you use GPT partitioning and 'da0' is your new boot disk
you may use the following command:

	gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

When done, I saw that the available space on the data zpool was still the same:

# zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
data   294G  65,3G   229G    22%  1.19x  ONLINE  -
tank  1,81T   300G  1,52T    16%  1.08x  ONLINE  -

This is due to the autoexpand property set to off by default and that should be set to on before replacing disks if this feature is desired.

# zpool set autoexpand=on data

Hopefully, it is possible to use the zpool(8)'s online command to make ZFS take into account the extra space available for the pools:

# zpool online -e data ada0p3
# zpool list
NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
data   460G  65,3G   395G    14%  1.20x  ONLINE  -
tank  1,81T   300G  1,52T    16%  1.06x  ONLINE  -

ZFS: unsupported ZFS version 14 (should be 13)

How I got there?

After updating to the latest development version of GNOME thanks' to MarcusCom ports I wanted to log out and back in but X refused to restart because of the in-heavy-development Nouveau video driver. I run in this problem every once a while and a full reboot solve this problem. However, the system did not boot:

ZFS: unsupported ZFS version 14 (should be 13)
ZFS: unsupported ZFS version 14 (should be 13)
NO ZFS pools located? can't boot

Okay, I recall running the following to update my full-ZFS system from ZFS 13 to 14 a while ago:

zpool upgrade -a

I completly forgot about updating the GPT bootloader accordingly and thus it was unable to use the version of ZFS on my disks.

The FreeBSD 8.0-STABLE livefs

The FreeBSD projects provides snapshots of -STABLE version of the Operating System (I am running FreeBSD 8.0-STABLE) and the livefs disk contains the fixit environement which would allow me to solve my problems.

I so used my laptop to look for such an ISO image and burn it, unfortunately, the bootp and dns servers are run by the machine that was down. Hopefully, I had a web browser oppenned on this system and it had the address of google's servers in it's cache. I could search for my ISP DNS and add them to /etc/resolv.conf. I was then able to download the image and burn it.

Fixing

By default, the livefs will not have ZFS support unless you ask for it before starting. In other word, at the boot menu, Escape to loader prompt and type:

OK load zfs
OK boot

When the sysinstall(8) menu appears, select Fixit / CDROM/DVD. At the Fixit# prompt, you will have to search for your zfs pools, mount your filesystems, locate the new version of the bootloader and install it. This is basically a transcript of what I did (I previously described my setup in this blog):

Fixit# zpool import -f data # at this point I had to leave and return in the Fixit environement
                            # because by ZFS root was hiding the livefs root.
Fixit# zfs set mountpoint=/mnt data
Fixit# gpart bootcode -p /mnt/boot/gptzfsboot -i 1 ad10
Fixit# gpart bootcode -p /mnt/boot/gptzfsboot -i 1 ad12
Fixit# zfs set mountpoint=legacy data
Fixit# zfs umount -a
Fixit# logout

Note to myself: in the future do not forget to update the bootloader when updating my ZFS filesystems.

OpenRD console access on FreeBSD.

This morning, I received the Open-RD I bought a few days ago and started playing with my new device. One of the first thing I wanted to access was (of course) the system console. While the default setup provides a SSH daemon and even GDM and a full desktop, my goal is to have FreeBSD on this device and move the services I run at home from my personal computer to this low-consumption computer (and I don't intend to switch to GNU/Linux).

Since the console is usually available on the serial port, I looked for some cable to connect the OpenRD to my computer. Unfortunately, I didn't have the required cable. I then spent about two hour with various serial cables, cables looking like serial cables but which where not serial cables, parallel SUB-D25 connectors, a serial SUB-D25 cross-over cable and a soldering iron to build a null-modem SUB-D9 serial cable. Unfortunately, when I connected my computer to the OpenRD, I was still not able to access the console using cu(1).

After reading a bit of documentation, I realised I missed the fact that the console was available on USB and not via the serial port. However, FreeBSD did not recognised the chip the OpenRD uses to provide serial over USB support.

Using usbconfig(8), I could get the vendor ID and product ID the uftdi(4) driver was lacking. I patches the source code and a few minutes later could enjoy a set of new devices when I plug the OpenRD on my computer:

% ls /dev/cuaU*
/dev/cuaU0
/dev/cuaU0.init
/dev/cuaU0.lock
/dev/cuaU1
/dev/cuaU1.init
/dev/cuaU1.lock

I can now access the OpenRD console using cu(1):

% cu -l /dev/cuaU1 -s 115200

\o/ I can boot a FreeBSD 8.0-STABLE arm kernel!

I filled-in the usb/140951 problem report in the FreeBSD GNATS with the patch. If you own an OpenRD and want to have access to the console, the patch is just waiting for you.

Introducing ZFS support in portshaker(8)

portshaker(8) is a tool designed for merging partial ports trees into the FreeBSD ports tree. In other words, it implements some kind of overlay for the FreeBSD ports.

When merging port trees, portshaker(8) first clones the upstream FreeBSD ports tree to the target location. For this purpose, portshaker(8) used to rely on rsync(1) because the target ports tree was supposed to be quite near to the source ports tree. With an UFS file-system, it took my computer circa 10 minutes to merge the 390 Mib of files.

I recently switched to full ZFS, and this gave the clone process a boost: less than four minutes where required to perform the clone action for the first time, and thanks to caching, a single minute was enought to clone a second time.

But ZFS provides cloning capabilities, thus, I added support for the ZFS file-system into portshaker(8). Enabling this feature is as simple as adding the following to /usr/local/etc/portshaker.conf:

use_zfs="yes"

There is however a prerequisite for this to work: the target ports tree's parent directory and the source directory have to be ZFS filesystems. For example, on my computer, the FreeBSD upstream ports are stored in /var/cache/portshaker/freebsd and I merge my ports to the usual /usr/ports directory. All these directories are ZFS filesystems:

romain@marvin ~ % zfs list /usr/ports /var/cache/portshaker /var/cache/portshaker/freebsd
NAME                                USED  AVAIL  REFER  MOUNTPOINT
data/usr/ports                     57,1M  80,9G   402M  /usr/ports
data/var/cache/portshaker           514M  80,9G  48,6M  /var/cache/portshaker
data/var/cache/portshaker/freebsd   427M  80,9G   390M  /var/cache/portshaker/freebsd

When merging, portshaker(8) snapshots the FreeBSD ports and clone them to the target ports tree. This requires only a few seconds to achieve.