Category Archives: Hardware

PC-BSD and 4K — Oh my!

I had recently seen some chatter on IRC about 4K monitor support. One of the people discussing stated that PC-BSD didn’t support 4K monitors, which was curious to me, since I hadn’t tried it yet. This week I did something about that, and ended up getting two 4K monitors and a new video card (With triple DisplayPort) to do some testing.

The Hardware

2x — ASUS PB287Q 28-Inch Screen LED-Lit 4K Monitor (Amazon)

Zotac GeForce GTX 960 2GB GDDR5 PCI Express 3.0 HDMI DVI DisplayPort (Amazon)

The Setup

The setup process was thus:

1. Remove the old monitors and video card

2. Connect the new monitors and video card

3. Go into monitor menus — Enable DisplayPort 1.2 mode

4. Boot up, selecting the “Run Display Wizard” option at the boot-loader.

5. Click “Accept”

That’s it! The display wizard properly detected both 4k monitors, and set them up in a proper dual-head configuration. Logging into KDE resulted in it detecting the new monitors, and automatically re-configuring the display to take advantage of the new resolution. At this point the system was fully usable, font-scaling even worked properly for Firefox and such.

Having 4K setup I did find a few minor issues, which are already being fixed in time for PC-BSD 10.1.2 next week, such as changing the mouse cursor to something which supports larger sizes /  resolutions better. The only real hiccup was figuring out that the monitor ships with DisplayPort set to 1.1 mode, only giving 30hz mode out of box. Changing that in the monitors menus to DisplayPort 1.2 mode, enabled 60hz, giving a silky-smooth desktop experience at 4K resolutions. Not sure if that is something typical of all 4K monitors, or just this model in particular, but if you can’t go beyond 30hz, you may want to check that in your monitor documentation.

Anyway, I hope this information is of use to anybody else who is thinking of taking the 4K plunge. Of course I would love additional feedback from other users who are running 4K, to see if there are other tweaks we can make for optimal support out of box. Let us know on the bug tracker, mailing lists or new forums!

Using TrueOS as a IPFW based home router

Setting up a FreeBSD / TrueOS home router with IPFW

Over the Christmas holidays I had some spare time and was ready to take the plunge and retire an old Asus router. It had begun getting rather slow, due to the increasing number of devices connected to our network, and of course I wanted peace of mind using a FreeBSD system I could be sure was up to date with security fixes. I used PC-BSD’s server release, TrueOS 10.1, because I wanted to use ZFS with boot-environments to ensure upgrading and replacing disks would be risk-free down the road. The following details how I setup TrueOS on the new box.

 

* Disclaimer *

This is by no means a comprehensive IPFW or NATd tutorial, but hopefully it can serve as a good reference for other users trying to do similar things.

 

The Hardware:

To replace it, I had an old AMD 64 x2 Dual-Core system which hadn’t been used in a while.

CPU: AMD Athlon™ 64 X2 Dual Core Processor 5200+ (2719.21-MHz K8-class CPU)
real memory = 4966055936 (4736 MB)
avail memory = 3974365184 (3790 MB)
80GB IDE disk drives (2x)
Intel PWLA83911GTBLK PRO/1000 GT network adapter (2x) — (Amazon Link)

This box is easily overkill, with more than enough horsepower to handle our internet traffic, which is currently 100Mb/s down, 10MB/s up, however it was something I had on hand which means I was only out the cost of a couple network cards. Also, I expect it will be able to keep up with future speed increases, since 200Mb/s down and 20MB/s up is supposed to be coming to our area in the near future.

The Installation:

To get started, I grabbed a copy of the CD-sized installer for TrueOS from PC-BSD’s website:
http://www.pcbsd.org/en/download.html

The installation was very straight forward. I picked the defaults to use ZFS on root, with GRUB so that boot-environments work properly. The only major change was to go to advanced disk setup and attach the second 80GB disk drive as a mirror of the primary (ada0). Since these disks aren’t super new I wanted the ability to easily replace them when they begin to fail down the road.

Post Install Configuration:

With the installation finished, the first thing I did was to edit /etc/rc.conf with the following lines:

# Disable some PC-BSD / TrueOS services we don’t need to run on a router
syscache_enable=“NO“
php_fpm_enable=“NO“
appcafe_enable=“NO”

This disabled the AppCafe / Warden web-interface, since I don’t plan on running any applications or jails on this box, only using it purely as a router. Of course if you plan on running jails or doing other package work, the web-interface could be useful and you can skip these lines.

Next I started the configuration of the network interfaces:

# Network configuration
hostname=“interdictor“
ifconfig_em0=“inet 192.168.1.1 netmask 255.255.255.0″ # LAN IP / Bottom NIC
ifconfig_em1=“inet XXX.XXX.XXX.XXX netmask XXX.XXX.XXX.XXX” # WAN IP / Top NIC
defaultrouter=“XXX.XXX.XXX.XXX”

# Enable gateway mode
gateway_enable=“YES”

A couple of comments help me keep straight which ports on the back of the system are the WAN / LAN. Of course I replaced the XXX lines with the static IP / gateway from my ISP.

Next, I was going to be using this router to assign local IP’s with DHCP, so I first installed the DHCP server from PC-BSD’s package repo:

# pkg install net/isc-dhcp43-server

Next, I configured my DHCP information in /usr/local/etc/dhcpd.conf

# option definitions common to all supported networks…
option domain-name “XXXXXXXXX”;
option domain-name-servers 8.8.8.8, 8.8.4.4;
default-lease-time 86400;
max-lease-time 90000;

# Use this to enble / disable dynamic dns updates globally.
#ddns-update-style none;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

# Home Network
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.100 192.168.1.254;
option domain-name-servers 8.8.8.8, 8.8.4.4;
option broadcast-address 192.168.1.255;
option routers 192.168.1.1;
default-lease-time 86400;
max-lease-time 90000;
}

And finally enable the service in /etc/rc.conf

# DHCPD config
dhcpd_enable=“YES“
dhdpd_ifaces=“em0”

With DHCPD now enabled, I next went and created /etc/natd.conf with the following:

interface em1
use_sockets yes
same_ports yes
dynamic yes
log_denied yes
log yes

Optionally if you run services behind NAT and need to forward ports, you can do so with syntax like the following:
# Sample redirect port
# redirect_port tcp 192.168.1.20:80 80

Where 192.168.1.20 is the local IP of your system hosting a service, and “:80 80” being the local server port, and port on your router to open.

For our last config, we now need to setup our firewall rules, which we will do by creating the file /etc/ipfw.rules with a setup something like this:

#!/bin/sh
# Flush out the list before we begin.
ipfw –q –f flush

# Set rules command prefix
cmd=“ipfw –q add“
pif=“em1” # interface name of NIC attached to Internet
iif=“em0” # interface name of NIC attached to LAN
server=“192.168.1.1″
inside=“192.168.1.0/24″
me=“XXX.XXX.XXX.XXX” # Public WAN address
ks=“keep-state” # Laziness
skip=“skipto 600″

# Allow everything through the local NIC
######################################
$cmd 00020 allow all from any to any via em0

######################################
# No restrictions on Loopback Interface
######################################
$cmd 00025 allow all from any to any via lo0

# NAT the inbound stuff
######################################
$cmd 00100 divert natd ip from any to any in via $pif # NAT any inbound packets

######################################
# Allow packet through if it matches existing entry in dynamic rules

######################################
$cmd 00105 check-state

######################################
# Allow all outgoing packets
######################################
$cmd 00110 $skip all from any to any out via $pif $ks

# Deny all inbound traffic from non-routable reserved address spaces
######################################
$cmd 00300 deny all from 192.168.0.0/16 to any in via $pif #RFC 1918 private IP
$cmd 00301 deny all from 172.16.0.0/12 to any in via $pif #RFC 1918 private IP
$cmd 00302 deny all from 10.0.0.0/8 to any in via $pif #RFC 1918 private IP
$cmd 00303 deny all from 127.0.0.0/8 to any in via $pif #loopback
$cmd 00304 deny all from 0.0.0.0/8 to any in via $pif #loopback
$cmd 00305 deny all from 169.254.0.0/16 to any in via $pif #DHCP auto-config
$cmd 00306 deny all from 192.0.2.0/24 to any in via $pif #reserved for doc’s
$cmd 00307 deny all from 204.152.64.0/23 to any in via $pif #Sun cluster interconnect
$cmd 00308 deny all from 224.0.0.0/3 to any in via $pif #Class D & E multicast
######################################

# Deny public pings
######################################
$cmd 00310 deny icmp from any to any in via $pif

# Log all the other troublemakers
$cmd 00550 deny log all from any to any

# Skip location for NATD
$cmd 600 divert natd ip from any to any out via $pif # skipto location for outbound stateful rules
$cmd 610 allow ip from any to any

 

Now that NATD / IPFW are setup, we can set them to run at boot with the following /etc/rc.conf entries:

# IPFW
firewall_enable=“YES“
firewall_logging=“YES“
firewall_script=”/etc/ipfw.rules”

# NATD
natd_enable=“YES“
natd_interface=“em1“
natd_flags=”-m –f /etc/natd.conf”

If you don’t want to reboot the server you can start the services with the following commands:

# service natd start
# service isc-dhcpd start
# service ipfw start

Also, the ipfw.rules file is just a shell-script, so you can always re-apply the firewall rules with “sh /etc/ipfw.rules”.

That’s it! At this point you should be able to replace your consumer router with the new system, and it’ll begin handling NAT traffic, assigning DHCP addresses, etc. Even during my network’s peak usage, I’ve yet to see the system drop below 95% idle.

 

 

Bonus content!

Some users may want to do NAT reflection for public facing services. This basically means you can have clients, such as your phone, connect to the same public address, regardless of being on the local LAN, or roaming outside the home. OpenBSD has a great explanation of this, along with the methods of solving it. I was able to easily do this with IPFW and TCP proxying. To get started, install tcpproxy:

# pkg install tcpproxy

Next, we have to create our /usr/local/etc/tcpproxy.conf file:

listen * 50080
{
  remote: 192.168.1.20 80;
  remote-resolv: ipv4;
};

In the example above, we are going to listen on port 50080 on the local router machine, and proxy this to the remote machine on the LAN: 192.168.1.20 on port 80. Next we will need to add a firewall rule to /etc/ipfw.rules causing it to redirect traffic coming from the LAN NIC to this service:

# Setup NAT reflection
$cmd 00002 fwd 127.0.0.1,50080 tcp from 192.168.1.0/24 to $me 80 in via em0

With those entries complete, you can now start the tcpproxy service, and restart IPFW to enable them.

 

10.1 New Update Manager Backend — Call for Testers

Hey testers a new beta update is available to upgrade your systems from 10.x to 10.1. The beta instructions have been sent out to the PC-BSD Testing e-mail list if you would like to participate in the testing. For more information you can email me @ [email protected] or sign up for the testing mailing list @ http://lists.pcbsd.org/mailman/listinfo/testing.

Thanks!

–Josh

whoami? I’m PC-BSD 10.0! — Weekly Feature Digest 15

The PC-BSD team has announced version 10.0 Joule Edition is now in official release status and is available for mass distribution. If you’ve used PC-BSD in the past you haven’t seen anything yet! PC-BSD 10.0’s feature rich front end runs beautifully without sacrificing the stability we all know and love from FreeBSD’s base system. This week we’re going to look back and highlight some of best new features now available in PC-BSD Version 10.0 — Joule Edition.

AMD / ATI graphics are now officially supported under PC-BSD. Although in the past AMD Radeon users have suffered from lack-of-driver-itis, those days are passed and a new age is upon us. I’m currently running PC-BSD on 3 different types of AMD graphics based systems and the performance is arguably better than some of my nvidia based test systems (sorry Nvidia fanboys :D). I was even more surprised when ATI hybrid graphics worked out of the box on my Samsung NP-350 Notebook computer.  Especially considering this laptop was designed for…Windows 8…*BARF*

0131142101a

You will notice there are a couple of quirks to be aware of with the new Radeon drivers that we are currently aware of. Firstly the CTRL F keys do not bring up the console. This is being actively remedied by FreeBSD and we expect there will be a fix committed in the near future. Also KDE support for Radeon is there, but i’ve noticed that I experience random freeze ups from time to time on any of my Radeon systems. Using any other desktop environment seemed to fix the issue for me.

More of your favorite Desktop Environments! We’ve now added Gnome 3, Cinnamon, and mate to the list of desktops available under PC-BSD 10.0. Mate has officially replaced Gnome 2 in version 10.0, but Gnome 3 and Cinnamon will remain as “unsupported� desktops currently until further development and additional testing are done to bring them up to speed in PC-BSD. That being said feel free to report issues with these desktops with the understanding that they are currently in a “beta� stage and are offered without any guarantee. For best results with your PC-BSD experience use supported desktops!

The Grub and FreeBSD boot loaders have both received updated support to work with PC-BSD version 10.0. Have your choice now during installation for whatever boot loader works best for you. As we continue to work with the Grub boot loader our support is evolving simultaneously to give you even better compatibility.

In other news this week if any users are still using RC’s 1–5 please update to PC-BSD 10.0 — Release before reporting bugs so we can make sure we aren’t duplicating tickets for issues that have already been submitted and /or resolved.

Special thanks to the entire PC-BSD team including our testers, committers, and administrators. Without you guys this release wouldn’t be what it is today and we sincerely thank you all for your dedication in making PC-BSD Joule Edition a success.

Best Regards,

Josh

 

 

PC-BSD Weekly Feature Digest 10/25/13

The big news of the week is finally here!  PC-BSD 10 images are ready for testing.   These images are offered for testing and development purposes only!  (Read disclaimer below).  They are still in alpha so by all means fire away, but expect breakage.  The more of you that can delve into this and get us feedback on bug reports the better!

PC-BSD 10.0 will offer a UEFI loader on installation media, as well as the newly polished life preserver system app.   Also the installer will now offer the ability to choose between GRUB or BSD bootloader.

Over the course of the next several weeks more packages and PBI’s will become available as they become available / working for us.  Keep an eye out on 10.0 updates here on the weekly feature digest.

Disclaimer
———————————————

This image is a very early cut of the new stable/10 branch of FreeBSD and is probably only suitable for advanced users or developers. A PKGNG repository is available for this release, however it is missing a number of packages we would normally ship with, such as virtualbox drivers, KDE accessories, etc. The AppCafe will currently appear empty, but we plan on populating it over the next few weeks.

 

 

PC-BSD Hardware Store

Josh Smith has announced the initial launch of the PC-BSD hardware store. This resource is meant to make it easier to find hardware that has been tested to work on PC-BSD.  The store itself is here. If you have hardware that you know works and which you don’t see listed, please add it to the Hardware section of the wiki so that it can be included.

How to charge an iPad with USB port and devd(8)

If you happen to have one of those Apple iPad gadgets, you’ve probably discovered at some point that iPads do not particularly like to charge themselves through your computer’s (or laptop’s) USB ports, even when the port is definitely capable of providing enough current (i.e., full powered).

In 2010, some clever soul sniffed the “magic” USB command, and wrote simple utility, on top of libusb-1.0, which enables charging from USB port. It can be found in the FreeBSD Ports Collection under sysutils/ipad_charge. Unfortunately, it’s a C program, with single hardcoded product id, so it doesn’t work with new iPads, for example, without patching. There is also no support for FreeBSD’s devd(8); original author only provides rule for Linuxish udev. (There are a few forks of the original utility available which support vast range of Apple devices, e.g. this one.)

Luckily, FreeBSD allows for simple things to be done in simple ways. Why bother with building and running some external binary, when usbconfig(8) conveniently provides do_request command to send hand-crafted requests to USB devices?

This is where the magic happens in the original C implementation:

#define CTRL_OUT (LIBUSB_REQUEST_TYPE_VENDOR | LIBUSB_ENDPOINT_OUT)
libusb_control_transfer(dev_handle, CTRL_OUT, 0x40, 0x6400,
    enable ? 0x6400 : 0, NULL, 0, 2000)

This can be translated into the following do_request (command-line example; replace ellipsis with actual iPad device name as it appears in your system per usbconfig list):

$ usbconfig -d ... do_request 0x40 0x40 0x6400 0x6400 0x0

Doing it manually is fine for initial testing, but ultimately we’d want devd(8) daemon to take care of this for us every time we plug in an iPad. I’ve come up with the following code, which I put into /etc/devd/ipad.conf file:

attach 0 {
    match "vendor" "0x05ac";
    match "product" "(0x129a|0x12ab)";
    action "usbconfig -d $device-name do_request 0x40 0x40 0x6400 0x6400 0x0";
};

With devd(8) hook, we no longer have to build anything, and can easily augment match "product" line for upcoming new models (provided that this USB hack still applies to them, but so far it seems to be true for all iPads from the first to the latest fourth; I’ve included product ids I was able to test myself). I guess that detach method could also be added, but it’s probably not necessary: once iPad is plugged out, the port gets reset or something.

Submit Your Hardware Compatibility Information

The PC-BSD team will be launching a new webstore designed to take the hassle out of knowing which hardware works well with PC-BSD. A wiki page has been created where users can add the models of the motherboards, video cards, network cards, wifi cards, and laptops they have found to work great with PC-BSD. When adding to the wiki, only add hardware that you have verified works as-is with no problem.

To add to the hardware list, create a wiki account so that you can login and edit the page. Since this list will get rather large, please add your entry alphabetically.

Status crypto cards HOWTO: problems with the card reader (support could be better)

After hours (spread over weeks) I come to the conclusion that there is a lot of potential to improve the documentation of card readers (but I doubt the card reader vendors will do it) and of the pcsc documentation. It is not easy to arrive at a point where you understand everything. The compatibility list does not help much, as the card readers are partly past their end of life and the models which replace them are not listed. Respectively the one I bought does not support all the features I need. I even ported the driver to FreeBSD (not committed, I wanted to test everything first) and a lot of stuff works, but one critical part is that I can not store a certificate on the crypto card as the card reader or the driver  does not support extended APDUs (needed to transfer more than 255 bytes to the card reader).

Well, the status so far:

  • I have a HOWTO what to install to use crypto cards in FreeBSD
  • I have a HOWOT what to install / configure in Windows
  • I have a HOWTO regarding creating keys on a openpgp v2 card and how to use this key with ssh on FreeBSD (or any other unix-like OS which can run pcsc)
  • I have a card reader which does not support extended APDUs
  • I want to make sure what I write in the HOWTOs is also suitable for the use with Windows / PuTTY
  • it seems Windows needs a certificate and not only a key when using the Windows CAPI (using the vendor supplied card reader driver) in PuTTY-CSC (works at work with a USB token)
  • the pcsc pkcs11 Windows DLL is not suitable yet for use on Windows 8 64bit
  • I contacted the card reader vendor if the card reader or the driver is the problem regarding the extended APDUs
  • I found problems in gpg4win / pcsc on Windows 8
  • I have send some money to the developers of gpg4win to support their work (if you use gnupg on Windows, try to send a few units of money to them, the work stagnated as they need to spend their time for paid work)

So either I need a new card reader, or have to wait for an update of the linux driver of the vendor… which probably means it may be a lot faster to buy a new card reader. When looking for one with at least a PIN pad, I either do not find anything which is listed as supported by pcsc on the vendor pages (it is incredible how hard it is to navigate the websites of some companies… a lot of buzzwords but no way to get to the real products), or they only list updated models where I do not know if they will work.

When I have something which works with FreeBSD and Windows, I will publish all the HOWTOs here at once.

Bookmark/FavoritesShare/Save

OpenPGP crypto cards ordered

I wrote in a previous blog post that I want to switch to crypto cards for use with ssh and GnuPG. After some research I settled on the OpenPGP cryto cards. I ordered them from kernelconcepts. As soon as they arrive (and I have some free time), I will start to use them and write down how to work with them with FreeBSD.

Bookmark/FavoritesShare/Save

Which crypto card to use with FreeBSD (ssh/gpg)

The recent security incident triggered a discussion how to secure ssh/gpg keys.

One way I want to focus on here (because it is the way I want to use at home), is to store the keys on a crypto card. I did some research for suitable crypto cards and found one which is called Feitian PKI Smartcard, and one which is called OpenPGP card. The OpenPGP card also exists in a USB version (basically a small version of the card is already integrated into a small USB card reader).

The Feitian card is reported to be able to handle RSA keys upto 2048 bits. They do not seem to handle DSA (or ECDSA) keys. The smartcard quick starter guide they have  (the Tuning smartcard file system part) tells how to change the parameters of the card to store upto 9 keys on it.

The spec of the OpenPGP card tells that it supports RSA keys upto 3072 bits, but there are reports that it is able to handle RSA keys upto 4096 bits (you need to have at least GPG 2.0.18 to handle that big keys on the crypto card). It looks to me like the card is not handle DSA (or ECDSA) cards. There are only slots for upto 3 keys on it.

If I go this way, I would also need a card reader. It seems a class 3 one (hardware PIN pad and display) would be the most “future-proof” way to go ahead. I found a Reiner SCT cyberJack secoder card reader, which is believed to be supported by OpenSC and seems to be a good balance between cost and features of the Reiner SCT card readers.

If anyone reading this can suggest a better crypto card (keys upto 4096 bits, more than 3 slots, and/or DSA/ECDSA  support), or a better card reader, or has any practical experience with any of those components on FreeBSD, please add a comment.

Bookmark/FavoritesShare/Save

Parallella on Kickstarter

Kickstarter is a great thing - it allows projects which limited appeal to be succesfully financed, which helps them succeed. One such project is the Parallela. It is basically an ARM-based highly-NUMA computer with 16-64 cores which can be used both to teach parallel programming and actually do some useful work with very little electrical power. The numbers cited as on the on the order of 45 GFLOPS/watt for the maximum configuration. As the Kickstarter deadline is approaching, I think it is a good time to use this opportunity to call on all enthusiasts to help fund this cool project!

Read more...

Book review: FreeBSD Device Drivers

In mid-April a woman from the marketing department of No Starch Press contacted me and asked if I am interested to do a public review of the FreeBSD Device Drivers book by Joseph Kong (no link to a book shop, go and have a look in your preferred one). Just this simple question, no strings attached.

I had my nose in some device drivers in the past, but I never wrote one, and never had a look at the big picture. I was interested to know how everything fits together, so this made me a good victim for a review (novice enough to learn something new and to have a look if enough is explained, and experienced enough to understand what is going on in the FreeBSD kernel).

Some minutes after I agreed to review it (but with a little notice that I do not know how long I need to review it), I had the PDF version of the book. That was faster than I expected (maybe I am too old-school and used to have paper versions of books in my hands).

Let the review begin… but bear with me, this is the first time I do a real public review of a book (instead of a technical review for an author). And as this is my very own personal opinion, I will not allow comments here. This page is all about my opinion while reading the book, questions I have while reading the book shall serve as a hint about the quality of the book and they should be answered in the book, not here.

In short, the book is not perfect, but it is a good book. There is room for improvement, but on a very high level. If you want to write a device driver for FreeBSD, this book is a must. I suggest to read it completely, even chapters which do not belong to the type of driver you want to write (specially the case studies of real drivers). The reason is that each chapter has some notes which may not only apply to the chapter in question, but to all kinds of device drivers. The long review follows now.

The first chapter is titled “Building and running modules”. The author begins with description of the usual device driver types (NIC driver, pseudo-device, …) and how they can be added to the kernel (statically linked in or as a module). The first code example is a small and easy kernel module, so that we do not have to reboot the system we use to develop a driver (except we make a fault during driver development which causes the machine to panic or hang). Every part of the example is well explained. This is followed by an overview about character devices (e.g. disks) and a simple character-device driver (so far a pseudo-device, as we do not have real hardware we access) which is not only as-well explained as the module-example, but there is also a note where the code was simplified and what should be done instead.

After reading this chapter you should be able to write your own kernel module in 5 minutes (well, after 5 minutes it will not be able to do a lot — just a “hello world” – but at least you can already load/unload/execute some code into/from/in the kernel).

I have not tried any example myself, but I compiled a lot of modules and drivers I modified in the past and remember to have seen the described parts.

The second chapter explains how to allocate and free memory in the kernel. There is the possibility to allocate maybe-contiguous memory (the normal case, when your hardware does not do DMA or does not have the requirement that the memory region it makes DMA from/too needs to be contiguous), and really contiguous. For the size argument of the freeing of the the contiguous memory there is the sentence “Generally, size should be equal the amount allocated.”. Immediately I wanted to know what happens if you specify a different size (as a non-native english speaker I understand this sentence in a way that I am allowed to specify a different size and as such are able to free only parts of the allocated memory). Unfortunately this is not answered. I had a look into the source, the kernel frees memory pages, so the size argument (and addr argument) will be rounded to include a full page. This means theoretically I am able to free parts of the allocated memory, but this is a source-maintenance nightmare (needs knowledge about the machine specific page boundaries and you need to make sure that you do the absolutely correct size calculations).  To me this looks more like as long as nobody is pointing a gun at my head and tells me to use a different size, specifying the same size as made during the allocation of this memory region is the way to go.

After reading this chapter you should know how to kill the system by allocating all the RAM in the kernel.

Again, I did not try to compile the examples in this chapter, but the difference of the memory allocation in the kernel compared with memory allocation in the userland is not that big.

The third chapter explains the device communication and control interfaces (ioctl/sysctl) of a driver. The ioctl part teached me some parts I always wanted to know when I touched some ioctls, but never bothered to find out before. Unfortunately this makes me a little bit nervous about the way ioctls are handled in the FreeBSD linuxulator, but this is not urgent ATM (and can probably be handled by a commend in the right place). The sysctl part takes a little bit longer to follow through, but there is also more to learn about it. If you just modify an existing driver with an existing sysctl interface, it probably just comes down to copy&paste with little modifications, but if you need to make more complex changes or want to add a sysctl interface to a driver, this part of the book is a good way to understand what is possible and how everything fits together. Personally I would have wished for a more detailed guide when to pick the ioctl interface and when the sysctl interface than what was written in the conclusion of the chapter, but it is probably not that easy to come up with a good list which fits most drivers.

After reading this chapter you should be able to get data in and out of the kernel in 10 minutes.

As before, I did not compile the examples in this chapter. I already added ioctls and sysctls in various places in the FreeBSD kernel.

Chapter 4 is about thread synchronization – mutexes, shared/exclusive locks, reader/writer locks and condition variables. For me this chapter is not as good as the previous ones. While I got a good explanation of everything, I missed a nice overview table which compares the various methods of thread synchronization. Brendan Gregg did a nice table to give an overview of DTrace variable types and when to use them. Something like this would have been nice in this chapter too. Apart from this I got all the info I need (but hey, I already wrote a NFS client for an experimental computer with more than 200000 CPUs in 1998, so I’m familiar with such synchronization primitives).

Delayed execution is explained in chapter 5. Most of the information presented there was new to me. While there where not much examples presented (there will be some in a later chapter), I got a good overview about what exists. This time there was even an overview when to use which type of delayed execution infrastructure. I would have preferred to have this overview in the beginning of the chapter, but that is maybe some kind of personal preference.

In chapter 6 a complete device driver is dissected. It is the virtual null modem terminal driver. The chapter provides real-world examples of event-handlers, callouts and taskqueues which where not demonstrated in chapter five. At the same time the chapter serves as a description of the functions a TTY driver needs to have.

Automated device detection with Newbus and the corresponding resource allocation (I/O ports, device memory and interrupts) are explained in chapter 7. It is easy… if you have a real device to play with. Unfortunately the chapter missed a paragraph or two about the suspend and resume methods. If you think about it, it is not hard to come up with what they are supposed to do, but a little explicit description of what they shall do, in what state the hardware should be put and what to assume when being called would have been nice.

Chapter 8 is about interrupts. It is easy to add an interrupt handler (or to remove one), the hard part is to generate an interrupt. The example code uses the parallel port, and the chapter also contains a little explanation how to generate an interrupt… if you are not afraid to touch real hardware (the parallel port) with a resistor.

In chapter 9 the lpt(4) driver is explained, as most of the topics discussed so far are used inside. The explanation how everything is used is good, but what I miss sometimes is why they are used. The most prominent (and only) example here for me is why are callouts used to catch stray interrupts? That callouts are a good way of handling this is clear to me, the big question is why can there be stray interrupts. Can this happen only for the parallel port (respectively a limited amount of devices), or does every driver for real interrupt driven hardware need to come with something like this? I assume this is something specific to the device, but a little explanation regarding this would have been nice.

Accessing I/O ports and I/O memory for devices are explained in chapter 10 based upon a driver for a LED device (turn on and off 2 LEDs on an ISA bus). All the functions to read and write data are well explained, just the part about the memory barrier is a little bit short. It is not clear why the CPU reordering of memory accesses matter to what looks like function calls. Those function calls may be macros, but this is not explained in the text. Some little examples when to use the barriers instead of an abstract description would also have been nice at this point.

Chapter 11 is similar to chapter 10, just that a PCI bus driver is discussed instead of an ISA bus driver. The differences are not that big, but important.

In chapter 12 it is explained how to do DMA in a driver. This part is not easy to understand. I would have wanted to have more examples and explanations of the DMA tag and DMA map parts. I am also surprised to see different supported architectures for the flags BUS_DMA_COHERENT and BUS_DMA_NOCACHE for different functions. Either this means FreeBSD is not coherent in those parts, or it is a bug in the book, or it is supposed to be like this and the reasons are not explained in the book. As there is no explicit note about this, it probably leads to confusion of readers which pay enough attention here. It would also have been nice to have an explanation when to use those flags which are only implemented on a subset of the architectures FreeBSD supports. Anyway, the explanations give enough information to understand what is going on and to be able to have a look at other device drivers for real-live examples and to get a deeper understanding of this topic.

Disk drivers and block I/O (bio) requests are described in chapter 13. With this chapter I have a little problem. The author used the word “undefined” in several places where I as a non-native speaker would have used “not set” or “set to 0″. The word “undefined” implies for me that there may be garbage inside, whereas from a technical point of view I can not imagine that some random value in those places would have the desired result. In my opinion each such place is obvious, so I do not expect that an experienced programmer would lose time/hairs/sanity over it, but inexperienced programmers which try to assemble the corresponding structures on the (uninitialized) heap (for whatever reason), may struggle with this.

Chapter 14 is about the CAM layer. While the previous chapter showed how to write a driver for a disk device, chapter 14 gave an overview about how to an HBA to the CAM layer. It is just an overview, it looks like CAM needs a book on its own to be fully described. The simple (and most important) cases are described, with the hardware-specific parts being an exercise for the person writing the device driver. I have the impression it gives enough details to let someone with hardware (or protocol), and more importantly documentation for this device, start writing a driver.

It would have been nice if chapter 13 and 14 would have had a little schematic which describes at which level of the kernel-subsystems the corresponding driver sits. And while I am at it, a schematic with all the driver components discussed in this book at the beginning as an overview, or in the end as an annex, would be great too.

An overview of USB drivers is given in chapter 15 with the USB printer driver as an example for the explanation of the USB driver interfaces. If USB would not be as complex as it is, it would be a nice chapter to start driver-writing experiments (due to the availability of various USB devices). Well… bad luck for curious people. BTW, the author gives pointers to the official USB docs, so if you are really curious, feel free to go ahead. :)

Chapter 16 is the first part about network drivers. It deals with ifnet (e.g. stuff needed for ifconfig), ifmedia (simplified: which kind of cable and speed is supported), mbufs and MSI(-X). As in other chapters before, a little overview and a little picture in the beginning would have been nice.

Finally, in chapter 17, the packet reception and transmission of network drivers is described. Large example code is broken up into several pieces here, for more easy discussion of related information.

One thing I miss after reaching the end of the book is a discussion of sound drivers. And this is surely not the only type of drivers which is not discussed, I can come up with crypto, firewire, gpio, watchdog, smb and iic devices within a few seconds. While I think that it is much more easy to understand all those drivers now after reading the book, it would have been nice to have at least a little overview of other driver types and maybe even a short description of their driver methods.

Conclusion: As I wrote already in the beginning, the book is not perfect, but it is good. While I have not written a device driver for FreeBSD, the book provided enough insight to be able to write one and to understand existing drivers. I really hope there will be a second edition which addresses the minor issues I had while reading it to make it a perfect book.

Bookmark/FavoritesShare/Save

Call for Testers: GEM/KMS Snapshot

A testing snapshot that integrates the new DRM/GEM/KMS work is now available to testers.

DANGER: this snapshot should be considered alpha quality and for testing purposes only!

This snapshot is meant to provide testers the opportunity to provide feedback regarding Intel, ATI and NVIDIA video cards which currently are not fully supported using the existing FreeBSD Xorg drivers. Please report any issues you find to the FreeBSD X11 mailing list so that the FreeBSDD Xorg porting team can address them.

As an added bonus, this snapshot contains the shiny new 9.1 installer. Feedback on the installer should be sent to the PC-BSD testing mailing list. Currently, the new installer is totally undocumented, but that should change over the next few weeks.

How big are the buffers in FreeBSD drivers?

Today I have read an interesting investigation and problem analysis from Jim Gettys.

It is a set of articles he wrote over several months and is not finished writing as of this writing (if you are deeply interested in it go and read them, the most interesting ones are from December and January and the comments to the articles are also contributing to the big picture). Basically he is telling that a lot of network problems users at home (with ADSL/cable or WLAN) experience  are because buffers in the network hardware or in operating systems are too big. He also proposes workarounds until this problem is attacked by OS vendors and equipment manufacturers.

Basically he is telling the network congestion algorithms can not do their work good, because the network buffers which are too big come into the way of their work (not reporting packet loss timely enough respectively try to not lose packets in situations where packet loss would be better because it would trigger action in the congestion algorithms).

He investigated the behavior of Linux, OS X and Windows (the system he had available). I wanted to have a quick look at the situation in FreeBSD regarding this, but it seems at least with my network card I am not able to see/find the corresponding size of the buffers in drivers in 30 seconds.

I think it would be very good if this issue is investigated in FreeBSD, and apart from maybe taking some action in the source also write some section for the handbook which explains the issue (one problem here is, that there are situations where you want/need to have such big buffers and as such we can not just downsize them) and how to benchmark and tune this.

Unfortunately I even have too much on my plate to even further look into this. :( I hope one of the network people in FreeBSD is picking up the ball and starts playing.

Share

Are USB memory sticks really that bad?

Last week my ZFS cache device — an USB memory stick — showed xxxM write errors. I got this stick for free as a promo, so I do not expect it to be of high quality (or wear-leveling or similar life-saving things). The stick survived about 9 months, during which it provided a nice speed-up for the access to the corresponding ZFS storage pool. I replaced it by another stick which I got for free as a promo. This new stick survived… one long weekend. It has now 8xxM write errors and the USB subsystem is not able to speak to it anymore. 30 minutes ago I issued an “usbconfig reset

10.140 MHz QRSs beacon

Got samples from analog.com.brNow to put them into use:

1) get geda installed

2) check/find the schematic symbols

3) check/find the pcb footprint

4) draw the schematc

5) layout a pcb

6) ask a friend to make the pcb

7) go into soldersmoke ( is a podcast actually)
8) make program for a pic or simply control it with an USB to lpt converter

#1 is done, to #2 though symbols needs to be created.