Category Archives: mips

bsdbox – take #1

One of the nice things about building embedded Linux "stuff" is busybox. The other is "uClibc", which isn't really the focus of this post.

I've fleshed out a basic busybox style thing based on the crunched binary stuff which generates sysinstall and the rescue binaries. Thankfully, most of the hard work (read: the hacks needed) are done for you - both in the rescue Makefile and the general FreeBSD build framework.

So, I give you "bsdbox" - a not-so-small static binary busybox style solution to help build a standalone image.

# ls -l /sbin/bsdbox
-rwxr-xr-x 1 0 0 2958668 Sep 16 17:44 /sbin/bsdbox
# /sbin/bsdbox
usage: bsdbox ..., where is one of:
ls cat dd df cp hostname kill mkdir sleep sh -sh dmesg sysctl init reboot
mount mdmfs mount_mfs mdconfig newfs ifconfig route ping true false hexdump
tail netstat chown chgrp arp hostapd hostapd_cli bsdbox

And with that, I have this TP-Link device happily running FreeBSD as a WPA-enabled wireless access point.

Combined with the GEOM uzip module, the entire filesystem (which is the above binary and a few shell scripts) - is just slightly above 1 megabyte in size. (geom_lzma will make that even smaller.)

The bsdbox stuff is in my GIT repository in "bsdbox". It's built as part of the base system and installed in /bsdbox/. The binary can then be cherry picked and built into an image with whatever symlinks are needed. There's more to do - more binaries for a start; but also making it a bit easier to configure bits than a single Makefile - but it's useful right now.

bsdbox – take #1

One of the nice things about building embedded Linux "stuff" is busybox. The other is "uClibc", which isn't really the focus of this post.I've fleshed out a basic busybox style thing based on the crunched binary stuff which generates sysinstall and the rescue binaries. Thankfully, most of the hard work (read: the hacks needed) are done for you - both in the rescue Makefile and the general FreeBSD build framework.So, I give you "bsdbox" - a not-so-small static binary busybox style solution to help build a standalone image.# ls -l /sbin/bsdbox-rwxr-xr-x 1 0 0 2958668 Sep 16 17:44 /sbin/bsdbox# /sbin/bsdbox usage: bsdbox ..., where is one of:ls cat dd df cp hostname kill mkdir sleep sh -sh dmesg sysctl init rebootmount mdmfs mount_mfs mdconfig newfs ifconfig route ping true false hexdumptail netstat chown chgrp arp hostapd hostapd_cli bsdboxAnd with that, I have this TP-Link device happily running FreeBSD as a WPA-enabled wireless access point.Combined with the GEOM uzip module, the entire filesystem (which is the above binary and a few shell scripts) - is just slightly above 1 megabyte in size. (geom_lzma will make that even smaller.)The bsdbox stuff is in my GIT repository in "bsdbox". It's built as part of the base system and installed in /bsdbox/. The binary can then be cherry picked and built into an image with whatever symlinks are needed. There's more to do - more binaries for a start; but also making it a bit easier to configure bits than a single Makefile - but it's useful right now.

FreeBSD/MIPS: AR9132 porting

I decided to avoid work a little by porting FreeBSD/MIPS to the AR9132 in the TP-LINK WN-1043nd 802.11n wireless AP I have here. There's existing OpenWRT code for it so I figured the port wouldn't be that difficult.

It turns out I was right. The AR91xx support took a couple of days to massage into a form which was good enough to commit to FreeBSD-HEAD. I also included some basic AR713x SoC support - not enough to be completely useful, but enough to get a head-start on porting.

I broke out various CPU specific operations into a set of function pointers which a CPU detection function sets up early during the MIPS init code. The main differences between the SoCs are:

* Different base frequencies for setting up RAM, peripheral bus (eg UART), etc
* Different register locations for a few things
* The AR713x has a PCIe bus; the AR71xx has a PCI bus; the AR91xx doesn't have a PCI bus
* The AR9132 has different PLL values for the gigabit ethernet MACs
* Each SoC has a different USB peripheral setup path
* Each SoC has different GPIO layouts
* There's slightly different on-board peripheral reset registers

I've tested the code on the AR71xx and AR9132 and things work fine. I now need to port the AR9100 wireless MAC support from Linux ath9k to complete things. The AR9100 looks a lot like the AR5416/AR9160 (802.11n, 3 radio chains) but it has a few annoying differences:

* It isn't on a PCI bus, so it has to be manually attached;
* A few registers differ in locations (one of which returns 0xdeadbeef if the AR5416 register location is read!)
* The EEPROM is not in the same location(s) as the AR5416 - it needs to be copied and then shoe-horned into the AR5416 eeprom access code.

Also, there's an RTL8133RB gigabit switch device on-board. There's also code in Linux/OpenWRT to support this. I'll look at porting this once the AR9100 support is done.

FreeBSD/MIPS: AR9132 porting

I decided to avoid work a little by porting FreeBSD/MIPS to the AR9132 in the TP-LINK WN-1043nd 802.11n wireless AP I have here. There's existing OpenWRT code for it so I figured the port wouldn't be that difficult.It turns out I was right. The AR91xx support took a couple of days to massage into a form which was good enough to commit to FreeBSD-HEAD. I also included some basic AR713x SoC support - not enough to be completely useful, but enough to get a head-start on porting.I broke out various CPU specific operations into a set of function pointers which a CPU detection function sets up early during the MIPS init code. The main differences between the SoCs are:* Different base frequencies for setting up RAM, peripheral bus (eg UART), etc* Different register locations for a few things* The AR713x has a PCIe bus; the AR71xx has a PCI bus; the AR91xx doesn't have a PCI bus* The AR9132 has different PLL values for the gigabit ethernet MACs* Each SoC has a different USB peripheral setup path* Each SoC has different GPIO layouts* There's slightly different on-board peripheral reset registersI've tested the code on the AR71xx and AR9132 and things work fine. I now need to port the AR9100 wireless MAC support from Linux ath9k to complete things. The AR9100 looks a lot like the AR5416/AR9160 (802.11n, 3 radio chains) but it has a few annoying differences:* It isn't on a PCI bus, so it has to be manually attached;* A few registers differ in locations (one of which returns 0xdeadbeef if the AR5416 register location is read!)* The EEPROM is not in the same location(s) as the AR5416 - it needs to be copied and then shoe-horned into the AR5416 eeprom access code.Also, there's an RTL8133RB gigabit switch device on-board. There's also code in Linux/OpenWRT to support this. I'll look at porting this once the AR9100 support is done.

FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!

I found and squished a small bug in the gigabit NIC driver in FreeBSD/MIPS for the AR71xx chipset. It wasn't all that complicated - TX buffers weren't being thoroughly enough checked for alignment/size constraints before being handed to the DMA engine.
It did however fix a few niggling issues. My tunneling stuff was fixed. IPv6 frames are now correctly handled. And ng_nat doesn't cause a panic in the NIC driver. So there's at least three people who are happy. :)

FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!

I found and squished a small bug in the gigabit NIC driver in FreeBSD/MIPS for the AR71xx chipset. It wasn't all that complicated - TX buffers weren't being thoroughly enough checked for alignment/size constraints before being handed to the DMA engine.

It did however fix a few niggling issues. My tunneling stuff was fixed. IPv6 frames are now correctly handled. And ng_nat doesn't cause a panic in the NIC driver. So there's at least three people who are happy. :)

Minor OpenOCD fixes

Back from the land of GUI software. I have bought one more Flyswatter JTAG recently and now have two boards connected to my home box. Unfortunately both Flyswatters got the same USB serial number so stock openocd opens only the first device it stumbles upon. Here is small patch that adds ft2232_index command to OpenOCD FTDI driver that allows to point at specific device to open. Works only with libftdi. In the same directory you can find my configs for AR71XX-based RouterStation Pro and Portwell's CAM-0010 device based on Octeon CN3010

Minor OpenOCD fixes

>Back from the land of GUI software. I have bought one more Flyswatter JTAG recently and now have two boards connected to my home box. Unfortunately both Flyswatters got the same USB serial number so stock openocd opens only the first device it stumbles upon. Here is small patch that adds ft2232_index command to OpenOCD FTDI driver that allows to point at specific device to open. Works only with libftdi. In the same directory you can find my configs for AR71XX-based RouterStation Pro and Portwell's CAM-0010 device based on Octeon CN3010

Minor OpenOCD fixes

Back from the land of GUI software. I have bought one more Flyswatter JTAG recently and now have two boards connected to my home box. Unfortunately both Flyswatters got the same USB serial number so stock openocd opens only the first device it stumbles upon. Here is small patch that adds ft2232_index command to OpenOCD FTDI driver that allows to point at specific device to open. Works only with libftdi. In the same directory you can find my configs for AR71XX-based RouterStation Pro and Portwell's CAM-0010 device based on Octeon CN3010

FreeBSD-current on the RouterStation Pro

I've been working on shoehorning FreeBSD-current onto the RouterStation Pro. It's a rather cheap but nice Atheros MIPS board by Ubiquity. It was tricky, but doable. I now have a cut down kernel and memory root filesystem stored on the on-board flash - enough to run it as a basic access point.FreeBSD-current has support for the chipset - it's the AR71XX kernel config file. I've added a few more devices (USB, disk, MSDOS, redboot) to the default kernel. The default cross-build system works just fine.I've been using TFTP loaded kernel+mdroot images to test out standalone functionality, along with TFTP + NFS root images to run a mostly-full development environment.The board uses RedBoot as a bootloader. There's a tool by Ubiquity which generates firmware images which are understood by the bootloader and will overwrite the non-system area of the on-board flash. This makes dumping an image onto the system highly trivial. The bootloader can also boot the (uncompresed) kernel+mdroot images via TFTP as well as having a compressed version written to flash - so I can easily test the standalone images without constant re-flashing.RedBoot makes it easy to TFTP upload a replacement flash image written out by the Ubiquity tool. It takes care of erasing, repartitioning and copying into the onboard flash without overwriting or damaging the RedBoot code, flash partition and configuration areas.There was a bug with the PCI probe code until a couple of days ago where PCI cards (ie, stuff like the mini-PCI radio cards!) wouldn't be enumerated unless bootverbose was specified. Gonzo traced it down to a missing DELAY() - Linux was waiting a lot longer between probes than FreeBSD was. PCI devices now probe and attach correctly.The geom_redboot module doesn't attach to the on-board flash (device "spi") - it only probes "cfi" flash devices. A quick patch to src/sys/geom/geom_redboot.c to also probe flash/spi has fixed this. I've asked Gonzo/Warner to investigate this and possibly commit a fix. With this in place, FreeBSD can mount a compressed, read-only filesystem from the "rootfs" flash slice rather than needing to pack a kernel+mdroot into the kernel flash slice.So far, so good. I've written out a basic image that can be coaxed to be an open (no authentication) access point.Interesting links:Instructions on re-flashing the unit (but it doesn't mention that the reset button must be held down during power on!): http://www.usualcoding.eu/post/2010/01/15/Flash-OpenWRT-on-the-Ubiquiti-RouterStation-ProMy FreeBSD router-station pro wiki: http://wiki.freebsd.org/AdrianChadd/UbiquityRouterstationProMy work-in-progress tarball of "stuff" for building firmware images for the RouterStation Pro: http://people.freebsd.org/~adrian/rspro/

FreeBSD-current on the RouterStation Pro

I've been working on shoehorning FreeBSD-current onto the RouterStation Pro. It's a rather cheap but nice Atheros MIPS board by Ubiquity. It was tricky, but doable. I now have a cut down kernel and memory root filesystem stored on the on-board flash - enough to run it as a basic access point.

FreeBSD-current has support for the chipset - it's the AR71XX kernel config file. I've added a few more devices (USB, disk, MSDOS, redboot) to the default kernel. The default cross-build system works just fine.

I've been using TFTP loaded kernel+mdroot images to test out standalone functionality, along with TFTP + NFS root images to run a mostly-full development environment.

The board uses RedBoot as a bootloader. There's a tool by Ubiquity which generates firmware images which are understood by the bootloader and will overwrite the non-system area of the on-board flash. This makes dumping an image onto the system highly trivial. The bootloader can also boot the (uncompresed) kernel+mdroot images via TFTP as well as having a compressed version written to flash - so I can easily test the standalone images without constant re-flashing.

RedBoot makes it easy to TFTP upload a replacement flash image written out by the Ubiquity tool. It takes care of erasing, repartitioning and copying into the onboard flash without overwriting or damaging the RedBoot code, flash partition and configuration areas.

There was a bug with the PCI probe code until a couple of days ago where PCI cards (ie, stuff like the mini-PCI radio cards!) wouldn't be enumerated unless bootverbose was specified. Gonzo traced it down to a missing DELAY() - Linux was waiting a lot longer between probes than FreeBSD was. PCI devices now probe and attach correctly.

The geom_redboot module doesn't attach to the on-board flash (device "spi") - it only probes "cfi" flash devices. A quick patch to src/sys/geom/geom_redboot.c to also probe flash/spi has fixed this. I've asked Gonzo/Warner to investigate this and possibly commit a fix. With this in place, FreeBSD can mount a compressed, read-only filesystem from the "rootfs" flash slice rather than needing to pack a kernel+mdroot into the kernel flash slice.

So far, so good. I've written out a basic image that can be coaxed to be an open (no authentication) access point.

Interesting links:

Instructions on re-flashing the unit (but it doesn't mention that the reset button must be held down during power on!): http://www.usualcoding.eu/post/2010/01/15/Flash-OpenWRT-on-the-Ubiquiti-RouterStation-Pro

My FreeBSD router-station pro wiki: http://wiki.freebsd.org/AdrianChadd/UbiquityRouterstationPro

My work-in-progress tarball of "stuff" for building firmware images for the RouterStation Pro: http://people.freebsd.org/~adrian/rspro/

Booting instructions for FreeBSD on Cavium Octeon

Here's a quick note on how to net boot the Cavium EBT3000 board running uboot. The Cavium kernel is still a work in progress as I restore all the fixes I made to an earlier version of this code that I was unable to release.You'll need to break into the boot sequence for this board. Usually that's just hitting ^C before uboot starts to load the kernel.Once you have a uboot prompt, similar to below, you'll be ready to start to fix the environment for netbooting.
U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxxOCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)DRAM: 2048 MBFlash: 8 MBIPD backpressure workaround verified, took 14 loopsClearing DRAM........ doneBIST check passed.Net: octeth0, octeth1, octeth2, octeth3 Bus 0 (CF Card): OK ide 0: Model: SanDisk SDCFB-1024 Firm: Rev 0.00 Ser#: X0308 20050726142909 Type: Removable Hard Disk Capacity: 977.4 MB = 0.9 GB (2001888 x 512)Octeon ebt3800#
It couldn't be easier to setup for netbooting. You'll need to setup an tftp server. I had to install the FreeBSD port from net/freebsd-tftpd to get a working tftpd, since there appears to be some issues with the stock FreeBSD tftpd. Once I had that, I set different addresses like so:
set ipaddr 10.0.0.30set netmask 255.255.255.0set serverip 10.0.0.10set gatewayip 10.0.0.1set bootdelay 0saveenv
This sets the board's address to 10.0.0.30/24. 10.0.0.1 is the router to the other networks. The tftp command will now use tftp from host 10.0.0.10 to get the named file. saveenv makes these setting permanent. I usually reboot at this point to make sure that the settings took.Next, you'll need to know how to load a FreeBSD kernel. This is also fairly straight-forward, although finding the right addresses took a bit of guesswork. You are tftping the ELF kernel to a chunk of memory. The bootoctlinux command then takes that image and copies it to the right place and starts executing it. So you have to be sure whereever you load it doesn't clash with uboot's memory use or with where the kernel will ultimately reside.
tftp a800000 kernel.octean1-32bootoctlinux a800000 numcores=1
As with any u-boot platform, it can often be useful to define macro commands. I did the following:
set fbsd 'tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1'
so that I don't have to keep typing it over and over. I can just type
run fbsd
to run my next test.The Cavium port is in a state of flux right now. The latest status as I am preparing this to be published is that there's an interrupt problem that's causing the serial port to be non-responsive after going to multi-user (eg, the login: prompt) and also the network stops responding. Also, I have to comment out the enableintr() call in mips_vector_init around line 365 of sys/mips/mips/machdep.c. I'm trying to hammer these issues out, and I think they are related. Earlier versions of the code worked with earlier versions of FreeBSD, so I'm sure this is just a shift in interfaces that needs to be accounted for.Here's the latest slightly trimmed log from a boot:
U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxxOCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)DRAM: 2048 MBFlash: 8 MBIPD backpressure workaround verified, took 32 loopsClearing DRAM........ doneBIST check passed.Net: octeth0, octeth1, octeth2, octeth3 Bus 0 (CF Card): OK ide 0: Model: 512MB CKS Firm: 2N3-0925 Ser#: 2435E7C34A52930007 Type: Removable Hard Disk Capacity: 491.6 MB = 0.4 GB (1006992 x 512)Octeon ebt3000# printbootdelay=0baudrate=115200download_baudrate=115200ls=fatls ide 0nuke_env=protect off BFBFE000 BFBFffff; erase BFBFE000 BFBFffffautoload=nipaddr=10.0.0.31gatewayip=10.0.0.1netmask=255.255.255.0serverip=10.0.0.16fbsd=tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1fbsd_cf=fatload ide 0 a800000 kernel.imp; bootoctlinux a800000 numcores=1coremask_override=0xffffnumcores=16stdin=serialstdout=serialstderr=serialEnvironment size: 1274/8188 bytesOcteon ebt3000# run fbsdocteth0: Down 10Mbs Half duplex, (port 16)Using octeth0 deviceTFTP from server 10.0.0.16; our IP address is 10.0.0.31Filename 'kernel.octeon1-32'.Load address: 0xa800000Loading: octeth0: Up 100Mbs Half duplex, (port 16)########################doneBytes transferred = 3370489 (336df9 hex)argv[2]: numcores=1ELF file is 32 bitSkipping non LOAD program header (type 0x6)Skipping non LOAD program header (type 0x3)Skipping non LOAD program header (type 0x70000000)Allocated memory for ELF segment: addr: 0x1000000, size 0x295a60Loading .text @ 0x81000000 (2064384 bytes)Loading .MIPS.stubs @ 0x811f8000 (16 bytes)Loading .rodata @ 0x811fa000 (33248 bytes)Loading .reginfo @ 0x812021e0 (24 bytes)Loading .rodata.str1.4 @ 0x812021f8 (189300 bytes)Loading set_sysctl_set @ 0x8123056c (3496 bytes)Loading set_sysinit_set @ 0x81231314 (1684 bytes)Loading set_sysuninit_set @ 0x812319a8 (952 bytes)Loading .interp @ 0x81231d60 (13 bytes)Loading .dynsym @ 0x81231d70 (78320 bytes)Loading .dynstr @ 0x81244f60 (72163 bytes)Loading .hash @ 0x81256944 (35984 bytes)Loading set_kdb_dbbe_set @ 0x8125f5d4 (8 bytes)Loading set_modmetadata_set @ 0x8125f5dc (288 bytes)Loading set_cons_set @ 0x8125f6fc (8 bytes)Loading .data @ 0x8125f704 (116476 bytes)Loading set_pcpu @ 0x8127be00 (3136 bytes)Loading .got @ 0x8127ca40 (5296 bytes)Loading .rld_map @ 0x8127def0 (4 bytes)Loading .sdata @ 0x8127def4 (12 bytes)Clearing .bss @ 0x8127df00 (97120 bytes)## Loading Linux kernel with entry point: 0x81000000 ...Bootloader: Done loading app on coremask: 0x1Boot Descriptor Ver: 6 -> 1/2 CPU clock: 500MHz Dram: 2048 MB Board Type: 2 Revision: 4/0 Octeon Chip: 0 Rev 0/0 Mac Address 00.0F.B7.10.1B.32 (14)octeon_dram == 80000000reduced to ram: 2048 MBReal memory bytes is 7ffff000phys_avail[0] = 1296000 phys_avail[1] = fffffffrealmem_bytes is now at 5ffff000Next block of memory goes from 20000000 to 7ffff000Total DRAM Size 0x80000000Bank 0 = 0x 1296000 -> 0x FFFFFFFBank 1 = 0x20000000 -> 0x7FFFF000physmem: 0x6ed69Cache info: picache_stride = 4096 picache_loopcount = 8 pdcache_stride = 128 pdcache_loopcount = 64cpu0: Cavium processor v1.0 MMU: Standard TLB, 32 entries L1 i-cache: 4 ways of 64 sets, 128 bytes per line L1 d-cache: 64 ways of 1 sets, 128 bytes per line Config1=0xbe3303da Config3=0x80Physical memory chunk(s):0x1296000 - 0xfffefff, 248942592 bytes (60777 pages)0x20000000 - 0x7fffefff, 1610608640 bytes (393215 pages)Maxmem is 0x7ffff000KDB: debugger backends: ddbKDB: current backend: ddbhz=100 cyl_per_hz:250000 cyl_per_usec:250 freq:250000000 cyl_per_hz:2500000 cyl_per_sec:250000000Copyright (c) 1992-2010 The FreeBSD Project.Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved.FreeBSD is a registered trademark of The FreeBSD Foundation.FreeBSD 9.0-CURRENT #141 r202986:202997M: Mon Jan 25 20:07:07 MST 2010 [email protected]:/tmp/imp/be-obj/mips/pe/imp/svn/head/sys/OCTEON1-32 mipsreal memory = 1859555328 (1815972K bytes)Physical memory chunk(s):0x013ab000 - 0x0fffefff, 247808000 bytes (60500 pages)0x20000000 - 0x7d830fff, 1568870400 bytes (383025 pages)avail memory = 1813180416 (1729MB)mem: nfslock: pseudo-devicenull: nexus0: Compact flash found in bootbus region 3 (8 bit).rgmii0: on nexus0rgmx0: Ethernet address: 00:0f:b7:10:1b:32rgmx0: type (null), full duplexrgmx1: Ethernet address: 00:0f:b7:10:1b:33rgmx1: type (null), full duplexrgmx2: Ethernet address: 00:0f:b7:10:1b:34rgmx2: type (null), full duplexrgmx3: Ethernet address: 00:0f:b7:10:1b:35rgmx3: type (null), full duplexrgmii0: [FILTER]cf0: on nexus0clock0: on nexus0clock0: [FILTER]obio0 at mem 0x1 flags 0x1 on nexus0uart1: on obio0uart1: [FILTER]uart1: fast interruptuart1: console (115200,n,8,1)uart0: on obio0uart0: [FILTER]uart0: fast interruptuart0: console (115200,n,8,1)Device configuration finished.Reducing kern.maxvnodes 117045 -> 100000Timecounter "MIPS32" frequency 250000000 Hz quality 800Timecounters tick every 10.000 msecGEOM: cf0: partition 2 does not start on a track boundary.GEOM: cf0: partition 2 does not end on a track boundary.GEOM: cf0: partition 1 does not start on a track boundary.GEOM: cf0: partition 1 does not end on a track boundary.GEOM: cf0s2: geometry does not match label (64h,32s != 16h,63s).Trying to mount root from ufs:cf0s2aWARNING: / was not properly dismountedwarning: no time-of-day clock registered, system time will not be set accuratelystart_init: trying /sbin/initSetting hostuuid: 1c99760e-1dd2-11b2-bd6a-00156dc12838.Setting hostid: 0xb0f278c1.No suitable dump device was found.Entropy harvesting:.Starting file system checks:t_delta 16.010fac783006c1a6 too long/dev/cf0s2a: 11100 files, 256973 used, 150946 free (426 frags, 18815 blocks, 0.1% fragmentation)Mounting local file systems:.t_delta 15.fece5003dd62224a too shortSetting hostname: ebt3000. SIOCSTIFFLAGS UP/RunningStarting Network: lo0 rgmx0 rgmx1 rgmx2 rgmx3.lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 rgmx0: flags=8843 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:32 inet 10.0.0.31 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselectrgmx1: flags=8802 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:33 media: Ethernet autoselectrgmx2: flags=8802 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:34 media: Ethernet autoselectrgmx3: flags=8802 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:35 media: Ethernet autoselectStarting devd.Creating and/or trimming log files.Starting syslogd./etc/rc: WARNING: Dump device does not exist. Savecore not run.ELF ldconfig path: /lib /usr/lib /usr/lib/compatClearing /tmp (X related).Updating motd:.Starting cron.Starting inetd.Starting background file system checks in 60 seconds.Tue Jan 26 02:07:16 UTC 2010FreeBSD/mips (ebt3000) (ttyu0)login:

Booting instructions for FreeBSD on Cavium Octeon

Here's a quick note on how to net boot the Cavium EBT3000 board running uboot. The Cavium kernel is still a work in progress as I restore all the fixes I made to an earlier version of this code that I was unable to release.

You'll need to break into the boot sequence for this board. Usually that's just hitting ^C before uboot starts to load the kernel.

Once you have a uboot prompt, similar to below, you'll be ready to start to fix the environment for netbooting.
U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 14 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: SanDisk SDCFB-1024 Firm: Rev 0.00 Ser#: X0308 20050726142909
Type: Removable Hard Disk
Capacity: 977.4 MB = 0.9 GB (2001888 x 512)

Octeon ebt3800#


It couldn't be easier to setup for netbooting. You'll need to setup an tftp server. I had to install the FreeBSD port from net/freebsd-tftpd to get a working tftpd, since there appears to be some issues with the stock FreeBSD tftpd. Once I had that, I set different addresses like so:
set ipaddr 10.0.0.30
set netmask 255.255.255.0
set serverip 10.0.0.10
set gatewayip 10.0.0.1
set bootdelay 0
saveenv

This sets the board's address to 10.0.0.30/24. 10.0.0.1 is the router to the other networks. The tftp command will now use tftp from host 10.0.0.10 to get the named file. saveenv makes these setting permanent. I usually reboot at this point to make sure that the settings took.

Next, you'll need to know how to load a FreeBSD kernel. This is also fairly straight-forward, although finding the right addresses took a bit of guesswork. You are tftping the ELF kernel to a chunk of memory. The bootoctlinux command then takes that image and copies it to the right place and starts executing it. So you have to be sure whereever you load it doesn't clash with uboot's memory use or with where the kernel will ultimately reside.
tftp a800000 kernel.octean1-32
bootoctlinux a800000 numcores=1


As with any u-boot platform, it can often be useful to define macro commands. I did the following:
set fbsd 'tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1'
so that I don't have to keep typing it over and over. I can just type
run fbsd
to run my next test.

The Cavium port is in a state of flux right now. The latest status as I am preparing this to be published is that there's an interrupt problem that's causing the serial port to be non-responsive after going to multi-user (eg, the login: prompt) and also the network stops responding. Also, I have to comment out the enableintr() call in mips_vector_init around line 365 of sys/mips/mips/machdep.c. I'm trying to hammer these issues out, and I think they are related. Earlier versions of the code worked with earlier versions of FreeBSD, so I'm sure this is just a shift in interfaces that needs to be accounted for.

Here's the latest slightly trimmed log from a boot:


U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 32 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: 512MB CKS Firm: 2N3-0925 Ser#: 2435E7C34A52930007
Type: Removable Hard Disk
Capacity: 491.6 MB = 0.4 GB (1006992 x 512)
Octeon ebt3000# print
bootdelay=0
baudrate=115200
download_baudrate=115200
ls=fatls ide 0
nuke_env=protect off BFBFE000 BFBFffff; erase BFBFE000 BFBFffff
autoload=n
ipaddr=10.0.0.31
gatewayip=10.0.0.1
netmask=255.255.255.0
serverip=10.0.0.16
fbsd=tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1
fbsd_cf=fatload ide 0 a800000 kernel.imp; bootoctlinux a800000 numcores=1
coremask_override=0xffff
numcores=16
stdin=serial
stdout=serial
stderr=serial

Environment size: 1274/8188 bytes
Octeon ebt3000# run fbsd
octeth0: Down 10Mbs Half duplex, (port 16)
Using octeth0 device
TFTP from server 10.0.0.16; our IP address is 10.0.0.31
Filename 'kernel.octeon1-32'.
Load address: 0xa800000
Loading: octeth0: Up 100Mbs Half duplex, (port 16)
########################
done
Bytes transferred = 3370489 (336df9 hex)
argv[2]: numcores=1
ELF file is 32 bit
Skipping non LOAD program header (type 0x6)
Skipping non LOAD program header (type 0x3)
Skipping non LOAD program header (type 0x70000000)
Allocated memory for ELF segment: addr: 0x1000000, size 0x295a60
Loading .text @ 0x81000000 (2064384 bytes)
Loading .MIPS.stubs @ 0x811f8000 (16 bytes)
Loading .rodata @ 0x811fa000 (33248 bytes)
Loading .reginfo @ 0x812021e0 (24 bytes)
Loading .rodata.str1.4 @ 0x812021f8 (189300 bytes)
Loading set_sysctl_set @ 0x8123056c (3496 bytes)
Loading set_sysinit_set @ 0x81231314 (1684 bytes)
Loading set_sysuninit_set @ 0x812319a8 (952 bytes)
Loading .interp @ 0x81231d60 (13 bytes)
Loading .dynsym @ 0x81231d70 (78320 bytes)
Loading .dynstr @ 0x81244f60 (72163 bytes)
Loading .hash @ 0x81256944 (35984 bytes)
Loading set_kdb_dbbe_set @ 0x8125f5d4 (8 bytes)
Loading set_modmetadata_set @ 0x8125f5dc (288 bytes)
Loading set_cons_set @ 0x8125f6fc (8 bytes)
Loading .data @ 0x8125f704 (116476 bytes)
Loading set_pcpu @ 0x8127be00 (3136 bytes)
Loading .got @ 0x8127ca40 (5296 bytes)
Loading .rld_map @ 0x8127def0 (4 bytes)
Loading .sdata @ 0x8127def4 (12 bytes)
Clearing .bss @ 0x8127df00 (97120 bytes)
## Loading Linux kernel with entry point: 0x81000000 ...
Bootloader: Done loading app on coremask: 0x1
Boot Descriptor Ver: 6 -> 1/2 CPU clock: 500MHz
Dram: 2048 MB Board Type: 2 Revision: 4/0
Octeon Chip: 0 Rev 0/0 Mac Address 00.0F.B7.10.1B.32 (14)
octeon_dram == 80000000
reduced to ram: 2048 MBReal memory bytes is 7ffff000
phys_avail[0] = 1296000 phys_avail[1] = fffffff
realmem_bytes is now at 5ffff000
Next block of memory goes from 20000000 to 7ffff000

Total DRAM Size 0x80000000
Bank 0 = 0x 1296000 -> 0x FFFFFFF
Bank 1 = 0x20000000 -> 0x7FFFF000

physmem: 0x6ed69Cache info:
picache_stride = 4096
picache_loopcount = 8
pdcache_stride = 128
pdcache_loopcount = 64
cpu0: Cavium processor v1.0
MMU: Standard TLB, 32 entries
L1 i-cache: 4 ways of 64 sets, 128 bytes per line
L1 d-cache: 64 ways of 1 sets, 128 bytes per line
Config1=0xbe3303da
Config3=0x80
Physical memory chunk(s):
0x1296000 - 0xfffefff, 248942592 bytes (60777 pages)
0x20000000 - 0x7fffefff, 1610608640 bytes (393215 pages)
Maxmem is 0x7ffff000
KDB: debugger backends: ddb
KDB: current backend: ddb
hz=100 cyl_per_hz:250000 cyl_per_usec:250 freq:250000000 cyl_per_hz:2500000 cyl_per_sec:250000000
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #141 r202986:202997M: Mon Jan 25 20:07:07 MST 2010
[email protected]:/tmp/imp/be-obj/mips/pe/imp/svn/head/sys/OCTEON1-32 mips
real memory = 1859555328 (1815972K bytes)
Physical memory chunk(s):
0x013ab000 - 0x0fffefff, 247808000 bytes (60500 pages)
0x20000000 - 0x7d830fff, 1568870400 bytes (383025 pages)
avail memory = 1813180416 (1729MB)
mem:
nfslock: pseudo-device
null:
nexus0:
Compact flash found in bootbus region 3 (8 bit).
rgmii0: on nexus0
rgmx0: Ethernet address: 00:0f:b7:10:1b:32
rgmx0: type (null), full duplex
rgmx1: Ethernet address: 00:0f:b7:10:1b:33
rgmx1: type (null), full duplex
rgmx2: Ethernet address: 00:0f:b7:10:1b:34
rgmx2: type (null), full duplex
rgmx3: Ethernet address: 00:0f:b7:10:1b:35
rgmx3: type (null), full duplex
rgmii0: [FILTER]
cf0: on nexus0
clock0: on nexus0
clock0: [FILTER]
obio0 at mem 0x1 flags 0x1 on nexus0
uart1: on obio0
uart1: [FILTER]
uart1: fast interrupt
uart1: console (115200,n,8,1)
uart0: on obio0
uart0: [FILTER]
uart0: fast interrupt
uart0: console (115200,n,8,1)
Device configuration finished.
Reducing kern.maxvnodes 117045 -> 100000
Timecounter "MIPS32" frequency 250000000 Hz quality 800
Timecounters tick every 10.000 msec
GEOM: cf0: partition 2 does not start on a track boundary.
GEOM: cf0: partition 2 does not end on a track boundary.
GEOM: cf0: partition 1 does not start on a track boundary.
GEOM: cf0: partition 1 does not end on a track boundary.
GEOM: cf0s2: geometry does not match label (64h,32s != 16h,63s).
Trying to mount root from ufs:cf0s2a
WARNING: / was not properly dismounted
warning: no time-of-day clock registered, system time will not be set accurately
start_init: trying /sbin/init
Setting hostuuid: 1c99760e-1dd2-11b2-bd6a-00156dc12838.
Setting hostid: 0xb0f278c1.
No suitable dump device was found.
Entropy harvesting:.
Starting file system checks:
t_delta 16.010fac783006c1a6 too long
/dev/cf0s2a: 11100 files, 256973 used, 150946 free (426 frags, 18815 blocks, 0.1% fragmentation)
Mounting local file systems:.
t_delta 15.fece5003dd62224a too short
Setting hostname: ebt3000.
SIOCSTIFFLAGS UP/Running
Starting Network: lo0 rgmx0 rgmx1 rgmx2 rgmx3.
lo0: flags=8049 metric 0 mtu 16384
options=3
inet 127.0.0.1 netmask 0xff000000
rgmx0: flags=8843 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:32
inet 10.0.0.31 netmask 0xffffff00 broadcast 10.0.0.255
media: Ethernet autoselect
rgmx1: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:33
media: Ethernet autoselect
rgmx2: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:34
media: Ethernet autoselect
rgmx3: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:35
media: Ethernet autoselect
Starting devd.
Creating and/or trimming log files.
Starting syslogd.
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Clearing /tmp (X related).
Updating motd:.
Starting cron.
Starting inetd.
Starting background file system checks in 60 seconds.

Tue Jan 26 02:07:16 UTC 2010

FreeBSD/mips (ebt3000) (ttyu0)

login:

Post-mortum on projects/mips branch

Greetings to one and all. As you have read elsewhere, I recently merged all the changes from the projects/mips branch onto head. In other reports, I've made cryptic reference to the branch being damaged. I thought I'd go through all the problems we encountered running this development effort on the projects/mips branch.First, we created the branch in the normal way:
svn cp svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/mips
and then started committing to it. So far so good.Then it came time to merge in changes from head. Here is where we made our first mistake. We merged just the kernel changes, and not the entire tree. The next time we merged the entire tree. The effect of this sequence meant that the changes between the creation and the first merge in the rest of the tree (outside of src/sys) were lost. I don't know if this is the result of a buggy svn client, or if they are a fundamental flaw in svn. So when it came time to collapse the results back into head, we couldn't just do a reverse integration, or even a more conventional merge.Next, we pulled in pieces of the projects/mips tree into head, either by hand or with the svn merge command. Doing both was a mistake, I think. Although svn coped with later merged from head into projects/mips fairly well, occasionally we'd have issues to sort out.So, when it came time to merge everything back into head, we were left with a number of difficult choices. We opted to copy the new files/directories in the repo (to preserve the history) and merge patches by hand with svn log entries. The latter went really well. There were no problems with it apart from the odd fuzz and .rej file to sort out (no different than a normal merge). The former, however, causes a lot of problems.First, it created svn:mergeinfo entries on the files I copied over. This conflicts with the project's "merge everything from sys" edict. This happened because I did the copy in the server rather than the client (which was an attempt to preserve history). It was a simple matter to delete these entries when it was brought to my attention.Next, somehow we created the files on the branch without the required svn:keywords, svn:eol-style and svn:mime-type properties. This caused problems when we went to commit changes to these files. The precommit complained, at this point, that the files lacked svn:keywords, and we had to add $FreeBSD$ by hand (since the copy didn't complaint that they were lacking). Also, I didn't discover these problems on my own. Other sharp-eyed individuals saw the problems and reported them to me.Finally, since I did the copies on the server, there was no way to batch them. When I copied a directory, I got the whole directory. When I copied a file, I got the file. Each one generated a commit message.I'm unsure what other damage was lurking in the projects/mips tree, but the lack of properties, the inability to easily use merge info and the missing commits lead me to the conclusion that it was easier to abandon the branch and create a new branch if needed in the future.The mips team plans to create more branches, that are more for special purposes, that will be shorter lived in the future. We have no further plans to have one monolithic mips branch that acts as a staging ground for commits into head.

Post-mortum on projects/mips branch

Greetings to one and all. As you have read elsewhere, I recently merged all the changes from the projects/mips branch onto head. In other reports, I've made cryptic reference to the branch being damaged. I thought I'd go through all the problems we encountered running this development effort on the projects/mips branch.

First, we created the branch in the normal way:
svn cp svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/mips

and then started committing to it. So far so good.

Then it came time to merge in changes from head. Here is where we made our first mistake. We merged just the kernel changes, and not the entire tree. The next time we merged the entire tree. The effect of this sequence meant that the changes between the creation and the first merge in the rest of the tree (outside of src/sys) were lost. I don't know if this is the result of a buggy svn client, or if they are a fundamental flaw in svn. So when it came time to collapse the results back into head, we couldn't just do a reverse integration, or even a more conventional merge.

Next, we pulled in pieces of the projects/mips tree into head, either by hand or with the svn merge command. Doing both was a mistake, I think. Although svn coped with later merged from head into projects/mips fairly well, occasionally we'd have issues to sort out.

So, when it came time to merge everything back into head, we were left with a number of difficult choices. We opted to copy the new files/directories in the repo (to preserve the history) and merge patches by hand with svn log entries. The latter went really well. There were no problems with it apart from the odd fuzz and .rej file to sort out (no different than a normal merge). The former, however, causes a lot of problems.

First, it created svn:mergeinfo entries on the files I copied over. This conflicts with the project's "merge everything from sys" edict. This happened because I did the copy in the server rather than the client (which was an attempt to preserve history). It was a simple matter to delete these entries when it was brought to my attention.

Next, somehow we created the files on the branch without the required svn:keywords, svn:eol-style and svn:mime-type properties. This caused problems when we went to commit changes to these files. The precommit complained, at this point, that the files lacked svn:keywords, and we had to add $FreeBSD$ by hand (since the copy didn't complaint that they were lacking). Also, I didn't discover these problems on my own. Other sharp-eyed individuals saw the problems and reported them to me.

Finally, since I did the copies on the server, there was no way to batch them. When I copied a directory, I got the whole directory. When I copied a file, I got the file. Each one generated a commit message.

I'm unsure what other damage was lurking in the projects/mips tree, but the lack of properties, the inability to easily use merge info and the missing commits lead me to the conclusion that it was easier to abandon the branch and create a new branch if needed in the future.

The mips team plans to create more branches, that are more for special purposes, that will be shorter lived in the future. We have no further plans to have one monolithic mips branch that acts as a staging ground for commits into head.