Category Archives: mips

… mmm emulators.

I occasionally get asked to test out FreeBSD/MIPS patches for people, as they don't have physical hardware present. I can understand that - the hardware is cheap and plentiful, but not everyone wants to have a spare access point around just to test out MIPS changes on.

However QEMU does a pretty good job of emulating MIPS if you're just testing out non-hardware patches. There's even instructions on the FreeBSD wiki for how to do this! So I decided to teach my wifi build system about the various QEMU MIPS emulator targets so it can spit out a kernel and mfsroot to use for QEMU.

Thus:

https://github.com/freebsd/freebsd-wifi-build/wiki/MipsQemuEmulatorImages

It turns out that it wasn't all that hard. The main trick was to use qemu-devel, not qemu. There are bugs in the non-development QEMU branch that mean it works great for Linux but not FreeBSD.

The kernel configurations in FreeBSD had bitrotted a little bit (they were missing the random device, for example) but besides that the build, install and QEMU startup just worked. I now have FreeBSD/MIPS of each variety (32 bit, 64 bit, Little-Endian, Big-Endian) running under QEMU and building FreeBSD-HEAD as a basic test.

Next is figuring out how to build gdb to target each of the above and have it speak to the QEMU GDB stub. That should make it very easy to do MIPS platform debugging.

I also hear rumours about this stuff working somewhat for ARM and PPC, so I'll see how hard it is to run QEMU for those platforms and whether FreeBSD will just boot and run on each.

The gymnastics required to just do a "HALT" for MIPS..

So, it turns out there's no nice, guaranteed way to implement a HALT style setup for MIPS in the idle thread in the kernel. I'll braindump what happened about two years ago to try and address this.

(And I do hope that the implementation actually works!)

When the kernel isn't running any work, it will schedule the "idle" thread to run. This has the simple(!) task of entering a sleep state, only to wake up when another interrupt fires. This saves power consumption and generates less heat.

However, it's a little trickier than just "enter idle state."

The only way to exit it is to receive an interrupt. Now, this can be device interrupts; this can be timer interrupts (and yes; the timer is a kind of device, tsk..) but without it, the CPU will stay halted. This isn't a big deal - the UNIX kernel tends to be one big event processing "thing" and these include both software and hardware events. If it's a software event that needs to happen "now", the kernel won't go into the idle loop - it'll just run the needed event. If it has to happen in the future, it'll schedule it to occur after some time has elapsed - and this will be driven by a timer interrupt. An interrupt would wake up the system from the HALT state. If the interrupt occured just before the HALT instruction was executed, there'd be a very small window where the interrupt would occur - the interrupt routine would be called, complete, and then the HALT instruction would next execute.

Now, imagine you're an ethernet driver on FreeBSD-10 where the interrupt handler just scheduled some ithread to run in the future. Here's one example of what may happen:

  • The idle thread is executed;
  • An interrupt occurs, say to signal completion of some ethernet frame transmission;
  • The CPU kicks off the interrupt code;
  • The interrupt code doesn't find a fast interrupt handler but finds an ithread; so it schedules the ithread to run;
  • The scheduler goes to choose another thread to run and finds an ithread scheduled at a higher priority, so it schedules that;
  • The ethernet driver code in the ithread runs;
  • The scheduler then exits and re-enters the idle loop.
Everything is fine, right?

But, with the event timer changes that came in during the 9.x time frame, the halt code is now in a critical section.

So now, the following happens:

  • The idle thread is executed;
  • critical_enter() is called;
  • An interrupt occurs, which calls the fast handler, which schedules the ithread to run;
  • .. but critical_enter() has set a flag that says "no preemption", so the ithread doesn't get to run;
  • Control is returned to the idle function;
  • The idle thread gets to the point where it executes "HALT";
  • The idle thread continues to run, executing HALT.
In this instance, the ithread is scheduled but can't run before HALT runs. The ithread only runs after the next interrupt occurs.

I noticed this was happening when doing traffic tests on FreeBSD-HEAD between 802.11 and ethernet interfaces. The atheros 802.11 hardware implemented interrupt moderation so there wouldn't be tens of thousands of interrupts a second being generated. But what I saw was occasionally the receive queue being filled by packets and not drained fast enough. When digging into it, I found that due to interrupt moderation, if the interrupt came in just before WAIT was executed, the 802.11 receive function was taking a long time (sometimes up to milliseconds) to run after the interrupt came in and the ithread was actually scheduled. If I had an interrupt for each received packet, the amount of time between interrupts would have been very small (20,000 packets a second, so around 1/20000 sec per interrupt) and this problem would've been masked. But with moderated interrupts, it would be 750 microseconds or so before the next receive interrupt was generated.

Now, this is messy. There's some hacks in the idle loop code to try and skip the halt bit if the scheduler detects there's something to run. But there's still a small race window there which needs to be closed.

How can this be solved?

Apparently - the two instructions STI;HLT on x86 are atomic. Ie, there's no race window between them. If an interrupt comes through, HLT doesn't run. This doesn't happen for MWAIT or the ACPI sleep states and I am concerned we're still possibly hitting this race window from time to time. The specific behaviour is that the STI causes interrupts to be enabled following the next instruction. So yes, there's no window.

All that's left now is to make sure that interrupts are disabled before you do the scheduler check so no new interrupt processing on that thread can be scheduled. Ie, when entering cpu_idle():

  • Call critical_enter();
  • Disable interrupts;
  • See if the scheduler has anything to do - if so, enable interrupts and skip calling the idle loop
  • If there's nothing to do - and since interrupts are disabled, nothing new will have happened (like an interrupt scheduling an ithread) then just call the idle function
    • Which may or may not enable interrupts before entering the idle loop (you enter ACPI with interrupts disabled, but you enter HLT with interrupts enabled)
  • .. and then call critical_exit(), which will let the kernel continue preempting.
For MIPS however, there's a clever hack. (No, not from me.)

Here's how it works:
  • In the idle loop, it calls mips_wait()
  • mips_wait() is a bit of assembly code that will:
    • disable interrupts
    • see if the scheduler sees anything running
    • .. and if so, it doesn't bother running the WAIT instruction! Just enable interrupts and jump over the WAIT;
  • .. but the bit of code that re-enables interrupts and calls WAIT is aligned to a 16 byte boundary and the address is a symbol (MipsWaitStart).
  • Then in the exception handling code (MipsKernIntr), it sees if the instruction pointer where the exception occured is in the 16 bytes (4 instructions) at MipsWaitStart.
  • .. if it is, it adjusts the return address from the interrupt to be after the WAIT instruction.
It's totally dirty and to be quite honest, I haven't at all tested it. Yes, it should be tested.


FreeBSD on PicoStation M2HP

After tplink, it was time to play with picostation m2hp.
image

image

I’ve learned a bunch of things from this experience and this entire bring up would have been impossible without the help from loos@ and adrian@

First of all, a usb-ttl serial adapter is needed to connect to the board:
image

Here,
black – GND
white – TX
green – RX

When you power it on, the boot up sequence looks like this:

U-Boot 1.1.4.2-s564 (Jul 19 2012 - 10:41:56)

Board: Ubiquiti Networks XM board (rev 1.0 e302)
DRAM: 32 MB
Flash: 8 MB
PCIe WLAN Module found (#1).
Net: eth0, eth1
Hit any key to stop autoboot: 0

Here is where you should top the autoboot and it will drop into uboot>.

ar7240> printenv
bootdelay=1
baudrate=115200
ethaddr=00:15:6d:0d:00:00
mtdids=nor0=ar7240-nor0
partition=nor0,0
mtddevnum=0
mtddevname=u-boot
filesize=10000
fileaddr=81000000
serverip=192.168.1.254
ethact=eth0
mtdparts=mtdparts=ar7240-nor0:256k(u-boot),64k(u-boot-env),1024k(kernel),5760k(rootfs),256k(cfg),64k(EEPROM)
bootcmd=bootm 0x9f050000
bootargs=console=tty0 root=31:03 rootfstype=squashfs init=/init
ipaddr=192.168.1.20
stdin=serial
stdout=serial
stderr=serial

Environment size: 452/65532 bytes
ar7240>

Again, mtdparts an important piece here to see how uboot expects the image layout:

mtdparts=ar7240-nor0:256k(u-boot),64k(u-boot-env),1024k(kernel),5760k(rootfs),256k(cfg),64k(EEPROM)

I picked up a working openwrt image and tried to load it.
My setup looks like this:
image

Black power adapter has 2 cables going to it:
o POE (yellow)- power over ethernet – which connects to the board
o LAN (green)- which connects to my working router

Laptop act as a tftp server here which is also connected to the router (via gray cable). This way laptop and board are both in the same network.
laptop has a tftp server running and I’ve assigned 192.168.1.254 to that network interface (em0 in my case) with “ifconfig alias”. This is because as you can see in uboot’s printenv, uboot expects the tftp server to be running at that address.
The board will obviously act as a tftp client.

Now, to transfer the image, after generating the image (which we will look into in a bit), copy that into /tftpboot on the server.

This is how you put board into “urescue” mode:

Board: Ubiquiti Networks XM board (rev 1.0 e302)
DRAM: 32 MB
Flash: 8 MB
PCIe WLAN Module found (#1).
Net: eth0, eth1
Hit any key to stop autoboot: 0
ar7240> ping 192.168.1.254
Using eth0 device
host 192.168.1.254 is alive
ar7240> urescue
Setting default IP 192.168.1.20
Starting TFTP server...
Using eth0 (192.168.1.20), address: 0x81000000
Waiting for connection: /

Now, board is waiting for image. Send it from the tftp server:

% tftp 192.168.1.20
tftp> bin
tftp> put PICOSTATION_M2HP.initial.img
Sent 5994346 bytes during 15.2 seconds in 11709 blocks
tftp>

board receiving the image:

Receiving file from 192.168.1.18:18401
Received 6040751 bytes
Firmware Version: XM.ar7240.FreeBSD

Alright this was the basics of how to upload a valid image onto the board. Fun part is to generate a *valid* image that board will accept.
I tried a bunch of different kenrconf’s available in the freebsd-wifi-build project but board was not accepting the generated images. After accepting the image, it used to fail like this:

Firmware check failed! (-2)

This error was coming from uboot which was proprietary and I could not find this error in any opensourced version of uboot.

General consensus about the reasons for this error was:
- bad layout of the firmware image
- wrong header (something that this uboot is not liking)

loos@ showed me a trick to look at images with hexdum. After looking at openwrt’s working image by “hexdump -c image”, I found out that uboot was expecting something along the lines of “XS2.ar7240.FreeBSD” as version string in its header. If it does not see “ar7240″ in the header, it would fail the check.

So, after all that and with ray@ suggestion of using lzma’ed kernel, I could boot up the board which later failed at mounting the rootfs:

U-Boot 1.1.4.2-s594 (Dec 5 2012 - 15:23:07)

Board: Ubiquiti Networks XM board (rev 1.0 e302)
DRAM: 32 MB
Flash: 8 MB
PCIe WLAN Module found (#1).
Net: eth0, eth1
Hit any key to stop autoboot: 0
ar7240> rescu
Unknown command 'rescu' - try 'help'
ar7240> uresc

Setting default IP 192.168.1.20
Starting TFTP server...
Using eth0 (192.168.1.20), address: 0x81000000
Waiting for connection: \
Receiving file from 192.168.1.254:57971
Received 5917237 bytes

Firmware Version: XS2.ar7240.FreeBSD
Setting U-Boot environment variables
Un-Protected 1 sectors
Erasing Flash.... done
Erased 1 sectors
Writing to Flash... done
Protected 1 sectors
Copying partition 'kernel' to flash memory:
erasing range 0x9F050000..0x9F12FFFF: .............. done
Erased 14 sectors
writing to address 0x9f050000, length 0x000e0000 ...
Copying partition 'rootfs' to flash memory:
erasing range 0x9F130000..0x9F5FFFFF: ............................................................................. done
Erased 77 sectors
writing to address 0x9f130000, length 0x004d0000 ...

Firmware update complete.

Resetting...

U-Boot 1.1.4.2-s594 (Dec 5 2012 - 15:23:07)

Board: Ubiquiti Networks XM board (rev 1.0 e302)
DRAM: 32 MB
Flash: 8 MB
PCIe WLAN Module found (#1).
Net: eth0, eth1
Hit any key to stop autoboot: 0
## Booting image at 9f050000 ...
Image Name: FreeBSD
Created: 2013-08-26 14:51:52 UTC

Image Type: MIPS Linux Kernel Image (lzma compressed)
Data Size: 893021 Bytes = 872.1 kB

Load Address: 80050000
Entry Point: 80050100
Verifying Checksum ... OK
Uncompressing Kernel Image ... OK

Starting kernel ...

CPU platform: Atheros AR7241 rev 1
CPU Frequency=390 MHz
CPU DDR Frequency=390 MHz
CPU AHB Frequency=195 MHz
platform frequency: 390000000
CPU reference clock: 5 MHz
arguments:
a0 = 00000006
a1 = a1f4bfb0
a2 = a1f4c450
a3 = 00000000

Cmd line:argv is invalid
Environment:
envp is invalid
Cache info:
picache_stride = 4096
picache_loopcount = 16
pdcache_stride = 4096
pdcache_loopcount = 8
cpu0: MIPS Technologies processor v116.147
MMU: Standard TLB, 16 entries
L1 i-cache: 4 ways of 512 sets, 32 bytes per line
L1 d-cache: 4 ways of 256 sets, 32 bytes per line
Config1=0x9ee3519e
Config3=0x20
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #0 r254676: Thu Aug 22 18:05:52 UTC 2013
[email protected]:/usr/home/hirenp/work/freebsd/head/obj/mipseb/mips.mips/usr/home/hirenp/work/freebsd/head/src/sys/AP91 mips

gcc version 4.2.1 20070831 patched [FreeBSD]
real memory = 16777216 (16384K bytes)
avail memory = 12087296 (11MB)

random device not loaded; using insecure entropy
nexus0:
nexus0: failed to add child: arge0
clock0: on nexus0
Timecounter "MIPS32" frequency 195000000 Hz quality 800
Event timer "MIPS32" frequency 195000000 Hz quality 800
argemdio0: at mem 0x19000000-0x19000fff on nexus0

mdio0: on argemdio0
mdioproxy0: on mdio0
arswitch0: on mdio0
arswitch0: attaching PHY 0 failed
arswitch0: attaching PHY 1 failed
arswitch0: attaching PHY 2 failed
arswitch0: attaching PHY 3 failed
device_attach: arswitch0 attach returned 6
apb0 at irq 4 on nexus0
uart0: on apb0
uart0: console (115200,n,8,1)
pcib0 at irq 0 on nexus0
pcib0: found EEPROM at 0x1fff1000 on 0.0.0
pcib0: EEPROM firmware: 0x1fff1000 @ 4096 bytes
pcib0: device EEPROM 'pcib.0.bus.0.0.0.eeprom_firmware' registered
pci0: on pcib0
pci0: at device 0.0 (no driver attached)
arge0: at mem 0x19000000-0x19000fff irq 2 on nexus0
miiproxy0: on arge0
arge0: can't attach proxy
arge0: finishing attachment, phymask 0010, proxy null
arge0: unable to attach PHY 4: 6
device_attach: arge0 attach returned 6
arge1: at mem 0x1a000000-0x1a000fff irq 3 on nexus0
arge1: finishing attachment, phymask 0000, proxy null
arge1: Ethernet address: 62:73:64:dd:03:41

spi0: at mem 0x1f000000-0x1f00000f on nexus0
spibus0: on spi0
mx25l0: at cs 0 on spibus0
mx25l0: w25q64, sector 65536 bytes, 128 sectors
ar71xx_wdog0: on nexus0
ar71xx_wdog0: Previous reset was due to watchdog timeout
Timecounters tick every 1.000 msec
Trying to mount root from ufs:/dev/map/rootfs.uncompress []...
mountroot: waiting for device /dev/map/rootfs.uncompress ...
Mounting from ufs:/dev/map/rootfs.uncompress failed with error 19.

Loader variables:

Manual root filesystem specification:
: [options]
Mount using filesystem
and with the specified (optional) option list.

eg. ufs:/dev/da0s1a
zfs:tank
cd9660:/dev/acd0 ro
(which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

? List valid disk boot devices
. Yield 1 second (for background tasks)
Abort manual input

mountroot>

jmg@ suspected that /dev/map/rootfs.uncompress doesn’t exist for some reason as error: 19 is ENODEV and asked me to enable debugging in g_uncompress.c
I did not find anything useful after enabling debugs:

flash/spi0.uncompress: media sectorsize 512, mediasize 8388608
flash/spi0.uncompress: no CLOOP magic
map/u-boot.uncompress: media sectorsize 512, mediasize 262144
map/u-boot.uncompress: no CLOOP magic
map/u-boot-env.uncompress: media sectorsize 512, mediasize 65536
map/u-boot-env.uncompress: no CLOOP magic
map/kernel.uncompress: media sectorsize 512, mediasize 1048576
map/kernel.uncompress: no CLOOP magic
map/rootfs.uncompress: media sectorsize 512, mediasize 6684672
map/rootfs.uncompress: no CLOOP magic
map/cfg.uncompress: media sectorsize 512, mediasize 262144
map/cfg.uncompress: no CLOOP magic
map/eeprom.uncompress: media sectorsize 512, mediasize 65536
map/eeprom.uncompress: no CLOOP magic

Trying to mount root from ufs:/dev/map/rootfs.uncompress []...
mountroot: waiting for device /dev/map/rootfs.uncompress ...
Mounting from ufs:/dev/map/rootfs.uncompress failed with error 19.

Loader variables:

Manual root filesystem specification:
: [options]
Mount using filesystem
and with the specified (optional) option list.

eg. ufs:/dev/da0s1a
zfs:tank
cd9660:/dev/acd0 ro
(which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

? List valid disk boot devices
. Yield 1 second (for background tasks)
Abort manual input

mountroot> ?

List of GEOM managed disk devices:
map/eeprom map/cfg map/rootfs map/kernel map/u-boot-env map/u-boot flash/spi0

mountroot>

Later loos@ caught that the problem was the rootfs start point in the hints file.

Specially we had to specify:
hint.map.3.start=0×130000

Because mtdparts specify 0×130000 as rootfs offset.

After this bring up part was done:

U-Boot 1.1.4.2-s594 (Dec 5 2012 - 15:23:07)

Board: Ubiquiti Networks XM board (rev 1.0 e302)
DRAM: 32 MB
Flash: 8 MB
PCIe WLAN Module found (#1).
Net: eth0, eth1
Hit any key to stop autoboot: 0
ar7240> ures

Setting default IP 192.168.1.20
Starting TFTP server...
Using eth0 (192.168.1.20), address: 0x81000000
Waiting for connection: |
Receiving file from 192.168.1.254:57971
Received 5966201 bytes
Firmware Version: XM.ar7240.FreeBSD

Setting U-Boot environment variables
Un-Protected 1 sectors
Erasing Flash.... done
Erased 1 sectors
Writing to Flash... done

Protected 1 sectors
Copying partition 'kernel' to flash memory:
erasing range 0x9F050000..0x9F12FFFF: .............. done
Erased 14 sectors
writing to address 0x9f050000, length 0x000e0000 ...
Copying partition 'rootfs' to flash memory:
erasing range 0x9F130000..0x9F5FFFFF: ............................................................................. done
Erased 77 sectors
writing to address 0x9f130000, length 0x004d0000 ...
Copying partition 'cfg' to flash memory:
erasing range 0x9F6F0000..0x9F6FFFFF: . done
Erased 1 sectors
writing to address 0x9f6f0000, length 0x00010000 ...

Firmware update complete.

Resetting...

U-Boot 1.1.4.2-s594 (Dec 5 2012 - 15:23:07)

Board: Ubiquiti Networks XM board (rev 1.0 e302)
DRAM: 32 MB
Flash: 8 MB
PCIe WLAN Module found (#1).
Net: eth0, eth1
Hit any key to stop autoboot: 0
## Booting image at 9f050000 ...
Image Name: FreeBSD
Created: 2013-08-27 11:12:34 UTC

Image Type: MIPS Linux Kernel Image (lzma compressed)
Data Size: 892769 Bytes = 871.8 kB
FreeBSD 10.0-CURRENT #6 r254676M: Tue Aug 27 11:11:27 UTC 2013

[email protected]:/usr/home/hirenp/work/freebsd/head/obj/mipseb/mips.mips/usr/home/hirenp/work/freebsd/head/src/sys/AP91 mips
gcc version 4.2.1 20070831 patched [FreeBSD]
real memory = 16777216 (16384K bytes)
avail memory = 12087296 (11MB)
random device not loaded; using insecure entropy
nexus0:
clock0: on nexus0
Timecounter "MIPS32" frequency 195000000 Hz quality 800
Event timer "MIPS32" frequency 195000000 Hz quality 800
argemdio0: at mem 0x1a000000-0x1a000fff on nexus0

mdio0: on argemdio0
mdioproxy0: on mdio0
arswitch0: on mdio0
miibus0: on arswitch0
ukphy0: PHY 0 on miibus0
ukphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus1: on arswitch0
ukphy1: PHY 1 on miibus1
ukphy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus2: on arswitch0
ukphy2: PHY 2 on miibus2
ukphy2: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
miibus3: on arswitch0
ukphy3: PHY 3 on miibus3
ukphy3: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
etherswitch0: on arswitch0
mdio1: on arswitch0
mdioproxy1: on mdio1

apb0 at irq 4 on nexus0
uart0: on apb0
uart0: console (115200,n,8,1)
pcib0 at irq 0 on nexus0
pcib0: found EEPROM at 0x1fff1000 on 0.0.0
pcib0: EEPROM firmware: 0x1fff1000 @ 4096 bytes
pcib0: device EEPROM 'pcib.0.bus.0.0.0.eeprom_firmware' registered
pci0: on pcib0
pci0: at device 0.0 (no driver attached)
arge0: at mem 0x19000000-0x19000fff irq 2 on nexus0
arge0: Overriding MAC from EEPROM
miiproxy0: on arge0
miiproxy0: attached to target mdio1
arge0: finishing attachment, phymask 0010, proxy set
miibus4: on miiproxy0
ukphy4: PHY 4 on miibus4
ukphy4: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
arge0: Ethernet address: 12:00:00:18:3c:02

arge1: at mem 0x1a000000-0x1a000fff irq 3 on nexus0
arge1: finishing attachment, phymask 0000, proxy null
arge1: Ethernet address: 12:00:00:18:3c:03

spi0: at mem 0x1f000000-0x1f00000f on nexus0
spibus0: on spi0
mx25l0: at cs 0 on spibus0
mx25l0: w25q64, sector 65536 bytes, 128 sectors
ar71xx_wdog0: on nexus0
ar71xx_wdog0: Previous reset was due to watchdog timeout
Timecounters tick every 1.000 msec
arswitch0port1: link state changed to DOWN
arswitch0port2: link state changed to DOWN
arswitch0port3: link state changed to DOWN
arswitch0port4: link state changed to DOWN
map/rootfs.uncompress: GEOM_ULZMA image found
map/rootfs.uncompress: 173 x 131072 blocks

Trying to mount root from ufs:/dev/map/rootfs.uncompress []...
warning: no time-of-day clock registered, system time will not be set accurately
Aug 27 11:11:45 init: login_getclass: unknown class 'daemon'
*** Populating /var ..
*** Loading configuration files ..
*** Restoring from /dev/map/cfg ..
gunzip: unknown compression format
0 blocks
*** Completed.
*** setting up hostname
*** Load kernel modules
random: initialized
*** bringing up loopback ..
*** Starting networking via /etc/rc.d/base/net
sysctl: unknown oid 'dev.ath.0.txq_mcastq_maxdepth': No such file or directory
sysctl: unknown oid 'dev.ath.1.txq_mcastq_maxdepth': No such file or directory
*** Interface: arge0: start
arge0: link state changed to UP
*** Interface: arge0: done
*** Interface: bridge0: start
bridge0: Ethernet address: d2:c4:a8:63:0e:57
arge0: promiscuous mode enabled
bridge0: link state changed to UP
*** Interface: bridge0: done
*** Default password/login databases ..
*** inetd
*** Done!

FreeBSD/mips (freebsd-wifi-build) (ttyu0)

login: root
No home directory.
Logging in with home = "/".
# uname -a
FreeBSD freebsd-wifi-build 10.0-CURRENT FreeBSD 10.0-CURRENT #6 r254676M: Tue Aug 27 11:11:27 UTC 2013 [email protected]:/usr/home/hirenp/work/freebsd/head/obj/mipseb/mips.mips/usr/home/hirenp/work/freebsd/head/src/sys/AP91 mips
# df -k
Filesystem 1024-blocks Used Avail Capacity Mounted on
/dev/map/rootfs.uncompress 21851 20633 -529 103% /
devfs 1 1 0 100% /dev
/dev/md0 828 8 756 1% /tmp
/dev/md1 828 56 708 7% /var
/dev/md2 828 436 328 57% /etc
# pciconf -lv
none0@pci0:0:0:0: class=0x028000 card=0xe3020777 chip=0x002a168c rev=0x01 hdr=0x00
vendor = 'Atheros Communications Inc.'
device = 'AR928X Wireless Network Adapter (PCI-Express)'
class = network
# ifconfig
arge0: flags=8943 metric 0 mtu 1500
options=80000
ether 12:00:00:18:3c:02
media: Ethernet autoselect (100baseTX )
status: active
arge1: flags=8802 metric 0 mtu 1500
ether 12:00:00:18:3c:03
media: Ethernet 1000baseT
status: active
lo0: flags=8049 metric 0 mtu 16384
options=600003
inet 127.0.0.1 netmask 0xff000000
bridge0: flags=8843 metric 0 mtu 1500
ether d2:c4:a8:63:0e:57
inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: arge0 flags=143
ifmaxaddr 0 port 5 priority 128 path cost 200000
#

Next up was figuring out networking.

I loaded all needed modules:

# kldstat
Id Refs Address Size Name
1 22 0x80050000 2d8f10 kernel
2 1 0xc0081000 1078 if_ath_pci.ko
3 1 0xc0083000 11e4bc if_ath.ko
4 4 0xc01a2000 606fc wlan.ko
5 2 0xc0203000 4924 bridgestp.ko
6 1 0xc0208000 6d90 if_bridge.ko
7 1 0xc020f000 a5e0 random.ko
8 1 0xc021a000 3ac wlan_xauth.ko
9 1 0xc021b000 27e4 wlan_tkip.ko
#

But if loading if_ath_pci failed in:

ath0: at device 0.0 on pci0
ath0: ath_pci_attach: EEPROM firmware @ 0x8046c000
[ath]: default pwr offset: -5 dBm != EEPROM pwr offset: 0 dBm; curves will be adjusted.
ath0: ath_getchannels: unable to collect channel list from hal, status 12
device_attach: ath0 attach returned 22

Turning up debugging level:

# sysctl hw.ath.hal.debug=25600000000
hw.ath.hal.debug: 0 -> 2147483647

# kldload if_ath_pci
ath0: mem 0x10000000-0x1000ffff irq 0 at device 0.0 on pci0

ath0: ath_pci_attach: EEPROM firmware @ 0x8046c000
ar9280Attach: sc 0xc090d000 st 0x803006b4 sh 0xb0000000
ar5416SetReset Applying descriptor swap
ar5416SetPowerMode: AWAKE -> AWAKE (set chip )
ar9280Attach: ID 0x802ff VERSION 0x2 TYPE 0x0 REVISION 0x2
ath_hal_v14EepromAttach Eeprom Version 14.22
v14EepromReadCTLInfo Numctls = 11
ar5416SetPowerMode: AWAKE -> AWAKE (set chip )
ar9280RfAttach: attach AR9280 radio
[ath]: default pwr offset: -5 dBm != EEPROM pwr offset: 0 dBm; curves will be adjusted.
enableAniMIBCounters: Enable mib counters: OfdmPhyErrBase 0xbffe0c cckPhyErrBase 0xbfff38
ar9280Attach: return
getchannels: cc 0 regDmn 0xf0 mode 0xffffff ecm
isEepromValid: invalid regulatory domain/country code 0x2a
getregstate: invalid EEPROM contents
ath0: ath_getchannels: unable to collect channel list from hal, status 12
ar5416Detach:
ar5416SetPowerMode: AWAKE -> AWAKE (set chip )
Detaching Ani
Disable MIB counters
ar5416SetPowerMode: AWAKE -> AWAKE (set chip )
ar5416SetReset Applying descriptor swap
ar5416SetPowerMode: AWAKE -> FULL-SLEEP (set chip )

device_attach: ath0 attach returned 22

Error here was: isEepromValid: invalid regulatory domain/country code 0x2a
What it meant was that hal regdomain needed to be updated with country code 0x2a

Just to hack that up for time being, adrian@ suggested to set it to 0×0 which worked.


Index: dev/ath/ath_hal/ah_regdomain.c
===================================================================
--- dev/ath/ath_hal/ah_regdomain.c (revision 255320)
+++ dev/ath/ath_hal/ah_regdomain.c (working copy)
@@ -169,6 +169,10 @@
if (regDomainPairs[i].regDmnEnum == rd)
return AH_TRUE;
}
+ if (rd == 42) {
+ rd = 0;
+ return AH_TRUE;
+ }
HALDEBUG(ah, HAL_DEBUG_REGDOMAIN,
"%s: invalid regulatory domain/country code 0x%x\n", __func__, rd);
return AH_FALSE;

The kernconf and hints file are committed as r255086.

This is how you can generate the image:

$ cd ~
$ mkdir -p work/freebsd/head
$ cd work/freebsd/head
$ svn checkout svn://svn.freebsd.org/base/head src
$ svn checkout http://freebsd-wifi-build.googlecode.com/svn/trunk/ build
$ cd build/programs/ubnt-mkfwimage
$ make
$ su
# make install
# exit
$ cd ~/work/freebsd/head/src
$ ../build/build/bin/build picostation_m2hp buildworld buildkernel
$ su
# mkdir /tftpboot
# ../build/build/bin/build picostation_m2hp installworld installkernel distribution mfsroot fsimage uboot ubnt
# exit

On successful completion, resulting image PICOSTATION_M2HP.initial.img will be in /tftpboot.

My entire effort/rant has been also recorded at freebsd-embedded@

Making TL-WR1043ND do networking

This is a followup post to Installing FreeBSD on TP-Link TL-WR1043ND

I’ve learned a few very basic networking things while trying to make this work.

After having FreeBSD run on the board, next thing was to make it do networking. How Adrian’s scripts work is, it has configuration in /etc/cfg/


# pwd
/etc/cfg
# ls
hostapd.wlan0.conf manifest rc.conf
#

Here,
manifest file has list of files to be stored in flash on “cfg_save”. Basic workflow is that you make changes you want and do “cfg_save” which writes things to flash and then “reboot”. So router comes up with the saved settings.

To create a wifi network for let’s say wlan0, we need to create a hostapd file for it:

# cat hostapd.wlan0.conf
interface=wlan0
driver=bsd
ssid=STRCDR_11N
wpa=3
wpa_key_mgmt=WPA-PSK
wpa_passphrase=NotAdminORPassword
wpa_pairwise=CCMP TKIP
ctrl_interface=/var/run/hostapd

As this is a new file and we want to preserve it across reboots, we should add an entry for this file into manifest file. This is how manifest file looks:

# cat manifest
etc/cfg/manifest
etc/master.passwd
etc/group
etc/cfg/rc.conf
etc/cfg/hostapd.wlan0.conf

rc.conf is where you specify your networking configuration:

# cat rc.conf
# Set the default system hostname
system_hostname="freebsd-wifi-build"

# Modules to load
kernel_modules="bridgestp if_bridge random"

# These interfaces are configured in-order
network_interfaces="arge0 wlan0 bridge0"

# Create arge0, no interface address
netif_arge0_enable="YES"
netif_arge0_type="ether"
netif_arge0_addrtype="none"
netif_arge0_descr="default"
netif_arge0_name="arge0"

# Create a bridge, flip on an IPv4 static address
netif_bridge0_type="bridge"
netif_bridge0_addrtype="static"
netif_bridge0_descr="default"
netif_bridge0_name="bridge0"
# These are bridge members w/ STP enabled
netif_bridge0_members_stp="arge0"
# These are bridge members w/ STP disabled
netif_bridge0_members="arge0 wlan0"
netif_bridge0_ipv4_address="192.168.1.20"
netif_bridge0_ipv4_netmask="255.255.255.0"
netif_wlan0_enable="YES"
netif_wlan0_type="wifi"
netif_wlan0_wifi_mode="hostap"
netif_wlan0_descr="default"
netif_wlan0_addrtype="none"
netif_wlan0_name="wlan0"
netif_wlan0_wifi_parent="ath0"
netif_wlan0_wifi_createargs1="country US regdomain FCC3"
netif_wlan0_wifi_createargs2="channel 1:ht/20 up"
netif_wlan0_wifi_hostapd_enable="yes"
netif_wlan0_wifi_hostapd_conf="/etc/cfg/hostapd.wlan0.conf"

A few things to notice here:
* arge0 is the ethernet interface
* wlan0 is the wifi interface
* bridge0 is the bridge interface connecting all of it together. It does the bridging of ethernet and wifi interfaces to send and receive bits (and bytes).

After making changes, to save them do:

# cfg_save
*** Storing configuration files from /etc/cfg/manifest -> /dev/map/cfg..
etc/cfg/manifest
etc/master.passwd
etc/group
etc/cfg/rc.conf
etc/cfg/hostapd.wlan0.conf
8 blocks
dd: /dev/map/cfg: end of device
0+2 records in
1+0 records out
65536 bytes transferred in 0.479322 secs (136726 bytes/sec)

And “reboot” the router. When it comes back, it _should_ have everything ready and setup which was not the case with my setup.

This is how interfaces looked:

# ifconfig
arge0: flags=8943 metric 0 mtu 1500
ether 64:70:02:cd:94:56
inet6 fe80::6670:2ff:fecd:9456%arge0 prefixlen 64 scopeid 0x6
nd6 options=21
media: Ethernet 1000baseT
status: active
arge1: flags=8802 metric 0 mtu 1500
ether 64:70:02:cd:94:57
nd6 options=21
media: Ethernet 100baseTX
status: active
ath0: flags=8843 metric 0 mtu 2290
ether 00:19:e0:66:66:68
nd6 options=21
media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng
status: running
lo0: flags=8049 metric 0 mtu 16384
options=600003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
inet 127.0.0.1 netmask 0xff000000
nd6 options=21
wlan0: flags=8943 metric 0 mtu 1500
ether 00:19:e0:66:66:68
inet6 fe80::219:e0ff:fe66:6668%wlan0 prefixlen 64 scopeid 0xa
nd6 options=21
media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng
status: running
ssid STRCDR_11N channel 1 (2412 MHz 11g ht/20) bssid 00:19:e0:66:66:68
regdomain FCC3 country US ecm authmode WPA1+WPA2/802.11i
privacy MIXED deftxkey 3 TKIP 2:128-bit TKIP 3:128-bit txpower 30
scanvalid 60 protmode CTS ampdulimit 64k ampdudensity 8 shortgi wme
burst dtimperiod 1 -dfs
bridge0: flags=8843 metric 0 mtu 1500
ether 96:ce:f3:c8:09:b4
inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=21
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: wlan0 flags=143
ifmaxaddr 0 port 10 priority 128 path cost 33333
member: arge0 flags=143
ifmaxaddr 0 port 6 priority 128 path cost 2000000
#

As you can see everything looked sane and I could see it advertizing wifi network but when a client tries to associate, it would get stuck on “Obtaining IP address” and even if you assign static IP, client does not receive any packets.

We then looked at “etherswitchcfg” which provides ethernet connectivity for the board:

# etherswitchcfg
port0:
vlangroup: 1
media: Ethernet autoselect (100baseTX )
status: active
port1:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port2:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port3:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port4:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port5:
vlangroup: 0
media: Ethernet 1000baseT
status: active
vlangroup0:
vlan: 1
members 1,2,3,4,5
vlangroup1:
vlan: 2
members 0,5t
#

Now, as you can see the ethernet cable in WAN (blue) port shows up as port0 and it is in vlangroup1. But vlangroup1 is in vlan 2 and it is tagged. Moving ethernet cable from WAN port to one of the LAN ports fixed the problem.


# etherswitchcfg
port0:
vlangroup: 1
media: Ethernet autoselect (none)
status: no carrier
port1:
vlangroup: 0
media: Ethernet autoselect (100baseTX )
status: active
port2:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port3:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port4:
vlangroup: 0
media: Ethernet autoselect (none)
status: no carrier
port5:
vlangroup: 0
media: Ethernet 1000baseT
status: active
vlangroup0:
vlan: 1
members 1,2,3,4,5
vlangroup1:
vlan: 2
members 0,5t
#

Now a client could associate and get an IP via dhcp from the incoming ethernet connection on the router. (No dhcpd is needed/running on the router itself).

To look at connected clients:

# ifconfig wlan0 list sta
ADDR AID CHAN RATE RSSI IDLE TXSEQ RXSEQ CAPS FLAG
08:11:96:f9:b2:ec 1 1 144M 38.5 0 4012 27616 EPS AQEHTRS RSN HTCAP WME

To debug, we can look at individual ports with “tcpdump -ni “.

Verbose debugging from ath wifi stack can be obtained by:

# wlandebug -i wlan1 +assoc
net.wlan.1.debug: 0x0 => 0x800000

Just as a side-note, hostapd proc is using the configuration we provided and is running:

# ps awwux | grep hostapd
root 176 0.0 6.6 13104 2176 - Ss 6:56PM 0:00.13 hostapd -P /var/run/hostapd.wlan0.pid -B /etc/cfg/hostapd.wlan0.conf

Again a *HUGE* Thanks to Mr FreeBSD Wifi (Adrian Chadd) for all the build scripts and help with the bring-up.

bsdbox – take #1

One of the nice things about building embedded Linux "stuff" is busybox. The other is "uClibc", which isn't really the focus of this post.

I've fleshed out a basic busybox style thing based on the crunched binary stuff which generates sysinstall and the rescue binaries. Thankfully, most of the hard work (read: the hacks needed) are done for you - both in the rescue Makefile and the general FreeBSD build framework.

So, I give you "bsdbox" - a not-so-small static binary busybox style solution to help build a standalone image.

# ls -l /sbin/bsdbox
-rwxr-xr-x 1 0 0 2958668 Sep 16 17:44 /sbin/bsdbox

# /sbin/bsdbox

usage: bsdbox ..., where is one of:

ls cat dd df cp hostname kill mkdir sleep sh -sh dmesg sysctl init reboot

mount mdmfs mount_mfs mdconfig newfs ifconfig route ping true false hexdump

tail netstat chown chgrp arp hostapd hostapd_cli bsdbox


And with that, I have this TP-Link device happily running FreeBSD as a WPA-enabled wireless access point.

Combined with the GEOM uzip module, the entire filesystem (which is the above binary and a few shell scripts) - is just slightly above 1 megabyte in size. (geom_lzma will make that even smaller.)

The bsdbox stuff is in my GIT repository in "bsdbox". It's built as part of the base system and installed in /bsdbox/. The binary can then be cherry picked and built into an image with whatever symlinks are needed. There's more to do - more binaries for a start; but also making it a bit easier to configure bits than a single Makefile - but it's useful right now.

bsdbox – take #1

One of the nice things about building embedded Linux "stuff" is busybox. The other is "uClibc", which isn't really the focus of this post.I've fleshed out a basic busybox style thing based on the crunched binary stuff which generates sysinstall and the rescue binaries. Thankfully, most of the hard work (read: the hacks needed) are done for you - both in the rescue Makefile and the general FreeBSD build framework.So, I give you "bsdbox" - a not-so-small static binary busybox style solution to help build a standalone image.# ls -l /sbin/bsdbox-rwxr-xr-x 1 0 0 2958668 Sep 16 17:44 /sbin/bsdbox# /sbin/bsdbox usage: bsdbox ..., where is one of:ls cat dd df cp hostname kill mkdir sleep sh -sh dmesg sysctl init rebootmount mdmfs mount_mfs mdconfig newfs ifconfig route ping true false hexdumptail netstat chown chgrp arp hostapd hostapd_cli bsdboxAnd with that, I have this TP-Link device happily running FreeBSD as a WPA-enabled wireless access point.Combined with the GEOM uzip module, the entire filesystem (which is the above binary and a few shell scripts) - is just slightly above 1 megabyte in size. (geom_lzma will make that even smaller.)The bsdbox stuff is in my GIT repository in "bsdbox". It's built as part of the base system and installed in /bsdbox/. The binary can then be cherry picked and built into an image with whatever symlinks are needed. There's more to do - more binaries for a start; but also making it a bit easier to configure bits than a single Makefile - but it's useful right now.

FreeBSD/MIPS: AR9132 porting

I decided to avoid work a little by porting FreeBSD/MIPS to the AR9132 in the TP-LINK WN-1043nd 802.11n wireless AP I have here. There's existing OpenWRT code for it so I figured the port wouldn't be that difficult.

It turns out I was right. The AR91xx support took a couple of days to massage into a form which was good enough to commit to FreeBSD-HEAD. I also included some basic AR713x SoC support - not enough to be completely useful, but enough to get a head-start on porting.

I broke out various CPU specific operations into a set of function pointers which a CPU detection function sets up early during the MIPS init code. The main differences between the SoCs are:

* Different base frequencies for setting up RAM, peripheral bus (eg UART), etc
* Different register locations for a few things
* The AR713x has a PCIe bus; the AR71xx has a PCI bus; the AR91xx doesn't have a PCI bus
* The AR9132 has different PLL values for the gigabit ethernet MACs
* Each SoC has a different USB peripheral setup path
* Each SoC has different GPIO layouts
* There's slightly different on-board peripheral reset registers

I've tested the code on the AR71xx and AR9132 and things work fine. I now need to port the AR9100 wireless MAC support from Linux ath9k to complete things. The AR9100 looks a lot like the AR5416/AR9160 (802.11n, 3 radio chains) but it has a few annoying differences:

* It isn't on a PCI bus, so it has to be manually attached;
* A few registers differ in locations (one of which returns 0xdeadbeef if the AR5416 register location is read!)
* The EEPROM is not in the same location(s) as the AR5416 - it needs to be copied and then shoe-horned into the AR5416 eeprom access code.

Also, there's an RTL8133RB gigabit switch device on-board. There's also code in Linux/OpenWRT to support this. I'll look at porting this once the AR9100 support is done.

FreeBSD/MIPS: AR9132 porting

I decided to avoid work a little by porting FreeBSD/MIPS to the AR9132 in the TP-LINK WN-1043nd 802.11n wireless AP I have here. There's existing OpenWRT code for it so I figured the port wouldn't be that difficult.It turns out I was right. The AR91xx support took a couple of days to massage into a form which was good enough to commit to FreeBSD-HEAD. I also included some basic AR713x SoC support - not enough to be completely useful, but enough to get a head-start on porting.I broke out various CPU specific operations into a set of function pointers which a CPU detection function sets up early during the MIPS init code. The main differences between the SoCs are:* Different base frequencies for setting up RAM, peripheral bus (eg UART), etc* Different register locations for a few things* The AR713x has a PCIe bus; the AR71xx has a PCI bus; the AR91xx doesn't have a PCI bus* The AR9132 has different PLL values for the gigabit ethernet MACs* Each SoC has a different USB peripheral setup path* Each SoC has different GPIO layouts* There's slightly different on-board peripheral reset registersI've tested the code on the AR71xx and AR9132 and things work fine. I now need to port the AR9100 wireless MAC support from Linux ath9k to complete things. The AR9100 looks a lot like the AR5416/AR9160 (802.11n, 3 radio chains) but it has a few annoying differences:* It isn't on a PCI bus, so it has to be manually attached;* A few registers differ in locations (one of which returns 0xdeadbeef if the AR5416 register location is read!)* The EEPROM is not in the same location(s) as the AR5416 - it needs to be copied and then shoe-horned into the AR5416 eeprom access code.Also, there's an RTL8133RB gigabit switch device on-board. There's also code in Linux/OpenWRT to support this. I'll look at porting this once the AR9100 support is done.

FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!

I found and squished a small bug in the gigabit NIC driver in FreeBSD/MIPS for the AR71xx chipset. It wasn't all that complicated - TX buffers weren't being thoroughly enough checked for alignment/size constraints before being handed to the DMA engine.
It did however fix a few niggling issues. My tunneling stuff was fixed. IPv6 frames are now correctly handled. And ng_nat doesn't cause a panic in the NIC driver. So there's at least three people who are happy. :)

FreeBSD/MIPS on AR71xx / Routerstation Pro: NIC alignment/size bug fixed!

I found and squished a small bug in the gigabit NIC driver in FreeBSD/MIPS for the AR71xx chipset. It wasn't all that complicated - TX buffers weren't being thoroughly enough checked for alignment/size constraints before being handed to the DMA engine.

It did however fix a few niggling issues. My tunneling stuff was fixed. IPv6 frames are now correctly handled. And ng_nat doesn't cause a panic in the NIC driver. So there's at least three people who are happy. :)

Minor OpenOCD fixes

Back from the land of GUI software. I have bought one more Flyswatter JTAG recently and now have two boards connected to my home box. Unfortunately both Flyswatters got the same USB serial number so stock openocd opens only the first device it stumbles upon. Here is small patch that adds ft2232_index command to OpenOCD FTDI driver that allows to point at specific device to open. Works only with libftdi. In the same directory you can find my configs for AR71XX-based RouterStation Pro and Portwell's CAM-0010 device based on Octeon CN3010

Minor OpenOCD fixes

>Back from the land of GUI software. I have bought one more Flyswatter JTAG recently and now have two boards connected to my home box. Unfortunately both Flyswatters got the same USB serial number so stock openocd opens only the first device it stumbles upon. Here is small patch that adds ft2232_index command to OpenOCD FTDI driver that allows to point at specific device to open. Works only with libftdi. In the same directory you can find my configs for AR71XX-based RouterStation Pro and Portwell's CAM-0010 device based on Octeon CN3010

Minor OpenOCD fixes

Back from the land of GUI software. I have bought one more Flyswatter JTAG recently and now have two boards connected to my home box. Unfortunately both Flyswatters got the same USB serial number so stock openocd opens only the first device it stumbles upon. Here is small patch that adds ft2232_index command to OpenOCD FTDI driver that allows to point at specific device to open. Works only with libftdi. In the same directory you can find my configs for AR71XX-based RouterStation Pro and Portwell's CAM-0010 device based on Octeon CN3010

FreeBSD-current on the RouterStation Pro

I've been working on shoehorning FreeBSD-current onto the RouterStation Pro. It's a rather cheap but nice Atheros MIPS board by Ubiquity. It was tricky, but doable. I now have a cut down kernel and memory root filesystem stored on the on-board flash - enough to run it as a basic access point.FreeBSD-current has support for the chipset - it's the AR71XX kernel config file. I've added a few more devices (USB, disk, MSDOS, redboot) to the default kernel. The default cross-build system works just fine.I've been using TFTP loaded kernel+mdroot images to test out standalone functionality, along with TFTP + NFS root images to run a mostly-full development environment.The board uses RedBoot as a bootloader. There's a tool by Ubiquity which generates firmware images which are understood by the bootloader and will overwrite the non-system area of the on-board flash. This makes dumping an image onto the system highly trivial. The bootloader can also boot the (uncompresed) kernel+mdroot images via TFTP as well as having a compressed version written to flash - so I can easily test the standalone images without constant re-flashing.RedBoot makes it easy to TFTP upload a replacement flash image written out by the Ubiquity tool. It takes care of erasing, repartitioning and copying into the onboard flash without overwriting or damaging the RedBoot code, flash partition and configuration areas.There was a bug with the PCI probe code until a couple of days ago where PCI cards (ie, stuff like the mini-PCI radio cards!) wouldn't be enumerated unless bootverbose was specified. Gonzo traced it down to a missing DELAY() - Linux was waiting a lot longer between probes than FreeBSD was. PCI devices now probe and attach correctly.The geom_redboot module doesn't attach to the on-board flash (device "spi") - it only probes "cfi" flash devices. A quick patch to src/sys/geom/geom_redboot.c to also probe flash/spi has fixed this. I've asked Gonzo/Warner to investigate this and possibly commit a fix. With this in place, FreeBSD can mount a compressed, read-only filesystem from the "rootfs" flash slice rather than needing to pack a kernel+mdroot into the kernel flash slice.So far, so good. I've written out a basic image that can be coaxed to be an open (no authentication) access point.Interesting links:Instructions on re-flashing the unit (but it doesn't mention that the reset button must be held down during power on!): http://www.usualcoding.eu/post/2010/01/15/Flash-OpenWRT-on-the-Ubiquiti-RouterStation-ProMy FreeBSD router-station pro wiki: http://wiki.freebsd.org/AdrianChadd/UbiquityRouterstationProMy work-in-progress tarball of "stuff" for building firmware images for the RouterStation Pro: http://people.freebsd.org/~adrian/rspro/

FreeBSD-current on the RouterStation Pro

I've been working on shoehorning FreeBSD-current onto the RouterStation Pro. It's a rather cheap but nice Atheros MIPS board by Ubiquity. It was tricky, but doable. I now have a cut down kernel and memory root filesystem stored on the on-board flash - enough to run it as a basic access point.

FreeBSD-current has support for the chipset - it's the AR71XX kernel config file. I've added a few more devices (USB, disk, MSDOS, redboot) to the default kernel. The default cross-build system works just fine.

I've been using TFTP loaded kernel+mdroot images to test out standalone functionality, along with TFTP + NFS root images to run a mostly-full development environment.

The board uses RedBoot as a bootloader. There's a tool by Ubiquity which generates firmware images which are understood by the bootloader and will overwrite the non-system area of the on-board flash. This makes dumping an image onto the system highly trivial. The bootloader can also boot the (uncompresed) kernel+mdroot images via TFTP as well as having a compressed version written to flash - so I can easily test the standalone images without constant re-flashing.

RedBoot makes it easy to TFTP upload a replacement flash image written out by the Ubiquity tool. It takes care of erasing, repartitioning and copying into the onboard flash without overwriting or damaging the RedBoot code, flash partition and configuration areas.

There was a bug with the PCI probe code until a couple of days ago where PCI cards (ie, stuff like the mini-PCI radio cards!) wouldn't be enumerated unless bootverbose was specified. Gonzo traced it down to a missing DELAY() - Linux was waiting a lot longer between probes than FreeBSD was. PCI devices now probe and attach correctly.

The geom_redboot module doesn't attach to the on-board flash (device "spi") - it only probes "cfi" flash devices. A quick patch to src/sys/geom/geom_redboot.c to also probe flash/spi has fixed this. I've asked Gonzo/Warner to investigate this and possibly commit a fix. With this in place, FreeBSD can mount a compressed, read-only filesystem from the "rootfs" flash slice rather than needing to pack a kernel+mdroot into the kernel flash slice.

So far, so good. I've written out a basic image that can be coaxed to be an open (no authentication) access point.

Interesting links:

Instructions on re-flashing the unit (but it doesn't mention that the reset button must be held down during power on!): http://www.usualcoding.eu/post/2010/01/15/Flash-OpenWRT-on-the-Ubiquiti-RouterStation-Pro

My FreeBSD router-station pro wiki: http://wiki.freebsd.org/AdrianChadd/UbiquityRouterstationPro

My work-in-progress tarball of "stuff" for building firmware images for the RouterStation Pro: http://people.freebsd.org/~adrian/rspro/

Booting instructions for FreeBSD on Cavium Octeon

Here's a quick note on how to net boot the Cavium EBT3000 board running uboot. The Cavium kernel is still a work in progress as I restore all the fixes I made to an earlier version of this code that I was unable to release.You'll need to break into the boot sequence for this board. Usually that's just hitting ^C before uboot starts to load the kernel.Once you have a uboot prompt, similar to below, you'll be ready to start to fix the environment for netbooting.
U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxxOCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)DRAM: 2048 MBFlash: 8 MBIPD backpressure workaround verified, took 14 loopsClearing DRAM........ doneBIST check passed.Net: octeth0, octeth1, octeth2, octeth3 Bus 0 (CF Card): OK ide 0: Model: SanDisk SDCFB-1024 Firm: Rev 0.00 Ser#: X0308 20050726142909 Type: Removable Hard Disk Capacity: 977.4 MB = 0.9 GB (2001888 x 512)Octeon ebt3800#
It couldn't be easier to setup for netbooting. You'll need to setup an tftp server. I had to install the FreeBSD port from net/freebsd-tftpd to get a working tftpd, since there appears to be some issues with the stock FreeBSD tftpd. Once I had that, I set different addresses like so:
set ipaddr 10.0.0.30set netmask 255.255.255.0set serverip 10.0.0.10set gatewayip 10.0.0.1set bootdelay 0saveenv
This sets the board's address to 10.0.0.30/24. 10.0.0.1 is the router to the other networks. The tftp command will now use tftp from host 10.0.0.10 to get the named file. saveenv makes these setting permanent. I usually reboot at this point to make sure that the settings took.Next, you'll need to know how to load a FreeBSD kernel. This is also fairly straight-forward, although finding the right addresses took a bit of guesswork. You are tftping the ELF kernel to a chunk of memory. The bootoctlinux command then takes that image and copies it to the right place and starts executing it. So you have to be sure whereever you load it doesn't clash with uboot's memory use or with where the kernel will ultimately reside.
tftp a800000 kernel.octean1-32bootoctlinux a800000 numcores=1
As with any u-boot platform, it can often be useful to define macro commands. I did the following:
set fbsd 'tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1'
so that I don't have to keep typing it over and over. I can just type
run fbsd
to run my next test.The Cavium port is in a state of flux right now. The latest status as I am preparing this to be published is that there's an interrupt problem that's causing the serial port to be non-responsive after going to multi-user (eg, the login: prompt) and also the network stops responding. Also, I have to comment out the enableintr() call in mips_vector_init around line 365 of sys/mips/mips/machdep.c. I'm trying to hammer these issues out, and I think they are related. Earlier versions of the code worked with earlier versions of FreeBSD, so I'm sure this is just a shift in interfaces that needs to be accounted for.Here's the latest slightly trimmed log from a boot:
U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxxOCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)DRAM: 2048 MBFlash: 8 MBIPD backpressure workaround verified, took 32 loopsClearing DRAM........ doneBIST check passed.Net: octeth0, octeth1, octeth2, octeth3 Bus 0 (CF Card): OK ide 0: Model: 512MB CKS Firm: 2N3-0925 Ser#: 2435E7C34A52930007 Type: Removable Hard Disk Capacity: 491.6 MB = 0.4 GB (1006992 x 512)Octeon ebt3000# printbootdelay=0baudrate=115200download_baudrate=115200ls=fatls ide 0nuke_env=protect off BFBFE000 BFBFffff; erase BFBFE000 BFBFffffautoload=nipaddr=10.0.0.31gatewayip=10.0.0.1netmask=255.255.255.0serverip=10.0.0.16fbsd=tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1fbsd_cf=fatload ide 0 a800000 kernel.imp; bootoctlinux a800000 numcores=1coremask_override=0xffffnumcores=16stdin=serialstdout=serialstderr=serialEnvironment size: 1274/8188 bytesOcteon ebt3000# run fbsdocteth0: Down 10Mbs Half duplex, (port 16)Using octeth0 deviceTFTP from server 10.0.0.16; our IP address is 10.0.0.31Filename 'kernel.octeon1-32'.Load address: 0xa800000Loading: octeth0: Up 100Mbs Half duplex, (port 16)########################doneBytes transferred = 3370489 (336df9 hex)argv[2]: numcores=1ELF file is 32 bitSkipping non LOAD program header (type 0x6)Skipping non LOAD program header (type 0x3)Skipping non LOAD program header (type 0x70000000)Allocated memory for ELF segment: addr: 0x1000000, size 0x295a60Loading .text @ 0x81000000 (2064384 bytes)Loading .MIPS.stubs @ 0x811f8000 (16 bytes)Loading .rodata @ 0x811fa000 (33248 bytes)Loading .reginfo @ 0x812021e0 (24 bytes)Loading .rodata.str1.4 @ 0x812021f8 (189300 bytes)Loading set_sysctl_set @ 0x8123056c (3496 bytes)Loading set_sysinit_set @ 0x81231314 (1684 bytes)Loading set_sysuninit_set @ 0x812319a8 (952 bytes)Loading .interp @ 0x81231d60 (13 bytes)Loading .dynsym @ 0x81231d70 (78320 bytes)Loading .dynstr @ 0x81244f60 (72163 bytes)Loading .hash @ 0x81256944 (35984 bytes)Loading set_kdb_dbbe_set @ 0x8125f5d4 (8 bytes)Loading set_modmetadata_set @ 0x8125f5dc (288 bytes)Loading set_cons_set @ 0x8125f6fc (8 bytes)Loading .data @ 0x8125f704 (116476 bytes)Loading set_pcpu @ 0x8127be00 (3136 bytes)Loading .got @ 0x8127ca40 (5296 bytes)Loading .rld_map @ 0x8127def0 (4 bytes)Loading .sdata @ 0x8127def4 (12 bytes)Clearing .bss @ 0x8127df00 (97120 bytes)## Loading Linux kernel with entry point: 0x81000000 ...Bootloader: Done loading app on coremask: 0x1Boot Descriptor Ver: 6 -> 1/2 CPU clock: 500MHz Dram: 2048 MB Board Type: 2 Revision: 4/0 Octeon Chip: 0 Rev 0/0 Mac Address 00.0F.B7.10.1B.32 (14)octeon_dram == 80000000reduced to ram: 2048 MBReal memory bytes is 7ffff000phys_avail[0] = 1296000 phys_avail[1] = fffffffrealmem_bytes is now at 5ffff000Next block of memory goes from 20000000 to 7ffff000Total DRAM Size 0x80000000Bank 0 = 0x 1296000 -> 0x FFFFFFFBank 1 = 0x20000000 -> 0x7FFFF000physmem: 0x6ed69Cache info: picache_stride = 4096 picache_loopcount = 8 pdcache_stride = 128 pdcache_loopcount = 64cpu0: Cavium processor v1.0 MMU: Standard TLB, 32 entries L1 i-cache: 4 ways of 64 sets, 128 bytes per line L1 d-cache: 64 ways of 1 sets, 128 bytes per line Config1=0xbe3303da Config3=0x80Physical memory chunk(s):0x1296000 - 0xfffefff, 248942592 bytes (60777 pages)0x20000000 - 0x7fffefff, 1610608640 bytes (393215 pages)Maxmem is 0x7ffff000KDB: debugger backends: ddbKDB: current backend: ddbhz=100 cyl_per_hz:250000 cyl_per_usec:250 freq:250000000 cyl_per_hz:2500000 cyl_per_sec:250000000Copyright (c) 1992-2010 The FreeBSD Project.Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved.FreeBSD is a registered trademark of The FreeBSD Foundation.FreeBSD 9.0-CURRENT #141 r202986:202997M: Mon Jan 25 20:07:07 MST 2010 [email protected]:/tmp/imp/be-obj/mips/pe/imp/svn/head/sys/OCTEON1-32 mipsreal memory = 1859555328 (1815972K bytes)Physical memory chunk(s):0x013ab000 - 0x0fffefff, 247808000 bytes (60500 pages)0x20000000 - 0x7d830fff, 1568870400 bytes (383025 pages)avail memory = 1813180416 (1729MB)mem: nfslock: pseudo-devicenull: nexus0: Compact flash found in bootbus region 3 (8 bit).rgmii0: on nexus0rgmx0: Ethernet address: 00:0f:b7:10:1b:32rgmx0: type (null), full duplexrgmx1: Ethernet address: 00:0f:b7:10:1b:33rgmx1: type (null), full duplexrgmx2: Ethernet address: 00:0f:b7:10:1b:34rgmx2: type (null), full duplexrgmx3: Ethernet address: 00:0f:b7:10:1b:35rgmx3: type (null), full duplexrgmii0: [FILTER]cf0: on nexus0clock0: on nexus0clock0: [FILTER]obio0 at mem 0x1 flags 0x1 on nexus0uart1: on obio0uart1: [FILTER]uart1: fast interruptuart1: console (115200,n,8,1)uart0: on obio0uart0: [FILTER]uart0: fast interruptuart0: console (115200,n,8,1)Device configuration finished.Reducing kern.maxvnodes 117045 -> 100000Timecounter "MIPS32" frequency 250000000 Hz quality 800Timecounters tick every 10.000 msecGEOM: cf0: partition 2 does not start on a track boundary.GEOM: cf0: partition 2 does not end on a track boundary.GEOM: cf0: partition 1 does not start on a track boundary.GEOM: cf0: partition 1 does not end on a track boundary.GEOM: cf0s2: geometry does not match label (64h,32s != 16h,63s).Trying to mount root from ufs:cf0s2aWARNING: / was not properly dismountedwarning: no time-of-day clock registered, system time will not be set accuratelystart_init: trying /sbin/initSetting hostuuid: 1c99760e-1dd2-11b2-bd6a-00156dc12838.Setting hostid: 0xb0f278c1.No suitable dump device was found.Entropy harvesting:.Starting file system checks:t_delta 16.010fac783006c1a6 too long/dev/cf0s2a: 11100 files, 256973 used, 150946 free (426 frags, 18815 blocks, 0.1% fragmentation)Mounting local file systems:.t_delta 15.fece5003dd62224a too shortSetting hostname: ebt3000. SIOCSTIFFLAGS UP/RunningStarting Network: lo0 rgmx0 rgmx1 rgmx2 rgmx3.lo0: flags=8049 metric 0 mtu 16384 options=3 inet 127.0.0.1 netmask 0xff000000 rgmx0: flags=8843 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:32 inet 10.0.0.31 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselectrgmx1: flags=8802 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:33 media: Ethernet autoselectrgmx2: flags=8802 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:34 media: Ethernet autoselectrgmx3: flags=8802 metric 0 mtu 1500 options=3 ether 00:0f:b7:10:1b:35 media: Ethernet autoselectStarting devd.Creating and/or trimming log files.Starting syslogd./etc/rc: WARNING: Dump device does not exist. Savecore not run.ELF ldconfig path: /lib /usr/lib /usr/lib/compatClearing /tmp (X related).Updating motd:.Starting cron.Starting inetd.Starting background file system checks in 60 seconds.Tue Jan 26 02:07:16 UTC 2010FreeBSD/mips (ebt3000) (ttyu0)login:

Booting instructions for FreeBSD on Cavium Octeon

Here's a quick note on how to net boot the Cavium EBT3000 board running uboot. The Cavium kernel is still a work in progress as I restore all the fixes I made to an earlier version of this code that I was unable to release.

You'll need to break into the boot sequence for this board. Usually that's just hitting ^C before uboot starts to load the kernel.

Once you have a uboot prompt, similar to below, you'll be ready to start to fix the environment for netbooting.
U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 14 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: SanDisk SDCFB-1024 Firm: Rev 0.00 Ser#: X0308 20050726142909
Type: Removable Hard Disk
Capacity: 977.4 MB = 0.9 GB (2001888 x 512)

Octeon ebt3800#


It couldn't be easier to setup for netbooting. You'll need to setup an tftp server. I had to install the FreeBSD port from net/freebsd-tftpd to get a working tftpd, since there appears to be some issues with the stock FreeBSD tftpd. Once I had that, I set different addresses like so:
set ipaddr 10.0.0.30
set netmask 255.255.255.0
set serverip 10.0.0.10
set gatewayip 10.0.0.1
set bootdelay 0
saveenv

This sets the board's address to 10.0.0.30/24. 10.0.0.1 is the router to the other networks. The tftp command will now use tftp from host 10.0.0.10 to get the named file. saveenv makes these setting permanent. I usually reboot at this point to make sure that the settings took.

Next, you'll need to know how to load a FreeBSD kernel. This is also fairly straight-forward, although finding the right addresses took a bit of guesswork. You are tftping the ELF kernel to a chunk of memory. The bootoctlinux command then takes that image and copies it to the right place and starts executing it. So you have to be sure whereever you load it doesn't clash with uboot's memory use or with where the kernel will ultimately reside.
tftp a800000 kernel.octean1-32
bootoctlinux a800000 numcores=1


As with any u-boot platform, it can often be useful to define macro commands. I did the following:
set fbsd 'tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1'
so that I don't have to keep typing it over and over. I can just type
run fbsd
to run my next test.

The Cavium port is in a state of flux right now. The latest status as I am preparing this to be published is that there's an interrupt problem that's causing the serial port to be non-responsive after going to multi-user (eg, the login: prompt) and also the network stops responding. Also, I have to comment out the enableintr() call in mips_vector_init around line 365 of sys/mips/mips/machdep.c. I'm trying to hammer these issues out, and I think they are related. Earlier versions of the code worked with earlier versions of FreeBSD, so I'm sure this is just a shift in interfaces that needs to be accounted for.

Here's the latest slightly trimmed log from a boot:


U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 32 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: 512MB CKS Firm: 2N3-0925 Ser#: 2435E7C34A52930007
Type: Removable Hard Disk
Capacity: 491.6 MB = 0.4 GB (1006992 x 512)
Octeon ebt3000# print
bootdelay=0
baudrate=115200
download_baudrate=115200
ls=fatls ide 0
nuke_env=protect off BFBFE000 BFBFffff; erase BFBFE000 BFBFffff
autoload=n
ipaddr=10.0.0.31
gatewayip=10.0.0.1
netmask=255.255.255.0
serverip=10.0.0.16
fbsd=tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1
fbsd_cf=fatload ide 0 a800000 kernel.imp; bootoctlinux a800000 numcores=1
coremask_override=0xffff
numcores=16
stdin=serial
stdout=serial
stderr=serial

Environment size: 1274/8188 bytes
Octeon ebt3000# run fbsd
octeth0: Down 10Mbs Half duplex, (port 16)
Using octeth0 device
TFTP from server 10.0.0.16; our IP address is 10.0.0.31
Filename 'kernel.octeon1-32'.
Load address: 0xa800000
Loading: octeth0: Up 100Mbs Half duplex, (port 16)
########################
done
Bytes transferred = 3370489 (336df9 hex)
argv[2]: numcores=1
ELF file is 32 bit
Skipping non LOAD program header (type 0x6)
Skipping non LOAD program header (type 0x3)
Skipping non LOAD program header (type 0x70000000)
Allocated memory for ELF segment: addr: 0x1000000, size 0x295a60
Loading .text @ 0x81000000 (2064384 bytes)
Loading .MIPS.stubs @ 0x811f8000 (16 bytes)
Loading .rodata @ 0x811fa000 (33248 bytes)
Loading .reginfo @ 0x812021e0 (24 bytes)
Loading .rodata.str1.4 @ 0x812021f8 (189300 bytes)
Loading set_sysctl_set @ 0x8123056c (3496 bytes)
Loading set_sysinit_set @ 0x81231314 (1684 bytes)
Loading set_sysuninit_set @ 0x812319a8 (952 bytes)
Loading .interp @ 0x81231d60 (13 bytes)
Loading .dynsym @ 0x81231d70 (78320 bytes)
Loading .dynstr @ 0x81244f60 (72163 bytes)
Loading .hash @ 0x81256944 (35984 bytes)
Loading set_kdb_dbbe_set @ 0x8125f5d4 (8 bytes)
Loading set_modmetadata_set @ 0x8125f5dc (288 bytes)
Loading set_cons_set @ 0x8125f6fc (8 bytes)
Loading .data @ 0x8125f704 (116476 bytes)
Loading set_pcpu @ 0x8127be00 (3136 bytes)
Loading .got @ 0x8127ca40 (5296 bytes)
Loading .rld_map @ 0x8127def0 (4 bytes)
Loading .sdata @ 0x8127def4 (12 bytes)
Clearing .bss @ 0x8127df00 (97120 bytes)
## Loading Linux kernel with entry point: 0x81000000 ...
Bootloader: Done loading app on coremask: 0x1
Boot Descriptor Ver: 6 -> 1/2 CPU clock: 500MHz
Dram: 2048 MB Board Type: 2 Revision: 4/0
Octeon Chip: 0 Rev 0/0 Mac Address 00.0F.B7.10.1B.32 (14)
octeon_dram == 80000000
reduced to ram: 2048 MBReal memory bytes is 7ffff000
phys_avail[0] = 1296000 phys_avail[1] = fffffff
realmem_bytes is now at 5ffff000
Next block of memory goes from 20000000 to 7ffff000

Total DRAM Size 0x80000000
Bank 0 = 0x 1296000 -> 0x FFFFFFF
Bank 1 = 0x20000000 -> 0x7FFFF000

physmem: 0x6ed69Cache info:
picache_stride = 4096
picache_loopcount = 8
pdcache_stride = 128
pdcache_loopcount = 64
cpu0: Cavium processor v1.0
MMU: Standard TLB, 32 entries
L1 i-cache: 4 ways of 64 sets, 128 bytes per line
L1 d-cache: 64 ways of 1 sets, 128 bytes per line
Config1=0xbe3303da
Config3=0x80
Physical memory chunk(s):
0x1296000 - 0xfffefff, 248942592 bytes (60777 pages)
0x20000000 - 0x7fffefff, 1610608640 bytes (393215 pages)
Maxmem is 0x7ffff000
KDB: debugger backends: ddb
KDB: current backend: ddb
hz=100 cyl_per_hz:250000 cyl_per_usec:250 freq:250000000 cyl_per_hz:2500000 cyl_per_sec:250000000
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #141 r202986:202997M: Mon Jan 25 20:07:07 MST 2010
[email protected]:/tmp/imp/be-obj/mips/pe/imp/svn/head/sys/OCTEON1-32 mips
real memory = 1859555328 (1815972K bytes)
Physical memory chunk(s):
0x013ab000 - 0x0fffefff, 247808000 bytes (60500 pages)
0x20000000 - 0x7d830fff, 1568870400 bytes (383025 pages)
avail memory = 1813180416 (1729MB)
mem:
nfslock: pseudo-device
null:
nexus0:
Compact flash found in bootbus region 3 (8 bit).
rgmii0: on nexus0
rgmx0: Ethernet address: 00:0f:b7:10:1b:32
rgmx0: type (null), full duplex
rgmx1: Ethernet address: 00:0f:b7:10:1b:33
rgmx1: type (null), full duplex
rgmx2: Ethernet address: 00:0f:b7:10:1b:34
rgmx2: type (null), full duplex
rgmx3: Ethernet address: 00:0f:b7:10:1b:35
rgmx3: type (null), full duplex
rgmii0: [FILTER]
cf0: on nexus0
clock0: on nexus0
clock0: [FILTER]
obio0 at mem 0x1 flags 0x1 on nexus0
uart1: on obio0
uart1: [FILTER]
uart1: fast interrupt
uart1: console (115200,n,8,1)
uart0: on obio0
uart0: [FILTER]
uart0: fast interrupt
uart0: console (115200,n,8,1)
Device configuration finished.
Reducing kern.maxvnodes 117045 -> 100000
Timecounter "MIPS32" frequency 250000000 Hz quality 800
Timecounters tick every 10.000 msec
GEOM: cf0: partition 2 does not start on a track boundary.
GEOM: cf0: partition 2 does not end on a track boundary.
GEOM: cf0: partition 1 does not start on a track boundary.
GEOM: cf0: partition 1 does not end on a track boundary.
GEOM: cf0s2: geometry does not match label (64h,32s != 16h,63s).
Trying to mount root from ufs:cf0s2a
WARNING: / was not properly dismounted
warning: no time-of-day clock registered, system time will not be set accurately
start_init: trying /sbin/init
Setting hostuuid: 1c99760e-1dd2-11b2-bd6a-00156dc12838.
Setting hostid: 0xb0f278c1.
No suitable dump device was found.
Entropy harvesting:.
Starting file system checks:
t_delta 16.010fac783006c1a6 too long
/dev/cf0s2a: 11100 files, 256973 used, 150946 free (426 frags, 18815 blocks, 0.1% fragmentation)
Mounting local file systems:.
t_delta 15.fece5003dd62224a too short
Setting hostname: ebt3000.
SIOCSTIFFLAGS UP/Running
Starting Network: lo0 rgmx0 rgmx1 rgmx2 rgmx3.
lo0: flags=8049 metric 0 mtu 16384
options=3
inet 127.0.0.1 netmask 0xff000000
rgmx0: flags=8843 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:32
inet 10.0.0.31 netmask 0xffffff00 broadcast 10.0.0.255
media: Ethernet autoselect
rgmx1: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:33
media: Ethernet autoselect
rgmx2: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:34
media: Ethernet autoselect
rgmx3: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:35
media: Ethernet autoselect
Starting devd.
Creating and/or trimming log files.
Starting syslogd.
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Clearing /tmp (X related).
Updating motd:.
Starting cron.
Starting inetd.
Starting background file system checks in 60 seconds.

Tue Jan 26 02:07:16 UTC 2010

FreeBSD/mips (ebt3000) (ttyu0)

login:

Post-mortum on projects/mips branch

Greetings to one and all. As you have read elsewhere, I recently merged all the changes from the projects/mips branch onto head. In other reports, I've made cryptic reference to the branch being damaged. I thought I'd go through all the problems we encountered running this development effort on the projects/mips branch.First, we created the branch in the normal way:
svn cp svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/mips
and then started committing to it. So far so good.Then it came time to merge in changes from head. Here is where we made our first mistake. We merged just the kernel changes, and not the entire tree. The next time we merged the entire tree. The effect of this sequence meant that the changes between the creation and the first merge in the rest of the tree (outside of src/sys) were lost. I don't know if this is the result of a buggy svn client, or if they are a fundamental flaw in svn. So when it came time to collapse the results back into head, we couldn't just do a reverse integration, or even a more conventional merge.Next, we pulled in pieces of the projects/mips tree into head, either by hand or with the svn merge command. Doing both was a mistake, I think. Although svn coped with later merged from head into projects/mips fairly well, occasionally we'd have issues to sort out.So, when it came time to merge everything back into head, we were left with a number of difficult choices. We opted to copy the new files/directories in the repo (to preserve the history) and merge patches by hand with svn log entries. The latter went really well. There were no problems with it apart from the odd fuzz and .rej file to sort out (no different than a normal merge). The former, however, causes a lot of problems.First, it created svn:mergeinfo entries on the files I copied over. This conflicts with the project's "merge everything from sys" edict. This happened because I did the copy in the server rather than the client (which was an attempt to preserve history). It was a simple matter to delete these entries when it was brought to my attention.Next, somehow we created the files on the branch without the required svn:keywords, svn:eol-style and svn:mime-type properties. This caused problems when we went to commit changes to these files. The precommit complained, at this point, that the files lacked svn:keywords, and we had to add $FreeBSD$ by hand (since the copy didn't complaint that they were lacking). Also, I didn't discover these problems on my own. Other sharp-eyed individuals saw the problems and reported them to me.Finally, since I did the copies on the server, there was no way to batch them. When I copied a directory, I got the whole directory. When I copied a file, I got the file. Each one generated a commit message.I'm unsure what other damage was lurking in the projects/mips tree, but the lack of properties, the inability to easily use merge info and the missing commits lead me to the conclusion that it was easier to abandon the branch and create a new branch if needed in the future.The mips team plans to create more branches, that are more for special purposes, that will be shorter lived in the future. We have no further plans to have one monolithic mips branch that acts as a staging ground for commits into head.

Post-mortum on projects/mips branch

Greetings to one and all. As you have read elsewhere, I recently merged all the changes from the projects/mips branch onto head. In other reports, I've made cryptic reference to the branch being damaged. I thought I'd go through all the problems we encountered running this development effort on the projects/mips branch.

First, we created the branch in the normal way:
svn cp svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/mips

and then started committing to it. So far so good.

Then it came time to merge in changes from head. Here is where we made our first mistake. We merged just the kernel changes, and not the entire tree. The next time we merged the entire tree. The effect of this sequence meant that the changes between the creation and the first merge in the rest of the tree (outside of src/sys) were lost. I don't know if this is the result of a buggy svn client, or if they are a fundamental flaw in svn. So when it came time to collapse the results back into head, we couldn't just do a reverse integration, or even a more conventional merge.

Next, we pulled in pieces of the projects/mips tree into head, either by hand or with the svn merge command. Doing both was a mistake, I think. Although svn coped with later merged from head into projects/mips fairly well, occasionally we'd have issues to sort out.

So, when it came time to merge everything back into head, we were left with a number of difficult choices. We opted to copy the new files/directories in the repo (to preserve the history) and merge patches by hand with svn log entries. The latter went really well. There were no problems with it apart from the odd fuzz and .rej file to sort out (no different than a normal merge). The former, however, causes a lot of problems.

First, it created svn:mergeinfo entries on the files I copied over. This conflicts with the project's "merge everything from sys" edict. This happened because I did the copy in the server rather than the client (which was an attempt to preserve history). It was a simple matter to delete these entries when it was brought to my attention.

Next, somehow we created the files on the branch without the required svn:keywords, svn:eol-style and svn:mime-type properties. This caused problems when we went to commit changes to these files. The precommit complained, at this point, that the files lacked svn:keywords, and we had to add $FreeBSD$ by hand (since the copy didn't complaint that they were lacking). Also, I didn't discover these problems on my own. Other sharp-eyed individuals saw the problems and reported them to me.

Finally, since I did the copies on the server, there was no way to batch them. When I copied a directory, I got the whole directory. When I copied a file, I got the file. Each one generated a commit message.

I'm unsure what other damage was lurking in the projects/mips tree, but the lack of properties, the inability to easily use merge info and the missing commits lead me to the conclusion that it was easier to abandon the branch and create a new branch if needed in the future.

The mips team plans to create more branches, that are more for special purposes, that will be shorter lived in the future. We have no further plans to have one monolithic mips branch that acts as a staging ground for commits into head.