Category Archives: Toolchain

Building old-school TiVo build tools on FreeBSD 9.2-stable

Intro

As long-term readers of my blog know, I've been nursing along a TiVo HR10-250 for the past few years. It works great for the low-quality material that my younger son loves to watch, plus I can harvest video off of it with ease.

Recently, we switched up how it was connected to our TV (and indeed, got a new TV too). During this process, we lost the ability to display subtitles. Since my wife and I like to watch The Daily Show and other similar shows upstairs without going downstairs to the big TV with the HD DirecTV player, and the environment upstairs can be a bit noisy, subtitles are quite useful.

So, after crawling around the DealDatabase forums for a bit, I found a good program called tivovbi. It works really well for displaying closed captioning. However, I used a binary I found on the forum. Nothing is more annoying than a program you download from a forum that randomly core dumps for reasons that are totally mysterious.

I can't even hack it to do anything since I have no TiVo build tools.  Looking for tools online, I can't really find anything that isn't just a Linux binary.  The Linux binaries have issues with the FreeBSD linux ABI implementation, owing in large part to their age (binaries from 10 years ago have some issues, I think with just the packages are needed having a subtle incompatibility).

So what should I do. I could create a virtualbox VM and run linux. But then I'd be running Linux, and copying back and forth to a VM is always a hassle in some way.

So, the other alternative is to build the tools from source. I thought it would be easy to do this, but there's a number of issues with it, so I thought I'd write up my experience. This is on a FreeBSD 9.2-stable system on amd64. That last bit will turn out to be important in a bit, because nothing was easy and simple on this project...

After reading through these instructions, I can't help but marvel at how this lack of integration is tolerated, but to be fair, this is building tools that are nearly a decade old at this point and I did have to kludge around lack of support for 64-bit x86 in gcc... This is nearly 60 separate commands to type. Yuck. Also, looking at the layout in chrome, many of the line breaks have liberties taken with them, so your best bet may be to cut and paste much of what I'm doing here...

Locating the tools

Tivo Utils has a bunch of useful links. Including the linux binaries that I've had issues with. Thankfully, there's a shell script called 'build_mips_x_compiler.sh' which builds all the tools and the basic libraries. I thought I could just run it and have everything built. As with everything else in this project, this wasn't so much the case. So I wound up doing a lot of things by hand.

The first bit of the script fetches all the source tar balls. The gnu stuff has had long-term stable paths, so they fetched. However the TiVo linux didn't transfer, since it was in a new location. In fact, it was also for 4.0, and my TiVo was running 6.4a. So, I had to grab that from TiVo linux downloads page. The Linux 6.4 download was exactly what I needed to grab. Once I had these in place, I was ready to start building. You also need binutils 2.13Gcc 3.0 (yes, 3.0!), glibc 2.2.3, and glibc 2.2.3 linuxthreads support.

Also, I'm installing all the tools in ~/tivo/tools tree, and building them from ~/tivo/toolchain. Fetch all of the above linked tarballs into ~/tivo/toolchain (or whatever you want). Now that we've found the tarballs, we can start with the builds. I'll assume the following environment variables are set. TARGET is set to "mips-TiVo-linux" and PREFIX is set to "$HOME/tivo/tools". I've also added ${PREFIX}/bin to my PATH (I didn't from the start, and got quite far before it mattered). I also had gnu make installed as gmake from ports.

Building Tools

Binutils 2.13

Binutils 2.13 is almost trivial to build:
cd ~imp/toolchain
tar xvfz binutils-2.13.tar.gz
mkdir build-buinutils
cd build-binutils
../binutils-2.13/configure  --target=$TARGET --prefix=$PREFIX
gmake -j 20 all
gmake install
cd ..

gcc 3.0 stage 1

gcc 3.0 stage 1 took a lot of trial and error to create. Turns out, there's no support for FreeBSD 9 in this tarball. That's trivial to add, and I've provided some patches. However, the next problem is that FreeBSD/amd64 isn't supported. This problem turns out to be more difficult to overcome. So you have to configure for FreeBSD/i386 (or backport amd64 support, which I thought would be too hard, so I didn't do it). This means you need to have a 32-bit compiler. After all the talk about making cc -m32 working, I thought I could just do gmake CC="cc -m32". This failed, however. The 32-bit applications dumped core, meaning that gcc couldn't complete its bootstrap process. I wound up having to use FreeBSD's xdev to do it:
pushd ~/FreeBSD/svn/stable/9
sudo make xdev XDEV=i386 XDEV_ARCH=i386 WITHOUT_CLANG=t
popd
Once we have that in place, we can build gcc. Except that it doesn't build quite right. We need to add some additional patches. One to work around a weirdness in the obstack implementation where it relied on an illegal lvalue autoincrement, and one to work around crazy sh that's likely bogus in fixincl. All of these are included in the above patch.
mkdir build-gcc
cd build-gcc
../gcc-3.0/configure --target $TARGET --prefix $PREFIX --without-headers --with-newlib --disable-shared --enable-languages=c --host i386-foo-freebsd9
gmake CC=/usr/i386-freebsd/usr/bin/cc all-gcc
gmake CC=/usr/i386-freebsd/usr/bin/cc install-gcc
If you wanted to see if -m32 worked, you could skip the make xdev step above and substitute CC="cc -m32" in the two gmake commands. I've also tested values up to 20 for -j for at least the first command.

Building the Linux Kernel headers

The following comes pretty much verbatim from the build_mips_x_compiler.sh and have been verified to work. You'll need to grab this patch for these instructions to work. It works around an expr difference, as well a really cheap kludge to get autoconf.h generated. Also, I had to install the bash port, and create a symlink from /bin/bash to /usr/local/bin/bash, which I've not put inline...
tar xf TiVo-6.4-linux-2.4.tar.gz
cd linux-2.4
patch -p0 < ../tivo-linux-2.4.diff
yes "" | gmake ARCH=mips CROSS_COMPILE=mips-TiVo-linux config
gmake ARCH=mips CROSS_COMPILE=mips-TiVo-linux include/linux/version.h
mkdir $PREFIX/$TARGET/include
cp -rf include/linux $PREFIX/$TARGET/include
cp -rf include/asm-mips $PREFIX/$TARGET/include/asm
which will be enough to get us to the next step...  Woof, maybe I should just make a script for all this stuff... With so many fiddly bits, I'm not sure that's a good idea.

Building gmake 3.80!

Newer versions of gmake don't grok the Makefiles that glibc 2.2.3 generates. All kinds of weird errors and odd behavior. So, to make progress, you'll need to build gmake.
fetch ftp://ftp.gnu.org/gnu/make/make-3.80.tar.gz
tar xf make-3.80.tar.gz
cd make-3.80
./configure
make
cp make ~/bin/gmake380
cd ..
I didn't bother installing it, since I just need it for this diversion so I copied into my bin dir, that I have in my path.

Building glibc 2.2.3

Now we're on to glibc. Things have been smooth sailing up to this point. With glibc, we have to extract, configure, build, fix the build oops, build again, fix some info files, then install. Woof! Sure makes for a difficult to reproduce experience. And those are the build on linux instructions... Apparently there's a lot of host leakage that's making things somewhat difficult to reproduce... but so far that appears to be only with gmake 3.82. gmake 3.80 works much better. More patches needed.
tar xf glibc-2.2.3.tar.gz
tar xf glibc-linuxthreads-2.2.3.tar.gz -C glibc-2.2.3
env CC=mips-TiVo-linux-gcc ../glibc-2.2.3/configure --host=$TARGET --prefix=$PREFIX --disable-debug --disable-profile --enable-add-ons --with-headers=${PREFIX}/${TARGET}/include
gmake380
sed -i.old -e 's/elf32-bigmips/elf32-tradbigmips/' elf/rtld-ldscript
gmake380
gmake380 install
cd ..

Munging things around before the second assault  on gcc for its stage 2

Don't know why the original instructions take this time rearrange the deck chairs, but it seems to be necessary.
mv ${PREFIX}/${TARGET}/include/asm ${PREFIX}/include
mv ${PREFIX}/${TARGET}/include/linux ${PREFIX}/include
rm -r ${PREFIX}/${TARGET}/include
rm -r ${PREFIX}/${TARGET}/lib
ln -s ${PREFIX}/include ${PREFIX}/${TARGET}/include
ln -s ${PREFIX}/lib ${PREFIX}/${TARGET}/lib

Build gcc 3.0 stage 2

Now that we have everything else setup, it is time to build the second stage of gcc. Alas, gcc 4.2.1 doesn't allow constructs like (a ? b : c) = d, which gcc 3.0 uses to implement C++. So, no g++ for me. If you want g++, you'd have to build the gcc34 port... The rest is almost boring after all the other hoops we jumped through:
mkdir build-gcc2
cd build-gcc2
../gcc-3.0/configure --target=$TARGET --prefix=$PREFIX --enable-languages=c --host=i386-foo-freebsd9
gmake CC=/usr/i386-freebsd/usr/bin/cc all
gmake CC=/usr/i386-freebsd/usr/bin/cc install
cd ..

Building stuff

Now, it is time build things...  I'll write more blog entries about that...

New Device Tree Compiler

This week, I imported a new device tree compiler, dtc(1). This is the tool that is used to translate between different representations of a Flattened Device Tree (FDT), a way of representing boot-time configuration information. The FDT contains more or less the same information as an OpenFirmware device tree, for example the locations of memory-mapped peripherals, reserved sections of RAM, interrupt routing information, and so on.

FreeBSD/ARM makes a lot of use of FDTs, as they’re the standard way of getting information from the bootloader. They are used in two ways. The ideal way is for the bootloader to provide the tree to the kernel at boot time. This allows a single kernel to be used with multiple SoCs. Alternatively, the device tree can be compiled into the kernel.

The device tree in both cases is in the form of a Device Tree Blob (DTB), a binary representation of the tree. The ‘flattened’ in the name comes from the fact that the tree is represented in a linear structured format, like HTML, with explicit delimiters for starts and ends of child nodes, rather than in a format with pointers between elements. The other representation, the Device Tree Source is a human-readable tree using braces to delimit children and is rather similar to the OpenStep property list format or JSON.

The device tree compiler is responsible for converting between these formats. In the FreeBSD tree, we have a number of DTS files that represent supported platforms where the bootloader doesn’t provide the DTB. During the build process, the DTB is generated and linked into the kernel.

Unfortunately, the existing tool was released under the GPL. We try to minimise the amount of GPL’d code installed by default, and intend to remove all of it by 10.0, so dtc was not installed unless your target platform used it (a bit silly, as you want to use it on your host platform when doing embedded device development). The new tool is a (BSD-licensed) from-scratch rewrite that I did over Christmas. It shares no code with the original, but works as a drop-in replacement in our build system.

It is now used by default, although the old GPL’d tool will remain available as an option for a while until I’m confident that we aren’t breaking out-of-tree dtc users. So, if you’re using FDTs and don’t have the DTS in the tree, please test it. Otherwise, this is just another step on the way to a fully GPL-free base system.

The New C++ Stack in 9.1

If you read the release announcement, then you’ll have seen that there is a new C++ stack in 9.1, but that it is marked optional and is not part of the default binary install. If you’re a C++ developer, you may want to play with it for a few reasons:

  • It supports C++11 (which is new, shiny, and buzzwordy)
  • It will be the only one shipped by default in 10.0, so if you care about forward compatibility then you will need to test with it or make sure that your code works with it, or depend on GNU libstdc++ from ports.

The new stack uses libcxxrt, which I originally wrote for PathScale and was since open sourced (funded by the FreeBSD and NetBSD Foundations).  This implements the low-level part of the C++ standard library, providing things like support for RTTI, dynamic_cast, exceptions, thread-safe initialisation of statics, and so on. In the old stack, libsupc++ does this.

On top of this sits the STL implementation. In the old stack, this was GNU libstdc++, in the new one it’s LLVM’s libc++. To make interoperability easier, in 9.1 we have made libstdc++ dynamically link against libsupc++. You can use libmap.conf(5) to tell it to link against libcxxrt instead, and then you can mix libraries that use libc++ with ones that use libstdc++ in the same program.

The new stack isn’t installed by default, but building and installing it is very easy:

# cd /usr/src/lib/libcxxrt
# make
# make install
# cd ../libc++
# make CXX=clang++
# make install

You should now have libc++.so installed in /usr/lib and the headers installed in /usr/include/c++/v1. You are now ready to compile C++ code with the new stack. You use them, clang has a command-line switch for selecting the stack to use. By default, it will still use the old stack:

$ clang++ hello.cc 
$ ./a.out 
Hello C++ World
$ ldd ./a.out 
./a.out:
    libstdc++.so.6 =&gt; /usr/lib/libstdc++.so.6 (0x800819000)
    libm.so.5 =&gt; /lib/libm.so.5 (0x800b29000)
    libc.so.7 =&gt; /lib/libc.so.7 (0x800d4a000)
    libgcc_s.so.1 =&gt; /lib/libgcc_s.so.1 (0x80109d000)

To enable the new stack, you must add -stdlib=libc++ to your CXXFLAGS and also to your LDFLAGS. The first ensures that we get the libc++ headers, the second that we link to libc++:

$ clang++ -stdlib=libc++ -std=c++11 hello.cc 
$ ./a.out 
Hello C++ World
$ ldd ./a.out 
./a.out:
    libc++.so.1 =&gt; /usr/lib/libc++.so.1 (0x80081a000)
    libm.so.5 =&gt; /lib/libm.so.5 (0x800ac9000)
    libgcc_s.so.1 =&gt; /lib/libgcc_s.so.1 (0x800cea000)
    libc.so.7 =&gt; /lib/libc.so.7 (0x800ef7000)
    libcxxrt.so.1 =&gt; /lib/libcxxrt.so.1 (0x80124a000)

Unfortunately, in 9.1 there is a small bug that prevents you from compiling in C++98 mode with libc++. The C11 quick_exit() and at_quick_exit() functions are exposed in our stdlib.h only in C11 and C++11 modes, but are referenced in libc++’s cstdlib in any mode.

Most code should work out-of-the-box in C++11 mode though, so there’s little reason not to try it. And, because of the availability of move semantics, some code using STL classes will be more efficient when compiled in C++11 mode.

If you want to test with a newer version, but don’t want to install -CURRENT, I’m putting x86-64 binaries of libc++ back-ported to 9.1 online for testing. I’ll update this whenever I pull a new version into -CURRENT.