EasyPBI Version 2 Available for Testing

Ken Moore has announced the availability of EasyPBI2:

I am pleased to announce that EasyPBI version 2.0 is now  available!

This has been a complete re-write of the original program code. It has a
more streamlined process for working with PBI modules, as well as a
brand new interface and many new features/abilities. See the bottom of
this post for simple instructions on how to get the new version of
EasyPBI, and also how to upgrade your pbi-manager tools (to utilize some
of the new abilities of EasyPBI).

New Features

  • Packaging a local directory into a PBI (not using FreeBSD ports)
  • New Logo (Thanks to Jennifer Rosenburg!)
  • Build 32-bit PBI’s on 64-bit systems (Additional build option)
  • Complete support for editing installation/wrapper scripts (as well as a basic template for creating new binary wrapper scripts)
  • Complete support for XDG desktop/menu entries with easy MIME type integration (full creation/editing of entries with a number of new options available for entries)
  • Switch to using the OptionsNG format for setting port build options by default (as well as using a multiple-line format for build options)

UI Improvements

  • New “Settings” dialog for setting/changing default directory paths, PBI build settings, and any external utilities.
  • New “Ports” dialog for downloading/updating the FreeBSD ports tree.
  • Displays the last time the ports tree was updated, and simplifies the process of using portsnap (or svn if previously setup that way) to update the system ports tree.
  • New “About” dialog for quickly viewing information about EasyPBI (like license information and development history)

Important Warnings

  • Make sure you are using the latest version of the pbi-manager tools before using the new “local sources” PBI build options (the default PC-BSD 9.1 tools do not have the proper version).
  • One of the install scripts (pre-pbicreate.sh) will also not be used unless you have the latest version of the pbi-manager tools.
  • Saved settings from earlier versions are not converted into the new format, you will need to reset all of your settings manually from the new “EasyPBI  Settings” menu option.
  • Desktop/Menu entries and external-links are no longer automatically generated on module creation. These can now be easily added from the module editor afterwards.
  • Since it has been added to SVN so recently, most of the translations have not been done yet. Translation is an ongoing process for the  PC-BSD sources and the current status can be checked on http://pootle.pcbsd.org

Updating Instructions

To get EasyPBI 2.0, you will need to have the Development-Qt system
package installed, as well as either the Subversion PBI or the Development-VCS system package. To build and install EasyPBI2, run the following commands as the superuser:

svn co svn://svn.pcbsd.org/pcbsd/current/src-qt4/EasyPBI EasyPBI-source

cd EasyPBI-source

qmake-qt4 *.pro

make install clean

Then, to update your version of the pbi-manager tools, run these commands:

svn co svn://svn.pcbsd.org/pcbsd/current/src-sh/pbi-manager

pbi-manager-source

cd pbi-manager-source

make install

Ports CVS End of Life on February 28th 2013

The development of FreeBSD ports is done in Subversion nowadays. By February 28th 2013, the FreeBSD ports tree will no longer be exported to CVS. Therefore ports tree updates via CVS, CVSup or csup(1) will no longer be available after that date. All users who use CVS, CVSup or csup(1) to update the ports tree are encouraged to switch to portsnap(8) or for users which need more control over their ports collection checkout, use Subversion directly. More information are available in the announcement mail on the FreeBSD ports announce mailing list.

VCHI driver, part 2

Some time ago I announced port of VCHI driver to FreeBSD. Since then it was re-licensed as BSD/GPL and I had high hopes for bringing it into the tree as a part of sys/contrib. This weekend I finally got around to it but turned out things had changed for worse. VCHI driver used to have this neat OS-abstraction wrapper, so overall porting process was quite simple: implement synch primitives, physical pages management, driver-specific initialization and you’re done. But… The layer was lost with driver update in October. The reason for it – OS compatibility shims are banned from Linux mainline kernel.

So now I face two options: create complete port of VCHI driver by replacing linux-specific parts with freebsd-specific code. Or create Linux compatibility shims and try to keep sources as close as possible to upstream. I’m going to try latter approach in order to minimize maintenance work. If it doesn’t work out – I’ll fall back to the former. But with any of these approaches code difference with upstream will be too significant to go to contrib tree and most likely driver will be distributed as a port.

Faces of FreeBSD – Thomas Abthorpe


Faces of FreeBSD
Are you aware of the tangible benefits derived from our support of the FreeBSD community? In conjunction with our fundraising efforts, we are spotlighting different people on our website and Facebook page who have received funding to work on development projects, run conferences, travel to conferences, and advocate for FreeBSD.
Please enjoy our third installment of our Faces of FreeBSD series!


Let us introduce you to Thomas Abthorpe. We helped him attend BSDCan 2009, 2011, and 2012 by helping with his travel expenses. Here’s his story.

Thomas' Story


My name is Thomas Abthorpe, and I am a Server Administrator working for the Canadian Government during the day. In my spare time I volunteer with a grassroots movement called Bicycles for Humanity and doing various odd jobs within the FreeBSD Project.  I became a Ports committer in August 2007, doing my own thing for a while, mostly keeping to myself, until I joined the Donations Team. 

The Donations Team was my first real non-ports function within the FreeBSD Project.  From there my functions evolved.  I was invited to take over as portmgr-secretary@ in March of 2010.  One year later my membership was upgraded to full voting member on portmgr@.  I was voted by my peers within the FreeBSD Project to join core@this past July.

In my five active years within FreeBSD, I was fortunate enough to be sponsored by The FreeBSD Foundation to attend BSDCan 2009, 2011, and 2012. At these conferences I met developers from around the world, attended DevSummits and advanced my knowledge of FreeBSD in general by attending the talks. 

The conference format lends itself well to learning, socializing and camaraderie. I attended my first conference just to learn the process and take in the experience.  At the next two conferences, I was not only there as a participant, but also to also represent portmgr@.

I have said time and time again that I am purely a hobbyist in FreeBSD. Because of the generosity of The FreeBSD Foundation, I have been able to meet with other like-minded people and do my part to share my knowledge and friendship.

Thomas Abthorpe

Donate today to help us continue and increase our support of the FreeBSD Project and community worldwide! Making a donation is quick and easy. To make a donation go to: http://www.freebsdfoundation.org/donate/

Faces of FreeBSD ? Thomas Abthorpe

We are excited to share our next story for our Faces of FreeBSD Series. This is a chance for us to spotlight different people who contribute to FreeBSD and have received funding from us to work on development projects, run conferences, travel to conferences, and advocate for FreeBSD.

Watchdogs

I’ve been having a lot of fun over the last few months with FreeBSD on BeagleBone.  Most recently, that’s involved working on the CPSW ethernet driver.

One nasty bug has been eluding me for quite some time: The controller just stops sending packets after about 20 minutes.

Eventually, I will track down this problem. However, I’ve managed to make the driver quite usable even with such an unpleasant bug: I can leave SSH sessions open for days, download port tarballs, use NFS mounts, and generally do the things you expect a network to do.

The key was to find a really good watchdog strategy. Even with the controller locking up regularly, a good watchdog notices this and resets the driver within a few seconds, fast enough that network protocols simply retry and keep going after the reset. A less effective watchdog can leave the controller non-functioning for a minute or more, resulting in failed transfers and dropped connections.

Network Driver Basics

Three functions that appear in every network driver play a part in the watchdog process:

  • The “start send” routine hands packets to the controller.
  • The “transmit completion interrupt” is invoked by the controller when a packet has finished being sent; this routine recycles the memory and other resources for subsequent packets.
  • The “watchdog ticker” wakes up once a second and decides whether or not to reset the controller.

I’ll refer to these three functions as the “start”, “interrupt”, and “watchdog” functions to match how they are named in the source code of most drivers.

The Standard Watchdog

The standard logic that appears in many FreeBSD network drivers uses a single “timer” variable that is updated in each of the above functions:

  • The start routine sets it to 5 whenever a new packet is added to the controller.
  • The interrupt routine sets it to zero whenever it reclaims the last outstanding packet.
  • The watchdog subtracts one and resets the controller when the counter changes from one to zero. (If the counter is already zero, it’s left alone.)

Remember the goal here is to detect when the network is no longer running.  That is, we want to know when the interrupt has stopped getting called.  In fact, because the interrupt isn’t getting invoked, we can entirely ignore that function for now.

To understand the standard logic, consider three different scenarios:

Scenario One:  An almost idle network, with more than 5 seconds between packets. In this case, the start routine queues some packet and sets the timer to 5. The watchdog function counts this down every second until it hits zero, then resets the controller.

Scenario Two:  A very busy network. It can take only a few milliseconds for a busy machine to completely fill the transmit queue. In this environment, each new packet will cause the timer to get reset to 5.  The watchdog may tick and reduce the timer to 4, but a new packet will immediately reset it to 5. Once the transmit queue is full, however, new packets stop getting added and the start function stops resetting the timer. Again, the watchdog function counts down and resets the controller fairly promptly.

Scenario Three:  A lightly used network. Suppose “ping” is running and the transmitter stops. In this case, one packet is getting sent every second. If the transmit queue holds 100 packets, it will take 100 seconds before the transmit queue fills up.  During that time, the start routine and the watchdog routine alternately set the timer count to 5 and 4.

This last scenario is troublesome. In the first two scenarios, the watchdog function detects and resets the failed transmitter in about 5 seconds.  But in the third scenario, the standard logic can leave the network broken for more than a minute, long enough for TCP sessions to time out and reset.

After a few days of not finding the cause for the transmitter stalls, I decided to spend a little time trying to improve the watchdog itself. I tried a variety of different approaches:  only a few handled all of these scenarios well.

A Better Watchdog

I spent almost a week experimenting with different watchdog logic before I formulated two key questions:

  • Is there something to be done?  If there’s no work to be done, then the watchdog should sit quietly.  For a network driver, this just requires checking whether there are packets in the queue waiting to be sent.
  • Has progress been made?  For network drivers, we have “progress” when any packet completes sending.

Translating these questions into code gives a watchdog function that looks something like this:

cpsw_tx_watchdog(struct cpsw_softc *sc)
{
  if (sc->tx_in_queue == 0) {
    sc->tx_wd_timer = 0; /* Nothing to do. */
  } else if (sc->tx_completed > sc->tx_completed_at_last_tick) {
    sc->tx_wd_timer = 0;  /* More stuff got done. */
  } else {
    /* Something should have been done! */
    ++sc->tx_wd_timer;
    if (sc->tx_wd_timer > 3) {
      ... reset controller ...
    }
  }
  sc->tx_completed_at_last_tick = sc->tx_completed;
}

Notice that the watchdog timer is no longer touched in either the start or interrupt routines.  The start and interrupt routines only need to maintain two statistics:  a count of how many packets have been taken off the queue by the interrupt routine (tx_completed), and a count of how many packets are still on the controller’s queue (tx_in_queue).

There’s another interesting feature of this new logic. The timer variable now has a comparatively simple interpretation: It is the number of seconds that have elapsed since we noticed work being missed. (As an exercise, try formulating a single sentence that accurately describes the timer variable in the standard logic.)

Most importantly, of course, this logic works well in all of the scenarios described above, including the lightly-used network scenario that causes problems for the standard logic.

Of course, all of the above will be considerably less interesting once I figure out how to keep the controller from stalling every 20 minutes…

Thank You for Helping Us Exceed Our Fundraising Goal!

We want to thank everyone for helping us exceed our fundraising goal for 2012. We are thrilled to report we raised $753,378. And, checks are still coming in! We should have final numbers by mid-January. Donation receipts will be emailed and mailed out by the end of January. If you made a donation and you can't find your name on our donor list, please let us know. We received so many donations in December, that we missed adding some names.

We can't thank you enough for your overwhelming support of the FreeBSD Foundation and Project. As we prepare our 2013 budget, we are very excited to be putting more funding in the Project Development area. In fact, we are always interested in hearing your project proposals. Do you have an idea of something you'd like to work on and need funding to help you accomplish it? Click here to find out about our project proposal process.

We also plan to help more developers attend BSD-based conferences. You can find out how to apply for a travel grant here:

http://www.freebsdfoundation.org/documents/TravelRequestForm.pdf

As the new year starts, we are grateful for the working with such passionate people who give their free time to the FreeBSD Project. The commitment that you all share in making FreeBSD the best OS out there inspires and drives us!

Consider making a donation for 2013! Click here to make a donation.

FreeBSD Development Snapshots – Testers Wanted

I am pleased to announce the re-availability of FreeBSD development snapshots provided by the FreeBSD Project.

As with any development branch, these snapshots are not intended for use on production systems. However, we do encourage testing on non-production systems as much as possible.

Users interested in testing the development branches are also encouraged to subscribe to the freebsd-snapshots@ mailing list, where new snapshot availability, including corresponding installation image checksums, and any additional noteworthy information about the images will be announced.

The list subscription URL is: http://lists.freebsd.org/mailman/listinfo/freebsd-snapshots

Snapshots may be downloaded from the corresponding architecture subdirectory over FTP: ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/

Problems, bug reports, or regression reports should be reported through the GNATS PR system or the appropriate mailing list, such as -current@ or -stable@ .

This directory contains FreeBSD installation snapshots for the head/ and stable/ branches. The snapshot installation media are organized in the same hierarchy as the FreeBSD release installers, for example:

snapshots/i386/i386/ISO-IMAGES/
snapshots/amd64/amd64/ISO-IMAGES/

The FTP structure is also available for bootonly.iso installers.

These snapshots do not include a package distribution, and do not include the release notes documentation.

Snapshots are expected to be generated weekly. Snapshot retention is expected to be the current week and three prior weeks. The FTP directory will always contain the distribution sets for the most recent snapshot.

Snapshots are named as:

FreeBSD-{BRANCH}-{ARCH}-{DATE}-{SVNREV}

where BRANCH is the official branch name (such as 10.0-CURRENT, or 9.2-STABLE), ARCH is the target architecure the release was built, DATE is the day the snapshot was built in YYYYMMDD format, and SVNREV is the svn revision number at the point in time the snapshot was generated.

The CHECKSUM.MD5 and CHECKSUM.SHA256 files are suffixed with the date the snapshots were generated.

In general, it can be expected that all snapshots will be built with the same subversion revision, as to avoid inconsistencies between images built on the same day.

Be sure to check the mailing lists for those architectures (e.g., freebsd-sparc64 mailing list for the sparc64 architecure) before taking one of those snapshots. The specific testing purpose that snapshot was generated for should be mentioned there. The snapshot ISOs in those architecture directories may not be suitable for general use.

Those who wish to keep up-to-date on the latest snapshot availability are advised to subscribe to the freebsd-snapshots@ mailing list, where new snapshots will be announced, including the image checksums and any notable information regarding a particular snapshot set.

The New C++ Stack in 9.1

If you read the release announcement, then you’ll have seen that there is a new C++ stack in 9.1, but that it is marked optional and is not part of the default binary install. If you’re a C++ developer, you may want to play with it for a few reasons:

  • It supports C++11 (which is new, shiny, and buzzwordy)
  • It will be the only one shipped by default in 10.0, so if you care about forward compatibility then you will need to test with it or make sure that your code works with it, or depend on GNU libstdc++ from ports.

The new stack uses libcxxrt, which I originally wrote for PathScale and was since open sourced (funded by the FreeBSD and NetBSD Foundations).  This implements the low-level part of the C++ standard library, providing things like support for RTTI, dynamic_cast, exceptions, thread-safe initialisation of statics, and so on. In the old stack, libsupc++ does this.

On top of this sits the STL implementation. In the old stack, this was GNU libstdc++, in the new one it’s LLVM’s libc++. To make interoperability easier, in 9.1 we have made libstdc++ dynamically link against libsupc++. You can use libmap.conf(5) to tell it to link against libcxxrt instead, and then you can mix libraries that use libc++ with ones that use libstdc++ in the same program.

The new stack isn’t installed by default, but building and installing it is very easy:

# cd /usr/src/lib/libcxxrt
# make
# make install
# cd ../libc++
# make CXX=clang++
# make install

You should now have libc++.so installed in /usr/lib and the headers installed in /usr/include/c++/v1. You are now ready to compile C++ code with the new stack. You use them, clang has a command-line switch for selecting the stack to use. By default, it will still use the old stack:

$ clang++ hello.cc 
$ ./a.out 
Hello C++ World
$ ldd ./a.out 
./a.out:
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x800819000)
    libm.so.5 => /lib/libm.so.5 (0x800b29000)
    libc.so.7 => /lib/libc.so.7 (0x800d4a000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80109d000)

To enable the new stack, you must add -stdlib=libc++ to your CXXFLAGS and also to your LDFLAGS. The first ensures that we get the libc++ headers, the second that we link to libc++:

$ clang++ -stdlib=libc++ -std=c++11 hello.cc 
$ ./a.out 
Hello C++ World
$ ldd ./a.out 
./a.out:
    libc++.so.1 => /usr/lib/libc++.so.1 (0x80081a000)
    libm.so.5 => /lib/libm.so.5 (0x800ac9000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x800cea000)
    libc.so.7 => /lib/libc.so.7 (0x800ef7000)
    libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x80124a000)

Unfortunately, in 9.1 there is a small bug that prevents you from compiling in C++98 mode with libc++. The C11 quick_exit() and at_quick_exit() functions are exposed in our stdlib.h only in C11 and C++11 modes, but are referenced in libc++’s cstdlib in any mode.

Most code should work out-of-the-box in C++11 mode though, so there’s little reason not to try it. And, because of the availability of move semantics, some code using STL classes will be more efficient when compiled in C++11 mode.

If you want to test with a newer version, but don’t want to install -CURRENT, I’m putting x86-64 binaries of libc++ back-ported to 9.1 online for testing. I’ll update this whenever I pull a new version into -CURRENT.

bsdtalk221 – Xenocara with Matthieu Herrb

An interview recorded by Michael Dexter at EuroBSDCon 2012 in Poland.  He speaks with Matthieu Herrb about what Xenocara is and what it is not.  More info at http://www.xenocara.org/

File Info: 11Min, 5MB.

Ogg Link: http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk221.ogg

FreeBSD 9.1-RELEASE Available

FreeBSD 9.1-RELEASE is now available. Please be sure to check the Release Notes (detailed version) and Release Errata before installation for any late-breaking news and/or issues with 9.1. More information about FreeBSD releases can be found on the Release Information page.

FreeBSD/armv6: what’s new and exciting?

It’s been a while since last update on the project status so it might seem as there was no progress in this area. The reality is: there is a bunch of activities happening with various levels of success. So I decided to give kind of end-of-the-year round-up of ongoing projects, plans and obstacles ARM hackers face.

First of all we tried switching default cache type from write-through to write-back type. It should have increased performance but instead opened a can of worms. Memory corruption debugging led to L2 cache driver on Pandaboard, EHCI driver code and subsequently to busdma code. Whole process took quite a few days full of hair-pulling and nagging various people and ended up in committing USB fixes and Ian Lepore’s busdma patches. PL310 (L2 cache controller) driver is being tested at this very moment. Original issue (WB caches) still stands and postponed till next year.

Then there are two projects by Andrew Turner aimed at modernizing FreeBSD/armv6 subsystem: switching to EABI and clang support for ARM. Daisuke Aoyama took both of them and produced working image for Raspberry Pi. He also fixed two issues with event timers on Raspberry Pi so now the platform is much more stable. I ran buildkernel in a loop overnight and by the morning Pi had survived 7 cycles and still was alive and kicking. I also managed to get python built and working on it. Didn’t have 100% success with perl 5.14/5.16, ports were built but failed at install stage segfaulting in do_clean_objs function.

My Pandaboard survived overnight buildkernel loop with L2 cache disabled, but acting up if I enable it. Investigating.

Then there are also several platform bring-ups in progress. Alexander Rybalko works on getting FreeBSD running on Efika MX Smartbook. Ganbold Tsagaankhuu hacks on Allwinner 10. Alexander Dutkowski’s hardware of choice is BeagleBoard-xM.
Ruslan Bukin experiments with Exynos4412 and Thomas Skibo reported about FreeBSD running on Zedboard (Xilinx Zynq-7000).

But what about devices/platform we have in tree? I have limited knowledge about some platforms so here is summary of the ones I’m aware of. If you have more information on any of these targets (or any other ARM-related projects) – let me know, I’ll update post.

  • BCM2835 Raspberry Pi the most accessible and therefore the one that gets the most exposure and testing. Pretty stable, considering. Supported devices: USB, network, MMC, GPIO, framebuffer, GPU. The rest is on ToDo list. VCHIQ driver is BSD-licensed now and I’m planning on getting it to sys/contrib. Userland bits of OpenGL ES should be added as a port though.
  • (update) LPC32x0 No first hand experience but judging by the code it supports MMC, FB, GPIO and USB
  • Marvel Armada XP I don’t have information about this one, sorry
  • Nvidia Tegra2 Just barebone boot stuff.
  • TI AM335x Examples: BeagleBone, TI Sitara EVM. Network was reported working but unstable on BeagleBone. USB is not supported. Haven’t tested GPIO yet.
  • TI OMAP3 Example: BeagleBoard-xM. See Alexander Dutkowski’s project
  • TI OMAP4 The hw I have – Pandaboard ES. Supported devices: USB, network, MMC, GPIO. Some issues with L2 cache
  • Versatile Platform Board Exists only as emulation target for QEMU. Supported hardware: PCI, network, framebuffer. Seems to be fairly stable, no extensive testing performed.

BeagleBone, PandaBoard ad Raspberry Pi images can be built using Tim Kientzle’s scripts.

Not really stellar list of supported peripherals I’d say. I tend to blame several things.

First – experimental and unstable state of FreeBSD/armv6 in general. It’s no fun adding new hardware support when you’re not confident in underlying subsystems stability. “I flush cache for this TX descriptor but is it really gets flushed?”. Been there, no fun at all. That’s why I believe task #1 for nearest future is maximum performance and rock-solid stability of what we have.

Then there is the case of syscons. It’s old, it’s inflexible and it’s mostly i386-centric. Just until recently most of our so-called embedded targets were headless so there were no pressure from this side to reorganize things. My experience with coding two framebuffer drivers or trying to add PS/2 keyboard support on non-i386 platform was not very pleasant. It’s messy and there is a lot of code duplication. newsyscons project may be the way to go, I haven’t looked at it yet. We just need someone(tm) to finish it and get into the tree.

Fix these two issues should make bring-up process easier. It leaves us with question of GPU support. But it’s different story for different post…

Happy New Year, everybody!

Alexander Leidinger » FreeBSD 2012-12-23 18:22:43

From time to time I convert videos to H264. When I do this I want to get the best quality out of a give filesize. This means I create VBR videos. The question here is, how big the target video file shall be.

After searching around a little bit in the net I found a formula which is supposed to give a hint about the target filesize. Naturally this depends on the encoder (or even the encoder-version), the encoder settings and even the video.

The formula is at least a good hint for my use, so I wrote a script which calculates several filesize values for a given video (based upon the output of mediainfo, which the scripts expects in a file with the filename as an argument to the script). It calculates a CBR and a VBR value for a given video based upon the width, height and duration. It should work on all system with a POSIX compatible shell.

Example output for a video from my HD-ready cam, original filesize 1.8 GB:

Width: 1280, Height: 720, FPS: 50.000, Time: 1424, Motion: 2
Per second: 6451200.000 bps / 6300 Kibps
Total CBR: 1148313600 bytes / 1121400 KiB / 1095 MiB
Total VBR: 861235200 bytes / 841050 KiB / 821 MiB
Width: 1280, Height: 720, FPS: 50.000, Time: 1424, Motion: 3
Per second: 9676800.000 bps / 9450 Kibps
Total CBR: 1722470400 bytes / 1682100 KiB / 1642 MiB
Total VBR: 1291852800 bytes / 1261575 KiB / 1232 MiB
Width: 1280, Height: 720, FPS: 50.000, Time: 1424, Motion: 4
Per second: 12902400.000 bps / 12600 Kibps
Total CBR: 2296627200 bytes / 2242800 KiB / 2190 MiB
Total VBR: 1722470400 bytes / 1682100 KiB / 1642 MiB

There are 3 sections, the difference is the “motion� value. It is a kind of multiplicator depending on the amount of motion in the video. For the videos I made myself (family videos, and even some videos of volley ball games), the first section seems to be just fine. So I reduced the original MP4 file to about 50% (not visible here is the audio size, normally I copy the original audio unmodified).

For the curious ones, the formula is

width_in_pixels * height_in_pixels * fps * motion_value * 0.07

for the bps value. The CBR value is

bps * playtime_in_seconds / 8

and the VBR value is

3 / 4 * CBR_value.

Share

Related Posts:

iXsystems Announces Release of PC-BSD 9.1 Isotope Edition

The press release for PC-BSD 9.1 is now available:

iXsystems is pleased to announce the arrival of PC-BSD 9.1 Isotope Edition, the latest release of the secure and user-friendly operating system based on FreeBSD 9.1. Several new components are introduced in PC-BSD 9.1 Isotope including a revamped Warden jail management tool, improved ZFS support, user interface enhancements, and the new server edition of PC-BSD named “TrueOS�.

The biggest change to come from this update is a complete overhaul of PC-BSD’s Warden jail management utility with support for multiple ports jails, meta-packages, Linux jails, and ZFS snapshot management. Advanced users can now enjoy unlimited FreeBSD ports sandboxes thanks to the integration of the Ports Jail utility with the Warden UI. In addition, the integration of the update manager into Warden and support for meta-packages allow users to install the complex programs available on the PC-BSD installation media, e.g., Samba and Apache, in jails. The ability to install Linux distributions, including Debian and Gentoo, in jails opens up new options for virtualization on PC-BSD. All of these functions are available from both the graphical and command line interfaces.

PC-BSD 9.1 improves ZFS support in the installer and throughout the system, adding many new features. The installer simplifies the task of disk layout, including support for ZFS mirror and up to triple-parity software RAID. ZFS users can now use the ‘beadm’ utility to back up the boot environment before an upgrade or major system change and restore it if necessary. In Warden on ZFS, entire jails (including Linux jails) may be cloned and rolled back. These advanced administrative tools help PC-BSD live up to its reputation as a powerful and versatile desktop operating system.

“PC-BSD has made stunning progress and is rapidly becoming the Unix workstation OS we have all been waiting for,” says Michael Dexter, the editor of CallForTesting.org and a long-time BSD lecturer and advocate. “From including ZFS and elegant management tools to being completely GUI-agnostic, PC-BSD embraces and extends FreeBSD in dramatic yet respectful ways and the result is a great desktop experience for not only end users but also administrators and developers.”

Several other improvements continue to ensure that PC-BSD 9.1 remains user-friendly and accessible to everyone. Set-up is easier than ever with the new, simplified installer that requires as few as four clicks for the default installation. The new installer also separates pre-installation and post-installation tasks, allowing OEMs to install the system and leaving final configuration to the end user. An “About� icon has been added to the Control Panel, making it easy to determine the PC-BSD version and which desktops and version of X Window System have been installed. The new release supports KDE 4.9.3 and improves support for wifi and Intel video.

One of PC-BSD 9.1’s most exciting and anticipated new features is the new server installation option. The regular installer now presents the option to install TrueOS, a custom server edition of PC-BSD. TrueOS provides command line versions of PC-BSD utilities (including Warden, Meta-package Manager, and PBI Manager tools) in addition to the base FreeBSD install. It’s an excellent choice for users who want to avoid the overhead of even the lightest-weight window manager but want to take advantage of the powerful tools available in PC-BSD. iXsystems offers Professional Software Support for TrueOS and PC-BSD.

“With the new TrueOS server option, system administrators and enterprise users of Linux will immediately feel more at home being able to install a system with packages such as Bash, Apache, or Samba available out of box,� says Kris Moore, founder and lead developer of the PC-BSD project. “This, coupled with command-line versions of the ‘Warden’ jail management tool, meta-package manager, update manager and others, makes running a BSD-based server easier than ever.�

For more information on PC-BSD or to download the new PC-BSD 9.1, please visit http://pcbsd.org. PC-BSD is also available for retail sale at http://www.freebsdmall.com in DVD form.

About PC-BSD
PC-BSD is a fully functional, user-friendly desktop operating system based on FreeBSD. It runs on the latest FreeBSD version 9.1 with a desktop interface of the user’s choice and graphical system installer. Its PBI system, developed for PC-BSD and also available on FreeBSD, allows users to download and install their applications in a self-extracting and self-installing format.

About iXsystems
iXsystems is the all-around FreeBSD company that builds FreeBSD-certified servers and storage solutions, oversees FreeNAS development, and is the corporate sponsor of the PC-BSD Project. iXsystems is an employee-owned and operated, open source-centric, customer focused organization dedicated to providing the highest-quality built-to-order enterprise rackmount server solutions, pre-configured server appliances, and scalable storage solutions to our customers around the globe.

FreeBSD Foundation Newsletter

The December issue of the FreeBSD Foundation Newsletter has been published:

FreeBSD has a flourishing community. Innovative technology is packed into every release. Thousands of products and services the world relies upon are built on FreeBSD, and the adoption rate of FreeBSD accelerates year after year. By all of these measures, FreeBSD is an amazingly successful open source project. But at the FreeBSD Foundation, we know today's success is just a step on a long path. We constantly ask, "How can we make FreeBSD even better?" "How can we take FreeBSD up that 'extra notch'?" Or as Nigel from 'This is Spinal Tap' would put it, "If we're playing maxed out at ten, how do we take it to eleven?"

Our answer for the 2013 year is to redefine the way the FreeBSD Foundation supports FreeBSD. Every year since I started the FreeBSD Foundation, we have grown our budget and activities by 10 to 25%, all managed by a single part-time employee. This next year we are investing in staff. Staff to bolster FreeBSD's amazing community of volunteers. Staff to scale the FreeBSD Foundation's funded development initiatives. Staff to double our capabilities in a single year.

This plan will increase the number and size of the development projects we fund. However, it is the FreeBSD Foundation's investment in "human infrastructure" that I believe will yield the largest dividends. As a volunteer myself, I know that no matter how deep my love for FreeBSD is, family and my "day job" must come first. Even with these other priorities, the volunteers that maintain the project's computer clusters, handle security incidents, tend our revision control and bug tracking systems, and generate our releases, show amazing skill and awe inspiring dedication. But our pool of volunteers for these areas hasn't grown as quickly as the rest of the project. By adding paid staff to support these roles, we will help reduce burn-out, provide continuity, and increase the effectiveness of FreeBSD's volunteers.

If things go as expected, by the end of 2013, the FreeBSD Foundation will have five employees, and an annual budget approaching $750,000. It will be a challenging transition, but a necessary step on the path of FreeBSD's continued success. Please consider giving us your support while we work to take FreeBSD all the way to eleven!

Justin T. Gibbs
President and Founder
The FreeBSD Foundation

Fundraising Update

Wow. I’ve been thoroughly overwhelmed by the outpouring of donations over the last few weeks! As of this publication we have raised $460,000 towards our goal of $500,000.

I want to thank you for everything you do to make this the best operating system around. There wouldn’t be a FreeBSD if we didn’t have you writing code, writing documentation, working on ports and releases, and educating current and future FreeBSD users.

We haven’t met our target yet, but we are getting close. Historically, each year, we have been half way to our fundraising goal at the start of December. Going forward, this is something that we plan on changing.

We unintentionally received some interesting press that disturbed a lot of FreeBSD users. This encouraged over 950 donations to come in, this past week. All I can say, is that it was incredible to see this support. One donor commented, “I don't use FreeBSD yet, but I've heard good things about the project, so that's why I wanted to support you.� How cool is that?

Your donations help us fund projects to improve FreeBSD, sponsor conferences and summits, purchase equipment to build infrastructure, promote FreeBSD, and provide legal counsel for the Project. In short, it helps us to provide the funding to make this the best OS available.

We’ve received some great lessons during this campaign. One thing we learned is that we need to advertise our fundraising needs outside of our FaceBook page, blog, and the FreeBSD Announce mailing list.

I hope that as you read through some of our accomplishments this year, you will consider making a donation to the Foundation. We can’t do it without you!

Efika SmartBook/SmartTop FreeBSD ARM Port

Tired of burning yourself with your laptop? If so, maybe it’s time for a change, such as switching to a platform that doesn’t require a large heat sink or loud cooling fans.

It is now possible to run FreeBSD on a low power, ARM-based, laptop: the Genesi Efika MX Smartbook. It has an ARM Cortex-A8 CPU inside, and, just like a cell phone, is energy efficient, consumes minimal power and uses passive cooling.

Our main project goal is to implement the missing pieces for the Freescale ARM SoC, including graphics, and to provide a great example of FreeBSD running on a low power device that developers can both use, and show off.

With the Efika MX code in the tree, we will have support for ARM-based laptops and even nettops (such as the Efika MX Smartop model) and we can continue pushing FreeBSD into ARM tablets and other ARM-based devices.

Capsicum Component Framework

Capsicum is a novel hybrid capability model that first appeared in FreeBSD 9.0, targeted at application compartmentalization: the mitigation of security vulnerabilities through decomposition of complex and risky applications into isolated components.

In the big picture, Capsicum provides a tight sandboxing mechanism where access to all global name spaces is restricted, and also allows actions that can be performed on file descriptors to be limited (file descriptors represent capabilities).

The purpose of my project was to make Capsicum more mature as well as easier to use by application developers.

The main part of the project was to create the Casper daemon, which is used by sandboxed processes to gain access to functionality that is not directly permitted within a sandbox. As an example, Casper provides a DNS service which can be use by sandboxed processes for host name resolution without the need to read /etc/resolv.conf or send any network packets to DNS servers.

Another part of the project was to remove capability wrapping descriptors - a special kind of file descriptors with a limited set of rights (e.g. a file descriptor which can be used only for reading, but not writing, fchmod, etc.). The capability wrapping descriptors were replaced with the ability to limit rights on any regular descriptor. This simplifies much of Capsicum's kernel support code and makes sandboxing more robust.

Initially the ioctl(2) system call was denied in capability mode (i.e. for sandboxed processes), because of its huge scope. Now, it is possible to limit the ioctl commands that can be performed within a sandbox, on a given descriptor. I also added the ability to control fcntl operations in a similar fashion.

Many bugs were fixed, many smaller improvements were made, and a lot of new regression tests were created along the way.

Capsicum is still on-going work, but thanks to funding from the FreeBSD Foundation and Google, it is maturing quickly and is starting to be used in real world applications (I personally use it in my other, FreeBSD Foundation sponsored, projects: HAST and auditdistd).

iSCSI Target

The goal of this project is to create a native, high performance, iSCSI target facility for FreeBSD. While configuration and connection setup and teardown are handled by a userland daemon, unlike previous target frameworks, all data-movement is performed in the kernel. The iSCSI target is fully integrated with the CAM Target Layer meaning that volumes can be backed by files or any block device. The hardware offload capabilities of modern network adapters will also be supported.

Much of the protocol and data movement logic can be shared between iSCSI target and initiator implementations. Once target support is robust, FreeBSD's existing iSCSI initiator will be updated to use many of the components developed for the target. This will improve initiator performance and add modern features such as InitialR2T.

The project is quickly progressing from a working prototype to a fully functional state; testing phase is expected to start in early January. After that the development will focus on implementing the hardware offload and performance tuning.

EuroBSDCon 2012

In October 2012, the 11th EuroBSDcon was held in the beautiful city of Warsaw in Poland. Four days of talking, learning about BSD, and of course exploring Polish culture (don't forget the Polish Hussars). The conference featured 2 tutorial days followed by two conference days on Saturday and Sunday. The conference consisted of a 2 track program, paralleled by a devsummit track on Saturday.

Traditionally, EuroBSDcon has a strong FreeBSD presence, but there is also enough room for the other BSD's. One of the reasons for the strong FreeBSD presence is the FreeBSD devsummit, which is held concurrently with the EuroBSDcon tutorial days.

The FreeBSD Foundation has been one of the regular sponsors over the years. This support plays an important role in making this an outstanding conference.

Besides sponsoring the conference, The FreeBSD Foundation also provides travel grants for visitors and developers who can not afford to go the conference on their own. This gives them the opportunity to visit the conference and exchange knowledge and ideas with other conference attendees, which greatly improves the value of having such conferences.

We are excited to announce that EuroBSDcon 2013 will be held September 28th-29th, 2013 (tutorials will be September 26th-27th) on Malta (St. Julian's area).

Cambridge Developer Summit 2012

Earlier this year, I had the pleasure of co-organizing the Cambridge DevSummit. Robert Watson, in contrast, had most of the stress, and others had the fun of running around and actually doing the work. This was a three-day event, with a documentation summit scheduled for the day before, organized primarily by Gavin Atkinson and Isabell Long.

This was the second such event hosted in Cambridge; the first in which no one fell in the river—possibly because the weather wasn't quite nice enough to encourage punting that week.

The three days of the main event were split into three sessions, with two tracks in each. I invited some visitors from ARM (who, conveniently, are just down the road) to attend one afternoon session. This was productive, and led to further engagement between the FreeBSD community and ARM. ARM is a customer-focused company, and those who may be interested in FreeBSD/ARM are potential customers of ARM's customers.

We finalized the schedule on the first day, filling an entire whiteboard with items that two or more people wanted to discuss and then splitting into groups. While a bit more advanced planning may have been helpful, our strategy worked relatively well this year. A short summary from each of the groups was presented in the final session and then published on the FreeBSD wiki.

The toolchain session, a big session, was of particular interest to me. We arrived at a tentative plan for throwing the switch to make clang the default compiler on FreeBSD-CURRENT. This was further discussed on the mailing list, and has now happened, bringing us one big step closer to a GPL-free FreeBSD 10.

An afternoon of short talks from researchers in the Cambridge Computer Lab involved either operating systems work in general or FreeBSD in particular. Robert Watson showed off a tablet running FreeBSD on a MIPS-compatible soft-core processor running on an Altera FPGA.

Cambridge, an old university town, predates the invention of hills and so is very amenable to cycling. We hired bikes for all non-local developers so they could get around easily.

The DevSummit dinner was hosted by St. John's College and cosponsored by Google and the FreeBSD Foundation. This enjoyable event, in the historic surroundings of a 500-year old college, provided a unique (and possibly never-to-be-repeated) opportunity of seeing a group of FreeBSD developers smartly dressed.

The day after the conference, Ollivier Robert organized a trip to Bletchley Park, which this year was celebrating Turing's centenary. Although it was an informative outing, I don't believe that it resulted in any concrete plans for a FreeBSD/Collossus port.

Bay Area Vendor Summit 2012

On November 8th The FreeBSD Foundation sponsored the FreeBSD Silicon Valley Vendor Summit. The summit was hosted by Yahoo at their Sunnyvale campus. We had over 50 participants from more than 20 different companies participate in the summit, which is a great turnout.

After a round of introductions we discussed the work that had been completed since the last summit, in May of 2012, including:

    Altera IP core drivers
    Amazon EC2 work
    Comprehensive test framework (ATF)
    DTrace on non-x86 (MIPS)
    Intel graphics
    Ivy Bridge hwpmc
    clang/llvm MIPS
    ZFS improvements
    growfs on live fs

After going over the completed and still to be done items, we had a brief set of comments from Tom Hanrahan of Microsoft on HyperV support for FreeBSD. Microsoft's intention is to have full hypervisor support for FreeBSD and they are actively engaging with the community to get the proper drivers in the right places in our tree so that FreeBSD is a first class citizen in their system.

The first major presentation of the day was Jim Harris talking about VTune. VTune is a system to help developers understand the performance of their code. A small number of us have been working with Intel to get this software available on FreeBSD, as, at the moment, it is only available on Windows and Linux.

VTune includes two major components, a data collector and a visualization program. The collector has been ported to FreeBSD and is available as a pre-beta to interested developers. The visualization program continues to only be available on Windows and Linux, but since the collector and the visualizer are separate it is easy to collect information about programs, including the kernel, running on a FreeBSD host and display the results on another system. One feature that VTune continues to lack, but which Intel is working on, is support for call graphs. Right now the system can show you which line of code is causing a performance problem, which is quite useful, but it does not show the call graph, basically how your program happened to get to that point. This is a feature that Intel knows it needs on FreeBSD for the system to be completely accepted by vendors and other folks building high performance systems with FreeBSD. That being said, it's still an impressive piece of software and will definitely help people who care about performance tuning.

Jim says that those who are interested in using VTune in its current form should contact him directly to get ahold of the beta driver: Jim Harris . I've put up a "VTune How To" page on our Wiki and will be populating it over the next week.

After the morning break Adrian Chadd brought people up to date on what's going on with embedded systems and FreeBSD. There has been a good deal of progress here, in particular the work done recently on ARM processor support, with FreeBSD now running on several systems, including the BeagelBone, ShivaPlug, and the now ubiquitous Raspberry Pi. MIPS support continues to improve, with a significant chunk of work coming from the folks in Cambridge, as well as companies that are integrating FreeBSD on MIPS into their products. PowerPC is also moving along, although there remains a dearth of hardware on which to work. A good deal of discussion accompanied this section, in particular about what we need to do to get the rest of the system, i.e. ports and packages, up on these architectures. Here the story is also improving with a set of packages existing for ARM processors as well. People interested in this area should check out recent mail threads on arm@ and embedded@ as well as others.

After talking about embedded systems, Daichi Goto updated us on efforts with vendors in Japan. He and Hiroki Sato have been busy working with companies in Japan and have set up a BSD Consulting company which is working with large Japanese companies to build business services on top of FreeBSD, with a great deal of success. Other work they're doing includes getting a good deal of FreeBSD documentation and information translated into Japanese, so that it is more easily accessible to developers in Japan.

After a lunch break a new, at least to me, FreeBSD Vendor, CacheIQ gave a presentation about their product and their use of FreeBSD. CacheIQ builds an SSD based NFS caching system on top of FreeBSD and are interested in following the project's development tree more closely. What followed was a discussion of the project's release policies and best practices relating to developing a product with FreeBSD.

From CacheIQ we moved on to testing and ATF. Garrett Cooper presented the work he's been doing, in collaboration with Simon Gerraty and Marcel Moolenaar, to get ATF integrated into FreeBSD. ATF was originally written for NetBSD and has been in use there for a couple of years now. The goal is to have a good framework for automated testing in FreeBSD and to improve the quality of our testing. As anyone who has worked with test frameworks knows, there are many bike sheds to be avoided here, and Garrett, Simon and Marcel have worked quite hard to keep away from picking colors, and have instead gotten the system into shape and working on FreeBSD. [As of the writing of this newsletter ATF is now in HEAD under contrib/atf.]

Our last session of the day was creating a new have/need list for vendors with the twist this time that we added a "co sponsor" list. The FreeBSD Foundation is going to put up funds to co-sponsor development projects with companies working on FreeBSD. This has already been successfully done in the past with projects such as the NAND Flash project (co-sponsored with Juniper Networks) and others. The Foundation can't sponsor all these projects, but where there is an ability to get cooperation on funding the development it makes sense to get a few interested parties together to get the work done. The list of possible co-development projects includes:

    NetConf Agent
    EFI Boot on amd64
    NDMP
    MIPS Super Pages
    SID Base Credentials and ACLs
    Xen Dom 0

Upcoming summits are in March at AsiaBSDCon and May at BSDCan. Mark your calendars.

2012 Grant and Travel Grant Recipients

Every year we sponsor FreeBSD related conferences and travel to these events for FreeBSD contributors. We believe that BSD-centered and FreeBSD-specific conferences play the dual roles of expanding the FreeBSD user community and supporting collaborative development. The FreeBSD Foundation's travel grant program helps to reduce financial roadblocks to participation in these events.

Our grant recipients often send us amazing tales of their experiences, proving the value of this program to the FreeBSD community. You can find these stories and trip reports on our blog.

Here is a list of projects, developers, and conferences we have sponsored for 2012.

2012 Conference Grant Recipients:

    AsiaBSDCon 2012 Conference
    BSDCan 2012 Conference
    Ottawa 2012 Developer Summit
    Ottawa 2012 Vendor Summit
    BSDDay 2012
    EuroBSDCon 2012 Conference
    Cambridge 2012 Developer Summit
    Bay Area 2012 Vendor Summit

2012 Project Grant Recipients:

    Edward Napierala - iSCSI Target project
    Pawel Jakub Dawidek - Capsicum Component Framework
    Edward Napierala - Growing Filesystmes Online
    Björn Zeeb - IPv6 Performance Analysis
    Pawel Jakub Dawidek - Implementing auditdistd
    Semihalf - NAND Flash Support
    Aleksandr Rybalko - Porting FreeBSD to Efika ARM platform

2012 Travel Grant Recipients:

    BSDCan - Hiren Panchasara, Adrian Chadd, Florian Smeets, Ben Haga, Marius Strobl, Brooks Davis, Julien Laffaye, Warren Block, Daichi Goto, Giovanni Trematerra, Davide Italiano, Thomas Abthorpe
    EuroBSDCon - Gabor Pali, Alberto Mijares, Gabor Kovesdan, Alexander Pronin
    Open Help - Warren Block
    MeetBSD - Mark Linimon

Faces of FreeBSD Series

Who are these people that receive money from the Foundation? How are your donations making a difference in the FreeBSD community? We've asked our grant recipients to share their stories with you in what we call the "Faces of FreeBSD" campaign. Tune in weekly to the FreeBSD Foundations's blog to hear how Foundation funding is used to run conferences, work on development projects, help with travel expenses to conferences, and to advocate for FreeBSD.

Looking for a recap of past stories? Here are our first two "Faces of FreeBSD":

Faces of FreeBSD - Dan Langille

Faces of FreeBSD - Alberto Mijares

Netflix

More than 30 million Netflix streaming members around the globe watch more than a billion hours of movies and TV shows each month. In the US, Netflix video streaming accounts for a third of peak downstream Internet traffic. Netflix created Open Connect, a single-purpose content delivery network, to help deliver these petabytes of data.

The main component of Open Connect is the Open Connect Appliance, a small-footprint network streaming device. The Open Connect Appliance is a 4U Intel-based server that is designed to economically maximize storage density in a space and power footprint that is ideal for both ISP data centers and metro-area network interchanges.

The Open Connect Appliance runs on FreeBSD 9. Netflix picked FreeBSD 9 because it is a high performing, low-maintenance and reliable operating system that is supported by major hardware vendors. FreeBSD 9 provides a foundation of reliability, performance, and hands-free manageability. Combined with NGINX, a light-weight and high performance Web server, FreeBSD 9 provides a simple but powerful solution that is capable of serving tens of thousands of simultaneous video streams across multiple 10Gbit fiber optic links.

Beyond its technical strengths, FreeBSD comes with an outstanding ecosystem of developers, vendors, and users who openly share expertise, talent, and technical improvements. Netflix has embraced this community and is committed to giving back its bug fixes and enhancements, thus completing the circle of community collaboration.

- David Fullagar, Director of Content Delivery Architecture, Netflix


Personal story of a ports committer

You all know that sometimes you end up dealing with all the ugly stuff instead of doing useful work. Over the last few months I was kept busy at $dayjob got assimilated by portmgr and had to look after redports. All of those new challenges are nice on it's own and I really enjoyed being part of the FreeBSD community and ecosystem but then 11/11 happened.

At that day quite a lot has changed for me since redports was isolated as a precaution and all ports building clusters of portmgr were effectively shut down. That situation was quite a mess since all automated systems and clusters were gone. No INDEX builds, no QAT, no pointyhat so also no exp-runs anymore. Whenever someone broke the ports tree we didn't even knew. It proved to be quite hard to get back on track again after that incident. INDEX checks and a very very limited QAT are already running again but pointyhat and redports are still dead. :(

The daily frustration and dealing with all that strange decisions that are taken because of the need to get stuff done is hard sometimes. But It's almost Christmas and without redports I have much more spare time so I try to calm down and focus on stuff that I can hack on my own. And that worked out quite nice so far ...

tvheadend:
I've noticed in the XBMC 12.0 release notes that they have included the PVR branch and thus support DVB-S2/C/T in XBMC. Well actually they only provide some backend configuration interfaces and rely on a backend like mythtv or tvheadend to handle the DVB stuff. Mythtv would be okay for that task but It's huge for such a small job. Tvheadend is a nice and small TV streaming server that suites perfectly and only does the bare minimum without a lot of dependencies. Configuration is done in a web based GUI or can be done in XBMC. So I started working on a tvheadend port. A few weeks later I'm at the point now where tvheadend compiles fine and also starts. I've just ripped out all that epoll stuff and linuxisms that I stumbled accross so it doesn't run properly yet. Adding kqueue support is the next step now.

virtualbox:
Due to redports being unavailable the vbox work has also frozen. I tried to collect all that patches and complains in my inbox so that they don't get lost. Since the situation did not improve I temporary created a github repository for the virtualbox ports and committed all the accumulated patches there: https://github.com/decke/freebsd-vbox

This includes almost all patches that were flying around on mailinglists and updates vbox to 4.2.6 / 4.1.24 but testing was very limited so take care if you give them a try. Testing will show us if we can commit it to the portstree by New Year's Eve.