10.0.2-PRERELEASE ISO Available for Testing

The next 10.0.2-PRERELEASE ISO is now available for testing and can be downloaded from
http://download.pcbsd.org/iso/10.0-RELEASE/testing/amd64/.

If you have a spare system or virtual machine, consider testing this image. If you find any bugs, report them at https://trac.pcbsd.org so we can take a look at fixing them before 10.0.2 is released later this month.

NOTE: if you plan to use AppCafe in this image, go to Configure -> Repository Settings and change it to “Edge”. Do this before attempting to upgrade within AppCafe; otherwise, if you reboot or logout, you will not be able to successfully log back in again.

Hacking on Receive Side Scaling (RSS) on FreeBSD

RSS is a Microsoft invention that tries to keep a given TCP or UDP flow (and I think IP, but I haven't yet tried that) on a given CPU core. The idea is to try and keep both flow-local data and flow-local locking on a single CPU core, increasing the chances that data is hot in the CPU core cache and reducing the chance of lock overhead.

You can find the RSS overview and programming details here:

http://msdn.microsoft.com/en-us/library/windows/hardware/ff567236(v=vs.85).aspx

RSS and supporting technology has been making its way into FreeBSD for quite some time but it's not in any real shape that application developers can take advantage of.

Firstly, there's "PCBGROUPS", which looks to group PCB (protocol control block) data for a connection local to a CPU. Instead of there being one global PCB table for the system (well, VIMAGE for FreeBSD - each virtual image instance has its own PCB table) with one lock protecting it, there's now multiple PCB tables, one per "thing". Here, the thing is whatever the kernel developer thinks is worth grouping them by.

http://www.ece.rice.edu/~willmann/pubs/paranet_usenix.pdf

Now, until the RSS work went in, this code was in FreeBSD but sat unused. A kernel developer could provide the hooks needed to map TCP (and maybe UDP later) flows to a "thing" and have that map to a PCB group table - but it required some glue to stamp incoming connections and outgoing packets with some identifier (which we call a "flowid" in FreeBSD) with something that can map to said "thing". Then whenever a PCB lookup was needed, it would first try the lookup in the table mapped to by the mapping between the "flowid" and "thing" - if it was successful, it wouldn't have to use the global PCB table to do the lookup.

This is only good for established connections - creating and destroying a connection still requires manipulating that global PCB table and the single PCB table lock. I'm going to ignore fixing that for now, as that is a bigger issue.

Then Robert Watson added the RSS work done under contract to Juniper Networks, Inc. RSS provides one kind of mapping between the flowid from the NIC and which CPU to run work on. So that part worked great - but there wasn't any way for the application user to take advantage of it. Additionally, there's no driver awareness of it yet - I'll discuss this shortly.

So I grabbed a bunch of this work whilst at Netflix and tried to make sense of it. It turns out that if you can keep the work local to a CPU, a lot of the lock contention in the networking stack melts away. Here's what's going on:

  • The receive thread(s) in the NIC driver processing packets are typically doing direct dispatch to the network stack - so they're running the receive side of the TCP stack;
  • .. and the receive side of the network stack includes ACKs, which triggers the transmit side of the network stack;
  • There's typically some deferred thread(s) in the NIC driver transmitting packets to each NIC queue;
  • There's also application threads trying to queue data to the TCP socket, which also can dig into the socket and TCP stack state, which involves grabbing locks;
  • And there's also timers firing to update state, and doing this involves grabbing locks.
Without RSS and without lining everything up on CPU cores, all the above can run on different cores. Whenever any of them try running at the same time, lock contention can occur and that particular task can stop. If the lock contention blocks the transmit or receive NIC threads, then not only is that connection affected - the whole NIC processing is affected.

There's still lock contention in the network stack - especially if you're doing a lot of new, short connections. The good folk at Verisign are working on that particular corner of the problem so I'm happy to defer to them.

So, I ended up doing a bunch of little pieces to get this lined up right:
  • The per-CPU timer callwheels can now be optionally pinned to their CPU cores, so timer events running on CPU X actually do run on CPU X (yes, that was amusing to find..);
  • There's support in the TCP stack for per-CPU timers, but it's not enabled by default;
  • ... and it also didn't query RSS, netisr or anything to figure out how to map a flowid to a given CPU to run a timer on;
  • Then to make matters worse, incoming TCP sessions didn't have a flowid assigned to the PCB until after the first data packet was read - which meant that the initial timer work would all assume CPU 0 and any queries on that particular PCB would return flowid=0 - so it would not find it in the right PCBGROUP.
So those are fixed in FreeBSD-HEAD. The per-CPU TCP timer and pinned-CPU timers aren't enabled by default - I'll only flip that on when I'm confident that the RSS stuff is working.

So that lets all the RSS stuff correctly work. But there wasn't a nice way to query the per-connection flowid or RSS information. So I then extended netstat to have 'R' as a flag - it returns the flowid and the flowid type. I'll add RSS information once I have a nice way to extract it out in bulk. It's still a good diagnostic tool to ensure that the IPv4/IPv6 hashing is working correctly.

Then I had to teach a driver about RSS so I could actually test it all out. I have some igb(4) hardware at home, so I did the minimal work required to teach it about the RSS key and assigning things to the correct CPUs. It's still incomplete but it's good enough to get off the ground. I'll go into more details about the driver requirements in a follow-up blogpost.

Finally, how are application developers supposed to use it? I'll cover that particular bit in another follow-up blog post as there's quite a lot to cover there.

FreeBSD 9.3-BETA2 Available

The second BETA build for the FreeBSD-9.3 release cycle is now available. ISO images for the amd64, i386, ia64, powerpc, powerpc64 and sparc64 architectures are available on most of our FreeBSD mirror sites.

Playing nice with others. git(1) and patches on #FreeBSD

I’ve been spending a lot of time massaging a branch of patches and other assorted bits and pieces for QEMU user mode on github

This led me down the path of being a good git user and contributor, so I’ll leave these notes for myself and others in the event you come into a situation where you need FreeBSD to play nice with people who are very git(1) centric.

After an update by [email protected] to the devel/git port, you can now install git(1) and have it work out of the box.  The most frustrating thing, after using git for like 5 minutes, is to figure out how to extract a patch out of it and send it all pretty-like to the mailing list(s) that would be consuming the patch.

In its simplest incarnation, you can simply reference a commit hash and us it to generate a patch via git format-patch, but this will give you the entire commit diff between the referenced version and HEAD.  This, in my case generated approximately 3000 patch files.

e.g. git format-patch –output-directory ~/patches –to=”[email protected]” c60a1f1b2823a4937535ecb97ddf21d06cfd3d3b

What I want, is a diff of one revision, which requires a start and ending hash:

format-patch –output-directory ~/patches –to=”[email protected]” c60a1f1b2823a4937535ecb97ddf21d06cfd3d3b…c6ad44bb288c1fe85d4695b6a48d89823823552b

Now I send this to the mailing lists via my client.  Here is where I kind of head-desked a bit.  If you are like me and run a mail server yourself and you use SSL with self-signed certs, then this little bit if for you.  I lost about an hour trying to figure this little bit out.

The way to dump patches from your patch director (~/patches) is to use:

git send-email patches/*

This will use the following variables in your git environment:

sendemail.smtpserver=mail.ignoranthack.me
sendemail.smtpencryption=ssl
sendemail.smtpuser=[email protected]
sendemail.smtpserverport=465
sendemail.smtpsslcertpath=
sendemail.annotate=yes

Notice the empty “sendemail.smtpcertpath” variable.  Without that set to EMPTY, git would repeatedly fail on the self-signed cert that I use.  So, I’m pretty sure something still isn’t setup correctly.  However, it must be set to EMPTY and not undefined.  Else, you will repeatedly fail with certificate validation errors.

Getting to know your portmgr@ – Steve Wills

It is my pleasure to introduce Steve Wills, the newest member of the portmgr team. Steve has done a tremendous work on the ports tree, especially in the field of testing and quality. Here is a short interview to get to know him better.

Name
Steve Wills

Committer name
swills

Inspiration for your IRC nick
Boring, it’s my userid.

TLD of origin
.us

Current TLD (if different from above)
Same.

Occupation
Sysadmin.

When did you join portmgr@
2014

Blog
Used to have one, use twitter more now (@swills)

Inspiration for using FreeBSD
Simplicty and learning.

Who was your first contact in FreeBSD
Can’t recall, it was ages ago.

Who was your mentor(s)
pgollucci

What was your most embarrassing moment in FreeBSD
Trying to migrate Ruby default version from 1.8 to 1.9 and having to roll back.

Boxers / Briefs / other
Heh, question assume survey taker is male, which I am, but I think we need to
work on diversity (but not in that “hey, let’s work on diversity and get some
women” way, but more in that we make something everyone wants to use)

What is your role in your circle of friends
The FreeBSD user. ;)

vi(m) / emacs / other
vi(m)

What keeps you motivated in FreeBSD
New users, new committers.

Favourite musician/band
I listen to a decent variety of stuff, but I suppose the thing I come back to
most is NIN.

What book do you have on your bedside table
I have an iPad by my bed, which I bought to read, but mostly I browse news on
it.

coffee / tea / other
Don’t drink caffeine, so don’t drink coffee much. I do drink good beer tho.

Do you have a guilty pleasure
Good dark chocolate. :)

How would you describe yourself
Mostly standard in many ways, husband, father, FreeBSD hacker, sysadmin, in
that order.

sendmail / postfix / other
Sendmail, tho dma is nice too.

Do you have a hobby outside of FreeBSD
Used to play guitar, still have one, don’t find time to pick it up much any
more.

What is your favourite TV show
Futurama

Claim to Fame
Ported Acidwarp from DOS to svgalib.

What did you have for breakfast today
Everything bagel with plain cream chese.

What sports team do you support
The only sport I watch is University of North Carolina Basketball.

What else do you do in the world of FreeBSD
ruby ports, perl ports sometimes

What can you tell us about yourself that most people don’t know
I was an employee at Red Hat way way back

Any parting words you want to share
Not really.

What is your .sig at the moment
Null.

Steve

FreeBSD 9.3-BETA1 Now Available

FreeBSD 9.3-BETA1 Now Available

The first BETA build of the 9.3-RELEASE release cycle is now available on the FTP servers for the amd64, i386, ia64, powerpc, powerpc64 and sparc64 architectures.

The image checksums can be found in the PGP-signed announcement email.

ISO images and, for architectures that support it, the memory stick images are available here:

    http://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/9.3/

(or any of the FreeBSD mirror sites).

If you notice problems you can report them through the normal GNATS PR system or on the -stable mailing list.

Please note, as the FreeBSD bug tracking system is undergoing maintenance, the PR system may be unavailable.  Problem reports submitted this maintenance period are being queued for later processing.

If you would like to use SVN to do a source based update of an existing system, use the "stable/9" branch.

A list of changes since 9.2-RELEASE are available on the stable/9 release notes page here:


The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases.  Systems running earlier FreeBSD releases can upgrade as follows:

    # freebsd-update upgrade -r 9.3-BETA1

During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly.

    # freebsd-update install

The system must be rebooted with the newly installed kernel before continuing.

    # shutdown -r now

After rebooting, freebsd-update needs to be run again to install the new userland components:

    # freebsd-update install

It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 8.x.  Alternatively, the misc/compat8x port can be installed to
provide other compatibility libraries, afterwards the system must be rebooted into the new userland:

    # shutdown -r now

Finally, after rebooting, freebsd-update needs to be run again to remove stale files:

    # freebsd-update install

Love FreeBSD?  Support this and future releases with a donation to the FreeBSD Foundation!  https://www.freebsdfoundation.org/donate/

BSDCan Trip Report: Michael Dexter

The next trip report is from Michael Dexter:

BSDCan 2014 was an amazing experience as always but one theme characterized this year more than any other: Coordination.

Never in my dozen years in the community have I seen such an active dialog between the BSD projects with attention being given to what each project is up to. From praise to constructive criticism, developers from all of the projects engaged each other in sessions and in the priceless hallway track. Beginning with a project that is close to my heart, Peter Grehan announced at the FreeBSD DevSummit that the bhyve hypervisor would soon support NetBSD, rounding out its support for OpenBSD, NetBSD and Linux virtual machines. I can think of no better way for developers to see first hand how each operating system works and to cross-validate code. Kudos to Peter, Neel Natu, John Baldwin and everyone else who has helped bhyve become such a useful feature in FreeBSD.

Continuing in the spirit of coordination, Abhishek Gupta of Microsoft's Hyper-V group was on hand to discuss with developers how to guarantee that FreeBSD is a first-class Hyper-V guest OS. From the sound of it, Microsoft appears to have more developers focusing on FreeBSD than Intel! Together, bhyve and Hyper-V represent compelling OS-native hypervisors and rest assured, Windows virtual machine support in bhyve is under active development.

Matt Ahrens of the OpenZFS project gave his annual update on what new ZFS features are making their way into FreeBSD in order to keep FreeBSD a first-class ZFS platform. Of these features, ZFS "bookmarks" will enable ZFS replication without relying on snapshots as a unit of history. Just how quickly the OpenZFS project transitioned from post-Sun Microsystems confusion to solid, OS-agnostic contributions is remarkable. We all owe Matt our gratitude for his active participation in the BSD community at events like BSDCan and AsiaBSDCon.

Other DevSummit highlights included a clarification of FreeBSD's "long term support" policy with the comforting recognition that the project had in fact been more or less adhering to the proposed 5-year policy. A formal affirmation of such a policy is a valuable marketing tool for everyone from vendors to end users. The idea was also raised about separating the FreeBSD base into packages to allow for modular updating and deployment. Done right, this could be of great value to embedded FreeBSD efforts.

Two notable highlights of the FreeBSD Doc Sprints were the participation of Ingo Schwarze of the mandoc project who committed FreeBSD's Igor documentation proofing tool to OpenBSD ports, and Allan Jude's formal entrance into the FreeBSD project with a documentation commit bit. Allan and Kris Moore have done a great job raising awareness of FreeBSD and other BSD projects with the BSDNow podcast and are demonstrating just how seamless community and code participation can be.

Though many of us were already exhausted from all-day discussions and late-night coding, it was finally time for the conference proper to begin. This saw an infusion of yet more wonderful people and continued engagement and coding. Security was a key topic this with the FreeBSD Address-space layout randomization (ASLR), Capsicum and LibreSSL talks standing out as must-see. Each talk was highly cross-pollinated by developers from different BSD projects with almost a sense of obligation to the Internet community as a whole, given BSD's key role in the development of the Internet.

The Embedded track comprised of ARM, MIPS64 and NAND flash storage talks and was also very timely given the changing nature of computing. Warner Losh went into great detail about how NAND flash storage works and how broad a range of reliability is available from the various flash technologies. This track even extended to a lunch time MIPS router hacking BOF lead by Sean Bruno. It is great that we have real Unix on really-affordable hardware.

The closing auction was fun as always and the clouds broke on Sunday, allowing quite a few attendees to walk around Ottawa and Parliament before heading home. Some brave systems administrators opted to take the first BSD Certification Group BSD Professional exam and the feedback I heard was very positive. The BSD Professional exam is a hands-on exam designed to compliment the BSD Associate exam that the BSDCG has offered for several years. This is an exciting development and is testament to the continued growth of the BSD community.

I would like to thank Dan and his team for putting on another great BSDCan and the FreeBSD Foundation for helping me attend this year.

FreeBSD 9.3-BETA1 Available

The first BETA build for the FreeBSD-9.3 release cycle is now available. ISO images for the amd64, i386, ia64, powerpc, powerpc64 and sparc64 architectures are available on most of our FreeBSD mirror sites.

Weekly Feature Digest 30

Hey PC-BSDers!  This week we’ve been gearing up for the next release of PC-BSD version 10.0.2.   In preparation for the next release we have been fine tuning some of the new features and making sure the loose ends are tied up.   We were also able to close out a good amount of trac tickets this week and commit the fixes for 10.0.2.

In other news / updates this week:

AppCafe

  • Fix a bug where the orphan package filter was also filtering out some base apps.
  • Randomize the browser home page so that it only show 10 random “recommended” and “highlighted” applications.
  •  Add a ton more recommended/highlighted applications to the repo file.
  • Fix some minor display bugs
  • Add menu option to view the recent vulnerability information for ports through freshports.
  • Fix the sizing information for installed meta-pkgs (will show the combined sizes of the direct dependencies instead)
  • Fix the sizing information for available applications (will now show the combined size of all the packages that need to be downloaded/installed for that app)

EasyPBI

  •  Add the ability to fetch/read the pkg-plist for a given pkg.
  • Add a “bulk” module creation side to EasyPBI which allows for creating PBI modules for an entire FreeBSD category at a time (with all sorts of filters and options)
  • Make EasyPBI automatically create up to 5 desktop/menu entries for graphical applications.
  • Make the application binaries detected/usable within the module editor for creating new desktop/menu entries.

Lumina

  • Quick fix for filenames that have spaces in them
  • Quick fix for making sure that when launching an app it is in the same general system environment. This allows apps like firefox/thunderbird to see other instances of themselves and act appropriately.
  • lumina-config - Make sure the menu options actually work

Miscellaneous Fixes / improvements

  • Fixed several warden bugs relating to new jail creation / package management
  • Imported the latest ports and Gnome3 / Cinnamon for 10.0.2
  • Fixed some issues prompting for GELI password from GRUB and then mountroot
  • Fixed a critical bug with new CUPS 1.7.0 breaking foomatic-rip and associated print drivers
  • Imported the latest PEFS code into 11-CURRENT and backported it to our 10-STABLE branches
  • Fixed bugs with system update tray notifier not showing freebsd-update” notifications
  • Migrated one of my build systems to 11-CURRENT and got it setup for doing PKG/ISO builds
  • Misc other trac tickets fixed / closed in cleanup process
  • Many other cosmetic / doc bugs fixes as Dru submitted them
  • Started investigating bug with BE/GRUB failing if the first dataset is destroyed

BSDCan Trip Report: Warren Block

BSDCan 2014 was held earlier this month and the Foundation provided travel grants to several committers. The first trip report is from Warren Block of the doceng@ team:

Every year, BSDCan is preceded by a developer summit, where FreeBSD committers and invited guests can get together to discuss proposals, difficulties, and plans.  Registration this year was at the intriguingly-named "Goat BOF".  There are stories behind this, but I'll just point you to https://twitter.com/GroffTheBSDGoat and https://plus.google.com/109575245711252585947/photos.

For the documentation developer summit group, Benedict Reuschling made the case for helping documentation translators from our outdated manual system to using gettext-based PO files.  These systems eliminate much of the manual work translators are forced to do with the current setup, allowing them to concentrate on translating. They also provide "translation memory", remembering phrases and sentences that have already been translated so it is not necessary to retranslate them when they appear in different documents.  The room had been fairly quiet up until Benedict began demonstrating this, at which point there was a loud "oooh!" from the back of the room.  There is still a bit of work to be done to fit these tools into our translator workflow, but the research is mostly done and the rest is just pounding it into a shape that can be used with our existing documents and setting up the first translation team to use it.

Each night, we had a "doc lounge", where people were welcome to come to learn about or work on documentation.  We split up the individual time with a few short presentations.  I showed how I used textproc/igor to proofread documentation changes, and it surprised me at how others were using it, and surprised them at how I was using it.  This in-person communication with a crowd of differing experience and viewpoints is one of the best features of BSDCan.

BSDCan itself began on Friday with a keynote session from Karl Lehenbauer, CTO of FlightAware.  This was one of the best presentations I've yet seen at any BSDCan, and worth the time to watch.   As in previous years, the FOSSLC group was there, making high-quality recordings of the presentations.  Compared to videos taken with a traditional camera, screens in the FOSSLC videos are easy to see and the speakers can be heard.

There were talks aimed at using FreeBSD on embedded hardware, with Warner Losh speaking about using NAND flash memory (apparently no video available, but an associated video is here) and Sean Bruno describing installation of FreeBSD on wireless routers with MIPS processors.

John-Mark Gurney showed how he had improved geli(8) encryption performance from less than 150MB/second to greater than 900MB/second.

Daichi Goto gave a talk called "Shellscripts and Commands", which was an interesting combination of traditional shell-based tools and fast hardware to process huge datasets.

Saturday morning, Ingo Schwarze from the OpenBSD project talked about "New trends in mandoc", the excellent full-text search abilities developed for this OpenBSD replacement for groff(1).  Ingo also attended several of our doc lounge
sessions, and we had some interesting comparisons between the document checking provided by igor and that in mandoc.

Vsevolod Stakhov talked about the new solver in pkg (no video available yet).  What I find particularly encouraging about this and other aspects of the new package system is the amount of research into other systems. That is the "good kind of lazy": the problem is difficult, and rather than jumping in and hacking together a solution that partly works, doing the research to find how other groups have done it.

FreeNAS is becoming increasingly popular, and John Hixson talked about how to add custom applications to it (no video yet).  Later, Fabio Balzano described a FreeBSD-driven ROV (remotely operated vehicle) using a Beaglebone Black ARM single-board computer.

At another doc lounge session, we covered the complete process to fix an error in the FreeBSD documentation, from installing the tools, to editing, checking, and build-testing the document, through to submitting a patch.  It's very good to note that some of the people we worked with have already had patches submitted and accepted since then.

FreeBSD developer Li-Wen Hsu was at several of the doc lounge sessions, and one night asked about integrating igor with the Jenkins continuous improvement framework.  I was skeptical about using igor for this, but we talked about some tests that would avoid false positives.  The next night, he returned.  Not only had he modified igor to produce the required output, he'd already set up a Jenkins test! It showed just how useful this continous automated testing can be, even if the test tool is not perfect.  In hindsight, I should have realized that this sort of thing is just an extended use of automation, which is the point behind igor: we have these nice computers, let's use automation to help us accomplish our goals.

Finally, Allan Jude of BSD Now (and many other FreeBSD-related things) had clearly been in line for a commit bit for some time now.  Benedict had a plan to keep it a secret and surprise him with the announcement during an interview. The full interview will be seen on a future episode of BSD Now.

This was all just a tiny part of BSDCan 2014.  There were numerous other talks that you should watch, like the already-famous one by OpenBSD's Bob Beck on LibreSSL, their fork of OpenSSL: http://www.youtube.co/watch?v=oM6S7FEUfkU, or http://www.youtube.com/watch?v=GnBbhXBDmwU.

All of the FOSSLC videos are here.

All the presentations and informal talks are still just a small part of BSDCan.  There is the "hallway track", where it is common to start talking with another person about something that's important to both of you... and then getting so caught up that you miss a presentation or two.  There are before- and after-hours talks with others on things that seem to have been overlooked, but it turns out were important to them also.  Lots of people you may only know by email address will be there, almost always looking completely unlike imagined.  At one point or another, almost everyone is drafted by Dan Langille to help carry boxes or set up power strips. There's lots of caffeine and more than a little sleep deprivation. Conferences like these help provide the motivation that drives projects throughout the rest of the year.

A big thanks goes to the FreeBSD Foundation for sponsoring my trip this year.  Thanks also to Dru Lavigne, Benedict Reuschling, Allan Jude, Dan Langille, and everyone who came to the developer summit, doc lounge, and BSDCan.  Your time and attention are appreciated.  Thank you all for helping to improve FreeBSD!

bsdtalk241 – Bob Beck

Interview at BSDCan 2014 with Bob Beck from the OpenBSD Project and the OpenBSD Foundation.

File Info: 26Min, 12MB.

Ogg Link: http://cis01.uma.edu/~wbackman/bsdtalk/bsdtalk241.ogg

BSDCam 2014

BSDCam 2014 (http://bsdcam.cl.cam.ac.uk/), University of Cambridge, Cambridge, United Kingdom 9 - 12 July, 2014. The Cambridge FreeBSD Developers Summit is an annual invite-only event focused on bringing together developers and vendors to discuss and build the future of the FreeBSD project. This years topics will include the desired feature set of FreeBSD 11, implementing the new release strategy for the 9 and 10 branches, packaging the base system, and building the infrastructure and tools to attract more embedded vendors to FreeBSD.

Frederic Culot takes over as portmgr-secretary@

It is with great pleasure that the FreeBSD Ports Management Team announces that Frederic (culot@) Culot will take over responsibilities of team secretary effective immediately.

Frederic became a ports committer in October 2010, and joined the ranks of portmgr-lurkers@ in March 2014 as the shadow secretary.

Please drop him a note and congratulate him (or offer condolences).

 

Thomas
on behalf of portmgr@