Author Archives: Dru Lavigne

BSDCan Trip Report: Li-Wen Hsu

The next trip report is from Li-Wen Hsu:

I am very excited for having the chance to join the most important and largest annual BSD conference. Thanks to the FreeBSD Foundation, it's my first year to attend BSDCan. The main motivation for attending is that I'm in part of the project started by Craig Rodrigues, Jenkins CI for FreeBSD, and and I am honored to be invited join that group.
 

I arrived in Ottawa on May 13th. After checking into Residence and taking a short nap to ease jet-lag, I went to the Royal Oak Pub to join the pre-party of the developers. Sean Bruno quickly recognized me and introduced me to other developers. I talked with Steve Wills, Mark Linimon, Gavin Atkinson, and met Peter Wemm, my roommate.
 

The first day of the Developer Summit started with presentations about changes to the support plan and brainstorming about FreeBSD 11. During the break, I spoke to Mark Johnston, who completed the last piece of axge(4), our first USB 3.0 to gigabit ethernet adapter driver. It is written mainly by Kevin Lo and I provided some fixes in rx/tx routines. During our chat, we discussed the performance issues of axge(4) where he discovered there might be a limitation of calling rx/tx routines numbers per second in the USB stack. This is done by just a few lines of DTrace code. I was totally shocked by that and decided that I should learn more about it.
 

In the afternoon, I joined The Java working group where Greg Lewis introduced the history, current status, and we discussed the future plans of the Java port. We talked about how to improve user experience, support for important Java software, and the known problems of Java on FreeBSD. There was also a discussion on how to to get more developers who want to develop Java applications on FreeBSD. We think that DTrace support might be attractive for people who run Java on FreeBSD.

We had Thai food in the Hacker Lounge for dinner. George V. Neville-Neil and I talked about how to make more people develop or support their software on FreeBSD. I showed him Travis CI which is used by many open source projects developed on GitHub for their continuous integration needs. However, it cannot support FreeBSD in the near future. Being a ports committer for several years, I feel that many projects are willing to support FreeBSD, but lack the environment and experience. I think we should work more with external communities to address this. We discussed the possibility of establishing such a service for FreeBSD, following the successful model of RedPorts. There are many tricky parts and security issues to consider. Furthermore, the most important part is manpower. If any reader is interested in helping, please contact me.
 

On the second day of the Developer Summit, I went to the Continuous Integration and Testing with Jenkins for FreeBSD working group in the morning. In the first part of the meeting, Craig introduced Jenkins and how it is utilized in the FreeBSD.org cluster. He also covered the internal architecture of jenkins.FreeBSD.org. In the second part of the meeting, we discussed the next steps to work on this year. Craig and I helped Julio Merino set up a Jenkins instance on his laptop and Julio quickly hacked Kyua to let it generate JUnit output which can be parsed by Jenkins. This is very exciting to us because it means that we can have a trackable and easy-to-read continuous integration report. We believe this can help developers and contributors to produce higher quality code and to find items they can start to work on.
 

In the afternoon, I joined the Documentation Translation System Session where Benedict Reuschling introduced a process to translate documentation just like using gettext for software i18n and l10n. This process is done by translating docbook XML files to .po files with po4a, then translators can use their favorite po file editor to concentrate on the content instead of struggling with non-human readable XML files. It is also possible to establish a "translation memory" to remember the phrases and sentences that have already been translated for sharing between documents, which reduces duplicated work from translators. We also talked about a wish: one web-based system where casual translators can fix a translation by clicking a mouse while the backend takes care of the rest. The doc committers or another contributor can commit the change back to the doc repository.
 

Finally, I asked about continuous integration for the doc tree. Warren Block suggested that we can run igor for checking the errors, however there are some false positives that would bother people. During BSDCan, I joined two of the Doc Sprints. One night I asked Warren about "safe parameters" for igor and I quickly hacked igor to generate the checkstyle format XML output and pass it to the Jenkins checkstyle plugin. I presented the proof of concept on the second night. It is really great that people thought it is useful and encouraged me to setup it as a job on jenkins.FreeBSD.org. Warren will help me with this. In the future, this could also integrate with Phabricator as a "lint" tool for being as a filter.
 

I always wanted to revive the Traditional Chinese Document Translation Project. Fortunately, about two weeks after BSDCan, a volunteer sent a mail to the freebsd-doc mailing list stating that he wants to contribute to the Traditional Chinese translation of the FreeBSD Handbook. After discussing with him and with the help from people on EFNet/#bsddocs and doceng, I converted zh_TW from Big5 to UTF-8 in the doc tree for making future translation easier. This is really a good restart and I hope more people can join and we can have a complete Traditional Chinese handbook and other documents soon.
 

The following two days were the BSDCan sessions. The starting keynote speaker was Karl Lehenbauer from FlightAware. The most rememberable might be the slide "A billion dollars + Linus <= good people with a rigorous engineering process doing BSD." Also, there are some slides about FreeBSD tuning at FlightAware.
 

In Luigi Rizzo's talk In-kernel OpenvSwitch on FreeBSD, I learned a lot about how to port the Linux kernel subsystem to FreeBSD. In his work, he provides netlink sockets for the FreeBSD kernel. This is very good news because I always heard that people who are familiar with Linux want this feature.
 

Patrick Kelsey gave the libuinet talk. This is used to port the FreeBSD TCP/IP stack to userland. It means that the resource needed for a connection is in userspace memory and the kernel only needs to provide a packet interface, such as netmap. This is useful for an application that handles lots of concurrent connections. Patrick became a committer recently and I hope more work from him can bring libuinet closer to HEAD soon.
 

Pawel Jakub Dawidek and Mariusz Zaborski talked about their work on Capsicum and Casper. In this talk, they presented the lack of traditional security mechanism (setuid(2), chroot(2), P_SUGID, setrlimit(2)) provided by the system, and how an easier protection is provided by Capsicum. The Casper daemon provides services to sandboxed processess which do not have the necessary rights.
 

The FreeBSD Developer Summit and BSDCan overlap for one day. May 16 is the public track of the dev summit and I attended two sessions. Michal Dubiel from Semihalf gave the status update of OpenStack and OpenContrail on a FreeBSD host. I am glad that there are companies which invest in cloud technologies for FreeBSD. I hope this can be in production soon and maybe the FreeBSD cluster can have some setup to enrich developer resources. Another session I attended is lightweight reference counting, by Gleb Smirnoff. Using counter(9) in FreeBSD 10 as a reference count is really a brilliant idea. I am looking forward to seeing performance improve with this solution.
 

Arun Thomas gave a good tutorial of ARM and how BSD supports it. For a person like myself with no experience in embedded systems, it was a good start. Now I can have more fun with my Raspberry Pi.
 

Julio Merino talked about The FreeBSD Test Suite, which is really important to me and the FreeBSD continuous integration group. He also announced the plan to combine Kyua and Jenkins in the session. He hopes that the tests can be more complete and the CI pipeline can be more mature. There are still lots of things to be done!
 

Matt Ahrens presented the goals of the OpenZFS project and its current status. With this project, platforms like Illumos, FreeBSD, Linux, and OS X can directly interact with a shared, platform-independent ZFS code base. This will greatly reduce the effort to port changes between platforms and the tests can also be shared. The future of the OpenZFS features are amazing, including resumable zfs send/recv and device removal, which can make a system administrator's€™ life much easier.
 

The ASLR talk by Shawn Webb was awesome and this is definitely a feature that a paranoid person like myself  hope will be merged to the main trunk soon. It seems to still have problems on ARM so he also asked for help from people with ARM experience.
 

On return home, I was surprised that Peter and I were on the same flight and sat next to each other. We talked about Linux containers and the projects that use it like Docker, which is the part that FreeBSD is not doing well. Currently, the resource limitation of the lightweight containers is not really complete. He said that the way we using servers, or the "computing nodes", are changing in lightspeed and we should not be left behind. We both agree that a modern operating system should put more effort in cloud and mobile solutions.
 

I would like to thank the FreeBSD Foundation again for sponsoring me to attend this great event. I made new friends and met people who only know each other on the Internet before. We shared many good ideas and it is really awesome to know there are so many people working on FreeBSD. I hope I can participate again next year!

BSDCan Trip Report: Michael Dexter

The next trip report is from Michael Dexter:

BSDCan 2014 was an amazing experience as always but one theme characterized this year more than any other: Coordination.

Never in my dozen years in the community have I seen such an active dialog between the BSD projects with attention being given to what each project is up to. From praise to constructive criticism, developers from all of the projects engaged each other in sessions and in the priceless hallway track. Beginning with a project that is close to my heart, Peter Grehan announced at the FreeBSD DevSummit that the bhyve hypervisor would soon support NetBSD, rounding out its support for OpenBSD, NetBSD and Linux virtual machines. I can think of no better way for developers to see first hand how each operating system works and to cross-validate code. Kudos to Peter, Neel Natu, John Baldwin and everyone else who has helped bhyve become such a useful feature in FreeBSD.

Continuing in the spirit of coordination, Abhishek Gupta of Microsoft's Hyper-V group was on hand to discuss with developers how to guarantee that FreeBSD is a first-class Hyper-V guest OS. From the sound of it, Microsoft appears to have more developers focusing on FreeBSD than Intel! Together, bhyve and Hyper-V represent compelling OS-native hypervisors and rest assured, Windows virtual machine support in bhyve is under active development.

Matt Ahrens of the OpenZFS project gave his annual update on what new ZFS features are making their way into FreeBSD in order to keep FreeBSD a first-class ZFS platform. Of these features, ZFS "bookmarks" will enable ZFS replication without relying on snapshots as a unit of history. Just how quickly the OpenZFS project transitioned from post-Sun Microsystems confusion to solid, OS-agnostic contributions is remarkable. We all owe Matt our gratitude for his active participation in the BSD community at events like BSDCan and AsiaBSDCon.

Other DevSummit highlights included a clarification of FreeBSD's "long term support" policy with the comforting recognition that the project had in fact been more or less adhering to the proposed 5-year policy. A formal affirmation of such a policy is a valuable marketing tool for everyone from vendors to end users. The idea was also raised about separating the FreeBSD base into packages to allow for modular updating and deployment. Done right, this could be of great value to embedded FreeBSD efforts.

Two notable highlights of the FreeBSD Doc Sprints were the participation of Ingo Schwarze of the mandoc project who committed FreeBSD's Igor documentation proofing tool to OpenBSD ports, and Allan Jude's formal entrance into the FreeBSD project with a documentation commit bit. Allan and Kris Moore have done a great job raising awareness of FreeBSD and other BSD projects with the BSDNow podcast and are demonstrating just how seamless community and code participation can be.

Though many of us were already exhausted from all-day discussions and late-night coding, it was finally time for the conference proper to begin. This saw an infusion of yet more wonderful people and continued engagement and coding. Security was a key topic this with the FreeBSD Address-space layout randomization (ASLR), Capsicum and LibreSSL talks standing out as must-see. Each talk was highly cross-pollinated by developers from different BSD projects with almost a sense of obligation to the Internet community as a whole, given BSD's key role in the development of the Internet.

The Embedded track comprised of ARM, MIPS64 and NAND flash storage talks and was also very timely given the changing nature of computing. Warner Losh went into great detail about how NAND flash storage works and how broad a range of reliability is available from the various flash technologies. This track even extended to a lunch time MIPS router hacking BOF lead by Sean Bruno. It is great that we have real Unix on really-affordable hardware.

The closing auction was fun as always and the clouds broke on Sunday, allowing quite a few attendees to walk around Ottawa and Parliament before heading home. Some brave systems administrators opted to take the first BSD Certification Group BSD Professional exam and the feedback I heard was very positive. The BSD Professional exam is a hands-on exam designed to compliment the BSD Associate exam that the BSDCG has offered for several years. This is an exciting development and is testament to the continued growth of the BSD community.

I would like to thank Dan and his team for putting on another great BSDCan and the FreeBSD Foundation for helping me attend this year.

BSDCan Trip Report: Warren Block

BSDCan 2014 was held earlier this month and the Foundation provided travel grants to several committers. The first trip report is from Warren Block of the doceng@ team:

Every year, BSDCan is preceded by a developer summit, where FreeBSD committers and invited guests can get together to discuss proposals, difficulties, and plans.  Registration this year was at the intriguingly-named "Goat BOF".  There are stories behind this, but I'll just point you to https://twitter.com/GroffTheBSDGoat and https://plus.google.com/109575245711252585947/photos.

For the documentation developer summit group, Benedict Reuschling made the case for helping documentation translators from our outdated manual system to using gettext-based PO files.  These systems eliminate much of the manual work translators are forced to do with the current setup, allowing them to concentrate on translating. They also provide "translation memory", remembering phrases and sentences that have already been translated so it is not necessary to retranslate them when they appear in different documents.  The room had been fairly quiet up until Benedict began demonstrating this, at which point there was a loud "oooh!" from the back of the room.  There is still a bit of work to be done to fit these tools into our translator workflow, but the research is mostly done and the rest is just pounding it into a shape that can be used with our existing documents and setting up the first translation team to use it.

Each night, we had a "doc lounge", where people were welcome to come to learn about or work on documentation.  We split up the individual time with a few short presentations.  I showed how I used textproc/igor to proofread documentation changes, and it surprised me at how others were using it, and surprised them at how I was using it.  This in-person communication with a crowd of differing experience and viewpoints is one of the best features of BSDCan.

BSDCan itself began on Friday with a keynote session from Karl Lehenbauer, CTO of FlightAware.  This was one of the best presentations I've yet seen at any BSDCan, and worth the time to watch.   As in previous years, the FOSSLC group was there, making high-quality recordings of the presentations.  Compared to videos taken with a traditional camera, screens in the FOSSLC videos are easy to see and the speakers can be heard.

There were talks aimed at using FreeBSD on embedded hardware, with Warner Losh speaking about using NAND flash memory (apparently no video available, but an associated video is here) and Sean Bruno describing installation of FreeBSD on wireless routers with MIPS processors.

John-Mark Gurney showed how he had improved geli(8) encryption performance from less than 150MB/second to greater than 900MB/second.

Daichi Goto gave a talk called "Shellscripts and Commands", which was an interesting combination of traditional shell-based tools and fast hardware to process huge datasets.

Saturday morning, Ingo Schwarze from the OpenBSD project talked about "New trends in mandoc", the excellent full-text search abilities developed for this OpenBSD replacement for groff(1).  Ingo also attended several of our doc lounge
sessions, and we had some interesting comparisons between the document checking provided by igor and that in mandoc.

Vsevolod Stakhov talked about the new solver in pkg (no video available yet).  What I find particularly encouraging about this and other aspects of the new package system is the amount of research into other systems. That is the "good kind of lazy": the problem is difficult, and rather than jumping in and hacking together a solution that partly works, doing the research to find how other groups have done it.

FreeNAS is becoming increasingly popular, and John Hixson talked about how to add custom applications to it (no video yet).  Later, Fabio Balzano described a FreeBSD-driven ROV (remotely operated vehicle) using a Beaglebone Black ARM single-board computer.

At another doc lounge session, we covered the complete process to fix an error in the FreeBSD documentation, from installing the tools, to editing, checking, and build-testing the document, through to submitting a patch.  It's very good to note that some of the people we worked with have already had patches submitted and accepted since then.

FreeBSD developer Li-Wen Hsu was at several of the doc lounge sessions, and one night asked about integrating igor with the Jenkins continuous improvement framework.  I was skeptical about using igor for this, but we talked about some tests that would avoid false positives.  The next night, he returned.  Not only had he modified igor to produce the required output, he'd already set up a Jenkins test! It showed just how useful this continous automated testing can be, even if the test tool is not perfect.  In hindsight, I should have realized that this sort of thing is just an extended use of automation, which is the point behind igor: we have these nice computers, let's use automation to help us accomplish our goals.

Finally, Allan Jude of BSD Now (and many other FreeBSD-related things) had clearly been in line for a commit bit for some time now.  Benedict had a plan to keep it a secret and surprise him with the announcement during an interview. The full interview will be seen on a future episode of BSD Now.

This was all just a tiny part of BSDCan 2014.  There were numerous other talks that you should watch, like the already-famous one by OpenBSD's Bob Beck on LibreSSL, their fork of OpenSSL: http://www.youtube.co/watch?v=oM6S7FEUfkU, or http://www.youtube.com/watch?v=GnBbhXBDmwU.

All of the FOSSLC videos are here.

All the presentations and informal talks are still just a small part of BSDCan.  There is the "hallway track", where it is common to start talking with another person about something that's important to both of you... and then getting so caught up that you miss a presentation or two.  There are before- and after-hours talks with others on things that seem to have been overlooked, but it turns out were important to them also.  Lots of people you may only know by email address will be there, almost always looking completely unlike imagined.  At one point or another, almost everyone is drafted by Dan Langille to help carry boxes or set up power strips. There's lots of caffeine and more than a little sleep deprivation. Conferences like these help provide the motivation that drives projects throughout the rest of the year.

A big thanks goes to the FreeBSD Foundation for sponsoring my trip this year.  Thanks also to Dru Lavigne, Benedict Reuschling, Allan Jude, Dan Langille, and everyone who came to the developer summit, doc lounge, and BSDCan.  Your time and attention are appreciated.  Thank you all for helping to improve FreeBSD!

FreeBSD Foundation Accepting Travel Grant Applications for BSDCan 2014

Calling all FreeBSD developers needing assistance with travel expenses to BSDCan 2014.

The FreeBSD Foundation will be providing a limited number of travel grants to individuals requesting assistance. Please fill out and submit  the Travel Grant Request Application by April 7th, 2014 to apply for this grant.

This program is open to FreeBSD developers of all sorts (kernel hackers, documentation authors, bugbusters, system administrators, etc).  In some  cases we are also able to fund non-developers, such as active community members and FreeBSD advocates. More details are available in the announcement.

vBSDCon Trip Report: John-Mark Gurney

The first trip report from vBSDCon is from John-Mark Gurney:

Thank you for the opportunity to go to vBSDcon.  vBSDcon was a great conference, and I was immediately and warmly welcomed by the host, Verisign, when I arrived.  They ran an exceptional conference and I hope this becomes an annual event.

One of the main reasons I attended the conference was to be able to meet FreeBSD developers I haven't seen for years and to meet new FreeBSD developers.  For the first time, I met Luigi Rizzo who I worked with over 15 years ago on FreeBSD's sound system and ISA UPnP code.  It was good to meet him and talk about some of his netmap work.

I also met some long time developers, Scott Long and George Neville-Neil, but I also met some that I haven't met before such as Adrian Chadd, David Chisnall (theraven), Ed Maste, Randall Stewart (rrs) and Baptiste Daroussin (bapt).  There are many more that I am forgetting, but that's simply because the conference was well attended.

I participated in the embedded BOF.  It was well attended and Adrian Chadd was the leader.  One of the issues that came up was the issue of how to make ARM and/or MIPS platforms Tier 1.  It was discussed how relevant the requirement that a new Tier 1 platform must be able to cross-build packages from an existing Tier 1 platform.  There was also discussions on which ARM board should be chosen was a reference platform.  The ones that were discussed as most viable were the BeagleBone Black, Raspberry Pi, a ChromeBook or a PandaBoard.

I attended the talk that Verisign gave on how they use FreeBSD to host the .net and .com domains.  It was impressive what they are able to do with netmap for UDP traffic, but they are also running issues w/ TCP connection performance on FreeBSD due to some lock scaling issues.

The hallway track is always interesting, and I was able to talk with David Chisnall about ways to possibly support changing functions at kernel load time to choose the best implementation.  One use of this would be to choose which AES implementation is best for the machine. If the machine has support for native AES instructions (AES-NI for example) it would use that, otherwise it could fall back to a software implementation.

Though unfortunately the hallway track made me miss Brad Davis's Speed Geeking talk on GELI, but it did allow me to spend time talking with Matt Olander, co-founder of iXsystems, Inc.  It was interesting to learn about the history of iX and find out about where they are interested in going with FreeBSD.  I may be able to help them with some work on FreeBSD in the future.

I attended the talk that Henning Brauer and Reyk Floeter gave on pf in OpenBSD.  The new features that pf has are very interesting and exciting. The features make possible reverse HTTP proxies and other packet steering techniques very easy to do.  I do wonder if using DXR for the table lookups that pf/ipfw do could be helpful, though the rule scanning overhead is still probably the largest overhead.

In summary, the conference was great and I was able to meet a large number of people and get some great ideas on how to help move FreeBSD forward.

Thank you again FreeBSD Foundation for providing me with the travel grant.

EuroBSDCon Trip Report: Mike Ma

The next trip report is from Mike Ma:

I recently had a fantastic trip to Malta in late September, and I attended FreeBSD devsummit together with EuroBSDcon 2013 for 4 days in total. I would like to first thank the FreeBSD Foundation. As a GSoC student this year, I was supported by the FreeBSD Foundation to make the trip happen. The weather was incredibly nice, so I had some extra summer time there since I'm in rather a cold country. It was the first time for me to attend a FOSS conference, well, (not?) surprisingly, I was the only Chinese person there. It was totally different from the research conferences I have attended, where I always see a lot of Asian faces.

One big thing I did at the devsummit was to present my GSoC project during a "student session" . My project was about porting glusterfs. There were four GSoC student talks where we talked about our biggest achievement as well as the problems/difficulties we had. I didn't have time to do a demostration of my project, and I illustrated and explained the experimental results I had at that point. I got very helpful advice on how to improve and extend my experiments.  I was seeing a big performance drop when using glusterfs in comparison to using the underlying filesystem, and people suggested for me to see if there's a similar performance drop on Linux. Then I figured out that the performance gap was normal, so I'm now ready to submit my changes to GlusterFS. If everything goes through, I'll then make it appear in our ports tree.

The most interesting talk to me at the devsummit was lldb on FreeBSD. Ed Maste did a brief demo and I was quite impressed by the smarter and more powerful debugger. I talked to Ed afterwards saying I'm willing to join the project, and I've now set up all the environment. I'll first look at some build failures and start with fixing some entrance-level issues.

I also liked the ZFS session on the first day. It gave me information about various issues and features about ZFS.  Sadly, I didn't manage to go to ZFS talk at EuroBSDcon as I was doing my own presentation at the same time.

Gavin and Gabor helped the GSoC students a lot at the conference.  They were very nice to introduce us to developers related to our project and interests. I also had some random talk with many FreeBSD developers, such as hps@, jhl@, Ilya, etc.  I got to know their projects and some FreeBSD history.

For the future work, I have talked with my mentor Sean Bruno. We agreed to look at some PRs that he is in charge of, which could be a good start for me. I also talked to Pedro, the mentor of the FUSE GSoC project during coding time.  He's also maintaining ext2fs and I'm now looking at fast symlink for our ext2fs implementation.

So far, it's quite a mess as there's a lot to do. Hopefully I'll have to time to do everything I'm interested in.

Again, many thanks to Sean, everyone who helped me with my project, and everyone I met in the conference. And a huge thanks to FreeBSD Foundation for funding my trip.

EuroBSDCon Trip Report: Isabell Long

The next trip report is from Isabell Long:

In the last week of September 2013, thanks to Foundation funding, I flew to Malta to partake in the FreeBSD DevSummit and EuroBSDcon. I spent a total of six days in Malta, experiencing the lovely weather, quite British culture, and food that on my first night included one of my favourites - rabbit - which turns out to be a national dish.

As a documentation committer, my main reason for attending the conference was to attend the documentation session and meet face-to-face with many people who have, for the last eighteen months, been just names, email addresses and IRC nicks. That session was productive as we used the time to hack on PRs and Handbook chapters. After all, it's important to have a balance between discussion and working on problems in-person in order to get things done. There was a discussion about the pressing and frustrating issue of the FreeBSD website design, where the pros and cons of frameworks such as Ruby on Rails were mentioned.

In the DevSummit track of the conference on the Saturday, the Google Summer of Code students' presentations of their stunningly complex work were great. At the end of the Saturday came the long awaited beach social event, which for some involved swimming at 11pm in the pleasantly warm sea. I can't swim, so I just paddled! On the Sunday, the keynote seemed scaremongering, all about how "nothing is secure", but it was nevertheless interesting. In the afternoon, as Netflix is a service I use, I attended one of their sessions. I was only just able to find a seat due to it being so popular and me being late due to the "hallway track" during the coffee break, but it was worth it - the statistics of the amount of US Internet traffic they account for were almost unbelievably high. Kirk McKusick's talk was one of three that ended the conference, and it was very informative for me because it seemed to touch on a lot of the basics.

I have come back and noticed how cold it is, but armed with a list of things to do on the documentation side, a new found love of learning interesting things, a wish to get involved in other areas of the FreeBSD project, and have more contact in terms of work with some other developers who I talked to at length over the dinners.

NYI Whitepaper Available

As part of our mission to serve the FreeBSD community, we have increased our efforts to educate people on FreeBSD. One of these efforts was to fund a project to produce white papers on using FreeBSD in different applications. Here's our first one called Managed Services Using FreeBSD at NYI by Joseph Kong. From the Executive Summary:

This white paper describes the challenges associated with being a managed services provider, which include interacting with the wider Internet, ensuring high availability, recovering from data loss, and compartmentalizing systems and data. To surmount these challenges, this white paper describes several FreeBSD-based solutions: PF, CARP, pfsync, HAProxy, GEOM mirroring, FreeNAS, ZFS, rsync, and jails. Each solution has been battle-tested by NYI, an ISP headquartered in New York, whose customers include Men’s Journal, Rolling Stone, and Us Magazine.

EuroBSDCon Trip Report: Marius Strobl

EuroBSDCon was recently held in Malta. The first trip report is from Marius Strobl:

First off, I would like to thank Andre Oppermann and his team for organizing such an amazing conference and developer summit! It would have been a great event even if it had not taken place in Malta but that location really made it outrageous. I totally did not expect to be able to go swimming in the sea at 9:30 PM as part of the social event with water that actually felt warmer than the surrounding air - just to mention one of the many highlights. Seeing all that people having fun in the sea at night with glow sticks around their wrists was a lovely sight, too. Thus, it speaks for itself that I am also very grateful to the FreeBSD Foundation for sponsoring me to attend EuroBSDcon 2013 as well as the associated FreeBSD Developer Summit, which both would have been unlikely undertakings without their generous support.

As with other BSD-related conferences, getting there turned out to be interesting as Ed Schouten had his seat next to mine across the aisle on that plane. Moreover, Ilya Bakulin and three additional OpenBSD folks working at genua GmbH that I know from other events also were on that flight. However, given that EuroBSDcon 2013 was not my first such conference, the list of usual suspects I have not met before gets considerably shorter over time. Nevertheless, I still managed to make contact with some nice people for the first time: Lars Engels, Andriy Gapon, Jeremie Le Hen, Navdeep Parhar, Colin Percival and Gleb
Smirnoff.

Things I achieved during EuroBSDcon 2013 and the associated FreeBSD Developer Summit include:
  • discussing the sparc64-specific aspects of the GET_STACK_USAGE macro with Alexander Motin (committed as r255937, later on merged verbatim to powerpc in r256007 by Nathan Whitehorn)
  • discussing problems with the current powerpc and sparc64 implementations of the counter(9) API with Andriy Gapon and Gleb Smirnoff (unfortunately, no good and obvious solution was found and I have to benchmark some possible approaches first)
  • getting a private status update about a GPL-free toolchain in NetBSD by Joerg Sonnenberger
  • getting my shiny new PGP key signed by Dag-Erling Smørgrav in his role as FreeBSD Security Officer, as well as by some others
  • talking to Colin Percival about getting freebsd-update(8) bits for powerpc and sparc64 in place now that we also and already have switched to cross-building the corresponding releases beginning with FreeBSD 9.2, which he intends to look over in the next couple of days
  • talking to Baptiste Daroussin about possibilities for improving the situation with ports and packages for sparc64
  • extending my English vocabulary by "wick" with the help of Isabell Long (one torch at the beach the social event took place was missing one)
The session that interested me the most at the FreeBSD Developer Summit was the "Networking" one. It was very informative to get a hold of the various performance issues in our network stack different parties are struggling with and how people intend to fix them. At EuroBSDcon, I probably enjoyed the talk "FreeBSD BHyve Hypervisor hosting Other Systems" by Peter Grehan best. Given that I last had a bit of look at BHyve as part of writing my master's thesis - which also has some FreeBSD/i386 PV on Xen coverage - around April, it was surprising to see what progress BHyve has made since then.

Before and afterwards, I had some nice and interesting chats with Peter, too. As it turned out, he also is concerned with the direction FreeBSD is going with clang/LLVM and the possibility that some - in his case 32-bit powerpc - platform support will be sacrificed along that road. Moreover, as an employee of NetApp he once more confirmed that GPLed toolchains are of no concern for such companies as these bits are only used for developing products but are not part of the latter themselves. Before, Marcel Moolenaar essentially raised the same worries about FreeBSD/ia64 and effectively said the same about GCC and friends being no problem for Juniper at EuroBSDcon 2011. So the question remains: which are the mysterious vendors who we shall nuke all GPLed source code out of the FreeBSD base at any cost for?

Cambridge DevSummit Trip Report: Aleksandr Rybalko

The next trip report is from Aleksandr Rybalko:

I recently returned from an exciting trip to Cambridge, United Kingdom, to attend the FreeBSD Developer Summit. Very warm and old town, and at the same time, one of the most historic academic centers in the world. Great support by the FreeBSD Foundation helps to make it realizable for me. It was a very interesting developers summit, even if some talks were difficult to understand because of English-to-English differences.

The main reason why I attended the DevSummit was to introduce the updated newcons project. Now, newcons works with i915 KMS and with the shiny new Radeon KMS too. Many thanks to Jean-Sébastien Pédron for that.

From the developers' reaction, it seems the most exciting part of newcons is a 2-color FreeBSD logo splash screen. Even though there's a lot of work in the underlying infrastructure, everything else is just letters and more letters on the screen.

While giving my demo, I found one more bug: connecting an SVGA projector produced a kernel panic (which was at least displayed nicely on the screen).

Anyway, the Cambridge Devsummit was a very good event. I met with many interesting people, including Adrian Chadd, my mentor (finally). He told me about PMC problems and points to me every time anyone asks about drivers for MIPS and ARM hardware. With Baptiste Daroussin and Vsevolod Stakhov, we discussed pkgng and ports problems and problems related to ports cross-building. I discussed a bit the lack of supported ARM features (like LPAE) with Andrew Turner.

Spent some free time hacking on Exynos4 based device together with Ruslan Bukin. Exynos4 has an interesting problem, but we still have not found a solution/workaround for it.

From the list of summit sessions I can highlight:
  • lldb - New, and I can say, more correct way to debug things. Thanks a lot to Ed Maste for that great work.
  • Capsicum by Pawel Jakub Dawidek. Exciting tool to limit everything you want.
  • pkgng - And of course I still don't understand how Baptiste got such a complex thing to work.
I enjoyed the devsummit a lot. Environment of the city of Cambridge is best for such activity, so developers are able to concentrate on things they need.

Many thanks to Robert, David, Jonathan, Bjoern and everyone involved in the devsummit organization. And of course, a huge thanks to the FreeBSD Foundation for sponsorship!

Cambridge DevSummit Trip Report: Mariusz Zaborski

The Cambridge DevSummit was held at the University of Cambridge Computer Laboratory Cambridge, UK August 24 - 28, 2013. Mariusz Zaborski writes about his experience:

I'm a GSoC student who was invited to the Cambridge DevSummit. Cambridge University is closely related to my GSoC project because this is the place where Capsicum was born and the initial implementation created by Robert Watson and his team.

We arrived late (around 7pm) on a Friday so we managed to only eat some dinner and go to sleep. We were accommodated in Sidney Sussex College. This is a very impressive College which is more than 400 years old. If you ever go to England, you should visit Cambridge and at least one of its colleges.

The first two days were devoted to the integration of the group. We had lunches and dinners together. Jonathan Anderson took us for a walk around Cambridge and to the Cambridge Museum of Technology. In the museum we saw a lot of steam engines and some very old printing machines.

The lightning talks and working groups started on Monday. We had a short talk by Ed Maste and Robert Watson about the FreeBSD Foundation. They told us how the Foundation works and its plans for the near future.

Next, David Drysdale, Pawel Jakub Dawidek and myself had a working group about capabilities and Capsicum in FreeBSD. Pawel talked about some new features and the problems he faced during the project.

After that, I attended the security discussion about /dev/random which was conducted by Mark Murray. I must admit that it was my favorite session. Mark told us how /dev/random works now and what he would like to change in the /dev/random algorithms. People attending this session proposed very useful ideas. That was a very educative and substantive discussion. If you are interested in learning more about this session, I recommend this link.

On Tuesday, we had a few more sessions. I again attended to the security discussion where we talked a lot about security in pkgng. After that we had some talklets. My favorite talklet was about lldb and about X86 Binary Code Analysis. The first presentation was given by Ed Maste. He has convinced me to give the lldb a try as a replacement for good old gdb. Now I think this is very promising project and I would like to examine it more closely. The presentation given by Warren Hunt was very very impressive. He told us about formal proofs in computer science on binary code level. After few talklets we had Pawel's presentation about Capsicum and Casper. He presented his last few months of work in this project. He also mentioned some of my work.

On Tuesday evening we had an official English dinner in Christ's College and this was a very exciting experience. After the dinner, we also managed to see old Darwin's office.

On Wednesday there were some more sessions, but unfortunately I had to return home. The DevSummit was a very intensive five days. I met a lot of very interesting people from all around the world and I came to home even more motivated and with a lot of new experiences.

I want to thank the the FreeBSD Foundation for sponsoring my trip to the Cambridge DevSummit, and my GSoC mentor Pawel for inviting me to it.

OpenZFS Project

The OpenZFS project was announced today. According to Justin Gibbs of the FreeBSD Foundation: this is a cross-platform effort to ensure the continued evolution of the ZFS file system. For developers and users of FreeBSD, the formation of OpenZFS clarifies the future of ZFS support for our platform.  The FreeBSD project is now an equal partner in defining the course for ZFS. OpenZFS combines the man power of the FreeBSD, Illumos, Linux, and MacOS communities to provide a level of test coverage, feature development, documenation, and support that wasn't possible with our separate efforts.  Most importantly, OpenZFS will improve platform interoperability and reduce fragmentation of ZFS implementations. Today is an exciting day for ZFS and the FreeBSD platform.  I encourage you to browse http://www.open-zfs.org and to get involved. You are officially invited to help make the future of OpenZFS!

I had a chance to listen to Matt Ahrens, co-founder the ZFS project at Sun, describe the OpenZFS projects goals during his session at LinuxCon. One of the interesting resources provided by OpenZFS will be Office Hours, a regular opportunity to chat live with a ZFS expert. The first Office Hours will be held the week of October 7 and the expert will be Matt Ahrens.

For those attending EuroBSDCon, Matt Ahrens will co-present with Martin Matuska "Open ZFS: Upcoming Features and Performance Enhancements with Illumos and FreeBSD joining Forces".

For those in the San Francisco Bay area, there will be an OpenZFS Day on November 18 and 19.

Glen Barber Interview in Today’s Episode of BSD Now

Today's episode of BSD Now features an interview with Glen Barber from the FreeBSD release engineering team. Glen will discuss how FreeBSD release engineering works and what his role is on the release engineering team, the new features in the upcoming 9.2-RELEASE, and his new role with the FreeBSD Foundation.

This episode can be streamed live at 2:00PM EDT, 18:00 UTC. Afterwards, the episode will be archived on the same web page.

Foundation at OLF

Several members of the Foundation will be attending Ohio LinuxFest, to be held in Columbus, OH from September 13-15. We're excited that Kirk McKusick will be delivering the closing keynote on Saturday, September 14. The topic of the keynote will be Building an Running an Open-Source Community: The FreeBSD Project.

There will also be a FreeBSD booth in the expo area on Saturday, September 14. Donations to the Foundation will be accepted at the booth. If you're at the conference, drop by and say hi!

Registration is required for this event, but registration is free.

Semi-Annual Foundation Newsletter

The Foundation has published their 2013 Semi-Annual Newsletter. This edition includes:

Foundation at FOSSCON

FOSSCON will be held at the Friends Center in Philadelphia, PA on Saturday, August 10 from 8:00-18:00. Registration is free for this event.

There will be a BSD booth in the Expo area. We will be giving out PC-BSD DVDs and other cool swag, as well as accepting donations to the FreeBSD Foundation.

 Dru Lavigne will present "Introducing FreeNAS 9.1" at 11:35 and Dan Langille will present "Bacula: The Networked Backup Open Source Solution" at 13:35.

Foundation at Indiana LinuxFest

Indiana LinuxFest will be held at the Wyndham Indianapolis West Hotel in Indianapolis, IN from Friday July 26 to Sunday July 28. Registration is free for this event.

There will be a FreeBSD booth in the Expo area on Saturday, July 27 from 8:00-18:30. Stop by to say hi, discuss the Foundation's work, or to donate to the FreeBSD Foundation.

Accepting Travel Grant Applications for EuroBSDCon 2013

Calling all FreeBSD developers needing assistance with travel expenses to EuroBSDCon 2013.

The FreeBSD Foundation will be providing a limited number of travel grants to individuals requesting assistance. Please fill out and submit  the Travel Grant Request Application by August 19th, 2013 to apply for this grant. Instructions can be found in the announcement.

This program is open to FreeBSD developers, kernel hackers,  documentation authors, bugbusters, and system administrators, etc.  In some  cases, we are also able to fund non-developers, such as active community members and FreeBSD advocates.

If you are a speaker at the conference, we expect the conference to cover your travel costs, and will most likely not approve your direct request to us.

The travel grant program is one of the most effective ways we can spend money to help support the FreeBSD Project, as it helps developers get together in the same place at the same time, and helps advertise and advocate FreeBSD in the larger community.

OpenHelp Trip Report: Warren Block

Open Help is an annual conference for people who write documentation for an open source project. The Foundation sponsored Warren Block from the FreeBSD documentation team to attend this year. He writes:

In June, I attended the Open Help Conference in Cincinnati, Ohio.  Open Help focuses on open source documentation.  Dru Lavigne attended also, and Tom Rhodes arrived for a doc sprint.

This year's attendance was larger than last year, and the changing group dynamics are always interesting.  There was a sign-in session on Friday night at a restaurant in downtown, on Fountain square.

The conference began in earnest on Saturday.  There were some very tantalizing presentations.

Jorge Castro, of Red Hat, spoke about their use of StackExchange for user support.  A key point was that forums and mailing lists are good for discussion, but not very good for user support.  StackExchange is good for user support because discussion is very limited.  The description of how this had improved user support for Red Hat was very interesting.  One problem with using this system for FreeBSD is that it is proprietary.  The StackExchange system is "software as a service", and the software is not open.  For that reason alone, it probably would not fly for FreeBSD.  Still, we could use many of the concepts, like the idea that outdated docs in search results can be worse than no results at all.  That idea was restated by several presenters and attendees.  It comes up a lot.

We have encountered it in FreeBSD, too.  One possible approach I have tried to combat this was about an outdated setting for X11 called AllowEmptyInput.  No amount of corrections seemed to make any difference... until I tried pulling instead of pushing.  Rather than just keep trying to correct the information, I decided I would make it famous.  So I wrote a small article that talked about the problem, the solution, and then ridiculed it.   Surprisingly, that
worked.  However, it does not scale well for all the bad advice out there.

So StackExchange sounds great, but would probably not work for FreeBSD. The very next talk was by Michael Verdi of Mozilla, about the Sumo Project, their homegrown support system.  It shared some of the features of StackExchange, but their Kitsune software is open, and built in Django.  This is something we really should test.  First we need to talk someone into creating a port, but that is a technicality.

Richard Bowen, also of Red Hat, made several interesting points in his presentation, "Listening to your Audience".  One of them was that regardless of your project, your default documentation is StackOverflow. I'm not sure that is true for FreeBSD, but it's an interesting concept. Another idea is that of finding how users got to your documents.  Did they find what they needed?  Or will they go elsewhere?

We also had group discussions and short presentations throughout the day.  Open Help is very informal and freewheeling.

On Sunday, more full presentations were given.  Again, they had areas which directly addressed problems the FreeBSD Documentation Project has encountered.  Lee Hunter spoke about using Drupal for technical communications, and component content management systems.  The idea of being able to write a technical section once, use and reuse it throughout documentation, and being able to revise it in one place really resonated.  For example, we have multiple sections in the Handbook that talk about partitioning hard drives.  They are all different, all incomplete, and difficult to update.  Imagine having one section that is simply included as an object in different parts of the documentation, yet is still only a single piece to maintain.

Jorge Castro gave a quick presentation on discourse.org, a site from one of the StackExchange founders.  It is aimed at reinventing the web forum with some very interesting concepts.

Siko Bouterse and Jake Orlowitz of Wikipedia spoke about their Teahouse project, trying to put the humanity back in technical support.  Their project is kind of the antithesis of StackExchange.  Rather than all-business user support with nothing else, it's a conversational, friendly tone to which moderators must agree beforehand.  Participation by women in this new style of support group is about ten times that of the normal forums, a stunning success.

Peter Coombe, also of Wikipedia, spoke about fixing Wikipedia help pages.  This was yet another presentation that dealt with topics the FreeBSD Documentation Project has encountered.  Help pages become long and complicated, addressing unusual situations that most users will not experience.  It can drive off the people looking for simple help.  I find our Handbook chapters on setting up X11, printing, and wireless networking in that situation.  There is lots of information, but 90% of users are looking for a common answer which can get lost in all the details.  We need to address this with simple quick-start introductions, answering the most common, simple questions first.

Also addressed in this talk was the concept of "sunk costs".  We have seen this, where even though some document is long obsolete, people are reluctant to retire it because of the effort that was put into creating it.  That can be addressed, but it takes understanding.  I think it will help to move old documents into an archive rather than just deleting them.  They will still be available, but also will not be confused with current documents.

Janet Swisher, of Mozilla, spoke about doing doc sprints and book sprints.  Dru has hosted more than a few FreeBSD doc sprints, and they do help.  I feel that in-person attendance gives the best results, and that is difficult for us because most contributors are not employed to work on FreeBSD.

Shaun McCance, organizer of the conference and Gnome documentation team leader, spoke about translations and showed his itstool (in ports as textproc/itstool) for extracting translatable content from XML files, then merging it back in after translation.  Gnome is using this to add subtitles and translated subtitles to help video.

We need to revamp the way we do translations for FreeBSD documentation. Translators already have a difficult task of translating very technical content into a second language--and sometimes a third.  They get no programmatic help to accomplish this, it's all manual.  Gnome has translations into 80 languages.  We do not.  If we can make it easier for translators, we can get more translations, faster, and into new languages.  This really needs to be a high priority, and we need to get a group of translators together to consider newer approaches.

I gave a small presentation about FreeBSD and what types of documentation we produce, and how.  We have some famous successes, like the Handbook and our man pages.  We also have many opportunities for improvement.  Some of those opportunities seem simple when viewed from the outside.  In many cases, difficulties arise due to the sheer length of time that FreeBSD has existed.  FreeBSD predates XML, for instance. But we can improve our documentation, and we can make the process of creating and translating it easier.

On Monday, Dru led a FreeBSD doc sprint.  And she did lead, we did a lot of things that needed to be accomplished to prepare for the upcoming print version of the Handbook.  Tom Rhodes also arrived on Monday, and between the three of us, we finished off more than could be reasonably expected of five.

Monday night, there was a bourbon tasting event, which was unusual and surprisingly educational.

And with that, Open Help 2013 came to a close.  A fun event in a great town with lots of ideas, concrete improvements during the doc sprint, and directions for future improvements to FreeBSD documentation and FreeBSD as a whole.

I want to thank the FreeBSD Foundation for sponsoring my trip to Open Help this year, and Tom Rhodes for his help, and most particularly Dru Lavigne for her help and guidance and refusal to accept excuses.

BSDCan Trip Report: Florian Smeets

The next trip report is from Florian Smeets. He writes:

I arrived in Ottawa on Tuesday afternoon. In the evening I went to the Royal Oak to meet other fellow developers.

Wednesday was the first day of the Devsummit. The day started with a presentation about the FreeBSD.org security incident and about how Netflix is using FreeBSD for their CDN.

In the afternoon I attended the ports session where we saw presentations about features that were added to the ports tree in the last year and also about stuff that people will be working on during the next year.

On Thursday I went to the VM I/O Concurrency and Virtualization sessions. In the first session, we talked about all the performance work that is currently happening that should be in the 10.0 release. In the second session, there was a lot of talk about Bhyve and we saw a demo of Linux running in Bhyve.

In the evening I attended the Vendor summit, which I find quite interesting, as you get a picture of the needs of other people and also what they are working on.

Friday was the first day of the actual conference. After talks about dtrace and git, I attended a talk about benchmarking FreeBSD. One of the slides even showed results I posted some time ago.

The next talk I saw was switching from Linux to FreeBSD, which gave some interesting insights into the differences of the two systems from someone else. Which was quite interesting as I've been working for a Linux based company for the last 6 months.

After the FreeBSD Birth to Death talk, I attended the Mozilla on OpenBSD talk. Landry, the OpenBSD Mozilla maintainer, is an OpenBSD and Mozilla committer. He helped us in committing a lot of our patches upstream. At the social event in the evening, I finally had a chance to meet him in person. We talked for a long time and made plans. Now, 4 weeks after the conference, we have a mozilla-central buildbot running in a Bhyve, so we will be informed about breakages automatically from now on. This will be a huge help, as we can try to fix those kind of things before the releases and fix them with our own patches in the ports tree.

On the last day of the conference I attended a talk about using Puppet to manage FreeBSD and about the FreeBSD.org cluster refit. In the afternoon I didn't attend talks, but took the time and sat down with some other developers to discuss future projects.

As my flight was only leaving on Sunday evening, I went into town with a couple of other developers to do some sightseeing. In the evening, the long journey back home started and an eventful and exhaustive week came to an end.