The next version of the Lumina Desktop Environment is now available! This version includes a significant number of updates, particularly to the main desktop session/interface, so I highly recommend that you update to the new version as soon as possible. While the full list of changes is posted at the bottom of the announcement, there are a few that I want to highlight here:
- The user button has received a significant speed boost, and can now be used for full browsing of the user’s home directory (files and directories).
- Desktop icons have received a large number of changes in styling, amount of visible text, and functionality. There is also a new feature to automatically generate plugins for items in the user’s Desktop directory – where each plugin may be individually moved/changed (not trapped within a container like the “desktopview” plugin).
- A new desktop plugin for monitoring the system hardware status (memory/CPU usage, CPU temperature, disk I/O). This functionality requires support for your particular OS, and is currently only available for: PC-BSD, FreeBSD, and Debian
- Lumina has now been fully translated to German, Russian, and Spanish, and almost-completely translated to Catalan (89%), Chinese (61%), Estonian (53%), Indonesian (76%), Polish (89%), Portuguese (89%), Portuguese-Brazilian (89%), Swedish (91%), and Turkish (88%). Thank you to all our translators for all your hard work!!
- To install the translation files on PC-BSD/FreeBSD, you will need to install the (new) x11/lumina-i18n port/package. These translations files are located in the lumina-i18n repository on GitHub, if you wish to package up the translations for your particular OS/distribution as well. Details about how to install those translation files are listed in the repository information.
- To contribute to the translation effort, you may create an account on the PC-BSD translation website and get started today!
- PC-BSD users currently running the EDGE package repo will be able to update their packages via the updater GUI or “pc-updatemanager” utility within the next couple days. Updates for users on the PC-BSD 10.1.2 / PRODUCTION repo will be available once 10.2-RELEASE is available later this year.
- FreeBSD users may now update Lumina directly from the FreeBSD ports tree (x11/lumina), or wait until the FreeBSD package repository is updated with the latest changes before updating with pkg.
- For other Linux/BSD users, please contact the packaging team for your distribution/OS to determine the availability of pre-built packages.
- The Lumina-DE source repository is available on GitHub, and contains a detailed list of instructions on how to build/install Lumina on various types of systems.
- A static archive of the sources for this release may also be downloaded directly from GitHub, to aid in the creation/distribution of pre-built packages for your particular OS.
- Found a bug in Lumina 0.8.5? Please report it (with as much detail as possible) to our bugs database. https://bugs.pcbsd.org
- The new system for desktop plugin settings requires that any desktop plugins will be reset back to defaults on upgrade to this version of Lumina.
- There is a known bug/conflict between Qt 5.4+ and Fluxbox 1.3.7 which results in the “close” button on unlocked desktop plugins having no effect when clicked. To work around this issue, you may right-click on the title for the plugin and select the “close” option from the menu to remove the desktop plugin. Alternatively, you may also remove desktop plugins from the Lumina configuration utility (lumina-config).
- IRC: #pcbsd and #Lumina-DE channels on Freenode
- Email: The PC-BSD mailing lists
- GitHub: The Lumina-DE source repository
Full list of changes since version 0.8.4:
- Desktop Icons:
- Always support two lines of text, and uniform icon sizes
- Allow the ability to remove files/dirs from ~/Desktop through the desktopview/applauncher plugins (as well as increase/decrease icon sizes).
- Add a new option to automatically generate plugins for each file/dir in the users Desktop directory. This can easily be turned on/off in lumina-config.
- Completely re-work how desktop plugins save their settings. Now it is extremely fast and much more reliable. NOTE: This change is NOT backwards compatible. On upgrade to this version of Lumina, all desktop plugins will have their settings reset to defaults.
- Completely re-work how desktop plugins are initially placed/sized. This ensures consistency/reliability
- New Plugins:
- System Monitor (Desktop): This will show the current CPU/Memory/Temperature of the system, as well as list the current disk I/O (if supported by the current OS).
- Quick Plugin (Desktop): This allows the user to load arbitrary QtQuick/QML scripts and display them within containers on the desktop. These scripts need to be located in “~/.lumina/quickplugins/quick-*.qml” or “<Lumina share directory>/quickplugins/quick-*.qml” (other .qml files may be in those directory for additional widget/scripting support, but are not considered full plugins without the “quick-” prefix on the filename). See the “quick-sample.qml” plugin for an example of how to provide additional information about your script/plugin when it is automatically made available in lumina-config.
- XDG Standards support
- Convert to the XDG autostart specifications completely for reading/setting/disabling auto-started applications. Previous settings will automatically be converted the new format on update to 0.8.5
- Assign internal “unknown/<filename/extension>” mimetypes to files that do not have a mimetype in the system database (so default applications may still be registered/used).
- Ensure that “Lumina” instead of “LUMINA” is used for XDG system registrations (where applicable).
- Session Startup Procedures:
- Add a small splashscreen showing the progress/stage of Lumina during the session initialization.
- Add 1/4 second, non-blocking delay between auto-started application startups (to prevent overloading the system all at once).
- Proceed immediately to the application autostart routines after session initialization (remove previous two-second delay).
- Lumina Configuration Utility:
- Change a number of options so that nothing is changed on the system until the “save” button is clicked (for uniformity).
- Add a new widget for managing panels – allowing up to 12 panels per screen to be setup in lumina-config.
- Add full list/add/remove support for desktop plugins (with multi-selection removals).
- Significantly updated themes, and some updates to color schemes as well.
- Add support for solid-color backgrounds
- Significantly speed up the loading of the userbutton on click, and add full home dir browsing/launching/favoriting support (files/dirs)
- Additional error detection for mis-configured luminaDesktop.conf settings.
- Make sure to put a line break between the time/date display if both are selected to be shown on the clock plugin.
- Additional checks/updates to the system tray routines to ensure that nothing gets “missed” on startup.
- Clean up some handling of “wine” applications in lumina-open.
- Add binaries in the lumina-search output for applications.
- Add thumbnail support to a number of plugins (userbutton, desktopbar, etc)
- Fix a number of issues in various places regarding uniform icons sizing/spacing.
- Add full desktop re-scaling support if switching to a new monitor/resolution between sessions.
- Force Fluxbox to completely restart when a monitor is added/removed in the middle of a session (bypassing a couple bugs in Fluxbox).
- Modify the luminaDesktop.conf file syntax a bit so it can be read/parsed by libUCL. The internal parsing routine does have backwards compatibility for the older syntax though.
- Bugfix for the favorites system: ensure the favorites directory exists and create it if not.
- Fix the initial brightness detection for new FreeBSD users.
- Clean up some case-sensitivity issues when parsing luminaDesktop.conf
- Remove some built-in Qt context menus on various toolbars.
- Adjust the initial window geometry check/fix routine to work better.
- Adjust a couple default key bindings (new users only):
- Pause key -> Lock screen (new binding)
- Tile windows left/right -> Alt+[left/right] instead of Ctrl+[left/right] (had conflict with existing Fluxbox shortcut).
FreeBSD-9 introduced basic NUMA awareness in the physical allocator (sys/vm/vm_phys.c.) It implemented first-touch page allocation, and then fell back to searching through the domains, round-robin style. It wasn't perfect, for some workloads it was apparently okay. But it had some shortcomings - it wasn't configurable, UMA and other subsystems didn't know about NUMA domains, and the scheduler really didn't know about NUMA domains. So I'm sure there are plenty of workloads which it didn't work for.
That was all ripped out before FreeBSD-10. FreeBSD-10 NUMA just implements round-robin physical page allocation. It still tracks the per-domain physical memory regions, but it doesn't do any kind of NUMA aware allocation. From what I can gather, it was removed until something 'better' would land.
However, nothing (yet) has landed. So I decided I'd take a look into it. I found that for a lot of simple workloads (ie, where you're doing lots of anonymous memory allocation - eg, you're doing math crunching) the FreeBSD-9 model works fine. It's also a perfectly good starting point for experimenting.
So all my NUMA work in -HEAD does is provide an API to exactly the above. It doesn't teach the kernel APIs about domain aware allocations - there's currently no way to ask for memory from a specific domain when calling UMA, or contigmalloc, etc. The scheduler doesn't know about NUMA, so threads/processes will migrate off-socket very quickly unless you explicitly limit things. Devices don't yet do NUMA local work - the ACPI code is in there to enumerate which NUMA domain they're in, but it's not used anywhere just yet.
Then what is it good for?
If you're doing math workloads where you read in data into memory, do a bunch of work, and spit it out - it works fine. If you're running bhyve instances, you can run them using numactl and have them pinned to a local NUMA domain. Those coarse-grained things work fine. You can also change the system default back to round-robin and use first-touch or fixed-domain for specific processes. It's useful for exactly the same subset of tasks as it was in FreeBSD-9, but now it's at least configurable.
So what's next?
Well, my main aim is to get the minimum done so kernel side work is NUMA aware. This includes UMA, contigmalloc, malloc, mbuf allocation and such. It'd be nice to tag VM objects with a domain allocation policy, but that's currently out of scope. I'd also like to plumb in domain configuration into devices and allow devices to allocate memory for different driver threads with different policies.
But the first thing that showed up is that KVA allocation and superpages get in the way of malloc/contigmalloc working. Allocating memory in FreeBSD first allocates KVA space, then back-fills it with pages. As far as malloc/contigmalloc is concerned, KVA is KVA and it finds the first available space in a time-fast way. It then backfills it with physical pages. The superpage reservation bits (sys/vm/vm_reserv.[ch]) join together regions that are contiguous and in the same superpage and turn it into an allocation from the same superpage. These have no idea about NUMA domains. So, if you allocate a 4KiB page via malloc() from domain 0 and then try to allocate a 4KiB page from domain 1, it will likely mess it up:
- First page gets allocated - first KVA, then the underlying 2mb superpage is allocated and a 4k page is returned - from physical memory domain 0;
- Second page gets allocated - first KVA, and if it's adjacent or within the same 2mb superpage as the above allocation, it'll "fake" the page allocation via refcounting and it'll really be that same underlying superpage - but it's from physical memory domain 0.
So, here's 11ish or so year old Adrian. It's the early 90s. I was hiding in my bedroom, trying to make another crystal set out of random parts and scraping away the paint at my windowsill. In walks my Aunty, who introduces her new boyfriend.
"Hi, I'm Julian." he said. That wasn't all that interesting.
"Oh, are you making a crystal set?" .. ok, so that was interesting.
And, that was that. Suddenly, someone role-model-y shows up in my life out of the blue. There I was, an 11 year old who felt very mostly alone most of the time, and someone shows up who I can look up to and think I can relate to. So, I'm a sponge for everything he shows me. Whenever he comes over, he has some new story to tell, some new thing to show me. He would show me better ways of building transistor switch circuits when I was in the "make large arcs with car alternator" phase of my early teens. And, when I saved up and bought a PC, he started to show me programming.
Now, I was already programming. My parents had saved up and bought me an Amstrad CPC464. We had a second-hand commodore 64 for a short while, but that eventually somehow stopped working and I didn't have the clue to fix it. But I was programming Locomotive BASIC and dabbling in Z80 assembly when I was 12, and had "upgraded" to Turbo Pascal 6 when I hit high school. (Yes, school taught Turbo Pascal at Grade 10 level, and I decided to learn it a bit earlier. That's .. wow, that dates me.) I hadn't yet really stumbled into C yet. I had heard about it, but I didn't have anything that could write it.
Julian explained task switching to me one day during a walk along the beach. He explained that computers can just appear to be doing multiple things at once - but the CPU only does one thing at a time, and you can just switch things really quickly to give the appearance that it's multitasking. With that bright spark planted in my head, I went home and started dreaming up ways to make my Z80 based CPC do something like this.
My mother dragged me to McDonalds to apply for a job the moment I was legally able to (14 years, 9 months) and I saw a computer at a second hand shop - it was a $500 IBM PC/AT, with EGA monitor, two floppy disks and a printer. We put down a down-payment and I paid it off myself with my minimum wage money. Once I had that home I quickly erm, "acquired" a copy of Turbo Pascal for home and was off drawing funny little fractals.
So yes - it's Julian's fault I discovered FreeBSD. Yes, this is Julian Elischer. One day he showed me his computer, running something called BSD. He was trying to explain bourne shell scripting and the installer. I nodded, very confused, and eventually went back to the VGA programming book he lent me. He also showed me fractint running in X on his monochome 486 DX2-50 laptop. I had no idea what was going on under the scenes, only that the fractals were much more interesting than the ones I was drawing. So I took the VGA book home and started learning how to use the higher resolutions available. One thing stuck in my mind: so much bit-plane work. Ugh. One other thing stuck in my mind - reading from VGA memory is one of the slowest things you can do. Don't do it. Ever. (Do you hear that console driver authors? Don't do it. It's bad.)
One day he explained pointers to me. I had erm, "acquired" a copy of Turbo C 2.0 from a friend after failing to make much traction with the less friendly versions (Tiny C, for example.) I had coded up a few things, but I didn't really "get" it. So he sat me down with a pen and paper, and drew diagrams to explain what was going on. I remember that lightbulb going off in the back of my mind, as I dimly connected the whole idea of types and sizes together - and that was it. I was off and doing bad things to C code.
I eventually saved up enough for an updated 286 motherboard, then an updated graphics card (full VGA!), then a sound blaster card, and finally a 486-DX33 motherboard. He introduced me to his friend Peter (who had, and I believe still has, a rather extensive electronics collection) and handed me a FreeBSD-1.1 CDROM. I took it home, put it in, and .. it didn't do anything. My 486 had a soundblaster pro + CD-ROM, and .. well, FreeBSD-1.1 didn't speak to that hardware. So, I eventually put Slackware Linux 3.0 on the thing, and became a Linux nerd for a bit.
I did eventually try FreeBSD-1.1 on it - after putting a lot of FreeBSD bits on a lot of floppies - but I couldn't figure out what to do when it booted. This is going to sound silly - but the lack of colorls turned me off. I know, it seems silly now, but that's honestly why I went back to Slackware.
I eventually went back to FreeBSD in the 2.x era once I had an IDE CDROM and I was working part time at an ISP after (high) school finished. Yes, I figured out how to get colorls to work, I got in trouble disagreeing with a Michael (O, not M) at iiNet about Squid on Linux versus FreeBSD, and well.. stuff. Here was this 17yo kid disagreeing with things and acting like he knew everything. I'm sure it was endearing.
Fast-forward a couple years, and I had been hacking on FreeBSD here and there. I got in a little erm, "trouble" before I finished high school, which phk reminded me of - when they granted me a commit bit. I forget when this was, but I wouldn't have been much older than 20.
So - this is why mentoring kids is important. It may seem like a waste of time; it may seem like they don't understand, but we were all there once. We wanted someone to relate to, someone to look up to, and something interesting to do. Julian was that person for me, and I owe both him and my mother (of course) pretty much everything about my existence in this silly little computer industry.
(This is also why you don't skimp on hardware support for popular, if cheaper platforms and "shiny" looking features if you want people to adopt your stuff - but that's a different rant.)
Ok, that's done. I'm going back to hacking on VGA/VESA boot loader support for FreeBSD-HEAD. That's long overdue, and I want my pretty splash screen.
Installation images are available for the amd64, armv6, i386, ia64, powerpc, powerpc64, and sparc64 architectures.
Additionally, FreeBSD 10.2-BETA1 is available on several third-party hosting providers.
See the PGP-signed announcement email for installation image checksums and more information.
Just quick (and overdue) update: HDMI for BBB is in HEAD as of r284534.
I came in on 12th June, the first day of the scheduled talks, after a grueling finals week at grad school, and was very sleep deprived, but the excitement and the energy at the conference kept me wide awake. I found each of the nine talks I attended very interesting. It was incredible listening to THE Steven Bourne talk about the creation of the Bourne shell at the plenary, and his many insights and inside stories to do with it. I especially loved the part where he explained how he convinced Dennis Ritchie to inculcate the ‘void’ type into C and his candor about the debugging of shell scripts.
I especially enjoyed the following talks, among the ones I attended:
FreeBSD Operations at LimeLight Networks Content Delivery Networks (CDN) are something that interests me, having had a research component on it at graduate school this year, and to see it in the operations perspective, with their usage of FreeBSD was very interesting. They deploy their own backbone and use FreeBSD on their backbone. For a big CDN, with many data centers, large scale installations of FreeBSD on the edge is an important factor. Such an operations workload requires fluidity in software and configuration changes and what is interesting is that only a handful of people are involved in the design and operations at Limelight Networks, as opposed to the usual model. The usage of FreeBSD allows both of these as you are pulled into the source tree and can get involved in the operations of the system. While deploying FreeBSD, their strategies involve upstreaming everything, using ports, and creating a src team. Tools used by Limelight were discussed, which I found informative Zabbix for monitoring to ensure API driven configuration management, monitoring in test, dev and QA to form an efficient feedback loop, OpenTSDB for storing time series metrics, SaltStack for configuration management using a declarative style where the system takes action based on policy using an orchestration bus, Vagrant for pushing FreeBSD on the edge servers.
CloudABI by Ed Schouten which is a Unix application binary interface which provides capability based security. It extends the concept of Capsicum, by providing a more compact representation with only about 60 system calls. This can be applied to cloud computing environments, instead of using full system virtualization or virtualized namespaces. CloudABI can be used in a far simpler way, with better performance, without complicated configurations, while providing security by controlling access to sockets, files and other resources. This was interesting to me, because it provides a whole new way of creating environments by building on capability based security to provide isolation and resource access control at the process level rather than complete isolation of environments to provide the level of security required.
An Introduction to the Implementation of ZFS by Kirk McKusick This was an interesting talk for more reasons than the fact that it was Kirk McKusick himself talking about ZFS there was also a fire alarm going off in the middle of the talk, followed by the arrival of fire trucks! I loved this talk for the clarity with which Dr. McKusick explained the ZFS block structure and checkpointing and made simple the rather complex concepts like freeing of filesystem and snapshot blocks in ZFS. The closing session hosted by Dan Langille was a lot of fun, and again the awesomeness of the BSD community was on shining display. I also attended the closing party at the nearby Lowertown Brewery, which was fun and a good opportunity for some networking as well.
Through this conference, it was great to finally meet Gavin Atkinson, who was the administrator for all of us GSOC students at FreeBSD last summer. I also got to converse with Diane Bruce, Ed Schouten, Deb Goodkin, Justin Gibbs, David Chisnall among others and every conversation was refreshingly unique in the perspectives it provided me with about the work done at FreeBSD, specific projects in FreeBSD, technology in general, the codebase, ways to contribute to FreeBSD, history, music and other myriad subjects. Ottawa was an incredible city to visit, and is so steeped in history and culture, and I soaked in the museums, art galleries and beautiful hiking trails it provided.
In terms of the work I’d like to do for FreeBSD in the future, besides wrapping up the testing of the BSNMP IPv6 project, I also learnt more about other projects I could potentially contribute to in the future, like further IPv6 support (userland cleanup, unification of ping and ping6, unification of traceroute and traceroute6), improvements to the Bluetooth module, 802.11 improvements and Space Communication Protocol, after exploring these with the people I spoke to. Even though I landed at Ottawa mentally exhausted, I returned completely excited and inspired, with many plans for future contributions to FreeBSD, and so very grateful to the FreeBSD Foundation for having provided me with this opportunity to attend my first (of many more, hopefully) BSDCan.
I arrived to Ottawa on Tuesday evening, I found the residence, checked in and dropped off my luggage. I had to look for a shop and do some shopping. It was too late for lounging about so I stayed for the rest of the day at the residence.
The first day of the Developer Summit / Tutorials for me started with “FreeBSD Storage for Sysadmin” tutorial by Michael W. Lucas. I experienced the jet lag, but it was stronger than I expected... During the brake I met Maciej Pasternacki from my country. Interestingly, Maciej was my neighbor on the plane from Montreal to Ottawa;) After the tutorial I joined the devsummit. My working group was “Designing Universal Configuration Files for FreeBSD” chaired by Allan Jude. There was a very interesting discussion about dividing system utilities for targets to USLify and not to ucl. I like the idea “show running config” for single compiled config for the entire system. In the end of the first devsummit / tutorial day we moved to the hacker lounge at the University residence, then during dinner I met Zbigniew Bodek, the third person from Poland at BSDCan.
On the second day of the Developer Summit / Tutorials for me started from the next tutorial about DNSSEC, unfortunately I missed Documentation and ARMv8 working groups, but I couldn’t be everywhere. After the Group Photo I joined the OpenZFS working group.
The main conference started on Friday. It was officially opened by Dan Langille, where Deb Goodkin told about the FreeBSD Foundation. The next step was an amazing keynote by Stephen Bourne about the history of Unix and /bin/sh. After the keynote we finished the devsummit track. Then I attended the Public track, where Kylie Lang explained why Microsoft loves FreeBSD and then Warren Block criticized the installers. The last subject was especially interesting for me, because I worked on installers last summer. During the break I was talking with Warren about profiles for installers. The next talk I've seen, was a legendary “A reimplementation of NetBSD using a MicroKernel” by Andy Tanenbaum, I heard very positive opinions about this talk on Sofia last year.
The lunch break was very productive, except the lunch itself, of course. I found Kris Moore and told him about my patch for freebsdupdate. He directed me to Colin Percival and finally with Allan Jude and Nathan Whitehorn supported Xin Li who closed bug 194746 in FreeBSD Bugzilla.
Then there was George Neville-Neil's talk about measuring network performance. After this talk people tried a cake brought by The FreeBSD Foundation, they were celebrating 15th birthday. Other talks I've seen were from the System Administration track, where Joseph Mingrone explained how scientist use FreeBSD in computation.
And the last one of that day the Olivier Cochard-Labbé’s talk entitled “Large Scale Plug & Play x86 Network Appliance Deployment Over Internet” was one of the most interesting ones for me. There was another event of the day after dinner in the hacker lounge I had a meeting with the mentor of my GSoC project – Warner Losh.
The last day of the conference for me started with the second George Neville-Neil's presentation about teaching advanced operating systems. I’m a student and it was very interesting how they do it in Cambridge. The next talk was given by Randi Harper about fighting harassment with open source tools. We really need more non-tech talks on conferences.
After lunch I attended the Daichi Goto’s talk: “FreeBSD for High Density Servers”. Daichi told how people from NEC install thousands of FreeBSD by PXE. The last two talks were about virtualization, the first given by Michael Dexter briefly described virtualization possibilities in FreeBSD, and the other presented by Maciej Pasternacki showed a lightweight virtualization technique based on FreeBSD jails and ZFS. During his talk we had a building evacuation, after which Maciej finished his talk about Jetpack.
I was really surprised by the charity auction that took place in the time of closing session, where Dan gave a great show. I had a great time during the closing party in a brewery in the Market, I’ve tried bacon in maple sauce for the first time.
The last point of the schedule was the Tourist Day, which we started at 10.00 AM, I’ve never seen Ottawa before, so I really enjoyed this short, but very interesting trip. We finished at about 2.00 PM and after dinner I came back and rented a bike for one more trip.
I think that attending the conferences like those is a huge motivation for work for new people. It was a great opportunity to meet people whom I had known only from the Internet. I hope I will be able to participate in devsummits and BSD conferences again in the future.
I arrived in Canada some time before the conference to see a bit of the country, and to shed any sign of jetlag before the start of the developer summit (a lesson I learned from my previous BSDCan trip). At the "Goat BoF" the day before the start of the developer summit I met many familiar faces, and was able to meet some people I knew by name but had never met in person before.
The next day was marked the start of the devsummit. It's hard to find a better place to get things done than in this inspiring atmosphere, single-mindedly focused on FreeBSD. Uninterrupted FreeBSD time is something I usually lack, so I managed to make a progress on a few of my long-standing TODO items, most notably the export of the OpenBSM repository from the FreeBSD Perforce server to git. The issues this presented on previous attempts were still there, but this time I was able to get it done and, after some more work after the conference, the repository has finally moved to GitHub. Another thing I explored was using BHyVe as a provider for Vagrant, a popular tool for creating reproducible development environments.
On the second day of the summit, I participated in the documentation working group. It was great to finally meet some of the most active people in the last few years. We had a productive day with Warren Block as ring leader, discussing topics ranging from translation, over the doc toolchain, to manpages. A special treat was the presence of Ingo Schwarze from the OpenBSD project and maintainer of the mandoc package, as well as Ryan Lortie from GNOME. Having outside experiences definitely benefited the discussion. One of my goals was discussing moving the release documentation from the source repository to the doc repository, to allow us to make corrections after a release. It was a good discussion, and Hiroki Sato explained some issues I had not considered beforehand.
The conference itself was a fantastic. It does not happen every day that you have people like Stephen Bourne, Andrew Tanenbaum and Kirk McKusick under the same roof. There were many great talks, however the most interesting one for me was "Molecular Evolution, Genomic Analysis and FreeBSD", given by Joseph Mingrone. I'm a graduate student in cancer genomics (the analysis of genetic data from cancer patients), which is a closely related field with many of the same issues and tools. Biological and medical research is dominated by Linux, with the problem that scientific researchers usually don't write the best and most portable code. This makes it more difficult to use FreeBSD, as oftentimes one wants to try out software from the latest research paper. Combined with the fact that biological/medical datasets are oftentimes huge, performance really matters, ruling out virtualization. As it turns out, Joseph had the same experience but found that bhyve appears to have low enough overhead to make this feasible. My hope is that we can work together to make FreeBSD a better platform for this kind of research.
In the evenings, I spent time in the hacker and doc lounges where the discussions and the work continued.
All in all, the conference and the devsummit were a great success for me. I learned a lot, and it's always astonishing how seeing the cool stuff other people have been working on motivates me to work on FreeBSD better myself. Thank you to the FreeBSD Foundation for making this trip possible!
By Warren Block: [email protected]
BSDCan, the large BSD conference in Ottawa, Ontario, was held in June this year. Each year, it becomes a bigger event. As in previous years, Andrew Ross was there with the FOSSLC Project, recording the presentations with excellent video and audio quality. As of this writing, some but not all of the recordings have made it to YouTube, and links are included for the ones that are known.
A FreeBSD developer summit is usually held in the two days prior to the conference itself. The summit gives developers a chance to plan projects and present ideas to other developers in person.
I was lucky enough to give the documentation working group's presentation. There is always concern about whether enough people will be interested in any given topic to make the meeting worthwhile. That turned out not only not to be a problem, but we actually had extra people show up. Better yet, they were all enthusiastic and interested.
We talked about revamping our translation system and tuning the documentation toolchain, possibly asking for help from the Foundation to do that.
Another major point was encouraging people to contribute to the documentation, and lowering the barriers to make such contributions easier. Live demonstrations are the computer equivalent of "Hold my beer and watch this!", yet I managed to show how Annotator (http://annotatorjs.org/) could be used to let people review and comment on our documents without having to know any of the complex markup languages. Help is still needed with porting a Storage component for this, please contact me if you are interested.
The FreeBSD wiki is underused, and we talked about ways to allow more people to contribute there without lowering our standards of quality. I actually forgot to mention my idea about that, where new wiki contributions would automatically generate reviews at https://reviews.freebsd.org/. Either a special group of wiki reviewers or any committer could approve them, spreading the load.
Another big issue was the FreeBSD.org website. Almost everyone agrees it needs to be updated. Where they disagree is the is the issue of how. The Foundation has updated their site recently, and might offer help with ours.
We had a total of three hours for the documentation working group, used all of it, and could have used more.
On nights after the developer summit and BSDCan, the FreeBSD documentation group holds an informal session where people can gather and learn about or work on documentation. We usually have a few short presentations and try to provide one-on-one time for attendees to talk with documentation committers. The trick is trying to work with all the people who show up. The doc committers are usually outnumbered, but people are always incredibly patient.
Only a few of the talks will be highlighted here, but go watch all of them online. Really.
The keynote presentation was Stephen Bourne's excellent talk on sh. It gave an inside view of software design within the limitations of available hardware. Along with that were some insights on things that might have been done differently in hindsight, or not at all.
The opening session ran somewhat long, and there was some confusion on schedules for the rest of the day. I gave a second presentation, called "Thinking About Installers: Discord and Happiness" about how a different view of installers could make the job of installing FreeBSD much easier, regardless of the amount of customization needed on the target system. Despite the schedule confusion and this talk not even being listed by name on the program, it was surprisingly well-attended. Some of that was certainly due to Kylie Lang's preceding talk about FreeBSD on Hyper-V and Azure. Still, the attendees were interested, asked good questions, and offered suggestions.
In the afternoon, Ryan Lortie of the Gnome project talked about jhbuild. Gnome has recently included FreeBSD as a supported platform, and this is their method of continuous testing and building. Initially met with not much interest, the interaction and cooperation has grown and benefitted both projects.
Later in the afternoon, Joseph Mingrone showed the use of FreeBSD clusters for genomic analysis. The application of clusters to this high-end scientific project was interesting, and combining it with down-to-earth sysadmin problems made for a unique presentation.
On the second day of talks, Kris Moore talked about shifting the PC-BSD management utilities to web applications. That was interesting, but then it got even more interesting when he described background updates, which make it possible to upgrade a machine transparently while in use. This builds on the concept of ZFS boot environments, and has serious advantages for end-user desktop systems.
Baptiste Daroussin gave a fascinating talk about packaging the base system. This will be a big step forward in being able to install and customize FreeBSD systems quickly and easily. People have been concerned about how this change might affect users, but the audience was very receptive and the talk anticipated and answered those concerns. One of my former mentors, FreeBSD Release Engineer Glen Barber, even spoke one short sentence into a microphone, probably breaking his previous record.
Later that day, Sevan Janiyan (who we met last year at the doc lounge) had a very interesting talk about building pkgsrc applications in unusual environments. pkgsrc is NetBSD's equivalent to ports, and he took that tree and attempted to just get it to build on numerous Unix variants and architectures. Even just trying to build the tree without succeeding caught bugs, often serious ones in the host systems. And that was from just trying to build the tree, not get the applications to run afterwards. Several of the talks this year pointed out how revealing testing could be, even trivial testing. Testing is good. We need more of it.
During this talk, we had our second schedule confusion. A fire alarm went off, and we had to evacuate the build for a short break. It turned out that someone had overheated something in a microwave on one of the higher floors (my guess is microwave popcorn, the bane of office workers everywhere). After the fire department left, we resumed the talks.
The closing talks and charity auction were somewhat rushed, but still great. The closing party at the nearby Lowertown Brewery was excellent. These informal events are great for meeting people that were only known by their email addresses previously.
The FreeBSD Foundation
The Foundation always has a table set up during BSDCan. They are too shy and polite to collar people walking by and ask them to help support FreeBSD with a donation. Unless you go to one of their talks, you might not even know that individual contributors are needed to show the auditors that the Foundation is community-supported, and even small donations increase the contributor count. If you see them at a conference, stop and talk to them. Make a small donation, or a large one if you prefer. Many FreeBSD improvements have been possible solely because of the Foundation's support. They continue to grow by adding good people. This year, they announced that Benedict Reuschling, my other mentor, has joined the board.
BSDCan is not what most people expect. It is not a boring computer conference. Sure, there are presentations and talks and the standard conference stuff. But this is a place where BSD nerds are the norm, not the exception; where the other people there speak your language, share and understand your problems, and know that you understand theirs. It is a feeling of family, a chance to share and solve problems, and gather inspiration for the rest of the year.
My thanks to the FreeBSD Foundation for making it possible to attend BSDCan, to Dan Langille for creating and running BSDCan, and to Dru Lavigne for always being there.
We sat down with Benedict to find out more about his background and what brought him to the Foundation. Please take a moment to see what he has to say and join us in welcoming him to the board!
Tell us a little about yourself, and how you got involved with FreeBSD?
During my undergraduate studies in computer science, I tried out many different Linux distributions. One day, I came across a FreeBSD Live-CD called FreeSBIE. I booted it and was intrigued by how quickly one could switch between terminals on the command line, whereas in all previous distros I saw, this had a noticeable delay. My thought was that if it is already faster on a Live-CD, how would it be when I actually install the system? So, I tried it on my desktop at home in a dual boot setup together with the Linux distro that I had been using. I learned more about FreeBSD by devouring the FreeBSD handbook, blog posts and lurking on mailing lists.
After a while, I realized that I had spent more time in the FreeBSD system than in my Linux partition. So, I decided one day to install FreeBSD as my only operating system and it has been with me ever since. Though I was a FreeBSD user now, I still was not interacting with the FreeBSD community.
That changed one day while being bored writing my master thesis. I read a blog post that the FreeBSD German Documentation team was looking for volunteers. I offered to help translate FreeBSD's documentation into german and was welcomed by the only person (and later mentor, jkois@) maintaining it at the time. I learned how to make changes to the documentation set, commit changes and sending emails back and forth to my mentor, who patiently explained everything. I spent a lot of time on it, which is not something I would recommend doing while writing your thesis (kids, don't try this yourselves). However, things worked out well. Not only did I finish my thesis on time and became a committer in the FreeBSD project, but luck would have it that EuroBSDcon was held in Karlsruhe, Germany that year. I took this opportunity to meet the people who I had only known by their postings on mailing lists. So I went to my first BSD conference and met a lot of nice BSD people there. That was a great experience and I became even more involved in the BSD community. It's been an exciting time and I look forward to what the future brings for FreeBSD. Helping to shape that future through my involvement in the FreeBSD Foundation is equally exciting.
Why are you passionate about serving on the FreeBSD Foundation Board?
The FreeBSD Foundation is a group of dedicated individuals with different backgrounds working together to serve the FreeBSD community. Their diverse set of skills and personalities are, in my opinion, a direct reflection of what the FreeBSD Project is as a whole. Everyone has something to contribute and the community is welcoming and helpful. The FreeBSD Foundation has enabled great things in the past and I hope my contributions as a board member will help continue and increase them further. The most exciting thing for me is that all members of the FreeBSD Foundation are friendly and open-minded people who listen to the problems in the community and look for ways to help. Standing on the shoulders of giants is a phrase that is often used in open source circles. There are a number of ways to get there, but in my opinion the most effective way is when the giant reaches down and helps you get up there. Enabling more people to contribute to the Project is one thing I'd like to help with in my role as member of the Board.
What excited you about our work?
During many conferences, I saw first hand how the Foundation has made it possible for people involved in FreeBSD to meet in person that would otherwise not have had that chance. For example, by providing travel grants to individuals who would not be able to afford going to BSD conferences. Although a lot of interaction is done via online mailing lists and IRC, nothing beats the face to face communication. Not only to discuss difficult problems that contributors and committers alike are working on, but also hanging out together as a group. I've seen many community members form real friendships. It's also nice to see when mentors and mentees meet for the first time. Once the right people are in the room, things start to happen and change, sometimes dramatically fast. Consensus is reached on finding the right solution to a problem, code is being written, activities are coordinated and new plans are made. This would not have happened if the FreeBSD Foundation had not sponsored that BSD conference financially or provided travel grants. Reading about all of the experiences that conference attendees have had in the trip reports that are published afterwards on the Foundation's blog is very satisfying for me.
The Foundation is also doing work that may seem tedious and boring, but needs doing anyway because they're important. Things like paperwork, financial planning and oversight, community management, legal issues, organizing meetings, and marketing and outreach are crucial to help for the Project to sustain and grow. The work that the Foundation does in dealing with that sort of bureaucracy is freeing developers from doing these tasks themselves and allows them to focus on their own work moving the Project forward.
Plus, the Foundation is spending money on infrastructure for the Project such as buying and setting up new build machines, as well as hiring people to support FreeBSD in areas like release engineering, marketing, and project development. Last, but certainly not least, I'm excited what kind of businesses are donating to the FreeBSD Foundation and what industry contacts they have formed over the years. The Foundation is a good bridge for companies and the Project by not directly influencing the Project's direction, but helping to steer it towards mutually beneficial goals.
What are you hoping to bring to the organization and the community through your new leadership role?
My skills include mentoring people into the project. I also have teaching experience in academia and giving talks at open source conferences. These skills tie into the activities of my fellow board members and will be expanded through me by covering more parts of Europe. Representing the Foundation at European conferences and events, telling people about FreeBSD, its features and community, is something I look forward to in my new role. Another thing I'm interested in is reaching out to new users, regardless of whether they via a company, academia, or individuals.
Many people think that it is difficult to contribute something to an open source project. In my opinion, having a will to learn and motivation are the best factors for anyone to bring something to the table. My mentoring experiences have shown me how easy it is for people to add value once they find something they like doing. I saw many of my mentees take on more responsibilities in the Project over time and learn new things I could not have taught them. By serving on the Board, I hope to enable more people to give something back to FreeBSD and experience the benefits it can have to contribute.
How do you see your background and experience complementing the current board?
My background with computer science and academia, as well as the work I’ve done with documentation is a good addition to the members of the Board who are already engaged in these areas. But these are not my only skills, and over time, new areas of interest will develop. I enjoy learning new things, but I also like teaching the things I know to others. In fact, this is one of the things I like about FreeBSD. With the regular influx of new features, constant learning is something that you want to do, rather than being forced to. Hence, I look forward to learning as much from my fellow Foundation members as I do to sharing with them my ideas, viewpoints and motivation.
The PC-BSD project is pleased to announce the availability of two new testing images: 10.2-PRERELEASE and 11.0-CURRENTJULY2015.
WARNING: These images are considered “bleeding-edge” and should be treated as such.
The DVD/USB ISO files can now be downloaded from the following URLs:
This is a great way to test features and report bugs well before the release cycle begins for the next major releases.
To report bugs in PC-BSD, use https://bugs.pcbsd.org.
To report FreeBSD / Port / Kernel / World bugs, use https://bugs.freebsd.org/bugzilla/enter_bug.cgi.
To update from 10.1-RELEASE:
# pc-updatemanager chbranch 10.2-RELEASE
# pc-updatemanager chbranch 11.0-CURRENTJULY2015
This process will take a while, downloading new packages / world / kernel for the system. When done you can reboot, and the updater will finish up the update process.
After a relatively long flight I finally arrived to Ottawa airport. On the spot I started noticing familiar faces and BSD logos here and there (we had plenty of time to stare while waiting in a huge line to immigration). BTW. Don’t ever forget your FreeBSD T-shirt and/or cap when attending BSD conference. They make you glow in the dark for other BSD-geeks so if you don’t have any - buy one.
The Developer Summit started on Wednesday morning with an interesting presentation by Nathan Dautenhahn about the nested kernel - just right to set up the “technical conference” mood. My main goal that day was to attend to a working group related to clocks and power domains in FreeBSD and meet up with guys working on ARMv8 project. And so after months of remote cooperation I was able to talk face to face to Andrew Wafaa (from ARM Ltd.), Ed Maste (from The FreeBSD Foundation) and Andrew Turner (from ABT Systems). Most of us went to ‘Clock and Power Domains’ session where we met with (i.a.) Justin Hibbits and John Baldwin - engineers who really knew what they were talking about. During the discussion I got acquainted with the general demands of the contemporary industry for the energy efficient systems, the ideas that ARM Ltd. recently developed for ARM architecture to prevail in these areas and what could we do to make FreeBSD keep up with the upcoming standards. The brainstorm was quite fruitful and some initial plan and goals for future work were established.
During the dinner on the same day I had a chance to have some less official conversations related to Semihalf’s part of the ARM64 support and an opportunity to perform few test runs of the FreeBSD on Cavium’s Thunder-X that I was supposed to present the following day.
The ARMv8 working group was scheduled for the second day of the DevSummit. That day I met with Larry Wikelius from Cavium from whom I got some feedback of Semihalf’s work so far. He also brought two Thunder-X based boards (or should I say beasts) that served as main attraction and photo/selfie spots. The session was lead by Andrew Wafaa and was one of the most populated working groups this year. We discussed the whole spectrum of topics starting with the current ARMv8 port state through problems that we may encounter when scaling to multiple cores, we talked about packages building, QEMU and future work around the power management, virtualization and etc. Semihalf’s presentation of the FreeBSD on Thunder-X was scheduled for the second part of the working group. The Thunder-X server board was located in Semihalf’s lab in Krakow and I was able to connect to it remotely. Thanks to my colleagues in Poland the board was up and running and all the necessary loader and kernel binaries were in place. It is truly a rare view of so many cores, that they barely fit in top(1) window :).
The main conference was held on Friday and Saturday. The opening lecture was given by famous Steve Bourne, the author of sh (the number of attendees was way above the capacity of the auditorium). Of course there were also some presentations on embedded and hacking tracks that drew my attention. Undoubtedly FreeBSD on ARMv8 (presented by Andrew Turner) was high on my list. The interest in the topic was quite high and after Andrew’s lecture we had some more discussions at which I met (i.a.) Julien Grall from Citrix who works on Xen for ARM and is interested in FreeBSD Xen support for ARMv8.
The DevSummit and conference gave me the opportunity to share Semihalf’s work on ARMv8 with the BSD community, exchange experience and gather other people’s feedback.
Software Engineer in Semihalf
During the BSDCan 2015 conference, I have attended the FreeBSD developers summit. I was particularly interested in the track called "Designing Universal Configuration Files for FreeBSD". As I'm the author of the library that was discussed, I did a talk about library internals and discussed some open questions with Jonathan Anderson, David Chisnall and Allan Jude. We have planned some proposal changes, the interaction with libnv and casper and the following integration of UCL into FreeBSD base system. We have also discussed the desired features and Jonathan suggested a reasonable approach to implement missing ones.
Moreover, during the conference I have finished the feature of flexible dependencies in `pkg'. We have discussed this feature among all pkg developers that were also on BSDCan (namely, bapt@, matthew@ and bdrewery@). I've proposed my view of the future packages dependencies that would resolve the vast majority of the current issues with dependencies and upgrades. I'm going to write a detailed report about this feature to the pkg@ mailing list (I was just too busy with other tasks after the conference).
Another question we've discussed was the problem of digital signatures for packages and distributions. We have concluded that moving from RSA to ed25519 algorithm would simplify pkg architecture by avoiding linking to openssl (which is quite complicated for all openssl versions supported).
Further, after Ted Uagnst presentation I have a conversation with him and John-Mark Gourney (jmg@) about digital signatures formats, compatibility with OpenBSD signify tool and packages signing questions including the ways of how to verify signatures of untrusted sources with potentially malleable signing algorithms.
Afterwards, I've talked with Colin Percival (cpersiva@) asking for his comments about streamlined signatures scheme proposed by D.J. Bernstein. He agreed that this scheme might work securely providing a more convenient users' tool for digital signatures verification and creation.
Among other topics, I have discussed cryptography and security with John-Mark Gourney. We have also talked about '/dev/random' and fortuna upcoming patch. We discussed numerous topics about FreeBSD packages and pkg tool in particular with Baptiste Daroussin, Bryan Drewery and Matthew Seaman.
I have also extracted a lot of valuable information from BSDCan topics, namely from 'CloudABI' given by Ed Schouten and 'Protecting FreeBSD with Secure Virtual Architecture' given by John Criswell.
I'd like to thank the FreeBSD Foundation for giving me the possibility to attend the BSDCan 2015!
File info: 14min, 6MB
Ogg Link: https://archive.org/download/bsdtalk254/bsdtalk254.ogg