10.2-PRE-RELEASE and 11.0-CURRENT Images Available for Testing

The PC-BSD project is pleased to announce the availability of two new testing images: 10.2-PRERELEASE and 11.0-CURRENTJULY2015.

WARNING: These images are considered “bleeding-edge” and should be treated as such.

The DVD/USB ISO files can now be downloaded from the following URLs:

http://download.pcbsd.org/iso/10.2-RELEASE/edge/amd64/
http://download.pcbsd.org/iso/11.0-CURRENTJULY2015/amd64/

This is a great way to test features and report bugs well before the release cycle begins for the next major releases.

To report bugs in PC-BSD, use https://bugs.pcbsd.org.

To report FreeBSD / Port / Kernel / World bugs, use https://bugs.freebsd.org/bugzilla/enter_bug.cgi.

To update from 10.1-RELEASE:

# pc-updatemanager chbranch 10.2-RELEASE
or
# pc-updatemanager chbranch 11.0-CURRENTJULY2015

This process will take a while, downloading new packages / world / kernel for the system. When done you can reboot, and the updater will finish up the update process.

BSDCan 2015 Trip Report: Zbigniew Bodek

Thanks to the FreeBSD Foundation and Semihalf I was able to attend FreeBSD DevSummit and BSDCan this year (2015).

After a relatively long flight I finally arrived to Ottawa airport. On the spot I started noticing familiar faces and BSD logos here and there (we had plenty of time to stare while waiting in a huge line to immigration). BTW. Don’t ever forget your FreeBSD T-shirt and/or cap when attending BSD conference. They make you glow in the dark for other BSD-geeks so if you don’t have any - buy one.

The Developer Summit started on Wednesday morning with an interesting presentation by Nathan Dautenhahn about the nested kernel - just right to set up the “technical conference” mood. My main goal that day was to attend to a working group related to clocks and power domains in FreeBSD and meet up with guys working on ARMv8 project. And so after months of remote cooperation I was able to talk face to face to Andrew Wafaa (from ARM Ltd.), Ed Maste (from The FreeBSD Foundation) and Andrew Turner (from ABT Systems). Most of us went to ‘Clock and Power Domains’ session where we met with (i.a.) Justin Hibbits and John Baldwin - engineers who really knew what they were talking about. During the discussion I got acquainted with the general demands of the contemporary industry for the energy efficient systems, the ideas that ARM Ltd. recently developed for ARM architecture to prevail in these areas and what could we do to make FreeBSD keep up with the upcoming standards. The brainstorm was quite fruitful and some initial plan and goals for future work were established.

During the dinner on the same day I had a chance to have some less official conversations related to Semihalf’s part of the ARM64 support and an opportunity to perform few test runs of the FreeBSD on Cavium’s Thunder-X that I was supposed to present the following day.

The ARMv8 working group was scheduled for the second day of the DevSummit. That day I met with Larry Wikelius from Cavium from whom I got some feedback of Semihalf’s work so far. He also brought two Thunder-X based boards (or should I say beasts) that served as main attraction and photo/selfie spots. The session was lead by Andrew Wafaa and was one of the most populated working groups this year. We discussed the whole spectrum of topics starting with the current ARMv8 port state through problems that we may encounter when scaling to multiple cores, we talked about packages building, QEMU and future work around the power management, virtualization and etc. Semihalf’s presentation of the FreeBSD on Thunder-X was scheduled for the second part of the working group. The Thunder-X server board was located in Semihalf’s lab in Krakow and I was able to connect to it remotely. Thanks to my colleagues in Poland the board was up and running and all the necessary loader and kernel binaries were in place. It is truly a rare view of so many cores, that they barely fit in top(1) window :).

The main conference was held on Friday and Saturday. The opening lecture was given by famous Steve Bourne, the author of sh (the number of attendees was way above the capacity of the auditorium). Of course there were also some presentations on embedded and hacking tracks that drew my attention. Undoubtedly FreeBSD on ARMv8 (presented by Andrew Turner) was high on my list. The interest in the topic was quite high and after Andrew’s lecture we had some more discussions at which I met (i.a.) Julien Grall from Citrix who works on Xen for ARM and is interested in FreeBSD Xen support for ARMv8.

The DevSummit and conference gave me the opportunity to share Semihalf’s work on ARMv8 with the BSD community, exchange experience and gather other people’s feedback.

Zbigniew Bodek
Software Engineer in Semihalf

BSDCan 2015 Trip Report: Vsevolod Stakhov

During the BSDCan 2015 conference, I have attended the FreeBSD developers summit. I was particularly interested in the track called "Designing Universal Configuration Files for FreeBSD". As I'm the author of the library that was discussed,  I did a talk about library internals and discussed some open questions with Jonathan Anderson, David Chisnall and Allan Jude. We have planned some proposal changes, the interaction with libnv and casper and the following integration of UCL into FreeBSD base system. We have also discussed the desired features and Jonathan suggested a reasonable approach to implement missing ones.

Moreover, during the conference I have finished the feature of flexible dependencies in `pkg'. We have discussed this feature among all pkg developers that were also on BSDCan (namely, bapt@, matthew@ and bdrewery@). I've proposed my view of the future packages dependencies that would resolve the vast majority of the current issues with dependencies and upgrades. I'm going to write a detailed report about this feature to the pkg@ mailing list (I was just too busy with other tasks after the conference).

Another question we've discussed was the problem of digital signatures for packages and distributions. We have concluded that moving from RSA to ed25519 algorithm would simplify pkg architecture by avoiding linking to openssl (which is quite complicated for all openssl versions supported).

Further, after Ted Uagnst presentation I have a conversation with him and John-Mark Gourney (jmg@) about digital signatures formats, compatibility with OpenBSD signify tool and packages signing questions including the ways of how to verify signatures of untrusted sources with potentially malleable signing algorithms.

Afterwards, I've talked with Colin Percival (cpersiva@) asking for his comments about streamlined signatures scheme proposed by D.J. Bernstein. He agreed that this scheme might work securely providing a more convenient users' tool for digital signatures verification and creation.

Among other topics, I have discussed cryptography and security with John-Mark Gourney. We have also talked about '/dev/random' and fortuna upcoming patch. We discussed numerous topics about FreeBSD packages and pkg tool in particular with Baptiste Daroussin, Bryan Drewery and Matthew Seaman.

I have also extracted a lot of valuable information from BSDCan topics, namely from 'CloudABI' given by Ed Schouten and 'Protecting FreeBSD with Secure Virtual Architecture' given by John Criswell.

I'd like to thank the FreeBSD Foundation for giving me the possibility to attend the BSDCan 2015!

RTL-SDR on FreeBSD, or "hey, cool, I live near an airport, I wonder if ADSB works.."

I bought one of those cheap RTL-SDR units a few months ago. There's no real kernel code required for it - all of the rtl-sdr code just uses the generic USB userland API which is shared between many operating systems.

So, getting it going was pretty easy:

# pkg install rtl-sdr

Then, using it to test ADSB is pretty easy:

# rtl_adsb -V -S 

.. this is verbose and listens to short packets.

Where I live (near San Jose Airport!) I receive a lot of ADSB transmissions. It's quite interesting.

Ok, so next - what about something more GUI like? Someone's already done it - https://github.com/antirez/dump1090 . There's already a package for it:

# pkg install dump1090
# dump1090 --net --aggressive

Then, point a webserver at http://localhost:8080/ and watch!

PC-BSD Documentation can now be Translated Using Pootle

Kris has finished integrating the source files for the PC-BSD Handbook documentation into Pootle, meaning that translators can now use their web browser to translate the Handbook into their native language. As translations are completed, we’ll make sure that the build server generates HTML copies and includes them in /usr/local/share/pcbsd/doc/html (right away for EDGE users and with the next release for PRODUCTION users).

To translate the documentation, go to http://translate.pcbsd.org/translate/, click the “All Projects” drop-down menu, and select “PC-BSD Handbook”. You can then click the link for the language to translate. Currently, German and French are available. If you want to translate to a different language, send an email to the translations mailing list and request that it be added.

When translating the documentation, be aware of the following:

  • At this time, some formatting tags are still displayed in raw text, as seen in the examples in Figures 1 and 2. It is IMPORTANT that you do not accidently remove the formatting as this can break the documentation build for that language. In the first example, it is OK to translate the phrase “Using the Text Installer” but care must be taken to not accidently remove any of the surrounding colons and backticks, or to change the text of the “ref” tag. In the second example, the asterisks are used to bold the word “install”. It is OK to translate “install”, but do not remove the asterisks.

Figure 1: Do Not Remove Formatting Characters

translate6

 

 

 

 

Figure 2: Another Formatting Characters Example

translate7

 

 

 

 

If you have any questions on how to use Pootle, have suggestions on how to streamline the translation process, or find any gotchas when translating, please send them to the translations mailing list so that the developers and other translators are aware of them.

BSDCan 2015 Trip Report: Ahmed Kamal

I come from Cairo, Egypt, and this was my very first BSD conference! Needless to say, it was a blast! It was said many times during the conference, and I believe it to be true, so here it is again. “I’m here because so many people smarter than me are here!”. My flight landed on the 11th, time since I left home was roughly 18 hours, it was a couple of long flights. I seriously should have been tired but the excitement was keeping me awake. I took a walk around the city for a couple of hours to freshen up. Li-Wen Hsu joined me for the walk. We had an interesting discussion about jenkins, documentation, ways to contribute to BSD. We also had an interesting discussion about CJK languages, and their special needs. Most of that was new to me, although arabic script is complex too, and can get quite wild!! Afterwards I went back to the campus, I joined a documentation session. It was a kickoff to how the freebsd handbook and similar documentation were written. I learnt some useful tricks which I know I will probably put to use soon contributing some documentation. At roughly 21:00 my eyes turned red, and body was calling on me to go to sleep. I decided to sleep early and be fresh on day-1 of the conference!

Day one began with excitement .. The plenary was just inspiring! I was made aware of multiple new projects where BSD is helping the world. The idea of building “OS Coursework” for university level studies was quite interesting to me. The opening keynote also included legendary Steven Bourne (of /bin/sh fame!) explaining how it all started. He even showed annotated unix source code printed on paper. His quotes included “Took 2 years to get shell quoting straight, and I'm not sure it's straight now” and about “shell shock” he mentioned he didn’t write that code :) The next session for me was “Embedded FreeBSD Development and Package Building via QEMU”. This is an interesting topic about cross compiling packages for embedded devices using the power of Qemu. Qemu began as a JIT translation engine, later the PC hardware was added and now it can be used to build packages for say ARM platforms. The speaker mentioned the flow should hopefully soon work on Apple hardware.

The next big talk to me, was “A reimplementation of NetBSD using a MicroKernel” by Andrew Tanenbaum. It was quite exciting to watch prof Andrew describe his work on micro-kernels and how he’s rebuilding netbsd-3 through the microkernel approach. The basic idea is to separate IO devices, so that for example a random driver (like an audio driver), is no longer running with unlimited powers on your machine. Such a driver can no longer write to disk as it should(n’t). Also the idea is to Isolate communication, restrict kernel calls on a per kernel component basis. This protection can be enforced by the MMU. Various user-mode servers provide the needed services to applications. This includes VFS, Process manager, Memory manager, .. etc. The Reincarnation server is the parent of all drivers and servers. When a driver or server dies, RS collects it. Checks the kernel config table and restarts it. For improving IPC reliability in the new kernel, fixed length messages are used everywhere to avoid potential buffer overflows. In contrast to modern OSs, “drivers” are basically untrusted code! heavily isolated. Cannot touch kernel data structures. Andy mentioned how Infinite loops are detected, and how the driver is appropriately restarted if hung. Andy later described a fault injection experiment where his team injected 800,000 faults on binary drivers, the sequence was 100 faults injected, wait 1s and the drivers crashed 18,000 times, but never crashed the OS. Kernel reliability “proven” to me! In order to port Minix3 to ARM platform, various things needed to be done like (added uBoot support, rewrote context switching, removed x86 segmentation code, imported netbsd arm headers, ported build.sh for cross toolchain, wrote drivers for sdcard!). Things to do include adding crucial missing system calls, port more pkgs (java, browsers...), get it running on raspi, port rump! Minix-3 is a microkernel reimplementation of netbsd, which was primarily motivated by providing that microkernels can be practical OSs that have lots of applications on top. It has proven that drivers belong in user-mode. Future features Andy’s team is working on include live kernel patching which he described how it should work. Upgrade os and change data structures in a live way, without restarting the running processes! This was one hell of a talk, it was very dense and packed with information

The next talk was “Measure Twice, Code Once. Network Performance Analysis for FreeBSD” by George Neville-Neil. George discussed performance of the network under FreeBSD. This is critical for supporting modern 10G ethernet networks. He introduced the conductor python framework he wrote to perform the testing. He joked about how others always say “Yeah, we have something similar that we’re not yet ready to release”. George showed performance graphs from various systems such as pfsense, freebsd, openbsd pf and Linux’s iptables. Various interesting and yet unanswered questions were revealed such as “Why is pfsense much faster” than FreeBSD on which it is based! What kind of patches went into pfsense to get that effect. Also it was found that iptables offers much better scalability for multi-core performance. Obviously iptables has been tuned for SMP performance, but this necessitates the question of why it is much more scalable. And how the BSDs can improve their packet filters. George mentioned that this is only the beginning. Those questions need a lot of work to be answered.

My next session was “Molecular Evolution, Genomic Analysis and FreeBSD” by Joseph Mingrone. The session was in the sysadmin track, something I can easily relate to. Joseph runs a freebsd based cluster for molecular evolution analysis at his university. Joseph began by giving an introduction to genomics, something way outside my comfort zone. It’s always fun to me whenever I discover new worlds like that :) He moved on to explain the IT infrastructure powering his clusters and how they moved from Solaris to FreeBSD 7.x and now at FreeBSD 10.1. He explained how he had a hard time just installing FreeBSD where sometimes the machines would just “eat” the CDs (yes they’re still in there many years later :) He currently uses Poudriere, enjoys ZFS on the storage server and NFS mounts that on the compute nodes.

The next session was “FreeBSD on ARMv8” by Andrew Turner. It was great see’ing the kind of effort that goes into supporting ARMv8. Everything we sort of take for granted like pagetables, MMU, enabling virtual addressing, calling into C code ...etc was being made to work step by step. Later-on pmap support was added, then it was possible to use a 4MB in kernel filesystem. Afterwards on it was possible to expand this storage area. Now it was time to play with the dynamic linker. After that, it was time to add ThunderX support (bus dma, ITS, …) Future work includes (ACPI, hw-pmc, dtrace, gem5). This was an advanced but certainly fun and educational session.


Day-2 now begins with a very interesting session “CloudABI: Cloud computing meets fine-grained capabilities” by Ed Schouten. The overall goal is to improve Unix security and portability. For example, a web server should only read files and write to network, but in reality it can do a lot more! (run a bitcoin miner, install cronjobs that call binaries in /tmp..etc) .. Problem #2, running a 3rd party application, whether running them directly or thru jails/docker is not really safe, a VM might be acceptable though! Problem 3 is that UNIX programs are hard to reuse and test as a whole. The idea is to extract sockets and file systems from being embedded inside the program to being passed as an argument. This allows better flexibility and testability! Then he discussed capsicum, how an app can call the kernel and tell it to “lock me up”! I no longer need to access new resources. Capsicum is awesome, works well .. however it doesn’t scale! There is no guidance when something doesn’t work. With CloudABI a cloud provider can simply promise to run customers’ applications, pass those applications specific file descriptors and leave the app to do what it wants without touching any other part of the app! Cloudlibc is a C library built on top of the low level API Goal is 90% POSIX compliant! You get compiler errors when using unsupported functions.

After that is a session I was waiting for, “New OpenZFS features supporting remote replication” by Matt Ahrens. I have a soft spot for ZFS :) Matt started by giving an introduction to zfs send and receive and why it rocks! Only parts of the ZFS trees modified “after” the time we are sending from, need to be read! The more interesting part Matt discussed is what are the unique features to OpenZFS. This includes send stream size estimation and monitor its progress! Big receive improvements like receiving file with holes in them. Another cool thing added is bookmarks. This allows free’ing data on the sending side by deleting the relevant snapshot, and really only keep a bookmark to it. Now the really cool part is the new stuff. Resumable send and receive, hell yes! Another cool new feature, is a new checksum has been added per every record. As opposed to the old state of sending one checksum at the end of the stream. If a bad checksum is hit, ZFS receive aborts, and naturally you can resume it later! One last new feature, is receive performance improvements. In a benchmark, Matt said it improved performance by 6x which is great!

The next was “Multipath TCP for FreeBSD” by Nigel Williams. This one got me really excited! Part of that is that I knew very little previously about mptcp before. This got me too excited that I kept googling for more mptcp info all day and after the conference. Nigel explained mptcp design, and how it was done the way it is to work with the how the Internet is, as it is today. That is to work with  current middleboxes and NAT functions. MPTCP handshake piggybacks on top of the 3 way regular handshake. The two sides negotiate mptcp then the server informs client of other addresses it is reachable over and new sub connections are started. To me this is sort of the holy grail of WAN networking. A single TCP connection spanning multiple links for performance and redundancy, wow! I guess we’ll all just need to wait some more till the kernel implementations are more mature.


Next was “Packaging FreeBSD base system” by Baptiste Daroussin. He discussed packaging the base through pkgng and the challenges faced. I’m not really much of a packaging person, so it was good to hear many details that need to be take into account. Next up was “An Introduction to the Implementation of ZFS” by Kirk McKusick. If someone could make you appreciate the complexity of ZFS and operating system concepts it’s McKusick! He contrasted various filesystem features as they existed in UFS, and how ZFS is different. There was sort of a fire incident in the middle of the talk, we had to go to the street and then be back to continue. However overall it was an enjoyable session. The closing talk by “Dan Langille” was so much fun. The stats, the growth in the community, and even the goodbye auction were all something I won’t forget. Thanks to everyone who helped make BSDCan 2015 as fun, informative and enjoyable as it was to me.

Ahmed Kamal

BSDCan 2015 Trip Report: Steven Douglas

With the Foundation's help, I was able to meet and network with new people. Where I live, there are only a handful of people that even know what BSD is, let alone can talk at a high level about it. That was one of my favorite things, being around like minded people. I made many new contacts, most notably Peter Toth. Peter is working on iocage, which is a modern jail management utility with some very impressive features. In speaking with Peter, I met Sean Chittenden from Groupon. Sean is looking to perhaps implement iocage in conjunction with a new orchestration software that he is testing. Other than the social aspect, I also got some much needed help with my GSOC project.

At the conference, there were opportunities to learn every hour of every day. An expert in a field is never more than arm's reach away. It is very helpful to have questions that can be answered by the people who wrote the code. All of the talks I went to were absolutely fantastic, and I can't wait to watch the ones I missed when they are posted. My favorite talks were Steven Bourne's talk about his past, Matt Ahren's talk about code flow between our community and OpenZFS, and Multipath TCP by Nigel Williams. The speaker's all did fantastic jobs, and I hope that I am able to speak in the future.

This was my first BSDCan, and first BSD conference. I enjoyed every minute of it, and it happened so quickly. I hope to be back to BSDCan next year, and hope to make it VBSDCon and EuroBSDCon. Thank you to the Foundation for the financial assistance to help get me there!

Steven Douglas

Conference Videos/Slides

Kris and Ken attended BSDCan 2015 last week and had an amazing conference. The videos of the conference were just put online within the last couple days, so if you were not able to attend, you might want to take a look!

  • BSDCan 2015 Videos from the conference (not all of them were video recorded, but a lot were).
  • The video of Ken’s session about Lumina-DE. There is a bit missing at the beginning of the talk, but not too much was missed. The video is fairly dark, but you can also find all of Ken’s slides on SlideShare.
  • It does not appear that a video was recorded for Kris’s session about package/jail management, but you can find the summary/paper on the conference website linked above.
  • EDIT: There was a video recording of Kris’s session, but it has just not been put up yet. Stay tuned to the BSDCCan playlist to catch it when it gets uploaded. You can also find Kris’s slides from the session here.

At the same time as BSDCan, Joshua Smith attended the SouthEast Linux Fest (SELF) and gave a presentation titled “PC-BSD 10.1.2: Whats New?”.

If you have the opportunity to attend one of these conference in the future I highly recommend going!

OpenStack on FreeBSD/Xen Proof of Concept

In my previos post I described how to run libvirt/libxl on the FreeBSD Xen dom0 host. Today we're going a little further and run OpenStack on top of that.

Screenshot showing the Ubuntu guest running on OpenStack on the FreeBSD host.

Setup Details

I'm running a slightly modified OpenStack stable/kilo version. Everything is deployed on two hosts: controller and compute.

Controller

Controller host is running FreeBSD -CURRENT. It has the following components:

  • MySQL 5.5
  • RabbitMQ 3.5
  • glance
  • keystone through apache httpd 2.4 w/ mod_wsgi

Everything here is installed through FreeBSD ports (except glance and keystone) and don't require any modifications.

For glance I wrote rc.d to have a convenient ways to start it:

(18:19) novel@kloomba:~ %> sudo service glance-api status
glance_api is running as pid 792.
(18:19) novel@kloomba:~ %> sudo service glance-registry status
glance_registry is running as pid 796.
(18:19) novel@kloomba:~ %>

Compute

Compute node is running the following:

  • libvirt from the git repo
  • nova-compute
  • nova-scheduler
  • nova-conductor
  • nova-network

This hosts is running FreeBSD -CURRENT as well. I also wrote some rc.d scripts for nova services except nova-network and nova-compute because I start it by hand and want to see logs right on the screen.

Nova-network is running in the FlatDHCP mode. For Nova I had to implement a FreeBSD version of the linux_net.LinuxNetInterfaceDriver that's responsible for bridge creation and plugging devices into it. It doesn't support vlans at this point though.

Additionally, I have implemented NoopFirewallManager to be used instead linux_net.IptablesManager and modified nova to allow to specify firewall driver to use.

Few more things I modified is fixing network.l3.NullL3 class mismatching interface and modified virt.libvirt to use the 'phy' driver for disks in libvirt domains XML.

And of course I had to disable a few things in nova.conf that obviously not work on FreeBSD.

I hope to put everything together and upload the code on github and create some wiki page documenting the deployment. It's definitely worth to note that things are very very far from being stable. There are some traces here and there, VMs sometimes fail to start, xenlight for some reason could start failing at VMs startup etc etc etc. So if you're looking at it as a production tool, you should definitely forget about it, at this point it's just a thing to hack on.

libvirt/libxl on FreeBSD

Few months ago FreeBSD Xen dom0 support was announced. There's even a guide available how to run it: http://wiki.xen.org/wiki/FreeBSD_Dom0.

I will not duplicate stuff described in that document, just suggest that if you're going to try it, it'd probably be better to use the port emulators/xen instead of compiling stuff manually from the git repo.. I'll just share some bits that probably could save some of your time.

X11 and Xen dom0

I wasn't able to make X11 work under dom0. When I startx with the x11/nvidia-driver enabled in xorg.conf, kernel panics. I tried to use an integrated Intel Haswell video, but it's not supported by x11-drivers/xf86-video-intel. It works with x11-driver/xf86-video-vesa, however, the vesa driver causes system lock up on shutdown that triggers fsck every time on the next boot and it's very annoying. Apparently, this behavior is the same even when not under Xen. I decided to stop wasting my time on trying to fix it and just started using it in a headless mode.

IOMMU

You should really not ignore the IOMMU requirement and check if your CPU supports that. If you boot Xen kernel and you don't have IOMMU support, it will fail to boot and you'll have to perform some boot loader tricks to disable Xen to boot your system (i.e. do unload xen and unset xen_kernel). Just google up your CPU name, e.g. 'i5-4690' and follow the link to ark.intel.com. Make sure that it lists VT-d as supported under the 'Advanced Technologies' section. Also, make sure it's enabled in BIOS as well.

UEFI

At the time of writing (May / June 2015), Xen doesn't work with the UEFI loader.

xl cannot allocate memory

You most likely will have to modify your /etc/login.conf to set memorylocked=unlimited for your login class, otherwise the xl tool will fail with some 'cannot allocate memory' error.

libvirt

It's very good that Xen provides the libxl toolkit. It should have been installed when you installed the emulators/xen port as a dependency. The actual port that installs it is sysutils/xen-tools. As the libvirt Xen driver supports libxl, there's not so much work required to make it work on FreeBSD. I made only a minor change to disable some Linux specific /proc checks inside libvirt to make it work on FreeBSD and pushed that to the 'master' branch of libvirt today.

If you want to test it, you'd need to checkout libvirt source code using git:

git clone git://libvirt.org/libvirt.git

and then run ./bootstrap. It will inform if it needs something that's not installed.

For my libxl test setup I configure libvirt this way:

./configure --without-polkit --with-libxl --without-xen --without-vmware --without-esx --without-bhyve CC=gcc48 CFLAGS=-I/usr/local/include LIBS=-L/usr/local/lib

The only really important part here is the '--with-libxl', other flags are more or less specific to my setup. After configure just run gmake and it should build fine. Now you can install everything and run the libvirtd daemon.

If everything went fine, you should be able to connect to it using:

virsh -c "xen://"

Now we can define some domains. Let's check these two examples:

The first one is for a simple pre-configured FreeBSD guest image. The second one defines CDROM device and hard disk devices. It's set to boot from CDROM to be able to install Linux. Both domains are configured to attach to the default libvirt network on the virbr0 bridge. Additionally, both domains support VNC.

You could get domain VNC display number using the vncdisplay command in virsh and then connect to a VM with your favorite VNC client.

I've been using this setup for a couple of days and it works fine. However, more testers are welcome, if you're using it and have some issues please drop me an email to novel@`uname -s`.org or poke me on twitter.

Gaining space on Android after ART->Dalvik switch (root access required)

I (still) use a Nexus S phone. I am using Cyanogenmod on it. After an article in a computer magazine I decided to give the ART-runtime a try instead of the default Dalvic-runtime. Unfortunately I do not have enough free space free (and all what I can is moved to the USB storage already) to really use the ART-runtime.

After switching back to the Dalvic-runtime, I had only 2/3 of the previously available space free. After a little bit of looking around I found /data/dalvik-cache. I deleted with a file manager the content of the directory (you will get some “app crashed/died” messages) and rebooted the phone (this is not the same as formatting the cache partition in the recovery system).

During boot it populated the directory again and now I have more than 4/3 of free space on the internal storage.

Share/Save

PC-BSD 10.1.2: an Interview with Kris Moore

PC-BSD 10.1.2 has been released, so we thought we’d talk to project lead, Kris Moore, to see what’s in store!

KrisMug2015

Q: What new features and improvements are available in PC-BSD 10.1.2?

This quarterly update got a LOT of new features, partly so we would have time to really play with them before the 10.2 release later this summer.  Most of them are security and privacy focused, but there are some other neat things as well.  In no particular order, these are some of the best ones:

* PersonaCrypt – Our new privacy utility that offers a few cool new features.

In its default mode, it can setup your entire user $HOME directory on a geli-backed external device, such as a fast SSD Flash Stick (Using the 256GB Corsair Flash Voyager GTX here). At the login screen, you will then enter your normal user password, along with a decryption password to mount your home-directory. The GELI key is split in two, so you can “pair” the memory stick with your system, so even if the stick is stolen, and somebody gets the password, it is still worthless without the “paired” system. Another benefit of this technology is that you can bring your work with you when you travel between desktops, or jump from a desktop to laptop. I’m using it now for conferences and being on the road, since all my development work and important data is on my $HOME, meaning I can just unplug and keep working on my laptop without  having to keep files in sync.

Another side of PersonaCrypt is something we call “Stealth” mode, which allows you to do a desktop login with a one-time GELI-key encrypted $HOME directory with no personal data. Think of it as privacy mode  in a web-browser, but for your entire desktop session. This also plays nicely with the following new feature.

* Tor transparent proxy support

10.1.2 includes an easy way to switch between your normal internet connectivity, and “Tor” mode by clicking a single button on the system tray. In Tor mode, the firewall acts as a transparent proxy, forcing all internet traffic to be routed through the Tor network, including DNS requests. All other traffic which may expose your system on the internet is blocked. This goes a step beyond just running Tor as a browser proxy, since you can’t always trust plugins (cough *flash/java*) and other apps to behave properly.

* 4K Monitor support

While most things already worked with 4K monitors, we did go through our toolchain and fixed a number of scaling bugs, mouse cursors and fonts which needed tweaking to look better “out of box”. I’m currently using an nvidia card with DisplayPort, and running in 4K at 60Hz, and yes my eyes thank me every day.

* Encrypted ZFS replication

Our Life-Preserver utility got an update to support doing backups to an encrypted zpool, using GELI and iSCSI. Both the zpool and GELI are initialized on the client side, so that when doing backups, nothing ever goes “over the wire” that isn’t already encrypted. LP also gives the ability to export the connection settings and GELI key to a password-protected GELI file, which means I don’t have to copy the plaintext key to another system for importing the pool / recovery. It also can be used by the install media to do a “bare-metal” restore from a backup, which is handy if your system dies badly. I’m currently using this feature to backup my PersonaCrypt home-directory.

LP was also updated to now do per-dataset replication, allowing us to do recursive replication with excludes, and preparing us for resumable ZFS send/recv in the near future.

* Media Center Support

Our installer now provides support to install Kodi or PlexHomeTheater.  Along with the automatic update feature, it makes PC-BSD great for the 10ft experience. My evil plan going forward is to add Steam Big Picture mode to this list, possibly for 11.

* LibreSSL / OpenNTPD

We switched all our ports to now using LibreSSL instead of OpenSSL, along with moving from NTP -> OpenNTPD. This is done in response to there being so many PR’s against this two particular pieces of code, hopefully reducing the number of PR’s we are vulnerable to.

* AppCafe web-interface

The new AppCafe web-interface is now live, allowing remote management of PBI/pkg, and basic jail support. This means you can run a  PC-BSD/TrueOS box headless, and manage your Applications, Jails, and Updates all via web-browser or mobile device. This is our first release of the web-based system, and we already have plans on adding more features to it, in particular to jail management and support for 3rd party repositories in
jails.

* IPFW Firewall

We’ve also switched our default firewall from PF over to IPFW, with the plan of enabling VIMAGE for 10.2 later this year. Once we ship with VIMAGE enabled, we will then be doing some planned updates to the Warden networking support.

Q: What feature(s) do you think will have the biggest impact for users?

Personally I think PersonaCrypt is just the coolest. Since I travel a bit, it is really nice to be able to bring my entire home development environment and workstation with me. I can actually shut the machine off at home now since all my files and work comes with me.  I can also see how that would be helpful for those who alternate between home and office, not needing to lug around a laptop to work from home.

From a marketing perspective though, we are hearing from a lot of dis-enfranchised Linux users. Over things like ShellShock, Heartbleed, SystemD and the like. While Tor mode isn’t for everybody, that coupled with LibreSSL, PersonaCrypt and the like helps demonstrate that PC-BSD is very security conscious, making some of the best security and privacy technology more user-friendly.

Q:  Did any of these features pose a particular challenge?

Switching over to LibreSSL was quite a challenge, I worked with  Bernard Spil (Whom should be getting a port-commit bit soon), to find and fix lots of packages that still used really old and vulnerable OpenSSL functionality. It was really eye-opening, and I think proves the point why we should somewhat frequently “cull” the old and obsolete code, because you never know what is still lurking around that uses it.

Q: What features do you plan on tackling next?

For 10.2 we will be working on getting VIMAGE enabled and better supported by Warden. Along with that will be updates to the AppCafe for more jail functionality, and beefing up the number of PBIs in our tree which have configuration UIs. I also plan on doing some infrastructure work, so that we can do more frequent builds of packages for -CURRENT and EDGE users, along with other architectures down the road.

Q: What does the upgrade path to 10.1.2 look like?

Upgrading from 10.1.1 is pretty easy. Our new updater went into the previous release, and it allows doing a “background” update of all your packages to the 10.1.2 set. Basically a new boot-environment is created, and fresh 10.1.2 packages are installed into it via a chroot. This means you can keep working throughout the update, and when its done you only have to reboot to get into the new BE and be on 10.1.2 with the new features.

Another Data Center Site Visit – NYI

No Systems Administrators Were Harmed While Writing This Blog Entry

Mmm...  Freshly-unboxed servers.  There really is nothing better to wake up to in the morning.

Well, okay, coffee.  But new servers - definitely second.

In late April, the FreeBSD Foundation generously purchased more machines to keep the FreeBSD.org infrastructure operating smoothly.  While the new servers are not yet in production (a task the Cluster Administrators will undertake while at BSDCan in June), we have planned far in advance what we intend to do with the new hardware.

In mid-May, I spent several days at our East-Coast US colocation facility, racking, cabling, installing, and configuring the new servers.

As They Say in Real-Estate: Colocation, Colocation, Colocation

The new hardware is located at New York Internet in Bridgewater, New Jersey, who generously provides colocation services to the FreeBSD Project.  They have an amazing staff, and whether we are on-site or working with them through their ticket system, are always friendly, knowledgeable, and of course, helpful.

New Hardware Specs

In total, 14 new SuperMicro-based machines were purchased from iXsystems, for this site. They are all 1U servers, each with four 1TB drives, redundant power supplies, and gigabit ethernet.

Ten of the machines are single-socket Intel(R) Xeon(R) E3-1230 v3 CPUs, running at 3.30GHz (4-core with hyper-threading, providing 8 logical CPU threads), with 32 GB RAM.

Four of the machines are dual-socket Intel(R) Xeon(R) E5-2630 v3 CPUs, running at 2.40GHz (8-core with hyper-threading, providing 16 logical CPU threads per socket, 32 threads total), with 64 GB RAM.

For the Inner Geek in All of Us

No blog post about new hardware would be complete without pictures, right?

This is the front view of the ten single-socket machines:


And this is the back view of the same machines:


(Yes, I really do label each end of the network and serial console cables.)

Then, this is the front view of the four dual-socket machines (sorry for the blurry picture, though):


And the back view of the dual-socket machines:


Last but not Least...

On behalf of the FreeBSD Foundation and the FreeBSD Cluster Administration Team, I would like to thank New York Internet for generously providing the colocation space for our east-coast site, the NYI network operations team for all of their assistance during my visit, and especially all of the community investors that have generously donated to the FreeBSD Foundation.  All of your support is greatly appreciated.

Thank you for helping make FreeBSD better!

Lumina Desktop Status Update/FAQ

With the amount of changes to the Lumina desktop environment and the increasing number of questions/comments we are receiving, I thought it would be a good time to post a status report about it and answer many of the commonly asked questions.

As always, please post any bug reports or feature requests on the PC-BSD Bug tracker.

If you wish to get involved in the development of the Lumina Desktop Environment, you can find the source code in our repository on GitHub.

Lumina Desktop FAQ/Status Update:

What is the Lumina Desktop Environment?

  • The Lumina Desktop Environment (sometimes referred to as Lumina-DE or just Lumina) is a BSD-licensed, FreeDesktop-compliant graphical interface for a desktop operating system.
  • It has been written from scratch in C++/Qt5 (it is not based on any existing DE code-base). It uses Fluxbox for the window management in the background and xscreensaver for screensaver/screenlocking functionality. The only other runtime dependencies are a couple of small X11 utilities (xnumlock, xbrightness, xterm, xrandr).
  • This results in a very lightweight, very smooth desktop experience with minimal system overhead.

What is the current development status?

  • While the current version (0.8.4) is still considered to be “Beta” quality, that is primarily because there are still a couple external utilities which have not had in-house replacements written yet (Fluxbox and xscreensaver primarily).
  • In its current state it is very stable and completely usable on a daily basis for both developers and non-developers alike.
  • The only caveat for its “beta” status is that since there are some backend systems which are still in major development, we reserve the right to break a user’s customized settings during the transition to these new systems (the window manager and screensaver settings are where I anticipate this happening once the replacement systems are in place). However, we already have a good track record of backwards compatibility with older settings formats, and will continue to work on ensuring a smooth transition between different versions of the desktop (even during heavy development/backend changes).

What are some of the features of Lumina?

  • Restore files from ZFS snapshots directly through the Insight file manager (lumina-fm).
  • View picture slideshows or play multimedia files through the Insight file manager.
  • Completely configurable – the interface is designed around a plugin-based mentality where everything you see/use is a compiled-in plugin (ensuring that every plugin is guaranteed to work). This means that you can create an interface custom-tailored to your specific needs.
  • Easy-to-use configuration utility (lumina-config). This single utility controls all the different configuration options for the desktop in one simple location.
  • Total system search capabilities through the lumina-search utility (without having a daemon eating memory in the background).
  • Screenshot functionality through the lumina-screenshot utility (also tied to the “print screen” key by default).
  • Multi-monitor support through xrandr (every monitor is treated as a distinct screen, with it’s own background, interface, etc..). Lumina also provides a graphical utility for adding/removing monitors from within the Lumina session (lumina-xconfig).
  • Simple system controls through the “System Dashboard” for things like audio volume, screen brightness, battery status/notifications, and workspace switching.
  • Easily select which application to use when opening a file and set that as the default for the next time. If you happen to remove that default application, it will automatically re-open the application selection dialog the next time you open that type of file again. This utility (lumina-open) is also tied into the commonly used xdg-open utility so that 3rd party applications will automatically use this utility for opening files outside of the application scope.

How is it different from other desktop environments?

  • Designed on PC-BSD, specifically for the BSD community at large (although it is easily ported to any OS, including Linux distros).
  • Does not require any of the commonly-used desktop implementation frameworks (DBUS, policykit, consolekit, systemd, HALD, etc..).
  • Does not come bundled with any “end-user” applications (web browsers, email clients, multimedia software, office suites, etc..). The only utilities that Lumina brings to the table by default are the ones written specifically for the project and are generally for background/utilitarian functionality (the largest utility is the file manager).
  • Simple text-based configuration file for setting system-wide defaults for new users. This allows distributors of the desktop to easily pre-set the system defaults/interface so it just works for the end user.
  • Plugin-based interface design. This allows the user to make the desktop as light/heavy as desired (within reason) simply by choosing which plugins to have running on their desktop/panels.

Are there any PC-BSD specific features?

  • Hardware-level screen brightness control (sysctl control through pc-sysconfig on PC-BSD 10.1.2+).
  • Links to the PC-BSD Control Panel and AppCafe are embedded directly into the Lumina interface for system control/modifications.
  • Detects when system updates are in progress and prevents the user from shutting down the system until that is complete (you can log out of Lumina and then shutdown the computer from the PCDM login manager if absolutely necessary) .
  • Designed to work hand-in-hand with the PC-BSD utilities (mounttray, AppCafe, Life Preserver, etc.) preventing conflicts between DE and OS utilities.

Is it available for any other operating systems?

  • There are currently source files in the project for building/using Lumina on: FreeBSD/PC-BSD, OpenBSD, DragonFlyBSD, kFreeBSD, Debian, and generic “Linux” distributions.
  • Please contact the package management/distribution team for your particular OS to determine the availability of pre-compiled packages.

What does it look like?

  • Due to the flexibility of the interface design and the ease with which the themes may be customized, the Lumina desktop can take many various forms. Here are a just a few of the variations that I have assembled on my system:
Click to view slideshow.
  • As we continue to create more plugins with every new version of Lumina, the variability of the interface just continues to grow. If you have a particular plugin/feature that you would like, please post a feature request on the PC-BSD bug tracker!

What plugins are currently available (as of version 0.8.4)?

  • Panel Plugins: AppLauncher, AppMenu, BatteryMonitor, Clock, DesktopBar, DesktopSwitcher, HomeButton, Spacer, SystemDashboard, SystemTray, TaskManager(with or without grouping), UserButton.
  • Desktop Plugins: AppLauncher, AudioPlayer, Calendar, DesktopView, Notepad
  • Menu Plugins: AppLauncher, AppMenu, LineBreak, OpenTerminal, OpenFileManager, SettingsMenu, WindowList
  • More plugins are being created all the time!

What is being worked on right now?

  • Right now, we are mainly focused on expanding the scope of the desktop through creation of new plugins and enhancement of the existing plugins.
  • The next “big” change planned is the move to a new window manager which is in the process of being written right now.  This change will correspond with version 0.9.0.
  • We are also working on tweaking the default themes and colors right now, and are looking into adding new themes & color schemes “out of box” for people to be able to simple select and use.

A new window manager? Is something wrong with Fluxbox?

  • Fluxbox is a great WM, and the Lumina project will be still be using it for quite some time yet while the replacement is being written. The main reason a new WM is planned is for better integration with the Lumina desktop in the following areas:
    • Interface/font scaling
    • Keyboard shortcuts (particularly with using the default application registrations in Lumina)
    • Addition of WM “modes” for use on various types of devices.
    • Allow compositing between windows (for better transparency effects)
    • Uniform Theme/Appearance
  • In addition, having a custom WM specifically for Lumina allows us to be able to combine the functionality of a few common background daemons into a single utility (since at their core they all revolve around responding to particular base system events).
    • Window Manager (ICCCM/EWMH events)
    • Screensaver/Screen Locker (All Input Events – transparent watcher)
    • Keyboard shortcuts (Keyboard Input Events)
    • Application message logger/viewer (some EWMH events – transparent watcher)
    • Power management (system events through devd/HALD/other)
  • Having all these systems within the same utility allows us to be able to create a much tighter level of inter-functionality between them. For a few quick examples:
    • Temporarily disable the screensaver while a window is full-screen (such as when watching a video)
    • While the screen is locked, prevent the window manager from showing any new windows on the screen and stop all keyboard shortcut handling (preventing possible security issues), while still allowing some notifications and background procedures.
    • Enable different levels of power management while the screen is locked or the screensaver is active.
  • This is a fairly massive undertaking, so I want to re-iterate that I expect it to be quite some time until this new window manager is ready for use and we will continue to use Fluxbox in the meantime. The Lumina desktop will remain within the 0.8.x series of version numbers while this new utility is in development.

How can I get involved?

  • If you are a C++/Qt developer, there are tons of areas where you could contribute! Some of them are: finding/fixing bugs, writing new interface plugins, adding functionality to existing plugins, and more!
  • If you are a CSS developer, then you might be interested in making new themes for Lumina (using QSS: it is very similar to CSS but specifically for Qt applications/widgets). These themes can be created/modified directly within the Lumina configuration utility, and also provides an easy way to directly implement the user’s current color scheme. Once you have a theme put together that you like, you can just send us that file (located in ~/.lumina/themes) and we can get that included with any future versions of the desktop.
  • If you want to create new color schemes, you can do so via the Lumina configuration utility. Once you have one that you like, just send in that file (located in ~/.lumina/colors) and we can get that included with any future versions of the desktop.
  • If you found a bug or have an idea for a cool new feature, you can post bug reports or feature requests on the PC-BSD bug tracker!

How can I contact the developer(s) with questions/comments/snide remarks?

 

HotFix release to 10.1.2 – Now available

A minor hotfix update to the 10.1.2 ISO’s has been released today. This includes fixes to advanced installation using raidz, cache and log devices, as well as a fix to the text-installer when booted in UEFI mode. Users who have already installed 10.1.2 will not need to download, and can instead online-update to install any fixes.

Download Now

PC-BSD 10.1.2 Released

The PC-BSD team is pleased to announce the availability of the next PC-BSD / TrueOS quarterly release, 10.1.2.

 

PC-BSD 10.1.2 Notable Changes:

  •  New PersonaCrypt Utility
    • Allows moving all of users $HOME directory to an encrypted USB Drive. This drive can be connected at login, and used across different systems
    • Stealth Mode — Allows login to a blank $HOME directory, which is encrypted with a one-time GELI key. This $HOME directory is then discarded at logout, or rendered unreadable after a reboot
  • Tor mode — Switch firewall to running transparent proxy, blocking all traffic except what is routed through Tor.
  • Migrated to IPFW firewall for enabling VIMAGE in 10.2
  • Added sound configuration via the first boot utility
  • Support for encrypted iSCSI backups via Life-Preserver, including support for bare-metal restores via installer media
  • New HTML handbook, updated via normal package updates
  • Media Center support allowing direct login to Kodi and PlexHomeTheater for the 10ft user experience
  • Switch to new AppCafe interface, with remote support via web-browser
  • Improvements to Online Updater, along with GRUB nested menus for Boot-Environments
  • Migrate all ports to using LibreSSL instead of OpenSSL
  • Switch from NTPD to OpenNTPD
  • Lumina desktop 0.8.4
  • Chromium 42.0.2311.135
  • Firefox 38.0
  • NVIDIA Driver 346.47
  • Pkg 1.5.2

 

Updating

Users currently running the 10.1.1 release can now apply the updates via their Update Manager utility. Users running TrueOS or CLI can start the update with the following command:

# pc-updatemanager pkgupdate

 

Getting media

10.1.2 DVD/USB media can be downloaded from the following URL via HTTP or Torrent. http://www.pcbsd.org/en/download.html

Reporting Bugs

Found a bug in 10.1.2? Please report it (with as much detail as possible) to our bugs database. https://bugs.pcbsd.org

Contacting Us