PC-BSD 10.0.3 Quarterly Package Update Released

The PC-BSD team is pleased to announce the availability of the next PC-BSD quarterly package update, version 10.0.3!

This update includes a number of important bug-fixes, as well as newer packages and desktops. Packages such as Chromium 37.0.2062.94, Cinnamon 2.2.14, Lumina 0.6.2 and more. This release also includes a CD-sized ISO of TrueOS, for users who want to install a server without X. For more details and updating instructions, refer to the notes below.

We are already hard at work on the next major release of PC-BSD, 10.1 later this fall, which will include FreeBSD 10.1-RELEASE under the hood. Users interested in following along with development should sign up for our Testing mailing list.

PC-BSD Notable Changes

* Cinnamon 2.2.14
* Chromium 37.0.2062.94
* NVIDIA Driver 340.24
* Lumina desktop 0.6.2-beta
* Pkg 1.3.7
* Various fixes to the Appcafe Qt UI
* Bugfixes to Warden / jail creation
* Fixed a bug with USB media not always being bootable
* Fixed several issues with Xorg setup
* Improved Boot-Environments to allow “beadm activate” to set default
* Support for jail “bulk” creation via Warden
* Fixes for relative ZFS dataset mount-point creation via Warden
* Support for full-disk (GELI) encryption without an unencrypted /boot partition

TrueOS

Along with our traditional PC-BSD DVD ISO image, we have also created a CD-sized ISO image of TrueOS, our server edition.

This is a text-based installer which includes FreeBSD 10.0-Release under the hood. It includes the following features:

* ZFS on Root installation
* Boot-Environment support
* Command-Line versions of PC-BSD utilities, such as Warden, Life-Preserver and more.
* Support for full-disk (GELI) encryption without an unencrypted /boot partition

We have some additional features also in the works for 10.1 and later, stay tuned this fall for more information.

Updating

Due to some changes with how pkgng works, it is recommended that all users update via the command-line using the following steps:

# pkg update –f
# pkg upgrade pkg
# pkg update –f
# pkg upgrade
# pc-extractoverlay ports
# reboot

PKGNG may need to re-install many of your packages to fix an issue with shared library version detection. If you run into issues doing this, or have conflicts, please open a bug report with the output of the above commands.

If you run into shared library issues running programs after upgrading, you may need to do a full-upgrade with the following:

# pkg upgrade –f

Getting media

10.0.3 DVD/USB media can be downloaded from this URL via HTTP or Torrent.

Reporting Bugs
Found a bug in 10.0.3? Please report it (with as much detail as possible) to our new RedMine Database.

BSDDay Argentina Trip Report: Damian Vicino

The Foundation recently sponsored Damian Vicino to attend BSDDay Argentina. Here is his trip report:

BSDday is the only BSD conference in South America as far as I know. The event's inception was in 2008 by 2 BSD Users Groups in Buenos Aires City. participated as part of the organisation committee from 2009 - 2012. In 2013, the event had no edition because of some big changes in the livesof the people participating in the organisation committee. In my case, I moved out of the country (and the continent). Thanks to the FreeBSD Foundation, I was able to return to South America for a few weeks this year to re-float the committee and the event, making possible the run of a 5th edition.

We started the preparation a few months before by coordinating remotely, but there was a lot of stuff to be done in-place, so I traveled 10 days earlier. In the days before the event, I coordinated with Universidad de Buenos Aires to finish the arrangements for the space to run the event and the supplies needed for the event. I worked as the main contact for the university and dealt with all the paperwork; being the largest university in Argentina, there is a lot of paperwork for everything. An interesting institutional plus this year is that the Faculty of Science and Department of Computer Science of Universidad de Buenos Aires declared officially the BSDday as an Event of Interest. Simultaneously, Hernan Constante and Matias Celani were coordinating accommodations for one of the speakers who traveled from Mar del Plata and making arrangements to have food & coffee for the event. Thanks for their help and also to Alejandro Lazaro who was helping in all he could remotely since he also moved out of Buenos Aires.

The quantity of proposals for talks received this year was about half the usual. We contacted previous speakers for feedback and we decided to include discussion spaces to find out why and how we can make it better for next year. On August 9th, a few minutes before the event started, the first speaker had family emergency. We decided to delay the opening talk and use the time for a first open discussion about the event and its future. The attendance was the lowest ever, so we focused the first discussion space on this topic. It appears to be a consensus that August is not a good month for the conference, because of the power outages in Buenos Aires in summer. From previous years, we knew that November is not good either. Another apparent reason is the break in continuity of the event (in 2013). Everyone in the room actively participated in the open discussion spaces. We noticed from discussions that the demographics of the event had changed. This time, we had a group of desktop users, mostly from FreeBSD, while in previous years we had mostly sysadmins from OpenBSD working in large companies or ISPs.

After the discussion, I did the opening talk with the help of Hernan Constante. The talk was also open to discussion so it extended a little longer than programmed; lucky for us, having only 1 track, it didn't affect the schedule much. The second talk was for 40 minutes, but was extended up to 2 hours and ended up in a different topic than the one it started with. We were tempted to stop it, but people were asking so many questions that we let it flow. We then had 4 more talks (including mine) and 2 more spaces for open discussion about anything-BSD where we collected opinions about the event, about BSD in Argentina, and the future of BSD advocacy actions. Since we didn't have sponsors for the food/coffee/supplies, we asked if anyone wanted to contribute at the end of the event. We were glad to see that everyone in the room put in money and we almost covered every expense for the event in this way. After the event, about 90% of the people moved to the bar across the street to share some beers and we kept discussing until the bar closed and kicked us out.

The week after the event, I met again with some organisers to discuss ideas for next year and do some analysis of what happened this year. One week later, I met with some companies and professionals to check sponsoring possibilities for next year's edition.

Last week, I collected and processed the materials we obtained from the event: videos, photos, and slides from every presentation. I still need to recover a few videos that we had to download to one of the organiser's computer (who left the country before me). In the following weeks, we will upload the videos, slides and pictures and formally close this year's event in order to start working for the 6th edition, expected to happen in 2016.

Once again, thank you very much to the FreeBSD Foundation for helping me with the expenses for this trip, to the University of Buenos Aires Faculty of Science and Computer Science department for giving us the space and support, to Hernan Constante, Alejandro Lazaro, Matias Celani, the speakers, and all those who helped to make this event possible once again.

New Lumina source repo and FreeBSD port

By popular demand, the source tree for the Lumina project has just been moved to its own repository within the main PC-BSD project tree on GitHub.

In addition to this, an official FreeBSD port for Lumina was just committed to the FreeBSD ports tree which uses the new repo.

 

By the way, here is a quick usage summary for those that are interested in how “light” Lumina 0.6.2 is on PC-BSD 10.0.3:

System: Netbook with a single 1.6GHz atom processor and 2GB of memory (Fresh installation of PC-BSD 10.0.3 with Lumina 0.6.2)

Usage: ~0.2–0.4% CPU and ~120MB active memory use (no apps running except an xterm with “top” after a couple minutes for the PC-BSD tray applications to start up and settle down)

 

10.0.3-RC2 Available for Testing

PC-BSD 10.0.3-RC2 ISO images are now available for testing.

Users on the EDGE package set, or 10.0.3-RC1 can update to the newer set with the following commands:

# pkg update –f
# pkg upgrade
# pc-extractoverlay ports

This update brings in the newer pkgng 1.3.7, which may need to re-install many of your packages in order to properly fix an issue with shared-library version detection in previous pkgng releases.

The current plan is to release 10.0.3 early next week, so please let us know of any issues right away via our RedMine bug tracker.

pkg(8) is now the only package management tool

The ports tree has been modified to only support pkg(8) as package management system for all supported version of FreeBSD.

if you were still using pkg_install (pkg_* tools) you will have to upgrade your system.

The simplest way is

cd /usr/ports/ports-mgmt/pkg
make install

then run

pkg2ng

You will have lots of warning, don’t be scared, they are expected, pkg_*  databases used to get easily mangled. pkg2ng is most of the time able to deal
with it.

If however you encounter a problem then please report to [email protected]

A tag has been applied to the ports tree if you need to get the latest ports tree before the EOL of pkg_install:

https://svn.FreeBSD.org/ports/tags/PKG_INSTALL_EOL

A branch has been created if some committers want to provides updates on the for pkg_install users:

https://svn.FreeBSD.org/ports/branches/pkg_install

Please note that this branch is not officially maintained and that we strongly recommend that you do migrate to pkg(8)

The ports tree is now stage only

The ports tree is now fully staged (only 2% has been left unstaged, marked as broken and will be removed from the ports tree if no PR to stage them are pending in bugzilla).

I would like to thank every committer and maintainers for their work on staging!
It allowed us to convert more than 23k packages to support stage in only 11 months!

Staging is a very important state, it allows us to right now be able to run quality testing scripts on the packages (which already allowed to fix tons of hidden problems) and it allows use to be able to build packages as a regular user!

It also opens the gates to new features that users have been requesting for many years:

  • flavors
  • multiple packages

Expect those features to happen in the near future.

FreeBSD 10.1-BETA1 Available

The first BETA build for the FreeBSD 10.1 release cycle is now available. ISO images for the amd64, armv6, i386, ia64, powerpc, powerpc64 and sparc64 architectures are available on most of our FreeBSD mirror sites.

helping out with VC4

I've had a couple of questions about whether there's a way for others to contribute to the VC4 driver project.  There is!  I haven't posted about it before because things aren't as ready as I'd like for others to do development (it has a tendency to lock up, and the X implementation isn't really ready yet so you don't get to see your results), but that shouldn't actually stop anyone.

To get your environment set up, build the kernel (https://github.com/anholt/linux.git vc4 branch), Mesa (git://anongit.freedesktop.org/mesa/mesa) with --with-gallium-drivers=vc4, and piglit (git://anongit.freedesktop.org/git/piglit).  For working on the Pi, I highly recommend having a serial cable and doing NFS root so that you don't have to write things to slow, unreliable SD cards.

You can run an existing piglit test that should work, to check your environment: env PIGLIT_PLATFORM=gbm VC4_DEBUG=qir ./bin/shader_runner tests/shaders/glsl-algebraic-add-add-1.shader_test -auto -fbo -- you should see a dump of the IR for this shader, and a pass report.  The kernel will make some noise about how it's rendered a frame.

Now the actual work:  I've left some of the TGSI opcodes unfinished (SCS, DST, DPH, and XPD, for example), so the driver just aborts when a shader tries to use them.  How they work is described in src/gallium/docs/source/tgsi.rst. The TGSI-to_QIR code is in vc4_program.c (where you'll find all the opcodes that are implemented currently), and vc4_qir.h has all the opcodes that are available to you and helpers for generating them.  Once it's in QIR (which I think should have all the opcodes you need for this work), vc4_qpu_emit.c will turn the QIR into actual QPU code like you find described in the chip specs.

You can dump the shaders being generated by the driver using VC4_DEBUG=tgsi,qir,qpu in the environment (that gets you 3/4 stages of code dumped -- at times you might want some subset of that just to quiet things down).

Since we've still got a lot of GPU hangs, and I don't have reset wokring, you can't even complete a piglit run to find all the problems or to test your changes to see if your changes are good.  What I can offer currently is that you could run PIGLIT_PLATFORM=gbm VC4_DEBUG=norast ./piglit-run.py tests/quick.py results/vc4-norast; piglit-summary-html.py --overwrite summary/mysum results/vc4-norast will get you a list of all the tests (which mostly failed, since we didn't render anything), some of which will have assertion failed.  Now that you have which tests were assertion failing from the opcode you worked on, you can run them manually, like PIGLIT_PLATFORM=gbm /home/anholt/src/piglit/bin/shader_runner /home/anholt/src/piglit/generated_tests/spec/glsl-1.10/execution/built-in-functions/vs-asin-vec4.shader_test -auto (copy-and-pasted from the results) or PIGLIT_PLATFORM=gbm PIGLIT_TEST="XPD test 2 (same src and dst arg)" ./bin/glean -o -v -v -v -t +vertProg1 --quick (also copy and pasted from the results, but note that you need the other env var for glean to pick out the subtest to run).

Other things you might want eventually: I do my development using cross-builds instead of on the Pi, install to a prefix in my homedir, then rsync that into my NFS root and use LD_LIBRARY_PATH/LIBGL_DRIVERS_PATH on the Pi to point my tests at the driver in the homedir prefix.  Cross-builds were a *huge* pain to set up (debian's multiarch doesn't ship the .so symlink with the libary, and the -dev packages that do install them don't install simultaneously for multiple arches), but it's worth it in the end.  If you look into cross-build, what I'm using is rpi-tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-gcc and you'll want --enable-malloc0returnsnull if you cross-build a bunch of X-related packages.

FreeBSD Foundation announces IPsec Enhancement Project

The Internet Protocol Security (IPsec) suite is used to implement virtual private networks on FreeBSD and other operating systems. As the networking world continues its transition from 1 to 10, to 40 gigabit per second speeds, and faster, improvements in IPsec’s cryptographic building blocks are necessary to keep pace. The FreeBSD Foundation is pleased to announce that long-time FreeBSD developer John-Mark Gurney is adding modern AES modes to FreeBSD’s cryptographic framework and IPsec. This project is co-sponsored by the FreeBSD Foundation and Netgate, a leading vendor of BSD-based firewalls and networking gear.

The project adds new encryption modes while also importing infrastructure updates from OpenBSD giving FreeBSD users unprecedented support for high performance, encrypted communications.  New modes include AES-CTR and AES-GCM with hardware acceleration using Intel’s AES-NI instructions. According to John-Mark, “on a modern 64-bit x86 CPU one core can process about 1 gigabyte per second of data” using the new AES-GCM mode.

Concurrent with this project, FreeBSD committer and pfSense employee Ermal Luçi will update the FreeBSD IPsec stack to take advantage of the new cryptographic modes.

Jim Thompson, a co-owner of both Netgate and ESF (the company behind pfSense), said “We are pleased to contribute to this project.  Our interest in high-performance IPsec is obvious, however we also recognize the importance of contributing this capability to the FreeBSD project. Not only because our own software is based on FreeBSD, but for the benefit it brings to the entire community.  We plan to have AES-GCM support for IPsec with AES-NI acceleration available in the 2.2 release of pfSense software.”

The project is currently in progress, with a planned completion at the end of September 2014.

PC-BSD 10.0.3 Preview: Lumina Desktop

As we are getting ready for PC-BSD 10.0.3, I wanted to share a little preview of what to expect with the Lumina desktop environment as you move from version 0.4.0 to 0.6.2.

To give you a quick summary, pretty much everything has been updated/refined, with several new utilities written specifically for Lumina. The major new utility is the “Insight” file manager: with ZFS snapshot integration, multimedia player, and image slideshow viewer capabilities built right in by default. It also has a new snapshot utility and the desktop configuration utility has been completely rewritten. I am going to be listing more details about all the updates between the versions below, but for those of you who are not interested in the details, you can just take a look at some screenshots instead.…  :-)

Lumina10-0-3--1

 

Lumina10-0-3--2

Lumina10-0-3--3

 

Lumina10-0-3--4

Lumina10-0-3--5

 

==== FULL UPDATE DETAILS ====

(Moving from 0.4.0 to 0.6.2)
Desktop

- A desktop plugin system has been implemented, with two plugins available at the moment (a calandar plugin, and an application launcher plugin).
– The panel plugin system has been refined quite a bit, with transparency support for the panel itself and automatic plugin resizing for example.
– A new panel plugin has been added: the system dashboard. This plugin allows control over the audio volume, screen brightness, and current workspace, while also displaying the current battery status (if applicable) and containing a button to let the user log out (or shutdown/restart the system).
– The user button panel plugin has been re-implemented as well, and incorporating the functionality of the desktopbar plugin. Now the user has quick access to files/application in the ~/Desktop folder, as well as the ability to add/remove shortcuts to system applications in the desktop folder with one click.
– New backgrounds wallpapers and project logo (courtesy of iXsystems).

NOTE: Users of the older versions of the Lumina desktop will have their configuration files returned to the defaults after logging in to the new version for the first time.


Utilities
The new file manager (lumina-fm, also called “Insight”):
Features:
– Browse the system and allow the bookmarking of favorite directories
– Simple multimedia player to allow playing/previewing multimedia files
– Image slideshow viewer for previewing image files
– Full ZFS file/directory restore functionality if ZFS snapshots are available
– Menu shortcuts to quickly browse attached/mounted devices
– Tabbing support for browsing multiple directories at once
– Standard file/directory management (copy/paste/delete/create)
– Supported multimedia and image formats are auto-detected on start, so if a particular file is not recognized, you just need to install the appropriate library or plugin on your system to provide support (none required by default).

The new screenshot utility (lumina-screenshot):
Features:
– Simple utility to create/save screenshots on the system.
– Can capture the entire system, or individual windows.
– Can delay the image capture for a few seconds as necessary
– Automatically assigned to the “Print Screen” keyboard shortcut by default, but also listed in the application registry under utilities.

The configuration utility (lumina-config):
Features:
– Competely new implementation
– Configure desktop appearance (background image, add desktop plugins)
– Configure panels (location, color/transparency, size, manage plugins, up to 2 panels supported per screen)
– Configure right-click menu plugins
– Manage/set global keyboard shortcuts (including shortcuts for adjusting audio volume or screen brightness)
– Manage/set default applications for the system by categories or individually
– Manage session options (enable numlock on log in, play audio chimes)
– Manage/set applications/files to be launched on log in
– Manage window system options (appearance, mouse focus policy, window placement policy, number of workspaces)

The application/file opener utility (lumina-open):
– Update the overall appearance of the application selector window.
– Fully support registered mime-types on the system now, and recommend those applications as appropriate.

ZFS support in libvirt

An upcoming release of libvirt, 1.2.8 that should be released early September, will include an initial support of managing ZFS volumes.

That means that it's possible to boot VMs and use ZFS volumes as disks. Additionally, it allows to control volumes using the libvirt API. Currently, supported operations are:

  • list volumes in a pool
  • create and delete volumes
  • upload and download volumes

It's not possible to create and delete pools yet, hope to implement that in the next release.

Defining a pool

Assume we have some pools and want to use one of them in libvirt:

# zpool list
NAME SIZE ALLOC FREE FRAG EXPANDSZ CAP DEDUP HEALTH ALTROOT
filepool 1,98G 56,5K 1,98G 0% - 0% 1.00x ONLINE -
test 186G 7,81G 178G 0% - 4% 1.00x ONLINE -

Let's take filepool and define it with libvirt. This could be done using this virsh command:

virsh # pool-define-as --name zfsfilepool --source-name filepool --type zfs
Pool zfsfilepool defined

virsh # pool-start zfsfilepool
Pool zfsfilepool started

virsh # pool-info zfsfilepool
Name: zfsfilepool
UUID: 5d1a33a9-d8b5-43d8-bebe-c585e9450176
State: running
Persistent: yes
Autostart: no
Capacity: 1,98 GiB
Allocation: 56,50 KiB
Available: 1,98 GiB

virsh #

As you can see, we specify a type of the pool, its source name, such as seen in zpool list output and a name for it in libvirt. We also need to start it using the pool-start command.

Managing volumes

Let's create a couple of volumes in our new pool.


virsh # vol-create-as --pool zfsfilepool --name vol1 --capacity 1G
Vol vol1 created

virsh # vol-create-as --pool zfsfilepool --name vol2 --capacity 700M
Vol vol2 created

virsh # vol-list zfsfilepool
Name Path
------------------------------------------------------------------------------
vol1 /dev/zvol/filepool/vol1
vol2 /dev/zvol/filepool/vol2

virsh #

Dropping a volume is also easy:

virsh # vol-delete --pool zfsfilepool vol2
Vol vol2 deleted

Uploading and downloading data

Let's upload an image to our new volume:

virsh # vol-upload --pool zfsfilepool --vol vol1 --file /home/novel/FreeBSD-10.0-RELEASE-amd64-memstick.img 

... and download

virsh # vol-download --pool zfsfilepool --vol vol1 --file /home/novel/zfsfilepool_vol1.img

Note: if you would check e.g. md5 sum of the downloaded files, the result would be different as downloaded file will be of the same size as a volume. However, if you trim zeros, it'll be the same.

$ md5 FreeBSD-10.0-RELEASE-amd64-memstick.img zfsfilepool_vol1.img 
MD5 (FreeBSD-10.0-RELEASE-amd64-memstick.img) = e8e7cbd41b80457957bd7981452ecf5c
MD5 (zfsfilepool_vol1.img) = a77c3b434b01a57ec091826f81ebbb97
$ truncate -r FreeBSD-10.0-RELEASE-amd64-memstick.img zfsfilepool_vol1.img
$ md5 FreeBSD-10.0-RELEASE-amd64-memstick.img zfsfilepool_vol1.img
MD5 (FreeBSD-10.0-RELEASE-amd64-memstick.img) = e8e7cbd41b80457957bd7981452ecf5c
MD5 (zfsfilepool_vol1.img) = e8e7cbd41b80457957bd7981452ecf5c
$

Booting a VM from volume

Finally got to the most important part. In use a volume as disk device for VM 'devices' section of the domain XML should be updated with something like this:


<disk type='volume' device='disk'>
<source pool='zfsfilepool' volume='vol1'/>
<target dev='vdb' bus='virtio'/>
</disk>

Few notes

Note #1: this code is just a few weeks old, so quite likely there are some rough edges. Feel free to report problems to novel%freebsd.org if you spot any problems.

Note #2: this code is FreeBSD-only for now. However, it should not be hard to make it work on Linux with zfsonlinux.org. Its developers were kind enough to add some useful missing flags in some of the CLI tools. However, these changes are not available in any released version so far. There are some more minor differences between zfs on Linux and FreeBSD, but that should not be hard to address. I was planning to get to it as soon as a new version of zfs on linux with the necessary flags is available. However, if you are interested in that and ready to help with testing -- feel free to poke me so it could be done sooner.


MeetBSD California 2014

MeetBSD California 2014 (https://www.meetbsd.com/), Western Digital Campus, San Jose, United States 1 - 2 November, 2014. MeetBSD 2014 uses a mixed unConference format featuring both scheduled talks and community-driven events such as birds-of-a-feather meetings, lightning talks, and speed geeking sessions.

X with glamor on vc4

Today I finally got X up on my vc4 driver using glamor.  As you can see, there are a bunch of visual issues, and what you can't see is that after a few frames of those gears the hardware locked up and didn't come back.  It's still major progress.
2014-08-21 16.16.37
The code can be found in my vc4 branch of mesa and linux-2.6, and the glamor branch of my xf86-video-modesetting.  I think the driver's at the point now that someone else could potentially participate.  I've intentionally left a bunch of easy problems -- things like supporting the SCS, DST, DPH, and XPD opcodes, for which we have piglit tests (in glean) and are just a matter of translating the math from TGSI's vec4 instruction set (documented in tgsi.rst) to the scalar QIR opcodes.

Happy 20th birthday FreeBSD ports tree!

It all started with this commit from Jordan Hubbard on August 21, 1994:

Commit my new ports make macros
Still not 100% complete yet by any means but fairly usable at this stage.

Twenty years later the ports tree is still there and actively
maintained. A video was prepared to celebrate the event and to thank
all of you who give some of their spare time and energy to the project!