Weekly Feature Digest 26 — The Lumina Project and preload

This week the PC-BSD team has ported over preload, which is an adaptive readahead daemon. It monitors applications that users run, and by analyzing this data, predicts what applications users might run, and fetches those applications and their dependencies to speed up program load times. You can look for preload in the next few days in edge packages and grab it for testing on your own system.

There is an early alpha version of the Lumina desktop environment that has been committed to ports / packages. Lumina is a lightweight, stable, fast-running desktop environment that has been developed by Ken Moore specifically for PC-BSD. Currently it builds and runs, but lacks many other features as it is still in very early development. Grab it from the edge packageset and let us know what you think, and how we can also improve it to better suit you as a user!

Other updates this week:

* Fixed some bugs in ZFS replication causing snapshot operations to take
far longer than necessary
* Fixed an issue with dconf creating files with incorrect permissions
causing browsers to fail
* Added Lumina desktop ports / packages to our build system
* PC-BSD Hindi translation 100% complete
* improvements to the update center app
* Update PCDM so that it will use “pw” to create a user’s home directory if it is missing but the login credentials were valid. This should solve one of the last reported issues with PCDM and Active Directory users.
* Bugfix for pc-mounttray so that it properly ignores the active FreeBSD swap partition as well.
* Another small batch of 10.x PBI updates/approvals.

Have you never been to BSDCan?

I remember a time when I’d never been to a conference related to my passions. Once I went, things changed. I realized that making strong working relationships with others who share my passion is important. Not only does this solidify the community of which you are a member, it also helps you personally. Every conference [...]

PC-BSD Weekly Feature Digest 25

Most of you have already heard of the Heartbleed vulnerability, the flaw in OpenSSL encryption. For any of you that may not be aware (which is probably precious few), the Heartbleed vulnerability is basically a flaw that may allow a malicious user to gain access to information that is supposed to be kept safe through OpenSSL. The good news is that the FreeBSD project and PC-BSD have both released fixes that will apply to versions 10.x. If you are currently running a machine with PC-BSD 9.x you are using an earlier version of openSSL that does not have the vulnerability, so no action is necessary to protect yourself from this. If you are running PC-BSD version 10.x make sure to use the “system updater” to apply the security patch to openSSL. After applying the fix reboot your computer and you should be good to go.

Kris has finished a new PBI run-time that will fix a number of stability issues users may have been experiencing while using PBI’s. The fix has also subsequently helped speed up load times for some of the larger PBI’s that may have been hanging or taking a long time to load.

Update Center is moving foward, and has received some fine-tuning this week to help bring it into PC-BSD as the one-stop utility for managing updates. We’d like to add a special thanks to the author Yuri for primary design and layout for the update center. Ken will also be working to help smooth out GUI design elements and help with integrating it fully into PC-BSD.

Other Updates / Bug Fixes:

* Updated openssl packages for 10.0 PRODUCTION/EDGE
* Patched issue with KRDC using FreeRDP version in ports
* A new 9.2 server has been spun up and building PBIs for 9.2 again. (Server failed earlier this week)
* Started work on PBI runtime for Linux compat applications
* Another large chunk of work on Lumina
* Bugfixes for pc-mixer (showing the proper icons)
* Life-Preserver bugfixes
* Large update to the available 10.x PBIs. All updates are finished, a few new applications were also added.
* Bugfixes on a number of PBI’s (waiting on rebuilds to test/approve the new fixed apps)
* Hindi translation project now about 75% complete

FreeBSD Foundation Spring Fundraising Campaign!

We're kicking off our Spring Fundraising Campaign! Our goal this year is to raise $1,000,000 with a spending budget of $900,000.

As we embark on our 15th year of serving the FreeBSD Project and community, we are proud of how we've helped FreeBSD become the most innovative, realiable, and high-performance operating system. We are doing this by:
  • funding development projects,
  • having an internal technical staff available to work on small and large projects, fixing problems, and areas of system administration and release engineering,
  • providing legal support,
  • funding conferences and summits that allow face-to-face interaction and collaboration between FreeBSD contributors, users, and advocates,
  • and advocating for and educating people about FreeBSD by providing high-quality brochures, white papers, and the FreeBSD Journal.

We can't do this without you! You can help by making a donation today.

Help spread the word by posting on FaceBook, Twitter, your blogs, and asking your company to help. Did you know there are thousands of companies that wil match their employee's donations? Check with your company to see if you can automatically double your donation by having your company match your donation.

Thanks for your support!

FreeBSD Journal Issue #2 is Now Available!



The FreeBSD Journal Issue #2 is now available! You can get it on Google Play, iTunes, and Amazon. In this issue you will find captivating articles on pkg(8), Poudriere, PBI Format, plus great pieces on hwpmc(4) and Journaled Soft-updates. If you haven't already subscribed, now is the time!

The positive feedback from both the FreeBSD and outside communities has been incredible. In less than two months, we have signed up over 1,000 subscribers. This shows the hunger the FreeBSD community has had for a FreeBSD focused publication. We are also working on a dynamic version of the magazine that can be read in many web browsers, including those that run on FreeBSD.

The Journal is guided by a dedicated and enthusiastic editorial board made up of people from across the FreeBSD community. The editorial board is responsible for the acquisition and vetting of content for the magazine.

You can find out more information about the Journal by going to https://www.freebsdfoundation.org/journal. Or, subscribe now by going to the following links for the device you'd like to download to:

amazon-apps-store





Available_on_the_Mac_App_Store_Badge

Google Button







Your subscriptions and the advertising revenue the Journal receives will help offset the costs of publishing this magazine. So, consider signing up for a subscription today! 

We know you are going to like what you see in the Journal! Please help us spread the word by tweeting, blogging, and posting on your FaceBook page. You can also help by asking your company to put an ad in the Journal. For advertising information contact [email protected]

And, don't forget you can support the Journal and FreeBSD by making a donation today!

OpenSSL Security Update

Many users have asked us about the recent OpenSSL Heartbleed bug.  This only applies to users of PC-BSD 10.0, users of 9.x and earlier will not be effected.

A patch has gone out this morning to correct the issue, which includes the following FreeBSD security advisories:

http://www.freebsd.org/security/advisories/FreeBSD-SA-14:06.openssl.asc
http://www.freebsd.org/security/advisories/FreeBSD-SA-14:05.nfsserver.asc

By running the graphical “System Updater” you can apply the bug fixes, or via “freebsd-update” at the command-prompt. After applying this fix, please reboot and the systems version should now show 10.0-RELEASE–p9

PC-BSD Weekly Feature Digest 24

Another week bites the dust and we’ve got some fantastic new features heading your way. Just a quick update this week so let’s get right to it. The FreeBSD mailing list has put a call out to the community to know if you are interested in having some custom DirectX patches applied to wine. You can view the e-mail here if it interests you. If you’d like to respond directly to the e-mail list you can do so @ [email protected]

New Features:

* Tons of new PBI updates for 10.0
* Committed the new PBI runtime implementation for 10.x
* Fixed some edge cases with new runtime and particular apps
* Added support for running 32bit apps in new PBI runtime
* Patched RTLD and pushed out freebsd-update to prepare systems
* Added improved callback functionality for PBIs to run system commands
* Added umplayer as the new out-of-box default CD audio / DVD video player
* Merged latest FreeBSD ports and Gnome3 / Cinnamon ports
* Added options to set exec= and suid= options on ZFS datasets to installer
* Added “vagrant” development environment utility to PC-BSD base system
* Began builds of EDGE packages with all the above fixes

Bug Fixes:

* Fixed issue with missing English dictionary in KDE text-processing apps
* Fixed bug with Life-Preserver which was pruning snapshots too
aggressively with replication enabled
* Don’t provide localization option to FAT mounting routine for english locales
* Clean up the usage of ntfslabel to make sure that extra outputs don’t get included in the name for Win8 NTFS devices.

The Short List #8: Using #lldb with a core file on #FreeBSD

Debugging qemu this evening and it took me a minute or two to figure out the syntax for debugging a core file with lldb.

lldb mips-bsd-user/qemu-mips -c /mipsbuild/qemu-mips.core

Make sure you have permissions to access both the binary and the core, else you get a super unhelpful error of:

error: Unable to find process plug-in for core file ‘/mipsbuild/qemu-mips.core’

But, after that, you can start poking around:

Core file ‘/mipsbuild/qemu-mips.core’ (x86_64) was loaded.

Process 0 stopped

* thread #1: tid = 0, 0x00000000601816fa qemu-mips`_kill + 10, name = ‘qemu-mips’, stop reason = signal SIGILL

frame #0: 0x00000000601816fa qemu-mips`_kill + 10

qemu-mips`_kill + 10:

-> 0x601816fa: jb 0x60182f5c ; .cerror

0×60181700: ret

0×60181701: nop

0×60181702: nop

(lldb) bt

* thread #1: tid = 0, 0x00000000601816fa qemu-mips`_kill + 10, name = ‘qemu-mips’, stop reason = signal SIGILL

* frame #0: 0x00000000601816fa qemu-mips`_kill + 10

frame #1: 0x000000006003753b qemu-mips`force_sig(target_sig=<unavailable>) + 283 at signal.c:352

frame #2: 0x00000000600376dc qemu-mips`queue_signal(env=<unavailable>, sig=4, info=0x00007ffffffe8878) + 380 at signal.c:395

frame #3: 0×0000000060035566 qemu-mips`cpu_loop [inlined] target_cpu_loop(env=<unavailable>) + 1266 at target_arch_cpu.h:239

frame #4: 0×0000000060035074 qemu-mips`cpu_loop(env=<unavailable>) + 20 at main.c:201

frame #5: 0x00000000600362ae qemu-mips`main(argc=1623883776, argv=0x00007fffffffd898) + 2542 at main.c:588

frame #6: 0x000000006000030f qemu-mips`_start + 367

 

How I killed 13 500 000 pages in the Google search engine

Talk about a loaded title, en par with the quality (or lack there of) of the various click bait titles on the postings I see on Facebook and friends...

I was told by my hosting provider that my index to the FreeBSD mailinglists at http://www.mavetju.org/mail/ was using more bandwidth alone than all of their public websites together. Now this is not much of a record, since they have only low-bandwidth websites, but still...

Looking through the logs, I saw that the Googlebot and the Bingbot and some bot from China were happily fighting over CPU and bandwidth to index all of the files. Going at it on a speed of about 50 requests per seconds for 24 hours per day.

So what could I do? Checking in Google for site:mavetju.org/mail/, I saw that there were about 13 500 000 pages indexed. For what goal? Not much anymore, I have stopped following all except the FreeBSD Announcement mailinglists a couple of years ago. I still use it on my laptops, but that is all.

So... That mailinglist archive has been shut down. You can still find the cached version of it in Google by using the above search terms, but that will disappear too.

And that is the story on how I killed 13 500 000 pages in Google. I wonder how much many computers in their data center that frees up for other things. Probably none...

Ports 2014Q2 branched

I am pleased to announce that we have created the 2014Q2 branch of the ports
tree.

Because the first 2014Q1 branch was experimental you might not have heard of it
yet.

January 2014 saw the release of the first quarterly branch, intended at
providing a stable and high-quality ports tree. Those stable branches are a
snapshot of the head ports tree taken every 3 months and currently supported
for three months, during which they receive security fixes as well as build and
runtime fixes.

Packages are built on regular basis on that branch (weekly) and published as
usual via pkg.FreeBSD.org (/quarterly instead of the usual /latest).

They are signed the same way the /latest branch is.

While packages for 2014Q1 were only built for 10 (i386 and amd64) 2014Q2 will be
built for both FreeBSD 9 and 10 (i386 and amd64).

The first build of 2014Q2 will started this morning (wednesday at 1 am UTC) and should
hit your closest mirrors very soon.

On behalf of the port management team
Bapt

Sometimes you have to sit down and write #FreeBSD documentation

When working on new projects or hacks, sometimes you just have to stop, think and start writing down your processes and discoveries. While working on bootstrapping the DLink DIR-825C1, I realized that I had accumulated a lot of new (to me) knowledge from the FreeBSD Community (namely, Adrian Chadd and Warner Losh).

There is a less than clear way of constructing images for these embedded devices that has an analogue in the Linux community under the OpenWRT project. Many of the processes are the same, but enough are different that I thought it wise to write down some of the processes into the beginning of a hacker’s guide to doing stuff and/or things in this space.

The first document I came up with was based on the idea that we can netboot these little devices without touching the on-board flash device. This is what you should use to get the machine bootstrapped and figure out where all the calibration data for the wireless adapters exist. This is crucial to not destroying your device. The wireless calibration data (ART) is unique to each device, destroying it will mean you have to RMA this device.

The second document I’ve created is a description of how to construct the flash device hints entries in the kernel hints file for FreeBSD. I found the kernel hints file to be cumbersome in comparison to the linux kernel way of using device specific C files for unique characteristics.

Its interesting stuff if you have the hankering to dig a bit deeper into systems that aren’t PC class machines.

Meraki Sparky boards, and constant resetting

There's a Mesh internet project at Sudo Room and they've been doing some great work getting a platform up and running. However, like a lot of volunteer projects, they're working with whatever time and equipment they've been donated.

A few months ago they were donated a few hundred Meraki Sparky boards. They're an Atheros AR2317 SoC based device with an integrated 2GHz 802.11bg radio, 10/100 ethernet and.. well, a hardware watchdog that resets the board after five minutes.

Now, annoyingly, this reset occurs inside of Redboot too - which precludes them from being (fully) flashed before the unit reboots. Once the unit was flashed with OpenWRT, the unit still reboots every five minutes.

So, I started down the path of trying to debug this.

What did I know?

Firstly, the AR2317 watchdog doesn't have a way of resetting things itself - instead, all it can do is post an interrupt. The AR7161 and later SoCs do indeed have a way to do a full hardware reset if the watchdog is tickled.

Secondly, redboot has a few tricksy ways to manipulate the hardware:

  • 'x' can examine registers. Since we need them in KSEG1 (unmapped, uncached) then the reset registers (0x11000xxx becomes 0xb1000xxx.) Since its hardware access, we should do them as DWORDS and not bytes.
  • 'mfill' can be used to write to registers.
Thirdly, there's an Atheros specific command - bdshow - which is surprisingly informative:

RedBoot> bdshow
name:     Meraki Outdoor 1.0
magic:    35333131
cksum:    2a1b
rev:      10
major:    1
minor:    0
pciid:    0013
wlan0:    yes 00:18:0a:50:7b:ae
wlan1:    no  00:00:00:00:00:00
enet0:    yes 00:18:0a:50:7b:ae
enet1:    no  00:00:00:00:00:00
uart0:    yes
sysled:   no, gpio 0
factory:  no, gpio 0
serclk:   internal
cpufreq:  calculated 184000000 Hz
sysfreq:  calculated 92000000 Hz
memcap:   disabled
watchdg:  disabled (WARNING: for debugging only!)

serialNo: Q2AJYS5XMYZ8
Watchdog Gpio pin: 6
secret number: e2f019a200ee517e30ded15cdbd27ba72f9e30c8


.. hm. Watchdog GPIO pin 6? What's that?

Next, I tried manually manipulating the watchdog registers but nothing actually happened.

Then I wondered - what about manipulating the GPIO registers? Maybe there's a hardware reset circuit hooked up to GPIO 6 that needs to be toggled to keep the board from resetting.

Board: ap61
RAM: 0x80000000-0x82000000, [0x8003ddd0-0x80fe1000] available
FLASH: 0xa8000000 - 0xa87e0000, 128 blocks of 0x00010000 bytes each.
== Executing boot script in 2.000 seconds - enter ^C to abort
^C
RedBoot> # set direction of gpio6 to out
RedBoot> mfill -b 0xb1000098 -l 4 -p 0x00000043
RedBoot> x -b 0xb1000098
B1000098: 00 00 00 43 00 00 00 00  00 00 00 00 00 00 00 03  |...C............|
B10000A8: FF EF F7 B9 7D DF 5F FF  00 00 00 00 00 00 00 00  |....}._.........|

RedBoot> # pat gpio6 - set it high, then low.
RedBoot> mfill -b 0xb1000090 -l 4 -p 0x00000042
RedBoot> mfill -b 0xb1000090 -l 4 -p 0x00000002

.. then I manually did this every minute or so.

RedBoot>
RedBoot> mfill -b 0xb1000090 -l 4 -p 0x00000042
RedBoot> mfill -b 0xb1000090 -l 4 -p 0x00000002
RedBoot> mfill -b 0xb1000090 -l 4 -p 0x00000042
RedBoot> mfill -b 0xb1000090 -l 4 -p 0x00000002

.. so, the solution here seems to be to "set gpio6 to be output", then "pat it every 60 seconds."

I hope this helps people bring OpenWRT up on this board finally. There seems to be a few of them out there!

The Short List #6: Make the CD drive do something useful on #FreeBSD

Noted today that while grip could access the CD drive on my machine, clemetine-player and xfburn could not.

Figure out which device node your CD drive is with camcontrol:

camcontrol devlist | grep cd
at scbus4 target 0 lun 0 (cd0,pass2)

Simply add the following to /etc/devfs.conf and restart devfs to get access to the CD device:

perm /dev/cd0 0666
perm /dev/xpt0 0666
perm /dev/pass2 0666

Now bear in mind, that this means any user of your machine has access to the device now. Hopefully, on a desktop computer, you know all the users of your machine.

Using Jenkins libvirt-slave-plugin with bhyve

I've played with libvirt-slave-plugin today to make it work with libvirt/bhyve and decided to document my steps in case it would be useful for somebody.

libvirt-slave-plugin

Assuming that you already have Jenkins up and running, installation of libvirt-slave-plugin is as follows. As we need a slightly modified version, we need to build it ourselves. I've made a fork which contains a required modification which could be cloned like that:

git clone -b bhyve [email protected]:jenkinsci/libvirt-slave-plugin.git

The only change I made is adding a single line with 'BHYVE' hypervisor type, you could find the pull request here. When that would be merged, this step will be not required.

So, getting back to the build. You'll need maven that could be installed from ports:

cd /usr/ports/devel/maven2 && make install clean

When it's installed, go back to the plugin we cloned and do:

mvn package -DskipTests=true

When done, login to the Jenkins web interface, go to Manage Jenkins -> Manage Plugins -> Advanced -> Upload Pluging. It'll ask to provide a path to the plugin. It would be target/libvirt-slave.hpi in our plugin directory.

After plugin is installed, please follow to Manage Jenkins -> Configure System -> Add new cloud. Then you'll need to specify hypervisor type BHYVE and configure credentials so Jenkins could reach your libvirtd using SSH. There's a handy 'Test Connection' you could use your configuration.

Once done with that, we can go to Manage Jenkins -> Manage Nodes -> New Node and choose 'libvirt' node type. Then you'll need to choose a libvirt domain to use for the node. From now on, node configuration is pretty straightforward, expect, probably an IP address of the slave. To find out an IP address, you'd need to find out its MAC address (just run virsh dumpxml and you'll find it there) and then find the corresponding file in dnsmasq/default.leases file.

Guest Preparation

The only thing guest OS needs is to have jdk installed. I preferred to download a package with java/openjdk7, but I had to configure network first. My VMs use bridged networking on virbr0, so NAT config looks like that in /etc/pf.conf:


ext_if="re0"
int_if="virbr0"

virt_net="192.168.122.0/24"

scrub all

nat on $ext_if from $virt_net to any -> ($ext_if)

Now openjdk could be installed from the guest using:

pkg install java/openjdk7

Finally, find the nodes in node management menu and press 'Launch slave agent' button. It should be ready for the builds now.

PS It might be useful to sync clock on both guest and host systems using ntpdate.

PPS libvirt version should be at least 1.2.2.


EuroBSDCon 2014

EuroBSDCon 2014 (http://2014.eurobsdcon.org/), InterExpo Congress Center, Sofia, Bulgaria 25 - 28 September, 2014. EuroBSDcon is the premier European conference on the open source BSD operating systems attracting about 250 highly skilled engineering professionals, software developers, computer science students and professors, and users from all over Europe and other parts of the world. The goal of EuroBSDcon is to exchange knowledge about the BSD operating systems, facilitate coordination and cooperation among users and developers. The dates for EuroBSDCon 2014 in Sofia have been set to September 25-26th for tutorials and September 27-28th for the main conference.

PC-BSD Weekly Feature Digest 23

Hey PC-BSDers! This week we’re coming at you with some pretty sweet updates to PC-BSD. The mount tray has seen some significant improvement and is now able to mount most audio / dvd formats without a problem. Also windows partition types are now showing up correctly on my test system after building the new mount tray from source. The mount tray will also prompt you to open your disc with a program and will offer you correct suggestions based on the proper package / PBI. Ultimately the mount tray will most likely replace the built in mounting systems in the desktop environments. This is still a little ways off in the future, but the direction we are heading in.

We heard that there were some users that were experiencing problems upgrading and believe we have found the guilty party. I was able to duplicate the same package upgrade problem that was causing updates to 10.0.1 to fail, and asked Allan over at Scale Engine to give us a hand. Allan was able to track down the issue to a faulty distribution server that was interrupting connections and preventing the upgrades randomly. This server has been removed from service at this time and further work is going into preventing this from happening again in the future.

Work has begun to localize PC-BSD into the Hindi language. We’d like to give a shout out to the newest member of our translation team Simran. Thanks for your help and we are excited at the prospect for even more people to be able to use PC-BSD. Our estimated date of completion is 3 weeks from now. If you have an interest in this language please help us spread the word!

Other News / Projects for this week:

* Merged latest ports and gnome3 patches into ‘master’
* Merged in latest VirtualBox versions
* Wrote a userland replacement for the FUSE module to execute PBIs in a faster and less unstable manner (about 90% complete)
* Kicked off new –STABLE builds
* Update 9.x PBI’s
* Add new XDG-compatibility classes in libpcbsd (scanning/listing/filtering system applications)
* New Utility: pc-systemflag (shell) — pc-systemflag is used to set a flag/message on the system for cross-application communication
* Rewrite the pc-systemupdatertray utility to use the new SystemFlagWatcher. Is much simpler and more streamlined now.
* Add system flag usage to pc-softwaremanager for PBI update availability
* Add system flag usage to the pbi-manager (“pbi_update –check-all” usage only)
* Add system flag usage to pc-updatemanager (for all package and system updates/checks)

bsdtalk239 – PkgNG with Baptiste Daroussin at vBSDCon

A recording of Baptiste Daroussin speaking at vBSDCon in October 2013.  He is a FreeBSD source committer and project developer for PkgNG.  PkgNG is a package management tool for FreeBSD. It is the replacement for the current pkg_info/pkg_create/pkg_add tools.

File Info: 55Min, 26MB.

Ogg Link: http://cis01.uma.edu/~wbackman/bsdtalk/bsdtalk239.ogg