Foundation at Texas LinuxFest

There will be a FreeBSD booth (booth #21) in the Expo area at Texas LinuxFest, to be held at the AT&T Conference Center in Austin, Texas on Saturday June 1. Registration is required for this event at a cost of $25 or $55.

Stop by the booth to discuss the Foundation's projects, check out the cool swag, or to make a donation to the FreeBSD Foundation.

PC-BSD at Texas LinuxFest

Several members of the PC-BSD team will be at Texas LinuxFest, to be held at the AT&T Conference Center in Austin, Texas from Friday May 31 to Saturday June 1. Registration is required for this event at a cost of $25 or $55.

There will be a BSD booth in the Expo (booth #21) on June 1 where we’ll be giving out PC-BSD DVDs, FreeNAS CDs and other cool swag. We can also accept donations to the FreeBSD Foundation.

Kris Moore will present “Automating the deployment of FreeBSD & PC-BSD® systems” at 11:25 on June 1. Dru Lavigne will present “TrueOS and Warden: Easy to Deploy FreeBSD Servers” at 13:30 on June 1.

The BSDA certification exam will be held at 13:00 on May 31. The cost for the exam is $75.

 

Craig Rodrigues, FreeBSD developer 2013-05-23 21:31:15

From May 15 – 18, I attended BSDCan and FreeBSD DevSummit in Ottawa, Canada.
These were both excellent conferences.

There were many excellent presentations during this conference, but my favorites were:

With these two technologies, the bar has definitely been raised for FreeBSD to be used in applications which require storage (ZFS) and virtualization (BHyve).

There were more presentations that I wanted to attend, but there was just not enough time!

BSDCan Trip Report: Eitan Adler

The FreeBSD Foundation sponsored 7 attendees of BSDCan 2013. The first trip report is from Eitan Adler, a doc committer, who attended the BSDCan Developers Summit. Eitan writes:

I arrived Tuesday night and met Colin Percival at the airport.  After dropping off luggage at the university, I met up with some of the other developers.

The first day, I attended the "Netflix and FreeBSD" session run by Scott Long.  It was interesting to see what kind of problems users of FreeBSD ran into when running at scale.

For the afternoon working group, I chose to attend the "ports and packages" session.  A variety of topics were discussed but the most discussed topic was cross-building ports across both versions and architectures.  This is a topic that came up repeatedly in prior
discussion and that would come up again in other working groups, so it was good to know about the latest work in this area.

The vendor summit came next.  In the past, the vendor summit focused on kernel work but this one revolved around the user land.  This is particularly important to me as I run FreeBSD on my laptop as my primary development machine.

At night I spent some time in the hacking lounge or other shared areas meeting people.  It was very nice to be able to meet the people I've been talking to for the past three years.

On Thursday I spent my morning in the "Desktop" session.  Getting FreeBSD running well on desktops is critical in attracting new developers in the future.  Kris Moore, from PCBSD, spoke a lot about the customizations that they made.  I pressed to share the improvements
that could be committed upstream.  Other issues discussed were packaging for the desktop and a graphical boot loader for FreeBSD/PC-BSD.

The afternoon session for me was "Documentation": a significant portion of the discussion was about the future print edition of the book and what sections need to be updated and improved.  In particular, how we could get more source committers involved in writing documentation.  We also discussed how to work going forward with other teams that need access to the documentation (e.g., portmgr and postmaster).   We also touched on the FAQ, translations, and the new toolchain.  The final topic we discussed was the automated QA and statistics tools we have (and don't have) and how we could improve in that area.

After dinner I did some work at the documentation hackathon.  I spent the remainder of the night at the hacker lounge discussing kernel internals with Peter Wemm, Sean Bruno, and others.

Unfortunately, I had to leave prior to the conference itself, but I felt that meeting people at the developer summit was well worth the time spent.

brd’s notes » FreeBSD 2013-05-21 16:58:17

First thing we will need a clone of https://github.com/xaque208/puppet-pkgng into /usr/local/etc/puppet/modules/.

This will be pushed out to the clients as long as: pluginsync = true

For me the next step is to create a manifests/init.pp in the new module directory. This is important to me because I want to sync out a /usr/local/etc/pkg.conf to all my machines so that they point to my internal poudriere repos. So I end up with something like this:

file { "/usr/local/etc/pkg.conf":
        mode => 755,
        owner => root,
        content => "packagesite: http://pkg/91-web/
",
}

Once that is done it is easy to use pkgng packages via:

package { "www/apache22":
        ensure => installed,
        provider => pkgng,
        require => File['/usr/local/etc/pkg.conf'],
}

PC-BSD Hardware Store

Josh Smith has announced the initial launch of the PC-BSD hardware store. This resource is meant to make it easier to find hardware that has been tested to work on PC-BSD.  The store itself is here. If you have hardware that you know works and which you don’t see listed, please add it to the Hardware section of the wiki so that it can be included.

FreeBSD Audio, quirks and such

At BSDCan this week, so I harassed one of the folks who maintains audio drivers. 

My Lenovo T520 has a good audio device on it, but I noticed that I could not get the onboard, external microphone to work.  I *could* record if I use a 4 position headset/microphone.  After a few moments, I learned quite a bit about the extensive FreeBSD hints/quirks system.  The sound devices supported by snd_hda has many, many, MANY things that can be adjusted.  I'll try and figure out how others can fix things, but for now, here's a set of quirks that binds both microphone input and fixes my problems.

loader.conf:
hint.hdaa.1.nid35.config="as=2"
hint.hdaa.1.nid27.config="as=2 seq=15"

brd’s notes » FreeBSD 2013-05-17 19:47:19

This is a bunch of links to the tools I talk about in my presenation

Tools:

Collectd: https://collectd.org/

Graphite: http://graphite.wikidot.com/
Nagios: http://www.nagios.org/

Poudriere: http://fossil.etoilebsd.net/poudriere

Config Management:
Salt Stack: http://saltstack.com/
Chef: http://www.opscode.com/chef/
Puppet: http://puppetlabs.com/

Subversion: http://subversion.apache.org/

LogStash: http://logstash.net/
Audit: http://www.freebsd.org/handbook/audit.html

CARP: http://www.freebsd.org/handbook/carp.html

OATH: http://www.openauthentication.org/

Serial Console: http://www.freebsd.org/handbook/serialconsole-setup.html

Generic Resources:
FreeBSD Handbook: http://freebsd.org/handbook
Everything Sysadmin Blog: http://everythingsysadmin.com/resources.html

FreeBSD Audio, Jack and gstreamer

I'm in the process of making it possible to move completely to FreeBSD for my primary workstation at home.  Currently, I'm using Linux for its ability to run IDJC http://idjc.sourceforge.net for its ability to let me be a shoutcast internet DJ for Radio KoL.  This has led me down into the FreeBSD Audio System a bit, which I'm exploring.

I normally run FreeBSD Current, so I came up with a couple of adjustments for my onboard microphones at BSDCan this year after I cornered the hdaa(4) audio maintainer and showed him my confusing.  So, for now, I can playback and record with audacity just fine.  This does not use the Jack audio system that is required by IDJC.

Jack installed from ports needs a .jackdrc --> "/usr/local/bin/jackd -r -T -d oss -p 2048"

You can test playback with the gstreamer jack plugin in ports along with my favorite audio player, Clementine.

bsdtalk225 – PC-BSD with Kris Moore

Interview during BSDCan 2013 with Kris Moore from iXsystems.  We talk about some of the new features of PC-BSD.

File Info: 12Min, 6MB.

Ogg Link
http://cis01.uma.edu/~wbackman/bsdtalk/bsdtalk225.ogg

What keramida said… » FreeBSD 2013-05-16 17:04:58

Regular expressions are a powerful text transformation tool. Any UNIX geek will tell you that. It’s so deeply ingrained into our culture, that we even make jokes about it. Another thing that we also love is having a powerful extension language at hand, and Lisp is one of the most powerful extension languages around (and of course, we make jokes about that too).

Emacs, one of the most famous Lisp applications today, has for a while now the ability to combine both of these, to reach entirely new levels of usefulness. Combining regular expressions and Lisp can do really magical things.

An example that I recently used a few times is parsing & de-humanizing numbers in dstat output. The output of dstat includes numbers that are printed with a suffix, like ‘B’ for bytes, ‘k’ for kilobytes and ‘M’ for megabytes, e.g.:

----system---- ----total-cpu-usage---- --net/eth0- -dsk/total- sda-
     time     |usr sys idl wai hiq siq| recv  send| read  writ|util
16-05 08:36:15|  2   3  96   0   0   0|  66B  178B|   0     0 |   0
16-05 08:36:16| 42  14  37   0   0   7|  92M 1268k|   0     0 |   0
16-05 08:36:17| 45  11  36   0   0   7|  76M 1135k|   0     0 |   0
16-05 08:36:18| 27  55   8   0   0  11|  67M  754k|   0    99M|79.6
16-05 08:36:19| 29  41  16   5   0  10| 113M 2079k|4096B   63M|59.6
16-05 08:36:20| 28  48  12   4   0   8|  58M  397k|   0    95M|76.0
16-05 08:36:21| 38  37  14   1   0  10| 114M 2620k|4096B   52M|23.2
16-05 08:36:22| 37  54   0   1   0   8|  76M 1506k|8192B   76M|33.6

So if you want to graph one of the columns, it’s useful to convert all the numbers in the same unit. Bytes would be nice in this case.

Separating all columns with ‘|’ characters is a good start, so you can use e.g. a CSV-capable graphing tool, or even simple awk scripts to extract a specific column. ‘C-x r t’ can do that in Emacs, and you end up with something like this:

|     time     |cpu|cpu|cpu|cpu|cpu|cpu|eth0 |eth0 | disk| disk|sda-|
|     time     |usr|sys|idl|wai|hiq|siq| recv| send| read| writ|util|
|16-05 08:36:15|  2|  3| 96|  0|  0|  0|  66B| 178B|   0 |   0 |   0|
|16-05 08:36:16| 42| 14| 37|  0|  0|  7|  92M|1268k|   0 |   0 |   0|
|16-05 08:36:17| 45| 11| 36|  0|  0|  7|  76M|1135k|   0 |   0 |   0|
|16-05 08:36:18| 27| 55|  8|  0|  0| 11|  67M| 754k|   0 |  99M|79.6|
|16-05 08:36:19| 29| 41| 16|  5|  0| 10| 113M|2079k|4096B|  63M|59.6|
|16-05 08:36:20| 28| 48| 12|  4|  0|  8|  58M| 397k|   0 |  95M|76.0|
|16-05 08:36:21| 38| 37| 14|  1|  0| 10| 114M|2620k|4096B|  52M|23.2|
|16-05 08:36:22| 37| 54|  0|  1|  0|  8|  76M|1506k|8192B|  76M|33.6|

The leading and trailing ‘|’ characters are there so we can later use orgtbl-mode, an awesome table editing and realignment tool of Emacs. Now to the really magical step: regular expressions and lisp working together.

What we would like to do is convert text like “408B” to just “408″, text like “1268k” to the value of (1268 * 1024), and finally text like “67M” to the value of (67 * 1024 * 1024). The first part is easy:

M-x replace-regexp RET \([0-9]+\)B RET \1 RET

This should just strip the “B” suffix from byte values.

For the kilobyte and megabyte values what we would like is to be able to evaluate an arithmetic expression that involves \1. Something like “replace \1 with the value of (expression \1)“. This is possible in Emacs by prefixing the substitution pattern with \,. This instructs Emacs to evaluate the rest of the substitution pattern as a Lisp expression, and use its string representation as the “real” substitution text.

So if we match all numeric values that are suffixed by ‘k’, we can use (string-to-number \1) to convert the matching digits to an integer, multiply by 1024 and insert the resulting value by using the following substitution pattern:

\,(* 1024 (string-to-number \1))

The full Emacs command would then become:

M-x replace-regexp RET \([0-9]+\)k RET \,(* 1024 (string-to-number \1)) RET

This, and the byte suffix removal, yield now the following text in our Emacs buffer:

|     time     |cpu|cpu|cpu|cpu|cpu|cpu|eth0 |eth0 | disk| disk|sda-|
|     time     |usr|sys|idl|wai|hiq|siq| recv| send| read| writ|util|
|16-05 08:36:15|  2|  3| 96|  0|  0|  0|  66| 178|   0 |   0 |   0|
|16-05 08:36:16| 42| 14| 37|  0|  0|  7|  92M|1298432|   0 |   0 |   0|
|16-05 08:36:17| 45| 11| 36|  0|  0|  7|  76M|1162240|   0 |   0 |   0|
|16-05 08:36:18| 27| 55|  8|  0|  0| 11|  67M| 772096|   0 |  99M|79.6|
|16-05 08:36:19| 29| 41| 16|  5|  0| 10| 113M|2128896|4096|  63M|59.6|
|16-05 08:36:20| 28| 48| 12|  4|  0|  8|  58M| 406528|   0 |  95M|76.0|
|16-05 08:36:21| 38| 37| 14|  1|  0| 10| 114M|2682880|4096|  52M|23.2|
|16-05 08:36:22| 37| 54|  0|  1|  0|  8|  76M|1542144|8192|  76M|33.6|

Note: Some of the columns are indeed not aligned very well. We’ll fix that later. On to the megabyte conversion:

M-x replace-regexp RET \([0-9]+\)M RET \,(* 1024 1024 (string-to-number \1)) RET

Which produces a version that has no suffixes at all:

|     time     |cpu|cpu|cpu|cpu|cpu|cpu|eth0 |eth0 | disk| disk|sda-|
|     time     |usr|sys|idl|wai|hiq|siq| recv| send| read| writ|util|
|16-05 08:36:15|  2|  3| 96|  0|  0|  0|  66| 178|   0 |   0 |   0|
|16-05 08:36:16| 42| 14| 37|  0|  0|  7|  96468992|1298432|   0 |   0 |   0|
|16-05 08:36:17| 45| 11| 36|  0|  0|  7|  79691776|1162240|   0 |   0 |   0|
|16-05 08:36:18| 27| 55|  8|  0|  0| 11|  70254592| 772096|   0 |  103809024|79.6|
|16-05 08:36:19| 29| 41| 16|  5|  0| 10| 118489088|2128896|4096|  66060288|59.6|
|16-05 08:36:20| 28| 48| 12|  4|  0|  8|  60817408| 406528|   0 |  99614720|76.0|
|16-05 08:36:21| 38| 37| 14|  1|  0| 10| 119537664|2682880|4096|  54525952|23.2|
|16-05 08:36:22| 37| 54|  0|  1|  0|  8|  79691776|1542144|8192|  79691776|33.6|

Finally, to align everything in neat, pipe-separated columns, we enable M-x orgtbl-mode, and type “C-c C-c” with the pointer somewhere inside the transformed dstat output. The buffer now becomes something usable for pretty-much any graphing tool out there:

| time           | cpu | cpu | cpu | cpu | cpu | cpu |      eth0 |    eth0 |  disk |      disk | sda- |
| time           | usr | sys | idl | wai | hiq | siq |      recv |    send |  read |      writ | util |
| 16-05 08:36:15 |   2 |   3 |  96 |   0 |   0 |   0 |        66 |     178 |     0 |         0 |    0 |
| 16-05 08:36:16 |  42 |  14 |  37 |   0 |   0 |   7 |  96468992 | 1298432 |     0 |         0 |    0 |
| 16-05 08:36:17 |  45 |  11 |  36 |   0 |   0 |   7 |  79691776 | 1162240 |     0 |         0 |    0 |
| 16-05 08:36:18 |  27 |  55 |   8 |   0 |   0 |  11 |  70254592 |  772096 |     0 | 103809024 | 79.6 |
| 16-05 08:36:19 |  29 |  41 |  16 |   5 |   0 |  10 | 118489088 | 2128896 |  4096 |  66060288 | 59.6 |
| 16-05 08:36:20 |  28 |  48 |  12 |   4 |   0 |   8 |  60817408 |  406528 |     0 |  99614720 | 76.0 |
| 16-05 08:36:21 |  38 |  37 |  14 |   1 |   0 |  10 | 119537664 | 2682880 |  4096 |  54525952 | 23.2 |
| 16-05 08:36:22 |  37 |  54 |   0 |   1 |   0 |   8 |  79691776 | 1542144 |  8192 |  79691776 | 33.6 |

The trick of combining arbitrary Lisp expressions with regexp substitution patterns like \1, \2 … \9 is something I have found immensely useful in Emacs. Now that you know how it works, I hope you can find even more amusing use-cases for it.

Update: The Emacs manual has a few more useful examples of \, in action, as pointed out by tunixman on Twitter.


Filed under: Computers, Emacs, Free software, FreeBSD, GNU/Linux, Lisp, Open source, Programming, Software Tagged: Computers, Emacs, Free software, FreeBSD, GNU/Linux, Lisp, Open source, Programming, Software

FreeBSD Foundation Announces Ed Maste is New Director of Project Development

The FreeBSD Foundation is pleased to announce Ed Maste's new role as the Foundation's part-time Director of Project Development.  Ed has served on the Foundation's board for two years, and has stepped down in order to accept this new position.

In this position, Ed will manage the Foundation's sponsored work, including projects funded under specific grants, operational support and project development undertaken by the Foundation's permanent technical staff. Working with the Foundation's Board of Directors, Ed will identify and document specific areas of future project work interest. This roadmap planning will include coordination with FreeBSD consumers and the FreeBSD community.

"2012 represented an inflection point in the Foundation's history,'' said Justin T. Gibbs, President of the FreeBSD Foundation. "The Foundation has a stated goal of investing in permanent staff through 2013. With Ed taking on this new position I'm excited by the Foundation's increased capacity to manage our project development and operational support.''

Ed has over ten years of experience in companies building products on FreeBSD, in both technical and managerial roles. He resides in Kitchener, Ontario, Canada.