On testing, part III

I just got word of an embarrassing bug in OpenPAM Nummularia. The is_upper() macro, which is supposed to evaluate to true if its argument is an upper-case letter in the ASCII character set, only evaluates to true for the letter A:

#define is_upper(ch)                            \
        (ch >= 'A' && ch <= 'A')

This macro is never used directly, but it is referenced by is_letter(), which is referenced by is_pfcs(), which is used to validate paths and path-like strings, i.e. service names and module names or paths. As a consequence, OpenPAM does not support services or modules which contain an upper-case letter other than A. I never noticed because a) none of the services or modules in use on the systems I use to develop and test OpenPAM have upper-case letters in their names and b) there are no unit or regression tests for the character classification macros, nor for any code path that uses them (except openpam_readword(), which uses is_lws() and is_ws()).

The obvious course of action is to add unit tests for the character classification macros (r760) and then fix the bug (r761). In this case, complete coverage is easy to achieve since there are only 256 possible inputs for each predicate.

I have merged the fix to FreeBSD head (r262529 and r262530). Impatient users can fix their system by running the following commands:

% cd /usr/src/contrib/openpam
% svn diff -r758:762 svn://svn.openpam.org/openpam/trunk | patch
% cd /usr/src/lib/libpam/libpam
% make && make install

Unsurprisingly, writing more unit tests for OpenPAM is moving up on my TODO list. Please contact me if you have the time and inclination to help out.

PC-BSD Weekly Feature Digest 18

Big news this week!   Kris has finished re-writing the code for handling how PC-BSD handles major updates.  The general consensus was that there were still many users out there that were having difficulty when upgrading to a new version (i.e 9.2 > 10).  For more information on the new PC-BSD update system, check out the article here.  To view Kris’ address to the testing mailing list click here.  Our goal is and has always been to have a reliable system to push out updates, and we think you will all be very pleased with the results!

Basic guidelines for PC-BSD utilities are continuing to evolve and we want your input!  If you have ideas for the development team on what should become standard practice we want to hear from you.  You can join the discussion on the the PC-BSD developer mailing list if you’d like to submit your ideas.  To check out Guidelines for PC-BSD utilities visit the PC-BSD 10.1 wiki’s “Become a Developer” section @ http://wiki.pcbsd.org/index.php/Become_a_Developer/10.1

A moderate batch of fixes for trac tickets were committed this week addressing minor issues with the warden.  Appcafe has also received a big update to it’s user interface allowing for a smoother experience for users.  If you want to give the 2 newest versions of these programs a try grab the PC-BSD source and let us know what you think!

That’s it for this week!  See you all next time!

Call For Testers: New Major Upgrade Methodology

Hello everyone!

Kris just posted a call for testers for a new methodology for major system upgrades (such as  9.2 -> 10.0 -> 10.1, etc..) and we are looking for people who are still running 9.2  to try it out. The full text of the call for testers is at the bottom of the post, but to give a bit of background we have been generally unimpressed with the reliability of the “standard” FreeBSD update tools (freebsd-update and pkg) when it comes to fetching uncorrupted update files through the internet. This new methodology takes those two utilities out of the general preparations for an update (download/verification of files), as well as a couple other upgrade steps so that there should no longer be an issue with starting an upgrade when only some of the upgrade files were actually retrieved successfully.

 

Please test it out and let us know how it goes!

Remember, always backup your data before doing any major upgrade like this!  The new methodology should automatically create a boot environment for you before doing the upgrade, but better safe than sorry!

 

Here is the full text of request from Kris to the developer mailing list:

I've just finished up some work on a major re-write of our updating
system when "upgrading" between major releases, I.E. 9.2 -> 10.0, 10.0
-> 10.1.

https://github.com/pcbsd/pcbsd/commit/b95e8a83c73511568ae4291a54e0f93f6c67ef30
https://github.com/pcbsd/pcbsd/commit/9a8b3d1945fa67db8e99b0e4e82280b5626aa895

It seems to work well here, but it needs some additional testing from
any users still running 9.2 who want to update to 10. To test this,
first grab the latest from git:

# git clone https://github.com/pcbsd/pcbsd.git pcbsd
# cd pcbsd/src-sh/pc-updatemanager
# make install

Then run:

# pc-updatemanager install fbsd-10.0-RELEASE

This should start the download / upgrade process. If anything fails
during the process, logs are kept in /root, which will assist me in
debugging.

Thanks!

PC-BSD Weekly Feature Digest 17

The PC-BSD development team has been abuzz this week with awesome suggestions on how we can standardize the way we write PC-BSD utilities and software.  One thing we’ve begun to realize is that as more people are contributing to the project, it is ever more important to make sure that there are clear standards for development.  Even our primary developers will admit it’s easy to forget to use the same icon pack, or file menu layout when you get busy writing the main program.  Going forward you can expect these standards to impact most of the PC-BSD utilities and programs you use everyday, although in a relatively minor way.  Everything will still function the exact same, but whether or not you are using AppCafe or the Warden you can expect the file menu layout / program layout to follow the same general rules.  For more information please check out “Becoming a Developer� in the PC-BSD 10.1 wiki.  If you’d like to join the discussion you can email [email protected]

I’ve seen some discussion lately about the life cycle of PC-BSD branches.  I sat down with Kris Moore in IRC and asked if he wouldn’t mind clarifying the release cycle for our users.   Kris answered the general rule of thumb you can use is a branch will continue to be supported for 6 months after the next branch is released.  The updates include all of the things you would expect like new PBI and security updates.  So for users of 9.2 you can expect support to continue through June of 2014.  9 Stable was a “experimental� branch and is no longer supported at this time.  Users of 9 Stable are encouraged to upgrade to 9.2 or 10.0 Release to continue to receive important updates.

You can expect to see tons of improvements coming up for PC-BSD 10.1.   One of the biggest being Kris and Yuri have been working to fix Linux jail support in the Warden.  A handful of commits went into the tree today that will address the previous problems users have been having with Linux jails.  Kris has continued to refine the Warden and PBI systems to fix some bugs that were causing major stability issues in certain scenarios.   Minor cosmetic changes are coming for most PC-BSD utilities to bring them up to the same standards outlined in the “Become a Developer� section in the PC-BSD 10.1 wiki.

That’s it for this week folks.  Lots of good things in the works so stay tuned to the blog for more important PC-BSD news!

 

bsdtalk238 – Voices from NYCBSDCon 2014

NYCBSDCon 2014 was great, and I recorded a series of voices from the attendees.

Also, the FreeBSD Journal release was announced at the conference.

File Info: 9Min, 4Mb.

Ogg Link: http://cis01.uma.edu/~wbackman/bsdtalk/bsdtalk238.ogg


The FreeBSD Journal is Now Available!

We are pleased to announce the FreeBSD Journal is now available! This is a new, FreeBSD focused, online publication. 

You can find out how to subscribe to the Journal by going to www.freebsdjournal.com. Or, go to the following links for the device you'd like to download to:

To get the Kindle App click here 
To get the Apple app click here 
To get the Android App click here

Here's the letter the editorial board wrote for this inaugural issue:

Welcome to the first issue of the FreeBSD Journal. A brand new, professionally produced, on-line magazine available from the various app stores, including Apple iTunes, Google Play, and Amazon Kindle.

What you will find in the upcoming issues are feature-length articles and columns that address the entirety of the FreeBSD community. Coverage will be diverse with topics including: managing large scale system deployments, application development, systems programming, embedded systems, academic research, and general software development. All using FreeBSD.

Issue #1 is dedicated to FreeBSD 10, the latest in the FreeBSD Project's line of major releases.  The release of FreeBSD 10 brings many new features not seen in other open source operating systems, including a brand new compiler toolchain based on LLVM, as well as mature support for ZFS as a first class kernel-based filesystem.

The Journal is guided by an editorial board made up of people from across the FreeBSD community, including, John Baldwin, Daichi Goto, Joseph Kong, Dru Lavigne, Michael Lucas, Marshall Kirk McKusick, George Neville-Neil, Hiroki Sato, and Robert Watson.  The editorial board is responsible for the acquisition and vetting of content for the magazine.  The editor is Jim Maurer who is the person with whom our authors work most closely getting their pieces into shape for publishing.  Jim has a long history in the technical publishing business, having worked with Scientific American and the Association for Computing Machinery.

The magazine will publish six issues per year.  The editorial board has planned the first year of issues that will cover topics including Networking, Virtualization, Development Tools, Support for new Hardware Features, and the new software packaging system for FreeBSD, pkgng.

The FreeBSD Journal is financially supported by the FreeBSD Foundation, the 501(c)3 charitable organization whose sole purpose is to help the FreeBSD Project grow and flourish.  Your subscriptions and the advertising revenue the Journal receives help to offset the money the Foundation needs to put in to support the Journal.

We know you'll like what you see in the Journal and hope that you'll let everyone you know about our new magazine.

Sincerley,
FreeBSD Journal Editorial Board

FreeBSD Journal First Edition Available

We are pleased to announce the availability of the FreeBSD Journal! It is the new online Journal, that the FreeBSD Foundation is publishing, that is all about FreeBSD. Click here to find out how to get the first issue that is focused on FreeBSD 10.

PC-BSD Weekly Feature Digest 16

Just a quick update for you fine folks this week!  The bulk of the last week was spent fixing many smaller bugs that were still stuck in the PC-BSD trac database. Our primary goal now that 10.0 has hit release status is to fine tune the PC-BSD experience to allow for the smoothest experience possible. Users are encouraged to submit even the most trivial bugs during this “fine tuning phase�.

Among the programs that received updates the Warden received a patch fixing some issues with warnings, creating jails, and fixing user interface inconsistencies. Also the Package Manager received updates to fix some issues with how it interacts with pkgng.  This should eliminate some of the minor inconsistencies some of you were seeing that was leading to communication issues between the two. Package manager and the User manager also received minor improvements to fix issues relating to their user interface.

Big news this week for the PBI format. Kris has confirmed that a patch has been implemented for the way PBI’s are being built that will shrink the size of PBI’s by approximately 50%. The patch has just been implemented and is currently in a testing phase, but stay tuned to the blog for more updates on how and when this patch will begin benefiting you in the very near future!

Thats it for this week folks.  As always thanks for being loyal PC-BSD users!

httperf tuning for #FreeBSD testing

Was playing around with httperf to excercise Apache / stunnel SSl benchmarks on FreeBSD this week and ran into the code that nerfs simultaneous connections down from the environment ulimit of maxfiles to the limit FD_SETSIZE as defined in <select.h>.

One can override this at compile time and push the system harder by passing in some ./configure foo:

env CC=”cc -DFD_SETSIZE=4096″

However, you will then be able to max out the number of ports in use very quickly if you try to use stunnel and apache in this configuration.  I noted that on our systems we raise the low port number and reduce the high port number for connections:

net.inet.ip.portrange.first=20000

net.inet.ip.portrange.last=49151

I set first down to 2000 and last up to 65534 for my testing.  This gives me quite a bit more ports to use in testing.  At this point I can run stunnel on 443 forwarding to apache on localhost:80 and get more than 8k simultaneous connections when using SSL accelerators on FreeBSD 10

 

Time to bid farewell to the old pkg_ tools

There comes a time in the life cycle of just about every software package that it has bee re-evaluated, refreshed, deprecated or just retired.

It is time that we bid farewell to the old pkg_* software that has been part of FreeBSD since the beginning, and has served us well. After years of development, testing, and playing, pkg(8) has become a suitable replacement.

Pkg is the Next Generation package management tool for FreeBSD. It is the replacement for the current pkg_info/pkg_create/pkg_add tools that ports use to register local packages and which provide remote packages. Its main goals are to facilitate remote binary package upgrades. It also works with ports without remote binary packages.

Pkg, combined with the quarterly release package sets, enables easy installation and safe upgrades for binary packages. Signed, binary packages are available for all supported FreeBSD releases on the i386 and  amd64 platforms from pkg.freebsd.org. Additionally, for those compiling ports from source, pkg’s new database format gives more fine-grained querying and management of installed software.

New features on the drawing board, like automatic pkg-plist generation, sub-packages, creating multiple packages containing different parts of a port from one build process, and flavours, being able to ask for e.g. a webserver, without directly specifying a specific one, cannot be implemented in the old pkg_* tools and those plans are currently on hold.

You are not obligated to switch to binary packages, if you still prefer to compile your own ports, it is a simple matter of installing ports-mgmt/pkg, run pkg2ng, add WITH_PKGNG=yes to your make.conf and use pkg <action> instead of pkg_<action>.

You can read more about pkgng on the FreeBSD wiki, https://wiki.freebsd.org/pkgng.

The decision has been made to allow the old pkg_* software to be EoL’d 6 months from now, at September 1, 2014 in all active FreeBSD branches.

Please start testing pkg(8) in your test environments before taking it live, you will find the benefits of full binary updates for your ports to be beneficial in a very short amount of time. Even if you prefer to compile from source, you will still reap the benefits of the modern packaging system.

whoami? I’m PC-BSD 10.0! — Weekly Feature Digest 15

The PC-BSD team has announced version 10.0 Joule Edition is now in official release status and is available for mass distribution. If you’ve used PC-BSD in the past you haven’t seen anything yet! PC-BSD 10.0’s feature rich front end runs beautifully without sacrificing the stability we all know and love from FreeBSD’s base system. This week we’re going to look back and highlight some of best new features now available in PC-BSD Version 10.0 — Joule Edition.

AMD / ATI graphics are now officially supported under PC-BSD. Although in the past AMD Radeon users have suffered from lack-of-driver-itis, those days are passed and a new age is upon us. I’m currently running PC-BSD on 3 different types of AMD graphics based systems and the performance is arguably better than some of my nvidia based test systems (sorry Nvidia fanboys :D). I was even more surprised when ATI hybrid graphics worked out of the box on my Samsung NP-350 Notebook computer.  Especially considering this laptop was designed for…Windows 8…*BARF*

0131142101a

You will notice there are a couple of quirks to be aware of with the new Radeon drivers that we are currently aware of. Firstly the CTRL F keys do not bring up the console. This is being actively remedied by FreeBSD and we expect there will be a fix committed in the near future. Also KDE support for Radeon is there, but i’ve noticed that I experience random freeze ups from time to time on any of my Radeon systems. Using any other desktop environment seemed to fix the issue for me.

More of your favorite Desktop Environments! We’ve now added Gnome 3, Cinnamon, and mate to the list of desktops available under PC-BSD 10.0. Mate has officially replaced Gnome 2 in version 10.0, but Gnome 3 and Cinnamon will remain as “unsupported� desktops currently until further development and additional testing are done to bring them up to speed in PC-BSD. That being said feel free to report issues with these desktops with the understanding that they are currently in a “beta� stage and are offered without any guarantee. For best results with your PC-BSD experience use supported desktops!

The Grub and FreeBSD boot loaders have both received updated support to work with PC-BSD version 10.0. Have your choice now during installation for whatever boot loader works best for you. As we continue to work with the Grub boot loader our support is evolving simultaneously to give you even better compatibility.

In other news this week if any users are still using RC’s 1–5 please update to PC-BSD 10.0 — Release before reporting bugs so we can make sure we aren’t duplicating tickets for issues that have already been submitted and /or resolved.

Special thanks to the entire PC-BSD team including our testers, committers, and administrators. Without you guys this release wouldn’t be what it is today and we sincerely thank you all for your dedication in making PC-BSD Joule Edition a success.

Best Regards,

Josh

 

 

FreeBSD Foundation Announces 2013 Fundraising Results

The board of directors had a very productive meeting in Berkeley in January. We worked on our strategic plan and other topics that greatly benefited by working together at an in-person meeting.

We thank you, the FreeBSD community and consumers, who support FreeBSD in many ways.  Thank you for your donations ranging from $5 to over $200,000. Thank you for your tireless efforts of supporting the Project by developing code, writing for and about FreeBSD, helping with conferences and summits, and advocating for FreeBSD.




We have our final fundraising results from 2013 and are pleased with the results. In 2013 we raised $768,562 from 1659 donors. If you compare the number with 2012, of $771,193 from 1855 donors, it was a little lower. The difference is primarily because 2012 blew out its goal of raising $500,000.

We have already started our 2014 fundraising efforts. As of the end of January we are just under $40,000. Our goal is to raise $1,000,000. We are currently finalizing our 2014 budget. We plan to publish both our 2013 financial report and our 2014 budget soon.

Please consider making a donation in 2014! It's easy, just click here to make your donation!

Thank you again for your support! We look forward to continuing our support of the FreeBSD Project and community in 2014.

PC-BSD 10.0-RELEASE is Now Available

PC-BSD 10.0-RELEASE is now available for download!

A special thank you to all the developers, testers, translators and docs team members who helped make this release possible.

10.0-RELEASE notable features

  • Includes FreeBSD 10.0-RELEASE
  • Updated KMS / AMD driver support
  • ISO file is a hybrid USB file, and can be “dd“ed to a USB media.
  • New text-based installer
  • Able to select between GRUB/BSD loaders during installation
  • New desktops! Gnome 3, Mate (Replaces Gnome2) and Cinnamon

For a more complete list, checkout the What’s New in 10.0.

Updating

Online updating to 10.0-RELEASE is now available for users running 9.2-RELEASE systems. As with any upgrade, please make sure to backup your critical data beforehand.

To get started, first ensure that your packages and world/kernel are up to date, and then apply the update to 10.0 using the following command at a root prompt:

pc-updatemanager install fbsd-10.0-RELEASE

NOTE: If you are on an older 9.2 install that still uses UFS, you will want to install 10.0 fresh and move to ZFS. Many of our new tools and utilities depend upon ZFS functionality.

Reporting Issues

Found a bug? Please report it, in as much detail as possible, to our Trac database.

The short list #5: coredumping with sudo on #FreeBSD

Things I learned from a misbehaving pam module managing our sudo context at work.  sudo, for security, will not dump core files if it hits a segfault.  You need to tell the kernel to allow set uid root binaries to core dump *and* you have to let sudo know that its ok via a sudo.conf entry.

DO NOT LEAVE THESE AS DEFAULTS

kern.sugid_coredump: 1

/etc/sudo.conf –> Set disable_coredump true

ref –> http://www.sudo.ws/sudo.man.html

 

PC-BSD Weekly Feature Digest

For those of you waiting for PC-BSD 10 to become a reality, the wait is almost over.  The PC-BSD team announced last Thursday that the project is now in 10.0 Release Candidate 5 (p4), and is available for mass consumption.  There are a couple of important things to remember when upgrading to the latest RC.  Kris announced that all users upgrading from RC4 or previous versions, will unfortunately have to upgrade all of their packages and PBI’s due to a recent ABI change in FreeBSD.  For more information on how this will impact you and what you can do to have a smooth transition, please see the PC-BSD 10.0 RC5 release notes here.

Also due to the ABI change and the inability to use our previously approved PBI’s for PC-BSD 10.0 RC5, PBI’s had to be completely rebuilt and re-approved for use in the version 10.0 AppCafe.  The PC-BSD team was able to get nearly 500 PBI’s rebuilt and re approved at the end of Friday the 24th, but there are still many more that will begin to show up in the AppCafe very soon.

 

Builds have started for final release, so keep your eyes peeled for the official release announcement over the next couple of weeks.  We do not have an exact date at this moment for when  the build will be finished so please wait patiently and allow us to finish testing to make sure that PC-BSD 10.0 release will be the best and most stable version yet.

  • Joe Maloney has been working tirelessly to get Gnome 3 into PC-BSD, and has been a huge help getting this Behemoth DE into a usable state.  Anyone who is interested in using / testing gnome 3 (it is still unsupported at this time) make sure to thank Joe for all the hard work he’s been putting in.
  • PBI’s have undergone even more detailed optimization further increasing their startup times.
  •  More tweaks and optimizations have been committed to PCDM, Firewall Manager, and ATI Hybrid graphics laptops.

PC-BSD 10.0-RC5 Now Available

The next PC-BSD 10.0-RELEASE-p4 image (based upon FreeBSD 10.0-RC5) is now available for download!

This will likely be our LAST RC before issuing the 10.0-FINAL release in a week or so. Please report any outstanding issues to our bug database.

*** NOTE to users of 10.0-RC4 or earlier ***

Due to an ABI change in FreeBSD between RC4 and RC5, you will need to re-install *all* packages and PBIs after updating  to RC5.

To update your packages, use the following command:

pkg-static upgrade –f

NOTE: This is best done immediately after the upgrade, before rebooting into RC5.

Once your desktop / server is updated, you can then update your PBIs via the “AppCafe� utility or using pbi_delete / pbi_add.

We are still finishing up a number of the newly-rebuilt RC5 PBIs, so if a particular application is not yet in the AppCafe, please check back in a day or two.

Disclaimer

This image is still a pre-release version of the upcoming 10.0-RELEASE. Use caution when deploying upon production systems.

Updating

Online updating to 10.0-RELEASE is now available for testing on 9.2-RELEASE systems. To get started, first ensure that your packages and world/kernel are up to date, and then follow these instructions:

http://trac.pcbsd.org/wiki/TestingUpdates

After making this change, the “pc-updatemanager branches� command will show the new 10.0-RELEASE branch as available for updating.

10.0-RELEASE notable features

  • Includes FreeBSD 10.0-RC5
  • Updated KMS / AMD driver support
  • ISO file is a hybrid USB file, and can be “dd“ed to a USB media.
  • New text-based installer
  • New UEFI loader on installation media
  • Able to select between GRUB/BSD loaders during installation
  • New desktops! Gnome 3, Mate (Replaces Gnome2) and Cinnamon
  • And much more!