Getting to know your portmgr-lurker — Frederic Culot

Name

Frederic Culot

Committer name

culot

Inspiration for your IRC nick

lack of inspiration actually…

TLD of origin

.fr

Current TLD (if different from above)

.lu

Occupation

IT consultant in the banking sector in Luxembourg, but I don’t always do IT.
I am also interested in business and management and my wife and I are working
on starting our own business.

When did you join portmgr@

Joined FreeBSD as a committer in October 2010 and the portmgr-lurkers program in
March 2014, but never been part of portmgr@.

Blog

http://people.freebsd.org/~culot is the closest thing I have to a blog

Inspiration for using FreeBSD

I was a longtime OpenBSD user until I worked in the same company as clement@
(former portmgr) who successfully managed to convert me to FreeBSD. I did not
feel the need to look into another system since then.

Who was your first contact in FreeBSD

clement@. But when I really started to get involved in FreeBSD it was jadawin@
who first contacted me. He is one of the kindest person I ever worked with and
while we’ve known each others for about 4 years now I’ve never been able to
meet him in person. But that’s the way it is with projects such as FreeBSD:
teams are virtual and gathering together might be difficult unfortunately.

Who was your mentor(s)

My mentors were sahil@ and wen@. Thanks to them I believe my mentorship at
FreeBSD was the best induction program I ever experienced. I was also amused to
realize that whereas companies spend huge amounts to design reward systems, it
is sometimes when nothing is to be expected in return that people are the most
caring and helpful.

What was your most embarrassing moment in FreeBSD

My first pointy hat: a bit after my first 700 commits when I started to feel
confident I finally managed to break INDEX :’(

Boxers / Briefs / other

Any 15-year old single material does it.

What is your role in your circle of friends

uncork the bottles usually…

vi(m) / emacs / other

vim

What keeps you motivated in FreeBSD

The people behind it. There are lots of great guys behind this project, and a
day when I could not meet with other developers on irc is a sad day for me :’(

But FreeBSD is also one of my sources of inspiration when it comes to how
organizations behave and innovate (which is a topic of interest I got into
during my MBA studies) and I find it very interesting to compare FreeBSD with
the for-profit companies I work for. I even wrote an article for BSDmag in case
some would also be interested in those aspects:

http://people.freebsd.org/~culot/BSDmag.pdf

> Favourite musician/band

I don’t listen to much music. The cause might be that I work in a very noisy
environment (large open-space), so I more and more enjoy silence and calm when
I’m back home. But recently when I listen to music I enjoy Moby’s “wait for me”
album (ambient edition), Erik Mongrain, or a bit of merengue to remind me of my
holidays.

What book do you have on your bedside table

Nietzsche’s Thus spoke Zarathoustra.

I even extracted my favorite quotes and created the
french/fortune-mod-zarathoustra port.

coffee / tea / other

Both, depends on the time of day

Do you have a guilty pleasure

To enjoy a 7-course meal with my wife at a 3-star michelin restaurant and
finish relaxing in a club chair in front of the fireplace with a 40 years old
armagnac.

How would you describe yourself

Sober, clever, and motivated in the morning.
Drunk, stupid, and depressed in the evening.
Or is it the opposite?

sendmail / postfix / other

sendmail as it’s in base, but not for long apparently so I could have to make a
more reasoned choice soon

Do you have a hobby outside of FreeBSD

Sports (I go to the gym almost everyday day, did quite some scuba diving and
snowboarding when I was younger), but I enjoy good food and wine when I’m done
training. I also enjoy traveling. My last trips were to India, Dominican
Republic, and Lapland: so many nice places to visit!

What is your favourite TV show

My favorites to day are twin peaks, the prisoner, and battlestar galactica

Claim to Fame

I spent one night at the pub with bapt@, and survived.

What did you have for breakfast today

oat flakes with water

What sports team do you support

If you want to torture me, just fasten me in front of the TV with a soccer game
on. I could even confess I enjoy tabthorpe’s jokes just to shorten the ordeal.

<Editors note: I have know idea what he means by this :>

What else do you do in the world of FreeBSD

Apart from my work on ports I also did some French translations (translated
the contributing-ports, linux-users, and building-products articles). I also
try to participate in IT exhibits and promote FreeBSD by managing booths such
as at Solutions Linux Paris for which I designed a poster to attract visitors:

http://people.freebsd.org/~culot/SolutionsLinux.png

But most importantly I offer beers and whisky to other FreeBSD developers when
I meet them :)

What can you tell us about yourself that most people don’t know

I actually enjoy tabthorpe’s jokes. Sometimes.

<Editor’s note: again, no clue what he is talking about :>

Any parting words you want to share

I repeat it but my main motivation to work on this project is to get in touch
with other FreeBSD enthusiasts, so do not hesitate to ping me on irc if you
feel like sharing some of your thoughts with me. I would be most pleased.

What is your .sig at the moment

Regards,
Frederic Culot

PC-BSD Weekly Feature Digest 22

The week is finally almost over and we’re back for another update on PC-BSD! The majority was spent squashing bugs and performing minor updates to PC-BSD utilities (as well as recovering from the Jet lag from AsiaBSDcon for Kris and Dru)! To check out pictures from the big event have a look at IXsystem’s facebook page here. For a list of some of the changes and updates this week have a look below.

Bug Fixes
* Fixed missing RDP support for krdc
* Fixed issue installing src / ports for server installs
* Enabled “lz4” compression on root FS by default
* Disabled some FUSE file-cache functionality in PBIFS
* Investigated issues with calls to “vflush” causing fuse to never finish unmounting
* Imported latest stable/10 and started builds
* Imported latest gnome3 / cinnamon changes
* Finished building next Edge package set
* Finished GUI updates and changes to bring them up to our new / current standards
* Added accessibility / shortcut keys for PC-BSD utilities

Adding chipset powersave support to FreeBSD’s Atheros driver

I've started adding some basic powersave support to the FreeBSD Atheros ath(4) driver. The NICs support putting parts of the device to sleep to conserve power but.. well, it's tricky.

In order to make things consistent, I either need to not do things when the NIC is asleep (for example, doing calibration when the NIC isn't running), but I also need to ensure that I force the NIC awake when the NIC may be asleep. During normal running, the NIC may have put itself into temporary sleep whilst waiting for some packets from the AP to signal that it needs to wake up. So I will also need to force the NIC awake before programming it.

So, before I start down the path of handling the whole dynamic power management stuff, I figured I'd tackle the initial bits - handling powering on the NIC at startup and powering it off when it's not in use. This includes powering it down during device detach and suspend, as well as when all of the VAPs are down.

This is turning out to be slightly more complicated than I'd like it to be.

The first really stupid thing I found was that during the interface down process, the VAP state change from RUN -> INIT would reset the BSS, which included re-programming the slot time. So, I have to wake up the hardware when programming that. It can then go back to sleep when I'm done with it.

Now there's some issues in the suspend path with the NIC being marked as asleep when it is being reset, which is confusing - the NIC should be woken up when ath_reset() is called. So, I'll have to debug these.

The really annoying bit is that if I read a register whilst the silicon is asleep, the reads return 0xDEADBEEF. So if I am storing the register contents anywhere, I'll end up storing and programming a potentially totally invalid value.

There's also some real problems with race conditions. I can put the power state changes behind a lock, but imagine something like this:

* ATH_LOCK; force awake; do something; ATH_UNLOCK .. ATH LOCK; do some more; put back to sleep; ATH_UNLOCK

Now, if a second thread puts the NIC back to sleep in between those two lock sections, the second "do some more" work may occur once the NIC was put to sleep by said second thread. So I have to correctly track if the NIC is being forced awake by refcounting how many times its being forced awake, then when the refcount hits zero and we can put it to sleep, put it back to sleep.

Once this is all done, I can start down the path of supporting proper network sleep - where the NIC stays asleep and wakes up to listen for beacons and received frames from the AP. I then choose to force the NIC awake and do more work. I have to make absolute sure that I don't queue things like transmitted frames or add more frames to the receive queue if it may fall asleep. There's also some mechanisms to have a transmit frame put the NIC to sleep - there's a bit that says "when this frame is transmitted, transition the NIC back to sleep." I have to go and figure out how that works and implement that.

But for now, let's keep it simple and debug just putting the NIC to sleep when it's not in use.

Upgrading Graphite

Recently swills@ upgraded Graphite and reconfigured how it works to fit more in to the FreeBSD file system layout.

So if you are upgrading from a graphite installation older than 0.9.12_1, you will need to follow the following instructions:

  1. Stop carbon
  2. Copy the old data from /usr/local/storage/whisper/* to /var/db/carbon/whisper/
  3. Copy the /usr/local/etc/carbon/carbon.conf.example over to carbon.conf
  4. Set the SECRET_KEY to something random in /usr/local/etc/graphite/local_settings.py
  5. Then follow the instructions after the install, including updating the httpd.conf per the message after the install
  6. Restart Carbon and Apache

Be careful that you do not miss any of the steps and you should have a working Graphite install.

Bhyve in libvirt

I continue my activities on improving libvirt FreeBSD support and I have some good news. Recent libvirt release, 1.2.2, is the first version to include the bhyve support!

Currently it's in its early stage and doesn't support some of the features and doesn't provide good flexibility, it's just a basic stuff at this point. I'll not provide a detailed description and instead will point you to the document: Libvirt: Bhyve driver. You'll find a sample domain XML which covers all the features currently supported by the driver.

TODO list

While there are lots and lots of things to be done, there are some specific ones I'm focusing on:

  • Console support through nmdm(4). This is very important feature for debugging and checking what's going on in the guest.
  • Domains autostart support. There's a patch already kindly provided by David Shane Holden that just needs review and testing.
  • A little more flexible slot ids allocation / device configuration.

Qemu/FreeBSD status

As a side note, I'll give an update what's changed since my previous blog post about qemu libvirt driver on FreeBSD. So, here's what's new:

  • Proper TAP interfaces cleanup
  • CPU affinity configuration support, check http://libvirt.org/formatdomain.html#elementsCPUAllocation for details
  • virsh console should now work if you run it from freebsd host and connect to libvirtd on Linux
  • Node status support (such as virsh nodecpustats, virsh nodememstats)

Some of these are available in already released versions, some are only in git version.


PC-BSD Weekly Feature Digest 21 — PCBSD 10.0.1 Released!

PC-BSD 10.0.1 Has been released! Check out the release notes from Kris below. Kris is currently out in the field attending AsiaBSDCon so make sure to stop by the FreeBSD booth if you’re in the area and show your support! Work has continued this week on the development of the new PC-BSD mixer although our primary goal this week was to get a bunch of trac tickets fixed, closed, or assigned to someone to take care of them. Thanks as always and enjoy the new updates!

- Josh

———————————————

The first PC-BSD 10.0 quarterly update is upon us, and 10.0.1 is now
available.

This update includes a number of important bugfixes, as well as newer
packages and desktops, such as KDE 4.12.2, Cinnamon 2.0 and more. For
more details and updating instructions, refer to the notes below.

Changes
———————————————

* KDE 4.12.2
* Cinnamon 2.0
* Samba 4.1.4
* Stability improvements to PBI subsystems
* Updated GRUB loader, fixing issues related to slow / hanging startup
* Updated AppCafe UI
* Updates to Life-Preserver, including “Classic” backup mode and
automatic snapshots
* Updated control panel with desktop settings buttons
* PulseAudio 5.0 integration
* Improved Video display auto-detection
* Bugfixes to mouse auto-detection
* Improved LDAP / AD support for login manager
* Misc other bugfixes

Updating
———————————————

Desktop users already running 10.0 can update via Control Panel ->
Package Manager -> Updates.
Server users can update via the “pc-updatemanager” utility.

If package updating fails due to conflict errors, please be sure to
apply all system updates first before trying again.

Installing
———————————————

10.0.1 DVD/USB media can be downloaded from the following URL:

http://www.pcbsd.org/en/download.html

Reporting Bugs
———————————————

Found a bug in 10.0.1? Please report it (in as much detail as possible)
to our Trac Database.

https://trac.pcbsd.org

FreeBSD Foundation Accepting Travel Grant Applications for BSDCan 2014

Calling all FreeBSD developers needing assistance with travel expenses to BSDCan 2014.

The FreeBSD Foundation will be providing a limited number of travel grants to individuals requesting assistance. Please fill out and submit  the Travel Grant Request Application by April 7th, 2014 to apply for this grant.

This program is open to FreeBSD developers of all sorts (kernel hackers, documentation authors, bugbusters, system administrators, etc).  In some  cases we are also able to fund non-developers, such as active community members and FreeBSD advocates. More details are available in the announcement.

Getting to know your portmgr-lurker — Alexy Dokuchaev

In this latest edition of Getting to know, we interview senior ports committer Alexy Dokuchaev as one of our newest portmgr-lurkers.

Name

Alexey Dokuchaev

Committer name

danfe@

TLD of origin

.ru (technically should be .su, but it’s now defunct)

Occupation

Software engineer and contractor

Inspiration for using FreeBSD

Wanted a Unix system that I could understand and that would not get bloated
as time goes by. In 1998 Linux was popular mount local folks, but I just
could not get it (trying to switch from DOS). Someone mentioned FreeBSD;
and it all started to make sense pretty much immediately. And even after
some 15 years, FreeBSD still feels like back in those good days.

Who was your first contact in FreeBSD

Max Khon (fjoe@), I guess…

Who was your mentor(s)

fjoe@ and krion@

vi(m) / emacs / other

nvi(1)

What keeps you motivated in FreeBSD

That in 2014 I can have a modern Unix system which I still can work with
like it’s 1999 again. I can still have text console. And start X11 with
startx(1). Configure things by editing /etc/rc.conf, but have support for
the latest hardware. Play sound via OSS, yet enjoy low-latency in-kernel
mixer. We’re so lucky to not have Lennart Poettering with his PulseAudio
and systemd crap…

coffee / tea / other

Both (but tea preferred) + beer

What is your favourite TV show

The X Files, I guess…

What is your .sig at the moment

./danfe (making it both answer and a sig this time)

FreeBSD Project to participate in Google Summer of Code 2014

The FreeBSD Project is pleased to announce its participation In Google's 2014 Summer of Code program, which funds summer students to participate in open source projects. This will be the FreeBSD Project's tenth year in the program, having mentored over 160 successful students through summer-long coding projects between 2005 and 2013.

Porting over the AR8327 support

It's been a while since I posted. I'll post about why that is at some point but for now I figure it's time I wrote up the latest little side project - the Atheros AR8327 switch support.

The AR8327 switch is like the previous generation Atheros switches except for a couple of very specific and annoying differences - the register layouts and locations have changed. So it's not just a case of pretending it's an AR8316 except for the hardware setup - there's some significant surgery to do. And no, I did try just ignoring all of that - the switch doesn't come up and pass packets.

So, the first thing was to survey the damage.

The Linux driver (ar8216.c) has a bunch of abstractions that the FreeBSD driver doesn't have, so that's a good starting point. The VLAN operations and VLAN port configuration stuff is all methods in the Linux driver, so that was a good starting point. I stubbed most of the VLAN stuff out (because I really didn't want it to get in the way) - this turned out to be more annoying than I wanted.

Next was the hardware setup path. There's more configurable stuff with the AR8327 - there's two physical ports that I can configure the PHY/MAC parameters on for either external or internal connectivity. I just took the code from Linux (which yes, I have permission to relicence under BSD, thanks to the driver authors!) and I made it use the defaults from OpenWRT for the DB120. The ports didn't properly come up.

I then realised that I was reading total garbage from the PHY register space, so I went looking at the datasheet and ar8216 driver for some inspiration. Sure enough, the AR8327 has the PHY MDIO bus registers in different locations. So after patching the arswitch PHY routines with this knowledge, the PHYs were probed and attached fine. Great. But it still didn't detect port status changes.

So, back to the ar8216 driver. It turns out that there were a few things that weren't methodized - and these were the bits that read the PHY status from the switch. Both drivers didn't just poll the PHYs directly - they read the switch registers which had a summary of the port status. So, I taught the driver about this and voila! Port status changes worked.

But, no traffic.

Well, there's a few reasons for this. It's a switch, so I don't have to setup anything terribly difficult. The trick here is to enable port learning and make sure they're all in the same VLAN group. Now, here's where I screwed up and I found a bug that needed working around.

The port setup code did enable learning and put things into a vlan group.

Firstly, I found this odd behaviour that I got traffic only when I switched the ethernet cable to another port. Then learning worked fine. I then found that the ar8216 driver actually triggers a forwarding table flush upon port status change, so I added that. This fixed that behaviour.

But then it was flooding traffic to all ports. This is kinda stupid. What did I screw up? I put each port in a separate vlangroup, rather than put them in the same vlangroup. Then, I programmed the "which ports can you see?" to include all the other ports. What this meant was:
  • The forwarding table (ie, what addresses were learnt) were linked to the vlangroup the port is in;
  • .. and when the switch did a lookup for a given MAC on another port, it wouldn't find it, as the address in the forwarding table showed it was for another vlangroup;
  • .. so it would do what switches do when faced with not knowing about the MAC (well, and how I had configured it) - it flooded traffic.
The solution was thankfully easy - I just had to change the vlangroup (well, "port vlan" here) to be '1', instead of the port id. Once this was done, all the ports came up perfectly and things worked great.

So, this now works great on the Atheros DB120 reference board. It's not working on other boards - there's likely some timing issues that need to be resolved. But we're making progress!

Finally, I spent a bunch of time porting over the port configuration and LED configuration stuff from OpenWRT so I didn't have the driver just hard-coded to the DB120 board. I'll update the configuration and code when I get my hands on other boards that use the AR8327 but for now this is all I have.

Enjoy!

bhyvecon 2014

bhyvecon 2014 (http://bhyvecon.org/), SAKURA Internet Research Center, Tokyo, Japan 12 March, 2014. See the bhyve hypervisor in action and ask a core bhyve developer your technical questions.

PC-BSD Weekly Feature Digest 20

New Sound Management

Work has began to fully port pulse audio into PC-BSD for 10.1, and we are quite pleased so far with the results.   Kris has been making headway this week getting pulse audio and it’s related utilities working.  In the meantime Ken has been working on an all new utility pc-mixer.  pc-mixer is a complete front-end to the FreeBSD “mixer” utility that will allow users a simple to use GUI and volume control for every day tasks.  There will also be an advanced tab allowing for more specific audio setups and control.

Other News

*New PBIs for 9.x versions and 10.x versions were released this week, so be sure to check out the AppCafe and see what’s new.

*Gnome 3 and Cinnamon 2.0 desktops have received updates this week.  These desktops are not 100% fully supported yet and as    such we can not make any guarantee on functionality.

*Grub 2.02 has been fully ported over and updated to GRUB 2.02-prerelease.

*Lastly The PC-BSD ports tree has been frozen in preparation for our quarterly package update.

Improvements for Life-Preserver
* Add new “Classic” backup dialog for custom exclusions and status updates
* Fix bug with restoring a file/dir into a missing directory on the main system.
* Clean up the restore tab

Bug Fixes
* Bugfixes to the FUSE “pbifs” file-system
* Fix bug showing HPLIP drivers in the main CUPS Manager.
* Fix seg-fault crash in EasyPBI when removing a non-selected item.

portmgr-lurkers@ March 1 edition

The first intake of portmgr-lurkers@ is complete, and it is now time to start with the second round of our -lurkers.  Please join us in welcoming Alexey (danfe@) Dokuchaev and Frédéric (culot@) Culot to our ranks.

During this -lurker round, culot@ will be the shadow portmgr-secretary@, learning the finer points of the roles and responsibilities of the job.

Thomas
on behalf of portmgr@

PC-BSD Weekly Feature Digest 19

Changes to PBI’s

As many of you know there was an issue with PBI’s causing them to freeze at random times during use. Kris went into full-blown hermit programmer mode to track down the issue and you’ll be glad to know a fix was committed that addresses this issue. Kris said of the fix: “it’s faster, cleaner, and allows proper access to all of the filesystem data. It can even be used by FreeBSD users who want to run different sets of packages in a location other than /usr/local”. To test out the new changes you will want to rebuild the pbi-manager backend. For those of you that may not know the pbi-manager utility is a backend that you never see, but is always there managing system interactions when running PBI’s. Follow the instructions below to grab the pc-bsd source and rebuild the pbi-manager to apply the fix.

1. Open a new terminal and paste: git clone https://github.com/pcbsd/pcbsd.git
This will create a directory for the PC-BSD source code

2. type: cd pcbsd/src-sh/pbi-manager/
This will browse directly to the pbi-manager source directory

3. type: sudo make install

4. Restart your system

And you’ve done it! Don’t forget to reset your system! PBI’s will not work until the system is reset. For more information, questions, or thoughts please post below.

Changes to Life Preserver

Life preserver has been updated to bring in some exciting new changes. New automatic snapshot schedules have been added along with new replication schedule options that will allow users more flexibility and control over their Life Preserver snapshot schedules (i.e. Hourly, 30 minutes, 10 minutes). New code has been added to allow the user to change the pop-up notification policy (all, only errors, none). A minor bug was also fixed that was causing non-error messages in the “Message” dialog.

Unifying PC-BSD Utility Chain

Work is continuing on standardizing the PC-BSD utility chain. More information has been added @ (http://wiki.pcbsd.org/index.php/Become_a_Developer/10.1). The changes will also bring in some new keyboard accessibility through hot keys and shortcut keys. There are currently several opportunities available to help update the tool chain, so if you’d like to lend a hand please let us know!

Important changes to Appcafe and PCDM (Release Notes)

AppCafe -

* Finish overhaul of the UI
* Add ability to email the port maintainer
* Add right-click action shortcuts for individual applications
* Add new browser home page with recommended applications
* Move the category browser to a seperate page
* Add ability to install custom PBI’s from your system via File->Add PBI (no internet/repository required)
* General improvements/bugfixes to the backend functions

PCDM -

* Fix backend detection of LDAP/Active Directory users (still needs verification/testing by people with this special type of setup)
* Add option to show an auto-login delay (in which time the user can cancel the auto-login if necessary)
* Add option to disable showing the system users and require that the username also have to be typed in.

Login Manager Configuration Utility (pc-dmconf)
* Update to reflect the new PCDM configuration options
* Fix a bug where a blank auto-login username could be set

We can patch it for you wholesale

…but remembering costs extra.

Every once in a while, I come across a patch someone sent me, or which I developed in response to a bug report I received, but it’s been weeks or months and I can’t for the life of me remember where it came from, or what it’s for.

Case in point—I’m typing this on a laptop I haven’t used in over two months, and one of the first things I found when I powered it on and opened Chrome was a tab with the following patch:

diff --git a/lib/libpam/modules/pam_login_access/pam_login_access.c b/lib/libpam/modules/pam_login_access/pam_login_access.c
index 945d5eb..b365aee 100644
--- a/lib/libpam/modules/pam_login_access/pam_login_access.c
+++ b/lib/libpam/modules/pam_login_access/pam_login_access.c
@@ -79,20 +79,23 @@ pam_sm_acct_mgmt(pam_handle_t *pamh, int flags __unused,

        gethostname(hostname, sizeof hostname);

-       if (rhost == NULL || *(const char *)rhost == '') {
+       if (tty != NULL && *(const char *)tty != '') {
                PAM_LOG("Checking login.access for user %s on tty %s",
                    (const char *)user, (const char *)tty);
                if (login_access(user, tty) != 0)
                        return (PAM_SUCCESS);
                PAM_VERBOSE_ERROR("%s is not allowed to log in on %s",
                    user, tty);
-       } else {
+       } else if (rhost != NULL && *(const char *)rhost != '') {
                PAM_LOG("Checking login.access for user %s from host %s",
                    (const char *)user, (const char *)rhost);
                if (login_access(user, rhost) != 0)
                        return (PAM_SUCCESS);
                PAM_VERBOSE_ERROR("%s is not allowed to log in from %s",
                    user, rhost);
+       } else {
+               PAM_VERBOSE_ERROR("neither host nor tty is set");
+               return (PAM_SUCCESS);
        }

        return (PAM_AUTH_ERR);

The patch fixes a long-standing bug in pam_login_access(8) (the code assumes that either PAM_TTY or PAM_RHOST is defined, and crashes if they are both NULL), but I only have the vaguest recollection of the conversation that led up to it. If you’re the author, please contact me so I can give proper credit when I commit it.

Burning all the bridges. Cleaning up jails with ezjail-admin on #FreeBSD

I noted that my updates on my jail host didn’t actually do a delete-old/delete-old-libs during the basejail process:

ezjail-admin update -i

I tend to update my jails with my base host svn updates to -current, so there’s a bit of churn and burn with regards to old files and such. This came to a head today as my src.conf on the base host declares WITHOUT_NIS to conserve my limited space.

The python port checks for the existence of the yp binaries to determine whether or not to build NIS support. So, if the old binaries are lying around and support for NIS is removed from your system, python’s build will abort with something like the following:

Install them as needed.
====
====> Compressing man pages (compress-man)
===> Installing for python27-2.7.6_2
===> Checking if lang/python27 already installed
===> Registering installation for python27-2.7.6_2 as automatic
pkg-static: lstat(/var/ports/basejail/usr/ports/lang/python27/work/stage/usr/local/lib/python2.7/lib-dynload/nis.so): No such file or directory
*** Error code 74

I realized that even though my host system was fairly clean (I do port rebuilds after each upgrade and delete-old delete-old-libs following that), the basejail was still filled with obsoleted files.

A super dangerous and super effective way to clean that up is the following:
yes | make delete-old DESTDIR=/usr/jails/basejail
yes | make delete-old-libs DESTDIR=/usr/jails/basejail

Dangerous, because you have to realize that your deleting binaries and libraries that might still be in use if you haven’t recompiled your ports packages. Effective, because it will cleanup and purge a lot of things if you haven’t done it in a while.

This also led me to understand that the /etc/src.conf tuneables WITHOUT_* don’t *stop* the buildsystem from creating the binaries and libraries. It doesn’t seem to shorten your build time. It *will* allow you to purge them from your system at install time with the delete-old make targets.

On testing, part III

I just got word of an embarrassing bug in OpenPAM Nummularia. The is_upper() macro, which is supposed to evaluate to true if its argument is an upper-case letter in the ASCII character set, only evaluates to true for the letter A:

#define is_upper(ch)                            \
        (ch >= 'A' && ch <= 'A')

This macro is never used directly, but it is referenced by is_letter(), which is referenced by is_pfcs(), which is used to validate paths and path-like strings, i.e. service names and module names or paths. As a consequence, OpenPAM does not support services or modules which contain an upper-case letter other than A. I never noticed because a) none of the services or modules in use on the systems I use to develop and test OpenPAM have upper-case letters in their names and b) there are no unit or regression tests for the character classification macros, nor for any code path that uses them (except openpam_readword(), which uses is_lws() and is_ws()).

The obvious course of action is to add unit tests for the character classification macros (r760) and then fix the bug (r761). In this case, complete coverage is easy to achieve since there are only 256 possible inputs for each predicate.

I have merged the fix to FreeBSD head (r262529 and r262530). Impatient users can fix their system by running the following commands:

% cd /usr/src/contrib/openpam
% svn diff -r758:762 svn://svn.openpam.org/openpam/trunk | patch
% cd /usr/src/lib/libpam/libpam
% make && make install

Unsurprisingly, writing more unit tests for OpenPAM is moving up on my TODO list. Please contact me if you have the time and inclination to help out.

PC-BSD Weekly Feature Digest 18

Big news this week!   Kris has finished re-writing the code for handling how PC-BSD handles major updates.  The general consensus was that there were still many users out there that were having difficulty when upgrading to a new version (i.e 9.2 > 10).  For more information on the new PC-BSD update system, check out the article here.  To view Kris’ address to the testing mailing list click here.  Our goal is and has always been to have a reliable system to push out updates, and we think you will all be very pleased with the results!

Basic guidelines for PC-BSD utilities are continuing to evolve and we want your input!  If you have ideas for the development team on what should become standard practice we want to hear from you.  You can join the discussion on the the PC-BSD developer mailing list if you’d like to submit your ideas.  To check out Guidelines for PC-BSD utilities visit the PC-BSD 10.1 wiki’s “Become a Developer” section @ http://wiki.pcbsd.org/index.php/Become_a_Developer/10.1

A moderate batch of fixes for trac tickets were committed this week addressing minor issues with the warden.  Appcafe has also received a big update to it’s user interface allowing for a smoother experience for users.  If you want to give the 2 newest versions of these programs a try grab the PC-BSD source and let us know what you think!

That’s it for this week!  See you all next time!

Call For Testers: New Major Upgrade Methodology

Hello everyone!

Kris just posted a call for testers for a new methodology for major system upgrades (such as  9.2 -> 10.0 -> 10.1, etc..) and we are looking for people who are still running 9.2  to try it out. The full text of the call for testers is at the bottom of the post, but to give a bit of background we have been generally unimpressed with the reliability of the “standard” FreeBSD update tools (freebsd-update and pkg) when it comes to fetching uncorrupted update files through the internet. This new methodology takes those two utilities out of the general preparations for an update (download/verification of files), as well as a couple other upgrade steps so that there should no longer be an issue with starting an upgrade when only some of the upgrade files were actually retrieved successfully.

 

Please test it out and let us know how it goes!

Remember, always backup your data before doing any major upgrade like this!  The new methodology should automatically create a boot environment for you before doing the upgrade, but better safe than sorry!

 

Here is the full text of request from Kris to the developer mailing list:

I've just finished up some work on a major re-write of our updating
system when "upgrading" between major releases, I.E. 9.2 -> 10.0, 10.0
-> 10.1.

https://github.com/pcbsd/pcbsd/commit/b95e8a83c73511568ae4291a54e0f93f6c67ef30
https://github.com/pcbsd/pcbsd/commit/9a8b3d1945fa67db8e99b0e4e82280b5626aa895

It seems to work well here, but it needs some additional testing from
any users still running 9.2 who want to update to 10. To test this,
first grab the latest from git:

# git clone https://github.com/pcbsd/pcbsd.git pcbsd
# cd pcbsd/src-sh/pc-updatemanager
# make install

Then run:

# pc-updatemanager install fbsd-10.0-RELEASE

This should start the download / upgrade process. If anything fails
during the process, logs are kept in /root, which will assist me in
debugging.

Thanks!

PC-BSD Weekly Feature Digest 17

The PC-BSD development team has been abuzz this week with awesome suggestions on how we can standardize the way we write PC-BSD utilities and software.  One thing we’ve begun to realize is that as more people are contributing to the project, it is ever more important to make sure that there are clear standards for development.  Even our primary developers will admit it’s easy to forget to use the same icon pack, or file menu layout when you get busy writing the main program.  Going forward you can expect these standards to impact most of the PC-BSD utilities and programs you use everyday, although in a relatively minor way.  Everything will still function the exact same, but whether or not you are using AppCafe or the Warden you can expect the file menu layout / program layout to follow the same general rules.  For more information please check out “Becoming a Developer� in the PC-BSD 10.1 wiki.  If you’d like to join the discussion you can email [email protected]

I’ve seen some discussion lately about the life cycle of PC-BSD branches.  I sat down with Kris Moore in IRC and asked if he wouldn’t mind clarifying the release cycle for our users.   Kris answered the general rule of thumb you can use is a branch will continue to be supported for 6 months after the next branch is released.  The updates include all of the things you would expect like new PBI and security updates.  So for users of 9.2 you can expect support to continue through June of 2014.  9 Stable was a “experimental� branch and is no longer supported at this time.  Users of 9 Stable are encouraged to upgrade to 9.2 or 10.0 Release to continue to receive important updates.

You can expect to see tons of improvements coming up for PC-BSD 10.1.   One of the biggest being Kris and Yuri have been working to fix Linux jail support in the Warden.  A handful of commits went into the tree today that will address the previous problems users have been having with Linux jails.  Kris has continued to refine the Warden and PBI systems to fix some bugs that were causing major stability issues in certain scenarios.   Minor cosmetic changes are coming for most PC-BSD utilities to bring them up to the same standards outlined in the “Become a Developer� section in the PC-BSD 10.1 wiki.

That’s it for this week folks.  Lots of good things in the works so stay tuned to the blog for more important PC-BSD news!