Author Archives: erwin

The Ports Management Team 2013-08-09 12:46:27

The FreeBSD project has provided pre-built ready-to-install binary packages for many years on a best-effort basis. While these packages do work in a large number of cases, there are too many inconsistencies and failure combinations, from the unpredictable update frequency to dependency handling across upgrades, for them to be used on a wider scale. After many months of work, we’re nearing a paradigm shift in both the format of the packages, and the building and distribution of the packages with the new PKGNG tools.

At the upcoming Developer Summit at the EuroBSDCon conference in Malta on September 26 and 27, there will be another Ports and Packages Summit, which will center on a round-table brainstorm that begins with a summary of the tremendous progress made in the last 12 months, and closes with a discussion of the roadmap on how to improve binary package creation, distribution, installation and upgrading. Please contact me if you have any topics you’d like to present or discuss. It will be an informal gathering, no formal slides or presentations are required.

As always, the DevSummit is an invitation-only event, so also contact me at [email protected] if you want to participate.

Droso

The FreeBSD project has provided pre-built ready-to-install binary packages for many years on a best-effort basis. While these packages do work in a large number of cases, there are too many inconsistencies and failure combinations, from the unpredictable update frequency to dependency handling across upgrades, for them to be used on a wider scale. After many months of work, we’re nearing a paradigm shift in both the format of the packages, and the building and distribution of the packages with the new PKGNG tools.

At the upcoming Developer Summit at the EuroBSDCon conference in Malta on September 26 and 27, there will be another Ports and Packages Summit, which will center on a round-table brainstorm that begins with a summary of the tremendous progress made in the last 12 months, and closes with a discussion of the roadmap on how to improve binary package creation, distribution, installation and upgrading. Please contact me if you have any topics you’d like to present or discuss. It will be an informal gathering, no formal slides or presentations are required.

As always, the DevSummit is an invitation-only event, so also contact me at [email protected] if you want to participate.

Droso » FreeBSD 2013-08-09 12:27:03

For the third year in a row, we’ll be organizing a Vendor Summit during the Developer Summit prior to the EuroBSDCon conference, this year in Malta on September 26 and 27. In previous years, we’ve had a number of presentations by companies, like NETASQ, pfSense and Netflix, on how they successfully built their products and services on top of FreeBSD, and how contributing code back to the community actually can save them money in the long term by reducing internal maintenance costs.

This year, I’d like to change to focus more towards an open discussion between vendors and other large consumers, not only so you can learn from each other, but also so the FreeBSD community can learn more on how their product is used. We’ve long heard that binary packages were the Achilles heal of FreeBSD. Are we on the right track with PKGNG? Java-support is another issue, but is there anything else you are missing that maybe some other company may have an existing solution for they may be able to contribute, but hadn’t thought about anyone else was looking for? Or maybe you have an problem that’s too big for one company to fix, but can be fixed if some of you together fund a project to do so. The FreeBSD Foundation might be able to help.

This is an invitation-only event, please contact me at [email protected] if you would like to attend. If you have anything to present or any topic you would like to discuss, please contact me as well. This will be an informal event, but we welcome a few short presentations.

Droso

With only three weeks to go, we so far have 7 people registered for the Ports and Packages Summit during the DevSummit at EuroBSDCon in Warsaw.
I’m sure that can’t be right. If you intend to come, please register (by sending an email to me) as soon as possible. If you don’t intend to come, please reconsider.

So far we have 4 main topics to discuss in 2 1,5 hour slots:
- Status of the move to subversion
- Status of the implementation and uptake of the new package tools
- Status and proposed schedule for scheduled releases of binary packages
- Quality assurance in all shapes and forms: QAT, redports, pointyhat

Please send any topics you’d like to discuss, presentations to present, and other items that should go on the schedule to me in the next week or two so I can prepare a draft agenda at least a week before.

Thank you and see you there!

Droso

In a few days, I’ll be heading to the FOSDEM conference in Brussels again this year. On Saturday, you’ll most likely find me around the FreeBSD booth representing the FreeBSD Foundation, so if you’re there drop by to say hi, discuss the Foundation’s work, pick up a Foundation flyer, check out the swag, or make a donation. On Sunday, I’ll be in the BSD DevRoom where there will be some interesting presentations and discussions. Remember, FOSDEM is free to attend. Hope to see you there!

Droso

Hetzner has a strange IPv6 routing setup where the default gateway is not in the same subnet as the host. Clearly, this is to avoid wasting space on glue nets, but it does not look pretty. Obviously, Linux just ignores the fact that it isn’t supposed to work, but no such luck on FreeBSD. As Bitmand wrote some time ago, it can be solved with a static route but the FreeBSD startup scripts, apply the default route before static routes, leaving it no other option but to reject the default route as it has no route to its target at that time. Instead of hacking the startup scripts, this can be solved easier with two static routes with fixed order, which is still a hack though not as ugly and less error prone when upgrading the system files. For reference, especially for myself next time I need this, here’s my full ipv6 startup configuration:

rc.conf:
ipv6_enable="YES"
ipv6_default_interface="re0"
ipv6_static_routes="defgw def"
ipv6_route_defgw="2a01:XXXX:XXXX:XXXX::1/59 -prefixlen 59 -iface re0"
ipv6_route_def="default 2a01:XXXX:XXXX:XXXX::1"
ipv6_ifconfig_re0="2a01:YYYY:YYYY:YYYY::2/64"

systctl.conf
net.inet6.ip6.accept_rtadv=1

So long HP Blade Cluster and thanks for all the packages

After many years of faithful service, today the FreeBSD Ports Management Team decided to decommission the HP Blade Cluster. When the 20-node BladeSystem was donated to the FreeBSD Foundation, by Hewlett-Packard back in 2005, it tripled the speed of the i386 package building process. Today, and several hardware generations later however, it is no longer profitable to keep the system running inside the cluster. The portmgr team has been very pleased with the system, especially the built-in out-of-band power management- and console system. The system has also proved to be very reliable; even with continuous high workloads for so many years, the only hardware failures we experienced were some of the disks. The i386 package cluster now consists of 5 Xeon-based servers hosted at ISC until the new clusters are fully online.

We again wish to thank HP for their generous donation and Yahoo! for hosting it in one of their datacenter.

Droso

Just a quick note to point to my slides that summarize the Ports and Packages Summit at the FreeBSD DevSummit during BSDCan 2011, which can be found here. Also, we looking forward to feedback on the PKGNG project that was announced earlier and will replace the current pkg_* tools to handle ports installation and package handling and which will be a focus for portmgr over the next few months.

New portmgr member: Baptiste Daroussin

Portmgr is pleased to announce that Baptiste Daroussin, bapt@, has joined the ranks of the Ports Management team. He has been working hard on some large infrastructure improvements, including a new OPTIONS framework and PKGNG that will replace the current pkg_* tools and bring them into the 21 century. We are very happy to have him onboard, where he will continue working on these and other much needed infrastructure improvements with the full power of the pointyhats.

Ports and Packages for Supported Releases

Portmgr published a new page on their website which describes the current support and EoL policies for the ports tree and released packages. The main take-home messages are:

  • Support of FreeBSD releases by ports and the ports infrastructure matches the policies set out by the FreeBSD Security Officer.
  • Package builds will use the oldest supported minor release within each major branch to ensure ABI and KBI backwards compatability within each major branch, and support all minor versions of each major branch, including -RELEASE and -STABLE.

See the full policy on the portmgr webpage.

Droso

In a few weeks, I’ll be heading to the FOSDEM conference in Brussels again this year. I’ll spend most of my time at the FreeBSD booth for the FreeBSD Foundation, so if you’re there drop by to say hi, discuss the Foundation’s work, pick up a Foundation flyer, check out the swag, or make a donation. There will also be a BSD DevRoom where there will be some interesting presentations and discussions that I might attend. Remember, FOSDEM is free to attend. Hope to see you there!

Droso

This weekend I had the pleasure of attending the third edition of the BSD community day at the NLLGG meeting in Utrecht, the Netherlands. I was happy to see that there were at least as many, if not more, attendees at the BSD track as the general track.

The BSD track featured 4 interesting talks. Rene Laden opened the day with a talk on porting ROS (Robot Operating System) to FreeBSD, detailing some of the difficulties of getting the core bits working, which already are in ports and some ideas and plans for future work. Ed Schouten was next with an update on integrating the clang compiler into FreeBSD. A lot of work has already been done here, but still more to come. The third talk by Paul Schenkeveld had some very interesting ideas of how to combine nanoBSD‘s image building features with ZFS snapshots as generalized way to upgrade software on servers, while minimizing downtown and providing an easy rollback when the upgrade doesn’t go as expected. The day ended with Otto Moerbeek’s overview of some of the security features in OpenBSD, with special focus on privilege separation in and between processes.

A big thanks to NLLGG for hosting the event, I certainly both enjoyed the day and learned some new things. We’ll see each other again next year at EuroBSDCon.

Droso

While the snow falls outside and the holidays approaching fast, it is time for the FreeBSD Foundations yearly End-of-Year fundraising campaign. This year again brought an impressive list of accomplishments by the Foundation, to mention a few:

  • Provided $100,000 in grants for projects that improve FreeBSD in the areas of:
    - DTrace support
    - High availability storage
    - Enhanced SNMP reporting
    - Virtualization and resource partitioning
    - Embedded device support
    - Networking stack improvements
  • Allocated $50,000 for equipment to enhance FreeBSD project infrastructure.
  • Sponsored 8 FreeBSD related conferences.
  • Funded 16 travel grants giving increased community and developer access to conferences.
  • Provided legal support to the FreeBSD project.

We are fortunate to already have reached half of this years fund-raising goal of $350,000, so please consider a donation, no matter how large or small, to help us reach that goal and help us continue supporting the FreeBSD community through next year as well.

Read the full letter by Justin Gibbs, President of the FreeBSD Foundation.

MD5 for distinfo has been deprecated

erwin@ committed http://www.freebsd.org/cgi/query-pr.cgi?pr=149657, based on work by dougb@ and rene@.   It deprecates the use of md5 checksums in distinfo.  So here on in, when you run make makesum, the md5 will no longer be generated, only the sha256 checksum.

Existing distinfo containing md5 info will silently be ignored.  So at this time there is no need re-create distinfo for the sake of removing it, just allow the regular flow of ports updates take care of it.

Droso

Last night, I committed a large update to the ports tree that deprecated MD5 checksums based on the work by Doug Barton and Rene Laden in ports/149657. For a long time we’ve had both MD5 and SHA256 checksums in the distinfo file, even though having multiple checksumming algorithms does not add any additional security. From today, MD5 checksums are no longer generated, but existing checksums will silently be ignored. For now, we won’t be doing large sweeps through the tree removing MD5, but let them slowly disappear when individual ports are updated, to avoid the churn on the cvs repository, mirrors, and package build infrastructure such large sweeps will cause.
The ports framework internals were also updated to reflect this change by renaming the MD5_FILE macro to DISTINFO_FILE. A lot of thanks to Dough and Rene!

Droso

In a few days, I’ll be heading off for another yearly EuroBSDCon, this year in Karlsruhe, Germany. Unfortunately, I will have to leave on Sunday, but on Saturday you might find me at the FreeBSD Foundation booth in the booth area where we’ll have Foundation brochures and swag. Please drop by to give feedback, ask questions, and/or make a donation. Hope to see you there!

Partial ports tree thaw

The ports tree is now tagged and partially thawed. Until 8.1 is released, sweeping commits still need explicit approval from portmgr to assure that tags can be slipped for potential security issues. For more information what constitutes a sweeping change, see the portmgr webpages.

Feature freeze for 8.1 now in effect

In preparation for 8.1-RELEASE, the ports tree is now in feature freeze.

Normal upgrade, new ports, and changes that only affect other branches are allowed without prior approval but with the extra Feature safe: yes tag in the commit message. Any commit that is sweeping, i.e. touches a large number of ports, infrastructural changes, commits to ports with unusually high number of dependent ports, and any other commit that requires the rebuilding of many packages is not allowed without prior explicit approval from portmgr after that date.

When in doubt, please do not hesitate to contact portmgr.

Droso

On behalf of portmgr, I am pleased to announce that portmgr has found a new secretary: Thomas Abthorpe. Thomas has been a FreeBSD ports committer since 2007 and has made more than 1000 commits since. He has previously served on the ports-security team and is currently a member of the KDE and donation teams. He has also mentored several new ports committers over the years.

In his role as portmgr secretary, Thomas will help portmgr keep track of ongoing issues, keeps the portmgr, and other bookkeeping work like organizing votes and stay in touch with other FreeBSD teams.

Please welcome him onboard!