pkg(8) is now the only package management tool

September 1, 2014 by · 1 Comment 

The ports tree has been modified to only support pkg(8) as package management system for all supported version of FreeBSD.

if you were still using pkg_install (pkg_* tools) you will have to upgrade your system.

The simplest way is

cd /usr/ports/ports-mgmt/pkg
make install

then run

pkg2ng

You will have lots of warning, don’t be scared, they are expected, pkg_*  databases used to get easily mangled. pkg2ng is most of the time able to deal
with it.

If however you encounter a problem then please report to [email protected]

A tag has been applied to the ports tree if you need to get the latest ports tree before the EOL of pkg_install:

https://svn.FreeBSD.org/ports/tags/PKG_INSTALL_EOL

A branch has been created if some committers want to provides updates on the for pkg_install users:

https://svn.FreeBSD.org/ports/branches/pkg_install

Please note that this branch is not officially maintained and that we strongly recommend that you do migrate to pkg(8)

The ports tree is now stage only

September 1, 2014 by · Leave a Comment 

The ports tree is now fully staged (only 2% has been left unstaged, marked as broken and will be removed from the ports tree if no PR to stage them are pending in bugzilla).

I would like to thank every committer and maintainers for their work on staging!
It allowed us to convert more than 23k packages to support stage in only 11 months!

Staging is a very important state, it allows us to right now be able to run quality testing scripts on the packages (which already allowed to fix tons of hidden problems) and it allows use to be able to build packages as a regular user!

It also opens the gates to new features that users have been requesting for many years:

  • flavors
  • multiple packages

Expect those features to happen in the near future.

Happy 20th birthday FreeBSD ports tree!

August 21, 2014 by · Leave a Comment 

It all started with this commit from Jordan Hubbard on August 21, 1994:

Commit my new ports make macros
Still not 100% complete yet by any means but fairly usable at this stage.

Twenty years later the ports tree is still there and actively
maintained. A video was prepared to celebrate the event and to thank
all of you who give some of their spare time and energy to the project!

Farewell beloved Canadian

August 12, 2014 by · Leave a Comment 

Last month, our beloved Canadian Thomas Abthorpe decided to step
down from his portmgr-secretary position. While I suspect this is secretly
related to his pool of Canadian jokes having dried up, the official reason is
that Thomas wants to focus more on his private and professional lives for the
moment. Needless to say, the whole ports community is in mourning.

pkg 1.3.0 out!

July 25, 2014 by · Leave a Comment 

Hi all,

I’m very please to announce the release of pkg 1.3.0
This version is the result of almost 9 month of hard work

Here are the statistics for the version:
- 373 files changed, 66973 insertions(+), 38512 deletions(-)
- 29 different contributors

Please not that for the first time I’m not the main contributor, and I would
like to particularly thanks Vsevold Stakhov for all the hard work he has done to
allow us to get this release out. I would like also to give a special thanks to
Andrej Zverev for the tons of hours spending on testing and cleaning the bug
tracker!

So much has happened that it is hard to summarize so I’ll try to highlight the
major points:
- New solver, now pkg has a real SAT solver able to automatically handle
conflicts and dynamically discover them. (yes pkg set -o is deprecated now)
- pkg install now able to install local files as well and resolve their
dependencies from the remote repositories
- Lots of parts of the code has been sandboxed
- Lots of rework to improve portability
- Package installation process has been reworked to be safer and handle properly
the schg flags
- Important modification of the locking system for finer grain locks
- Massive usage of libucl
- Simplification of the API
- Lots of improvements on the UI to provide a better user experience.
- Lots of improvements in multi repository mode
- pkg audit code has been moved into the library
- pkg -o A=B that will overwrite configuration file from cli
- The ui now support long options
- The unicity of a package is not anymore origin
- Tons of bug fixes
- Tons of behaviours fixes
- Way more!

Thank you to all contributors:
Alberto Villa, Alexandre Perrin, Andrej Zverev, Antoine Brodin, Brad Davis,
Bryan Drewery, Dag-Erling Smørgrav, Dmitry Marakasov, Elvira Khabirova, Jamie
Landeg Jones, Jilles Tjoelker, John Marino, Julien Laffaye, Mathieu Arnold,
Matthew Seaman, Maximilian Gaß, Michael Gehring, Michael Gmelin, Nicolas Szalay,
Rodrigo Osorio, Roman Naumann, Rui Paulo, Sean Channel, Stanislav E. Putrya,
Vsevolod Stakhov, Xin Li, coctic

Regards,
Bapt on behalf of the pkg@

BSDCan 2014 – Ports and Packages WG

July 18, 2014 by · Leave a Comment 

Baptiste Daroussin started the session with a status update on package building. All packages are now built with poudriere. The FreeBSD Foundation sponsored some large machines on which it takes around 16 hours to build a full tree. Each Wednesday at 01:00UTC the tree is snapshot and an incremental build is started for all supported released, the 2 stable branches (9 and 10) and quarterly branches for 9.x-RELEASE and 10.x-RELEASE. The catalogue is signed on a dedicated signing machine before upload. Packages can be downloaded from 4 mirrors (us-west, us-east, UK, and Russia) and feedback so far has been very positive.

He went on to note that ports people need better coordination with src people on ABI breakage. We currently only support i386 and amd64, with future plans for ARM and a MIPS variant. Distfiles are not currently mirrored (since fixed), and while it has seen no progress, it’s still a good idea to build a pkg of the ports tree itself.

pkg 1.3 will include a new solver, which will help 'pkg upgrade' understand that an old packages needs to be replaced with a newer one, with no more need for 'pkg set' and other chicanery. Cross building ports has been added to the ports tree, but is waiting for pkg-1.3. All the dangerous operations in pkg have now been sandboxed as well.

EOL for pkg_tools has been set for September 1st. An errata notice has gone out that adds a default pkg.conf and keys to all supported branches, and nagging delays have been added to ports.

Quarterly branches based on 3 month support cycle has been started on an evaluation basis. We’re still unsure about the manpower needed to maintain those. Every quarter a snapshot of the tree is created and only security fixed, build and runtime fixed, and upgrades to pkg are allowed to be committed to it. Using the MFH tag in a commit message will automatically send an approval request to portmgr and an mfh script on Tools/ makes it easy to do the merge.

Experience so far has been good, some minor issues to the insufficient testing. MFHs should only contain the above mentioned fixes; cleanups and other improvements should be done in separate commits only to HEAD. A policy needs to be written and announced about this. Do we want to automatically merge VuXML commits, or just remove VuXML from the branch and only use the one in HEAD?

A large number of new infrastructure changes have been introduces over the past few months, some of which require a huge migration of all ports. To speed these changes up, a new policy was set to allow some specific fixes to be committed without maintainer approval. Experience so far has been good, things actually are being fixed faster than before and not many maintainers have complained. There was agreement that the list of fixes allowed to be committed without explicit approval should be a specific whitelist published by portmgr, and not made too broad in scope.

Erwin Lansing quickly measured the temperature of the room on changing the default protocol for fetching distils from MASTER_SITE_BACKUP from ftp to http. Agreement all around and erwin committed the change.

Ben Kaduk gave an introduction and update on MIT’s Athena Environment with some food for thought. While currently not FreeBSD based, he would like to see it become so. Based on debian/ubuntu and rolled out on hundreds of machines, it now has it’s software split into about 150 different packages and metapackages.

Dag-Erling Smørgrav discussed changes to how dependencies are handled, especially splitting dependencies that are needed at install time (or staging time) and those needed at run time. This may break several things, but pkg-1.3 will come with better dependency tracking solving part of the problem.

Ed Maste presented the idea of “package transparency”, loosely based on Google’s Certificate Transparency. By logging certificate issuance to a log server, which can be publicly checked, domain owners can search for certificates issued for their domains, and notice when a certificate is issued without their authority. Can this model be extended to packages? Mostly useful for individually signed packages, while we currently only sign the catalogue. Can we do this with the current infrastructure?

Stacy Son gave an update on Qemu user mode, which is now working with Qemu 2.0.0. Both static and dynamic binaries are supported, though only a handful of system call are supported.

Baptiste introduced the idea of having pre-/post-install scripts be a library of services, like Casper, for common actions. This reduces the ability of maintainers to perform arbitrary actions and can be sandboxed easily. This would be a huge security improvement and could also enhance performance.

Cross building is coming along quite well and most of the tree should be able to be build by a simple 'make package'. Major blockers include perl and python.

Bryan Drewery talked about a design for a PortsCI system. The idea is that committer easily can schedule a build, be it an exp-run, reference, QAT, or other, either via a web interface or something similar to a pull request, which can fire off a build.

Steve Wills talked about using Jenkins for ports. The current system polls SVN for commits and batches several changes together for a build. It uses 8 bhyve VMs instances, but is slow. Sean Bruno commented that there are several package building clusters right now, can they be unified? Also how much hardware would be needed to speed up Jenkins? We could duse Jenkins as a fronted for the system Bryan just talked about. Also, it should be able to integrate with phabricator.

Erwin opened up the floor to talk about freebsd-version(1) once more. It was introduced as a mechanism to find out the version of user land currently running as uname -r only represents the kernel version, and would thus miss updates of the base system that do no touch the kernel. Unfortunately, freebsd-version(1) cannot really be used like this in all cases, it may work for freebsd-update, but not in general. No real solution was found this time either.

The session ended with a discussion about packaging the base system. It’s a target for FreeBSD 11, but lots of questions are still to be answered. What granularity to use? What should be packages into how many packages? How to handle options? Where do we put the metadata for this? How do upgrades work? How to replace shared libraries in multiuser mode? This part also included the quote of the day: “Our buildsystem is not a paragon of configurability, but a bunch of hacks that annoyed people the most.”

Thanks to all who participated in the working group, and thanks again to DK Hostmaster for sponsoring my trip to BSDCan this year, and see you at the Ports and Packages WG meet up at EuroBSDCon in Sofia in September.

Monthly dashboard

July 13, 2014 by · Leave a Comment 

Thursday morning, 5 AM, I couldn’t sleep. I thought I could use the time before
work to do something useful, so I started handling a few PRs for FreeBSD. After
a couple of commits, a warm shower, and just before heading to work, I quickly
browsed through my irc backlog and suddently got very sad: someone was angrily
asking why bug reports were being ignored for such a long time, pushing for his
own PR to finally be given some consideration.

Thinking about it in the bus to work I realized that this guy was right to
complain: when a bug is reported it should ideally be fixed right away. Still I
was feeling sad because being on the other side of the fence I know how much
dedication volunteers put into FreeBSD, but I was not sure everybody was aware
of this. I had to find something to express this dedication.

That’s how the idea of the monthly dashboard came: simple figures that can tell
a whole story. See for yourself with this dashboard that can be found in portmgr
monthly report for June 2014:


Monthly dashboard

Number of messages to portmgr@: 564 (+53%)
Number of commits on ports: 3,717 (+17%)
Number of ports PRs closed: 873 (+25%)
Active ports committers: 147 (+10%)


 

Isn’t it amazing? Nearly 4,000 updates on the ports tree and nearly 900 problem
reports closed in a single month!

That’s a tremendous amount of work done by our committers. Take Linux for
example: with more than twice as much contributors during the same period four
times less commits were applied to the Linux kernel than to the FreeBSD ports
tree [1].

Those figures pay tribute to our committers, and I am pleased to see that the
activity keeps growing. I personnally believe there has never been a better time
to start contributing to the FreeBSD ports tree with all those new features
currently being introduced. So come and join the party!


[1] Statistics taken from http://www.ohloh.net/p/linux:
962 commits done by 344 contributors (activity recorded from Jun 9 2014 to Jul 9 2014)

Getting to know your portmgr-lurker: William Grzybowski

July 5, 2014 by · Leave a Comment 

From July to the end of October two new lurkers will have the opportunity to get insights into FreeBSD portmgr internals, namely William Grzybowski and Nicola Vitale. William was the first to answer our interview so let’s get to know him a bit better.

 

Name

William Grzybowski

Committer name

wg

Inspiration for your IRC nick

my name initials

 TLD of origin

.br

 Current TLD (if different from above)

Occupation

Software engineer

 Blog

None

Inspiration for using FreeBSD

Stability and simplicity

Who was your first contact in FreeBSD

I don’t recall, too long ago!

Who was your mentor(s)

culot, jpaetzel

What was your most embarrassing moment in FreeBSD

Breaking INDEX, but hey, who ever didn’t? ;)

vi(m) /  emacs / other

vim

What keeps you motivated in FreeBSD

The passion of everyone in the zoo about it. You can very easily see a
guy angry by someone’s else commit like stealing candy from his son :)

Favorite musician/band

AC/DC

What book do you have on your bedside table

book? what is that? :)

coffee / tea / other

coffee, beer (but real beer, no corn!)

How would you describe yourself

Calm and mind-centered

sendmail / postfix / other

postfix

Do you have a hobby outside of FreeBSD

Play soccer, gym

What is your favorite TV show

Fringe

Claim to Fame

I was mentored by culot and I survived! Just kidding, I have none, I am a joke!

What did you have for breakfast today

An orange and a slice of yogurt cake

What sports team do you support

Gremio Foot-Ball Porto Alegrense, brazillian soccer team

What else do you do in the world of FreeBSD

I do work mostly in python ports and help closing PRs.

2014Q3 Branched

July 1, 2014 by · Leave a Comment 

The 2014Q3 branch has just been branched and the package builder has been
updated to use that branch. This means that the next update on the quarterly
packages will be on the 2014Q3 branch.

What happened during the last 3 months:
- 177 different committers have participated
- 9918 commits happened
- diffstat says: 23646 files changed, 554070 insertions(+), 577210 deletions(-)

What does that means for users:
- default Java is now 1.7
- massive conversion to stagedir (93% of the ports are now properly staged)
- massive improvement of the usage of libtool (which reduces a lot overlinking)
- new USES: mono, objc, drupal, gecko, cpe, gssapi, makeinfo
- new Keywords for plist: @sample, @shell
- LibreOffice has been updated to 4.2.5
- Firefox has been updated to 30.0
- Firefox-esr has been updated to 24.6
- Default postgresql has moved from 9.0 to 9.2
- nginx has been updated to 1.6.0
- Default lua is 5.2
- subversion has been split into multiple ports for each features
- On FreeBSD 9-STABLE and 10-STABLE the default xorg 1.12.4 (for default binary
packages it is still 1.7.7)
- Improved QA checking in the infrastructure
- Info files are handle correctly even if base has been built WITHOUT_INFO
- Ancient emacs version has been cleaned out

Getting to know your portmgr@ – Steve Wills

June 4, 2014 by · Leave a Comment 

It is my pleasure to introduce Steve Wills, the newest member of the portmgr team. Steve has done a tremendous work on the ports tree, especially in the field of testing and quality. Here is a short interview to get to know him better.

Name
Steve Wills

Committer name
swills

Inspiration for your IRC nick
Boring, it’s my userid.

TLD of origin
.us

Current TLD (if different from above)
Same.

Occupation
Sysadmin.

When did you join portmgr@
2014

Blog
Used to have one, use twitter more now (@swills)

Inspiration for using FreeBSD
Simplicty and learning.

Who was your first contact in FreeBSD
Can’t recall, it was ages ago.

Who was your mentor(s)
pgollucci

What was your most embarrassing moment in FreeBSD
Trying to migrate Ruby default version from 1.8 to 1.9 and having to roll back.

Boxers / Briefs / other
Heh, question assume survey taker is male, which I am, but I think we need to
work on diversity (but not in that “hey, let’s work on diversity and get some
women” way, but more in that we make something everyone wants to use)

What is your role in your circle of friends
The FreeBSD user. ;)

vi(m) / emacs / other
vi(m)

What keeps you motivated in FreeBSD
New users, new committers.

Favourite musician/band
I listen to a decent variety of stuff, but I suppose the thing I come back to
most is NIN.

What book do you have on your bedside table
I have an iPad by my bed, which I bought to read, but mostly I browse news on
it.

coffee / tea / other
Don’t drink caffeine, so don’t drink coffee much. I do drink good beer tho.

Do you have a guilty pleasure
Good dark chocolate. :)

How would you describe yourself
Mostly standard in many ways, husband, father, FreeBSD hacker, sysadmin, in
that order.

sendmail / postfix / other
Sendmail, tho dma is nice too.

Do you have a hobby outside of FreeBSD
Used to play guitar, still have one, don’t find time to pick it up much any
more.

What is your favourite TV show
Futurama

Claim to Fame
Ported Acidwarp from DOS to svgalib.

What did you have for breakfast today
Everything bagel with plain cream chese.

What sports team do you support
The only sport I watch is University of North Carolina Basketball.

What else do you do in the world of FreeBSD
ruby ports, perl ports sometimes

What can you tell us about yourself that most people don’t know
I was an employee at Red Hat way way back

Any parting words you want to share
Not really.

What is your .sig at the moment
Null.

Steve

Next Page »