Time to bid farewell to the old pkg_ tools

February 3, 2014 by · 14 Comments 

There comes a time in the life cycle of just about every software package that it has bee re-evaluated, refreshed, deprecated or just retired.

It is time that we bid farewell to the old pkg_* software that has been part of FreeBSD since the beginning, and has served us well. After years of development, testing, and playing, pkg(8) has become a suitable replacement.

Pkg is the Next Generation package management tool for FreeBSD. It is the replacement for the current pkg_info/pkg_create/pkg_add tools that ports use to register local packages and which provide remote packages. Its main goals are to facilitate remote binary package upgrades. It also works with ports without remote binary packages.

Pkg, combined with the quarterly release package sets, enables easy installation and safe upgrades for binary packages. Signed, binary packages are available for all supported FreeBSD releases on the i386 and  amd64 platforms from pkg.freebsd.org. Additionally, for those compiling ports from source, pkg’s new database format gives more fine-grained querying and management of installed software.

New features on the drawing board, like automatic pkg-plist generation, sub-packages, creating multiple packages containing different parts of a port from one build process, and flavours, being able to ask for e.g. a webserver, without directly specifying a specific one, cannot be implemented in the old pkg_* tools and those plans are currently on hold.

You are not obligated to switch to binary packages, if you still prefer to compile your own ports, it is a simple matter of installing ports-mgmt/pkg, run pkg2ng, add WITH_PKGNG=yes to your make.conf and use pkg <action> instead of pkg_<action>.

You can read more about pkgng on the FreeBSD wiki, https://wiki.freebsd.org/pkgng.

The decision has been made to allow the old pkg_* software to be EoL’d 6 months from now, at September 1, 2014 in all active FreeBSD branches.

Please start testing pkg(8) in your test environments before taking it live, you will find the benefits of full binary updates for your ports to be beneficial in a very short amount of time. Even if you prefer to compile from source, you will still reap the benefits of the modern packaging system.

Be Sociable, Share!

About tabthorpe

Comments

14 Responses to “Time to bid farewell to the old pkg_ tools”
  1. Walter Hop says:

    Excellent. pkg is already running very smoothly. It will make the lives of those managing large(ish) deployments much better. Thanks to the smart design of the base pkg bootstrapper and keeping pkg proper in ports, it will be easy to roll out many more improvements in the future.

    Personally, I’m longing for multi-repository use with priorities (e.g. first try the local repo, then fall back to the FreeBSD repo) which should alleviate the necessity of building large repositories. You could just build that single package with that custom option that you need.

    Something like port ‘variants’ or ‘option sets’ would also be highly desirable, so for instance a user might install svn+apache24 or php5+apache24 binary packages which include the necessary Apache module (if I’m correct, the official packages don’t allow this right now, but it’s a huge use case for ISPs). At the other hand, maybe this could be handled without architectural changes by hacking together meta-ports…

    Both of these features could get tricky though, as they can affect the dependency graph, so I guess it’s harder than it seems. For now, running your own pkg repository is the best way to get everything out of pkg. Fortunately, that’s super easy.

  2. Thorsten says:

    Hi,

    does EOL mean, pkg_* will be removed or will only be deactived as default?

    Thanks,
    Thorsten

  3. Bapt says:

    Support will be totally removed

  4. Helge says:

    But will it *stay* in the base system? Or will it be physically removed? If the latter, in which release will the pkg_* tools be removed, and in which will they be kept?

  5. GreenReaper says:

    Won’t compile for FreeBSD 8.1 (what my jail is on – don’t control the host) due to the issues here:
    http://michaeljmaguire.mjws.net/2013/06/28/pkg-not-building-on-freebsd-8/

    Then there’s some other things, probably resulting in part from being on the latest gcc:
    annotate.c: In function ‘read_input’:
    annotate.c:178 error: void value not ignored as it ought to be

    I’m guessing that before 8.3 (or 8.2) sbuf_finish returned void, not int. Presumably Dragonfly has the same issue given the ifdef in there..

    I gave up when it started talking about archive_read_free and archive_write_free. Could probably have got it to work, but clearly it’s not meant to be. I guess I should hassle our host about upgrading! :-)

  6. Clueless User says:

    This is all great. How about an explanation ANYWHERE on how to use it?!?! I assume this is great if you are using a newer release where I assume everything is setup for you. But how about those of us on older systems that are still supported, such as FreeBSD 8.3?!?!?!

    after spending a bunch of time looking around google, I’ve determined I need to install /usr/ports/ports-mgmt/pkg. Great.

    add WITH_PKGNG=yes to your make.conf as you specify above. Great.

    Run pkg2ng. Great. Wow that spits out a lot of scary messages, which I assume because I can’t find information anywhere, that I can safely ignore.

    Great, let’s see how this new amazing technology works!
    pkg install myfavoritepkg:
    Updating repository catalogue
    No valid repository found.
    @#$@#$ Now what?!?!

    Go looking for an answer. not finding anything that means much. No helpful howto Information for getting started quickly. Start taking stabs in the dark based on results from google.

    copy pkg.conf.example to pkg.conf, add PACKAGESITE: http://pkgbeta.freebsd.org/freebsd-8-i386/latest
    based off the result found for 9-amd64, after looking and thinking it seems sane. (wondering why everything points to pkgbeta?!?!?!? That seems scary.)

    pkg install myfavoritepkg:
    pkg: PACKAGESITE in pkg.conf is deprecated. Please create a repository configuration file
    Updating repository catalogue
    pkg: Warning: use of http:// URL scheme with SRV records is deprecated: switch to pkg+http://
    pkg: http://pkgbeta.freebsd.org/freebsd-8-i386/latest/digests.txz: Not Found
    pkg: Unable to find catalogs
    @#$@$#@!#$!#@$!!!

    Now thinking this new shiny awesome thing needs to die in a fire!

    So I decide to post here. Please note, a lot of my comments are for comic relief. It’s great that the team wants to the force the change, it seems like it may be some decent kit, but could someone please PLEASE update this post / add a comment to add useful information on how to properly get started, for those of us who aren’t dedicated FreeBSD engineers, who wish to comply with the scary Upgrade Now or die in a fire message that ports give while trying to upgrade.

    (BTW, I found all kinds of great information on creating my own pkg repo. But that’s not what I want to do “yet”).

    Please don’t take this post personally, it’s just venting my frustration. If the ports tree tells you to go to this url for information, there should be some helpful information on getting started. installing ports-mgmt/pkg should have some more helpful output such as: to enable the default package portal, do this: blah blah blah.

    It’s just not a good end user experience..
    … I’m off to start man’ing everything related to pkg to see if I can figure out what it is I’m expected to do to enable the default repo…

  7. Clueless User says:

    Meh, so it seems I glossed over the linked article which did tell specifically what needed to be added, so here is the direct url: https://wiki.freebsd.org/pkgng#Availability_of_binary_pkgs_for_Download (which could be better highlighted/named, such as “Adding the FreeBSD Default Repository”. pkg primer was eye catching, sounded exactly what I was looking for, but was irrelevant to a newb. Lead to “Switching to pkgng”, which also let this poor newb down. and incorrectly tells you to set PACKAGESITE, but doesn’t tell you to what.)

    and for those who just want to get it installed and not have to read a lot of documentation. I found the begging of this article to be an exact step-by-step procedure. Just follow it along until you get to “Add pkgng as package provider in Puppet”, enjoy your working pkg manager: http://www.fitzdsl.net/en/2013/11/utiliser-pkgng-sous-freebsd-avec-puppet/

  8. Jeremy Chadwick says:

    I came here because a “WARNING” message began to be emit by the ports/Mk infrastructure at some point in time. I was shocked to see a “random blog post” being mentioned in such an official repo, rather than something on freebsd.org directly.

    Two questions/comments:

    1. The directions provided by Thomas for those who “want to use pkg with ports” (ex. install ports-mgmt/pkg, pkg2ng, etc.) aren’t anywhere on the freebsd.org pkgng wiki page. In fact, this is the first time I’ve seen mention of this procedure. Why is that? That should really be documented somewhere official on freebsd.org.

    2. Have the pkg and/or SQLite-related problems been solved? An example from over a year ago:

    http://lists.freebsd.org/pipermail/freebsd-ports/2012-December/080001.html

    Many of the lower-level problems mentioned on the mailing list about pkg and/or SQLite get responses then die off without any mention of solution or resolution. These kinds of things need follow-ups, if not by bapt@ then by someone else involved in the project. I know time/resources are limited, but we’re talking about a complete package replacement system that’s now being imposed. There is now a responsibility to treat this like production.

  9. inof says:

    With the old pkg system, I can type “ls -lrt /var/db/pkg/*/+DESC” to see the ports in the order in which they were installed, with the lattest at the bottom. In fact I have a shell alias for that command. Is this possible with pkgNG, too? Can I list ports installed on one particular day?

    And second, I often “fake” a port update by simply renaming the directory in /var/db/pkg and editing the +CONTENTS and +REQUIRED_BY files of affected ports (yeah, I have a script for this, too). For example, I do this when the version of a port was bumped because a dependency was updated, but I have disabled that dependency in the port’s options anyway, so rebuilding the port would change nothing and just be a waste of time and resources. But I don’t want the port to stay in the “ports that need to be updated” list that gets mailed regularly, so I “fake” the update. When switching to pkgNG, I assume I have to manipulate the SQL database directly in order to achieve the same thing, right?

    BTW, what would be the proper place to ask questions like this (mailing list, forum, …)?

  10. Mark says:

    /usr/ports/ports-mgmt/pkg/work/pkg-1.2.6/libpkg/../external/sqlite/sqlite3.c:23598: error: ‘posix_fallocate’ undeclared here (not in a function)

  11. Gerhard Schmidt says:

    Sorry To hear this. The Old Package format is one of the Main reasons i have to use FreeBSD. Because it’s rock stable and one corrupt file doesn’t kill your whole Package system. I Know querieing a database is much easier than scanning thousands of files. But losing one of these thousends of file is an inconvience. Losing the database is a major disaster.

    I Know everyone is proud of the software the write, but why force people to move from a well tested an not failing system.

    I’m in the progress of migrating our build and autoinstall system to FreeBSD 10. An the major problem I’m encountering is the pkgng system. If it’s the default for 10 why isn’t it in the base system. Oh there is a pkg in the base but it does only try to fetch the real pkg off the net.

    Our auto install system does install a machine whit FreeBSD from via PXE Boot and NFS and after the Reboot the system is Production ready.

    The First Problem was upgrading from FreeBSD 8 to 9 because BSDInstall has no functions to install Packages. This i solved by simply calling pkg_add from the install script and everything worked fine.

    Now in FreeBSD 10 even pkg_add is gone and the pkg command there needs the internet to do it’s work, and needs the user to answer a question, not so good in an closed install environment without user supervision.

    The only way i found was to unpack the pkgng package manually on the system to be installed and forced reinstalling it after the unpack to register it properly.

    The old system was convenient, rock stable and well tested. So why the move.

  12. Gerhard Schmidt says:

    And I just missed another problem. I have software in my production environment that still needs python2.4. up til now i can use an old ports three where python2.4 still in and produce packages for the actual FreeBSD and install them. With removing the old pkg_* tools from newer systems this will no longer work and we a cut of from the Main development in FreeBSD because we can’t upgrade any without much more inconvenience.

    I didn’t find a way to do this a newly installed FreeBSD 10 machine. as the Old ports Tree doesn’t support pkgng and the new ports three doesn’t support python2.4.

    is it possible to install the pkg_* tools on a fresh installed FreeBSD 10 machine

  13. Happy Valentines Day says:

    Maybe I’m dumb but I didn’t feel like you answered Thorsten’s question. Are you wiping it from FreeBSD versions post-9/2014 or will it still be there, but pkg will be there as well as default and as the recommended use. Support means lots of things. I understand in the context of FreeBSD updates maybe it means one thing to you and others, but I am just dumb. Thanks.

Trackbacks

Check out what others are saying about this post...