Staging

October 3, 2013 by · Leave a Comment 

You may has notice that staging has hit the ports tree, staging is something really important, all packages system are using that feature for eons, sometime called DESTDIR sometime called FAKEDIR.

Staging is consistent in adding a new step while building packages: install everything into ${STAGEDIR}. Then we can directly create packages out of that directory without having to install into /. What the implementation does is:

With pkg_install (legacy package tools):

  • Create a package from the stage directory
  • Install that package.

With pkgng:

  • Create a package from that stage directory

OR

  • Install to / (pkgng can consider the stage directory as if it is a package)

What it means is, that it is the end of the bad plist, only things in plist are really installed, this is the end of the broken packages that break the system because they fail in the middle of make install as a package will only be installed once we are sure the build process properly pass.

It allows right now to build and create a package without the need to be root.
It gives us new targets:

  • make makeplist (to create the pkg-plist)
  • make check-orphans (to print what is in the stage directory but not in pkg-plist)

It reduces code duplication: in the end the installation is being done via a package installation meaning that PKG-MESSAGE is automatically printed, all pre/post installation scripts are executed.

It reduces patches: no need anymore to patch upstream Makefiles to avoid installing the DOCS just do not list them in pkg-plist or conditionnaly list them in the pkg-plist.

This also allows lots of new features to come:

  • Allow to create sub-packages
  • Allow to create debuginfo packages
  • Allow to do a lot of sanity check in the staging area to improve our QA

To convert your ports refer to: https://wiki.freebsd.org/ports/StageDir

Along with stage you have noticed that MAN* and MLINKS has gone, and now all the manpages should be added to the plist. BTW MANCOMPRESSED is also not necessary now.

There is 3 reason behind making it go and not being replaced:

  1. The initial goal of those macros was to be able to compress the right manpages and to handle the different hardlinks/symlinks between those pages. Because it was directly installed in / rather than using a stage directory, it was necessary to at a point list them. and to properly track the different links in MLINKS (which wasn’t done properly in most ports btw :)). With stage, we have a new compress-man which does it properly on its own without the need to get the list of the manpages, without the need of an explicit declaration of the links. This syntax to handle localized man pages was also terrific :) and if you have a port mixing manpages in “non regular” and “regular” localtion, it was totally messing
  2. Consistency, a port is basically: some metadata, a plist and some operations to perform. for metadata all metadata are in the Makefile, all operations are in the Makefile but plist can be a mix of pkg-plist and Makefile, this is inconsistent, it makes sometime hard to figure out why a file has been added to the plist etc. Also this makes us having tons of targets define to handle those extras entries and results in highly inefficient make(1) usage.
  3. Stage is also a first step to go to sub-packages, sub-packages will basically be: 1 port able to produce N packages, to be able to do this we will use multiple plist, having the files properly defined already in plist will help that. Having files defined in macros on Makefile will make it hard todetermine which one should go in which plist.

Last thing I would like to add about it: I don’t see the difference personnaly between listing N lines of manpages into Makefile MAN* macros where btw you have to manually define the categories where to put them and adding those N lines
directly into the plist where make makeplist and/or make check-orphans can help you getting the exact lines automatically?