Category Archives: Ports

Build just one FreeBSD port

I usually build my own packages with poudriere but it’s not fun to do on tiny boxes so I just do ‘pkg install ‘ on them and use upstream packages. One downside is, that package is build with default options. I recently ran into a situation where I wanted to change some options for just a single port.

Now, what is the minimal set of things in /usr/ports/ that I need to checkout to be able to config/build just one port?
Turns out to be:


/usr/ports/Mk
/usr/ports/Keywords
/usr/ports/Templates
/usr/ports/ports-mgmt/dialog4ports

And the port I want to actually build. Now I can ‘make config’, change the options and build/install that port without checking out entire ports tree.

Getting to know your portmgr@ – Steve Wills

It is my pleasure to introduce Steve Wills, the newest member of the portmgr team. Steve has done a tremendous work on the ports tree, especially in the field of testing and quality. Here is a short interview to get to know him better.

Name
Steve Wills

Committer name
swills

Inspiration for your IRC nick
Boring, it’s my userid.

TLD of origin
.us

Current TLD (if different from above)
Same.

Occupation
Sysadmin.

When did you join portmgr@
2014

Blog
Used to have one, use twitter more now (@swills)

Inspiration for using FreeBSD
Simplicty and learning.

Who was your first contact in FreeBSD
Can’t recall, it was ages ago.

Who was your mentor(s)
pgollucci

What was your most embarrassing moment in FreeBSD
Trying to migrate Ruby default version from 1.8 to 1.9 and having to roll back.

Boxers / Briefs / other
Heh, question assume survey taker is male, which I am, but I think we need to
work on diversity (but not in that “hey, let’s work on diversity and get some
women” way, but more in that we make something everyone wants to use)

What is your role in your circle of friends
The FreeBSD user. ;)

vi(m) / emacs / other
vi(m)

What keeps you motivated in FreeBSD
New users, new committers.

Favourite musician/band
I listen to a decent variety of stuff, but I suppose the thing I come back to
most is NIN.

What book do you have on your bedside table
I have an iPad by my bed, which I bought to read, but mostly I browse news on
it.

coffee / tea / other
Don’t drink caffeine, so don’t drink coffee much. I do drink good beer tho.

Do you have a guilty pleasure
Good dark chocolate. :)

How would you describe yourself
Mostly standard in many ways, husband, father, FreeBSD hacker, sysadmin, in
that order.

sendmail / postfix / other
Sendmail, tho dma is nice too.

Do you have a hobby outside of FreeBSD
Used to play guitar, still have one, don’t find time to pick it up much any
more.

What is your favourite TV show
Futurama

Claim to Fame
Ported Acidwarp from DOS to svgalib.

What did you have for breakfast today
Everything bagel with plain cream chese.

What sports team do you support
The only sport I watch is University of North Carolina Basketball.

What else do you do in the world of FreeBSD
ruby ports, perl ports sometimes

What can you tell us about yourself that most people don’t know
I was an employee at Red Hat way way back

Any parting words you want to share
Not really.

What is your .sig at the moment
Null.

Steve

Frederic Culot takes over as portmgr-secretary@

It is with great pleasure that the FreeBSD Ports Management Team announces that Frederic (culot@) Culot will take over responsibilities of team secretary effective immediately.

Frederic became a ports committer in October 2010, and joined the ranks of portmgr-lurkers@ in March 2014 as the shadow secretary.

Please drop him a note and congratulate him (or offer condolences).

 

Thomas
on behalf of portmgr@

 

portmgr-lurkers@ March 1 edition

The first intake of portmgr-lurkers@ is complete, and it is now time to start with the second round of our -lurkers.  Please join us in welcoming Alexey (danfe@) Dokuchaev and Frédéric (culot@) Culot to our ranks.

During this -lurker round, culot@ will be the shadow portmgr-secretary@, learning the finer points of the roles and responsibilities of the job.

Thomas
on behalf of portmgr@

Time to bid farewell to the old pkg_ tools

There comes a time in the life cycle of just about every software package that it has bee re-evaluated, refreshed, deprecated or just retired.

It is time that we bid farewell to the old pkg_* software that has been part of FreeBSD since the beginning, and has served us well. After years of development, testing, and playing, pkg(8) has become a suitable replacement.

Pkg is the Next Generation package management tool for FreeBSD. It is the replacement for the current pkg_info/pkg_create/pkg_add tools that ports use to register local packages and which provide remote packages. Its main goals are to facilitate remote binary package upgrades. It also works with ports without remote binary packages.

Pkg, combined with the quarterly release package sets, enables easy installation and safe upgrades for binary packages. Signed, binary packages are available for all supported FreeBSD releases on the i386 and  amd64 platforms from pkg.freebsd.org. Additionally, for those compiling ports from source, pkg’s new database format gives more fine-grained querying and management of installed software.

New features on the drawing board, like automatic pkg-plist generation, sub-packages, creating multiple packages containing different parts of a port from one build process, and flavours, being able to ask for e.g. a webserver, without directly specifying a specific one, cannot be implemented in the old pkg_* tools and those plans are currently on hold.

You are not obligated to switch to binary packages, if you still prefer to compile your own ports, it is a simple matter of installing ports-mgmt/pkg, run pkg2ng, add WITH_PKGNG=yes to your make.conf and use pkg <action> instead of pkg_<action>.

You can read more about pkgng on the FreeBSD wiki, https://wiki.freebsd.org/pkgng.

The decision has been made to allow the old pkg_* software to be EoL’d 6 months from now, at September 1, 2014 in all active FreeBSD branches.

Please start testing pkg(8) in your test environments before taking it live, you will find the benefits of full binary updates for your ports to be beneficial in a very short amount of time. Even if you prefer to compile from source, you will still reap the benefits of the modern packaging system.

Blanket approval to modernize the Ports Tree

In years gone by, and I am thinking of FreeBSD 7.0 specifically, portmgr@
gave some latitude to *ALL* committers to “just fix things” to get a port
into shape. In the case of 7.0, it was making ports build for gcc4.

What we have laying ahead of us is a ports tree in various states of modern
preparedness (new style USES=, stagefication, etc) and the old way of doing
ports (boo!).

We would like committers, and contributors to generate a PR and/or “just
fix” the old ports to update them to the new way of doing things regardless
of maintainership. We are looking for fixes in the following areas

- Convert to LIB_DEPENDS
- stagify ports
- convert things like USE_GMAKE -> USES=gmake USE_DOS2unix -> USES=dos2unix
etc

This can be done with implicit portmgr@ blanket approval, and without
maintainer approval.

Please, however, respect some boundaries, do not change ports belonging to
kde@, gnome@ or x11@. These teams work in private repos that may have
changes pending.

Also, cross reference GNATS, to see if a port has an open PR that you can
factor into the fix. It is important to stress here that we *DO NOT* want
to invalidate existing patches that a maintainer has offered up or already
approved.

If the change is very trivial AND has been tested, “just fix it”. One of
the strengths of the Ports Collection is it’s volunteer maintainers, if you
make a change, regardless of how trivial, just send a courtesy email to the
maintainer.

The Ports Management Team 2013-11-25 15:38:23

In this interview, we talk to Martin Wilke, aka miwi, the high quality automated PR and commit bot.

Name

Martin Wilke

Committer name

miwi

Inspiration for your IRC nick

MartIn WIlke

TLD of origin

Ger

Current TLD (if different from above)

My

Occupation

System Administrator

When did you join portmgr@

Jun 4 08:35:57 UTC 2009 (actually a few hours earlier because we did
know that already)

Blog

http://miwi.cc (I did not write anything in ages ^^)

Inspiration for using FreeBSD

Can I use a shout cart?
http://miwi.cc/2012/03/why-freebsd-my-personal-view/ that should explain it
quite well I think.

Who was your first contact in FreeBSD

To be honest, I think many because I spam GNATS quiet heavily :)

Who was your mentor(s)

Krion (wave)

What was your most embarrassing moment in FreeBSD

If you want to know, see my commit history, you will find a lot :(

Boxers / Briefs / other

Boxers

vi(m) / emacs / other

vi(m)/mcedit :)

What keeps you motivated in FreeBSD

Still the community

Favourite musician/band

Gerneral I still like and listen a lot to of German Hip Hop (I still feel
very young!)

What book do you have on your bedside table.

Book? Can I eat that?

coffee / tea / other

black coffee

Do you have a guilty pleasure

Hm..

How would you describe yourself

Shy :)

sendmail / postfix / other

postfix

Do you have a hobby outside of FreeBSD

When you have kids there is not much time left for hobbies :)

What is your favourite TV show

Stargate*, House

Claim to Fame

miwibot and friends ..

What did you have for breakfast today

Rotti Chanai, Ice Milo

What sports team do you support

I keep on following F1 and hope a other German Driver will win ..

What else do you do in the world of FreeBSD

Trying to clean up bapts stuff *gg* (joking), lately I helped a lot
on the cluster setup / package packaging.

What can you tell us about yourself that most people don’t know

I canâ~@~Yt else you will know :)

<Editor note: this is a copy and paste, something may have been lost in translation>

Any parting words you want to share.

Have a nice day!

What is your .sig at the moment

Mess with the best, die like the rest :).

The Ports Management Team 2013-10-03 14:05:32

For the sake of being able to have clean working binary packages, pkgng has had to use a dirty hack: origin is used has an identifier instead of the package name.

Why: How to determine than ImageMagick-nox11 and ImageMagick are the same package? and if I don’t know they are the same package how to upgrade them properly? same goes how the package manager can know that mysql-client-5.2.1 is not an upgrade of mysql-client-5.0.1?

How can pkgng determine which subversion is the right version to use: ‘pkg install subversion’ will propose all possible subversion, in that case the higher version is probably the default version, but in perl case, the higher version is probably not the one you want.

Some packages are totally different project but have the same package name… How can pkgng differenciate them?

So please stop using LATEST_LINK to differenciate the different port but directly make _unique_ package names, pkgng will switch back to package name as soon as possible (that will also solve the ugly and stupid pkg set -o).

There is multiple possibility to make sure to properly handle multiple versions of packages:

  1. suffix everything with part of the version (like tcl)
  2. suffix everything but the default with part of the version
  3. have only 2 or 3 different version: project-legacy project project-devel

Please stop renaming the packages based on options! Options are now registered with the package with both pkgng and pkg_install, please stop adding suffix base on options!

Think about the end user, he will try to install a package, it should be as transparent for him as possible.

I is really important to change that as quick as possible. This page on the wiki reference all the port with conflicting names.

The Ports Management Team 2013-10-03 14:05:12

You may has notice that staging has hit the ports tree, staging is something really important, all packages system are using that feature for eons, sometime called DESTDIR sometime called FAKEDIR.

Staging is consistent in adding a new step while building packages: install everything into ${STAGEDIR}. Then we can directly create packages out of that directory without having to install into /. What the implementation does is:

With pkg_install (legacy package tools):

  • Create a package from the stage directory
  • Install that package.

With pkgng:

  • Create a package from that stage directory

OR

  • Install to / (pkgng can consider the stage directory as if it is a package)

What it means is, that it is the end of the bad plist, only things in plist are really installed, this is the end of the broken packages that break the system because they fail in the middle of make install as a package will only be installed once we are sure the build process properly pass.

It allows right now to build and create a package without the need to be root.
It gives us new targets:

  • make makeplist (to create the pkg-plist)
  • make check-orphans (to print what is in the stage directory but not in pkg-plist)

It reduces code duplication: in the end the installation is being done via a package installation meaning that PKG-MESSAGE is automatically printed, all pre/post installation scripts are executed.

It reduces patches: no need anymore to patch upstream Makefiles to avoid installing the DOCS just do not list them in pkg-plist or conditionnaly list them in the pkg-plist.

This also allows lots of new features to come:

  • Allow to create sub-packages
  • Allow to create debuginfo packages
  • Allow to do a lot of sanity check in the staging area to improve our QA

To convert your ports refer to: https://wiki.freebsd.org/ports/StageDir

Along with stage you have noticed that MAN* and MLINKS has gone, and now all the manpages should be added to the plist. BTW MANCOMPRESSED is also not necessary now.

There is 3 reason behind making it go and not being replaced:

  1. The initial goal of those macros was to be able to compress the right manpages and to handle the different hardlinks/symlinks between those pages. Because it was directly installed in / rather than using a stage directory, it was necessary to at a point list them. and to properly track the different links in MLINKS (which wasn’t done properly in most ports btw :) ). With stage, we have a new compress-man which does it properly on its own without the need to get the list of the manpages, without the need of an explicit declaration of the links. This syntax to handle localized man pages was also terrific :) and if you have a port mixing manpages in “non regular” and “regular” localtion, it was totally messing
  2. Consistency, a port is basically: some metadata, a plist and some operations to perform. for metadata all metadata are in the Makefile, all operations are in the Makefile but plist can be a mix of pkg-plist and Makefile, this is inconsistent, it makes sometime hard to figure out why a file has been added to the plist etc. Also this makes us having tons of targets define to handle those extras entries and results in highly inefficient make(1) usage.
  3. Stage is also a first step to go to sub-packages, sub-packages will basically be: 1 port able to produce N packages, to be able to do this we will use multiple plist, having the files properly defined already in plist will help that. Having files defined in macros on Makefile will make it hard todetermine which one should go in which plist.

Last thing I would like to add about it: I don’t see the difference personnaly between listing N lines of manpages into Makefile MAN* macros where btw you have to manually define the categories where to put them and adding those N lines
directly into the plist where make makeplist and/or make check-orphans can help you getting the exact lines automatically?

Stash for svn

When hacking on the ports tree or on the sources, you often have unfinished patches you are testing step by step.

I'm also hacking on something unfinished and then some other area needs some fixes with a higher priority and in the same time some people are asking for some testing/review on their own patches. So I need to quickly interrupt what I was working on get back to a clean tree, and switch from patches to patches.

While doing this is easy with git, fossil or mercurial it is more complicated with svn. The feature in particular I use on fossil/git for that is the stash feature.

So I wrote my own stash for svn, and because of mobility I was willing to be able to share my patches across boxes, so I have made stash able to be under a vcs itself, with support for git, hg, fossil and svn as a vcs for the stash.

How does it works: The stash command will discover that .svn of the working copy you are working on and will create a patches subdirectory.

Now imagine that directory itself contains a .hg, .fslckout, a .git or a .svn then stash will know it is being under vcs control.

$ stash save <name> [-u] [files...]

This will create a .svn/patches/.patch file using svn diff --git on the specified files (if none is provided it will diff the current directory the stash command is being run on).

Once the diff created it will rollback the tree (on specified files or current directory) to the clean state before any modification.)

By default it will not overwrite a patch with the same name except is -u is provided on the command line.

If the stash directory is under a vcs control then a add/commit (or just commit in case of update) will be performed in the stash directory.

$ stash ls

List all the available patches.

$ stash show <name>

Print on stdout the content of a given patch

$ stash apply <name>

Apply a specified patch on the working copy using svn patch --strip 1 from the root of the working copy.

$ stash rm <name>

Remove/destroy a patch from the stash directory. In case the stash directory is under vcs control then the proper rm command followed by the needed commit will be performed.

$ stash pop <name>

It is the equivalent of stash apply followed by stash rm this is useful when your patch is finished and you want to commit it directly.

$ stash push <name>

Push (scp) the patch on a remote site (currently freefall is hard coded :)

$ stash sync <name>

This command is only useful if the stash directory is under vcs control, it performs the necessary pull/push mechanism depending on the VCS used.

I use fossil to maintain the stash script And here is for example my repository of patches for the ports tree

No git svn won't have worked in my case for multiple reason: 1/ want something flexible which can also only work with svn 2/ the ports tree can work properly with git svn (properties setting adding new files etc will not work as one expect) 3/ I want to use fossil for the stash, other might prefer svn or hg.

Disclaimer: hg and git support hasn't been tested yet, patches welcome to fix them if needed. if you want to add support for your own favorite vcs just them me the patches I'll integrate them.

Conversion to new options: done

It took a year but at least it is done! Huge thanks to all people involved in the new Option Framework.

So now what we have:

  • A consisten way to set options in make.conf (OPTIONS_SET, OPTIONS_UNSET, ${UNIQUENAME}_SET, ${UNIQUENAME}_UNSET )
  • 0-1 options
  • only 1 options
  • 0 or N options
  • 1 or N options
  • helpers to sanely create slaves (OPTIONS_SLAVE aand OPTIONS_EXCLUDE)

What still needs to be done:

  • Stop popping the dialog only for global options
  • Fix the very old bug about OPTIONSFILE name dansing (PKGNAMEPREFIX changing)
  • Convert NOPORTEXAMPLES, NOPORTDOCS and WITHOUT_NLS to options.

With the previous done, my 2 main priorities for FreeBSD now are:

  • Get pkg 1.1.0 released
  • Introduce the next big and long awaited change to the ports tree: stage directory

The Ports Management Team 2013-03-06 17:25:04

The FreeBSD Ports Management Team wishes to remind users that February 28 was the end of support for the Ports Collection for both FreeBSD 7.4 RELEASE and the FreeBSD 7.x STABLE branch. Neither the infrastructure nor individual ports are guaranteed to work on these FreeBSD versions after that date. A tag has be created for users who cannot upgrade for some reason, at which time these users are advised to stop tracking the latest ports repository and use the RELEASE_7_EOL tag instead.

Read more at http://lists.freebsd.org/pipermail/freebsd-ports-announce/2013-March/000051.html

The Ports Management Team 2013-02-21 04:09:42

Mark Linimon, aka linimon@,  recently stepped down from his duties on the FreeBSD Ports Management Team.

Mark joined the team back in 2004, providing nine years of continuity to the Ports Infrastructure.  Among the many things Mark did, was maintaining and documenting the current portbuild system that the team uses for -exp runs and package building.  With his other bugbuster@ hat on, he played middle man contacting maintainers of BROKEN and DEPRECATED ports.

On behalf of the Ports Management team, we would like to thank Mark for his many years of service and dedication, his contributions will be greatly missed.

 

Thomas

on behalf of portmgr@

The Ports Management Team 2012-10-19 15:28:17

The FreeBSD Ports Management team is pleased to welcome Bernhard Froelich, aka decke@, to it’s ranks.

Bernhard was a long time ports contributor, and received his ports commit bit back in March 2010.

More recently, Bernhard was the one responsible for bringing us Redports.org shared tinderbox.

Please join me in welcoming decke@ to the team.

Thomas
on behalf of portmgr@

The Ports Management Team 2012-10-19 15:25:36

Pav Lucistnik, aka pav@, recently stepped down from his roll on the FreeBSD Ports Management team.

Pav started on portmgr back in November 2006, he was the one responsible for many of the -exp runs over the years. His most dubious claim to fame was talking over the responsibility of krismails. We all looked forwward to our pavmails, right?

On behalf of the Ports Management team, we want to thank Pav for his years of  service, he will be missed.
Thomas
on behalf of portmgr@

The Ports Management Team 2012-10-10 19:41:24

FreeBSD 9.1 RC2 has been pulicly announced, it is now time for the the Ports Feature Freeze.

Normal upgrade, new ports, and changes that do not affect other ports will be allowed without prior approval, but with the extra

Feature safe: yes

tag in the commit message. Any commit that is sweeping, that is, touches a large number of ports, infrastructural changes, commits to ports with unusually high number of dependencies, and any other commit that requires the rebuilding of many packages will not be allowed without prior explicit approval from portmgr@.

Check out http://www.freebsd.org/portmgr/implementation.html#sweeping_changes for what constiutes a sweeping change.
Thomas
on behalf of portmgr@

The Ports Management Team 2012-09-17 16:21:45

It was recently posted on, http://blogs.freebsdish.org/portmgr/2012/09/01/change-to-the-header-in-ports-makefiles/ that we would adopt a new header for the ports Makefiles. The initial discussion seemed to show enough support for the idea of completely stripping the header, leaving only the $FreeBSD$ tag. After the announcement was made, more people stated strong feelings that when and where possible attribution be maintained in the header.

A private discussion was held among ports committers, and while opinions were as varied as the individuals who shared them, it was decided to unify on a two line header.

# Created by: J.Q. Public <[email protected]>
# $FreeBSD$

The Whom line from the classic six line header becomes Created By.

Sometimes, as a result of a repocopy, or changed maintainership, the Created By and MAINTAINER is no longer in synchronisation. To avoid confusion, the first line can be removed, optionally leaving us with a one line header.

# $FreeBSD$

Removing the line of attribution is to be done only at the consent/request of the original contributor.

As before, we ask this header only be updated in conjunction with a regular update, as we do not want any unnecessary churn to the repo prior to the pending Ports Feature Freeze.

Thomas
on behalf of portmgr@