Getting to know your portmgr-lurker: ehaupt@

November 8, 2014 by · 1 Comment 

Let us welcome Emanuel, our second lurker who will learn a bit more about portmgr duties for the next four months and who started by answering our usual questionnaire.

 

Name

Emanuel Haupt

Committer name

ehaupt

Inspiration for your IRC nick

Same as my default UID so that people can find me.

TLD of origin

.ch

Occupation

System Engineer

When did you join portmgr@

Beginning of november 2014 as a lurker.

Inspiration for using FreeBSD

It’s been my primary server/desktop OS since years. I always liked the
documentation and found things generally easier to achieve than with
Linux. I was also fascinated by ports. At the time I was manually
downloading solaris packages from “sun freeware” when someone showed me
ports. I think it is no surprise that I switched to FreeBSD. I always
found the community to be very friendly and helpful. Finally with pkgng
I feel the same sense of excitement all over again.

Who was your first contact in FreeBSD

Pav Lucistnik (pav)

Who was your mentor(s)

Roman Bogorodskiy (novel)

What was your most embarrassing moment in FreeBSD

Can’t think of any particular one. In general breaking things tends to
be embarrassing.

Boxers / Briefs / other

Boxers

vi(m) /  emacs / other

Mostly nvi but more and more vim.

What keeps you motivated in FreeBSD

pkgng, poudriere, the friendly and helpful community, ZFS, geli,
stability of the OS to name just a few.

What book do you have on your bedside table

Arnaldur Indriðason – The Draining Lake

coffee / tea / other

Coffee!!!

Do you have a guilty pleasure

Reddit and coin mining.

How would you describe yourself

Sysadmin, traveller, adventurer, motorcycler, dog person.

sendmail / postfix / other

sendmail.

Do you have a hobby outside of FreeBSD

I am a passionate motorcycler. I love riding my motorcycle in the more
mountainous regions of Europe. After a long day at work you often see
me on my motorcycle riding towards the sunset. I also have a
fascination for the nordic culture and literature. I’m taking Swedish
lessons since 2011.

Claim to Fame

Driving from Oberstaufen, Germany to Amman/Jordan using no highways in
3 weeks in a 20 year old Audi A4. Maintaining 195 ports and keeping
them all up to date and working.

What did you have for breakfast today

Swiss-Muesli with Coffee

What sports team do you support

None.

What else do you do in the world of FreeBSD

Porting and maintaining ports that I’m interested in.

Any parting words you want to share

I’m just glad to have the opportunity to work with so many highly
skilled people on the FreeBSD project.

Getting to know your portmgr-lurker: ak@

November 4, 2014 by · Leave a Comment 

From now on and for the next four months the FreeBSD ports team is pleased to welcome two new portmgr-lurkers: ak@ and ehaupt@. Alex was the first to answer our questionnaire so let’s get to know him a bit better.

 

Name
Alex (Олександр)

Committer name
ak

Inspiration for your IRC nick
Committer name

TLD of origin
ua

Current TLD (if different from above)
es

Occupation
Independent contractor

When did you join portmgr@
Just a lurker atm, 2014-11-01

Blog
None

Inspiration for using FreeBSD
I tried linux first.

Who was your first contact in FreeBSD
itetcu

Who was your mentor(s)
eadler, itetcu. Thanks, guys.

What was your most embarrassing moment in FreeBSD
I broke the INDEX, twice.

vi(m) /  emacs / other
vi(m)

What keeps you motivated in FreeBSD
It sucks less.

Favourite musician/band
Ritchie Blackmore/Queen

What book do you have on your bedside table
A Night in the Lonesome October (The Halloween was a few days ago).

coffee / tea / other
Tea, green.

Do you have a guilty pleasure
Sometimes I watch The Muppets instead of doing something useful.

How would you describe yourself
Too lazy.

sendmail / postfix / other
exim

Do you have a hobby outside of FreeBSD
Tons. The bikes, skydiving, amateur Martial Arts, poetry.

What is your favourite TV show
The Muppets

Claim to Fame
Nothing so far.

What did you have for breakfast today
Fried rice with mushrooms

What sports team do you support
I don’t support teams, I play when I can (regrettably, not that often lately).

What else do you do in the world of FreeBSD
Fiddling with portlint, xorg, zfs filestorages, various odd jobs.

Any parting words you want to share
Have more fun.

What is your .sig at the moment
Alex

pkg(8) is now the only package management tool

September 1, 2014 by · 2 Comments 

The ports tree has been modified to only support pkg(8) as package management system for all supported version of FreeBSD.

if you were still using pkg_install (pkg_* tools) you will have to upgrade your system.

The simplest way is

cd /usr/ports/ports-mgmt/pkg
make install

then run

pkg2ng

You will have lots of warning, don’t be scared, they are expected, pkg_*  databases used to get easily mangled. pkg2ng is most of the time able to deal
with it.

If however you encounter a problem then please report to [email protected]

A tag has been applied to the ports tree if you need to get the latest ports tree before the EOL of pkg_install:

https://svn.FreeBSD.org/ports/tags/PKG_INSTALL_EOL

A branch has been created if some committers want to provides updates on the for pkg_install users:

https://svn.FreeBSD.org/ports/branches/pkg_install

Please note that this branch is not officially maintained and that we strongly recommend that you do migrate to pkg(8)

The ports tree is now stage only

September 1, 2014 by · Leave a Comment 

The ports tree is now fully staged (only 2% has been left unstaged, marked as broken and will be removed from the ports tree if no PR to stage them are pending in bugzilla).

I would like to thank every committer and maintainers for their work on staging!
It allowed us to convert more than 23k packages to support stage in only 11 months!

Staging is a very important state, it allows us to right now be able to run quality testing scripts on the packages (which already allowed to fix tons of hidden problems) and it allows use to be able to build packages as a regular user!

It also opens the gates to new features that users have been requesting for many years:

  • flavors
  • multiple packages

Expect those features to happen in the near future.

Happy 20th birthday FreeBSD ports tree!

August 21, 2014 by · Leave a Comment 

It all started with this commit from Jordan Hubbard on August 21, 1994:

Commit my new ports make macros
Still not 100% complete yet by any means but fairly usable at this stage.

Twenty years later the ports tree is still there and actively
maintained. A video was prepared to celebrate the event and to thank
all of you who give some of their spare time and energy to the project!

Farewell beloved Canadian

August 12, 2014 by · Leave a Comment 

Last month, our beloved Canadian Thomas Abthorpe decided to step
down from his portmgr-secretary position. While I suspect this is secretly
related to his pool of Canadian jokes having dried up, the official reason is
that Thomas wants to focus more on his private and professional lives for the
moment. Needless to say, the whole ports community is in mourning.

pkg 1.3.0 out!

July 25, 2014 by · Leave a Comment 

Hi all,

I’m very please to announce the release of pkg 1.3.0
This version is the result of almost 9 month of hard work

Here are the statistics for the version:
- 373 files changed, 66973 insertions(+), 38512 deletions(-)
- 29 different contributors

Please not that for the first time I’m not the main contributor, and I would
like to particularly thanks Vsevold Stakhov for all the hard work he has done to
allow us to get this release out. I would like also to give a special thanks to
Andrej Zverev for the tons of hours spending on testing and cleaning the bug
tracker!

So much has happened that it is hard to summarize so I’ll try to highlight the
major points:
- New solver, now pkg has a real SAT solver able to automatically handle
conflicts and dynamically discover them. (yes pkg set -o is deprecated now)
- pkg install now able to install local files as well and resolve their
dependencies from the remote repositories
- Lots of parts of the code has been sandboxed
- Lots of rework to improve portability
- Package installation process has been reworked to be safer and handle properly
the schg flags
- Important modification of the locking system for finer grain locks
- Massive usage of libucl
- Simplification of the API
- Lots of improvements on the UI to provide a better user experience.
- Lots of improvements in multi repository mode
- pkg audit code has been moved into the library
- pkg -o A=B that will overwrite configuration file from cli
- The ui now support long options
- The unicity of a package is not anymore origin
- Tons of bug fixes
- Tons of behaviours fixes
- Way more!

Thank you to all contributors:
Alberto Villa, Alexandre Perrin, Andrej Zverev, Antoine Brodin, Brad Davis,
Bryan Drewery, Dag-Erling Smørgrav, Dmitry Marakasov, Elvira Khabirova, Jamie
Landeg Jones, Jilles Tjoelker, John Marino, Julien Laffaye, Mathieu Arnold,
Matthew Seaman, Maximilian Gaß, Michael Gehring, Michael Gmelin, Nicolas Szalay,
Rodrigo Osorio, Roman Naumann, Rui Paulo, Sean Channel, Stanislav E. Putrya,
Vsevolod Stakhov, Xin Li, coctic

Regards,
Bapt on behalf of the pkg@

BSDCan 2014 – Ports and Packages WG

July 18, 2014 by · Leave a Comment 

Baptiste Daroussin started the session with a status update on package building. All packages are now built with poudriere. The FreeBSD Foundation sponsored some large machines on which it takes around 16 hours to build a full tree. Each Wednesday at 01:00UTC the tree is snapshot and an incremental build is started for all supported released, the 2 stable branches (9 and 10) and quarterly branches for 9.x-RELEASE and 10.x-RELEASE. The catalogue is signed on a dedicated signing machine before upload. Packages can be downloaded from 4 mirrors (us-west, us-east, UK, and Russia) and feedback so far has been very positive.

He went on to note that ports people need better coordination with src people on ABI breakage. We currently only support i386 and amd64, with future plans for ARM and a MIPS variant. Distfiles are not currently mirrored (since fixed), and while it has seen no progress, it’s still a good idea to build a pkg of the ports tree itself.

pkg 1.3 will include a new solver, which will help 'pkg upgrade' understand that an old packages needs to be replaced with a newer one, with no more need for 'pkg set' and other chicanery. Cross building ports has been added to the ports tree, but is waiting for pkg-1.3. All the dangerous operations in pkg have now been sandboxed as well.

EOL for pkg_tools has been set for September 1st. An errata notice has gone out that adds a default pkg.conf and keys to all supported branches, and nagging delays have been added to ports.

Quarterly branches based on 3 month support cycle has been started on an evaluation basis. We’re still unsure about the manpower needed to maintain those. Every quarter a snapshot of the tree is created and only security fixed, build and runtime fixed, and upgrades to pkg are allowed to be committed to it. Using the MFH tag in a commit message will automatically send an approval request to portmgr and an mfh script on Tools/ makes it easy to do the merge.

Experience so far has been good, some minor issues to the insufficient testing. MFHs should only contain the above mentioned fixes; cleanups and other improvements should be done in separate commits only to HEAD. A policy needs to be written and announced about this. Do we want to automatically merge VuXML commits, or just remove VuXML from the branch and only use the one in HEAD?

A large number of new infrastructure changes have been introduces over the past few months, some of which require a huge migration of all ports. To speed these changes up, a new policy was set to allow some specific fixes to be committed without maintainer approval. Experience so far has been good, things actually are being fixed faster than before and not many maintainers have complained. There was agreement that the list of fixes allowed to be committed without explicit approval should be a specific whitelist published by portmgr, and not made too broad in scope.

Erwin Lansing quickly measured the temperature of the room on changing the default protocol for fetching distils from MASTER_SITE_BACKUP from ftp to http. Agreement all around and erwin committed the change.

Ben Kaduk gave an introduction and update on MIT’s Athena Environment with some food for thought. While currently not FreeBSD based, he would like to see it become so. Based on debian/ubuntu and rolled out on hundreds of machines, it now has it’s software split into about 150 different packages and metapackages.

Dag-Erling Smørgrav discussed changes to how dependencies are handled, especially splitting dependencies that are needed at install time (or staging time) and those needed at run time. This may break several things, but pkg-1.3 will come with better dependency tracking solving part of the problem.

Ed Maste presented the idea of “package transparency”, loosely based on Google’s Certificate Transparency. By logging certificate issuance to a log server, which can be publicly checked, domain owners can search for certificates issued for their domains, and notice when a certificate is issued without their authority. Can this model be extended to packages? Mostly useful for individually signed packages, while we currently only sign the catalogue. Can we do this with the current infrastructure?

Stacy Son gave an update on Qemu user mode, which is now working with Qemu 2.0.0. Both static and dynamic binaries are supported, though only a handful of system call are supported.

Baptiste introduced the idea of having pre-/post-install scripts be a library of services, like Casper, for common actions. This reduces the ability of maintainers to perform arbitrary actions and can be sandboxed easily. This would be a huge security improvement and could also enhance performance.

Cross building is coming along quite well and most of the tree should be able to be build by a simple 'make package'. Major blockers include perl and python.

Bryan Drewery talked about a design for a PortsCI system. The idea is that committer easily can schedule a build, be it an exp-run, reference, QAT, or other, either via a web interface or something similar to a pull request, which can fire off a build.

Steve Wills talked about using Jenkins for ports. The current system polls SVN for commits and batches several changes together for a build. It uses 8 bhyve VMs instances, but is slow. Sean Bruno commented that there are several package building clusters right now, can they be unified? Also how much hardware would be needed to speed up Jenkins? We could duse Jenkins as a fronted for the system Bryan just talked about. Also, it should be able to integrate with phabricator.

Erwin opened up the floor to talk about freebsd-version(1) once more. It was introduced as a mechanism to find out the version of user land currently running as uname -r only represents the kernel version, and would thus miss updates of the base system that do no touch the kernel. Unfortunately, freebsd-version(1) cannot really be used like this in all cases, it may work for freebsd-update, but not in general. No real solution was found this time either.

The session ended with a discussion about packaging the base system. It’s a target for FreeBSD 11, but lots of questions are still to be answered. What granularity to use? What should be packages into how many packages? How to handle options? Where do we put the metadata for this? How do upgrades work? How to replace shared libraries in multiuser mode? This part also included the quote of the day: “Our buildsystem is not a paragon of configurability, but a bunch of hacks that annoyed people the most.”

Thanks to all who participated in the working group, and thanks again to DK Hostmaster for sponsoring my trip to BSDCan this year, and see you at the Ports and Packages WG meet up at EuroBSDCon in Sofia in September.

Monthly dashboard

July 13, 2014 by · Leave a Comment 

Thursday morning, 5 AM, I couldn’t sleep. I thought I could use the time before
work to do something useful, so I started handling a few PRs for FreeBSD. After
a couple of commits, a warm shower, and just before heading to work, I quickly
browsed through my irc backlog and suddently got very sad: someone was angrily
asking why bug reports were being ignored for such a long time, pushing for his
own PR to finally be given some consideration.

Thinking about it in the bus to work I realized that this guy was right to
complain: when a bug is reported it should ideally be fixed right away. Still I
was feeling sad because being on the other side of the fence I know how much
dedication volunteers put into FreeBSD, but I was not sure everybody was aware
of this. I had to find something to express this dedication.

That’s how the idea of the monthly dashboard came: simple figures that can tell
a whole story. See for yourself with this dashboard that can be found in portmgr
monthly report for June 2014:


Monthly dashboard

Number of messages to portmgr@: 564 (+53%)
Number of commits on ports: 3,717 (+17%)
Number of ports PRs closed: 873 (+25%)
Active ports committers: 147 (+10%)


 

Isn’t it amazing? Nearly 4,000 updates on the ports tree and nearly 900 problem
reports closed in a single month!

That’s a tremendous amount of work done by our committers. Take Linux for
example: with more than twice as much contributors during the same period four
times less commits were applied to the Linux kernel than to the FreeBSD ports
tree [1].

Those figures pay tribute to our committers, and I am pleased to see that the
activity keeps growing. I personnally believe there has never been a better time
to start contributing to the FreeBSD ports tree with all those new features
currently being introduced. So come and join the party!


[1] Statistics taken from http://www.ohloh.net/p/linux:
962 commits done by 344 contributors (activity recorded from Jun 9 2014 to Jul 9 2014)

Getting to know your portmgr-lurker: William Grzybowski

July 5, 2014 by · Leave a Comment 

From July to the end of October two new lurkers will have the opportunity to get insights into FreeBSD portmgr internals, namely William Grzybowski and Nicola Vitale. William was the first to answer our interview so let’s get to know him a bit better.

 

Name

William Grzybowski

Committer name

wg

Inspiration for your IRC nick

my name initials

 TLD of origin

.br

 Current TLD (if different from above)

Occupation

Software engineer

 Blog

None

Inspiration for using FreeBSD

Stability and simplicity

Who was your first contact in FreeBSD

I don’t recall, too long ago!

Who was your mentor(s)

culot, jpaetzel

What was your most embarrassing moment in FreeBSD

Breaking INDEX, but hey, who ever didn’t? ;)

vi(m) /  emacs / other

vim

What keeps you motivated in FreeBSD

The passion of everyone in the zoo about it. You can very easily see a
guy angry by someone’s else commit like stealing candy from his son :)

Favorite musician/band

AC/DC

What book do you have on your bedside table

book? what is that? :)

coffee / tea / other

coffee, beer (but real beer, no corn!)

How would you describe yourself

Calm and mind-centered

sendmail / postfix / other

postfix

Do you have a hobby outside of FreeBSD

Play soccer, gym

What is your favorite TV show

Fringe

Claim to Fame

I was mentored by culot and I survived! Just kidding, I have none, I am a joke!

What did you have for breakfast today

An orange and a slice of yogurt cake

What sports team do you support

Gremio Foot-Ball Porto Alegrense, brazillian soccer team

What else do you do in the world of FreeBSD

I do work mostly in python ports and help closing PRs.

Next Page »