pkg 1.3.0 out!

July 25, 2014 by · Leave a Comment 

Hi all,

I’m very please to announce the release of pkg 1.3.0
This version is the result of almost 9 month of hard work

Here are the statistics for the version:
- 373 files changed, 66973 insertions(+), 38512 deletions(-)
- 29 different contributors

Please not that for the first time I’m not the main contributor, and I would
like to particularly thanks Vsevold Stakhov for all the hard work he has done to
allow us to get this release out. I would like also to give a special thanks to
Andrej Zverev for the tons of hours spending on testing and cleaning the bug
tracker!

So much has happened that it is hard to summarize so I’ll try to highlight the
major points:
- New solver, now pkg has a real SAT solver able to automatically handle
conflicts and dynamically discover them. (yes pkg set -o is deprecated now)
- pkg install now able to install local files as well and resolve their
dependencies from the remote repositories
- Lots of parts of the code has been sandboxed
- Lots of rework to improve portability
- Package installation process has been reworked to be safer and handle properly
the schg flags
- Important modification of the locking system for finer grain locks
- Massive usage of libucl
- Simplification of the API
- Lots of improvements on the UI to provide a better user experience.
- Lots of improvements in multi repository mode
- pkg audit code has been moved into the library
- pkg -o A=B that will overwrite configuration file from cli
- The ui now support long options
- The unicity of a package is not anymore origin
- Tons of bug fixes
- Tons of behaviours fixes
- Way more!

Thank you to all contributors:
Alberto Villa, Alexandre Perrin, Andrej Zverev, Antoine Brodin, Brad Davis,
Bryan Drewery, Dag-Erling Smørgrav, Dmitry Marakasov, Elvira Khabirova, Jamie
Landeg Jones, Jilles Tjoelker, John Marino, Julien Laffaye, Mathieu Arnold,
Matthew Seaman, Maximilian Gaß, Michael Gehring, Michael Gmelin, Nicolas Szalay,
Rodrigo Osorio, Roman Naumann, Rui Paulo, Sean Channel, Stanislav E. Putrya,
Vsevolod Stakhov, Xin Li, coctic

Regards,
Bapt on behalf of the pkg@

BSDCan 2014 – Ports and Packages WG

July 18, 2014 by · Leave a Comment 

Baptiste Daroussin started the session with a status update on package building. All packages are now built with poudriere. The FreeBSD Foundation sponsored some large machines on which it takes around 16 hours to build a full tree. Each Wednesday at 01:00UTC the tree is snapshot and an incremental build is started for all supported released, the 2 stable branches (9 and 10) and quarterly branches for 9.x-RELEASE and 10.x-RELEASE. The catalogue is signed on a dedicated signing machine before upload. Packages can be downloaded from 4 mirrors (us-west, us-east, UK, and Russia) and feedback so far has been very positive.

He went on to note that ports people need better coordination with src people on ABI breakage. We currently only support i386 and amd64, with future plans for ARM and a MIPS variant. Distfiles are not currently mirrored (since fixed), and while it has seen no progress, it’s still a good idea to build a pkg of the ports tree itself.

pkg 1.3 will include a new solver, which will help 'pkg upgrade' understand that an old packages needs to be replaced with a newer one, with no more need for 'pkg set' and other chicanery. Cross building ports has been added to the ports tree, but is waiting for pkg-1.3. All the dangerous operations in pkg have now been sandboxed as well.

EOL for pkg_tools has been set for September 1st. An errata notice has gone out that adds a default pkg.conf and keys to all supported branches, and nagging delays have been added to ports.

Quarterly branches based on 3 month support cycle has been started on an evaluation basis. We’re still unsure about the manpower needed to maintain those. Every quarter a snapshot of the tree is created and only security fixed, build and runtime fixed, and upgrades to pkg are allowed to be committed to it. Using the MFH tag in a commit message will automatically send an approval request to portmgr and an mfh script on Tools/ makes it easy to do the merge.

Experience so far has been good, some minor issues to the insufficient testing. MFHs should only contain the above mentioned fixes; cleanups and other improvements should be done in separate commits only to HEAD. A policy needs to be written and announced about this. Do we want to automatically merge VuXML commits, or just remove VuXML from the branch and only use the one in HEAD?

A large number of new infrastructure changes have been introduces over the past few months, some of which require a huge migration of all ports. To speed these changes up, a new policy was set to allow some specific fixes to be committed without maintainer approval. Experience so far has been good, things actually are being fixed faster than before and not many maintainers have complained. There was agreement that the list of fixes allowed to be committed without explicit approval should be a specific whitelist published by portmgr, and not made too broad in scope.

Erwin Lansing quickly measured the temperature of the room on changing the default protocol for fetching distils from MASTER_SITE_BACKUP from ftp to http. Agreement all around and erwin committed the change.

Ben Kaduk gave an introduction and update on MIT’s Athena Environment with some food for thought. While currently not FreeBSD based, he would like to see it become so. Based on debian/ubuntu and rolled out on hundreds of machines, it now has it’s software split into about 150 different packages and metapackages.

Dag-Erling Smørgrav discussed changes to how dependencies are handled, especially splitting dependencies that are needed at install time (or staging time) and those needed at run time. This may break several things, but pkg-1.3 will come with better dependency tracking solving part of the problem.

Ed Maste presented the idea of “package transparency”, loosely based on Google’s Certificate Transparency. By logging certificate issuance to a log server, which can be publicly checked, domain owners can search for certificates issued for their domains, and notice when a certificate is issued without their authority. Can this model be extended to packages? Mostly useful for individually signed packages, while we currently only sign the catalogue. Can we do this with the current infrastructure?

Stacy Son gave an update on Qemu user mode, which is now working with Qemu 2.0.0. Both static and dynamic binaries are supported, though only a handful of system call are supported.

Baptiste introduced the idea of having pre-/post-install scripts be a library of services, like Casper, for common actions. This reduces the ability of maintainers to perform arbitrary actions and can be sandboxed easily. This would be a huge security improvement and could also enhance performance.

Cross building is coming along quite well and most of the tree should be able to be build by a simple 'make package'. Major blockers include perl and python.

Bryan Drewery talked about a design for a PortsCI system. The idea is that committer easily can schedule a build, be it an exp-run, reference, QAT, or other, either via a web interface or something similar to a pull request, which can fire off a build.

Steve Wills talked about using Jenkins for ports. The current system polls SVN for commits and batches several changes together for a build. It uses 8 bhyve VMs instances, but is slow. Sean Bruno commented that there are several package building clusters right now, can they be unified? Also how much hardware would be needed to speed up Jenkins? We could duse Jenkins as a fronted for the system Bryan just talked about. Also, it should be able to integrate with phabricator.

Erwin opened up the floor to talk about freebsd-version(1) once more. It was introduced as a mechanism to find out the version of user land currently running as uname -r only represents the kernel version, and would thus miss updates of the base system that do no touch the kernel. Unfortunately, freebsd-version(1) cannot really be used like this in all cases, it may work for freebsd-update, but not in general. No real solution was found this time either.

The session ended with a discussion about packaging the base system. It’s a target for FreeBSD 11, but lots of questions are still to be answered. What granularity to use? What should be packages into how many packages? How to handle options? Where do we put the metadata for this? How do upgrades work? How to replace shared libraries in multiuser mode? This part also included the quote of the day: “Our buildsystem is not a paragon of configurability, but a bunch of hacks that annoyed people the most.”

Thanks to all who participated in the working group, and thanks again to DK Hostmaster for sponsoring my trip to BSDCan this year, and see you at the Ports and Packages WG meet up at EuroBSDCon in Sofia in September.

Monthly dashboard

July 13, 2014 by · Leave a Comment 

Thursday morning, 5 AM, I couldn’t sleep. I thought I could use the time before
work to do something useful, so I started handling a few PRs for FreeBSD. After
a couple of commits, a warm shower, and just before heading to work, I quickly
browsed through my irc backlog and suddently got very sad: someone was angrily
asking why bug reports were being ignored for such a long time, pushing for his
own PR to finally be given some consideration.

Thinking about it in the bus to work I realized that this guy was right to
complain: when a bug is reported it should ideally be fixed right away. Still I
was feeling sad because being on the other side of the fence I know how much
dedication volunteers put into FreeBSD, but I was not sure everybody was aware
of this. I had to find something to express this dedication.

That’s how the idea of the monthly dashboard came: simple figures that can tell
a whole story. See for yourself with this dashboard that can be found in portmgr
monthly report for June 2014:


Monthly dashboard

Number of messages to portmgr@: 564 (+53%)
Number of commits on ports: 3,717 (+17%)
Number of ports PRs closed: 873 (+25%)
Active ports committers: 147 (+10%)


 

Isn’t it amazing? Nearly 4,000 updates on the ports tree and nearly 900 problem
reports closed in a single month!

That’s a tremendous amount of work done by our committers. Take Linux for
example: with more than twice as much contributors during the same period four
times less commits were applied to the Linux kernel than to the FreeBSD ports
tree [1].

Those figures pay tribute to our committers, and I am pleased to see that the
activity keeps growing. I personnally believe there has never been a better time
to start contributing to the FreeBSD ports tree with all those new features
currently being introduced. So come and join the party!


[1] Statistics taken from http://www.ohloh.net/p/linux:
962 commits done by 344 contributors (activity recorded from Jun 9 2014 to Jul 9 2014)

Getting to know your portmgr-lurker: William Grzybowski

July 5, 2014 by · Leave a Comment 

From July to the end of October two new lurkers will have the opportunity to get insights into FreeBSD portmgr internals, namely William Grzybowski and Nicola Vitale. William was the first to answer our interview so let’s get to know him a bit better.

 

Name

William Grzybowski

Committer name

wg

Inspiration for your IRC nick

my name initials

 TLD of origin

.br

 Current TLD (if different from above)

Occupation

Software engineer

 Blog

None

Inspiration for using FreeBSD

Stability and simplicity

Who was your first contact in FreeBSD

I don’t recall, too long ago!

Who was your mentor(s)

culot, jpaetzel

What was your most embarrassing moment in FreeBSD

Breaking INDEX, but hey, who ever didn’t? ;)

vi(m) /  emacs / other

vim

What keeps you motivated in FreeBSD

The passion of everyone in the zoo about it. You can very easily see a
guy angry by someone’s else commit like stealing candy from his son :)

Favorite musician/band

AC/DC

What book do you have on your bedside table

book? what is that? :)

coffee / tea / other

coffee, beer (but real beer, no corn!)

How would you describe yourself

Calm and mind-centered

sendmail / postfix / other

postfix

Do you have a hobby outside of FreeBSD

Play soccer, gym

What is your favorite TV show

Fringe

Claim to Fame

I was mentored by culot and I survived! Just kidding, I have none, I am a joke!

What did you have for breakfast today

An orange and a slice of yogurt cake

What sports team do you support

Gremio Foot-Ball Porto Alegrense, brazillian soccer team

What else do you do in the world of FreeBSD

I do work mostly in python ports and help closing PRs.

2014Q3 Branched

July 1, 2014 by · Leave a Comment 

The 2014Q3 branch has just been branched and the package builder has been
updated to use that branch. This means that the next update on the quarterly
packages will be on the 2014Q3 branch.

What happened during the last 3 months:
- 177 different committers have participated
- 9918 commits happened
- diffstat says: 23646 files changed, 554070 insertions(+), 577210 deletions(-)

What does that means for users:
- default Java is now 1.7
- massive conversion to stagedir (93% of the ports are now properly staged)
- massive improvement of the usage of libtool (which reduces a lot overlinking)
- new USES: mono, objc, drupal, gecko, cpe, gssapi, makeinfo
- new Keywords for plist: @sample, @shell
- LibreOffice has been updated to 4.2.5
- Firefox has been updated to 30.0
- Firefox-esr has been updated to 24.6
- Default postgresql has moved from 9.0 to 9.2
- nginx has been updated to 1.6.0
- Default lua is 5.2
- subversion has been split into multiple ports for each features
- On FreeBSD 9-STABLE and 10-STABLE the default xorg 1.12.4 (for default binary
packages it is still 1.7.7)
- Improved QA checking in the infrastructure
- Info files are handle correctly even if base has been built WITHOUT_INFO
- Ancient emacs version has been cleaned out

Getting to know your portmgr@ – Steve Wills

June 4, 2014 by · Leave a Comment 

It is my pleasure to introduce Steve Wills, the newest member of the portmgr team. Steve has done a tremendous work on the ports tree, especially in the field of testing and quality. Here is a short interview to get to know him better.

Name
Steve Wills

Committer name
swills

Inspiration for your IRC nick
Boring, it’s my userid.

TLD of origin
.us

Current TLD (if different from above)
Same.

Occupation
Sysadmin.

When did you join portmgr@
2014

Blog
Used to have one, use twitter more now (@swills)

Inspiration for using FreeBSD
Simplicty and learning.

Who was your first contact in FreeBSD
Can’t recall, it was ages ago.

Who was your mentor(s)
pgollucci

What was your most embarrassing moment in FreeBSD
Trying to migrate Ruby default version from 1.8 to 1.9 and having to roll back.

Boxers / Briefs / other
Heh, question assume survey taker is male, which I am, but I think we need to
work on diversity (but not in that “hey, let’s work on diversity and get some
women” way, but more in that we make something everyone wants to use)

What is your role in your circle of friends
The FreeBSD user. ;)

vi(m) / emacs / other
vi(m)

What keeps you motivated in FreeBSD
New users, new committers.

Favourite musician/band
I listen to a decent variety of stuff, but I suppose the thing I come back to
most is NIN.

What book do you have on your bedside table
I have an iPad by my bed, which I bought to read, but mostly I browse news on
it.

coffee / tea / other
Don’t drink caffeine, so don’t drink coffee much. I do drink good beer tho.

Do you have a guilty pleasure
Good dark chocolate. :)

How would you describe yourself
Mostly standard in many ways, husband, father, FreeBSD hacker, sysadmin, in
that order.

sendmail / postfix / other
Sendmail, tho dma is nice too.

Do you have a hobby outside of FreeBSD
Used to play guitar, still have one, don’t find time to pick it up much any
more.

What is your favourite TV show
Futurama

Claim to Fame
Ported Acidwarp from DOS to svgalib.

What did you have for breakfast today
Everything bagel with plain cream chese.

What sports team do you support
The only sport I watch is University of North Carolina Basketball.

What else do you do in the world of FreeBSD
ruby ports, perl ports sometimes

What can you tell us about yourself that most people don’t know
I was an employee at Red Hat way way back

Any parting words you want to share
Not really.

What is your .sig at the moment
Null.

Steve

Frederic Culot takes over as portmgr-secretary@

May 16, 2014 by · Leave a Comment 

It is with great pleasure that the FreeBSD Ports Management Team announces that Frederic (culot@) Culot will take over responsibilities of team secretary effective immediately.

Frederic became a ports committer in October 2010, and joined the ranks of portmgr-lurkers@ in March 2014 as the shadow secretary.

Please drop him a note and congratulate him (or offer condolences).

 

Thomas
on behalf of portmgr@

 

Ports 2014Q2 branched

April 2, 2014 by · Leave a Comment 

I am pleased to announce that we have created the 2014Q2 branch of the ports
tree.

Because the first 2014Q1 branch was experimental you might not have heard of it
yet.

January 2014 saw the release of the first quarterly branch, intended at
providing a stable and high-quality ports tree. Those stable branches are a
snapshot of the head ports tree taken every 3 months and currently supported
for three months, during which they receive security fixes as well as build and
runtime fixes.

Packages are built on regular basis on that branch (weekly) and published as
usual via pkg.FreeBSD.org (/quarterly instead of the usual /latest).

They are signed the same way the /latest branch is.

While packages for 2014Q1 were only built for 10 (i386 and amd64) 2014Q2 will be
built for both FreeBSD 9 and 10 (i386 and amd64).

The first build of 2014Q2 will started this morning (wednesday at 1 am UTC) and should
hit your closest mirrors very soon.

On behalf of the port management team
Bapt

Getting to know your portmgr-lurker — Frederic Culot

March 25, 2014 by · Leave a Comment 

Name

Frederic Culot

Committer name

culot

Inspiration for your IRC nick

lack of inspiration actually…

TLD of origin

.fr

Current TLD (if different from above)

.lu

Occupation

IT consultant in the banking sector in Luxembourg, but I don’t always do IT.
I am also interested in business and management and my wife and I are working
on starting our own business.

When did you join portmgr@

Joined FreeBSD as a committer in October 2010 and the portmgr-lurkers program in
March 2014, but never been part of portmgr@.

Blog

http://people.freebsd.org/~culot is the closest thing I have to a blog

Inspiration for using FreeBSD

I was a longtime OpenBSD user until I worked in the same company as clement@
(former portmgr) who successfully managed to convert me to FreeBSD. I did not
feel the need to look into another system since then.

Who was your first contact in FreeBSD

clement@. But when I really started to get involved in FreeBSD it was jadawin@
who first contacted me. He is one of the kindest person I ever worked with and
while we’ve known each others for about 4 years now I’ve never been able to
meet him in person. But that’s the way it is with projects such as FreeBSD:
teams are virtual and gathering together might be difficult unfortunately.

Who was your mentor(s)

My mentors were sahil@ and wen@. Thanks to them I believe my mentorship at
FreeBSD was the best induction program I ever experienced. I was also amused to
realize that whereas companies spend huge amounts to design reward systems, it
is sometimes when nothing is to be expected in return that people are the most
caring and helpful.

What was your most embarrassing moment in FreeBSD

My first pointy hat: a bit after my first 700 commits when I started to feel
confident I finally managed to break INDEX :’(

Boxers / Briefs / other

Any 15-year old single material does it.

What is your role in your circle of friends

uncork the bottles usually…

vi(m) / emacs / other

vim

What keeps you motivated in FreeBSD

The people behind it. There are lots of great guys behind this project, and a
day when I could not meet with other developers on irc is a sad day for me :’(

But FreeBSD is also one of my sources of inspiration when it comes to how
organizations behave and innovate (which is a topic of interest I got into
during my MBA studies) and I find it very interesting to compare FreeBSD with
the for-profit companies I work for. I even wrote an article for BSDmag in case
some would also be interested in those aspects:

http://people.freebsd.org/~culot/BSDmag.pdf

> Favourite musician/band

I don’t listen to much music. The cause might be that I work in a very noisy
environment (large open-space), so I more and more enjoy silence and calm when
I’m back home. But recently when I listen to music I enjoy Moby’s “wait for me”
album (ambient edition), Erik Mongrain, or a bit of merengue to remind me of my
holidays.

What book do you have on your bedside table

Nietzsche’s Thus spoke Zarathoustra.

I even extracted my favorite quotes and created the
french/fortune-mod-zarathoustra port.

coffee / tea / other

Both, depends on the time of day

Do you have a guilty pleasure

To enjoy a 7-course meal with my wife at a 3-star michelin restaurant and
finish relaxing in a club chair in front of the fireplace with a 40 years old
armagnac.

How would you describe yourself

Sober, clever, and motivated in the morning.
Drunk, stupid, and depressed in the evening.
Or is it the opposite?

sendmail / postfix / other

sendmail as it’s in base, but not for long apparently so I could have to make a
more reasoned choice soon

Do you have a hobby outside of FreeBSD

Sports (I go to the gym almost everyday day, did quite some scuba diving and
snowboarding when I was younger), but I enjoy good food and wine when I’m done
training. I also enjoy traveling. My last trips were to India, Dominican
Republic, and Lapland: so many nice places to visit!

What is your favourite TV show

My favorites to day are twin peaks, the prisoner, and battlestar galactica

Claim to Fame

I spent one night at the pub with bapt@, and survived.

What did you have for breakfast today

oat flakes with water

What sports team do you support

If you want to torture me, just fasten me in front of the TV with a soccer game
on. I could even confess I enjoy tabthorpe’s jokes just to shorten the ordeal.

<Editors note: I have know idea what he means by this :>

What else do you do in the world of FreeBSD

Apart from my work on ports I also did some French translations (translated
the contributing-ports, linux-users, and building-products articles). I also
try to participate in IT exhibits and promote FreeBSD by managing booths such
as at Solutions Linux Paris for which I designed a poster to attract visitors:

http://people.freebsd.org/~culot/SolutionsLinux.png

But most importantly I offer beers and whisky to other FreeBSD developers when
I meet them :)

What can you tell us about yourself that most people don’t know

I actually enjoy tabthorpe’s jokes. Sometimes.

<Editor’s note: again, no clue what he is talking about :>

Any parting words you want to share

I repeat it but my main motivation to work on this project is to get in touch
with other FreeBSD enthusiasts, so do not hesitate to ping me on irc if you
feel like sharing some of your thoughts with me. I would be most pleased.

What is your .sig at the moment

Regards,
Frederic Culot

Getting to know your portmgr-lurker — Alexy Dokuchaev

March 11, 2014 by · Leave a Comment 

In this latest edition of Getting to know, we interview senior ports committer Alexy Dokuchaev as one of our newest portmgr-lurkers.

Name

Alexey Dokuchaev

Committer name

danfe@

TLD of origin

.ru (technically should be .su, but it’s now defunct)

Occupation

Software engineer and contractor

Inspiration for using FreeBSD

Wanted a Unix system that I could understand and that would not get bloated
as time goes by. In 1998 Linux was popular mount local folks, but I just
could not get it (trying to switch from DOS). Someone mentioned FreeBSD;
and it all started to make sense pretty much immediately. And even after
some 15 years, FreeBSD still feels like back in those good days.

Who was your first contact in FreeBSD

Max Khon (fjoe@), I guess…

Who was your mentor(s)

fjoe@ and krion@

vi(m) / emacs / other

nvi(1)

What keeps you motivated in FreeBSD

That in 2014 I can have a modern Unix system which I still can work with
like it’s 1999 again. I can still have text console. And start X11 with
startx(1). Configure things by editing /etc/rc.conf, but have support for
the latest hardware. Play sound via OSS, yet enjoy low-latency in-kernel
mixer. We’re so lucky to not have Lennart Poettering with his PulseAudio
and systemd crap…

coffee / tea / other

Both (but tea preferred) + beer

What is your favourite TV show

The X Files, I guess…

What is your .sig at the moment

./danfe (making it both answer and a sig this time)

Next Page »