Monthly Archives: August 2011

HOWTO create a new linux_base port

FreeBSD is in need of a new linux_base port. It is on my TODO list since a long time, but I do not get the time to create one. I still do not have the time to work on a new one, but when you read this, I managed to get the time to create a HOWTO which describes what needs to be done to create a new linux_base port.

I will not describe how to create a new linux_base port from scratch, I will just describe how you can copy the last one and update it to something newer based upon the existing infrastructure for RPM packages.

Specific questions which come up during porting a new Linux release should be asked on [email protected],  there are more people which can answer questions than here in my blog. I will add useful information to this HOWTO if necessary.

In the easy case most of the work is searching the right RPMs and their dependencies to use, and to create the plist.

Why do we need a new linux_base port?

The current linux_base port is based upon Fedora 10, which is end of life since December 2009. Even Fedora 13 is already end of life. Fedora 16 is supposed to be released this year. From a support point of view, Fedora 15 or maybe even Fedora 16 would be a good target for the next linux_base port. Other alternatives would be to use an extended lifetime release of another RPM based distribution, like for example CentOS 6 (which seems to be based upon Fedora 12 with backports from Fedora 13 and 14). Using a Linux release which is told to be supported for at least 10 years, sounds nice from a FreeBSD point of view (only minor changes to the linux ports in such a case, instead of creating a complete new linux_base each N+2 releases like with Fedora), but it also means additional work if you want to create the first linux_base port for it.

The mysteries you have to conquer if you want to create a new linux_base port

What we do not know is, if Fedora 15/16, CentOS 6, or any other Linux release will work in a supported FreeBSD release. There are two ways to find this out.

The first one is to take an existing Linux system, chroot into it (either via NFS or after making a copy into a directory of a FreeBSD system), and to run a lot of programs (acroread, skype, shells, scripts, …). The LTP testsuite is not that much useful here, as it will test mostly kernel features, but we do not know which kernel features are mandatory for a given userland of a Linux release.

The second way of testing if a given Linux release works on FreeBSD is to actually create a new linux_base port for it and test it without chrooting.

The first way is faster, if you are only interested in testing if something works. The second way provides an easy to setup testbed for FreeBSD kernel developers to fix the Linuxulator so that it works with the new linux_base port. Both ways have their merits, but it is up to the person doing the work to decide which way to go.

The meat: HOWTO create a new linux_base port

First off, you need a system (or a jail) without any linux_base port installed. After that you can create a new linux_base port (= lbN), by just making a copy of the latest one (= lbO). In lbN you need to add lbO as a CONFLICT, and in all other existing linux_base ports, you need to add lbN as a conflict.

Change the PORTNAME, PORTVERSION, reset the PORTREVISION in lbN, and set LINUX_DIST_VER  to the new Linux-release version in the lbN Makefile (this is used in PORTSDIR/Mk/bsd.linux-rpm.mk and PORTSDIR/Mk/bsd.linux-apps.mk).

If you do not stay with Fedora, there is some more work to do before you can have a look at chosing RPMs for installation. You need to have a look at PORTSDIR/Mk/bsd.linux-rpm.mk and add some cases for the new LINUX_DIST you want to use. Do not forget to set LINUX_DIST in the lbN Makefile to the name of the distribution you use. You also need to augment the LINUX_DIST_VER check in PORTSDIR/Mk/bsd.linux-rpm.mk with some LINUX_DIST conditionals. If you are lucky, the directory structure for downloads is similar to the Fedora structure, and there is not a lot to do here.

When this is done, you can have a look at the BIN_DISTFILES variable in the lbN Makefile. Try to find similar RPMs for the new Linux release you want to port. Some may not be available, and it may also be the case that different ones are needed instead. I suggest to first work with the ones which are available (make makesum, test install and create plist). After that you need to find out what the replacement RPMs for non-existing ones are. You are on your own here. Search around the net, and/or have a look at the dependencies in the RPMs of lbO to determine if something was added as a dependency of something else or not (if not, forget about it ATM). When you managed to find replacement RPMs, you can now have a look at the dependencies of the RPMs in lbN. Do not add blindly all dependencies, not all are needed in FreeBSD (the linux_base ports are not supposed to create an environment which you can chroot into, they are supposed to augment the FreeBSD system to be able to run Linux programs in ports like they where FreeBSD native programs). What you need in the linux_base ports are libraries, config and data files which do not exist in FreeBSD or have a different syntax than in FreeBSD (those config or data files which are just in a different place, can be symlinked), and basic shell commands (which commands are needed or not… well… good question, in the past we made decisions what to include based upon problem reports from users). Now for the things which are not available and where not added as a dependency. Those are things which are either used during install, or where useful to have in the past. Find out by what it was replaced and have a look if this replacement can easily be used instead. If it can be used, add it. If not, well… bad luck, we (the FreeBSD community) will see how to handle this somehow.

If you think that you have all you need in BIN_DISTFILES, please update SRC_DISTFILES accordingly and generate the distfile via  make –DPACKAGE_BUILDING makesum to have the checksums of the sources (for legal reasons we need them on our mirrors).

The next step is to have a look at REMOVE_DIRS, REMOVE_FILES and ADD_DIRS if something needs to be modified. Most of them are there to fall back to the corresponding FreeBSD directories/files, or because they are not needed at all (REMOVE_*). Do not remove directories from ADD_DIRS, they are created here to fix some edge conditions (I do not remember exactly why we had to add them, and I do not take the time ATM to search in the CVS history).

If you are lucky, this is all (make sure the plist is correct). If you are not lucky and you need to make some modifications to files, have a look at the do-build target in the Makefile, this is the place where some changes are done to create a nice user experience.

If you arrive here while creating a new linux_base port, lean back and feel a bit proud. You managed to create a new linux_base port. It is not very well tested at this moment, and it is far from everything which needs to be done to have the complete Linux infrastructure for a given Linux release, but the most important part is done. Please notify [email protected] and call for testers.

What is missing?

The full Linuxulator infrastructure for the FreeBSD Ports Collection has some more ports around a linux_base port. Most of the infrastructure for this is handled in Mk/bsd.linux-apps.mk.

UPDATE: I got some time to write how to update the Linux-infrastructure ports.

Share

Alberto Villa, FreeBSD committer

I’m happy to announce my first mentee, Raphael Kubo da Costa (rakuco@). If you look at my commit history, you’ll understand I was getting tired of committing patches “Submitted by: Raphael Kubo da Costa”, so it was time to punish him with a ports commit bit and let him do the hard job on his […]

Alberto Villa, FreeBSD committer

It’s time for another roadmap. And let’s make it fast. What’s keeping KDE SC 4.7 still out of FreeBSD? Well, the high number of tarballs (and thus ports) in which it was split for this release. Actually they’re not too much, but they require a deep scan to update dependencies. We’ll probably be ready for […]

Two New Videos: SuperPages and NanoBSD

Thanks to Kirk McKusick, I'm happy to announce two new fully edited high quality videos from BSDCan 2011 in the BSD Conferences YouTube channel. I've also created a new playlist for the BSDCan 2011 videos.

The first talk is "Superpages in FreeBSD" by McKusick, and it describes the addition of superpage support to the FreeBSD 8 kernel on the Intel PC architecture. Superpages aggregate together standard-sized hardware pages into much larger "superpages". Each superpage requires only one entry in the page table replacing the numerous entries used by the standard-sized hardware pages.



The second talk is "Updates from NanoBSD: FreeNAS drives NanoBSD development" from Warner Losh, and it describes the basics of NanoBSD and how FreeNAS moved over to NanoBSD.



We now have 108 high-quality videos in the BSD Conferences channel. These videos have been watched in aggregate over 400,000 times, and our most popular video remains McKusick's FreeBSD Kernel Internals Lecture.

As a reminder, this channel was setup specifically for the BSD technical community and does not have the standard limitations on video size for other types of YouTube uploads. If you have additional video content from a conference, presentation, or class about BSD Unix please get in touch and I'd be happy to help you publish the content here.

Accepting Travel Grant Applications for EuroBSDCon 2011

Calling all FreeBSD developers needing assistance with travel expensesto EuroBSDCon 2011.

The FreeBSD Foundation will be providing a limited number of travelgrants to individuals requesting assistance. Please fill out and submitthe Travel Grant Request Application bySeptember 5, 2011 to apply for this grant.

How it works:

This program is open to FreeBSD developers of all sorts (kernel hackers,documentation authors, bugbusters, system administrators, etc). In somecases we are also able to fund non-developers, such as active communitymembers and FreeBSD advocates.

(1) You request funding based on a realistic and economical estimate oftravel costs (economy airfare, trainfare, ...), accommodations(conference hotel and sharing a room), and registration or tutorialfees. If there are other sponsors willing to cover costs, such as youremployer or the conference, we prefer you talk to them first, as ourbudget is limited. We are happy to split costs with you or anothersponsor, such as just covering airfare or board.

If you are a speaker at the conference, we expect the conference tocover your travel costs, and will most likely not approve your directrequest to us.

(2) We review your application and if approved, authorize you to seekreimbursement up to a limit. We consider several factors, including ouroverall and per-event budgets, and (quite importantly) the benefit tothe community by funding your travel.
Most rejected applications are rejected because of an over-all limit ontravel budget for the event or year, due to unrealistic or uneconomicalcosting, or because there is an unclear or unconvincing argument thatfunding the applicant will directly benefit the FreeBSD Project.Please take these points into consideration when writing your application.

(3) We reimburse costs based on actuals (receipts), and by check or banktransfer. And, we do not cover your costs if you end up having to cancel your trip. We require you to submit a report on your trip, which we mayshow to current or potential sponsors, and may include in our semi-annual newsletter.

There's some flexibility in the mechanism, so talk to us if somethingabout the model doesn't quite work for you or if you have any questions.The travel grant program is one of the most effective ways we canspend money to help support the FreeBSD Project, as it helps developersget together in the same place at the same time, and helps advertise andadvocate FreeBSD in the larger community.

Semi-Annual Newsletter Available

We are pleased to announce the publication of The FreeBSD Foundation's 2011 Semi-Annual Newsletter. In this edition:

  • Letter From the President
  • Fundraising Update
  • Development Project Updates
  • IPv6 support in FreeBSD and PC-BSD
  • Implementing support of GEM, KMS, and DRI for Intel Drivers
  • Resource Containers Project
  • Feed-Forward Clock Synchronization Algorithms Project
  • Five New TCP Project
  • libcxxrt C++ Runtime Available Under BSD License
  • Conference Updates
  • AsiaBSDCon 2011
  • BSDCan 2011
  • 2011 Grant and Travel Grant Recipients
  • NYI Testimonial
  • Foundation Update
  • Financials
You can help to support the FreeBSD Project by making a donation to the Foundation.

    bsd_day(2011)

    bsd_day(2011) (http://bsdday.eu/2011), Slovak University of Technology, Bratislava, Slovakia 5 November, 2011. A new BSD-Day is approaching again to gather Central European BSD people to meet. The event features developers so they can popularize their work and communicate with users and potential future partners. There are no formalities, papers, registration or participation fee, however the invited developers are encouraged to give a brief talk about their favorite BSD-related topic. The goal is to motivate everybody, especially university students to work with BSD systems.

    bsdtalk207 – ArabBSD with Mohammed Farrag

    Interview with Mohammed Farrag about the ArabBSD project.

    The ArabBSD website https://sites.google.com/site/arabbsd/

    Contact us (Calling for Volunteers )webpage: https://sites.google.com/site/arabbsd/contacts

    FreeBSD Summer Course Tutorials https://sites.google.com/site/arabbsd/freebsd-summer-course

    File Info: 15min, 7MB.

    Ogg Link:
    http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk207.ogg

    FreeBSD 9.0-BETA1 Available

    The first test build for the FreeBSD-9.0 release cycle is now available. ISO images for the architectures amd64, i386, ia64, powerpc, powerpc64, and sparc64 are available on most of our FreeBSD mirror sites. One of the many new features in 9.0 we would like to be tested is the new installer, so we encourage our users to do fresh installation on test systems.