Monthly Archives: March 2009

Multi processor compilations for everyone

Two days ago, I have checked in probably most requested feature of last few years. Ports framework now systematically supports building ports on multiple processing cores. It is achieved by passing -jX flag to make(1) running on vendor code. Of course not all ports handle this well, experimental run on pointyhat with this flag globally enabled turned up shy of 400 failures. Because of that, the feature was designed as a whitelist. Individual ports need to be enabled, and indeed, fellow developers took on and already started adding required declarations to popular ports like Firefox and others.

If you are FreeBSD ports user, you don’t need to do anything to enable the new feature. Whitelisted ports will automatically make use of all processors available in your computer. If you want, for some reasons, to disable this feature, put DISABLE_MAKE_JOBS=yes to your /etc/make.conf. By default, the level of parallelization will be equal to a number of processing cores in your machine. If you want to override this number, use for example MAKE_JOBS_NUMBER=6, again in /etc/make.conf. And if you are extra brave, or you want to check out all the yet unmarked ports, if they will build, you can define FORCE_MAKE_JOBS=yes in /etc/make.conf.

If you are FreeBSD port maintainer, nothing changes for you, if you don’t want. If you want to enable the use of multiple cores in your port, add MAKE_JOBS_SAFE=yes to a block somewhere below dependency declarations. If you know your port does not handle -jX well, and want to disable it from using -jX even when user forces this feature, use MAKE_JOBS_UNSAFE=yes. And that’s all to it.

Gleb Kurtsou 2009-03-24 09:26:46

Haven’t posted about progress with lyear2 filtering for a while. One notable improvement is addition of ethernet address masks to dummynet.

Just configure a pipe. New masks available: src-ether and dst-ether (and a shortcut for specifying both of them: ether)
# ipfw pipe 1 config bw 1Mb mask ether

And use it:
# ipfw add 1100 pipe 1 src-ether 00:11:11:11:11:11 dst-ether 00:22:22:22:22:22 out via bridge0 layer2
# ipfw add 1200 pipe 1 dst-ether 00:11:11:11:11:11 src-ether 00:22:22:22:22:22 out via bridge0 layer2


# ipfw pipe show
00001: 1.000 Mbit/s 0 ms 50 sl. 2 queues (64 buckets) droptail
mask: ff:ff:ff:ff:ff:ff -> ff:ff:ff:ff:ff:ff tag: 0x0000
BKT _Source Ether Addr_ _Dest. Ether Addr__ Tag Tot_pkt/bytes Pkt/Byte Drp
40 00:11:11:11:11:11 00:22:22:22:22:22 0 2 196 0 0 0
43 00:22:22:22:22:22 00:11:11:11:11:11 0 2 196 0 0 0

Besides, masking packet by tag is also there:
# ipfw add 200 pipe 1 ip from any to any tagged 1-1000 via bridge0 layer2

As several tags per packet supported, it is necessary to specify desired tag range, tag, or any tag:
# ipfw add 200 pipe 1 ip from any to any tagged any via bridge0 layer2

Patches for current and 7-stable available:
l2filter-current-2009-03-24
l2filter-stable-2009-03-24 (updated link)

Slowly but Steadily Getting There


The Greek FreeBSD translation team has been working on and off on the Greek translation of the FreeBSD documentation set for a long time now. We started getting a lot of commit actions when Manolis joined the team, and he is now an undisputed “overlord” of the Greek Handbook. He gates patches from other contributors, and has submitted far more text than me at this point:

$ pwd
/hg/doc/el
$ hg churn
ncvs                                627371 ******************************************
sonicy                              114351 *************
keramida                             23019 ***

We have Greek versions of more than half of the English documentation tree, and there are 18 files marked for update with special comments in the source of each file:

$ find en_US.ISO8859-1 -type f | wc -l
     342
$ find el_GR.ISO8859-7 -type f | wc -l
     125
$ find el_GR.ISO8859-7 | checkupdate -c | nl
     1  1.39       -> 1.60       el_GR.ISO8859-7/articles/Makefile
     2  1.49       -> 1.51       el_GR.ISO8859-7/articles/new-users/article.sgml
     3  1.43       -> 1.61       el_GR.ISO8859-7/articles/problem-reports/article.sgml
     4  1.6        -> 1.19       el_GR.ISO8859-7/articles/releng-packages/article.sgml
     5  1.18       -> 1.19       el_GR.ISO8859-7/articles/releng/Makefile
     6  1.48       -> 1.81       el_GR.ISO8859-7/articles/releng/article.sgml
     7  1.1103     -> 1.1110     el_GR.ISO8859-7/books/faq/book.sgml
     8  1.414      -> 1.421      el_GR.ISO8859-7/books/handbook/advanced-networking/chapter.sgml
     9  1.1        -> NONE       el_GR.ISO8859-7/books/handbook/appendix.decl
    10  1.1        -> NONE       el_GR.ISO8859-7/books/handbook/chapter.decl
    11  1.233      -> 1.236      el_GR.ISO8859-7/books/handbook/config/chapter.sgml
    12  1.288      -> 1.290      el_GR.ISO8859-7/books/handbook/disks/chapter.sgml
    13  1.73       -> 1.74       el_GR.ISO8859-7/books/handbook/mac/chapter.sgml
    14  1.1        -> 1.116      el_GR.ISO8859-7/books/handbook/network-servers/chapter.sgml
    15  1.323      -> 1.334      el_GR.ISO8859-7/books/handbook/security/chapter.sgml
    16  1.45       -> 1.46       el_GR.ISO8859-7/books/handbook/vinum/chapter.sgml
    17  NO-%SRCID% -> 1.34       el_GR.ISO8859-7/share/sgml/glossary/freebsd-glossary.sgml
    18  1.43       -> 1.44       el_GR.ISO8859-7/share/sgml/trademarks.ent
$

Tonight, I think I am going to work on bringing the “new-users” and “problem-reports” articles up to date. Here’s to hope that the “checkupdate” output list will be shorter tomorrow :)

Posted in Computers, Free software, FreeBSD, FreeBSD people, Open source, Software Tagged: Computers, Free software, FreeBSD, FreeBSD people, Open source, Software

AsiaBSDCon 2009

I just presented a paper at AsiaBSDCon 2009 on the FreeBSD/mips status. There's a lot going on with the FreeBSD/mips project these days. You can find out most of the details from my paper, which is available in letter or a4 format.In summary, there's a number of new ports underway. There's Cavium Octeon, Atheros 7xxx/9xxxx, RMI Au1xxxx and a couple others.AsiaBSDCon is always fun. There's many Japanese developers that don't get a chance to go to other conferences who I only get to see in Japan. This conference allows me to see them, as well as talk to them to see what they are using BSD in general, and FreeBSD specifically. The Japanese have often lead the way in many areas, and seeing their current projects is very interesting.Also, I could cut and paste large parts of my paper here, but I'm not sure of the value of doing that. Can people that have strong opinions leave feedback on this issue.

AsiaBSDCon 2009

I just presented a paper at AsiaBSDCon 2009 on the FreeBSD/mips status. There's a lot going on with the FreeBSD/mips project these days. You can find out most of the details from my paper, which is available in letter or a4 format.

In summary, there's a number of new ports underway. There's Cavium Octeon, Atheros 7xxx/9xxxx, RMI Au1xxxx and a couple others.

AsiaBSDCon is always fun. There's many Japanese developers that don't get a chance to go to other conferences who I only get to see in Japan. This conference allows me to see them, as well as talk to them to see what they are using BSD in general, and FreeBSD specifically. The Japanese have often lead the way in many areas, and seeing their current projects is very interesting.

Also, I could cut and paste large parts of my paper here, but I'm not sure of the value of doing that. Can people that have strong opinions leave feedback on this issue.

port-tags on github

Some years ago I’ve made a little web application which allowed one to browse FreeBSD ports collection by tags, à la delicious.

The tags were not created by users but were instead generated from a couple of fields taken from every port’s Makefile, so it was not exactly a “social” software.

There was some limited amount of discussion on FreeBSD mailing lists, and a publicly accessible readonly SVN repository was created by my friend Erwin, but the overall interest was rather low.

Over time I moved on and basically stopped working on the project, but recently I had an idea - not exactly to re-surrect it, but to make it more easy for people who are interested to contribute.

Enter port-tags at github. Github is a tool to host git repositories of your open-source projects. Anybody can easily clone your repository, fork it completely, or submit their changes back to you. I only started using it today, so I cannot say much about its features and how convenient they are, but from what I’ve heard, it is very very nice.

So, if you are interested, and have got round tuits to spare, please hack on port-tags - maybe some good will eventually come out of it.

BSD Certification at AsiaBSDCon

19:54 JST

Just in case people are reading the Planets more than the conference websites, I thought I'd spam this here. :-)

If you are interested in taking the BSD Certification exam at AsiaBSDCon 2009 in Tokyo this Saturday (14 March 2009), please sign up for it on the website. I still have five exams "too many" if people want to sit them.

It would be silly to make empty exams travel 10 000km... :-)

Sign up now.

Gábor’s Blog 2009-03-09 11:09:29

I haven’t made too much noise in these days because I’m quite busy but this doesn’t mean that I’ve completely abandoned FreeBSD. A summary of my recent progress:

  • BSD grep: Finally, it seems working and compatible with GNU grep. A portbuild test is necessary to make sure nothing breaks with this and if the test performs well, we don’t have any barriers to have an own grep implementation, which is independent from GNU, smaller and cleaner.
  • NLS in libc: NLS support was disabled in libc a long time ago due to some issues with the catalog handling code. Now they are resolved but NLS was kept off. I’ve collected some catalogs, made a patch to turn it on and made some tests. It seems that I can commit this stuff soon.
  • BSD sort: I’ve started a sort implementation from scratch. It performs well and there are few missing features so I expect this to be ready soon. Not so soon as the former two items, but quite soon. Stay tuned!