Monthly Archives: March 2011

BSDTalk 203 – BSDCan and PGCon with Dan Langille

Interview with Dan Langille. We talk about BSDCan and PGCon 2011. More information on these conferences at http://www.bsdcan.org/2011/ and http://www.pgcon.org/2011/File info: 14min, 7MB.Ogg Link:http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk203.ogg

BSDTalk 203 – BSDCan and PGCon with Dan Langille

Interview with Dan Langille. We talk about BSDCan and PGCon 2011. More information on these conferences at http://www.bsdcan.org/2011/ and http://www.pgcon.org/2011/

File info: 14min, 7MB.

Ogg Link:
http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk203.ogg

We are not only committers, we are also humans (3)

Sofian Brabez is now ports committer and miwi@ accepted to co-mentor him with me. As usual, I sent him few questions. Welcome Sofian!

1- Can you introduce yourself?
My name is Sofian Brabez, I'm 26 years old and I come from Paris, France. I discovered FreeBSD in 2004. My nickname is sbz and I try to use it everywhere as my login name but when it's impossible I use sbrabez.

2- When was your first pr and why?
My first pr was ports/118088 in 2007 to try to fix eclipse desktop integration in gnome but it wasn't commit :).

3- Beer/whiskey?
Beer.

4- Are you in a LUG/BSD association?
I'm in a French association named GCU-Squad which is a Free Unices advocacy and support group.

5- What is the most embarassing thing you will admit to having on your mp3/ogg player?
Papa Pingouin

6- Vim or Emacs?
Vim.

7- What is the air-speed velocity of an unladen swallow?
"African or European?" :)

8 - What would you like to achieve as a Ports committer?
Take care of security and python ports.

9- Coffee, tea, other?
Coffee because there is no life before coffee !

10- Are you active on other opensource projects too? Which one?
I'm semi-active in some opensource projects:
- W3af where I'm responsible for the FreeBSD port.
- Inguma where I enjoy writing python, I joined the team recently.
- For others I have my ohloh account.


Thanks and have fun sbz@

…and fresh blood came to kde@

Actually, the blog post by miwi@ referenced in the title has nothing to do with this, but it’s nice to see how, few days after that call to arms, the FreeBSD KDE Team gets a new member! David Naylor was invited to join the team after that he spent more than a year contributing patches [...]

Ground Labs Announces FreeBSD Support

From the press release:

Ground Labs, a global leader in the development of security and auditing software for the payment card industry, recently announced the introduction of native support for FreeBSD within its cardholder data discovery products for PCI compliance.

"Our goal is to provide support for all major operating systems that are used to store, transmit or process cardholder data," said Stephen Cavey, director of corporate development for Ground Labs. "FreeBSD is used in mission-critical environments worldwide. It is therefore a perfect addition to our portfolio of supported platforms."

Many large organisations, including large web hosting providers rely on FreeBSD to achieve high levels of uptime. "FreeBSD is known for being a reliable and robust operating system," said Peter Duthie, chief architect for Ground Labs. "By offering native support for FreeBSD within our cardholder data discovery products we can enable more organisations to identify non-compliant instances of cardholder data storage and facilitate compliance with PCI DSS 2.0."

Ground Labs’ flagship products, Card Recon and Enterprise Recon, previously supported 5 operating systems, including Windows, Linux, Solaris, AIX and HPUX. Card Recon can now be used on FreeBSD systems to perform accurate cardholder data discovery scans. Enterprise Recon users will also benefit now that Enterprise Recon Node Agents can be deployed on remote FreeBSD systems within larger environments to achieve centralised monitoring and visibility of PCI compliant cardholder data storage practices.

"We’re very pleased to have Ground Labs offer FreeBSD support within its cardholder data discovery products," said Erwin Lansing, FreeBSD developer and member of the Ports Management team. "While FreeBSD is widely deployed throughout the industry, enterprise-grade commercially supported software tools are only just starting to appear. Given the growing importance of the PCI Compliance Standards, Ground Labs products add valuable tools for FreeBSD users, who will benefit from this new level of support."

FreeBSD versions of Card Recon and Enterprise Recon Node Agents are available at no additional charge for download by existing and new customers.

About Ground Labs

Ground Labs is a global leader in the development of security and auditing software solutions for PCI compliance. Its flagship products, Card Recon and Enterprise Recon, identify and analyze cardholder data storage risks on thousands of computer systems worldwide. Merchants, acquirers and schemes use Ground Labs products to achieve and maintain PCI compliance, while QSAs use those same products to validate compliance and produce accurate reports.

Ground Labs Announces FreeBSD Support

From the press release:

Ground Labs, a global leader in the development of security and auditing software for the payment card industry, recently announced the introduction of native support for FreeBSD within its cardholder data discovery products for PCI compliance.

"Our goal is to provide support for all major operating systems that are used to store, transmit or process cardholder data," said Stephen Cavey, director of corporate development for Ground Labs. "FreeBSD is used in mission-critical environments worldwide. It is therefore a perfect addition to our portfolio of supported platforms."

Many large organisations, including large web hosting providers rely on FreeBSD to achieve high levels of uptime. "FreeBSD is known for being a reliable and robust operating system," said Peter Duthie, chief architect for Ground Labs. "By offering native support for FreeBSD within our cardholder data discovery products we can enable more organisations to identify non-compliant instances of cardholder data storage and facilitate compliance with PCI DSS 2.0."

Ground Labs’ flagship products, Card Recon and Enterprise Recon, previously supported 5 operating systems, including Windows, Linux, Solaris, AIX and HPUX. Card Recon can now be used on FreeBSD systems to perform accurate cardholder data discovery scans. Enterprise Recon users will also benefit now that Enterprise Recon Node Agents can be deployed on remote FreeBSD systems within larger environments to achieve centralised monitoring and visibility of PCI compliant cardholder data storage practices.

"We’re very pleased to have Ground Labs offer FreeBSD support within its cardholder data discovery products," said Erwin Lansing, FreeBSD developer and member of the Ports Management team. "While FreeBSD is widely deployed throughout the industry, enterprise-grade commercially supported software tools are only just starting to appear. Given the growing importance of the PCI Compliance Standards, Ground Labs products add valuable tools for FreeBSD users, who will benefit from this new level of support."

FreeBSD versions of Card Recon and Enterprise Recon Node Agents are available at no additional charge for download by existing and new customers.

About Ground Labs

Ground Labs is a global leader in the development of security and auditing software solutions for PCI compliance. Its flagship products, Card Recon and Enterprise Recon, identify and analyze cardholder data storage risks on thousands of computer systems worldwide. Merchants, acquirers and schemes use Ground Labs products to achieve and maintain PCI compliance, while QSAs use those same products to validate compliance and produce accurate reports.

More wireless hackery

I've had a busy week in the land of FreeBSD wireless driver hacking.
I've been porting over the AR9280 TX calibration changes from ath9k. The AR9280 NIC in my laptop loves it - but the AR9220 based SR71-12 is still unhappy. I'll go poke around and see if I can figure out what the problem is. I have some other AR9280 series NICs to try out.
I've been using hostap mode on the AR5416 and AR9160 and it still seems quite fine and stable.
I've ported over a number of changes to the AR9285 codebase from Linux ath9k. These include:
  • Fixing the TX power calibration curve code;
  • Handling a TX power offset at the beginning of the calibration curve;
  • Updating how the board is initially calibrated on reset;
  • Adding PA calibration.
I'm also trying to remove some of the duplicate code in the HAL that has crept up as chipset support was added.
I'm waiting to hear back from some other users about what effects it has on their setup. I still have to port over the (completely rewritten) radio board configuration code from ath9k that sets up all kinds of radio related gain, offset and various things. There's also software antenna diversity to port over. I hope that this all fixes the issues users have been reporting with AR9285 NICs just plain not working in a lot of instances - but working fine in others. I also hope it fixes the AR2427 support.
I'm still very unclear what may be up with the AR9280 support. The Ubiquiti high-powered cards tend to have strange EEPROM calibration settings, so it's quite possible FreeBSD just handles one of the options badly. That's going to take some time to figure out.
There's no point in getting 802.11n support enabled if I can't TX and RX cleanly, so I'm leaving 802.11n alone (for the most part) until the radios and MACs are behaving themselves.

More wireless hackery

I've had a busy week in the land of FreeBSD wireless driver hacking.

I've been porting over the AR9280 TX calibration changes from ath9k. The AR9280 NIC in my laptop loves it - but the AR9220 based SR71-12 is still unhappy. I'll go poke around and see if I can figure out what the problem is. I have some other AR9280 series NICs to try out.

I've been using hostap mode on the AR5416 and AR9160 and it still seems quite fine and stable.

I've ported over a number of changes to the AR9285 codebase from Linux ath9k. These include:
  • Fixing the TX power calibration curve code;
  • Handling a TX power offset at the beginning of the calibration curve;
  • Updating how the board is initially calibrated on reset;
  • Adding PA calibration.
I'm also trying to remove some of the duplicate code in the HAL that has crept up as chipset support was added.

I'm waiting to hear back from some other users about what effects it has on their setup. I still have to port over the (completely rewritten) radio board configuration code from ath9k that sets up all kinds of radio related gain, offset and various things. There's also software antenna diversity to port over. I hope that this all fixes the issues users have been reporting with AR9285 NICs just plain not working in a lot of instances - but working fine in others. I also hope it fixes the AR2427 support.

I'm still very unclear what may be up with the AR9280 support. The Ubiquiti high-powered cards tend to have strange EEPROM calibration settings, so it's quite possible FreeBSD just handles one of the options badly. That's going to take some time to figure out.

There's no point in getting 802.11n support enabled if I can't TX and RX cleanly, so I'm leaving 802.11n alone (for the most part) until the radios and MACs are behaving themselves.

[ECFT] drm/dri/mesa/xorg-server update [Part 1]

Hi,

First of all, note that this is very experimental, so you really have to know what
you’re doing. We managed to get drm/dri with the newer xorg-server to work,
and we have removed the support for WITHOUT_NOUVEAU.

We have just updated the xorg-dev repo:

– libdrm -> 2.4.24
– libGL to 7.10.1
– libGLU to 7.10.1
– libGLUw to 7.10.1
– libglut to 7.10.1
– xproto to 7.0.17
– libXaw to 1.0.9
– libXt to 1.1.0
– libX11 to 1.4.1
– xorg-server to 1.9.4

After installing these, you will have to rebuild the following ports:

– your graphic driver
– keybord driver
– mouse/synaptics driver

Upon rebuilt, restart them.
So to get the xorg stuff you will need to:
run

svn co https://trillian.chruetertee.ch/svn/ports/branches/xorg-dev

A small merge script to merge the svn checkout into the real portstree can
be found here:

http://people.freebsd.org/~miwi/xorg/xorgmerge

The script is a modified version of the kdemerge script. Please set the KDEDIR
variable to the path of your X.org ports.

After merging, run one of the following command, depending on which tool you use
to manage your installed packages.

portupgrade -af \*
portmaster -af

Please report any problems and issues to x11 (at) FreeBSD.org.

Again, please be aware that this is very experimental, and
I personally haven’t tested any 3D things yet, but we want
to share our work and start testing to get early feedback
for improvements. We plan to update Xorg fully to 7.6 after
we get some feedback for update part 1. It will be much easier
for us to figure out what the problems are with the updates
being separated in 3 parts. Please make sure you know what
you’re doing.

Thanks to Piter (gahr@) for helping me to get it compiled with our
base gcc version.

- Martin

PS: ECFT -> Experimental Call for Testing :P

Summary of Five New TCP Congestion Control Algorithms Project

Grenville Armitage has provided a summary of the completed TCP Congestion Control Algorithms project:

Background

TCP is a crucial part of any modern operating system. FreeBSD's standard "NewReno" congestion control (CC) is not able to fully utilize the high capacity links available today. A range of newer CC algorithms have emerged (and continue to emerge) from the networking research community over the past 15+ years. These include traditional loss-based algorithms (where packet losses indicate network congestion) and delay-based algorithms (where changes in Round Trip Time, RTT, are used to infer network congestion).

However, to date FreeBSD's TCP stack has not had an easy-to-use mechanism for introducing new CC algorithms. In recent years the Centre for Advanced Internet Architectures (CAIA) at Swinburne University of Technology has (with the support of the Cisco University Research Program Fund at Community Foundation Silicon Valley) been developing a range of extensions to the FreeBSD TCP stack. These included a modular framework for adding new CC algorithms and new modular implementations of the existing NewReno algorithm, four other algorithms from the literature (H-TCP, CUBIC, Vegas and HD) and a novel algorithm developed at CAIA (CHD). In mid-2010 the FreeBSD Foundation funded CAIA to complete, tidy up and commit a number of these key enhancements to the FreeBSD TCP stack.

Delivered

Our project, "Five New TCP Congestion Control Algorithms for FreeBSD", has delivered
the following enhancements to FreeBSD's TCP stack:
  • Modular congestion control framework.
  • Khelp (Kernel Helper) and Hhook (Helper Hook) frameworks.
  • Basic Khelp/Hhook (Kernel help/hook) integration with the TCP stack.
  • ERTT (Enhanced Round Trip Time) Khelp module for delay-based TCP algorithms.
  • Modularised implementations of NewReno, CUBIC and HTCP loss-based TCP CC algorithms.
  • Modularised implementations of Vegas, "HD" and "CHD" delay-based TCP CC algorithms.
  • Technical report comparing the computational overhead associated with TCP before and after integrating the new frameworks and modularised NewReno algorithm

Benefits

Each congestion control algorithm is implemented as a loadable kernel module. Algorithms can be selected to suit the application/network characteristics and requirements of the host's installation. The modular CC framework also makes it much easier for developers to implement new algorithms, allowing FreeBSD's TCP to be at the forefront of advancements in this area, while still maintaining the stability of its network stack.

CUBIC and HTCP are variants of TCP that provide significant performance improvements (relative to NewReno) over high bandwidth, high latency paths. Vegas, HD, and CHD utilise RTT fluctuations to provide a more timely indication of network congestion -- by not forcing network queues to overflow, delay-based CC algorithms can help to keep queuing delays low along a network path. CHD is also tolerant of packet losses that are unrelated to congestion (such as can occur over wireless links).

In addition, the Khelp/Hhook frameworks provide useful kernel infrastructure which are not specific to the TCP stack and we anticipate they will be used elsewhere in the kernel in the future to provide other unrelated enhancements to FreeBSD.

Participants

Code development, testing, and documentation: David Hayes and Lawrence Stewart

Editorial review of code and documentation: Rui Paulo and Bjoern Zeeb

Project supervision: Grenville Armitage

Project URL: http://caia.swin.edu.au/freebsd/5cc/

Summary of Five New TCP Congestion Control Algorithms Project

Grenville Armitage has provided a summary of the completed TCP Congestion Control Algorithms project:

Background

TCP is a crucial part of any modern operating system. FreeBSD's standard "NewReno" congestion control (CC) is not able to fully utilize the high capacity links available today. A range of newer CC algorithms have emerged (and continue to emerge) from the networking research community over the past 15+ years. These include traditional loss-based algorithms (where packet losses indicate network congestion) and delay-based algorithms (where changes in Round Trip Time, RTT, are used to infer network congestion).

However, to date FreeBSD's TCP stack has not had an easy-to-use mechanism for introducing new CC algorithms. In recent years the Centre for Advanced Internet Architectures (CAIA) at Swinburne University of Technology has (with the support of the Cisco University Research Program Fund at Community Foundation Silicon Valley) been developing a range of extensions to the FreeBSD TCP stack. These included a modular framework for adding new CC algorithms and new modular implementations of the existing NewReno algorithm, four other algorithms from the literature (H-TCP, CUBIC, Vegas and HD) and a novel algorithm developed at CAIA (CHD). In mid-2010 the FreeBSD Foundation funded CAIA to complete, tidy up and commit a number of these key enhancements to the FreeBSD TCP stack.

Delivered

Our project, "Five New TCP Congestion Control Algorithms for FreeBSD", has delivered
the following enhancements to FreeBSD's TCP stack:
  • Modular congestion control framework.
  • Khelp (Kernel Helper) and Hhook (Helper Hook) frameworks.
  • Basic Khelp/Hhook (Kernel help/hook) integration with the TCP stack.
  • ERTT (Enhanced Round Trip Time) Khelp module for delay-based TCP algorithms.
  • Modularised implementations of NewReno, CUBIC and HTCP loss-based TCP CC algorithms.
  • Modularised implementations of Vegas, "HD" and "CHD" delay-based TCP CC algorithms.
  • Technical report comparing the computational overhead associated with TCP before and after integrating the new frameworks and modularised NewReno algorithm

Benefits

Each congestion control algorithm is implemented as a loadable kernel module. Algorithms can be selected to suit the application/network characteristics and requirements of the host's installation. The modular CC framework also makes it much easier for developers to implement new algorithms, allowing FreeBSD's TCP to be at the forefront of advancements in this area, while still maintaining the stability of its network stack.

CUBIC and HTCP are variants of TCP that provide significant performance improvements (relative to NewReno) over high bandwidth, high latency paths. Vegas, HD, and CHD utilise RTT fluctuations to provide a more timely indication of network congestion -- by not forcing network queues to overflow, delay-based CC algorithms can help to keep queuing delays low along a network path. CHD is also tolerant of packet losses that are unrelated to congestion (such as can occur over wireless links).

In addition, the Khelp/Hhook frameworks provide useful kernel infrastructure which are not specific to the TCP stack and we anticipate they will be used elsewhere in the kernel in the future to provide other unrelated enhancements to FreeBSD.

Participants

Code development, testing, and documentation: David Hayes and Lawrence Stewart

Editorial review of code and documentation: Rui Paulo and Bjoern Zeeb

Project supervision: Grenville Armitage

Project URL: http://caia.swin.edu.au/freebsd/5cc/

mdcached

For some years (many more than really necessary) I have been workin on and off on a pet project of mine, mdcached - a cache server similar to memcached. I've figured it's time to finish it and bring it to usable state now, so (I hope) I'll be writing up the progress here. For now, I have some really nice benchmark results which show off its strengths compared to memcached.

Read more...

mdcached

For some years (many more than really necessary) I have been workin on and off on a pet project of mine, mdcached - a cache server similar to memcached. I've figured it's time to finish it and bring it to usable state now, so (I hope) I'll be writing up the progress here. For now, I have some really nice benchmark results which show off its strengths compared to memcached.

Read more...

FreeBSD needs fresh Blood!

Oh well, it’s time to write some nice job offer, of course it’s all
for free, and you can’t earn any money out of it, but you’ll get a
big thanks, hugs and love from the community. Ask your self, how
long have you been using FreeBSD. Months? Years? Decades? And you
love using it because of whatever reason but at the same time
you’re feeling a bit guilty to use it all for free without giving
anything back? Well now you’ll have the chance to change that.
We at FreeBSD are always in need of new people who are willing
to spare some of their time and effort into FreeBSD development.

Let me share a bit of my experience. I have (re)built a lot of
teams in the past, such as gecko@, kde@, python@, and I was
involved in the creation of FreeBSD vbox@ team. I have always
managed to get assistance from a lot of people, but recently more
and more people have started to complain about the slowness,
broken commits and requested for more Call for Testing. And that
is actually a big problem. I am the kind of person who like to
call for test, but I am also the kind of person who easily gets
disapointed when I’m not getting much feedbacks. The best example
here is ATI, Xorg and Xfce update. I did a call for testing because
Xorg and Driver updates is always a big issue because there are so
many different hardware involved with various configurations. From
the call for testing, we managed to get a total of 19 mails of
positive feedback and after 2 weeks I’ve committed the update.
What happened after that was I received a lot of complains for
not conducting much testing, yadda, yadda. Well I say it ain’t
my fault for not testing much, but it is also your fault for not
helping us. It is always easy to blame instead of helping. Ask
yourself why have you not helped us in testing properly and give
us feedbacks. Complaining is fine when it is done in the right
way, with the right tone.

While I’m talking about Xorg, the FreeBSD Xorg Team is currently
a one man show effort, supported by kwm@ and fluffy@. Xorg alone
is too big to get worked on. Plus you should not think that it is
affecting the ports only, but it affects the kernel as well, which
we are having the most problems at the moment. And of course I
would like to call for help on that as well. Based on my last call
for help, it is funny to see how many people wanted to offer some
help, but after knowing the amount of work involved, I have stopped
hearing from these guys. I understand that to update Xorg is always
a crappy job but I love doing it, because it is nice to get more and
more experience in understanding how things work, and it helps to
improve my skills a lot.

Lets a talk a bit about our FreeBSD KDE Team. KDE is nice, but it
really is a fat project. It needs a lot of love, and maintenance
time. Currently it’s a 4 people project, namely makc@, fluffy@ and
avilla@. While for support Raphael Kubo da Costa is handling it
actively. The thing is, KDE involves more than just KDE packages.
It includes Qt, PY-Qt, KOffice and Cmake as well. It is a big
project too and it would be nice to find more people to contribute
in the development.

And now lets talk about gecko@. gecko@ includes all Mozilla Project,
namely Firefox, Thunderbird and Seamonkey. It is currently maintained
by beat@ and decke@, and supported by flo@ and andreas. So again,
I’d like to see some fresh faces for this project as well. If you are willing
to help, do ping us via mail :p.

As for FreeBSD Gnome Team, well I can’t say much about gnome but
whenever I see the cvs commits in marcuscome tree, it seems like
most work for the upcoming gnome3 is done by kwm@, and supported
by marcus@, mezz@ and avl@. Gnome includes not only Gnome things
but it also include gtk and cairo, the one that always cause
problems in a major update. I think the team would love to have
some fresh blood in the team.

Okay, all of these need an understanding of programming and
scripting. If you think that you can’t do any of that, testing would
also help much. FreeBSD is one of the best documented open source
project, so that’s another area that could use some help too. Check
if FreeBSD.org is available in your language, or start helping to
improve the FreeBSD documents in your language. It would be very
helpful and the community will thank you for that. So if you would
like to offer some help, ping me in irc/jabber/mail :-)

- Martin

PCBSD sponsor FreeBSD KDE Build box

I’d like to note that the KDE FreeBSD Team is now
receiving assistance from PCBSD again. Since I left
the KDE Team, things have gone terribly wrong. After
having a few conversations with my old team, I finally
figured out what the problems were. But the main problem
was missing a package build system, and I managed to find
a solution for that with Kris’s help. Apparently this was
a root cause to some other issues and having it solved
helped to speed up other processes.

I’d like to say thanks to Kris Moore, Josh Paetzel,
Dru Lavigne and of course the iXsystem for their
generous donation to build a test machine for KDE.

PS: This means not i jump back to the KDE Team
but I’m always willing to Help when i can.