Category Archives: Weblog

Tarsnap backup script

Tarsnap

Tarsnap is an advanced online-backup facility, entirely encrypted. The only copy of the keys used to encrypt and decrypt archives are in your own possession, so things that should be kept safe, are (in the current form) safe. Tarsnap makes extensive use of the Amazon EC2 and Amazon S3 for storage.

Tarsnap is originally written by the FreeBSD Security Officer Emiritus’ Colin Percival, on topics that he periodic gives talks about at various conferences. If you are able, you should seriously attend one of those talks

Script

Recently I rewrote a tarsnap backup script from Tim Bishop http://www.bishnet.net/tim/blog/2009/01/28/automating-tarsnap-backups/ to a more suitable script for us.

Tim backups his data via Tarsnap, all via the same way. That works well for him, but for our hosting company that is more tricky. We do not want to keep large amounts of data for our customers (which tend to change rapidly, for example emails that come in and go out and get deleted etc.). Instead we want to keep the minimal amount of data for these customers, and we want to offer them more advanced backup strategies for which we calculate an increased price (the minimal backup strategy is free).

After collaborating, we decided that next to the free strategy, we would like to offer a medium-term backup strategy, and a maximum-term backup strategy, where the former is a month of backups (7 weekdays, 4 weeks), and the latter is three months of backups (7 weekdays, 4 weeks, 3 months), so that going back in time is doable. If customers want to have a customized strategy, that would ofcourse be possible if we add that to the script.

Since we are keen on open source we would like to offer you the option to download the script, and if possible even enhance it more so that we can all benefit from it. Do note that we didn’t try to complicate the script, but instead keeping it as simple as possible. That means that we add more lines then likely needed, but it is very readable. One comment from Colin we got so far is that Tarsnap is capable of removing more files in one go (tarsnap -d -f -f ) and that is not yet implemented in the script. We will consider doing so.. ofcourse :-)

The script can be found here, tarsnap.script.

20131013
Updated the script with the update from Tim, this had been tested and works fine for us so far. Thanks Tim ! I shamelessly used the code in our code ;-)

One less hat :-)

So. Today we can congratulate George Neville-Neil as the new FreeBSD Security Team Secretary. It seems that I had been doing the job for around 5 years and 8 months (although not the entire time officially nor documented); which is a very long time. I decided to start reducing on the amount of hats that I carry so that I can focus more on the things that I want to focus on within FreeBSD.

Slowly but surely I am returning to my roots:

- Maintain the nl_NL tree

- Keep VuXML as up to date as possible

- Commit low hanging fruit from src/ so that the developers can focus on their development instead of being distracted by easier things. I will also try and merge for example usb/ related things from hps@.

That said: I will remain a doc committer, src committer and member of secteam .. please applaud George in his new task, it’s a thankless job and you really need to keep your head together :-)

Can the following be done with Postfix and LDAP?

Dear readers of my blog,

I have a “simple” question for you. I Would like to do the following, can someone that reads this and has suggestions and ideas respond to me at [email protected]

I have three various mailrelays, I would like to finish off mail that shouldn’t get in at the border relays. For this I have setup LDAP so that all three relays can query this LDAP Server. To fill the LDAP I use the Virtualmin application to make this as automatic as possible.

Currently the Virtual-addresses and Aliases are all in LDAP, as well as the useraccounts that receive email. No specific tag is added for local users.

I would like to have the relays do the following:

- Receive mail from XXX
- do RBL checks
- do postscreen checks and the like
- resolve the destination address (expand alias or virtual account)
if the resolved destination address lives outside of my domain (mailforwarding accounts) i would like to deliver it there immediately.
- check whether the resolved destination address is listed as local user and send it to the internal mailserver
(The internal mailserver will receive mail for local-user and only has to do spam checks for this user, no need to expand aliases etc).

Suggestions are welcome :))

FIXED: FreeBSD Jails PHP dirname WordPress

Dear Reader,

I had fixed the issue. Instead of using nullfs to get access to the /usr/home directories, I am using unionfs, which basically does the same for my goals (unless someone corrects me in misunderstanding things) and this does not seem to generate the same issues. Various sites are now running happily behind the WWW Jail. Time to finish my document on how I did setup the entire beast.

Thanks all for listening, helping, and giving tips (Alexander and Miroslav!)

HELP: FreeBSD Jails PHP dirname WordPress

So, I am still building up my jail structure and the last few evenings I was testing the FreeBSD jail wrt. PHP, Apache22-mpm-itk and wordpress.

Things started to break when I redirected external traffic to the jail. It seemed that require_once(dirname(dirname(__FILE__))) . ‘/wp-load.php’; does not work from within the jail.

I decided to do a little test and testing reveals that in a stand alone configuration the dirnames behave exactly the same, in both the host and the jail. Printing the directive within WordPress (when loading the admin pages f.ex.) reveals a ‘.’ instead of the ‘/path’ . It is resolvable by adding a ‘.’ to the directive so that wp-admin/admin.php loads the ../wp-load.php file instead of ‘/path/to/wordpress/wp-load.php’. Though this sounds very sily todo.

Did someone else encounter this? I Do not want to change enforcement of the statfs to some other value since the defaults should be good enough (given the testsript).

Relevant details: the /usr/home where the public_html files live, are nullfs rw mounted from the host and are available in the jail. The jail does username/group lookups through Ldap, and can see the various users. Apache had been build with the ITK patches so that every host runs under his/her own user. I do not see obvious differences between the regular host and the jail, the only real difference is the internal/external addresses used in the vhost configuration, but that is kinda obvious to me.

Let me know :-)

Family news…

Dear all,

It is with very great pleasure that I would like to tell you, that we (Denise, Luca and myself) are expecting our second child. Currently we are around 12 weeks and everything is looking good.

We saw the first images of our soon kid and new FreeBSD Hacker? ;-) He or she is looking beautiful already. We do not yet know the gender, but we are expecting the kid around the beginning of September.

Luca is also happy with these developments, so everything is in the works (actively as we speak) to move his room to the upperfloor, and we are going to prepare his old room for the baby.

Leave a message so that I can read them later on (I would like that): do realise that it’s moderated and that it might take a little before I can acknowledge your message :-)

FreeBSD: jails, ezjail, pfSense

During the last couple of days I am intensively using ezjail to administer several jails on my machines. They are currently IPv6 only (internet-facing) and are used to build pfsense images to test locally (still setting this up, need to cross compile to i386 from amd64), offer a testjail to a collegue to work together on a Opsview implementation on FreeBSD, whether or not we are going to succeed in that, and I just installed a test environment for my webservices. They are all contained in their own little box, having IPv4 connectivity outgoing through NAT, and native IPv6 connectivity from my “Vendor” on an extra subnet that I obtained.

I like this, so I am probably going to setup some more services here and there to perform some magic for me that might need external access. I will also tie them together with LDAP and the like so that it’s an uniform base. At the moment I do not have additional ideas about moving production services towards jails as well though.

Thanks to FreeBSD this all is damned easy. You should try it, or poke me in case you want to know more! :)

Happy New Year – 2011

In just one minute it will be 2011 (hey, scheduling things is fun, this gives you something to read while I am jumping around, celebrating with Luca, Denise, Rik and Larissa the coming of the new year, and perhaps drink a beer, or more but enough about that.

It’s my tradition to have a new years post, and this year I decided to schedule it for the first year. I wont be able to write a post before tommorrow or perhaps even later so “Sad but true”.

The last year saw a lot of sad things, sad changes and sad news, deaths and so on. Please take a minute to remember the persons you lost this year, think about the bad and the good things you shared. Cheerish those good moments, you can be upset about the negative things, but it will only make you more grumpy, which isn’t worth it. Life is too short!

OK So we considered the negative things of the last year, but ending the year with a negative thing is not right, right? So also take a minute to remember the positive things, positive changes and positive news, the birth’s you saw this year, the news that people are pregnant and are expecting a child, the new job, consider it and remember it.

From my position I would like to offer you my very best wishes for the upcoming year, I hope that you will see the positive things of life, respect eachother, and that you are healthy and can remain healthy (and your relatives).

Ofcourse my new years post wouldn’t be the same without mentioning my beloved FreeBSD. The last year we saw a few new releases, saw a lot of hard work, had to deal with the economic crises and loads of more things. This year we will get generous donations from you… right? So that we can build even more funky stuff, and keep the best operating system!

Welcome.. 2011!

Remko Lodder

Now that I started working for Ziggo in the Hague time is flying even quicker then it normally does. The traveltime increased, and traffic between Rotterdam and The Hague is dense. BUT, I have a bunch of nice collegue’s, nice working environment and a nice assignment so far. There are not many things to complain about (Darn, I am a dutchy, I neeeedddd to complain!) ;-)

It remains difficult to combine my opensource activities with work, but I am trying. I try to do as much security work as I can because that is the most important thing I can do for FreeBSD, but I would like to be a little more involved :-) .

I recently got notified that the 11th of December will see the light of another NLLGG meeting in Utrecht! Be there! I will try to be there as well, not presenting for the first time (well last time I was unable to present at all due to private circumstances, but I wrote a presentation then). I am too busy :-( .

Well, more updates to follow, “Massohl” :-)

Sting – Symphonica in Rosso

Well.. WELL!!! I just bought tickets for Denise and myself to go to Sting on the day before my birthday (15 okt). I like sting very much, I know a lot of songs from him.. and I am pleased about it already.. So if you are going to Arnhem on the 15th of oct and will visit Sting, I would be glad to know, perhaps we can even say hi.. I have standing places in the arena itself, so jumping is a requirement! Denise is already excited as well, or rather she told me “subtle” that she thought that it might be a very good concert :-) :-) so this is really cool!!

Sting – Symphonica in Rosso

Well.. WELL!!! I just bought tickets for Denise and myself to go to Sting on the day before my birthday (15 okt). I like sting very much, I know a lot of songs from him.. and I am pleased about it already.. So if you are going to Arnhem on the 15th of oct and will visit Sting, I would be glad to know, perhaps we can even say hi.. I have standing places in the arena itself, so jumping is a requirement! Denise is already excited as well, or rather she told me “subtle” that she thought that it might be a very good concert

evil-admin

Do you remember that? I was working on “Evilcoder-Admin” for a while. Now the project had been renamed to Evil-Admin and is very much alive again. Why? Because I “regret” the way certain hosting packages work. I would like to have a very simple program that delivers me everything I need. I can buy various packages like DirectAdmin or plesk, but they are all just not entirely that.

They all depend on some software version, or some specific library, or some specific way on how things are being dealt with. I am not entirely opposed to that, but I want to have the freedom to upgrade packages and my system where needed, instead of waiting for new versioned / static libraries. So, as long as functionality remains the same, I can use it.

I started developping evil-admin from scratch. Everything I had (including working DNS adminstration) was wiped (well preserved into my versioning control system, subversion) and I first started to design the application (read my previous post on why i might be doing this, it’s my job now!) and fill in requirements. Justin (my partner at JR-Hosting.nl, heh consider buying one of our products, it makes our world better, or at least mine haha) proposed that he can help with the frontend things, as long as I produce a working backend.

The entire idea is to store all information in a database (What else is new, we all do that!); extract it periodically and write it to flat files. That way the used system does not rely on the information retrieval speed, nor does it rely on databases in general.

There are different configurations for each machine in the cluster, like mail-relay settings, dns duplication etc. I am considering: Apache, (Bind or PowerDNS), ProFTPD, Exim, Postfix (incoming filters, then drop to exim since that has flexible delivery locations), MySQL, PostgreSQL, etc, all build on my FreeBSD servers.

If you would like to propose some ideas, just let me know and I will see what I can do. Remember that I just recently started recoding the application and that basically only a ”design.txt” is there, and some simple backend scripts are written. It takes time, it will take time, it wont be done yesterday, yadayada. But it feels good to write again!

evil-admin

Do you remember that? I was working on “Evilcoder-Admin” for a while. Now the project had been renamed to Evil-Admin and is very much alive again. Why? Because I “regret” the way certain hosting packages work. I would like to have a very simple program that delivers me everything I need. I can buy various packages like DirectAdmin or plesk, but they are all just not entirely that.

They all depend on some software version, or some specific library, or some specific way on how things are being dealt with. I am not entirely opposed to that, but I want to have the freedom to upgrade packages and my system where needed, instead of waiting for new versioned / static libraries. So, as long as functionality remains the same, I can use it.

I started developping evil-admin from scratch. Everything I had (including working DNS adminstration) was wiped (well preserved into my versioning control system, subversion) and I first started to design the application (read my previous post on why i might be doing this, it’s my job now!) and fill in requirements. Justin (my partner at JR-Hosting.nl, heh consider buying one of our products, it makes our world better, or at least mine haha) proposed that he can help with the frontend things, as long as I produce a working backend.

The entire idea is to store all information in a database (What else is new, we all do that!); extract it periodically and write it to flat files. That way the used system does not rely on the information retrieval speed, nor does it rely on databases in general.

There are different configurations for each machine in the cluster, like mail-relay settings, dns duplication etc. I am considering: Apache, (Bind or PowerDNS), ProFTPD, Exim, Postfix (incoming filters, then drop to exim since that has flexible delivery locations), MySQL, PostgreSQL, etc, all build on my FreeBSD servers.

If you would like to propose some ideas, just let me know and I will see what I can do. Remember that I just recently started recoding the application and that basically only

Busy with work

As people might have noticed within the FreeBSD world, my blog world, and personally. I am very busy with work lately :-) . Reason for that is that I was given the opportunity to become a Technical Designer, which means that I need to translate functional designs, into technical designs, write the proper documentation for it, go into meetings about what and how to approach certain things (I dont know everything, luckily :-) ), and help the engineer (Lemar, you are doing a great job!) in implementing the proposed infrastructure.

We (lemar and I) were both new to this kind of work and the way it is handled, and I think we are doing a really good job so far. Things are not always great, but then we sit together one way or the other, and choose to make the fogg less foggy and clarify a few things.

We already have established one of the smaller goals of our project, and more to come very quickly. I just hope that the partner that we are doing this for, is as happy with the situation as we are. but I think we are fine :-)

Anyway; I will try to blogpost a little more often, show my grumpy feelings about the decreasing privacy, tell happy things (like this one) and well whatever pops to mind.

Busy with work

As people might have noticed within the FreeBSD world, my blog world, and personally. I am very busy with work lately :-) . Reason for that is that I was given the opportunity to become a Technical Designer, which means that I need to translate functional designs, into technical designs, write the proper documentation for it, go into meetings about what and how to approach certain things (I dont know everything, luckily :-) ), and help the engineer (Lemar, you are doing a great job!) in implementing the proposed infrastructure.

We (lemar and I) were both new to this kind of work and the way it is handled, and I think we are doing a really good job so far. Things are not always great, but then we sit together one way or the other, and choose to make the fogg less foggy and clarify a few things.

We already have established one of the smaller goals of our project, and more to come very quickly. I just hope that the partner that we are doing this for, is as happy with the situation as we are. but I think we are fine

NL: Co2 onder mijn dorp

Ik heb zojuist een filmpje gezien van Netwerk.tv waar gesproken wordt over de opslag van Co2 onder Barendrecht, het dorp waar ik inwoon. Specifiek gaat het over de overburen in Carnisselande en dat er NOOIT EN TE NIMMER goed getest is met Co2, dat er rapporten achtergehouden worden etc.

De bevolking geeft (Democratisch!) aan dat ze het niet willen en niet vertrouwen, en de overheid, Cramer, luister je mee? Je hebt nooit normaal gereageerd op mijn verzoek voor meer informatie, concreet kreeg ik het volgende terug van uw medewerker:


Hartelijk dank voor uw e-mail van 28 januari aan minister Cramer waarin u uw zienswijze en uw bezorgdheid met betrekking tot de risico’s van CO2-opslag kenbaar maakt. Hierbij geef ik u, na overleg met de beleidsafdeling, een reactie op uw e-mail.

Vorig jaar zomer hebben wij geluisterd naar de bezwaren en gevoelens van het gemeentebestuur en de bevolking van Barendrecht. Een belangrijke uitkomst van deze gesprekken was dat er nog onvoldoende gegevens op tafel lagen om een besluit op te baseren. Daarom hebben wij aanvullend onderzoek laten doen op het gebied van locatiekeuze, veiligheid en gezondheidsklachten. Deze onderzoeken bevestigen de eerdere onderzoeken en tonen nogmaals aan dat Barendrecht op dit moment inderdaad de beste keuze voor kleinschalige CO2-opslag op land is en dat het echt veilig kan.

Dat neemt niet weg dat u er nog steeds niet gerust op bent dat dit project veilig kan worden uitgevoerd. U heeft gelijk als u zegt dat geen enkel risico is terug te brengen tot het absolute nulpunt. In het veiligheidsonderzoek heeft echter toetsing plaatsgevonden aan de hand van de strengste normen die wettelijk gesteld zijn. De conclusie hiervan is dat de omvang van de risico

NL: Co2 onder mijn dorp

Ik heb zojuist een filmpje gezien van Netwerk.tv (http://www.netwerk.tv/uitzending/2010-04-06/hoe-veilig-de-co2-opslag-barendrecht?page=1) waar gesproken wordt over de opslag van Co2 onder Barendrecht, het dorp waar ik inwoon. Specifiek gaat het over de overburen in Carnisselande en dat er NOOIT EN TE NIMMER goed getest is met Co2, dat er rapporten achtergehouden worden etc.

De bevolking geeft (Democratisch!) aan dat ze het niet willen en niet vertrouwen, en de overheid, Cramer, luister je mee? Je hebt nooit normaal gereageerd op mijn verzoek voor meer informatie, concreet kreeg ik het volgende terug van uw medewerker:


Hartelijk dank voor uw e-mail van 28 januari aan minister Cramer waarin u uw zienswijze en uw bezorgdheid met betrekking tot de risico’s van CO2-opslag kenbaar maakt. Hierbij geef ik u, na overleg met de beleidsafdeling, een reactie op uw e-mail.

Vorig jaar zomer hebben wij geluisterd naar de bezwaren en gevoelens van het gemeentebestuur en de bevolking van Barendrecht. Een belangrijke uitkomst van deze gesprekken was dat er nog onvoldoende gegevens op tafel lagen om een besluit op te baseren. Daarom hebben wij aanvullend onderzoek laten doen op het gebied van locatiekeuze, veiligheid en gezondheidsklachten. Deze onderzoeken bevestigen de eerdere onderzoeken en tonen nogmaals aan dat Barendrecht op dit moment inderdaad de beste keuze voor kleinschalige CO2-opslag op land is en dat het echt veilig kan.

Dat neemt niet weg dat u er nog steeds niet gerust op bent dat dit project veilig kan worden uitgevoerd. U heeft gelijk als u zegt dat geen enkel risico is terug te brengen tot het absolute nulpunt. In het veiligheidsonderzoek heeft echter toetsing plaatsgevonden aan de hand van de strengste normen die wettelijk gesteld zijn. De conclusie hiervan is dat de omvang van de risico

FreeBSD 7.3 released

As mentioned recently; 7.3 was supposed to go out of the door today.. and what do you think? IT GOT RELEASED! Download it as soon as possible to enjoy the new candy and fixes. The highlights from Ken Smith’s release form:

Some of the highlights:

- ZFS updated to version 13
- new boot loader gptzfsboot supports GPT and ZFS
- hwpmc(4) enhancements including support for core2/i7 processor
and pmcannotate(8)
- new mfiutil and mptutil tools for widely used RAID controllers
- NULL pointer vulnerability mitigation
- bind updated to 9.4-ESV
- Gnome updated to 2.28.2
- KDE updated to 4.3.5
- Perl updated to 5.10

New WAP4410N v2

After my latest replacement we noticed that my WAP4410N was a first generation version, having some problems with gigabit ethernet connections. After contacting Cisco we found out that the version should be replaced by a v2 instead.

Some difficulties with the replacement procedure (follow this, get a different ticket within the box, needing to get the arrangement with the carrier yourself) etc. I am now running on a fine working (so far at least) version2 WAP4410N.

Hurray for Cisco for their continued support!

New WAP4410N v2

After my latest replacement we noticed that my WAP4410N was a first generation version, having some problems with gigabit ethernet connections. After contacting Cisco we found out that the version should be replaced by a v2 instead.

Some difficulties with the replacement procedure (follow this, get a different ticket within the box, needing to get the arrangement with the carrier yourself) etc. I am now running on a fine working (so far at least) version2 WAP4410N.

Hurray for Cisco in their