Author Archives: Remko Lodder

Tarsnap backup script

Tarsnap

Tarsnap is an advanced online-backup facility, entirely encrypted. The only copy of the keys used to encrypt and decrypt archives are in your own possession, so things that should be kept safe, are (in the current form) safe. Tarsnap makes extensive use of the Amazon EC2 and Amazon S3 for storage.

Tarsnap is originally written by the FreeBSD Security Officer Emiritus’ Colin Percival, on topics that he periodic gives talks about at various conferences. If you are able, you should seriously attend one of those talks

Script

Recently I rewrote a tarsnap backup script from Tim Bishop http://www.bishnet.net/tim/blog/2009/01/28/automating-tarsnap-backups/ to a more suitable script for us.

Tim backups his data via Tarsnap, all via the same way. That works well for him, but for our hosting company that is more tricky. We do not want to keep large amounts of data for our customers (which tend to change rapidly, for example emails that come in and go out and get deleted etc.). Instead we want to keep the minimal amount of data for these customers, and we want to offer them more advanced backup strategies for which we calculate an increased price (the minimal backup strategy is free).

After collaborating, we decided that next to the free strategy, we would like to offer a medium-term backup strategy, and a maximum-term backup strategy, where the former is a month of backups (7 weekdays, 4 weeks), and the latter is three months of backups (7 weekdays, 4 weeks, 3 months), so that going back in time is doable. If customers want to have a customized strategy, that would ofcourse be possible if we add that to the script.

Since we are keen on open source we would like to offer you the option to download the script, and if possible even enhance it more so that we can all benefit from it. Do note that we didn’t try to complicate the script, but instead keeping it as simple as possible. That means that we add more lines then likely needed, but it is very readable. One comment from Colin we got so far is that Tarsnap is capable of removing more files in one go (tarsnap -d -f -f ) and that is not yet implemented in the script. We will consider doing so.. ofcourse :-)

The script can be found here, tarsnap.script.

20131013
Updated the script with the update from Tim, this had been tested and works fine for us so far. Thanks Tim ! I shamelessly used the code in our code ;-)

One less hat :-)

So. Today we can congratulate George Neville-Neil as the new FreeBSD Security Team Secretary. It seems that I had been doing the job for around 5 years and 8 months (although not the entire time officially nor documented); which is a very long time. I decided to start reducing on the amount of hats that I carry so that I can focus more on the things that I want to focus on within FreeBSD.

Slowly but surely I am returning to my roots:

- Maintain the nl_NL tree

- Keep VuXML as up to date as possible

- Commit low hanging fruit from src/ so that the developers can focus on their development instead of being distracted by easier things. I will also try and merge for example usb/ related things from hps@.

That said: I will remain a doc committer, src committer and member of secteam .. please applaud George in his new task, it’s a thankless job and you really need to keep your head together :-)

Can the following be done with Postfix and LDAP?

Dear readers of my blog,

I have a “simple” question for you. I Would like to do the following, can someone that reads this and has suggestions and ideas respond to me at [email protected]

I have three various mailrelays, I would like to finish off mail that shouldn’t get in at the border relays. For this I have setup LDAP so that all three relays can query this LDAP Server. To fill the LDAP I use the Virtualmin application to make this as automatic as possible.

Currently the Virtual-addresses and Aliases are all in LDAP, as well as the useraccounts that receive email. No specific tag is added for local users.

I would like to have the relays do the following:

- Receive mail from XXX
- do RBL checks
- do postscreen checks and the like
- resolve the destination address (expand alias or virtual account)
if the resolved destination address lives outside of my domain (mailforwarding accounts) i would like to deliver it there immediately.
- check whether the resolved destination address is listed as local user and send it to the internal mailserver
(The internal mailserver will receive mail for local-user and only has to do spam checks for this user, no need to expand aliases etc).

Suggestions are welcome :))

DSPAM

Since recent (with the very great help of Ion-Mihai Tetu, a fellow FreeBSD committer and developer for dspam) we (JR-Hosting) are running our anti-spam infrastructure on DSPAM. We stopped using SpamAssassin after some testing and resolving problems. The interesting fact is that we share most directories through nullfs so that both the webjail and the mailjail share data and our users are able to modify settings, see their stats etc. Very great and after overcoming our issues (local delivery was not OK in the beginning and the webjail was not able to properly use the MySQL database backend at first, which was odd because the main system WAS looking into it and the webjail wasn’t), it works just fine. Ofcourse it is still learning but it seems that it finds spam efficiently and quick, and it’s footprint is much much lower then SpamAssassin was. I might want to figure out how to run the daemonized version as per advise of Ion-Mihai, till then it works as a deliveryagent.

I am writing a ‘hosting environment howto’ (or something that will largely look like that) in which I will write about the setup as well.

FIXED: FreeBSD Jails PHP dirname WordPress

Dear Reader,

I had fixed the issue. Instead of using nullfs to get access to the /usr/home directories, I am using unionfs, which basically does the same for my goals (unless someone corrects me in misunderstanding things) and this does not seem to generate the same issues. Various sites are now running happily behind the WWW Jail. Time to finish my document on how I did setup the entire beast.

Thanks all for listening, helping, and giving tips (Alexander and Miroslav!)

HELP: FreeBSD Jails PHP dirname WordPress

So, I am still building up my jail structure and the last few evenings I was testing the FreeBSD jail wrt. PHP, Apache22-mpm-itk and wordpress.

Things started to break when I redirected external traffic to the jail. It seemed that require_once(dirname(dirname(__FILE__))) . ‘/wp-load.php’; does not work from within the jail.

I decided to do a little test and testing reveals that in a stand alone configuration the dirnames behave exactly the same, in both the host and the jail. Printing the directive within WordPress (when loading the admin pages f.ex.) reveals a ‘.’ instead of the ‘/path’ . It is resolvable by adding a ‘.’ to the directive so that wp-admin/admin.php loads the ../wp-load.php file instead of ‘/path/to/wordpress/wp-load.php’. Though this sounds very sily todo.

Did someone else encounter this? I Do not want to change enforcement of the statfs to some other value since the defaults should be good enough (given the testsript).

Relevant details: the /usr/home where the public_html files live, are nullfs rw mounted from the host and are available in the jail. The jail does username/group lookups through Ldap, and can see the various users. Apache had been build with the ITK patches so that every host runs under his/her own user. I do not see obvious differences between the regular host and the jail, the only real difference is the internal/external addresses used in the vhost configuration, but that is kinda obvious to me.

Let me know :-)

FreeBSD Activity++

Lately I have become more active then the last year in total for FreeBSD. I committed several enhancements that were in my queue already for a long time, but finally came into the FreeBSD tree. Some too late for 8.3 which is upcoming and some not.

I also understand merging much better then before, taking the time for a commit and making a few mistakes really helps, and ofcourse the community is not too shy to mention my faults :-)

One of the things that got committed is the force setting of carp, whether it’s a master or backup node (make sure pre-empt is disabled if you do this, else it will just rollback within seconds :-)), which eases maintenance for example. One of the other things is that I am using an extensive network of local jails now that service my needs, most seperated items. I also started writing an Howto (or bsdmag article if there is interest) to demonstrate and tell how we setup most items. One of the things that is still causing me headaches is that we have nullfs rw mounts of /home to the jails because mailservers need to write, ftp servers need to write etc. But if we were sharing these information sources via NFS we would have had the same challenges :-)

I feel good in taking the time for FreeBSD again, and I would like to hear recommendations on what targets I can persue in FreeBSD (low hanging fruit is good enough for the time being) and additional things, also please comment on the nullfs mounts (rw,nosuid) to enhance my security level and which makes my article even better :)

Family news…

Dear all,

It is with very great pleasure that I would like to tell you, that we (Denise, Luca and myself) are expecting our second child. Currently we are around 12 weeks and everything is looking good.

We saw the first images of our soon kid and new FreeBSD Hacker? ;-) He or she is looking beautiful already. We do not yet know the gender, but we are expecting the kid around the beginning of September.

Luca is also happy with these developments, so everything is in the works (actively as we speak) to move his room to the upperfloor, and we are going to prepare his old room for the baby.

Leave a message so that I can read them later on (I would like that): do realise that it’s moderated and that it might take a little before I can acknowledge your message :-)

FreeBSD: jails, ezjail, pfSense

During the last couple of days I am intensively using ezjail to administer several jails on my machines. They are currently IPv6 only (internet-facing) and are used to build pfsense images to test locally (still setting this up, need to cross compile to i386 from amd64), offer a testjail to a collegue to work together on a Opsview implementation on FreeBSD, whether or not we are going to succeed in that, and I just installed a test environment for my webservices. They are all contained in their own little box, having IPv4 connectivity outgoing through NAT, and native IPv6 connectivity from my “Vendor” on an extra subnet that I obtained.

I like this, so I am probably going to setup some more services here and there to perform some magic for me that might need external access. I will also tie them together with LDAP and the like so that it’s an uniform base. At the moment I do not have additional ideas about moving production services towards jails as well though.

Thanks to FreeBSD this all is damned easy. You should try it, or poke me in case you want to know more! :)

Happy New Year – 2011

In just one minute it will be 2011 (hey, scheduling things is fun, this gives you something to read while I am jumping around, celebrating with Luca, Denise, Rik and Larissa the coming of the new year, and perhaps drink a beer, or more but enough about that.

It’s my tradition to have a new years post, and this year I decided to schedule it for the first year. I wont be able to write a post before tommorrow or perhaps even later so “Sad but true”.

The last year saw a lot of sad things, sad changes and sad news, deaths and so on. Please take a minute to remember the persons you lost this year, think about the bad and the good things you shared. Cheerish those good moments, you can be upset about the negative things, but it will only make you more grumpy, which isn’t worth it. Life is too short!

OK So we considered the negative things of the last year, but ending the year with a negative thing is not right, right? So also take a minute to remember the positive things, positive changes and positive news, the birth’s you saw this year, the news that people are pregnant and are expecting a child, the new job, consider it and remember it.

From my position I would like to offer you my very best wishes for the upcoming year, I hope that you will see the positive things of life, respect eachother, and that you are healthy and can remain healthy (and your relatives).

Ofcourse my new years post wouldn’t be the same without mentioning my beloved FreeBSD. The last year we saw a few new releases, saw a lot of hard work, had to deal with the economic crises and loads of more things. This year we will get generous donations from you… right? So that we can build even more funky stuff, and keep the best operating system!

Welcome.. 2011!

Remko Lodder

Now that I started working for Ziggo in the Hague time is flying even quicker then it normally does. The traveltime increased, and traffic between Rotterdam and The Hague is dense. BUT, I have a bunch of nice collegue’s, nice working environment and a nice assignment so far. There are not many things to complain about (Darn, I am a dutchy, I neeeedddd to complain!) ;-)

It remains difficult to combine my opensource activities with work, but I am trying. I try to do as much security work as I can because that is the most important thing I can do for FreeBSD, but I would like to be a little more involved :-) .

I recently got notified that the 11th of December will see the light of another NLLGG meeting in Utrecht! Be there! I will try to be there as well, not presenting for the first time (well last time I was unable to present at all due to private circumstances, but I wrote a presentation then). I am too busy :-( .

Well, more updates to follow, “Massohl” :-)

Evilcoder

FreeBSD’s Security Officer Colin Percival send an update (2 july) to the various FreeBSD Mailinglists recently, stating the EoL of FreeBSD 7.2

For convience sake, I pasted it below:

Hello Everyone,

The branches supported by the FreeBSD Security Officer have been updated
to reflect the EoL (end-of-life) of FreeBSD 7.2. The new list is below
and at .

Users of FreeBSD 7.2 are advised to upgrade promptly to a newer release,
either by downloading an updated source tree and building updates manually,
or (for i386 and amd64 systems) using the FreeBSD Update utility as
described in the relevant release announcement.

[Excerpt from http://security.freebsd.org/ follows]

Supported FreeBSD Releases

The FreeBSD Security Officer provides security advisories for
several branches of FreeBSD development. These are the -STABLE
Branches and the Security Branches. (Advisories are not issued for
the -CURRENT Branch.)

* The -STABLE branch tags have names like RELENG_7. The
corresponding builds have names like FreeBSD 7.0-STABLE.

* Each FreeBSD Release has an associated Security Branch. The
Security Branch tags have names like RELENG_7_0. The
corresponding builds have names like FreeBSD 7.0-RELEASE-p1.

Isses affecting the FreeBSD Ports Collection are covered in the
FreeBSD VuXML document.

Each branch is supported by the Security Officer for a limited
time only, and is designated as one of `Early adopter’, `Normal’,
or `Extended’. The designation is used as a guideline for
determining the lifetime of the branch as follows.

Early adopter
Releases which are published from the -CURRENT branch will be
supported by the Security Officer for a minimum of 6 months
after the release.

Normal
Releases which are published from a -STABLE branch will be
supported by the Security Officer for a minimum of 12 months
after the release, and for sufficient additional time (if
needed) to ensure that there is a newer release for at least
3 months before the older Normal release expires.

Extended
Selected releases (normally every second release plus the last
release from each -STABLE branch) will be supported by the
Security Officer for a minimum of 24 months after the release,
and for sufficient additional time (if needed) to ensure that
there is a newer Extended release for at least 3 months before
the older Extended release expires.

The current designation and estimated lifetimes of the currently
supported branches are given below. The Estimated EoL (end-of-life)
column gives the earliest date on which that branch is likely to be
dropped. Please note that these dates may be extended into the
future, but only extenuating circumstances would lead to a branch’s
support being dropped earlier than the date listed.

+——————————————————————–+
| Branch | Release | Type | Release date | Estimated EoL |
|———–+———–+——–+—————–+—————–|
|RELENG_6 |n/a |n/a |n/a |November 30, 2010|
|———–+———–+——–+—————–+—————–|
|RELENG_6_4 |6.4-RELEASE|Extended|November 28, 2008|November 30, 2010|
|———–+———–+——–+—————–+—————–|
|RELENG_7 |n/a |n/a |n/a |last release + 2y|
|———–+———–+——–+—————–+—————–|
|RELENG_7_1 |7.1-RELEASE|Extended|January 4, 2009 |January 31, 2011 |
|———–+———–+——–+—————–+—————–|
|RELENG_7_3 |7.3-RELEASE|Extended|March 23, 2010 |March 31, 2012 |
|———–+———–+——–+—————–+—————–|
|RELENG_8 |n/a |n/a |n/a |last release + 2y|
|———–+———–+——–+—————–+—————–|
|RELENG_8_0 |8.0-RELEASE|Normal |November 25, 2009|November 30, 2010|
|———–+———–+——–+—————–+—————–|
|RELENG_8_1 |8.1-RELEASE|Extended|not yet |release + 2 years|
+——————————————————————–+

[End excerpt]