Checking access to files on a server

August 2nd, 2009 by Ryan French

So recently I put some files up on a file server I rent with a couple of mates, mostly ones used in my CV so that prospective employees can check papers etc that I have written. In an attempt to try and figure out who has accessed these files I, with the help of my friend Paul wrote a smal script that uses the access log, grep and whois to figure out the domains that have accessed the file. To be honest its a small script, and if I had more experience with bash I probably could have written it myself. In fact, if Paul had wanted to, I know he could have written it no problems, but it was all experience. In case someone wants to use it, here it is.

#!/bin/bash

FILE=”/tmp/$(basename $0).$RANDOM.$$.txt”;

echo Searching for access to files with $1;

sudo cat /var/log/apache2/access.log | grep -i *PUT YOUR NAME HERE* | grep $1 > $FILE;

if [ -z "$2" ]
then
echo “no exclusions”;
else
echo “excluding files containing $2″;
cat $FILE | grep -v $2 > $FILE;
fi

cat $FILE | cut -d ‘ ‘ -f 1 | sort | uniq -c > $FILE;

for i in $(cat $FILE)
do
echo -ne “$i    -       “; whois $i -H | egrep ‘OrgName|descr’ | head -n 1 | cut -d ‘:’ -f 2;
done

To use it, just copy the script (replacing the part that says *PUT YOUR NAME HERE* with your username) into a file, set it as an executable using  chmod +x, then run it. The first argument is the string you are looking for access to, e.g. pdf will show all pdf’s, and there is a second optional string for if you want to exclude files with names containg a certain string.

Mono Migration – Stage 1

July 14th, 2009 by Ryan French

Well I promised I would be updating on how the effort was going to migrate our VB.Net/ASP/SQL Server 2000 product here at work to Mono, and as the first steps are now underway, I have a few things to talk about.

For starters we are looking at porting our code to C#, rather than VB.Net in Mono, and removing the ASP stuff from it etc. We are also looking at implementing localisation of the site. Originally when looking at this in Visual Studio we would have to look at purchasing a tool to pull out all the translatable strings in the program. Luckily, I found out mono-develop does this for free. Awesomeness.

At this stage the idea is that we are going to go for a proof of concept and move the code over to Mono as it is in it’s VB.Net/ASP form, then we will look at doing the actual port, as time is a little critical for us.

The first step is to setup a virtual machine for running Ubuntu (our OS of choice for the cloud) and install Mono/PostgreSQL etc on it. I choose to use VirtualBox for 2 reasons, 1) I hadnt used it before and it looked very interesting and 2) VMWare was causing my machine to go into an infinite “Cannot load Drivers” loop on the windows host, so it wasnt really an option at all.

Once I had completed this I downloaded and ran MoMA. This tool will scan the assemblies for your program and tell you what problems there are with running the code on Mono. We have a few problems on ours, but most of it is SQL calls that aren’t implemented, so I’ll find a way around this.

At the end of the MoMA tool there are links to a few sites that have definately helped me in moving the code over for the first test. Now there is just one last step before attempting to build the code in Mono and see how badly it breaks.

Clean the code. It is pointless trying to port everything over unless you need everything. I have currently removed about 50k of code from the site that is either no longer used and, thanks to my predescessor, has never been removed. I also found thousands of table entries and a good 30+ tables and another few hundred stored procedures that are either not used or obsolete. I’m still in the process of cleaning up before I try the build, but once that is done I shall let you know how things go.

My new big project (sadly it’s for work)

July 10th, 2009 by Ryan French

Here at my new job we develop in ASP/VB.Net, running off of an SQL Server 2000 (yes, it horrified me too when I found out). The company has been around since ~2000 and we have decided now that it is a time for a change. We are hoping in the next few months to be moving away from a single server based model that we have now, onto something a little more modern, a cloud.

At the same time as we do this, we are wanting to make the move to using OSS. This is going to be quite a move, with our development enivronment being firmly embedded in MS technologies at the moment. So we have a few steps that we are going to be going through. I am hoping over the next few months to share my experiences in moving our product from the MS environment to running on Ubuntu server using Mono (the C# variety) and Postgresql. It will be interesting, with the need for a demonstration of the new product by mid-August.

Also, on a side note, I have moved out of where I was staying temporarily into a new place, and once I have sorted out my internet issues I shall be looking at finally getting around to working on MPLS again.

Where I’ve been…

June 10th, 2009 by Ryan French

So it has been a while since my last post, and while thats nothing really unusual, I actually have things to post about, so I thought I better get round to doing them.

First off, my new job. As I said in my last post I had a pretty quick turn about from the first interview to starting work (phone call on the Tuesday concerning an interview, started work the following Monday). Things are going not too badly here, now that I’m settling into the role. The job is working for a company called TechCertain, who develop, maintain and operate a system which handles the paper work for insurance companies online (issue proposals and policies, automatic quoting/underwriting e.t.c.). My role is the companies Business Analyst. My main job is to work with the clients to determine what it is they need, and then work out how we are going to acheive this. I also do a lot of the companies research and development (more on that later), write the front end for the clients, and manage the development team (read: the other guy who works mostly on the backend haha). The language is VB .net and Microsofts SQL, which is actually pretty funny because the day I started this job I had never worked on or studied either of these, so I pretty much had to learn all of that on top of everything else that comes with a new job.

I mentioned before that the company had some R&D going on. Seems strange for a simple web application company like us to have R&D but it’s true. Currently we are looking into porting our code over to mono, and we are looking into running our service on a cloud rather than the server it is currently on, so at least that is keeping me busy.

In the next few weeks I am also moving into a house up here in Auckland, which will be great because I have been living in my friends parents garage since moving up here (it’s not nearly as bad as it sounds) but it’ll still be nice to have a real room e.t.c.

One last thing. I just wanted to say good work to the team that made Ubuntu Rescue Remix. A friend of mine has an 8 year old Acer Travelmate 240 sitting at home, and recently it started telling him that there was imminent drive failure, and refused to log into windows anymore. Using the rescue remix I was able to create a live cd, boot into the command line, plug in my flash drive, and rescue thousands of files off his laptop for him. Now all he has to do is go through and sort them all manuallly.

What a week…

April 20th, 2009 by Ryan French

Last week I received a phone call on Tuesday about a possible position at a software company in Auckland. The next day I travelled up for an interview and by the time I returned that night I had my first job out of university. I moved up yesterday and started my new job today, so it’s been a very hectic week as I’m sure you can imagine. I am now the Client Services Officer and Business Analyst for TechCertain, a very small company who sells ‘software as a service’ to a major insurance vendor here in New Zealand. I’m not entirely sure how much I’m allowed to say of what it is we actually do, so to stay on the safe side of things I’ll leave it up to you to search for more information on them if you are intereseted.

In terms of time spent doing my own things it means I am having to dedicate all my free time to trying to find a place to live, as at the moment I am staying with friends while looking. This basically means that I will have no time for anything until I find a place. Even once this is done I am not sure how much time I will be able to spend on personal projects, as I am not sure how much of my time this job will take.  In saying that if anyone is interested in having a look at the MPLS in FreeBSD code let me know, as I would greatly appreciate some help in not only trying to get the ball rolling on this project again, but also someone to help motivate me into finding the time to work on the project.

Just a Quick Update

March 18th, 2009 by Ryan French

This is just a quick post to let people know that I have joined LinkedIn. If anyone out there wants to then feel free to add me on there.

Also, the job front is looking good. I have a job lined up in Wellington that I shall hopefully hear back on in the next few weeks. and once everything is sorted I will be able to start working on MPLS again.

Thats all for now.

Statistics overload

January 18th, 2009 by Ryan French

So this morning I installed StatPress-Reloaded on my private blog, and within a few hours I was told about Google Analytics. Seeing as I had noticed that this was already installed on my FreeBSD blog, so I have now removed StatPress, and have signed up for GA on both of my blogs. I’m hoping this will give me an idea of who is reading my blog, and checking on my insights into the world.

In other news I shall be posting an update in a few days about where I am at the moment in terms of my life, and why I havent been posting that much since university finished.

The end of an era…

October 28th, 2008 by Ryan French

Last Friday I handed in what was hopefully the last piece of assessment that I will ever have to do for university. It’s not to say that I wouldnt like to go back next year and look at doing a Masters or a PhD, but I feel after 5 years at university it’s about time to move on and start my career. Tomorrow I will be moving back to my home town, where I will be staying with my parents while searching for that elusive first computer science job. I have already had a few places that have been interested, but none of them have worked out for some reason or another, the main one being that I was unable to start until this week, so hopefully now it shouldnt take too long to find one.

So what does this mean for my MPLS project with FreeBSD? Basically I will not have internet for a while, at least until I can get a connection setup at my parents house, but once this is done I will continue to work on MPLS. At this stage I am going to start looking at how to implement the routing table. I was thinking of using a hash map of some form, due to the fact that the MPLS labels means that this will probably be the most efficient way of storing them. Given that I will have a lot of free time on my hands now I will hopefully be able to dedicate a lot more time to this than I have this past year.

The final countdown and what I’m working on

September 24th, 2008 by Ryan French

So I figured that even though the Summer of Code has finished I am still going to keep posting here. Mostly I will be posting about the progress of MPLS in FreeBSD, but I will also write a few other posts about other interesting, and not so interesting depending on your point of view, things that I am up to.

As I am writing this I currently have just under 4 weeks till I have finished university, hopefully for good. I have been going through the process of trying to find a job that I can step into once I complete my degree but unfortunately the graduate IT jobs in New Zealand are few and far between, so much so that I have started applying for jobs overseas.

The biggest problem with this time of year is the amount of work that is due in a matter of a few weeks. I have been working a lot on getting my assignments done. The majority of them are reasonably standard and uninteresting, like the work I have been doing in XNA using Vertex and Pixel Shaders, but some of it is a lot more interesting. The biggest project I have at the moment, besides my Implementation of MPLS in FreeBSD which is due in 5 weeks, is a Distributed File System using FUSE. This file system, if successful, may be looked at for using on the University of Waikato’s cluster computer.

Hopefully within the next week I will have some serious time to go over MPLS. I am up to the stage now of testing the code for receiving and sending of packets, and hopefully that works pretty quickly so I can start on routing table implementation and then maybe even move onto LDP, but I very much doubt I will have the time before the dissertation on the project is due (approximately October 24th).

Final Summer of Code Report

August 18th, 2008 by Ryan French

Summer of Code has now officially finished. Well, at least the coding part of it has. This is my final GSoC report, but hopefully will not be my last post here. As it stands at the moment, I have not managed to get sending and receiving working, but I am very close to it. I honestly am unsure as to why it is not working,  but I believe it may be to do with how I have tied my code into the kernel, and I dont think it is getting initialized or compiled with the right preprocessor defines or something. However, I will still be working on getting this going, as not only is it my dissertation for university this year to do this, but it has become somewhat of a personal challenge to prove to myself that I can do it.

With that in mind I will keep on working on this at every oppurtunity I get, and will definately have at least sending/receiving working by October, which is when the dissertation is due. So here is a run down of what I have accomplished so far for the GSoC, and what I plan to implement in the future.

Accomplished:

MPLS-needle – this is a small tool I built for testing of MPLS. It takes a configuration file and uses that to define and build an MPLS packet, which is then inserted onto the networking stack using a BPF device.

Basic port of OpenBSD/Ayame code – I have been working solidly for the last few months on porting the OpenBSD code, which is based on Ayame, to FreeBSD. This was a much bigger task than I orginally intended. As I have had no previous experience with kernel coding this has involved a huge learning curve, not only in just kernel coding itself, but in how the FreeBSD networking code works, and how it differs to OpenBSD (by the way, it is a lot more than I thought it would be). The code has been ported over and changed accordingly, but still does not work. However, the kernel does compile and run, it just appears that the MPLS code does not run when a packet is received. I think this is because of incorrect #defines, as I’m not entirely sure where I need to define these (in the Makefile or somewhere in my code). I will continue to work on this and attempt to get it working asap.

Future Plans:

Once basic sending/receiving is acheived, I will be attempting to implement the following features. I believe this will be a lot easier as I will be working mostly on my own code, independant of the kernel code itself, but I have been known to be wrong before.

Routing Table Manipulation – create a way to store, insert, edit and delete MPLS routes in a table.

Static Label Switching – being able to receive a packet, swap the label, and send it out using a statically defined route in the routing table.

LDP Implementation – implement the LDP daemon for autmatic label distribution and routing table building.

Tunnels – implement MPLS L2 and L3 tunnels with first virtual interfaces, and then real interfaces.

At least thats the plan for now, and I’m really hoping that it becomes a reality.

I would like to take this chance to thank everyone that has helped me through this transition into kernel hacking and opensource hacking. It has been a lot of fun, with a lot of frustration, but very much worth it. If anyone would like to take a look at my work and provide some feedback, whether it be stuff I need to change, or help on completing my goals, it would be welcomed and very much appreciated. The code can be viewed here on Perforce.

Thank you

-Ryan French.