We have been busy working on open source libraries recently. A lot of great new stuff has come out of the last few weeks of work along with some rearrangements of the repository. To see what we've been up to just go to http://opensource.thinkrelevance.com.

As promised, here is the first admin script. It's not fancy, but it solves the disk space warning problem.

#!/bin/sh

MAILTO=you@yourdomain.com

df -h | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do
  consumed=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
  partition=$(echo $output | awk '{ print $2 }' )
  if [ $consumed -ge 95 ]; then
    echo "Running out of disk space on \"$partition [$consumed% consumed]\" on $(hostname)" |
        mail -s "[ALERT]: $(hostname) is almost out of disk space" $MAILTO
  fi
done

This will take a look at your disk usage and send you an email warning if you hit or cross your threshold. Obviously tailor to your liking and system specific implementations. Once you are satisfied, simply chmod +x the file, and add it to a cron task so that it runs as much as you need it to. The following example will run this script every 30 minutes.

*/30 * * * * cd /path/to/script && sh script

This script was created and tested on OpenBSD 4.1. I have also put this script up on the Relevance Open Source Repository. You can check it and eventually others at https://opensource.thinkrelevance.com/svn/admin_scripts

I am sick and f^$king tired of reading about application service provider's Ruby on Rails sites going down for periods of time. I understand that things happen and every now and again there will be a bit of downtime. The thing that pisses me off to no end is that the number one cause of downtime I have read about lately is that their mongrels started acting up after their slice ran out of disk space. Seriously? WTF? You want to take people's money but you don't have the wherewithal to monitor your server for disk usage? Let me guess, you probably run your server processes as root along with your database connection too. If you can't figure out how to administer a VPS, either hire someone to do it or find a company that has those services baked into their hosting plan and pay for it. If you are a hosting company that claims to offer full stack Ruby on Rails hosting and doesn't back it up with these types of services, you're dead to me. Grow up and figure it out. It is really hard for me not to name names right now, but you know who you are.

In an effort to bring everyone's level of competence up a bit I am going to start posting some admin scripts to help you get started. I will put the first script up tomorrow that will include simple things like disk space usage / cleanup / warning. If you are reading this and thinking to yourself, hey, I pay $300 a month for Ruby on Rails hosting and I never get any disk space usage warnings, or hey my app went down and it took me 20 minutes to figure out that the problem was disk space usage, it's time to find another hosting company. You can get that level of service at a $30 a month do it yourself VPS company. I will continue to follow this up with administration tips and tricks as well as rules to live by so that you can have a decent reference. There will always be different ways of doing the things I post, but at least it's better than crashing for no explained reason!

Sometime last fall I decided to switch car insurance companies because the one i was using was pretty costly. I made this decision because I really had not had any accidents and I was just paying a lot of money for nothing. I switched to ESurance after seeing many commercials about how they were so much cheaper I decided to give them a shot. They indeed were cheaper so I signed up for a policy and went on with my life. Ironically no more than three or four days after switching policies I was parked at Panera working from there for the morning when I came out to realize that someone had hit my car and took off. Some people just suck at life. I filed a claim that day, gave the information, and went about me business. Estimators came out to my place, got things prepared, and otherwise had things looking like they were all ready to get fixed up. Then I got the phone call. A claims investigator called me and asked me to repeat what happened over a recorded telephone conversation. After I was finished he informed me that my claim was going to the special investigations unit and would be put on hold until further notice. What, you ask? Why would they put my claim on hold for investigation? Simple. I had no way to prove that I was where I said I was on the day that I filed the claim. The police wouldn't do anything because it was a private parking lot, and I had no receipt from the Panera I was at that day, so according to ESurance, my claim wasn't valid and unless I could prove that I was at that Panera on that day when my car got hit, they were not going to cover my claim. I went back and forth for months with these people trying to get things patched up and did everything they asked. I let them call people that talked to me that day, I gave them every piece of info they asked for and still they denied my claim. The last kick in the teeth came when I called to cancel my policy just the other day after being took. They made me pay a cancellation fee. So I gave my money to a company to provide a service which they refused to provide, and when I wanted to stop the service they never provided they made me pay them more money. ESurance, you are dead to me. Hopefully I deter enough people with this post to cost them more money than they would have had to spend just fixing my car. This might be a far shot from reality, but at least I can let you all know NOT to switch to them ever.

I want to start doing some parallel programming blogging, and thought I would start with a simple example of some distributed code that computes PI. Contrary to the concerns of the state of Indiana circa 1897 the value of PI is not and never will be 3.2. Having said that let's take a look at how to do this using MPI Ruby.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

PI25DT = 3.141592653589793238462643 NINTERVALS = 10000 rank = MPI::Comm::WORLD.rank() size = MPI::Comm::WORLD.size() startwtime = MPI.wtime() h = 1.0 / NINTERVALS sum = 0.0 (rank + 1).step(NINTERVALS, size) do |i| x = h * (i - 0.5) sum += (4.0 / (1.0 + x**2)) end mypi = h * sum pi = MPI::Comm::WORLD.reduce(mypi, MPI::Op::SUM, 0) if rank == 0 then printf "pi is ~= %.16f, error = %.16f\n", pi, (pi - PI25DT).abs endwtime = MPI.wtime() puts "wallclock time = #{endwtime-startwtime}" end

Then run the script

% mpirun -np 2 mpi_ruby pi.rb

Where -np 2 is the number of processors you would like the script to use. After running you should get an output similar to this.

pi is ~= 3.1415926544231318, error = 0.0000000008333383
wallclock time = 0.0111241340637207

This is the first of many posts that will become increasingly more useful as time goes on.

I was reminiscing about some good old parallel code I found while cleaning up one of my computers the other day. It reminded me of how cool it was to get down with multiple processors when you need to do a lot of computing. I started looking to see if there were any Ruby implementations for MPI which was my favorite library to use. I stumbled across the mpi_ruby library but was sad to see that it seemed to have died out quite a while ago. The last development on it was done in '01 and consequently only worked with Ruby 1.6.4. I tried emailing the people that were in charge of the project according to the info on the site, but all the emails just bounced back.

I spent a little bit of time and made some rather trivial modifications to the code and got it working with Ruby 1.8.6. It should work with 1.8.x, but I haven't tried it with other versions yet. For all of you HPC and math geeks out there here's a good chance to help out. You can have all of the qualities of parallel goodness along with the elegance of ruby. Give it a shot. You can find the installation instructions over at http://opensource.thinkrelevance.com/wiki/mpi_ruby. The code itself can be found at https://opensource.thinkrelevance.com/svn/incubator/mpi_ruby.

If you were part of the original project it would be great to hear from you. I would love to keep things going and would like to see what other people are doing with Ruby and MPI.