Introducing YippieMove '09. Easy email transfers. Now open for all destinations.
Jan
28.
Comments Off
Comments
Category: Technology

We recently had to decide on a configuration format for one of our internal utilities. In this post I’ll talk a little about why we picked YAML as the format and the reasoning behind it.

WireLoad has a couple of servers, each running a number of different services. For a long time we had almost one backup script per service, all hand hacked in bash to fit the requirements of the application. This wasn’t great because it meant we repeated a lot of work. To make the situation a little more manageable we developed BackupWire, a simple backup utility in a single file with a minimal number of dependencies.

The design goals of BackupWire were,

  • Minimal footprint: BackupWire shouldn’t be much heavier than the bash scripts we already had. Why? Because if it was huge and difficult to deploy we might end up writing little bash scripts instead!
  • Minimal dependencies: BackupWire should not have many dependencies. This is for the same reason as in the previous point. BackupWire needs to be easy to install.
  • Readable configuration: One of the problems with bash scripts is that once they’re a little complicated it gets hard to see what’s happening. BackupWire’s real purpose is to alleviate that headache by distilling most backup jobs down to a few lines of configuration.

In order to make everything dead simple the configuration was stored in the BackupWire script itself. This would make it easier to relocate the script, and it would decrease the chance that a config file was not found due to things such as the cron environment being sparse. However, this design decision made it hard to update the script with new versions. In addition, the configuration format became a little cumbersome because it was just a set of Python class instantiations. Hence this feature went against the third design goal of BackupWire. The latest version now uses a configuration file instead.

Thinking that the world really doesn’t need another arbitrary configuration syntax, I wanted to pick a standardized configuration format. So I read up on Wikipedia’s entry on configuration files and found the top three contenders: Lua, XML and YAML.

Lua, being a programming language these days, looked like it would add too many dependencies to BackupWire. BackupWire is written in Python, which we already have on all servers, but Lua we don’t use for anything else so it would be a new requirement which would have to be installed on each server. Also, it just struck me as a little excessive to have a full blown second programming language as a configuration format unless the application was really complex.

The other problem with Lua was that googling Lua config tutorial didn’t really give that many good results, making me think that perhaps the focus of the language has shifted from configuration to something else over time.

XML was immediately off the table, perhaps obviously to some of our readers. Most importantly XML is not a very readable language with it’s abundance of symbols and markup. But also, it’s not very easy to write for the same reason. The people behind Django’s documentation put it best when they said, “Making humans edit XML is sadistic!”

YAML is readable and easy to write both. There is also a light weight Python module called PyYAML to read the format. Using YAML the new BackupWire configuration files are definitely to the point and concise without being complicated to edit. Here is an example of the new configuration format we developed in YAML syntax:

name:       "Sample Backup"         
to:         "/backup/"                   
frequency:  "daily"                 

tasks:
 - run: 
   command: 'df -h'
   log_output: True
 - archive:
   name: "etc.tbz"
   contents: ["/etc/", "/opt/etc/"]
 - archive:
   name: "tmp.tbz"
   contents: [ "/tmp/" ]
# Dump a database using a run task with 
# %(targetFolder)s to locate the destination.
 - run:
   command: 'mysqldump --quick --extended-insert 
     --compact --single-transaction 
     -u backup --databases sample 
     | bzip2 >%(targetFolder)s/mysql-sample.sql.bz2'
---

Not too bad as far as readability goes and all standardized YAML to spare the world from yet one more syntax.

Author: Tags:
Introducing YippieMove '09. Easy email transfers. Now open for all destinations.

What’s better than having a job that you love? How about not having to work, and still make money? We want to give you this opportunity. Ok, we might not be able to make you a millionaire, but if you are successful, you will be able to make decent money while literally doing nothing.

The way you would be able to do this is though our brand new affiliate program. It’s dead simple. You link to us on your website using your own custom link that we will provide you with. Then, every time a customer comes to us from your site and we make a sale, we will give you 10% of the sale. Are we great or what? Other affiliates programs, such as Amazon’s only gives you 4%.

Affiliate status page

The Affiliate Program status page.

Convinced yet? If so, just head over and sign up right away.

Author: Tags: ,
Introducing YippieMove '09. Easy email transfers. Now open for all destinations.
Jan
13.
Comments Off
Comments
Category: Uncategorized

Shortly after getting iReport running using the fix described in this article, we ran into a new problem — the Postgresql support. After some Googeling, it became clear that one had to add the the driver for Postgresql by hand from here. If not, you will encounter this error message:

Error: java.lang.ClassNotFoundException: org.postgresql.Driver

When adding the driver, beware of what version you’re selecting. After banging our heads in the wall for a while, we realized that we picked the wrong version. On Mac OS X 10.5.6 (with all latest updates installed) the version you’re looking for is the ‘JDBC3′ branch (assuming you’re using the java version that comes with OS X). Once we figured this out, the installation was easy. First, download this file. Then you need to move it into the ‘lib’ folder in iReport.

If you’re a console user like us, here are the steps:

cd iReport-3.0.0/lib
wget http://jdbc.postgresql.org/download/postgresql-8.3-604.jdbc3.jar

Easy as pie.

Author: Tags: , , ,
Introducing YippieMove '09. Easy email transfers. Now open for all destinations.
Jan
12.
Comments Off
Comments
Category: Uncategorized

If you’ve never heard of iReport and Jasper before, you really ought to take a look at it. It’s a really impressive suite of reporting tools that can generate reports from pretty much any data source out there.

Almost exactly a year ago, we wrote a similar article on how to fix the launcher in version 2.0.x. While I’m sure Jasper improved iReport a lot during this time, they also managed to break the launcher in new ways with version 3.0.0.

This is how you can get it working (assuming you’ve downloaded it):

tar xvfz iReport-3.0.0.tar.gz
cd iReport-3.0.0/bin
awk ‘{ sub(“r$”, “”); print }’ startup.sh > startup2.sh
chmod +x startup2.sh
./startup2.sh

The commands above fixes two problems with the launcher. First we convert the line feed from DOS format to UNIX format (the awk-part). The second problem was that the launcher was not executable. If you just make the original launcher executable (chmod +x startup.sh), you will end up with this:

-bash: ./startup.sh: /bin/sh^M: bad interpreter: No such file or directory

Good luck, and have fun generating all those new cool reports.

Author: Tags: , ,

© 2006-2009 WireLoad, LLC.
Logo photo by William Picard. Theme based on BlueMod © 2005 - 2009 FrederikM.de, based on blueblog_DE by Oliver Wunder.
Sitemap