Excellent suggestion – our situation isn’t quite like that though. We actually have lots of different sqlite databases, one per transfer job we do. And those guys are only accessed by one host at a time, the one currently working that account. For ‘the big database’ which is accessed by lots of hosts we use Postgres today.

I’m really interested to hear about your success with ZFS and cache devices. We have been looking at ZFS for quite some time, but haven’t dared to take the plunge since it’s fairly new. What performance boost did you get with that SSD hooked up? Sounds like a fantastic setup.

If that is the case, that might not be the best way to go with any kind of significant database writes happening. Multiple hosts locking/unlocking an sqlite database file constantly does not sound good.

I like sqlite and its performance is very nice for simpler things where one process on one machine is utilizing the sqlite database on its local disk, but if I were getting into using the database from multiple machines simultaneously I would start using a database server instead.

If you’ve switched from a mess of flat files to sqlite you’re already most of the way there, you’ve already SQL-ized your app. Pick your favorite SQL server and your journey towards the dark side will be complete. I know it’s a bit boring and common to just slap pgsql or mysql on a server and start using it but… it sounds like a very logical step up from sqlite with locking.

Badly-written overreaching SQL queries being executed often and lack of I/O performance are two real common things to watch out for when you go to use a regular old database server.

If your SQL load is significant and you are concerned about its performance there are plenty of ways to handle that… If your database fits in RAM you are probably good to go. If you’re up to the tens of gigs, RELENG_7 also has ZFS v13 which supports cache devices. I’m using it, I can tell you that just having a 32GB SATA SSD cache device assigned to your zpool is amazing. I am quite curious to test how mysqld databases cached on the SSD will perform.

Good point. While most companies like to downplay issues being open develops a sense of trust with your customers.

You’d be right not to see this article as a side by side review of VMware and Jails – it’s more a sharing of our experience: what we did, what problem we ran into and what worked for us.

I think chances are good we’ll give VMware ESXi a shot down the road. It’ll be interesting to see how that affects the IO throughput.