Sunday, August 19th, 2007

Building a modern IT infrastructure for a small company (10 clients) with a sub-$3,000 budget

Introduction
No way! That’s impossible.” Well, actually it’s not. Using Open Source technology, it’s actually possible to create a competitive IT infrastructure at very low costs. Not only does Open Source software enable you to create more customized solutions to better fit your needs, but it also means that you can spend your budget on hardware - not software.

Last month I was asked by a company to figure out how to ‘modernize‘ their IT infrastructure with a minimal (almost non-existing) budget. After plenty of thinking and research, I came to realize that the only way to do this was to use some kind of thin-client solution. The solution that I found to fit my needs the best was Linux Terminal Server Project (LTSP). LTSP utilizes network boot (we will use PXE) to boot the clients directly from the server. Therefore, we can use obsolete clients without hard drives (to reduce the noise) as thin clients. The only thing we need on all the clients is a fast network adapter with PXE-support.

Some of the widest adoption of LTSP has been within K12 the education field. Since many educational institutions are working with very tight budgets, LTSP has a strong advantage. It is a way to save costs without having to compromise too much on usability. Edubuntu is a Linux distribution that targets educational institution. What makes Edubuntu very interesting is that it comes out-of-box with LTSP support, which enables system administrators with limited knowledge of Linux to get a thin-client setup running with little effort. Since Edubuntu is closely related to the regular Ubuntu, it’s not very hard to get Ubuntu up and running as an LTSP server.

Infrastructure

This layout is quite typical for a LTSP setup. You might also want to add a couple of network printers.

The Hardware
With a very limited budget, I realized that a thin-client solution would be the most realistic approach. As the name implies, the clients are thin and most of the load will be on the server. Therefore we will spend most of the money on a solid server. After a good amount of research to find a cheap, yet powerful and expandable server, I found the HP ProLiant ML115. The server comes with a 64 bit 2.2GHz Dual Core AMD Opteron CPU, which will serve our needs well. However, it only comes with 512Mb of RAM, which is insufficient for the number of users we intend to have. Therefore we’ll need to purchase some additional RAM. The RAM consumption estimates varies across different LTSP projects, ranging from 256Mb + 32Mb per client to 1024Mb + 64Mb per client. However, since I’d rather be on the safe side, I’d recommend that you purchase another 2Gb of RAM (total 2.5Gb) and put in the server (1024Mb + 150Mb per client).

The next thing we need to add is a Gigabit switch to reduce the possibility of having the network as a possible bottleneck (note that I’m not sure if a 10/100 Mbit network would actually create a bottleneck in this setup, but I rather be safe than sorry). Since Gigabit is cheap today, going Gigabit all the way seems like a reasonable move. Therefore, I’ve budgeted for 10 Gigabit network adapters (with PXE support) and a 16 ports Gigabit switch (the HP server comes with Gigabit network adapter).

Now we need somewhere to store the users’ data with high security and performance. Since we’re on a limited budget, we will use a software RAID solution rather than a high-end hardware RAID solution. A RAID-5 setup on three SATA disks using Linux’s software RAID is probably the cheapest and most reliable for the price. This will allow us to increase performance while we also gain protection against loss of any one drive without loss of any data. Moreover, we also use a separate OS disk to reduce the I/O load.

Because everything is both stored and running on the server, it’s crucial that we protect the equipment from power failures and spikes (the server by itself is a single point of failure). Therefore I’ve added an UPS to the budget (1100 VA) that will not only protect our hardware but also reduce server downtime.

The last thing we need to add is the clients. It’s quite likely that you have a many retired PCs (preferably P2>). If not, you’re likely to find a many of these computers for sale (or for free) in your local classified adds section. If Craigslist is available where you live, this is a great source to find this kind of hardware. Even though many of you will get these computers for free, I’ve budgeted $50 per client.

Budget

Here’s the complete budget for the project. As you can see, the total is just below $3,000

The Software
I’ve divided this section into two parts: Server and Desktop. Although all software will be running on the server, the users will only see the software on the Desktop side, which is why I have separated them.

Server
As already touched upon, the server will be running Linux. To be more precise, I’ve decided to use Ubuntu Server 6.06.1 LTS Server (64-bit). The reason why I didn’t chose to use the most recent version (7.04) is to minimize the administrative effort. Since 6.06 is the Long Time Support (LTS) version, the Ubuntu team will supply this version with security patches and updates for a longer period than the 7.04 release (6.06 LTS Server will be supported until 2011).

Installing Ubuntu is very straight forward. The only thing to consider in this setup is to make sure we install the system on the hard drive that came with the server and not the RAID-5 array.

Configuring the RAID 5 can be done though a couple of different ways. You can either do this during the installation (with Ubuntu’s graphical utility), or wait and set it up afterward in the console (see the Software RAID HOWTO for details). After setting up the RAID, go ahead and mount it to /raid or something similar.

Now it’s time to set up LTSP, which is the foundation of our cost-saving solution. There are a couple of different ways to do this, but the one I found to be most useful was to follow a guide from Novell (strangely enough) available here. You might also want to take a look at Ubunut’s Thin-client documentation.

Before you start the installation, go ahead and symlink /opt, which is where LTSP will be installed, to within you RAID array (ln -s /opt /raid/opt). This will install all the packages on the RAID array instead of the system disk. Finally, what you will want to do is to add a test-user (or a real user - it’s your call). This is done by simply using the user-management tool in the Ubuntu. Note that you probably want to have the home-dir of the users on the RAID array. To achieve this, you can either symlink the entire /home to /raid/home or just set the home-dir to /raid/home/user in the user-creation process.

Desktop
The LTSP setup comes with the most common software used in a corporate environment. This includes:
* FireFox - A great web browser
* Open Office - A Microsoft Office replacement
* Evolution - A Microsoft Exchange replacement
* The Gimp - An Adobe Photoshop replacement (arguably less powerful)
Plus a long line of other applications such as PDF-viewer etc.

You might also want to install is Wine. Wine is a Windows emulator which will run (legally) without any Windows license. Although it does not run all Windows software, many applications work very well.

If you have needs outside of the applications listed above, there’s more available. Any software that runs in a Linux environment (pretty much) will run on these thin clients. Although I haven’t tested it, you should be able to run a fully emulated Windows environment using a virtualization software such as VMWare Workstation or CPU/RAM intense softwares such as MatLab or CAD software.

Screenshot of LTSP in Action Running Open Office in Swedish

Here’s a screenshot of LTSP in action (from the client-side) running Open Office and the Gimp in Swedish

The Clients
The last part is to get the clients to actually boot over the network. If you decide to use a different network card than the one specified in the budget above, make sure it supports PXE booting. Many budget NICs don’t support this feature. There are also other ways to boot but I’m not going to cover that in this article (such as floppy, CD etc.).

The Pros and The Cons
Although this approach is a very good way to create an updated desktop environment while at the same time minimizing the administrator’s job, it does come with some drawbacks. Unfortunately many companies today are stuck with custom software that only runs on Windows. Although Wine offers a great emulation software, you might be forced to purchase a license of VMWare Workstation (and Windows) to run some specialized applications. If you’re lucky, your custom software was written in Java, and it will actually run on Linux as well.

Another thing to consider is the transition from the old environment (quite likely from Windows) to this new environment. Although the transition is likely to be smooth for a crew of young people, members of the older generation (40+) are likely to require some training before being able to use the system fully. Both Open Office and Firefox work very similar to their Windows-counterparts, but Evolution is slightly different.

Another pro is the lack of viruses on Linux. Since we’ve left Windows behind, the likeliness of being infected by a virus is almost zero.

In summary, utilizing a solution such as this might or might not suite your needs. If you do have the flexibility to use Open Source software in your organization and are able to run your customized software either emulated or use a web-based interface, this is a great way to reduce costs from the IT budget and spend them in a way that benefits the company better.

Update: I’ve now actually deployed this solution at a company. For information about how it went, go ahead and read “Deploying the sub-$3,000 IT-infrastructure.”

Friday, August 10th, 2007

Cleversafe Presents Alternative to Google’s Triple Storage Storage Philosophy at LinuxWorld

At least in my personal opinion, one of the strongest trends seen at the LinuxWorld expo in San Francisco over the last years has been virtualization. This year many exhibitors had taken the next step and were actually using VMware products on their exhibit computers to simulate a number of servers in a network. For instance, Hyperic demoed their systems management software with a set of virtual servers.

Whenever virtualization comes up, the idea of grid computing isn’t far away as enterprises wish to maximize server utilization by turning their data centers into grids that each deliver the ‘services’ of CPU, memory, ports and so on. But in this brave new world of virtual machines and grid processing there is an element missing. If you’re moving your computing over to a grid computing model, why is there no corresponding grid storage model?

The commercial open source startup Cleversafe has that corresponding model. By employing a mathematical algorithm known as an Information Dispersal Algorithm, found in the cryptographic field of research, Cleversafe separates data into slices that can be distributed to different servers, even across the world. But it’s much more than just slicing and dicing: the algorithm adds redundancy and security as it goes about its task. When the algorithm is done, each individual slice is useless in isolation, and yet not all slices are needed to reconstruct the original data. In other words, your data is safer both in terms of security and in terms of reliability.

Cleversafe is not the first entity to come up with such a scheme. The idea of an Information Dispersal Algorithm is known from Adi Shamir’s paper ‘How to Share a Secret’ and other publications. When we met up with Cleversafe’s Chairman and CTO Chris Gladwin at LinuxWorld, he mentioned that the Information Dispersal Algorithm had been used in many applications before - even to store launch codes for nuclear weapons securely.

The scheme is different from a simple parity scheme in that you can configure how many redundant pieces you want. With parity as found in common RAID setups, you can lose any one storage unit in the set. With an Information Dispersal Algorithm, you can make your system resistant to failure or corruption of any one, two or indeed any number of units in the set. If there’s a strike in your data center in Texas, and your German data center is on fire, your data will still be fully accessible through the remaining servers provided you began with a sufficient number of servers. And as opposed to the brute force solution of multiple mirrors of the data, the dispersal algorithm has a much smaller overhead.

Google is a well known proponent of the brute force solution: the Google File System implementation suggests that the best method to keep your data continuously available is to keep three copies of it at all times. Cleversafe is a smarter system. If you have 16 slice servers (known as pillars in Cleversafe terminology) with a redundancy of 4 slices (known as the threshold) you can lose up to four servers simultaneously and still retain your data. At the same time the total overhead in storage space is only 4/12 - 33% of the space. The advantage as compared to Google’s three copies method is clear: with three copies you only protect yourself against the failure of any two servers and yet you pay a much greater price with a total of 200% storage and bandwidth overhead. And that’s not all. While you’re storing two additional copies of your data, you have effectively tripled the risk of that data being stolen. When a careless system administrator forgets the backup tapes in his car over night and the car gets stolen, all those credit card numbers or what have you will be out in the wild, even that only one out of three locations was compromised. In our Cleversafe example, 12 separate servers would have to simultaneously be compromised - quite unlikely by comparison.

Cleversafe is not alone and there are other actors on the software market such as the PASIS system. PASIS’ home page describes functionality very similar to Cleversafe’s: “PASIS is a survivable storage system. Survivable storage systems can guarantee the confidentiality, integrity, and availability of stored data even when some storage nodes fail or are compromised by an intruder.” None the less, Cleversafe appears to be a step ahead of its competitors at this time and is poised to be the first to deliver grid storage to a wider market.

While the Cleversafe software is developed as open source through the Cleversafe Open Source Community at cleversafe.org, there is a commercial company behind Cleversafe: Cleversafe, Inc. Cleversafe, Inc. plans to generate revenue by offering a storage grid for rent based on the Cleversafe technology. “The market for a more secure, more cost effective storage solution is enormous,” says Jon Zakin, CEO of Cleversafe in a press release issued in May.

The Cleversafe project is available as Open Source under the GPL 2.0 License. The version online is apparently an early alpha version and is not ready for production use. According to Mr. Gladwin, there will most likely be a new version within a month, and sometime in the beginning of the next year Cleversafe may be ready for production use. In the meantime, you can download the current alpha version of the software at the Cleversafe Open Source Website. You can read more about the algorithm at Cleversafe.org’s wiki, and there’s also a flash video describing the idea available.

Saturday, July 28th, 2007

Yahoo!’s YSlow Helps You Optimize Your Site

Yahoo! recently released a new Firefox extension called YSlow. This article describes how to get started and what we did at Playing With Wire to get our front page to load in almost half the time.

YSlow is a handy little tool for analyzing the performance of your websites. It will give you vital statistics and grade your site on 13 performance points with helpful hints for what you may be able to do to improve the loading speed of your pages.

Installation

YSlow is actually a plugin to a plugin in Firefox. To use it, you need Firefox and the Firebug web developer plugin. Both are easily installable using links from YSlow’s homepage however.

Once you’ve restarted Firefox with the new plugins installed, all you have to do is to activate Firebug for a particular site and you’re ready to go. Normally, this means surfing to the site and then revealing Firebug from its icon in the Firefox status bar. Just click the Firebug icon and it will reveal it’s main view. In it, it will most likely say that Firebug is disabled. Just click ‘Enable Firebug for this web site’ and you’re ready to start dissecting it’s performance with Yahoo!’s YSlow.

This is what Firebug looks like.
Firebug revealed.

Taking your site apart

Once you have Firebug enabled, switch to the ‘Performance’ tab and you’ll get a grade on your website’s loading performance. The grade breaks down into several subcomponents where each one corresponds to a point in Yahoo!’s Thirteen Simple Rules for Speeding Up Your Web site. The grading is fairly arbitrary and should be taken with a grain of salt. For example, if you have 35 downloads for your page and just four of them don’t have an Expires header, YSlow will give you a harsh F in that category.

YSlow gives Playing With Wire an F for Expires headers.
Hello, is this Google? YSlow is giving me an F in Expires headers. Could you reconfigure your ad servers for me?

None the less, the sub-points of the grade-sheet are great hints for what you can do with your site. While you’ll probably be forced to ignore the ‘grades’ if you have externally sourced ad units like we do, you can still work your way through the list and fix everything that you do have control over. This is what we did with Playing With Wire and astonishingly enough we reduced the download size to about half of what it used to be. Below are the best tricks we learnt or revisited after using YSlow.

Eliminate HTTP requests

This is a well known method that we had already worked into the design of Playing With Wire. The idea is to have as few CSS, image and Javascript files as possible. Most browsers will only download two files at a time and there’s always some overhead associated with the download of a new file. If you can combine files you reduce this overhead.

YSlow’s Components page let us know exactly what we were bringing in through links and we could eliminate an external Javascript we were no longer using.

Add Expires headers

Expires headers are important to let web browsers know that once they’ve cached an image, CSS include or Javascript file, they can keep using it for a while. Without these headers most browsers will keep downloading the same files over and over out of fear that they may change frequently. Again, YSlow’s Components tab reveals relevant information: the Expires column lets you know what Expiry date your web server is broadcasting for each downloaded file. If you notice files with a value in the Expires column, it may be time to go into your web server configuration file. In Apache, a section like this one might just do the trick:

<virtualhost ...>
...

<directory ...>
  ...
  ExpiresByType text/css "access plus 1 week"
  ExpiresByType text/javascript "access plus 1 week"
  ExpiresByType image/gif "access plus 1 week"
  ExpiresByType image/jpg "access plus 1 week"
  ExpiresByType image/jpeg "access plus 1 week"
  ExpiresByType image/png "access plus 1 week"
</directory>

  ExpiresActive On
</virtualhost>

Enable compression

Most modern clients support streaming compression. This is a feature that lets the web server compress data before sending it to the client. This reduces the download time of the page at the expense of some CPU time on the server. While most graphics can’t be compressed much, this turns out to work out great for HTML, CSS and Javascript. All of these files can often be reduced to as little as a third of their original size. The best part is that the web server won’t try to compress the data unless it already knows the client can handle it.

How to set up compression depends on your application and web server software. If you’re using Apache, you can have the server do it for you for normal files. For dynamic content such as that generated by PHP it depends. In WordPress there’s a switch in the Options tab. If you’re using wp-cache like Playing With Wire is, it may take some more work to get things up and running, but it’s well worth the effort.

Conclusion

Between these tricks and a couple more, Playing With Wire came out about half as heavy for the main HTML, CSS and Javascript, and as a result felt much more responsive. All in all YSlow was a helpful utility, especially thanks to its ‘Components’ tab which made it easy to see what parts of the page were being cached and compressed properly, and which ones were not. The grading system wasn’t very helpful, not all of the grade hints were applicable, and none of the hints were unknown in the field. Still, the list of suggestions was a useful as a kind of laundry list of things to do for the site, combined with data specific to your site. At the end of the day, I’ve found a new partner for optimizing websites with Yahoo!’s YSlow.

Monday, July 23rd, 2007

How to get your P990i to sync with iSync

Finally! Ever since I bought my P990i about 7 months ago, I’ve been struggling with keeping my contacts and calendars in sync. Since there is no native support for the P990i in iSync (blame both Apple and Sony Ericsson, since none of them seems to care to solve the problem), I’ve been spending hours on the web trying to find nifty hacks to bypass this problem.

Until a couple of days ago, the most successful solution was to use Goosync to synchronize my calendar with my Google Calendar, and then simply import and export vCards back and forth to keep the address book in sync (note that you must use vCard v2.1 to achieve this). However, as you might imagine this gets tiresome quite quickly.

A benefit with GooSync was that I could synchronize anytime and anywhere as long as I had access to WiFi. The downside however was that I needed to remember to sync the phone every day. Another downside was that I needed to use Google Calendar instead of my personal preference, Apple’s Calendar. (Although it is possible to configure Apple’s Calendar to access Google Calendar in a read-only fashion when offline.)

Ok, lets cut to the chase: how to get your P990i to work with iSync. The solution to the problem is found here. The solution is a plugin written by ‘mate,’ who posted the plugin at My-Symbian.com. The plugin is still quite buggy, and neither we nor the author of the plugin should be held liable for any problems that might occur to your P990i or your data. It can be quite a bit of headache to get the plugin to work properly, so please follow the following steps carefully.

  • Back up your data
  • Make sure you back up all your data. If you have access to a Windows PC (which you will need in the next step anyways), use Sony Ericsson’s Backup-software from the PC Suite. Also, use the phone’s internal backup-software to perform a backup to the Memory Stick on the Calendar (Calendar -> Calendar Manager -> Backup) and the Contacts (Contacts -> Contact Manager -> Backup).

  • Update the phone to the latest firmware
  • In order to get this plugin to work, your phone need to run the latest firmware. Unfortunately there’s no way to do this on a Mac (maybe using Parallel’s might work, but I didn’t try this), so you need to have a PC running Windows to do this update. Just download the latest version of Sony Ericsson Update Service and install it on your PC. Since you’ve already backed up all your data, you can just go ahead and run the update right away. Note that the update takes quite some time, and that it will wipe all the data off your phone.

  • Restore your data from your Memory Stick
  • Assuming that the update went fine, you can just go ahead and restore your Calendar (Calendar -> Calendar Manager -> Restore) and the Contacts (Contacts -> Contact Manager -> Restore) from your Memory Stick. Don’t recover the data using Sony Ericsson’s backup utility, since you won’t be able to perform a sync if you do. We only took that backup to make sure that we have an additional backup in case something goes wrong.

  • Delete your old bluetooth pairing
  • Now let’s move over to the Mac to connect the phone with your Mac. The first thing we want to do is to delete the (possible) prior pairing with the phone. To do this go “Apple” -> “System Preferences” -> “Bluetooth” and Select the Devices tab. Now select your phone and press the delete button.
    bluetooth_thumb.png

  • Install the plugin
  • It’s finally time to install the plugin. Download the file from here (or our local mirror). When the download is completed, just go ahead and click on the dmg-file and run the installation.

    bt-add_device_thumb.png

  • Pair the phone with your Mac
  • The first thing you need to do is to enable bluetooth on your phone. Click Menu (far lower left corner) -> Connections -> Bluetooth -> “Bluetooth On”. Now let’s move over to the Mac and click on the Bluetooth icon up in the very right corner and select “Setup new bluetooth device.” In the wizard, select Phone and select your phone from the list. In the very last step, deselect Dial Up Networking, and just leave Address Book and Contacts selected.

  • Perform your first sync
  • Now, before you hit that sync button in iSync, you should make some changes to the settings. First, go ahead and deselect Calendar, and just perform a Contact sync for the initial sync. If your sync is successful, start by selecting one calendar, and then two calendars in the next sync. I’ve personally had problems with syncing more then one calendar, so you may or may not be able to sync several calendars.

properties_thumb.png
Congratulations, you should now finally be able to sync your P990i with you Mac. As you might notice, the plugin is still in early beta, so it’s not unlikely that you’ll receive some weird error messages or have some failed syncs. Regardless, this is by far the best solution that I’ve run across. If you however do run in to some trouble, I’ve compiled a small troubleshooting guide that you might find useful.

    Troubleshooting

  • My phone crashes and reboots when I try to sync
  • There’s a couple of ways this might happen. I actually ran into this myself, but was able to solve it. If you do run into this problem, try these things:

    * Make sure that you’re running the latest firmware.

    * Deselect the calendar-sync and only sync the Contacts.

    * If you can’t even perform a Contact-sync, you should try to make a Master Reset on your phone. After you’ve performed the Master Reset, try to sync before you restore any data.

  • My phone restarts when I try to sync more than one calendar
  • I actually experienced this too. For some reason some of my calendars won’t sync. At this point I really don’t know why some of my calendars won’t sync, but fortunately my most important calendars synced without problems. What you want to do if your phone crashes when syncing calendars is to select the calendars one by one, and see which calendar(s) cause the sync to fail.

For more troubleshooting ideas, please see the forum over at My-Symbian.com where the plugin was originally posted.

    Other useful tools to make the P990i more Mac friendly

  • USB File mode enabler
  • This little kernel-patch enables you to use the USB cable to connect your P990i to your Mac. Previously I was forced to either use bluetooth or a Memory Stick-reader to transfer data to the P990i. With this nifty tool you are not only able to access the files on the Memory Stick, but the cable also charges the phone.

  • iTunesMyWalkman
  • iTunesMyWalkman is a really nifty tool that works well with the USB enabler mentioned above. The software enables you to easily sync both photos and music to your phone. It also comes with some handy Apple Scripts that you can access from inside iTunes to make your phone act almost like an iPod.

That’s it. Hopefully this article is enough to help making your P990i more Mac friendly. Welcome to the world of synchronized data =)

As a side note I should probably mention that the guide above should be enough to get the m600, P1 and the w950 to get up running with iSync. The only difference (as far as I know) is that you use this plugin for the m600, this plugin for the w950 and this plugin for the P1.

Update: I guess I underestimated Sony Ericsson. As it turns out, the patch/driver mentioned in the article actually turns out to be made by Sony Ericsson (and only leaked, and possibly modified by ‘mate’). Sony Ericsson has now released the driver, which can be found here. The official download offers a new version which is also more stable.

Monday, July 16th, 2007

Famous Mac Software Company Parallels Forgets About Safari

Parallels makes a virtual PC type of software for the Mac which allows you to run Windows on the Mac. Great software, but the company has a little bit of a history of quality control problems. Today the company launched a new design of their website. Unfortunately the company forgot about supporting the default Mac browser, Safari!

Parallels website shows a dropdown in the wrong place in Safari.

Nothing big: a drop down menu is showing out of place. The site actually seems to start working after you resize it for the first time, or click a single link. It’s likely to be fixed by the time many people read this, but it’s still a little bit ironic that a company with a major Mac market would not check their site in Safari.