Introduction
“No way! That’s impossible.” Well, actually it’s not. Using Open Source technology, it’s actually possible to create a competitive IT infrastructure at very low costs. Not only does Open Source software enable you to create more customized solutions to better fit your needs, but it also means that you can spend your budget on hardware – not software.
Last month I was asked by a company to figure out how to ‘modernize‘ their IT infrastructure with a minimal (almost non-existing) budget. After plenty of thinking and research, I came to realize that the only way to do this was to use some kind of thin-client solution. The solution that I found to fit my needs the best was Linux Terminal Server Project (LTSP). LTSP utilizes network boot (we will use PXE) to boot the clients directly from the server. Therefore, we can use obsolete clients without hard drives (to reduce the noise) as thin clients. The only thing we need on all the clients is a fast network adapter with PXE-support.
Some of the widest adoption of LTSP has been within K12 the education field. Since many educational institutions are working with very tight budgets, LTSP has a strong advantage. It is a way to save costs without having to compromise too much on usability. Edubuntu is a Linux distribution that targets educational institution. What makes Edubuntu very interesting is that it comes out-of-box with LTSP support, which enables system administrators with limited knowledge of Linux to get a thin-client setup running with little effort. Since Edubuntu is closely related to the regular Ubuntu, it’s not very hard to get Ubuntu up and running as an LTSP server.
The Hardware
With a very limited budget, I realized that a thin-client solution would be the most realistic approach. As the name implies, the clients are thin and most of the load will be on the server. Therefore we will spend most of the money on a solid server. After a good amount of research to find a cheap, yet powerful and expandable server, I found the HP ProLiant ML115. The server comes with a 64 bit 2.2GHz Dual Core AMD Opteron CPU, which will serve our needs well. However, it only comes with 512Mb of RAM, which is insufficient for the number of users we intend to have. Therefore we’ll need to purchase some additional RAM. The RAM consumption estimates varies across different LTSP projects, ranging from 256Mb + 32Mb per client to 1024Mb + 64Mb per client. However, since I’d rather be on the safe side, I’d recommend that you purchase another 2Gb of RAM (total 2.5Gb) and put in the server (1024Mb + 150Mb per client).
The next thing we need to add is a Gigabit switch to reduce the possibility of having the network as a possible bottleneck (note that I’m not sure if a 10/100 Mbit network would actually create a bottleneck in this setup, but I rather be safe than sorry). Since Gigabit is cheap today, going Gigabit all the way seems like a reasonable move. Therefore, I’ve budgeted for 10 Gigabit network adapters (with PXE support) and a 16 ports Gigabit switch (the HP server comes with Gigabit network adapter).
Now we need somewhere to store the users’ data with high security and performance. Since we’re on a limited budget, we will use a software RAID solution rather than a high-end hardware RAID solution. A RAID-5 setup on three SATA disks using Linux’s software RAID is probably the cheapest and most reliable for the price. This will allow us to increase performance while we also gain protection against loss of any one drive without loss of any data. Moreover, we also use a separate OS disk to reduce the I/O load.
Because everything is both stored and running on the server, it’s crucial that we protect the equipment from power failures and spikes (the server by itself is a single point of failure). Therefore I’ve added an UPS to the budget (1100 VA) that will not only protect our hardware but also reduce server downtime.
The last thing we need to add is the clients. It’s quite likely that you have a many retired PCs (preferably P2>). If not, you’re likely to find a many of these computers for sale (or for free) in your local classified adds section. If Craigslist is available where you live, this is a great source to find this kind of hardware. Even though many of you will get these computers for free, I’ve budgeted $50 per client.
The Software
I’ve divided this section into two parts: Server and Desktop. Although all software will be running on the server, the users will only see the software on the Desktop side, which is why I have separated them.
Server
As already touched upon, the server will be running Linux. To be more precise, I’ve decided to use Ubuntu Server 6.06.1 LTS Server (64-bit). The reason why I didn’t chose to use the most recent version (7.04) is to minimize the administrative effort. Since 6.06 is the Long Time Support (LTS) version, the Ubuntu team will supply this version with security patches and updates for a longer period than the 7.04 release (6.06 LTS Server will be supported until 2011).
Installing Ubuntu is very straight forward. The only thing to consider in this setup is to make sure we install the system on the hard drive that came with the server and not the RAID-5 array.
Configuring the RAID 5 can be done though a couple of different ways. You can either do this during the installation (with Ubuntu’s graphical utility), or wait and set it up afterward in the console (see the Software RAID HOWTO for details). After setting up the RAID, go ahead and mount it to /raid or something similar.
Now it’s time to set up LTSP, which is the foundation of our cost-saving solution. There are a couple of different ways to do this, but the one I found to be most useful was to follow a guide from Novell (strangely enough) available here. You might also want to take a look at Ubunut’s Thin-client documentation.
Before you start the installation, go ahead and symlink /opt, which is where LTSP will be installed, to within you RAID array (ln -s /opt /raid/opt). This will install all the packages on the RAID array instead of the system disk. Finally, what you will want to do is to add a test-user (or a real user – it’s your call). This is done by simply using the user-management tool in the Ubuntu. Note that you probably want to have the home-dir of the users on the RAID array. To achieve this, you can either symlink the entire /home to /raid/home or just set the home-dir to /raid/home/user in the user-creation process.
Desktop
The LTSP setup comes with the most common software used in a corporate environment. This includes:
* FireFox – A great web browser
* Open Office – A Microsoft Office replacement
* Evolution – A Microsoft Exchange replacement
* The Gimp – An Adobe Photoshop replacement (arguably less powerful)
Plus a long line of other applications such as PDF-viewer etc.
You might also want to install is Wine. Wine is a Windows emulator which will run (legally) without any Windows license. Although it does not run all Windows software, many applications work very well.
If you have needs outside of the applications listed above, there’s more available. Any software that runs in a Linux environment (pretty much) will run on these thin clients. Although I haven’t tested it, you should be able to run a fully emulated Windows environment using a virtualization software such as VMWare Workstation or CPU/RAM intense softwares such as MatLab or CAD software.
The Clients
The last part is to get the clients to actually boot over the network. If you decide to use a different network card than the one specified in the budget above, make sure it supports PXE booting. Many budget NICs don’t support this feature. There are also other ways to boot but I’m not going to cover that in this article (such as floppy, CD etc.).
The Pros and The Cons
Although this approach is a very good way to create an updated desktop environment while at the same time minimizing the administrator’s job, it does come with some drawbacks. Unfortunately many companies today are stuck with custom software that only runs on Windows. Although Wine offers a great emulation software, you might be forced to purchase a license of VMWare Workstation (and Windows) to run some specialized applications. If you’re lucky, your custom software was written in Java, and it will actually run on Linux as well.
Another thing to consider is the transition from the old environment (quite likely from Windows) to this new environment. Although the transition is likely to be smooth for a crew of young people, members of the older generation (40+) are likely to require some training before being able to use the system fully. Both Open Office and Firefox work very similar to their Windows-counterparts, but Evolution is slightly different.
Another pro is the lack of viruses on Linux. Since we’ve left Windows behind, the likeliness of being infected by a virus is almost zero.
In summary, utilizing a solution such as this might or might not suite your needs. If you do have the flexibility to use Open Source software in your organization and are able to run your customized software either emulated or use a web-based interface, this is a great way to reduce costs from the IT budget and spend them in a way that benefits the company better.
Update: I’ve now actually deployed this solution at a company. For information about how it went, go ahead and read “Deploying the sub-$3,000 IT-infrastructure.”
Good article, but evolution is an outlook replacement, not exchange mail server. If your looking for more advance exchange replacements, look to zimbra, open-exchange, or scalix.
The big advantage of software raid is that you don’t have to worry about raid controller failure. If a hardware raid controller fails you need to find a compatible replacement or you can lose all your data. This can be a real problem.
You haven’t speced out a complete solution because you’ve not included backup. Raid is no substitute for backup. Backup means you can accidently delete a file, discover the problem a week or a month later, and still recover. Take one of your $50 boxes and put some big hard drives in it. Make a file system with lots of extra inodes. (I find 1 inode per 4096 byte block sufficient.) Then write a rsync backup script with –hard-links or use something like dirvish to get daily backups that share hard drive space so you can have hundreds of days of daily backup using only 2 or 3 times the disk space of the backed up filesystem. Get fancy and put the backups read-only on the network with samba or nfs.
Then make another backup server and put it in a different building and connect to it via the Internet. You’ve now got disaster recovery in case of flood or fire.
And don’t forget to use lvm on the server to make life easy, and on the backup box to make one big filesystem that spans many drives.
mattm,
Yeah, you’re right. The reason why I chose to not mention Zimbra was that I was looking for a desktop replacement. Zimbra is a great product, but the benefit to use Evolution is that it is easier to deploy and supports import mailboxes and settings from Outlook (I’m not sure if Zimbra does this).
Karl,
You’re right about the software raid. That’s an aspect that I actually didn’t think of. Another benefit of using sw-raid instead of hw-raid is that it is easier to troubleshoot remotely.
As for the backup, your suggestion of turning one of the thin clients into a backup-unit is not a bad idea. As you point out, a simple rsync-script can take care of the backup quite easily. Also, an off-site backup solution is a great suggestion too.
HEllo,
how did you solve the problem with ERP. Most of Small Business here in Germany use cheap windows applications for bookkeeping and management. Itś hard to find a similar solution for Linux. There are good OSS Projects like adempiere or openbravo, but the are lightyears away to be “lightweight”. An other problem ist the “GOB”-attestation. This attestation must given from a certified public accountant like PricewaterhouseCoopers or KPMG. without your in danger to get in trouble wit financial authorities.
olli,
I’m actually in the same situation. I’m about to deploy this solution in Sweden, where most SMBs use a similar Windows software for ERP. The software is made by a software company called SPCS and is Windows only. Fortunately the software does run under Wine, which is how I’m intending to solve this problem.
As far as ERP/CRM etc. softwares for Linux, there’s a couple of alternatives. You already mentioned OpenBravo, which I have no experience of, but saw a demonstration at LinuxWorld and it seems quite solid. Another software you might also want to look into is SugarCRM.
What happened to the router? Looks like it is in the image but no mention of it. Just curious.
Regards,
Nix
I’ve deployed this sort of setup for a few Australian small businesses now, and all have been very pleased with the outcome. In the past I’ve used Gentoo simply because of its marvellous “USE flags” which enable you to trim down the install size and memory footprint of binaries greatly (these things add up, with 20+ users on a system).
One location currently has 25 simultaneous users on a Dual AthlonMP 1800+ (1.533GHz) with 2GB of RAM, and barely creeps over a load average of 1.0. They use a mixture of Firefox, Thunderbird, OpenOffice 2, GIMP, Inkscape and other tools without any problems. Thin clients are left-over boxes from old upgrades, and cost next to nothing. Overall it’s been a huge cost saving for the business, and reliability of the system is much higher than other systems which require constant patching, rebooting, anti-virus, and other time wasters.
Nix,
You’re right. I did mention it in a draft, but it must have deleted it. The reason why I neglected to mention the router was because it’s hard to budget for since many companies get free routers from their ISP.
elvis,
Very interesting. You’re probably right about that you can get better performance from a Gentoo machine, but I decided to go for Ubuntu because of the low administrative maintenance with the LTS-release.
I’ve seen various estimates and case studies from K12 environments, but very few business-cases. If you manage to run that many clients on one server with that kind of heavy apps running, my fear of having an underpowered server was for nothing.
Just out of curiosity, are you running LTSP 4 or 5?
Also, are you running on a gigabit or 10/100 network?
I really appreciate your input.
Nice roundup! I guess, with such a budget you can convince even some of the most “windows-infected” managers.
A couple of questions:
– What about Monitors? Are they included in the $50/client? Nowadays, I think, you can’t expect employees to use 15in tubes anymore.
– What are the average specs (RAM/CPU) for the clients?
– What about cabling? (Ethernet and power)
As for the backup solution, I suggest you have a look at the “backup2l” package, which is available in the “universe” repository. Very easy to configure and maintain.
Christopher,
I hope these answers will clarify the details further.
– What about Monitors? Are they included in the $50/client? Nowadays, I think, you can’t expect employees to use 15in tubes anymore.
In my calculations I actually expected those to be included in the $50 budget. However, you’re right about that most employees today expect TFT/LCD monitors, but with this budget, that’s almost impossible to solve. I guess that the cheapest TFT/LCD monitors you can find today is in the $100 range.
– What are the average specs (RAM/CPU) for the clients?
I’m not sure, but I don’t think it matters too much. As long as the clients has an available PCI-slot for the NIC, I think they’re good. However, I thing a P2 or newer with 128>Mb ram would be reasonable.
– What about cabling? (Ethernet and power)
I’ve intentionally left this out, because it’s really depends on how your office looks like.
As for backup solution, I need to look into your idea further. It might be a good idea that’s easy to deploy.
The network cards for the clients might be a problem. If you are buying old computers it is very possible that they will not have PCI Express slots.
Have you found any other “cheap” PCI PXE enabled NICs?
Rudiger,
Good point. Didn’t think of that actually. Well, I know Intel has their Pro 10/100/1000 Desktop adapter for PCI that supports PXE. I guess that would be an option. Also, I bet 3com offer some similar card. However, I think most of the “cheap” NICs fails to support PXE.
Why can’t you use normal PCI cards? Is normal PCI too slow for gigabit ethernet?
I have found gigabit ethernet PCI NICs with PXE support on German ebay for ~11EUR incl. shipping, so I guess you can save a little bit on the budget here.
> I guess that the cheapest TFT/LCD monitors you can find today is in the $100 range
Yes, most cheap 15in TFTs I see on ebay are around 60-70 EUR plus shipping.
But from a workplace ergonomics point of view, a good monitor is crucial and should be worth to stretch the budget a bit. I f an employer is not willing to spend sufficient money for that, I wouldn’t want to work there.
Great Article.
If you do have to use VMWare you don’t need to buy VMWare Workstation, you can download VMWare Server for free. I run it at home and at work with no trouble.
Christopher,
Regarding gigabit and PCI…there’s no problems finding Gigabit PCI NICs. There are two things to consider tough: 1) Make sure the NIC supports PXE-booting 2) Make sure it’s a PCI card and not PCI Express. It’s quite likely that your clients won’t have PCI Express-slots.
As for the ergonomics I don’t think this is an optimal setup. It is however one of the cheapest setups possible. If you can extend the budget and add ergonomic hardware (better displays, keyboards, mice etc.) I think that’s a great investment that’s going to pay off in the long-run.
Harley,
Good point. I forgot that VMWare gives that one away for free. However, I’m not sure how well a virtual instance of Windows will work with multiple users. I would imagine that you would need one instance of windows (and license) for each active user.
PS: assorted addendae
Re-scanning the budget — you could (can?) have TFTs by stretching it up a bit and dropping/downgrading those parts which are overkill or overpriced: GigE NICs (then 15×100+1×1000 switch), 500G disks and strangely expensive memory.
BTW D-Link switches aren’t exactly reliable, that is one might work flawlessly for years but you never know when you get ports failed.
> As for the ergonomics I don’t think
> this is an optimal setup.
We went for compaq deskpro sff (small form factor), very nice and they do PXE booting with updated BIOS flashed in where needed. These include onboard ATI chip, ESS sound (with loudspeaker even), and CD/FDD. We’ve only added USB2 PCI cards in some of them.
Ah, and client specs are OK with 64M RAM and something like PII up (probably PI too, didn’t see working one for some year or so, ours are PIII-500). The worst that can hit the terminal is browser inflating X server pixmap cache (taking the whole thin client down) but you should really have networked swapping activated anyways.
—
Re Windows, you might want to use translation tools to read syselegance.kiev.ua — folks have implemented a custom terminal server solution employing a single WinXP Pro as a terminal server with Linux clients; it seems to have passed legal scrutiny for at least some of their wary customers. If you badly need win32 environment, going that way might be more cost efficient than struggling with vmware in the first place.
Here is a bit more about remote network booting that I found interesting.
http://www.linux.com/feature/118636
You can also add ClamAV to the mix. Works well on my Edubuntu setup.
way to go open source
We implemented LTSP 4.1 on a CentOS-based server using existing CAT5 cabling that was not capable of gigabit. The server has a gigabit NIC and is hooked to a gigabit hub, but the thin-clients are all using 1000. One time as a test we ran CAT6 cable to one thin client and got a gigabit connection, but we saw no difference in speed. For the record, our server has 2 dual-core 3.2GHz Xeons and 6 GB RAM, and we are running almost 30 clients with little problem. We are running Windows Server 2003 with Terminal Services (on a second server – 1 GHz, 512 MB RAM) for our minimal Windows requirements, but we recently ran across XP Unlimited (xpunlimited.com) which offers Terminal Services on XP for a site license, and seems to work well with VMware Server or Xen in our testing, and could be quite a cost savings if you already have an XP license you can use.
WINE = WINE is not an emulator
its an acronym, and not an emulator