Our email transfer service YippieMove is essentially software as a service. The customer pays us to run some custom software on fast machines with a lot of bandwidth. We initially picked VMware virtualization technology for our back-end deployment because we desired to isolate individual runs, to simplify maintenance and to make scaling dead easy. VMware was ultimately proven to be the wrong choice for these requirements.

Ever since the launch over a year ago we used VMware Server 1 for instantiating the YippieMove back-end software. For that year performance was not a huge concern because there were many other things we were prioritizing on for YippieMove ’09. Then, towards the end of development we began doing performance work. We switched from a data storage model best described as “a huge pile of files” to a much cleaner sqlite3 design. The reason for this was technical: the email mover process opened so many files at the same time that we’d hit various limits on simultaneously open file descriptors. While running sqlite over NFS posed its own set of challenges, they were not as insurmountable as juggling hundreds of thousands of files in a single folder.

The new sqlite3 system worked great in testing – and then promptly bogged down on the production virtual machines.

Tough CPU week on a server running VMWare

We had heard before that I/O performance and disk performance are the weaknesses of virtualization but we thought we could work around that by putting the job databases on an NFS export from a non virtualized server. Instead the slowness we saw blew our minds. The core servers spent a constant 70% of CPU time with system tasks and despite an uninterrupted 100% CPU usage we could not transfer more than 400KBit/s worth of IMAP traffic per physical machine. This was off by a magnitude from our expected throughput.

Obviously something was wrong. We doubled the amount of memory per server, we quadrupled sqlite’s internal buffers, we turned off sqlite auto-vacuuming, we turned off synchronization, we added more database indexes. These things helped but not enough. We twiddled endlessly with NFS block sizes but that gave nothing. We were confused. Certainly we had expected a performance difference between running our software in a VM compared to running on the metal, but that it could be as much as 10X was a wake-up call.

At this point we realized that no amount of tweaking was likely to get  our new sqlite3 version out of its performance hole. The raw performance just wasn’t there. We suspected at least part of the problem was that we were running FreeBSD guests in VMware. We checked that we were using the right network card driver (yes we were). We checked the OS version – 7.1, yep that one was supposedly the best you could get for VMware. We tuned various sysctl values according to guides we found online. Nothing helped.

We had the ability to switch to a more VM friendly client OS such as Ubuntu and hope it would improve performance. But what if that wouldn’t resolve the situation? That’s when FreeBSD jails came up.

Jails are a sort of lightweight virtualization technique available on the FreeBSD platform. They are like a chroot environment on steroids where not only the file system is isolated out but individual processes are confined to a virtual environment – like a virtual machine without the machine part. The host and the jails use the same hardware but the operating system puts a clever disguise on the hardware resources to make the jail seem like its own isolated system.

Since nobody could think of an argument against using jails we gave them a shot. Jails feature all the things we wanted to get out of VMware virtualization:

• Ease of management: you can pack up a whole jail and duplicate it easily
• Isolation: you can reboot a jail if you have to without affecting the rest of the machine
• Simple scaling: it’s easy to give a new instance an IP and get it going

At the same time jails don’t come with half the memory overhead. And theoretically IO performance should be a lot better since there was no emulated harddrive.

And sure enough, system CPU usage dropped by half. That CPU time was immediately put to good use by our software. And so even that we still ran at 100% CPU usage overall throughput was much higher – up to 2.5MBit/s. Sure there was still space for us to get closer to the theoretical maximum performance but now we were in the right ballpark at least.

More expensive versions of VMware offer process migration and better resource pooling, something we’ll be keen to look into when we grow. It’s very likely our VMware setup had some problems, and perhaps they could have been resolved by using fancier VMware software or porting our software to run in Ubuntu (which would be fairly easy). But why cross the river for water? For our needs today the answer was right in front of us in FreeBSD: jails offer a much more lightweight virtualization solution and in this particular case it was a smash hit performance win.

Today we are really happy to be able to add support for Yahoo Mail in YippieMove. For the first time email can easily be moved in and out of Yahoo Mail accounts. Yahoo Mail, the worldʼs largest provider of email services, does not by itself provide any method for users to transfer email between different accounts and providers. YippieMove breaks down this barrier and enables average users to migrate their own email to or from Yahoo Mail accounts.

Since we launched YippieMove, our customers have been asking for support for Yahoo Mail. Today we are finally able to offer our customers this service.

While many modern email providers, such as Googleʼs Gmail, have opened up to allow easy access to a userʼs email, Yahoo Mail users are still locked in. This is why we are proud to announce support for Yahoo Mail in YippieMove. Finally, this vendor lock-in is broken, and users are free to move their emails in and out of Yahoo.

YippieMove now has a potential reach of over a third of the worldʼs email users. Users can transfer mailboxes to and from Yahoo Mail as well as any of the more than 100 different email providers currently supported.

Guidelines for Yahoo Mail users looking to move

With this new addition to YippieMove, we’ve solved one of the main issues with switching email provider, namely moving the actual emails.

With the emails moved, let’s move on to the next problem — incoming mail. The best way to deal with this is email forwarding. Unfortunately this service requires Yahoo Mail Plus which costs $19.95 per year. If you’re not willing to spend that, you can use the free vacation response-feature and write a note telling everyone emailing you that you’ve switched email address.

The next thing you might want to move is your contacts. To export your contacts is quite straight forward. However, in order to access the export feature, you need to switch to Yahoo Mail Classic. Once you’ve exported your contacts you can import that file into your new email provider.

For more information about YippieMove, visit YippieMove.com

‘If Joomla! is Linux, then WordPress is Mac OS X. WordPress might offer only 90% of the features of Joomla!, but in most cases WordPress is both easier to use and faster to get up and running.’

Over the course of the last few years, I’ve been in charge of putting up a number of websites for various companies, often as favors for friends. In most cases, I’ve ended up using one out of two solutions: Joomla! and WordPress. While both of these projects have evolved greatly over the last few years, they are vastly different. Joomla! has always been intended as a ‘fit-all-your-possible-needs’-kind of CMS solution, while WordPress was developed as a blog with CMS capabilities. Recently WordPress has opened up to allow its users to set up a site with static-only material (with the option of a blog-page), without having to hack the code. Hence it’s one step closer to being a direct competitor to Joomla!.

Joomla!’s Control Panel	WordPress’ Dashboard

I will probably step on a few peoples feelings here, but I will argue that Joomla! is an example of a poorly managed open source project and that WordPress is a very successfully managed one. Certainly I don’t mean that Joomla! is a useless piece of junk, but that the lead developers have quite a bit to learn from WordPress. The main thing that Joomla! is vastly behind on is usability. While it is true that Joomla! 1.5 is a step in the right direction, it is still light years behind WordPress. Let me illustrate with two examples of common tasks.

Example 1: Create a blog-post with an image

Joomla!

1. From the ‘Control Panel,’ click ‘Add New Article.’
2. There are two image buttons. If you use the wrong one, you won’t be able to upload an image (as you will only browse the existing images). You must use the one below the text field.
3. Select a ‘Title,’ the right ‘Section’, and then the right ‘Category.’
4. Write the content and save.

WordPress

1. Select QuickPress in the Dashboard.
2. Click on the image icon and upload the image.
3. Select title, write your content and press publish.

‘Add New Article’ in Joomla!	‘QuickPress’ in WordPress

Example 2: Create a static page accessible from the menu

Joomla!

1. From the ‘Control Panel,’ click ‘Add New Article.’
2. Select a ‘Title,’ the right ‘Section’, and then the right ‘Category.’
3. Write the content and save it.
4. From the top-menu, select ‘Menu’ and ‘Main Menu’ (assuming you want to add it to the main menu.)
5. Click ‘New.’
6. Select ‘Internal link,’ and ‘Articles,’ and then finally ‘Article Layout.’
7. Fill in the title of the object as well as the parent item.
8. In the column to the right, you now need to browse your list of articles and select the desired article.
9. Press ‘Save.’

WordPress

1. From the Dashboard, click ‘Pages.’
2. Select ‘Add New.’
3. Fill in the title and contents.
4. Select the parent item (if other than root.)
5. Click ‘Publish.’

‘Add New Article’ in Joomla!	‘Add New Page’ in WordPress

Let’s step back for a minute and imagine the following scenario: you’re in charge of putting up a website for a company. They might want to put up about 10 or so pages with various information. According to my experience this is a pretty common situation. You can do this with either Joomla! or WordPress – both are fully capable of delivering this. Assuming you’re going to buy a template to solve the design issue, it will probably take you about an hour with either of the software to get to the first draft (assuming you’ve been working with them in the past.) So far so good. This is where they start to differ. With WordPress you’re pretty much done by now. However, with Joomla!, you’ll probably have to spend another hour or two just trying to re-organize the different modules to fit the template you bought (in many cases, just to get the basics to work.) Next you will end up spending even more time trying to figure out how to re-organize the different menus. You need to link up a particular document to a particular menu-entry (as illustrated above.) If you want a blog-feed, you need to set up a dedicated section or category (I still don’t really know the difference between the two.) Moreover, you need to select the ‘style’ of blog you want.

‘New menu item’ in Joomla!	‘Modules’ in Joomla!

Let’s say you managed to figure all of that out and that you got the site ready. Now it’s time to hand over the site to the customer. There will obviously be some training involved, and here’s another crucial difference between Joomla! and WordPress. Training someone to learn WordPress takes (in my experience) less than 30 minutes, and they truly understand it. Training someone to use Joomla! takes at least an hour, and they still don’t really understand it.

Again, I’m not saying that Joomla! is useless, it’s that WordPress is a more intuitive piece of software. Let me throw an analogy out there that will probably help you better understand my point. If Joomla! is Linux, then WordPress is Mac OS X. WordPress might offer only 90% of the features of Joomla!, but in most cases WordPress is both easier to use and faster to get up and running. I use and love Linux, it just doesn’t have that elegant touch to Mac OS X does.

To Joomla!’s defense, there are at least two scenarios I can think of where Joomla! is a better fit than WordPress. The first one would be eCommerce. If you install VirtueMart on Joomla! you can be up running with an eCommerce site pretty quickly. However, the problem is that it does not feel like it is a part of Joomla!, but rather as a 3rd party module that works in Joomla! (which is pretty much what it is.) The second one would be a site where you need to have multiple levels of permissions (ie. an extranet). WordPress only offers three levels of permission (public, private, and password protected), while Joomla! is much more flexible.

‘Global Configuration’ in Joomla!	‘General Settings’ in WordPress

Joomla! is not doomed. It just has a long way to go when it comes to usability. WordPress has really been developed by the KISS-principle, while Joomla! appears to have been developed to solve every problem on Earth (by engineers, for engineers). Going back to the two problems I mentioned above, where Joomla! beats WordPress. I think it would actually be less of a challenge to add support for eCommerce and more permission levels to WordPress, than it would be to improve the usability in Joomla! to reach WordPress’ level.

Just as a side note, a quick line-count on the latest versions of both software reveals that Joomla 1.5.9 has 350,975 lines of codes, while WordPress 2.7.1 has a mere 159,682 (might not be completely accurate, but that’s what ‘wc -l’ said). Hence, even if WordPress only offers 70% of the features of Joomla!, which I am pretty sure it does, their code is written much more efficiently.

WireLoad is proud to today introduce YippieMove Complete, the latest addition to YippieMove, which adds the missing piece of the puzzle for many small and mid-size companies (SMB) that are considering switching email provider. YippieMove Complete is a package that includes everything needed for a smooth transition, including:

• Unbiased technical consulting. What is really the best email solution/provider for your needs?
• Creation of user accounts on the destination server according to your needs
• Redirecting the email from your old server to the new
• Transfer of your old emails from the old server to the new server

Since we launched YippieMove, we’ve been contacted by many SMBs that are planning to change email provider for various reasons. Whatever your reason for switching is, YippieMove Complete can help you. We will do all the work for you and let you focus on what’s important to you – your business.

What does email downtime cost you per hour? If you’re an SMB, it’s likely that it costs you thousands of dollars an hour in lost productivity. YippieMove Complete can make sure that the transition goes as smoothly as possible, and in most cases, without any email downtime at all.

YippieMove Complete is starting at only $499.95 with 10 transfers included. If you ened more transfer than that our normal rate applies (including volume discount).

Requirements

The requirements are simple; we need access to your old email server and your DNS records. If you don’t know anything about this, our sales team can provide you with further details. We can even contact your provider directly as a last resort. The second requirement is related to the transfer of the old emails. In order for us to successfully transfer your old emails, the old email server needs to support IMAP.

More info

For more information about YippieMove Complete, visit the product page.

I think it’s fair to say that few people outside of Google have more experience of working with Google’s IMAP implementation (GIMAP) than we have. Since we launched YippieMove more than six months ago, we’ve performed a lot of transfers to Google using IMAP. Truth be told, we’ve spent a lot of time trying to find workarounds for bugs in Gmail IMAP implementation. In this brief blog-post, we will explore two bugs that we’ve reported to Google, but which Google seems to have little interest in fixing.

Inconsistency between SELECT and CREATE

The first bug might not be very juicy but it took us a while to recognize it. What we first thought was a bug in our system turned out to be a bug in GIMAP. As it turns out, in GIMAP, SELECT is case sensitive, while CREATE is not. Here’s a brief example to illustrate the bug:


0001 SELECT "INBOX/Sales Invoices"
0001 NO Unknown Mailbox: INBOX/Sales Invoices (Failure)
0002 CREATE "INBOX/Sales Invoices"
0002 NO Folder name conflicts with existing folder name. (Failure)


In this case, there is already a folder named ‘INBOX/Sales invoices’, but since SELECT is case sensitive, and CREATE is not, we were unable to select the folder and with an upper case ‘I’ and at the same time unable to create the folder.

We first encountered this bug when a user migrated from a case sensitive IMAP server to Gmail. Hence we could have two folders named ‘foo’ and ‘Foo’ without it being any problem until we tried to copy those to Gmail.

Rejection of random messages

The next bug is of far more serious nature and it is a bug that we run into everyday. We’ve spent a significant amount of time trying to narrow down this problem, but almost entirely without luck. What happens is that GIMAP decides to reject certain emails. Sometimes the upload works when retrying later, sometimes not. To make this even more interesting, GIMAP can reject emails that it just gave us. For instance, let’s say we’re copying messages from [email protected] to [email protected]. Even that a GIMAP just gave us a particular email, another GIMAP server rejects the same email. Strange, isn’t it?

When I said that we’ve spent a serious amount of time trying to narrow down the problem, I was not kidding. We’ve done statistical analysis on tens of thousands of messages trying to find some kind of pattern (including trying to find a correlation between the content in the header and the body), and still no luck.

That said, the failure rate is quite low. Out of all the messages we upload, only a small fraction gets rejected (a quick database query reveals that it’s currently at 0.04%). And if a message gets rejected, we clearly state what message we failed to upload in the Transfer Report that we send out to our customers upon completion.

In Google’s defense, they do state that they do not officially support ‘upload of messages,’ but that is a quite weak argument, as pretty much any other IMAP-enabled service on the market supports this. Not to mention that without APPEND, simple drag and drop operations may fail in email programs.

For the curious geeks reading this article, our software does comply with RFC 3501, as well as other related RFCs.

As a side note, we’re not the only ones experiencing the APPEND bug. The Google Group discussion for Gmail IMAP and POP is full with threads regarding this. The only official-looking response given multiple times by wár17 § is to upload in small batches as you otherwise is likely to hit the bandwidth limit. However, we can testify that this is not the case as the number of messages and the size of the mailbox seem to have little effect. We’ve had this problem with small mailboxes (<10 messages and only a few hundred bytes) and at the same time had mailboxes that exceed several Gigabytes running through flawlessly.

Update: While it seems like the rejection of emails is somewhat random, there are certain emails that are more likely to get rejected. The other day we ran a transfer of a folder including 600-something bounced messages. They were all rejected.