I think it’s fair to say that few people outside of Google have more experience of working with Google’s IMAP implementation (GIMAP) than we have. Since we launched YippieMove more than six months ago, we’ve performed a lot of transfers to Google using IMAP. Truth be told, we’ve spent a lot of time trying to find workarounds for bugs in Gmail IMAP implementation. In this brief blog-post, we will explore two bugs that we’ve reported to Google, but which Google seems to have little interest in fixing.

Inconsistency between SELECT and CREATE

The first bug might not be very juicy but it took us a while to recognize it. What we first thought was a bug in our system turned out to be a bug in GIMAP. As it turns out, in GIMAP, SELECT is case sensitive, while CREATE is not. Here’s a brief example to illustrate the bug:


0001 SELECT "INBOX/Sales Invoices"
0001 NO Unknown Mailbox: INBOX/Sales Invoices (Failure)
0002 CREATE "INBOX/Sales Invoices"
0002 NO Folder name conflicts with existing folder name. (Failure)


In this case, there is already a folder named ‘INBOX/Sales invoices’, but since SELECT is case sensitive, and CREATE is not, we were unable to select the folder and with an upper case ‘I’ and at the same time unable to create the folder.

We first encountered this bug when a user migrated from a case sensitive IMAP server to Gmail. Hence we could have two folders named ‘foo’ and ‘Foo’ without it being any problem until we tried to copy those to Gmail.

Rejection of random messages

The next bug is of far more serious nature and it is a bug that we run into everyday. We’ve spent a significant amount of time trying to narrow down this problem, but almost entirely without luck. What happens is that GIMAP decides to reject certain emails. Sometimes the upload works when retrying later, sometimes not. To make this even more interesting, GIMAP can reject emails that it just gave us. For instance, let’s say we’re copying messages from [email protected] to [email protected]. Even that a GIMAP just gave us a particular email, another GIMAP server rejects the same email. Strange, isn’t it?

When I said that we’ve spent a serious amount of time trying to narrow down the problem, I was not kidding. We’ve done statistical analysis on tens of thousands of messages trying to find some kind of pattern (including trying to find a correlation between the content in the header and the body), and still no luck.

That said, the failure rate is quite low. Out of all the messages we upload, only a small fraction gets rejected (a quick database query reveals that it’s currently at 0.04%). And if a message gets rejected, we clearly state what message we failed to upload in the Transfer Report that we send out to our customers upon completion.

In Google’s defense, they do state that they do not officially support ‘upload of messages,’ but that is a quite weak argument, as pretty much any other IMAP-enabled service on the market supports this. Not to mention that without APPEND, simple drag and drop operations may fail in email programs.

For the curious geeks reading this article, our software does comply with RFC 3501, as well as other related RFCs.

As a side note, we’re not the only ones experiencing the APPEND bug. The Google Group discussion for Gmail IMAP and POP is full with threads regarding this. The only official-looking response given multiple times by wár17 § is to upload in small batches as you otherwise is likely to hit the bandwidth limit. However, we can testify that this is not the case as the number of messages and the size of the mailbox seem to have little effect. We’ve had this problem with small mailboxes (<10 messages and only a few hundred bytes) and at the same time had mailboxes that exceed several Gigabytes running through flawlessly.

Update: While it seems like the rejection of emails is somewhat random, there are certain emails that are more likely to get rejected. The other day we ran a transfer of a folder including 600-something bounced messages. They were all rejected.