• Review of LifeIO – A first look at this email and social networking aggregation site.
• Taking Thunderbird 3 Beta 4 for Mac OS X out for a spin – A thorough look at the latest version of Mozilla’s Thunderbird.
• Can’t Access Your Hotmail with Outlook, Outlook Express or Entourage? – A quick tip on how to get back on track after Microsoft’s recent service changes.
• Bank emails wrong person, sues – When just receiving an email is enough to get you into trouble.

We’ll definitely be adding to this list quickly in the coming week so make sure to check back often if you’re interested in the email space.

Thanks to you guys’ feedback, we’ve now decided to remove the default prefix in ‘Step 3′. That is, in the past, when you’ve made a transfer with YippieMove from, let’s say, Yahoo Mail, all the transferred folders would by default end up under a sub-folder on the destination side named ‘yahoo’. However, as many of you guys pointed out, that is not a preference. Instead, a you would rather see a seamless migration (ie. the old Inbox would end up in the destination Inbox).

For those of you who do prefer to still utilize our ‘prefix’ feature, that is still possible. Simply click on the ‘Bulk action’ text below the folders and select ‘Use a name pattern’. A window will now pop up where you can enter your prefix (eg. some-prefix/$SOURCE_NAME$).

Again, let me reiterate how much we value your feedback. If there’s anything you like or do not like about YippieMove, please let us know!

ESG Front Page

You could spend a few days doing research and browse through various pay-for-the-top-spot top-lists. Another, and much more efficient option, is to check out brand new service named Email Service Guide. With this tool you can find the best possible provider based on your criterions. With Email Service Guide, you are able to answer questions like:

• Who is the cheapest provider that supports IMAP and gives me 8GB in storage per user?
• What providers offers Exchange hosting with 99.999% or more Service License Agreement (SLA)
• Where can I find email hosting powered by renewable energy?

At this point, our database features over 100 different email plans from all of the leading email providers on the market. Email Service Guide enables you to not only find the best provider according to your needs, but it also allows you to compare the matches objectively side-by-side.

Remember that, once you have found the best email provider for you needs, don’t forget to use YippieMove to move your old emails with you.

Since we were unable to take on the job due to the bug, we didn’t pay much attention to First Class and assumed that they would fix the bug in the next release.

Last week, we took the time to install a local copy of First Class to see if they had resolved the problem. Unfortunately the bug still remained in the latest version (9.1). Because of that, we thought it might be appropriate to write a brief blog post explaining the bug.

About the bug

The bug is pretty straight forward. It occurs when a message has a subject line that includes one or more quotation marks (“). The problem is not that a quotation mark is an illegal character according to the RFC’s, but rather that FirstClass fails to escape them ( ‘ ” ‘ should be ‘ \” ‘). Because of this, when we issue the FETCH command, it’s impossible to tell where the different different segments of the respond ends.

To put this in a real example, here’s an example of the FETCH response First Class is sending:

212 FETCH (ENVELOPE (“Fri, 30 May 2008 15:11:42 GMT” “Re: Fwd: lorem ipsum “foobar”” ((“Replace Replace” NIL “replacel” “replace.com”)) ((“Replace Replace” NIL “replacel” “replace.com”)) ((“Replace Replace” NIL “replacel” “replace.com”)) ((“Repl Repl” NIL “repla” “replace.com”)) ((“Rep Rerpla” NIL “replace” “replace.com”)) NIL NIL ““) RFC822.SIZE 5120 INTERNALDATE “30-May-2008 11:11:42 -0400″ FLAGS (\Seen) UID 89423624)

If this was properly formatted, the response should read:

212 FETCH (ENVELOPE (“Fri, 30 May 2008 15:11:42 GMT” “Re: Fwd: lorem ipsum \”foobar\”” ((“Replace Replace” NIL “replacel” “replace.com”)) ((“Replace Replace” NIL “replacel” “replace.com”)) ((“Replace Replace” NIL “replacel” “replace.com”)) ((“Repl Repl” NIL “repla” “replace.com”)) ((“Rep Rerpla” NIL “replace” “replace.com”)) NIL NIL ““) RFC822.SIZE 5120 INTERNALDATE “30-May-2008 11:11:42 -0400″ FLAGS (\Seen) UID 89423624)

While you might not consider this a very serious but, it really is. FETCH is a very important feature in IMAP and more emails than you’d imagine includes a quotation mark in the subject line. Let’s just hope Open Text Corporation will have this fixed in the next release.

After looking back on the humble beginnings of the communication medium that changed the way we see the world today in the article ‘45 Years of Email – From Military Tool to Mass Market Communication,’ let us try to catch a glimpse of where it is heading at.

Authoritative and comprehensive, yet easy to use, portable and instant—these are just some of the reasons why electronic mailing has been a runaway success for the past years. With the unwavering support of its users and subscribers, the use of e-mail has been a web for communication especially for people trying to bridge the gap of distant locations. Thus, the cycle of email seems never ending and innovative as it grows and grows each year settling new and various controls on the world wide web.

As the future of technology per se soars to greater heights and so is the future of electronic mailing in particular. Adding to the fact that latest trends like instant messaging and video conferencing further improved the ways on how people from all walks of life reach out to the world, such widespread penetration can be seen on the millions of people being hooked in using email as a means to connect to people to whatever technological and communicative endeavors they face: whether it be for personal or for business use. Also, a variety of communication choices have emerged over the past few years and each of the options have their own special appeal—this is made possible because of the discovery and further enhancements of electronic mailing.

Just as electronic mailing revolutionized how the world accessed information and communicated through the years, the ongoing development in speed, bandwidth, and functionality will surely continue to cause fundamental changes on how our world will operate for decades to come. So what’s actually cooking now in the world of innovations? When no one could really tell, but we can always speculate in the light of the trends that we observe today. For one, mobility will be the name of the game. We have already seen wireless networking and we are going to see a lot more broadband roaming in the future. This trend has actually begun! Back in January, Novatell Wireless filled a patent or a technology that allows users to flip back and forth between wireless LAN and mobile phone networks, even at high speed.

Rapid-switching technology

With these developments, distance was not the only thing man was able to conquer, even time. Internet and email can be accessed real time, even when traveling—not to mention the impressive speed. This is yet another feat for mankind. The influence of e-mail globalism will generally become the main reason for the stay of e-mail as it continually distributes globally the information and knowledge for the people at lower costs that lift the need of the community for accessible information. Another reason why it will continue to be a popular communication option is because of the emergence of communities that continually unite to provide information networks and different relationships that will be further established in the cyberspace may this be for political organizations or social networking. This will provide increasingly wide choices to individuals who wish to participate in local communities that share their ideals, thoughts and interests. Electronic mailing integration with an increasing number of other technologies is as natural as a musician’s experimentation with notes. These will cause email to become increasingly integrated with phones, televisions, home appliances, portable digital gadgets, and a range of other small hardware and software devices. Hence, e-mail is one core component of an emerging unified messaging solution set as evidenced by convergence of corporate email and instant messaging.

Will email still matter in the coming years? Absolutely! The unprecedented and rapid growth and impact of email with its growing and improving features will constantly evolve and garner significant respects in the future. However there may be some threat to its future use. A concrete example of this is the Google Wave—the freshest and the coolest communication and collaboration tool today initiated by Google. It is not out yet, but it is the most talked about innovation in the world of information technology.

Google Wave demo

With Innovations such as Google Wave, which by the way is expected to be released later this year, it is interesting to see how people will embrace the new features that Google Wave is promising. For sure, it will serve as a global and worldwide virtual and technological resource that will cater to the information and communication needs of the humankind. Furthermore, e-mail applications will not only become better and better to reflect the natural world, they will also have the fluidity, flexibility, and speed of the digital world. E-mail architecture has definitely just begun as the new horizons set to its future needs are being driven by the constant need for humans to interact, to communicate and to connect to every part of the universe.
It’s unquestionable then that surely we will foresee the growing success of email as the primary hub for communication, media and networking in the future. Indeed electronic mailing catapulted the unsettling success of information technology with its’ generatively and innovative character. It has gained a respective mainstream acceptance from the people and this acceptance will be the bread and butter of its’ future and will determine its life span as the provider of secure methods of communication, especially now that such thing as Google Wave is coming out. We could only gauge its success once it is fully launched

Scalix Admin

Ok, let’s get started.

First, install CentOS 5.3 as normal. In the software selection, we only selected ‘Server’ and deselect everything else (why people run X-windows on a server is beyond me). The only other thing to keep in mind is to use a static IP and a fully qualified domain name (FQDN).

Once you got your system up and running, we need to make a few changes to the default setup. Login using SSH or the console and run:

# system-config-securitylevel-tui

You will need to disable ‘SELinux’ as well as configuring the firewall according to your needs.

Next we need to trick Scalix into believing that you are running RHEL instead of CentOS. This is pretty easy:

# cp /etc/redhat-release /etc/redhat-release.orig
# echo ‘Red Hat Enterprise Linux Server release 5 (Tikanga)’ > /etc/redhat-release

We also need to make sure your hostname is listed in /etc/hosts under its IP (and not under 127.0.0.1). An example of a proper line is :

xxx.yyy.zzz.nnn server.example.org server

Now, let’s get started with installing Scalix. We start by upgrading the system and installing a few dependencies.

# yum -y upgrade
# yum install -y tk cyrus-sasl-md5 cyrus-sasl-plain sendmail-cf postgresql-server postgresql-libs mx compat-libstdc++-296

Assuming you’ve already downloaded Scalix onto the machine, all you need to launch the installer

# chmod +x scalix-11.4.4-GA-enterprise-redhat-intel.bin && ./scalix-11.4.4-GA-enterprise-redhat-intel.bin

After you are done answering all the questions, you should be able to access your brand new Scalix installation from your browser.

Good Luck!

Update: As Michael points out, the step of ‘tricking Scalix that you are running RHEL’ is not necessary. Thanks Michael.

Update 2: If you want to take Scalix for a spin without actually having to install it, we have created a Virtual Machine with Scalix installed that can be downloaded here.

“Adding support for Scalix was pretty straight forward. It basically came down to filtering out a few unsupported IMAP flags,” says Alexander Ljungberg, head of YippieMove’s software development team.

With Scalix added to the list of supported groupware, YippieMove now supports migrating emails between all major groupware. Combine that with our Batch Migration service and jumping from one groupware to another could not get easier.

Old meets new: early version of Hotmail displayed in Safari 3

Ever since the dawn of human existence, it has been an instinct of man to try to find ways to reach out and to be able to be heard. Speech and language proved to be the most important tools of the ancient human. Humankind has used these tools to survive and to exchange thoughts; which later evolved into other forms that made use of written symbols, drawings, runes and many other variations. The natural next step in this desire to innovate was the attempts to conquer distance — man wanted to still be in touch besides being apart. Carrier pigeons and smoke signals paved the way for a line of development aiming for an infinite means of communication. Thus came the discovery and birth of technologies like the telegraph, telephone, and ultimately, the e-mail system.

Before we can talk about the creation of electronic mail, we should first understand the beginnings of the internet itself. Many people have probably heard that the Internet began in some military computers in the famous Pentagon, that it was called Arpanet, and that the year was 1969. The theory goes on to suggest that the network was designed to survive a nuclear attack. True enough, the Internet was designed in part to provide a communications network that would work even if some of the sites were destroyed by nuclear attack. In 1969, the US Department of Defense wanted a communication system that could not be destroyed in the event of an emergency. They linked computers over telephone lines so that if one computer failed to work, the others could still communicate with each other. This system was called then as ARPANET.

ARPANET (Advanced Research Projects Agency Networks) was the network that became the basis for the development of the INTERNET. It was developed under the direction of the U.S. Advanced Research Projects Agency (ARPA). In 1969, the idea became a modest reality with the interconnection of four university computers. The initial purpose was to communicate with and share computer resources among mainly scientific users at the connected institutions. ARPANET took advantage of the new idea of sending information in small units called packets that could be routed on different paths and reconstructed at their destination. The main concept in packet switching was the idea of making use of circuits that are switched like in the old type of typical telephone circuit, where a dedicated circuit is tied up for the duration of the call and communication is only possible with the single party on the other end of the circuit.

ARPANET logic map, March 1977 (From Computer Science Museum)

The starting point for host-to-host communication on the ARPANET was the 1822 protocol which defined the way that a host sent messages to an ARPANET IMP. The message format was designed to work unambiguously with a broad range of computer architectures. Essentially, an 1822—message consisted of a message type, a numeric host address, and a data field. To send a data message to another host, the sending host would format a data message containing the destination host’s address and the data to be sent, and transmit the message through the 1822 hardware interface. The IMP would see that the message was delivered to its destination, either by delivering it to a locally-connected host or by delivering it to another IMP. When the message was ultimately delivered to the destination host, the IMP would send an acknowledgment message (called Ready for Next Message or RFNM) to the sending host.

So, how exactly did email evolve from the classic ARPANET? The answer comes from the name of Raymond Tomlinson. Tomlinson, born in 1941, is a programmer who implemented an email system in 1971 on the ARPANet. Email had been previously sent on other networks. Before internetworking began, email could only be used to send messages to various users of the same computer. Once computers began talking to each other over networks, however, the problem became a little more complex—they needed to be able to put a message in an envelope and address it. To do this, they needed a means to indicate which mails go to whom in a way that the electronic posts understood. This is the same as the conventional postal system: they need a way to indicate an address for a particular mail. The AUDOTIN was the first system able to send mail between users on different hosts connected to the Arpanet. To achieve this, Tomlinson used the @ sign to separate the user from their machine, the “commercial at” symbol to combine the user and host names, providing the naturally meaningful notation “user@host”—that is the standard for email addressing today. This has been used in email addresses ever since. These early programs had simple functionality and were command line-driven, but established the basic transactional model that still defines the technology: email gets sent to someone’s mailbox.

The first important email standard was called SMTP, or simple message transfer protocol. SMTP was very simple and is still in use – however, as we will hear later in this series, SMTP was a fairly naïve protocol, and made no attempt to find out whether the person claiming to send a message was the person they purported to be. Forgery was (and still is) very easy in email addresses. These basic flaws in the protocol were later exploited by viruses and worms, and by security frauds and spammers forging identities. But as it developed, email started to take on some pretty neat features. One of the first good commercial systems was Eudora, developed by Steve Dorner in 1988. When Internet standards for email began to mature the POP (or Post Office Protocol) servers began to appear as a standard; before that, each server was a little different. POP was an important standard which allowed users to develop mail systems. These were the days of per-minute charges for email of individual dialup users. For most people on the Internet in those days, email and email discussion groups were its main uses. There were many hundreds of these on a wide variety of topics, and as a body of newsgroups, they became known as USENET.

Raymond Tomlinson

With the World Wide Web, email started to be made available with friendly web interfaces by providers such as Yahoo and Hotmail. Usually this was without charge. Now that email was affordable, everyone wanted at least one email address, and the medium was adopted by not just millions, but hundreds of millions of people. This only proves how emailing has reached new horizons in helping people to connect with the virtual world.

Though it is undeniable that emailing has gone a long way since it was first conceptualized, conceived and born, it now faces more problems than ever. While one cannot question the importance of email and instant messaging nowadays, one question remains: what is in store for electronic mailing in the future? In this age when everyone is aware of the emergence of so many communication options, does email still makes sense? What’s worse, e-mail has become a tool for criminal hackers ready to show off their technical skills. Recently, organized crime has become more of a force in the spam arena. They developed a series of get-rich-quick schemes and have also leveraged spam as an entry point into collecting and then misusing individuals’ personal financial information. As a result, it is estimated that spam represents 80% to more than 90% of all e-mail messages. Consequently, some businesspeople are flocking to these new communications options to rid themselves of the tedious task of constantly hitting the delete button. Can one still trust the reliability of emailing in connecting and communicating with other people?

The answer lies on the platform that emailing has founded. The flaw that makes email so easy to abuse for spam and other nefarious activities is also it’s strength: it’s easy to get an e-mail address and nearly everyone has one. E-mail will continue to be a popular communication option precisely because it is so popular. Emailing continues to be a communications option that generates billions of messages each year.

Yes, new communication channels have emerged, they appeal to different sort of users, and they will be in use at times instead of e-mail. However, e-mail still has the broadest range of supporters and will continue to be the primary communications media for most businesspersons for the foreseeable future; especially now that there is a boom in the online business industry. Email has proven to be an efficient marketing and advertising medium. The increase in electronic commerce or e-commerce has once again pressured the mail developers to improve and better the functionalities of email systems.

For sure things will change and just like all else, e-mail will surely evolve, but its use as a communication medium is still unparalleled. It was after all created on top of technology built to survive.

Ever since the launch over a year ago we used VMware Server 1 for instantiating the YippieMove back-end software. For that year performance was not a huge concern because there were many other things we were prioritizing on for YippieMove ’09. Then, towards the end of development we began doing performance work. We switched from a data storage model best described as “a huge pile of files” to a much cleaner sqlite3 design. The reason for this was technical: the email mover process opened so many files at the same time that we’d hit various limits on simultaneously open file descriptors. While running sqlite over NFS posed its own set of challenges, they were not as insurmountable as juggling hundreds of thousands of files in a single folder.

The new sqlite3 system worked great in testing – and then promptly bogged down on the production virtual machines.

Tough CPU week on a server running VMWare

We had heard before that I/O performance and disk performance are the weaknesses of virtualization but we thought we could work around that by putting the job databases on an NFS export from a non virtualized server. Instead the slowness we saw blew our minds. The core servers spent a constant 70% of CPU time with system tasks and despite an uninterrupted 100% CPU usage we could not transfer more than 400KBit/s worth of IMAP traffic per physical machine. This was off by a magnitude from our expected throughput.

Obviously something was wrong. We doubled the amount of memory per server, we quadrupled sqlite’s internal buffers, we turned off sqlite auto-vacuuming, we turned off synchronization, we added more database indexes. These things helped but not enough. We twiddled endlessly with NFS block sizes but that gave nothing. We were confused. Certainly we had expected a performance difference between running our software in a VM compared to running on the metal, but that it could be as much as 10X was a wake-up call.

At this point we realized that no amount of tweaking was likely to get  our new sqlite3 version out of its performance hole. The raw performance just wasn’t there. We suspected at least part of the problem was that we were running FreeBSD guests in VMware. We checked that we were using the right network card driver (yes we were). We checked the OS version – 7.1, yep that one was supposedly the best you could get for VMware. We tuned various sysctl values according to guides we found online. Nothing helped.

We had the ability to switch to a more VM friendly client OS such as Ubuntu and hope it would improve performance. But what if that wouldn’t resolve the situation? That’s when FreeBSD jails came up.

Jails are a sort of lightweight virtualization technique available on the FreeBSD platform. They are like a chroot environment on steroids where not only the file system is isolated out but individual processes are confined to a virtual environment – like a virtual machine without the machine part. The host and the jails use the same hardware but the operating system puts a clever disguise on the hardware resources to make the jail seem like its own isolated system.

Since nobody could think of an argument against using jails we gave them a shot. Jails feature all the things we wanted to get out of VMware virtualization:

• Ease of management: you can pack up a whole jail and duplicate it easily
• Isolation: you can reboot a jail if you have to without affecting the rest of the machine
• Simple scaling: it’s easy to give a new instance an IP and get it going

At the same time jails don’t come with half the memory overhead. And theoretically IO performance should be a lot better since there was no emulated harddrive.

And sure enough, system CPU usage dropped by half. That CPU time was immediately put to good use by our software. And so even that we still ran at 100% CPU usage overall throughput was much higher – up to 2.5MBit/s. Sure there was still space for us to get closer to the theoretical maximum performance but now we were in the right ballpark at least.

More expensive versions of VMware offer process migration and better resource pooling, something we’ll be keen to look into when we grow. It’s very likely our VMware setup had some problems, and perhaps they could have been resolved by using fancier VMware software or porting our software to run in Ubuntu (which would be fairly easy). But why cross the river for water? For our needs today the answer was right in front of us in FreeBSD: jails offer a much more lightweight virtualization solution and in this particular case it was a smash hit performance win.

Again, we apologize for the inconvenience this may have caused you.