I have a friend who is an attorney who keeps tons of email. He's been having issues with reliablilty on the shared hosting account he is on so I suggested we move him to Google Apps. Then I discovered he has nearly 15,000 email messages stored. A bit of a shocker, but it's Google. They have made migrating to Google Apps a breeze, right?
It has been a complete nightmare. We tried just dragging and dropping folders using his Apple Mail client. It was moving one message every three to five minutes. At that rate it would have taken just about one month to migrate all 15,000 messages.
So we tried Google's tools. Big mistake. The migration tool available on the apps dashboard pretends like it's working, but it doesn't. After 2.5 days, it reported having moved all his email except it didn't. It moved two out of eight folders and only about 8,000 of the 15,000 messages. Nice job, Google.
I called Google support—one of the perks of being a paying apps customer. They suggested I use a Windows desktop tool that is supposed to be better, but I couldn't make it work. It refused to accept his secret key and would bail without doing anything. Once, apparently randomly, it actually got past the secret key check, but then couldn't create any of the necessary labels and quit. I read and re-read all the support docs I could find, but could not find anything that explained why the secret key was being rejected. I tried calling support back, only to have the support rep read back to me the support pages I had already read through.
Next I tried talking to these guys on the advice of the one of the IT guys at work. Apparently we're using them for a big domain migration we're undergoing right now. Supposedly they are the best 3rd-party migration service out there. There's just one catch. Their software is licensed on a per person basis and you have to buy a minimum of 10 licenses. I contacted them and asked if their free trial would work to get one person migrated. They responded that I could buy a license for one person. Except that you can't. The web site won't let you buy fewer than 10. I tried chatting with someone on their web site to see if I was missing something. He told me the minimum was 25. Do they not even know how their own site operates? I contacted them by email again saying I couldn't see how to order just one license. I explained that I was happy to pay for one person, but I didn't want to pay for 10 if I only needed one. Their response? Nothing. So, if you're small potatoes, don't waste your time.
I had some PHP code lying around from a time in the past when I needed to migrate email from one server to another. I didn't use it then, but decided that was my next best option. Problem: it was apparently some seat-of-the-pants coding. It was mostly broken and got some things flat out wrong. So after several hours and much trial and error, I finally got it working. Seemed to work like a charm. The folder I was testing with had about 1200 messages in it and moved to Gmail in just a few minutes. New problem: in the message list view, all the dates were being displayed based on the date the messages were moved, not the date of the last message in the conversation. Even though you can specify an "internal date" in the PHP funciton call—which some seemed to think should do the trick—it wasn't working. Even with the date parameter set and properly formatted, Gmail kept right on displaying the date of the copy, not the date of the last message. To be clear, the actual dates on the message were not affected. You could open the message and see the correct date, but that's just annoying.
This morning, I started Googling for a solution to the date problem...ironic isn't it? Apparently this has been an issue for years. This thread was started in 2009. TL;DR: use Thunderbird. So after weeks of fussing with Google's migration tools, several support calls to Google, and a failed attempt to engage with a third party, I fired up Thunderbird, pointed it at both accounts and in less than a day I have all 15,000 messages moved to Gmail. Apparently not all IMAP clients are created equal. For whatever reason Thunderbird plays nicely with Gmail.
What really bugs me about this whole thing was the long, painful process. It wasn't until I tried to write my own code, ran into an issue with my code and started looking for a solution to that specific problem that I finally found a workable solution. Why, when I spoke to the first support rep at Google about migrating this enormous quanity of email, didn't he just say, "Hey, our tools suck. Just use Thunderbird. You'll be much happier."
Hopefully, some day in the future, someone else will be trying to migrate to Gmail and will find this post sooner rather than later in his or her process.