Don’t get stuck in the manual migration abyss

Many archive platforms provide a set of native utilities that allow some form of export and import of messages via PST, EML or MSG files. These inbuilt toolsets have given even the most competent IT managers a feeling of empowerment which unfortunately is very short lived. We regularly get asked by our customers “why shouldn’t we use the native tools to complete our migration?”. Depending on the environment, the reasons are many and varied. From slow performance and the inherent stability issues with large PSTs, to poor user experience and the risks of human error. However the real problems stem from a few of the never ending processes that these customers will generally find themselves in. More on this later.

First let’s squash the idea that EML or MSG is viable way to migrate. Unless you are doing a migration of a journal mailbox, using either of these targets to complete your migration will likely cause your users a lot of issues. When these items are exported, folder structure will not be retained which means that once this mail is finally in your new system, users will not be able to find their emails in the location they would have expected. If in fact an export of the journal archive is being performed, fields like BCC and other memberships of distribution lists will be lost (also the case when exporting to PST).

The key issues you will face when trying to export and import PSTs using the native utilities are;

  • Large PST files are often required due to large source archives. Such files are prone to corruption
  • PSTs are space-inefficient
  • PST extraction with native tools is VERY slow
  • Needs constant monitoring
  • No error management
  • Loss of important compliance data (e.g. BCC fields)
  • No auditing or chain of custody
  • No ability to handle shortcuts

If none of the above points are a concern and you still wish to proceed then we should circle back to those never ending tasks.

Exporting the data to PST

I’m sure, like most businesses, your archive has probably been around for a few years, has had a couple of minor issues and gone through a upgrade or two.  Chances are that there will be corruptions in your archive (interesting blog about silent data corruption). This isn’t necessarily a big issue, except that your archive wasn’t designed to deal with data corruptions and, when it is trying to export the corrupted emails to PST with its native tools, it’s going to fail. Logic dictates that it will try and process the corrupted item and when it can’t it will move onto the next one. Invariably not. This logic seems to fall over with pretty much every archive platform we have worked with. What will happen is that you are going to find yourself tracking down this corrupt item and then trying to find a resolution. In all likelihood, the item is corrupt and there is nothing you can do. If you are lucky enough, you will be able to remove the corrupted item and get your archive platform to reindex the data you can start again. Just be aware you will have to start the export again from the beginning, and chances are this will not be the only corrupt item that you will face, at which point you will repeat the above process again and again and again……….

Importing the PST

Once you have managed to get through the export process and you are getting ready for the import, you should take a holiday so that you can come back prepared for the same perpetual issue. In some cases the export process doesn’t do any form of data integrity checking, so when it writes the email to the PST some of those corrupted emails do manage to sneak through. In others, your users archive was so big that when it was exported to a large PST which becomes unstable and subsequently corrupt. As with the export, the native tools for importing generally work on the assumption that the data is clean (not corrupted) and thus there is very little integrity checking, so when those pesky corrupted messages get imported, the process will hang. Not only are the reporting mechanisms poor at assisting you to find which message has caused the hang, they also are terrible at even letting you know that they have hung. Once you have worked out that the process has stopped you have two choices – try and find the message that is causing the problem or run the export and import again and again and again……….


Because of these two frustrating aspects of a manual migration, you will find yourself stuck in a third perpetual task, monitoring. The single biggest bane of any IT admin is manual monitoring, constantly having to log in and check that things are actually occurring as you expect and require. Like Homer Simpson, you are going to find yourself distracted by gadgets and gizmos and you will miss the failures around you.

Now my intention wasn’t to be doomsayer, but rather prepare you (or the poor individual you are going to instruct to do this work) for the constant head banging process that will be undertaken when using native tools for migrations.

Simon Alit > Data Migrations specialist

Simon Altit

Director – EMEA





+61 433 232 349

+44 7403 599 817


1 reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *