Data Migration Best Practices

If you use a Document Management System (DMS) to manage document based content you will already know that one of the key issues faced is how to move existing content into the new system.

On paper document migration looks relatively easy. A team of people could spend the next six months copying files and documents from their existing location into the DMS. However in order to achieve an effective document migration project it is essential that you consider:

•    How many files can one person move in one day? A document migration project of only 100,000 pages might take anywhere between 10 to 100 days to complete.
•    What happens during this move? Do you place a content freeze over all of the files and documents for the duration of the migration project? What about moving attached documents and handling internal and external links? How can you incorporate this functionality into your document migration strategy?

A document migration does not simply mean changing a few attributes and then placing the content into a slightly different information architecture. Document migrations involve the wholesale change of properties, storage, information architecture and content lifecycle, therefore document migration projects tend to involve hundreds of minor changes to very large volumes of data. This can potentially become incredibly time-consuming and frustrating. To reduce this confusion and to improve data quality, eliminate redundant, obsolete and redundant information and match the requirements of your new system, you really need a tried and tested method.

Vamosa Consultancy Practice, with their collective experience gained on over one hundred client projects, can assist in implementing best practice content quality processes; such as when selected to undertake the migration of the website and corporate Intranet for the UK’s Department of Environment, Food and Rural Affairs (Defra).

Working with integration partner, digital media giant Euro RSCG, Vamosa’s solutions and expertise was the obvious choice to deliver a web and Intranet content migration for Defra.

Gregory Roekens, Technology Director at Euro RSCG said ‘We were asked to provide a best of breed and best value solution to Defra’s business content migration problem and Vamosa’s tools and methods have the reputation as the Central Government standard’.

Vamosa Consulting was able to successfully bring control and structure to all documents across Defra’s content stores, providing, de-duplicated and clean content. As a part of an ongoing Enterprise Content Governance strategy, Vamosa ECoG Suite for Documents also uniquely pinpoints breaches of policy for all document governance areas, and offers an automated process to resolve each breach, allowing document stores to be kept up to code.

As an experienced Consultant, I find it’s difficult to say when considering a data migration strategy which step in the process is most important. The success of a data migration methodology really depends on all the components of a solution working well from beginning to end. But it’s certainly true that a successful data migration project cannot take place without a robust means to push content into its new home, whatever that might be.

Since virtually every content management system (CMS) on the market is different, there is no silver bullet for loading content quickly and dependably. Each application programming interface (API) is different and can vary greatly in terms of quality style and completeness. Some may require a custom web service, deployed on the target environment and called remotely.

But this solution isn’t quite optimal. What if the client’s target environment is completely inaccessible for some reason? Perhaps the client’s security model forbids deploying foreign services. Microsoft’s SharePoint 2010 CMS circumvents the necessity to deploy remote services with its client object model. After getting your hands on the required libraries the SharePoint 2010 API is suddenly at your fingertips. Using this technique, a data migration can be accomplished using a locally deployed custom service after supplying the required credentials!

While I found SharePoint’s client object model to provide a promising new way to connect to a CMS, I thought the API was incomplete and sometimes poorly documented. Luckily, the out-of-the-box web services packaged with MOSS provided the methods I required. I am excited at the prospect that more CMSs will start packaging up libraries that provide the tools necessary to connect to an environment with a remote machine. It simply provides a safer solution for the data migration and one that doesn’t require deploying anything on the client’s machines! The big upshot of the client object model implies that projects are less likely to face resource bottlenecks because additional access to secure systems is not required. A smaller gap between the development and testing periods allows more time for refinements and a better quality data migration solution.

It seems that Microsoft is leading the way in this regard.

  Download our Data Migration - Seven Steps to Success White Paper to gain a further understanding of the data migration best practices that should be considered when beginning a migration project.