Document Migration

The challenges of content governance are constantly evolving as the volume of digital content published increases exponentially every day and new publishing channels emerge. Who, 12 months ago, for example, could have predicted that Twitter would become such a publishing phenomenon?

Vamosa understands this complexity and we know that dealing with the end-to-end life cycle of content in separate parts can make implementing an effective strategy for managing enterprise content more complex.

In response to enterprise needs, coupled with our understanding of the obstacles to enterprise-wide content management, we have launched two configurable software platforms that incorporate this end-to-end approach to content: Vamosa Enterprise Content Governance (ECoG) Suite for Documents and ECoG Suite for Web. 

ECoG Suite for Documents

ECoG Suite for Documents has been designed to enable the transformation of document repositories into clean, usable content stores that can be governed by ECoG policies. The suite automates the life cycle of documents, from creation to on-going maintenance, supporting taxonomy and metadata management, versioning, policies for records management and archiving, and so on.

ECoG Suite for Web

ECoG Suite for Web allows clients to take back control of web content by automating policies for all stages of the content lifecycle, from accessibility to tagging and from broken links to ECMS migration, making it findable, compliant and more usable. By adopting a SaaS approach to web maintenance, web properties are monitored 24/7 so any breach of policy is trapped and resolved.

Most importantly for enterprises, the suites offer all the functionality previously available in separate Vamosa products, including analysis, data cleansing and migration but bundled into a single installation that is then configurable to the customer’s environment and specific ECoG policies.

If you use a Document Management System (DMS) to manage document based content you will already know that one of the key issues faced is how to move existing content into the new system.

On paper document migration looks relatively easy. A team of people could spend the next six months copying files and documents from their existing location into the DMS. However in order to achieve an effective document migration project it is essential that you consider:

•    How many files can one person move in one day? A document migration project of only 100,000 pages might take anywhere between 10 to 100 days to complete.
•    What happens during this move? Do you place a content freeze over all of the files and documents for the duration of the migration project? What about moving attached documents and handling internal and external links? How can you incorporate this functionality into your document migration strategy?

A document migration does not simply mean changing a few attributes and then placing the content into a slightly different information architecture. Document migrations involve the wholesale change of properties, storage, information architecture and content lifecycle, therefore document migration projects tend to involve hundreds of minor changes to very large volumes of data. This can potentially become incredibly time-consuming and frustrating. To reduce this confusion and to improve data quality, eliminate redundant, obsolete and redundant information and match the requirements of your new system, you really need a tried and tested method.

Vamosa Consultancy Practice, with their collective experience gained on over one hundred client projects, can assist in implementing best practice content quality processes; such as when selected to undertake the migration of the website and corporate Intranet for the UK’s Department of Environment, Food and Rural Affairs (Defra).

Working with integration partner, digital media giant Euro RSCG, Vamosa’s solutions and expertise was the obvious choice to deliver a web and Intranet content migration for Defra.

Gregory Roekens, Technology Director at Euro RSCG said ‘We were asked to provide a best of breed and best value solution to Defra’s business content migration problem and Vamosa’s tools and methods have the reputation as the Central Government standard’.

Vamosa Consulting was able to successfully bring control and structure to all documents across Defra’s content stores, providing, de-duplicated and clean content. As a part of an ongoing Enterprise Content Governance strategy, Vamosa ECoG Suite for Documents also uniquely pinpoints breaches of policy for all document governance areas, and offers an automated process to resolve each breach, allowing document stores to be kept up to code.

Organizations use SharePoint for a variety of things, from Intranets to Extranets, from Customer Portals to document management and for team collaboration.

Recently there has been a significant amount of discussion around exactly what SharePoint is and what it offers. For example, a recent AIIM article highlights 8 things SharePoint 2010 needs to be a true ECM system. A blog by the Microsoft Team describes SharePoint 2010 in 1 sentence, 8 categories and 40 features. The Rez's SharePoint blog provides a comprehensive comparison of features between SharePoint 2007 and SharePoint 2010.

One element thats there has been great excitement around, is the SharePoint 2010 taxonomy implementation and management across sites and site collections. Organizations must spend time and effort building the appropriate foundation for utilizing the content taxonomy. That is all well and good when a green field project is started, but considerations must be given to data that is already stored in previous versions of SharePoint; how can that information be classified correctly when undertaking a SharePoint Migration? Microsoft does not provide a tool or utility which can be used to apply a new taxonomy and as a result many organizations may become stuck during a MOSS migration.

Vamosa recently announced a partnership with MetaVis to help both existing and new customers get a better ROI and increased value when moving into SharePoint 2010 from SharePoint 2003 or SharePoint 2007 environments, by ensuring the new taxonomy values can be applied to documents.

The MetaVis Suite does not simply help organizations migrate their existing content into the SharePoint 2010 platform; it also provides features such as  a graphical top down view of the web estate and it highlights dependencies between objects. This provides the administrator with a complete view of how their sites are structured. The 'Live Compare' feature of MetaVis provides useful functionality as it allows comparisons of two sites to be undertaken in real-time, listing the differences between sites at the granular level of identifying column differences between content types.

Synchronizing two SharePoint environments at the click of a button is only one of many features offered by Metavis that will help SharePoint Administrators perform their day to day tasks more effectively and efficiently, while ensuring their organization has an effective SharePoint Governance strategy.

MetaVis Migrator for SharePoint allows to use filters to copy content to new location or to tag existing content setting new values for the existing fields. These properties enable an interesting technique to perform a data migration strategy in a slow pace – at the same time as preparing new environment, training users or organizing the existing content.

Sometimes copying data from one SharePoint site to the other one is a quick process that could be done by copying the whole list or library from one place to the other one. However, in other cases this is more complex routine involving copying different items in different locations or migrating portion of data in one day and other portion of data in the other day when users of this content are ready to use new environment. In this later case it could be useful to use combination of some artificial field marking the migrated content in the source site, MetaVis Migrator copy filters and SharePoint views hiding the content that was already migrated to the destination site.

The idea of this technique is simple. First, we will create an artificial Yes/No field in the source list that will indicate that an item is copied to the destination site. By default this field will be “No” for all items / documents in the list.


Then we will go ahead with our SharePoint migration activity selecting portions of data to copy based on some filter condition. The filter will always include the condition to exclude the content which is already migrated based on the state on our artificial columns that we created earlier. Below in an example of such a filter (this time we copy all documents with MK Region equals to Netherlands):


After the copy is completed we use the very same filter that we just used to copy content but this time – to tag selected source data setting our artificial Migrated field to Yes – indicating that this content has been already copied to the destination site. Using the same filter will ensure that all items just copied using this filter will be marked as Migrated.

The fact that all copied content has been marked with certain criteria could be used in a variety of ways. First use is to skip the content which was already copied before when performing new copies to the other places. The second use it to create list view that will hide copied content from the viewers. Below is the example of such a condition in a list view.


Having this condition enabled for the list view will point out that the content is already located in the new destination.
















To learn more about how Metavis MIGRATOR can help you simplify yor SharePoint migration, download the MetaVis Migrator Business Results Sheet.

Content Provided by MetaVis for Vamosa

As noted in a previous post  by Nic Archer it is anticipated that a third of organizations will in time employ a MOSS migration strategy and migrate to the new SharePoint 2010 platform. However in order to maximize organizational efficiency it is essential that all data is prepared prior to the SharePoint 2010 deployment to ensure a smooth content migration strategy.

By defining efficient content taxonomy structures prior to deployment, organizations are able to more easily move content into their new SharePoint environment resulting in improved site architecture and navigation within the new site. This task in itself can involve a significant amount of work before data is ready to be migrated.  If this stage is not meticulously carried out, the value of the content in the new store will be significantly depleted.

The suite of MetaVis tools for SharePoint, now allows SharePoint administrators to reclassify content by assigning new metadata values and new content types during a migration, allowing organizations to migrate more efficiently. Even post migration, MetaVis allows administrators to bulk update metadata, should this be required, further enhancing the findability of data for the user.

With MetaVis, SharePoint administrators can utilize the new Term Store, to graphically re-design and re-architect their content, significantly improving search and discovery of local documents for organizations that are located globally.
By utilizing the capabilities of the new SharePoint environment through MetaVis, administrators will be able transform the web experience, lower the cost of ownership of content management and deliver error free automated migrations, while creating effective content structures in SharePoint 2010 – the ultimate goal for all SharePoint administrators.

To learn more about the suite of MetaVis tools download the MetaVis Architect Suite Buisness Results Sheet.

As an experienced Consultant, I find it’s difficult to say when considering a data migration strategy which step in the process is most important. The success of a data migration methodology really depends on all the components of a solution working well from beginning to end. But it’s certainly true that a successful data migration project cannot take place without a robust means to push content into its new home, whatever that might be.

Since virtually every content management system (CMS) on the market is different, there is no silver bullet for loading content quickly and dependably. Each application programming interface (API) is different and can vary greatly in terms of quality style and completeness. Some may require a custom web service, deployed on the target environment and called remotely.

But this solution isn’t quite optimal. What if the client’s target environment is completely inaccessible for some reason? Perhaps the client’s security model forbids deploying foreign services. Microsoft’s SharePoint 2010 CMS circumvents the necessity to deploy remote services with its client object model. After getting your hands on the required libraries the SharePoint 2010 API is suddenly at your fingertips. Using this technique, a data migration can be accomplished using a locally deployed custom service after supplying the required credentials!

While I found SharePoint’s client object model to provide a promising new way to connect to a CMS, I thought the API was incomplete and sometimes poorly documented. Luckily, the out-of-the-box web services packaged with MOSS provided the methods I required. I am excited at the prospect that more CMSs will start packaging up libraries that provide the tools necessary to connect to an environment with a remote machine. It simply provides a safer solution for the data migration and one that doesn’t require deploying anything on the client’s machines! The big upshot of the client object model implies that projects are less likely to face resource bottlenecks because additional access to secure systems is not required. A smaller gap between the development and testing periods allows more time for refinements and a better quality data migration solution.

It seems that Microsoft is leading the way in this regard.

  Download our Data Migration - Seven Steps to Success White Paper to gain a further understanding of the data migration best practices that should be considered when beginning a migration project.

In short: when its in a zoo… Bare with me. Common Eland in Zoo

A document is a record of something that has been observed. Such a record can be anything – a utility bill, an employment contract, a sculpture in a museum, or a painting on a wall. All are examples of documents describing something else. The utility bill records and describes your usage of gas and electricity. The employment contract records and describes the details of the handshake you gave at that final job interview. The sculpture or painting documents – there’s that word – a historical event. All these examples of documents are records of something observed by something or someone else.

Paul Otlet (1868-1944), the father of information science, is known for his observation that documents could be three dimensional. As examples of such “documents” Otlet cites natural objects, artifacts, objects bearing traces of human activity (such as archaeological finds), explanatory models, educational games, and works of art.

Suzanne Briet (1894-1989), also known as “Madame Documentation”, states her case through the enumeration of six objects:

•     Is the star in the sky a document? No.
•     Is the photo of the star in the sky a document? Yes.
•     Is the stone in the river a document? No.
•     Is the stone in the museum a document? Yes.
•     Is the antelope in the wild a document? No.
•     Is the antelope in the zoo a document? Yes.

Suzanne Briet rules: an antelope running wild on the plains of Africa should not be considered a document. But if it were to be captured, taken to a zoo and made an object of study, it has been made into a document, it has been made evidence. So there is a process involved in making “the something” into a document – we call it documentation.

As humans, we’ve invented all kinds of devices to aid in the process of documentation: library cards, folders, URLs, bibliographies, tags, taxonomies, reference documents. They form part of the discipline that is documentation and the basis for content management.

With the advent of content management systems we seem to have lost some of the high-level abstract concepts that were clearly laid out in the early parts of the 20th century. As an industry sector, involved in content management, we’ve become too focussed on the implementation details of content management systems and the limitations that these systems face.

Context

“What is metadata? What is a document?” These questions typically go hand-in-hand and are often naively answered by: “the document is a file or a blob that is stored in database but is difficult too manipulate, so the metadata, table rows and columns, are used to facilitate manipulation and describe the document”.

Metadata provides context with which to consume the document. You’ll have seen this in a zoo. You walk up to the antelope enclosure and there’s plaque containing the name, Latin name and a map of the world with a particular part of Africa highlighted describing the antelope and its origin – metadata. The zoo is giving you context with which to understand the antelope document.

The same holds true for documents in a content management system. Documents are stored in a particular context described by their metadata. The folder, the author, draft/publish status, tags, taxonomy are all pieces of metadata to aid the consumer in consuming the document.

That consumer may be the content management system itself as it responds to the query “give me all documents in the /marketing folder” on behalf of a web visitor. The consumer can also be a records management system archiving documents “in a published state and that are older than 24 months”.

Documents never exist without metadata, without context. For example, the print-out of sales figures that I’ve thrown in the wastebasket is a fully-fledged document of our company’s sales figures telling the person that picks it out the wastebasket to treat (read “consume”) the document as a discarded document.

I’ve seen this catch people out on a few content migration projects when they try and de-duplicate content repositories. They classify documents as duplicates based on their contents alone, without ever taking context into account. De-duplication is tricky business because in doing so you are destroying metadata that is right-or-wrongfully been created to help consume documents.

The accurate consumption or manipulation of documents is intrinsically tied to the accuracy and completeness of their metadata. Is the print-out of sales data in the wastebasket to be trusted? Is the sales data accurate? How should the reader consume the document? Look at the metadata! Its in the wastebasket. This opens up the possibility: did I mean to throw the print-out in the wastebasket? Is the metadata accurate? The reader can only make that decision with more metadata. The reader could phone or email me and ask: did you intend to discard that print-out? Thereby creating more metadata and a better context with which to consume the document.

Content management systems merely store metadata, human beings create metadata – often by hand, sometimes using automated tools. The process of generating metadata or maintaining its accuracy is a human process. Computers don’t care about accuracy or completeness.

Adriaan Bloem, analyst at CMSWatch, touches on this by labeling enterprise search as a “brute force” approach. Adriaan also points out that metadata or context is neccessary to communicate. He’s right – otherwise how do we make sense of a document ?

What if metadata contains a document, i.e. when one document describes another? Doesn’t this form of reasoning collapse in on itself?

What if you took a photograph of the antelope and attached it to the information plaque outside the enclosure? So when the antelope is having an off-day and its hiding in the undergrowth, passers-by can still learn about it by reading the plaque. Now you’ve got one document (the photograph) describing another (the antelope), haven’t you? Aren’t both documents? Wrong.

We can describe documents with other documents. Suzanne Briet would argue that the antelope in the zoo is the primary document and any scholarly articles written about it are secondary documents. They provide context around the primary document. There’s is a document and there is context with which to interpret that document – metadata. Nothing else. Document… Metadata… Document… Metadata.

In an English language sentence “things” can be both subjects as well objects, yet can’t be both at the same time. In one situation the photograph is a document, described by metadata from a digital camera (exposure, shutterspeed), in the other situation it is metadata describing the antelope.

Confused? What is metadata ? In any given situation, ask yourself what the document is and by exclusion all that isn’t is metadata.

So what does this means for content management systems ? Are they all broken? Do we need metadata management processes as well as content management processes? Do we need a separate metadata lifecycle to run alongside a content lifecycle ?

The answer to those questions is unfortunately – yes. Yes, we do need separate metadata management processes. Yes, we do need a separate metadata lifecycle. Unless… we stop building content management systems in the naive fashion of blobs for documents and table rows and columns for metadata. We need to start building these systems so that there is no technical distinction between the content store and the metadata store. Having separate stores for content and metadata causes us to duplicate our efforts, causing us to define duplicate processes to support the lifecycle of both document and metadata.

Ironic, since a promise of content management is the removal of duplication.

In part 1 of the series we highlighted some of the steps that can help you obtain maximum value from your content. In this issue, we will further explore some of the solutions that will allow you to overcome the business challenges associated with the digital landfill.

1) Make Sure Your Content isn’t Duplicated

While reducing the noise on your content store is hugely beneficial you need to go one step further; you must ensure that there are no duplication issues.  A Swiss Bank client once told me that attaching a PowerPoint presentation of 1Mb to an email within that bank increases corporate storage requirements by at least 1 GB and produces on average ten versions of the file.  In one search, a single document could exist in 100 different places. That single fact alone highlights how difficult it is to sort out the knowledge from the noise. By de-duplicating your content, you’ll already be streamlining the process. Anecdotally this can reduce the file count by between 50% and 80% – significant in anybody’s language.

2) Find the Holes within Your Knowledge

Look for existing entities in your classification that do not exist within your metadata – the information you have about your content. Holes provide you with the ability to clean information, ensure metadata management is considered by content creators; and if it’s not to get those content creators in line, by setting out content complaince guidelines. That way you’ll be able to ensure content is relevant, findable and clean.

3) Share Knowledge

Content Management Systems work optimally when the content that they are custodians of is focused, relevant and classified and as a result this content compliance issues should be addressed. You need to assess how your content is being consumed and with this information drive your choice of platform as required. When undertaking a data migration strategy, for example emails, a real opportunity exists to be selective about where you store content. One customer was migrating 120,000 user email accounts from Lotus Notes to Microsoft Exchange. When discussing the migration, it was identified that 70% of the storage requirement was generated by attachments. Moving all of the attachments to SharePoint instantly enabled ‘share-ability’ and de-duplication. You don’t need to follow the obvious route when sharing knowledge, rather choose a platform based upon what your business requires.

4) Maintaining Your Knowledge Assets

It is essential to keep on top of your knowledge. If you allow your content to become chaotic, it’s not easy to repair; if you manage it on a weekly basis, maintenance is much easier. Once your content is at a point of usefulness, where it is relevant and findable, keep it there. If you don’t’ look at it for a year, you’ll then see some major gaps in the metadata, incorrect naming conventions, incorrect storage locations etc. If you invest in a pragmatic care and maintenance program, then spotting minor deviations from the documented standard will be easy to monitor, highlight and resolve, allowing you to continually obtain maximum value from your digital landfill.







To learn more about getting the most from your digital landfill,
download the Five Steps to Discovering the Real Shape of your Digital Content White Paper.

In a CNET Q&A with Microsoft Business Division President, Stephen Elop was asked how Microsoft will get businesses to upgrade from the older version of SharePoint they’re already using and ‘in a nutshell, what’s Microsoft’s pitch to large businesses this time around?’ Stephen replied by discussing productivity across the PC, the phone, and the browser and the focus on making sure that people can work better together, taking advantage of the social-networking capabilities, the rich cloud technologies and a variety of other things.

He was also asked about the specific features that the average productivity worker will get with Office 2010 that they couldn’t get in either Google Docs or a prior version of Office. Elop commented that for users of Microsoft’s Outlook product for e-mail and various forms of communication, there are all sorts of capabilities that have been introduced to help people more effectively manage their communications, whether it’s ignoring threads of communication that are annoying, whether it is the ability to work with advanced conversation views to deal with complex communication patterns, or the interoperability established between Outlook and a number of the social-networking environments, such as LinkedIn, MySpace, and Facebook.

The benefits of SharePoint are certainly there and according to Forrester, a third of big enterprises plan to upgrade to SharePoint 2010 within a year, and that in turn will drive the corporate success of the Office 2010. But while Stephen is thinking of ways to convince enterprises to buy, he’s not thinking about or being asked about how Microsoft suggests large enterprises conduct a MOSS migration and migrate all of their content including documents to SharePoint 2010.

Migration is often viewed as an obstacle to enjoying the full benefits of a new product, but it needn’t be and Microsoft should recognise this early or face low adoption.

There are a few steps that need to occur prior to the upgrade to make it smooth and ensure it is a success.  These include content analysis to understand exactly what exists and a process to clean that content up. SharePoint 2010 is being received well, but it won’t solve many of the issues faced by enterprises unless a clear content migration strategy for upgrade is adopted and the required process of content migration is carried out properly.







MetaVis MIGRATOR is a simple tool for migrating content and objects between SharePoint sites, site collection or servers. Whether you are consolidating, upgrading or simply re-organizing your SharePoint environment. Download the MetaVis MIGRATOR business results sheet of have a free trial.

Content Migration is complex; it is not simply a case of ‘lifting and shifting’ content from one content management system (CMS) to another. Web content is often inconsistently structured and spread across multiple locations and sites.

With this in mind, there is a growing requirement for content to be analyzed, enhanced and standardized prior to migration. The cost involved in manually reviewing and migrating content can be significant; but it can be minimized through the use of software to perform automatic analysis and migration – a solution recently provided by Vamosa for the Department of Health and Ageing of Australian Government (DoHA).

DoHA needed to migrate from their legacy CMS to IBM WCM within just eight weeks in order to achieve better control of their internet sites. Vamosa recommended a solution that involved an automatic cleanup and enhancement of their content. This process firstly involved identifying all of the content that was required to be migrated.

Using Vamosa Content Analyzer Vamosa Expert Services gained a clear understanding of DoHA’s content inventories and content management activities. The results showed that DoHA had 40 static websites containing 50,000 ‘must have’ pages that were to be migrated.

Vamosa Content Migrator then extracted all of the required content, copying it into the staging repository while allowing business to continue as normal for all DoHA’s employees. The next stage of the process involved exposing all of the content to Vamosa’s rules engine, Vamosa Content Quality Builder, allowing all of the content to be modified both for business requirements and to satisfy the requirements of the target system. The content was then loaded into the target system already ‘fit for purpose’ and ready for productive use.

Vamosa Content Migrator was used to simply, automatically and quickly migrate the required web content and linked documents to IBM Lotus WCM in the stated timescales. The Vamosa toolset in the migration process, compared with the manual alternative, lead to the project being completed

    * Four times faster
    * At a quarter of the price
    * With zero impact on day to day operations

In Part 1 of this series, we highlighted the hidden challenges of merging digital content in the context of an acquisition. Acquired content often undercuts unsuspecting organizations by delivering blows across a range of exposed areas: from the content itself, to the technology on which it is served, to its audience – whether employees or customers. In this issue, we explore some of the solutions Vamosa provides to help organizations overcome these integration challenges and achieve their acquisition objectives.

Branding of Acquired Properties

The most obvious way in which acquired content negatively impacts an organization is by eroding its corporate identity. This is most apparent in the case of branding assets: logos, color palettes, and font choices, but many more signals of incomplete integration lurk below the surface. These signals may be acute but unobtrusive – contact email addresses pointing to pre-acquisition domains, obsolete product names – or subtly pervasive – material at sharply different reading levels, non-compliance with adopted accessibility and web standards. Through a combination of services and technology, Vamosa allows organizations to close the brand gap and ensure brand governance standards are adhered to. Taking advantage of Vamosa’s policy-driven rules engine, our Consultancy practice can design and configure a tailored package of content policies using Vamosa's ECoG Suite for Web, precisely targeting an organization’s most pressing content issues.

System Consolidation

It’s easy for companies to make a connection between public content and sales, but the burden of supporting post-acquisition content has deep implications for costs as well. While there are clear – and important – differences between content management systems, all are designed to facilitate the flow of information in a collaborative environment. That’s fine as a concept using one CMS but when you have multiple Content Management Systems – be careful, it can actual restrict collaboration. Reducing the number of content management systems required by the combined organization benefits both production and consumption; at a business level, this translates into elimination of sources of waste: licensing fees, hardware, lines of application code. Vamosa’s ECoG Suite for Web – with or without the deployment of our Consulting practice – allows organizations to accurately size their potential savings through systems consolidation, and then achieve them through migration into a unified platform. When decommissioning systems is not an option, Vamosa’s ECoG Suite for Web can apply metadata dimensions to content in place, enabling portal integration to make content findable or push it directly to relevant consumers.

Users of Content – Enabling Access

Lastly, acquisitions yield major challenges for the users of content. In the context of restructuring a company, it’s common for content ownership to change as departments are split and merged, and much of the content itself – internal HR documents, mission statements, functional group sites – is likely to become redundant or irrelevant. At an access level, reorganization manifests itself in changes to security groups. People often focus on security’s role in preventing information from getting into the wrong hands, but it’s just as important to ensure that new employees are quickly granted access to company information; neglecting this basic privilege is likely to precipitate a morale nosedive. Vamosa’s family of products and services allow business to quickly identify and eliminate swaths of redundant content while at the same time updating links to point to their corresponding active pages. Additionally, Vamosa’s ability to reassign content to new owners or groups ensures that information is editable by and available to the proper channels, breaking down barriers to collaboration and empowering an organization to be greater than the sum of its parts.








To learn more about CMS consolidation and CMS Migration, down the Content Migration: Seven Steps to Success White Paper.