01273 264503 info@mootpoint.org

Data Migration

As the web has matured, the need to migrate data from one system to another has become more obvious. You may be on your second or third website, and familiar with the pain of transferring content from the old to the new. Perhaps you cut and pasted page content from a Word document into a CMS editor? For sites with substantial content, some sort of automated data migration is essential to reduce the resources required to move to a new system.

Data migration can often be achieved by exporting the content from the old system as a CSV (comma separated values) file which can then be imported into the new. This process can vastly reduce the man-hours required to transfer site content, although care must be taken to transfer meta data such as categories, tags, post dates accurately. There may also be differences in data format between the 2 systems, for example dates may be stored in a different format, or text encoding may be different.

There are often plugins or modules available to help get data in or out of a CMS, and these can be very useful. Sometimes it is simplest to export directly from the database itself, particularly when retrieving meta data like content categories.

Data Integration

A related requirement is to connect the data from multiple systems, linking databases together. Some common data integration requirements:

  • Integrating warehouse inventory levels with the ecommerce web site to provide accurate stock levels online
  • Publishing content from a back-office intranet onto a web site
  • Connecting the staff area of a website to gather timesheet information and integrate with a payroll system
  • Integrating ecommerce orders into a fulfilment process at a warehouse

There are several approaches to solve data integration problems:

  • The most popular method of connecting disparate systems is via an API (application programme interface) which allow authenticated connections to transfer data via REST or SOAP.
  • Sometimes, albeit rarely, databases can be directly connected i.e. both systems can read from the same database.
  • It is sometimes possible to set up replication between 2 databases so changes in one are immediately reflected in another. This process can be one-way or two-way.
  • Sometimes it is necessary to set up a scheduled automated process where data is exported in a common format (often XML), transferred to the web site via FTP, and imported into the site.

Find Out More

Moot Point has extensive experience in data migration and integration. Find out how we can save you time and money.

Recent Data Wrangling Projects