Skip to Content

Why RPA is the perfect tool for fast, high-quality data migration

Capgemini
2019-10-30

Robotics or Robotic Process Automation (RPA) is still one of the hot topics when it comes to automating established processes. Regardless of functional scope, type of company or industry specifics, automation of daily operations is almost always at the heart of the RPA initiative. One-off activities, such as data migration and test data generation are often neglected although RPA may be particularly helpful for these purposes due to its ability to effortlessly connect systems without the need for time- and cost-intensive programming of interfaces and stand-alone applications.

Typical migration challenges

Migrating data between systems is neither a new challenge nor will it cease to remain one. Triggers for migration are manifold – be it a merger, system/infrastructure consolidation or strategic ventures such as introduction of cloud strategies. Regardless of what your trigger might be, it always boils down to moving data from one system to another. The typical challenges associated with migration projects can be summarized by three simple questions:

  1. What does my migration data look like?
    One of the key migration struggles is the variety of source systems and, hence, the variety of formats the data comes in. Complex mapping tables, interim formats, and data cleansings are frequently applied to solve this issue. These activities not only drive migration costs but easily reach a level at which the complexity seems unsurmountable.
  2. How do I add the migration data into my target system?
    Once you have solved the puzzle of converting all migration data into a common format, you are presented with the challenge of getting the data into your target system. Custom interfaces are typically applied here. Again, this includes development/customizing efforts that typically have no further usage once the migration is completed.
  3. How can I verify that my migration was successful?
    Most custom interfaces do not offer a detailed overview of which elements have been migrated successfully and which ones have not. They only generate very basic logs outlining the duration of the migration – information that is hardly useful to assess the success of the migration making it almost impossible to find underlying errors in case of failed/incomplete migrations.

In line with the automation of routine tasks, RPA offers a simple, cost-efficient and effective solution for all of those challenges.

RPA and what it can do for you – A showcase

As we have established in other articles of our RPA blog, RPA is highly adaptable with a sheer endless number of use cases. (For a selected range of examples, please see here).

Recently, we completed a project with a large German asset manager, which impressively showed the potential of RPA for data migration. It has been achieved to migrate some 45,000 depots with about 300,000 data objects in a fraction of time and money compared to a manual migration. The goal was to migrate accounts from another bank’s depot system into the asset manager’s depot system in only ten days and without any errors in order to provide a smooth transition for operations and customers.

Due to the structure of the asset manager’s highly customized database, the different elements and parts of the depots that were to be migrated had to be added individually in a predefined order. To do so, we developed several artifacts that did not only read the input files and pasted them into the system one-by-one in the required sequence but also tracked the progress of the migration and the migrated objects, respectively. This included a detailed status and optional error message to allow for quality and success monitoring.

Furthermore, we managed to use another key benefit of RPA to our favor: scalability. As with almost any migration, the asset manager’s migration was time critical. However, we had to find a way to migrate all data within the given timeframe without overpowering the database. By scaling up the number of robots and using them simultaneously to add accounts to the system, we ensured a timely migration process.

Given the artifact features and ease of use, they were fine-tuned to improve speed and stability while the detailed documentation of transferred objects made it possible to detect incomplete and faulty datasets that could be corrected before completing the actual migration. Hence, the tracking file more than served its intended purpose and even allowed for data quality improvements.

During tests in the test environment as part of the preparation for the actual migration, weaknesses of the database as well as the unforeseen opportunities to supply the system with an abundance of quickly added test cases were discovered. RPA trainings, knowledge transfers and the re-use of artifacts for testing purposes of the system by the database maintenance team left the client with a lasting benefit.

Why RPA is the perfect fit for data migrations

High flexibility – Low price

  • RPA is designed to cope with a sheer unlimited variety of systems, interfaces and data formats. This functionality is at the heart of the various RPA software products and comes “out of the box“. Any tailoring to client specifics can be done quickly and easily.

Garbage in – Quality data out

  • As RPA operates using the existing GUIs instead of customized interfaces, any inconsistencies and lacks in data quality are immediately discovered by plausibility checks of the underlying systems. As such, your migration data can be scanned and rectified before it is transferred to the target systems. Needless to say that RPA may also be used quite easily to generate any required amount of test data to further improve the quality routines.

Traceability? Traceability!

  • Due to its flexibility, RPA software can create almost any logfile as desired in the particular setting.
  • Specifically, RPA can generate logfiles in any desired level of detail and file format (e.g. txt, Word, Excel, PDF, etc.) and propagate the results as required (storage on a network drive or SharePoint, distribution by mail, etc.)

Overall, RPA was the ideal method for migrating the data objects in this case. It impressively solved every key challenge that typically comes with migration projects and may be of use in a variety of other cases in your specific setting.

Find out more about this topic:

Author detailsNA Contact details
Jörg Becker
joerg.becker@capgemini.com		Wolfgang Enders
wolfgang.enders@capgemini.com