Whitepaper

The Masterplan to
Optimize your Data
Migration Journey to
the Cloud
01

Abstract
Enterprises are witnessing two major growth trends:

Increasing demand for business insights from data to gain and maintain
their competitive edge

An ever-increasing amount of data and data sources, both internal and

external where these insights need to be derived from

This in turn has put an extremely high load on their existing systems of storage and
analytics, typically residing on their in-house and on-prem data platforms. These data
platforms were built to scale decades ago but are unable to stretch their limits in terms
of storage capacity, processing power or analytical ability in line with the two trends
mentioned above.

Cloud has emerged as the solution to this problem with its practically infinite storage
capacity, very high processing power and modern, scalable and high-speed data and
analytics platforms.

Key Takeaways

02 03
Business challenges prompting Use cases for data migration
the need for a right data to cloud
migration strategy

05 06
Solution tenets Solution architecture

07
TechM’s Best Practices for
Data Migration Journey
02

Introduction
As organizations want to migrate from their legacy systems to cloud, there is no ‘one size
fits all’ approach. Every case is unique, and the right answer depends upon a multitude of
factors, with key ones as follows:

Current source systems and data growth rate

Future source systems and data growth rate

Business needs and use cases

Variety, frequency, and recency of analytics required

Business Challenges Prompting the Need for

the Right Data Migration Strategy
Cost and time have emerged as the primary challenge when it comes to enterprise data
migration. Poor data quality and lack of visibility into associated issues have been cited
as the most common factors behind delays and increasing cost. Another key challenge
is choosing the right platform.

We help organizations develop a well-planned migration strategy by consulting with

their business and technology stakeholders to outline the scope of data migration, the
timeline, and the availability of resources in the new system. This enables organizations
to develop a plan that is in alignment with overall business goals and ensures seamless
and timely migration.

Few pointers that need attention while an organization are on their data migration journey:

Choosing the right tool is always a challenge when

Platform
considering hybrid or multi cloud migration solution and it
Selection
requires additional impact analysis on cost and compatibility

Data Quality Identifying right data cleansing rules is challenging as it

and Cleansing requires context of the data and good business knowledge

Monitoring For big migration programme across hybrid environments the

and Predictive monitoring and maintenance is complex in nature and thus
Maintenance requires specialized tools
03

Data encryption /decryption at rest and motion with

proper authentication and authorisation could become
Security and
very cumbersome. Additionally, security management for
Compliance
test data requires use case based planning and
specialized tools for execution.

Considering the varied nature of stakeholders involved in

Program a data migration, it becomes a very complex and proper
Management strategy should be employed for project, communication,
and risk management

System Specialized job planning and architecture consideration

Performance
including accelerators are needed for performance scalability
and Data
Throughput which required additional cost and effort

Data Extracting common meta data from unstructured and semi

Transformation structure data is complicated due to varied sources and some
and Conversion
time requires specialized tool for conversion.

Verification For larger volume of data verification and validation

and Validation become a major challenge and automation may require
of Data for better productivity.

Data Migration Use Cases

Several use cases and situations can prompt the need to migrate data. While one of the
foremost requirements stems from businesses undertaking a large digital transformation
initiative where movement to cloud is all but necessary, an organization could be moving
data to cloud just to get long term cost advantage or leverage more modern technologies;
it could also be a sudden need to moving and merging bulk data because of mergers and
acquisitions. We have seen organizations that have required us to create a migration
factory and implement migration-as-a-service for parallel migrations running across
several geographies. Besides business needs for data migration, we have seen the need
of data migration from varied data sources like migration of unstructured data or file
systems. There are several trends on the approach of data migration as well – such as
API and web-service-based migration or batch load.

To boost our customers’ strategy, we complement it with different IPs and accelerators.
Our home-grown frameworks, solutions, and IPs benefit our customers to choose a
cost effective yet modern migration approach that suits them best to achieve their
business goals.
04

Digital Transformation Merger and Acquisitions

Moving database/data Bulk migration for M&A

warehouse to cloud
Data merge
Setting up future proof data and
analytics platforms on cloud Data audit

Identification of the right Address and revenue validation

migration tool

Building an automation framework

Migration -as- a- Service

Leveraged for migration factory model

Creating a common data model and exposing services from the common data model
to consuming system

Flexibility in migration depending on consumer need with embedded data quality and
data governance

Transformation and Migration Application Migration

Heterogeneous database migration Data migration for application such

as ERP, CRM, and SAP
OLTP/OLAP to NoSQL
This is not only data migration but
Create a transformation layer and map need to consider business process
to target DB with automation. and logic as well

Structured Bulk
DB Lift &
Data Load
Files Shift

File
Unstructured Batch
systems Staged
Data and Delta
(Json, loading
XML, txt)

Semi API/Web
Structured Data Services
05

Our Pillars
Accelerators Frameworks Automation TDM

Solution Tenets
We follow a few key principles for data migration and make sure these principles are
benefiting the organization while migrating their data to the target platform.

The most critical one is productizing the data. It is just not migrating the data, but we look at
how that data will be consumed how can we deliver the data to the point of consumption
with ease. The solution take cares of these aspects and we provide a framework that
enables our customers discover the data that they are looking to consume.

We also provide an error resolution framework that takes care of every failure scenario with
proper mitigation and reconciliation plan including provisioning of a self-healing mechanism.

Another important component of our framework deals with existing data issues and
identification of process glitches. This helps restricting those issues migrating to the new
system. The solution also establishes required cleansing and reporting processes to enable
automated, governed and report based manual cleansing by users.

Our solution tenets not only bring the clarity of the process but also makes the organization
future ready with ensuring business agility and modern trends of technology.

Use Case Based Data Product

Data arranged as per business and consumption system-based use cases to
ensure consistent governance and quality framework while delivering the data.

Data Availability
Each business product data made discoverable and shared by critical
filtering elements such as Country and Language, using tagging and lineage
with data access and security framework

Error Handling & Reconciliation

Establish error framework at each point of failure to ensure proper
mitigation and reconciliation including self-healing mechanism

Data Cleansing And Data Quality As A Service

Establish required cleansing and reporting process to enable automated,
governed and report based manual cleansing by users. Combination of business
requirements and industry best practices to provide high quality data

Multi-platform, Multi Format Usable Framework

Establish once, use many times, enable delta loads in both
API and ETL format through global framework

Optimized & Robust Cut Over Execution Plan & Mock Load
Decouple Data availability from data sharing
06

Accelerators
Utilize TechM IP tools SPRINTER, CDIF, UDMF and INFOWISE for metadata
management, profiling, mapping, transformation and filtering

Verification and Validation

Automated test script execution for data validation and verification based upon
sample test data management, security and use cases

Solution Architecture
Discover Prepare Explore Realize Deploy Go Live

System Data Data Migration Design & Build Data Verification Data Ready Operating
Analysis Strategy Fallout Management
Management

Data Validation, Mapping,

RDBMS Cleanse & Transformation
Validation

Mapping Transformation
Finance
Cleansing
Load
Data Load to Data Load Cleansed &
Intermediate to Staging
Operations Data Transformed
tables Layer data
Extraction
File Extracts Data Target Ready
.csv / .xlsx Staging Data Schema
DW
Target
Data Profiling Automated System
DQ changes Data Load
Summary
Extract
configuration
& Post Load
Staging DB Layer Reconciliation
Data
Share Profiling reports Service
Source with data owners to Error Handling, Recon
Systems correct data at Source Monitoring & Tuning Engine

Error Handling Framework

Correction Report
Error Logging Error Classification Automated Recovery Communication
Ticket Creation

The Way Forward

Data migration is a necessary part of maturing a company as a data-driven organization. It
is always challenging and if we don’t carefully strategize and set up the right migration
model that aligns with the business requirements, it’s all too easy for a promising mission to
fail, stopping businesses from achieving digital transformation.

Selecting a deployment model that aligns with business requirements is essential to make
sure that any data migration is both smooth and successful and delivers business value.
07

TechM’s Best Practices for Data Migration

Data migration is a necessary part of maturing a company as a data-driven organization.
It is always challenging and if we don’t carefully strategize and set up the right migration
model that aligns with the business requirements, it’s all too easy for a promising mission
to fail, stopping businesses from achieving digital transformation.

Selecting a deployment model that aligns with business requirements is essential to make
sure that any data migration is both smooth and successful and delivers business value.

Data Data Movement

Discovery and Reusability
The first step in data migration is not Use a flexible, metadata-driven
field-level mapping, but entity-level architecture that standardizes and reuses
analysis to determine the master data definitions across platforms and projects,
entities needed for the target for e.g., global ingestion framework
application. Identify the source of
product, customer, vendor data, and Establish error framework at each point
validate it against other sources of failure to ensure proper mitigation and
reconciliation including self-healing
Identify data quality problems
through data profiling to reduce
testing and reconciliation effort
Execution
Identify data security requirement
to ensure regulatory compliance and Optimized and robust cut over
sensitive data tagging for test and execution plan including mock loads,
production data verification process, offline vs online
load, and planning for parallel jobs

Program Infrastructure
Management Planning
Efficient program management to
Scalable infrastructure planning
establish a program governance
based on the data growth rate
organization to identify and manage
dependency across stakeholders Tool selection for hybrid and multi
from business to end consumers cloud for cost effectiveness

Author

Saurabh Jha
SVP & Global Head – Data & Analytics

Copyright © Tech Mahindra 2022. All Rights Reserved.

Disclaimer. Brand names, logos and trademarks used herein remain the property of their respective owners.