What is Data Migration?
Data Migration is the process of transferring data from one system to another while changing the storage, database or application. In reference to the ETL (Extract-Transform-Load) process, data migration always requires at least Extract and Load steps.
When do we need Data Migration in Salesforce?
Legacy System to Salesforce.
Typically data migration occurs during an upgrade of existing hardware or transfer to a completely new system.
Examples include: Migration to or from hardware platform, Upgrading a database or migrating to new software.
Prerequisites of Data Migration:
Understanding the Source and Destination: Before you start with Data Migration its important to understand both the structure’s i.e legacy system and the new system. Once you are clear with the source structure it would be easy to create a model on the destination system.
Check List:
Identifying The Tool to Migrate
- Migration tools plays a major role in Data Migration. As most of the business logic or data manipulation has to be done in the tool.
- There are many tools available in the market like Talend, SmartConnect, MuleSoft, Boomi etc.
- Identify the tool which is user friendly and where you can implement your data manipulation logic’s wisely.
Before Data Migration Activity
- Data Cleaning on the legacy system.
- Data Cleaning is to define which cleaning rules will be carried out manually and which will be automated.
- Splitting the rules into two enables the organisation’s domain experts to concentrate on the manual process, while the migration experts design and develop the automated cleaning.
- Typically, the manual cleaning will be carried out before the migration, while the automated cleaning may be carried out before the migration or as part of the migration’s initial phase.
- Create a workbook to maintain mapping of the legacy system (fields, datatype, purpose and respective destination field mapping).
- Most mappings include data transformations, because source and target data models are always different. Transformations of legacy data tend to be complex, because some source fields contain multiple values that must be teased out and mapped separately. Furthermore, legacy data will have missing values that can be filled in from other sources.
- Identify the required fields.
- Identify the legacy Ids to be maintained (External Id’s in SFDC).
- Order of Migration in which you want to migrate the Objects like moving Users first then Accounts then Contacts and their related Objects.
During Migration
- Source and destination Data-type should be in sync (specially Date fields).
- Maintaining the count and the details which you have migrated in a workbook.
- Initially test with a small set of data.
- When source data sets are large, it’s more efficient to work with a subset that is a small but representative sample of source data.
Post Migration
- Create a workbook to maintain the count of records of both the systems
- Verify the counts
- Verify the mapping of all the fields.
- Verify the related objects.
- 100% verification using an automated tool is recommended. Ensure that this automated testing tool is independent of the migration tool.
These checklist items can help you prepare and execute a successful large volume data migration.