Home
/
Stories
/
November 20, 2023
Salesforce

How to do Salesforce Data Migration Like a Pro

Detailed practical steps for Salesforce developers to perform a mindful data migration – proper tools, preparation, and safety tips included.

Understanding data migration

Data migration in Salesforce means transferring a certain amount of data from one storage, platform, or system to another with one or both of them being Salesforce-based. It is a common task that occurs both in daily admin operations in the CRM and in large strategic business processes. This is also a complex task that requires dealing with big volumes of data and data structures.

Act 1. Choosing the right tools

The data migration tools you choose are just as important as the process itself. A solid tool will save a great deal of time and resources, and prevent any unnecessary headaches.

Here are a few points you should consider when choosing the tools for data migration:

  • The majority of your data will be in .xlsx or .csv, so the tools should support these formats
  • The tools have to provide you with the ability to handle large volumes of data
  • The tools should be able to perform bulk operations within Salesforce

Some professional recommendations from the Noltic dev team include:

Excel + Salesforce Inspector

Yes, Microsoft Excel is exceptional in dealing with data, and it’s a great data migration tool in Salesforce. It supports all the needed data formats, the data is displayed in a readable table which you can sort and filter, and formulas are great for data transformation and linking related records.

Salesforce Inspector Chrome extension, on the other hand, provides you with a good and convenient Salesforce interface allowing you to easily work with the Salesforce database. The best part is that Salesforce Inspector supports Excel data format. It means you can omit the pile of intermediate import/export files, and copy data directly into Excel instead. Then, from Excel, you can transfer it right into Salesforce Inspector.

SFDX Data Move Utility plugin

SFDX Data Move Utility plugin (SFDMU) is an sfdx plugin created specifically for salesforce migration. Its features are well documented, and it can handle related records migrations. It’s great for SF-to-SF migrations

Programming languages

With sufficient knowledge of almost any programming language, you can easily apply it in data migration projects. You can write a script that transforms the data, handles related records, and uses Salesforce REST API to communicate with Salesforce.

This approach provides you with the best flexibility possible and allows you to automate the whole process. It is, however, the most complex to use, as it requires specific skills, and may take up a lot of resources. It’s worth considering if you plan to perform multiple migrations.

Act 2. Preparing for the data migration

To achieve the desired outcome, a good amount of time should be allocated to planning the data migration process in Salesforce. The first step of preparation includes analyzing business data and understanding the level of its complexity.

Then, you’ll need to go through the target data model to identify the desired future structure and the various relationships between objects that need to be taken into account.

It’s at this point that you’ll need to select the right data migration tool.

After that, you can start laying down the migration pipeline, describing where you will take source data, how you will transform and map records, and how you will insert them into the target environment.

Reviewing The Source Data

First of all, let's take a look at the source data to understand its structure. The table below shows a common example of old data that should be imported into Salesforce. We will use this data set as an example in the following parts of the article. Please note that in a real migration scenario, you will have to deal with hundreds or thousands of records, and the number of columns will be much greater.

Company Comments Customer Name Customer Email Last Call Contract Amount Contract End Date
Company 1 most likely will renew the contract Client 11 email11@mail.com Sep 12, 2023 $12,000 Dec 30, 2023
Client 12 email12@mail.com Oct 20, 2023
Client 13 email13@mail.com
Company 2 might cancel Client 21 email21@mail.com Jun 5, 2023 $8,500 Dec 30, 2023
Client 22 email22@mail.com
Company 3 a call is scheduled for Dec 21 Client 31 email31@mail.com Aug 29, 2023 $13,000 Dec 30, 2023
Company 4 will be renewing Client 41 email41@mail.com Sep 30, 2023 $23,000 Jan 30, 2024
Client 42 email42@mail.com Dec 26, 2022

Defining The Scope

Let’s assume you were told that the data should be imported as Account, Contact, and Contract (those objects are selected as examples, it could be any set of standard/custom objects) records and that corresponding custom fields were created.

Given the requirements, you can now summarize that:

  • You need to move a set of company records
  • You need to move a set of customer records
  • You need to move a set of contract records
  • Customer records should be linked with the company records
  • One company record can have multiple related customer records
  • Contract records should be linked with the company records

Analyzing your Salesforce data model

Now take a look at the target environment. As per requirements, you need to import data as Account, Contact, and Contract. Here’s a visual representation of a sample destination Salesforce data model.

It is quite simple – you have one parent object (Account) and two independent related objects (Contact and Contract). However, if you compare it with your source data, you’ll notice that they are completely different. What to do?

As you cannot import data in its current state, you’ll need to transform it.

Separate the big table by columns and group those columns by their destination:

  • The Company and Comments columns will go into the Account
  • The Customer Name, Email, and Last Call look like they will fit into the contact section
  • The Contract Amount and End Date are self-explanatory

Company Comments Customer Name Customer Email Last Call Contract Amount Contract End Date
Company 1 most likely will renew the contract Client 11 email11@mail.com Sep 12, 2023 $12,000 Dec 30, 2023
Client 12 email12@mail.com Oct 20, 2023
Client 13 email13@mail.com
Company 2 might cancel Client 21 email21@mail.com Jun 5, 2023 $8,500 Dec 30, 2023
Client 22 email22@mail.com
Company 3 a call is scheduled for Dec 21 Client 31 email31@mail.com Aug 29, 2023 $13,000 Dec 30, 2023
Company 4 will be renewing Client 41 email41@mail.com Sep 30, 2023 $23,000 Jan 30, 2024
Client 42 email42@mail.com Dec 26, 2022

By doing that, you’ll end up with 3 separate tables:

Accounts:

Company Comments
Company 1 most likely will renew the contract
email/call in Dec
Company 2 might cancel
Company 3 a call is scheduled for Dec 21
Company 4 will be renewing

Contacts:

Customer Name Customer Email Last Call
Client 11 email11@mail.com Sep 12, 2023
Client 12 email12@mail.com Oct 20, 2023
Client 13 email13@mail.com
Client 21 email21@mail.com Jun 5, 2023
Client 22 email22@mail.com
Client 31 email31@mail.com Aug 29, 2023
Client 41 email41@mail.com Sep 30, 2023
Client 42 email42@mail.com Dec 26, 2022

Contracts:

Contract Amount Contract End Date
$12,000 Dec 30, 2023
$8,500 Dec 30, 2023
$13,000 Dec 30, 2023
$23,000 Jan 30, 2024

Relationships between Salesforce records in data migration

Act 3. Migrating records with relations

With all the preparation done, now you can match your data with the desired data model, and all the data should fit perfectly.

However, a crucial field is missing – AccountId.

Troubleshooting

As a result of splitting the source data, the connections between records were lost. Also, there is no Id column for Companies in your data – we have to get it from Salesforce.

It comes down to two key points.

  1. Insert data gradually to obtain Id’s for populating lookups in the child records

To determine the order for uploading the data, you need to look at how many parent records each object has.

The Account does not have parent objects, so it will be first:

Contacts and Contracts have a shared parent record – the Account, hence they should be uploaded after the Accounts.

If, for example, the Contract also had a Signed By field, which was a lookup to the Contact, the order could look like this:

In some cases, you might encounter circular dependencies. As an example, an Account might have a Primary Contact lookup. To handle this, you need to upload the parent record, then his related records, and after that – update the parent record with Id’s from its related records.

Generally, the more steps there are, the easier it is later to debug and amend the whole process as you can branch off at any step. But, at the same time, more steps equal more time to complete the task, and more room for error (especially if the step requires human actions).

  1. While separating data into tables, keep the connections between related records using a unique field

You could use one of the existing fields (e.g. email or name), but they may have duplicates you are not aware of or your data just doesn’t have matching fields.

A reliable way would be to introduce migration id – a unique text+number identifier. The number will guarantee uniqueness as it will be automatically generated, and text will make it human-friendly (e.g. Account-000001).

Migration id is needed only for parent records, but in some cases, you might need to upload additional records that are related to current child records. To have it covered, try creating a migration id for each migrated object to keep the connection between the old and new records.

In the target org, create a text field to contain the migration id. In source data, use any available tool to generate unique values (e.g. Auto Number fields in SF-to-SF migration)

Now, let's apply this knowledge to your data. First, create a migration id field for each object in Salesforce. Then, generate migration id’s for your source data and separate it into tables while keeping the connection.

Accounts:

Company Comments MigrationId
Company 1 most likely will renew the contract
email/call in Dec
ACC-001
Company 2 might cancel ACC-002
Company 3 a call is scheduled for Dec 21 ACC-003
Company 4 will be renewing ACC-004

Contacts:

Customer Name Customer Email Last Call AccountId MigrationId
Client 11 email11@mail.com Sep 12, 2023 ACC-001 CNT-001
Client 12 email12@mail.com Oct 20, 2023 ACC-001 CNT-002
Client 13 email13@mail.com ACC-001 CNT-003
Client 21 email21@mail.com Jun 5, 2023 ACC-002 CNT-004
Client 22 email22@mail.com ACC-002 CNT-005
Client 31 email31@mail.com Aug 29, 2023 ACC-003 CNT-006
Client 41 email41@mail.com Sep 30, 2023 ACC-004 CNT-007
Client 42 email42@mail.com Dec 26, 2022 ACC-004 CNT-008

Contracts:

Contract Amount Contract End Date AccountId MigrationId
$12,000 Dec 30, 2023 ACC-001 CNR-001
$8,500 Dec 30, 2023 ACC-002 CNR-002
$13,000 Dec 30, 2023 ACC-003 CNR-003
$23,000 Jan 30, 2024 ACC-004 CNR-004

At this point, the AccountId column in the related records is populated with the migration id of the parent record. Now you can extract inserted accounts with their Salesforce Id and migration id, and replace them in child records tables.

Moving data safely

You have prepared your data, but before the migration, you have to make sure that it’s not going to compromise anything. Errors occur all the time, and they are an inevitable part of the process, but every developer needs to know how to handle them. The main precaution you should take if you plan to modify existing data is to create a backup. It’ll allow you to restore the data’s original state.

Also, sometimes the error is noticeable only after the migration. In some cases, you’ll need to manually update the faulted records to fix them or even delete them completely and reupload them. To do that, you have to be able to extract only the records you migrated. Migration id can help with this, but to make sure you can add one more field called Migration Flag, the checkbox should be set to ‘true’ in your source data only.

Lastly, depending on your Salesforce configuration, you might want to disable any related automation to omit validation or limit errors.

Populating System Fields

You might want to set up certain system fields on migrated records, e.g. Created Date or Owner. Keep in mind that you should do it before uploading the data. Also, make sure to provide the Salesforce user who performs the migration with all the required permissions.

Act 4. The migration

With the right tools chosen, with the action sequence in mind, and with troubleshooting steps in direct reach, you are now ready to perform the migration.

You can first take a small set of records and run them through the process to test it and to confirm that, indeed, all records are being uploaded, corresponding fields are populated with correct values and related records are linked

During this, you should consider the edge cases that may appear: what if there’s no email? what if there’s no company name? what if the date is in the wrong format? etc., and find a business logic-based solution to them.

The number of edge cases you will encounter depends on the data quality. The cleaner the data – the less unexpected issues you will get. Still, during the QA/UAT phase of your data migration, you’ll usually discover that there were business cases you missed, but that’s also a part of the process. What matters most is your ability to cooperate with the business-oriented part of your team.

As a bonus, here is a checklist for you to be all set to perform a successful data migration:

  • Select the right tool
  • Create a backup for the data
  • Create migration id’s for our source data
  • Structure and map the data to match the Salesforce data model
  • Set the migration flag and system fields on your records
  • Disable Salesforce automation if needed
  • Upload the records from Step 1 (see Troubleshooting section)
  • Extract the uploaded records with Id’s
  • Populate lookups for related records
  • Repeat the three previous steps until all records are uploaded
  • Check the results
  • Enable SF automation if needed
  • Pass to QA for testing
Share:
/ More news
February 24, 2024
About Noltic
The full-scale war in Ukraine marks two years of exceptional bravery
A message from Noltic with gratitude, sorrow, and hope.
Read more
January 22, 2024
Salesforce
Unlocking Business Synergy: Mastering Slack to Salesforce Integration
Discover how the integration of Slack and Salesforce can unlock significant business synergy and streamline your team's communication and workflow.
Read more