Abstract
Great! Let’s do it! In this chapter I put you in my shoes and walk you step-by-step through the data migration component of a typical project. We will follow the Basic Upsert-No Delete pattern (discussed briefly in Chapter 5), which is the standard migration pattern and can be used for just about any Salesforce data migration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
They are not affiliated with Universal Containers. Their containers are way too specialized to be used universally.
- 2.
Tool selection and various options were described in Chapter 3. Just because I am using SSIS and KingswaySoft does not in any way mean that these are the right tools for your project. This is a sample scenario only.
- 3.
If you don’t know anything about SQL, you can still follow along, but you will have to be more focused on the text that explains what the code does.
- 4.
Amazon EC2 for Microsoft Windows: https://aws.amazon.com/windows/products/ec2/ .
SQL Server Developer Edition: https://www.microsoft.com/en-us/sql-server/sql-server-downloads .
SQL Server Management Studio: https://docs.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms .
SQL Server Data Tools, the SSIS interactive development environment (IDE): https://docs.microsoft.com/en-us/sql/ssdt/download-sql-server-data-tools-ssdt .
KingswaySoft’s SSIS Integration toolkit (free for developer use): http://www.kingswaysoft.com/products/ssis-integration-toolkit-for-salesforce .
- 5.
For more information, see https://docs.microsoft.com/en-us/sql/relational-databases/in-memory-oltp/cross-database-queries .
- 6.
Remember, we don’t want to modify the SC-CRM database in any way. Each week we will get a new copy with updated data from the client and will restore them over our local copy. Any changes we make to it will be lost.
- 7.
If you prefer to use configuration files for this kind of stuff, that’s fine. I like to use tables because I can include them in SQL code, as we will see shortly.
- 8.
I’m actually using an OLE DB target, for irrelevant reasons. For more information, see https://social.msdn.microsoft.com/Forums/sqlserver/en-US/9f9208ad-6371-4cdf-aed4-778b8b7eea4c/sql-server-destination-vs-ole-db-destination .
- 9.
If you really have to exceed ten0, you can use two connections with different users to run 20 flows in parallel, or 3 and 30!
- 10.
Here you can define larger as “big enough to warrant the use of the Bulk API.” I know this is somewhat vague, but determining when to use the Bull API is not an exact science. Refer back to Chapter 3.
- 11.
I know I said I wouldn’t mention indexing again. This is the last time. I promise.
- 12.
If you run into performance issues, you may need to add an index to the source database to resolve it. (This is common because performance is one of the reasons people decide to move off their current CRM.) Of course, adding an index to the source database violates our rule of not touching the source database. We lose that change on the next restore! If you have this need, I recommend you keep a single SQL file that adds all the needed indexes to the source database, and include running the script as part of the restore process. If you are a purist and refuse to have that additional manual step, you can add a data flow to your SSIS package to move that table over to your staging database as is, and index it there.
- 13.
We don’t do Select *. We list all our fields. This has nothing to do with Salesforce or SSIS; it’s just good database code practice. For more information, see https://www.google.com/search?q=is+select+*+bad .
- 14.
With Kingsway, once you map a field, it drops out of the Input column drop-down, so all we have to do is click that and confirm it’s empty!
- 15.
It’s not enabled by default and should not be enabled unless you plan on using it, because it introduces overhead. To enable it, see https://help.salesforce.com/articleView?id=shared_contacts_overview.htm&type=5 .
- 16.
For more information, see https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_objects_accountcontactrelation.htm .
- 17.
My technical editor (Jarett) took issue with me saying this stuff is easy, and he’s right. It’s not easy. This is me trying to be encouraging, so I will take this opportunity to repeat what I said at the beginning on this chapter:
“I intentionally picked what I felt was a moderately complex project. I want this to feel like a real-world project with real-world problems, as opposed to an easy, straightforward one. You will have a lot to absorb in a single chapter (I get that), but don’t be discouraged if you have to reread this chapter a few times. I could have given you the example we studied in Chapter 3 and left it at that. Technically, it’s a full migration, but it’s not a realistic scenario. It’s overly simplified. So again, if you do not “get” this chapter on your first read, that’s okay. I’d rather you understand 65% of a real-world example than 100% of an overly simplified one.”
- 18.
Salesforce archives events automatically that are more than a year old, and does the same for tasks that are more than a year old and are closed.
- 19.
Like AccountContactRelation, the TaskRelation and EventRelation objects are not enabled by default and should not be enabled unless you plan on using them, because their use introduces overhead. To enable them, see https://help.salesforce.com/articleView?id=activities_enable_shared_activities.htm&type=5 .
- 20.
Of course, if this is the first time we are running the code, no data are returned.
- 21.
For more information on base64 encoding, see https://stackoverflow.com/questions/201479/what-is-base-64-encoding-used-for .
- 22.
No? See the third footnote of Chapter 8.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2019 David Masri
About this chapter
Cite this chapter
Masri, D. (2019). Putting It All Together: A Sample Data Migration. In: Developing Data Migrations and Integrations with Salesforce. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-4209-4_7
Download citation
DOI: https://doi.org/10.1007/978-1-4842-4209-4_7
Published:
Publisher Name: Apress, Berkeley, CA
Print ISBN: 978-1-4842-4208-7
Online ISBN: 978-1-4842-4209-4
eBook Packages: Professional and Applied ComputingProfessional and Applied Computing (R0)Apress Access Books