Improving Data Quality by Analyzing Data Migrations

Salesforce Building San MateoData migrations are a necessary part of transitioning from your current system to the Salesforce platform. Because of this, there is a strong case to be made for tools that can easily move data from your current system to Salesforce with a simple click of a button. However, this dream of a single click migration ignores a very fundamental fact about your data – it might not be worth migrating.

There are likely numerous reasons you are moving from your current system over to Salesforce.com such as: user experience, reporting and analytics, mobile, reducing IT costs (moving on-premise to the web), etc. These factors all point to the fact that your current system is not able to meet the needs of your business, and the data that is stored might not either.

That isn’t to say that none of your existing data is valuable. Rather, it means that you need to take the time to analyze your existing data and to understand what is valuable, what needs to be cleaned up, what requires translation from your old process to your new business process in Salesforce, and what should just be left where it is.

Correlation between data quality and users

As a business it is much easier to define the data points that you need but much more difficult to define how to collect and maintain that information. In many cases the responsibility of collecting this information is left up to your users. In addition to their daily duties, how much information you require your users to maintain has a direct impact on data quality and usability.

Let’s consider a simple scenario of how some simple data requirements can quickly cause deterioration in your data quality:

Jane is a sales executive and a top producer at Company X. She exceeds her monthly quota consistently and has the highest percentage of return customers. Company X has just implemented a new software system that promises to improve productivity and provide the business with some key metrics on their sales and marketing channels.

Company X sat down with all of their directors in sales, marketing, and support. Each department had a list of things they needed to know about their customers in order to perform their duties. Company X then set out to customize their system of choice to meet these needs and releases the new system to all their users and spared no expense on training and making sure everyone understood how to use the system. Jane was a part of this training and was very receptive to the new system. She could see the value that the system brought to the company and seemed to move through the system easily during the training sessions.

It is now the first week with the new system and Jane is processing orders for her clients. As she is preparing their orders Jane is now required to ask them a few more questions to complete their order. This information was deemed required by management for every customer so Jane must collect this information before she can place his or her order. Jane is finding that some customers do not want to provide the additional information or simply do not have an answer to give. She doesn’t want to lose the sale so she puts something random into the system and moves on. 

Jane is now trying to update an existing customer’s contact information. She easily finds the record and starts making changes. However, the system doesn’t seem to accept the new phone number. It seems the system checks the format of the phone number but only recognizes U.S. phone numbers and Jane’s customer has relocated to China. Not wanting to lose the information she puts the new phone number into a note field.

Jane also has some regular customers she has worked with for years. They have such a great relationship that they simply call up and ask for their regular orders and hang up. Jane usually just enters the order herself without the customer on the phone. But again she has to enter additional information about the customer to complete the order and again she just picks some values for the required fields and moves on leaving many of the other fields empty.

Additionally, Jane is spending a lot more time on the phone with customers. She used to be able to enter orders within a few minutes but now consistently is taking 3-4 minutes to enter an order. Not wanting to miss her numbers she decides to stop asking customers for the additional information and just enters what is minimally required for an order to get entered into the system.

Jane reports to management that the additional information is causing her to lose productivity. Company X agrees and decides to change their process to optimize for critical information to be required and all other data to be optional. In most cases, users decide not to enter the optional information.

While the scenario above is fictional, it is based on a collage of true stories. These scenarios are what I see play out every time I analyze a database for migration: there are required fields that have gibberish or some random value entered. On regular occasions I need to parse out some information in a note field to get to important information, and I typically find that many records have as little as 25% of their fields populated with data.

What this illustrates is the need to analyze the data you are migrating. This analysis can help you discover how your users are really using the system and answer some important questions such as:

  • Have you asked your users to collect too much information impacting productivity?
  • Are your users really following your process or simply working around it?
  • Are you really using this data to make business decisions or is it part of an old process that was abandoned?
  • Is the information you collected in your previous system valuable and thus worth migrating over to Salesforce?

We’ve seen why problems can occur in your data. In our next post we’ll look at some techniques on how to mitigate the issue of poor data quality and value.

Author credit: This is two-part guest blog post by John De Santiago of Iterative Logic and podcast co-host of Good Day, Sir!

This entry was posted in Guest Posts. Bookmark the permalink.