If high quality customer data is critical to your enterprise operations, then match-merge calibration is one of the most effective techniques at one's disposal. There is no more significant enterprise information management (EIM) activity that occurs within Life Sciences Organizations (LSOs) than the accurate identification and enumeration of its customers. Whether it is the accurate identification of credentialed practitioners or the proper distinction between two group practices with similar names in the same hospital complex with the same address line 1, city, state, and zip code; uniquely identifying customers and associating valuable internal and external 3rd party data to them is a critical step in unlocking insights and intelligence about them. Augmented with social media, socioeconomic factors, and big data (narrow, high-volume data), targeted data governance and applied leading edge technologies, these information assets can become much more valuable to the enterprise and help propel organizational initiatives that grow revenue, reduce cost and manage risk. This series is designed to discuss the implications of specific approaches to customer MDM matching which seriously impact data quality and data value. It is also designed for individuals who are new to the customer MDM world and seek to become more conversant in it based on role, position, or responsibility.
Some of the biggest challenges customer MDM data owners and processors face are:
Lengthy MDM cycles - the time it takes to process batches of customer data which causes latency in publishing data, failure to meet SLAs, and severely impacts data quality in downstream processes.
Over-matching - linking customer profile candidates to a cluster of records where they do not belong; essentially making a match that is not a true match.
Under-matching - not linking customer profile candidates to a cluster of records where they actually belong by being too conservative in your grouping criteria; essentially not making a match where there is a true match.
Singletons - validating records as true profile candidates but not being able to associate them with a cluster or a trusted reference data source which prevents enrichment with other high-value data sources.
Data “not sticking” - Valuable updates of your customer data assets that end up being ignored or "overwritten" after MDM processing and creating golden profiles.
It is in matching and merging that a critical MDM calibration activity takes place: managing the Razor’s Edge between over- and under-matching profiles in the enterprise customer MDM environment. The reason it is a calibration exercise is that it is an informed decision to select and automate specific levels, tolerances and thresholds based on your insight about the volume, voracity, variety and inherent value of your data sources and your knowledge about requirements of your data consumers. Organizations must leverage their understanding of, and tolerance for risk and determine how much they want to allow automation to match and merge data. Match too much and data stewards must break links between existing customer entities and newly submitted entities. Match too little to your customer MDM instance and data stewards must toil through large work queues to manually link near matches to existing customer entities. In addition, they will have to decide when to create new customer entities through splits and broken links and when to remediate customer profiles to submit to automation again. This is costly and time-consuming leading to unnecessary hours of data recycling and the problem of data “not sticking” - valuable updates of your customer data assets that you end up overwriting after reprocessing. This is akin to throwing a great block for your running back, only to tackle your own running back yourself. Think of the notorious "butt fumble" perpetrated by the New York Jets. Although linking technology has improved greatly in recent years, many firms rely on tried-and-true methods such as deterministic and probabilistic matching, which often conflict with each other because the very rules created to determine who a unique customer is must be broken repeatedly to keep the system functioning properly. But because there are so many different internal and external sources and channels of customer information, the largest information management challenge the industry faces is achieving and maintaining the ability to confidently define, identify and describe customers.
I would love to hear about the challenges you face in upstream MDM processing and ways in which you have managed it. The next entry will discuss details around the terminology used herein and relate it to everyday challenges and approaches to mastering data.
Comments