{"id":13173,"date":"2021-05-12T10:00:00","date_gmt":"2021-05-12T14:00:00","guid":{"rendered":"https:\/\/enterprise-knowledge.com\/?p=13173"},"modified":"2025-11-17T16:34:01","modified_gmt":"2025-11-17T21:34:01","slug":"360-view-of-a-consumer-deduplicating-and-constructing-consumer-data-using-an-identity-graph","status":"publish","type":"post","link":"https:\/\/enterprise-knowledge.com\/360-view-of-a-consumer-deduplicating-and-constructing-consumer-data-using-an-identity-graph\/","title":{"rendered":"360-view of a Consumer: Deduplicating and Constructing Consumer Data using an Identity Graph"},"content":{"rendered":"\r\n
\r\n
\"\"<\/figure>\r\n
\r\n

The Challenge<\/h2>\r\n\r\n\r\n\r\n

For the last 30 years, a large global digital marketing and technology firm has been collecting consumer data on adults in the United States from hundreds of primary sources to build consumer marketing databases. Because data comes from different sources, variations in data formats,\u00a0typos, misspellings, missing data, and incorrect information make linking records for the same real world customer an arduous task. As a result, even after attempts to deduplicate customer data, these databases contain records for more than 2 billion distinct consumers (reflecting an estimated 240 million real world individuals). This makes it almost impossible to obtain a 360-view of a consumer, since data about an individual is split across multiple records and products.<\/p>\r\n\r\n\r\n\r\n

With the goal to integrate the intelligence gathered from different data sources and products, Enterprise Knowledge worked with one of our technology partners for graph-based data catalog systems to engineer a solution that would successfully link records across data products that refer to the same individual.<\/p>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n

\r\n
\"\"<\/figure>\r\n
\r\n

The Solution<\/h2>\r\n\r\n\r\n\r\n

In order to deduplicate the records and associate them to identifiable, unique individuals, EK’s team of experts started from the bottom up, conducting an exhaustive exploratory data analysis of each attribute in the data. Using the completeness and uniqueness of data fields, we then prioritized fields for further exploration, allowing us to quickly deliver value by focusing on the data attributes with the highest relevance for matching individuals.<\/p>\r\n\r\n\r\n\r\n

Working with our partner, we developed a data processing pipeline to take messy input data and create a graph of linked records that corresponded to real world individuals. Our pipeline focused on the following steps:<\/p>\r\n\r\n\r\n\r\n