Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for our Email Newsletter
Click on the LinkedIn logo to join the new Shopper Technology Institute Discussion Group
Understand Technical Framework of Social Media
To Gain Customer Insight

By Duane Lyons

Leveraging both internal and external data is important to understand your consumer better and create insights that impact the bottom-line. After all, most CPG and retail companies understand the benefits of having greater insight into their customers’ behavior and interactions on social media. So, the real question is what’s standing in their way?

To support social-media analytics initiatives, most firms will need to modify their technical environments. Existing information management (IM) and analytics tools may not be adequate for the demands of integrating, organizing and analyzing structured and unstructured customer data. For many CPG organizations that are accustomed to working with distributors, the transition to an ecosystem enabling direct consumer engagement is substantial.

Handling the diverse and dynamic nature of social and other unstructured data sources requires an information management and analytics architecture that is:

  • Modular – Not a “one size fits all” solution, but one that can be tailored to many specific business use cases, following proven technologies that work seamlessly with existing technology investments.
  • Scalable – To accommodate the large volumes of structured and unstructured customer data while maintaining performance, reliability and depth of analytical capability.
  • Flexible – Allowing firms to leverage packaged tools and applications to accelerate time-to-value without sacrificing the ability to rapidly accommodate evolving business requirements.
  • Platform-agnostic – To leverage existing and emerging capabilities in a dynamic market.
  • To address these requirements, many CPG and retail companies are evolving their IM and analytics ecosystems into layered models defined by Acquisition and Ingestion, Content Management, Entity Resolution, Rationalization and Engagement.

The demands of a Consumer 720 initiative which involves integrating external social consumer data with enterprise structured and unstructured consumer data, introduces new elements and new tiers to the ecosystem including: 

1. Acquisition and Ingestion Layer: This foundational layer captures and integrates raw content from internal and external providers and sources. The volume, velocity, variety and veracity of interactive social content differ from enterprise data; therefore, the acquisition tier should be schema-agnostic to manage complex and evolving document formats. Type-safe, failure-free ingestion is optimized to capture and store consumer interactions while minimizing infrastructure workload and operational maintenance requirements.

Key requirements for this layer include:

  • Accommodating batch, micro-batch and real-time deployment approaches.
  • Allowing provider integration that considers enterprise, interactive and vendor-supplied content.
  • Tuning to reduce latency and performance impacts on the data management tier while optimizing analytical capabilities.

2. Content Management Layer: Social interaction generates massive volumes of complex data of variable structures, requiring careful consideration for both cost-effective management and usage patterns. Storage formats, retrieval access and compression must be optimized for the specific needs of entity resolution and feature extraction.

Key requirements for this layer include:

  • Optimizing access for constant-time retrieval of source-contextual data.
  • Defining a scalable and cost-effective data management layer.
  • Tuning to flexible schemas and document storage to manage near-constant schema changes.
  • Designing storage and structure to optimize access patterns specific to entity resolution and feature extraction.

3. Entity Resolution Layer: This layer resolves common attributes for entities and interactions into structured, identified and classified enterprise definitions. Matching techniques tuned for specific nuances of social profile data are used to resolve consumer identity across multiple social platforms, and internal interactions. Similarly, contextual classifiers are applied to extract features from interactive behaviors, relationships and affinities, providing a detailed perspective of personalized micro-segments of consumer attributes. A great example of contextual classification is the analysis of the Facebook pages that are “liked” by Facebook users. Assuming a brand identifies approximately 80 standard classification attributes, contextual classification can be used to analyze the millions of potential Facebook page likes to determine which of the 80 standard classification attributes are impacted.

Entity resolution and feature extraction both require computationally expensive combinatorial-pattern and similarity-recognition algorithms. Architecting on top of a massively parallel processing foundation enables CPG manufacturers to constantly tune and enhance programs to optimize outcomes through both directed analysis and closed-loop feedback. Driven by a dedicated data science lab that scales linearly with infrastructure, this layer serves a dual purpose; for programmatic data management and exploratory analysis, supporting ongoing optimization.

Key requirements for this layer include:

  • Employing a tunable and extensible entity-resolution approach that encourages optimization and enhancement.
  • Integrating analysis lab capabilities with a data-quality framework for ongoing activity classification prioritized by business needs and use cases.

4. Rationalization and Enrichment Layer: The rationalization and enrichment layer manages the structured view of the consumer, providing manufacturers with a single high-performing access layer. While unstructured data analysis may sound like an oxymoron, the first step in support of analyzing unstructured data requires the application of structure and quantitative measurement. At this point, unstructured data management becomes a form of structured data analysis.
The rationalization and enrichment tier provides a high-performance, structured, semantic layer leveraging enterprise-data definitions to provide a comprehensive 720-degree enterprise and social-media view of the customer. Key requirements for this layer include:
  • Defining standardized, structured and governed semantic access to qualified individuals enhanced with actionable features.
  • Enabling seamless integration with third-party tools.
  • Designing with consumer-engagement processes and technologies in mind.
  • Optimizing for interactive performance accessible to business users.

5. Consumer Engagement Layer: Effective consumer engagement requires a longitudinal view of consumer lifecycle and lifestyle across a broad portfolio of attributes. Promoting positive engagement is a science as much as an art, requiring a platform that puts the consumer at the center.

Key requirements for this layer include:

  • Ensuring closed-loop optimization to enhance and evolve consumer relationships (for example, incorporating campaign response data back into the Consumer720 solution).
  • Two-way communication that adapts to the consumer’s unique preferences and behaviors.
  • Ability to integrate with industry-leading campaign management tools.
  • Incorporating quantified A/B testing and feedback control through machine learning and test design which involves the ability to build predictive models to target the customers most likely to respond to an offer.

Sounding the Horn for Customer720: Call to Action
Major brands and retailers must be able to engage with consumers on social media to compete in today’s market. The digital world is becoming the biggest driver in influencing shopping behavior and brand loyalty. Being “social-media-aware” means leveraging knowledge obtained from social channels to improve the level of personalization within each interaction. Gaining a 720-degree view of your customers means engaging with them constantly and meaningfully on multiple digital channels.

Prior to actual implementation of Consumer720, it is important that CPG manufacturers lay the right groundwork. This should include:

  • Actively driving consumers to your own website and encouraging them to create user profiles.   While the vast majority of retailers already do this, CPG brands have room to improve.   Furthermore, allowing the consumers to log-in with their existing social identities such as Facebook is a best practice.
  • Providing timely and valuable content and incentives to consumers via a branded presence on Facebook with the goal of getting the consumer to “Like” your page. Benefits include improved organic reach results, as well as the ability to know all the Facebook users that like your page.
  • Giving consumers a real reason to follow your brand on Twitter. Twitter provides a robust set of Application Programming Interfaces or APIs that allow access to the historical tweets for specific users. In addition, Twitter’s new analytics provide robust access to engagement, not just impressions.
By taking the time to lay the groundwork, CPG manufacturers will be well on their way to establishing the foundation for Consumer720.  The time to act is now and the opportunity is great. In fact, over the next 30 years, roughly $140 trillion in new customer spending will be up for grabs. “The winners will be enterprises that create an aligned, integrated experience for their customers on a foundation of exceptional service,” according to the Gallup BusinessJournal, July 15, 2014.

To stay relevant, CPG and retail companies must continue to invest in digital marketing and related technologies. To win in the months and years ahead, they must also invest in truly understanding what the digital, social-media world can reveal about their customers, products and practices. Adding the second 360-degree customer view to an arsenal of business intelligence prepares firms to acquire new customers, retain high-value customers and boost profitability. Deep customer insight can only be revealed and understood if it comes from thorough analysis of all meaningful sources available.

Duane Lyons is Practice Leader at Clarity Solution Group, a data and analytics consulting firm.