Custom AI Application IMPACT MEASUREMENT TOOL
A premium shoe manufacturer, specializing in eco-friendly products, wanted to boost brand visibility and broaden its consumer base, being a new entrant in the market. While the brand was performing well across multiple channels, the decentralized nature of functions proved it difficult to pinpoint the exact causes that were prompting impactful results. This made it difficult for the brand to choose focus areas for investments to accelerate brand promotions and drive tangible results.
The global sustainable apparel industry grew by almost 508 percent, between 2016 and 2018. Research shows that customers are observed to not be as sensitive to high prices in the sustainable apparel category when compared to its counterpart. It’s only natural that the industry has garnered the attention of new entrants, as well as traditional fashion retailers who are introducing exclusive sustainable product lines to keep up with the trends and times, and also keep their existing market shares. Marketing has never been more relevant.
The fiercely competitive market has retailers tapping into marketing far and wide, with spends observed to be relatively higher in the sustainable apparel industry when compared with the regular apparel industry. These costs are certainly a point of concern for sustainable apparel brands, given that sustainable wear also requires expensive raw materials such as organic cotton, which shoots up production costs. It’s natural for decision-makers to want to cut down on the marketing cost center, so they don’t have to compromise on product quality or transfer the costs onto the customer. More so for brands that are relatively new in the market and are yet to establish a brand following that would be unswerving in the event of price mark-ups. All eyes are on measuring marketing efficacy and channeling funds only to the most effective channels.
There’s a clear delineation between how marketing efficacy is measured across online and offline channels – by design – whether it’s page views, or CTRs on ads, in-store check-ins, among others. It also becomes essential for brands to tap into customer information to segment into different brackets to build a long-term customer loyalty strategy and customize marketing efforts to their personas. From social listening to paid advertising, customer surveys, email interactions, and more, many channels are used to build rich buyer profiles and disseminate personalized communication through automated CRM platforms and advertising channels – to nudge the customer to purchase/engage with the brand, and build stronger brand loyalty over time.
Customer acquisition is another focus area for retailers, where attributing the exact role of marketing in converting customers becomes difficult, given how there are innumerable variables that can influence acquisition - thereby making it difficult to explain the steep costs that tail these exercises. While most online channels, more so paid channels, have a provision to measure marketing impact, these standalone metrics may not be telling of the bigger picture, where consumers interactions with a brand overlap across touchpoints and channels, making it difficult to pinpoint which of these channels is proving to be an effective marketing investment.
It’s key to evaluate marketing activities in real time from a vantage point, and tweak marketing bids, promotional offers etc., depending on their efficacy, to optimize spends. This can shed light on whether or not decision-makers need to invest more in certain marketing campaigns/vehicles in the long run or even rethink their strategy in a specific geography.
There is no dearth of customer data in the retail industry, as brands have multiple channels and opportunities to interact with customers search engines, email, organic channels, affiliates, social media, comparison shopping engines, billboards, advertisements, radio jingles, and many more. However, the number of retailers that tap into data to drive decisions, is not as promising a figure. A survey conducted on 350 manufacturers revealed that only 16 percent of the respondents consider themselves to be experts in data-driven decision-making, despite the fact that over 80 percent of the respondents gathered data, while 76 percent agreed to the criticality data can bring in retail decision-making and ramping sales performance. It’s natural to arrive at a conclusion that most of the surveyed retailers do not have the data capabilities or knowhow to drive decisions led by analytics. Intuitive, easy-to-use analytics solutions are the evident next step for retailers.
There’s been a significant uptick in the use of streaming analytics in the recent past, as more and more businesses rely on live analytical insights to make on-the-go decisions. A good example of this would be influencing customer behavior with dynamic discounting and recommendations. Analytics plays an important role in gauging the performance of existing campaigns, gauging customer engagement as well as intent to purchase. Typically, revenue management teams plan sales and in turn, production by relying on forecasts across multiple levels. The interdependency between marketing, pricing, sales and CRM functions, necessitate free flowing of insights across departments rather than siloed efforts that do not paint the whole picture. The interdependencies and complexities only grow deeper, with large retailers who have multiple product categories, speckled across geographies.
An Impact Measurement Framework - a methodology of evaluation metrics - helps organizations gain perspective on the best channels to invest funds to derive maximum impact. An impact measurement analytics tool that allows marketers to run simulated experiments, can help in generating real-time insights that aid decision-making, and thereby improve operational efficiency and maximise ROI.
Productization allows analytics to permeate across different levels and functions in the organization, so an analytics synergy drives business decisions. This can help in expanding the outreach of analytics from silos to across the bandwidth of the organization. The resulting tool should essential be easy-to-use for stakeholders across the organization, so they can identify the real-time impact of operational functions, while also build on and/or experiment with different recommendations that improve operational efficiency. By offering a bird’s view of multiple operational aspects, business can also cut costs by reducing dependency on multiple tools/frameworks to gauge performance of individual siloed departments, proving cost-effective, while serving as the single watchtower to supervise operations.
An impact measurement framework can be treated as a custom AI application, which can help identify models with the greatest marketing impact, through real time experiments. Such frameworks help understand the overall working nature of the existing operational model in a company. Using impact measurement tools, flaws can be identified, and the points of improvement can be targeted efficiently instead of at random.
Let’s take a look at typical features of a well-rounded impact assessment framework:
A growth funnel visualization offers keen insights on how customers are distributed from the top to the bottom of the sales funnel, across awareness, acquisition, conversion and procurement stages. Comparisons of funnel movement over different timelines can shed light on the most productive marketing campaigns and investments, and align new targeting, re-targeting and brand loyalty efforts.
These models help to link sales/business performance to marketing activities. It is essential to evaluate historical data on marketing spend vs. performance in these models. Typically, an MMM utilizes a vast array of data around industries, categories, competitors, pricing, promotions, economy, sales, profits, revenues and so on, making predictions on the most suitable marketing variables using regression. The model can then predict how market outcomes such as sales, profits or revenue will be affected when the input variables are changed, helping brands identify marketing vehicles that fuel growth.
Essentially a lift is a represented as the ratio of target response of a model with the average. This helps companies identify shortcomings of existing models that need to be revisited, alongside high-performing models in the existing ecosystem, across functions, on a real-time basis. The tool can be utilized by analysts, operators and marketers, to quickly get a read on marketing efforts versus results, and thereby optimize spend, while a campaign is still in flight.
A dashboard that offers an overview of the brand’s performance metrics such as sales, in-store walk-ins, website visits etc., over different periods of time, against marketing triggers such as campaigns, promotions, advertising and other events etc., across geographies. The dashboard can be used to identify the most valuable avenues contributing to sales and brand visibility, helping brand/marketing/sales managers plan budgets and direct future investments in the most fruitful/rewarding vehicles.
An impact measurement framework is inputted with data collected across touchpoints, from customer experience & NPS surveys to paid media engagement, organic website visits to emails, and more – segregated as new and returning customers, across awareness, acquisition, conversion and fulfilment segments of the sales funnel. The input data may be refreshed on a daily or real-time basis, depending on the nature of data infrastructure.
Past campaign data such as start date, end date, conversion channel, marketing cost, campaign efficacy etc., can be leveraged as a baseline for validation while building the market mix model in an impact measurement framework. Performance metrics are subjected to statistical analysis to unravel insights. The modelling approach may involve combining trends as well as measurement against control, or even use a causal framework.
The impact measurement application houses forecasting algorithms that predict the performance of various metrics across experiment variable manipulations. Since there are multiple variables at play, all possible permutation and combinations of these variables are taken into account while developing the tool, so the user has the lenience to choose the filters that would shed light on KPIs of interest.
The granularity of the filters would vary depending on the focus areas; for ex: location filters can be restricted to state for one client, while may have to run deeper to accommodate city and sub-areas in the case of another client who has hyper-local marketing strategies. Filters can allow for quantification of results across business channels, customer persona, product categories and more.
When the user makes selections on the event parameters and control variables, the application triggers the models to run in Python and retrieve the results that are displayed in a rich visual graph format. The tool reports a lift of x.x %.
Applications with socialization capabilities, can allow businesses to analyze the impact of co-dependent variables and factors. This makes way for well-rounded and realistic evaluations. The application can house comparison metrics so custodians can evaluate performance across peer categories, geographies etc., and replicate marketing success.
We’ll now do a deep-dive into a sample solution that was built by TheMathCompany to enable the premium shoe manufacturer measure impact of marketing activities across geographies. The real-life example will help in understanding how an AI asset is built and mobilized to measure retail marketing efficacy and chalk out pipeline strategies through simulated experiments.
An upcoming premium shoe manufacturer wanted to increase the brand’s effectiveness and outreach. While they were the fastest growing brand in the niche given their unique, eco-friendly products, they were relatively new to the market and wanted to further improve the brand presence. The existing decision-making process was decentralized, and it was challenging to identify activities generating high ROI. The time taken to arrive at decisions is key, and the client wanted to speed up the decision-making process to achieve high-impact results at pace.
TheMathCompany developed an impact measurement tool that could measure impact of experiments/ changes implemented in near real time and suggest changes to the experiment, if necessary, thereby, speeding up the decision-making process as well. The optimum level of data granularity had to be identified for the tool. Once functional, the tool would identify scenarios where lift measurement was essential, provide insights into real time impact and recommendations on experiments that would help in improving operational efficiency.
To identify activities that are generating favourable results,
- The granularity of tests was cross-referenced with data and key performance indicators were identified
- Appropriate algorithms were utilized for forecasting and developing accurate metrics
- User flow and wireframe of the application were defined
- Upon assessing the features of the client’s cloud platform and taking the real-time usage into consideration, data pipelines were configured
- After extensive functional testing and user acceptance testing, the application was deployed on a cloud platform
The resulting application comprised a multi-layer architecture:
- A presentation layer that comprised action-oriented results visualization along with individual user access
- The application server layer, i.e., an interface connecting the presentation layer and backend application builder
- A model runner where statistical models running to measure the impact could be altered as and when the need arose
- A data layer made up of input data with a provision to store results
When using the tool, first, measurement KPIs and granularity of measurement had to be identified. Then, parameters had to be defined for events whose impact had to be measured. A control strategy determining factors like geography, had to be chosen for the event. Upon setting up these filters, impact could be assessed, and the results could be visualized.
By leveraging Co.dx, our proprietary AI master engine, TheMathCompany was able to configure a custom impact measurement application with nearly 60% faster build turnaround time. Through Co.dx we were able to re-use prebuilt impact measurement application workflows that can measure impact of experiments/changes implemented in near real time and recommend changes, if necessary, thereby speeding up the decision-making process as well. Customizations were aligned to nuances unique to the retailer. For instance, repurchase frequency of products was observed to be very low, which required shifting from the use of standard RFM metrics at customer level to identifying other relevant metrics to measure impact of events. Once deployed, the nuanced application would identify scenarios where lift measurement was essential, provide insights into real time impact and recommendations on experiments that would help in improving operational efficiency.
We use a combination of cutting-edge technologies to uproot analytical roadblocks and create sustainable, insight-driven ecosystems for all our partners.
- Cost of measuring the impact was reduced by more than 80%
- Real time measurement of event impact was possible as all the data inputs were streamed real time and multiple algorithms ensured accurate results.
- Experiments were recommended based on historical results and this enabled decision makers to perform smarter trials
- Implemented impact measurement framework to measure real-time impact of 100+ experiments run across marketing, pricing, and CRM functions
- Deployed market mix application to measure the impact of various marketing activities and optimize spends across channels which resulted in incremental marketing impact of 6 percentage points
The intricacies in customer behaviour and diversity in marketing triggers, call for an intelligent consolidated system, that offers a 360-degree view of how each cog contributes to the overall results. With a bird’s view of operations, performance and costs involved, decision-makers are better placed to expend/allocate funds to the most fruitful investment vehicles.