Fast Segmentation Analysis to Diagnose Changing Customer Behavior

By Laura Shores - July 22, 2020

If you’re an analyst, you know the days of providing weekly revenue updates are long gone. These days, as the company continuously runs experiments and promotions to stay competitive, the business demands real-time analysis to support operational decisions. The good news is that you have more data available than ever before. The bad news is that it’s a struggle to find the answers at the pace the business requires. Whether your conducting complex segmentation analysis to understand changing customer behavior or evolving product preferences, there are simply too many variables to check by hand. 

Let’s take a look at how this struggle plays out for an analyst at a large retail company. The company is increasingly competing with online-first brands, and they’re laser-focused on meeting rising customer expectations and delivering a hyper-personalized experience. 

Now, when the marketing team runs a promotion, they don’t just want to understand the general performance of the promotion against their KPI, they want to understand “why” the promotion performed well or not, and with whom. And they need answers, fast. 

At Sisu we’re focused on getting data analysts to the “why” faster and more comprehensively than ever before, and we’re unlocking new levels of detail for experiment and segmentation analysis. Let’s explore how we’d find the answers for your marketing team in Sisu.

Go from the “what” to the “why” faster with segmentation analysis

The average transaction-level dataset includes hundreds of columns – everything from acquisition channels, customer demographics, product SKUs, and time spent in-store or on digital properties. Typically, you’d have to simplify or aggregate this data before you could work with it, but with Sisu you can automatically analyze the millions of possible factors in your data for any given KPI and finally leverage all the rich, wide data you’ve been capturing.

For this example, let’s take a look at how the marketing team’s December promo code campaign impacted transaction amount (txn_amount), their primary KPI.

Before we start, we need to identify where the data lives. In this case, it’s an existing table called public.purchases. 

Then, we’ll set up a comparison test, looking at the performance of the December promo code (December Promo) against populations that didn’t use a promo code or used other promo codes (No / Other Promo). Setting this up only takes a minute or two, and all that’s left to do is to “Get the facts.”

(Product images use mock data.) 

Check millions of hypotheses in seconds

A typical segmentation analysis only compares the performance of two groups. You still don’t know “why” one group outperformed another, which means even when analysts know which group performed best, they’ll still have to spend hours manually digging through hundreds of hypotheses to try and answer this question. 

With our new tools for comprehensive segmentation analysis, once you hit Get the facts, Sisu automates that work for you. In seconds it tests every factor in the data and uncovers the subpopulations that are having the biggest impact on the two groups (December Promo and No / Other Promo). 

Averages across populations are misleading; focus attention where it matters

At a glance, we can see the average transaction amount is roughly $46.00 across the two groups, and on the surface it appears that the December Promo didn’t have an effect. A dashboard would show you a relatively flat line, but with Sisu we can see the specific subpopulations where the promo code had a significant impact. 

Almost immediately you can see differences in the ratio of the average, which is how the average transaction amount differs between your two groups. 

For example, in the fact below we see that for bronze loyalty program members that were shown the week 52 advertisement, the December Promo had a positive impact and lifted the average transaction amount up to $56.20. This is nearly 2 times the transaction amount of the No / Other Promo group that has an average transaction amount of only $30.90. 

This also tells us that if this bronze loyalty member subpopulation was not present in the data, the overall transaction amount for the December Promo group would be almost a dollar lower. You can see this in the impact column, which directs analysts to the highest impact factors in the data so you can focus on the biggest areas of opportunity to keep the business on track. 

Know where the opportunities are – and where they’re not. 

No campaign is going to be perfect, so it’s important that you’re able to quickly provide the marketing team with valuable data on where their December Promo was ineffective without having to rerun the analysis. To do so, we simply flip our view of the ratio of the average and can quickly see subpopulations where the December Promo performed worse than No / Other Promo.  

Here we can see that silver loyalty program members that were shown the week 52 advertisement had a transaction amount of only $23.00 using the December Promo

For this subgroup, the December Promo had a negative impact on txn_amount.  But since you’ve uncovered this answer quickly, this negative can be turned into a positive when your marketing team is able to quickly respond and stop offering the December Promo to this subpopulation. 

Voila! In just a few seconds we conducted deep segmentation analysis and identified specific subpopulations that are significantly impacting your transaction amount. 

For a refresher on how to use Sisu for fast and comprehensive analysis of complex KPIs, check out “Diagnosing Session-Level Data for a Streaming Service in Seconds.”  To learn how customers like Samsung use Sisu to drive customer upsells, retail sales, and campaign performance, head to our customers page or get in touch for a custom demo

Read more

Diagnosing Session-Level Data for a Streaming Service in Seconds

Learn how to go from the “what” to the “why” faster and more comprehensively. In this post, we’ll walk through how a data analyst at a large streaming services company can use Sisu to find the facts in session-level data.

Read more

Designing Datasets: Four Principles to Advance Data Diagnosis

With more transactional data in cloud-native warehouses than ever before, analysts should stop aggregating their data for business intelligence tools. To help, here are four principles on designing datasets for cloud-native diagnosis.

Read more