A segmentation is a function from a customer to a different action
The working definition we use, and it settles most arguments before they start: a segmentation is useful exactly when two customers who land in different segments get treated differently, that difference is something the business is able and willing to do, and somebody can tell which segment a customer is in at the moment the treatment has to be chosen. If any of the three is missing, what you have is a description of the customer base. Descriptions are interesting. They are not segmentations, and they do not survive a change of leadership.
You are probably here because
- There are five personas on a slide and no system anywhere can tell you which one a given account is
- The segments changed materially when someone reran the notebook with a different random seed
- Marketing and finance each have a segmentation and they disagree about the same customers
- Everyone agrees the segments are interesting and nobody can name a decision that changed
These are all the same failure at different stages. The three tests below are the cheapest way to catch it before the work is done rather than after.
Test one: differentiated action. Write the sentence "customers in segment A get ___ and customers in segment B get ___" for every pair, using verbs the company can execute this quarter. A different email subject line counts. A different onboarding path counts. A different price counts if pricing is willing. "A deeper understanding of their needs" does not count, and the inability to finish that sentence is the single most common reason segmentation work dies.
Test two: assignability at decision time. The segment must be computable from data you hold at the moment the treatment is applied. A segmentation built on twelve months of trailing spend cannot target new customers, because a new customer has no trailing spend. A segmentation built on post-purchase support behaviour cannot drive acquisition. This constraint eliminates more candidate designs than any statistical criterion, and it is usually discovered last.
Test three: enough stability to act on. If a third of your customers change segment every month, a segment-specific campaign is chasing a moving target and the measurement afterwards will be uninterpretable. Stability requirements differ by use case: a pricing tier needs to hold for a contract term, a next-best-action can update weekly.
Clustering never fails, and that is the whole problem
Ask k-means for five clusters on uniform noise and it returns five clusters, each with a centroid, a size, and a set of averages that a person can narrate a story about. The algorithm has no way to report that there was no structure. Every other partitioning method behaves the same way. The output looks identical whether the data has natural groups or does not.
So the burden of proof sits outside the algorithm. Before believing a partition, do three things. Run it on a permuted version of your own data, where each column is shuffled independently to destroy the joint structure while preserving every marginal distribution, and compare the cluster quality scores. If the real data scores close to the permuted data, the groups are an artifact of the method. Refit on bootstrap resamples and measure how often pairs of customers land together; a stable structure keeps its pairs, a fragile one shuffles them. And resist reading the elbow plot as an answer, because on continuous data there is often no elbow, only a gentle curve, and the number of segments is then a business choice about how many distinct treatments the organisation can actually run.
That last point deserves saying plainly. If the marketing team can maintain three creative variants, the answer is three segments, and no silhouette score should be permitted to argue for eleven.
| Approach | Fits when | What it costs you |
|---|---|---|
| Rule-based tiers thresholds on two or three known drivers | The action is already tied to a driver everyone agrees on | Nothing statistical. Arguments about where to put the cut |
| k-means on scaled numerics | Continuous behavioural features, roughly similar scales, spherical groups | Sensitive to scaling and outliers; assumes equal-sized round clusters |
| Gaussian mixture | Overlapping groups where a soft membership is useful | More parameters, needs more data, harder to explain in a meeting |
| Hierarchical with Gower distance | Mixed numeric and categorical fields, few thousand rows | Quadratic memory; the distance definition becomes a design decision |
| Supervised buckets predict the outcome, then cut the prediction | There is a named outcome and the action is tied to it | Needs labels and a holdout; usually the right first attempt |
| Uplift buckets model the response to the treatment | You can run an experiment and the treatment costs money | Experimental data, and roughly four times the sample of a response model |
Choosing the features is choosing the answer
Nothing in clustering is as consequential as the feature set, and nothing gets less scrutiny. Distance-based methods answer whichever question the input columns encode, and the columns are usually chosen by availability.
Scale decides weight. Revenue in dollars and tenure in years, unscaled, produces a segmentation of revenue with tenure as garnish, because the dollar column has thousands of times the variance. Standardise, or accept that you have chosen a weighting by accident.
Correlated inputs vote twice. Order count, total spend, total items and session count are one underlying quantity wearing four hats. Include all four and that quantity gets four votes against a single behavioural signal that might have carried the real distinction. Either reduce them first or pick one deliberately.
Encoded categories dominate quietly. One-hot encoding a field with forty values adds forty columns whose combined variance can outweigh every continuous feature in the set. The segmentation becomes a segmentation of that field, which you could have produced with a GROUP BY.
More columns is worse, not better. As dimensions grow, distances between points converge and the notion of a nearest neighbour degrades. Sixty features do not produce a richer picture than twelve well-chosen ones; they produce a flatter one with a more impressive-looking pipeline in front of it.
What decides whether a segmentation gets used — our weighting
Weights sum to 100. Our starting allocation of effort, not a measurement. The last row is where most projects spend the most time.
Assignability is where good designs go to die
Work backwards from the moment of treatment. A customer arrives, something has to be decided, and the only inputs available are whatever the system holds at that instant.
For an acquisition decision, that is often nothing but a channel, a device and a landing page. For an onboarding decision it might be a signup form and the first session. For a renewal decision it is everything, which is why renewal segmentations are easy to build and acquisition segmentations are hard. If the segment definition needs ninety days of behaviour, then for the first ninety days every customer sits in an unassigned bucket, and unless that bucket has its own defined treatment it silently becomes the largest segment in the system.
The practical shape that survives is a two-layer one. A coarse layer assignable on day one from what you observe immediately, and a refined layer that takes over once enough history exists. Write down the promotion rule between them, because otherwise a customer's segment changes on an implicit trigger and every downstream measurement inherits a discontinuity nobody can find later.
Stability, measured rather than assumed
Recompute the assignment monthly for six months and build the transition matrix: what fraction of each segment is still in that segment a month later. This is a half-day of work and it is the most informative thing in the project.
What the matrix tells you is what the segmentation is allowed to drive. Retention above roughly ninety percent per month supports a contract-length commitment such as a pricing tier or a service level. Something in the seventies supports campaign targeting but not a promise. Under half is not a segmentation, it is a state that fluctuates, and the honest response is to treat the underlying quantity as a continuous score rather than pretending it is a category.
Watch specifically for churn between two adjacent segments that sit either side of a threshold. Customers oscillating across a boundary receive alternating treatments, which is worse than either treatment applied consistently. Hysteresis fixes it: require a larger move to leave a segment than to enter it.
The definition lives in a table, not a slide
A segmentation that exists only as a description in a document will be reimplemented slightly differently by every team that needs it, and within a year three systems will disagree about the same customer. Materialise assignments to a table with an as-of date, keep the code that produces it in version control beside everything else, and let downstream systems join to it rather than reconstructing the logic. When the definition changes, that table gets a new version and the old rows stay, so a campaign run last quarter can still be explained.
Send the segment definitions and we will tell you if they hold.
Email the feature list, the number of segments and the action each one is supposed to receive to contact@precisionfederal.com. You get back a short written note: whether the partition survives a permutation test, whether it is assignable at decision time, and the one change we would make first. One business day. No charge and no meeting.
contact@precisionfederal.comThe supervised version most teams should try first
If there is a named outcome the segmentation is meant to influence — expected value over the next year, likelihood to renew, probability of adopting a second product — skip the unsupervised step. Model the outcome directly, then cut the predicted value into bands and treat the bands as segments.
Three advantages, and they are large. The result is validated the ordinary way, on a holdout, so you can state how well it works instead of arguing about it. The bands are ordered, which means the treatment ladder is obvious rather than negotiated. And the feature set can be wide without being carefully weighted, because a supervised model learns the weighting from the outcome instead of inheriting it from your scaling decisions.
The step past that, when there is money on the treatment itself, is to model the response to the treatment rather than the outcome. The customers most likely to renew are frequently the ones who would have renewed anyway, so a retention budget aimed at the top of a propensity model spends most of itself on people who needed nothing. Uplift modelling separates the persuadable from the already-persuaded, but it requires experimental data with real random assignment and it needs substantially more of it — a working rule is around four times the sample of an equivalent response model, because you are estimating a difference between two noisy quantities. Run the experiment first, then build it. Related ground on the retention case sits in churn models and what to do with them.
Three kinds of segmentation that get confused
Value-based. Groups by what a customer is worth: current revenue, expected lifetime value, margin. Drives investment decisions such as service levels and account coverage. Easy to compute, easy to defend, and says nothing about why anyone behaves the way they do.
Needs-based. Groups by what the customer is trying to accomplish. Drives product and messaging. Usually requires research data rather than transaction logs, which is why it tends to fail test two: survey responses are not attached to the accounts in your database at the moment a decision has to be made. The repair is to build a model that predicts the research-derived group from behavioural data you do hold, and to report how accurately it does so.
Behavioural. Groups by observed usage patterns. Drives lifecycle and in-product decisions. Assignable, stable enough over short windows, and the variety most likely to survive contact with a production system.
Most organisations need two of these at once and try to serve both with one partition. That produces a compromise that satisfies neither team. Two clearly named segmentations, each tied to its own decision, cost less to maintain than one that has to be re-explained in every meeting.
Cost to change once the segmentation is wired into systems
Difficulty as we rank it, driven by how much downstream work each change invalidates. Decide the count and the assignment path before anything is built on top.
Validating it the only way that counts
Statistical separation is not validation. Two groups can be beautifully separated in feature space and respond identically to everything you are able to do about it, in which case the separation is a fact about the data and not about the business.
The validation is behavioural. Take one segment pair, apply the two differentiated treatments, hold back a randomised control within each segment, and measure whether the difference in response between segments is larger than the noise. If it is not, the segmentation may still be describing something real, but it is not yet driving anything, and continuing to invest in refining the partition is the wrong move. This is where the arithmetic in experiment design when traffic is thin becomes relevant, because segment-level tests split an already small sample.
Do this on one pair before rolling the segmentation across every channel. The team that tests the full scheme at once learns only whether the whole thing worked, which is not an actionable answer either way.
The mistakes we are called in to fix
- Five personas with names and photographs, and no field in any system that stores which one an account is
- Unscaled features, producing a revenue segmentation that everyone believes is behavioural
- The elbow plot treated as an answer when the organisation can only run three treatments
- No permutation or bootstrap check, so nobody knows whether the structure is real
- Segments requiring ninety days of history, silently placing every new customer in an undefined state
- Customers oscillating across a boundary and receiving alternating treatments as a result
- A retention budget aimed at high-propensity accounts who were going to renew regardless
- Two teams maintaining two definitions, both called the segmentation, disagreeing on a third of accounts
A four-week pass
Segmentation Design Pass
Before you present it
- Every segment pair has a written, fundable difference in treatment
- Assignment uses only data available at the moment of the decision
- An unassigned bucket exists and has its own defined treatment
- The partition beat a permutation of your own data on the same score
- Bootstrap refits keep customers together at a rate you can quote
- The segment count equals the number of treatments the business can run
- A transition matrix over at least six periods has been read, not assumed
- Assignments are materialised with an as-of date and versioned definitions
- One differentiated treatment has been tested against a randomised control
- Where a named outcome exists, the supervised alternative was tried and compared
Bottom line
Segmentation is one of the few analytical exercises that produces a confident-looking result on data with no structure at all, which makes discipline about the surrounding questions more important than sophistication in the method. Start from the action and work backwards to the data available when that action is chosen. Fix the number of segments to the number of things the organisation can genuinely do differently. Check stability before promising anything that has to hold for a quarter. And when there is a named outcome, model it directly and cut the prediction, because that version can be validated and the unsupervised version mostly cannot.
Frequently asked questions
The number of genuinely different treatments the organisation can build and maintain, which in most companies is three to five and almost never eleven. Statistical criteria on continuous behavioural data rarely produce a clear answer anyway, so letting an operational constraint decide costs you very little and saves a long argument. Adding a segment nobody has a distinct action for adds maintenance and no value.
Two checks. Shuffle each column independently to destroy the joint structure while keeping every marginal distribution, cluster the shuffled data, and compare quality scores against the real data; if they are close, there is no structure to find. Then refit on bootstrap resamples and measure how often the same pairs of customers land together. A partition that survives both is worth building on, and one that fails either is a story about randomness.
Usually after, and often instead. If the decision is how much to invest in an account, predicted value cut into bands is a segmentation that can be validated on a holdout and explained in one sentence. Reach for unsupervised methods when the goal is to discover distinct patterns of behaviour and there is no single outcome that captures what you care about.
No, because segment sizes can hold steady while the membership churns underneath. Read the transition matrix rather than the size chart: what fraction of each segment is still there a month later. Aggregate stability with high individual movement is the pattern that quietly ruins campaign measurement, since the group you treated is not the group you measured.
It works well for generating candidate dimensions from unstructured text, which is genuinely hard to do by hand at scale. It does not remove any of the three tests. Whatever comes out still needs a differentiated action, an assignment path that runs at decision time on every customer rather than only on the ones who wrote in, and measured stability. Treat the output as a hypothesis about which dimensions matter, then verify it in the structured data.