The interview most buyers run tests the wrong thing
A typical selection process asks about stack, methods, degrees and past logos. All four are easy to answer well and none of them separates the engagement that ships from the one that produces a handsome deck and a repository nobody can run. The separating factors are unglamorous: whether the person in the room is the person who writes the code, whether your data is in a state anyone can work with, whether "done" was defined in a sentence two people would score the same way, and what you are left holding when the invoices stop.
What follows is the set of questions we would ask if we were on the buying side, roughly in the order they are useful. Several of them are uncomfortable to ask and all of them are easy to answer if the answer is good. A consultant who becomes evasive on the ownership question or the staffing question has told you something important at no cost to you.
What will you do in the first two weeks, specifically?
This question does more work than any other, because it cannot be answered from a template. A weak answer describes a phase: discovery, alignment, stakeholder interviews, a current-state assessment. A strong answer names artifacts: by the end of week two you will have a written description of the three data sources, a query that reproduces last quarter's number from raw records, a list of every place the definition of a customer differs between systems, and a one-page recommendation on whether this is worth continuing.
The difference matters because it tells you what the consultant thinks their job is. Some genuinely believe the first two weeks are for building consensus. That is a real service and occasionally the right purchase. It is not the same purchase as engineering, and the two get sold with identical vocabulary.
Follow-up worth asking: what would make you tell us to stop? Anyone who cannot describe a finding that would end the engagement in week two is describing a project with no failure mode, which means it will run until the budget does.
Who does the work, by name, and what else are they doing?
The oldest problem in professional services is that the people who win the work are not the people who do it. It is not always a bait and switch; often it is genuine and structural, because the senior person's job is selling. Either way you need three facts in writing: the names of the people who will write code and run analysis, the percentage of their week that is yours, and how much of the engagement is theirs versus somebody more junior.
Ask what happens if that person becomes unavailable. Illness, a resignation, another client's emergency. A firm with a real answer will describe how work is documented and handed over. A firm with no answer is telling you the engagement is one person's calendar, which is fine at a low price and a serious risk at a high one.
One more, and it is the most revealing version of this question: can I meet the person who will actually be writing the code, before we sign? Not for a technical interrogation. Fifteen minutes. You will learn whether they have understood the problem or heard about it.
What do you need from us, and what happens if we cannot provide it?
This is the question buyers most often skip and most often pay for. Data work is a joint project whether or not the contract says so. Someone on your side has to grant access, explain what a column means, decide which of two conflicting figures is authoritative, and answer questions within a day rather than a week.
Ask for the dependency list in writing before signing: named systems, named people, access that must be granted, decisions that must be made and by whom, and the response time the schedule assumes. Then check it against reality. If the plan assumes a two-day turnaround on questions and your data owner is on vacation for three weeks in the middle, the schedule is already wrong and everybody can know it now rather than in month two.
The honest version of this from the consultant's side sounds like a warning, and you should want to hear it. Access to a production database at a mid-sized company frequently takes two to six weeks from first ask to working credentials, because it goes through security review, a ticket queue and someone's manager. That is not anybody's incompetence. It is the normal cost of controls, and a plan that budgets zero for it will slip by exactly that much.
What actually predicts a good outcome, in our experience
Our ranking, not a study. The bottom two are what most selection processes weight hardest.
What is the deliverable, and how will we know it is finished?
"A model that predicts churn" is not a deliverable. It is a topic. A deliverable is a thing that exists, in a place, that somebody can accept or reject. Ask for the sentence that decides that, and read it as if you were on the other side of a disagreement.
A workable acceptance sentence has four parts: the artifact, where it lives, the measurement, and the data it is measured on. For example: a scoring service running in your environment, achieving recall of at least 0.6 at a precision of at least 0.35 on the twelve months of history held out at the start, with a written runbook. Two reasonable people would score that the same way. "A production-ready model that improves retention" would produce a two-week argument.
Then ask the harder question: what if the number is not achievable? Sometimes the signal is not in the data, and no amount of skill produces the target. A good consultant will tell you what they will deliver in that case, which is usually a written explanation of why, with the evidence attached, and that is worth real money. A consultant who cannot imagine that outcome will hit the number by moving the goalposts.
What do we keep when this ends?
Get this into the contract, not the conversation. Four things are in play and they are often treated differently: the code, the trained model or artifacts, the data you supplied, and anything derived from it such as labels, features and documentation.
The common arrangement in bespoke work is that you own the deliverables outright, the consultant retains their pre-existing tools and general know-how, and any third-party components come with their own licences that you should see a list of. That is fair on both sides. What is not fair, and appears more often than you would think, is a clause where the consultant retains rights to models trained on your data, or where the working code lives permanently in their environment and you have a report.
The practical test cuts through the legal language. Ask this: if we ended the relationship on Friday, what exactly would we have on Monday, and could a new engineer run it? Then ask for it to be demonstrated at the halfway point rather than at the end. A handover rehearsed in the middle of an engagement is a handover. A handover promised for the last week is a hope.
How do you charge, and what does a change cost?
The rate is the least interesting part of the money conversation, though it is the part everyone starts with. For reference, the ranges we see quoted in the US market run roughly $150 to $300 an hour for independent senior practitioners and small specialist firms, and materially higher on a blended basis at large consultancies, where the number reflects a team with several layers rather than one person. A low rate on a long timeline and a high rate on a short one frequently land in the same place, which is why the total and the shape matter more than the hourly figure.
| Shape | What it is good for | Who carries the risk | How it goes wrong |
|---|---|---|---|
| Fixed-price discovery 2–4 weeks, small | Finding out whether the real project is possible and what it costs | Shared, and the amount is small enough not to matter | It becomes the whole relationship and nothing is ever built |
| Time and materials | Work whose shape will change as you learn | You | No natural stopping point; spend grows quietly |
| Capped time and materials | Most real engineering work | Shared, with a ceiling you can plan around | The cap is hit and the scope conversation happens late |
| Fixed price, fixed scope | Well-understood work with stable requirements | Them, which they price for | Every change becomes a negotiation, including good ones |
| Milestone-based | Longer builds where you want off-ramps | Shared per milestone | Milestones written as dates rather than as artifacts |
| Retainer | Ongoing operation and improvement of something that exists | You | Bought before there is anything to retain |
Whatever the shape, ask how a change is handled. Not whether there is a change process, but what it costs to invoke one and how long it takes. A change process that requires a week and a signature at two levels means small good ideas get abandoned rather than adopted, which is its own expense. A change process that costs nothing means the scope is a suggestion.
What is the part of this that is not data science?
In most analytics and machine-learning work, the modelling is a minority of the effort. The majority is access, plumbing, definitions that disagree between systems, records that were entered by hand for eleven years, deployment into an environment somebody else owns, and the work of getting a person to actually use the output. Ask the consultant to estimate that split for your project.
The answer tells you whether they have done this before. Someone who says the modelling is most of the work has either been unusually lucky or has not shipped into a real organization. Someone who says the data preparation and integration will take the bulk of the schedule is describing the normal case, and you should ask them to price it that way rather than discovering it in week five.
"What did you have to do yourselves?"
On a reference call, that question outperforms every other. It surfaces the gap between what was sold and what was delivered without asking anyone to criticize a colleague. The answers are concrete and easy to act on: we had to write the deployment ourselves, we had to build the monitoring, we had to redo the documentation before a new hire could use it. Whatever the reference had to do, plan to do it or price it in.
What would you refuse to do?
Ask it plainly. The answer separates a professional from an order-taker, and it is one of the few questions with no comfortable evasion.
Good answers exist and they are specific. We would not put a model into a decision that affects people without a documented review path. We would not build on a data source we have not been allowed to look at. We would not promise a number before seeing the data. We would not take a four-week deadline on a twelve-week project, because the result would be something you cannot maintain.
An answer of "we can do whatever you need" is not flexibility. It means either that they have not encountered the situations where the right response is no, or that they will say yes to a deadline they cannot meet and manage the consequences later, which lands on you.
What do you think our problem is?
Ask this before describing your preferred solution, and ask it in the first meeting. You are testing whether the consultant listens for the problem or for the shape of a project they already know how to sell.
Watch for the reframe. The most valuable answer often disagrees with your premise. If you asked for a forecasting model and the honest read is that your demand data is aggregated at a level that makes forecasting meaningless until an upstream change is made, you want to hear it in the first meeting from someone who is arguing themselves out of revenue. That is a rare and reliable signal.
The inverse is worth naming too. If every consultant you speak to independently reframes the problem the same way, they are probably right and the internal framing is the thing to revisit.
The mistakes buyers make with this hire
- Interviewing on tools, which are learnable in a week, instead of on judgment, which is not
- Never meeting the person who will do the work before signing
- Accepting an outcome as a deliverable, so that acceptance becomes a matter of opinion
- Assuming data access is instant, and losing a month of a three-month project to a ticket queue
- Leaving IP and code custody to the conversation rather than the contract
- Buying a retainer before there is anything to maintain
- Choosing on hourly rate between two proposals that priced different amounts of work
- Skipping the halfway handover rehearsal, then discovering in the last week that nobody internal can run it
A selection process that fits in three weeks
Selection
Step six is deliberately last and deliberately in that order. Once the acceptance sentence and the ownership terms are settled, the rate conversation is about a known quantity of work, and it takes twenty minutes. Done in the other order, you negotiate a price for something neither side has defined, and every subsequent disagreement is expensive.
Before you sign
- You have met the people who will do the work and know their allocation
- The first two weeks produce named artifacts, not a phase name
- Acceptance is one sentence two people would score identically
- The dependency list exists, with names and response times
- Data access is either granted or on a dated plan before the start
- Code, models, data and derived artifacts are assigned in the contract
- A handover rehearsal is scheduled at the midpoint
- You know what a change costs and how long it takes to approve
- Someone on your side owns this, by name, with time protected for it
Bottom line
The failure modes in this hire are boringly consistent. The work was defined as a topic rather than an artifact. The person who sold it was not the person who did it. Data access took six weeks nobody had planned for. And when it ended, what remained was a repository that one person could run and a report that summarized it. Every one of those is prevented by a question asked before signing, and none of the questions require any technical knowledge to ask or to evaluate. Ask them. A consultant worth hiring will be relieved that somebody finally did.
Frequently asked questions
A scoped discovery engagement of two to four weeks, ending in a written recommendation and a real look at your data, is a common and sensible first purchase. Ranges vary widely by market and seniority, so anchor on the shape rather than the number: a small fixed price, a defined artifact at the end, and an explicit option not to continue.
Name the artifact, where it runs, the metric and threshold, and the dataset the metric is measured on, which should be held out and agreed before work starts. Also write what is delivered if the threshold turns out to be unachievable, because that outcome is common and needs a defined value rather than an argument.
For bespoke work the normal arrangement is that you own the deliverables, the consultant keeps their pre-existing tools and general knowledge, and third-party components arrive with a disclosed licence list. Put it in the contract, and test it with a midpoint handover rather than trusting it at the end.
Usually a minority of it. Access, integration, reconciling definitions that differ between systems, cleaning historical records and deployment normally consume the bulk of a schedule. Ask any candidate for their estimate of that split; a candidate who says modelling dominates has probably not shipped into an organization like yours.
Sometimes, and less often than buyers assume. Industry familiarity shortens the vocabulary learning curve and is genuinely valuable where regulation shapes the work. It matters far less than whether the person doing the work is senior, present, and able to say no. Weight domain knowledge, but do not let it outrank staffing and acceptance terms.