Start by measuring the people
Before anyone quotes a camera, run a study on the inspectors you already have. Take sixty parts — a mix of good, clearly bad, and borderline — and have two or three inspectors grade them, twice each, on different days, without knowing they are being compared. Two numbers come out. How consistent an inspector is with themselves, and how consistent inspectors are with each other. The results are usually uncomfortable, and they are the most valuable thing produced in the first month of any inspection project.
What that study typically reveals is that the clear defects are graded consistently by everyone, and the borderline ones are not — a scuff that one inspector scraps and another passes, a cosmetic mark near the edge of the specification. That disagreement is not a people problem. It means the acceptance criterion is written in language that does not resolve to a decision, and it has a hard consequence for any software project: a model cannot be trained to be more consistent than its labels, and it cannot be evaluated against a ground truth that shifts by inspector.
So the first deliverable is often not a model at all. It is a boundary-sample set: physical parts, photographed under the production fixture, agreed by quality as accept or reject, kept in a drawer, and used to settle arguments. Once that exists, everything downstream becomes measurable. Without it, every accuracy claim anyone makes is untethered.
You are probably here because
- A customer complaint came back on a defect that should have been caught at final inspection
- You cannot staff inspection on third shift and quality varies by who is on the line
- A vendor ran a proof of concept on a tray of parts and it looked excellent, and nobody knows what to conclude from that
- You installed a system last year and operations quietly stopped using it
The last two have the same root: the pilot measured detection on a curated tray, and production measures whether the line keeps running.
Pick the defects from the cost data, not from the tour
Ask which defects to detect and you will be walked down the line and shown the interesting ones. Instead, pull the scrap and rework codes for the last twelve months, the customer complaint log, and the warranty or return data if you have it, and rank by cost. That ranked list is frequently not the list the plant would have named, and the top item is often something dull like a mislabelled carton or a missing component rather than a subtle surface flaw.
Then split that list honestly into three groups. Defects visible on a surface a camera can see, which are candidates. Defects that are only visible from an angle or a lighting condition you cannot achieve in the available space, which need a fixturing conversation before anything else. And defects that are not visible at all — internal voids, contamination, insufficient torque, a bond that looks fine and is not. Saying out loud in week one which of the top ten cannot be caught by vision buys enormous credibility and prevents the failure where a system is installed and the complaints keep coming because the real problem was never visible.
Optics and fixturing beat architecture, every time
The largest single accuracy gain available on most inspection problems is lighting. A scratch that is invisible under a diffuse dome light appears sharply under low-angle illumination, because the geometry of the defect scatters light differently from the geometry of the surface. Changing the light can turn an unsolvable problem into a threshold, and it costs a few hundred dollars and an afternoon.
Presentation repeatability matters nearly as much. If the part arrives in front of the camera in the same position and orientation every time, within a millimetre or so, the problem is dramatically easier than if it arrives anywhere on a moving belt at any rotation. A simple mechanical guide is worth more than a great deal of model capability. Teams staffed with software engineers tend to spend the first two weeks on architecture and the fixturing conversation never happens, and that is the wrong order.
The timing budget is the other physical constraint. A line running twelve seconds per unit gives you room for almost anything. A line running four hundred units a minute gives you on the order of a hundred milliseconds from trigger to decision, and the decision has to reach the reject actuator before the part passes it. Track parts by encoder position rather than by elapsed time, because belts slip and lines slow down, and a timer-based reject that was tuned at full speed will eject the wrong unit at start-up.
Rare defects change the technique, not just the sample size
How rare the defect is determines which family of methods can work at all.
| Defect rate | What you can collect | Reasonable approach |
|---|---|---|
| Above about 1% | Hundreds of examples in weeks | Supervised detection, evaluated normally |
| Around 0.1% to 1% | Dozens per week on a fast line | Supervised, with heavy augmentation and careful splits by lot and shift |
| Tens of parts per million | A handful a month, if that | Model normal appearance and flag departures; treat the threshold as a business setting |
| Never seen but feared | Nothing | Do not build a detector. Build the boundary-sample library first and revisit |
When you are modelling normal appearance rather than learning defect classes, one consequence follows immediately: the system produces a score, and someone has to choose where to cut it. That choice is not a technical decision. It trades escapes against false rejects, and both have prices that finance can compute. It belongs to quality and operations jointly, it should be revisited monthly at first, and it should be written down with the reasoning, because the person who has to defend it in six months will not remember.
The false-reject math is what kills these projects
Here is the arithmetic that decides everything, and it is routinely skipped. Suppose a line makes forty thousand units per shift and the system runs at a half percent false reject rate. That is two hundred good units pulled per shift. Whether that is fine or fatal depends entirely on what happens to a rejected unit. If reject means an automatic diverter into a re-inspect tote that someone works down at end of shift, two hundred is annoying. If reject means scrap, two hundred good units is a real loss. If reject means a line stop and an operator decision, two hundred is roughly one interruption every two minutes and the line will not run.
This is the actual reason inspection systems get bypassed. Nobody decides to disable it. The reject bin overflows on nights, the operator has a production number to hit, and the system gets switched to advisory and then forgotten. Design the reject path before the model: where the part goes, who touches it, how long the queue can get, and what happens when the bin is full. Then set the threshold to a false-reject rate the reject path can absorb, and report detection at that operating point instead of quoting a curve.
What decides whether the system is still running in a year — our weighting
Weights we use when scoping. The line everyone expects to dominate is the smallest, and that ordering has held on every line we have worked on.
Shadow mode is the practice that saves the project
Run the system for several weeks making decisions it does not act on. Log every image, every score, and the disposition the current process gave the same unit. At the end you have a real comparison on real production, across shifts, lots, suppliers and ambient conditions, instead of a curated tray.
What comes out of shadow mode is usually surprising in a useful direction. The disagreements cluster: a particular product variant, a particular supplier's material, the first hour after a changeover, the shift when the overhead door is open. Each cluster is either a model problem, a fixturing problem, or a discovery that the current process is missing things nobody knew about. All three are worth knowing before the system has authority to stop a line.
Shadow mode is also how you earn the right to ask for the reject path. A quality manager presented with three weeks of production data showing what would have been caught and what would have been wrongly pulled can make a decision. The same manager presented with a validation accuracy figure cannot, and will reasonably say no.
Give the operator a button that says this reject is wrong
One physical button or one large target on the screen, which saves the image, the score, the timestamp and a reason. It costs a day to build and it does two jobs. It gives you the highest-value training data you will ever collect, because operator disagreements are exactly the borderline cases. And it converts the operator from someone the system is imposed on into someone whose judgment the system asks for. Lines where operators have that button do not quietly bypass the system, because they have a legitimate way to push back.
Things drift, and the plant already knows how to check
A vision system that was correct in March is not automatically correct in September. The resin supplier changed. The vendor changed the gloss on a coating without telling anyone because it was within specification. Someone bumped the camera during a changeover. Sunlight through a skylight moves across the station in the afternoon in a way it did not in winter. The line speed was increased.
Statistical monitoring of the input distribution helps, and it is worth having. But the practice that works best in a plant is one the plant already understands: a physical tray of known parts, run at the start of every shift, with the results logged. If the scores on the golden samples move, something changed, and you find out that morning rather than from a customer. It is a routine an operator can own, it needs no interpretation, and it is checkable by anyone.
Pair it with a plain trend of the reject rate by hour, shift and product. A reject rate that jumps at 14:00 every day in August is a lighting problem, and no amount of retraining will fix a skylight.
Send us your scrap codes and a few hundred images.
Twelve months of scrap and rework codes with costs, plus whatever images you already have, is enough for a first read. You get back a written note: which of your top defects are plausibly detectable, which are not, what the false-reject budget needs to be for your line, and whether the honest next step is a camera or a process change. Email contact@precisionfederal.com.
contact@precisionfederal.comIntegration is a controls problem, not a web problem
The decision has to reach the line controller deterministically. That usually means a discrete output or a fieldbus tag, not a request over a network that might retry. Build the interface so a failure of the inspection station is a defined state the controller handles — pass everything and alarm, or stop and alarm — chosen deliberately with quality, and tested by unplugging the station while the line runs. That test finds things.
Keep the inspection network separate from the office network and keep the station's storage local, with images pushed out on a schedule rather than pulled in real time. A plant floor that depends on a corporate network link to make reject decisions has acquired a new way to stop production. And in a validated plant, changing an inspection method is a change-control event with requalification behind it, so the release process for a model update needs to be designed alongside the model, not discovered after the first retrain.
When the answer is not a camera
Some inspection projects should be process projects. If a mould produces a flash defect on four percent of shots, a vision system is an expensive way to keep making bad parts and sorting them. Fixing the mould costs less than the cameras and eliminates the defect instead of catching it. Ask what the process capability is on the characteristic before assuming detection is the answer.
Some are not projects at all. If the defect is reliably caught downstream at low cost and the escape rate is already tiny, automating the check buys very little. And if you are inspecting because a customer demanded a hundred percent check on a characteristic that has never failed, the honest move is a conversation with the customer, not a capital request.
The scale question matters too. A single station with cameras, lighting, fixturing and a controller interface tends to land somewhere in the low tens of thousands of dollars in hardware, with the engineering to a shadow-mode result typically running six to twelve weeks. Rolling that across twenty stations is not twenty times the engineering, but it is a year of plant time, and most of that year is scheduling access during changeovers rather than writing software.
The mistakes we get called in to fix
- An accuracy figure with no defined operating point, quoted from a curated tray of parts
- No reject path designed, so the system is bypassed within weeks of going live
- Labels from one inspector, treated as ground truth, with the disagreement never measured
- Two weeks on architecture and zero on lighting, on a defect that a low-angle light makes obvious
- Timer-based reject actuation, ejecting the wrong unit whenever the line runs slow
- No operator feedback mechanism, so the people closest to the parts have no legitimate way to disagree
- No golden-sample routine, so drift is discovered by a customer complaint
- A model update process invented after the first retrain, in a plant with formal change control
A realistic first station
Typical sequence to a defensible go-live decision
Before you go live
- Inspector agreement measured and borderline criteria resolved into physical samples
- Detection reported at the operating point you will actually run, not as a curve
- Reject path designed, staffed and stress-tested at the false-reject rate you chose
- Encoder-based part tracking, verified at reduced line speed
- A defined station-failure state, agreed with quality and tested by pulling the plug
- Golden-sample tray and a shift-start routine with someone named to run it
- An operator disagreement button that writes images back for review
- A written model-change process that fits your plant's change control
Bottom line
Line inspection is an operations project with a model inside it. Measure your inspectors before you measure a model, rank defects by what they cost rather than by what is interesting, spend the early weeks on lighting and fixturing, and choose the threshold from what your reject path can absorb. Run in shadow mode long enough to see a changeover, a supplier lot and a night shift. And give the operator a real way to disagree, because the alternative is that they disagree by switching it off.
Frequently asked questions
It depends on the rate more than the count. Above roughly one percent you will collect enough for supervised detection within weeks. At tens of parts per million you will never collect a supervised dataset, and the right approach is to model normal appearance and flag departures, which needs a large set of good parts rather than a large set of bad ones.
Whatever your reject path can absorb. Multiply the rate by your shift volume to get units pulled per shift, then ask who handles them and what happens when the tote is full. That answer sets the threshold. Setting the threshold from a metric first and discovering the operational cost later is the most common way these systems get bypassed.
Usually because it interrupted production more than the defect cost, and nobody had authority or a process to retune it. Look at the reject rate by shift and the number of line interruptions per hour. The other common cause is drift with no monitoring, so the system became wrong and the operators noticed before the engineers did.
For a specific characteristic on a specific surface, often yes and more consistently than a person at hour seven of a shift. For general judgment about whether a part looks right, no. Most successful installations shift inspectors toward handling the reject queue and the ambiguous cases, which is also where their judgment is worth the most.
If the task is a standard measurement, a presence check or a code read, buy a smart camera and configure it. Custom work earns its place on subtle or variable surface defects, on multi-station coordination, and where the decision needs process context the camera cannot see. Ask any vendor to run their offering in shadow mode on your line before you commit.