Start with the records the law already forced you to keep
Pharmaceutical manufacturing is one of the few industries where the data problem was solved before the analytics problem was posed. 21 CFR 211.188 requires a batch production and control record for every batch: actual quantities of each component, actual yield at each phase, identification of the major equipment used, in-process control results, and the initials of the person who performed and the person who checked each significant step. 21 CFR 211.192 requires that any unexplained discrepancy or out-of-specification result be thoroughly investigated and the investigation written down, whether or not the batch was released. 21 CFR 211.180(e) requires an annual review of records for every product. A site that has run for ten years has ten years of all of it, indexed by lot.
That is a rare starting position. Most industrial AI projects begin with a sensor archive nobody trusts and no ground truth. A GMP plant begins with a signed, dated, human-verified account of what happened to every unit of product it ever made. The problem is almost never that the data is missing. The problem is that it sits in five systems that were bought in different decades and were never designed to be joined.
The five systems, and what each will really hand you
Before scoping anything, get an honest inventory of the source systems and the export path out of each. In our experience the export path is the schedule risk, not the modeling.
| System | What it holds | Realistic volume, one commercial site |
|---|---|---|
| MES / electronic batch record (Körber PAS-X, Rockwell PharmaSuite, Emerson Syncade) | Executed batch records, phase-level step confirmations, weigh-and-dispense, equipment and material genealogy | 2,000–6,000 batches a year for solid oral dose; 100–400 for biologics. Executed records run 100–400 pages each |
| Process historian (AVEVA PI System, Aspen InfoPlus.21) | Time-series tags: temperature, pressure, agitation, dissolved oxygen, pH, differential pressure, compression force | 5,000–50,000 tags per site. A 14-day, 2,000 L bioreactor run at 300 tags and one-minute resolution is about 6 million values |
| LIMS (LabWare, LabVantage, STARLIMS, SampleManager) | Release and stability testing, assay, dissolution, impurities, bioburden, environmental monitoring | 100,000–500,000 individual results a year, each one a small row with a specification attached |
| Quality management system (Veeva Vault QMS, MasterControl, TrackWise) | Deviations, CAPAs, change controls, complaints, audit findings, effectiveness checks | 500–5,000 deviations a year. Each carries several hundred to a few thousand words of free text |
| Regulatory publishing (eCTD archive, submission management) | Module 3 quality sections, commitments, post-approval changes, health-authority questions and responses | Tens of thousands of PDF and XML leaves per product family, spanning every market the product is registered in |
Look at the middle column again. It is small. A biologics site producing 400 batches a year, each with 6 million historian values, generates roughly 15–25 GB of process time series annually once it is written to Parquet. Ten years of deviations is maybe 300 MB of text. The entire analyzable history of a large commercial site fits on a laptop. This matters because it sets expectations: there is no cluster to buy and no streaming platform to stand up. The engineering effort goes into joining and cleaning, not scaling.
Where AI actually pays, in order
The uses that return money in a GMP plant are the ones that attack review labor and investigation cycle time. They are unglamorous and they are large.
Use-case readiness in GMP manufacturing
Editorial weighting from public regulatory sources and practitioner reading; illustrative, not a measured statistic.
Deviation triage. A site running 2,000 deviations a year at ten to fifteen hours of quality and technical labor each is spending somewhere between $2M and $3M annually on investigations at a loaded rate near $95 an hour. Most of that time is classification, routing, gathering the same five attachments, and searching for whether this has happened before. Retrieval over ten years of closed investigations, surfacing the four most similar prior events and their approved root causes, cuts the search step from hours to minutes. A 15–25% reduction in investigation labor is worth $350K–$750K a year at that volume.
Batch record review by exception. Reviewing a 300-page executed record takes four to eight hours. The reviewer checks arithmetic, confirms every entry has a signature and a date, verifies in-process results sit inside limits, and hunts the handful of pages carrying a comment or correction. A vision model reading page images can flag the 12 pages that need human eyes out of 300 and pass the rest with a completeness check. At 5,000 batches a year and six hours each, shaving 40% off review time is roughly 12,000 hours.
Submission traceability. When a health authority asks why the batch formula in section 3.2.P.3.2 differs from the record it saw during inspection, the answer lives in a change control from four years ago. eCTD has been mandatory for NDAs, ANDAs, and BLAs since May 2017 and for commercial INDs since May 2018, so the archive is machine-readable, and the linkage from a stated commitment back to the source record is a graph problem the team can build once and reuse for every question that follows.
Continued process verification. FDA's 2011 process validation guidance made Stage 3 an ongoing obligation, and most sites meet it with control charts on a handful of critical quality attributes. Multivariate comparison against a golden-batch profile finds drift that univariate charts miss. The value is avoided loss: a commercial biologics lot can be worth $500K to several million dollars, so catching one drifting campaign a year pays for the work many times over.
The regulations that shape the architecture
Nothing about the compute is hard. The constraints come from the records themselves.
21 CFR Part 11. Electronic records that a predicate rule requires you to keep must have secure, computer-generated, time-stamped audit trails, must be readable and copyable throughout the retention period, and the systems that create them must be validated. FDA's 2003 scope and application guidance narrowed enforcement, but the underlying obligation is unchanged for anything that becomes part of the GMP record.
Data integrity expectations. FDA's December 2018 final guidance on data integrity and CGMP codified the ALCOA expectations that inspectors apply: attributable, legible, contemporaneous, original, and accurate, with completeness, consistency, endurance, and availability added by common practice. A model output that influences a quality decision has to be attributable to a specific model version and reproducible on demand.
Validation practice. GAMP 5 Second Edition, published by ISPE in July 2022, added guidance on machine learning and moved the field toward risk-based, critical-thinking validation. FDA's computer software assurance guidance for production and quality system software, first issued in draft in September 2022, pushes the same direction: put assurance effort where the patient risk is, and stop generating test scripts for their own sake.
Where the regulators are heading on AI specifically. CDER published a discussion paper on artificial intelligence in drug manufacturing in March 2023, asking directly how AI models used in production would be assessed and whether a model would count as an established condition under ICH Q12. In January 2025 FDA issued draft guidance on the use of AI to support regulatory decision-making for drug and biological products, built around a context-of-use statement and a risk-based credibility assessment. In 2025 the EU's GMP inspectors circulated a draft Annex 22 on artificial intelligence for public consultation alongside a revision of Annex 11 on computerised systems. The direction of travel is consistent: define the context of use narrowly, assess credibility in proportion to risk, and keep a human accountable for the decision.
The read-only boundary is the design decision that matters
The single architectural choice that determines how expensive a system is to validate is whether the model writes to the GMP record. Keep it read-only and the qualification burden stays proportionate. The model reads a copy of the executed record and the deviation history, produces a recommendation with citations back to source pages, and a named human makes the determination that enters the quality system. The model's output is an aid to the reviewer, and the reviewer's signature is the record.
Cross that line and the picture changes. The moment a model's output is what gets relied upon, without independent human verification, the system falls inside the validated boundary, needs full Part 11 treatment including audit trails and access controls on the model itself, and every model update becomes a change control with a revalidation assessment attached. That is a legitimate destination for a mature program. It is a terrible place to start.
Retention drives the second design decision. Under 21 CFR 211.180, batch records are kept at least one year past the expiration date, and for OTC products without expiration dating, three years after distribution. If a model output supports a decision recorded in a batch record, you owe an explanation of that output for as long as the record lives. In practice that means pinning the container digest and the model version alongside every output, storing the full prompt and response as the artifact, and never assuming you can regenerate the answer later. Frontier models are retired, and even at temperature zero, regeneration is not a guarantee. Store the output. Do not plan to recreate it.
Data quality decides the outcome; model choice rarely does
Teams arrive asking which model to use. That is close to the least consequential question on the list. On the record-understanding tasks, several current frontier models will land within a few points of each other, and the difference between a project that works and one that stalls is almost always in the table you built before any model saw the data.
- Batch genealogy breaks. Linking a released lot back through bulk, granulation, and API lots crosses MES, ERP, and paper. One missing hop and every downstream feature is wrong.
- Tag renaming across historian migrations. The same thermocouple has three names over eight years. Nobody documented the cutover date.
- Timestamp drift. Historian in UTC, MES in local time with daylight saving, LIMS in the analyst's entry time rather than the sampling time.
- Inconsistent deviation categories. Two reviewers classify the same event differently, and the category list was restructured in a system upgrade, so your labels change meaning mid-dataset.
- Recipe versions. The process changed twice in the period you are modeling. Those are three processes, not one, and pooling them hides the signal.
- Copy-forward free text. Root-cause fields pasted from a prior investigation, which makes retrieval look brilliant and prediction look impossible.
- Units and significant figures. Mg versus mg/mL, percent versus fraction, and rounding applied at different points in different systems.
- Specification changes. A limit tightened in year six, so a result that was in-specification then is out-of-specification against today's limit.
Every one of these is boring, and every one of these will sink a model quietly rather than loudly. The work of building a trustworthy batch-level feature table, one row per lot with verified genealogy and aligned time, is typically 60–70% of the effort in a first engagement. Budget for it openly. A vendor who does not mention it has not done this before.
Your positives are rare, and that changes what you can promise
A well-run commercial site rejects well under 1% of batches. Out-of-specification results run in the fraction-of-a-percent range against total tests. That is a success story for the plant and a serious constraint on supervised learning. If a site made 400 batches last year and rejected two, no method recovers a reliable failure predictor from two positive examples and 400 candidate features. This is why the batch-failure prediction bar above sits at 62% while the retrieval and review bars sit near 90.
The productive reframing is to stop predicting the rare event and start predicting the abundant one. Deviations are plentiful. In-process results near a limit are plentiful. Yield variance is continuous and available on every batch. Models that flag an atypical batch for human attention, rank investigations by likely complexity, or find the prior events most similar to the one in front of the reviewer all train on data you actually have. They deliver the same operational benefit without asking a model to learn from two examples.
Latency and cost, in real numbers
Almost nothing in this domain needs to be fast. A deviation investigation runs on a 30-day clock in most quality systems. Batch record review happens over days. Continued process verification reporting is monthly or quarterly. A model that answers in 30 seconds is indistinguishable from one that answers in 300 milliseconds, which removes an entire category of infrastructure cost. The one genuine low-latency case is in-line process analytical technology under FDA's 2004 PAT framework, where spectroscopic feedback drives a control loop in tens of milliseconds. That is a control-systems problem under its own qualification regime, and it is not what a language model is for.
Inference cost is similarly undramatic. Three thousand deviations a year at roughly 4,000 input and 1,000 output tokens each is 12 million input and 3 million output tokens annually. At the few-dollars-per-million-tokens list prices frontier models carry in 2026, that is a rounding error against a single quality reviewer's fully loaded cost. The heaviest realistic workload is batch record page images: 5,000 batches at 300 pages is 1.5 million page reads a year, landing in the $10,000 to $40,000 range of annual model spend. Set against 12,000 hours of reviewer time, the model bill is the smallest line on the page. The real costs are data engineering, qualification, and change control.
Where federal money touches this problem
Domestic pharmaceutical and biomanufacturing capacity is an active federal funding area, and the data problems are the same ones commercial sites have. NIIMBL, the National Institute for Innovation in Manufacturing Biopharmaceuticals, has operated as a Manufacturing USA institute under a NIST cooperative agreement since 2017 and funds process and data projects directly. ASPR and BARDA fund industrial base and manufacturing readiness work, and NIH runs SBIR and STTR programs across institutes that touch manufacturing science and quality. FDA's own Emerging Technology Program, established within CDER in 2014, lets sponsors discuss novel manufacturing technology with the agency before it appears in a submission. That is the right first conversation for anyone planning to put a model near a regulatory filing.
How to scope a first engagement that can fail cheaply
The failure mode we see most often is a program that spends nine months building a platform before anyone learns whether the underlying data supports the claim. Invert it. Buy an eight-week, fixed-price answer to one narrow question, on one product family, at one site, against a read-only extract, and make the deliverable a written decision rather than a system.
An eight-week first engagement
Three details make this a real option rather than a formality. First, the baseline is stated before the model runs, and it is the honest one: how accurately your reviewers classify a deviation today, and how long the search step takes today. Second, the holdout is blind and your people score it. Third, a no-go is written into the statement of work as an acceptable outcome, because a documented finding that ten years of deviation categories are too inconsistent to support automated routing is worth the eight weeks. It stops a program that would otherwise spend a year discovering the same thing.
Insist that the deliverable include the data-quality report even if the model result is good. That report is the asset. It tells the quality organization exactly what to fix, and it is what makes the second engagement cheaper than the first.
Bottom line
Pharmaceutical manufacturing gives an engineering team something most industries cannot: a decade of signed, dated, human-verified records of what happened to every lot. The wins are in review labor and investigation cycle time, they are measured in thousands of hours rather than percentage points on a benchmark, and they are reachable with a read-only architecture that keeps a qualified human accountable for every decision that enters the record. Get the batch table right, keep the model outside the GMP boundary until it has earned its way in, and buy the first answer in eight weeks rather than the first platform in nine months.
Frequently asked questions
It applies to the electronic records a predicate rule requires you to keep. If the model reads a copy of those records and a qualified human makes and signs the determination, the model sits outside the validated record boundary. If the model's output is what gets relied upon, the system falls inside it, and audit trails, access controls, and change control on model versions all follow.
For retrieval and review tasks, two to three years of closed deviations and a few hundred executed batch records is enough to get a defensible read. For predicting rare failures, no realistic volume is enough, because a well-run site produces almost no failures. Pick the abundant target instead of the rare one.
Less than most buyers expect. A site's full analyzable history is tens of gigabytes, nothing needs sub-second response, and annual model spend for the heaviest realistic workload lands in the tens of thousands of dollars. Data engineering, qualification, and change control are where the budget goes.
On record-understanding tasks the leading frontier models cluster closely, and swapping between them moves results far less than fixing batch genealogy, timestamp alignment, or inconsistent deviation categories. Choose on deployment posture, retention terms, and validation support rather than on benchmark scores.
Store the artifact, not the recipe for the artifact. Pin the model version and container digest, save the full input and output alongside the record, and cite the specific source pages the output relied on. Batch records live at least a year past expiration under 21 CFR 211.180, and by then the model that produced the answer may no longer be available to re-run.