Skip to main content
Systems & Data Migration

Data migration without downtime

Replacing the system your business runs on is mostly a data problem wearing a software project's clothes. The cutover weekend gets all the attention. The four months before it decide how the weekend goes.

Start with what downtime actually costs you

"No downtime" is a slogan until somebody prices it. For a wholesale distributor, downtime means the pick line stops and trucks leave short, which costs a known number of dollars per hour and an unknown amount of goodwill. For a nine-site medical group it means the front desk cannot check anyone in, and the practical answer is that there is no acceptable window during business hours at all. For a machine shop running one shift, the plant shutdown week in July is a gift most companies never get. Before anyone designs a migration, write down the length of window you can actually buy and what it costs per hour past that. Every decision below falls out of that one number.

Be honest about the second half of the question too, which is what "down" means to each department. A warehouse can often live for six hours on printed pick tickets and a clipboard, as long as somebody keys the results in afterwards. A call center cannot. An accounts receivable team can wait a day. A retail point of sale cannot wait ten minutes. These are different tolerances inside the same company, and a plan that assumes one number for all of them will fail in whichever department has the tightest one.

You are probably here because

  • A vendor gave you a go-live date and you cannot tell whether it is real
  • The last migration you lived through took three weeks to recover from
  • Somebody said "we will just export to CSV and import" and you got a bad feeling
  • You are buying a company and inheriting a system nobody in your building has seen

The profiling section is where to start. Almost every failed migration we have been asked to rescue skipped it and discovered the real state of the data in the third rehearsal, or in production.

Two shapes, and most projects should not pick the exciting one

There are really only two shapes. A cutover freezes the old system, moves everything, and starts the new one. A parallel runs both for a period, with the business writing to one and the data flowing to the other, until you switch which one is authoritative.

Cutover is simpler, cheaper, and correct for most mid-size companies. It needs a window, and the window is bounded by the slowest thing in your load. Parallel avoids the window but introduces the hardest engineering problem in this whole article — keeping two systems agreeing while both are being written to — and it usually imposes double entry on staff, which they will quietly stop doing in week two.

The shape most teams end up wanting is a cutover with the history pre-loaded. Move ninety-five percent of the volume weeks early, while everyone is still working normally, then move only the changes during the window. That turns a fourteen-hour load into a forty-minute one, and it is the decision that changes the plan more than any other.

ApproachFits whenWhat it costs youWhere it goes wrong
Single cutover
Freeze, load, start
You can buy a weekend and the business tolerates a queueA hard window and a real back-out planThe load runs long and the window closes with you mid-way
Pre-load plus delta
History early, changes at the window
Most mid-size replacementsChange tracking on the old system, and a second reconciliationThe delta misses a table nobody flagged as changing
Phased by entity
One branch, region or division at a time
Operationally separable units with few shared recordsMonths of running two systems and a bridge between themShared customers and shared inventory make "separable" untrue
Full parallel
Both live, one authoritative
You genuinely cannot stop, and can fund itDouble entry or two-way sync, plus daily reconciliationStaff stop dual-entering and the shadow system silently rots

Profile the data before you agree to a date

The most common cause of a blown migration is that everyone planned against the data they believed they had. Two weeks of profiling, done before a date is committed, changes the shape of the entire project.

What profiling turns up is consistent across industries and still surprises the owner every time. A distributor's customer file with 61,000 records where 9,000 are duplicates created by the phone team, another 400 carry a second shipping address typed into a notes field, and the state column contains eleven spellings of Illinois. An item master where the unit-of-measure conversion for a whole product line was entered backwards in 2019 and everyone has been compensating manually. A medical group where 6 percent of patient records have no date of birth because the old system never made it required. A rental company where assets show as on-rent in the system and are physically sitting in the yard.

Run this as a set of counts, not a browse: null rates by column, distinct-value counts, orphaned foreign keys, records violating a rule the new system will enforce, and the top twenty values in every code field. Then take that report to the people who use the system and ask what each anomaly means. Half of them are known workarounds with a business reason. Those are the ones that will break your load.

Every field the old system allowed to be blank is a field somebody used for something else. Find out what, before the new system refuses it.

Decide what does not move

The reflex is to move everything, and it is usually wrong. Fifteen years of transaction history triples the load time, triples the reconciliation surface, and imports fifteen years of bad codes into a clean system. The better pattern for most companies: move open items and enough closed history to run the business — commonly twenty-four to thirty-six months of transactions, all master records, and full balances — and leave the rest in a read-only archive that finance and legal can query.

That archive is a real deliverable, not a shrug. A copy of the old database, a small search screen over it, and a written retention decision signed by whoever owns records retention at your company. It costs a fraction of migrating the history and it satisfies the audit, the tax question, and the customer who calls about a 2016 warranty.

Set the rule per entity, not globally. Customers and items: everything. Open orders, open invoices, open work orders, on-hand inventory, serial and lot records with any remaining life: everything. Closed transactions: a window. Attachments and scanned documents: this is where projects blow up on volume, so measure the total size before promising anything — a twenty-year document store can be several terabytes of scans nobody has opened since 2014.

Where the hours actually go on a system-of-record replacement

Profiling, cleansing and business mapping decisions
30
Reconciliation harness and closing the differences
22
Extract, transform and load code
18
Integrations and peripherals nobody listed
14
Rehearsals, on a clock
10
The cutover window itself
6

Weights sum to 100. Our planning split for the data workstream, which itself typically runs a quarter to two-fifths of total project hours. Judgment from projects we have run and rescued, not a survey.

Mapping is a business decision wearing technical clothes

Field mapping looks like a spreadsheet exercise and it is not. The old system has four customer types and the new one has three. The old payment terms table has 61 entries and 22 have been used since 2020. Two of your branches use the same numeric code to mean different things. Somebody has to decide what each of those becomes, and it cannot be the integrator, because the integrator does not know that terms code 14 is the arrangement with the co-op group and losing it will cost you the account.

Put a name against each mapping decision — an actual person in your company, in a column on the mapping sheet — and give them a deadline. Track the open decisions like a punch list. A migration that slips almost always slips here, and the slip is invisible until the load fails, because unmade decisions do not generate an error until you run the transform.

Two mapping traps are worth naming. First, identifiers: if customer numbers change, every downstream thing that carries a customer number changes with them — the labels on the shelf, the customer's own purchase orders, the electronic trading documents, the reports someone built. Keeping the old identifier is almost always worth it, and where it is impossible, keep it as a searchable alternate. Second, dates and money in text fields: legacy systems store both as strings more often than anyone believes, and a string date of "3/4/19" is ambiguous in a way that will put a hundred records in the wrong fiscal year.

The reconciliation harness is the deliverable

Loading data is easy. Proving you loaded the right data is the job. Build the reconciliation as code before you build the load, and run it after every single rehearsal.

Reconcile at three levels. Counts: rows in, rows out, rows rejected, and the rejects have to be looked at rather than counted. Control totals: accounts receivable balance, accounts payable balance, inventory quantity and extended value, open order value, general ledger trial balance by account. Sampled detail: pick a hundred records with a spread of ugly cases — the customer with three ship-tos and a credit hold, the item with a lot-controlled serial history, the order that is half shipped — and compare them field by field between the systems.

Agree the tolerance in advance and in writing, with the person who signs the financials. Money-bearing control totals reconcile to zero, full stop. Counts of transactional history can reasonably carry a small variance if you can name every excluded record and why. Anything else is negotiated before go-live, not argued about at two in the morning during the window.

Practice Note

Reconcile to the old system's own reports, not to your extract

Compare the new system's trial balance to the report finance already prints out of the old system, not to a query you wrote against the old database. Your query embeds the same assumptions your extract does, so if the assumption is wrong the two agree beautifully and both are wrong. This is the single most common way a migration passes its own checks and still lands with the balances off.

Rehearse it, on a clock, at least three times

A rehearsal is a full run against a fresh copy of production, timed, with the reconciliation executed at the end and the results written down. Not a partial load. Not a load into a database somebody already loaded once. Fresh copy, from the top, with a stopwatch.

Expect three to six of them. The first is always a disaster and that is the point. The second finds the mapping decisions nobody made. By the third, the timings become predictable and you can start optimising — which is when pre-loading history stops being an idea and becomes the plan. Our rule for calling a plan ready: the final rehearsal completes in under half the available window, with reconciliation passing, and the runbook was followed by someone who did not write it.

Rehearse the manual steps too. Somebody has to restart an interface, somebody has to print a final report from the old system, somebody has to physically count the four pallets in the receiving dock. In a real cutover those steps take two to three times as long as anyone estimates, because they happen at four in the morning with people who have been awake for nineteen hours.

The window: what actually happens in it

Write a runbook with a time column, an owner column, and a checkbox column. Not a plan document — a checklist somebody reads out loud in a room. It should be detailed enough that the person running it does not need to make a judgment call.

Freeze. Name exactly who is locked out of what and how. Locking users out is not the same as locking out the interfaces: the electronic order feed from your largest customer does not read your email. Disable the integrations first, by name, from a list.

Final extract and load. If you pre-loaded, this is the delta, and the delta needs its own reconciliation, because a delta that misses a table is invisible until a customer calls.

Verify before you open the doors. Run the control totals. Then have real users do real work: enter an order, receive a shipment, pick and ship it, invoice it, take a payment, close a work order, run payroll if payroll is in scope. Twenty minutes of a person doing their actual job finds more than an hour of automated checks.

Reconnect the outside world in a deliberate order. Payments and electronic trading documents last, because those are the ones that reach a customer or a bank and cannot be un-sent.

Back-out is a real plan or it is a prayer

Every plan has a back-out section and most of them are fiction. The honest version says two things: exactly what has to be true for a rollback to still work, and the specific moment after which it no longer does.

That moment is usually earlier than people think. Once real transactions have been entered in the new system — a shift of orders, a day of receipts, a payroll run — rolling back means re-entering them by hand or losing them. In practice the point of no return arrives a few hours after you let users in, not at the end of the weekend. So place a go / no-go checkpoint before that moment, with named decision-makers and pre-agreed criteria, and give people permission to use it. A migration postponed by two weeks costs a fraction of one that half-worked.

After the point of no return, the plan is not rollback. It is a written list of degraded-mode procedures: how to ship without the system, how to take a payment on paper, who is allowed to authorise a manual invoice, and how all of it gets keyed in afterwards.

Send us the plan and the record counts.

Email the cutover plan, the list of entities being moved with approximate row counts, and the length of window you can buy, to contact@precisionfederal.com. You get back a written note naming the three things most likely to run long, and what we would pre-load. One business day, no charge, no meeting.

contact@precisionfederal.com

The week after go-live is part of the project

Budget it explicitly, because it is where the goodwill is won or lost. The first week produces a predictable set of problems: a report that existed in the old system and does not exist in the new one, a customer whose invoice came out with the wrong tax because their exemption certificate did not map, a picker who cannot find a bin, a month-end process nobody has run yet.

Staff for it. Have the people who built the load available, not moved onto the next thing. Put a single intake channel in place so problems get triaged rather than shouted, and publish a daily list of what is fixed and what is known. And plan for the first month-end close specifically — it is the first time the accounting side gets exercised end to end, it happens three weeks after everyone has stopped paying attention, and it is where balance errors surface.

The things nobody puts on the list

Every migration we have joined had a scope document that omitted at least three of these. They are not edge cases; they are Monday morning.

  • Label and document printers with hard-coded formats, and the pick ticket the warehouse has used for nine years
  • Electronic trading connections — each customer's mapping is its own small project, and their test cycle is on their calendar, not yours
  • Bank files: positive pay, lockbox, remittance formats, the file layout your bank last changed in 2011
  • Scales, scanners, meters and time clocks that talk to the old system over something ancient
  • The spreadsheet somebody built that pulls from the old database directly and feeds a commission calculation
  • Tax rates and exemption certificates, which do not survive a naive customer-record copy
  • User accounts, roles and approval limits, which are usually recreated by hand and usually too generously
  • Attachments — signed delivery tickets, certificates, photos — measured in terabytes and forgotten until the last month

Where you do not need any of this

If you are moving a few thousand records between two systems that both have decent import and export, the whole apparatus above is overhead. Export, import, spot check, done in a weekend by two people. The machinery in this article earns its cost when the data has ledger consequences, when the window is short, when the record counts run into the millions, or when the old system has been in place long enough for people to have invented workarounds inside it.

And if the honest reason for the migration is that a few reports are bad or one module is clumsy, replacing the system of record is an expensive way to fix that. Look at what it would cost to fix the reports first.

A sixteen-week shape that usually holds

Migration Data Workstream

1
Profile every source table; publish the anomaly report and walk it with the people who use the system
Weeks 1–2
2
Scope decisions: what moves, what is archived, retention signed by a named owner
Week 3
3
Mapping sheet with an owner and a due date per decision; cleanse at the source where it is cheaper
Weeks 3–7
4
Build the reconciliation harness first, then the load; rehearsal one against a fresh copy
Weeks 6–9
5
Rehearsals two and three on the clock; add the delta path so history pre-loads
Weeks 10–13
6
Final rehearsal run by someone who did not write the runbook; go / no-go criteria agreed in writing
Weeks 14–15
7
Window, then a staffed first week and a rehearsed first month-end close
Week 16+

Before you commit to the weekend

  • The cost of an hour of downtime is written down, per department
  • Profiling is complete and the anomaly report has been walked with users
  • What is not being moved is decided, with an archive that someone can search
  • Every mapping decision has a named business owner and a date
  • Reconciliation is code, runs after every rehearsal, and its tolerances are signed
  • Control totals are compared to the old system's own reports, not to your extract
  • The last rehearsal finished in under half the window, reconciled clean
  • Integrations, printers, bank files and attachments are on the list by name
  • Go / no-go criteria and decision-makers are agreed before the night
  • The point of no return is written down, and degraded-mode procedures exist past it

Bottom line

A migration is decided months before the window. Price your downtime honestly, profile the data before you accept a date, move less history than you think you need, put a named business owner on every mapping decision, and build the reconciliation before the load. Then rehearse on a clock until the run is boring. The teams who do that have a quiet weekend and a busy week after. The teams who skip profiling and rehearsal discover the same problems in the same order, except at three in the morning with the phone ringing.

Frequently asked questions

How much history should we bring into the new system?

All master records and all open items, plus roughly twenty-four to thirty-six months of closed transactions. Older history belongs in a read-only archive with a search screen, backed by a written retention decision. Migrating a decade of transactions multiplies the load time and the reconciliation surface while adding almost nothing anyone uses day to day.

How many rehearsals are enough?

Three at an absolute minimum, and four to six is common. Each one runs against a fresh copy of production, on a stopwatch, ending with the full reconciliation. The plan is ready when the last rehearsal finishes in under half the available window, reconciles clean, and was executed by someone who did not write the runbook.

Should we clean the data before or during the migration?

Before, in the old system, wherever the people who own the records can do it. Cleansing in the transform means the fix exists only in code, has to be re-verified after every change, and leaves the old system wrong if you ever have to fall back. Duplicates, missing fields and dead codes are best fixed by the team that created them, starting the week profiling finishes.

Can we run both systems in parallel to be safe?

Sometimes, but it is the most expensive option and it fails in a specific way: staff stop dual-entering under pressure, and the shadow system quietly stops matching, so the safety net you paid for is not there when you reach for it. If you cannot buy any window at all, parallel by division or by branch is usually more workable than parallel across the whole company.

Who should own the migration internally?

Someone from the business with authority to make mapping decisions, not the IT lead and not the vendor. The critical path runs through decisions like which payment terms survive and whether customer numbers change, and those need a person who can settle them in an afternoon. The best outcomes we have seen had a full-time internal owner for the duration.

1 business day response

Have a cutover date you are not sure about?

Send the plan, the entities and row counts, and the window you can actually buy. Our engineers will write back with what we would profile first, what we would pre-load, and where we think the timing breaks. Email bo@precisionfederal.com.

Email an engineerCapabilitiesMore insights →
Data MigrationSystems ReplacementReconciliationCutover Planning