From spreadsheet to system: migrating your data without losing your mind
Quality Assurance · 5 min read
From spreadsheet to system: migrating your data without losing your mind
The new system is built, tested, ready — and then comes the moment everyone underestimated: getting years of data out of the old spreadsheets and into it. Data migration is where good projects earn their reputation or lose it, because the first thing users do in a new system is look up something they already know. If it’s wrong, trust dies on day one. Here’s how we keep that from happening.
Your data is messier than you think — and that’s normal
Every spreadsheet that’s lived a few years contains the same fossils: the customer entered three ways (“ACME”, “Acme Inc”, “acme houston”), dates in two formats, the notes column doing the work of five real fields, and rows nobody dares delete. None of this means your team was careless — spreadsheets simply don’t enforce rules. The migration is the moment those rules finally get applied, so we start with a profiling pass that surfaces every inconsistency before anyone writes import code.
Decide what moves, what merges, what dies
Not everything deserves to make the trip. Duplicates get merged by rules you approve (“same email = same customer”). Dead records get archived, not imported. Ambiguous rows go on an exceptions list for a human — usually far shorter than feared — instead of silently importing as garbage. The output of this stage is a written mapping: every old column, where it lands, and what happens to the misfits. It’s the requirement document of the migration.
Rehearse it like a release
We treat migrations exactly like software releases, because they are: run the full import on a copy, verify the counts (every row accounted for — imported, merged, or exception-listed), then have the people who know the data spot-check the records they know best. The go-live migration should be a rerun of a rehearsal that already passed — boring by design. And the old spreadsheet? Kept read-only, as the audit trail, never as the working copy.
The rule that saves migrations: nothing disappears silently. Every record is imported, merged by an approved rule, or on a named exceptions list. If the numbers don’t reconcile, you don’t go live.
Why this is a QA story
In our warehouse work, migrated inventory data had to match the physical shelves — there’s no hiding from a floor team with a scanner. That standard travels to every project: the migration is tested against reality, not against “it ran without errors.” Plan it early, budget real time for it, and day one becomes an anticlimax — which is exactly what you want.