A data migration is scoped by what the system can't do yet

A migration estimate prices the copy. Three capabilities the system had before the plan existed decide the rest, and whatever it lacks is the project.

Share
A data migration is scoped by what the system can't do yet
A migration estimate prices only the copy — volume and cutover steps. Three capabilities the system already had decide the real cost: Split, Stream, and Absorb. Whatever's missing is the project — without split the whole store moves at once, without stream it moves in a downtime window, without absorb the copy's load reaches users.

A data migration runs on capabilities the system already has.

Volume sizes how long the data takes to copy across. Service count and cutover steps size the coordination around it. What decides whether the migration goes quietly is three capabilities the system either grew or didn't, long before anyone opened this plan.

Split: the data divides into units that cut over one at a time, with no operation or invariant that has to hold across the divide.

Stream: the source hands over its changes while it keeps answering.

Absorb: the caller survives the store pushing back.

None of the three is the copy. All three decide what the copy costs.

A capability in place leaves no trace on the plan

The line saying the data splits would read like stating the obvious, so it never gets written. The plan lists what will change, and the properties that decide how the change goes are missing from it because they were already true when someone opened the document.

What's left to count is the part the plan can already see.

Where one is missing, rehearsal doesn't close the gap. A runbook can't install a capability the system lacks. The absence costs time, or it costs uptime, or it widens the blast radius. All three capabilities permit an uneventful migration without promising one.

The pressure behind all three is one source that has to keep taking production work while the copy changes its state and competes for its capacity. Move a platform instead and the list changes. Even for data it doesn't stop at three. A way back after the cutover is its own capability, and so is keeping the old store's readers in step. A unit that cuts over while they still read the old side has only half moved.

The check has to be able to come back absent

Take the store you'd move first.

Name the unit you could move next week, then name everything that would have to reach across the boundary while it's gone. A read, a write, a uniqueness constraint, a nightly job that sweeps the whole set. The first name is easy and proves nothing. The second list is the test.

Ask what the destination reports about how far it has applied, then ask what would prove the two sides agree. A position tells you how far the copy got. Whether both sides hold the same records is a separate question with a separate answer, and a migration that never asks it signs off on data that was never compared.

Then find the last time the store pushed back, whatever form that took, and trace one instance to what the client did with it. An empty log answers nothing. A store that has never been pushed to its limit produces the same silence as a client that handles it perfectly, and the first of those is what absence looks like.

What you can't answer is either absent or unmeasured, and those cost differently. Unmeasured needs instrumentation, or one bounded experiment against a corner of the system. Absent needs building.

Installing a missing capability is a migration of its own

The install lands on a system that's still serving, which puts it under the same requirements as the migration that needs it. Treating it as a line item hides a second project in front of the first, with its own cutover and its own way of going wrong.

Installing split means partitioning data that was never partitioned. You choose a boundary and move records across it while queries keep arriving. Every access path that assumed one set has to be scoped to a single unit, and the ones that can't be are what keep the units from moving alone. The paths that get found are updated. The paths that get missed keep returning results, just smaller ones. A count comes back lower than it should. A lookup misses a record that exists. Nothing throws, nothing pages, and the check written to catch it queries the same partial view and agrees. Any check that survives this one has to reach the data by a route the change didn't touch.

The three don't cost the same, and each one has something you can count before you commit to a date. The count won't give you the effort, only its shape. Absorb turns on distinct ways the system pushes back, and those parallelize: a shared client can cover hundreds of call sites at once, and a single path with its own retry logic, running only when a nightly job fails twice, can outweigh all of them. Split turns on access paths and the invariants that cross units, and it doesn't parallelize the same way, because each one is a correctness decision and the mistakes are the silent kind. Stream is binary before it's anything else. Either the source can produce the change history the destination needs, with enough retention to cover the copy, or it can't, and you're building one against a live system.

None of them produces visible progress toward the migration, which is why they get deferred behind work that does, and deferring moves them into the window where nothing is left to spend on doing them carefully. Split is the one to keep out of that window.

Split is answered one boundary at a time

The invariants don't distribute evenly. One part of the data separates cleanly, and another is held together by a constraint or a background job that reads the whole set. So asking whether the system splits gets you nothing. Ask it of the boundary you're about to draw, and ask again at the next one.

Ten boundaries that held do not prove the eleventh. A wrong guess on one costs a unit on the new side, and on the old side it costs whatever the copy took from a store the rest are still reading, which is why the copy's rate is the first number you set and the last one you raise.

What you install keeps paying

The install is project work. What it leaves behind belongs to the system, so it doesn't expire at cutover, and it's still there the next time something has to move.

There is always a next time. A version reaches end of life, a region gets added, the platform underneath moves on its own schedule. Each one moves something while it keeps serving, and each runs on whatever the last migration left behind. Come through with all three and the next change is routine work someone else can run.

Absorb's payoff doesn't wait for the next migration. Where capacity is provisioned, missing absorb sets its permanent floor. If every refusal becomes an error, the safe allocation has to cover the peak. When eligible work can wait, retry without duplicating effects, and still finish inside its deadline, capacity can sit at sustained load plus enough headroom to drain a spike. The headroom above that line was never buying throughput. It was buying the absence of an error path.

A system missing one of the three still gets migrated, and the price was set before anyone wrote the plan. Without split, the whole store moves at once. Without stream, it moves inside a window where nothing is serving. Without absorb, the copy's own load reaches the people using the system.

None of that leaves the system easier to move than it was. The same price waits for the next migration, and the one after that, until building the capability costs less than paying that price again.