Dirty data is the real deadline

Jason Ihaia · June 9, 2026 · 8 min read

Every AI migration demo maps a clean schema in ninety seconds. Real migrations don't die on schema — they die on data quality, the months of analyst cleanup the demo never shows. "Agentic data quality" is having a moment for exactly that reason. Here's the reframe: AI isn't a mapping toy, it's the thing that compresses that cleanup from months into days.

Watch any AI data-migration demo and you'll see the same ninety seconds. Two schemas side by side. A prompt. The model draws the join lines — customer_id to cust_no, created_at to signup_date — and the room nods. Mapping solved. Migration solved. Ship it.

It's a magic trick, and the trick is in what's offstage. Schema mapping is the part of a migration that was already easy. The part that actually consumes the calendar — the part that turns a "six-week" project into a nine-month one — is the data underneath the schema. And the demo never, ever shows you that.

Nobody's migration slipped because they couldn't match two column names.

The thing that slips is the data

Schema is the easy 20%

Map the columns and you've described where data should go. You haven't touched the question of whether the data is fit to go there. That's a different job, and it's the big one:

  • A status column with seven canonical values in the spec and forty-one in the wild — ACTIVE, active, Actv, A, active, actve, null.
  • Three source systems that each "own" the customer, disagreeing on the address for a third of them.
  • Dates as strings, in four formats, two of them ambiguous about month vs. day.
  • Foreign keys that point at rows that were deleted in 2019.
  • A free-text notes field that, it turns out, is where the business has been quietly storing the real account owner for a decade.

None of that is a schema problem. The schema is fine. The data is the deadline.

Important

A migration plan that budgets for mapping and treats cleanup as "we'll handle edge cases as they come up" isn't a plan with a risk in it. It is the risk. Data quality isn't the edge case — for most of these projects it's the majority of the work, and it's the part with no schema diagram to make it look finished.

This is why migrations die quietly. The mapping is done in week one and everyone feels fast. Then someone runs the load, the constraints reject a few hundred thousand rows, and the project disappears into a months-long swamp of analysts reconciling values by hand — the work that was always going to be the work, just discovered too late to have been priced.

Why "agentic data quality" is suddenly everywhere

If it feels like every data vendor started saying "agentic data quality" at once, that's not a coincidence of marketing. It's that the shape of the problem finally fits the shape of the tool.

Data cleanup is the canonical bad job for deterministic code and the canonical good job for a capable model. It's a million small judgment calls — is Actv the same as ACTIVE? does this 2019 address or this 2024 one win? is this free-text note an owner name or a reminder? — each one easy for a human, none of them worth a human's afternoon, and all of them, in aggregate, worth a quarter. That's precisely the work agents are good at: high-volume, context-heavy, individually-cheap reasoning that you'd never write a rule for.

What changed Before Now
Normalizing messy values Rules per field, brittle Model infers intent from context · Generalizes
Reconciling conflicting sources Analyst judgment, one row at a time Agent proposes, cites, ranks confidence · Scales
Mining free-text fields Mostly ignored Read and structured at volume · New capability
Reviewing the cleanup Spot-check a sample, hope Every change diffed + attributable · Still needs a human

The capability didn't exist in usable form two years ago. Now it does — and the data world is, correctly, having a moment about it.

The reframe: not a mapping toy, an analyst-cleanup compressor

Here's the move. Stop selling AI as the thing that draws the join lines. Anyone can demo that, and it was never the hard part. Sell it as the thing that does the part nobody demos: the months of analyst cleanup, compressed into days.

The mental model isn't "AI matched my schema." It's "the AI did the cleanup a team of analysts would have spent a quarter on, and left a reviewable trail of every decision." The agent doesn't replace the analyst's judgment — it applies that judgment a hundred thousand times, drafts every fix, and hands a human the diff instead of the drudgery.

flowchart TD
    A[Profile the source<br/>find the real mess]:::flow --> B[Agent proposes a fix<br/>per value, with citations]:::flow
    B --> C{Confidence &<br/>blast radius}:::flow
    C -->|Clear-cut, high confidence| D[Batch for review]:::flow
    C -->|Ambiguous or high-impact| E[Escalate to analyst]:::risk
    D --> F[[Human reviews diffs<br/>by pattern, not by row]]:::human
    E --> F
    F -->|Approve| G[Apply to target]:::good
    F -->|Reject / correct| H[Send back, agent learns the rule]:::flow
    H --> B
    G --> I[(Append to<br/>cleanup audit log)]:::audit
    F -.every decision.-> I

The analyst stops cleaning rows and starts approving patterns. The agent does the volume; the human keeps the judgment — and every call is logged.

The leverage isn't that the agent is smarter than your analyst. It's that your analyst reviews patterns — "normalize these 41 spellings to these 7, here's the mapping, approve?" — instead of editing values one cell at a time. A quarter of linear cleanup becomes a few days of high-level review.

The reviewer should approve decisions, not do them

The whole compression comes from changing what the human touches. If your tool makes an analyst confirm every individual cell, you've automated nothing — you've added a step. The win is the agent collapsing a hundred thousand edits into a few dozen reviewable rules, each one citing the evidence behind it.

Cleanup rules you can hand to an auditor

The judgment the agent applies shouldn't live in a prompt nobody can see. It should be configuration — legible, version-controlled, and the same artifact you show a client to explain exactly what was done to their data.

cleanup.policy.yamlyaml
version: 1
job: customer-migration

# Normalize messy values toward a canonical set. The agent infers intent;
# anything it isn't sure about goes to a human instead of guessing.
normalize:
  status:
    canonical: [active, suspended, closed, pending]
    min_confidence: 0.90        # below this, escalate — never silently coerce
  dates:
    target_format: iso8601
    ambiguous_dmy: escalate     # 03/04/2026 is a question, not an answer

# When sources disagree, prefer by rule — and record that they disagreed.
reconcile:
  address:
    prefer: most_recent_verified
    on_conflict: flag_for_review
  owner_name:
    mine_from: [notes, legacy_owner_field]   # the real owner hides in free text

# Nothing is applied without an attributable human approval.
gates:
  require_human_approval: true
  auto_apply: false
  review_unit: pattern          # approve mappings, not individual rows

evidence:
  require_citations: true       # every fix points back to its source value
  retain_audit_log: append_only

The point isn't this exact schema. It's that "we cleaned your data" stops being a claim and becomes an inspectable record of which rule, on what evidence, approved by whom.

Audit-ready cleanup, because data quality work is never trusted on faith

A clean target table proves nothing on its own. The first question any serious data owner asks after a migration is what did you change, and how do I know it's right? If your answer is "trust the model," you've lost them.

So every agent-proposed fix should leave an entry like this — written once, never edited:

cleanup-audit-entry.jsonjson
{
  "change_id": "fix_2a7e91",
  "record": { "table": "customer", "id": "C-880142" },
  "field": "status",
  "before": "Actv",
  "after": "active",
  "rule": "normalize.status",
  "proposed_by": "agent:cleanup@v2.1",
  "confidence": 0.97,
  "sources": ["source:crm.status_raw"],
  "reviewed_by": "a.analyst@nutility.example",
  "decision": "approved",
  "decided_at": "2026-06-18T16:40:02Z",
  "reversible": true
}

Read it the way the data owner will. You can see what changed, from what to what, which rule drove it, how confident the agent was, what source it cited, which human approved it, and that it can be rolled back. That object is the difference between "we cleaned it up" and "here's the receipt."

The failure mode to design against

The dangerous pattern isn't an agent that's occasionally wrong about a value. It's an agent whose changes are indistinguishable from the original data once they're applied. If you can't tell what the cleanup touched, you can't validate it, can't explain it, and can't undo a bad call cleanly. Attribution isn't paperwork — it's what makes the speed safe.

Where the compression quietly breaks

Under deadline, the temptation is to let the agent auto-apply the "obviously safe" fixes and only review the hard ones. Don't. The moment some changes skip the gate, your clean table has edits no one signed off on — and you won't find them until the data owner does. Either every fix is reviewed and attributable, or the cleanup you're proud of has a hole in it.

The economics this sets up

Now the timeline math changes, and it changes the thing buyers actually care about. When the cleanup is the unknown — months of analyst time you can't size until you're deep in it — nobody can quote a migration honestly. The dirty data is precisely what makes "it depends" the only safe answer, and "it depends" is what turns into time-and-materials and a slipping date.

Note

Compress the cleanup and you don't just go faster — you make the previously un-sizable part sizable. The work that used to be an open-ended swamp becomes a profiled, bounded, reviewable job. That's the real unlock: not the speed itself, but that the speed is predictable.

When the longest, least-predictable phase of a migration collapses from a quarter into days — and every change in it is profiled up front and attributable after — you can finally say what the project costs and when it's done, before you start. Fixed price. In weeks. Not because you're brave, but because the part that used to make that promise reckless is the part you just compressed.

The schema was never the deadline. The data was. Compress the data work and the deadline becomes a date you can actually name.

Why "in weeks" stops being a gamble


Migrations don't slip on schema — they slip on the data underneath it. That's the work Nutility is built to compress: agent-driven cleanup, every change human-checked and audit-ready by default. Talk to us about what your dirty data is really costing you.

0 comments

Comments are closed for this post.