← Forward Deployed Engineer
Forward Deployed Engineer

Messy Real Data vs. Clean Demo Data

28 Oct 2026·3 min read

Demo data is clean because you made it. Customer data is messy because nobody made it for you — it was typed by hand over ten years, migrated twice, and half the fields mean something slightly different than the docs say.

What actually goes wrong, in order

  • Fields that are technically filled in but wrong — a phone number sitting in the email field, left over from an old migration.
  • Duplicates that look different enough on the surface that simple matching misses them completely.
  • Values nobody told you exist — a "status" field with forty distinct values when the docs list only six.
Why this is not a code quality problemNone of the three above are bugs in the usual sense — the code works exactly as written. The gap is between what the data was supposed to look like and what it actually looks like after years of real use. No amount of clean code fixes a wrong assumption about the data.
Do this in 60 secondsDownload any public dataset from Kaggle or data.gov. Spend 20 minutes just looking — count distinct values in one "category"-style column, check for near-duplicate rows, and check whether any field that should always be filled in has blanks. Write down the three ugliest things you find before writing a single line of cleanup code.
Today’s AI angleAn LLM trained to answer from a customer’s data inherits every one of these problems silently — a confidently wrong answer from messy data is harder to catch than an outright crash.
Interview prepWalk me through how you would validate a dataset before building anything on top of it. I would first sample real rows, not just read the schema, and look for fields that are technically filled but semantically wrong. Then I would count distinct values on fields the docs describe narrowly, because that is where undocumented reality usually hides. Only after that would I write any cleanup or ingestion code — writing code first just means rewriting it once the real shape shows up.
Messy Real Data vs. Clean Demo Data · Ali Huzaifa