Protect Your Reports and AI from Unexpected Data Changes

Data engineering guide

Protect Your Reports and AI from Unexpected Data Changes

A Practical Guide to Data Contracts

A source system can change while a report still refreshes successfully. The result may omit invoices, misstate an account picture or pass incomplete information to an AI workflow. This guide shows how a data contract connects producer and consumer expectations with executable checks, clear ownership and a controlled response.

Read the practical approach

Protect Your Reports and AI from Unexpected Data Changes guide banner

A practical guide to agreeing, checking and safely changing the data that feeds reports and AI. Includes a worked invoice example and a single page release checklist.

Inside the guide

  • A practical agreement for one source to consumer flow
  • An invoice example that shows exactly where matching fails
  • Checks for structure, meaning, quality and delivery
  • A change and recovery process with a single page release checklist
8 page guideWorked invoice exampleRelease checklist

The practical approach

A successful refresh can still leave invoices out

Imagine a daily ERP invoice feed used in customer account summaries. The source team changes customer identifiers from text to numbers. The identifier 00123 becomes 123, while the customer master still holds 00123. If a landing step converts the number back to text, the missing zeroes do not return. An exact text match may now fail even while the feed arrives and the report refreshes.

A data contract records what the producer provides and what the consumer needs. For this flow, it should identify the owners, the meaning of the customer key, the expected delivery, the checks that must pass and the action when they fail. The Open Data Contract Standard includes structure, quality, ownership, support and service expectations. The specific rules below are KCM recommendations for this flow.

Agree on the boundary before the next change

Start with one source, one business output and the people who own each side. State that customer ID is a five character text identifier and that leading zeroes matter. Name the ERP owner, the account reporting owner, the person who accepts a release and the support contact. Record the business date, time zone, expected batch and how corrections are identified.

Make the agreement versioned. Give consumers notice of a proposed change, identify the reports and AI inputs that use the field, and test the proposed version against the actual consumer logic. A technical compatibility result does not establish that account totals and matches are still correct.

Put the checks where the data travels

  • At input: Reject a numeric customer ID when the active agreement requires text. Check required fields and the expected contract version before conversion can erase the evidence.
  • Before publication: Check missing and duplicate identifiers, unmatched invoices, customer matches, source counts and amounts. Include the unmatched records in reconciliation.
  • At delivery: Confirm the expected period and batch arrived by the agreed deadline. A missing result is not a healthy result.
  • During release testing: Include normal loads, manual files, corrections and replay. Deliberately send invalid data and prove the report stays on hold and the right owner is notified.

A written rule needs an execution point. Confirm that checks run on every route to the consumer. Confluent documents route and executor limits for its own implementation; the general lesson is to test the route you actually use.

Invoice batch

$2,600 arrived. Only $600 matched.

Three invoices use customer keys 00123, 00456 and 12000, with amounts of $1,200, $800 and $600 CAD. After a numeric conversion, the first two keys become 123 and 456. An exact text inner join retains only the third invoice. The other two, totaling $2,000, are unmatched.

Reconcile all three source invoices against both matched and unmatched groups. Testing only the joined rows hides the loss. Hold the affected summary and obtain a correct reexport or approved mapping from the authoritative source. Guessing the missing zeroes could attach an invoice to the wrong customer.

Make recovery part of the release decision

Before cutover, record the current and proposed contract versions, affected consumers, test results, approval and contingency. If a check fails, reject or quarantine invalid input, preserve the batch and failure evidence, and hold affected publication. After authoritative correction, replay with batch and invoice identifiers that prevent duplicates. Reconcile the counts and amounts before the consumer accepts the result.

If inaccurate data was already published, identify the affected outputs and periods, tell their owners, and correct or withdraw them through the agreed process. Refresh dependent summaries and AI data sources where needed. Close the incident after consumers verify the corrected result.

This guide focuses on changes at one producer to consumer boundary. A contract cannot prevent every data problem or prove an AI answer is correct. For monitoring after release, read Catch Data Problems Before Your Users Do. For agreed business measure definitions, read the metric contracts guide.

Get the full guide and release checklist

Use the worked example, agreement template and single page checklist to review one important source change.

Download the guide

Bring one important data flow and its next change to the table.

Explore KCM Data Engineering