Codabra
← Deep SQL & Production Data Engineering

Data contracts — the column rename that didn't take down prod

A versioned schema agreement between producer and consumer, enforced in CI. The lesson where `column rename` becomes a tracked event, not a 2 AM page.

What is a contract

A data contract is a versioned, machine-readable schema that the producer of a table publishes and the consumer relies on. It states: "this table will have these columns, with these types, this nullability, and these invariants — and we will tell you (and bump the version) before we change it."

It's the same idea as a REST API contract or a gRPC .proto file — applied to tables. The point isn't to prevent changes; it's to make changes visible and give consumers time to adapt.

Producer / contract / consumer / CI

Three relationships matter:

  • Producer ↔ contract: the producer owns the contract. Any code change that touches the schema must update the contract in the same PR.
  • Consumer ↔ contract: the consumer reads the table through the contract. If a column it relies on is missing in the YAML, the consumer's CI fails before deploy.
  • PR ↔ CI gate: the gate compares the live schema to the YAML on every PR. Breaking changes (rename, type change, drop) fail unless the contract version is bumped — which itself requires consumer sign-off in the social half of the contract.
A minimal contract — checked in next to the producer's code
# contracts/oltp/orders.yml
table: orders
schema: oltp
owner: backend-team@codabra.app
version: 3                                # bump when the schema changes
columns:
  - name: order_id
    type: bigint
    nullable: false
    primary_key: true
  - name: customer_id
    type: bigint
    nullable: false
    references: customers.customer_id
  - name: status
    type: text
    nullable: false
    accepted_values: [pending, paid, refunded, cancelled]
  - name: total_cents
    type: integer
    nullable: false
    invariant: ">= 0"
  - name: created_at
    type: timestamptz
    nullable: false
freshness:
  max_lag_minutes: 15                     # producer commits to this
breaking_changes_require: 14_days_notice  # the social half of the contract

How the contract becomes load-bearing

A contract sitting in a YAML file is a wish. It becomes load-bearing when CI checks two things on every PR that touches the producer's code:

  1. Schema diff: actual schema (from information_schema.columns) matches the contract. Adding a column with nullable: true is a non-breaking additive change. Removing a column, renaming, or changing a type is breaking — fail CI unless the contract version is bumped and the consumer side has been notified.
  2. Behavioral checks at runtime: accepted_values, invariant, freshness SLO. These run nightly; alert (not block) on failures.

The rename outage from Module 00 is exactly this: with a contract, the rename PR fails CI; without one, it ships and breaks dashboards at 2 AM.

Implement an `accepted_values` check on `orders.status`. The contract says status must be in `('pending', 'paid', 'refunded', 'cancelled')`. Return one row, two columns: `bad_rows` (count of rows with status outside the set) and `passes` (true when bad_rows = 0).

Takeaway: a contract is a versioned schema published by the producer, checked by CI on every change, and enforced by behavioral tests at runtime. With it, schema changes become tracked events. Without it, they become 2 AM pages.