Master data

Under the hood

Free attribute evolution and native query speed at the same time: a fixed table plus JSON attributes, with the declared ones promoted to indexed generated columns

Master data has one hard requirement that pulls in two directions. The attribute set belongs to the application, so it must be able to change without a platform migration. And the entities are read on hot paths - a checkout, a form's type-ahead, a join in a report - so lookups must be indexed and fast.

The usual answers pick a side. A rigid schema means every new product attribute is a migration somebody has to write for you. A pure key-value store means every filter is a full scan.

The storage shape

Each entity is a real table with a fixed frame - code, activity, timestamps, audit - plus an attributes column holding the rest as JSON. Attributes the application declares as filterable, sortable or joinable are promoted to generated columns and indexed.

So both properties hold at once: an attribute nobody declared can still be written and read immediately, and an attribute that matters is a native indexed column with the query plan to match. Promotion is a declaration in the application, not a request to us.

It is a table in your database, not a service behind an API

Entity tables live in a schema of their own, per organisation, in the same cluster as the rest of that organisation's data.

That is the decision the whole block exists to make. Master data reachable only over another service's HTTP interface means every consumer pays a network hop, cannot index what it filters on, cannot join, and inherits that service's availability on paths that cannot afford it. Here a report joins products to sales in one query, and a process reads a counterparty in the query that reads the request.

The schema is separate from the platform's own, deliberately: tables whose structure is owned by an application must not sit among tables whose structure is owned by platform migrations.

Guardrails on app-declared structure

Letting an application define tables is powerful and needs to be boring in exactly the right places.

  • Identifiers are validated in shape, and reserved prefixes are refused - for entity and attribute names, where an application chooses them.
  • Every reference is schema-qualified. An entity whose name collides with a platform table cannot silently resolve to the wrong one - a failure that is invisible until it is expensive.
  • A type change on a live attribute is refused, as is a change to an entity's key policy. Both are ways to fork or orphan existing codes without touching a single row.
  • The key pattern is a restricted grammar, so an application cannot supply an expression that is cheap to write and catastrophic to evaluate.

Codes are immutable, entries deactivate

An entry's code is its identity everywhere else - in a balance, in a task field, in a report's dimension - and those references cross service boundaries where no database can enforce them.

So the code cannot be changed, and an entry is deactivated rather than deleted. A renamed or removed code would not fail loudly; it would silently orphan whatever pointed at it, and the damage would surface as a number nobody can explain.

Seeding follows the same logic. When an application is applied, each seed row is inserted only if its code is absent, and never overwrites a row that exists. Two consequences, both intended: an operator's correction survives every subsequent update, and a seed row added in a later version does reach installations that already existed. Seeding runs on every apply, not only on first install, so a seed row that once failed validation can arrive later, after the author fixed it.

Where it joins the rest of the platform

Master data earns its keep by being read.

Processes read it as they route. A field on a request binds to an entity, so the form offers the real suppliers and the route can branch on a counterparty's category without a copy of that list living inside the process. See Processes.

Analytics joins it as a dimension. Sales by product category, movements by warehouse, requests by counterparty type - a reference entity is exactly the shape an analytical model wants on the other side of a join, and because it is a real table in the same place, the join is ordinary. See Analytics and the data engine that runs it.

The ledger keys balances by its codes. How much of something there is belongs to the ledger; what the something is belongs here. The ledger holds a code as an opaque string and never resolves it, which is what lets one register serve stock, points and settlements without knowing what any of them are.

Spreadsheets bind columns to it, so a plan typed in a grid names the same warehouses as the system it will be executed by. See Spreadsheets.

Pages and portals render and drive navigation from it. An entity gets management screens without a hand-authored page, and a menu of categories can be the category list, not a copy of it. See Portals and pages.

People are next door, not here. Employees, positions and departments have their own lifecycle, their own substitution rules and their own resolution logic - a different subject with a different shape. See Org structure.

One honest boundary worth naming: because the ledger is a separate service, there is no database-level foreign key between a balance's dimension value and a catalogue entry. A mistyped code opens a real account with a real balance and the ledger's own zero-sum check will not notice, because both sides of the posting still net. Canonicalising codes and binding an operation to a catalogue is the mitigation, and it is an application-level one - which is a weaker guarantee than a real foreign key, and worth knowing rather than discovering.