Tutorial
A test database is a web of references. An order points at a customer, a claim points at a member and a provider, a transaction points at an account. Referential integrity is the simple rule that every one of those pointers lands on a record that actually exists.
In production, the database engine usually enforces this for you. In test data, the rule is easy to break without noticing, because subsetting removes records and masking rewrites the very keys that link them together.
Text description
- Parents: customer (key C-1001) → account (key A-2001 → C-1001)
Connection: Every child key resolves to a parent; masked keys stay consistent
- Children: transaction → A-2001 → card → A-2001 → statement → A-2001
Why it matters
Broken references produce defects that are not real. A screen crashes because an order has no customer, a batch job skips records it cannot join, and testers spend hours chasing problems that only exist in the test copy. Worse, some breakage is silent: a report simply shows fewer rows and nobody questions it. References also cross system boundaries. A customer key in the CRM must match the same customer in billing and support, or end-to-end tests in SIT and UAT stop meaning anything. Integrity is therefore not a nice-to-have; it decides whether test results can be trusted.
Text description
- Subsetting (Your environment): Select driving entities → Follow keys to parents → Include required children → Close the reference set → Verify no dangling keys
Example
Consider a small synthetic banking dataset (all values invented for this tutorial):
| Table | Key | References |
|---|---|---|
| customer | C-1001 | — |
| account | A-2001 | customer C-1001 |
| card | K-3001 | account A-2001 |
| transaction | T-4001 | account A-2001 |
Two things can go wrong:
- Subsetting without closure. A filter keeps transaction T-4001 but drops account A-2001. The transaction now points at nothing.
- Inconsistent masking. The customer key C-1001 is masked to C-7734 in the customer table but to C-5120 in the CRM export. Each table looks fine on its own; the join across systems returns no rows.
The fix for the first is Referential closure: follow references until nothing dangles. The fix for the second is deterministic, keyed masking, so the same input always produces the same output everywhere it appears.
How DataNivra approaches it
Subsets are built from a Business entity outward. The engine follows declared and discovered relationships to bring in required parents and children, then verifies there are no dangling keys before the dataset moves on. Industry packs contribute relationship maps for common domains, so links that are not declared as database constraints are still respected.
Masking policies use deterministic, keyed transformations. The key lives in your secret store and is referenced by URI, never sent to the control plane, and every system masked with the same policy version produces matching substitutes. Referential integrity is also one of the certification gates: if any check fails, the dataset is blocked and never provisioned. All of this runs as customer-resident processing; the control plane only receives counts and gate outcomes. See data subsetting and dataset certification.
Common pitfalls
- Trusting declared constraints only. Many applications enforce relationships in code, triggers or batch jobs rather than foreign keys. Those relationships must be declared to the subsetting step or the subset will leave orphans.
- Relationships across systems. A customer id in the CRM, the billing database and a file export is one relationship even though no database knows about it. Deterministic masking with one key keeps them aligned.
- File sources carry no constraints. Parquet and CSV extracts have no foreign keys at all, so relationships must come from policy or an industry pack.
- Masking keys with collisions. A pseudonymisation that maps two different ids to the same value silently merges two customers. Use collision-free strategies for keys.
- Checking integrity by eye. Orphan detection should be automatic and should block a dataset, not produce a warning someone reads later.
The subset-size calculator among the free tools shows how relationship fan-out affects size. The guides explain how relationships are discovered per database, and the integrations page lists the available sources. See the documentation, or start free and explore relationships in the synthetic sandbox.
Key takeaways
- Every reference must resolve, inside a database and across systems.
- Subsetting needs closure; masking needs determinism.
- Integrity should be verified automatically before anyone uses the data.