Integrations · Streaming

Streaming test data management

Event streams carry the same customers and transactions as databases, one message at a time. Testing consumers needs realistic events, in order, with keys that match the systems they update.

Unbounded consumption is the risk: a test-data job must read a defined window, not follow a topic forever.

What test data management needs here

  • Bounded snapshot or replay windows defined by offsets or time.
  • Schema-registry-aware decoding and deterministic transformations of keys and payload fields.
  • No offset commits on the consumer groups that production applications use.

How DataNivra approaches it

  • The Kafka connector reads the topics you declare as a bounded snapshot window: from the log start or a start time up to the last stable offset when the job starts. It never joins a consumer group or commits offsets, and it proves from the broker ACLs that its credential cannot write before reading.
  • JSON fields become columns for the same classification, deterministic masking and certification as database columns. Avro and Protobuf with a Schema Registry are not decoded yet.
  • Whatever the system, row-level work happens in the DataNivra agent inside your network: only metadata, aggregates and evidence reach DataNivra Cloud.

Streaming: connector status (1)

1 of these connectors can be used today (available or in preview); the others are shown with their honest status. Statuses are derived from recorded conformance evidence.

Available now

Apache Kafka

Shipped in the standard agent and backed by recorded conformance evidence.

Kafka connector: declared topics read as a bounded snapshot window (log start or a start time up to the last stable offset when the job starts), with JSON fields, text or binary values, record metadata columns and offset-range row counts. It never joins a consumer group and never commits offsets.

Available now — details about Apache Kafka

Learn, try, then start

See the whole workflow — discovery, classification, masking, subsetting and certification — in the synthetic playground, then start a free trial. The product overview explains how the customer-resident agent and DataNivra Cloud divide the work.

Other categories: Databases · Data warehouses · Lakehouses · Object storage · Files · NoSQL · SaaS applications · Mainframe · APIs.