Integrations · Object storage

Object storage test data management

Object storage is the most common hand-off point between systems: nightly extracts, partner files and analytics exports land as Parquet or CSV under a bucket prefix.

Those files are often forgotten copies of production data, so a test-data process should read them in place, mask them, and help retire them.

What test data management needs here

  • Reading Parquet and CSV in place, with declared table names so that data-derived file names never leave your network.
  • Least-privilege read access; object stores rarely expose permissions in a way a client can verify.
  • Declared relationships between files, because files carry no foreign keys.

How DataNivra approaches it

  • File sets map operator-declared logical tables to paths; undeclared names become opaque hashes.
  • Where read-only access cannot be introspected, the agent requires an explicit attestation and fails closed without it.
  • Whatever the system, row-level work happens in the DataNivra agent inside your network: only metadata, aggregates and evidence reach DataNivra Cloud.

Object storage: connector status (3)

2 of these connectors can be used today (available or in preview); the others are shown with their honest status. Statuses are derived from recorded conformance evidence.

Available now

Azure Blob Storage / Data Lake Storage

Shipped in the standard agent and backed by recorded conformance evidence.

Built-in file-set connector for Parquet or CSV files in an Azure Blob container or ADLS Gen2 filesystem, with account-key, SAS or service-principal authentication.

Available now — details about Azure Blob Storage / Data Lake Storage

Coming next

Google Cloud Storage

Active implementation; not yet available for production use.

File-set connector for Parquet, CSV or JSON-lines objects under a Cloud Storage bucket prefix, read through the Cloud Storage JSON API with bounded listings and ranged reads, into the same discovery, masking, subsetting and certification path as the other object stores.

Coming next — details about Google Cloud Storage

Learn, try, then start

See the whole workflow — discovery, classification, masking, subsetting and certification — in the synthetic playground, then start a free trial. The product overview explains how the customer-resident agent and DataNivra Cloud divide the work.

Other categories: Databases · Data warehouses · Lakehouses · Files · NoSQL · SaaS applications · Mainframe · Streaming · APIs.