Documentation

Agent installation

Install, verify and run the customer-resident DataNivra agent — packages for Docker, Compose, Kubernetes, Azure and AWS, network, identity, secrets, the doctor and optional isolated execution.

What the agent is

The agent is a container you run inside your VPC, VNet, Kubernetes cluster or data centre. It hosts the connectors, the TDM engine and the industry packs, and it is the only DataNivra component that ever reads your source data.

Network requirements

  • Outbound HTTPS from the agent to your DataNivra control-plane endpoint. Proxies are supported.
  • No inbound ports. The control plane never connects into your network; the agent polls for work.
  • Network reachability from the agent to the sources it reads (read-only) and the targets it provisions.

Identity and enrollment

  1. An administrator with the agent-management permission creates a one-time enrollment token in the console. It is short-lived and single-use.
  2. On first start the agent generates its own key pair and enrolls with that token, registering only its public key.
  3. From then on the agent exchanges a signed assertion for short-lived access tokens. Revoking the agent in the console takes effect at its next token exchange, heartbeat or lease request, and a revoked agent stops all work.

Secrets

Source and target credentials, and masking keys, stay in your secret store. DataNivra configuration refers to them by URI:

SchemeExampleStatus
env://env://DATANIVRA_TDM_READER_PASSWORDBuilt in (names must start with an allowed prefix, DATANIVRA_ by default)
file://file://readerBuilt in (under the agent's secrets directory; file mode 0600 or 0400)
k8s://k8s://tdm/reader-credentialsBuilt in (mounted Kubernetes secrets)
azure-kv://azure-kv://tdm-vault/qa-db-passwordNeeds a provider plugin in a derived agent image, or stage the secret into a file
aws-sm://aws-sm://prod/tdm/readerNeeds a provider plugin in a derived agent image, or stage the secret into a file
vault://vault://secret/tdm#readerNeeds a provider plugin in a derived agent image, or stage the secret into a file

The agent resolves these locally at run time. The resolved values never leave your environment.

Masking key format

Masking rules that need deterministic output (HMAC_PSEUDONYMIZE, TOKENIZE, any rule with preserve_linkage) reference a keyset — a JSON document you create and own:

{"primary": 1, "keys": {"1": "<base64url-encoded random bytes, ≥ 32>"}}

Generate one and write it to the agent's secrets directory:

python3 -c '
import base64, json, secrets
doc = {"primary": 1, "keys": {"1": base64.urlsafe_b64encode(secrets.token_bytes(32)).decode()}}
print(json.dumps(doc, separators=(",", ":")))
' | sudo install -m 0600 -o 10001 -g 10001 /dev/stdin /run/datanivra-secrets/masking-key

Then reference it as file://masking-key in the masking policy wizard. For rotation, add a second version ("2": "...") and change "primary" to 2; old datasets stay linkable by keeping the old version in the keys object.

The agent returns SECRET_NOT_JSON_OBJECT if the file is not JSON and KEYSET_INVALID if it is JSON but not in the keyset shape above. Neither error is logged to the control plane; check the agent's local log (docker logs <agent>) for the precise code.

Check the host: datanivra-agent doctor

Run datanivra-agent doctor where the agent runs, before or after enrollment. It checks control-plane DNS, outbound HTTPS, TLS and health, clock skew, local storage and permissions, the agent and protocol versions, and — for each --source-ref or --target-ref you pass — that the secret reference resolves, the connector ships, the source opens and its read-only proof passes, or that the target is writable. Each result has a status, a stable reason code and a link to its fix in the support center; nothing it prints contains a secret value or a row.

Failure behaviour

If the control plane is unreachable, the agent finishes or aborts in-flight work safely, never publishes uncertified output, and does not start new work without a valid lease.

Install it

Every release on Downloads lists a signed, digest-pinned multi-architecture image (linux/amd64 and linux/arm64), the Helm chart and one package per install method, each with its checksum.

MethodPackageBest for
Docker (one command)datanivra-agent-docker-<version>.tar.gzone Linux host
Docker Composedatanivra-agent-docker-compose-<version>.tar.gzone Linux host, restart policy and volumes declared
Kubernetes (Helm)datanivra-agent-<version>.tgz + datanivra-agent-<version>-helm-values.yamlclusters; restricted security context and NetworkPolicy by default
Kubernetes (kustomize)datanivra-agent-kubernetes-generic-<version>.tar.gzclusters without Helm
Azure VNetdatanivra-agent-azure-customer-vnet-<version>.tar.gza VM in your VNet, no public IP
AWS VPCdatanivra-agent-aws-customer-vpc-<version>.tar.gzECS Fargate with encrypted EFS

The steps are the same everywhere:

  1. In the console, open Agents → Register an agent and create a one-time enrollment token.
  2. Download the package and verify it (below).
  3. Store the token in your secret store and point DATANIVRA_AGENT_ENROLLMENT_TOKEN_REF at it; set DATANIVRA_AGENT_CONTROL_PLANE_URL.
  4. Start the agent with the package's command (the console's Agents page shows it pre-filled), then run datanivra-agent doctor.

Verify what you download

# 1. The checksum file was signed by the DataNivra release workflow for this version
cosign verify-blob --bundle SHA256SUMS.cosign.bundle \
  --certificate-identity 'https://github.com/manuelbomi/datanivra-platform/.github/workflows/release-agent.yml@refs/tags/agent-v<version>' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' SHA256SUMS
# 2. Every downloaded file matches
sha256sum -c --ignore-missing SHA256SUMS
# 3. Deploy the image by the digest in release-manifest.json, never by a tag
cosign verify ghcr.io/manuelbomi/datanivra-agent@sha256:<digest> \
  --certificate-identity 'https://github.com/manuelbomi/datanivra-platform/.github/workflows/release-agent.yml@refs/tags/agent-v<version>' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com'

Upgrade and roll back

Upgrade by deploying the new release's image digest with the same state volume: the agent keeps its identity and resumes interrupted work from verified checkpoints. Roll back by deploying the previous digest. The control plane can require a minimum agent version, and it never dispatches a job to an agent that does not advertise what the job needs.

Viewing provisioned datasets with DBeaver

DBeaver Community is a free, open-source SQL client that connects directly to the target PostgreSQL database in your environment. It is a convenient way to browse provisioned test datasets without writing SQL.

Connect to a lab target:

  1. Open DBeaver and choose Database → New Database Connection.
  2. Select PostgreSQL and click Next.
  3. Enter the target host, port (5432), database name, username and password from your target credentials file (e.g. lab-target-qa.json).
  4. Click Test Connection to verify, then Finish.

The connection is made from your machine directly to the target database — no data passes through DataNivra Cloud. Use read-only credentials to browse provisioned datasets; use a write-capable account only when you need to reset or repopulate a target.

DBeaver resolves container names only on the agent's Docker network. If the target runs as a Docker container (e.g. lab-target), connect from inside the Docker network or map the port to localhost (e.g. add -p 127.0.0.1:5433:5432 to the target container's docker run command and connect to localhost:5433).

Optional: isolated execution

For stronger separation between row-level work and the agent itself, the agent can run every stage in a fresh, hardened container, a gVisor sandbox or a Firecracker microVM, and your organization can require it. See Isolated execution.

← All documentation