Test data in CI/CD pipelines

The pattern

Every recipe is a thin wrapper over the same CLI or REST calls — there is no vendor-specific engine:

1. Authenticate — a CI workload identity (datanivra login --workload-identity …, see CI workload identity; off by default) or a user session (DATANIVRA_TOKEN).
2. Request a pinned profile version — datanivra test-data request <manifest> --ci-from-env --wait -o json. --ci-from-env adds the repository, pull/merge request, run and commit, so a re-run attaches to the live request instead of creating a second one, and derives a deterministic idempotency key for the run attempt.
3. Run tests on a runner that can reach the approved target (a self-hosted or network-connected runner). Target credentials are resolved by your agent and your runner; they never pass through DataNivra.
4. Release, always — on success, failure and cancel (if: always(), after_script, post { always }, a shell trap). If the runner dies, the lease's TTL tears the copy down.

Exit codes of datanivra test-data request --wait: 0 READY, 24 denied, 25 still waiting for a human approval at the timeout, 26 ended without becoming READY, 27 timed out.

Status of each platform

The index is examples/integrations/README.md. The earlier run and lease modes of examples/ci/datanivra_ci.py remain available.

GitHub Actions

name: integration
on: pull_request
permissions:
  contents: read
  id-token: write
jobs:
  integration:
    runs-on: [self-hosted, qa-network]
    env:
      DATANIVRA_BASE_URL: ${{ vars.DATANIVRA_BASE_URL }}
    steps:
      - uses: actions/checkout@<FULL_COMMIT_SHA>
      - run: datanivra login --workload-identity ${{ vars.DATANIVRA_WORKLOAD_IDENTITY_ID }} --tenant acme --audience datanivra:acme
      - run: datanivra test-data request test-data/pr-regression.yaml --ci-from-env --wait -o json > request.json
      - run: ./run-integration-tests.sh
      - if: always()
        run: datanivra test-data release "$(jq -r .id request.json)" --reason-code TEST_RUN_FINISHED

Fork pull requests: GitHub does not give id-token: write to pull_request workflows from forks, and trusts never accept pull_request_target, so a fork cannot obtain a workload session.

GitLab CI

integration-tests:
  tags: [qa-network]
  id_tokens:
    DATANIVRA_ID_TOKEN:
      aud: datanivra:acme
  script:
    - datanivra login --workload-identity "$DATANIVRA_WORKLOAD_IDENTITY_ID" --tenant acme --audience datanivra:acme
    - datanivra test-data request test-data/pr-regression.yaml --ci-from-env --wait -o json > request.json
    - ./run-integration-tests.sh
  after_script:
    - datanivra test-data release "$(jq -r .id request.json)" --reason-code TEST_RUN_FINISHED

Use --protected-refs-only on the trust when only protected branches may obtain test data.

Any other runner (REST)

REQUEST=$(curl -fsS -X POST "$DATANIVRA_BASE_URL/v1/test-data/requests" \
  -H "Authorization: Bearer $DATANIVRA_TOKEN" -H "Content-Type: application/json" \
  -H "Idempotency-Key: ci-$BUILD_ID" \
  -d '{"profile":{"name":"healthcare-integration","version":1},"purpose_ref":"QA-1234"}')
ID=$(echo "$REQUEST" | jq -r .id)
trap 'curl -fsS -X POST "$DATANIVRA_BASE_URL/v1/test-data/requests/$ID/release" -H "Authorization: Bearer $DATANIVRA_TOKEN" -H "Content-Type: application/json" -d "{\"reason_code\":\"TEST_RUN_FINISHED\"}"' EXIT
until [ "$(curl -fsS "$DATANIVRA_BASE_URL/v1/test-data/requests/$ID" -H "Authorization: Bearer $DATANIVRA_TOKEN" | jq -r .state)" = READY ]; do sleep 5; done
./run-integration-tests.sh

What never appears in CI logs

API responses carry ids, states, counts, checksums and reason codes — never rows, samples or credentials. Workload tokens are never printed by the CLI; validation errors name fields, not values.

Related

• Test Data API overview
• Lease vs revoke and troubleshooting

← All documentation