RENEWA Knowledge Base

Runbook: Secret Management Operations

2 min read

Runbook: Secret Management Operations

How to change, read, and reason about secrets during operations — including mid-incident, when third-party logins may not be available.

Source of truth

Infisical EU (https://eu.infisical.com, project renewa-one) holds all runtime app secrets. Delivery per environment:

EnvDelivery
development / staging / productionInfisical → Fly native sync (Auto-Sync + Auto Redeploy ON)
pr-previewCI fetches via OIDC (Infisical/secrets-action) → flyctl secrets set --stage on the ephemeral app
localInfisical CLI (make env-pull, .infisical.json)

GitHub Actions secrets hold CI-infra credentials only (FLY_API_TOKEN, FLY_MPG_TOKEN, …) — never app runtime secrets.

Changing a secret

  1. Update the value in Infisical (correct env scope).
  2. Synced envs redeploy automatically (Auto Redeploy); pr-preview picks it up on next deploy.

NEVER flyctl secrets set on a synced app — the next Infisical sync overwrites it, and in the window before that you have split-brain config. Same rule for workflows: GH-driven secret pushes would race the sync.

Reading current secrets mid-incident (no Infisical login required)

Ops/recovery procedures deliberately avoid a third-service dependency. Fly secrets are write-only via flyctl, but the running machines have them in env:

flyctl ssh console -a <app> -C 'printenv DATABASE_URL'

This is how rollback-deploy.sh Path B reads the active DB URLs. Operator credentials needed: only the Fly token you already hold.

Connecting to the database

MPG hostnames (*.flympg.net) resolve only on Fly’s private network — public DNS does not even resolve them. From a laptop, always tunnel:

flyctl mpg proxy <cluster-id> -p 16380
psql "postgres://app-user:<pw>@localhost:16380/fly-db"

Key facts

  • ENCRYPTION_KEY: AES-256-GCM master key for data at rest (backend/src/lib/encryption.ts reads this name). Rotation tooling: I#1950.
  • E2E bot passwords (E2E_ADMIN_PASSWORD/E2E_EMPLOYEE_PASSWORD): independent values per scope (local/pr-preview/development), absent in staging+production.
  • After a Path B database restore, the cluster host changes → DATABASE_URL/DATABASE_URL_MIGRATION must be re-pointed in Infisical; see Deployment Rollback.

Graph View

Backlinks