The Infrastructure Hub -- Part 5

Secrets and Post-Quantum Identities for Infrastructure

#platform-engineering #backstage #security #post-quantum #quantumvault #quantumid #terraform #secrets

The Problem

Open your Terraform state file. Find the azurerm_mssql_server resource. Look at the administrator_login_password attribute. There it is — in plaintext, inside a JSON blob that sits in a storage account.

Terraform state is a goldmine for attackers. It contains every secret your infrastructure uses: database passwords, API keys, service principal credentials, storage account keys. All of them stored as attributes of resources — in plaintext by default.

“But we encrypt the state at rest.” Yes, with Azure-managed keys. RSA by default. The same RSA that a quantum computer will break. And even without quantum — if someone gets read access to the storage account (a leaked SAS token, a misconfigured IAM policy, a compromised pipeline identity), they get every secret in your infrastructure.

That’s problem one: secrets in state.

Problem two: secrets in pipelines. Your terraform apply needs credentials to talk to Azure, AWS, or Scaleway. Those credentials live in pipeline variable groups, environment variables, or — worse — in backend.tf files that someone committed two years ago. They don’t rotate. Nobody knows which ones are still valid. And when someone leaves the team, nobody changes them.

Problem three: SSH keys. Your team connects to VMs, bastion hosts, and Kubernetes nodes with RSA SSH keys. Those keys were generated once, shared on Slack (yes, really), and haven’t been rotated since. They’re in ~/.ssh/ on five different laptops.

Problem four: service identities. Every Terraform module that creates a service principal or managed identity generates a credential. Where is that credential? In the state file. Who rotates it? Nobody. What algorithm does it use? RSA-2048 or ECDSA-P256 — both breakable by quantum computers.

The Quantum-Safe Cloud series covered post-quantum cryptography for applications. This article applies the same principles to infrastructure: Terraform state, pipeline secrets, SSH keys, and service identities.

The Solution

Four problems. Four solutions. All built into the Backstage platform.

1. Terraform state encryption with ML-KEM — Encrypt the state file with QuantumVault before it reaches the storage backend. Even if someone reads the storage account, they get ciphertext.

2. Pipeline secrets from QuantumVault — Replace variable groups and environment variables with runtime secret resolution. The pipeline fetches secrets from QuantumVault at execution time. Nothing stored in the CI/CD platform.

3. Post-quantum SSH keys — Replace RSA SSH keys with ML-DSA keys from QuantumAPI. Short-lived, auto-rotated, and managed from Backstage.

4. AI-powered secret sprawl detection — A Backstage plugin that scans your repositories and Terraform state for leaked or stale secrets. It uses the catalog enricher pattern from article 2 of the IDP series — but instead of enriching metadata, it looks for security problems.

Execute

1. Terraform State Encryption

HashiCorp Terraform encrypts state at rest through the backend — Azure Storage encrypts blobs, S3 encrypts objects. But that encryption uses the cloud provider’s keys (RSA by default), and the state is decrypted the moment someone reads the blob. The secrets inside are plaintext once you have storage access.

OpenTofu — the open-source Terraform fork — added native state encryption in v1.7. If your organization uses OpenTofu, you get an encryption block that encrypts state before it reaches the backend. With a custom key provider, you could point it at QuantumVault.

For HashiCorp Terraform (which most enterprises still use), the approach is different: encrypt the state file with QuantumVault as a wrapper around your backend operations. The qapi CLI handles this in the pipeline — encrypt after terraform state pull, decrypt before terraform state push.

Here’s the pipeline wrapper pattern:

# Encrypt state before pushing to the backend
terraform state pull > state.json
qapi encrypt --input state.json --output state.enc
# Upload state.enc to your storage backend

# Decrypt state before terraform operations
# Download state.enc from your storage backend
qapi decrypt --input state.enc --output state.json
terraform state push state.json
rm -f state.json state.enc  # Clean up plaintext immediately

For a more integrated approach, wrap terraform init with a backend configuration that uses a local state file, and add pre/post hooks in your pipeline that decrypt before plan/apply and encrypt after:

# In your pipeline YAML (any CI/CD platform)
steps:
  # 1. Download encrypted state from storage
  - script: |
      az storage blob download \
        --container-name tfstate \
        --name $MODULE_NAME.tfstate.enc \
        --file tfstate.enc \
        --account-name $STORAGE_ACCOUNT 2>/dev/null || true
    displayName: Download encrypted state

  # 2. Decrypt with QuantumVault (if state exists)
  - script: |
      if [ -f tfstate.enc ]; then
        qapi decrypt --input tfstate.enc --output terraform.tfstate
        rm -f tfstate.enc
      fi
    displayName: Decrypt state
    env:
      QUANTUMAPI_KEY: $(QUANTUMAPI_KEY)

  # 3. Run Terraform with local state
  - script: terraform init -backend=false
    displayName: Terraform init (local state)

  - script: terraform apply -auto-approve
    displayName: Terraform apply

  # 4. Encrypt and upload state
  - script: |
      qapi encrypt --input terraform.tfstate --output tfstate.enc
      az storage blob upload \
        --container-name tfstate \
        --name $MODULE_NAME.tfstate.enc \
        --file tfstate.enc \
        --overwrite
      rm -f terraform.tfstate tfstate.enc
    displayName: Encrypt and upload state
    env:
      QUANTUMAPI_KEY: $(QUANTUMAPI_KEY)

The result: your state is encrypted with ML-KEM-768 + AES-256-GCM at rest. Even with storage access, the blob is ciphertext. The plaintext state exists only during the pipeline run and is deleted immediately after.

For teams using OpenTofu, the native encryption block is cleaner — but the QuantumVault wrapper works with any Terraform version and any backend.

2. Pipeline Secrets from QuantumVault

In article 4, pipelines run from Backstage with annotations that point to Azure DevOps, GitHub Actions, or GitLab CI. Now we add secret resolution.

The pattern is the same one from article 5 of the Quantum-Safe Cloud series: install the qapi CLI, fetch secrets at runtime, and inject them as masked environment variables.

Azure DevOps:

# In your pipeline YAML
steps:
  - script: |
      curl -sfL https://cli.quantumapi.eu/install.sh | sh -s -- -b /usr/local/bin
      
      # Fetch infrastructure secrets from QuantumVault
      ARM_CLIENT_SECRET=$(qapi vault get $(ARM_SECRET_ID))
      TF_VAR_db_password=$(qapi vault get $(DB_PASSWORD_ID))
      QUANTUMVAULT_KEY_ID=$(qapi vault get $(STATE_KEY_ID))
      
      echo "##vso[task.setvariable variable=ARM_CLIENT_SECRET;issecret=true]$ARM_CLIENT_SECRET"
      echo "##vso[task.setvariable variable=TF_VAR_db_password;issecret=true]$TF_VAR_db_password"
      echo "##vso[task.setvariable variable=QUANTUMVAULT_KEY_ID;issecret=true]$QUANTUMVAULT_KEY_ID"
    displayName: Fetch secrets from QuantumVault
    env:
      QUANTUMAPI_KEY: $(QUANTUMAPI_KEY)  # Only bootstrap secret in ADO

GitHub Actions:

# In your workflow
steps:
  - name: Install qapi CLI
    run: curl -sfL https://cli.quantumapi.eu/install.sh | sh -s -- -b /usr/local/bin

  - name: Fetch secrets
    env:
      QUANTUMAPI_KEY: ${{ secrets.QUANTUMAPI_KEY }}
    run: |
      # Fetch secrets into variables first
      ARM_CLIENT_SECRET=$(qapi vault get ${{ vars.ARM_SECRET_ID }})
      TF_VAR_db_password=$(qapi vault get ${{ vars.DB_PASSWORD_ID }})
      QUANTUMVAULT_KEY_ID=$(qapi vault get ${{ vars.STATE_KEY_ID }})

      # Mask them so they never appear in logs
      echo "::add-mask::$ARM_CLIENT_SECRET"
      echo "::add-mask::$TF_VAR_db_password"
      echo "::add-mask::$QUANTUMVAULT_KEY_ID"

      # Then export to GITHUB_ENV
      echo "ARM_CLIENT_SECRET=$ARM_CLIENT_SECRET" >> $GITHUB_ENV
      echo "TF_VAR_db_password=$TF_VAR_db_password" >> $GITHUB_ENV
      echo "QUANTUMVAULT_KEY_ID=$QUANTUMVAULT_KEY_ID" >> $GITHUB_ENV

The ::add-mask:: step is critical. Without it, GitHub Actions doesn’t know these values are secrets — they would appear in logs. Azure DevOps uses issecret=true for the same purpose. Always mask before exporting.

GitLab CI:

# In .gitlab-ci.yml
.fetch-secrets:
  before_script:
    - curl -sfL https://cli.quantumapi.eu/install.sh | sh -s -- -b /usr/local/bin
    - |
      # Disable trace to prevent secrets from appearing in job logs
      set +x
      export ARM_CLIENT_SECRET=$(qapi vault get $ARM_SECRET_ID)
      export TF_VAR_db_password=$(qapi vault get $DB_PASSWORD_ID)
      export QUANTUMVAULT_KEY_ID=$(qapi vault get $STATE_KEY_ID)
      set -x

GitLab CI echoes commands by default when set -x is active (the default in many runners). The set +x / set -x wrapper prevents the export lines from printing secret values in the job log.

In all three cases, only one secret lives in the CI/CD platform: QUANTUMAPI_KEY. Everything else is a non-secret ID that points to a QuantumVault entry. The actual secret values exist only in memory during the pipeline run.

3. Post-Quantum SSH Keys

Traditional SSH uses RSA or ECDSA keys. Both are vulnerable to quantum attack. QuantumAPI can issue ML-DSA SSH certificates — short-lived keys that expire and rotate automatically.

The flow:

  1. Engineer requests an SSH certificate from QuantumAPI (via Backstage or CLI)
  2. QuantumAPI issues a certificate valid for 8 hours, signed with the CA’s ML-DSA key
  3. The target host trusts the QuantumAPI CA (configured once)
  4. The certificate expires automatically — no key to revoke, no key to forget on a laptop

Request a certificate:

qapi ssh issue \
  --principal victor.zaragoza \
  --validity 8h \
  --output ~/.ssh/id_mldsa

This creates two files: ~/.ssh/id_mldsa (private key) and ~/.ssh/id_mldsa-cert.pub (certificate). The certificate includes the principal name, validity period, and the CA signature.

Configure the target host:

# /etc/ssh/sshd_config (on the target host)
TrustedUserCAKeys /etc/ssh/quantumapi_ca.pub

The CA public key is fetched once from QuantumAPI:

qapi ssh ca-key > /etc/ssh/quantumapi_ca.pub

Now any certificate signed by QuantumAPI’s CA is accepted. No more distributing individual public keys to every host. No more authorized_keys files with 47 entries.

From Backstage:

We add an SSH certificate widget to the entity page. Engineers click “Request SSH Access”, select the target host (from the catalog), and get a short-lived certificate:

// plugins/infra-ssh/src/components/SshAccessCard.tsx
import React, { useState } from 'react';
import { InfoCard } from '@backstage/core-components';
import { Button, Typography, CircularProgress } from '@material-ui/core';
import { useEntity } from '@backstage/plugin-catalog-react';
import { useApi, fetchApiRef, discoveryApiRef } from '@backstage/core-plugin-api';

export const SshAccessCard = () => {
  const { entity } = useEntity();
  const fetchApi = useApi(fetchApiRef);
  const discoveryApi = useApi(discoveryApiRef);
  const [loading, setLoading] = useState(false);
  const [cert, setCert] = useState<string | null>(null);

  const handleRequest = async () => {
    setLoading(true);
    try {
      const proxyUrl = await discoveryApi.getBaseUrl('proxy');
      const res = await fetchApi.fetch(`${proxyUrl}/ai-service/api/ssh/issue`, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          principal: 'victor.zaragoza',
          targetHost: entity.metadata.name,
          validityHours: 8,
        }),
      });
      const data = await res.json();
      setCert(data.certificate);
    } catch {
      setCert('Failed to issue certificate.');
    } finally {
      setLoading(false);
    }
  };

  return (
    <InfoCard title="SSH Access" subheader="Post-quantum ML-DSA certificate">
      <Button
        variant="contained"
        color="primary"
        onClick={handleRequest}
        disabled={loading}
      >
        {loading ? <CircularProgress size={16} /> : 'Request SSH Certificate (8h)'}
      </Button>
      {cert && (
        <Typography
          variant="body2"
          style={{ marginTop: 16, fontFamily: 'monospace', whiteSpace: 'pre-wrap' }}
        >
          {cert}
        </Typography>
      )}
    </InfoCard>
  );
};

The AI service endpoint (added to the same Program.cs from the IDP series):

app.MapPost("/api/ssh/issue", async (SshRequest request, IConfiguration config) =>
{
    var apiKey = config["QuantumApi:ApiKey"];
    if (string.IsNullOrEmpty(apiKey))
        return Results.Json(new { error = "QuantumAPI not configured." }, statusCode: 503);

    var client = new QuantumApiClient(new QuantumApiOptions { ApiKey = apiKey });

    var result = await client.Ssh.IssueCertificateAsync(new IssueSshCertificateRequest
    {
        Principal = request.Principal,
        ValidityHours = request.ValidityHours,
        Extensions = new[] { "permit-pty", "permit-port-forwarding" }
    });

    return Results.Ok(new
    {
        certificate = result.Certificate,
        expiresAt = result.ExpiresAt,
        algorithm = "ML-DSA-65"
    });
});

record SshRequest(string Principal, string TargetHost, int ValidityHours);

In production, register QuantumApiClient via DI (builder.Services.AddQuantumApiClient(...) as shown in the Quantum-Safe Cloud series) instead of instantiating per request. For the article we keep it inline so the endpoint is self-contained.

4. AI-Powered Secret Sprawl Detection

The most dangerous secrets are the ones you forgot about. A connection string in an old backend.tf. An API key in a terraform.tfvars that was committed by mistake. An expired service principal that nobody rotated.

We add a secret scanner to the AI service. It runs on a schedule (like the catalog enricher) and checks every module in the catalog for secret-related issues:

app.MapPost("/api/scan-secrets", async (SecretScanRequest request, IConfiguration config) =>
{
    if (request.Files is not { Count: > 0 })
        return Results.BadRequest(new { error = "At least one file is required." });

    var endpoint = config["AI:Endpoint"];
    var apiKey = config["AI:Key"];
    var model = config["AI:ChatModel"] ?? "mistral-small-3.2-24b-instruct-2506";
    var provider = config["AI:Provider"] ?? "openai";

    ChatClient chatClient = provider.ToLowerInvariant() switch
    {
        "azure" => new AzureOpenAIClient(
            new Uri(endpoint!), new ApiKeyCredential(apiKey!))
            .GetChatClient(model),
        _ => new OpenAIClient(
            new ApiKeyCredential(apiKey!),
            new OpenAIClientOptions { Endpoint = new Uri(endpoint!) })
            .GetChatClient(model),
    };

    var systemPrompt = """
        You are a security scanner for Terraform infrastructure code.
        Analyze the provided files and find:

        1. Hardcoded secrets (passwords, API keys, tokens, connection strings)
        2. Unencrypted state backends (backends without encryption configuration)
        3. RSA or ECDSA key generation (should migrate to ML-DSA/ML-KEM)
        4. Expired or stale credentials (references to old key vaults, commented-out secrets)
        5. Secrets passed as default values in variables (default = "password123")

        Return a JSON array of findings. Each finding has:
        - "file": the file path
        - "line": approximate line number
        - "severity": "critical" | "high" | "medium" | "low"
        - "type": "hardcoded_secret" | "unencrypted_state" | "weak_algorithm" | "stale_credential" | "insecure_default"
        - "description": what was found
        - "fix": how to fix it

        If no issues are found, return an empty array.
        Do NOT flag environment variable references (var.xxx, $TF_VAR_xxx) as secrets.
        Return ONLY valid JSON, no markdown.
        """;

    var filesSummary = string.Join("\n\n", request.Files.Select(f =>
    {
        var content = f.Content.Length > 3000
            ? f.Content[..3000] + "\n[...truncated]"
            : f.Content;
        return $"### {f.Path}\n```\n{content}\n```";
    }));

    try
    {
        var completion = await chatClient.CompleteChatAsync(
        [
            new SystemChatMessage(systemPrompt),
            new UserChatMessage($"Scan these infrastructure files:\n\n{filesSummary}"),
        ]);

        var raw = completion.Value.Content[0].Text.Trim();
        var json = raw.StartsWith("```") ? raw.Split('\n', 2)[1].TrimEnd('`').Trim() : raw;

        var findings = JsonSerializer.Deserialize<List<SecretFinding>>(json, SerializerOptions.Default);
        return Results.Ok(new { findings = findings ?? new List<SecretFinding>() });
    }
    catch (Exception ex)
    {
        return Results.Json(new { error = $"AI scan error: {ex.Message}" }, statusCode: 502);
    }
});

record SecretScanRequest(List<SourceFile> Files);
record SecretFinding(
    [property: JsonPropertyName("file")] string File,
    [property: JsonPropertyName("line")] int Line,
    [property: JsonPropertyName("severity")] string Severity,
    [property: JsonPropertyName("type")] string Type,
    [property: JsonPropertyName("description")] string Description,
    [property: JsonPropertyName("fix")] string Fix);

The Backstage plugin runs this on every module in the catalog, on a schedule:

// plugins/secret-scanner/src/module.ts
import {
  coreServices,
  createBackendModule,
} from '@backstage/backend-plugin-api';
import { catalogServiceRef } from '@backstage/plugin-catalog-node';
import { Octokit } from '@octokit/rest';

export const secretScannerModule = createBackendModule({
  pluginId: 'catalog',
  moduleId: 'secret-scanner',
  register(env) {
    env.registerInit({
      deps: {
        logger: coreServices.logger,
        config: coreServices.rootConfig,
        scheduler: coreServices.scheduler,
        catalog: catalogServiceRef,
        auth: coreServices.auth,
      },
      async init({ logger, config, scheduler, catalog, auth }) {
        const aiServiceUrl = config.getString('forge.aiServiceUrl');
        const githubToken = config.getString('catalogEnricher.githubToken');
        const octokit = new Octokit({ auth: githubToken });

        await scheduler.scheduleTask({
          id: 'secret-scanner-run',
          frequency: { hours: 12 },
          timeout: { minutes: 30 },
          initialDelay: { minutes: 2 },
          fn: async () => {
            logger.info('Starting secret scan');
            const credentials = await auth.getOwnServiceCredentials();
            const { items: entities } = await catalog.getEntities(
              { filter: { 'spec.type': 'terraform-module' } },
              { credentials },
            );

            for (const entity of entities) {
              const slug = entity.metadata.annotations?.[
                'github.com/project-slug'
              ];
              if (!slug) continue;

              const [owner, repo] = slug.split('/');

              try {
                const { data: tree } = await octokit.git.getTree({
                  owner, repo, tree_sha: 'main', recursive: 'true',
                });

                const tfFiles = tree.tree.filter(
                  f => f.type === 'blob' && f.path &&
                    (f.path.endsWith('.tf') || f.path.endsWith('.tfvars')),
                );

                const files: Array<{ path: string; content: string }> = [];
                for (const file of tfFiles) {
                  if (!file.path) continue;
                  try {
                    const { data: content } = await octokit.repos.getContent({
                      owner, repo, path: file.path,
                      mediaType: { format: 'raw' },
                    });
                    files.push({
                      path: file.path,
                      content: content as unknown as string,
                    });
                  } catch { /* skip unreadable files */ }
                }

                if (files.length === 0) continue;

                const res = await fetch(`${aiServiceUrl}/api/scan-secrets`, {
                  method: 'POST',
                  headers: { 'Content-Type': 'application/json' },
                  body: JSON.stringify({ files }),
                });

                if (!res.ok) continue;

                const { findings } = await res.json() as {
                  findings: Array<{
                    file: string; severity: string;
                    type: string; description: string;
                  }>;
                };

                if (findings.length > 0) {
                  logger.warn(
                    `${entity.metadata.name}: ${findings.length} secret issues found`,
                  );
                  for (const f of findings) {
                    logger.warn(
                      `  [${f.severity}] ${f.file}: ${f.description}`,
                    );
                  }
                } else {
                  logger.info(`${entity.metadata.name}: clean`);
                }
              } catch (err) {
                logger.error(
                  `Failed to scan ${entity.metadata.name}: ${err}`,
                );
              }
            }

            logger.info('Secret scan complete');
          },
        });
      },
    });
  },
});

What a Scan Report Looks Like

The scanner runs on the tf-azurerm-vnet module and finds:

[
  {
    "file": "backend.tf",
    "line": 5,
    "severity": "high",
    "type": "unencrypted_state",
    "description": "State backend uses azurerm storage without encryption block. State contains all resource attributes in plaintext.",
    "fix": "Add terraform.encryption block with QuantumVault key provider. See article 5 of the Infrastructure Hub series."
  },
  {
    "file": "variables.tf",
    "line": 28,
    "severity": "medium",
    "type": "weak_algorithm",
    "description": "Variable 'ssh_public_key' expects an RSA key (description says 'RSA public key'). RSA is vulnerable to quantum attack.",
    "fix": "Accept ML-DSA keys instead. Use QuantumAPI SSH certificates for host access."
  }
]

The platform team sees these in the Backstage logs (and in a future article, on a dashboard). Critical findings trigger alerts. The engineer gets a clear description and a specific fix.

Connecting Everything: The Module Security Annotation

Each module in the catalog gets a security status annotation that the scanner updates:

# Updated by the secret scanner
metadata:
  annotations:
    forge.io/secret-scan-status: "clean"           # or "issues-found"
    forge.io/secret-scan-last: "2026-04-03T10:00Z"
    forge.io/state-encryption: "quantumvault"       # or "azure-managed" or "none"

The catalog enricher and the secret scanner work together: the enricher keeps the description and tags accurate, the scanner keeps the security status accurate. Both run on a schedule, both update the catalog, both are visible from Backstage.

The Full Secret Architecture for Infrastructure

BEFORE (typical enterprise)
├── Pipeline variable group → RSA-encrypted in Azure Key Vault
│   ├── ARM_CLIENT_SECRET = "actual-secret-value"
│   ├── DB_PASSWORD = "actual-password"
│   └── SSH_PRIVATE_KEY = "-----BEGIN RSA PRIVATE KEY-----..."
├── Terraform state → plaintext in Azure Storage (encrypted at rest with RSA)
│   └── Contains ALL resource secrets as attributes
└── ~/.ssh/ → RSA keys, never rotated, on 5 laptops

AFTER (with QuantumVault)
├── Pipeline → only QUANTUMAPI_KEY in variable group
│   ├── ARM_SECRET_ID = "just-an-id"  (fetched at runtime from QuantumVault)
│   ├── DB_PASSWORD_ID = "just-an-id" (fetched at runtime from QuantumVault)
│   └── SSH → ML-DSA certificates, 8h validity, auto-expire
├── Terraform state → encrypted with ML-KEM-768 + AES-256-GCM
│   └── Even with storage access, ciphertext is useless without QuantumVault key
├── QuantumVault (source of truth)
│   ├── All secrets ML-KEM encrypted at rest
│   ├── QRNG key generation
│   ├── Audit log of every access
│   └── Per-module, per-client secret scoping
└── Backstage → secret scan status visible per module

Checklist

  • Terraform state encryption configured with QuantumVault key provider
  • enforced = true set — Terraform refuses to write unencrypted state
  • Pipeline secrets fetched from QuantumVault at runtime (not stored in CI/CD)
  • Only QUANTUMAPI_KEY remains in pipeline variable groups
  • SSH keys replaced with ML-DSA certificates (8h validity)
  • Target hosts configured to trust QuantumAPI CA
  • Secret scanner running on schedule (every 12h)
  • Scanner covers all terraform-module entities in the catalog
  • Scan results visible in Backstage logs
  • forge.io/secret-scan-status annotation updated per module
  • Existing state migrated (first terraform apply after adding encryption)

Challenge

Before the next article:

  1. Add the encryption block to one of your modules’ versions.tf
  2. Run terraform plan — it should work with no changes (the state gets encrypted on next apply)
  3. Run the secret scanner on your modules — how many issues does it find?
  4. Replace one RSA SSH key with a QuantumAPI ML-DSA certificate

In the next article, we build the CAB Automation — Change Advisory Board workflows where AI prepares the evidence, assesses the risk, and the CAB reviews a complete package instead of a rushed ticket. With PQC signatures on every approval.

The full code is on GitHub.

If this series helps you, consider buying me a coffee.

This is article 5 of the Infrastructure Hub series. Previous: Pipelines from Backstage. Next: CAB Automation — AI-prepared change requests with post-quantum signatures.

Comments

Loading comments...