Verifying Deployed Assets Against a Hash Manifest

Permalink to "Verifying Deployed Assets Against a Hash Manifest"

A build-time manifest only proves what the bundler wrote to disk, so the gates described in CI/CD Integrity Gates are not finished until something fetches each asset back off the live origin and the CDN edge and confirms the digest still matches. This page builds that post-deploy smoke test, explains how to read a mismatch, and wires the result into a gate that can roll the release back automatically.

Quick Reference

Permalink to "Quick Reference"
Control Value Effect
Manifest entry "/app.abc123.js": "sha384-…" Path to full SRI digest string, produced at build time
Hash input Decoded response body Matches what a browser hashes after content decoding
Cache-Control: no-cache Request header Asks intermediaries to revalidate; CDNs may ignore it
CF-Cache-Status / X-Cache Response header HIT or MISS at the edge node that answered
CF-Ray / X-Served-By Response header Contains the POP code that served the response
Cache-Status Response header (RFC 9211) Standardised cache reporting; support is still patchy
curl --compressed Flag Negotiates and transparently decodes gzip/Brotli
curl --resolve host:443:IP Flag Pins the connection to a specific edge address
Exit code 0 all match, 1 any mismatch Drives the deploy gate and the rollback job

Default order of operations: verify origin first, then each edge region, then gate on the combined result.

The mental model

Permalink to "The mental model"

The manifest is a claim made by the build about bytes that no longer exist anywhere the build can see. Between the bundler writing app.abc123.js and a browser executing it, the file passes through an upload step, an object store, an origin web server, one or more CDN caches, and possibly an edge worker that is allowed to rewrite responses. Every one of those hops is a place where the bytes can change without anyone deliberately changing them. The verification pass closes that gap by treating the deployed site as an untrusted third party: it fetches the asset the way a browser would, computes the digest the way a browser would, and compares the result to the claim the build recorded.

That framing matters because it decides what you fetch. You are not verifying the file in your object store — you already trust that, because you put it there. You are verifying the response that a real client receives, which means following the same hostname, the same protocol, and the same caching layers a user would. A check that reads the artifact out of the deploy bucket proves nothing about the edge transform that mangles it thirty milliseconds later. The interesting failures all live downstream of the thing that is easy to test.

Post-deploy verification data flow The build manifest feeds a verifier that fetches and decodes each asset from the live origin and from two CDN edge nodes, and the three resulting digests are compared against the manifest entry. build manifest sha384 per path fetch + decode hash the body live origin cache bypassed edge POP: IAD cached copy edge POP: FRA cached copy compare digests

The manifest itself has to travel as a build artifact, not as a file you download from the deployed site. A manifest fetched from the same host you are testing describes whatever is currently deployed, so it agrees with the assets by construction and the check becomes a no-op. Keep it in the CI artifact store alongside the bundle, exactly as Generating an SRI Manifest in GitHub Actions produces it, and hand it to the verification job through job outputs or an artifact download.

Canonical example: the post-deploy verification script

Permalink to "Canonical example: the post-deploy verification script"

This script reads a manifest of the form { "/assets/app.abc123.js": "sha384-…" }, fetches every entry from a base URL, hashes the decoded body, and prints a per-asset pass/fail table before exiting with a gate-friendly status code. It runs on Node 20 or newer with no dependencies.

// scripts/verify-deployed.mjs
// usage: node scripts/verify-deployed.mjs dist/sri-manifest.json
import { createHash } from 'node:crypto';
import { readFile } from 'node:fs/promises';

const manifestPath = process.argv[2] ?? 'dist/sri-manifest.json';
const base = process.env.VERIFY_BASE_URL ?? 'https://www.example.com';
const manifest = JSON.parse(await readFile(manifestPath, 'utf8'));

function sriDigest(bytes, algorithm) {
  return `${algorithm}-${createHash(algorithm).update(bytes).digest('base64')}`;
}

async function probe(url) {
  const res = await fetch(url, {
    redirect: 'error',
    headers: {
      'Cache-Control': 'no-cache',
      'Pragma': 'no-cache',
      'User-Agent': 'deploy-verifier/1.0'
    }
  });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  // arrayBuffer() returns the body AFTER content decoding — the same bytes
  // a browser hashes for its integrity check.
  return {
    body: Buffer.from(await res.arrayBuffer()),
    pop: res.headers.get('cf-ray') ?? res.headers.get('x-served-by') ?? '-',
    cache: res.headers.get('cf-cache-status') ?? res.headers.get('x-cache') ?? '-'
  };
}

const results = [];
for (const [path, expected] of Object.entries(manifest)) {
  const url = new URL(path, base).href;
  try {
    const { body, pop, cache } = await probe(url);
    const algorithm = expected.split('-')[0];
    const actual = sriDigest(body, algorithm);
    results.push({ url, ok: actual === expected, expected, actual, bytes: body.byteLength, pop, cache });
  } catch (err) {
    results.push({ url, ok: false, expected, actual: `ERROR ${err.message}`, bytes: 0, pop: '-', cache: '-' });
  }
}

for (const r of results) {
  console.log(`${r.ok ? 'PASS' : 'FAIL'}  ${r.url}  ${r.bytes}B  cache=${r.cache}  pop=${r.pop}`);
  if (!r.ok) {
    console.log(`        expected  ${r.expected}`);
    console.log(`        actual    ${r.actual}`);
  }
}

const failed = results.filter((r) => !r.ok).length;
console.log(`\n${results.length - failed}/${results.length} assets verified against ${manifestPath}`);
process.exit(failed > 0 ? 1 : 0);

Three details carry the weight. redirect: 'error' refuses to follow a redirect, because an asset that has quietly moved is a finding rather than something to chase silently. The algorithm is read from the manifest entry instead of being hard-coded, so a manifest that mixes sha256 and sha384 still verifies correctly, and the comparison is on the whole prefixed string rather than the bare base64 — a digest that is right but labelled with the wrong algorithm is still wrong. Finally the POP and cache-status headers are recorded on every row, pass or fail, because those two fields are what turn a red line into a diagnosis.

Verification request sequence The CI verifier sends a cache-defeating GET to the CDN edge, the edge revalidates with the origin, the origin returns the body, the edge returns an encoded response with cache headers, and the verifier decompresses before hashing. verifier (CI) CDN edge POP origin server GET /app.abc123.js Cache-Control: no-cache miss or revalidate 200 + stored bytes 200, content-encoding: br cf-cache-status, cf-ray decompress body first, then sha384, base64, compare record cache status and POP code on every row, pass or fail

The decoding step is the one people get wrong. Integrity is checked against the response body after content decoding, so a Content-Encoding: br response and a Content-Encoding: gzip response of the same file produce the same digest in a browser even though the wire bytes differ completely. Response.arrayBuffer() in Node hands you the already-decompressed body, so the script above is correct by default — but if you ever reach for a raw socket, a proxy, or a tool that hands back the encoded stream, you will get a digest that changes every time the edge recompresses at a different quality setting. That is a false alarm, and false alarms are how integrity gates get disabled.

The digest the script compares against is the same string the page carries in its markup, which is what makes the check meaningful in the first place:

<script src="https://cdn.example.com/app.abc123.js"
        integrity="sha384-oqVuAfXRKap7fdgcCY5uykM6+R9GqQ8K/uxy9rx7HNQlGYl1kPzQho1wx4JwY8wC"
        crossorigin="anonymous"></script>

If the manifest and the tag disagree, the browser rejects the script and you have an outage; if the manifest and the deployed bytes disagree, you also have an outage. The verification job exists to find the second case before real traffic does. The exact base64 alphabet and padding rules that make two visually similar digests compare unequal are covered in Base64 Encoding Rules for SRI Hashes.

Variants

Permalink to "Variants"

Checking several edge nodes

Permalink to "Checking several edge nodes"

Anycast routing chooses a POP from the network location of the client, so you cannot select one by IP from a single runner — the address you dial is the same address everywhere. The practical options are to run the same job from runners in several regions, or, where the provider publishes per-POP hostnames, to pin the connection explicitly:

# Same asset, forced through a specific edge address
curl -sS --compressed --resolve cdn.example.com:443:203.0.113.10 \
     -H 'Cache-Control: no-cache' \
     -D headers.txt -o body.bin \
     https://cdn.example.com/app.abc123.js

echo "sha384-$(openssl dgst -sha384 -binary body.bin | openssl base64 -A)"
grep -iE '^(cf-ray|cf-cache-status|x-served-by|x-cache):' headers.txt

--compressed makes curl negotiate an encoding and then decode transparently, so body.bin holds the decoded payload. In a workflow, express the regions as a job matrix and let each runner emit its own report; a single region that disagrees is a very different signal from every region disagreeing.

Shell-only verification loop

Permalink to "Shell-only verification loop"

Where Node is not available in the deploy image, jq and openssl are enough:

#!/usr/bin/env bash
set -uo pipefail
BASE="${VERIFY_BASE_URL:?}"
fail=0

while IFS=$'\t' read -r path expected; do
  actual="sha384-$(curl -sSf --compressed -H 'Cache-Control: no-cache' \
      "${BASE}${path}" | openssl dgst -sha384 -binary | openssl base64 -A)"
  if [ "$actual" = "$expected" ]; then
    printf 'PASS  %s\n' "$path"
  else
    printf 'FAIL  %s\n        expected %s\n        actual   %s\n' "$path" "$expected" "$actual"
    fail=1
  fi
done < <(jq -r 'to_entries[] | "\(.key)\t\(.value)"' dist/sri-manifest.json)

exit "$fail"

This assumes an all-sha384 manifest; the deliberate simplification is why the Node version reads the algorithm per entry. Related digest-from-the-command-line recipes live in Generating SRI Hashes with OpenSSL and shasum.

Gating the deploy with an automatic rollback

Permalink to "Gating the deploy with an automatic rollback"

Split deploy, verify, and rollback into three jobs so the rollback can key off the verify result. Give propagation a moment and retry before declaring failure, because an object store that is still replicating is not the same thing as a corrupted asset.

# .github/workflows/deploy.yml
jobs:
  deploy:
    runs-on: ubuntu-latest
    outputs:
      previous: ${{ steps.record.outputs.previous }}
    steps:
      - uses: actions/checkout@v4
      - id: record
        run: echo "previous=$(./scripts/current-release.sh)" >> "$GITHUB_OUTPUT"
      - run: ./scripts/deploy.sh
      - uses: actions/upload-artifact@v4
        with:
          name: sri-manifest
          path: dist/sri-manifest.json

  verify:
    needs: deploy
    runs-on: ubuntu-latest
    strategy:
      matrix:
        region: [us-east, eu-west]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/download-artifact@v4
        with:
          name: sri-manifest
          path: dist
      - name: Verify deployed assets
        env:
          VERIFY_BASE_URL: https://www.example.com
        run: |
          for attempt in 1 2 3; do
            node scripts/verify-deployed.mjs dist/sri-manifest.json && exit 0
            echo "attempt ${attempt} failed, waiting for propagation"
            sleep 20
          done
          exit 1

  rollback:
    needs: [deploy, verify]
    if: failure()
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/rollback.sh "${{ needs.deploy.outputs.previous }}"

if: failure() on a job that lists both deploy and verify in needs runs only when one of them failed, which is exactly the trigger you want. Keep rollback.sh to whatever restore primitive your platform already exposes — repointing an alias, re-uploading the previous artifact set, promoting the prior release — and have it re-run the verification against the restored manifest so a failed rollback is loud rather than silent.

Triaging a mismatch

Permalink to "Triaging a mismatch"

A red line tells you the bytes differ; it does not tell you why. Four causes account for nearly everything, and they are cheap to separate if you ask the questions in the right order. Start with the origin, because if the origin is wrong nothing downstream matters, then ask whether the edges agree with each other, because uniform disagreement and isolated disagreement have different causes.

Mismatch triage decision tree A digest mismatch branches on whether the origin copy matches the manifest, and then on whether every edge node agrees, separating a bad deploy from a stale cache and from a uniform edge transform. digest mismatch origin digest matches manifest? no yes bad deploy or compromised build all POPs agree with each other? no yes stale cache at a POP purge and re-check edge transform minify or rewrite

Bad deploy is the common case and the boring one: a partial upload, a race between two pipelines, or a manifest generated from a different build than the one that shipped. It shows up as the origin disagreeing with the manifest, usually for a subset of files, and it is fixed by redeploying from the same artifact the manifest describes.

Stale edge cache shows up as one or two POPs disagreeing while the origin and every other POP agree. The disagreeing node is still serving the previous build because its stored object had not expired and the purge did not land there. Purge that path, wait for the purge to propagate, and re-run. Content-hashed filenames make this nearly impossible, which is a good argument for them.

Edge transform shows up as every POP agreeing with each other and disagreeing with the origin by the same digest. Something in the edge configuration is rewriting responses — automatic minification, HTML rewriting, an image optimiser, or a worker that appends a header script. This is a configuration problem rather than an incident, and it is covered in depth in SRI with Cloudflare and Fastly Edge Transforms.

Compromise is the residual: the origin disagrees, the artifact in CI matches its own recorded digest, and nobody deployed. It is rare, which is precisely why it must be the explicit fourth branch rather than an unstated assumption — a triage rule that ends at “probably a cache thing” will classify a real intrusion as a cache thing. Escalate when a mismatch survives a confirmed purge, appears at unrelated POPs simultaneously, or affects a file that no recent build touched.

Gotchas and Edge Cases

Permalink to "Gotchas and Edge Cases"
  • A manifest fetched from the site under test proves nothing. If the verification job downloads sri-manifest.json from the deployed origin, it is comparing the deploy against itself and will pass even when every asset was replaced. Pass the manifest through the CI artifact store so it comes from the build that produced the bytes, and treat a missing artifact as a hard failure rather than a skip.

  • Cache-Control: no-cache is a request, not a guarantee. Providers vary in whether they honour a client no-cache, and several ignore it on purpose so that clients cannot force origin traffic. Send it anyway, then read CF-Cache-Status, X-Cache, or the RFC 9211 Cache-Status header to find out what actually happened. A run where every response says HIT has not tested the origin at all.

  • Hashing the wire bytes produces phantom failures. If the tool you use returns the compressed stream rather than the decoded body, the digest will change whenever the edge switches Brotli quality or falls back to gzip. Browsers hash after decoding, so the compressed bytes are not the thing under test. Prefer curl --compressed or Response.arrayBuffer() and never a raw socket read.

  • Omitting crossorigin="anonymous" breaks the tag even when the digest is right. For a cross-origin script or stylesheet, the browser needs a CORS-enabled fetch to expose the body to the integrity check; without the attribute the request is made in no-cors mode and the resource is blocked regardless of whether the hash matches. A green verification run says nothing about this, because the script fetches with CORS semantics by default — check the rendered markup separately.

  • HTML documents are a bad fit for this check. Server-rendered pages carry per-request nonces, cache-busting query strings, and A/B markers, so their digests legitimately differ on every request. Verify immutable, content-hashed assets — scripts, stylesheets, fonts, wasm modules — and leave documents to a different test.

Verification Steps

Permalink to "Verification Steps"

1. Run the check against a known-good deploy

Permalink to "1. Run the check against a known-good deploy"
VERIFY_BASE_URL=https://www.example.com node scripts/verify-deployed.mjs dist/sri-manifest.json

Expected output on a clean rollout:

PASS  https://www.example.com/assets/app.abc123.js  184213B  cache=HIT  pop=IAD
PASS  https://www.example.com/assets/app.abc123.css  22140B  cache=HIT  pop=IAD

2/2 assets verified against dist/sri-manifest.json

2. Prove the check can actually fail

Permalink to "2. Prove the check can actually fail"

Corrupt one manifest entry and confirm the script reports it and exits non-zero:

jq '.["/assets/app.abc123.js"] = "sha384-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"' \
  dist/sri-manifest.json > /tmp/bad.json
node scripts/verify-deployed.mjs /tmp/bad.json; echo "exit: $?"

The corrupted row prints as FAIL with both digests and the run ends with exit: 1. A gate that has never been observed failing is not a gate.

3. Confirm you are hashing the decoded body

Permalink to "3. Confirm you are hashing the decoded body"

Fetch the same URL twice, once decoded and once as raw encoded bytes, and compare:

curl -sS --compressed https://www.example.com/assets/app.abc123.js \
  | openssl dgst -sha384 -binary | openssl base64 -A; echo
curl -sS -H 'Accept-Encoding: gzip' https://www.example.com/assets/app.abc123.js \
  | openssl dgst -sha384 -binary | openssl base64 -A; echo

The first digest must equal the manifest value. The second will differ whenever the server compresses — that difference is the mistake this step exists to make visible.

4. Confirm the gate blocks and the rollback fires

Permalink to "4. Confirm the gate blocks and the rollback fires"

Push a branch that deploys a deliberately mismatched manifest and watch the workflow: the verify job must exhaust its retries and fail, and the rollback job must start on if: failure() and restore the previous release. Confirm the restored deploy passes verification, then check that the failure surfaced wherever your team actually looks. The complementary build-time gate that catches drift before anything ships is described in Failing CI on SRI Hash Drift.

Frequently Asked Questions

Permalink to "Frequently Asked Questions"
Should I hash the compressed bytes or the decompressed body?

Always the decompressed body. Browsers apply the integrity check after content decoding, so a gzip or Brotli layer never changes the digest a browser computes. Hashing the wire bytes would make your check fail every time the edge recompresses at a different quality level, which is a routine and harmless event.

Does a Cache-Control: no-cache request header really bypass the CDN?

Not reliably. Some CDNs revalidate on a client no-cache, and many deliberately ignore it so clients cannot stampede the origin. Send the header, but read the response cache-status header to learn what actually happened, and use a separate origin-direct request when you need to isolate the origin copy.

How do I check more than one CDN POP?

Anycast routing picks the POP from the network location of the client, so you cannot select one by IP address from a single runner. Run the same verification job from runners in several regions, or point curl --resolve at a provider-published per-POP hostname where one exists. Record the POP identifier from the response headers with every result.

What does a digest mismatch on only one POP mean?

Almost always a stale cached object at that node rather than a compromise. One POP still holds the previous build because the object had not expired and the purge did not reach it. Purge that path, re-run the check, and only escalate if the mismatch survives a confirmed purge or appears at several unrelated POPs at once.

Can this check replace the integrity attribute in the page?

No. The smoke test samples the asset once, from your infrastructure, at deploy time. The integrity attribute is enforced by every browser on every load, including loads that happen days later against a poisoned cache. The manifest check catches a bad rollout early; the attribute is what actually protects users.

Permalink to "Related"

Related Articles

Failing CI on SRI Hash Drift
Generating an SRI Manifest in GitHub Actions
CI/CD Integrity Gates Asset Hashing & Dynamic Script…