Parag Mali - tag: web-security

Two Identical Requests: How Web Bot Auth and HTTP Message Signatures Let Servers Trust the Right Bots

noreply@paragmali.com (Parag Mali) — Mon, 20 Jul 2026 00:00:00 GMT

For thirty years the web authenticated humans with ever-stronger cryptography while governing bots with etiquette (`robots.txt`), self-declared names (`User-Agent`), and network guesswork (IP allowlists) -- none of which can tell a legitimate AI agent apart from a scraper. RFC 9421 (HTTP Message Signatures, February 2024) [@rfc9421] and the Cloudflare and Google **Web Bot Auth** drafts [@draft-webbotauth-arch] fix this: a bot signs selected *components* of each request with an Ed25519 key and publishes its public keys in a discoverable JWKS directory, so any origin can verify a stranger bot on first contact. The genuine novelty is not the cryptography -- machines have signed requests since AWS SigV4 and ActivityPub [@cf-blog-webbotauth] -- but a **standardized, directory-backed, open-web identity anyone can look up**. And it proves exactly one thing: *who signed*, never *whether they are allowed*.

1. Two Identical Requests

Two HTTP requests hit your server in the same second. Both carry the identical Chrome User-Agent. Both arrive from the same shared cloud IP range. Both ask for the same page. One is OpenAI's Operator, fetching that page on behalf of a paying customer. The other is a scraper, harvesting your content to resell. You have milliseconds to decide which one to admit -- and with everything the web handed you across its first thirty years, you literally cannot tell them apart.

That is not a failure of effort. It is a failure of design. For decades the web learned to authenticate humans with steadily stronger cryptography: passwords gave way to asymmetric keys, and asymmetric keys gave way to passkeys bound to a specific origin. Machines got none of that. The web still governs automated clients the way a library governs its patrons -- with a posted set of rules and the hope that visitors behave. This article is the story of the layer that finally closes the gap -- and the one thing it still refuses to prove.

Start with why the problem is genuinely hard. Cloudflare, whose network sits in front of a large fraction of the web, framed it bluntly in a 2025 proposal titled "Forget IPs": the three signals an origin has today are each broken in a different way [@cf-blog-webbotauth].

The User-Agent string is spoofable -- just text the client types, and agents deliberately send a real Chrome string. The source IP is brittle -- one address is routinely shared across unrelated users, privacy proxies, and whole companies renting the same cloud. A pre-shared secret, the classic API-key answer, is pairwise -- it forces every bot to arrange a separate credential with every site it might visit. None of the three answers the only question that matters: who is actually sending this request?

The set of Cloudflare- and Google-authored IETF Internet-Drafts that profile RFC 9421 (HTTP Message Signatures) for the specific job of bot identity, and add the piece the RFC leaves out -- a discoverable public key directory. Together they let an origin cryptographically verify *which* automated client sent a request, with no prior arrangement.

Before going further, fix the cast of characters. An AI agent here means an orchestrated browser -- often a real, automated instance of Chrome -- acting on a person's behalf: booking a flight, comparing prices, reading a page and summarizing it. From the server's side it looks exactly like a human's browser, because it frequently is one. That is precisely why the old signals collapse: the agent is not pretending to be Chrome, it is Chrome, driven by software instead of hands.

flowchart TD A["AI agent acting for a user
real Chrome User-Agent, shared cloud IP"] --> O{"Origin: which do I admit?"} B["Scraper reselling content
real Chrome User-Agent, shared cloud IP"] --> O O --> Q["No verifiable identity signal
the two requests are indistinguishable"]

There is a deeper idea hiding in that question, and the whole article turns on it. "Who is sending this?" is a question about identity. "Should I let them in?" is a question about authorization. They feel like one decision at the door, but they are two, and conflating them is the mistake that makes bot management so miserable. Everything that follows gives the web a way to answer the first question with cryptographic certainty -- while being scrupulously honest that it never answers the second.

Key idea: A signature can prove who sent a request without proving the sender is allowed to make it. Verifiable identity and authorization are different problems. Web Bot Auth solves the first and, by design, leaves the second to policy. Hold on to that distinction -- it is the hinge every later section turns on.

So the refined claim this article defends is narrower and more interesting than the headline. After decades of hardening how it authenticates people, the web is finally giving agents a cryptographic identity that is standardized and discoverable -- not the first-ever cryptography for machines, but the first kind any origin can simply look up. If you cannot trust what a bot says it is, and you cannot trust where it comes from, what is left to trust? The answer took the web thirty years to build. To see why it was so hard, we have to go back to 1994.

2. Identity for Humans, Politeness for Machines

In 1993 and 1994, the first web robots "swamped servers with rapid-fire requests" and wandered into corners of sites they should have left alone [@robotstxt-orig]. There was no way for a server to tell a robot where not to go. Martijn Koster, working with a mailing list of the era's robot authors, proposed a fix and the group reached consensus on it on 30 June 1994. His fix was not a lock. It was a polite note taped to the door.

That note is robots.txt: a single file at a well-known path listing the URL paths a crawler is requested not to fetch. A well-behaved crawler downloads it once, reads the section addressed to its own self-declared name, and voluntarily stays out of the listed areas.

The founding text is remarkably candid about what it is not. The protocol, it says, "is not an official standard backed by a standards body," and it "is not enforced by anybody, and there [is] no guarantee that all current and future robots will use it" [@robotstxt-orig]. It was standardized twenty-eight years later as RFC 9309, which records that the protocol was "originally defined by Martijn Koster in 1994" [@rfc9309].

Read those words carefully, because they contain the distinction the rest of this article depends on. robots.txt never authenticates anyone. The server advertises a policy and hopes; it never learns who is asking. RFC 9309 draws the line in one sentence: its rules "are not a form of access authorization" [@rfc9309]. This is etiquette, not identity, and certainly not enforcement. A hostile scraper reads Disallow: /, crawls the whole site anyway, and the origin is left with neither a barrier nor a name.

While machines got etiquette, people got cryptography -- and steadily more of it. The web moved from passwords (a shared secret the server stores and the user retypes), to asymmetric second factors, to [WebAuthn and passkeys](/blog/webauthn-and-passkeys-on-windows-from-ctap-to-the-credential/), where the browser holds a private key bound to a specific origin and proves possession without ever transmitting the secret. That is the exact shape of guarantee -- a private key that signs, a public key that verifies, tied to an identity -- that bots never received. This blog's field guide on passkeys covers the human side; Web Bot Auth is, in a real sense, the same idea arriving decades late for machines.

If a policy file cannot identify anyone, the obvious next step is to let the client say who it is. The web already had a place for that: the User-Agent header, present since HTTP/1.0 in 1996 [@rfc1945] and given its current normative definition in RFC 9110, section 10.1.5, in June 2022 [@rfc9110].

A crawler announces itself -- User-Agent: MyBotCrawler/1.1 -- and the origin can, in principle, allow the crawlers it trusts and block the rest. For the first time the server has an identity claim it can act on, per request, without relying on the client to police its own behavior.

There is one catch, and it is fatal: the claim is unauthenticated. A header is just a string, and anyone can send any string.

The specification itself concedes the point. If a user agent "masquerades as a different user agent," it notes, recipients simply have to assume the client meant to be treated as whatever it claimed [@rfc9110]. Cloudflare states the operational consequence without euphemism: user-agent strings "alone are easily spoofed and are therefore insufficient for reliable identification," and agents "often use the Chrome user agent for this very reason" [@cf-blog-webbotauth].The masquerade language lives in RFC 9110 section 10.1.5; the same sentence is reproduced, with that section citation, in Cloudflare's "Forget IPs" post [@cf-blog-webbotauth], which is where many practitioners first meet it.

So the second generation of bot identity answers a subtly wrong question. User-Agent tells you who the client says it is, never who it is. And that is exactly the hook from the previous section: a legitimate agent and a scraper both send a real Chrome string, and the origin cannot separate them. A name anyone can type is not an identity. So perhaps the answer is to stop trusting what the bot writes and start trusting something it cannot simply type -- the network it comes from.

3. Anchoring Identity to the Network

If a bot can lie about its name, tie its identity to something it cannot simply write down: the network it lives on. A random impostor can set any User-Agent, but it cannot easily forge control of a vendor's DNS or occupy a specific block of IP addresses. For roughly a decade, this was the strongest bot-verification signal the web had.

Google pioneered the technique for Googlebot, and the canonical version has two composable checks. The first is forward-confirmed reverse DNS.

Run a reverse (PTR) lookup on the source IP to get a hostname -- for Googlebot, something like `crawl-66-249-66-1.googlebot.com` -- then forward-resolve that hostname and confirm it maps back to the same IP. Passing both directions proves the request came from network infrastructure the claimed operator controls. It proves network control, not identity.

Forging that requires controlling both directions of DNS for the vendor's domain, which a casual spoofer cannot do [@google-verify]. The second check is a published IP allowlist: the vendor publishes its address ranges as CIDR blocks in a file, and the origin checks whether the source IP falls inside them [@cf-blog-webbotauth]. Google documents both paths today -- common crawlers reverse-resolve under googlebot.com and appear in a published common-crawlers.json, while special-case fetchers use a separate google.com mask [@google-verify].

This was a real improvement. Identity was now anchored to something the requester could not self-assert; the request had to originate from Google's footprint. "Verify Googlebot" became standard operating procedure, and for years it worked.The mechanism is a little older than any dated write-up, but Google's "How to verify Googlebot" post, dated 20 September 2006, is the first primary source that spells it out and calls reverse-plus-forward DNS "an official way to authenticate Googlebot" [@google-2006-blog]. That is the debut this article pins to.

And then it broke -- not at the edges, but at its foundation. Reverse DNS and IP lists prove network location, and location is not identity. The entire premise is that a source IP maps to a single, identifiable operator, and modern infrastructure dissolves that mapping.

Cloudflare puts the failure plainly: the same IP address "might be shared by multiple users or multiple services within the same company, or even by multiple companies when hosting infrastructure is shared (like Cloudflare Workers)" [@cf-blog-webbotauth]. Privacy proxies and VPNs interpose addresses that belong to neither the operator nor any stable identity. Serverless platforms hand out egress addresses that churn with the underlying infrastructure, which is why vendors resort to constantly-updated CIDR lists [@cf-blog-webbotauth].

Note: Every network-anchored check answers the question "what network did this packet come from?" That was a useful proxy for identity when an operator owned its addresses. The moment agents began running on shared clouds, privacy relays, and serverless platforms, the proxy stopped pointing at anything. The signal did not weaken -- it started measuring the wrong thing.

Picture the concrete case that matters now. An AI agent runs on a serverless platform and shares its outbound address with thousands of unrelated tenants. A reverse lookup on that address resolves to the cloud provider, not to the agent's operator. The origin can confirm "this came from a big cloud" and learns nothing about which automated client sent it.

Worse, the scheme is per-vendor and brittle in both directions: every operator must publish and maintain its own ranges, and every origin must track every operator's ever-shifting lists. Onboarding a new verified crawler historically meant hand-checking IP ranges and autonomous-system numbers -- exactly the manual toil the cryptographic approach was later built to retire [@cf-blog-verifiedbots].

The flaw, then, is not the mechanism. It is the premise. An IP address was never an identity; it was a stand-in for one, and stand-ins leak. To bind identity to the operator itself -- independent of whatever address it happens to borrow today -- you need something only that operator holds and can present on demand. You need a secret. Or, better still, a key.

4. Generation by Generation

Step back from the individual mechanisms and a pattern appears: each one failed in a way that named its successor. robots.txt had no identity, so we added a self-declared name. The name was spoofable, so we anchored to the network. The network proved location instead of identity, so we reached for a secret. Six generations, each a better answer to the same stubborn question, each breaking in a way that forced the next.

Generation	Mechanism	What it added	The failure that forced the next
1 (1994)	`robots.txt` honor system	A shared vocabulary for addressing crawlers	No authentication, no enforcement -- the server never learns who is asking [@robotstxt-orig]
2 (1996)	`User-Agent` header	A per-request identity claim to act on	Self-declared and trivially spoofed; a real Chrome string hides everything [@rfc9110]
3 (~2006)	Reverse DNS + IP allowlists	Identity anchored to network infrastructure	Proves location, not who; collapses on shared clouds, VPNs, serverless [@cf-blog-webbotauth]
4 (2012)	Bearer tokens / API keys	Portable, IP-independent possession	Possession is not proof, and secrets are pairwise -- N x M distribution [@rfc6750]
5 (2013-19)	Per-vendor request signing (SigV4, draft-cavage, Stripe, mTLS)	Real asymmetric cryptography plus integrity	No shared format, no interoperable key discovery -- every scheme an island [@cf-blog-webbotauth]
6 (2024-26)	RFC 9421 + Web Bot Auth + JWKS directory	A standard signature and a discoverable public key	Not superseded; its limit is fundamental -- it proves who, never whether-allowed [@draft-webbotauth-arch]

Generation 4 deserves a moment, because it is where cryptography first almost arrived. A Bearer token or API key is a secret string issued out of band; the bot presents it on every request and the origin checks it against what it handed out. Identity now travels with the credential instead of the address, so changing IPs no longer breaks it.

But RFC 6750 is unusually honest about the model's ceiling: "Any party in possession of a bearer token (a 'bearer') can use it to get access to the associated resources (without demonstrating possession of a cryptographic key)" [@rfc6750]. Possession is not proof -- anyone who intercepts or leaks the token can replay it.

And the secret is shared between exactly two parties, so a bot must arrange a distinct one with every origin it visits. Cloudflare names the wall directly: "You go to every website and share a secret ... This is impractical at scale because it requires developers to maintain separate tokens for each website their bot will visit" [@cf-blog-webbotauth].

flowchart LR G1["1994
robots.txt
honor system"] --> G2["1996
User-Agent
self-declared"] G2 --> G3["2006
reverse DNS + IP
network-anchored"] G3 --> G4["2012
Bearer / API keys
shared secret"] G4 --> G5["2013-2019
SigV4, cavage, Stripe, mTLS
fragmented crypto"] G5 --> G6["2024-2026
RFC 9421 + Web Bot Auth
standardized + discoverable"]

The fix for a leakable, pairwise secret is an asymmetric keypair: the bot signs with a private key it never discloses, and any origin verifies with the matching public key. One keypair works everywhere, interception no longer grants impersonation, and the secret never leaves the signer.

And here is the part most summaries get wrong -- the industry built exactly this, repeatedly, more than a decade before Web Bot Auth. AWS Signature Version 4 signed canonical requests at cloud scale [@aws-sigv4]. Stripe authenticated webhooks with an HMAC over a timestamped payload [@stripe-webhooks]. draft-cavage-http-signatures, first published in May 2013, signed selected HTTP headers with an asymmetric key and was adopted de-facto across the ActivityPub and Mastodon Fediverse [@draft-cavage]. Mutual TLS presented client certificates. Cryptographic request identity for machines was not missing. It was everywhere -- and useless for the open web.

"It's not the only way that developers can sign requests ... AWS has used Signature v4, and Stripe has a framework for authenticating webhooks -- but Message Signatures is a published standard." -- Cloudflare, "Forget IPs" [@cf-blog-webbotauth]

Why useless, if the cryptography was sound? Because every scheme was an island, and two gaps blocked open-web identity. First, no shared signature format: SigV4, Stripe, and draft-cavage each canonicalize and encode differently, so a stranger origin has no common way even to parse an unknown crawler's signature.

Second -- and this is the decisive gap -- no interoperable key discovery. The symmetric schemes hide a shared secret, which drags Generation 4's pairwise problem right back in. mTLS demands client-certificate provisioning that is famously awkward on the open web. And draft-cavage, though genuinely asymmetric, discovered keys in an application-specific way: a Mastodon server fetches the sender's actor document and reads its publicKeyPem field, which works only inside the Fediverse, not for an arbitrary origin meeting an arbitrary bot [@w3c-activitypub].

The successor spec states the gap in standards language: RFC 9421 "assumes verifiers have prior knowledge of signers' key material, requiring out-of-band key distribution ... This creates deployment friction" [@draft-httpsig-directory].AWS's documentation describes the SigV4 protocol but never states a launch year; it is widely dated to roughly 2013-2014, and this article keeps that hedge rather than asserting a precise date [@aws-sigv4].

Read the chain along two axes and the story sharpens. Unforgeability climbs the whole way down: from a string anyone can write, to a network position, to a leakable secret, to an unleakable private key. Interoperability does the opposite -- it regresses at Generation 5, which has the best cryptography of any generation before it and yet fails precisely because it has no common standard and no discovery.

The entire arc is a migration in key-distribution cost, from $O(N \times M)$ pairwise secrets and brittle per-vendor state toward a single $O(1)$ keypair backed by an $O(k)$ public directory. There is even a literal human through-line: Manu Sporny co-authored both the 2013 draft-cavage [@draft-cavage] and the 2024 RFC 9421 [@rfc9421] -- the same person carrying the same idea from pre-standard sketch to real standard.

Key idea: The breakthrough was never "add cryptography to bot requests" -- machines had signed HTTP requests since 2013. The novelty is two acts of standardization: a signature construction any verifier can parse (RFC 9421), and a public directory any origin can look up (Web Bot Auth). Interoperability and discovery, not signing, are the news.

Generation 5 had the cryptography right a full decade early. So why did signing a request fail to give bots an open-web identity? Because no two islands agreed on how to build a signature -- and the fix starts with one deceptively simple decision about what a signature should even cover.

5. RFC 9421 and the Signature Base

The genius of RFC 9421 is a single, almost counterintuitive decision: do not sign the message. Sign a description of the message. Once you see why, the whole design of Generation 6 falls into place.

An IETF Standards-Track mechanism, published February 2024, for creating and verifying signatures over *selected components* of an HTTP message -- specific fields and derived values the signer chooses -- rather than over the raw byte stream of the request.

Here is the problem with signing raw bytes. An HTTP request rarely reaches its destination unchanged. Proxies reorder headers, add their own, re-encode bodies, and normalize whitespace. A signature over the literal bytes would shatter the moment any intermediary touched the request. So RFC 9421 inverts the approach: the signer names exactly which parts of the message it wants to commit to, and signs a canonical rendering of just those parts.

A signable pseudo-field computed from the message itself rather than sent as a header -- `@method`, `@authority`, `@path`, `@query`, `@target-uri`, `@status`. The leading `@` marks it as derived, distinguishing it from ordinary HTTP fields like `content-type`.

The signer chooses its covered components -- any mix of ordinary fields and derived components -- and appends one trailing line, @signature-params, that carries the metadata: created, expires, keyid, an optional alg, an optional nonce, and a tag. All of it serializes into a canonical string.

The canonical, newline-delimited string assembled from the covered components plus the trailing `@signature-params` line. The signature base -- not the raw request -- is precisely what the private key signs and what the verifier independently reconstructs to check.

A signature base for a simple request looks like this. Each covered component is one line; the last line records the recipe itself:

"@method": GET
"@authority": example.com
"@path": /articles/latest
"@signature-params": ("@method" "@authority" "@path");created=1618884473;expires=1618884773;keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U";tag="web-bot-auth"

That block is the entire mechanism in miniature. The private key signs exactly those four lines. The result travels in two Structured-Field headers: Signature-Input, the @signature-params recipe so a verifier knows what to rebuild, and Signature, which carries the signature bytes [@rfc9421].

When the request has a body worth protecting, the signer adds a content-digest field -- a hash of the body defined by RFC 9530 -- to the covered set, so the body is committed to as well [@rfc9530]. The serialization rules all come from Structured Field Values, RFC 8941, which give headers a strict, unambiguous grammar of items, lists, and dictionaries [@rfc8941].Without a strict grammar, two implementations could serialize the same header differently and never agree on the signature base -- which is precisely how the Generation 5 islands failed to interoperate.

flowchart TD R["HTTP request"] --> C["Pick covered components
@method, @authority, @path, chosen fields"] C --> P["Append @signature-params
created, expires, keyid, tag"] P --> B["Canonical signature base
one line per covered component"] B --> S["Ed25519 sign the base"] S --> H["Emit Signature-Input (recipe) and Signature (bytes)"]

Now the payoff, and it is the aha of the whole standard. Because the signature commits to chosen components rather than raw bytes, it survives intermediaries, re-encodings, and proxies -- the signer decides exactly what the signature guarantees and what it deliberately leaves free to change.

This is the precise property every Generation 5 island lacked. SigV4 and draft-cavage each had their own private notion of what to canonicalize; RFC 9421 makes that notion a published standard, so any conforming verifier -- in any language, at any origin -- can reconstruct the base and check the signature without prior agreement with the signer.

A fast, deterministic elliptic-curve signature scheme -- EdDSA using the edwards25519 curve, the twisted-Edwards curve birationally equivalent to Curve25519 -- with compact 32-byte keys. It is Web Bot Auth's deployed default; this blog's field guide on digital signatures covers the underlying curve mathematics.

RFC 9421 is deliberately algorithm-agnostic. Ed25519 is the deployed default because it is fast, deterministic, and small enough to run per request at internet scale, but the standard commits to no single scheme.

The IANA "HTTP Signature Algorithms" registry -- Specification Required, so it can grow -- also profiles RSA-PSS-SHA512, RSA-PKCS1-v1.5-SHA256, ECDSA over P-256 and P-384, and HMAC-SHA256 [@iana-httpsig-registry]. The alg is a property of the key, not a mandate of the protocol, which matters later when quantum-resistant schemes need a home.RFC 9421 was edited by Annabelle Backman (Amazon), Justin Richer (Bespoke Engineering), and Manu Sporny (Digital Bazaar) -- the same Sporny who co-authored the 2013 draft-cavage, which is why the standard is best read as a revival rather than an invention [@rfc9421].

One more precision, because the record is muddled. RFC 9421 is a Standards-Track document published in February 2024.Several AI-generated summaries date RFC 9421 to June 2023; the RFC Editor and IETF Datatracker both settle it as February 2024 [@rfc9421]. When a source and a chatbot disagree on a standard's date, trust the RFC Editor.

RFC 9421 tells you how to sign a request so that anyone can verify it. But "anyone can verify it" hides a trapdoor. To check a signature, a verifier needs the signer's public key -- and RFC 9421 says nothing about where to find it. It "assumes verifiers have prior knowledge of signers' key material, requiring out-of-band key distribution" [@draft-httpsig-directory]. For a paying customer's agent meeting an origin it has never contacted, that missing half is not a detail. It is the entire problem. Solving it is what turns a signature into an identity.

6. Web Bot Auth: From a General Tool to a Bot Identity

RFC 9421 is a general engine. Web Bot Auth is what happens when Cloudflare and Google point that engine at one specific job -- proving which bot sent a request -- and bolt on the piece the RFC deliberately left out. The result is authored by Thibault Meunier of Cloudflare and Sandor Major of Google, and it answers the trapdoor question from the last section head-on: how does a stranger find your key?

Start with the cast. Web Bot Auth defines a three-actor model. A User delegates a task. An Agent carries it out and signs each outbound request with its private key. An Origin receives the request and verifies the signature. The clean separation matters: the party that signs is the agent operator, not the human, and that distinction returns to haunt the governance discussion later.Because the operator signs, one key can sit in front of thousands of different end users -- the seed of the intermediary-trust problem that resurfaces in the open problems.

On top of RFC 9421's freedom, the profile pins down the parts that must be fixed for strangers to interoperate.

Component or parameter	RFC 9421 says	Web Bot Auth requires
Covered components	Signer's free choice	MUST cover at least `@authority` (or `@target-uri`) [@draft-webbotauth-arch]
`created` / `expires`	Optional metadata	Both MUST be present; expiry RECOMMENDED ≤ 24 hours [@draft-webbotauth-arch]
`keyid`	Opaque signer-chosen string	MUST be the base64url JWK SHA-256 Thumbprint [@rfc7638]
`tag`	Optional label	MUST be `web-bot-auth` [@draft-webbotauth-arch]
`nonce`	Optional	A 64-byte nonce is the recommended anti-replay tool [@draft-webbotauth-arch]
Key discovery	Out of scope	JWKS at a well-known path, found via `Signature-Agent` [@draft-httpsig-directory]

The last row is the whole point -- the genuine novelty of the entire effort.

A JSON document listing public keys, each expressed as a JSON Web Key. The Web Bot Auth directory is a JWKS: a signer's public keys, published for anyone to fetch.

A signer publishes its public keys as a JWKS at the well-known path /.well-known/http-message-signatures-directory, and points to that directory in-band with a request header.

An in-band request header whose value is a Structured-Dictionary URI pointing at the signer's key directory. Because Web Bot Auth lists `Signature-Agent` among the covered components, an attacker cannot swap the pointer without invalidating the signature.

That self-protecting loop is the elegant part. The header says where the keys live; the signature covers the header; so the pointer to the keys is itself signed. And the directory response must in turn be signed -- with tag="http-message-signatures-directory", one signature per listed key -- so no one can mirror your directory at their own host and enroll as you [@cf-docs-webbotauth]. The keyid closes the loop by naming the exact key without any external registry.

A stable, self-certifying key identifier: the base64url-encoded SHA-256 hash over a JSON Web Key's required members in canonical form. It is what `keyid` carries, so a key effectively names itself.

You can watch a keyid derive itself. The snippet below takes the real Ed25519 public key served by Cloudflare's live debug directory, canonicalizes its required members exactly as RFC 7638 prescribes, hashes, and base64url-encodes the result -- and out falls the same kid the directory advertises [@cf-live-directory].

{const NL = String.fromCharCode(10); function rotr(n, x) { return (x >>> n) | (x << (32 - n)); } function sha256(bytes) { const K = [ 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5,0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5, 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3,0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174, 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc,0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da, 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7,0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967, 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13,0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85, 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3,0xd192e819,0xd6990624,0xf40e3585,0x106aa070, 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5,0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3, 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208,0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2]; let h = [0x6a09e667,0xbb67ae85,0x3c6ef372,0xa54ff53a,0x510e527f,0x9b05688c,0x1f83d9ab,0x5be0cd19]; const l = bytes.length, withOne = l + 1; const pad = (56 - (withOne % 64) + 64) % 64, total = withOne + pad + 8; const m = new Uint8Array(total); m.set(bytes); m[l] = 0x80; const dv = new DataView(m.buffer); dv.setUint32(total - 4, (l * 8) >>> 0); dv.setUint32(total - 8, Math.floor(l * 8 / 0x100000000)); const w = new Uint32Array(64); for (let i = 0; i < total; i += 64) { for (let t = 0; t < 16; t++) w[t] = dv.getUint32(i + t * 4); for (let t = 16; t < 64; t++) { const s0 = rotr(7,w[t-15]) ^ rotr(18,w[t-15]) ^ (w[t-15] >>> 3); const s1 = rotr(17,w[t-2]) ^ rotr(19,w[t-2]) ^ (w[t-2] >>> 10); w[t] = (w[t-16] + s0 + w[t-7] + s1) >>> 0; } let a = h[0], b = h[1], c = h[2], d = h[3], e = h[4], f = h[5], g = h[6], hh = h[7]; for (let t = 0; t < 64; t++) { const S1 = rotr(6,e) ^ rotr(11,e) ^ rotr(25,e); const t1 = (hh + S1 + ((e & f) ^ (~e & g)) + K[t] + w[t]) >>> 0; const S0 = rotr(2,a) ^ rotr(13,a) ^ rotr(22,a); const t2 = (S0 + ((a & b) ^ (a & c) ^ (b & c))) >>> 0; hh=g; g=f; f=e; e=(d+t1)>>>0; d=c; c=b; b=a; a=(t1+t2)>>>0; } h = [(h[0]+a)>>>0,(h[1]+b)>>>0,(h[2]+c)>>>0,(h[3]+d)>>>0,(h[4]+e)>>>0,(h[5]+f)>>>0,(h[6]+g)>>>0,(h[7]+hh)>>>0]; } const out = new Uint8Array(32), odv = new DataView(out.buffer); for (let i = 0; i < 8; i++) odv.setUint32(i*4, h[i]); return out; } function base64url(bytes) { let s = ''; for (const b of bytes) s += String.fromCharCode(b); return btoa(s).split('+').join('-').split('/').join('_').split('=').join(''); } // The Ed25519 public key served by Cloudflare's live Web Bot Auth directory: const jwk = { crv: 'Ed25519', kty: 'OKP', x: 'JrQLj5P_89iXES9-vFgrIy29clF9CC_oPPsw3c5D0bs' }; // RFC 7638: keep only the required members, sort them, no whitespace. const canonical = '{"crv":"' + jwk.crv + '","kty":"' + jwk.kty + '","x":"' + jwk.x + '"}'; const thumbprint = base64url(sha256(new TextEncoder().encode(canonical))); console.log('Canonical JWK : ' + canonical); console.log('Thumbprint : ' + thumbprint); console.log('Directory kid : poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U'); console.log('Identical? : ' + (thumbprint === 'poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U'));}

With the signature standard and the directory in place, the verifier's job is mechanical: fetch the directory, match the keyid thumbprint, reconstruct the signature base, verify the Ed25519 signature, and enforce the validity window. Only then -- as a wholly separate step -- does it decide policy.

sequenceDiagram participant U as User participant A as Agent participant O as Origin participant D as Key directory U->>A: Delegate a task, fetch this page A->>A: Build signature base, Ed25519 sign A->>O: Send request with Signature-Input, Signature, Signature-Agent O->>D: Fetch or read cached directory at the well-known path D-->>O: Return signed JWKS of public keys O->>O: Match keyid thumbprint, reconstruct base, verify O->>O: Enforce created ≤ now ≤ expires O-->>A: Identity established, then apply policy separately

Ed25519 is the deployed default here, but the mechanism stays algorithm-agnostic: the same registry that lets RFC 9421 carry RSA-PSS or ECDSA lets Web Bot Auth carry them too [@iana-httpsig-registry]. Deployment is real but early, and precision matters.The directory draft registers the plural path /.well-known/http-message-signatures-directory [@draft-httpsig-directory], while some Cloudflare prose uses a singular alias; the live server answers on both, and this article uses the registered plural form [@cf-blog-verifiedbots]. The strongest independent signal that this is not vendor theater is OpenAI.

"With HTTP Message Signatures (RFC 9421), OpenAI signs all Operator requests so site owners can verify they genuinely originate from Operator and haven't been tampered with." -- Eugenio, Engineer, OpenAI [@cf-blog-webbotauth] Cloudflare moved this from a closed beta announced in May 2025 [@cf-blog-webbotauth] to integration in its Verified Bots Program in July 2025, where its edge automatically validates signatures and marks matching traffic as verified [@cf-blog-verifiedbots]. As of a July 2026 taxonomy update, Web Bot Auth is one of two accepted validation methods for becoming a Verified bot [@cf-docs-verifiedbots]. But the profile is still a set of Individual Internet-Drafts, not RFCs, and edge validation is ramping rather than generally available. This is production-adjacent, not finished.

Note: Cloudflare's production implementation pins to specific earlier draft revisions -- roughly directory-03 and architecture-02 -- while the latest published drafts are at -05, and the architecture draft was renamed mid-flight to draft-meunier-webbotauth-httpsig-protocol [@cf-docs-webbotauth] [@draft-webbotauth-protocol]. Interop can break silently across revisions, so verify which draft your counterparty implements before assuming compatibility.

With a signature and a directory, an origin can verify a bot it has never met, on first contact, with zero prior arrangement. That is the headline. But Cloudflare shipped a second proposal in the very same blog, and there is a whole family of rival ways to prove a machine's identity. Why did signatures win?

7. Signatures, Certificates, and the Other Signers

Cloudflare proposed two mechanisms in the same post, and the one it did not prioritize explains why the one it did is pulling ahead. The road not taken is request mTLS.

A handshake in which both peers present X.509 certificates, so each authenticates the other at the transport layer, beneath HTTP. The resulting identity is bound to the TLS *connection*, not to any individual request on it.

Classic mTLS is decades old and rock-solid inside a managed setting where both sides already run a private certificate authority. The problem is the open web. If an origin asks for a client certificate and the client has none, the browser shows "an inscrutable" and unskippable error [@cf-blog-webbotauth]. To soften that, draft-jhoyla-req-mtls-flag adds a TLS flag -- an experimental extension -- by which a client signals it holds a suitable certificate, so the server can request one only when it will not break an ordinary visitor [@draft-jhoyla-mtls].

It is a real, live parallel track. But Cloudflare is candid about ranking it second: it prioritizes Message Signatures because that approach "relies on the previously adopted RFC 9421 with several reference implementations, and works at the HTTP layer" [@cf-blog-webbotauth]. The reference-implementation point is concrete -- RFC 9421 already has an implementations hub and independent libraries to build on [@httpsig-org].

flowchart TD subgraph HL["HTTP layer -- Web Bot Auth"] A1["Signature travels inside headers"] --> A2["Survives CDNs and reverse proxies"] end subgraph TL["Transport layer -- request mTLS"] B1["Identity bound to the TLS connection"] --> B2["Lost when a proxy terminates TLS upstream"] end

The layer difference is not cosmetic. A signature lives in HTTP headers, so it rides through content delivery networks and reverse proxies untouched and can be verified at the edge or the origin. A client certificate binds to the connection, so a reverse proxy that terminates TLS -- how much of the web is commonly served -- strips the identity before the request reaches the application. For an internet built on intermediaries, HTTP-layer identity is simply the one that arrives.

Web Bot Auth also clarifies its relationship to the Generation 5 signers. It keeps draft-cavage's asymmetric idea but replaces the Fediverse-specific publicKeyPem lookup with a standard format and an open-web directory [@draft-cavage] [@w3c-activitypub]; and unlike SigV4 or Stripe, it needs no shared secret, only a public key the origin can fetch. Every path so far -- reverse DNS, mTLS, SigV4, Web Bot Auth -- differs less in its cryptography than in the row of the table that has no cryptography at all: how a verifier discovers the key.

Dimension	A. Web Bot Auth	B. Request mTLS	C. Reverse DNS + IP	D. Privacy Pass
What it proves	Who signed, plus integrity of covered parts	Who holds the client certificate	Which network sent the packet	A property, unlinkably
Layer	HTTP (application)	TLS (transport)	Network / DNS	HTTP (application)
Cryptography	Asymmetric, Ed25519 default	Asymmetric, X.509	None	Blind signatures / VOPRF
Key discovery	Public JWKS + `Signature-Agent`, in-band	PKI / CA, out-of-band	Published range files, per-vendor	Issuer directory
Replay protection	Validity window + optional nonce	TLS channel binding	None at the IP layer	Token single-use rules
Survives reverse proxies	Yes, an HTTP header	No, if TLS is terminated upstream	Only with trusted forwarding	Yes, an HTTP header
Privacy	Identifying by design	Identifying	Coarsely identifying	Unlinkable by design
Best suited for	Open-web stranger identity	Managed B2B and API	Legacy search crawlers	Privacy-preserving attestation

Method D in that table is not really a rival at all -- it answers the opposite question.

Privacy Pass and Apple's Private Access Tokens, standardized in June 2024 as RFCs 9576, 9577, and 9578, prove a *property* about a request -- "a human or an attested device stands behind this" -- without revealing who, and without letting the origin link one request to another [@rfc9576] [@rfc9577] [@rfc9578]. That is the mirror image of a signature. Web Bot Auth maximizes accountable identity; Privacy Pass maximizes unlinkable anonymity. They sit at opposite ends of the same axis and coexist because they serve different jobs: one names the bot, the other refuses to.

Line the methods up by how a verifier learns the key and the pattern is stark. mTLS and SigV4 distribute keys out of band. IP schemes distribute a public list of ranges. Only Web Bot Auth carries an in-band pointer to a standardized, discoverable directory -- the one design choice, containing no new cryptography, that makes stranger-to-stranger verification possible on the open web.

And yet notice what unites every row, signatures included. Each mechanism proves exactly one kind of thing: who, or what network, or what property. Not one of them, however strong its cryptography, can tell you whether the bot should be admitted. That gap is not an engineering oversight to be patched in the next draft. It is a limit on what a signature is -- and it is time to look at it directly.

8. What a Signature Cannot Prove

Here is the sentence the whole article has been walking toward. A valid signature proves who signed and what they covered -- and nothing else. Not that the bot is well-behaved. Not that it respects your rate limits. Not that it is allowed anywhere near your origin.

This is not a shortcoming to be engineered away in a later draft; it is the definition of what a signature is. A digital signature is a proof of key possession over a covered message. It can establish authenticity -- which key signed -- and integrity -- that the covered components were not altered. It is information-theoretically silent on everything else.

"Allowed" is a predicate over policy and behavior, not over bytes, so no amount of cryptographic strengthening can make a signature answer it. The web already learned this once: RFC 9309 says the rules in a robots.txt file "are not a form of access authorization" [@rfc9309]. Web Bot Auth simply makes the identity half verifiable while leaving that same authorization line exactly where it was.

flowchart TD R["Signed request arrives"] --> V["Verify signature and validity window"] V --> I["Identity established: who signed, what was covered"] I --> G{"Policy gate, not cryptography"} G -->|trusted and well-behaved| Allow["Admit"] G -->|unknown or abusive| Deny["Deny or rate-limit"]

Three concrete limits make the boundary vivid, and each is fundamental rather than incidental.

First, replay. A signature with a validity window can be captured and replayed by anyone until it expires. The only defenses are a short window and a server-tracked nonce -- and the moment the server tracks a nonce, it is keeping state. This is a genuine lower bound, not a missing feature: stateless anti-replay is impossible. You trade window size against verifier memory, and Web Bot Auth's mandatory created/expires plus optional 64-byte nonce sit exactly on that trade-off. A verifier that admits a request only when $created \le now \le expires$ has narrowed the replay opportunity, never closed it [@cf-blog-webbotauth].

Second, uncovered components. A signature commits only to what it lists. Anything the signer left out of the covered set can be altered in flight without breaking verification. That is why the profile insists on covering @authority and Signature-Agent -- so the target host and the pointer to the key directory cannot be quietly swapped. Coverage is a choice, and everything outside it is unprotected by construction.

Third, the trust-anchor problem, which is the deepest of the three. A verified signature does not create trust; it relocates it. "Do I trust this request?" becomes "do I trust the directory behind this key?" Cryptography cannot manufacture the first bit of trust in which signers matter -- that decision has to come from somewhere outside the math. It is the same unanswered question that sits under the entire web public-key infrastructure: who gets to be a certificate authority? Web Bot Auth relocates that question to "whose directory do I honor?" and, by design, leaves it open [@draft-httpsig-directory].

One more frontier the standards make concrete by existing side by side: Web Bot Auth points toward accountable, linkable identity, Privacy Pass toward unlinkable anonymity [@rfc9576]. You can pick a point on that axis, but not both endpoints in one token -- a design decision, not a bug awaiting a fix.

A valid signature proves who signed and what was covered -- never whether the bot is allowed. Signing makes a bad bot accountable, not good.

It is worth naming what these limits are not. This domain has no impossibility theorems in the Turing sense, no proven complexity lower bounds, no hardness results. The honest "bounds" here are conceptual: they describe what a signature can attest, not how expensive it is to compute. Pretending otherwise -- dressing a category boundary up as a big-O result -- would be a fabrication. The real limit is cleaner and more absolute than any asymptotic one.

Key idea: Identity and authorization are not two ends of one dial. They are different categories. A signature is silent on "allowed" the way a photograph is silent on "true" -- not because it is a weak signature, but because "allowed" is not the kind of thing a signature can encode. This is a boundary, not a backlog item.

So where does "allowed" get decided? Not here.

Deciding whether an identified agent may perform an action is the job of an authorization system -- the concern of standards like [AuthZEN](/blog/who-authorized-this-tool-call-openid-authzen-the-mcp-profile/), covered in this blog's agent-authorization post. The clean handoff is the point: cryptography answers "who," and policy answers "allowed," and keeping them separate is what lets each be reasoned about honestly. A further layer sits beyond "allowed" -- proving *what* a user authorized an agent to do, such as a payment -- the domain of this blog's companion post on agentic payment intent ([FIDO AP2 / Verifiable Intent](/blog/the-mandate-that-replaced-the-checkout-button-how-fido-ap2-a/)). A signed request is the beginning of an access decision, never the whole of it.

If a signature only proves who, then "who do we trust, and who decides?" becomes the entire game. And nobody has fully answered it. That unfinished question opens onto the frontier of governance.

9. Governance, Revocation, and the Legitimacy Gap

Verifiable identity solves the easy half of the problem. The hard half -- who deserves trust, and how you take it back -- is wide open, and almost none of it is a cryptography question. Here are the eight problems that will decide whether Web Bot Auth becomes the default or a footnote.

Signer accreditation. A signature proves who signed, never whether that signer deserves trust. Someone still has to decide which directories an origin honors. Cloudflare's Verified Bots Program, backed by its BotBase and Radar data, is a de-facto answer -- but a single-vendor one [@cf-docs-verifiedbots]. The open problem is a neutral, multi-party accreditation model, without which every origin rebuilds the per-vendor allowlist that this whole effort set out to retire.
Revocation and rotation at web scale. A leaked key needs to be pulled fast and globally. Short expires windows bound the exposure, and a signer can drop a key from its JWKS, but there is no low-latency revocation or transparency mechanism -- nothing like the certificate world's revocation lists or transparency logs -- for these directories yet [@draft-httpsig-directory].
Directory availability and denial of service. Verification depends on fetching a third party's well-known directory. When that directory is slow, down, or flooded, the request hot path inherits a new availability and amplification surface. Signing the directory response makes cached copies tamper-evident, but cache-lifetime and denial-of-service guidance is still thin [@cf-docs-webbotauth].
The legitimacy gap. A signed bot can still scrape aggressively, ignore robots.txt, or blow past rate limits. Signing makes it accountable, not good. Cloudflare's response is to bundle identity with a behavioral bar -- a Verified bot must both self-identify honestly and be non-abusive, obeying robots.txt at reasonable rates [@cf-docs-verifiedbots]. That is policy bolted on top of identity, exactly as the identity-versus-authorization spine predicts.
Transitive trust for intermediary agents. An "agent mode" browser is driven by many different end users, so trusting the operator's key is not the same as trusting every person behind it. The party that signs is not the party ultimately responsible for a given action. Cloudflare's July 2026 taxonomy adds a Direct-versus-Intermediary label and is experimenting with forwarding end-user information in the standard Forwarded header, so an origin can apply policy to the responsible party [@cf-docs-verifiedbots].
Adoption bootstrapping. Signers gain little until origins verify, and origins gain little until signers sign -- the classic two-sided chicken-and-egg. The escape is asymmetric scale: a large reverse proxy verifying by default seeds millions of origins at once [@cf-blog-verifiedbots], while a marquee signer like OpenAI's Operator seeds the client side [@cf-blog-webbotauth].
Standardization status and version skew. The profile is a set of Individual Internet-Drafts, not RFCs, and it is a moving target: the architecture draft was renamed mid-flight to draft-meunier-webbotauth-httpsig-protocol [@draft-webbotauth-protocol], while production pins to earlier revisions than the latest. Interop can break silently between draft versions until a working group freezes the text.
Post-quantum migration. Ed25519, RSA, and ECDSA are all breakable by a sufficiently large quantum computer. A long-lived signing infrastructure should become crypto-agile before it must. RFC 9421's algorithm registry is extensible by design, and post-quantum signature schemes are being explored for it, but none is the deployed default today [@iana-httpsig-registry].

Note: Notice that not one item on this list is fixed by a better signature. Accreditation, revocation, availability, legitimacy, delegation, adoption, standardization, and quantum-readiness are questions of governance and process. Cryptography set the identity floor; everything above it is human coordination -- and coordination moves slower than code.The datatracker landing pages for these drafts are version-agnostic and stable even as the documents are renamed, which is why this article cites them rather than any single versioned snapshot [@draft-webbotauth-protocol].

None of these are reasons to wait. The identity layer they sit on is already real, already standardized at its base, and already signing live traffic from a marquee agent. Which leaves one intensely practical question, the one an engineer actually needs answered: how do you do this, on the wire?

10. Doing It on the Wire

Enough theory. Here is the whole protocol as an engineer meets it, in two roles.

If you are building a bot, there are four steps. Generate an Ed25519 keypair. Publish your public keys as a JWKS at /.well-known/http-message-signatures-directory, and sign that directory response itself with tag="http-message-signatures-directory", one signature per key [@cf-docs-webbotauth]. On each outbound request, build a signature base that covers at least @authority and signature-agent, set a short created/expires window, set keyid to your key's JWK Thumbprint [@rfc7638], and set tag="web-bot-auth". Then emit three headers -- Signature-Agent, Signature-Input, and Signature. On the wire, using the current directory draft's Structured-Dictionary URI for Signature-Agent [@draft-httpsig-directory], that is:

Signature-Agent: sig="https://signer.example.com"
Signature-Input: sig=("@authority" "signature-agent";key="sig");created=1700000000;expires=1700011111;keyid="ba3e64==";tag="web-bot-auth"
Signature: sig=abc==

Three lines carry the entire identity claim [@cf-blog-webbotauth]. Signature-Agent says where the keys live, Signature-Input is the recipe, and Signature is the proof.

If you are the origin, the flow reverses. Parse Signature-Agent to learn where the directory is. Fetch it -- or reuse a cached copy -- and match the request's keyid to a published key. Reconstruct the signature base byte-for-byte from the same covered components, verify the Ed25519 signature against the matched key, and enforce that the request falls inside its created/expires window. Only after all of that succeeds do you reach the separate, non-cryptographic step: apply policy.

The one part worth running yourself is the signature base, because byte-identical reconstruction is where implementations most often fail. The snippet below assembles exactly the string a signer signs, then runs the validity-window check the verifier performs.

{const NL = String.fromCharCode(10); // Build the RFC 9421 signature base: the exact string a signer signs // and a verifier must reconstruct byte-for-byte. function componentId(c) { return '"' + c.name + '"' + (c.key ? ';key="' + c.key + '"' : ''); } function buildSignatureBase(covered, values, params) { const lines = []; for (const c of covered) { lines.push(componentId(c) + ': ' + values[c.name]); } const innerList = '(' + covered.map(componentId).join(' ') + ')'; const sigParams = innerList + ';created=' + params.created + ';expires=' + params.expires + ';keyid="' + params.keyid + '"' + ';tag="' + params.tag + '"'; lines.push('"@signature-params": ' + sigParams); return lines.join(NL); } // Web Bot Auth MUST cover @authority; draft-05 covers the signature-agent ;key="sig" member. const covered = [ { name: '@authority' }, { name: 'signature-agent', key: 'sig' } ]; const values = { '@authority': 'example.com', 'signature-agent': '"https://signer.example.com"' }; const params = { created: 1700000000, expires: 1700000300, keyid: 'poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U', tag: 'web-bot-auth' }; const base = buildSignatureBase(covered, values, params); console.log('--- signature base (input to Ed25519) ---'); console.log(base); // The origin enforces the validity window BEFORE trusting the signature. const now = 1700000300; const withinWindow = params.created <= now && now <= params.expires; console.log(''); console.log('created <= now <= expires : ' + (withinWindow ? 'VALID' : 'REJECT')); console.log('the real signing is then one call: ed25519.sign(privateKey, base)');}

That is genuinely the whole computation. The actual signing is a single library call on the string the snippet builds, and verification is the same string plus one verify call.

Note: Sign the directory response itself, or an attacker can mirror your directory and enroll as you [@cf-docs-webbotauth]. Reconstruct the signature base byte-for-byte -- the Structured Field grammar of RFC 8941 is strict, and a stray space or a reordered field breaks verification [@rfc8941]. Always cover @authority and Signature-Agent so the host and the key-directory pointer cannot be swapped. Watch clock skew against short windows -- a five-minute expiry is safer against replay but less forgiving of unsynchronized clocks. Pin the draft revision your counterparty implements. And deploy Ed25519: the standard is algorithm-agile, but Ed25519 is what interoperates today [@draft-webbotauth-arch].

You do not have to build this from scratch. The reference implementations are first-party: an npm web-bot-auth package [@npm-webbotauth], a Rust crate, a Cloudflare Workers verifier, and a Caddy plugin, all in the public repositories [@gh-cloudflareresearch] [@gh-cloudflare-webbotauth], plus the broader RFC 9421 implementations hub [@httpsig-org]. Best of all, there is a live directory you can hit right now.Cloudflare runs a live debug directory at http-message-signatures-example.research.cloudflare.com; its well-known path serves a real Ed25519 JWK whose kid is poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U -- the very key the earlier thumbprint snippet reproduces [@cf-live-directory].

Fetch that directory, feed its key into the verifier, and you have reproduced the entire mechanism end to end. The discovery layer that RFC 9421 left out is the part you can now watch working, live [@draft-httpsig-directory].

Run `curl` against the debug host's well-known path -- `http-message-signatures-example.research.cloudflare.com/.well-known/http-message-signatures-directory` -- and the response is a signed JWKS containing a single Ed25519 key. Its `kid` is the exact thumbprint the earlier snippet computes [@cf-live-directory].

Stand back and the mechanics are almost anticlimactic: a keypair, a JSON file at a well-known URL, and two request headers. The hard part was never the code. It was getting the entire web to agree on the same keypair format, the same JSON document, the same two headers -- so that an origin and a bot that have never met can still understand each other. That agreement is the thirty-year story you just read.

11. Frequently Asked Questions

No. Mutual TLS authenticates at the transport layer with a per-connection X.509 certificate, and it struggles on the open web: a client without a certificate gets an inscrutable browser error, provisioning and rotation are heavy, and the identity does not survive a reverse proxy that terminates TLS. Web Bot Auth signs at the HTTP layer, per request, so it rides through content delivery networks and can be verified at the edge [@cf-blog-webbotauth]. No. A signature proves who signed and what was covered. "Allowed" is a separate policy decision -- identity is not authorization. The web drew this exact line for `robots.txt`, whose rules "are not a form of access authorization" [@rfc9309]. A signature makes the sender *nameable*, not *trustworthy* -- it hands you someone to hold responsible, not a reason to say yes. No. RFC 9421 is algorithm-agnostic, and its IANA registry also profiles RSA-PSS, ECDSA over P-256 and P-384, and HMAC [@iana-httpsig-registry]. Ed25519 is the deployed default for interoperability today, not a mandate. No, and this is the point most coverage gets wrong. AWS Signature v4, `draft-cavage` HTTP Signatures across the Fediverse, Stripe webhook signatures, and mTLS all predate it [@cf-blog-webbotauth] [@draft-cavage]. The novelty is a standardized, discoverable, open-web identity -- not signing itself. No. `robots.txt` says *what* an origin permits; Web Bot Auth establishes *who* is asking. Identity makes etiquette enforceable against a named party for the first time, so the two are complementary rather than competing [@rfc9309]. Partly. RFC 9421 is a published standard, but the Web Bot Auth profile is a set of Individual Internet-Drafts, not RFCs, and Cloudflare's edge validation is ramping from beta into its Verified Bots Program rather than being generally available [@cf-blog-verifiedbots]. OpenAI's Operator, however, signs in production today [@cf-blog-webbotauth]. A short `created`/`expires` window plus an optional server-tracked 64-byte nonce [@draft-webbotauth-arch]. But stateless anti-replay is impossible: you trade a smaller window against the verifier having to remember nonces. The window shrinks the opportunity, it never eliminates it.

After Thirty Years, an Identity

Return to the two requests from the opening -- the agent and the scraper, identical down to the byte. The web spent its first three decades unable to separate them, not for lack of trying but because every tool it reached for answered the wrong question. robots.txt asked bots to behave. User-Agent let them name themselves. Reverse DNS pinned them to a network. Bearer tokens and per-vendor signatures added real cryptography but locked it inside islands no stranger could reach. Each was a better answer to "which bot is this?" and each broke in a way that named its successor.

RFC 9421 and Web Bot Auth close the loop with two acts of standardization the previous generations never managed: a signature construction any verifier can parse, and a public directory any origin can look up. That is the honest shape of the claim. Not the first cryptography for machines -- the first cryptographic identity for machines that is standardized, discoverable, and open to the whole web. A bot generates one keypair, publishes one signed JSON document at one well-known URL, and any origin on earth can verify it on first contact, with no prior arrangement.

After decades of hardening how it authenticates humans, the web has finally given agents a verifiable cryptographic identity -- finally standardized and discoverable, not first-ever. And it proves exactly one thing: who signed, never whether they are allowed.

That last clause is the discipline the whole article kept. A signature makes a bot accountable, not good. It relocates trust, it does not manufacture it. Everything above the identity floor -- who deserves trust, how you revoke it, what "allowed" even means for an agent acting for a thousand different users -- is governance, and governance is unfinished. The door now knows who is knocking. Deciding whether to open it is a different, still-unwritten story, and it begins exactly where this one ends.

A Perfect Signature for a Certificate That Should Never Have Existed: A Field Guide to X.509 and PKI

noreply@paragmali.com (Parag Mali) — Wed, 08 Jul 2026 12:12:39 GMT

A certificate is a signed statement, and "verifying" it is a one-line signature check -- which is exactly why almost every famous PKI disaster left the cryptography flawless. DigiNotar's rogue `*.google.com` certificate, Flame's forged Microsoft signature, and Debian's guessable-but-valid keys were failures of *delegation*, not math: whom a trusted authority vouched for, what it was allowed to say, and whether anyone could take that trust back. This field guide hands you one lens -- *whom am I trusting, to say what, and how would I know if they lied?* -- and uses it to walk chain building versus path validation, the authorization-extension bug family, revocation and why it stayed broken, and Certificate Transparency's pivot from *preventing* mis-issuance to merely *detecting* it. It closes with the 2024-2026 answer -- short-lived and 47-day certificates, CRLite, multi-perspective issuance, tiled logs, and post-quantum certificates -- and a design-review checklist for using X.509 correctly in 2026.

1. The Certificate That Should Not Have Existed

In the late summer of 2011, hundreds of thousands of people in Iran opened Gmail over what looked like a perfectly secure connection. The padlock was closed. The certificate for *.google.com chained to a trusted root. Every signature in that chain verified flawlessly. It was also completely fake: minted by a breached Dutch certificate authority called DigiNotar, it let someone read all of that traffic, and nothing in any browser's validation logic so much as blinked [@foxit-diginotar]. The RSA math was perfect. The trust was the bug.

Sit with the paradox, because the rest of this article lives inside it. A certificate that is cryptographically perfect and a total compromise at the same time is not a contradiction the mathematics can resolve. Modular exponentiation did what it promised; SHA hashed what it was given; the signature over the certificate was valid in the strict, checkable sense that it matched the issuer's public key. And a browser trusting that certificate handed an eavesdropper a live feed of people's email.

Here is the reframing that dissolves it. A certificate is not identity, and it is not magic. It is a signed statement: one party vouches, by signature, that a particular public key belongs to a particular name. "Verifying" a certificate means checking that signature -- the easy, automatable part. Deciding whether to believe the statement it signs is the hard part, and no signature check can do it for you.

At DigiNotar, the statement was a lie, signed by an authority the whole Web had agreed to trust. The variable was never the cryptography. It was delegation -- whom a trusted authority chose to vouch for.

That gives us the single diagnostic tool this field guide is built around, a sentence to ask of any trust system you will ever design or debug:

Whom am I trusting, to say what, and how would I know if they lied?

By the end of this article you will drop any PKI disaster into one of those buckets on sight. DigiNotar is a whom did I trust failure -- the answer turned out to be an attacker. Flame, a year later, forged a Microsoft code-signing certificate with a novel hash collision, a what were they allowed to say failure reaching into a corner of Microsoft's own PKI [@flame-counter-crypto-2013]. Debian's OpenSSL, in 2008, generated certificates that were structurally flawless and yet trivially guessable [@cve-2008-0166]. None of them broke a cipher. Every one of them broke a delegation.

This is Part 4 of a field guide for protocol designers, and it assumes the earlier parts' habits of mind. DigiNotar is, in the language of Part 1, a wrong-adversary-model story: path validation was secure against exactly the attacker it was never going to face. Debian belongs to Part 2's account of what a weak key is. And the parsing bugs we will meet in the enforcement code are Part 3's serialization hazards wearing certificates.

If neither RSA's modular exponentiation nor SHA's collision resistance was broken at DigiNotar, then what, exactly, failed? To answer that with any precision, you first have to know what a certificate actually is -- and what a signature over one does, and does not, promise.

2. What an X.509 Certificate Actually Is

Strip away the mystique and an X.509 certificate is a small, rigidly-encoded data structure in which one key vouches, by signature, for a binding between another key and a name, plus a set of machine-readable limits on what that key may do.

A signed data structure that binds a public key to a name (and to a set of usage constraints), so that anyone holding the issuer's public key can verify the binding offline. Its format is defined by ITU-T Recommendation X.509 and profiled for the Internet by RFC 5280.

The signed part of the certificate is called the tbsCertificate -- "to be signed." RFC 5280 fixes its fields: a version number, a serial number, the issuer's name, a validity window (notBefore and notAfter), the subject's name, the subject's public key (subjectPublicKeyInfo), and, since version 3, a list of extensions [@rfc5280]. The issuer then hashes that whole body and signs the hash. The certificate you receive on the wire is the body plus that one signature.

flowchart TD subgraph TBS["tbsCertificate (the signed body)"] V["version, serial number"] I["issuer name"] VAL["validity: notBefore, notAfter"] SUB["subject name"] SPKI["subjectPublicKeyInfo: the bound key"] EXT["extensions: v3 authorization scope"] end TBS --> H["hash the entire body"] H --> SIG["issuer signs the hash with its private key"] SIG --> CERT["certificate: body plus one signature"]

What does verifying that signature actually establish? Almost exactly one thing. Let $H$ be the hash function and $\sigma$ the signature; a relying party recomputes $H(\text{tbsCertificate})$, runs the signature-verification algorithm against the issuer's public key $pk$, and confirms the clock is inside the window:

$$\text{Verify}(pk,, H(\text{tbsCertificate}),, \sigma) = 1 \quad\text{and}\quad t_{\text{notBefore}} \le t_{\text{now}} \le t_{\text{notAfter}}.$$

That is an $O(1)$ operation over a single certificate -- one hash, one signature check, two date comparisons. A student can implement it in an afternoon.The cost of verifying one certificate does not grow with anything: it is a fixed number of operations regardless of how many certificates exist in the world. The hard part of PKI is never this check. It is deciding whether to believe the statement the check confirms was signed.

This is the load-bearing observation of the whole field, worth stating flatly: the signature check is the easy, automatable part, and deciding whether to believe the signed statement is the hard, unautomatable part. A valid signature tells you that whoever holds the issuer's private key asserted this binding. It tells you nothing about whether that assertion is true, or whether that issuer should have been trusted to make it. DigiNotar's forged certificate passed the check perfectly.

The bytes themselves are encoded with ASN.1 using the Distinguished Encoding Rules.

ASN.1 is a language for describing structured data; DER is a canonical binary encoding of it in which every value has exactly one valid byte representation. X.509 certificates are DER-encoded so that the signature covers a single, unambiguous sequence of bytes.

That canonical-encoding requirement is not pedantry. If two different byte sequences could parse to the "same" certificate, an attacker could sign one and present another, and the whole scheme unravels -- which is why certificate parsers are a recurring attack surface, a hazard Part 3 treats in depth [@itu-x690].

The format grew in three steps. Version 1 (1988) carried only the basic fields; version 2 added issuer and subject unique identifiers; version 3 (1996) added the extension mechanism -- a list of {OID, critical flag, value} entries that finally let a certificate carry its own authorization scope [@rfc5280]. That extension seam is where the "trusted to say what" question gets its answer, and where a two-decade family of enforcement bugs would later live.

Note: This series calls each installment a field guide to "a primitive," but X.509 and PKI are not a cryptographic primitive in the way a hash or a signature scheme is. They are trust infrastructure composed of primitives -- signatures, hashes, and encodings wired together into a system for delegating and checking claims. Every failure in this article lives in that wiring, never in the primitives underneath.

So a certificate is just a signed claim, verifiable offline by anyone holding the issuer's key. That idea raises the question the next forty years of engineering tried to answer: where did it come from, and why did it take so long to make it work on the open Internet?

3. Where the Idea Came From: The Trust Gap in Public-Key Cryptography

Public-key cryptography did not create trust. It created a gap. When Whitfield Diffie and Martin Hellman published "New Directions in Cryptography" in 1976, they removed the need for two parties to share a secret before they could communicate securely. But their own paper leaned on a trustworthy "Public File" -- a directory you would query to learn someone's public key [@diffie-hellman-1976]. Knowing that a key really belongs to the party you think it does was, and remains, a trust problem, not a math problem.

Querying a live directory for every correspondent was a bottleneck and a hazard: the directory was online, central, and had to be trusted in real time. Two years later, an MIT undergraduate named Loren Kohnfelder proposed the fix in his 1978 bachelor's thesis. Replace the live lookup with a signed object -- a certificate -- that anyone can verify offline by checking one signature [@kohnfelder-1978]. That single move, from an online lookup to an offline signed token, is the germ of every certificate authority that has existed since.

timeline title From the public file to the modern trust window 1976 : Diffie-Hellman assume a trustworthy public file of keys 1978 : Kohnfelder replaces the lookup with an offline certificate 1988 : X.509 v1 standardizes the format under the X.500 directory dream 1993 to 1996 : PEM single-root hierarchy fails : X.509 v3 adds extensions 1994 to 1999 : Netscape SSL brings many roots and the browser root store : PKIX profiles X.509 2011 to 2012 : DigiNotar and Flame break the many-root trust model 2013 to 2018 : Certificate Transparency pivots from prevention to detection 2024 to 2026 : short lifetimes, multi-perspective issuance, tiled logs, PQ certificates

The concept needed a standard, and in 1988 the CCITT (renamed the ITU-T in 1993) published X.509 as the authentication layer of the X.500 directory series. That parentage is the format's original sin. X.500 envisioned one global directory with one global namespace, and X.509 inherited both assumptions [@itu-x509]. The global directory was never built. The format outlived it, carrying forward a picture of the world -- one namespace, one hierarchy -- that the messy, multi-vendor Internet would spend decades constraining.

The edition years (v1 in 1988, v3 in 1996) are corroborated here through the normative references of RFC 5280 rather than read from the ITU text directly -- and it is RFC 5280 that carries the profile browsers actually implement [@rfc5280].

The 1990s taught two hard lessons about how many authorities the Web should trust. The first attempt, Privacy-Enhanced Mail, specified a single global certification hierarchy: one root, the Internet Policy Registration Authority, atop everything [@rfc1422]. It was undeployable. No one could agree on who should hold the one root key, and the model died. The opposite approach won by shipping. When Netscape put X.509 certificates under its new Secure Sockets Layer protocol, it bundled many independent root certificates into the browser itself, and the commercial certificate authority industry -- and the browser root store -- was born [@rfc6101].Taher Elgamal, Netscape's chief scientist, is often called the "father of SSL," and he did lead the effort. But SSL 3.0, the version that survived, was substantially redesigned with Paul Kocher and Phil Karlton, and SSL 1.0 never shipped publicly [@rfc6101]. Treat the epithet as an honorific, not a claim of sole authorship.

That left a standards problem: raw X.509 was too permissive for the open Web. The IETF's PKIX working group produced a constrained Internet profile, first as RFC 2459 in 1999 and then as the RFC 5280 that still governs Web certificates today [@rfc2459]. PKIX also did something subtle the next section depends on: it separated the messy job of finding a chain of certificates from the precise job of checking one.

Other trust models were proposed and lost. PGP's web of trust asked users to vouch for each other's keys directly, but it foundered on usability [@whitten-tygar-1999, @wikipedia-web-of-trust]. SPKI/SDSI tried authorization without global names and never saw adoption [@rfc2693]. DANE later offered to anchor certificate trust in DNSSEC instead, a parallel path browsers never shipped [@rfc6698].

By the 2000s the machinery worked at Internet scale -- hundreds of roots, automatic chaining, a padlock in every browser. And then, one break at a time, the Web discovered that a certificate whose signature verifies is not the same as a certificate you should trust. To see why every one of those breaks is the same kind of break, you need a single lens.

4. The Three Questions of Delegated Trust

Here is the entire article in one sentence, and the tool you will apply to every mechanism below and to tomorrow's certificate CVE: whom am I trusting, to say what, and how would I know if they lied?

Those clauses are not rhetorical. Each names a question that any public-key infrastructure has to answer, each maps to a specific mechanism in X.509, and each has its own catalog of real-world breaks. The sentence names three questions outright, and answering the last one honestly forces a fourth into the open.

flowchart TD Q["Whom am I trusting, to say what, and how would I know if they lied?"] Q --> B["1. Binding: is this key really that name?"] Q --> A["2. Authorization: trusted to say what?"] Q --> W["3. Withdrawal: how do I take the trust back?"] Q --> D["4. Detection: how would I even know they lied?"]

Binding is the "whom am I trusting" question: is this key really that name? It lives in issuance, in chain building, and in path validation, and it fails when a certificate is forged, collided, guessable, or signed by an authority that should not have signed it. Authorization is the "to say what" question: even granting that this key is that name, was that name trusted to make this particular claim? It lives in the version-3 extensions and fails when a relying party skips or mis-parses them.

Withdrawal is the question everyone assumes is solved: once you know a certificate is bad, how do you stop trusting it? It lives in revocation, and it has almost never worked. Detection, the fourth question, is the one the field arrived at last and reluctantly: if you cannot prevent a trusted authority from lying, how would you even find out that it did?

Question	What it asks	The X.509 mechanism	Named breaks
Binding	Is this key really that name?	Issuance, path building, path validation	DigiNotar, Flame, Debian, CurveBall
Authorization	Trusted to say what?	Basic Constraints, Key Usage, EKU, Name Constraints	MS02-050, CVE-2021-3450, Spooky SSL
Withdrawal	How do I stop trusting them?	CRL, OCSP, stapling, Must-Staple	soft-fail, Heartbleed CRL costs
Detection	How would I know if they lied?	Certificate Transparency logs and SCTs	the answer, not a break

Key idea: Every X.509 and PKI failure lands in one of these four questions, and never in the signature or hash math underneath. When you meet a new certificate incident, do not ask "which algorithm broke?" Ask: whom did someone trust, to say what, could anyone take it back, and would anyone have known? The answer is always in the delegation, not the arithmetic.

Start with the first question -- is this key really that name? -- because before a relying party can enforce anything at all, it has to turn a pile of certificates into a single chain it is willing to trust. That turns out to be two very different jobs, and the industry keeps confusing them for one.

5. Binding, Part One: Chain Building Is Not Path Validation

A browser is handed a leaf certificate and a bag of intermediates. Getting from there to "trusted" is not one algorithm but two, and treating them as one is a source of bugs.

A trust anchor is a certificate authority public key that a relying party trusts a priori -- there is no certificate above it to check. The root store is the curated set of trust anchors that a browser or operating-system root program ships and maintains.

The first job is discovery. Because certificate authorities cross-certify each other, reissue intermediates, and publish "Authority Information Access" pointers to their issuers, the certificates in the wild form a graph with multiple edges and even cycles. Finding a route from your leaf to some trust anchor is a search over that graph.

The graph search that discovers a candidate chain from a leaf certificate up to some trust anchor, navigating cross-certificates, reissued intermediates, and Authority Information Access pointers. On an adversarial cross-certified graph it is worst-case exponential, which is why RFC 4158 is essentially a catalog of pruning heuristics.

Path building is a depth-first search with loop detection [@rfc4158]. Its output is not a verdict -- it is a candidate: one chain that plausibly reaches a root. The second job is to decide whether that specific chain is any good.

The linear state machine that checks one chosen chain: it verifies each signature, checks each validity window, and enforces the authorization extensions (Basic Constraints, Key Usage, Extended Key Usage, Name Constraints, and policy) down the path. A critical extension the verifier does not understand must cause rejection.

Validation is a single pass, so its cost is $O(n)$ in the chain length $n$: for each certificate, check the signature against the next issuer up, confirm the dates, and update the running authorization state from the extensions [@rfc5280]. The rule that trips up naive implementations is the last one in the definition: if a certificate marks an extension critical and the verifier does not recognize it, the verifier must reject rather than ignore it. Silence is not consent.

flowchart TD Leaf["Leaf certificate plus a bag of intermediates"] Leaf --> Build["Path BUILDING (RFC 4158): graph search over cross-certs and AIA pointers to find one chain to a trust anchor"] Build --> Chain["One candidate chain: leaf to intermediate to root"] Chain --> Val["Path VALIDATION (RFC 5280 Section 6): linear state machine over the chosen chain"] Val --> C1["verify each signature and validity window"] Val --> C2["enforce Basic Constraints, Key Usage, EKU"] Val --> C3["intersect Name Constraints down the path"] Val --> Verdict["well-formed and authorized? accept or reject"]

Keeping the two apart matters because they fail differently. Building can blow up on a pathological cross-certified graph -- worst-case exponential search -- but a slow build is a denial-of-service risk, not a trust breach. Validation is where trust is decided, and its limits are the subject of the rest of this article.

Here is the first and deepest of them, the observation that seeds every failure to come: path validation can prove a chain is well-formed; it can never prove the issuing authority was honest. Everything RFC 5280 Section 6 checks is a property of the certificates. None of it is a property of the decision to sign them.

To make the difference between "verifies" and "authorizes" concrete, here is a certificate as a decoded object, with a few lines that print what it actually permits. Notice that nothing here checks a signature -- it reads the claims.

{` // A decoded X.509 certificate as a plain object. No crypto library needed: // verifying the signature is easy. Reading what it CLAIMS is the point. var cert = { subject: "CN=example.com", issuer: "CN=Example Root CA", notBefore: "2026-01-01", notAfter: "2026-01-07", basicConstraints: { cA: false }, extendedKeyUsage: ["serverAuth"], subjectAltName: ["example.com", "www.example.com"] };

describeAuthorization(cert); `}

The binding model got to this point in six recognizable steps, each answering a limitation of the one before:

Approach	Year	Key idea	Strength	Why it was superseded or limited
Online public-key directory	1976	Live name-to-key lookup	Distributes public keys at all	Online single point of failure; no offline authenticity
X.509 v1/v2 certificate	1988	Signed offline name-to-key token	Portable, verifiable offline	No authorization scope; X.500 global namespace
X.509 v3 certificate	1996	Extensions carry authorization scope	Still the live format	Only as good as the verifier's enforcement
Path building	2005	Graph search from leaf to a root	Finds a path amid cross-certs and AIA	Ambiguity is a bug class; worst-case exponential
Path validation	2008 (RFC 5280; algorithm since RFC 2459, 1999)	Section 6 state machine over one chain	Deterministic, fully specified checks	Cannot detect a dishonest CA
Web PKI (many roots)	1994	Any-root chaining, browser root store	Deployed at Internet scale	Trust equals the trust of the weakest CA

Validation answers "is this chain well-formed?" But well-formed for what? A chain can be flawless and still end in a leaf certificate quietly claiming the right to sign for the entire Internet. That is the second question -- trusted to say what? -- and it has its own two-decade catalog of enforcement bugs.

6. Binding, Part Two: Trusted to Say What?

Version 3's extensions finally let a certificate carry its own limits. Four of them do the heavy lifting for authorization, and each is a field that path validation is supposed to enforce as it walks the chain [@rfc5280].

The extension that says whether a certificate's key may act as a certificate authority (`cA` = TRUE or FALSE) and, if so, how many sub-CAs may appear below it (`pathLenConstraint`). A leaf certificate has `cA` = FALSE and must not be able to sign other certificates. The machine-readable list of purposes a key is permitted to serve -- TLS server authentication, TLS client authentication, code signing, email protection, and so on. A certificate valid for `serverAuth` is not thereby valid for `codeSigning`. The extension that limits a sub-CA to issuing only within permitted name subtrees (and outside excluded ones). During validation the permitted and excluded sets are intersected down the path, so a constrained sub-CA cannot widen its own authority.

Key Usage is the fourth: a lower-level bitfield (digital signature, key encipherment, certificate signing) that further narrows what the key may do. Together these fields answer the "trusted to say what" question in a form a machine can check. And here is the catch, the sentence that generates a two-decade bug family: a declarative authorization field is worthless if the relying party's code skips, misorders, or mis-parses the check. The certificate can be flawless. If the verifier does not enforce the limit, the limit does not exist.

The archetype is MS02-050, disclosed in 2002. Microsoft's CryptoAPI simply did not check the cA flag in Basic Constraints, so any valid leaf certificate could sign certificates for other names, and every Microsoft application that used CryptoAPI would accept them [@cve-2002-0862]. Moxie Marlinspike weaponized it in a tool called sslsniff: obtain an ordinary certificate for a domain you control, then use it to mint a certificate for any other domain, and Internet Explorer would trust it [@sslsniff-moxie]. The certificate authority had done nothing wrong. The verifier had.

You might expect that lesson to stay learned. It did not. In March 2021, OpenSSL CVE-2021-3450 landed: a regression in strict mode meant the library would accept a non-CA certificate as a CA -- the exact same bug class, nineteen years later, in the most widely used TLS library on Earth [@openssl-secadv-20210325].

The failure family is structural, not incidental. Enforcement code is written by humans, and the check that a certificate is not allowed to do something is precisely the check that is easy to forget. Apple's 2014 "goto fail" bug made the same point in a single line -- a duplicated goto statement that skipped the signature check altogether [@cve-2014-1266].CurveBall (CVE-2020-0601, 2020) is a cousin worth knowing: Windows CryptoAPI accepted attacker-supplied explicit elliptic-curve parameters, so a signature could verify against an attacker-chosen curve rather than the standard one. The signature was "valid," but the validity was meaningless -- a binding-adjacent verification bug in the same spirit as skipping Basic Constraints [@cve-2020-0601].

The machinery meant to enforce these limits can itself become the attack surface. In 2022, "Spooky SSL" (CVE-2022-3602 and CVE-2022-3786) was a stack buffer overflow reached during Name-Constraint checking of punycode-encoded email addresses in OpenSSL 3.0 [@cve-2022-3602]. The code enforcing an authorization limit had a memory-safety bug in its parser -- exactly the serialization hazard Part 3 warns about, now living inside the authorization check itself [@openssl-secadv-20221101].

To feel how thin the line is, here are two validators over the same forged chain. They differ by one if statement.

{` // A leaf certificate (cA:false) is trying to sign a certificate for a bank. var attackerLeaf = { subject: "CN=mallory.example", basicConstraints: { cA: false } }; var forged = { subject: "CN=login.yourbank.com", issuedBy: attackerLeaf };

function accepts(issuer, opts) { var signatureMatches = true; // assume the crypto checks out -- it usually does if (opts.checkBasicConstraints && issuer.basicConstraints.cA !== true) { return false; // issuer is not a CA, so reject. This is the whole defense. } return signatureMatches; }

console.log("Strict validator (checks cA): " + (accepts(forged.issuedBy, { checkBasicConstraints: true }) ? "ACCEPT" : "REJECT")); console.log("Lax validator (skips cA): " + (accepts(forged.issuedBy, { checkBasicConstraints: false }) ? "ACCEPT" : "REJECT")); console.log("The lax path just let a leaf sign for " + forged.subject + "."); `}

Note: Use a maintained validation library and turn on strict path validation. Do not skip Basic Constraints, Key Usage, EKU, or Name Constraints, and never treat a critical extension the verifier does not understand as benign -- RFC 5280 requires rejection. Most of the failure catalog in this article is not broken cryptography; it is a verifier that let something through.

The enforcement that fails so often in public code is a gift when you run it deliberately. If you operate an internal certificate authority, issue its root with Name Constraints so it can *only* sign for your own namespace -- `*.corp.example.com` and nothing else. Even if that root key leaks, it cannot mint a certificate for `google.com`, because conforming validators intersect the constraint down every path. You trade public trust for tight, auditable scope, which is exactly the trade Active Directory Certificate Services operators wrestle with in [this companion post](/blog/certified-pre-owned-ad-cs-and-active-directorys-second-trust).

Every one of these breaks had a fix and a lesson, and the Web learned them. But they share a comforting property: the certificate authority was honest, and the code was wrong. What happens when the code is right, the signature verifies, and the authority itself is the adversary?

7. The Binding Failure Catalog: When the Authority Vouched for the Wrong Key

There are three ways a certificate can be perfectly valid and perfectly compromised, and none of them is a bug in the mathematics. Each is a failure of binding -- the "is this key really that name?" question -- and each taught the Web something it did not want to learn.

DigiNotar (2011) is the pure case. Attackers who had breached the Dutch certificate authority used its fully trusted key to issue a valid *.google.com certificate, along with more than 500 others, and someone used the Google certificate to intercept the traffic of roughly 300,000 users in Iran [@foxit-diginotar]. Every signature verified. Path validation flagged nothing, because a trusted authority had chosen to sign. The forensic report, aptly titled "Black Tulip," traced the intrusion; the company, owned by VASCO, was distrusted by every major browser and declared bankruptcy in September 2011 [@wikipedia-diginotar].

Key idea: DigiNotar has no cryptographic tell. There was nothing in the certificate, the chain, or the signature for path validation to catch, because the failure was not in any of them -- it was in the decision to sign. Prevention failed at the root of trust itself. When a trusted authority turns adversary, the only defense left is detection: making the lie visible after the fact. Hold that thought; it is the reason Certificate Transparency exists.

Flame (2012) made the same point on a different trust path. The Flame espionage platform carried a certificate that chained to Microsoft and appeared valid for code signing, letting its components masquerade as legitimate Windows Update content. The certificate was forged using a novel chosen-prefix MD5 collision -- a new variant, distinct from the 2008 technique -- against a corner of Microsoft's Terminal Server Licensing infrastructure still signing with MD5 [@flame-counter-crypto-2013].

Marc Stevens' "counter-cryptanalysis" later reconstructed it forensically. A hash-function weakness, reached through a forgotten sub-CA, subverted code-signing trust at nation-state scale; the trust path Flame abused is the subject of this companion post.A chosen-prefix collision takes two attacker-chosen prefixes and appends carefully computed bytes so their MD5 digests come out identical. The attacker gets a benign-looking certificate signed and transplants the signature onto a malicious one with the same digest. The underlying "valid signature, forged message" hazard is the theme of Part 3.

Debian OpenSSL (2008) is the case that proves the binding failure need not involve the authority at all. A well-meaning Debian patch removed code the maintainer thought was uninitialized-memory noise, and in doing so gutted the entropy of OpenSSL's random number generator. Keys generated on affected systems came from a space of only about 32,767 possibilities [@cve-2008-0166]. The X.509 structure was flawless; the private key underneath was one you could guess.With the entropy source reduced to essentially the process ID, affected keys had roughly 15 bits of randomness -- around 32,767 candidates for a given key type and size, enumerable in an afternoon. This is a randomness failure, catalogued under weak keys in Part 2, not a defect in certificate logic. The certificate did its job perfectly; the job was worthless.

Flame did not appear from nowhere. Its research ancestor is the 2008 rogue CA demonstration, in which Marc Stevens and six co-authors used a chosen-prefix MD5 collision -- on the order of 2^49 MD5 compression-function evaluations -- to obtain a working intermediate CA certificate from a RapidSSL branch that still signed with MD5 [@rogue-ca-md5-2009]. That was a controlled proof of concept. Flame was the same idea, refined, in the wild.

Incident	Year	Root cause	Question it breaks	The lesson
DigiNotar	2011	Compromised trusted CA issued valid rogue certificates	Binding (whom)	You cannot prevent a trusted CA from lying -- only detect it
Flame	2012	Novel MD5 chosen-prefix collision forged a code-signing cert	Binding (whom)	A hash break subverts code signing; retire weak hashes in CA signing
Debian OpenSSL	2008	PRNG entropy gutted, leaving ~15-bit guessable keys	Binding (weak key)	A perfect certificate over a guessable key is worthless
2008 rogue CA (RapidSSL)	2008	Chosen-prefix MD5 collision, about `2^49` compressions	Binding (whom)	MD5 in CA signing is fatal; the research ancestor of Flame

DigiNotar was the sharpest instance of a recurring pattern, not a one-off. Across the 2010s, other trusted authorities were caught mis-issuing certificates or were distrusted for related failures -- names like Comodo [@comodo-fraud-2011], TURKTRUST [@turktrust-2013], Symantec [@symantec-distrust-2017], and WoSign [@wosign-distrust-2016] recur in any serious history of the Web PKI. The details differ; the shape does not. Each time, a certificate that path validation would happily accept turned out to rest on a delegation that should never have been made. And the modern analog is a *stolen signing key* rather than a rogue authority, as in the [Storm-0558 incident](/blog/forged-from-2016-how-storm-0558-turned-one-stolen-signing-ke), where one compromised key forged tokens at cloud scale.

DigiNotar proved you cannot prevent a trusted authority from lying. That leaves two questions the field is still wrestling with: once a certificate is known to be bad, how do you take the trust back -- and if you cannot, how would you even find out? Take withdrawal first, because it is the one everyone assumes is solved and almost nobody has working.

8. Withdrawal: How Do I Stop Trusting Them, and Why That Is So Hard

Ask an engineer how certificate revocation works and you will get a clear description of a mechanism. Ask whether it works and the honest answer is: for most of the Web, for most of its history, it did not. Revocation is where the "how do I take the trust back?" question goes to fail, and it fails in a genealogy of increasingly clever attempts.

Any connection-time revocation mechanism wants three things at once: *freshness* (the answer reflects reality now), *privacy* (checking does not tell anyone what you are browsing), and *scalability* (it works at Web scale without huge cost). No single online mechanism achieved all three. CRLs sacrificed freshness and size; OCSP sacrificed privacy; stapling sacrificed enforceability. The field eventually stopped trying to satisfy the trilemma and started dissolving the need for it.

The original mechanism is the Certificate Revocation List, a periodically published, CA-signed list of revoked serial numbers, in X.509 since 1988 and profiled by RFC 5280 Section 5 [@rfc5280]. The problem is size. A CRL grows with the number of revocations, routinely reaching megabytes, and downloading a multi-megabyte list to visit one site is untenable, so browsers cached them (reintroducing staleness) or stopped fetching them entirely [@mozilla-crlite-2020].

The cost became concrete after Heartbleed triggered mass revocation in 2014. Serving its CA partner GlobalSign's suddenly-4.7 MB CRL, Cloudflare estimated, would have added about $400,000 a month in bandwidth -- roughly $953,000 at AWS pricing -- a load Cloudflare largely absorbed rather than a bill it paid [@cloudflare-heartbleed]. Online revocation was still being reaffirmed as ineffective years afterward [@langley-revocationagain-2014].

The fix seemed obvious: instead of downloading the whole list, ask about the one certificate you care about. The Online Certificate Status Protocol does exactly that -- a per-certificate query to a CA responder that returns a signed good, revoked, or unknown [@rfc6960]. It has three fatal properties.

First, soft-fail: because the client cannot always reach the responder -- captive portals, responder downtime, or an attacker who simply blocks the request -- a failed check is ignored. That single design choice makes it useless against the attacker it targets, because a man-in-the-middle who can intercept your HTTPS can also block your OCSP query [@langley-crlsets-2012, @langley-revocation-2011].

"So soft-fail revocation checks are like a seat-belt that snaps when you crash. Even though it works 99% of the time, it's worthless because it only works when you don't need it." -- Adam Langley

Second, OCSP is a privacy leak: every check tells the certificate authority that a particular user, at a particular IP, is visiting a particular site, in real time -- reviving the online-directory hazard that certificates were invented to escape [@langley-crlsets-2012]. Third, it adds latency: a median around 300 milliseconds on every first connection [@langley-crlsets-2012], with more than 7% of checks timing out entirely [@mozilla-crlite-2020], which is why Chrome disabled online revocation checking by default.Soft-fail is not a careless default; it is a forced one. If you make OCSP hard-fail instead -- block the connection whenever the check does not complete -- then any responder outage becomes an outage for every site that depends on it. So you either honor a blocked check (worthless against a MITM) or you refuse it (a self-inflicted denial of service). There is no third option at connection time.

Two fixes branched from OCSP. OCSP stapling moves the query off the client: the server fetches a fresh signed response and staples it into the TLS handshake, so the CA never sees the visitor and there is no per-connection round-trip [@rfc6066]. But stapling is optional, so it inherits soft-fail -- an attacker just presents a certificate with no staple, and the client falls back to accepting it.

OCSP Must-Staple tried to close that gap with a flag in the certificate demanding a valid staple, turning a missing staple into a hard failure [@rfc7633]. Hard-fail, though, means any hiccup in obtaining a staple takes the site down, so adoption stayed near zero and the feature became a footgun.

flowchart TD CRL["CRL 1988: periodic signed lists"] OCSP["OCSP 1999: per-cert online query"] STAP["OCSP stapling 2003: server fetches the response"] MUST["Must-Staple 2015: certificate demands a staple"] PUSH["Pushed and dissolved 2012 to 2026: CRLite plus short-lived certs"] CRL -->|"lists too large, went stale"| OCSP OCSP -->|"soft-fail, privacy leak, latency"| STAP STAP -->|"optional, so still soft-fails"| MUST MUST -->|"hard-fail self-DoS, near-zero adoption"| PUSH

Then the whole branch was retired. Let's Encrypt stopped issuing Must-Staple certificates on 30 January 2025, dropped OCSP URLs from its certificates around 7 May 2025, and turned its OCSP responders off on 6 August 2025 [@le-ending-ocsp-2024] -- responders that at peak had served roughly 340 billion requests per month [@le-ocsp-eol-2025]. The CA/Browser Forum made OCSP optional [@cabforum-br-documents]. The stated rationale was user privacy and reliability, not mere cleanup [@abetterinternet-ending-ocsp]. This is not a mechanism you tune; it is one the industry decommissioned.

Mechanism	Client cost	Freshness	Privacy	Fail mode versus MITM	Status in 2026
CRL	Download the whole list (MBs)	Publication interval	Good (offline)	Stale or skipped	Fallback substrate
OCSP	About 300 ms per check	Real-time	Leaks browsing to the CA	Soft-fail is worthless	Off (2025-08-06)
Stapling / Must-Staple	Zero client round-trips	Staple window	Good	Missing staple soft-fails, or hard-fail self-DoS	Legacy, effectively dead
CRLite	O(1) local lookup	Push interval (4x/day)	Good (offline)	Cannot be suppressed	Active (Firefox)
Short-lived certs	Zero (no revocation data)	Certificate expires in ~6 days	Good	Nothing to check	Active (GA Jan 2026)

Note: OCSP, OCSP stapling, and OCSP Must-Staple are legacy. The responders are off, Must-Staple is a self-inflicted denial-of-service risk, and the CA/Browser Forum has made OCSP optional. Design revocation around CRLite and CRLs as the offline fallback, plus short-lived certificates -- never around OCSP.

Every mechanism that tried to let you withdraw trust at connection time failed on soft-fail, privacy, latency, or brittleness. So the Web made a different move -- one that does not fix revocation at all. But before we get there, it had to answer DigiNotar's unanswered question: if you cannot prevent a rogue certificate, how would you even know one exists?

9. Detection: How Would I Know If They Lied? Certificate Transparency

The most important idea in modern PKI is an admission of defeat. You cannot stop a trusted authority from lying, DigiNotar proved that, so stop trying to prevent it and make lying impossible to hide instead.

A system of public, append-only Merkle-tree logs that record every certificate a trusted authority issues. Browsers require proof that a certificate was logged before they will trust it, so mis-issuance leaves a permanent, public record that anyone can monitor.

Certificate Transparency, introduced by Ben Laurie, Adam Langley, and Emilia Kasper of Google in RFC 6962 in 2013, works like this: before a certificate can be trusted, it must be submitted to public logs, and each log returns a small signed receipt [@rfc6962].

A log's signed promise to include a certificate in its append-only tree within a bounded time. Browsers require one or more SCTs, from independent logs, as a precondition for trusting a certificate.

A CT log is an append-only Merkle tree, and its power is that two things about it are cheaply provable. An inclusion proof (or audit proof) shows that a specific certificate is in the tree, using only about $\log_2 n$ hashes for a log of $n$ entries. A consistency proof shows that an earlier version of the tree is a prefix of a later one -- that the log only appended and never rewrote history -- also in $O(\log n)$ hashes [@rfc6962]. Those logarithmic proofs are what make a log at Internet scale auditable by a small client [@rfc9162].

sequenceDiagram participant CA as Certificate Authority participant Log as CT Log participant Server as TLS Server participant Browser as Browser participant Monitor as Monitor CA->>Log: submit newly issued certificate Log-->>CA: return a Signed Certificate Timestamp CA-->>Server: certificate with embedded SCT Server->>Browser: TLS handshake with certificate and SCT Browser->>Browser: require a valid SCT before trusting Monitor->>Log: read the append-only log Monitor-->>Monitor: alert the domain owner on mis-issuance

The SCT reaches the browser by one of three paths: embedded in the certificate itself, delivered in a TLS handshake extension, or -- historically -- stapled into an OCSP response.The OCSP-stapled delivery path for SCTs is dying along with OCSP itself, which leaves the embedded and TLS-extension paths as the durable ones. The most common in practice is embedding, because it requires no server configuration. Whichever path carries it, the browser refuses to trust a publicly-trusted certificate that cannot prove it was logged. And because the logs are public, independent monitors read them continuously, so a domain owner can discover a mis-issued certificate for their own name, often within minutes, even though nothing stopped it from being issued [@wikipedia-ct].

Here is the mechanism under an SCT: recomputing a Merkle root from a leaf and a short audit path, the check a browser or auditor performs.

{` // Recompute a Merkle root from one leaf plus an audit path of sibling hashes. // Toy hash shows the SHAPE; a real CT log uses SHA-256 over the same structure. function h(x) { return "H(" + x + ")"; }

var leaf = h("cert:example.com"); // the certificate we want to prove var auditPath = [ // one sibling hash per tree level { side: "right", hash: "H(cert:sibling)" }, { side: "right", hash: "H(subtree-2)" }, { side: "left", hash: "H(subtree-3)" } ];

function recomputeRoot(node, path) { for (var i = 0; i < path.length; i++) { var s = path[i]; node = (s.side === "left") ? h(s.hash + "|" + node) : h(node + "|" + s.hash); } return node; }

var treeSize = 8; // eight certificates in this toy log console.log("Log holds " + treeSize + " certificates."); console.log("Proof length: " + auditPath.length + " hashes, about log2 of " + treeSize + "."); console.log("Recomputed root: " + recomputeRoot(leaf, auditPath)); console.log("If it equals the log's signed root, inclusion is proven, in O(log n)."); `}

CT became mandatory for public trust when Chrome began requiring SCTs for certificates issued after 30 April 2018 [@chromium-ct-policy], and Apple followed for Safari on 15 October 2018 [@apple-ct-requirements]. Browsers accept SCTs only from logs admitted to their transparency programs [@google-ct-requirements]. The design goal was stated plainly in the RFC:

Publicly logging certificates "allows anyone to audit certificate authority activity and notice the issuance of suspect certificates as well as to audit the certificate logs themselves." -- RFC 6962

But read the verbs carefully. Audit. Notice. Not prevent.

Note: Certificate Transparency does not stop a rogue certificate from being issued or trusted. It makes the certificate leave a public, permanent record. A logged rogue certificate is still valid and trusted until a human notices it and gets it distrusted. CT shrinks the undetected window, not the exploitable one -- it is the honest ceiling of what an open, many-root system can do.

CT closes the loop DigiNotar opened, but only as far as detection. A detected rogue certificate is still a valid, trusted certificate until it is revoked, and we have just seen that revocation is broken. That unresolved tension -- detected but still valid, and hard to take back -- is exactly what the 2024-2026 frontier attacks, and it does so with one surprisingly simple move.

10. State of the Art, 2024 to 2026

The whole modern frontier is one strategic move wearing five hats: shrink the trust window until withdrawal barely matters, corroborate issuance from many vantage points, keep every act of delegation public, and make the whole apparatus quantum-durable. Here are the layers, each sitting on the settled PKIX substrate.

Short-lived certificates are the dissolving move. Let's Encrypt's roughly six-day profile, which reached general availability in January 2026, issues certificates that carry no OCSP or CRL URLs at all -- a certificate that mis-issues simply expires before revocation would have mattered [@le-6day-2025]. Revocation is not fixed; it is designed out. The price is a renewal cadence that only automation can sustain.

The 47-day validity schedule turns that from a boutique option into Web-wide policy. CA/Browser Forum Ballot SC-081v3, proposed by Apple (endorsed by Sectigo, Google, and Mozilla) and approved on 11 April 2025, steps the maximum TLS certificate validity down in stages: 398 days now, 200 days from 15 March 2026, 100 days from 15 March 2027, and 47 days from 15 March 2029, with domain-validation data reuse falling to just 10 days [@cabforum-sc081-47day].

All four major browser vendors voted yes, and the schedule binds publicly trusted authorities through the CA/Browser Forum Baseline Requirements [@cabforum-br-documents]. Its real function is a forcing function: certificates this short cannot be managed by hand, so the schedule mandates ACME-style automation across the entire Web [@digicert-47day].

CRLite handles the certificates that still live long enough to need revoking. It ingests every certificate from CT plus Internet scans, sorts them into revoked and valid, and encodes the answer as a cascade of Bloom filters, compressing roughly 300 MB of revocation data into about 1 MB that is refreshed four times a day and checked locally, offline, with zero false negatives [@mozilla-crlite-2020, @crlite-pdf]. Lookups leak nothing and cannot be suppressed by a man-in-the-middle, which is the property OCSP never had [@crlite-ieee-2017]. After OCSP's retirement, plain CRLs are again the universal fallback beneath it.

Multi-Perspective Issuance Corroboration (MPIC) hardens the binding step at issuance time. Under CA/Browser Forum Ballot SC-067v3, a certificate authority must perform domain-control and CAA checks from several network vantage points, and the remote perspectives must corroborate the primary result or issuance is blocked -- which defeats localized BGP hijacks that could fool validation from a single location [@cabforum-sc067-mpic].MPIC revives the "check from many vantage points" idea of the Perspectives and Convergence notary projects from around 2008 to 2011, which failed as user-facing products. The revival works because it runs inside the certificate authority at issuance time, not on the user's machine at connection time.

The requirement was adopted in 2024; enforcement began with two corroborating perspectives on 15 September 2025, ramping to three by 15 March 2026, four by 15 June 2026, and five by 15 December 2026, with perspectives required to sit at least 500 km apart [@sectigo-mpic-faq].

Static, tile-based CT makes universal logging affordable. The Sunlight design, built by Let's Encrypt and Filippo Valsorda with the interoperable Static CT API, publishes the Merkle tree as immutable 256-element tiles served as plain static files from object storage, with no per-request database. The scale motivation is concrete: Let's Encrypt issues over four million certificates per day, and its Oak log passed 700 million entries, sizes that buckled the older relational design [@le-sunlight-2024]. Tiling turns every proof read into a cache hit [@static-ct-api].

Merkle Tree Certificates and post-quantum certificates are the frontier. NIST's ML-DSA (FIPS 204) and SLH-DSA (FIPS 205) are the standardized post-quantum signatures a certificate will eventually use [@nist-fips-204], and they hit a wall made of bytes.

At the AES-128 security level, post-quantum signatures dwarf today's Ed25519 (32-byte key, 64-byte signature). ML-DSA-44 is 1,312 bytes of key and 2,420 of signature; SLH-DSA-128s is a 7,856-byte signature [@cloudflare-pq-2024]. A single TLS handshake carries several signatures -- the leaf, the CA chain, the SCTs -- so post-quantum multiplies the on-the-wire cost and can burst a handshake past what ossified middleboxes tolerate. Merkle Tree Certificates dodge this by batching many certificates under *one* post-quantum signature over a Merkle root and shipping each subscriber a small inclusion proof instead of a full per-certificate signature [@mtc-draft]. It is Certificate Transparency's logarithmic proof structure, reused as the delivery format. For the migration itself, see the post-quantum posts [here](/blog/post-quantum-cryptography-on-windows-the-thirty-year-migrati) and [here](/blog/the-thirty-year-migration-ships-in-a-pip-install-how-post-qu).

Scheme	Post-quantum	Public key (bytes)	Signature (bytes)	Note
Ed25519	no	32	64	today's default, the bar to beat
RSA-2048	no	256	256	still common
ML-DSA-44	yes	1,312	2,420	the default post-quantum pick (FIPS 204)
FN-DSA-512	yes	897	666	smallest, but floating-point-dangerous to sign safely
SLH-DSA-128s	yes	32	7,856	most conservative security, worst size and speed (FIPS 205)

Post-quantum key agreement is already live -- around 1.8% of Cloudflare's TLS 1.3 connections in early 2024 -- but post-quantum certificates are not, and no certificate authority was expected to issue one before 2026 [@cloudflare-pq-2024]. Put the layers side by side and the strategy is legible at a glance.

Method	What it does	What it buys	Status in 2026
Short-lived certificates	Certificates expire in days	Makes revocation barely matter	GA January 2026 (Let's Encrypt)
47-day validity schedule	Steps max validity toward 47 days by 2029	Forces automation Web-wide	Approved April 2025 (CA/B Forum)
CRLite	Pushes all revocations, compressed	Fresh, private, offline withdrawal	Active (Firefox)
MPIC	Corroborates issuance from many vantage points	Defeats BGP-hijack mis-issuance	Enforcing since 2025-09-15
Tiled / Static CT	Serves the Merkle tree as static tiles	Cheap, scalable transparency	Active (Chrome accepts)
Merkle Tree Certificates	Batches certs under one PQ signature	Dodges the post-quantum size wall	Draft (IETF PLANTS WG)

Key idea: The 2024-2026 frontier is a single move seen from five angles: shrink the trust window so withdrawal barely matters, corroborate issuance so mis-binding is harder, keep every delegation public so lies are visible, and make it all quantum-durable. Notice what is absent from that list -- a better cipher. The math was never the variable. Delegation always was.

Note: Automate issuance with ACME. Adopt short-lived or at most 47-day certificates. Rely on Certificate Transparency plus CRLite and CRLs for withdrawal. Monitor CT for your own domains. Publish CAA records. Do not configure OCSP or Must-Staple. Deploy post-quantum key agreement now and wait on post-quantum certificates.

This is the closest reachable approximation to a perfect trust layer. "Closest reachable" carries weight, because some of what stands between here and perfect is not an engineering backlog. It is a wall.

11. The Walls: Theoretical Limits and Open Problems

Some of this you cannot out-engineer. Underneath the 2024-2026 machinery are four structural limits, and no ballot or algorithm removes them.

The first is the deepest. You cannot prevent a rogue trusted authority; you can only detect one. In a system with hundreds of equally powerful roots, the trust of the whole equals the trust of its weakest member, and nothing in path validation flags a certificate that a trusted authority chose to sign. Certificate Transparency is the field's formal admission of this ceiling: it does not optimize prevention, because prevention is unavailable under an open many-root model. It optimizes detection latency [@rfc6962].

That is the reader's final turn: after CT and CRLite and short-lived certs, it is tempting to conclude the problem is solved. It is not. The exploit window is shrunk, not closed, and the reason it can never reach zero is structural, not a missing feature.

The second is that the soft-fail/hard-fail fork of Section 8 is a structural wall, not an implementation gap. No connection-time status check escapes it: an attacker who can block the check defeats soft-fail, and hard-fail turns any responder outage into a self-inflicted denial of service for every dependent site. That is why the OCSP branch was retired rather than repaired [@langley-crlsets-2012].

The third is the revocation trilemma -- freshness, privacy, and scalability cannot be maximized together by any single online mechanism. This one is practically resolved by pre-distribution plus short lifetimes, but that is an escape from the trilemma, not a refutation of it; taken together, the historical record and the CRLite result suggest the trilemma is real and the field simply stopped paying its toll [@mozilla-crlite-2020].

The fourth is that post-quantum signatures have no free lunch. No standardized post-quantum scheme matches Ed25519 on size and speed and conservative security at once; the three NIST standards each win a different corner, and the gap to an ideal is large enough that NIST opened a signatures onramp to search for better ones [@cloudflare-pq-2024].An "onramp" is NIST's term for a supplementary call for new candidate schemes after the main standardization round -- an admission that the standardized set does not yet contain a signature good enough for the certificate use case.

Around those walls sit the genuinely open problems. Preventing, rather than merely detecting, mis-issuance remains unsolved for the open Web; MPIC hardens one attack path, but the DigiNotar ceiling stands. Certificate Transparency has its own unfinished business: a misbehaving log can in principle show different trees to different clients -- a split view -- unless clients gossip their observations, and a complete, deployed gossip protocol that defeats split views without leaking privacy is not finished [@rfc9162].

Fresh, private, complete, and cheap revocation at full Web scale has no consensus mechanism; CRLite is the best answer and still ships a bounded-staleness dataset [@mozilla-crlite-2020]. The post-quantum size wall is unsolved for the general handshake, which is what Merkle Tree Certificates exist to dodge [@mtc-draft]. And name-constraint completeness over adversarial, worst-case-exponential path-building graphs is still an open corner of the validation problem [@rfc4158].

Two of these deserve special mention because they are the new risks the SOTA created. The first is operational: short lifetimes convert "revocation is broken" into "your renewal automation must never fail," making the ACME pipeline a fresh single point of failure. No named large-scale post-mortem documents this yet; it is a structural consequence of the six-day and 47-day moves, flagged here rather than sourced to an incident [@le-6day-2025].

The second is not technical at all, and it is the one the whole subject circles back to. I will state it as a judgment rather than a theorem: the hardest limit in PKI is who gets to be a trust anchor and who decides when to remove one. Root programs at Mozilla, Google, Apple, and Microsoft are the de-facto governors, and there is no technical mechanism that substitutes for that human judgment. The deepest limit is not mathematical. It is organizational, and consolidation makes it more fragile, not less.

Which means the practical question is never "is PKI secure?" but "am I using it in a way that respects what it can and cannot promise?" Here is how to do exactly that in 2026.

12. Using X.509 Correctly in 2026

Everything above collapses into one decision procedure and one misuse checklist you can run in a design review. Walk the tree in order.

flowchart TD Start["Designing certificate handling in 2026"] Start --> Q1{"Am I validating certificates myself?"} Q1 -->|"yes"| R1["Use a maintained library, enable strict path validation"] Q1 -->|"no, I operate servers"| Q2{"Public or private trust?"} Q2 -->|"public"| R2["Automate with ACME, short-lived or 47-day certs, match SAN not CN"] Q2 -->|"private"| R3["Name-constrained root, enforce EKU, keep off the public path"] R2 --> Q3{"Withdrawal or detection?"} Q3 -->|"withdrawal"| R4["CRLite plus CRLs, never OCSP or Must-Staple"] Q3 -->|"detection"| R5["Monitor CT logs, publish CAA records"]

Now the rules behind the tree, with the parameters that matter.

Never hand-roll validation. Most of this article's failure catalog is verifier bugs, and the classic study of non-browser software found certificate validation "completely broken" across a wide range of applications that rolled their own [@georgiev-2012]. Use a maintained library, enable strict path validation, and do not skip Basic Constraints, Key Usage, EKU, or Name Constraints. Treat a critical extension you do not understand as fatal [@rfc5280].
Match the SAN, not the CN. Validate hostnames against subjectAltName. The legacy commonName-as-hostname path is deprecated and is the root of classic misuse -- null-prefix names and wildcard abuse [@rfc5280].
Make revocation a non-problem. Automate issuance with ACME and adopt short-lived (six-day) or 47-day-or-shorter certificates so a bad certificate expires on its own [@le-6day-2025]. For the certificates that still live long, rely on CRLite and CRLs [@mozilla-crlite-2020]. Do not deploy OCSP or Must-Staple as new work; the responders are off [@le-ocsp-eol-2025].
If you run a private authority, constrain it. Issue a name-constrained root so it can only sign your namespace, enforce EKU, and keep it out of the public trust path [@rfc5280]. This is the deliberate use of the enforcement that public code so often skips.ACME clients -- Certbot, lego, acme.sh, and Caddy's automatic HTTPS -- are the renewal substrate the 47-day schedule quietly assumes. Choosing and monitoring one is now part of certificate design, not an afterthought.
Authorize issuance with CAA. Publish DNS Certification Authority Authorization records naming which authorities may issue for your domain [@rfc8659]. Under MPIC, those records are now checked from multiple network perspectives, so they are harder to spoof [@cabforum-sc067-mpic].
Detect mis-issuance for your own names. Run or subscribe to a Certificate Transparency monitor, reading the logs through the Static CT tiles [@static-ct-api]. This is how you personally exercise the detection that CT makes possible [@chromium-ct-policy].
Plan for post-quantum without over-rotating. Deploy post-quantum key agreement (an ML-KEM hybrid) now, but wait on post-quantum certificates and track the Merkle Tree Certificate and ML-DSA rollout instead [@cloudflare-pq-2024]. There is no reason to ship kilobyte-scale certificate signatures before the format that tames them lands [@mtc-draft].
Do not pin keys. Prefer CT monitoring plus CAA over HPKP-style public-key pinning, which browsers deprecated in 2018 after it proved to be a self-inflicted denial-of-service and ransom risk [@rfc7469].

Each misuse below maps one-to-one to a named break earlier in this article and to its fix. If you recognize your own system in the left column, you already know which section to reread.

Common misuse	Maps to which named break	The fix
Skipping Basic Constraints, accepting a leaf as a CA	MS02-050, CVE-2021-3450	Enable strict path validation
Matching the CN instead of the SAN	null-prefix and wildcard abuse	Match `subjectAltName` only
Deploying OCSP or Must-Staple as new work	soft-fail, Must-Staple self-DoS	Short-lived certs plus CRLite and CRLs
Treating a public CA-signed cert as inherently "safe"	DigiNotar	Monitor CT; question the delegation
Long-lived certificates and manual renewal	Heartbleed mass-revocation cost	Automate with ACME, 47 days or shorter
Explicit ECC parameters, hand-rolled crypto	CurveBall (CVE-2020-0601)	Maintained libraries, standard curves
Pinning keys with HPKP	HPKP self-DoS and ransom pinning	CT monitoring plus CAA

Notice that every rule is the same rule wearing different clothes: know whom you are trusting, to say what, and how you would know if they lied. The folklore that gets this wrong is dense, and it causes real incidents. Here are the misconceptions worth dismantling.

13. Seven Half-Truths That Cause Real Incidents

Historically, mostly not. Online checks soft-fail, which makes them worthless against the man-in-the-middle they target, and they leak your browsing to the certificate authority while adding latency [@langley-crlsets-2012]. The 2026 answer is not better revocation but *short-lived certificates* that dissolve the need [@le-6day-2025] plus *CRLite and CRLs* that push the remaining data offline [@mozilla-crlite-2020]. Do not build new systems on OCSP -- its responders are off [@le-ocsp-eol-2025]. No. The padlock means the connection is encrypted and authenticated to *a* name that passed domain-control validation. It does not mean the operator is honest, the content safe, or the business real -- DigiNotar's forged Google certificate produced a perfectly closed padlock [@foxit-diginotar]. Whom you are trusting, and to say what, is a separate question the padlock cannot answer. No. A CA-signed certificate is only as trustworthy as the authority that signed it and the validation that checked it, and DigiNotar's rogue certificates were fully CA-signed [@wikipedia-diginotar]. For internal use, a *name-constrained* private root can be safer than a public one, because it physically cannot issue outside your namespace [@rfc5280]. "Signed by a CA" is not a synonym for "safe." No. Modern validation matches the `subjectAltName` extension, not the Common Name, and CN-as-hostname is deprecated precisely because it was the root of null-prefix and wildcard abuse [@rfc5280]. If your tooling still relies on the CN field for hostname matching, it is running a validation path the standards left behind. No -- it is a legacy footgun on top of a retired substrate. Must-Staple hard-fails, so any hiccup in obtaining a staple takes your site down [@rfc7633], and it sits on OCSP responders that Let's Encrypt turned off in August 2025 [@le-ocsp-eol-2025]. Present it historically. Design revocation around short lifetimes and CRLite instead. No -- it *detects* it. CT makes every issued certificate leave a public, permanent record so a rogue certificate can be *noticed*, but it does not stop the certificate from being issued or trusted [@rfc6962]. Detection, not prevention, is the achievable ceiling for an open, many-root system. A logged rogue certificate is valid until a human gets it distrusted. Not in the way people assume. This is my read of the evidence rather than a theorem, but Extended Validation buys little practical protection once its distinct browser indicator is gone, and extra intermediate certificates add attack surface rather than safety. Chain *length* is not chain *trustworthiness* -- what matters is whom the chain delegates to, not how many hops it takes.

Every one of these dissolves into the same sentence -- which is the conclusion's job to deliver, back where the article began.

The Trust Was the Bug

Go back to that DigiNotar certificate one last time. It was cryptographically perfect. It was trusted by every browser on Earth. And it was a total compromise -- because the question that mattered was never "does this verify?" [@foxit-diginotar]

Look at the whole catalog with that in mind. DigiNotar, Flame, the 2008 rogue CA, Debian's guessable keys, MS02-050, CurveBall, OpenSSL's strict-mode regression, Spooky SSL -- in every single case, the RSA, ECDSA, and SHA math did exactly what it promised [@flame-counter-crypto-2013]. The signatures verified. The hashes hashed. The variable was always delegation: whom a trusted party vouched for [@cve-2008-0166], what it was allowed to say [@cve-2002-0862], and whether anyone could take the trust back. Not once was the primitive the weak link.

And read the 2024-2026 answer the same way, because it is one move, not five. The Web could not make certificate authorities honest, so it made them public with Certificate Transparency [@rfc6962], corroborated with multi-perspective issuance [@cabforum-sc067-mpic], and short-lived with the 47-day schedule [@cabforum-sc081-47day] -- shrinking the trust window until a lie expires before it can spread. The engineering did not find a better cipher. It changed what it demanded of delegation.

That is the same payoff the rest of this series keeps arriving at from different directions. The RSA math was always fine, as Part 1 argued about adversary models. The primitive was never the variable, as Part 3 showed about serialization. Here it is again, in the machinery of trust itself: the cryptography was innocent, and the delegation was guilty.

So take the one tool this article was built to give you. When the next certificate incident crosses your feed -- and there will be a next one -- do not reach for the algorithm. Reach for the sentence: whom am I trusting, to say what, and how would I know if they lied? It will sort the break into binding, authorization, withdrawal, or detection before you have finished reading the headline.

That lens outlives every algorithm transition and every CA/Browser Forum ballot. RSA will yield to ML-DSA; OCSP already yielded to CRLite; 398-day certificates are yielding to 47-day ones. The three questions do not change. Which is exactly why this is Part 4 of a field guide to protocol design, and not a chapter on parsing certificates.