Parag Mali - tag: applied-cryptography

How Elliptic Curves and Diffie-Hellman Break in Real Life: The Discrete Log Never Fell

noreply@paragmali.com (Parag Mali) — Sat, 18 Jul 2026 00:00:00 GMT

On a strong curve or a 2048-bit group, no one has ever solved the discrete logarithm in the field -- yet PlayStation 3 consoles, Android Bitcoin wallets, tamper-resistant TPMs, Windows certificate validation, and live TLS sessions all fell. Every one of those breaks attacked a layer the discrete-log math depends on but does not control: the per-signature nonce (PS3, Android, Minerva, TPM-Fail, LadderLeak), the curve and certificate parameters (CurveBall), or the negotiated strength (Logjam). A *reused* nonce surrenders the key from just two signatures; a *biased* nonce needs a statistical attack over many. The only discrete log ever computed here was Logjam's, against a downgrade-forced, export-crippled, shared 512-bit prime -- never a strong deployed curve. The fix is never a bigger curve. It is hedged, constant-time signing, validated parameters, and retiring export and shared groups.

1. Seven Breaks, One Unbroken Problem

On a well-chosen elliptic curve, the best attack anyone has ever fielded against the discrete logarithm is a square-root-time random walk that has barely moved since the 1970s [@hac-ch11][@teske-2001]. No deployed P-256, Curve25519, or secp256k1 key, and no standard 2048-bit Diffie-Hellman group, has ever had its discrete log solved in public. By every measurement we can make, the mathematics is holding.

And yet. PlayStation 3 consoles were jailbroken [@fail0verflow-ps3-2010]. Android Bitcoin wallets were emptied [@bitcoin-android-2013]. Tamper-resistant TPMs certified to government standards surrendered their signing keys [@tpmfail-2020]. Windows accepted forged certificates that chained to a counterfeit authority [@curveball-cve-2020-0601]. Live TLS sessions were downgraded and decrypted in transit [@logjam-weakdh-2015]. Every one of those systems ran the discrete-log math exactly as specified. The math was never where they broke.

That is the paradox this article resolves, and the resolution fits in one sentence: none of these attacks solved a discrete logarithm. Each attacked a layer the math depends on but does not control. There are exactly three such layers, and they organize everything that follows:

The per-signature nonce -- the throwaway secret ECDSA needs for every signature. PS3, Android, Minerva, TPM-Fail, and LadderLeak all landed here.
The curve and certificate parameters -- which curve, and whose generator, you are actually talking to. CurveBall landed here.
The negotiated strength -- how strong a key exchange two parties agree to use before they exchange anything. Logjam landed here.

Key idea: The discrete-logarithm assumption protects the private key against being directly computed -- and nothing else. "Nobody solved the discrete log" and "somebody recovered the private key" are not contradictory statements, because the discrete-log problem (genuinely hard) and a discrete-log deployment (a nonce, some parameters, a negotiation) are different things. Every real break in this article lives in the gap between them.

A note on scope. This piece treats the hardness of the elliptic-curve discrete logarithm as a black box: assume it holds, and watch everything around it fail. The structural mathematics of curve choice -- why one curve is weak and another is strong, how the number field sieve and Shor's algorithm actually work -- belongs to a companion article, "How the Discrete Log Would Break: Elliptic Curves, Finite Fields, and Why Curve Choice Is the Weak Part." Here, we assume the strongbox is uncrackable and ask how the key kept getting out anyway.

To see why a hard problem protects so little, you first have to see exactly what it promises -- and what it quietly leaves to the deployment.

2. What the Math Actually Promises

In 1976, Whitfield Diffie and Martin Hellman reduced the ancient problem of exchanging a secret over an open wire to a single piece of hard arithmetic: the discrete logarithm [@diffie-hellman-1976]. Two parties each pick a private exponent, exchange public powers of a shared generator, and combine them into a key that an eavesdropper cannot reconstruct without solving for one of those exponents.

Nine years later, Victor Miller and, independently, Neal Koblitz moved the same hard problem onto elliptic curves [@miller-1985][@koblitz-1987]. Koblitz argued the curve version was "likely to be harder than the classical discrete logarithm problem" [@koblitz-1987] -- which is why a 256-bit curve buys security that a finite field needs thousands of bits to match. A clean, triumphant origin. So where did the promise leak?

Given a generator `G` and a public point `Q = dG` on an elliptic curve (or a base `g` and `g^d` in a finite field), recovering the secret integer `d` is the discrete logarithm problem. The security of ECDSA, ECDH, and Diffie-Hellman rests on the assumption that this recovery is computationally infeasible for well-chosen parameters. It is the one thing the math promises -- and, as we will see, the only thing.

Signatures are where the story starts, because signatures introduce a second secret. The Elliptic Curve Digital Signature Algorithm (ECDSA) signs a message hash z under a long-term private key d by generating a fresh random value k, computing a point kG, taking its x-coordinate as r, and producing:

$$s = k^{-1},(z + r,d) \bmod n$$

where n is the order of the group. The verifier only ever sees (r, s). The private key d is buried inside, protected by the discrete-log assumption -- nobody can read d off the signature. But look at what else is in that equation.

The `k` in ECDSA is a *nonce*: a number used once. It must be secret, statistically uniform, and unique to each signature. It is not the private key, and it is thrown away the instant the signature is produced. It is also the single most load-bearing prerequisite in the whole scheme -- because `k` and `d` appear together in one linear equation, anything an attacker learns about `k` turns directly into knowledge of `d`.

The discrete-log assumption guards d against being computed from the public key. It says nothing whatsoever about three things the assumption quietly depends on but does not itself enforce:

The nonce k must be secret, uniform, and unique. The math assumes a perfect k and never checks that you supplied one.
The parameters must be the standard, named ones. The math assumes you are on the curve you think you are, with the generator you think you have.
The negotiated strength must not be crippled or shared. The math assumes the group you agreed to use is actually strong.

Each of those is the deployment's responsibility, not the primitive's. And each one became a pillar of failure.

flowchart TD DLP["(EC)DLP is hard: d cannot be computed from the public key dG"] DLP --> N["Depends on the per-signature nonce k"] DLP --> P["Depends on the curve and certificate parameters"] DLP --> G["Depends on the negotiated group strength"] N --> P1["Pillar 1 nonce: PS3, Android, Minerva, TPM-Fail, LadderLeak"] P --> P2["Pillar 2 validation: CurveBall"] G --> P3["Pillar 3 downgrade: Logjam"]

The curve migration itself is not our subject.Miller (1985) and Koblitz (1987) are mentioned here only to fix the primitive's birth. Whether a particular curve is structurally weak -- anomalous curves, small embedding degrees, the number field sieve, Shor's algorithm -- is the structural mathematics deferred to the companion article. This article assumes the curve is strong and attacks everything around it. The math guards the key against being computed. It says nothing about the key being handed over. Three contracts govern that handover, and each one broke.

3. Three Contracts the Deployment Signs and Never Reads

Every cryptographic deployment signs a contract it never reads. There are three clauses, one for each layer the math leans on, and the fine print on all three is where real products sank.

The nonce contract: k must be fresh

Take the ECDSA signing equation and ask the cruelest possible question: what if k is not unique? Suppose two different messages, with hashes z1 and z2, are signed under the same nonce k. Because r is the x-coordinate of kG, the same k produces the same r in both signatures. Now write them down side by side:

$$s_1 = k^{-1}(z_1 + r,d), \qquad s_2 = k^{-1}(z_2 + r,d) \pmod n$$

Subtract. The rd term is identical in both, so it cancels, and k falls straight out:

$$s_1 - s_2 = k^{-1}(z_1 - z_2), \qquad k = \frac{z_1 - z_2}{s_1 - s_2} \bmod n$$

Once you know k, the first equation has exactly one unknown left, and you solve for the private key by hand:

$$d = \frac{s_1,k - z_1}{r} \bmod n$$

No lattice. No side channel. No discrete log. Two signatures that share a nonce are two linear equations in two unknowns, and a middle-schooler with modular arithmetic can finish the job.

Note: The discrete-logarithm assumption protects d against direct computation from the public key -- and nothing else. It does not protect d against being algebraically extracted when the nonce misbehaves. The moment two signatures reuse k, the strongbox is irrelevant, because the deployment left a second door wide open.

The contract, then, is strict: k must be secret, statistically uniform, and never reused or partially leaked. Break any word of that clause and the algebra above (or its statistical cousin, later) takes your key.

The parameter contract: talk to the curve you think you are

The second clause is about identity. A named curve -- "P-256," "secp256k1" -- is an agreement to use one specific set of domain parameters, and in particular one specific generator, the base point G that every public key is measured against.

Every named elliptic curve fixes a single public point `G` from which all key material is derived: a private key `d` corresponds to the public point `dG`. The generator is not a secret -- it is a published constant. But it is load-bearing: "the public key is `dG`" is only meaningful if both sides agree on which `G`. A certificate can specify its curve two ways. A *named curve* is an object identifier pointing at standardized domain parameters (fixed `a`, `b`, `p`, `G`, `n`). *Explicit parameters* spell out all of those numbers inline, which lets the certificate carry a *custom* generator. Accepting explicit parameters without checking them against a known named curve means trusting an attacker to tell you what `G` is.

The contract: never trust attacker-supplied explicit parameters. A validator that checks a public point but not the generator it was measured against has, in effect, let the attacker choose the ruler and then declared the measurement correct. That is precisely the door CurveBall walked through [@curveball-cve-2020-0601].

The negotiation contract: agree to strength, not to a museum piece

The third clause governs what two parties agree to before they exchange anything. In ephemeral Diffie-Hellman, each side picks a fresh secret per connection, which is supposed to guarantee that compromising one session cannot retroactively expose others.

In ephemeral DH, the private exponents are generated fresh for each connection and discarded afterward, so a key recovered later cannot decrypt past sessions. This property is called forward secrecy. It is the reason "just use a fresh ephemeral key" feels like a complete defense -- and, as Logjam showed, the reason that feeling is dangerously incomplete. In the 1990s, US export regulations capped the strength of cryptography shipped abroad, producing deliberately weakened cipher suites such as 512-bit `DHE_EXPORT` Diffie-Hellman [@logjam-weakdh-2015]. The policy was abandoned, but the code was not: two decades later, servers and libraries still spoke these crippled suites on request.

The contract: the negotiated group must be strong, fresh, and unshared. The fine print that sank it: many servers ship the same standardized 512-bit prime, so breaking one group breaks all of them at once [@logjam-weakdh-2015]. A fresh ephemeral exponent does not save you when the prime itself is weak and shared.

Contract	What it requires	What violates it	Field break
Nonce	`k` secret, uniform, unique per signature	a constant, repeated, or biased `k`	PS3, Android, Minerva, TPM-Fail, LadderLeak
Parameter	named curve, standard generator `G`	trusting attacker-supplied explicit parameters	CurveBall
Negotiation	strong, fresh, unshared group	downgrade to a crippled, shared group	Logjam

Three contracts, three clauses of fine print. Over fifteen years, every clause was breached in the field -- and the breaches fall into exactly three piles.

4. The Failure Catalog: Seven Breaks, Three Layers, One Shape

None of the seven incidents below solved a discrete logarithm. Every one of them recovered a private key or forged a trusted one. Watch where each attack actually landed -- because the landing spot, not the marquee name, is the whole lesson.

timeline title EC and DH in the field, the assumption versus the breaks 1976 : Diffie-Hellman, the assumption is born 1985-1987 : Miller and Koblitz, the curve migration 2010 : PS3 constant nonce, reuse 2013 : Android SecureRandom, reuse 2015 : Logjam, downgrade plus shared-prime NFS 2019-2020 : Minerva and TPM-Fail, lattice bias 2020 : LadderLeak, Bleichenbacher-Fourier sub-bit : CurveBall, unvalidated generator

Here is the entire corpus on one page. The rightmost column is the thesis in a single word, repeated seven times.

Incident	Year	Pillar	Mechanism	Effort to break	What fell	Discrete log solved?
PS3	2010	Nonce (reuse)	a constant `k`	2 signatures	ECDSA firmware-signing key	No
Android wallets	2013	Nonce (reuse)	repeated `k` from a broken RNG	2 reused signatures	Bitcoin wallet keys	No
Logjam	2015	Downgrade	forced 512-bit shared prime	1 NFS precompute per prime	TLS session secrecy	Only a crippled, shared 512-bit export prime
TPM-Fail	2019-2020	Nonce (bias)	timing leak of nonce bit-length	many signatures + lattice	256-bit TPM keys	No
Minerva	2020	Nonce (bias)	timing leak of nonce bit-length	about 2100 signatures + lattice	ECDSA key	No
LadderLeak	2020	Nonce (bias)	sub-bit ladder leak	many signatures + Bleichenbacher/Fourier	ECDSA key	No
CurveBall	2020	Validation	an unchecked generator	1 forged certificate	certificate trust itself	No

Six "No"s and one heavily qualified "Yes." Let us walk the three pillars in turn.

Pillar 1a: the nonce, reused (certain, from two signatures)

In December 2010, at the 27th Chaos Communication Congress, the group fail0verflow presented "Console Hacking 2010" and revealed that Sony had signed PlayStation 3 firmware using ECDSA with a constant nonce [@fail0verflow-ps3-2010]. Not a weak random k. The same k, every time. Every signature Sony ever produced shared a nonce with every other, which means any two of them collapse to the algebra from Section 3.Keep two PS3 events distinct. fail0verflow's 27C3 talk exposed the constant-nonce mechanism. George Hotz released a separate PS3 root key weeks later, in January 2011 [@hotz-ps3-2011]. The teaching case -- a hardcoded constant nonce -- is fail0verflow's. The console's chain of trust, the thing Sony spent years engineering, came apart because a random number generator was told to always return the same number.

flowchart LR A["Sig 1: s1 = k^-1 (z1 + r d)"] --> C["Same k means same r"] B["Sig 2: s2 = k^-1 (z2 + r d)"] --> C C --> D["Subtract, k cancels: k = (z1 - z2) / (s1 - s2) mod n"] D --> E["Solve for d = (s1 k - z1) / r mod n"] E --> F["Private key recovered, no discrete log solved"]

This is not a metaphor. Here is the recovery, executable. Two signatures share a nonce, and a dozen lines of BigInt arithmetic reproduce the private key exactly:

{` // secp256k1 group order const n = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141n; const mod = (a, m) => ((a % m) + m) % m;

// Modular inverse via the extended Euclidean algorithm function inv(a, m) { let [oldr, r] = [mod(a, m), m]; let [olds, s] = [1n, 0n]; while (r !== 0n) { const q = oldr / r; [oldr, r] = [r, oldr - q * r]; [olds, s] = [s, olds - q * s]; } return mod(olds, m); }

// A victim signs two different messages with the SAME nonce k const d = 0x00c0ffee1234567890abcdef00c0ffee1234567890abcdef00c0ffee12345678n; // secret const k = 0x0000deadbeef0000feed0000deadbeef0000feed0000deadbeef0000feed0001n; // reused nonce const r = mod(0x1122334455667788112233445566778811223344556677881122334455667788n, n); const z1 = 0x1111111111111111111111111111111111111111111111111111111111111111n; const z2 = 0x2222222222222222222222222222222222222222222222222222222222222222n; const s1 = mod(inv(k, n) * mod(z1 + mod(r * d, n), n), n); const s2 = mod(inv(k, n) * mod(z2 + mod(r * d, n), n), n);

// The attacker sees only (r, s1, z1) and (r, s2, z2). Recover k, then d. const kRec = mod((z1 - z2) * inv(mod(s1 - s2, n), n), n); const dRec = mod((mod(s1 * kRec, n) - z1) * inv(r, n), n); console.log('recovered nonce matches: ' + (kRec === k)); console.log('recovered private key: ' + (dRec === d)); console.log('d = 0x' + dRec.toString(16)); `}

Three years later, the same algebra emptied wallets. In August 2013, an advisory on bitcoin.org warned that "a component of Android responsible for generating secure random numbers contains critical weaknesses" that rendered "all Android wallets" vulnerable to theft [@bitcoin-android-2013]. The root cause, per Google's own postmortem, was that the platform's SecureRandom "may not receive cryptographically strong values" because the underlying PRNG was improperly initialized [@android-securerandom-2013].

The consequence was nonce repetition across transactions. And because every Bitcoin signature is broadcast to a public ledger, anyone could scan the chain for two signatures sharing an r and drain the wallet behind them.The bitcoin.org advisory names the flaw and the affected wallets but states no total loss figure [@bitcoin-android-2013]. Any "coins stolen" number you have seen is reported, not established -- treat it accordingly.

The detector that would have caught both PS3 and Android is embarrassingly short. Group public signatures by r; any collision is an exposed key:

{` const n = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141n; const mod = (a, m) => ((a % m) + m) % m; function inv(a, m) { let [oldr, r] = [mod(a, m), m]; let [olds, s] = [1n, 0n]; while (r !== 0n) { const q = oldr / r; [oldr, r] = [r, oldr - q * r]; [olds, s] = [s, olds - q * s]; } return mod(olds, m); } // Build a public feed in which one signer reused a nonce, then forget the secret const dSecret = 777n, kReused = 424242n, rReused = 987654321n; const mk = (z) => ({ r: rReused, s: mod(inv(kReused, n) * mod(z + mod(rReused * dSecret, n), n), n), z }); const feed = [ { r: 111n, s: 22n, z: 5n }, mk(1001n), { r: 222n, s: 44n, z: 9n }, mk(2002n), { r: 333n, s: 66n, z: 12n } ];

Two incidents, two different root causes -- a hardcoded constant and a starved RNG -- and one identical, certain outcome. This is the crudest form of the whole thesis: the deployment lost the key while the discrete log stood untouched.

Note: A repeated nonce is certain key recovery from just two signatures, by elementary algebra. A biased nonce -- one that merely leans, statistically, toward certain values -- leaks only a partial statistic and requires a statistical attack over many signatures. "One biased signature leaks your key" is false. The distinction is the difference between the next two subsections, and blurring it is the single most common error in this area.

Dimension	Reused nonce	Biased nonce
What leaks	the whole nonce (it repeats)	a partial statistic about the nonce
Signatures needed	exactly 2	many (Minerva about 2100; PuTTY about 60 [@putty-cve-2024-31497])
Attack	elementary algebra	Hidden Number Problem (lattice or Fourier)
Certainty	certain	statistical
Examples	PS3, Android	Minerva, TPM-Fail, LadderLeak, PuTTY

Pillar 1b: the nonce, biased (statistical, over many signatures)

Reuse is loud. The subtler failure is a nonce that never repeats but is not quite uniform -- one whose top bit is a little too often zero, say. It turns out that even this surrenders the key, and the theory predicting it predates the exploits by two decades.

Given many samples that each leak a few of the most-significant bits of `k`, together with the public signature data, recover the fixed secret `d`. Boneh and Venkatesan defined this problem in 1996 and proved it tractable [@boneh-venkatesan-1996]. Every biased-nonce attack in this article -- Minerva, TPM-Fail, LadderLeak, PuTTY -- is an instance of it.

Nguyen and Shparlinski turned the abstract HNP into a concrete lattice attack on (EC)DSA with partially known nonces [@nguyen-shparlinski-2003], and Brumley and Tuveri showed in 2011 that the necessary bit-leak could be harvested remotely, over a network, from an OpenSSL implementation whose scalar multiplication was not constant-time [@brumley-tuveri-2011]. Then the field split into two solver regimes, and telling them apart is essential.

When the timing channel leaks a few bits of the nonce -- typically its exact bit-length -- the HNP is solved by lattice reduction. Minerva (CHES 2020) demonstrated this against smartcards and libraries whose scalar multiplication ran in time correlated with the nonce's length, reporting roughly "2100 signatures to recover the private key on the standard secp256r1 curve" [@minerva-2020].

TPM-Fail (USENIX Security 2020) carried the same idea into hardware roots of trust. A timing leak in Intel firmware TPMs (CVE-2019-11090 [@cve-2019-11090]) and STMicroelectronics ST33 chips (CVE-2019-16863 [@cve-2019-16863]) recovered a 256-bit ECDSA key from an Intel fTPM in 4 to 20 minutes, and a VPN authentication key remotely in about five hours [@tpmfail-2020].Two honesty notes. Minerva's project page retracted its original EdDSA claim; the confirmed result is ECDSA-only [@minerva-2020]. And the TPM-Fail targets were not toys -- they held Common Criteria EAL4+ and FIPS 140-2 certification, and the leak survived that scrutiny [@tpmfail-2020].

flowchart TD L["A timing channel leaks partial information about the nonce k"] --> Q{"How much leaks per signature?"} Q -->|"at least 2 bits"| LAT["Lattice reduction of the HNP: Minerva, TPM-Fail"] Q -->|"under 1 bit"| FFT["Bleichenbacher and Fourier HNP: LadderLeak"] LAT --> R["Private key d recovered over MANY signatures"] FFT --> R

The second regime is where people get it wrong. When the leak drops below a single bit -- LadderLeak revealed the most-significant bit of the nonce "but with probability less than one" -- lattices become intractable, and the attack instead revives a Fourier-analysis method the paper credits to Daniel Bleichenbacher [@ladderleak-2020].

The LadderLeak authors note that "most of the attacks so far have relied on at least two bits of nonce bias," precisely the lattice regime. Their contribution was to break below it, over curves sect163r1 and NIST P-192, using the Fourier solver rather than a lattice [@ladderleak-2020]. Calling LadderLeak a lattice attack is the field's classic mistake.

"Breaking ECDSA With Less Than One Bit Of Nonce Leakage" -- the title of the LadderLeak paper, and the most compact statement of how little a nonce must betray before the key is gone.

Notice what happened across this subsection. Every one of these attacks beat a signer that had already removed the reuse bug. The leak had moved from how the nonce was generated to how it was used -- a variable-time scalar multiplication -- and the cure had to move with it. Hold that thought; it is the pivot of the entire article. But reuse or bias, lattice or Fourier, two signatures or two thousand, the outcome is the same and the discrete logarithm is never touched: the deployment hands over d, and the strongbox stays locked.

Pillar 2: the parameters, unvalidated (CurveBall)

The nonce pillar is about a secret the deployment mishandles. The second pillar is about an identity the deployment fails to check. On 14 January 2020, Microsoft patched CVE-2020-0601, reported by the NSA: "a spoofing vulnerability exists in the way Windows CryptoAPI (Crypt32.dll) validates Elliptic Curve Cryptography (ECC) certificates" [@curveball-cve-2020-0601]. It became known as CurveBall, and it is the cleanest example in the whole corpus of a break that has nothing to do with the mathematics of ECC.

Recall the parameter contract. A public key is the point dG -- meaningful only if everyone agrees on G. CryptoAPI, when handed a certificate that specified its curve with explicit parameters, checked that the certificate's public point matched a trusted root's public key. What it did not check was the generator.

So an attacker could craft a curve with a custom generator G', pick any private key d' they liked, and compute G' such that d' * G' equals the exact public point of a trusted root certificate authority. The forged certificate then presents the right public point on the wrong curve. Windows, comparing only the point, accepts it -- while a library that pins named curves rejects it instantly, because G' is not the standard base point.

flowchart TD A["Attacker picks explicit curve parameters with a custom generator G-prime"] --> B["Chooses private key d-prime so that d-prime times G-prime equals a trusted root public point Q"] B --> C["Forged certificate presents point Q on the attacker curve"] C --> D{"Windows CryptoAPI validation"} D -->|"Public point matches a trusted root? YES"| E["Accepted, but the generator was never checked"] D -.->|"A named-curve-pinning library also checks G"| F["Rejected, G-prime is not the standard base point"] E --> G["Forged code-signing or TLS certificate chains to a trusted root"]

The ECDLP was never in play. The attacker chose their own private key; they never had to recover anyone's. The failure is a single missing comparison -- generator against named curve -- buried in a certificate validation stack that a billion machines trusted. CurveBall is a validation bug, not a curve break, and the fix is exactly what the missing check would have been: reject explicit parameters, or fully re-validate them against the claimed named curve.

Pillar 3: the negotiation, downgraded (Logjam)

The third pillar is the one honest exception, and handled carelessly it looks like a refutation of the whole thesis. In 2015, the Logjam team (CVE-2015-4000) showed that a man-in-the-middle could rewrite a TLS handshake -- "rewriting a ClientHello with DHE replaced by DHE_EXPORT" -- to force the connection onto a 512-bit export-grade Diffie-Hellman group [@cve-2015-4000]. And here, unlike everywhere else, the attacker really does compute a discrete logarithm.

The trick is that the expensive part of the number field sieve has to be done only once per prime, and servers share primes.

The fastest known classical algorithm for the finite-field discrete logarithm (and for integer factoring). Its decisive feature here: its costly first stage depends only on the *prime* being attacked, not on any particular exchanged key. Precompute once per prime, and every session under that prime becomes cheap to break. The internals belong to the companion article; what matters here is that one structural fact.

The Logjam researchers found that "8.4% of the Top 1 Million domains were initially vulnerable," and that because so many servers reused a handful of standardized 512-bit primes, "the first step in the number field sieve ... is dependent only on this prime" [@logjam-weakdh-2015]. One precomputation, amortized across every server that shipped that prime.

"The first step in the number field sieve ... is dependent only on this prime." One precomputation breaks every server that shares the group. sequenceDiagram participant C as Client participant M as MITM participant S as Server Note over M: One-time NFS precomputation per shared 512-bit prime C->>M: ClientHello offering DHE M->>S: ClientHello rewritten to DHE_EXPORT S->>M: 512-bit export DH parameters, a shared standardized prime M->>C: forwards the weak parameters Note over M: Precomputation done, so this connection solves fast M-->>C: session decrypted in near real time The 512-bit `DHE_EXPORT` groups existed because 1990s US export regulations forbade shipping strong cryptography abroad, mandating deliberately breakable key exchange for foreign markets [@logjam-weakdh-2015]. The policy was abandoned by the early 2000s, but the cipher suites were never removed from server software. Logjam, in 2015, exploited a legal compromise that had been dead for over a decade and a codebase that never got the memo. The math did not decay -- the deployment simply kept a museum piece plugged in.

Read carefully, Logjam is not a counterexample; it is the thesis in its most literal form. A discrete log was computed, but only against a 512-bit prime that a downgrade forced the connection onto and that policy had left crippled and shared. The deployment set the weak, shared group on the table; the number field sieve merely finished the job. No strong, standard-sized group ever fell. The fix is the same shape as the break: remove export ciphers, negotiate vetted named groups (RFC 7919 [@rfc7919]), and adopt TLS 1.3, which removes static Diffie-Hellman entirely [@rfc8446].

Three pillars, seven breaks, one shape: every attack hit a layer the discrete-log math depends on but does not control. Name the shape, and you have the whole discipline.

5. The Realization: Fix the Contracts, Not the Curve

There was no single eureka here. The insight accumulated across fifteen years of incidents and then snapped into one sentence: every field break attacked a layer the discrete-log math depends on but does not control -- the nonce, then the parameters, then the negotiated strength -- and every fix changed how the primitive is generated, used, validated, or negotiated, never the curve and never the log.

The sharpest crystallization hides inside the word "bias." For years it was treated as one problem with one fix: remove the random number generator, derive the nonce deterministically (RFC 6979 [@rfc6979], later blessed by FIPS 186-5 [@fips-186-5]), and the biased-nonce class is solved. But Minerva, TPM-Fail, and LadderLeak all recovered keys from signers that had already done that. The reason is that "bias" was concealing two distinct root causes with two distinct cures.

flowchart TD B["One word: nonce bias"] --> G["Generation defect: the RNG returns a constant, repeated, or predictable k"] B --> U["Usage defect: variable-time scalar multiplication leaks the bit-length of k"] G --> GC["Cure: deterministic and hedged nonces, RFC 6979 and FIPS 186-5"] U --> UC["Cure: constant-time, exception-free scalar multiplication"] GC --> BOTH["You need BOTH, they are orthogonal"] UC --> BOTH

A generation defect is a bad nonce coming out of the sampler -- a constant (PS3), a repeat (Android), a predictable value. Determinism and hedging cure it. A usage defect is a perfectly good nonce leaking through the timing of the scalar multiplication that consumes it -- the bit-length falling out through a side channel (Minerva, TPM-Fail) or a sub-bit correlation (LadderLeak). Only constant-time code cures that. The two cures are orthogonal, and you need both.

Key idea: Generalize the two-cures fork and you get the entire discipline of this field: four contracts, not one curve. Generate the nonce so it cannot be predicted or repeated; use it in constant time so it cannot leak; validate your parameters so no attacker chooses your generator; negotiate only strong, unshared groups so no downgrade can strand you. Every documented break is a breach of one of those four. No break in the corpus is a break of the discrete logarithm.

This also dissolves the apparent contradiction the article opened with. "Nobody solved the discrete log" and "somebody recovered the key" only sound incompatible if you conflate the discrete-log problem with a discrete-log deployment. The problem is a statement about computational hardness; the deployment is a nonce, a set of parameters, and a negotiation, wrapped around that hardness. The problem stayed hard. The wrapping tore.

If you remember one operational thing from this article, make it this: deterministic ECDSA and constant-time scalar multiplication are not two names for the same safeguard. A team can adopt RFC 6979, check the box marked "nonce," and still ship a signer that a co-located attacker times to death. Ask both questions separately -- how is the nonce generated, and how is it used -- because they have separate answers.

If the discipline is "honor the four contracts, not the curve," what does a deployment that honors all four actually look like in 2026?

6. What Correct EC/DH Deployment Looks Like in 2026

The state of the art in this field is not a better discrete-log solver -- no such thing exists. It is a deployment discipline at the three layers the math never controlled. Here it is, pillar by pillar.

Nonce: two orthogonal cures, and a primitive that folds in both

Derive the nonce `k` deterministically from the private key and the message hash via an HMAC-based construction, so the signer "do[es] not need access to a source of high-quality randomness" [@rfc6979]. Identical key-and-message pairs always yield the same `k`; distinct messages yield unrelated nonces. This closes the *generation* defect -- the PS3 and Android reuse classes cannot occur.

RFC 6979 also quietly records the deeper lesson. It insists on comparing each candidate nonce against the group order rather than reducing it modulo the order, warning that a "simple modular reduction would induce biases that would be detrimental to signature security" [@rfc6979]. That is a standards document, in 2013, acknowledging that bias, not just reuse, is fatal. You can watch the effect yourself:

{// A raw sampler emits values in [0, 256). We need a nonce in [0, q). const q = 200n, B = 256n, trials = 400000; const raw = () => BigInt(Math.floor(Number(B) * Math.random())); let biasedHigh = 0, fairHigh = 0; const half = q / 2n; for (let i = 0; i < trials; i++) { const reduced = raw() % q; // reduce mod q -> BIASED if (reduced >= half) biasedHigh++; let fair; do { fair = raw(); } while (fair >= q); // compare to q, retry -> FAIR if (fair >= half) fairHigh++; } console.log('reduce mod q -> P(nonce in top half): ' + (biasedHigh / trials).toFixed(4)); console.log('compare to q -> P(nonce in top half): ' + (fairHigh / trials).toFixed(4)); console.log('That gap from 0.5 is a free bit for a Hidden Number Problem attacker.');}

Determinism, though, fixes only generation. On hardware it also adds a risk.

Compute the nonce from the private key, the message hash, *and* a few bytes of fresh randomness. A dead RNG falls back to the safe deterministic core; a differential fault, which needs two identical signings to differ, is defeated by the fresh input. FIPS 186-5 recommends this for hardware because bare determinism is "of particular concern for ... deterministic signature schemes, as well as embedded or IoT devices and smartcards" [@fips-186-5]. Compute `kG` with a data-independent trace of instructions, branches, and memory accesses, so the runtime never depends on the bit-length or value of `k`. This is the *usage* cure -- the only defense against Minerva, TPM-Fail, and LadderLeak, whose leaks all live in the multiplication, not the sampler. Its precursor is Brumley and Tuveri's 2011 remote attack on a non-constant-time OpenSSL ladder [@brumley-tuveri-2011].

Note: Hedged signing and constant-time multiplication are the two cures of the fork, and they do not substitute for each other. A hardware signer needs both: hedge the generation, and make the usage constant-time. Checking one box leaves the other break wide open.

New designs can sidestep the whole fork by choosing a misuse-resistant primitive. Ed25519 (RFC 8032) derives its nonce as a hash of the secret and message -- deterministic by construction, no per-signature RNG -- and signs with a single complete, constant-time addition law that has no exceptional cases to mishandle [@rfc8032]. It folds both cures into the primitive and removes the dangerous choices from the caller's hands.Misuse-resistant is not misuse-proof. PuTTY derived P-521 ECDSA nonces such that "the first 9 bits of each ECDSA nonce are zero," allowing "full secret key recovery in roughly 60 signatures" (CVE-2024-31497) [@putty-cve-2024-31497]. Determinism implemented carelessly re-introduced exactly the bias it was meant to prevent.

Validation: reject what you cannot vouch for

The CurveBall cure is direct: bind a certificate's curve to the standard parameters of its named curve, and reject explicit parameters or fully re-validate them. The January 2020 patch made crypt32.dll check the generator, not merely the point [@curveball-cve-2020-0601].

For key exchange with static keys, the discipline is fuller. Before using a received point, confirm it is not the identity, lies on the curve, and sits in the correct prime-order subgroup -- the full public-key validation of NIST SP 800-56A, which blocks invalid-curve and small-subgroup attacks that would otherwise bleed a static key out one query at a time [@sp-800-56a-r3].

The most durable answer is to make invalid inputs unrepresentable. X25519 works on a Montgomery curve chosen so that every 32-byte string is an acceptable public key; the RFC even instructs implementations to "accept non-canonical values," and to run the conditional swap "in constant time" [@rfc7748]. There is no explicit-parameter path to abuse and no on-curve check to forget.This is the "make invalid states unrepresentable" principle applied to cryptography: the safest validation code is the code you deleted because the type system already guarantees the property. But note the boundary carefully.

Note: X25519's scalar "clamping" fixes the cofactor and twist problems for ephemeral key exchange. It does not replace named-curve pinning in the X.509 world, and some protocols still need a contributory-behavior check on the shared secret. Twist-secure curves solve the problems they were designed for -- not every parameter problem.

Negotiation: remove the weak option entirely

The downgrade pillar's fix is structural. RFC 7919 replaced ad-hoc, shared Diffie-Hellman primes with a menu of vetted named groups negotiated like curves [@rfc7919]. TLS 1.3 went further and deleted the weak options outright, so there is nothing to downgrade to.

"Static RSA and Diffie-Hellman cipher suites have been removed; all public-key based key exchange mechanisms now provide forward secrecy." -- RFC 8446, the TLS 1.3 specification [@rfc8446].

Because TLS 1.3 permits only strong ephemeral groups and binds the negotiation into the handshake transcript, the Logjam rewrite is not merely discouraged -- it is structurally impossible [@rfc8446]. The end state of the negotiation pillar is a protocol in which the crippled, shared 512-bit group Logjam relied on simply cannot be selected.

Each of these is a choice, and where there is a choice, engineers argue. Which discipline actually wins in which threat model?

7. Competing Approaches: Which Defense Answers Which Adversary

These defenses are not rungs on one ladder. They are answers to different adversaries, and the honest way to compare them is one pillar at a time.

Note: A constant-time multiplier and a named-curve pin do not compete -- they defend different layers, and a real system needs one from each pillar. The comparisons below rank options within a pillar. Picking the best nonce discipline does nothing for your parameter validation.

Nonce: which signing discipline stops which adversary

The load-bearing insight is that no single column stops every row. Generation cures (deterministic [@rfc6979], hedged [@fips-186-5]) and the usage cure (constant-time) are orthogonal, which is why hedged-plus-constant-time became the practitioner default and why Ed25519 [@rfc8032] folds most of both into the primitive.

Adversary / failure	Randomized (bare RNG)	Deterministic (RFC 6979)	Hedged (FIPS 186-5)	Constant-time mult	Ed25519
RNG returns constant or repeat (PS3, Android)	fails	stops	stops	orthogonal	stops
Differential fault on a deterministic signer	randomness helps	fails	stops	orthogonal	inherits the caveat
Timing leak of nonce bit-length (Minerva, TPM-Fail)	fails	fails	fails	stops	stops (uniform ladder)
Sub-bit ladder leak (LadderLeak)	fails	fails	fails	stops	stops (uniform ladder)

Read the rows, not the columns: reuse fails only the first row and is certain from two signatures; the timing leaks fail three of the five columns and are cured on a different axis entirely. That is the two-cures fork, rendered as a matrix.

Validation: three philosophies of not trusting your input

Dimension	Named-curve pinning	Full validation (SP 800-56A)	All-inputs-valid (X25519)
Philosophy	reject non-standard parameters	validate every received input	make invalid inputs unrepresentable
Stops CurveBall (unvalidated generator)	yes, its exact fix	yes, re-derives the generator	yes, no explicit-parameter path
Stops invalid-curve / small-subgroup	partial	yes, on-curve plus subgroup	yes, twist-secure
Per-operation cost	O(1) table compare	O(1) up to one scalar mult	one constant-time ladder
Best suited for	X.509, code-signing, PKI	static-key ECDH, HSMs	ephemeral ECDH

Named-curve pinning is the precise CurveBall fix [@curveball-cve-2020-0601]; SP 800-56A validation is the belt-and-suspenders choice for static keys [@sp-800-56a-r3]; X25519 deletes the problem by construction [@rfc7748]. They converge on stopping CurveBall and diverge on everything else.

Downgrade: hardening, deletion, or hedging against tomorrow

Dimension	Config hardening + FFDHE	TLS 1.3	Hybrid post-quantum
Removes the 512-bit export rung	yes, within FFDHE	yes, structurally	yes, inherits 1.3
Removes shared or unknown primes	yes, named menu	yes, fixed groups	yes
Downgrade protection in the handshake	no, still in-band	yes, transcript-bound	yes
Defends against a future Shor adversary	no	no	yes, if ML-KEM holds
Residual weakness	the legacy long tail	non-TLS 1024-bit primes	a KEM is not a NIKE, young PQC

RFC 7919 named groups [@rfc7919] are the transitional hardening for TLS 1.2; TLS 1.3 is the structural end state [@rfc8446]; hybrid X25519 with ML-KEM (RFC 9954 [@rfc9954], ML-KEM standardized as FIPS 203 [@fips-203]) is the forward hedge against a quantum adversary. Only the last column defends against a break that would be genuine mathematics rather than a deployment slip.

Every column in every table changed the deployment, not the curve. So is the discrete log itself simply unbreakable? Almost -- and the "almost" is the most honest paragraph in this article.

8. Theoretical Limits: What Is Provably True on Both Sides

This is the honest concession the thesis has been earning the right to make. There are two provable sides here, and the argument needs both.

The hardness side: no room for a strong curve to fall

On a well-chosen group, the discrete logarithm is genuinely hard, and we can say so with unusual precision. The best classical attack is Pollard's rho, a square-root-time random walk that needs negligible memory [@teske-2001]. In the strict generic model its expected cost is about $\sqrt{\pi n/2}$, roughly 1.25 times $\sqrt{n}$ group operations; on an elliptic curve, the negation map (where $P$ and $-P$ share an x-coordinate) trims that to about 0.886 times $\sqrt{n}$. Either way the exponent is the same, matching baby-step giant-step's $O(\sqrt{n})$ time [@hac-ch11].

Against that upper bound, Shoup proved a generic lower bound of $\Omega(\sqrt{p})$: no algorithm restricted to the group operation can do better [@shoup-1997]. The two meet at $\Theta(\sqrt{n})$. The generic gap is essentially closed -- which is exactly why a 256-bit curve delivers around 128-bit security, and why no strong deployed curve's discrete log has fallen.

Quantity	Value	Note
Best classical DL attack	Pollard's rho, about 1.25 times sqrt(n) generic; 0.886 times sqrt(n) on curves	negligible space
Baby-step giant-step	O(sqrt(n)) time and O(sqrt(n)) space	the space-heavy twin
Generic lower bound	Omega(sqrt(p)), provably no faster generic algorithm	Shoup 1997
Finite-field DH (non-generic)	sub-exponential via the number field sieve	why DH needs about 3072-bit primes (2048-bit minimum)
Strong deployed curve or standard group ever solved	none, only Logjam's crippled shared 512-bit prime	the thesis in one row

The last two rows carry the concession. Finite-field Diffie-Hellman is not generic: the number field sieve solves it in sub-exponential time, which is why finite-field DH needs primes of about 3072 bits to match the security a 256-bit curve gives (2048 bits is the absolute minimum), and why Logjam's downgrade to 512 bits was finishable at all [@logjam-weakdh-2015]. The number-field-sieve mathematics belongs to the companion article.

Key idea: Logjam is the only break in this corpus that reached the mathematics, and the reason is structural, not accidental. A finite field carries extra arithmetic structure that a generic group lacks, which lets the number field sieve run in sub-exponential time, faster than the square-root barrier that protects a strong elliptic curve. That structural gap is the sole opening the discrete log ever offered, and even it was not enough alone: the computation became feasible only after the prime was cut to 512 export-grade bits. Raise the group back to a standard 2048-bit prime, or move to a curve where no such shortcut exists, and the opening closes.

The leakage side: no room to hide a little bias

The other provable side indicts the deployment just as sharply. The Hidden Number Problem makes partial leakage fatal by construction: Boneh and Venkatesan proved that recovering a secret from many small most-significant-bit leaks is tractable [@boneh-venkatesan-1996], and Nguyen and Shparlinski turned that into a working lattice attack on (EC)DSA with partial nonces [@nguyen-shparlinski-2003].

There is no "a little bias is tolerable" regime: any consistent partial leak of k, integrated over enough signatures, provably recovers d. Two or more leaked bits fall to a lattice; a sub-bit leak falls to Bleichenbacher's Fourier method [@ladderleak-2020]. "k is secret, uniform, and unique" is not a best practice -- it is a hard boundary.

Note: The math is safe at the sizes we deploy (the generic gap is closed), and leakage is fatal at any size (the HNP admits no safe margin). Both proofs point the same way: the primitive is not the weak part. The nonce, the parameters, and the negotiation are.

Two clouds sit on the horizon. First, the shared-prime structure Logjam exploited at 512 bits revives at 1024. The Logjam team estimated that precomputing the single most common 1024-bit prime would expose roughly 18% of the top HTTPS sites; a second common prime would reach about 66% of IPsec VPNs and around 26% of SSH servers [@logjam-weakdh-2015] -- the same trick, resized, and a resourced adversary's project. Second, a large fault-tolerant quantum computer running Shor's algorithm would break the discrete logarithm outright, curve and finite field alike. That is the one future break that *is* the mathematics, and it is the subject of the companion article, not this one. The deployment answer, already shipping, is hybrid key exchange.

So where, concretely, does EC and DH still break tomorrow?

9. Where EC and DH Still Break in the Field

The incidents are settled. The classes are not. Every open problem below is a documented pattern waiting for its next instance, and each one maps to a pillar you have already met.

The next SecureRandom (nonce generation, Pillar 1). A signer needs a unique, secret nonce at the exact moment its entropy is weakest: first boot, a headless IoT device with no entropy events, or a freshly cloned virtual machine carrying a duplicated seed. This is the mechanism behind Android 2013 [@android-securerandom-2013], and it recurs whenever hardware is provisioned faster than it can gather randomness.Cloned-VM state duplication is an especially nasty modern form: two hosts that never coordinated can emit the same "random" nonce because they booted from one snapshot. The signing path can be hedged [@fips-186-5], but key generation and ephemeral key exchange still need a live, well-seeded source.

The next Minerva or LadderLeak (nonce usage, Pillar 1). Constant-time scalar multiplication must be re-established against every new microarchitecture -- caches, execution ports, prefetchers, simultaneous multithreading -- and on every fragile incomplete-addition curve. The bit-length leak of Minerva [@minerva-2020] and the sub-bit leak of LadderLeak [@ladderleak-2020] both post-date deterministic nonces precisely because the leak had moved to usage. Formally verified primitives close the known channels on the chosen curve; FIPS-mandated P-curves and secp256k1 keep hand-written code in production.

The long tail of shared primes (downgrade, Pillar 3). TLS 1.3 closes the web protocol layer, but export-grade and shared standardized primes outlived their policy elsewhere. IKEv1 VPNs and older SSH still ship shared 1024-bit Diffie-Hellman groups, and a single precomputation per shared prime amortizes across every server using it [@logjam-weakdh-2015]. The consensus recommendation is blunt: retire 1024-bit DH now.

The next CurveBall (validation, Pillar 2). CurveBall was one unchecked generator in one library [@curveball-cve-2020-0601]; the class is "somewhere in a large validation stack, attacker-supplied parameters are trusted." Every new parser, certificate profile, or post-quantum key format reopens the class, and a single such bug in a widely embedded validator forges trust across everything that depends on it.

Post-quantum migration ahead of Shor (cross-pillar). Hybrid key exchange -- running X25519 alongside ML-KEM so the session stays secure if either component holds -- is standardized (RFC 9954 [@rfc9954]; concrete groups such as X25519MLKEM768 in the ongoing ECDHE-MLKEM work [@draft-ietf-tls-ecdhe-mlkem]; ML-KEM standardized as FIPS 203 [@fips-203]) and is already deployed at scale: it protected a majority of the human web traffic Cloudflare measured by late 2025 [@cloudflare-pq-radar]. But the migration carries a trap that has nothing to do with the discrete log.

Note: ML-KEM is a key-encapsulation mechanism, not a non-interactive key agreement. It does not reproduce X25519's static Diffie-Hellman semantics, where two long-term public keys combine without any live exchange. Systems that quietly assume "post-quantum DH" is a drop-in replacement for classical DH will find the shape of the primitive has changed underneath them. Migrate deliberately, not by search-and-replace.

None of these needs new mathematics to break you. Each needs exactly one deployment mistake. Here is how not to make it.

10. What to Do on Monday

Every item below maps to a specific incident in this article. Do these, and you do not ship the next one.

Never put a bare runtime RNG in the signing path. Use deterministic (RFC 6979 [@rfc6979]) or, on hardware, hedged (FIPS 186-5 [@fips-186-5]) nonces. Prevents: PS3, Android.
Demand constant-time, exception-free scalar multiplication. This is a separate control from the nonce discipline, not a duplicate of it. Prevents: Minerva, TPM-Fail, LadderLeak.
Prefer misuse-resistant primitives. Ed25519 for new signing [@rfc8032], X25519 for ephemeral key exchange [@rfc7748]; for static-key ECDH, run full public-key validation per SP 800-56A [@sp-800-56a-r3]. Prevents: whole classes of implementer error.
Reject explicit curve parameters; require named curves. Pin the curve, and the generator with it. Prevents: CurveBall [@curveball-cve-2020-0601].
Disable export ciphers and use vetted groups. RFC 7919 named groups [@rfc7919] for residual TLS 1.2 DHE, or move to TLS 1.3 [@rfc8446]. Prevents: Logjam.
Never assume a fresh ephemeral rescues a shared prime. The number-field-sieve precomputation is per-prime, not per-connection [@logjam-weakdh-2015]. Prevents: the subtle half of Logjam.
Plan the post-quantum migration as hybrid, not pure. Run a classical curve alongside ML-KEM (RFC 9954 [@rfc9954]); remember a KEM is not a NIKE. Prevents: harvest-now-decrypt-later exposure.

Note: New signing design, free to pick the curve: Ed25519. Must use ECDSA on a NIST or secp256k1 curve: hedged RFC 6979 and a vetted constant-time multiply. Hardware or embedded signer: hedged, never bare-deterministic. Ephemeral key exchange: X25519. Static-key ECDH: full SP 800-56A validation. Certificate or PKI validation: pin named curves. Any TLS: version 1.3, and if you are stuck on 1.2 DHE, RFC 7919 groups with export ciphers off. Long-lived confidentiality: hybrid X25519 with ML-KEM.

1. **Nonce:** For every signature, can you show the nonce is both unpredictable (hedged or deterministic) *and* consumed in constant time? If you can only answer one half, you are exposed to the other. 2. **Parameters:** Does your certificate and key-exchange path reject attacker-supplied explicit parameters and validate every received point against the named curve? 3. **Negotiation:** Can a man-in-the-middle force any weaker group or version than you intend? If you cannot prove "no," assume "yes."

Notice that not one item on this list is "use a bigger curve." For every break except Logjam's downgrade, key size is irrelevant -- a 384-bit curve reuses a nonce exactly as fatally as a 256-bit one. The controls are about generation, usage, validation, and negotiation, because those are the four contracts, and the contracts are where reality broke.

Two questions decide whether you got this right, and they happen to be the two most-repeated errors in the field.

11. FAQ: Misconceptions, Precisely Corrected

Once, and only against a deliberately crippled target. Logjam computed discrete logarithms against a 512-bit export-grade Diffie-Hellman prime -- one that a man-in-the-middle downgrade could select and that many servers had in common [@logjam-weakdh-2015][@cve-2015-4000]. No strong deployed curve -- P-256, Curve25519, secp256k1 -- and no standard-sized group of 2048 bits or more has ever had its discrete log solved in public. The ceiling of the entire corpus is a 512-bit prime that policy left broken. No, and this is the most common error. A *repeated* nonce recovers the key with certainty from exactly two signatures, by elementary algebra. A merely *biased* nonce emits only a faint statistical signal per signature, so recovery takes many of them plus a dedicated lattice or Fourier attack -- about 2100 for Minerva on secp256r1 [@minerva-2020], roughly 60 for PuTTY's nine-zero-bit bias [@putty-cve-2024-31497]. One biased signature leaks almost nothing. No. LadderLeak's leak is under a single bit per signature, which lattice methods cannot exploit; it revives Bleichenbacher's Fourier-analysis solver instead [@ladderleak-2020]. The lattice regime is Minerva and TPM-Fail, whose leaks are two bits or more [@minerva-2020][@tpmfail-2020]. Calling all three "lattice attacks" collapses a distinction the attacks themselves are built on. No. Padding oracles are an RSA and CBC-mode phenomenon; there is no elliptic-curve or Diffie-Hellman padding-oracle exemplar, and you should not go looking to manufacture one. The EC and DH attack surface is exactly the three pillars here -- the nonce, parameter validation, and downgrade. RSA's padding oracles, and the fact that RSA signing has no per-signature secret nonce to reuse, are covered in the companion piece "How RSA Breaks in Real Life." Only against half the problem. RFC 6979 removes the random number generator from the signing path and eliminates the reuse class [@rfc6979], but it does nothing about a timing leak during nonce *usage* -- which is exactly how Minerva, TPM-Fail, and LadderLeak recovered keys from deterministic signers. The complete answer is hedged signing plus constant-time scalar multiplication [@fips-186-5], the two orthogonal cures. Almost never. The nonce, validation, and downgrade defects are all indifferent to key size: a forged CurveBall certificate is accepted no matter how large the curve is, because the flaw is a skipped generator check rather than a hard curve [@curveball-cve-2020-0601], and a reused or leaked nonce surrenders the private key whether that key is 256 or 384 bits. The one place size mattered was the export-group downgrade, and there the fix was to stop negotiating the 512-bit group at all [@logjam-weakdh-2015], not to enlarge it. The primitive, no. The discrete logarithm on a strong group remains as hard as it was in the 1970s. What broke, repeatedly, was the deployment around it. Fix the nonce path, validate your parameters against named curves, and retire export and shared groups, and you have closed every break in this article. CurveBall was a validation bug, not a curve break [@curveball-cve-2020-0601] -- and that distinction is the whole point.

Half a century after Diffie and Hellman reduced key exchange to one hard problem, the scoreboard is unambiguous: seven documented field breaks across three deployment layers, and a discrete-log ceiling that only ever reached a deliberately crippled, shared 512-bit export group -- never a strong deployed curve, never a standard-sized group. The mathematics kept its promise. The deployments did not.

The discrete log never fell. The nonce did, the parameters did, and the negotiation did -- and every one of those is the deployment's job, not the curve's.

For the other half of the story -- how a curve itself could be structurally weak, how the number field sieve and Shor's algorithm actually break the math -- see the structural companion, "How the Discrete Log Would Break." For the primitive that has no per-signature nonce to reuse and its own padding-oracle history, see the series sibling, "How RSA Breaks in Real Life." This article was about the gap between a hard problem and a shipped product. That gap is where the keys keep getting out.

Correct, Constant-Time, and Still Owned: A Field Guide to Side Channels, Faults, and Key Custody

noreply@paragmali.com (Parag Mali) — Tue, 14 Jul 2026 01:57:15 GMT

**Correct algorithms still leak.** Timing, cache, power, electromagnetic, speculative-execution, and induced-fault channels hand attackers keys that the mathematical proof never modeled -- and constant-time coding closes only the channels you can name. But relative to those exotic breaks, and relative to breaking the ciphers themselves, deployed systems fail most often at the least glamorous layer: key and secret management. Put the root key in hardware, use it without exporting it, envelope-wrap and auto-rotate everything else, and keep no long-lived secret sitting in software.

1. The Algorithm Was Correct. The Key Left Anyway.

In early 2024, researchers pulled private keys out of Apple's newest laptops -- not from buggy code, but from OpenSSL's Diffie-Hellman, Go's RSA, and the freshly standardized post-quantum algorithms CRYSTALS-Kyber and CRYSTALS-Dilithium, every one of them written to run in constant time [@gofetch-2024]. Twenty-eight years earlier, Paul Kocher had recovered a different private key by doing nothing more exotic than watching how long the math took [@kocher-1996]. Same lesson, two eras: the proof modeled the algorithm; the attacker measured the machine.

This is the twenty-fifth and final part of a field guide that spent twenty-four installments proving, primitive by primitive, that the mathematics holds. Each of those proofs came with a silent condition attached. RSA is secure if you blind the exponent. Elliptic-curve signatures are unforgeable if the nonce is uniform and the scalar multiplication is data-independent. AES-GCM is authenticated if you never reuse a nonce and you compare tags in constant time. Twenty-four times, the phrase "if the caller does X" carried the whole argument. This part is about what happens when that "if" meets a physical machine and a running organization.

Key idea: The math held; the machine and the operations handed the secret away. A cryptographic proof reasons about an algorithm -- an input/output relation. An attack happens to an implementation -- a physical, timed, power-drawing, speculating, operated process. The distance between those two things is the entire subject of this article.

That distance has a name, the abstraction gap, and it opens onto five surfaces where a correct algorithm sheds its secret. The first four are the exotic ones that make headlines: timing and cache channels, power and electromagnetic emanations, microarchitectural and transient-execution leaks, and induced faults. The fifth is the unglamorous one that ends most real incidents: the operational handling of keys and secrets themselves. The first four are where clever attackers earn conference papers. The fifth is where ordinary systems quietly die.

So this is not an encyclopedia of channels. It is an argument, backed by a catalog: every famous break in the record violates exactly one of four invariants -- a secret touched time, a secret touched an address, a fault skipped a check, or a key outlived or escaped its custody. Seen through those four invariants, the sprawling zoo of attacks collapses into a map that tells you where to spend your defensive budget.

To see why a correct algorithm leaks, though, you first have to see the gap the proofs never modeled. And it has a birthday.

2. Naming the Gap

Ask a cryptographer from 1990 what it means for a cryptosystem to be secure and you would get a clean answer: an adversary who sees only the inputs and outputs -- the ciphertexts, the signatures, the public keys -- cannot do better than guessing. The whole edifice of provable security, the IND-CPA and EUF-CMA definitions this series has used throughout, rests on that framing. The adversary is a mathematical object that submits queries and reads replies. It has no stopwatch, no oscilloscope, and no physical access to the box doing the computing.

That framing quietly omitted an entire dimension. A real computer does not merely map inputs to outputs. It takes time. It draws power. It radiates. It caches. It speculates. It can be made to stumble. None of those behaviors appears in the security definition, which means none of them was ever proven absent. The proof was silent about them not because they were safe, but because the model could not see them.

Information leaked by a computation's physical or microarchitectural behavior -- its running time, power draw, electromagnetic emanation, or cache state -- rather than by its defined input/output relation. A side channel is real even when the algorithm is mathematically perfect, because it lives in the machine, not the math.

There was already a Cold-War counterexample sitting in the open literature. In 1985, Wim van Eck published the first unclassified demonstration that a video display's electromagnetic emanations could be intercepted from a distance and its screen contents reconstructed with modest equipment [@van-eck-1985]. Militaries had worried about "compromising emanations" under the codename TEMPEST for decades; van Eck moved the problem into public view. The lesson was blunt: computation emits exploitable physical signals whether or not the theory chooses to look.Van Eck's result is not itself a cryptographic attack -- it reconstructs a screen, not a key -- but it is the moment compromising emanations entered the open scientific record, which is why it anchors the side-channel lineage [@van-eck-1985].

flowchart TD A["Algorithm: an input/output relation the proof reasons about"] -->|realized as| B["Implementation: a physical, timed, operated process"] A --> H["Output the proof models: ciphertext or signature"] B --> C["Leak channel: time and cache addresses"] B --> D["Leak channel: power and EM"] B --> E["Leak channel: speculation and prefetch"] B --> F["Leak channel: induced faults"] B --> G["Leak channel: key custody and secret handling"]

The eureka: time is an output

The gap got its name in 1996. Paul Kocher, then an independent cryptographer, made a single almost impudent observation: if the running time of a correct RSA or Diffie-Hellman exponentiation depends on the bits of the secret exponent, then merely timing the operation leaks the key [@kocher-1996]. No implementation bug is required. The code can be a flawless transcription of the textbook algorithm. The leak is in the fact that textbook modular exponentiation does more work for a one bit than for a zero bit.

The distance between an algorithm -- an input/output relation a proof reasons about -- and an implementation, which is a physical, timed, power-drawing, speculating, operated process. Every side-channel and key-management break in this article is a way of falling through that gap.

The canonical exponentiation loop makes the leak concrete. Square-and-multiply walks the secret exponent bit by bit: it squares at every step, and it performs an extra multiplication only when the current bit is one. The total running time therefore tracks the number of one bits, and finer measurements over chosen inputs recover the bits individually.

sequenceDiagram participant E as Exponent bit participant C as CPU participant T as Wall-clock time E->>C: current bit is 0 C->>C: square only C->>T: short step E->>C: current bit is 1 C->>C: square then multiply C->>T: longer step Note over C,T: total time tracks the number of one bits in the secret exponent

Kocher's insight was not a new algorithm. It was the naming of a missing dimension. And once you accept that wall-clock time is an output the proof forgot to model, the same argument applies to every other physical observable: power, electromagnetic field, cache occupancy, speculative footprint, the very correctness of the result. The defensive program for the next thirty years becomes a single sentence: find each channel and remove the secret's influence over it.Paul Kocher bookends this whole field. He named the timing channel in 1996, co-invented Differential Power Analysis in 1999, and two decades later was a lead author of Spectre in 2018 [@kocher-1996][@dpa-1999][@spectre-2019]. One researcher's career traces the arc from a stopwatch to speculative execution.

That program organizes the rest of this article around those four invariants -- time, address, fault, custody. Hold them in mind. If time was an output the proof ignored, so was every other physical observable, and the attackers were only getting started.

3. The First Defenses, and Their First Breaks

Here is a fact that ought to feel counterintuitive: within one year of Kocher showing you could learn a key by measuring a correct machine, two teams showed you did not even need the machine to be correct. You could make it compute wrong on purpose and read the key out of the error.

The first generation of defenses was a scramble of point countermeasures, one patch per channel. Against timing, Kocher himself proposed blinding: before the secret operation, randomize the input so its running time carries no information about the true message. For RSA decryption you pick a random $r$, multiply the ciphertext by $r^e$, decrypt, and divide out $r$. The identity that makes it work is

$$(m \cdot r^e)^d \equiv m^d \cdot r^{ed} \equiv m^d \cdot r \pmod{n},$$

so multiplying the result by $r^{-1} \bmod n$ recovers $m^d$ while the timing now depends on the random blind, not the secret. Blinding was a proposal in 1996; it became a default only after David Brumley and Dan Boneh showed in 2003 that an OpenSSL RSA key could be recovered over a network by timing decryptions, ending the comfortable assumption that timing attacks needed a local probe [@brumley-boneh-2003]. OpenSSL turned blinding on by default in response: a theoretical nicety became a shipped mitigation the moment someone proved the threat was remote.

Breaking the machine on purpose

While the timing people were blinding, the fault people arrived. In 1997, Dan Boneh, Richard DeMillo, and Richard Lipton at Bellcore showed that a single faulty RSA signature computed with the Chinese Remainder Theorem optimization lets an attacker factor the modulus with one greatest-common-divisor computation [@bellcore-1997]. The RSA-CRT speedup computes the signature modulo each prime separately and recombines the halves; corrupt just one half-computation -- a voltage glitch, a clock spike, a stray cosmic ray -- and the faulty signature $s'$ comes out right modulo one prime but wrong modulo the other. Raising it to the public exponent should return the message $m$; instead $s'^e - m$ shares exactly one of the two prime factors with $n$, so $\gcd(s'^e - m \bmod n, n)$ hands you that factor. One bad signature, and the private key falls out.

An attack that induces a computational error -- through a voltage or clock glitch, an electromagnetic or laser pulse, or a software-triggered disturbance like Rowhammer -- so that the faulty output reveals the secret. The Bellcore result is the archetype: one incorrect RSA-CRT signature factors the modulus.

The same year, Eli Biham and Adi Shamir extended the idea from public-key to symmetric ciphers with Differential Fault Analysis: inject faults during a DES computation, difference the correct and faulty ciphertexts, and the secret key emerges [@biham-shamir-1997]. The countermeasures wrote themselves in principle -- verify before you output a CRT signature, add redundancy and recompute, sense the glitch and refuse -- but each one added cost and each one had gaps.

Then, in 1999, Kocher returned with Joshua Jaffe and Benjamin Jun and opened the power rail. Simple and Differential Power Analysis showed that a device's instantaneous power draw depends on the data it switches, and that correlating power traces across many operations extracts a key even when the per-trace signal is buried in noise [@dpa-1999]. A smartcard running AES or RSA was leaking its key through its power pins to anyone with an oscilloscope and patience.

A statistical attack that correlates a device's measured power consumption across many operations with a hypothesized intermediate value, extracting the key even under heavy noise. Its countermeasures are masking (split secrets into random shares) and hiding (flatten or randomize the power signal).

The pattern is the argument. Each attack bred a defense -- blinding, verify-before-output, masking and hiding -- and each defense's gaps bred the next attack. This is whack-a-mole, and playing it one mole at a time was already visibly losing. Three structural cracks doomed the patch-by-patch approach: the channels were multiplying faster than the patches, each patch protected one primitive rather than the class, and the deepest crack was about to open -- the leak was preparing to jump from the value of the secret to the addresses the secret makes the processor touch.

There is a third way implementations betray correct algorithms that this article names but does not re-derive: the [random-number generator](/blog/predictable-or-repeated-the-only-two-ways-cryptographic-rand/). The 2008 Debian OpenSSL disaster (a commented-out line collapsed key entropy to a few thousand possibilities) [@debian-openssl-2008], the 2010 PlayStation 3 ECDSA break (a reused signing nonce) [@ps3-ecdsa-2010], and 2017's ROCA (a structured-prime library flaw) [@roca-2017] are all implementation and operational failures, not broken math. They belong to the randomness and [nonce-reuse](/blog/one-number-used-twice-how-a-repeated-nonce-hands-over-your-p/) story told in Parts 2 and 8 of this series. Keep them in view as a fourth failure family; we return to their operational cousin -- key custody -- as this article's payoff.

4. The Channels Multiply

The next twenty years were a single sentence repeated in new dialects: the secret leaked through a channel the proof never modeled. What changed each time was the channel. Here is the whole family tree, and the timeline that carries it.

timeline title Thirty years of the abstraction gap 1985 : van Eck EM emanations 1996 : Kocher timing attack 1997 : Bellcore RSA-CRT fault : Biham-Shamir DFA 1999 : Kocher-Jaffe-Jun DPA 2003 : Brumley-Boneh remote timing 2005 : Bernstein AES cache : Percival hyperthreading 2013 : Lucky Thirteen breaks TLS 2014 : Yarom-Falkner Flush and Reload : Rowhammer 2018 : Meltdown and Spectre 2020 : Minerva : TPM-Fail 2022 : Hertzbleed via DVFS 2024 : GoFetch : KyberSlash

From the value to the address

The deepest move in the whole story is the one that happened between 2005 and 2014: the leak stopped being about the value of the secret and became about the addresses the secret makes the processor touch. Daniel Bernstein's 2005 cache-timing attack on AES showed that software using secret-dependent table lookups -- the standard fast AES implementation, indexing precomputed T-tables by key-dependent bytes -- leaks the key through cache timing, because which table entries end up cached depends on which ones the victim accessed [@bernstein-2005].

The same year, Colin Percival showed that Intel Hyper-Threading's shared L1 cache lets one thread spy on a co-resident thread's access pattern, recovering RSA key bits from a concurrent OpenSSL process [@percival-2005]. In 2006, Dag Arne Osvik, Adi Shamir, and Eran Tromer named and formalized the reusable primitives the field still uses [@osvik-shamir-tromer-2006]: Prime+Probe fills (primes) a cache set with the attacker's own lines, lets the victim run, then re-accesses (probes) those lines and times its own reload -- a slow reload reveals the victim evicted a line, so it needs no shared memory, while Evict+Time measures the victim directly.

A leak that recovers which memory addresses a victim touched by measuring shared-cache timing -- a hit is fast, a miss is slow. Flush+Reload is the sharpest variant: an attacker flushes a shared line, waits, then times the reload to learn whether the victim accessed it in the interval.

The technique reached its high-resolution form in 2014, when Yuval Yarom and Katrina Falkner delivered Flush+Reload: flush a shared cache line with clflush, wait, then time the reload to learn whether the victim touched it, all through the last-level cache shared across cores [@flush-reload-2014]. It recovered GnuPG RSA keys across cores and became the workhorse primitive behind the transient-execution attacks that followed. The countermeasure had crystallized into a principle, not a patch: never let a secret decide which address you touch.

From local to remote, and a flaw in the spec

Parallel to the address story ran the reach story. Brumley and Boneh had already made timing a network threat in 2003. Then, in 2013, Nadhem AlFardan and Kenneth Paterson published Lucky Thirteen, and it stung in a new way: the non-constant-time MAC-and-padding check in TLS and DTLS CBC-mode record processing is a timing oracle, and a man-in-the-middle can recover plaintext from it [@lucky13-2013]. The wound was not one library's bug. The TLS specification mandated a MAC-then-pad-then-encrypt construction whose safe verification is genuinely hard to do in constant time. This was a spec-level flaw -- a reminder that "write it in constant time" is sometimes fighting the protocol design itself.

Below the instruction set

For a decade the discipline looked like it might be enough. Then, on 3 January 2018, the floor gave way. Meltdown and Spectre showed that transient execution -- the out-of-order and speculative machinery every fast CPU uses -- runs secret-dependent code paths that the architectural program never takes, and leaks their traces through the cache [@meltdown-2018][@spectre-2019][@projectzero-2018]. Meltdown let an unprivileged process transiently read kernel memory. Spectre poisoned branch prediction so that even correct, constant-time code speculatively executed a secret-dependent path and left it in the cache for a Flush+Reload probe to recover.

Instructions a CPU runs speculatively or out of order and then discards architecturally, so they never affect the program's official result -- but whose microarchitectural traces, chiefly cache state, survive and leak. Spectre and Meltdown proved that constant-time source code can be defeated by what the silicon does beneath it.

This was the qualitative break. Constant-time is a property of the instruction stream the programmer wrote. Speculation executes an instruction stream the programmer did not write, chosen by a branch predictor the attacker can train. The abstraction did not spring a leak; it was breached from underneath.Minerva (2020) is the miniature of the whole pattern: many ECDSA implementations leak the bit-length of the per-signature nonce during a non-constant-time scalar multiplication. Gather a couple thousand signatures, feed the tiny leaks to a lattice attack, and the long-term key falls out. On an Athena IDProtect smartcard the team needed about 2,100 signatures and recovered a P-256 key in minutes [@minerva-2020].

The machine keeps inventing channels

If Spectre broke constant-time from below through speculation, the next wave broke it from the side, through physics the instruction set never promised to hold still. In 2022, Yingchen Wang, Riccardo Paccagnella, and colleagues published Hertzbleed: modern x86 dynamic voltage and frequency scaling makes the CPU's clock frequency depend on the data being processed, so a program's wall-clock time depends on its data values -- turning a power side channel into a remote timing channel with no power measurement at all [@hertzbleed-2022]. They extracted a full key from a constant-time implementation of the post-quantum candidate SIKE, remotely.SIKE died twice. Hertzbleed broke its constant-time implementation in 2022 through frequency scaling [@hertzbleed-2022]. Weeks later, Wouter Castryck and Thomas Decru broke the mathematics of SIDH outright with a classical key-recovery attack, and SIKE was withdrawn [@castryck-decru-2023]. The math break is out of this article's implementation scope, but the coincidence is a useful reminder that a primitive can fall from either direction.

Then, in 2024, came the break that opens this article. Apple's M-series CPUs contain a Data Memory-dependent Prefetcher (DMP) that inspects values loaded from memory and, when one "looks like" a pointer, dereferences it speculatively to prefetch what it points to [@gofetch-2024]. That behavior is precisely the data-to-address coupling constant-time forbids. By crafting chosen inputs so that an intermediate value looks like a pointer only when a guess about a key bit is correct, the GoFetch team achieved end-to-end key extraction from constant-time OpenSSL Diffie-Hellman, Go RSA, and -- the part that should worry every protocol designer -- the post-quantum CRYSTALS-Kyber and CRYSTALS-Dilithium.

sequenceDiagram participant A as Attacker participant M as Victim memory participant DMP as Data-memory prefetcher participant Cache as Cache state A->>M: craft input so a value looks pointer-like only when the key-bit guess is right M->>DMP: value flows through a constant-time load DMP->>DMP: value resembles a pointer, so dereference it DMP->>Cache: prefetch pulls the pointed-to line into the cache A->>Cache: time the reload to confirm the guess Note over A,Cache: data influenced an address, the exact coupling constant-time forbids

The same year, KyberSlash showed the discipline is unfinished in a humbler way. Several Kyber implementations contained a single line that divides a secret numerator by a public denominator; because integer division takes a data-dependent number of cycles on many CPUs, that one statement leaks ML-KEM secret bits into timing, sometimes exploitably [@kyberslash-2024]. Not an exotic prefetcher -- just a division, in a freshly standardized algorithm, that no one flagged.

The fault lineage, in parallel

Running alongside the leakage story the whole time was the fault story, and it too grew more remote. Classical fault injection meant physically glitching a chip with voltage, clock, electromagnetic, or laser pulses. In 2014, Rowhammer changed the threat model: repeatedly activating one DRAM row flips bits in physically adjacent rows without accessing them, a hardware fault triggered entirely in software [@rowhammer-2014]. Plundervolt then showed you could undervolt an Intel CPU from software to induce faults inside an SGX enclave, corrupting cryptographic computations from the operating system [@plundervolt-2019]. The fault, like the leak, had escaped the need for physical possession.

One lens for the whole catalog

Set every named break side by side and re-read each as exactly one violated invariant. The zoo becomes a table.

Break (year)	Surface / channel	Root cause	Violated invariant	Lesson
Kocher timing (1996) [@kocher-1996]	Timing	Secret-dependent modular-exponentiation time	secret touched time	time is an unmodeled output; blind the operand
Bellcore RSA-CRT (1997) [@bellcore-1997]	Fault	One faulty CRT signature, then a single GCD factors n	fault skipped a check	verify before output
DFA (1997) [@biham-shamir-1997]	Fault	Fault plus differencing recovers the DES key	fault skipped a check	redundancy and verification
DPA (1999) [@dpa-1999]	Power	Data-dependent power draw over many traces	secret touched power	masking and hiding
Brumley-Boneh (2003) [@brumley-boneh-2003]	Remote timing	OpenSSL RSA decryption timing over a network	secret touched time, remotely	blinding on by default
Bernstein AES cache (2005) [@bernstein-2005]	Cache	Secret-indexed T-table lookups	secret touched an address	constant-time, AES-NI, bitslicing
Percival (2005) [@percival-2005]	Cache under SMT	Shared L1 under Hyper-Threading	secret touched an address	co-residency threat model
Lucky Thirteen (2013) [@lucky13-2013]	Timing	Non-constant-time MAC-and-pad check in TLS CBC	secret touched time	constant-time MAC verification, a spec fix
Flush+Reload (2014) [@flush-reload-2014]	Cache	Flush plus reload on shared last-level cache	secret touched an address	high-resolution shared-cache primitive
Rowhammer (2014) [@rowhammer-2014]	Fault, software-triggered	DRAM disturbance flips adjacent-row bits	fault skipped a check, no physical access	software can induce hardware faults
Meltdown and Spectre (2018) [@meltdown-2018][@spectre-2019]	Transient execution	Speculation runs secret-dependent paths, leaks via cache	secret touched an address below the ISA	constant-time broken beneath the ISA
Minerva (2020) [@minerva-2020]	Timing	Non-constant-time ECDSA nonce bit-length plus lattice	secret touched time	constant-time scalar multiplication
TPM-Fail (2020) [@tpm-fail-2020]	Timing	Secret-dependent ECDSA time in a certified TPM	secret touched time	certified is not side-channel-safe
Hertzbleed (2022) [@hertzbleed-2022]	Power to remote timing via DVFS	Frequency scaling makes wall-time data-dependent	secret touched time, remotely	power channels can go remote
GoFetch (2024) [@gofetch-2024]	Cache via DMP prefetcher	Prefetcher dereferences pointer-like data	secret touched an address below the ISA	constant-time unfinished, reaches PQC
KyberSlash (2024) [@kyberslash-2024]	Timing via division	Secret numerator over public denominator, variable-time divide	secret touched time	constant-time incompletely applied to ML-KEM
Storm-0558 (2023) [@csrb-storm0558]	Operational key custody	Unrotated, software-resident 2016 signing key	a key outlived and escaped its custody	custody, not math

Note: Every row above reduces to one of four sentences. A secret touched time. A secret touched an address. A fault skipped a check. A key outlived or escaped its custody. This is not a mnemonic; it is the map. Each invariant names both the break and the defense that closes it, and the last one -- custody -- is where the whole article is heading.

You can only close the channels you model, and the machine keeps inventing new ones.

Read the catalog through those four invariants and the pattern snaps into focus. Three of the four -- time, address, fault -- are the exotic surfaces, and they share one defense strategy. The fourth is different in kind, and it is where deployed systems actually break. That split is the breakthrough.

5. Two Breakthroughs That Arrive Together

After a decade of one-patch-per-channel, the field found the single rule that closes an entire family of channels at the source. And in almost the same breath, it learned the rule was necessary but not sufficient -- which forced a second, operational breakthrough to sit beside the first. The two arrive together because each is the other's admission of limits.

The software breakthrough: constant-time as a discipline

State it precisely, because the precision is the whole point. Constant-time code has no secret-dependent branches, no secret-dependent memory indices, and no data-dependent variable-time instructions. That is it. Crucially -- and this is the single most misunderstood idea in the field -- constant-time does not mean fixed-duration execution. It means data-independent execution: the same instruction stream and the same sequence of memory addresses regardless of the secret's value. A routine can finish in wildly different absolute times on different inputs and still be constant-time, so long as the variation never depends on a secret.

Code whose instruction sequence and memory-address trace do not depend on secret values. It is NOT fixed-duration execution. The rule has three clauses: no branch on a secret, no memory index computed from a secret, and no variable-latency instruction (early-exit compare, data-dependent division) applied to a secret.

In practice the discipline replaces three forbidden patterns. A secret-dependent if becomes a branchless select computed with masks and arithmetic. A secret-indexed table lookup, table[secret], becomes a full scan of the table with a constant-time select, or is dissolved entirely by bitslicing the algorithm into boolean operations that touch no data-dependent address [@bernstein-2005]. A variable-time operation on a secret -- an early-exit memcmp on a MAC, a division with a secret numerator -- is banned outright. Whole primitives were redesigned to make the discipline natural: Curve25519's Montgomery ladder runs the identical steps for every bit of the scalar, so there is no secret-dependent path left to leak.

The comparison case is worth seeing in code, because it is the exact mistake behind Lucky Thirteen and a hundred MAC-verification bugs.

{` // Early-exit compare: returns as soon as two bytes differ. // The iteration count reveals how long a prefix matched -- a timing leak. function earlyExitEqual(a, b) { if (a.length !== b.length) return false; for (let i = 0; i < a.length; i++) { if (a[i] !== b[i]) return false; // stops at the first difference } return true; }

// Constant-time compare: always scans every byte, accumulating differences. // Running time depends only on length, never on the secret contents. function constantTimeEqual(a, b) { if (a.length !== b.length) return false; let diff = 0; for (let i = 0; i < a.length; i++) { diff |= a[i] ^ b[i]; // no early exit, no branch on the data } return diff === 0; }

const tag = [1, 2, 3, 4, 5, 6, 7, 8]; const nearMiss = [1, 2, 3, 4, 5, 6, 7, 9]; // matches 7 of 8 bytes const farMiss = [9, 2, 3, 4, 5, 6, 7, 8]; // differs at the very first byte // earlyExitEqual iterates 8 times for nearMiss but only once for farMiss: // that difference is the leak an attacker times to forge a tag byte by byte. console.log('constant-time, near miss:', constantTimeEqual(tag, nearMiss)); console.log('constant-time, far miss :', constantTimeEqual(tag, farMiss)); `}

The other half of the breakthrough is that the property became checkable, and even provable. Oscar Reparaz and colleagues built dudect, which runs a routine on fixed-versus-random secret inputs and applies a Welch t-test: a stable t-statistic past the usual threshold flags a data-dependent timing leak, while a genuinely constant-time routine stays flat [@dudect-2017]. On the formal side, ct-verif encodes constant-timeness as a property over pairs of executions differing only in secrets and discharges it mechanically [@ct-verif-2016]. Constant-time stopped being a matter of the author's vigilance and became something a machine could test in CI or prove outright.

For the physical surface -- power, electromagnetic, glitch -- the corresponding discipline is masking, and it too became principled rather than ad hoc.

Masking splits every secret intermediate into $d+1$ random shares so that any $d$ probed values are statistically independent of the secret; the computation runs on shares. Threshold implementations add non-completeness and uniformity so the construction stays secure even under hardware glitches, which transiently leak the unmasked value [@threshold-impl-2006].

But masking closes only the physical channel, and constant-time closes only the modeled ISA-level channels. Spectre, Hertzbleed, GoFetch, and KyberSlash all live below the abstraction the discipline reasons about. You can be perfectly constant-time in the instruction stream you wrote and still leak, because the silicon runs an instruction stream you did not write. That is the wall the software breakthrough hits -- and it is the reason for the second breakthrough.

The operational breakthrough: key custody as an architecture

If you can never prove the whole machine leak-free, stop trying to eliminate leakage and instead bound what a leak can cost. That is the pivot. Concretely: put the root key inside a hardware boundary that generates and uses it without ever exporting it, and reach your data through an envelope.

A tamper-resistant boundary -- a dedicated appliance, a cloud Managed HSM, a TPM, or an embedded secure element -- that generates and uses keys without releasing them in plaintext. Software sends in a request (sign this, unwrap that) and gets back a result, never the key bytes. FIPS 140-3 validates such modules, including some non-invasive side-channel requirements. A pattern in which a cheap Data-Encryption Key (DEK) encrypts the bulk data, and a Key-Encryption Key (KEK) that never leaves the HSM or KMS wraps the DEK. The wrapped DEK is stored right beside the ciphertext; only the KMS can unwrap it. Standardized authenticated key wrapping (NIST SP 800-38F) supplies the wrap operation [@nist-sp800-38f].

The economics are what make it deployable. A hardware module is slow and expensive, so you do not run all your data through it -- you run one small wrap and unwrap through it, and let a cheap symmetric DEK do the bulk work. A leaked DEK exposes only the data it wrapped; the KEK, and therefore every other DEK, stays safe inside the boundary. Rotation becomes automatic and its blast radius bounded. This is the move from "make the code not leak" to "make a leak not matter," and it maps directly onto how NIST SP 800-57 tells you to think about key lifecycles and crypto-periods [@nist-sp800-57].

The junction

The two breakthroughs are two chains of the same story, and they meet at one move. Chain A drives the code toward not leaking, generation by generation, until it hits the microarchitectural wall. Chain B accepts that wall and shrinks what any residual leak is worth. The thesis of this whole series is the junction of the two: the math held; Chain A closes the channels you can model, and Chain B bounds the cost of the ones you cannot.

flowchart TD A0["A0: correctness is enough"] --> A1["A1: point patches, blinding, verify, masking"] A1 --> A2["A2: constant-time discipline"] A2 --> A3["A3: verified plus hardware-assisted constant-time"] B0["B0: keys as data, hardcoded or in config"] --> B1["B1: OS or software keystore"] B1 --> B2["B2: TPM, HSM, secure element"] B2 --> B3["B3: cloud KMS, envelope, automatic rotation"] A3 --> T["Thesis: the math held. Close the modeled channels, bound the cost of the rest"] B3 --> T

Key idea: Defense evolves from "make the code not leak" to "make a leak not matter." The first is Chain A and it is necessary. The second is Chain B and it is what saves you when Chain A inevitably has a gap the machine invented last week.

Both breakthroughs are engineering realities you can deploy today: verified constant-time libraries on one side, cloud KMS with envelope encryption and automatic rotation on the other. So what does the 2024 to 2026 state of the art actually look like when you ship both at once -- and where is it still losing?

6. The State of the Art, 2024 to 2026

There is no single best implementation. There are two parallel frontiers, and the 2024-to-2026 practitioner ships both at once. Call them Chain A, leakage defense, and Chain B, key custody.

Chain A: the leakage-defense frontier

At the base sits the constant-time-by-construction library. libsodium, Google's BoringSSL, Thomas Pornin's BearSSL, and Rust's ring obey the three-clause rule everywhere a secret is touched. This is table stakes now, not a differentiator -- if a general-purpose crypto library is not constant-time by default in 2026, it is not a serious library.

Above that sits the highest-assurance floor: formally verified constant-time crypto, where a machine-checked proof replaces human review.

Cryptographic code whose secret-independence is established by a machine-checked mathematical proof, not by review or testing. HACL* and EverCrypt (written in a subset of F*) and the Jasmin and libjade toolchain (with EasyCrypt) are the two production lineages; the proof itself is the constant-time argument, discharged once and re-checked automatically [@hacl-star].

This is not a laboratory curiosity. HACL* ships verified Curve25519 in Mozilla Firefox through NSS, and a formally verified Curve25519 rode into the Linux kernel alongside WireGuard; verified SHA-3 and HMAC from the same project ship inside CPython [@hacl-star]. Proof-carrying crypto is in the browser and the operating system you are reading this on.

Verifying and testing the property is its own layered toolchain, and no single tool dominates. A 2023 systematic evaluation quantified the trade-offs across real libraries [@sidetool-eval-2023]; the shape of the field is a ladder from cheap-and-empirical to expensive-and-proven.

Tool	Technique	Level analyzed	Guarantee	Catches compiler-introduced leaks	Effort	Microarchitecture
dudect [@dudect-2017]	Welch t-test on timing	Black-box, runtime	Empirical, tested paths	Only if exercised	Very low, CI drop-in	Not modeled
ctgrind / TIMECOP	Dynamic taint over Valgrind	Binary, dynamic	Empirical, executed paths	Yes, when observed	Low	Not modeled
ct-verif [@ct-verif-2016]	Formal self-composition	LLVM IR	All paths at the IR	No, runs before compilation	Medium, needs annotations	Not modeled
Binsec/Rel [@binsec-rel]	Relational symbolic execution	Final binary	All paths, bounded depth	Yes	High, setup and path explosion	Not modeled
HACL* / Jasmin [@hacl-star]	Proof in F* or EasyCrypt	Source down to C or assembly	Whole-program theorem	Verified assembly, yes	Very high, proof engineering	Not modeled

Read the last column: every tool shares the same blind spot. None models the microarchitecture, because none can -- a limit we return to in Section 8. The empirical tools like dudect are the ones you actually gate CI with, and the Welch t-test at their heart is simple enough to see in a few lines.

{// Simulate timing samples for two input classes: a "fixed" secret and "random" secrets. // A variable-time operation reacts differently to the two classes; a constant-time one does not. function mean(xs) { return xs.reduce((s, x) => s + x, 0) / xs.length; } function variance(xs, m) { return xs.reduce((s, x) => s + (x - m) * (x - m), 0) / (xs.length - 1); } // Welch t-statistic for two samples with unequal variance. function welchT(a, b) { const ma = mean(a), mb = mean(b); const va = variance(a, ma), vb = variance(b, mb); return (ma - mb) / Math.sqrt(va / a.length + vb / b.length); } function samples(n, base, classShift) { const out = []; for (let i = 0; i < n; i++) out.push(base + Math.random() * 5 + classShift); return out; } const N = 5000; // Variable-time op: the random class runs a few units longer on average, a real leak. const leakFixed = samples(N, 100, 0), leakRandom = samples(N, 100, 8); // Constant-time op: both classes share one distribution, no leak. const ctFixed = samples(N, 100, 0), ctRandom = samples(N, 100, 0); console.log('variable-time absolute t:', Math.abs(welchT(leakFixed, leakRandom)).toFixed(1), 'past 4.5 threshold, flagged'); console.log('constant-time absolute t:', Math.abs(welchT(ctFixed, ctRandom)).toFixed(1), 'below 4.5, clean');}

Beneath the software sits hardware backing. Intel's Data Operand Independent Timing Mode (DOITM) and Arm's Data Independent Timing (DIT) bit promise that an enumerated set of instructions runs with data-independent latency, re-establishing at the silicon level the guarantee the source could only assert [@intel-doitm][@arm-dit]. But their coverage is exactly their limit. The lists are vendor-specific and enumerated, and they pointedly exclude integer and floating-point division and square root -- which is why KyberSlash's secret division leaks -- and they do not model the prefetcher or frequency scaling, which is why GoFetch and Hertzbleed operate outside their guarantee entirely [@kyberslash-2024][@gofetch-2024].

For the physical surface -- power, electromagnetic, glitch -- masking and threshold implementations plus on-die fault sensors remain the answer, and the live research frontier is masking the post-quantum algorithms at practical cost.

Note: Reach for a verified or constant-time-first library -- libsodium, BoringSSL, BearSSL, ring, or the verified HACL* and libcrux -- and gate it in CI with dudect or TIMECOP [@hacl-star][@dudect-2017]. Home-rolled constant-time code is wrong until a tool says otherwise, and even expert-written code needs re-checking after every compiler upgrade, because optimizers reintroduce the branches you removed.

Chain B: the key-custody frontier

The operational state of the art is cloud KMS plus envelope encryption plus automatic rotation. AWS KMS, Azure Key Vault and Managed HSM, and Google Cloud KMS all keep a small number of Key-Encryption Keys inside a hardware boundary and wrap an unbounded number of cheap Data-Encryption Keys under them, using the authenticated key-wrapping of NIST SP 800-38F and the lifecycle discipline of NIST SP 800-57 [@nist-sp800-38f][@nist-sp800-57].AWS puts a concrete number on the crypto-period: keys that AWS manages rotate on a required schedule of every year, approximately 365 days, and customer-managed keys support optional automatic rotation on a period you choose [@aws-kms]. Rotation is O(1) to request; the hard part is re-wrapping and inventorying what each key protects.

For root keys -- certificate authorities, code-signing, payment keys -- the anchor is a FIPS 140-3 validated module.

The current U.S. standard for validating cryptographic modules, superseding FIPS 140-2. Unlike its predecessor's reputation, FIPS 140-3 explicitly addresses non-invasive security -- that is, side-channel resistance -- among its requirements, across four ascending security levels with increasing physical tamper response [@fips-140-3].

The third pillar is the least technical and the most often fatal: secrets management and non-human-identity hygiene. Application secrets -- API keys, database credentials, tokens, signing keys -- must stay out of code, config files, container images, and CI logs. OWASP made cryptographic-usage and keying failures the number-two web-application risk in 2021, and the 2025 OWASP Non-Human Identities Top 10 put improper offboarding (NHI1) and secret leakage (NHI2) at the very top of the list, with long-lived secrets also named (NHI7) [@owasp-a02-2021][@owasp-nhi-2025]. The tooling exists -- secrets managers that issue short-lived attested credentials, CI secret-scanning, workload federation -- but it demands organizational discipline no library can supply.

The concrete arc: Storm-0558 to Managed HSM

The clearest picture of Chain B's state of the art is the failure that forced it and the remedy that answered it. In 2023, the threat actor Storm-0558 forged Microsoft consumer authentication tokens using a 2016 signing key that was software-resident, held outside any HSM, and had never been rotated; a token-validation flaw let that consumer key mint enterprise tokens, and the actor downloaded roughly 60,000 State Department emails (about ten accounts), part of an intrusion that reached around two dozen organizations [@csrb-storm0558][@msrc-storm0558]. The U.S. Cyber Safety Review Board judged the intrusion preventable. No cipher was broken. A key simply outlived and escaped its custody.

Microsoft's remedy, under its Secure Future Initiative, is the Chain B playbook made concrete: the MSA and Entra ID signing keys moved into Azure Managed HSM with automatic rotation, and the signing service itself moved into confidential-computing VMs [@microsoft-sfi-2024]. Put the root key in hardware, use it without exporting it, rotate it automatically. That is the entire prescription, and it is exactly what was missing in 2016.

The honest caveat closes the section. Hardware custody is not a magic floor. A 2025 taxonomy catalogs how cloud HSM and TPM custody actually fails in practice, and TPM-Fail already showed that certified modules leak keys through timing [@hsm-tpm-failures-2025][@tpm-fail-2020]. Hardware moves the trust boundary; it does not remove it.

A few commands turn the abstract advice into an audit. Confirm a KMS key actually rotates with `aws kms get-key-rotation-status --key-id KEY_ID`. List which principals can call decrypt on it by reading the key policy with `aws kms get-key-policy --key-id KEY_ID --policy-name default`. Scan a repository's history for committed secrets with `gitleaks detect --source .` or `trufflehog filesystem .`. Gate a C library's tag comparison in CI by building it under `valgrind` with the TIMECOP macros, or by wiring a dudect harness around the routine. None of these is exotic; all of them catch the mistakes that end real incidents.

You now know the tools on both frontiers. The hard part is the two decisions nobody hands you: where do you put the defense, and where do you put the key?

7. Where Do You Put the Defense, and Where the Key?

Every choice in this field trades assurance against cost, portability, and threat model. Two decision axes organize the whole thing, and the good news is that within each axis the options mostly compose rather than compete.

Axis one: defense placement

The four leakage defenses are not rivals fighting for one slot. They stack. Ship verified constant-time as the floor, enable hardware data-independent-timing as backing, and add masking only when the adversary can physically hold and instrument the device.

Approach	Assurance	Runtime cost	Portability	Defends against	Blind to	Best suited for
Software constant-time	Review-level	Near-constant factor	High, any CPU	Timing and cache	Microarchitecture: Spectre, DVFS, DMP	Every deployment, the floor
Verified constant-time [@hacl-star]	Machine-checked proof	Same as software	High, generated C	Timing and cache, proven	Below the verified boundary; microarchitecture	Long-lived high-assurance primitives
Hardware DIT / DOITM [@intel-doitm][@arm-dit]	Vendor guarantee, listed instructions	Near zero for covered instructions	Low, vendor-specific	Operand timing of listed instructions	Prefetcher, frequency scaling, division	Defense-in-depth beneath software
Masking / threshold [@isw-2003]	Provable order-d in the probing model	Quadratic in the order	Hardware-oriented, costly in software	Power, EM, glitch faults	Higher-order analysis; cost	Physical-possession adversary

The table's lesson is the composition, not any single winner. Each defense covers what the one above it cannot; masking, in particular, is the only thing that helps once an attacker owns the silicon, and you pay for it by the square of the order. None of them closes the microarchitectural channels alone -- which is the structural reason Chain B has to exist.

Axis two: the key-custody spectrum

Custody is a ladder, and the single most useful skill in this whole article is matching the rung to the adversary and the asset value.

Rung	Key location	Exportable?	Blast radius on host compromise	Cost / latency	Use when	Anchor
0	Hardcoded, config, env var, repo	Yes, in plaintext	Total, the key is the leak	None	Never, an anti-pattern	Storm-0558 [@csrb-storm0558]
1	OS or software keystore	Yes, to root or admin	High	Low	Low-value single-host secrets	OWASP A02 [@owasp-a02-2021]
2	TPM or secure element	No, use without export	Low, but see TPM-Fail	Low to medium	Device-bound keys, measured boot	TPM-Fail [@tpm-fail-2020]
3	Cloud KMS plus envelope	KEK no, DEK wrapped	Bounded to unwrapped DEKs	One KMS round-trip per DEK	Data at rest at scale	AWS KMS [@aws-kms]
4	Dedicated or Managed HSM	No	Very low	Higher	Root, CA, code-signing, payment keys	FIPS 140-3 [@fips-140-3]
5	Threshold or MPC custody	No single holder	Requires several compromises	Highest, protocol rounds	No single point of trust, see Part 22	NIST SP 800-57 [@nist-sp800-57]

Here is the capstone's punchline, sitting in one table. The exotic Chain-A attacks -- a timing leak against a certified TPM at rung 2, say -- are real but rare. The overwhelming majority of real-world breaks cluster at rungs 0 and 1: keys hardcoded, sprawled across config and CI logs, software-resident, and never rotated. Storm-0558 was a rung-0 failure with a nation-state on the other end. The state-of-the-art move for almost everyone is simply to climb to rung 3 or 4 with automatic rotation, and to accept the custody regress that waits at the top.

Note: A key in source, in a config file, in an environment variable baked into an image, or in a repository is not "low assurance." It is the leak, waiting for any code, log, or memory disclosure to complete it. There is no asset value low enough to justify rung 0. Move to a secrets manager and rotate.

It is tempting to read a Common Criteria or FIPS certificate as "side-channel-proof." It is not. TPM-Fail recovered 256-bit ECDSA keys from an Intel firmware TPM and from a Common Criteria EAL4+ and FIPS 140-2 Level 2 certified STMicroelectronics chip, in minutes locally and about five hours across a network [@tpm-fail-2020]. Minerva broke FIPS-certified smartcards from a couple thousand signatures [@minerva-2020]. Certification validates a *modeled* threat surface against a defined protection profile; it bounds process, not physics. When the modeled surface omits the channel the attacker uses, the certificate is silent. "Certification has failed to protect the product against an attack that is considered by the protection profile." -- Moghimi et al., the TPM-Fail disclosure [@tpm-fail-2020]

These tables tell you what to reach for. But can leakage ever be eliminated, or are we only ever bounding its cost? The honest answer is uncomfortable, and it comes in two parts.

8. Can Leakage Be Eliminated?

The field owes you two very different answers. One is a clean, provable bound. The other is an honest impossibility that is not a theorem at all but a structural fact about how computers are specified.

The masking cost curve is a real bound, and it is not a wall

For the physical channel, the mathematics is unusually crisp. In the Ishai-Sahai-Wagner probing model, protecting a secret against an adversary who can observe up to $d$ internal wires requires at least $d+1$ random shares, and that bound is tight -- you cannot buy order-$d$ security with fewer [@isw-2003]. Securely masking a single multiplication in that model costs $O(d^2)$ in circuit size and randomness. So security grows linearly in the order while cost grows quadratically: doubling your resistance roughly quadruples the work.

Under a physically realistic noisy-leakage model, the picture gets better in one specific way. Duc, Dziembowski, and Faust proved that masking's security amplifies exponentially in the number of shares, provided each share carries enough noise, by reducing noisy leakage back to the clean probing model [@ddf-2014]. Put the two results together and you get the defining shape of the whole physical-defense story: leakage is a cost curve, not a wall. You can push the attacker's required number of traces arbitrarily high by paying $O(d^2)$ and ensuring noise. You can never push it to zero.

Whole-machine constant-time is an impossibility, not a theorem

For the timing and cache channels, there is no comparable happy bound, and the reason is almost embarrassingly simple. Constant-time cannot be guaranteed at the microarchitectural level because the instruction set architecture does not specify timing. A constant-time proof is a statement about ISA-visible behavior. The silicon underneath is free to introduce data-dependence the ISA never promised to forbid, and it repeatedly does.

Key idea: The ISA does not specify timing, so a constant-time proof only constrains architecture-visible behavior. You can only close the channels you model, and the set of channels is open-ended and adversary-discovered. Speculation, frequency scaling, and data-memory-dependent prefetching each reopened it after the proof was done.

This is why Spectre, Hertzbleed, GoFetch, and KyberSlash are not a series of fixable bugs but a series of demonstrations of the same structural gap. Each exploited a microarchitectural behavior -- speculation, DVFS, a prefetcher, variable-time division -- that no source-level or even ISA-level guarantee covered. There is no proof that a real machine leaks nothing through timing, and, because the ISA is silent on timing, none is possible with today's hardware contracts. The gap is not small and closing. It is structurally open until a complete, cross-vendor data-independent-timing contract exists, which DIT and DOITM only partially provide.

The custody regress

The operational half has its own impossibility, and it is the reason this article lands on key custody rather than on some final leak-proof coprocessor. Someone must ultimately hold a root secret. Hardware roots -- HSMs, TPMs, Managed HSMs -- move the trust boundary inward and defend it far better, but they do not remove it, and that smaller boundary has its own documented failure modes, from TPM-Fail's timing leak to the 2025 taxonomy of cloud HSM and TPM failures [@tpm-fail-2020][@hsm-tpm-failures-2025]. There is no keyless bottom. There is only a smaller, better-defended custodian, and certification of that custodian is necessary but never sufficient.

Hardware moves the trust boundary; it does not remove it.

So perfection is off the table on both halves. What replaces it is not despair but a different target.

Note: The honest goal is not zero leakage and not an unbreakable key. It is cost-optimality: make leakage so expensive to exploit, and each secret so short-lived and tightly bounded, that no leak is worth the attacker's effort. Masking raises the price of a power trace; short key lifetimes shrink the value of any single leaked bit; envelope encryption bounds the blast radius. You are not building a wall. You are pricing the attacker out.

The purely theoretical program of leakage-resilient cryptography pushes on the same questions from the model side, but relative to deployed masking and hardware custody it remains largely academic. If perfection is unreachable, the frontier turns into a list of concrete engineering fights -- and, refreshingly, they are winnable one channel at a time.

9. The 2026 Frontier

Name the open problems precisely and the field stops looking like whack-a-mole and starts looking like a research agenda. Each of these is calibrated engineering, not undetectable magic.

A complete microarchitectural timing contract. DIT and DOITM enumerate data-independent instructions, but they are vendor-specific and exclude division, square root, the prefetcher, and frequency scaling [@intel-doitm][@arm-dit]. Every constant-time proof is therefore conditional on an unmodeled microarchitecture, which is exactly the seam GoFetch and Hertzbleed pried open. The open problem is an ISA-level guarantee that extends a constant-time proof below the ISA, across vendors.

Speculative constant-time at scale. Spectre-class transient execution breaks architecturally constant-time code from beneath [@spectre-2019]. Retpoline, LFENCE barriers, and Speculative Load Hardening are point defenses with real, often double-digit-percent overhead [@llvm-slh]. A scalable, low-cost compiler that guarantees data-independence even under speculation, for whole libraries, does not yet exist.The community conjecture is that without hardware speculation-taint tracking, a software-only speculative-constant-time pass cannot be both complete and cheap. You either pay in overhead or accept coverage gaps.

Whole-library verification through the compiler. Compiler lowering reintroduces leaks that IR-level proofs miss, which is the entire motivation for binary-level tools like Binsec/Rel [@binsec-rel]. Today we have per-primitive proofs and per-binary bounded checks; 2024 work such as CT-Prover targets the scalability wall, and benchmarks map the tool terrain [@ct-prover-2024][@sidetool-eval-2023]. A push-button, whole-library, binary-level verifier usable by non-experts remains out of reach -- and usability studies show even experts misconfigure the current tools.

Constant-time and fault-resistant post-quantum code. KyberSlash and GoFetch proved the discipline is unfinished for the freshly standardized lattice schemes [@kyberslash-2024][@gofetch-2024]. Shipping ML-KEM and ML-DSA implementations that are simultaneously constant-time, masked against physical attack, and fast across diverse CPUs -- and proving it -- is an active frontier, because the arithmetic-heavy PQC code has larger, fresher leakage surfaces than classical crypto.

Secretless architectures and minimal key lifetimes. The dominant real-world failure surface is long-lived, sprawled, unrotated secrets [@owasp-nhi-2025]. Rotation, workload attestation, and short-lived derived credentials shrink the custody surface, but the custody regress forbids a truly secretless bottom. The open question is quantitative: how small can the irreducible custodian be made?

Post-quantum key management at fleet scale. Larger keys, hybrid classical-and-PQ modes, and harvest-now-decrypt-later urgency make the key lifecycle harder precisely as it becomes critical. Zero-downtime migration across a whole estate, with a cryptographic bill of materials for inventory, anchored on the lifecycle discipline of NIST SP 800-57, is an open operational problem handed off to the crypto-agility part of this series [@nist-sp800-57].

Note: Every item above is a bounded engineering fight with measurable partial results, not a wish for undetectable magic. That framing matters: it tells you which problems to fund and which "solutions" to distrust. Anyone selling a complete, cheap, whole-machine leak-free implementation today is selling the impossibility from Section 8.

This capstone deliberately defers depth to its siblings. Randomness and RNG failures live in Part 2; padding oracles and the Lucky Thirteen mechanics in Part 6; nonce reuse and the Minerva and PS3 stories in Part 8; RSA-CRT and Bleichenbacher detail in Part 14; scalar-multiplication leakage of the nonce in Part 15; deterministic-versus-hedged signing and its fault risk in Part 17; threshold and MPC custody, the top rung of the spectrum, in Part 22; ML-KEM and ML-DSA internals in Part 23; and rotation-at-fleet-scale, crypto-agility, and the cryptographic bill of materials in Part 24. Read as a set, the twenty-five parts are one argument.

The research agenda is clear. The deployment agenda is clearer still. Here is what to do on Monday.

10. What To Do on Monday

Everything above collapses into a handful of rules you can apply without a research budget.

The constant-time checklist

Anything that touches a secret gets three prohibitions: no branch on it, no memory index computed from it, no variable-time operation applied to it -- and "variable-time" specifically includes integer division and early-exit comparison. Test the result with dudect or TIMECOP in CI, prefer a verified library over your own code, and re-check after every toolchain upgrade, because compilers betray branchless code back into branches. That single decision rule fits in a diagram.

flowchart TD S["A value in your code"] --> Q{"Does it touch a secret?"} Q -->|no| OK["Ordinary code is fine"] Q -->|yes| R["Forbid all three"] R --> B["No branch on it"] R --> M["No memory index from it"] R --> V["No variable-time op: no divide, no early-exit compare"]

Threat-model routing for leakage defense

Match the defense to the adversary. Everyone starts from the same floor and adds as the threat model widens.

Adversary	Add to the floor	Because
Remote network	Verified or constant-time crypto, constant-time MAC checks, no padding oracles	This is the floor for everyone; it closes Lucky Thirteen and the timing family
Co-tenant, shared cache (cloud)	Cache-hardening, per-tenant isolation, hardware CT where honored	Prime+Probe and Flush+Reload cross tenant boundaries
Physical possession (smartcard, wallet, IoT)	Masking or threshold implementations, fault sensors, a secure element	The attacker owns and can instrument the silicon
Assume speculation, DVFS, DMP exist	Treat CT as necessary-not-sufficient; isolate secrets across process and VM boundaries; keep key lifetimes short	These channels live below the ISA and cannot be closed in source

The key-management playbook

The custody decision is a lookup table keyed on asset and threat. A low-value single-host secret belongs in the OS keystore. A device-bound key with measured boot belongs in a TPM or secure element. Lots of data at rest belongs behind cloud KMS with envelope encryption and automatic rotation [@aws-kms]. A root, certificate-authority, code-signing, or payment key belongs in a dedicated or Managed FIPS 140-3 HSM [@fips-140-3].

A key that must survive with no single point of trust belongs in threshold or MPC custody, the subject of Part 22. And any application secret belongs in a secrets manager issuing short-lived attested credentials, with CI secret-scanning behind it -- never hardcoded, always rotated, and stale non-human identities always offboarded [@owasp-nhi-2025][@nist-sp800-57].

One caution about rotation, because it is oversold. Rotation bounds the blast radius of a leaked key -- it caps how much a single compromised key is worth and for how long -- but it is not a substitute for custody. Storm-0558's key was both never rotated and software-resident; rotation alone would have shortened the exposure window, but keeping the key in an HSM is what stops the export in the first place. Rotate and custody; the two do different jobs.

The common-misuse catalog

Most real leaks are not exotic. They are the same dozen mistakes, made again.One misuse deserves special note because the fix for a different problem creates it: deterministic ECDSA (RFC 6979) removes the nonce-RNG risk [@rfc6979], but a fault during signing then yields two signatures on the same nonce, which recovers the key [@poddebniak-2018]. Hedged signing -- mixing in fresh randomness -- restores fault resistance without reopening the RNG risk. Part 17 treats this trade in full.

Misuse	Why it leaks	Fix
`memcmp` on a MAC or tag	Early-exit timing oracle, the Lucky Thirteen class [@lucky13-2013]	Constant-time compare (CRYPTO_memcmp, sodium_memcmp)
Secret-indexed table lookup	Cache-timing leak of the accessed address [@bernstein-2005]	AES-NI or bitslicing
Secret-dependent division or branch	Variable-time; division is not a data-independent instruction [@kyberslash-2024]	Replace with multiply, mask, or branchless select
Compiler betrayal	Optimizer restores a branch, or elides a memset that scrubs a secret	Verify the binary, use secure-zero intrinsics, re-check after upgrades
Skipping RSA-CRT verification	Reopens the Bellcore fault; one bad signature factors n [@bellcore-1997]	Verify the signature before releasing it
Home-rolled constant-time	Assume it is not constant-time until a tool proves otherwise	dudect or TIMECOP, plus a verified library
Hardcoded or sprawled secrets	Rung 0; the key is the leak [@csrb-storm0558]	Secrets manager, rotation, offboarding
Reusing a KEK as a DEK	Couples bulk data directly to the root key	Keep DEK and KEK distinct; wrap, do not encrypt with the KEK
Non-constant-time post-quantum code	Fresh, unhardened leakage surface [@gofetch-2024]	Patched, verified PQC libraries

The envelope pattern at the bottom of the custody advice is worth seeing in code, because the blast-radius argument is the whole point.

{` // Illustrative only. Real systems use AES-GCM to encrypt and AES-KW to wrap, // and the KEK lives inside an HSM or cloud KMS that performs wrap and unwrap on request. function xorCrypt(bytes, key) { return bytes.map((b, i) => b ^ key[i % key.length]); } function randomKey(n) { return Array.from({ length: n }, () => Math.floor(Math.random() * 256)); }

// The KEK never leaves this boundary; software only sends wrap and unwrap requests. const KEK = randomKey(16); function wrap(dek) { return xorCrypt(dek, KEK); } // stands in for a KMS wrap call function unwrap(wd) { return xorCrypt(wd, KEK); } // stands in for a KMS unwrap call

// Encrypt: fresh DEK per object, encrypt the data, wrap the DEK, store the wrapped DEK beside it. const data = [72, 105, 33]; const dek = randomKey(16); const ciphertext = xorCrypt(data, dek); const wrappedDek = wrap(dek); // this, not the plaintext DEK, is stored with the ciphertext

// Decrypt: ask the boundary to unwrap the DEK, use it, discard it. const plaintext = xorCrypt(ciphertext, unwrap(wrappedDek)); console.log('round-trips:', plaintext.join(',') === data.join(',')); // Blast radius: a leaked wrappedDek without the KEK reveals nothing; a leaked DEK exposes only its one object. `}

Note: Print the table above and run it against any code that touches a key. Every row has ended a real system. The first three are the constant-time family, the next four are custody, and the last two are the modern frontier. If your review cannot rule out all nine, you have not finished reviewing.

Note: Ship verified constant-time as the floor, back it with hardware timing modes and, for physical adversaries, masking -- then spend the rest of your budget where systems actually break, on key custody. Close the channels you can model; bound the cost of the ones you cannot.

11. Seven Beliefs That Ship to Production and Get People Owned

No, and this is the single most consequential misunderstanding in the field. Constant-time means *data-independent* execution: the same instruction stream and the same memory-address trace regardless of the secret's value. It does not mean fixed duration. A routine can run for different absolute times on different inputs and still be constant-time, as long as the variation never depends on a secret. Conversely, padding a function to a fixed wall-clock time does not make it constant-time if it still branches on a secret underneath. No. Certification validates a modeled threat surface, not physics. TPM-Fail recovered 256-bit ECDSA keys from an Intel firmware TPM and a Common Criteria EAL4+ and FIPS-certified STMicroelectronics chip via a timing-and-lattice attack -- minutes locally, hours remotely [@tpm-fail-2020]. A certificate tells you the module passed a defined evaluation; it does not tell you the evaluation covered the channel your attacker will use. Necessary, not sufficient. Constant-time is a property of the instruction stream you wrote; the silicon runs one you did not. Spectre executes secret-dependent paths speculatively, Hertzbleed turns data-dependent frequency into remote timing, GoFetch's prefetcher dereferences pointer-like secrets, and KyberSlash leaks through a variable-time division -- all beneath the constant-time abstraction [@spectre-2019][@hertzbleed-2022][@gofetch-2024][@kyberslash-2024]. Constant-time is the floor, not the ceiling. Mostly true, and you should threat-model it honestly: if no adversary can hold or instrument your device, the classical power and EM channels are lower priority than the remote ones. But the boundary is not absolute. Hertzbleed turned a power side channel into a *remote* timing attack by exploiting dynamic frequency scaling, extracting a key from constant-time SIKE over a network with no physical probe at all [@hertzbleed-2022]. Rotation bounds the blast radius of a leak; it does not prevent the leak. It caps how long a compromised key stays valid and how much it protects, which is real and valuable. But it is not custody. Storm-0558's key was never rotated *and* software-resident; rotation would have shortened the window, while hardware custody would have stopped the export that started the whole incident [@csrb-storm0558]. Do both. No. It is custody plus performance, not redundancy. A cheap Data-Encryption Key does the bulk work while the Key-Encryption Key that wraps it never leaves the HSM or KMS, using standardized key wrapping [@nist-sp800-38f]. The payoff is a bounded blast radius: a leaked DEK exposes only the one object it encrypted, and the root key stays in hardware. Double encryption gives you two layers of the same risk; envelope encryption gives you a small, cheap, disposable key over a small, protected, permanent one. No -- they are broken implementations and mismanaged keys. Scope the claim carefully: relative to breaking the primitives and relative to the exotic side channels this article catalogs, operational key and secret management is the dominant failure surface, evidenced by cryptographic failures ranking as the number-two web risk and by Storm-0558's uncracked-but-stolen key [@owasp-a02-2021][@csrb-storm0558]. It is *not* a claim that key management outranks every software vulnerability class -- memory-safety bugs lead by raw count. The math almost never breaks. The handling of the math does.

The Proof Was Conditional

Twenty-four times, this series proved that the mathematics holds. Every one of those proofs carried a silent condition: supply a fresh nonce, validate the point, derive a uniform key, compare tags in constant time, keep the secret in its custody. This final part is the condition, standing alone and made visible. The math held. The machine and the operations are what handed the secret away.

You now have the lens to read any future break the same way, without waiting for someone to explain it. When the next headline attack lands, ask which of four invariants it violated. Did a secret touch time -- a branch, a division, a data-dependent frequency? Did a secret touch an address -- a table index, a cache line, a prefetched pointer? Did a fault skip a check -- a glitch, a hammered row, an undervolted enclave? Or did a key outlive or escape its custody -- hardcoded, sprawled, unrotated, software-resident? Kocher's stopwatch in 1996 and Apple's prefetcher in 2024 are the same sentence in two dialects, and Storm-0558 is the fourth invariant in its purest form: no cipher broken, a key simply gone.

This article has claimed that key and secret management is where deployed systems most often break, and that claim needs its fences. It is scoped: *relative to breaking the primitives, and relative to the exotic side channels catalogued here*, operational custody is the dominant failure surface -- supported by cryptographic failures ranking as the number-two web-application risk and by nation-state incidents like Storm-0558 that cracked no math [@owasp-a02-2021][@csrb-storm0558]. It is *not* a measured majority over all software vulnerability classes; memory-safety bugs lead by raw count. The umbrella term is "key and secret management," and the honest version of the thesis never says "most of all breaches" -- it says "the dominant *operational* failure surface for cryptographic systems."

So end the series where the evidence points. Spend real effort on the exotic channels, because verified constant-time and hardware timing modes are now deployable and the microarchitecture will keep testing them. But spend the larger share of your budget on the unglamorous layer, because that is where the incidents actually cluster: put the root key in hardware, use it without exporting it, envelope-wrap and auto-rotate everything else, and keep no long-lived secret sitting in software. The twenty-five parts of this guide are one argument, and this is its last line.

The math held. Make sure the machine, and the people running it, do not hand the secret away.

The Discrete Log Held; The Proofs Leaked: A Field Guide to Commitments and Sigma Protocols

noreply@paragmali.com (Parag Mali) — Mon, 13 Jul 2026 20:58:24 GMT

A discrete-log zero-knowledge proof is one three-move conversation -- commit, challenge, respond -- and for forty years every real break landed on one of those moves, never on the discrete logarithm underneath. Move 1, the commitment: Pedersen hides perfectly and binds on the discrete log, but a trapdoor-holder can equivocate. Move 2, the Fiat-Shamir challenge: hash too little and you get weak Fiat-Shamir, the Frozen Heart bug that forged "zero-knowledge" proofs in Helios (2012) and in Bulletproofs and PlonK (2022) with the math untouched. Move 3, the response: reuse the nonce and the witness falls out by algebra; exploit its linearity across concurrent sessions and the ROS attack forges naive multisignatures in polynomial time. This is the field guide to those three moves -- Pedersen, Schnorr, Sigma protocols, Fiat-Shamir -- how they compose into Bulletproofs, MuSig2, FROST, Signal's credentials, and Privacy Pass, and every documented way each has been gotten wrong.

1. The Math Was Fine. The Proofs Were Forgeable.

In April 2022, Trail of Bits published a coordinated disclosure with an unsettling shape [@tob-part1]. Across three unrelated "zero-knowledge" proof systems -- Girault's classic proof of knowledge, Bulletproofs (the range proof securing confidential Monero transactions), and PlonK (a modern general-purpose SNARK) -- the proofs were forgeable. An attacker could produce a convincing proof of a statement that was simply false [@tob-part1], [@tob-girault], [@tob-bp], [@tob-plonk].

And in none of the three cases had anyone broken the discrete logarithm, or factoring, or any hard problem. The math was untouched. The bug was that a single hash had omitted part of the statement it was supposed to be about [@tob-part1].

Note: In every Frozen Heart case, the discrete logarithm was never solved. The forgery lived entirely in what a hash function failed to bind. Soundness turned out to be a property of the protocol's plumbing, not of the hard problem beneath it.

The Bulletproofs instance was the sharpest. Trail of Bits traced it to an insecure Fiat-Shamir generation recommended in the original 2017 Bulletproofs paper itself -- the flaw was in the specification, not a careless implementer's shortcut [@tob-bp], [@bulletproofs]. It received a CVE: CVE-2022-29566 [@cve]. A vulnerability class was born, and Trail of Bits named it. The word "Frozen" is the acronym -- FoRging Of ZEro-kNowledge proofs; "Heart" nods to the Fiat-Shamir transform that sits at the heart of these proof systems [@tob-part1].

Then the twist that makes it a pattern, not an incident: this exact bug had already been named and published a decade earlier. In 2012, David Bernhard, Olivier Pereira, and Bogdan Warinschi showed that the Helios internet-voting system used the same shortcut -- and that its "zero-knowledge" ballot proofs could be transplanted and forged the same way [@bpw]. Two breaks, ten years apart, identical root cause, and the second one shipped in production behind a CVE.

Now widen the aperture. The discrete logarithm -- the hard problem under Diffie-Hellman, Schnorr, ElGamal, and the whole family -- has resisted forty years of attack; nobody broke the discrete log, as Part 18 argued at length. Yet the proof systems built on top of it break with some regularity. Why?

Because a discrete-log zero-knowledge proof is not one thing. It is a three-move conversation -- commit, challenge, respond -- and each move is a separate attack surface. The prover sends a commitment, the verifier sends a random challenge, the prover sends a response, and one equation decides the outcome.

Frozen Heart and Helios are failures of the second move, the challenge. Reusing a signature nonce -- the catastrophe that hands over a private key when one number repeats (Part 8) -- is a failure of the third move, the response. A trapdoor that lets a committer change its mind after the fact is a failure of the first move, the commitment. Three moves, three ways to leak, and not one of them is the discrete logarithm.

That is the whole thesis of this article, stated once so you can hold it through everything that follows: the discrete log held; the three moves leaked. The hard problem underneath these systems has never fallen. The conversation built on top of it has failed, repeatedly, in ways that are precise, named, and preventable.

The reach of these three moves is easy to underestimate. The same commit-challenge-respond skeleton runs under Bitcoin's Taproot multisignatures, under Monero's confidential transaction amounts, under Signal's private group membership, and under the digital-identity credentials now standardizing at the IETF. Master the three moves and you have a lens that resolves an entire branch of applied cryptography -- and a checklist that separates a deployment that ships from one that ships a CVE.

Those three moves were not invented together, and the cleanest one was invented first and deployed last -- frozen for nearly two decades by a patent [@schnorr-patent]. To see why the conversation has exactly three moves, and why each one leaks, we start where the vocabulary was born.

2. The Three-Move Conversation

Strip away the applications -- the multisignatures, the range proofs, the credentials -- and there is one shape underneath all of them. Learn that shape precisely, and every guarantee and every failure in this article becomes a statement about one of its three parts.

The archetype is Schnorr's protocol for proving you know a discrete logarithm. You have a public key $y = g^x$ in a group of prime order $q$ generated by $g$, and you want to convince a verifier that you know the secret exponent $x$ without revealing it. The conversation has exactly three moves:

Commit. Pick a fresh random nonce $r$ and send the commitment $a = g^r$.
Challenge. The verifier picks a random $e$ and sends it back.
Respond. Send $z = r + e\cdot x$.

The verifier accepts if and only if $g^z = a\cdot y^e$. Check the algebra: $g^{r + e x} = g^r\cdot (g^x)^e = a\cdot y^e$. It works whenever the prover really knows $x$.

A three-move public-coin proof -- commitment, then a random challenge, then a response -- in which the verifier checks a single equation. It must satisfy two properties: special soundness and special honest-verifier zero-knowledge. Schnorr's proof of knowledge of a discrete logarithm is the archetype [@damgard]. sequenceDiagram participant P as Prover (knows x) participant V as Verifier (knows y) Note over P: pick fresh random nonce r Note over P: commitment a holds g to the r P->>V: send commitment a Note over V: pick random challenge e V->>P: send challenge e Note over P: response z holds r plus e times x P->>V: send response z Note over V: check g to the z equals a times y to the e

Why is this a proof, and why does it reveal nothing? Both answers are visible by inspection, which is exactly why Schnorr is the teaching example.

Two accepting transcripts hand you the secret

Suppose a prover produces two accepting conversations that share the same commitment $a$ but have different challenges $e_1 \ne e_2$, with responses $z_1$ and $z_2$. Then $g^{z_1} = a\cdot y^{e_1}$ and $g^{z_2} = a\cdot y^{e_2}$. Divide the two equations and the $a$ cancels: $g^{z_1 - z_2} = y^{e_1 - e_2}$. Since $y = g^x$, the exponents must match, and you can solve for the witness:

$$x = \frac{z_1 - z_2}{e_1 - e_2} \pmod q.$$

This is special soundness, and the algorithm that recovers $x$ is called the extractor. It is what makes Schnorr a proof of knowledge: a prover who can answer two different challenges on the same commitment demonstrably possesses $x$, because we could run the extractor and read it out [@damgard].

The property that two accepting transcripts sharing a commitment but with different challenges yield the witness through an efficient extractor. It is what upgrades a Sigma protocol from "the statement is true" to "the prover actually knows a witness" [@damgard].

Hold on to that extractor. In Section 6 we will watch an attacker run exactly this formula -- same $a$, two different $e$ values -- against a signer who reused a nonce, and read out the private key. The defender's soundness proof and the attacker's key-recovery are the same three lines of algebra.

One accepting transcript hands you nothing

Now run the trick the other way. Pick the response $z$ and the challenge $e$ first, both at random, and then define the commitment as $a = g^z\cdot y^{-e}$. By construction $g^z = a\cdot y^e$, so this transcript passes verification perfectly -- and you built it without knowing $x$ at all. The distribution of $(a, e, z)$ produced this way is identical to a real run.

The property that any accepting transcript can be produced without the witness by choosing the response and challenge first and back-solving for the commitment. The algorithm that does this is the simulator, and its existence proves the exchange reveals nothing about the secret [@damgard].

That algorithm is the simulator, and its existence is the proof that the conversation leaks nothing about $x$: anything a verifier sees, it could have generated by itself. The whole edifice of zero knowledge -- the idea that "reveals nothing" can be made mathematically precise by demanding a simulator -- traces to Goldwasser, Micali, and Rackoff in 1985 [@gmr]. Part 1 covers those security definitions in full; here we take them as the bedrock they are.

Key idea: A Sigma protocol is one three-move conversation -- commit, challenge, respond -- whose two guarantees are both visible by inspection. Two accepting transcripts on the same commitment extract the secret (special soundness); a single transcript can be simulated without the secret (special honest-verifier zero knowledge). Two transcripts hand you the key; one transcript hands you nothing. That knife-edge is the entire game.

The distinction the extractor forces is worth naming. A Sigma protocol is a proof of knowledge -- the prover provably possesses a witness -- not merely a proof of membership, which would only assert that the statement is true. The gap matters: "this public key has a corresponding private key" is trivially true for every well-formed key, but "I know that private key" is the thing a signature must establish.

A proof that the prover actually possesses a witness, formalized by the existence of an extractor that could recover the witness from a successful prover. It is strictly stronger than a proof of membership, which asserts only that the statement is true [@damgard]. The simulator is not just a proof device; it is a construction tool. To prove "I know the secret behind public key A OR the one behind B" without revealing which, the prover *honestly* runs the branch it can answer and *simulates* the branch it cannot -- choosing that branch's challenge and response first and back-solving for its commitment, exactly as SHVZK allows. A shared challenge, split between the branches, ties them together so a cheater cannot simulate both at once. That is the Cramer-Damgard-Schoenmakers OR-proof, and it is why the toolkit composes into disjunctions, ring signatures, and selective-disclosure credentials [@cds].

Here is that whole story in code. The extractor is not a metaphor; it is a five-line function that turns two transcripts into a private key.

{` // A toy prime-order group: the order-q subgroup of the integers mod p. // p = 2039 is prime, q = 1019 is prime, and g = 4 has order exactly q. const p = 2039n, q = 1019n, g = 4n; const mod = (a, m) => ((a % m) + m) % m; function powmod(base, exp, m) { let r = 1n; base = mod(base, m); while (exp > 0n) { if (exp & 1n) r = mod(r * base, m); base = mod(base * base, m); exp >>= 1n; } return r; } const invModQ = (a) => powmod(a, q - 2n, q); // inverse mod prime q, via Fermat

const x = 777n; // the secret witness (never sent) const y = powmod(g, x, p); // public key y = g^x

// One honest run with nonce r and challenge e: function transcript(r, e) { const a = powmod(g, r, p); // commit const z = mod(r + e * x, q); // respond return { a, e, z }; } const verify = (t) => powmod(g, t.z, p) === mod(t.a * powmod(y, t.e, p), p);

const r = 314n; // ONE nonce, reused across two challenges const t1 = transcript(r, 11n); const t2 = transcript(r, 90n); // same a, different e -- the fatal pair console.log('t1 verifies = ' + verify(t1) + ', t2 verifies = ' + verify(t2));

// The extractor: x = (z1 - z2) / (e1 - e2) mod q const xRecovered = mod((t1.z - t2.z) * invModQ(mod(t1.e - t2.e, q)), q); console.log('recovered x = ' + xRecovered + ', true x = ' + x + ', match = ' + (xRecovered === x)); `}

That is the knife-edge in code: two transcripts hand you the key, one hands you nothing -- so every guarantee and every failure ahead is a statement about one of the three moves. But this clean picture took eleven years and five separate ideas to assemble, and the cleanest idea of all sat frozen behind a patent.

3. Five Ideas and a Patent (1985-1996)

The three-move conversation was assembled one idea at a time, mostly by people who were not trying to build it. And the sharpest tension in the story is this: the cleanest, smallest scheme in the whole toolkit was invented in 1989 and could not be legally deployed at scale until its patent expired around 2008 [@schnorr-patent]. The best piece shipped last. Here is how five disconnected papers became a coherent toolkit.

Goldwasser, Micali, and Rackoff (1985) gave the field its vocabulary. Their paper on the knowledge complexity of interactive proof systems defined zero knowledge through the simulator argument -- a proof reveals nothing if a simulator can produce the same transcript without the secret [@gmr]. Without this definition, none of the three moves would be analyzable; "reveals nothing" would be a slogan rather than a theorem.

Fiat and Shamir (1986) contributed two things in one paper. First, a practical zero-knowledge identification scheme based on factoring. Second, and far more consequential, the transform that bears their name: replace the verifier's random challenge with the output of a hash function, and the interactive protocol collapses into a non-interactive one -- a signature [@fiat-shamir]. This is the birth of Move 2, and it arrived as a heuristic, a plausible trick without a proof. That gap between "plausible" and "proven" is where three decades of trouble would later live.

Schnorr (1989) produced the efficient discrete-log archetype we dissected in Section 2: three moves, special soundness by inspection, and -- once you apply Fiat-Shamir -- the compact Schnorr signature.Schnorr's US Patent 4,995,082, granted in 1991, is widely reported to have expired around 2008. That is why patent-free DSA and later ECDSA colonized the internet through the 1990s, 2000s, and 2010s while Schnorr-proper waited, and why Bitcoin's BIP-340 could finally standardize native Schnorr signatures only in 2020 [@schnorr-patent], [@bip340]. Part 17 tells that signature story. The irony is total: the scheme with the cleanest security argument was the one the market could not touch, so the industry standardized on messier alternatives and lived with them for twenty years.

Pedersen (1991) was working on verifiable secret sharing, not commitments, when he handed the field its Move 1 workhorse.The Pedersen commitment debuted as a byproduct of a non-interactive, information-theoretically secure verifiable secret sharing scheme [@pedersen]. The toolkit's most-used commitment arrived as a side effect of a paper about something else -- a recurring pattern in this history. His commitment $C = g^m\cdot h^r$ is perfectly hiding, computationally binding, and additively homomorphic [@pedersen] -- three properties we will spend all of Section 4 unpacking, because the entire range-proof line stands on the third one.

Cramer, Damgard, and Schoenmakers (1994) turned Schnorr from a single protocol into a toolkit. Their proofs-of-partial-knowledge construction composes Sigma protocols with AND and OR: prove you know one of two secrets without revealing which, by honestly answering the branch you know and simulating the branch you do not [@cds].The name "Sigma protocol" was coined later, in Cramer's 1996 doctoral thesis. The letter is said to trace the three-move zig-zag of the conversation -- commitment down, challenge across, response back -- though the mnemonic is folklore. The OR-proof trick reuses the very simulator that proves zero knowledge, now as a constructive building block [@cds]. This is the deep reuse the article keeps returning to -- the simulator that proves a single proof leaks nothing is the same object that fakes the clauses you cannot answer in a disjunction. Okamoto's 1992 identification scheme sharpens the point: it is the witness-indistinguishable sibling of Schnorr, with a two-generator, Pedersen-shaped first move $y = g^{x_1} h^{x_2}$.Because many witness pairs $(x_1, x_2)$ open the same $y = g^{x_1} h^{x_2}$, an Okamoto transcript never reveals which pair the prover used -- the scheme is witness-indistinguishable by construction [@okamoto]. That is exactly the property a CDS OR-proof leans on to hide which clause is the real one, and it is why the Pedersen shape shows up again the moment you want to hide a witness among many.

Pointcheval and Stern (1996) finally proved that Fiat-Shamir signatures are unforgeable -- in the random-oracle model -- with the forking lemma: rewind a successful forger, feed it a different challenge on the same commitment, and its two answers extract the discrete log, contradicting hardness [@pointcheval-stern-96], [@pointcheval-stern]. The proof works, but it is loose: it loses a quadratic factor, and it does not extend cleanly to many sessions running at once. That looseness is not a blemish to be polished away later -- it is the first visible crack of the concurrency failure that Section 6 tears wide open.

flowchart LR A["1982 Chaum blinding"] --> B["1985 GMR zero knowledge"] B --> C["1986 Fiat-Shamir transform"] C --> D["1989 Schnorr identification"] D --> E["1991 Pedersen commitment"] E --> F["1994 CDS OR-proofs"] F --> G["1996 forking lemma"] G --> H["2018 Bulletproofs"] H --> I["2021 ROS and MuSig2"] I --> J["2024 FROST RFC 9591"] J --> K["2026 BBS drafts"]

By 1996 the conversation was complete and had a name. The rest of the story is what happened when people started building with it -- and the failures organize themselves, cleanly, one per move. Start with the move you make first: the commitment.

4. Move 1: What You Bind

Every proof begins by pinning down a value you are not yet ready to reveal. A commitment is the cryptographic envelope: you seal a value now and open it later, and the sealing must satisfy two opposing demands. It must be hiding -- the sealed envelope leaks nothing about what is inside -- and binding -- you cannot swap the contents after sealing. There are two standard ways to build one, and they sit at opposite corners of a trade-off you cannot escape.

A protocol to lock in a value now and reveal it later, such that the commitment leaks nothing about the value (hiding) and cannot be opened to a different value than the one committed (binding). No scheme can make both properties perfect at once [@damgard].

The Pedersen corner

Pedersen's commitment lives in a group of prime order $q$ with two generators $g$ and $h$, where nobody knows the discrete logarithm $\log_g h$. To commit to a message $m$, pick a random blinding scalar $r$ and publish

$$C = g^m\cdot h^r.$$

It has three properties, and each one matters downstream [@pedersen]:

Perfectly hiding. Because $r$ is uniform, $h^r$ is a uniform group element, so $C$ is uniformly distributed regardless of $m$. Even an adversary with unlimited computing power learns nothing about $m$ from $C$. This is information-theoretic, not merely computational.
Computationally binding. Opening $C$ to two different messages means finding $(m, r) \ne (m', r')$ with $g^m h^r = g^{m'} h^{r'}$, which rearranges to $\log_g h = (m - m')/(r' - r)$. In other words, breaking binding is computing a discrete logarithm. The binding reduction is not near the hard problem; it is the hard problem [@pedersen].
Additively homomorphic. Multiply two commitments and the exponents add: $C(m_1, r_1)\cdot C(m_2, r_2) = C(m_1 + m_2,\ r_1 + r_2)$. You can add committed values without opening them.

That third property is the one the whole range-proof line stands on, so hold it in mind. But first, the precision that must never be smoothed over.

Note: A Pedersen commitment is perfectly hiding and only computationally binding. Anyone who knows the trapdoor $\log_g h$ can equivocate: open a single commitment to any message they like, by solving one linear equation for a matching blinder. The security of the whole scheme rests on nobody knowing that trapdoor -- which makes the generation of $h$ a security-critical step, not a formality.

Equivocation is the entire Move-1 failure class. If the party who set up the parameters secretly chose $h = g^t$ and kept $t$, then for any commitment $C = g^m h^r$ they can produce a blinder $r'$ that opens it to a different message $m'$ -- because $C = g^{m + t r}$, and they can solve $m' + t r' \equiv m + t r$ for $r'$. The defense is to make the trapdoor un-knowable to everyone.The second generator $h$ must have unknown $\log_g h$, so it is derived by a nothing-up-my-sleeve procedure -- hash a fixed string to a curve point, so that no party could have planted a trapdoor. Part 16 covers hashing a string into the group; the takeaway here is that "pick $h$ honestly" is a real engineering requirement, not a throwaway line.

The hash corner

The dual construction throws away the algebra. Commit to $m$ by publishing H(m ‖ r) for a random $r$ and a collision-resistant hash H. This is computationally hiding and computationally binding -- and its binding rests on collision resistance, not on the discrete logarithm, which makes it the post-quantum-friendly choice. What it gives up is homomorphism: there is no way to add two hash commitments and get a commitment to the sum. Hashes shred structure by design.

Why can you not have everything -- perfect hiding and perfect binding? Because it is impossible. If a commitment is perfectly hiding, then $C$ is statistically independent of $m$, which means some opening to every message must exist; a computationally unbounded opener could always equivocate, so binding cannot also be perfect [@damgard]. Pedersen sits at the perfectly-hiding end; hash commitments sit near the perfectly-binding end; nothing sits at both.

flowchart TD Q{"What does the job need?"} Q -->|"add or range-prove values"| PED["Pedersen commitment g to the m times h to the r"] Q -->|"post-quantum binding, no algebra"| HASH["Hash commitment H of m with r"] PED --> P1["perfectly hiding"] PED --> P2["computationally binding on the discrete log"] PED --> P3["additively homomorphic"] PED --> P4["trapdoor holder can equivocate"] HASH --> H1["computationally hiding and binding"] HASH --> H2["not homomorphic"] HASH --> H3["post-quantum from collision resistance"]

Here is the corner choice as a table you can keep next to a design review.

Property	Pedersen `g^m · h^r`	Hash `H(m ‖ r)`
Hiding	Perfect (information-theoretic)	Computational
Binding	Computational (discrete log)	Computational (collision resistance)
Homomorphic	Yes, additively	No
Post-quantum	No -- Shor breaks binding	Yes -- binding survives Shor
Commitment size	~32 bytes (one ristretto255 point)	~32 bytes (one SHA-256 output)
Equivocation risk	Trapdoor holder of `log_g h` opens to anything	None -- needs a hash collision
Best suited for	Confidential amounts, range proofs, sums	PQ binding, timestamps, no algebra needed

The homomorphism is not a curiosity; it is the concrete cash-in of Move 1. Committed values that add correctly are exactly what confidential transactions need -- inputs and outputs whose hidden amounts must balance -- and they are the value commitments that a range proof later proves lie in a valid range. Section 7 follows that thread from Pedersen to Bulletproofs. First, watch the homomorphism and the equivocation trapdoor in code.

{` const p = 2039n, q = 1019n, g = 4n; const mod = (a, m) => ((a % m) + m) % m; function powmod(base, exp, m) { let r = 1n; base = mod(base, m); while (exp > 0n) { if (exp & 1n) r = mod(r * base, m); base = mod(base * base, m); exp >>= 1n; } return r; } // In a real system NOBODY knows t; we plant it here only to demonstrate equivocation. const t = 503n; // the trapdoor: t = log_g h const h = powmod(g, t, p); const commit = (m, r) => mod(powmod(g, m, p) * powmod(h, r, p), p);

// 1) Additive homomorphism: C(m1,r1) * C(m2,r2) opens to (m1+m2, r1+r2) const C1 = commit(100n, 7n), C2 = commit(250n, 9n); const product = mod(C1 * C2, p); const summed = commit(mod(100n + 250n, q), mod(7n + 9n, q)); console.log('homomorphism holds = ' + (product === summed));

// 2) Equivocation: the trapdoor holder opens ONE commitment to a different message const m = 100n, r = 7n, C = commit(m, r); const mFake = 999n; const rFake = mod(r + (m - mFake) * powmod(t, q - 2n, q), q); // solve m'+t r' = m+t r console.log('same C reopened to ' + mFake + ' = ' + (commit(mFake, rFake) === C)); `}

A Pedersen commitment binds a value the verifier cannot see. But an interactive proof about it needs a live verifier to send a random challenge -- and live verifiers do not scale to signatures or blockchains. The fix is to fake the verifier with a hash. That single fix is the most-broken move in the toolkit.

5. Move 2: What Goes Into the Hash

The transform that makes the whole toolkit shippable is one line of code -- and that one line has been written wrong, in production, for thirty years.

The interactive Sigma protocol needs a live verifier to supply a random challenge $e$. That is fine for a login handshake and useless for a signature or a blockchain, where there is no verifier standing by. Fiat and Shamir's fix was to derive the challenge from a hash of the transcript so far: compute $e = H(\dots)$ yourself and proceed [@fiat-shamir]. In the random-oracle model -- where the hash is treated as a truly random function -- the hash is the verifier, and the interactive proof becomes a non-interactive one.

A method to make an interactive Sigma protocol non-interactive by replacing the verifier's random challenge with a hash of the transcript. The prover computes the challenge itself, turning a proof of knowledge into a signature or an on-chain proof [@fiat-shamir]. An idealization that treats a hash function as a truly random function answering consistently to repeated queries. It is the model in which Fiat-Shamir soundness is provable; Part 10 covers the [hash machinery](/blog/the-fingerprint-two-files-shared-a-field-guide-to-cryptograp/) it idealizes [@fiat-shamir].

Everything now rides on what you put inside that hash. And this is where the failures cluster, because the answer is not obvious and the wrong answer still runs.

The decision rule

Strong Fiat-Shamir binds everything the proof is about:

$$e = H(\texttt{domain_separator} ,|, \text{statement} ,|, \text{public params} ,|, a ,|, \text{transcript}).$$

Weak Fiat-Shamir hashes only the commitment, $e = H(a)$, and omits the statement. That omission is the bug. When the challenge does not depend on the statement, a proof built for one statement can be transplanted onto another -- a statement it was never made for -- and it still verifies [@bpw]. The extreme case is a signature: if the message is not in the hash, a signature on "pay 5" is a valid signature on "pay 5000," because verification never touches the message.

flowchart TD A["Sigma commitment a and statement x"] --> B{"What enters the challenge hash?"} B -->|"weak FS: only a"| W["e is H of a"] B -->|"strong FS: statement, params, a"| S["e is H of x with params and a"] W --> W2["challenge ignores the statement"] W2 --> W3["transplant the proof onto a false x prime"] W3 --> W4["forgery verifies"] S --> S2["challenge binds the exact statement"] S2 --> S3["transplant recomputes a different e"] S3 --> S4["forgery rejected"]

You can watch the transplant happen and then watch the fix stop it. In the model below, the "statement" is the signed message; weak Fiat-Shamir lets one signature verify against any message, and strong Fiat-Shamir binds it.

{` const p = 2039n, q = 1019n, g = 4n; const mod = (a, m) => ((a % m) + m) % m; function powmod(base, exp, m) { let r = 1n; base = mod(base, m); while (exp > 0n) { if (exp & 1n) r = mod(r * base, m); base = mod(base * base, m); exp >>= 1n; } return r; } function H(str) { let h = 0n; for (const ch of str) h = mod(h * 131n + BigInt(ch.charCodeAt(0)), q); return mod(h, q); } const x = 777n, y = powmod(g, x, p); const r = 421n; // nonce hardcoded only so the demo is deterministic

// WEAK Fiat-Shamir: the challenge hashes only the commitment a (message omitted) const signWeak = () => { const a = powmod(g, r, p); const e = H(a.toString()); return { a, z: mod(r + e * x, q) }; }; const verifyWeak = (msg, s) => { const e = H(s.a.toString()); return powmod(g, s.z, p) === mod(s.a * powmod(y, e, p), p); }; const sigW = signWeak(); console.log('weak valid on original = ' + verifyWeak('alice pays 5', sigW)); console.log('weak valid on FORGED = ' + verifyWeak('alice pays 5000', sigW)); // true -> universal forgery

// STRONG Fiat-Shamir: the challenge binds the public key AND the message AND a const signStrong = (msg) => { const a = powmod(g, r, p); const e = H(y.toString() + '|' + msg + '|' + a.toString()); return { a, z: mod(r + e * x, q) }; }; const verifyStrong = (msg, s) => { const e = H(y.toString() + '|' + msg + '|' + s.a.toString()); return powmod(g, s.z, p) === mod(s.a * powmod(y, e, p), p); }; const sigS = signStrong('alice pays 5'); console.log('strong valid on original = ' + verifyStrong('alice pays 5', sigS)); console.log('strong valid on FORGED = ' + verifyStrong('alice pays 5000', sigS)); // false -> rejected `}

The same bug, forged twice, a decade apart

This is not a hypothetical. It is the single most-repeated real-world break in the toolkit, and it has a body count.

In 2012, Bernhard, Pereira, and Warinschi formalized the weak-versus-strong distinction and showed that the Helios internet-voting system used weak Fiat-Shamir. Its ballot proofs -- the "zero-knowledge" proofs that a vote is well-formed -- could be transplanted and forged [@bpw]. The paper's title is a warning label: How Not to Prove Yourself.

Ten years later, in April 2022, Trail of Bits disclosed the same class of bug in three live systems at once: Girault's proof of knowledge, Bulletproofs, and PlonK [@tob-part1], [@tob-girault], [@tob-bp], [@tob-plonk]. The Bulletproofs instance was traced to an insecure Fiat-Shamir generation recommended in the original 2017 paper, and it carried CVE-2022-29566 [@tob-bp], [@cve], [@bulletproofs].Note the attribution carefully: "Girault" is the name of an affected scheme (a proof of knowledge in an RSA group), not the researcher. The disclosure was by Jim Miller at Trail of Bits [@tob-part1], [@tob-girault].

"We've dubbed this class of vulnerabilities Frozen Heart. The word frozen is an acronym for FoRging Of ZEro kNowledge proofs ... a mistake in the original academic paper, in which the authors recommend an insecure Fiat-Shamir generation." -- Trail of Bits [@tob-part1]

Note: Helios in 2012 and Frozen Heart in 2022 are the same failure: a challenge hash that omitted the statement. In neither case did anyone break the discrete logarithm. The security lived in one line of the hash, and that line was wrong -- in a shipped e-voting system and in a peer-reviewed range-proof paper -- ten years apart [@bpw], [@tob-part1].

Bind the whole statement and the challenge is safe. But Fiat-Shamir also turns the Schnorr identification protocol into the Schnorr signature -- and that hands the third move, the response, a fresh way to leak. This one does not even need a mistake in the hash. It needs a mistake in a single random number.

6. Move 3: How the Response Behaves

The response $z = r + e\cdot x$ is the only message in the whole conversation that touches the secret, and it protects that secret with exactly one thing: a fresh, secret, one-time nonce $r$. Everything the third move can suffer is a story about that nonce -- a number that is never even transmitted, yet is where the key most often escapes.

Start with the bridge that Fiat-Shamir just built. Apply the transform to Schnorr's identification protocol and you get the Schnorr signature: the commitment $a$ and response $z$ are the signature, and the challenge is $e = H(y ,|, m ,|, a)$. Read that carefully, because it is one of the article's quiet revelations: a Schnorr signature is a non-interactive proof of knowledge of the private key. Signing is proving you know $x$; the message rides along in the challenge hash [@bip340].

Reuse the nonce, lose the key

Now recall the extractor from Section 2. Two accepting transcripts that share the commitment $a$ but differ in the challenge $e$ yield the witness $x = (z_1 - z_2)/(e_1 - e_2)$. That was the defender's proof that Schnorr is a proof of knowledge. Turn it around: if a signer produces two signatures with the same nonce $r$, they produce two transcripts with the same $a = g^r$ and different challenges. The attacker runs the same three lines of algebra the extractor runs, and the private key falls out.

This is the identical failure as a reused ECDSA nonce -- the catastrophe covered in Parts 8 and 17 -- because both signature schemes hide the key behind a per-signature nonce, and both surrender it the moment the nonce repeats.There is a sharp distinction worth carrying. The reused-nonce break in Part 8 happens within a single scheme when one nonce is used for two signatures. The ROS break below happens across many concurrent sessions of a multi-party scheme, exploiting the linearity of the response rather than a literal repeat. Same move, two different exploits. A biased nonce -- not repeated, just slightly non-uniform -- leaks the key more slowly, through lattice attacks that aggregate many partial leaks. The discipline is absolute: nonces must be fresh, secret, and uniform, whether drawn from a well-seeded CSPRNG or derived deterministically as Part 2 describes.

Why concurrency is the hard part

Pointcheval and Stern's forking lemma proves single-session Schnorr unforgeable in the random-oracle model: rewind a forger, give it a fresh challenge on the same commitment, and its two answers extract the discrete log, which is assumed hard [@pointcheval-stern]. The proof works, but it pays a price -- a quadratic loss in tightness -- and, more importantly, the rewinding trick does not compose when many sessions run at once. You cannot cleanly rewind one of a hundred interleaved conversations without disturbing the rest.

The rewinding technique of Pointcheval and Stern that turns a random-oracle forger into a discrete-log solver by running it twice with different challenges on the same commitment. It gives Schnorr and Fiat-Shamir their ROM security proof, but loosely -- with a quadratic security loss -- and only for the single-session setting [@pointcheval-stern].

That gap between "one session is provably secure" and "many concurrent sessions are secure" stayed hidden for two decades. Then it was pried open.

The ROS attack

In 2021, Benhamouda, Lepoint, Loss, Orru, and Raykova published a polynomial-time solution to the ROS problem, and with it broke naive two-round Schnorr multisignatures and blind Schnorr signatures [@ros]. The mechanism is pure linear algebra. Open enough concurrent signing sessions -- more than about $\log_2 p$ of them -- collect the responses, and set up a linear system whose solution forges a signature on a message the signers never agreed to. No discrete logarithm is solved anywhere in the attack.

"Random inhomogeneities in an Overdetermined, Solvable system of linear equations." It is solvable in polynomial time once the number of concurrent sessions exceeds roughly $\log_2 p$, which is exactly what breaks naive linear Schnorr multi-signatures and blind signatures. It is emphatically NOT "Random Oracle Substitution," a common web mistranslation [@ros].

The 2021 result had a predecessor. In 2019, Drijvers and co-authors had already shown, at IEEE S&P, that the same naive two-round multisignatures were insecure under concurrency -- their attack ran in sub-exponential time using Wagner's generalized-birthday (k-sum) algorithm [@drijvers], [@wagner]. The 2021 ROS work sharpened sub-exponential into polynomial. Together they closed the case: naive linear Schnorr multisignatures are not merely hard to prove secure under concurrency; they are concretely broken.

flowchart TD Z["response z holds r plus e times x"] --> B{"how is the nonce misused?"} B -->|"reuse or bias one nonce"| R1["two transcripts share commitment a"] R1 --> R2["extractor recovers x from two responses"] B -->|"open many concurrent sessions"| C1["responses are linear across sessions"] C1 --> C2["ROS solves the linear system"] C2 --> C3["forge a signature, no discrete log solved"] No discrete log is ever solved. The response simply leaks across concurrent sessions, and linear algebra does the rest.

Note: The simple "everyone sends a nonce, everyone sends a response" two-round multisignature is forgeable under concurrent sessions -- sub-exponentially since Drijvers (S&P 2019), and in polynomial time since ROS (2021). Use MuSig2 (n-of-n) or FROST (t-of-n), which are built specifically to defeat this attack [@drijvers], [@ros].

Move 1 can be equivocated, Move 2 can be transplanted, Move 3 can be solved by linear algebra -- and not one of these touched the discrete logarithm. That is the thesis, proven three times. The story from here is how each sub-lineage climbed out of its break, while the three moves kept running underneath.

7. What Superseded What (and What Never Did)

Here is the classic mistake this article exists to correct: there is no single "what superseded what" ladder for this toolkit. The three moves are a substrate. Supersession happens in the composites built on top of them, and there are exactly three construction ladders -- one per application: range proofs, multi-party Schnorr, and anonymous credentials. Every rung was forced by a break or a cost on a specific move, never by an advance against the discrete logarithm.

The one change to the foundation itself -- weak Fiat-Shamir hardened to strong -- is not a fourth ladder but a fix applied in place, so we take it first.

flowchart TD subgraph FOUND["Foundations, never superseded, Move 2 hardened weak to strong in place"] F1["Move 1 Pedersen"] F2["Move 2 Fiat-Shamir, hardened weak to strong"] F3["Move 3 Schnorr response"] end FOUND --> RP["Range proofs"] FOUND --> MP["Multi-party Schnorr"] FOUND --> AC["Anonymous credentials"] RP --> RP1["Borromean 2015"] RP1 -->|"superseded by"| RP2["Bulletproofs 2018"] RP2 -->|"superseded by"| RP3["Bulletproofs plus 2022"] MP --> MP1["naive two-round"] MP1 -->|"superseded by"| MP2["three-round"] MP2 -->|"superseded by"| MP3["MuSig2 and FROST"] AC --> AC1["Chaum blind signature"] AC1 -->|"superseded by"| AC2["CL-Idemix and U-Prove"] AC2 -->|"shipped instead"| AC3["KVAC, BBS, Privacy Pass"]

Foundation hardening: weak Fiat-Shamir becomes strong

The one part of the foundation that changed was never superseded -- it was hardened in place. Weak Fiat-Shamir was the default misreading of the heuristic; Helios in 2012 and Frozen Heart in 2022 forged proofs under it [@bpw], [@tob-part1], [@cve]. Strong Fiat-Shamir -- bind the whole statement -- is what the random-oracle proof had silently assumed all along. The two generations differ only in what enters one hash, and the discrete logarithm is untouched in both. Move 2 was not replaced by a better construction; it was corrected to the instantiation it always needed.

Ladder A -- range proofs: cashing in the homomorphism

This ladder cashes in Move 1's homomorphism, and it is the one with dramatic size numbers. The textbook starting point is a bit-decomposition OR-proof: to show a committed value lies in $[0, 2^n)$, prove each of its $n$ bits is either 0 or 1 with an OR-proof. It works and it is $O(n)$ -- linear in the bit width, which is a lot of bytes.

Monero's RingCT initially shipped Borromean ring range proofs (Maxwell and Poelstra, 2015), which packed the bit proofs into ring signatures at roughly 6 kB per output -- the dominant cost of a confidential transaction [@monero]. Then Bulletproofs (Bunz, Bootle, Boneh, Poelstra, Wuille, and Maxwell, 2018) reduced the range statement to an inner-product relation and compressed it with a recursive argument that halves the vectors each round, yielding an $O(\log n)$ proof of about 600 to 700 bytes -- with no trusted setup [@bulletproofs], [@monero].

Monero's own documentation reports the deployment cut transaction size by at least 80 percent [@monero]. Bulletproofs+ (2022) shrank the constant further with a weighted inner-product argument [@bulletproofs-plus].

The honest footnote on this ladder is the one Section 5 set up: Bulletproofs also broke, via weak Fiat-Shamir, as CVE-2022-29566 [@tob-bp], [@cve]. That was a Move-2 failure inside a Move-1 construction, and it was fixed by strong Fiat-Shamir -- not by Bulletproofs+. The size ladder and the security fix are independent axes on the same object.

Ladder B -- multi-party Schnorr: climbing out of the ROS break

This ladder is Move 3 recovering from the concurrency break of Section 6. The naive two-round multisignature -- everyone sums nonces, everyone sums responses -- fell to Drijvers (2019) and then ROS (2021) [@drijvers], [@ros]. The first fix was to spend a round: three-round commit-then-reveal schemes make each signer commit to its nonce before revealing it, which restores concurrent security at the cost of an extra round trip [@musig2].

MuSig2 (Nick, Ruffing, and Seurin, 2021) got back to two rounds by a clever trick: each signer publishes two nonces, and the effective nonce is a hash-weighted combination of them, so the combined nonce is a nonlinear function of the inputs -- exactly what defeats the linear-algebra ROS forgery [@musig2]. FROST (standardized as RFC 9591 in June 2024) then added genuine threshold signing [@rfc9591].

The distinction that matters: MuSig2, standardized as BIP-327, is n-of-n -- every signer must participate, and the output is a single 64-byte signature indistinguishable from a single-key one [@bip327], [@musig2]. FROST is the t-of-n threshold scheme, where any $t$ of $n$ signers suffice [@rfc9591]. They are different tools for different policies, not two names for the same thing.

Ladder C -- anonymous credentials: the elegant generation that lost

This ladder is the strange one, because the most elegant generation is the one that did not win. It begins with Chaum's blind signatures -- blind, let the issuer sign, unblind -- introduced in 1982 [@chaum-blind] and set out as a whole transaction system in his 1985 paper, the root of unlinkable credentials, though single-show [@chaum-cacm]. It climbs to the multi-attribute systems: Camenisch-Lysyanskaya (Idemix) and Brands (U-Prove), which let a holder prove statements about many attributes and were genuinely beautiful cryptography [@cl-2001], [@cl-2004], [@uprove]. And then, mostly, it did not ship.

Camenisch-Lysyanskaya / Idemix and Brands / U-Prove were the theoretically complete multi-attribute credentials of the 2000s -- standardized (U-Prove under ISO/IEC 18370-2 and released under Microsoft's Open Specification Promise) and mathematically lovely [@uprove], [@cl-2001], [@cl-2004]. Yet they were largely never deployed at population scale, losing on cost, complexity, and integration friction. The systems that shipped instead were pragmatic ones -- Signal's keyed-verification credentials, BBS, and Privacy Pass. This is the rare case in cryptography where the most elegant design is not the one that wins -- a deployment failure, not a cryptographic break. The exception proves the rule: Direct Anonymous Attestation, a Camenisch-Lysyanskaya-family credential presented through a Sigma proof of possession, shipped in billions of TPM security chips -- precisely because its target was fixed hardware with a captive issuer, exactly the friction the general-purpose systems could not overcome [@daa], [@tpm2-spec].

The winners are three deployed corners, not one. Signal KVAC (2020) is a keyed-verification credential where the issuer is also the verifier, built on an algebraic MAC and a Sigma-protocol presentation proof, deployed in Signal's private groups [@signal], [@kvac]. BBS signatures are the publicly-verifiable corner, a multi-message credential now in IETF drafts [@bbs-draft].

And Privacy Pass / Apple Private Access Tokens, standardized at the IETF in June 2024, are the unlinkable single-use authorization-token corner -- the deployed realization of Chaum's blind-token idea, in which a client redeems an anonymously issued token at an origin that cannot link it back to issuance [@rfc9576], [@cloudflare-pat]. All three are Sigma proofs over the same three moves, filled in for three different trust models; Section 8 attaches the parameters to each.

Key idea: Every real-world break lives on one of the three moves. The discrete logarithm was never broken. The commitment equivocates, the challenge transplants, the response leaks -- and the composites climb their ladders while Pedersen, Fiat-Shamir, and Schnorr keep running underneath, never superseded.

Here is the whole picture as one matrix.

Approach	Year	Lineage / move	Key idea	Size or cost	Status
Weak Fiat-Shamir	1986	Move 2	hash only the commitment	--	Superseded (insecure)
Strong Fiat-Shamir	2012	Move 2	bind the whole statement	--	Active
Bit-decomposition range proof	classic	Move 1	OR-prove each bit	O(n)	Historical
Borromean range proof	2015	Move 1	ring signatures over bits	~6 kB/output	Superseded
Bulletproofs	2018	Move 1	recursive inner-product	~600-700 B, O(log n)	Active
Bulletproofs+	2022	Move 1	weighted inner-product	shorter constant	Active
Naive two-round multisig	2010s	Move 3	sum nonces and responses	2 rounds, broken	Superseded
Three-round multisig	2006	Move 3	commit-then-reveal nonces	3 rounds	Superseded
MuSig2 (BIP-327)	2021	Move 3	two nonces, nonlinear combine	2 rounds, n-of-n	Active
FROST (RFC 9591)	2024	Move 3	threshold nonce commitments	2 rounds, t-of-n	Active
Chaum blind signature	1982	Credentials	blind, sign, unblind	single-show	Historical
CL-Idemix / U-Prove	2001	Credentials	multi-attribute proofs	multi-show (Idemix); single-show (U-Prove)	Niche (undeployed)
Signal KVAC	2020	Credentials	algebraic MAC, issuer verifies	deployed	Active
BBS	2023	Credentials	pairing, multi-message	~80 B signature	Standardizing
Privacy Pass / Apple PAT	2024	Credentials	unlinkable single-use token	RFC 9576/9577/9578	Deployed

Three ladders, three break-driven climbs, over a foundation that was itself hardened in place -- three ascents on one unchanging three-move base. Which rung is at the top of each ladder today -- and with which exact parameters -- is the difference between a design review that passes and one that ships a CVE.

8. State of the Art (2024-2026): What Actually Ships

The frontier is not one best algorithm. It is a layered stack: three foundational moves that were never superseded, and above them a handful of composites that ship at scale, each with a standard and a parameter set. If you have to build with this in 2026, here is the state of the art with the numbers attached.

The moves, still the state of the art as primitives. A Pedersen commitment is a single 32-byte point on ristretto255. Strong Fiat-Shamir is the correct instantiation now baked into every standard below. And Schnorr-proper, standardized as BIP-340, is a 64-byte signature [@bip340]; Bitcoin's Taproot upgrade activated it at block 709,632 in November 2021 [@bip341], [@taproot-wiki], and native Schnorr has been available for every spend since. These are not legacy pieces waiting to be replaced; they are the load-bearing base of everything else.

BIP-340 is Schnorr-proper over the secp256k1 curve [@bip340]. EdDSA and its instance Ed25519 are also Schnorr-*family* signatures -- same linear response structure -- but with different encoding, different nonce derivation, and a different curve. They are cousins, not the same standard, and conflating them causes real interoperability and security-review confusion. When someone says "Schnorr signature," ask which one.

MuSig2 / BIP-327 is the deployed answer for n-of-n multisignatures: a two-round protocol producing a single 64-byte Schnorr signature that is indistinguishable on-chain from a single-key spend, implemented in libsecp256k1 and secp256k1-zkp [@bip327], [@musig2]. Every co-signer participates, and the verifier does one ordinary Schnorr check.

FROST / RFC 9591 (June 2024) is the most consequential new standard in the window: a two-round t-of-n threshold Schnorr signature, specified across five ciphersuites -- Ed25519, ristretto255, Ed448, P-256, and secp256k1 [@rfc9591]. On its own, FROST assumes signers do not vanish mid-protocol; ROAST (CCS 2022) wraps it to keep signing alive even when some signers are faulty or offline, giving fault tolerance for asynchronous deployments [@roast].

Scheme	Nonce structure	Rounds	Concurrent-secure	Fault-tolerant	Signature	Status
Naive two-round	sum the nonces	2	No (Drijvers, ROS)	No	64 B	Broken
Three-round	commit then reveal	3	Yes	No	64 B	Superseded
MuSig2 (BIP-327)	two nonces, nonlinear	2	Yes	No	64 B (n-of-n)	Deployed
FROST (RFC 9591)	threshold commitments	2	Yes	No	64 B (t-of-n)	Standardized
FROST + ROAST	ROAST wrapper	2+	Yes	Yes	64 B (t-of-n)	Active

A zero-knowledge proof that a committed value lies in a fixed interval such as $[0, 2^n)$ without revealing the value. Bulletproofs are the deployed transparent construction, proving a Pedersen-committed amount is in range in about 600 to 700 bytes [@bulletproofs].

Bulletproofs and Bulletproofs+ are the state of the art for transparent range proofs -- logarithmic size, no trusted setup -- and they secure confidential amounts in Monero, where Bulletproofs deployed in October 2018 and Bulletproofs+ followed in 2022 [@bulletproofs], [@bulletproofs-plus], [@monero]. They are the default whenever you need a range proof without a setup ceremony and can accept linear verification time. This is the immediate cash-in of Move 1's homomorphism at production scale.

On the credential side, three systems ship, and they occupy different economic corners.

An anonymous credential in which the issuer and the verifier are the same party, so the credential can be built on a fast algebraic MAC instead of a public-key signature. The holder presents it with a Sigma-protocol proof of knowledge, revealing only the attributes it chooses [@kvac], [@signal].

Signal KVAC is the keyed-verification credential in production in Signal's private group system: the issuer is also the verifier, so it runs on an algebraic MAC with no pairings, and presentation is a Sigma-protocol proof of knowledge [@kvac], [@signal]. It is the standing counter-example to "elegant but undeployed."

BBS and Blind BBSBBS takes its name from the 2004 Boneh-Boyen-Shacham short group signature scheme it descends from, by way of the later BBS+ variant [@bbs-origin]. are the publicly-verifiable answer: multi-message, selective-disclosure credentials over the BLS12-381 pairing curve, with roughly 80-byte signatures, now in draft-irtf-cfrg-bbs-signatures-10 (8 January 2026) and Blind BBS draft-03 [@bbs-draft], [@blind-bbs]. BBS is the standardizing choice for W3C Verifiable Credentials and identity wallets, where a third party must verify what an issuer signed. KVAC and BBS are parallel answers -- private-issuer economy versus public verifier -- not one replacing the other.

Privacy Pass, deployed at population scale as Apple Private Access Tokens, is the third corner: a different shape from the other two -- an unlinkable single-use authorization token rather than a multi-attribute credential. A client anonymously obtains a token backed by some attestation and later redeems it at an origin that cannot link the redemption to issuance.

It was standardized at the IETF in June 2024 across three documents -- RFC 9576 for the architecture, RFC 9578 for issuance, and RFC 9577 for the PrivateToken HTTP authentication scheme [@rfc9576], [@rfc9578], [@rfc9577]. Apple shipped it in iOS 16 and macOS Ventura in 2022 with hardware-backed device attestation, and Cloudflare uses it to drop the CAPTCHA for attested devices [@cloudflare-pat].Direct Anonymous Attestation (2004), the deployed exception detailed in Section 7, ships in two hardware generations -- TPM 1.2 RSA-DAA and TPM 2.0 ECDAA -- and adds per-verifier basename linkability, so a device proves it is genuine without becoming trackable [@daa], [@tpm2-spec].

Underneath, it is still commit, challenge, respond over the discrete log. Keep it distinct from KVAC and BBS: it is single-show authorization, not multi-attribute selective disclosure, and its privately-verifiable issuance is a verifiable oblivious PRF whose Chaum-Pedersen mechanism Part 20 owns and this section only signposts.

System	Verifiability	Cryptographic base	Size	Deployment
U-Prove / Idemix	Public	Discrete log, RSA, pairings	multi-kB	Standardized, largely undeployed
Signal KVAC	Keyed (issuer verifies)	Algebraic MAC, no pairings	Compact	Deployed in Signal
BBS / Blind BBS	Public	Pairings on BLS12-381	~80 B signature	Standardizing (IETF)
Privacy Pass / Apple PAT	Public or keyed, per token type	VOPRF or blind RSA issuance	Small single-use token	Deployed (iOS 16, macOS Ventura)

This is the same primitive Part 20 built on: an oblivious PRF's verifiability rests on a Chaum-Pedersen Sigma proof -- and Privacy Pass's privately-verifiable issuance is exactly that VOPRF. The deployed anonymous-credential systems are Sigma proofs of knowledge of a signature, and Direct Anonymous Attestation is the Sigma-protocol credential already shipping in TPM silicon. Every composite in this section is the three moves with the parameters filled in.

Every one of these is a defensible point on a Pareto frontier, not a single winner -- which is exactly why the toolkit is a toolkit. To choose between them, and between the whole family and a general-purpose proof system, you need the line where Sigma plus Fiat-Shamir stops being the right tool.

9. Competing Approaches and Where the Line Is

The toolkit's real competition is the general-purpose proof system -- the SNARK or STARK that can prove any circuit. Knowing when not to reach for one is half of using this toolkit well.

A bespoke Sigma proof plus strong Fiat-Shamir is transparent (it needs only the random-oracle assumption, no setup ceremony), tiny for narrow statements (a constant number of group elements, growing only with the number of clauses), and cheap to prove. A pairing-based SNARK gives a constant-size proof and near-constant verification, but at the price of a trusted setup: Groth16 lands near 200 bytes and PLONK near 400 [@groth16], [@plonk].

A STARK is transparent and plausibly post-quantum, but its proofs run to tens or hundreds of kilobytes. The Bulletproofs authors made the transparency contrast explicit: their range proofs need no trusted setup, unlike the succinct pairing SNARKs [@bulletproofs].

The decision rule falls out of the shape of your statement:

A fixed algebraic relation -- "I know a discrete log," "this Pedersen value is in range," "one of these $n$ public keys signed" -- wants a hand-verified Sigma proof with strong Fiat-Shamir. It is smaller, faster, and setup-free.
An arbitrary circuit -- "this value is the output of running this program" -- is where you reach for a SNARK or STARK, accepting the setup or the proof size in exchange for expressiveness you cannot get from a fixed relation.

Approach	Proof size (64-bit range)	Trusted setup	Prover	Verify	Post-quantum	Aggregation
Bit-decomposition OR	O(n), large	No	O(n)	O(n)	No	Weak
Borromean	~6 kB/output	No	O(n)	O(n)	No	No
Bulletproofs	~0.6-0.7 kB, O(log n)	No	O(n)	O(n), batchable	No	Yes
Bulletproofs+	smaller constant, O(log n)	No	O(n)	O(n), batchable	No	Yes
SNARK (Groth16/PLONK)	~0.2 kB (Groth16) to ~0.4 kB (PLONK), O(1)	Yes	O(circuit)	~O(1)	No	Circuit-dependent
STARK	tens to hundreds of kB	No	O(n log n)	polylog	Plausibly	Limited

Two more axes decide the corner. Commitment choice is the Section 4 recap: need to add committed values, pick Pedersen; need post-quantum binding or no algebra, pick a hash commitment. And interactivity is a genuine fork: an interactive Sigma protocol is deniable -- because anyone could have simulated the transcript, it proves nothing to a third party -- and needs no random-oracle assumption, while Fiat-Shamir buys you a signature or an on-chain proof at the cost of making the proof transferable to everyone forever. Deniability versus non-interactivity is a real tension we return to in the open problems.

The challenge move is the shared weak point of proof systems well beyond this toolkit. PlonK -- a general-purpose SNARK, not a Sigma protocol -- was hit by the *same* weak Fiat-Shamir bug as Bulletproofs in the 2022 Frozen Heart disclosure [@tob-plonk], [@tob-part1]. And in 2025, Khovratovich, Rothblum, and Soukhanov demonstrated a *practical* soundness break against a GKR-based commercial proof system, tricking it into certifying false statements regardless of which hash function instantiates Fiat-Shamir [@khovratovich], [@quanta]. This boundary keeps the article honest: weak-challenge failures are not unique to Sigma protocols -- the move that most often leaks is the same across the whole family of proof systems.

The line is drawn by the shape of your statement and the trust you can assume. But even the right tool, used perfectly, runs into limits that no parameter choice can buy its way out of.

10. Theoretical Limits

The discrete log has held for forty years. But "the hard problem is hard" is not the same claim as "your proof is secure," and the gap between them is where this section lives. Every limit below survives even if the discrete logarithm stays hard forever.

Start with the good news, because it is genuinely good. For the interactive protocol, special soundness gives a tight, unconditional extractor -- two transcripts and three lines of algebra recover the witness, no assumptions -- and SHVZK gives a perfect simulator [@damgard]. Interactive Schnorr is about as close to an ideal proof of knowledge as anything in cryptography gets. The trouble starts when you make it non-interactive, or run many copies at once.

Soundness is only ROM-deep

Fiat-Shamir soundness is provable, but only in the random-oracle model, via the forking lemma [@pointcheval-stern]. In the standard model it is uninstantiable: Goldwasser and Kalai (FOCS 2003) exhibited schemes whose Fiat-Shamir transform is insecure for every concrete hash function you could plug in [@gk-focs]. That was a theoretical construction for two decades. Then in 2025, Khovratovich, Rothblum, and Soukhanov turned it practical, making a GKR-based commercial proof system accept false statements regardless of the hash instantiating the transform [@khovratovich], [@quanta].

Note: The Khovratovich-Rothblum-Soukhanov result targets Fiat-Shamir applied to GKR-style succinct arguments, not the short algebraic Sigma statements that strong Fiat-Shamir fully binds [@khovratovich]. It does not forge a Schnorr signature. What it does is harden the general lesson: Fiat-Shamir soundness is a property of the random-oracle idealization, and the more your proof leans on the hash doing something a real hash may not do, the thinner the ice [@gk-focs].

Forking is loose, and looseness is the seam

The random-oracle reduction is not tight. Rewinding a forger to extract the discrete log loses roughly a quadratic factor in the security bound [@pointcheval-stern]. This is not sloppy proof engineering; it is inherent to the rewinding technique.The quadratic looseness and the concurrency failure are the same fact seen twice. Rewinding extracts from one session by replaying it with a fresh challenge; it does not compose when many sessions interleave, because you cannot rewind one without disturbing the others. That is precisely the seam the ROS attack pries open [@pointcheval-stern], [@ros]. One line of work sidesteps rewinding entirely: the Fischlin transform (2005) uses an online, straight-line extractor that reads the witness out of a single transcript by inspecting the prover's oracle queries, so its extraction composes concurrently and carries more cleanly into the quantum random-oracle model -- at the price of larger proofs, which is why it coexists with Fiat-Shamir rather than replacing it [@fischlin]. And because rewinding does not compose across concurrent sessions, the single-session forking proof simply does not extend to the multi-party concurrent setting.

That gap has a sharp quantitative edge. Naive linear Schnorr multisignatures and blind signatures are secure only while the number of concurrent open sessions stays below roughly $\log_2 p$; above that threshold, the ROS forgery runs in polynomial time with no discrete log solved [@ros]. MuSig2's two-nonce nonlinear combination is the construction that evades the bound -- not a discrete-log advance, just a change that makes the response nonlinear so the linear system has no solution [@musig2].

Two settled negatives

Some limits are not open questions; they are proven impossibilities you must design around.

No commitment is both perfectly hiding and perfectly binding. The Move-1 trade-off of Section 4 is fundamental, not an artifact of Pedersen [@damgard]. You choose a corner; you do not get both.
None of this is post-quantum. Every guarantee in the article rests on the hardness of the discrete logarithm or of pairing problems, and Shor's algorithm breaks all of them on a sufficiently large quantum computer. No parameter choice rescues it -- a bigger group buys nothing against Shor. The hard problem that held against classical attackers for forty years is exactly the one a quantum computer dissolves.

The discrete log is hard, and this multisignature is secure under concurrency, are simply different claims -- and the second one was false until MuSig2 made the combined nonce nonlinear.

Those are the settled negatives. The open frontiers -- where the field is still moving -- are next, and the sharpest is the quantum cliff the whole toolkit stands at the edge of.

11. Open Problems

The toolkit is deployed and load-bearing, and it still has open edges sharp enough to end careers on. Here is what is genuinely unsolved, and which of it should keep a protocol designer awake.

Standard-model Fiat-Shamir. A universal, hash-instantiable, tight Fiat-Shamir is believed impossible in general. The 2003 uninstantiability result and the 2025 practical break together bound the ambition: the realistic goal is not a universal fix but per-protocol correlation intractability -- proving a specific hash is good enough for a specific proof system [@gk-focs], [@khovratovich]. Straight-line alternatives such as the Fischlin transform trade larger proofs for cleaner, rewinding-free extraction, but they are not a universal drop-in replacement [@fischlin]. For the short algebraic statements in this article, strong Fiat-Shamir remains the right practice; for succinct arguments, the question is live.

Post-quantum Sigma protocols. This is the sharpest frontier, because it is where the whole toolkit meets its deadline. Discrete-log zero knowledge dies to Shor -- but the three-move template does not. It reappears over other hardness assumptions.The Sigma template outlives its discrete-log instantiation. Fiat-Shamir-with-aborts over lattices is the shape behind Dilithium-style signatures, and MPC-in-the-head -- instantiated as Picnic -- builds a zero-knowledge signature from a hash and a secret-sharing protocol, needing no structured hardness assumption at all [@picnic]. Commit, challenge, respond survives the move to a post-quantum world; only the algebra underneath changes. What does not yet transfer cleanly is the efficient, verifiable, partially-oblivious machinery: post-quantum range proofs and post-quantum multi-show credentials exist but are large and immature compared to Bulletproofs and BBS. The thirty-year migration to post-quantum cryptography is the sibling part that tracks this deadline in full.

Round-optimal blind Schnorr after ROS. Plain two-round blind Schnorr is inherently ROS-vulnerable [@ros]. Getting a secure blind signature in few rounds needs extra structure: clause or boosting variants that break the linearity, a proof in the algebraic group model (Fuchsbauer, Plouviez, and Seurin), or a different scheme entirely [@fps]. There is no known plain, standard-model, round-optimal blind Schnorr that survives concurrency.

Threshold Schnorr, hardened. RFC 9591 standardized two-round threshold Schnorr for custody in June 2024, and FROST is now entering production custody [@rfc9591], but stronger guarantees remain research targets: adaptive security (against an attacker who corrupts signers mid-protocol), proofs that do not lean on the algebraic group model, and distributed key generation that scales asynchronously to large signer sets.

Anonymous-credential revocation at population scale without breaking unlinkability is still mostly finessed rather than solved -- deployed systems lean on short-lived credentials that simply expire, because a true privacy-preserving revocation list that scales to millions of holders is hard to make both efficient and unlinkable.

Deniability versus non-interactivity. A Fiat-Shamir'd proof is transferable -- anyone who receives it is convinced, forever. An interactive Sigma proof is deniable -- because it could have been simulated, it convinces only the live verifier. Getting both properties at once is a per-application tension, not a solved problem.

None of these block you from shipping today; they tell you what to watch. What you ship comes down to a small, unglamorous set of rules that separate a correct deployment from a forged one.

12. Use X With These Parameters In Case Y

Here are the rules a reviewer can apply without re-deriving anything, organized -- of course -- by the three moves, in the order you make them.

Move 1: pick the commitment by the job

Need to add or range-prove committed values -- confidential amounts, homomorphic tallies? Use a Pedersen commitment on ristretto255, and derive the second generator $h$ by hashing a fixed string to a curve point so that nobody holds the trapdoor $\log_g h$. A known trapdoor is silent equivocation; the generator setup is security-critical, not boilerplate. Need post-quantum binding, or you are committing to something with no algebra -- a Merkle leaf, a coin flip? Use a hash commitment H(m ‖ r).

Move 2: get the challenge right, every time

Use strong Fiat-Shamir: hash the full statement, all public parameters, the prior transcript, and a domain separator. The operational habit that prevents the entire Frozen Heart class is one sentence: for every proof, write down "what is this proof about?" and confirm each item on that list is inside the hash. The statement is about a public key? The public key goes in the hash. It is about a specific message or ciphertext? That goes in too. Weak Fiat-Shamir is what happens when that list is incomplete [@tob-part1].

Move 3: nonce discipline

Nonces must be fresh, secret, and uniform per session. Draw them from a well-seeded CSPRNG, or derive them deterministically with the tagged-hash discipline BIP-340 specifies so a broken RNG cannot repeat or bias them [@bip340]. In MuSig2 and FROST, honor the two-nonce and preprocessing rules exactly, and never reuse a nonce commitment across sessions -- the multi-party schemes defeat ROS only if the nonce rules are followed [@musig2], [@rfc9591].

Note: Move 1 -- is your second generator $h$ from a nothing-up-my-sleeve hash, with no known trapdoor? Move 2 -- write down what the proof is about, and verify every item is inside the challenge hash, behind a domain separator. Move 3 -- is every nonce fresh, secret, and uniform, and are the MuSig2 or FROST nonce rules followed to the letter? Three questions, in order. Most Frozen-Heart and nonce-reuse incidents fail one of them.

Then pick the composite

flowchart TD START{"What are you building?"} START -->|"commit to a value"| COMMIT{"need to add or range-prove it?"} COMMIT -->|"yes"| C1["Pedersen on ristretto255"] COMMIT -->|"no, need PQ binding"| C2["Hash commitment"] START -->|"prove a range"| C3["Bulletproofs plus, aggregated, strong FS"] START -->|"multi-party signature"| SIG{"all signers or t of n?"} SIG -->|"all n"| C4["MuSig2 BIP-327"] SIG -->|"t of n"| C5["FROST RFC 9591 plus ROAST"] START -->|"anonymous credential"| CRED{"who verifies?"} CRED -->|"the issuer itself"| C6["Signal KVAC"] CRED -->|"any third party"| C7["BBS"] START -->|"arbitrary circuit"| C8["SNARK or STARK"]

The composite rules, stated plainly:

Range proof, small and transparent: Bulletproofs+, aggregated, with strong Fiat-Shamir binding [@bulletproofs-plus].
All parties co-sign one key (n-of-n): MuSig2, standardized as BIP-327 [@bip327].
Any t of n signers suffice: FROST, RFC 9591, run with a distributed key generation -- and add ROAST if a faulty signer must not be able to stall the protocol [@rfc9591], [@roast].
Authenticate your own users anonymously (you issue and verify): Signal KVAC [@kvac], [@signal].
A third party must verify selective disclosure: BBS [@bbs-draft].
A fixed algebraic relation, tiny and setup-free: a hand-rolled Sigma proof with strong Fiat-Shamir, using a vetted library -- never your own group arithmetic.
An arbitrary circuit: a SNARK or STARK, accepting the setup or proof-size cost.
Anything that must stay secure through the quantum transition: do not rely on these guarantees; plan a post-quantum migration.

For the implementations, use the libraries that encode these defaults: bitcoin-core/secp256k1 and secp256k1-zkp for BIP-340 and MuSig2 [@bip340], [@bip327]; FROST implementations validated against the RFC 9591 test vectors [@rfc9591]; dalek-cryptography for Bulletproofs; Signal's libsignal and zkgroup for KVAC [@signal]; BBS libraries tracking the CFRG drafts [@bbs-draft]; and Damgard's notes together with Trail of Bits' ZKDocs when you need the correct Fiat-Shamir recipe written out [@damgard], [@tob-part1].

Most of the ways this goes wrong are the same handful of mistakes phrased for different audiences -- and they are worth naming head-on, because reviewers find them in real code every week.

13. Common Misuse, and the Questions Engineers Ask

The antipattern catalog is short because the failures are few and repeat. Reviewers find these in real code, and every one maps to a move.

Misuse	Move	Root cause	Fix
Weak Fiat-Shamir in real code	Move 2	statement omitted from the challenge hash	bind the full statement and parameters (strong FS)
Reused or biased nonce	Move 3	two transcripts share a commitment; extractor recovers the key	fresh, uniform, per-session nonces; hedged derivation
Assuming Pedersen is perfectly binding	Move 1	it is only computationally binding	design for computational binding; protect the trapdoor
Equivocation via a known `log_g h`	Move 1	someone knows the trapdoor	derive `h` by nothing-up-my-sleeve hash-to-curve
Rolling your own OR-proof or CDS composition	all three	simulator and challenge-splitting are subtle	use a vetted implementation
Naive linear multisig under concurrency	Move 3	linear responses solved by ROS	MuSig2 (n-of-n) or FROST (t-of-n)
Treating credentials as "just signatures"	Move 1-3	they are Sigma proofs with selective disclosure	model unlinkability and presentation proofs
"ROS means Random Oracle Substitution"	Move 3	mistranslated acronym	it is an overdetermined, solvable linear system
Conflating BIP-340 with EdDSA	Move 3	different encoding, nonce, and curve	pin the exact standard you mean

No. A Pedersen commitment is perfectly *hiding* and only *computationally* binding. A party who knows the trapdoor $\log_g h$ can open a single commitment to any message they like, by solving one linear equation for a matching blinder [@pedersen], [@damgard]. This is why the second generator must be produced so that nobody knows that trapdoor, and why "Pedersen is perfectly binding" is a dangerous thing to write in a spec. The full statement, all public parameters, the prior transcript, and a domain separator -- not just the commitment [@bpw]. Omitting the statement is weak Fiat-Shamir, the bug behind both the Helios voting break in 2012 and the Frozen Heart disclosures in 2022 [@bpw], [@tob-part1]. The habit that prevents it: write down what the proof is about, and confirm each item is inside the hash. Yes, and seeing that is worth the price of admission. A Schnorr signature is Fiat-Shamir applied to the Schnorr identification protocol, with the verifier's challenge replaced by a hash of the public key, message, and commitment [@fiat-shamir], [@bip340]. A signature *is* a non-interactive proof of knowledge of the private key; the message just rides along in the challenge. Don't. The CDS construction for AND/OR composition relies on splitting the challenge and simulating the branches you cannot answer, and getting the challenge algebra or the simulator subtly wrong breaks soundness or zero knowledge [@cds]. Use a vetted implementation; this is exactly the code that looks simple and is not. No. It is forgeable under concurrent sessions -- sub-exponentially since Drijvers at S&P 2019, and in polynomial time since the ROS result in 2021 [@drijvers], [@ros]. Use MuSig2 for n-of-n signing or FROST for t-of-n threshold signing; both are engineered specifically to defeat that attack [@musig2], [@rfc9591]. No. They are Sigma-protocol proofs of knowledge of a signature or an algebraic MAC, and they support selective disclosure and unlinkability -- a plain signature has neither [@bbs-draft], [@signal]. Treating a credential presentation as an ordinary signature verification misses the entire privacy machinery that is the point of the primitive. Privacy Pass and Apple's Private Access Tokens are the single-*show* authorization-token cousin -- unlinkable, but a one-time "you are allowed" token rather than a multi-attribute selective-disclosure credential [@rfc9576]. No, and the mistranslation is everywhere online. ROS is "Random inhomogeneities in an Overdetermined, Solvable system of linear equations" -- a linear-algebra object, not an oracle trick [@ros]. The attack forges by solving a system of equations over responses collected across concurrent sessions, with no discrete logarithm ever solved.

The discrete log held; the three moves leaked

Follow the whole failure catalog back and it lands in one place. In April 2022, three unrelated proof systems were forgeable at once, and the discrete logarithm was untouched in all three [@tob-part1]. A decade earlier, the same weak-Fiat-Shamir bug had forged ballot proofs in Helios [@bpw]. A reused nonce hands over a private key by the same algebra that proves Schnorr sound [@bip340]. The ROS attack forges naive multisignatures in polynomial time by solving a linear system, no hard problem broken [@ros]. A Pedersen commitment equivocates the instant someone knows a trapdoor [@pedersen].

Four decades, and the discrete logarithm never fell. Every named break -- Frozen Heart, Helios, nonce reuse, ROS, Drijvers, equivocation -- landed on one of the three moves built on top of it.

And the composites climbed while the foundations stayed put. Borromean gave way to Bulletproofs and Bulletproofs+ [@bulletproofs], [@bulletproofs-plus]; naive multisignatures gave way to MuSig2 and FROST [@musig2], [@rfc9591]; Chaum's blind signatures gave way, in deployment, to Signal KVAC, BBS, and Privacy Pass [@signal], [@bbs-draft], [@rfc9576]. Not one of those ladders replaced Pedersen, Schnorr, or Fiat-Shamir. The three moves keep running underneath every rung, exactly as they were written between 1985 and 1996 -- and where the foundation moved at all, it was Move 2 being hardened in place, weak Fiat-Shamir corrected to strong, never a piece torn out and replaced.

This is the frame the series keeps returning to from a new angle each time. Part 18 argued that nobody broke the discrete log; Part 17 that the math held and the interface leaked. Here, the math held and the conversation leaked -- because a discrete-log zero-knowledge proof was never one thing. It was always three moves, and each one is a separate promise that can be broken without touching the problem underneath.

The discrete log held. The three moves leaked.

The Server Helped, and Never Saw: A Field Guide to Oblivious Pseudorandom Functions

noreply@paragmali.com (Parag Mali) — Mon, 13 Jul 2026 10:23:11 GMT

**An oblivious pseudorandom function (OPRF) lets a server apply its secret key to your secret input so that it learns neither the input nor the output.** That single move is the engine under OPAQUE logins, Privacy Pass tokens, and private password-breach checks [@rfc9497]. The trick is David Chaum's 1982 blinding transplanted from RSA into a Diffie-Hellman group [@chaum82]: hash your input to a curve point, multiply by a fresh random blind, let the server multiply by its key, then divide the blind back out. RFC 9497 (2023) pins this down as three modes -- OPRF, VOPRF, and POPRF -- that answer three questions: is it blind, which key produced the answer, and how many times can one key be asked before it bleeds out [@rfc9497]. This is the field guide to that engine: every generation that built it, every real break that shaped it, and every way it still gets misused in production code.

1. The Server Hardened Your Password Without Ever Seeing It

You want to make a stolen password database worthless. The standard defense is to key every stored password with a secret the attacker cannot exfiltrate -- a "pepper" that lives only in your server's memory -- so a dumped hash is uncrackable without also breaking the keyholder. This is the logic behind server-assisted password hardening, and it is exactly what OPAQUE was designed to formalize [@opaque-paper].

But look closely at the obvious way to apply that key. To hash a password under the pepper, the server has to be handed the password. That is the one thing you were trying not to do. The service that hardens your secret must first be shown your secret. It feels like a law of nature: to key a value, you have to see the value.

It is not a law of nature. Here is the whole resolution, and it fits in four lines.

The client hashes its password to a point on an elliptic curve, $P = H(pw)$. It picks a fresh uniformly random scalar $r$ and sends not $P$ but $r \cdot P$, a uniformly random point. The server multiplies by its secret key: $k \cdot (r \cdot P)$. The client divides the blind back out to get $k \cdot P$, and hashes that: $F(k, pw) = H_2(pw,, k \cdot H(pw))$.

Now state exactly what the server saw. It saw $r \cdot P$: a uniformly random point on the curve, and nothing else. It never saw the password. It never saw the output. It applied its key to a value it could not read, and handed back a value it could not read either.

A two-party protocol that computes $F(k, x)$ where the server holds the key $k$, the client holds the input $x$, and when the protocol finishes the server has learned neither $x$ nor the output, while the client has learned only $F(k, x)$ and nothing about $k$ [@rfc9497]. A keyed function $F(k, x)$ whose outputs are indistinguishable from those of a truly random function to anyone who does not hold $k$. It is the "keyed lock": only the keyholder can compute it, and its outputs reveal nothing about the key [@rfc9497]. The fresh, uniformly random scalar $r$ that the client multiplies into $H(x)$ before sending it. Because $r$ is uniform and independent of the input, the transmitted point $r \cdot H(x)$ is itself uniform and carries no information about $x$ [@rfc9497]. sequenceDiagram participant C as Client (holds input x) participant S as Server (holds key k) Note over C: P holds HashToGroup(x) Note over C: pick fresh random blind r Note over C: blindedElement holds r times P C->>S: send blindedElement, a uniform random point Note over S: evaluatedElement holds k times blindedElement S->>C: return evaluatedElement Note over C: divide out r to recover k times P Note over C: output holds Hash of x with that point

Key idea: The oblivious PRF applies a secret key to a secret input so that neither party learns the other's secret. Everything else in this guide is a variation on that one move, or a story about what happens when one half of it is done wrong.

That two-line trick is not a party piece. The same two lines are how Privacy Pass hands you a token that a server can validate but cannot link back to you [@dgstv18]; how Google's Password Checkup tells you a credential appears in a breach corpus without Google learning the credential [@thomas19]; and how two organizations compute the intersection of their customer lists without either revealing its list [@fipr05]. Replace "the component that used to have to see the secret" with a blind, verifiable, rate-limited oracle that does not, and you have described every one of these systems at once.

Because the move is always the same, you can interrogate any deployment with the same three questions. Is it blind? Does the server ever see the input, or link two evaluations of it? Which key? Can the client prove the answer was produced by the one key the server committed to? How many asks? How many blind evaluations can one key answer before an attacker can reconstruct it? Hold onto those three questions. They are the spine of this article, and by the end you will see that every named failure in the primitive's history is one of them answered wrong.

The 2HashDH shape $F(k, x) = H_2(x,, H_1(x)^k)$ we just wrote by hand is the actual construction that ships today [@jkk14]. If it feels like Diffie-Hellman run one-sided, that is exactly right: Part 18 argued that nobody has broken the discrete logarithm, and this is its constructive twin -- the same group structure, run so that a server applies a secret rather than agreeing on one. If you are wondering why we harden passwords at all, Part 12 is the prerequisite. Here we take the hardening as given and ask how a server can perform it blind.

{` // STRUCTURAL MODEL, NOT REAL CRYPTO. It shows only the SHAPE of blind/evaluate/unblind. // Model an elliptic-curve group of prime order q: a "point" is an integer mod q, // and "scalar multiplication" scalar*point is multiplication mod q. BigInt avoids overflow. const q = 2147483647n; // a prime group order (toy) const mod = (a) => ((a % q) + q) % q; const scalarMul = (s, point) => mod(s * point); function invScalar(a) { // inverse of a scalar mod q (q prime), via Fermat let r = 1n, b = mod(a), e = q - 2n; while (e > 0n) { if (e & 1n) r = mod(r * b); b = mod(b * b); e >>= 1n; } return r; }

const k = 1337n; // SERVER secret key (client never learns it) const Hx = 991n; // H(x): the hashed input, as a group element const r = 42424243n; // CLIENT fresh random blind

const blinded = scalarMul(r, Hx); // client SENDS rH(x): looks uniformly random const evaluated = scalarMul(k, blinded); // server returns k(rH(x)) const unblinded = scalarMul(invScalar(r), evaluated); // client removes r: recovers kH(x) const direct = scalarMul(k, Hx); // what F would be if the server had seen x

console.log('server saw = ' + blinded.toString() + ' (independent of H(x)=' + Hx.toString() + ')'); console.log('client recovered = ' + unblinded.toString()); console.log('equals direct k*H(x) = ' + (unblinded === direct)); `}

That trick is forty years old, and for its first two decades it did not even have a name. To understand why it is secure, and why there turned out to be three versions of it, we have to start where the blinding started: in 1982, with digital cash.

2. Blinding, Before It Had a Name

The move is older than the problem it now solves. In 1982, David Chaum wanted untraceable digital cash: a way for a bank to sign a coin without learning which coin it signed, so the coin could later be spent without the bank linking it back to the withdrawal. His solution was blinding [@chaum82]. The client multiplies its message by a random factor, the bank signs the blinded message under its RSA key, and the client divides the factor out to recover a valid signature on the original. Blind, let the keyholder act, unblind. Structurally, that is the exact template an OPRF runs on.Chaum's 1982 construction used multiplicative blinding in an RSA group, and its output was a publicly verifiable signature. The OPRF performs the same blind-act-unblind dance in a Diffie-Hellman group, and its output is a pseudorandom value only the querier can read. Part 14 covers the RSA lineage in full.

So if the trick existed in 1982, why did the OPRF take until 2005 to be named and 2023 to be standardized? Because Chaum's version produced the wrong kind of output, and the right kind needed a second ingredient that did not exist yet.

Watch the shape of what Chaum built. A blind signature is public: anyone with the bank's public key can verify it. That is perfect for a coin you want a merchant to check, and exactly wrong for a pepper you want no one to be able to reproduce. To harden a password, you do not want a verifiable signature on it; you want a pseudorandom value that only the keyholder could have produced and only the querier can read.

Chaum had the blinding. He did not have a pseudorandom thing to blind. This is why "Blind RSA," standardized decades later as RFC 9474, is not an OPRF but its publicly verifiable cousin -- a distinction that will matter enormously when we reach Privacy Pass [@rfc9474].

The second ingredient arrived in 1997. Moni Naor and Omer Reingold built a pseudorandom function out of pure number theory: $F_k(x)$ computed as a single group exponentiation whose exponent is a subset-product of the key, $g^{\prod a_i^{x_i}}$, provably secure under the Decisional Diffie-Hellman assumption [@naorreingold97]. What mattered was not the security proof but the shape. Because the function was built from group exponentiations, it had enough algebraic structure that a blinded input could pass through it and come out blinded -- the homomorphic property that makes oblivious evaluation possible at all. A random hash function scrambles its input beyond recovery; an algebraically structured PRF lets the blind survive the trip.

Now the two ingredients were on the table. In 2005, Michael Freedman, Yuval Ishai, Benny Pinkas, and Omer Reingold fused them, gave first constructions, and coined the term: the Oblivious Pseudorandom Function [@fipr05]. They were not thinking about passwords. They wanted private keyword search and private set intersection -- letting one party query another's database, or intersect two sets, without either side revealing its contents.

The abstraction they named is the one we still use: transplant Chaum's blinding out of RSA, point it at a Diffie-Hellman-structured PRF, and you get a value that is pseudorandom rather than a signature, and blind rather than seen.RFC 9497 standardizes multiplicative blinding, the variant described here: its Blind step computes blindedElement = blind * inputElement, and that multiplicative form is what ships.

flowchart LR A["1982 Chaum blinding in RSA"] --> B["1986 Fiat-Shamir transform"] B --> C["1992 Chaum-Pedersen DLEQ proof"] C --> D["1997 Naor-Reingold algebraic PRF"] D --> E["2005 FIPR names the OPRF"] E --> F["2009 Jarecki-Liu efficient DH OPRF"] F --> G["2014 2HashDH, the workhorse"] G --> H["2018 Privacy Pass VOPRF"] H --> I["2022 3HashSDHI POPRF"] I --> J["2023 RFC 9497 standard"] J --> K["2025 RFC 9807 OPAQUE"]

Two other pieces on that timeline were quietly waiting to become load-bearing. In 1986, Fiat and Shamir showed how to turn an interactive proof into a one-shot, non-interactive one by replacing the verifier's random challenge with a hash of the transcript [@fiatshamir86]. In 1992, Chaum and Pedersen gave a zero-knowledge proof that two discrete logarithms are equal [@chaumped92]. Neither was about OPRFs. Both become the machinery of verifiability when, twenty-six years later, Privacy Pass needs a client to check which key the server used. Keep them in view; they return in Section 5.

The problem was defined and it had a name. What it did not have was a construction cheap enough to ship. And the first one that looked practical turned out to have a slow poison built into it.

3. The First Construction That Got Weaker With Use

Naming a primitive is not the same as having a recipe worth shipping. The constructions that came with the 2005 definition were general and powerful and, for a deployed service, awkward. The oblivious-transfer and oblivious-polynomial-evaluation approaches were round-heavy. The Naor-Reingold-based ones inherited the PRF's per-bit structure: they consumed the input one bit at a time, so their cost scaled with the input length rather than staying flat.The linear-in-input-length cost of the early Naor-Reingold-style OPRFs is an inference from that PRF's per-bit construction rather than a benchmark quoted from the 2005 paper; treat it as a structural signpost, not a measured figure. For private set intersection in a research prototype, fine. For a server answering millions of blind queries a second, not yet.

The efficient turn came in 2009. Stanislaw Jarecki and Xiaomin Liu built the first OPRF that was both fast and committed -- verifiable -- in a prime-order group [@jareckiliu09]. Instead of the exponentiation-per-bit structure, it used a Dodis-Yampolskiy-style PRF of the form $g^{1/(k+x)}$: the server inverts $k + x$ in the exponent, so evaluation is a single group operation regardless of input length.

And because the construction committed to the key, the client got assurance not just that some key was applied but that it was the server's key. In the language of our three questions, this one answered Q1 and Q2 at once. It looked like the deployable OPRF.

Then look at the assumption it rested on. Security of the $g^{1/(k+x)}$ family reduces to a q-strong-Diffie-Hellman assumption: hardness holds as long as the attacker cannot exploit a growing collection of key-dependent samples. And three years earlier, in 2006, Jung Hee Cheon had shown precisely how to exploit them. Cheon's analysis proved that the strong-Diffie-Hellman family weakens as the attacker obtains more samples tied to the same secret -- security degrades with the number of queries [@cheon06].

Sit with what that means for an OPRF specifically. An OPRF's entire job is to be offered as a service: to answer blind query after blind query, indefinitely. But every answer this construction gave was another key-dependent sample, and Cheon's result said each sample chipped away at the secret. The primitive got weaker the more it was used. A lock that loosens every time you turn the key is a strange thing to install on a vault you intend to open a billion times.

Note: Most cryptographic primitives are static: a hash function is exactly as strong after a trillion evaluations as after one. The 2009 Jarecki-Liu OPRF was not. Its security was a budget that the act of answering queries spent down, because each blind evaluation handed the attacker another sample against the strong-Diffie-Hellman assumption it relied on [@cheon06]. For a primitive whose whole purpose is to be queried without limit, that is a design-level contradiction, not a tuning problem. This is our third question -- how many asks? -- appearing for the first time, and appearing as a wound rather than a footnote.

This is worth marking as special: it is the only true assumption-level break in the whole story. Everything else that supersedes something in the lineage does so because it is cheaper, or answers one more question. Only here does a construction fall because the mathematical ground under it gave way. The strong-Diffie-Hellman assumption was the wrong foundation, and no engineering on top could fix a foundation that eroded by design.

The fix was not a cleverer attack defense. It was to stop building on an assumption that decays -- to hash the input to one group element and re-base the whole construction on an assumption that does not weaken sample by sample. That single stroke produced the workhorse that every deployment now runs, and it is where the OPRF stops being a research object and becomes infrastructure.

4. Moving the Server From Trusted to Blind

Here is the whole next fifteen years compressed into one sentence: each generation moves the server one step from "must be trusted, sees the secret" toward "blind, verifiable, query-limited," and -- the part that surprises people -- most of those steps are additive. A later construction rarely throws out its predecessor. It bolts one more guarantee onto it. To read the lineage correctly you have to separate two kinds of arrow: superseded by, a genuine replacement, and extended by, an additive answer to one more question with the old construction still running underneath.

The workhorse: 2HashDH

In 2014, Stanislaw Jarecki, Aggelos Kiayias, and Hugo Krawczyk wrote down the construction we built by hand in Section 1: $F(k, x) = H_2(x,, H_1(x)^k)$ [@jkk14]. Hash the input into the group with $H_1$, exponentiate by the key, hash out with $H_2$.

It is round-optimal -- one message each way, and you provably cannot do it in fewer. And its security rests on the One-More Gap Diffie-Hellman assumption rather than the strong-Diffie-Hellman family, which is the whole point: One-More Gap security is not a function of a growing pile of key-dependent samples, so it sidesteps the Cheon erosion that doomed the 2009 construction."One-More" assumptions bound how much an attacker can produce relative to how many genuine evaluations it was given -- it cannot get "one more" than it was handed. This framing is what breaks the dependence on a growing q-tuple that made the strong-Diffie-Hellman family decay under Cheon's attack. Because the input is hashed to a single group element, cost is flat in input length. Jarecki, Kiayias, Krawczyk, and Jiayu Xu put it on composable footing two years later [@jkkx16].

The construction $F(k, x) = H_2(x,\, H_1(x)^k)$: hash the input into a prime-order group, raise it to the secret key, and hash the result back out with the input bound in. It is the OPRF and VOPRF core standardized in RFC 9497 [@jkk14], [@rfc9497].

This is the OPRF that ships. But notice what it does not do. It answers Q1 -- the server never sees the input -- and nothing more. The client has no way to tell which key the server used. A malicious server can quietly assign a different key to each client and, later, use the distinct responses to tell which client was which. In an anonymity system that is fatal: the primitive designed to hide you becomes the tag that identifies you. 2HashDH left Q2 open. The next generation did not replace it; it extended it.

Verifiability you can batch: the VOPRF

In 2018, Alex Davidson, Ian Goldberg, Nick Sullivan, George Tankersley, and Filippo Valsorda shipped Privacy Pass, and with it the deployed VOPRF [@dgstv18]. The addition is precisely the 1992 machinery we set aside. The server publishes a commitment to its key -- a public key $pk_S = k \cdot G$ -- and alongside each response attaches a Chaum-Pedersen proof that the same $k$ relates its public key and its answer, made non-interactive by Fiat-Shamir [@chaumped92], [@fiatshamir86]. The client verifies before it accepts. Answering Q2, at last.

The engineering move that made it deployable was batching: many evaluations fold into a single proof via a random linear combination, so a client redeeming a thousand tokens verifies one proof, not a thousand [@dgstv18]. That is what let Privacy Pass run at content-delivery-network scale. That scale is what surfaced the primitive's real-world failure modes -- key consistency, tagging, rotation, double-spend -- that no research prototype had stress-tested.

An OPRF augmented with a proof that the server applied the key committed in its published public key, so a client can reject an answer computed under any other key. In RFC 9497 the proof is a batchable Chaum-Pedersen DLEQ [@rfc9497], [@dgstv18].

The VOPRF still left one gap. Its verifiability binds one key to one public key, which means one key cannot cleanly serve many domains, tenants, or rotation epochs -- to separate them you would need a different key per context, and a different public key per context is, again, a tag. What was missing was a way to fold a public label into the evaluation without changing the key.

Partial obliviousness: heavy, then light

The first answer came early and expensive. In 2015, Adam Everspaugh, Rahul Chatterjee, Samuel Scott, Ari Juels, and Thomas Ristenpart built Pythia, the first partially oblivious PRF: a public per-tenant tweak let one server key serve many domains and support bulk key rotation, while the password stayed oblivious [@pythia]. It worked, but it needed bilinear pairings -- special curves, heavier operations, a bigger dependency. Powerful, but not the thing you reach for by default.

The light answer came in 2022. Nirvan Tyagi, Sofia Celi, Thomas Ristenpart, Nick Sullivan, Stefano Tessaro, and Christopher Wood built 3HashSDHI: the first POPRF that needs no pairings [@tcrstw22]. It folds the public input info into the key by inversion. The server computes $t = k + H(\text{info})$ and returns $t^{-1} \cdot \text{blindedElement}$, with a DLEQ proof against the tweaked key. The private input stays oblivious; info is public and known to both sides. And it runs, in the authors' words, as fast as the standards-track 2HashDH protocol [@tcrstw22].

A VOPRF that also binds a *public* input `info` into the evaluation, computing $F(k, x, \text{info})$ while keeping $x$ oblivious. One published key can then serve many public contexts -- tenants, rotation epochs, rate-limit buckets. A POPRF with a fixed `info` is functionally a VOPRF [@rfc9497], [@tcrstw22].

Here is the detail worth savoring. To fold in info, 3HashSDHI uses key inversion -- $t^{-1}$ where $t = k + H(\text{info})$ -- which is the same Dodis-Yampolskiy inversion structure Jarecki and Liu used back in 2009 [@tcrstw22], [@jareckiliu09]. The mechanism that got the field burned by Cheon's attack was never actually wrong. What was wrong was the assumption it had been analyzed under. Tyagi and co-authors re-proved the same inversion under a One-More Gap assumption that reduces to plain discrete log in the algebraic group model, and the old idea walked back onto the field rehabilitated [@tcrstw22].

The lesson is one every protocol designer should internalize: a construction can be sound while the security assumption used to justify it is not, and better analysis can resurrect a "broken" mechanism untouched.

The other lineage, in one breath

Not every OPRF is a Diffie-Hellman OPRF. In 2016, Vladimir Kolesnikov, Ranjit Kumaresan, Mike Rosulek, and Ni Trieu built a completely different one: a batched OPRF from oblivious-transfer extension, symmetric-key rather than public-key, and enormously fast in bulk [@kkrt16]. It is not verifiable, not publicly reusable, and only semi-honest secure, but it intersects two sets of a million ($2^{20}$) elements each in about 3.8 seconds [@kkrt16], and OT-extension OPRFs of this kind were for years the throughput workhorse of high-volume semi-honest PSI, until VOLE-based PSI overtook them in communication for very large sets [@rindalvolepsi]. The takeaway for a practitioner: "OPRF" names a goal, and at least two very different mechanisms reach it. Do not assume every system labeled OPRF is the standardized Diffie-Hellman family.

Approach	Year	Key idea	Questions answered	Fate
Chaum blind signature	1982	Blind, sign, unblind in RSA	none (a signature, not a PRF)	Ancestor; became Blind RSA [@chaum82]
FIPR generic OPRF	2005	Obliviously evaluate an algebraic PRF; names "OPRF"	Q1	Superseded on cost [@fipr05]
Jarecki-Liu	2009	$g^{1/(k+x)}$, committed	Q1 + Q2	Superseded: strong-DH erodes with use [@jareckiliu09]
2HashDH	2014	$H_2(x, H_1(x)^k)$; One-More Gap CDH	Q1	Active core of RFC 9497 [@jkk14]
VOPRF	2018	2HashDH + batched Chaum-Pedersen DLEQ	Q1 + Q2	Active; extends 2HashDH [@dgstv18]
Pythia	2015	Verifiable partially oblivious PRF, pairings	Q1 + Q2 + info	Superseded by 3HashSDHI [@pythia]
3HashSDHI	2022	$(k + H(\text{info}))^{-1}$; no pairings	Q1 + Q2 + info	Active POPRF of RFC 9497 [@tcrstw22]
KKRT16	2016	Batched OPRF from OT-extension (symmetric)	Q1 (relaxed, semi-honest)	Active, niche: high-volume PSI [@kkrt16]

Twenty years of constructions now answered all three questions. But each was a separate paper with its own parameters, its own group choices, and its own foot-guns. The last move was not a new construction. It was a standard that unified them and, unusually for a specification, wrote the one unavoidable cost down as a number.

5. RFC 9497: Three Modes, One Interface

In December 2023, the IRTF's Crypto Forum Research Group published RFC 9497, written by Alex Davidson, Armando Faz-Hernandez, Nick Sullivan, and Christopher Wood [@rfc9497]. It did what twenty years of individual papers had not: it turned the whole family into a single misuse-resistant interface, fixed five ciphersuites, mandated the safety checks, and -- the honest move no construction paper had made -- wrote the one attack it cannot design away down as a number.

Start with the interface, because all three modes share it. Over a prime-order group with generator $G$, and using the hash-to-group machinery of RFC 9380, the protocol is three calls [@rfc9497], [@rfc9380]:

Blind(x):
    blind           <- random nonzero scalar               # fresh, uniform, per call
    blindedElement  <- blind * HashToGroup(x)              # a uniformly random point
    if blindedElement == Identity: abort                   # element validation
    return blind, blindedElement

BlindEvaluate(k, blindedElement, [info]):
    OPRF / VOPRF:  evaluatedElement <- k * blindedElement
    POPRF:         t <- k + HashToScalar(info)
                   evaluatedElement <- t^(-1) * blindedElement
    # VOPRF and POPRF also emit a DLEQ proof (see below)

Finalize(x, blind, evaluatedElement, [info]):
    N <- blind^(-1) * evaluatedElement    # OPRF/VOPRF: = k * HashToGroup(x); POPRF: = HashToGroup(x)^(1/(k+H(info)))
    return Hash( len(x) || x || [len(info) || info ||] len(N) || N || "Finalize" )

The server sees only blindedElement, a uniformly random point independent of $x$; the client recovers exactly one PRF value and nothing about $k$ [@rfc9497]. That is the same 2HashDH we have been circling, now with every byte of the output hash pinned down.HashToGroup is RFC 9380 hash-to-curve, the subject of Part 16. Its defining property is that the output point has an unknown discrete logarithm: no one, including the client, knows a scalar relating it to the generator. That unknown-discrete-log property is exactly why blinding hides the input and why the server cannot invert the map.

The three modes are three answers

This is where the second shift in understanding lands. The three modes are not three products. They are three points on one axis, distinguished by exactly which of our questions they answer.

The base OPRF answers Q1 and stops. BlindEvaluate is just $k \cdot \text{blindedElement}$, with no proof. The client trusts that some key was applied; it gets no assurance about which. Cheapest mode, smallest messages, right only when a wrong key can hurt no one but the querier [@rfc9497].

The VOPRF answers Q1 and Q2. The server commits to a public key $pk_S = k \cdot G$ and attaches a proof; the client verifies before finalizing [@rfc9497]. The POPRF answers Q1 and Q2 and binds a public info, via the key-inversion tweak. A POPRF with a fixed info collapses back to a VOPRF [@rfc9497]. Base to verifiable to partially oblivious: blind, then blind-plus-which-key, then blind-plus-which-key-plus-public-tweak.

Key idea: Three modes are three answers to three questions. OPRF answers "is it blind?" VOPRF adds "which key?" POPRF adds "and bind this public label." You do not choose a mode by taste; you choose it by counting how many of the three questions your threat model forces you to answer.

flowchart LR Q1["Q1: is it blind?"] --> OPRF["OPRF: blind only"] OPRF -->|add DLEQ proof against one committed key| VOPRF["VOPRF: blind plus which key"] VOPRF -->|fold in a public info tweak| POPRF["POPRF: blind, which key, public label"]

Verifiability, in depth

The proof is the 1992 machinery finally doing its job. A Chaum-Pedersen DLEQ proof establishes that two discrete logarithms are equal: that the $k$ relating $pk_S$ to $G$ is the same $k$ relating evaluatedElement to blindedElement [@rfc9497], [@chaumped92]. If the server used any other key, no valid proof exists. Fiat-Shamir turns the interactive proof into a single non-interactive object the client checks offline, so verifiability costs no extra round [@fiatshamir86].The DLEQ proof is a sigma protocol made non-interactive by Fiat-Shamir -- the same transform behind Schnorr signatures, which Part 17 covers. The client recomputes the challenge as a hash of the transcript and checks one equation; there is no back-and-forth with the server.

A non-interactive zero-knowledge proof that two discrete logarithms are equal -- here, that the server's public key and its response were produced under the same secret key. It is what makes a VOPRF verifiable, and via a random linear combination it can prove a whole batch of responses with one object [@rfc9497], [@chaumped92].

Batching is what makes verifiability affordable. Many evaluations fold into one proof through a random linear combination, so a client redeeming a batch verifies a single object, not one proof per element [@rfc9497], [@dgstv18]. That is one proof object per batch, not $n$ -- but verifying it still forms one linear combination across the batch, so the win is skipping $n$ separate proof checks, not making verification constant-time in the batch size. That is what let anonymous-token issuance scale.

There is a subtlety that quietly carries the entire security of any anonymity system built on a VOPRF: verifiability is only as strong as key consistency. Proving the server used the key in its public key is worthless if the server hands a different public key to every client, because then the unique key is itself a deanonymizing tag. Verifiability protects you only when everyone is served under one transparently published key [@rfc9497]. We will return to this in the practical guide; for now, note that "add a proof" and "pin one key" are two requirements, not one.

sequenceDiagram participant C as Client (input x) participant S as Server (key k, public pkS) Note over C: blindedElement holds blind times HashToGroup(x) C->>S: send blindedElement Note over S: evaluatedElement holds k times blindedElement Note over S: GenerateProof binds pkS and the answer to one k S->>C: return evaluatedElement and DLEQ proof Note over C: VerifyProof against pkS, abort on failure Note over C: unblind, then Finalize with domain separation

The machinery that makes it safe

A specification earns "misuse-resistant" in the details, and RFC 9497's details are where the accumulated scar tissue lives. Hashing and group operations are delegated, not reinvented: HashToGroup and HashToScalar come from RFC 9380, and the groups themselves from RFC 9496 -- ristretto255 and decaf448 -- or the NIST curves [@rfc9497], [@rfc9380], [@rfc9496].ristretto255 and decaf448 are prime-order abstractions built over Edwards curves whose raw order has a small cofactor. They remove the cofactor pitfalls that Part 15 covers -- the small-subgroup and equivalent-point traps -- which is exactly why RFC 9497 prefers them and why it mandates rejecting the identity element on deserialization.

Three requirements are worth naming because dropping any one silently breaks the guarantee:

Note: RFC 9497 requires that every deserialized element be validated -- on-curve, in the right range, and not the identity -- or an attacker can probe the key with malformed inputs [@rfc9497]. It requires that operations on secret data, including GenerateProof and BlindEvaluate, run in constant time, or the server leaks its key through timing [@rfc9497]. And it requires domain separation: the literal "Finalize" tag and the mode context string keep one application's evaluations from colliding with another's [@rfc9497]. None of these is optional hardening; each is load-bearing for a property the primitive claims.

The domain separation is the discipline Part 13 argued for, applied here: the output hash binds the input length, the input, the point, and a fixed label, so no two contexts can be coaxed into producing the same value. Inputs are bounded -- shorter than $2^{16}-1$ bytes, with a two-byte length prefix -- so longer inputs must be pre-hashed [@rfc9497].

The guarantee that the server cannot link two evaluations of the same input, nor learn anything about the input. Because the blind is fresh and uniform, this holds *even against an adversary with unbounded computation* -- it is information-theoretic, not merely computational [@rfc9497], [@dgstv18].

{` // STRUCTURAL MODEL. Shows why Finalize binds lengths + a "Finalize" tag, not just N. function toyHash(str) { // a tiny non-cryptographic stand-in let h = 5381 >>> 0; for (const ch of str) h = (Math.imul(h, 33) ^ ch.charCodeAt(0)) >>> 0; return (h >>> 0).toString(16).padStart(8, '0'); } // length-prefixed encoding: len(x) || x, so boundaries are unambiguous function lp(s) { return String(s.length).padStart(4, '0') + s; }

// RFC 9497 Finalize binds: len(x) || x || len(N) || N || "Finalize" function finalize(x, N) { return toyHash(lp(x) + lp(N) + 'Finalize'); }

// Without length prefixes, ("ab","c") and ("a","bc") would collide. With them, they don't: console.log('naive ab|c =', toyHash('ab' + 'c')); console.log('naive a|bc =', toyHash('a' + 'bc'), '(collides above)'); console.log('finalize ab,c =', finalize('ab', 'c')); console.log('finalize a,bc =', finalize('a', 'bc'), '(distinct)'); `}

Finally, the honesty. RFC 9497 section 7.2.3 states outright that every BlindEvaluate is a static-Diffie-Hellman oracle sample, and quantifies the erosion: the best-known attacks reduce the group's security by $\log_2(Q)/2$ bits, where $Q$ is the number of calls [@rfc9497], [@browngallant04], [@cheon06]. That is the domesticated ghost of the very failure that killed the 2009 construction -- no longer a surprise, but a budget line. We give it the full treatment it deserves in Section 8. First: who actually runs this, and what looks like it but is not.

6. Who Runs This Today

The OPRF is quietly load-bearing. You almost certainly used one this week without knowing it, because it lives one layer below the protocols you do know. Here are the four deployed consumers, each at "exactly where and why it calls the OPRF," and each paired with the look-alike an expert must not confuse it with.

OPAQUE, standardized as RFC 9807 in July 2025. This is the flagship. OPAQUE is an augmented password-authenticated key exchange: the server authenticates you by password without ever seeing the password, even at registration, and with no cleartext salt an attacker could precompute against [@rfc9807], [@opaque-paper]. The OPRF is the tool that makes the password oblivious, and it is what defeats precomputation attacks -- there is nothing static to build a rainbow table against.

RFC 9807 derives a per-user OPRF key from a single seed, keyed by each credential_identifier, so a server compromise does not let an attacker enumerate accounts, and it discusses a threshold-OPRF defense that forces an attacker online before any dictionary attack can even begin [@rfc9807]. The RFC names WhatsApp's end-to-end encrypted backups as a driving use case, and Meta's audited opaque-ke is the reference implementation [@rfc9807], [@opaque-ke]. If Part 12 explained why offline dictionary attacks are the thing to fear, OPAQUE is the OPRF's answer to them.

Privacy Pass, RFCs 9576, 9577, and 9578, June 2024. The in-scope token type is 0x0001 = VOPRF(P-384, SHA-384); RFC 9578 says plainly that its privately verifiable issuance is based on the OPRF [@rfc9578], [@rfc9576], [@rfc9577]. This is the VOPRF most engineers meet by name. But Privacy Pass is not one thing, and this is the boundary the whole article has been protecting.

RFC 9578 defines two interchangeable token types. Token `0x0001` is a VOPRF, and it is an OPRF. Token `0x0002` is Blind RSA (2048-bit), standardized in RFC 9474, and it is a blind *signature* -- Chaum's 1982 construction, publicly verifiable, and emphatically *not* an OPRF [@rfc9578], [@rfc9474]. The most visible Privacy Pass deployment in the world, Apple's Private Access Tokens, uses the Blind-RSA token, not the VOPRF one. The two make a real, standardized trade-off: VOPRF tokens are issuer-verifiable only, small, and easy to rotate; Blind-RSA tokens are publicly verifiable with the RSA public key but larger and heavier to rotate [@rfc9578]. So never write "Privacy Pass equals OPRF." One of its two token types is not an OPRF at all -- a point the earlier field-guide entry on anonymous credentials shipping develops further.

Password breach checking. Google's Password Checkup performs a blinded, OPRF-based lookup: it checks your credential against a corpus of breached ones without Google learning the credential [@thomas19], [@google-blog]. Microsoft's Password Monitor in Edge does something related but not identical, and the difference matters.

It is tempting to describe Microsoft Password Monitor as "an OPRF breach check," but that undersells the machinery. The OPRF is one layer; the actual set-matching runs on *homomorphic encryption* -- Labeled Private Set Intersection built on the Microsoft SEAL library, evaluating the match under encryption [@msft-monitor]. The OPRF hides the query; the fully homomorphic layer performs the private lookup against the breach set. Do not attribute the set intersection to the OPRF alone. And note the well-known *non-OPRF* alternative for the same job: Have I Been Pwned's k-anonymity range API, where the client sends the first five hex digits of a SHA-1 hash and receives the whole bucket to search locally [@cloudflare-kanon]. That leaks a hash prefix by design; it is prefix-bucketed hashing, simpler and cache-friendly, but not oblivious. The earlier entry on [Edge's two password cryptographies](/blog/edge-two-password-cryptographies/) traces this contrast in detail.

OPRF-based private set intersection. OPRF-PSI is one approach among several -- plain Diffie-Hellman PSI, oblivious-transfer and circuit-based PSI, and homomorphic-encryption PSI all compete for different workloads [@fipr05]. Even within OPRF-PSI, the two mechanisms from Section 4 occupy different corners: the public-key 2HashDH family when you want few messages and verifiability, and the symmetric KKRT16 OPRF when you want raw throughput over enormous sets [@kkrt16].

Consumer	What the OPRF does there	The non-OPRF look-alike to not confuse it with
OPAQUE (RFC 9807)	Makes the password oblivious; defeats precomputation; per-user keys block enumeration [@rfc9807]	A plain salted or peppered hash, where the server must see the password
Privacy Pass token `0x0001`	VOPRF issuance of unlinkable, issuer-verifiable tokens [@rfc9578]	Token `0x0002` = Blind RSA, a publicly verifiable blind signature (Apple PAT) [@rfc9474]
Google Password Checkup	Blinded OPRF lookup against a breach corpus [@thomas19]	HIBP k-anonymity range API: leaks a SHA-1 prefix, not oblivious [@cloudflare-kanon]
Microsoft Password Monitor	OPRF hides the query; FHE Labeled PSI does the match [@msft-monitor]	Assuming the OPRF alone does the set intersection
OPRF-based PSI	Oblivious set membership via 2HashDH or KKRT16 [@kkrt16]	DH-PSI, OT/circuit-PSI, and FHE-PSI, which are different approaches

The production substrates are a short list: Cloudflare's CIRCL in Go, the voprf crate and opaque-ke in Rust, and curve25519-dalek providing the ristretto255 group backend [@circl], [@voprf-crate], [@opaque-ke], [@dalek]. Four deployments, one primitive, three look-alikes that are emphatically not it. When you actually have to choose, the decision comes down to a small matrix.

7. The Decision Matrix

Given a real problem, each decision has a crisp rule. Here they are, head to head.

Which mode?

The primary choice is the mode, and it falls straight out of the three questions.

Dimension	OPRF (base)	VOPRF	POPRF
Questions answered	Q1 (blind)	Q1 + Q2 (which key)	Q1 + Q2 + public tweak
Extra output	none	DLEQ proof (batchable)	DLEQ proof (batchable)
Public input `info`	no	no	yes, $F(k, x, \text{info})$
Construction	2HashDH	2HashDH + DLEQ	3HashSDHI
Assumption	One-More Gap CDH	One-More Gap CDH	One-More Gap SDHI (to q-DL, AGM)
Client trust in key	must trust	verifies	verifies
Best suited for	trusted-key internal oracle	anonymity systems, OPAQUE	per-tenant, rotation, buckets

Sources: RFC 9497 sections 3.3.1 through 3.3.3, plus DGSTV18 and TCRSTW22 [@rfc9497], [@dgstv18], [@tcrstw22]. The rule: if you trust the server's key by construction and need only obliviousness, use the base OPRF. If any adversarial server could hand you a wrong or per-client key -- anonymity systems, OPAQUE -- use the VOPRF. If you need to bind a public label such as a per-tenant salt, rotation epoch, or rate-limit bucket, use the POPRF, and never put a secret in info.

Which ciphersuite?

Pick the ciphersuite by your query budget, because the group size is what buys you static-Diffie-Hellman headroom (Section 8).

Ciphersuite	Group	Nominal security	Use when
ristretto255-SHA512	ristretto255	128-bit	Default: prime-order, fast, no cofactor traps
decaf448-SHAKE256	decaf448	224-bit	Internet-facing oracle needing static-DH headroom
P-256-SHA256	NIST P-256	128-bit	FIPS or interop constraints
P-384-SHA384	NIST P-384	192-bit	Higher budget; Privacy Pass `0x0001` uses this
P-521-SHA512	NIST P-521	256-bit	Maximum static-Diffie-Hellman headroom

Source: RFC 9497 sections 4.1 through 4.5 [@rfc9497]. The rule: default to ristretto255-SHA512. If you expose an unmetered public oracle on a 128-bit group, the $\log_2(Q)/2$ erosion argues for decaf448, P-384, or P-521 -- RFC 9497 recommends exactly these for applications that cannot tolerate discrete-logarithm security below 128 bits [@rfc9497]. Choose P-256-SHA256 only when FIPS or interoperability forces it. And never roll your own group or hash-to-curve; use RFC 9380 and RFC 9496 [@rfc9380], [@rfc9496].

Which token cryptography?

If you are issuing anonymous tokens, the choice between the two Privacy Pass token types is a genuine trade-off, not a default.

Dimension	VOPRF token `0x0001`	Blind-RSA token `0x0002`
Primitive	OPRF (VOPRF, P-384/SHA-384)	Blind signature (RSA-2048), not an OPRF
Verifiability	Issuer-verifiable only (secret key)	Publicly verifiable (RSA public key)
Token size	Small	Larger
Key rotation	Easy	Heavier (RSA keypair)
Flagship deployment	CDN anonymous rate-limiting	Apple Private Access Tokens
Spec	RFC 9578, RFC 9497	RFC 9578, RFC 9474

Sources: RFC 9578 and RFC 9474 [@rfc9578], [@rfc9474]. The rule: if only the issuer needs to verify tokens and you want small, easily rotated tokens, use the VOPRF token. If a third party must verify a token without contacting the issuer, you need public verifiability, and that means a blind signature -- the Blind-RSA token -- which is not an OPRF at all.

The remaining two, briefly

For private set intersection, choose the public-key 2HashDH OPRF when you want few messages and malicious-secure verifiability, and the symmetric KKRT16 OPRF when you want to intersect enormous sets fast under a semi-honest model [@jkk14], [@kkrt16]. For breach checking, choose an OPRF when query privacy is a hard requirement, and k-anonymity when simplicity and cache-friendliness outweigh the leak of a hash prefix [@thomas19], [@cloudflare-kanon].

Every row above assumes one thing worth making explicit, because it is the property the whole family exists to deliver. Against a plain salted or peppered server-side hash, the server must see the secret to key it. That is the single compromise the OPRF removes, and each decision in this matrix is a variation on how much structure you are willing to add on top of "the server never saw it." Which is worth asking: what, precisely, can this primitive prove -- and what can it provably not?

8. What the Assumptions Buy, and What They Cost

If the discrete logarithm has never been broken, what can still go wrong with an OPRF -- and what can never be fixed? The answer separates cleanly into what the primitive provably delivers, what it rests on, and the one price it can never stop paying.

What it delivers. RFC 9497 section 7.1 enumerates the guarantees [@rfc9497]. Pseudorandomness: for a random key, the output is indistinguishable from a random function. Nonmalleability: you cannot turn one evaluation into another. Verifiability, in the VOPRF and POPRF modes. Partial obliviousness, in the POPRF. And the one worth pausing on -- unconditional input secrecy. Because the blind is fresh and uniform, the point the server receives is uniform and independent of the input, so input secrecy holds even against an adversary with unbounded computation [@rfc9497]. That is not "secure until someone finds a faster algorithm." It is information-theoretic. The secret was never in the message, so there is nothing for computation to extract.

What it rests on. The OPRF and VOPRF rest on the well-studied One-More Gap Computational Diffie-Hellman assumption; the POPRF on One-More Gap Strong-Diffie-Hellman-Inversion [@rfc9497], [@jkk14], [@tcrstw22]. The POPRF's assumption comes with a load-bearing result: it is shown to be implied by the plain $q$-discrete-logarithm assumption in the algebraic group model [@tcrstw22]. All three are discrete-log-family assumptions believed hard in the groups actually used, so the deployed OPRF is anchored to the same discrete-log problem Part 18 argued has stood unbroken for half a century. That is about as good a foundation as applied cryptography offers.

What it costs, forever. Now the third question comes due. Every BlindEvaluate is, by construction, a static-Diffie-Hellman oracle sample: the server computes $x \mapsto x^k$ for a fixed secret $k$, and Brown-Gallant and Cheon showed that collecting such samples recovers $k$ at a cost that drops with the count [@rfc9497], [@browngallant04], [@cheon06].

A service that, for a fixed secret key $k$, answers queries of the form $x \mapsto x^k$. Each answer is a sample that erodes the key: after $Q$ queries, the best-known attacks recover $k$ at a cost reduced by $\log_2(Q)/2$ bits (the Brown-Gallant and Cheon bound). Offering an OPRF *is* offering this oracle [@browngallant04], [@cheon06].

The specification does not bury this. It states it:

"Best-known attacks reduce the security of the prime-order group instantiation by log_2(Q) / 2 bits, where Q is the number of BlindEvaluate calls." -- RFC 9497, section 7.2.3

Work the arithmetic and the design pressure becomes concrete. On a 128-bit group, effective key security is $128 - \log_2(Q)/2$ bits. Answer $2^{40}$ queries and you have shed 20 bits, down to 108. Answer $2^{60}$ and you are at 98. This is not an implementation flaw you can patch; it is a lower bound on the attacker's advantage that no construction in this family removes, because it follows from the primitive doing the one thing it exists to do.

BlindEvaluate calls $Q$	Bits lost, $\log_2(Q)/2$	Effective security on a 128-bit group
$2^{20}$	10	118 bits
$2^{40}$	20	108 bits
$2^{60}$	30	98 bits
$2^{80}$	40	88 bits

Note: "Keep this key secret forever" and "answer unlimited public blind queries on a 128-bit group" cannot both hold. The static-Diffie-Hellman erosion is inherent to exposing the oracle, so an unmetered public OPRF on a 128-bit group provably cannot retain 128-bit key security [@rfc9497], [@browngallant04]. The only resolutions are operational: rate-limit BlindEvaluate, rotate the key on a schedule, or move to a larger group (decaf448, P-384, P-521). You are choosing a budget, not eliminating a cost.

Look at what just happened across five sections. The strong-Diffie-Hellman decay that doomed the 2009 Jarecki-Liu construction was treated then as a fatal flaw -- the reason to abandon a whole generation [@jareckiliu09], [@cheon06]. The *same physics* reappears in RFC 9497, but now it is a written-down budget with a formula and a mitigation list [@rfc9497]. Nothing about the mathematics changed. What changed is the engineering posture: a phenomenon that was a silent, disqualifying weakness became an explicit, managed policy knob. This is often how a field matures -- not by defeating a limit, but by naming it precisely enough to live with it.

Two more facts bound the theory. Round-optimality: two messages is provably the minimum, and 2HashDH achieves it, so there is no efficiency left to win there.The two-message floor is structural: the client must send at least one blinded value and receive at least one response to recover an output, so no protocol of this shape can use fewer round-trips. And two honest open edges. RFC 9497 admits that the exact multi-key, batched protocol the industry actually deploys has no security proof:

"There is currently no security analysis available for the VOPRF protocol described in this document in a setting with multiple server keys or batching." -- RFC 9497, section 7.2.1

The deployed reality -- many issuer keys, batched proofs -- outruns the single-key proof it is built on [@rfc9497]. And the whole family rests on discrete log, so a scalable quantum computer running Shor's algorithm breaks obliviousness and unforgeability alike. Two of these limits are not settled facts but open frontiers -- and the sharpest one opened with a construction that was broken within a year of being proposed.

9. Where It Is Still Moving

For the single-key, non-batched, classical setting, the theory is essentially closed: round-optimal, constant communication, unconditional input secrecy, a clean assumption reducing to discrete log. The live research is almost entirely about the edges the deployments already stand on.

The post-quantum frontier, which opened with a cautionary tale. Because every deployed OPRF falls to Shor, the sharpest open problem is a post-quantum replacement -- and the first serious attempt is exactly why the field is cautious.

In 2020, Dan Boneh, Dmitry Kogan, and Katharine Woo built the first substantial post-quantum OPRF from supersingular isogenies [@bkw20]. Within a year, Andrea Basso, Peter Kutas, Simon-Philipp Merz, Christophe Petit, and Antonio Sanso cryptanalyzed it, giving a polynomial-time attack and a subexponential one that survived the obvious countermeasures -- together breaking the parameters the authors had proposed [@basso21].

A construction proven secure under a named assumption was undone because the assumption itself did not hold. It is the cleanest possible reminder that a proof is only as good as what it reduces to.The isogeny paper actually offered two constructions. Its second one revived the 1997 Naor-Reingold PRF, reinterpreted over commutative group actions (the CSIDH setting) [@bkw20], [@naorreingold97]. The oldest algebraic PRF in this story turned out to be the one with a plausible post-quantum future -- a twenty-three-year-old idea made newly relevant by a change of group.

The other post-quantum directions are advancing but heavy. Martin Albrecht, Alex Davidson, Amit Deo, and Nigel Smart gave a round-optimal verifiable OPRF from ideal lattices in 2021 [@adds21]; Albrecht and Kamil Doruk Gur made it "practical-ish and thresholdisable" in 2024, cutting its lattice bandwidth roughly fourfold and adding a threshold variant [@albrechtgur24]. And a general framework arrived in 2025: Ward Beullens, Lucas Dodgson, Sebastian Faller, and Julia Hesse compiled quantum-safe OPRFs from secure multi-party evaluation of simple functions, reporting a working construction in about 0.57 seconds with under a megabyte of communication [@beullens25].

That is a genuine milestone -- and it is still orders of magnitude heavier than the sub-millisecond classical evaluation it aims to replace. An efficient, verifiable, partially oblivious post-quantum OPRF, matching the classical profile on all three counts at once, does not yet exist.

The proof the industry is running ahead of. Section 8's second pull quote is also an open problem: prove the multi-key, batched VOPRF that Privacy Pass actually deploys [@rfc9497]. Batching is a well-used heuristic, but the formal guarantee still covers only the single-key case.

Removing the single point of failure. A threshold or distributed OPRF splits the key across several servers so no one of them holds it, which is the structural answer to server compromise. RFC 9807 names exactly this as the defense that forces an attacker online before any dictionary attack can start [@rfc9807]. Round-optimal, misuse-resistant threshold OPRF specifications are still maturing.

The limits that are policy, not cryptanalysis. Two open problems are engineering rather than mathematics. The static-Diffie-Hellman budget is a permanent tax on any public oracle; the open question is operational -- what query cap and rotation cadence keep a given group above 128 bits -- not whether the bound can be improved, since Brown-Gallant and Cheon are believed essentially tight for generic groups [@browngallant04], [@cheon06].

And verifiability quietly collapses if a malicious issuer hands each user a different public key, so making "one key for everyone" auditable needs key-transparency infrastructure that does not yet ship as a turnkey component [@rfc9497]. Formally verified constant-time implementations sit in the same category: RFC 9497 mandates constant-time secret operations, but the guarantee is only as strong as the machine-checked code behind it [@rfc9497].

None of these block you from shipping today. They tell you what to watch. What you ship today has a small, boring set of rules that separate a correct deployment from a broken one -- and unlike the open problems, those rules are entirely settled.

10. The Rules, Made Operational

Here is the whole deployment discipline in a form a reviewer can apply without rereading the RFC. It has three parts: pick two things, always do a handful, and never do another handful.

Pick the mode by the questions. Trust the key by construction: base OPRF. Must verify which key: VOPRF. Must bind a public label: POPRF, and never smuggle a secret into info, which is public by definition [@rfc9497], [@tcrstw22]. Pick the ciphersuite by the query budget. Default to ristretto255-SHA512; for an internet-facing unmetered oracle move to decaf448, P-384, or P-521; for FIPS or interop use P-256-SHA256. Never roll your own group or hash-to-curve -- use RFC 9380 and RFC 9496 [@rfc9497], [@rfc9380], [@rfc9496].

flowchart TD Start["Choose an OPRF deployment"] --> A{"Could the server be adversarial about which key it uses?"} A -->|no, key trusted| B["Base OPRF"] A -->|yes| C{"Must you bind a public label such as tenant or epoch?"} C -->|no| D["VOPRF"] C -->|yes| E["POPRF"] B --> F{"Unmetered public oracle?"} D --> F E --> F F -->|no| G["ristretto255-SHA512 default"] F -->|yes| H["decaf448 or P-384 or P-521"]

Note: Every call: draw a fresh, uniform blind, and a fresh nonce for the DLEQ proof. Every received element: validate it -- on the curve, in range, and not the identity -- before you touch it [@rfc9497]. Every secret operation, including BlindEvaluate and GenerateProof: run it in constant time [@rfc9497]. Every output: domain-separate with the "Finalize" tag and mode context, and keep inputs within the length limit [@rfc9497]. For a VOPRF or POPRF: publish and pin one public key, transparently, for everyone. For a POPRF: treat an InverseError as a signal that the tweak collided and it is time to rotate [@tcrstw22]. And bound $Q$: rate-limit BlindEvaluate or rotate keys so the static-Diffie-Hellman loss stays inside budget [@rfc9497], [@browngallant04].

The "never" list is just the failure catalog from the last nine sections, re-read as review rules. Each entry maps to one of the three questions answered wrong.

Anti-pattern	Why it breaks	Question violated	The fix
Sending the un-blinded input	The server sees the input; obliviousness is gone	Q1	Always apply the fresh blind before sending [@rfc9497]
A MAC or HKDF "used as an OPRF"	If the server computes it, it saw the input	Q1	Use a real two-party OPRF; the blind is the point [@rfc9497]
Reusing the blind across calls	Two evaluations become linkable, and the input can leak	Q1	Fresh, uniform blind every call [@rfc9497]
Skipping element validation	Malformed points enable key-probing attacks	Q1/Q3	Reject off-curve, out-of-range, and identity elements [@rfc9497]
Dropping or unsoundly batching the DLEQ	A malicious server can use a per-client key and tag you	Q2	Verify the proof; combine batches soundly [@rfc9497], [@dgstv18]
A per-user public key	The unique key is itself a deanonymizing tag	Q2	Pin one transparently published key for all [@rfc9497]
Non-constant-time secret ops	The key leaks through timing	Q3	Constant-time group ops, proof, and evaluate [@rfc9497]
Treating POPRF `info` as secret	`info` is public by definition; obliviousness assumed of it fails	Q1	Put only public labels in `info` [@tcrstw22]
An unmetered oracle on a 128-bit group	The key erodes by $\log_2(Q)/2$ bits, unbounded	Q3	Rate-limit, rotate, or use a bigger group [@rfc9497], [@browngallant04]
Using an OPRF where you need public verifiability	A third party cannot verify a VOPRF token	(wrong primitive)	Use a blind signature (Blind RSA), not an OPRF [@rfc9474]

{` // STRUCTURAL MODEL. Shows why reusing the blind across calls destroys unlinkability. // Same additive group model: a "point" is an integer mod q, scalar*point is mod q. const q = 2147483647; const mod = (a) => ((a % q) + q) % q; const scalarMul = (s, point) => mod(s * point);

const Hx = 991; // H(x): the SAME secret input, evaluated twice const blindedWith = (r) => scalarMul(r, Hx); // what goes on the wire: r*H(x)

// BAD: reuse the same blind r for both calls const rFixed = 55555; const wire1_bad = blindedWith(rFixed); const wire2_bad = blindedWith(rFixed); console.log('reused blind -> wire equal? ', wire1_bad === wire2_bad, '(server links the two as one input)');

// GOOD: a fresh uniform blind each call const wire1_ok = blindedWith(37337); const wire2_ok = blindedWith(88123); console.log('fresh blind -> wire equal? ', wire1_ok === wire2_ok, '(server cannot link them)'); `}

Most of those "nevers" are the same handful of mistakes phrased for different audiences: do not let the server see the input, do not let it pick a per-client key, and do not let it answer unlimited queries. It is worth answering the questions engineers actually ask about them head on.

The `voprf` Rust crate and Cloudflare's CIRCL both implement all three modes against the RFC 9497 test vectors [@voprf-crate], [@circl]. A good first exercise: run the base OPRF, print the blinded element on two evaluations of the same input, and confirm the two wire values differ because the blind is fresh. Then switch to the verifiable mode, tamper with one byte of the server's response, and watch `VerifyProof` reject it. That single afternoon teaches more about Q1 and Q2 than any diagram.

11. Questions Engineers Actually Ask

No, and the difference is the whole primitive. If the server computes `HMAC(k, x)` for you, it *saw* `x`. That is exactly the compromise the OPRF removes: it applies the key to a blinded value it cannot read, so it never sees the input or the output [@rfc9497]. An HMAC that the server evaluates on your plaintext input is not oblivious by any definition. In any setting with an untrusted server, yes, decisively. The base OPRF answers only "is it blind?" Without the DLEQ proof, a malicious server can hand each client a *different* key and later use the distinct responses to tell clients apart -- deanonymizing the very users an anonymity system is meant to protect [@rfc9497], [@dgstv18]. The VOPRF's proof, plus one transparently pinned public key, is what closes that gap. Because every `BlindEvaluate` is a static-Diffie-Hellman oracle sample, and collecting samples recovers the key at a cost that drops by $\log_2(Q)/2$ bits after $Q$ calls [@rfc9497], [@browngallant04], [@cheon06]. It is inherent to offering a Diffie-Hellman oracle, not an implementation defect. Rate-limit, rotate keys, or use a larger group to keep the loss inside budget. Half of it. Privacy Pass token type `0x0001` is a VOPRF, which is an OPRF. Token type `0x0002` is Blind RSA -- a blind *signature*, not an OPRF -- and it is the type behind Apple's Private Access Tokens, the most visible deployment [@rfc9578], [@rfc9474]. So "Privacy Pass equals OPRF" is wrong; one of its two token types is a different primitive entirely. No. Checkup performs a blinded, OPRF-based lookup, so Google does not learn the credential being checked [@thomas19]. Have I Been Pwned's range API uses k-anonymity: you send the first five hex digits of a SHA-1 hash and receive the whole bucket to search locally [@cloudflare-kanon]. That leaks a hash prefix by design. It is simpler and cache-friendly, but it is not oblivious. No. Reusing the blind across calls makes two evaluations of the same input produce the same on-wire value, so the server can link them, and in some settings it can leak the input [@rfc9497]. The blind and the proof nonce must be freshly and uniformly drawn every single call. This is non-negotiable. No. Every RFC 9497 ciphersuite rests on discrete-log hardness, which Shor's algorithm breaks, so a scalable quantum computer defeats both obliviousness and unforgeability [@rfc9497]. Post-quantum OPRFs are an open frontier: the first serious isogeny construction was broken within a year [@bkw20], [@basso21], and lattice and MPC candidates remain far heavier than classical evaluation [@albrechtgur24], [@beullens25]. And to preempt the related question: no, a POPRF's `info` is not secret -- it is public by definition [@tcrstw22].

Read back across the whole story and the shape is unmistakable. Every named failure was one of three questions answered wrong. FIPR's early constructions answered "is it blind?" but too expensively to ship. Jarecki and Liu answered "which key?" too, and then the ground under "how many asks?" gave way beneath them. A base OPRF deployed where a VOPRF was needed answers "which key?" wrong and tags its users. A reused blind answers "is it blind?" wrong. An unmetered public oracle answers "how many asks?" wrong. The broken isogeny construction answered all three -- until its assumption did not hold. The diagnostic is not a teaching device bolted onto the history; it is the history.

And the resolution is a single move, assembled from primitives this series has already built. Take Chaum's 1982 blinding from the RSA world of Part 14, run it in the Diffie-Hellman group of Part 18 -- the discrete log that Part 18 argued nobody has broken, here run one-sided so a server applies a secret rather than agreeing on one -- hash the input into the group with the hash-to-curve of Part 16, and pin the key with the equal-discrete-log proof descended from the sigma protocols of Part 17.

Four earlier building blocks, one new primitive. Later parts of this guide return to its consumers as topics in their own right -- the augmented PAKE, anonymous credentials, private set intersection -- because each is, underneath, this same engine.

The server helped, and it still never saw.

That is the sentence to keep. A pseudorandom function is a lock only the keyholder can open, which is exactly what made it dangerous to offer as a service. The oblivious PRF removed the danger without removing the help: it lets a server apply its secret key to your secret input, prove it used the right key, and do so within a query budget it is honest about -- all without ever seeing what it operated on. Blind, verifiable, rate-limited. The server helped, and it still never saw.

The Standard Was Boring; That Was the Point: A Field Guide to HPKE (RFC 9180)

noreply@paragmali.com (Parag Mali) — Mon, 13 Jul 2026 08:42:43 GMT

**"Encrypt to a public key" sounds like the most basic promise in cryptography, yet the industry standardized it four incompatible ways.** ANSI X9.63, IEEE 1363a, ISO/IEC 18033-2, and SEC 1 all shipped a version of ECIES; two conforming implementations could produce ciphertexts that would not decrypt for each other, none came with shared test vectors, and one standard openly documented that an attacker could forge a second valid ciphertext for the same plaintext and named it "benign malleability" [@sec1], [@maea10]. HPKE (RFC 9180, 2022) is the boring, correct fix: one KEM-DEM construction where a Key Encapsulation Mechanism *generates* a shared secret and its encapsulation, an HKDF key schedule with every public key bound in derives the working keys, and an AEAD seals the data [@rfc9180]. It is IND-CCA2-proven, machine-checked across all four modes (`base`, `psk`, `auth`, `auth_psk`), and shipped with per-ciphersuite test vectors [@lipp20], [@abhklr20]. Because the primitive is correct by construction, HPKE has no known catastrophic real-world break; the entire discipline is knowing its four modes and its non-goals. It does *not* provide forward secrecy against recipient compromise in any mode, `auth` mode is *not* a signature, and its AEADs are *not* key-committing -- and every real HPKE incident to date is one of those non-goals assumed away or a check an implementation dropped [@rfc9180], [@ghsa]. It is the sealing primitive behind Encrypted Client Hello (RFC 9849), MLS, Oblivious HTTP, and Oblivious DNS; the post-quantum path is to swap the KEM (X-Wing / ML-KEM), not to redesign the scheme [@rfc9849], [@xwing]. The math held; the risk moved to the interface.

1. An Elementary Task the Industry Got Wrong Four Times

"Encrypt this to their public key" is the first promise public-key cryptography ever made. It is the one-line pitch of RSA, the thing every engineer assumes was solved and made boring decades ago, a settled corner of the field you reach for without thinking. Yet for roughly twenty-five years there was no single correct way to do it.

At least four "standard" versions of the Elliptic Curve Integrated Encryption Scheme (ECIES) shipped: ANSI X9.63 in 2001, IEEE 1363a in 2004, ISO/IEC 18033-2 in 2006, and SECG's SEC 1 version 2 in 2009 [@maea10], [@rfc9180]. They disagreed on the key-derivation function, the message authentication code, whether elliptic-curve points were compressed or uncompressed, and which optional fields went into the hash. Two conforming implementations, handed the same curve and message, could produce ciphertexts that would not decrypt for each other.

None of the four came with shared test vectors, so two libraries had no ground truth against which to check that they agreed. And one standard openly documented that an attacker could take a valid ciphertext and manufacture a second, different valid ciphertext for the same plaintext. It named the phenomenon "benign malleability" and deemed it harmless [@sec1].

The hard part of "encrypt to a public key," it turned out, was never the cryptography. It was getting everyone to do the same correct thing, with a proof that it was correct and a way to check that two implementations interoperated. That failure was tolerable when public-key encryption was a batch operation between parties who could negotiate a format. It stopped being tolerable the moment the modern web needed to encrypt to a server it had never spoken to.

Consider what TLS Encrypted Client Hello and Oblivious HTTP actually require. A client must encrypt the sensitive part of its connection -- in Encrypted Client Hello, the server name it is trying to reach -- to a public key it fetched moments earlier, for a server it has exchanged no messages with, with no round trip in which to negotiate anything [@rfc9849], [@rfc9458]. There is no handshake to agree on a KDF or a point encoding. The format has to be right the first time, on the first byte, or the connection reveals exactly what it was trying to hide.

That is the setting Hybrid Public Key Encryption was built for. HPKE, specified in RFC 9180 by the IRTF's Crypto Forum Research Group in February 2022, is the boring, correct answer [@rfc9180]. Its headline achievements sound almost too unglamorous to matter: it publishes test vectors, and it comes with a proof that it is secure against chosen-ciphertext attack.

Those two boring facts are exactly why HPKE matters. Once you accept them, the interesting question is not whether HPKE is secure but a sharper one, worth carrying through every section that follows: you have a stranger's public key and one shot to send them a secret, with no handshake and no reply -- what does HPKE buy you, and what does it silently refuse to give?

This is Part 19 of a field guide for people who design protocols, and it is the part where the primitives from earlier parts -- authenticated encryption, key derivation, elliptic curves, the KEM-DEM idea, digital signatures -- stop being separate ingredients and compose into a single shipped building block. By the end you will be able to look at any "encrypt to a recipient's public key" design and name, on sight, which of HPKE's modes it is using and which of HPKE's non-goals it is quietly assuming away.

The fix is so unglamorous it is tempting to skip straight to the API. Do not: the shape of HPKE is written in the failures of the four standards it replaced, and you cannot use it safely without seeing what "done right" was correcting. Start with the one idea all four were circling and none quite pinned down: the difference between encrypting a key and encapsulating one.

2. Encapsulate the Key, Do Not Encrypt It

RFC 9180 describes what it does differently in a single line, and that line is the whole conceptual pivot of this article. The traditional combination, it says, has been to "encrypt the symmetric key with the public key." HPKE instead will "generate the symmetric key and its encapsulation with the public key" [@rfc9180]. That one-word swap -- from encrypt to encapsulate -- is the difference between the padding-oracle-prone lineage that broke RSA key transport and a clean, composable, provable construction.

The traditional combination has been "encrypt the symmetric key with the public key." HPKE instead will "generate the symmetric key and its encapsulation with the public key." -- RFC 9180

Build the mental model in three moves. Derive only what the rest of the article needs.

Move one: KEM-DEM in one picture

The scheme standardized in RFC 9180 for encrypting arbitrary-length messages to a recipient's public key. It is built from three interchangeable ingredients -- a Key Encapsulation Mechanism (KEM), a Key Derivation Function (KDF), and an Authenticated Encryption with Associated Data (AEAD) scheme -- named together as a ciphersuite.

The word "hybrid" means it splits the work between public-key and symmetric cryptography, and the split follows the KEM-DEM pattern. The KEM does the public-key half; the DEM does the symmetric half.

A public-key primitive that does not transport a key you chose. Called with the recipient's public key, `Encap(pkR)` *generates* a fresh random shared secret together with an encapsulation `enc` of it, returning the pair `(shared_secret, enc)`. The recipient runs `Decap(skR, enc)` to recover the same shared secret. You never pick the key; the public-key operation produces it. The symmetric half of a hybrid scheme: an AEAD that bulk-encrypts the actual payload under a key derived from the KEM's shared secret. In HPKE the DEM is a single AEAD scheme, and its authenticity tag is intrinsic rather than bolted on.

If you have read Part 14 on getting RSA right, this is the same KEM-DEM skeleton, now instantiated on elliptic curves rather than RSA. The reason to generate the key rather than encrypt one is the entire lesson of that part: the moment you encrypt an attacker-influenced key under a public-key operation, you invite a decryption oracle to leak information one query at a time. A KEM removes the attacker's chosen input from the public-key step. There is nothing to pad, and nothing to probe.

flowchart TD subgraph LEFT["Encrypt the key (RSA and ElGamal key transport)"] A1["Choose a random symmetric key k"] --> A2["Encrypt k under the recipient public key"] A2 --> A3["Ship wrapped k plus the symmetric ciphertext"] end subgraph RIGHT["Encapsulate the key (KEM-DEM, HPKE)"] B1["Encap(pkR) generates a fresh shared secret"] --> B2["Returns shared_secret and enc together"] B2 --> B3["Derive keys, then AEAD-seal the payload"] end

Move two: DHKEM, concretely

HPKE's deployed KEM is Diffie-Hellman-based, and it is worth seeing the mechanism once rather than treating it as a black box.

HPKE's Diffie-Hellman-based KEM. `Encap(pkR)` generates an ephemeral key pair, computes a Diffie-Hellman value against the recipient's public key, and derives the shared secret from it; the encapsulation `enc` is the serialized ephemeral public key. The authenticated variants `AuthEncap`/`AuthDecap` additionally fold in a Diffie-Hellman value computed with the sender's *static* key.

Concretely, Encap(pkR) generates an ephemeral pair $(sk_E, pk_E)$, computes $\mathrm{DH}(sk_E, pk_R)$, sets enc to the serialization of $pk_E$, and returns shared_secret = ExtractAndExpand(dh, kem_context), where kem_context already contains both enc and the recipient key $pk_R$ [@rfc9180]. The recipient recomputes the identical Diffie-Hellman value as $\mathrm{DH}(sk_R, pk_E)$ and derives the same secret. This is exactly the ephemeral-static shape that DHIES introduced in the 1990s, now wearing a modern KDF -- and the decision to fold $pk_R$ into kem_context is the first appearance of the idea that later kills benign malleability.

sequenceDiagram participant S as Sender (holds pkR) participant R as Recipient (holds skR) Note over S: generate ephemeral pair skE, pkE Note over S: dh holds DH(skE, pkR) Note over S: enc holds Serialize(pkE) Note over S: shared_secret from ExtractAndExpand(dh, kem_context with enc and pkR) S->>R: send enc (the serialized ephemeral key) Note over R: dh holds DH(skR, pkE), the same value Note over R: recompute the identical shared_secret

Move three: the key schedule and the Context

The shared secret is not used directly. Both sides run it through a key schedule.

The HKDF Extract-then-Expand process that turns the KEM's `shared_secret` into working keys. It *extracts* a pseudorandom key from the shared secret (and, in PSK modes, the PSK), then *expands* that key over a context of the mode identifier, a hash of the pre-shared-key identifier (`psk_id_hash`), and a hash of the application `info` string (`info_hash`) to output the AEAD `key`, a `base_nonce`, and an `exporter_secret`. A symmetric primitive that provides confidentiality and integrity at once: it encrypts a plaintext and authenticates both the ciphertext and some associated data, so any tampering is detected on decryption. In HPKE it is the DEM that seals the payload; AES-128-GCM, AES-256-GCM, and ChaCha20-Poly1305 are the standard choices.

The HPKE key schedule is, quite literally, a worked example of the HKDF construction from Part 13: extract a pseudorandom key from the shared secret, then expand it into the specific bytes each downstream primitive needs. Its outputs assemble into a Context. The Context exposes Seal/Open -- a stateful pair in which an incrementing sequence number is combined with base_nonce (by exclusive-or) to give every message a distinct nonce -- and Export.

An HPKE interface that derives fresh, independent secrets of any length from the established context, for keying *other* protocols. It behaves as a variable-length pseudorandom function, so exported secrets are independent of the AEAD key and of each other. Oblivious HTTP and Oblivious DNS use it to key their own message framing instead of calling `Seal` directly.

For a single message you do not need to hold a Context open at all: the single-shot Seal/Open API wraps setup, one encryption, and teardown into one call. Everything above -- encapsulate, derive, seal -- is what those single-shot calls do underneath.

flowchart TD PKR["Recipient public key pkR"] --> ENCAP["Encap"] ENCAP --> SS["shared_secret"] ENCAP --> ENCV["enc (sent to recipient)"] SS --> KS["key_schedule (HKDF Extract-then-Expand)"] MODE["mode identifier"] --> KS PSK["psk_id_hash"] --> KS INFO["info_hash"] --> KS KS --> KEY["AEAD key"] KS --> NON["base_nonce"] KS --> EXP["exporter_secret"] KEY --> CTX["Context"] NON --> CTX EXP --> CTX CTX --> SEAL["Seal / Open (per-message nonce)"] CTX --> EXPORT["Export (key other protocols)"]

The structural model below is a toy, not real cryptography, but it lets you feel the difference from wrapping a chosen key: encap invents a random secret and its encapsulation, folds in the recipient's public key, and shows that a different recipient key yields a different secret from the same encapsulation.

{` // STRUCTURAL MODEL, NOT REAL CRYPTO. It only shows the SHAPE of "generate the key". function toyMix(str) { // a tiny non-cryptographic mixer let h = 2166136261 >>> 0; for (const ch of str) { h ^= ch.charCodeAt(0); h = Math.imul(h, 16777619) >>> 0; } return (h >>> 0).toString(16); } function randInt() { return Math.floor(Math.random() * 1e9); }

// Encap: the PUBLIC-KEY op GENERATES the shared secret. The caller never picks it. function encap(pkR) { const skE = randInt(); // fresh ephemeral secret const enc = 'pkE(' + skE + ')'; // its public value IS the encapsulation const dh = toyMix(enc + '|' + pkR); // stands in for DH(skE, pkR) // kem_context binds BOTH enc and pkR into the derivation -- the anti-malleability trick const shared_secret = toyMix('ss|' + dh + '|' + enc + '|' + pkR); return { shared_secret, enc }; }

const pkR = 'recipient-key-A'; const { shared_secret, enc } = encap(pkR); console.log('enc on the wire = ' + enc); console.log('shared_secret = ' + shared_secret);

// Same enc, a DIFFERENT recipient key -> a DIFFERENT secret. A swapped or mangled // public key simply cannot derive the key that decrypts, because pkR is bound in. const swapped = toyMix('ss|' + toyMix(enc + '|recipient-key-B') + '|' + enc + '|recipient-key-B'); console.log('same enc, key B = ' + swapped); console.log('secrets differ = ' + (swapped !== shared_secret)); `}

Now the payoff, and the first place your understanding should shift.

Key idea: Encapsulate, do not encrypt. Let the public-key operation generate the symmetric key and its encapsulation together (a KEM), then AEAD-encrypt under a derived key (a DEM). Because the composite is secure against chosen-ciphertext attack whenever both halves are, "encrypt to a public key" stops being an ad-hoc scheme you must re-prove each time and becomes a composition of two well-understood parts.

The modularity is the whole point. Cramer and Shoup proved in 2001 that a KEM and a DEM, each secure against chosen-ciphertext attack, compose into a hybrid scheme that is too [@cs01]; a few years later the converse was pinned down, showing both halves essentially have to be that strong for the composite to be [@hhk06]. Those results turn "is this encrypt-to-a-public-key scheme secure?" into two smaller, reusable questions.RFC 9180 notes in passing that its Base mode and the original Cramer-Shoup scheme are both "named HPKE" -- the name has KEM-DEM in its bones [@rfc9180], [@cs01].

Note: RFC 9180 is an Informational IRTF/CFRG document, not a Standards-Track one. Yet it is normatively required by Standards-Track protocols -- MLS (RFC 9420), Oblivious HTTP (RFC 9458), and Encrypted Client Hello (RFC 9849) all cite it as a mandatory dependency. Its category label undersells how much of the modern stack rests on it [@rfc9180].

This clean picture -- encapsulate, derive, seal -- looks obvious in hindsight. It was not. The KEM-DEM idea took two decades to take its final shape, and the Diffie-Hellman instantiation it rests on fragmented into the four broken standards from the last section before anyone consolidated it. To understand why HPKE makes the exact choices it does, walk the twenty-five years it is the endpoint of.

3. From "Encrypt the Key" to "Encapsulate the Key"

The story is a twenty-five-year march from encrypt the key to encapsulate the key, starting, as RFC 9180 notes, in the early days of public-key cryptography [@rfc9180]. Diffie and Hellman defined the idea of a public key in 1976 but did not say how to encrypt a long message to one [@dh76]. The answers accumulated in layers, each fixing the previous one's sharpest failure.

timeline title From encrypt-the-key to encapsulate-the-key, 1976 to 2026 1976 : Diffie and Hellman define public-key cryptography 1993 : PEM (RFC 1421) encrypts the content key under the recipient public key 1999 : DHIES derives the key from Diffie-Hellman instead of encrypting it 2001 : Cramer and Shoup formalize the KEM-DEM pattern 2009 : SEC 1 v2 completes the four-way ECIES split and names benign malleability 2022 : RFC 9180 consolidates KEM-DEM with proofs and test vectors 2026 : Post-quantum hybrid KEMs such as X-Wing drop into the KEM slot

PEM (RFC 1421, 1993): the traditional combination. Privacy Enhanced Mail defined the canonical early hybrid. Pick a content-encryption key, encrypt that key under the recipient's public key, and bulk-encrypt the message symmetrically. RFC 9180 opens by naming this exact tradition and positioning HPKE as the departure from it [@rfc1421], [@rfc9180]. It is the naive model most engineers still carry, and the one the previous section asked you to give up.

ElGamal (1985) and why Diffie-Hellman resists "encrypt a chosen key." ElGamal's public-key encryption predates the hybrid schemes and shows the friction directly [@elgamal85]. To "encrypt" a chosen symmetric key under a Diffie-Hellman public key, you must first encode that key as a group element, then pay a roughly twofold ciphertext expansion -- and the result is malleable rather than secure against chosen-ciphertext attack. That malleability is a reasoned inference here, anchored to DHIES's own claim to offer "stronger security properties than ElGamal," not a verbatim quote [@dhies_ms].

The RSA branch of the same idea failed even more loudly. Encrypting a chosen key under RSA with PKCS#1 v1.5 padding is precisely the construction that Bleichenbacher's attack turned into a decryption oracle, revived two decades later as ROBOT against nearly a third of the top 100 web domains [@rfc8017], [@robot_site], [@robot_eprint]. The part of this series on getting RSA right tells that story in full; both branches teach the same lesson: stop choosing a key and wrapping it.

DHIES and DHAES (Abdalla, Bellare, Rogaway, 1999): derive, do not encrypt. The academic root of ECIES made the decisive move. Instead of encrypting a key, run an ephemeral-static Diffie-Hellman exchange, push the result through a KDF to get an encryption key and a MAC key, then Encrypt-then-MAC the payload. No key is ever encrypted; it is agreed.

The authors proved the scheme secure against chosen-ciphertext attack under the Oracle Diffie-Hellman assumption, with "no random oracles involved," and called it "an attractive starting point for developing public-key encryption standards" [@dhaes], [@dhies_ms].DHIES has no single crisp date: the DHAES manuscript appeared as ePrint 1999/007 in 1999, was later renamed DHIES, and its formal analysis was published at CT-RSA 2001. Cite the range, not one year [@dhaes], [@dhies_ms]. That parenthetical -- "an attractive starting point for developing standards" -- turned out to be prophetic in the worst way.

Cramer and Shoup (2001): the modular abstraction. DHIES hinted at a structure; Cramer and Shoup formalized it. A KEM transports a random key, a DEM encrypts the payload, and the composite is secure against chosen-ciphertext attack if both halves are [@cs01]. A few years later the necessity direction was worked out: for the standard hybrid, both halves essentially have to reach that bar [@hhk06]. This is the theory HPKE would eventually standardize. It existed, complete, in 2001.

Note: By 2001 the field had both a clean, proof-backed academic scheme (DHIES) and the modular abstraction that made its security compositional (KEM-DEM). The striking fact about HPKE is not that someone invented a new idea in 2022. It is that the correct idea sat published and unadopted as a standard for two decades [@cs01], [@rfc9180].

So by 2001 everything HPKE needed already existed: a concrete, provably secure ephemeral-static scheme, and an abstraction that made "encrypt to a public key" a matter of plugging two proven pieces together. What happened over the following eight years was not progress. It was fragmentation. The same good idea was handed to four different standards bodies, and it came out four incompatible ways.

4. The ECIES Zoo

Between 2001 and 2009 the DHIES construction was standardized four times -- ANSI X9.63, IEEE 1363a, ISO/IEC 18033-2, and SEC 1 v2 -- collectively called ECIES, and collectively a mess that RFC 9180 politely calls "numerous competing and non-interoperable standards" [@rfc9180], [@maea10]. This is the hook's promise, paid off in detail. Three defects run through the era, and HPKE was built to close all three.

flowchart TD D["DHIES (Abdalla, Bellare, Rogaway)"] --> A["ANSI X9.63 (2001)"] D --> I["IEEE 1363a (2004)"] D --> S["ISO/IEC 18033-2 (2006)"] D --> C["SEC 1 v2 (2009)"] A --> Z["The ECIES zoo: incompatible KDFs, MACs, point encodings, no shared test vectors"] I --> Z S --> Z C --> Z Z --> H["RFC 9180 (2022): one construction, proven, test-vectored"]

Fragmentation: the "which ECIES?" problem. The four standards disagree on the key-derivation function, the MAC, whether elliptic-curve points are compressed or uncompressed, and whether the optional SharedInfo fields are present. So two conforming implementations often cannot interoperate: an ECIES ciphertext built with the ANSI X9.63 KDF and one built to ISO/IEC 18033-2, for the same curve and message, are byte-incompatible [@rfc9180], [@maea10].The concrete interoperability failure is not hypothetical: swap the KDF or the point-encoding convention between two of the four standards and a ciphertext valid under one is undecryptable under another, for identical inputs [@maea10].

The four standards -- ANSI X9.63 (2001), IEEE 1363a (2004), ISO/IEC 18033-2 (2006), and SEC 1 version 2 (2009) -- differ across enough independent knobs (KDF, MAC, point-encoding, `SharedInfo`) that "ECIES" names a family of mutually unintelligible dialects rather than one scheme. RFC 9180 points readers to the Gayoso Martinez et al. survey "for a thorough comparison." That survey and three of the four standards are paywalled, so this article cites them through RFC 9180's reference section rather than by live link [@rfc9180], [@maea10], [@sec1].

Benign malleability. SEC 1 version 2 documents a specific defect and then waves it off.

A property of the ECIES standards in which an attacker can transform a valid ciphertext into a different valid ciphertext for the *same* plaintext -- concretely, by replacing the ephemeral point $R$ with $-R$, which shares the same x-coordinate. SEC 1 called it "benign malleability" and "generally deemed harmless," and offered binding $R$ into `SharedInfo` only as an *optional* countermeasure. Because a ciphertext can be mauled into a second valid ciphertext, the scheme as written is not strictly secure against chosen-ciphertext attack [@sec1].

The word "benign" is doing a great deal of work. A scheme that lets anyone produce a second valid ciphertext for a plaintext they cannot read is, by definition, malleable, and malleability is exactly what security against chosen-ciphertext attack forbids. SEC 1 made the fix -- include $R$ in the hashed context -- optional. Hold that thought: it is the single design decision the next section shows HPKE making mandatory.

No proofs, and no test vectors. RFC 9180 is blunt about the state it inherited: prior schemes "rely on outdated primitives, lack proofs of IND-CCA2 security, or fail to provide test vectors" [@rfc9180]. The missing test vectors are not a documentation nicety. Without a published set of known-good inputs and outputs, two implementers have no way to discover that their libraries disagree until a ciphertext fails to decrypt in production -- which is precisely how "conformant" implementations drifted apart.

Note: The ECIES era is the standing proof that a primitive can be widely deployed and still be under-specified, unprovable, and mutually incompatible. "We implemented the standard" told you nothing about whether your ciphertext would decrypt on the other side, or whether the scheme met a modern security definition. Both had to be established separately, and neither was [@rfc9180], [@maea10].

The comparison below is the historical spine as a grid. Read the last two columns -- proof status and test vectors -- and the shape of the fix becomes obvious.

Approach	Core idea	Main weakness	Proof and test vectors	Status
RSA key transport (PKCS#1 v1.5)	Encrypt a chosen key under RSA	Padding oracle (Bleichenbacher, ROBOT)	No CCA2 proof; padding fragile	Deprecated in TLS 1.3
ElGamal encryption	Encode a chosen key as a group element	Malleable; roughly 2x expansion	No CCA2 proof	Historical
DHIES / DHAES	Derive the key from ephemeral-static DH	Underspecified as a deployable format	CCA2 proof (ODH); no shared vectors	Academic root of ECIES
ECIES (four standards)	Standardize DHIES	Fragmented; benign malleability	Proof gaps; no shared vectors	Legacy interop only
HPKE (RFC 9180)	KEM-DEM, all public keys bound in	Non-goals must be respected	IND-CCA2 proof; per-suite vectors	Current default
HPKE + X-Wing	Swap DHKEM for a hybrid PQ KEM	Larger ciphertexts; draft-stage	Hybrid PQ proof; draft vectors	Emerging

Every one of these defects points at the same missing discipline: pin one construction, bind the ambiguous inputs so malleability becomes impossible, prove the result, and publish vectors so anyone can check interoperation. That list is not a wishlist. It is, almost line for line, the changelog of RFC 9180. Here is what "done right" actually changed.

5. "Done Right" Was Discipline, Not Genius

There is no eureka moment in this story, and that is the point. HPKE's virtue is not a clever new algorithm -- the algorithm, KEM-DEM, was twenty years old when RFC 9180 shipped. Its virtue is four consolidating decisions, made once, correctly, with the proof and the test vectors published in the same body of work. Each decision closes one of the previous section's defects.

Decision one: abstract the KEM so the primitive is agile. Instead of freezing one curve, one KDF, and one MAC into a single scheme, HPKE is parameterized over a ciphersuite of three identifiers: kem_id, kdf_id, and aead_id [@rfc9180]. This looks like a small thing and is not. Fragmentation was the disease of the ECIES era, where each standards body froze a different combination.

HPKE replaces uncontrolled fragmentation with managed agility: many ciphersuites, but one construction and one set of rules governing all of them. When a component needs replacing -- as the post-quantum transition will require -- you change an identifier, not the scheme.

Decision two: bind every public key into the key schedule. This is the decisive fix, and RFC 9180 states it in one sentence.

"HPKE mitigates malleability problems (called benign malleability) in prior public key encryption standards based on ECIES by including all public keys in the context of the key schedule." -- RFC 9180, Section 9.1

What SEC 1 made optional, HPKE makes mandatory [@rfc9180], [@sec1]. Recall the benign-malleability trick: replace the ephemeral point $R$ with $-R$ to get a second valid ciphertext. In HPKE the public keys are inputs to the derived keys, so a mauled encapsulation derives a different key schedule and simply fails to produce a valid decryption. The attack does not become hard; it becomes impossible to express. The malleability is not patched -- it is designed out.

Decision three: mandate a single AEAD as the DEM. Gone is the ad-hoc two-key Encrypt-then-MAC with per-standard knobs for the cipher, the MAC, and the tag order. HPKE specifies one AEAD scheme per ciphersuite, so authenticity is intrinsic to the primitive rather than assembled by the caller [@rfc9180]. If the series' part on safe authenticated-encryption composition has a single thesis -- compose it wrong and the tag verifies while the forgery goes through -- then HPKE is simply that "compose it correctly" rule frozen into a specification.

Decision four: ship proofs and test vectors, developed alongside the standard. This is the part the ECIES era never had. HPKE began as an individual draft in 2019 and was analyzed as it was written [@hpke_draft]. Lipp's CryptoVerif analysis mechanically checked all four modes; the analysis by Alwen, Blanchet, Hauck, Kiltz, Lipp, and Riepel gave exact security bounds for the authenticated mode and was published at EUROCRYPT 2021; and the two pre-shared-key modes were later closed under the Gap Diffie-Hellman assumption [@lipp20], [@abhklr20], [@psk1480]. Every ciphersuite ships machine-checkable test vectors in the RFC itself [@rfc9180].

The standard security goal for public-key encryption. An adversary who may submit ciphertexts of its choice to a decryption oracle -- adaptively, even after seeing the challenge -- still cannot tell which of two chosen plaintexts a challenge ciphertext encrypts. It is the bar the ECIES standards could not prove and that malleability violates, and the bar HPKE's composition meets when its KEM and AEAD do [@cs01], [@hhk06].

Put the four together and your sense of what happened should shift.

Key idea: "Done right" was consolidation, not invention. Abstract the KEM for agility, bind every public key into the key schedule so benign malleability cannot happen, mandate one AEAD instead of ad-hoc Encrypt-then-MAC, and ship machine-checked proofs and per-ciphersuite test vectors in the same document. The innovation is rigor -- which is exactly why HPKE is boring, and why boring is the achievement.

Note: The KEM-DEM theory was published in 2001 and simply not adopted as the standard until 2022. HPKE is the merge of a two-decade-old proof track with a standards track: it did not out-think the ECIES authors so much as out-discipline them, pinning one construction and refusing to ship it without proofs and vectors [@cs01], [@rfc9180].

There is even a small tell that HPKE was designed to be built on, not just used directly.HPKE reserves an export-only AEAD identifier (the value 0xFFFF) for ciphersuites that never call Seal and only ever use the Export interface -- a strong hint that the authors expected HPKE to be keyed into other protocols rather than used standalone [@rfc9180].

Four decisions turn a fragmented, unprovable family into one boring, correct primitive. But "one primitive" is not "one behavior." HPKE ships a small menu -- four modes -- and choosing the wrong one is the first and most common way to misuse it. The menu is where the design stops being about the math and starts being about what you are trying to authenticate.

6. The Four Modes: HPKE's Core Menu

RFC 9180, Section 5.1, defines exactly four modes, and the difference between them reduces to a single question: what extra secret or identity gets folded into the key schedule? Get the answer wrong and you either fail to authenticate a sender you needed to, or -- worse -- come to believe you authenticated one you did not [@rfc9180]. There are four modes, never three: the pre-shared-key-plus-authenticated combination is a real, distinct mode that is easy to forget.

Mode	ID	Extra input	What it authenticates	Use when
`mode_base`	0x00	none	Only that the ciphertext is for this recipient key	You have just the recipient's public key
`mode_psk`	0x01	a pre-shared key	Recipient, plus possession of the PSK	A symmetric secret already exists out of band
`mode_auth`	0x02	sender's static key	That a specific sender key holder sent it	The recipient must know which sender key sealed it
`mode_auth_psk`	0x03	static key and PSK	Both a sender identity and PSK possession	You need sender identity and a shared secret

mode_base (0x00): nothing extra. The sender is anonymous; the derivation authenticates only that the ciphertext was sealed to this recipient's key. It is the default -- and, tellingly, the mode that every flagship deployment in the next section uses.

mode_psk (0x01): a pre-shared key. A symmetric secret known to both parties is folded into the schedule, so a valid decryption additionally proves the sender possessed the PSK.

Note: mode_psk authenticates possession of the PSK, which means a weak PSK is an authentication that anyone who guesses it can forge. RFC 9180 requires a PSK with at least 32 bytes of entropy, and the reason is not arbitrary: a low-entropy PSK opens a partitioning-oracle attack that the non-goals section quantifies. Treat the PSK as key material, never as a human-chosen secret [@rfc9180].

mode_auth (0x02): the sender's static key. Here Encap/Decap are replaced by AuthEncap/AuthDecap, which fold a Diffie-Hellman value between the sender's static key and the recipient into the shared secret. A valid decryption now proves the message came from the holder of that specific sender key [@rfc9180].

mode_auth_psk (0x03): both. A static sender identity and a shared secret, folded together.

flowchart TD B["mode_base 0x00: nothing extra"] --> KS["key_schedule context"] P["mode_psk 0x01: plus a pre-shared key"] --> KS A["mode_auth 0x02: plus sender static key via AuthEncap"] --> KS AP["mode_auth_psk 0x03: plus PSK and sender static key"] --> KS KS --> OUT["Same construction, four different derived key schedules"]

Now the subtlety that trips up careful engineers. The authenticated modes are a form of signcryption, and they carry a security distinction imported from that theory.

A primitive that combines encryption and sender authentication in one operation, cheaper than encrypt-then-sign. Its authentication is defined relative to *insiders* and *outsiders*: an outsider is any third party, while an insider is a party who holds one of the long-term keys. HPKE's authenticated modes are signcryption, so their guarantees are stated in these terms rather than as a public, third-party-verifiable signature.

The consequence matters: mode_auth is not a digital signature. Because the recipient shares the Diffie-Hellman secret that authenticated the sender, the recipient could have produced the same ciphertext, so it cannot prove to any third party who the author was -- there is no non-repudiation. The analysis by Alwen, Blanchet, Hauck, Kiltz, Lipp, and Riepel makes this precise with an insider-versus-outsider treatment lifted from signcryption theory [@abhklr20], [@signcryption]. If you want a third party to verify authorship, you want the digital-signature part of this series, not HPKE's auth mode.

The security properties proven for each mode, from RFC 9180's Table 6 via Lipp's mechanized analysis, are worth seeing side by side [@rfc9180], [@lipp20]:

Property	Base	PSK	Auth	AuthPSK
Message confidentiality (IND-CCA2)	Yes	Yes	Yes	Yes
Export secrecy (variable-length PRF)	Yes	Yes	Yes	Yes
Sender authentication	N/A	Yes	Yes	Yes
Extra secret assumed	recipient key secret	plus PSK secret	plus sender key secret	plus sender key and PSK

The structural model below makes the "same machinery, different inputs" claim concrete: it builds a stand-in key-schedule context for each mode and shows the derived key differs, though nothing about the construction changed.

{` // STRUCTURAL MODEL, NOT REAL CRYPTO. Shows only that mode changes the derived key. function toyMix(s){let h=2166136261>>>0;for(const ch of s){h^=ch.charCodeAt(0);h=Math.imul(h,16777619)>>>0;}return (h>>>0).toString(16);} const shared_secret = 'ss-from-KEM';

// Real HPKE: Extract a PRK from shared_secret, then Expand over mode + psk_id_hash + info_hash. // This toy just mixes them (plus the sender key for auth modes) to show the mode changes the key. function keySchedule(mode, pskId, senderKey) { const psk_id_hash = toyMix('pskid:' + (pskId || '')); const info_hash = toyMix('info:my-app-v1'); const ss = senderKey ? toyMix(shared_secret + '|auth:' + senderKey) : shared_secret; return toyMix(mode + '|' + psk_id_hash + '|' + info_hash + '|' + ss); }

console.log('base -> ' + keySchedule('0x00', null, null)); console.log('psk -> ' + keySchedule('0x01', 'shared-psk', null)); console.log('auth -> ' + keySchedule('0x02', null, 'sender-static-key')); console.log('auth_psk -> ' + keySchedule('0x03', 'shared-psk', 'sender-static-key')); console.log('all four derive different keys from identical machinery'); `}

Choosing a mode answers "who am I authenticating?" It says nothing about which curve, which KDF, which AEAD, and how not to shoot yourself with the nonce. That is the ciphersuite-and-usage layer -- where the test vectors finally earn their keep, and where the known HPKE implementation advisories actually live.

7. Ciphersuites and Correct Usage

A mode plus a ciphersuite is a complete HPKE configuration. The ciphersuite is three identifiers -- a KEM, a KDF, and an AEAD -- and the correct-usage rules are mostly about a single thing: never let the same nonce protect two messages.

Slot	Options	How to choose
KEM	DHKEM(X25519, HKDF-SHA256); DHKEM(P-256/P-384/P-521, HKDF-SHA256/384/512); DHKEM(X448, HKDF-SHA512)	X25519 unless policy requires NIST curves; encapsulation `enc` is 32 bytes for X25519, 65 for P-256
KDF	HKDF-SHA256, HKDF-SHA384, HKDF-SHA512	Match the KDF hash to the KEM and AEAD strength
AEAD	AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305, Export-only (0xFFFF)	AES-GCM with AES hardware; ChaCha20-Poly1305 without; export-only when you never call `Seal`

The three menus map onto earlier parts of this series. DHKEM's curves are the elliptic curves of the curves part; pick X25519 for new designs, NIST curves only when a compliance regime demands them. The KDF slot is the HKDF of the key-derivation part, used exactly as designed. The AEAD slot is the decision from the AEAD part: AES-GCM where the CPU has AES instructions, ChaCha20-Poly1305 where it does not. The tag is 16 bytes in every case.

The nonce and sequence discipline is the crux. Inside a Context, the per-message nonce is computed as nonce = base_nonce XOR seq, where seq is an incrementing sequence number. The Context is therefore stateful, and the sequence number must never repeat and must never overflow, because AEAD nonce reuse is catastrophic: for AES-GCM it can leak the authentication key, and for any of these AEADs it breaks confidentiality outright [@rfc9180]. This is the same hazard the nonce-reuse part of this series covers for AES-GCM, arriving here as a counter you must not let wrap.

The structural model below computes a few per-message nonces and then deliberately overflows a narrow counter so you can watch a nonce repeat -- which is, in miniature, exactly the implementation bug the misuse section names.

{` // STRUCTURAL MODEL, NOT REAL CRYPTO. Toy 8-bit nonce; real HPKE nonces are 96 bits. const base_nonce = 0xA7; function nonceFor(seq) { return (base_nonce ^ (seq & 255)).toString(16); }

for (let seq = 0; seq !== 5; seq++) { console.log('seq ' + seq + ' -> nonce ' + nonceFor(seq)); }

// A deliberately NARROW counter overflows after 256 messages and REPEATS a nonce. const width = 256; const seqA = 3; const seqWrap = seqA + width; // a later message with the same low 8 bits console.log('seq ' + seqA + ' -> nonce ' + nonceFor(seqA)); console.log('seq ' + seqWrap + ' -> nonce ' + nonceFor(seqWrap) + ' (wrapped)'); console.log('nonce reused after overflow = ' + (nonceFor(seqA) === nonceFor(seqWrap))); `}

Note: Prefer single-shot Seal/Open for independent messages -- it sets up, encrypts once, and tears down, so there is no counter to mismanage. If you hold a Context open, Seal strictly in order, respect the per-ciphersuite AEAD message-count and input-length limits, and never reuse a Context across two independent streams. A repeated or overflowed sequence number is a repeated nonce, and a repeated nonce is the end of your confidentiality [@rfc9180].

Two operational notes round out the usage rules. First, the AEAD message-count and plaintext-length limits are per-ciphersuite constants, and single-shot Seal/Open sidesteps the streaming bookkeeping entirely.When you need to protect many messages or key a different protocol's framing, use the Export interface to derive an independent secret rather than looping Seal on one long-lived Context -- this is exactly what Oblivious HTTP and Oblivious DNS do [@rfc9180]. Second, the one default to remember: mode_base with DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, and AES-128-GCM is the interop-safe choice, with ChaCha20-Poly1305 substituted on hardware without AES acceleration.

These rules read like an interop checklist because that is precisely what they are, and the only reason they can be a checklist is the test vectors: you can prove your library agrees with the specification byte for byte before you ship. But a checklist for using HPKE says nothing about where it has already won. The strongest evidence that HPKE is the right default is that the modern privacy stack is built on it.

8. Deployments as the Proof

The best argument that HPKE is state of the art in 2026 is not a benchmark. It is a deployment list. Every "encrypt to a party you have never spoken to" corner of the modern IETF stack runs on HPKE, and they almost all run on the same mode.

Deployment	Specification	What HPKE does	Mode and API
Encrypted Client Hello	RFC 9849 (2026)	Encrypts the inner ClientHello, including the server name	`mode_base`, `SetupBaseS`/`SetupBaseR`
Messaging Layer Security	RFC 9420 (2023)	Encrypts path secrets to tree nodes in TreeKEM	`mode_base`, `SealBase`
Oblivious HTTP	RFC 9458 (2024)	Encapsulates requests and responses to a gateway	`mode_base` plus `Export`
Oblivious DNS over HTTPS	RFC 9230 (2022)	Encrypts DNS queries to a target through a proxy	`mode_base` plus `Export`

Encrypted Client Hello. ECH encrypts the sensitive inner ClientHello -- including the server name a censor would want to read -- to the server's HPKE public key, published in an ECHConfig and fetched ahead of time. It uses SetupBaseS on the client and SetupBaseR on the server [@rfc9849].Currency pin: ECH is now Standards-Track RFC 9849, published in March 2026 by Rescorla, Oku, Sullivan, and Wood, and formerly known as draft-ietf-tls-esni. It is an RFC, not a draft, and its text references HPKE throughout [@rfc9849].

Messaging Layer Security. MLS uses HPKE inside TreeKEM: when a group member updates the ratchet tree, it encrypts the new path secrets to the public keys of the relevant tree nodes, using SealBase [@rfc9420]. Group-scale forward secrecy and post-compromise security come from the tree and the ratchet, not from HPKE.

Oblivious HTTP and Oblivious DNS. Both split trust between a relay and a gateway so that no single party sees both who you are and what you asked. The client seals its request to the gateway's HPKE key; the relay forwards ciphertext it cannot read; the gateway decapsulates and, crucially, uses Export to key the response rather than looping Seal [@rfc9458], [@rfc9230]. Oblivious DNS, in 2022, was the first shipped RFC deployment of HPKE.

flowchart LR C["Client seals request with SetupBaseS"] --> REL["Relay sees ciphertext and client address, not content"] REL --> GW["Gateway runs SetupBaseR, decrypts, Export-keys the reply"] GW --> TGT["Target sees the request, never the client address"]

Privacy Pass and its Private Access Tokens belong on the list too, at one remove: some deployments run token issuance over Oblivious HTTP (RFC 9578 does not require it), and therefore over HPKE [@rfc9578], [@rfc9458].

Two observations reinforce the article's spine. First, every marquee deployment uses mode_base: sender authentication and forward secrecy are supplied at the protocol layer -- by the TLS handshake, by the MLS ratchet -- leaving HPKE to be the boring sealing primitive it was designed to be. Second, Oblivious HTTP and Oblivious DNS reach for Export to key their own framing instead of looping Seal, exactly the extensibility the export interface was built for. The deployments did not stretch HPKE; they used the parts the design anticipated.

DHKEM is classical-only, which means every HPKE ciphertext sealed today is exposed to "harvest now, decrypt later": an adversary can store it and decrypt it once a quantum computer can solve the elliptic-curve Diffie-Hellman problem. The fix is the payoff of Decision one -- swap the KEM, do not redesign HPKE. The leading candidate is X-Wing, a hybrid KEM combining X25519 and ML-KEM-768 that stays secure if *either* component holds, and it comes with a dedicated "Use in HPKE" mapping [@xwing], [@xwing_draft], [@fips203]. The cost is size: an X-Wing encapsulation is roughly 1120 bytes against 32 for DHKEM(X25519) [@xwing_draft06]. Draft-stage ciphersuites also include a pure ML-KEM option, a generic hybrid-KEM framework, and a combined post-quantum suite with a SHA-3-based KDF [@mlkem_draft], [@hybrid_draft], [@barnes_pq]. One caveat matters for the modes section: X-Wing is *not* an authenticated KEM, so the post-quantum path powers `base` and `psk` but not yet `auth` or `auth_psk` [@xwing_draft06].

HPKE won the IETF privacy stack. But "won" invites the question a careful engineer actually asks: won against what, and when would I still reach for something else? The honest comparison comes next -- along with the one construction people keep miscategorizing as HPKE.

9. Competing Approaches, and One That Is Not HPKE

HPKE is the right default for new protocol design, but it is not the only encrypt-to-a-public-key tool, and it is not always the right one. Because all of these building blocks cost about the same -- roughly two elliptic-curve scalar multiplications for the sender, plus an ephemeral value and an AEAD tag of overhead -- the honest comparison is not speed. It is security model, agility, and proof discipline.

Tool	Construction	Agility and modes	Proof and shared vectors	Best suited for
HPKE (RFC 9180)	KEM-DEM (DHKEM + KDF + AEAD)	Full agility; four modes	Machine-checked; per-suite vectors	New protocol design
libsodium `crypto_box` / seal	Fixed X25519 + XSalsa20-Poly1305	None; mutual-auth, repudiable	No mechanized proof; partial vectors	Both ends yours, zero knobs
JOSE `ECDH-ES` (RFC 7518)	Ephemeral-static ECDH + Concat KDF	Some agility; encryption only	No mechanized proof; partial vectors	JWE and JWT stacks
`age`	X25519 + HKDF-SHA256 + ChaCha20-Poly1305	None; file-oriented	No mechanized proof; has vectors	Encrypting files to a key
Raw ECIES	Ephemeral-static ECDH + Encrypt-then-MAC	Per-standard knobs	Proof gaps; no shared vectors	Legacy and blockchain interop

libsodium crypto_box and sealed boxes. One fixed suite -- X25519, XSalsa20, Poly1305 -- simple and misuse-resistant, but with no agility, no formal mode menu, and no HPKE-grade test vectors; raw crypto_box also makes the caller manage nonces, and it is deliberately repudiable and mutually authenticated [@nacl]. Reach for it when you own both ends and want zero knobs.

JOSE and COSE ECDH-ES. Ephemeral-static ECDH with the Concat KDF, native to JWE and JWT [@rfc7518]. It has no auth or psk modes, uses Concat-KDF rather than HKDF, and ships nothing like HPKE's vectors. It is chosen by platform gravity, and a COSE-HPKE binding is emerging to bring HPKE into that world [@cose_hpke].

age. Wraps a random file key to X25519 recipients with HKDF-SHA256 and ChaCha20-Poly1305 -- the same primitive family as DHKEM(X25519) plus an AEAD, but fixed and file-oriented [@age]. Reach for it to encrypt files to a public key.

Raw ECIES survives only for interop with existing deployments, such as blockchain wallets and legacy PKI [@sec1]. And RSA-KEM (RFC 5990) is worth one line: it proves the KEM-DEM pattern generalizes beyond Diffie-Hellman, but RFC 9180 defines no RSA-KEM ciphersuite, so it stays a niche [@rfc5990].

Now the boundary marker, because it is the single most common miscategorization in this space.

Apple's PQ3 protocol for iMessage is *KEM-based* -- as Apple describes it, the design combines a hybrid post-quantum KEM (ML-KEM/Kyber-1024 with P-256 ECDH) through HKDF-SHA384, inside a triple-ratchet message scheme [@apple_pq3] -- but it is **not** an RFC 9180 HPKE deployment, and it never invokes HPKE. It solves a different problem: stateful, forward-secret, *interactive* messaging with post-compromise security, where HPKE is single-shot public-key encryption to a static key. PQ3 belongs here only to mark the single-shot-versus-ratchet boundary and to keep it off the HPKE deployment list, where it does not belong. Do not cite it as evidence that HPKE ships in iMessage; it does not.

The ranking has a pattern. The fixed-suite tools win on simplicity, ECDH-ES wins on platform gravity, and everything else loses to HPKE for new design, because HPKE has the agility, the modes, the proofs, and the vectors. That makes HPKE sound like a strict upgrade. It is not. Its power comes from a set of things it deliberately refuses to do -- and mistaking those refusals for oversights is where every real HPKE incident begins.

10. The Refusals That Look Like Features

HPKE's non-goals are not gaps someone forgot to fill. Several are impossibilities for a single-shot public-key primitive, and RFC 9180 states them plainly so that no one mistakes the boundary for a bug. Knowing them is the entire discipline this article is teaching.

No forward secrecy against recipient compromise, in any mode. RFC 9180 is explicit: HPKE ciphertexts "are not forward secret with respect to recipient compromise in any mode," because only long-term secrets are used on the recipient's side [@rfc9180]. This is structural, not an oversight.

The property that compromising a long-term key does not expose past messages. HPKE has none with respect to recipient compromise: because a single-shot ciphertext is encrypted to a static recipient key, anyone who later obtains that key can decrypt every ciphertext ever sent to it. Forward secrecy must come from a ratchet or key rotation *outside* HPKE -- which is exactly why MLS and TLS layer it on top.

auth mode is not a signature. As the modes section showed, the authenticated modes are signcryption with insider and outsider guarantees, not third-party non-repudiation. The recipient who shares the Diffie-Hellman secret could have produced the same ciphertext, so it cannot prove authorship to anyone else [@abhklr20], [@rfc9180]. If you need a portable proof of who signed, use a digital signature.

Not key-committing by default. HPKE's AEADs bind a ciphertext to a plaintext, but not to a unique key.

A property by which a ciphertext can be opened successfully under exactly one key. HPKE's standard AEADs are not key-committing by default, which enables a partitioning-oracle attack: an attacker who can submit ciphertexts and observe success or failure can binary-search a low-entropy secret. The eventual fix is a committing AEAD, whose properties are codified in RFC 9771 [@aead_props].

The concrete danger is a low-entropy PSK. RFC 9180, applying the partitioning-oracle result of Len, Grubbs, and Ristenpart, notes that an attacker can recover a PSK drawn from a set of size $m \cdot k$ in roughly $m + \log k$ oracle queries [@rfc9180], [@lgr20]. That logarithmic search is precisely why the PSK requirement is a floor on entropy, at least 32 bytes, and not a note about password strength. The committing-AEAD countermeasure is standardized as a set of properties in RFC 9771, awaiting a committing aead_id [@aead_props].

No replay, reordering, downgrade, or metadata protection. HPKE does not detect a replayed ciphertext, enforce message ordering across independent contexts, prevent ciphersuite downgrade, or hide the length or existence of a message. Every one of these is deliberately left to the surrounding protocol [@rfc9180].

Classical-only by default. This is the single primitive-level gap -- and, as the deployment section showed, the only non-goal you close by swapping a component rather than redesigning the scheme.

It is worth being precise about what is and is not finished. The core security question is closed: sufficiency and necessity together establish that the KEM-DEM composite reaches IND-CCA2 exactly when its halves do, and all four modes have been machine-checked [@cs01], [@hhk06], [@lipp20], [@abhklr20], [@psk1480].

The residual gaps are about model and API coverage, not the core: the tightest authenticated-mode proof covers single-shot use rather than many messages, some results live in the random-oracle model rather than the standard model, and the post-quantum composition result is tightest for the authenticated mode, with the psk and auth_psk modes since covered by the PSK-modes analysis and a fully general standard-model post-quantum proof across all four modes still open [@abhklr20], [@psk1480]. Even the definition of IND-CCA for KEMs has known subtleties the analyses had to navigate [@rfc9180], [@bhk09].

Key idea: The math is finished; the risk moved to the interface. HPKE has no known catastrophic break, so its hazards are misuse of its four modes and its non-goals -- and the post-quantum swap, which changes the KEM and not the scheme, is the proof that the architecture was the achievement. The discipline is knowing what HPKE deliberately does not do.

Note: HPKE's most important non-goals look like features waiting to be used: there is no forward secrecy against recipient compromise in any mode, auth is not a signature, and the AEADs are not key-committing. Assuming any one of the three is present is where real incidents begin [@rfc9180].

Every non-goal above is a sentence a designer might wrongly assume was a yes. So the honest failure catalog for HPKE is not a list of broken math -- there is none -- it is a list of those wrong assumptions, made in real code. Here they are, named.

11. The Failure and Misuse Catalog

HPKE itself has no known catastrophic real-world break, and that absence is the finding, not a gap in the research. This is a deliberately hedged absence-of-evidence claim: it rests on three machine-checked analyses and on the public advisories to date being implementation bugs, not design flaws -- never a claim that HPKE is "proven secure" for all time [@lipp20], [@abhklr20], [@psk1480]. The catalog is two things: the predecessor ECIES defects HPKE fixed, and the misuse patterns that appear when a non-goal is assumed away.

Misuse	Root cause	Non-goal assumed away	Fix or lesson
Sequence counter overflow	Fixed-width counter wraps, nonce repeats	AEAD nonce uniqueness is the caller's job	Wide counter; single-shot; the hpke-rs `u64` fix
Static-key forward-secrecy illusion	Expecting past secrecy from a static key	No FS against recipient compromise	Ratchet or rotate keys above HPKE
Low-entropy PSK	Guessable PSK enables partitioning oracle	AEADs are not key-committing	PSK with at least 32 bytes of entropy
Bad ephemeral randomness	Weak CSPRNG in `Encap`	HPKE assumes good randomness	Use a vetted CSPRNG
`auth` treated as a signature	Insider can forge to a third party	`auth` is signcryption, not non-repudiation	Use a digital signature for authorship
Out-of-order or reused `Open`	Ordering not enforced by HPKE	No reordering or replay protection	`Seal` in order; put ordering in the AAD
Export-only misuse	Calling `Seal` on an export-only suite	Interface contract not enforced	Error, do not truncate; the hpke-rs fix

The first advisory: hpke-rs, GHSA-g433-pq76-6cmf. In February 2026 the hpke-rs and hpke-rs-rust-crypto crates received a high-severity advisory bundling four fixes, and the reason it vindicates this article's spine rather than refuting it is that every fix maps onto a non-goal RFC 9180 had already flagged [@ghsa], [@rfc9180], [@hpke_rs]:

Sequence-number overflow leading to nonce reuse. The context counter could overflow; the fix widened it to a 64-bit integer. This is exactly the scenario the earlier structural model wraps in miniature, and exactly the AEAD nonce-uniqueness hazard the usage section warned about. It was reported by Kobeissi.
Export-only misuse and silent truncation. Calling Seal/Open on an export-only suite, and requesting an over-long Export, previously truncated silently; both now return errors. Reported by Arciszewski.
Missing X25519 zero and low-order key check. The library did not reject all-zero keys; it now errors on them, closing a small-subgroup foot-gun.
A post-quantum KEM-identifier mapping bug. A confusion in the draft X-Wing KEM-id mapping -- harmless in effect, but a foretaste of the ciphersuite-management care the post-quantum transition will demand.

The second advisory: @hpke/core / hpke-js, CVE-2025-64767. Three months earlier, in November 2025, the @hpke/core package -- the core of the hpke-js library in the survey below -- drew a critical advisory (GHSA-73g8-5h73-26h4, CVSS 9.1) for reusing AEAD nonces. The root cause was a concurrency race rather than a counter overflow: the public SenderContext Seal() API let two concurrent asynchronous seal() calls read the same sequence number, so both derived the same nonce and reused it in the AEAD, forfeiting confidentiality and integrity outright. Versions @hpke/core 1.7.4 and earlier are affected; the fix shipped in 1.7.5. The lesson is the misuse table's first row reached from a new direction -- nonce uniqueness is the caller's job -- so a single Context must never have Seal called on it concurrently: serialize per-context sealing, or prefer single-shot. It is a second, independent instance of the exact nonce-reuse hazard the usage section named, arrived at by a race instead of an overflow [@hpke_core_cve], [@rfc9180], [@hpke_js].

Notice what is not on either advisory: no broken cipher, no discrete-log break, no flaw in the KEM-DEM composition -- only dropped or mishandled checks and a concurrency race, each mapping onto a non-goal or requirement the specification had already named.

The rest of the catalog is misuse in application code, and it maps onto the table above. Expecting forward secrecy from a static recipient key yields none, in any mode [@rfc9180]. A low-entropy PSK turns mode_psk into a partitioning oracle, so the PSK needs at least 32 bytes of entropy [@rfc9180], [@lgr20]. Bad ephemeral randomness can collapse base-mode confidentiality outright, because a predictable ephemeral key undoes the encapsulation.

Treating auth as a signature walks into the insider-versus-outsider trap [@abhklr20], and out-of-order Open or context reuse breaks because applications, not HPKE, must preserve Seal order and put ordering in the associated data [@rfc9180]. The predecessor ECIES defects -- benign malleability, missing proofs, missing test vectors -- round out the list, each already closed by the four design decisions.

HPKE has no catastrophic break because the risk moved up a layer. Master the four modes and the non-goals and you have a boring, safe default. The math held; the risk moved to the interface.

Read the catalog back to front and a single pattern falls out: every entry is a non-goal treated as a goal, or an implementation dropping a check the specification demanded. None is a broken cipher. That means the "how to use HPKE safely" guide is not a research problem. It is the non-goals, turned into rules.

12. A Practical Decision Guide

Everything above collapses into a short decision procedure and a shorter list of things never to do.

flowchart TD START["Encrypt to a public key"] --> Q1{"New protocol needing agility, modes, proofs?"} Q1 -->|Yes| HPKE["Use HPKE"] Q1 -->|No| Q2{"Both ends yours, zero knobs?"} Q2 -->|Yes| BOX["libsodium sealed box"] Q2 -->|No| Q3{"JWE or JWT stack?"} Q3 -->|Yes| JOSE["ECDH-ES, or COSE-HPKE as it matures"] Q3 -->|No| Q4{"Encrypting files?"} Q4 -->|Yes| AGE["age"] Q4 -->|No| ECIES["Matched ECIES, legacy interop only"] HPKE --> M{"Which mode?"} M -->|Only the recipient key| BASE["mode_base"] M -->|A shared secret exists| PSK["mode_psk, PSK at least 32 bytes"] M -->|Need a sender identity| AUTH["mode_auth"] M -->|Both| AUTHPSK["mode_auth_psk"]

Which building block? A new protocol or standard that needs agility, proofs, and modes takes HPKE. Both ends under your control with no options wanted takes a libsodium sealed box. The JWE and JWT world takes ECDH-ES, or COSE-HPKE as it matures [@cose_hpke]. Encrypting files takes age. Interop with an existing ECIES deployment takes a matched ECIES configuration and nothing new [@rfc9180].

Which HPKE mode? Only the recipient's public key in hand: mode_base, the deployment default. A pre-shared secret already exists: mode_psk, with a PSK carrying at least 32 bytes of entropy. The recipient must know that a specific sender key sealed the message: mode_auth. Both a sender identity and a shared secret: mode_auth_psk. Never reach for an authenticated mode expecting a third-party-verifiable signature.

Which ciphersuite? The interop-safe classical default is DHKEM(X25519, HKDF-SHA256) with HKDF-SHA256 and AES-128-GCM, substituting ChaCha20-Poly1305 where AES is unaccelerated. If you only ever export, use the export-only AEAD (0xFFFF). If you are planning for quantum resistance, track the X-Wing and ML-KEM drafts, budget roughly 1.1 KB per encapsulation, and remember that the post-quantum suites give you base and psk but not yet auth [@xwing_draft06].

A worked path to shipping. Start with a single-shot Seal/Open for one message, which removes the entire counter-management problem. Reach for Export only when you are keying another protocol's framing. And before you trust your integration, reproduce one of RFC 9180's published test vectors end to end: if your library produces the specified bytes, you have proof it interoperates -- the very check the ECIES era could not run. The mature libraries below make this easy.

Library	Language	Notes
OpenSSL 3.2+	C	Native HPKE via `OSSL_HPKE_CTX` [@openssl_blog], [@openssl_api]
BoringSSL	C	HPKE used in the Chrome and ECH stack [@boringssl]
Cloudflare CIRCL	Go	Early production HPKE; powers OHTTP and ODoH [@cloudflare]
rust-hpke	Rust	RFC 9180-conformant crate [@rust_hpke]
hpke-rs	Rust	Formally oriented; ships a draft X-Wing KEM id [@hpke_rs]
hpke-js	JavaScript	Browser, Node.js, and Deno; full RFC 9180 suites; require `@hpke/core` 1.7.5 or later (fixes CVE-2025-64767) [@hpke_js], [@hpke_core_cve]

Note: Use mode_base single-shot with DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, and AES-128-GCM -- or ChaCha20-Poly1305 without AES hardware. Use Export to key your own framing rather than looping Seal. Track X-Wing and ML-KEM for the post-quantum transition, and budget about 1.1 KB per encapsulation when it lands [@rfc9180], [@xwing_draft06].

The list of nevers is short and worth memorizing: never expect forward secrecy from a static recipient key; never treat auth as a signature; never reuse a Context across independent streams or let the sequence counter overflow; never use a low-entropy PSK; never open one ciphertext under multiple keys and assume it committed to one; and never skip your library's test-vector check.

Run your library against RFC 9180's Appendix A vectors for the exact ciphersuite you ship. A mismatch on the first byte tells you -- before any traffic flows -- that your point encoding, your `info` handling, or your KDF wiring disagrees with the standard. This is the single check the four ECIES standards could never offer, and it is the reason "conformant" now implies "interoperable."

The checklist is short because the lesson is one sentence. Before restating it, clear away the confident-but-wrong beliefs that keep regenerating the misuse catalog in code review.

13. The Misconceptions, Corrected

HPKE gets misused mostly because a handful of confident, wrong sentences keep getting said in design meetings. Here they are, corrected.

No -- not with respect to recipient compromise, in any mode. RFC 9180 states that HPKE ciphertexts are not forward secret against recipient compromise, because only long-term secrets are used on the recipient's side [@rfc9180]. Forward secrecy has to come from a ratchet or key rotation layered above HPKE, which is exactly what MLS and TLS do. No. `auth` mode is signcryption, not a third-party-verifiable signature: the recipient cannot prove authorship to anyone else, for the reason the modes section works through. If you need portable proof of who signed, use a digital signature [@abhklr20]. A `Context` is stateful: an incrementing sequence number derives each per-message nonce. Seal strictly in order, never reuse a context across independent streams, respect the per-ciphersuite AEAD message-count limit, and never let the sequence counter overflow -- any of those repeats a nonce, which is catastrophic [@rfc9180]. For independent messages, prefer single-shot `Seal`/`Open` and avoid the bookkeeping entirely. Not by default -- DHKEM is classical-only. The migration path is to drop a hybrid KEM such as X-Wing (X25519 with ML-KEM-768) into the KEM slot, secure if either component holds [@xwing], [@fips203]. Those suites are still draft-stage, add roughly 1.1 KB per encapsulation, and are not authenticated KEMs yet, so they cover `base` and `psk` but not `auth` [@xwing_draft06]. `crypto_box` is one fixed suite (X25519, XSalsa20, Poly1305), deliberately repudiable and mutually authenticated, with manual nonce handling for the raw API [@nacl]. It is excellent when you own both ends and want no options. HPKE wins for new protocol design, because it brings algorithm agility, four formal modes, machine-checked proofs, and test vectors [@rfc9180]. No -- PQ3 is KEM-based but not HPKE; it is a ratcheted, interactive scheme rather than single-shot encryption, as the earlier PQ3 aside details [@apple_pq3]. Do not list it as an HPKE deployment. Only if you use low-entropy pre-shared keys or open one ciphertext under multiple candidate keys. HPKE's AEADs are not key-committing by default, which is why a weak PSK enables a partitioning oracle; keep the PSK entropy at 32 bytes or more [@rfc9180], [@lgr20]. A committing AEAD, whose properties are codified in RFC 9771, may eventually arrive as a new `aead_id` [@aead_props].

Every correction points at the same root: the primitive was never the risk. It is time to say the sentence the whole article was built to earn.

14. The Math Held; The Risk Moved to the Interface

Return to where this started: an elementary task -- encrypt to a public key -- that the industry got subtly wrong four incompatible times, with no proofs and no shared test vectors, one standard shrugging off a second valid ciphertext as "benign" [@sec1], [@maea10]. HPKE is what that task looks like when it is finally done right.

The evidence has accumulated into a single claim. HPKE is one KEM-DEM construction: a Key Encapsulation Mechanism generates a shared secret and its encapsulation, an HKDF key schedule with every public key bound in derives the working keys, and an AEAD seals the payload.

It is proven secure against adaptive chosen-ciphertext attack under standard assumptions, machine-checked across all four modes, test-vectored per ciphersuite, and it designs benign malleability out by making the public-key binding mandatory rather than optional [@rfc9180], [@cs01], [@lipp20], [@abhklr20]. The consequence is the whole thesis: HPKE has no known catastrophic break, so the discipline it demands is not "how do I make it secure" but "do I know its four modes and its non-goals?"

Every entry in the misuse catalog proved the point. Forward secrecy expected from a static key, a signature expected from auth mode, safety expected from a non-committing AEAD, a fresh nonce expected from an overflowed counter -- each was a non-goal assumed away, not a broken cipher [@rfc9180], [@ghsa].

And the post-quantum migration is the architecture's final vindication: the one primitive-level gap, quantum resistance, is closed by swapping the KEM for a hybrid like X-Wing, not by redesigning the scheme [@xwing]. That is the exact payoff of having abstracted the KEM in the first place. It echoes this series' running lesson from the KEM-DEM and digital-signature parts: the math held, and the interface is where the work -- and the risk -- actually lives.

So carry one question out of this article and apply it to any encrypt-to-a-public-key design you review.

Which of the four modes is this, and which of HPKE's non-goals am I about to assume it has?

The primitive was never the weak link, and the post-quantum swap shows it never will be. That is why this is Part 19 of a field guide to protocol design, not a chapter about any one cipher. HPKE is the clean seam where the primitives of the earlier parts -- authenticated encryption, key derivation, elliptic curves, KEM-DEM, signatures -- become a protocol. The math is finished, the vectors are published, and the only thing left to get wrong is you.

Nobody Broke the Discrete Log: A Field Guide to Diffie-Hellman and Forward Secrecy

noreply@paragmali.com (Parag Mali) — Mon, 13 Jul 2026 02:35:50 GMT

**Diffie-Hellman's hardness -- the discrete logarithm -- has held for fifty years and almost never breaks in the field.** What breaks is one of four conditions the caller must supply, plus one discipline beneath them all.

Forward secrecy needs a genuinely fresh ephemeral: reuse it and the property silently evaporates, which is why the paper that named the problem is titled Imperfect Forward Secrecy. The group must be strong and unshared: finite-field DH has a sub-exponential attack whose expensive step depends only on the prime, so millions of servers sharing one 1024-bit prime let a nation-state precompute once and read everyone, while a 256-bit curve escapes this -- which is why X25519 keys are 32 bytes. The peer's public value must be validated: an unvalidated point turns your reused key into a key-extraction oracle, and X25519 makes that check structural through twist security, so clamping is not validation. The exchange must be authenticated: raw DH stops a passive eavesdropper and not an active man-in-the-middle. Beneath all four, the raw shared secret is a group element, not a key: run it through a KDF.

The fifty-year evolution -- finite-field DH in 1976, then elliptic-curve DH, then X25519, then the TLS 1.3 ephemeral mandate, then today's X25519MLKEM768 hybrid -- is one sustained move to bake each condition into the primitive, the named group, and the protocol, so the simple thing to do is the safe thing to do. Every one of these still falls to Shor, which is why ECDH is now being wrapped, not replaced. The math held; the deployment handed the secret away.

1. They Precomputed the Prime Once, and Read Everyone

In 2015 a team of researchers pointed out something that should have been impossible: millions of HTTPS, SSH, and VPN servers were sharing the same handful of 1024-bit Diffie-Hellman primes. That sharing did not merely weaken those servers -- it changed the economics of breaking them. The most expensive step of the best known algorithm for computing discrete logarithms depends only on the prime, not on any particular connection, so an adversary who could afford the precomputation for one common prime could then decrypt, passively and forever, every connection that used it [@weakdh].

Breaking the single most common 1024-bit group would have exposed an estimated 18% of the top million HTTPS sites, and a second common prime an estimated 66% of VPNs [@weakdh]. Nobody had to break the math. They precomputed the prime once and read everyone.

Here is the part that should unsettle you. The people running those servers were not careless. They believed they were doing the one thing everyone agrees you must do: generate a brand-new key for every connection. The researchers put it plainly -- practitioners "believed this was safe as long as new key exchange messages were generated for every connection" -- and they were wrong, because a shared prime and a fresh per-connection secret are two different conditions, and only one of them was being met [@weakdh].

The attack, named Logjam, actually had a second half: an active attacker could force a browser and server to negotiate a deliberately breakable 512-bit "export-grade" group, so that even a modest adversary could solve the discrete log in real time [@weakdh]. Two coupled failures -- a downgrade and a precomputation -- and neither one touched the underlying hardness assumption. That active downgrade even had an exact RSA twin the same year, FREAK, which forced a browser down to breakable 512-bit "export" RSA instead of Diffie-Hellman [@freak]. The paper's title says the whole thing in two words: Imperfect Forward Secrecy.

The math held; the deployment handed the secret away.

This article is about that gap, and about closing it. The security a system actually gets from Diffie-Hellman is never simply "the discrete logarithm is hard." It is a stack of four conditions the caller must independently satisfy -- a genuinely fresh ephemeral secret, a strong and unshared group, a validated peer value, and an authenticated exchange -- plus one discipline underneath them: the raw output is a group element, not a key, so it must pass through a key derivation function.

Every world-scale Diffie-Hellman break you half-remember -- Logjam, small-subgroup key recovery, invalid-curve attacks on TLS, Raccoon, the anonymous man-in-the-middle -- is exactly one of those conditions quietly violated. The same unbroken math spawned a whole catalog of disasters, and the fifty-year history of the primitive is one sustained effort to pull those conditions back inside the primitive, the named group, and the protocol, so that a caller cannot forget them.

To see why a shared prime is a catastrophe -- and why "we generate a fresh key every connection" was true yet still left everyone exposed -- you first have to be precise about what a Diffie-Hellman exchange promises, and the four jobs it quietly delegates to you.

2. What Diffie-Hellman Actually Is, and the Four Things You Must Supply

Two strangers who have never met, standing on either side of a wire the whole world can read, end up sharing a secret that no eavesdropper can compute. That is one of the deepest ideas in computer science, and it hides four decisions the scheme refuses to make for you.

The mechanism, with just enough math

Fix a public group with a generator $g$. Alice picks a secret number $a$ and sends $g^a$; Bob picks a secret $b$ and sends $g^b$. Each raises what they received to their own secret, and because exponents commute, they land on the same value:

$$g^{ab} = (g^a)^b = (g^b)^a.$$

An eavesdropper sees $g$, $g^a$, and $g^b$, but never $a$ or $b$. A student can implement the whole thing in three lines of modular exponentiation.

A protocol in which two parties exchange the public values $g^a$ and $g^b$ over an open channel and each computes the shared secret $g^{ab}$, which an eavesdropper who sees only $g$, $g^a$, and $g^b$ cannot compute.

{// Toy Diffie-Hellman over a tiny prime field. Illustrative sizes, NOT secure. const p = 2147483647n; // a small 31-bit prime; a real DH prime is >= 3072 bits const g = 7n; // a public generator element function powmod(base, exp, m){ let r = 1n; base %= m; while (exp > 0n){ if (exp & 1n) r = (r * base) % m; base = (base * base) % m; exp >>= 1n; } return r; } const a = 1234567n; // Alice's fresh ephemeral secret (never sent) const b = 7654321n; // Bob's fresh ephemeral secret (never sent) const A = powmod(g, a, p); // Alice sends this over the open wire const B = powmod(g, b, p); // Bob sends this over the open wire const secretAlice = powmod(B, a, p); // Alice computes (g^b)^a const secretBob = powmod(A, b, p); // Bob computes (g^a)^b console.log('Alice g^ab =', secretAlice.toString()); console.log('Bob g^ab =', secretBob.toString()); console.log('do they agree? ->', secretAlice === secretBob); // The raw shared value is a group element, NOT a key. Real code never uses it // directly: it feeds g^ab into a KDF (HKDF-SHA256) to extract a uniform key. const toyKey = (secretAlice * 2654435761n) % (2n ** 32n); // stand-in for HKDF only console.log('derived key (toy stand-in) =', toyKey.toString());}

Why it is hard to break

The security rests on the discrete logarithm problem: given $g$ and $g^a$, recover the exponent $a$. In a well-chosen group this is believed intractable.

Given a generator $g$ and an element $g^a$ in a large group, the problem of recovering the exponent $a$. It is believed hard in carefully chosen groups, and it is the foundation Diffie-Hellman rests on.

But hardness of the discrete log is not quite the property we need. Recovering $a$ is the discrete-log problem; computing the shared secret $g^{ab}$ from $g^a$ and $g^b$ without recovering either exponent is the Computational Diffie-Hellman (CDH) assumption, a slightly weaker thing to assume hard. Sharper still: "the shared secret is indistinguishable from a random group element" is the Decisional Diffie-Hellman (DDH) assumption, and it is strictly stronger than CDH [@cryptobook]. The distance between "hard to compute" and "looks random" is exactly why the raw output of the exchange is not yet a key.

CDH is the assumption that computing $g^{ab}$ from $g^a$ and $g^b$ is hard. DDH is the stronger assumption that $g^{ab}$ cannot even be distinguished from a random group element. DDH can fail in groups where CDH still holds, which is why the raw shared value is not treated as a uniform key.

There are real groups -- pairing-friendly elliptic curves -- where DDH is easy even though CDH is plausibly hard, so $g^{ab}$ is computable-hard yet distinguishable from random.This is why "looks random" is the decisional assumption, not the computational one. The one-line consequence, paid off in the forward-secrecy and theory sections, is that you must extract a key from the shared value with a KDF rather than use it raw. Keep that fact in your pocket; it becomes a theorem later.

The thesis, stated as the primitive's interface

Now look at what the exchange quietly assumes but never enforces. Raw finite-field Diffie-Hellman outsources all of the following to the caller, at once:

Freshness. The secret $a$ must be genuinely fresh per session, then destroyed. This is what buys forward secrecy.
A strong, unshared group. The group must be large enough and not shared in a way that lets an attacker amortize work across everyone who uses it.
A validated peer value. The incoming $g^b$ must actually be a well-formed element of the right group, not a hostile input chosen to leak your secret.
Authentication. Something must bind $g^a$ and $g^b$ to identities, or an active attacker sits in the middle.

And beneath all four, a fifth discipline: the raw $g^{ab}$ is a structured group element, not a key, so it must go through a key derivation function before anyone calls it a key [@rfc5869]. The original 1976 construction supplies none of these; it hands you a shared secret and trusts you to remember the rest [@dh1976].

flowchart TD A["Alice: fresh secret a, send g^a"] --> S["Shared value g^(ab)"] B["Bob: fresh secret b, send g^b"] --> S S --> K["KDF turns g^(ab) into a real key"] F1["1. Freshness: is a truly ephemeral and destroyed"] -.-> A F2["2. Strong and unshared group"] -.-> S F3["3. Validate the peer value g^b"] -.-> B F4["4. Authenticate the exchange"] -.-> S

This table is the entire article compressed into a grid: each row a condition, each column showing who historically lifted that burden off the caller. It fills in from left (the caller does everything by hand) to right (the primitive, the named group, or the protocol does it).

Condition	What the caller must supply	Where it eventually gets pulled inside
1. Freshness	A per-session ephemeral secret, destroyed after use	The protocol: TLS 1.3 mandates ephemeral (EC)DHE
2. Strong, unshared group	A large prime or curve nobody can precompute against	The named group: RFC 7919 FFDHE safe primes; strong curves
3. Validated peer value	An on-curve, correct-subgroup public value	The primitive: X25519 twist security; or a hand-written check for classic ECDH
4. Authentication	A signature or MAC binding the transcript to identities	The protocol: STS/SIGMA, then TLS 1.3
5. KDF discipline	Extract a uniform key from the raw shared value	Every modern design: HKDF over the shared secret

Key idea: The security a system gets from Diffie-Hellman is never "the discrete log is hard." It is four conditions the caller must supply -- fresh, strong-and-unshared, validated, authenticated -- and then a KDF beneath them. Every named break in this article is exactly one of those quietly missing.

Note: The raw $g^{ab}$ (or, on a curve, the shared x-coordinate) is a group element, not a key. Run it through a KDF such as HKDF before you ever call it a key. This is the discipline beneath all four conditions, and later it turns out to be a theorem, not a style preference.

Three of those four conditions were not obvious in 1976. Each was discovered the hard way, by a named break. To understand why a fifty-year-old idea still hands them to you, go back to the moment key agreement was only an idea -- an idea that, it turns out, had already been discovered in secret.

3. Where This Came From: 1976, and the Secret That Came First

In 1976 two Stanford researchers opened a paper with a sentence that has aged remarkably well: "We stand today on the brink of a revolution in cryptography" [@dh1976]. Whitfield Diffie and Martin Hellman were right about the revolution. They were also, unknown to them, second.

The problem before the answer

Before public key agreement, two parties who wanted to talk securely had to already share a secret key, delivered out of band -- by a trusted courier, a locked briefcase, a prior meeting. That does not scale. A network of $n$ parties who each want to talk privately needs on the order of $n^2$ pairwise keys, every one of them distributed and stored in advance. The cost of moving keys, not the cost of encryption, is what throttled commercial cryptography for decades.

The public origin

Diffie and Hellman's "New Directions in Cryptography" broke the deadlock. Their key exchange let two parties agree on a shared secret over a channel an adversary could fully observe: fix a public generator $g$ over a finite field, keep the exponents private, exchange $g^a$ and $g^b$, and let both sides compute $g^{ab}$, with security resting on the discrete logarithm modulo a large prime [@dh1976]. The same paper credits Ralph Merkle's independent "puzzles" idea as a partial solution to the same problem, which is the seed of the argument -- pressed for years by Hellman -- that the scheme should be called Diffie-Hellman-Merkle [@dh1976], [@merkle]. Merkle's own framing was that his construction "forces any enemy to expend an amount of work which increases as the square of the work required of the two communicants" [@cacm-merkle].

Merkle conceived his puzzles around 1974 and saw a 1975 write-up rejected; the work was not published until 1978. Yet Diffie and Hellman's 1976 paper already cites it as "submitted to the CACM," which fixed the idea in the literature before its own publication -- part of why Hellman argued for the three-name credit [@merkle].

The classified precursor

Here is the part most textbooks skip. Inside the United Kingdom's signals-intelligence agency, GCHQ, and its information-security arm CESG, the same ideas had already arrived, and stayed locked in a safe. James Ellis conceived what he called "non-secret encryption" at the end of the 1960s -- the recipient, he realized, only needs to be in a different position from the interceptor, not a secret one [@ellis]. Clifford Cocks worked out the trapdoor that we would now recognize as RSA in 1973, and Malcolm Williamson derived the discrete-exponentiation key exchange -- Diffie-Hellman in substance -- in 1974 [@wikipedia-cocks]. All of it remained classified until December 1997 [@wikipedia-cocks].

The counterfactual sharpens the whole thesis of this article. The idea of public key agreement existed inside a government agency and changed nothing about how the world built systems, because it was never published, never standardized, never turned into a named group or a protocol anyone could deploy. An idea that no caller can reach protects no one.

The GCHQ dates are genuinely disputed. Ellis's own account traces the origin only to "the 60's"; the widely repeated "1970" is the date of his internal report. The primary CESG and gchq.gov.uk pages that once hosted this history are gone, so the record here rests on the archived Ellis document plus corroborating biography, not on any live government URL [@ellis], [@wikipedia-cocks].

timeline title From a secret idea to a mandated protocol 1969 : Ellis conceives non-secret encryption inside GCHQ 1974 : Williamson derives DH-style key exchange, kept classified 1976 : Diffie and Hellman publish New Directions, finite-field DH 1985 : Miller proposes elliptic-curve cryptography (Koblitz independently, published 1987) 1992 : Station-to-Station signs the DH values for authentication 2006 : Bernstein publishes Curve25519 and the X25519 function 2015 : Logjam shows shared 1024-bit primes are a mass liability 2018 : TLS 1.3 mandates ephemeral key exchange, removes static DH 2024 : X25519MLKEM768 hybrid ships by default in OpenSSH 9.9

Diffie and Hellman had shown that two strangers could agree on a secret against an eavesdropper who only listens. They said almost nothing about an adversary who acts. That omission is not a flaw in the paper; it is the second of the four conditions, and closing it took another two decades.

4. The Anonymous-DH Gap: Why Raw DH Is Only Half a Protocol

Raw ephemeral Diffie-Hellman defeats a passive eavesdropper completely, and an active attacker not at all. The whole difference is one adversary who can inject packets, and it is the entire reason "just do a DH exchange" is never the whole answer.

The man-in-the-middle, mechanically

Suppose an attacker sits on the wire and can not only read but rewrite. When Alice sends $g^a$ toward Bob, the attacker keeps it and sends its own $g^m$ onward. When Bob replies with $g^b$, the attacker keeps that too and sends $g^m$ back to Alice. Now the attacker shares one secret, $g^{am}$, with Alice, and a different secret, $g^{bm}$, with Bob. It decrypts whatever Alice sends, reads or modifies it, re-encrypts under the Bob-side secret, and forwards. Neither party sees anything wrong: both completed a textbook-perfect exchange. The primitive simply never promised anything about who is on the other end.

sequenceDiagram participant A as Alice participant M as Attacker participant B as Bob A->>M: sends g^a intended for Bob M->>B: substitutes g^m B->>M: sends g^b intended for Alice M->>A: substitutes g^m Note over A,M: Alice agrees on secret g^am with the attacker Note over M,B: Bob agrees on a different secret g^bm Note over M: Attacker decrypts, reads, re-encrypts both directions

This is a design gap, not an implementation bug. It falls to an active adversary specifically; a passive listener still learns nothing.

{// Toy anonymous-DH MITM. Illustrative sizes, NOT secure. const p = 2147483647n, g = 7n; function powmod(base, exp, m){ let r = 1n; base %= m; while (exp > 0n){ if (exp & 1n) r = (r*base) % m; base = (base*base) % m; exp >>= 1n; } return r; } const a = 111111n, b = 999999n; // honest ephemeral secrets const A = powmod(g, a, p); // Alice's message (intercepted) const B = powmod(g, b, p); // Bob's message (intercepted) const m = 424242n; // the attacker's own secret const M = powmod(g, m, p); // the attacker forwards THIS to both sides const aliceSide = powmod(M, a, p); // Alice computes (g^m)^a const attackerA = powmod(A, m, p); // attacker computes (g^a)^m -> equal const bobSide = powmod(M, b, p); // Bob computes (g^m)^b const attackerB = powmod(B, m, p); // attacker computes (g^b)^m -> equal console.log('Alice-attacker secret agrees?', aliceSide === attackerA); console.log('Bob-attacker secret agrees? ', bobSide === attackerB); console.log('Do Alice and Bob share a key?', aliceSide === bobSide); // false // The attacker now holds both session secrets and relays, reading everything.}

A key-agreement protocol that also binds the exchanged public values to identities, so that an active attacker cannot substitute its own values undetected. It is what raw Diffie-Hellman is missing.

The precision that carries the rest of the article

Authentication defeats the active adversary, and only the active adversary. It is orthogonal to the hardness of the group and to input validation. The sections ahead each violate a different condition, and it is tempting but wrong to think a signature would have saved them.

Note: Authentication closes the man-in-the-middle, and only that. It does not make a weak prime strong, an unvalidated point safe, or a reused ephemeral fresh. Those are three other conditions, each with its own catalog of breaks.

The fix, historically

Naming the goal came first. Christoph Guenther's 1989 identity-based key-exchange work is where the term perfect forward secrecy is generally said to first appear, attached to the goal that a compromised long-term key should not expose past sessions [@gunther1989]. Diffie, van Oorschot, and Wiener turned it into a concrete protocol in 1992 with Station-to-Station, which authenticates the exchange by having each side sign the Diffie-Hellman values, and which gave the standard, most-cited formal treatment of the property [@sts1992]. A decade later Hugo Krawczyk distilled the pattern into SIGMA, the "SIGn-and-MAc" template that underlies IKE and, in spirit, the TLS 1.3 handshake; the SIGMA protocols "provide perfect forward secrecy" via a Diffie-Hellman exchange authenticated with digital signatures [@sigma2003]. The authenticated-DH family is treated as standard material in the reference texts [@cryptobook], [@hac].

The coinage is disputed and worth stating carefully: the term "perfect forward secrecy" traces to Guenther (1989/1990), while Diffie, van Oorschot, and Wiener (1992) gave the standard, most-cited formal treatment via Station-to-Station. Present both and pick neither silently [@gunther1989], [@sts1992].

Signing the DH values closes the man-in-the-middle. It does nothing for the group the exchange runs in -- and the original group, the multiplicative integers modulo a prime, turned out to hand the caller two more conditions at once, plus a nation-state-scale way to violate them.

5. Generation 1: Finite-Field DH and the Tyranny of the Shared Prime

Here is the mechanism underneath Logjam, the small-subgroup CVEs, and those 3072-bit key sizes you have wondered about -- and here is why the original finite-field group hands the caller more traps than any generation since.

Finite-field DH, mechanically

Pick a large prime $p$, ideally a safe prime of the form $p = 2q + 1$ with $q$ also prime, and a generator $g$ of the large prime-order subgroup. Alice and Bob exchange $g^a \bmod p$ and $g^b \bmod p$ and compute $g^{ab} \bmod p$. The safe-prime shape is not cosmetic: it forces the cofactor down to 2, leaving only the trivial order-two subgroup (the values $1$ and $p-1$). So a small-subgroup attack can pin a key to at most one bit, which is exactly why an implementation must still reject $0$, $1$, and $p-1$ -- a point that becomes concrete below.

A prime of the form $p = 2q + 1$ where $q$ is also prime. The multiplicative group modulo a safe prime has one large prime-order subgroup of order $q$ and only the trivial order-two subgroup (the values $1$ and $p-1$), a cofactor of 2. That confines any small-subgroup attack to at most one bit, which is why implementations still reject $0$, $1$, and $p-1$.

The cost that shaped everything

Finite-field discrete logs have a sub-exponential attack. Index calculus, and its modern form the number field sieve (NFS), runs in heuristic complexity

$$L_p[1/3,\ (64/9)^{1/3}] = \exp!\left((1.923 + o(1)),(\ln p)^{1/3}(\ln\ln p)^{2/3}\right),$$

far faster than brute force. The practical consequence: to reach roughly 128-bit security you need a 3072-bit prime, an order of magnitude larger than the 256-bit curve that buys the same margin [@sp80057]. That size penalty is the whole reason finite-field DH lost.

The best known sub-exponential algorithm for factoring and for finite-field discrete logarithms. Its most expensive stage depends only on the modulus (the prime), not on any individual target, which is why a shared prime is a shared weakness.

Logjam, in full

That "depends only on the prime" clause is the entire disaster. The NFS's expensive first step "is dependent only on this prime," so if millions of hosts share a handful of 1024-bit primes -- and in 2015 they did -- an adversary precomputes once and then decrypts every connection on that prime cheaply, passively, and indefinitely [@weakdh]. The Logjam researchers estimated that breaking the single most common 1024-bit prime would expose about 18% of the top million HTTPS sites, and a second common prime about 66% of VPNs and 26% of SSH servers [@weakdh]. And there was an active half: an attacker could downgrade a TLS handshake to 512-bit DHE_EXPORT parameters, which were breakable in near-real time, with 8.4% of the top million initially vulnerable [@weakdh].

The paper priced that precomputation directly, and the numbers are the point. Breaking a 512-bit export prime was demonstrated, not merely estimated: about 89,000 core-hours of work -- "slightly over one week" of wall-clock time spread across a couple of thousand cores -- after which each individual discrete log took a median of 70 seconds [@weakdh]. A 768-bit prime was an academic reach at roughly 36,500 core-years. And a 1024-bit prime was a nation-state one at roughly 45 million core-years of one-time precomputation, a figure the authors judged "not necessarily out of reach for a nation state" [@weakdh]. Every number there is a fixed, per-prime cost that reading one more connection barely adds to.

flowchart TD P["One shared 1024-bit prime"] --> NFS["NFS first step: precompute per prime, about 45M core-years at 1024-bit, done once"] NFS --> DB["Reusable precomputation for that prime"] DB --> D["Per-connection descent: about 70 s at 512-bit"] D --> R["Passively decrypt every session on that prime"] P -.-> X["Active downgrade to 512-bit export group"] X --> R

Notice what never happens in that diagram: nobody solves a fresh discrete logarithm. The costly work is done once, against the prime, and amortized across everyone who shares it.

Practitioners "believed this was safe as long as new key exchange messages were generated for every connection" -- and were wrong.

They were wrong because a fresh per-connection secret satisfies the freshness condition while doing nothing about the unshared-group condition. Both had to hold; only one did [@weakdh].

Key idea: The attacker never computed a fresh discrete logarithm. They precomputed the shared prime once and read everyone on it. The math held; the deployment handed the secret away.

The RSA twin, in one breath

Logjam did not travel alone. The same year, and the same relic of 1990s U.S. export-crypto policy, produced its RSA counterpart: FREAK, for "Factoring RSA Export Keys," publicly disclosed on 3 March 2015 by Karthikeyan Bhargavan and the miTLS team, with the client-side flaw tracked as CVE-2015-0204 [@freak]. A man-in-the-middle downgrades a client to 512-bit export RSA, factors that small modulus cheaply on rented hardware, and reads the session. At disclosure 36.7% of HTTPS servers with browser-trusted certificates still accepted export RSA -- falling to 6.5% as patches landed -- along with 8.5% of the Alexa Top-1M [@freak].

FREAK downgrades RSA key transport and is fundamentally an implementation bug, a client-side state-machine slip that accepts a key it never asked for; Logjam downgrades Diffie-Hellman parameters and is, in the weakdh.org authors' own words, "due to a flaw in the TLS protocol rather than an implementation vulnerability" [@weakdh]. RSA key transport is the RSA installment's territory, so FREAK stays a mention -- but it is proof that the export-downgrade wound had two matching edges, one for each key-exchange family.

Two precisions to keep straight. First, Logjam is two coupled failures, not one: the 512-bit export downgrade and the shared-prime precomputation. Second, FREAK is its RSA sibling, not the same attack -- both weaponize leftover 512-bit export keys, but FREAK is an implementation bug in RSA key transport while Logjam is a protocol flaw in Diffie-Hellman parameter negotiation [@weakdh], [@freak].

Small-subgroup confinement

The safe-prime rule has teeth. If $p$ is not a safe prime, the group has small subgroups, and an attacker can weaponize them. The attacker sends a carefully chosen element of small order $r$ as its "public key." The victim raises it to its private exponent $x$, producing an element that depends only on $x \bmod r$, and the attacker -- watching whether the handshake succeeds, or reading the resulting ciphertext -- recovers $x \bmod r$ by trying the $r$ possibilities. Repeat across several small factors and combine with the Chinese Remainder Theorem, and the private exponent falls out.

This is the small-subgroup confinement attack of Lim and Lee (CRYPTO '97), and it works only when the victim reuses the private exponent long enough to be probed [@limlee1997], [@twist]. It went live as CVE-2016-0701, where OpenSSL failed to run the subgroup check on X9.42-style (RFC 5114) non-safe-prime parameters while reusing a DH key, letting an attacker "discover a private DH exponent" [@openssl-cve], [@nvd-0701].

An attack that sends a small-order group element as a public key so that the victim's response depends only on the private exponent reduced modulo a small number. Repeating across several small factors and applying the Chinese Remainder Theorem recovers a reused private exponent.

{// A NON-safe prime: p is prime but p-1 = 2^3 * 13 * 19 * 53. Insecure sizes. const p = 104729n; const g = 12n; // a generator of the full multiplicative group function powmod(b, e, m){ let r = 1n; b %= m; while (e > 0n){ if (e & 1n) r = (r*b)%m; b = (b*b)%m; e >>= 1n; } return r; } const x = 8675n; // Bob's REUSED private exponent -- the fatal choice const smalls = [13n, 19n, 53n]; // small prime factors of p-1 const residues = []; for (const r of smalls){ const h = powmod(g, (p - 1n) / r, p); // an element of small order r const target = powmod(h, x, p); // the victim returns h^x = h^(x mod r) let e = 0n; for (let k = 0n; k < r; k++){ if (powmod(h, k, p) === target){ e = k; break; } } // tiny search residues.push([r, e]); console.log('recovered x mod ' + r + ' = ' + e); } // Combine the residues with the Chinese Remainder Theorem: let Mtot = 1n; for (const r of smalls) Mtot *= r; let X = 0n; for (const [r, e] of residues){ const Mi = Mtot / r; let inv = 1n; for (let k = 1n; k < r; k++){ if ((Mi * k) % r === 1n){ inv = k; break; } } X = (X + e * Mi * inv) % Mtot; } console.log('reconstructed private exponent x = ' + X + ' (true value ' + x + ')');}

Keep this attack distinct from the one in the next section. This is small-subgroup confinement: one group, a low-order element, Lim-Lee 1997, on finite fields. The invalid-curve attack of Section 6 uses many curves and an off-curve point, Biehl-Meyer-Mueller 2000, on elliptic curves. They are routinely conflated and they are not the same [@twist].

The fix: take "which prime" away from the caller

The response was to stop letting callers choose primes. RFC 7919 defines a fixed menu of named FFDHE groups (ffdhe2048 through ffdhe8192), every one a safe prime, so a custom, shared, weak, or backdoored modulus is simply not expressible on the wire [@rfc7919]. That is condition #2 pulled inside a named group.

Why take the choice away so completely? Because a caller-chosen modulus can fail in a way no inspection reveals. In early 2016 the socat networking tool was found to have shipped a hard-coded 1024-bit Diffie-Hellman p parameter that was not prime at all, making a key exchange weaker than one over a genuine prime [@socat-secadv7]. The advisory (MSVR-1499) is worth quoting for its candor: because "there is no indication of how these parameters were chosen, the existence of a trapdoor ... cannot be ruled out" [@socat-secadv7].

The repair generated a fresh 2048-bit prime with OpenSSL's dhparam, and the flaw was reported by Santiago Zanella-Beguelin -- a Logjam co-author -- together with Microsoft Vulnerability Research [@socat-secadv7]. A bring-your-own modulus can be weak or backdoored, and you cannot tell which by looking at it. That single degree of freedom is exactly what named safe-prime groups take away.

Note: Do not generate or ship a custom finite-field DH prime, and never one below 2048 bits. A custom modulus can be weak or even backdoored and you cannot tell by looking -- in 2016 socat shipped a 1024-bit "prime" that was not prime at all. Use a named safe-prime group (ffdhe3072 or larger), or, better for most new work, move to a curve.

The number field sieve has an expensive precomputation stage whose cost "is dependent only on this prime," followed by a cheap per-target descent [@weakdh]. So the marginal cost of reading one more connection on a broken prime is tiny; the fixed cost is paid once. When a handful of 1024-bit primes cover a large fraction of the internet, that fixed cost buys mass surveillance: the single most common prime makes roughly 18% of top HTTPS sites readable, and a second prime roughly 66% of VPNs [@weakdh].

The Logjam paper put a concrete price on the 1024-bit precomputation, and it lands squarely in nation-state territory. Roughly 45 million core-years of one-time work is "a huge computational effort," the authors wrote, but "not necessarily out of reach for a nation state": about $8 million of custom ASIC hardware would buy enough capacity to finish the sieving stage in a year, and the linear-algebra hardware was estimated "plausibly on the order of hundreds of millions of dollars" -- on general-purpose machines, the Titan supercomputer's cores would need 117 years for that stage alone [@weakdh].

Set that against the roughly $10.5 billion the paper cites for the NSA's 2012 consolidated cryptologic-program budget, and the arithmetic is uncomfortable [@weakdh]. As a public, measured front-door datapoint for the same algorithm one size down, the RSA-250 factoring record fell to the number field sieve in February 2020 at a cost of about 2,700 core-years, so the 1024-bit discrete-log estimate extrapolates a demonstrated technique rather than a hypothetical one [@rsa250].

Two conditions leaked at once here: the group was neither strong-per-bit nor unshared, and a reused key turned the exchange into an oracle. The obvious escape is a group where the discrete log is harder per bit and there is no cheap shared-prime precomputation. That group is an elliptic curve -- and it introduced a sharper oracle of its own.

6. Generation 2: Elliptic-Curve DH, and the New Oracle

Move the same Diffie-Hellman protocol onto an elliptic curve and the number field sieve vanishes. A 256-bit key now buys what a 3072-bit prime bought. But the smaller, structure-free group sharpened one of the four conditions into a live, key-recovering oracle.

ECDH, mechanically

The curve internals belong to the elliptic-curves installment of this series; here we only need the shape. Replace "exponentiate $g$" with "scalar-multiply the base point $G$": a private key is a scalar $d$, the public key is the point $Q = dG$, and the shared secret is the point $d_A d_B G$, from which both sides take the x-coordinate [@sp80056a]. The payoff is in the attack cost. On a strong curve the best known attack is the generic Pollard rho algorithm, running in about $\sqrt{n}$ steps with no sub-exponential index calculus available, so a 256-bit curve delivers roughly 128-bit security -- concretely, rho against P-256 costs about $2^{127.8}$ operations [@rho]. Victor Miller and Neal Koblitz independently proposed elliptic-curve cryptography in the mid-1980s for exactly this reason [@miller1985], [@koblitz1987].

The new failure class

The smaller group came with a sharper edge. The peer's point is attacker-controlled, and the standard scalar-multiplication formulas "do not involve the constant coefficient $b$" of the curve equation [@twist]. So if an attacker submits a point that satisfies everything except lying on the intended curve, the victim's arithmetic processes it without complaint -- effectively doing the computation on a different, weaker curve whose group has small subgroups. Scalar-multiplying such a point by a reused private key turns the operation into a key-extraction oracle, one small residue at a time, combined with the Chinese Remainder Theorem across many curves. This is the invalid-curve attack of Biehl, Meyer, and Mueller (CRYPTO 2000) [@bmm2000], [@twist]. Jager, Schwenk, and Somorovsky made it practical against real TLS servers doing static or reused ECDH at ESORICS 2015, recovering the server's private key [@jss2015].

An attack that submits a point lying on a different, weaker curve than the intended one. Because scalar-multiplication formulas ignore the curve's constant term, the victim computes on the wrong curve and, if it reuses its private key, leaks that key one small residue at a time. flowchart TD A["Attacker sends a point not on the intended curve"] --> B["Scalar multiply ignores the curve constant b"] B --> C["Computation lands in a weak subgroup of another curve"] C --> D["Reused key leaks d modulo a small order"] D --> E["Repeat across many curves, then CRT to the full key d"]

This is the place readers most often blur two attacks, so once more, at the point of maximum confusion: Lim-Lee small-subgroup confinement (Section 5) is finite-field, one group, a low-order element; Biehl-Meyer-Mueller invalid-curve (here) is elliptic-curve, many curves, an off-curve point. And the practical TLS demonstration is Jager-Schwenk-Somorovsky's ESORICS 2015 paper, which is distinct from the same authors' separate USENIX 2015 work [@twist], [@jss2015].

The lesson, made mandatory

For classic cofactor ECDH the caller must validate every incoming point: it has to lie on the intended curve and in the correct prime-order subgroup. NIST SP 800-56A codifies this as full or partial public-key validation [@sp80056a]. It is an explicit, easy-to-forget check the smaller group pushed back onto the implementer -- exactly the caller-facing condition this article tracks.

Note: Reject any peer point that is not on the curve and not in the prime-order subgroup, every time. Miss the check once, on a server that reuses its key, and you have built a private-key-recovery oracle.

A related but separate ECC failure is worth one pointer so you do not file it under the wrong heading: Curveball (CVE-2020-0601) was a certificate-validation bug in Windows CryptoAPI, not an ECDH key-agreement break [@nvd-0601]. Same family of "validate the parameter you were handed," different layer.

So ECDH shrank the key but still demanded a hand-written on-curve check and constant-time code that an implementer could botch. The next move makes both of those unnecessary by construction -- a curve where every 32-byte string is a valid input and the arithmetic is constant-time by design.

7. Generation 3: X25519, or "Make the Simple Implementation the Safe One"

Everything classic ECDH asked the caller to remember -- validate the point, write the arithmetic in constant time -- one curve was designed to make impossible to get wrong. That is the whole point of X25519.

What X25519 is

X25519 is the Diffie-Hellman function over Curve25519, a Montgomery curve over the prime field $p = 2^{255} - 19$, computing x-coordinate-only DH with a uniform Montgomery ladder. Daniel Bernstein introduced it in 2006, and it was standardized as RFC 7748 in 2016 [@ecdh], [@rfc7748]. Two design decisions are the entire construction, and each one pulls a caller-facing condition inside the primitive.

A scalar-multiplication algorithm that processes the secret scalar in a fixed sequence of steps, performing the same field operations at every bit regardless of the secret's value. It computes only the x-coordinate of the result and has no secret-dependent branches.

Constant-time by construction

The ladder is a fixed sequence of roughly 255 iterations, each doing the same field operations no matter what the secret bits are: no secret-dependent branch, no secret-indexed table lookup [@rfc7748]. The timing side channel that a hand-written P-256 ladder can leak simply has no place to live. The primitive absorbed the constant-time requirement so the implementer cannot forget it.

Twist-security

The second decision is subtler and more beautiful. Curve25519 and its quadratic twist both have near-prime order, so an attacker who submits an off-curve x-coordinate does not land in a weak subgroup -- it lands in another strong group where there is nothing to confine a key into. The invalid-curve attack of Section 6 finds no purchase, and it needs no explicit on-curve check to stop it. That is why "the Curve25519 function was carefully designed to allow all 32-byte strings as Diffie-Hellman public keys," and why Bernstein's guidance on validating public keys is, bluntly, "Don't" [@ecdh], [@twist].

A property of a curve whose quadratic twist also has near-prime order. An attacker who supplies an off-curve x-coordinate is forced into another strong group instead of a weak subgroup, so invalid-curve attacks fail without any explicit point validation. flowchart TD In["Any 32-byte peer value, trusted or hostile"] --> L["Montgomery ladder: same steps for every secret bit"] L --> Tw["An off-curve input lands on the twist, another strong group"] Tw --> Out["No weak subgroup to confine a key: validation is structural"] L --> Ct["No secret-dependent branch or lookup: the timing channel is closed"]

Note: X25519 is not merely a faster curve. It takes two conditions that classic ECDH left on the implementer -- validate the input, run in constant time -- and makes them structural properties of the primitive. The safety is by construction, not by discipline. That is the shape of the entire history in miniature.

The two rules that trip people up

Two details cause most X25519 confusion.

First, clamping. Before use, the secret scalar is masked with k[0] &= 248; k[31] &= 127; k[31] |= 64 [@rfc7748]. Clamping constrains your own scalar so your side avoids cofactor and small-order pitfalls. It is emphatically not input validation, and it does nothing to a malicious peer value. X25519 is safe against hostile inputs because of twist security, not because of clamping.

The bit-masking applied to an X25519 secret scalar before use, which fixes certain high and low bits. It shapes your own scalar to sidestep cofactor issues, and it performs no check whatsoever on the peer's public value.

Second, the cofactor-8 all-zero check. Curve25519 has cofactor 8, so a low-order peer input can drive the shared output to the all-zero value. RFC 7748 says implementations "MAY check ... whether the resulting shared secret is the all-zero value and abort if so" -- a MAY, tied to the nuance of contributory behavior, not a MUST [@rfc7748]. Most protocols do not need it because they mix identities and transcripts into the KDF anyway.

Note: Clamping constrains your own scalar; it says nothing about the peer's value. If you find yourself reasoning that clamping makes an incoming point safe, stop -- the safety comes from twist security. Treat clamping and validation as different jobs.

A precision that catches even careful readers: X25519 and its larger sibling X448 are the Montgomery, x-coordinate-only variant of elliptic-curve DH, not the SP 800-56A cofactor ECDH of Section 6, even though "ECDH" is the umbrella term for both. X448 targets a more conservative security level near 224 bits and is defined in the same RFC 7748 [@rfc7748].

Even here the KDF discipline is stated in the primitive's own voice: "Both of you can then hash this shared secret and use the result as a key," never the raw x-coordinate [@ecdh].

X25519 makes the primitive misuse-resistant: validated by construction, constant-time by design. It says nothing about the other two conditions -- freshness and authentication -- which remain the protocol's to enforce. And the first of those, freshness, is the exact hinge on which the property everyone wants from Diffie-Hellman either holds or silently evaporates.

8. The Breakthrough: Ephemeral Exchange and (Imperfect) Forward Secrecy

This is the property that makes people say "use DHE, not static," and the one most often stated as if it were automatic. It is not. It is a conjunction, and the failure catalog is its list of unmet conjuncts.

Forward secrecy, stated conditionally

If the Diffie-Hellman secret is freshly generated per session, never reused, destroyed after use, and run over a strong unshared group, then later compromise of the long-term authentication key cannot recover past session keys -- because the ephemeral secret those keys were derived from no longer exists. The long-term key's only job is to authenticate the exchange; the session's confidentiality dies with the ephemeral. Every word of that conditional is load-bearing. Drop "never reused" and you get a static key in disguise; drop "strong unshared group" and Logjam reads you anyway.

The guarantee that compromising a long-term key does not expose past session keys. It holds only when the per-session ephemeral secret was freshly generated, never reused, destroyed after use, and run over a strong, unshared group. It is a conjunction of those conditions, not an automatic consequence of using DH. An ephemeral DH key is generated fresh for a single session and discarded immediately after. A static key is long-lived and reused across sessions, providing no forward secrecy. A cached or reused "ephemeral" is, for security purposes, a static key. flowchart TD E["Fresh ephemeral secret, used then destroyed"] --> H["Handshake, DH values signed by the long-term key"] LT["Long-term key authenticates only"] --> H H --> K["Session key derived from the ephemeral"] K --> D["Ephemeral destroyed when the session ends"] C["Attacker later steals the long-term key"] --> P["Past sessions stay secret: the ephemeral is gone"] ST["Static DH: one reused key does everything"] --> O["Its compromise opens every past session at once"]

Mode	Fresh per session?	Forward secrecy
Static DH	No -- one long-lived key	None; a single key compromise opens all past sessions
Reused or cached "ephemeral"	No -- shared across sessions	Collapses for every session that shares it; also enables Raccoon
True ephemeral	Yes -- destroyed after use	Yes, conditional on a strong unshared group

The honest other half: how the property fails

The conditional fails in three documented ways, and each is a row in the catalog.

Ephemeral-key reuse or caching. Sold as a performance optimization, reusing an ephemeral turns it into a static key and collapses forward secrecy for every session that shares it. Worse, reuse is the precondition for Raccoon (CVE-2020-1968): in TLS 1.2 and earlier, the premaster secret had its leading zero bytes stripped before the key derivation step, so the exact timing of the KDF leaked the most significant bits of the shared secret, recoverable as an instance of the Hidden Number Problem -- but only when the server reused its DH key long enough to gather timing samples [@raccoon].

Shared-prime precomputation. This is Logjam again, from the other direction: even a genuinely fresh per-connection ephemeral does not save you if the prime is shared and precomputed [@weakdh]. It is the reason the paper is titled Imperfect Forward Secrecy.

Static DH. No ephemeral at all, so no forward secrecy at all -- the baseline failure that TLS 1.3 later abolished [@rfc8446].

Note: Caching or reusing a DH "ephemeral" collapses forward secrecy for every session that shares it, and reuse is exactly the condition Raccoon needs to mount its timing oracle. If it is not fresh and destroyed, it is not ephemeral.

The Raccoon authors are blunt about the root condition: reusing a DH key "was already considered bad practice before the attack, as it does not provide forward secrecy" [@raccoon]. The timing oracle was a second penalty on top of the forward-secrecy loss.

The discipline beneath it all

Why must the raw shared value always be hashed? Because a Diffie-Hellman value "is NOT a uniformly random or pseudorandom string" -- it is a structured group element with detectable bias -- so it must be run through an extract-and-expand KDF before it becomes key material [@rfc5869]. This is the formal echo of the CDH-versus-DDH gap from Section 2, now stated as a standards requirement.

A Diffie-Hellman value "is NOT a uniformly random or pseudorandom string." -- RFC 5869, Section 3.3 It is easy to over-claim what forward secrecy buys. It bounds the reach of a *future* compromise: an attacker who steals your long-term key tomorrow cannot decrypt the sessions you completed and destroyed today.

It does nothing about the present. An attacker who owns your endpoint during a live session, or who grabs the ephemeral before you delete it, reads that session in full. Nor does it provide post-compromise security -- the guarantee that sessions after a compromise heal once the attacker loses access. That healing is a different property, achieved by ratcheting constructions such as the one in the Signal protocol, and it is out of scope for a single DH exchange. Forward secrecy is a statement about deleted secrets, and only about the past.

Freshness, a strong group, validated inputs, authentication, and a KDF -- five things, and the caller was trusted to remember all of them. The last move in this story stops trusting the caller: it bakes the conditions into named objects and a protocol that refuses to run without them.

9. State of the Art (2024-2026): X25519 Everywhere, and the Quantum Clock

Here is what a practitioner actually deploys today -- and the single overlay that reframes all of it: every discrete-log DH in this article has an expiration date, and a well-funded adversary can start the clock by simply recording your traffic now.

The deployed reality

X25519 is the dominant key agreement across TLS 1.3, OpenSSH, WireGuard, and Signal, and the word "dominant" now rests on measured deployment rather than defaults alone. Cloudflare, which terminates a large fraction of the world's HTTPS and can therefore see it, reports that X25519 "now secures the vast majority of all Internet connections" [@cloudflare-pq]; OpenSSH has shipped Curve25519 key exchange as "the default when both the client and server support it" since version 6.5, back in 2014 [@openssh65], [@ecdh]; and WireGuard and Signal are built on the same primitive [@wireguard], [@pqxdh]. Finite-field DH survives mainly as deprecated-but-present legacy in older TLS and IPsec [@weakdh], [@rfc8446].

TLS 1.3 turned this whole article's argument into a requirement: RFC 8446 removed static RSA and static Diffie-Hellman key transport, requires ephemeral (EC)DHE, and restricts negotiation to named groups. The specification states the result directly: "Static RSA and Diffie-Hellman cipher suites have been removed; all public-key based key exchange mechanisms now provide forward secrecy" [@rfc8446]. That is conditions #1, #2, and #4 -- freshness, a vetted group, and authentication -- pulled inside the protocol as non-optional. The fifty-year move is complete: the four conditions now live in the primitive, the named group, and the protocol, not in the caller's memory.

The quantum overlay

There is one problem no key size solves. All discrete-log Diffie-Hellman, finite-field and elliptic-curve alike, falls in polynomial time to Shor's algorithm on a sufficiently large quantum computer, and "harvest-now, decrypt-later" means an adversary can record your ECDH traffic today and decrypt it once such a machine exists [@rfc9794]. So the field is wrapping ECDH rather than replacing it.

An attack strategy in which an adversary records encrypted traffic today and stores it, intending to decrypt it later once a cryptographically relevant quantum computer exists. It makes today's ECDH confidentiality a future liability.

The wrapper is X25519MLKEM768: it runs an X25519 exchange and an ML-KEM-768 key encapsulation in parallel and feeds both shared secrets into the KDF, so the session stays secure as long as either problem remains hard [@ecdhe-mlkem], [@fips203]. Concretely, the client key share is 1216 bytes (1184 for ML-KEM plus 32 for X25519), the server share is 1120 bytes, the negotiated combined secret is 64 bytes, and the TLS codepoint is 0x11EC [@ecdhe-mlkem].

A public-key primitive in which a sender encapsulates a fresh random key to a recipient's public key, producing a ciphertext the recipient decapsulates. Unlike Diffie-Hellman, a KEM is inherently interactive and has no non-interactive agreement between two static public keys.

The transition was fast. OpenSSH 9.9, released on 2024-09-19, enabled mlkem768x25519-sha256 by default [@openssh99]. OpenSSL 3.5, a long-term-support release from April 2025, ships a DEFAULT TLS group list that "selects X25519MLKEM768 as one of the predicted keyshares" [@openssl35-release], [@openssl-groups]. Chrome and Cloudflare deployed post-quantum key agreement across their edges [@cloudflare-pq].

The classical primitives do not vanish. FFDHE and P-256 persist wherever FIPS validation or vendor interop compels them, and FIPS-flavored hybrids such as `SecP256r1MLKEM768` exist for exactly that constraint [@sp80056a]. There is also a plumbing wrinkle: the roughly 1.2 KB hybrid ClientHello may no longer fit in a single TCP segment, which OpenSSL's documentation calls out along with guidance on ordering group tuples to avoid a silent downgrade [@openssl-groups].

Two precisions that keep the story honest

First, a KEM is not a drop-in Diffie-Hellman. It has no non-interactive static-static agreement, so X25519MLKEM768 is a hybrid KEM handshake, not "post-quantum Diffie-Hellman." Second, in messaging, Signal's PQXDH (2023) is the analog: it adds an ML-KEM encapsulation to the older X3DH design but "still relies on the hardness of the discrete log problem for mutual authentication in this revision," so X25519 is retained, not retired [@pqxdh], [@x3dh]. Signal described it as "an upgrade to the X3DH specification which we are calling PQXDH" [@pqxdh-blog].

State the adoption figure carefully. Cloudflare measured post-quantum key agreement at nearly 2% of TLS 1.3 connections in early 2024, with double-digit adoption projected by year-end rather than measured -- so "roughly 2% in early 2024, rising" is the defensible phrasing without a newer source [@cloudflare-pq].

X25519 is the default, TLS 1.3 made the discipline mandatory, and the hybrid buys time against Shor. So which primitive, with which parameters, in which case? That is a decision matrix -- and it is not a single winner.

10. Competing Approaches: The Decision Matrix and the AKE Families

There is no single best key-agreement primitive. The state of the art is a small set of deployed constructions, each dominant in a niche, wrapped by a small set of authentication families. The honest answer is "it depends."

The primitive decision matrix

Primitive	Best classical attack	Classical security	Post-quantum	Public value size	Peer-input validation	Best suited for
FFDHE-3072	Sub-exponential NFS	about 128-bit	No	about 384 bytes	Subgroup check on a safe prime	Legacy FIPS or interop only
ECDH P-256	Generic rho, about $2^{127.8}$	about 128-bit	No	65 bytes	Hand-written on-curve and subgroup MUST	FIPS, HSM, existing deployments
X25519	Generic rho, about $2^{125.8}$	about 128-bit	No	32 bytes	Structural, via twist security	New classical designs
X25519MLKEM768	Generic rho plus lattice attacks on ML-KEM	about 128-bit and post-quantum	Yes, if either half holds	about 1.2 KB (client share)	X25519 structural plus KEM ciphertext check	The post-quantum default going forward

Two rows carry the weight. "Best classical attack" separates FFDHE's sub-exponential tax from the curves' generic-only attack, which is why a curve needs a quarter of the bits [@rho], [@sp80057]. "Peer-input validation" separates a hand-written MUST for P-256 and a subgroup check for FFDHE from the structurally-unnecessary case of X25519 [@sp80056a], [@rfc7748]. The head-to-head rules fall out cleanly: X25519 for new designs; ECDHE with P-256 or FFDHE at 3072 bits and up only where FIPS or interop compels it; and ephemeral, always, for forward secrecy.

The authentication families

A primitive is only half a deployment; something must authenticate the exchange. Four families cover essentially all real usage.

Family	How DH is authenticated	Deployed in	Trade-off
Signed-DH	Each side signs the transcript or the DH values	TLS 1.3, IKEv2, STS and SIGMA	Simple and well understood; needs a PKI or pinned keys
Implicitly-authenticated DH	Long-term and ephemeral keys combine inside the shared-secret formula	MQV and HMQV, parts of IKE	Compact, no signatures; subtle to get exactly right
DH-pattern frameworks	Compose several static and ephemeral DH operations	Noise, WireGuard's Noise_IK	Mutual auth plus identity hiding plus KCI resistance
Asynchronous AKE	Prekeys let a recipient be offline	Signal's X3DH and PQXDH	Deniable and offline-friendly; more moving parts

Signed-DH dominates the open web through TLS 1.3, whose handshake is SIGMA in spirit [@rfc8446], [@sigma2003]. WireGuard composes several X25519 operations in the Noise_IK pattern to get mutual authentication, forward secrecy, key-compromise-impersonation resistance, and identity hiding at once [@wireguard]. Signal's X3DH and its post-quantum successor PQXDH deliver asynchronous, deniable key agreement to an offline recipient [@x3dh], [@pqxdh].

The composition hazard

One more lesson hides in those families, and it is easy to miss: sub-protocols each proven secure in isolation can still compose into an insecure whole. The Triple Handshake attack (Bhargavan, Delignat-Lavaud, Fournet, Pironti, and Strub, IEEE Symposium on Security and Privacy 2014) chained a TLS key exchange with session resumption and renegotiation so that two distinct connections could be driven to share the same master secret, which broke client authentication and defeated the countermeasures of the day [@triple-handshake].

Paired with key-compromise impersonation -- where stealing your long-term key lets an attacker impersonate others to you -- it is the standing reminder that a secure handshake is not simply the sum of secure steps. It is also the concrete reason TLS 1.3 binds the entire transcript: the Finished MAC covers the whole handshake, so two runs that differ anywhere cannot be coerced into agreeing [@rfc8446].

The Triple Handshake attack (2014) and key-compromise impersonation are adjacent AKE-composition pitfalls, not breaks of the signed-DH template itself. They are why TLS 1.3 authenticates the full handshake transcript rather than individual messages [@triple-handshake].

Signposts, not sections

Several approaches deserve a pointer rather than a treatment. MQV and HMQV are the implicitly-authenticated DH family, with narrow deployment in some IKE and Suite B profiles [@sp80056a]. DH-based PAKEs and OPRFs -- CPace, OPAQUE, SPAKE2 -- use Diffie-Hellman for password-authenticated and oblivious key agreement and are the subject of the hash-to-curve and PAKE installment of this series. CSIDH is the isogeny-based candidate for a genuinely post-quantum Diffie-Hellman analog, and its cautionary neighbor SIDH/SIKE was broken outright in 2022 [@sike-break]. ElGamal is DH's public-key encryption cousin, not a key-agreement scheme. And X448 is the conservative higher-security sibling of X25519. Each is one sentence here by design.

Every row of those tables assumes the underlying hardness holds. That assumption has a precise shape on each track -- and one hard ceiling above both.

11. Theoretical Limits: Where the Hardness Lives, and Its Ceiling

Here is what a first course skips: the two Diffie-Hellman tracks do not have the same kind of hardness, the raw shared secret is provably not a good key, and every scheme in this article dies to the same quantum algorithm.

Finite-field DLP is sub-exponential

The general number field sieve solves finite-field discrete logs in $L_p[1/3,\ (64/9)^{1/3}]$, and its first step depends only on the prime -- the engine behind Logjam. Because the attack is sub-exponential rather than exponential, key sizes must grow quickly to stay ahead: 3072 bits for 128-bit security, 7680 for 192-bit [@weakdh], [@sp80057]. The Logjam paper's own field estimates trace that curve concretely: a 512-bit prime falls in near-real time after a one-week, 89,000-core-hour precomputation (a median of 70 seconds per subsequent log); a 768-bit prime is an academic reach at about 36,500 core-years; and a shared 1024-bit prime is a standing nation-state liability at about 45 million core-years of one-time precomputation [@weakdh]. As an independent measured anchor one size down, the RSA-250 factoring record fell to the number field sieve in February 2020 at about 2,700 core-years [@rsa250].

Elliptic-curve DLP is generic square-root, and no better

For a strong curve the best known attack is generic Pollard rho at about $0.886\sqrt{\ell}$ operations, and -- this is the deep part -- that matches Shoup's generic lower bound of $\Omega(\sqrt{\ell})$ [@shoup1997]. For a well-chosen curve the gap between the best attack and the proven floor is closed: Curve25519 sits at about $2^{125.8}$ and P-256 at about $2^{127.8}$ [@rho]. The only escapes are specific bad families -- pairing (MOV and Frey-Ruck) reductions and anomalous curves -- which is why rigidity and vetted parameters matter, and which the elliptic-curves installment of this series treats in full.

Target security	Finite-field DH prime	Elliptic curve (rho cost)	Best classical attack
128-bit	3072-bit	Curve25519 about $2^{125.8}$, P-256 about $2^{127.8}$	Sub-exponential NFS vs generic rho
192-bit	7680-bit	P-384 about $2^{191.8}$	Sub-exponential NFS vs generic rho

Rho also amortizes across targets: breaking $T$ keys at once costs only about $\sqrt{T}$ times a single break, the elliptic-curve echo of Logjam's shared-prime amortization, and another reason to prefer a generous subgroup order [@rho].

The CDH-DDH gap is why the KDF is a theorem

The assumptions form a ladder, but mind the direction: an algorithm that solves the left solves the right, so DLP is at least as hard as CDH, which is at least as hard as DDH (read instead as hardness assumptions, the implication runs the other way, with DDH-hardness implying CDH-hardness implying DLP-hardness). That last gap is real and not merely conservative. On pairing-friendly curves, DDH is easy while CDH is plausibly still hard, so the raw $g^{ab}$ can be distinguished from uniform even when it cannot be computed by an attacker [@cryptobook]. That is precisely why RFC 5869 forbids skipping the extract step and insists the shared value be run through a KDF before use [@rfc5869]. Condition #5 is not a style preference; it is a consequence of a proven gap between two assumptions.

Forward secrecy's precise limits

Stated formally, forward secrecy protects the past, not the present or future, and only under the conjunction from Section 8: the ephemeral must have been ephemeral and the group strong. Its unmet conjuncts are, exactly, the failure catalog.

The ceiling: Shor

Above both tracks sits one algorithm that does not care which group you chose. Shor's algorithm solves discrete logarithms -- finite-field and elliptic-curve -- in polynomial time on a sufficiently large quantum computer, and no key size escapes it [@rfc9794]. This is the one break in the entire article that is a break of the math rather than a leaked condition.

Key idea: Every other failure in this article was a leaked condition -- a shared prime, an unvalidated point, a reused ephemeral, a missing signature. Shor is the only one that breaks the math itself. That is why the answer to Shor is a new problem, ML-KEM wrapped around X25519, and not simply a bigger curve.

If the theory is this settled -- a closed generic gap on curves, a sub-exponential tax on fields, a proven quantum cliff above both -- then what is left is genuinely open. And what is open is not the math. It is the interface, again.

12. Open Problems: What Remains Genuinely Unsettled

The discrete log is not the frontier. The frontier is what "post-quantum Diffie-Hellman" even means, whether constant-time can ever be a settled property, and how much broken cryptography is still in the field.

A practical post-quantum NIKE

Diffie-Hellman is a non-interactive key exchange: two parties who have each merely published a static public key can agree on a shared secret with no further interaction, and that single property is what makes asynchronous and static-static designs -- X3DH, Noise's static-static patterns -- possible. A KEM cannot do this. It is inherently interactive: one side encapsulates to the other's key and returns a ciphertext, so there is no agreement between two published static keys [@ecdhe-mlkem].

A scheme in which two parties agree on a shared secret purely from each other's published static public keys, with no live message exchange. Diffie-Hellman is a NIKE; a KEM is not, which is why KEMs cannot directly replace DH in static-static and asynchronous designs.

The only serious candidate for a genuinely post-quantum NIKE is the isogeny scheme CSIDH (Castryck, Lange, Martindale, Panny, and Renes, ASIACRYPT 2018), a commutative supersingular-isogeny group action that offers Diffie-Hellman's exact static-static, non-interactive interface. Its appeal is size: the paper advertises "public keys of only 64 bytes at a conjectured AES-128 security level, matching NIST's post-quantum security category I" [@csidh]. But that quantum-security claim is contested. Bonnetain and Schrottenloher analyzed the underlying quantum abelian-hidden-shift attack and concluded that "the parameters proposed by the authors of CSIDH do not meet their expected quantum security" (EUROCRYPT 2020), a finding Peikert reached independently the same year [@bs2020], [@peikert2020].

The consensus repair is to grow the parameters: the SQALE authors set out to "optimize large CSIDH parameters ... while still achieving the NIST security levels 1, 2, and 3," which pushes the prime to 1024 bits and beyond, with correspondingly slower group actions [@sqale]. And CSIDH's close relative SIDH/SIKE was broken outright in 2022, when Castryck and Decru recovered keys against SIKEp434 "in about ten minutes on a single core" [@sike-break]. That is a reminder that young isogeny assumptions can fall catastrophically, and it is why today's hybrids sidestep the missing PQ-NIKE rather than solve it [@pqxdh].

Key idea: "Post-quantum Diffie-Hellman" is not a solved problem. A KEM is not a drop-in for DH's non-interactive interface, and the closest true analog -- 64-byte CSIDH -- buys much less quantum security than its key size advertises, so restoring the intended level pushes it to far larger, far slower parameters. The hybrid buys time; it does not resurrect the NIKE.

Deniability versus post-quantum authentication

Diffie-Hellman gives messaging protocols a property signatures cannot: deniability. Because a session secret in X3DH is derived from DH values rather than a signature, neither party can later prove to a third party who they talked to -- the transcript could have been produced by either side alone [@x3dh]. That is a deliberate feature, and it is in tension with the post-quantum migration. Signal's PQXDH ships post-quantum confidentiality first, but in its own words it "provides post-quantum forward secrecy ... but still relies on the hardness of the discrete log problem for mutual authentication in this revision" [@pqxdh]. The reason it keeps a DH operation for the authentication step is precisely the open problem: the obvious post-quantum fix -- a post-quantum signature -- is inherently non-deniable, and a deniable post-quantum authentication scheme is not yet a settled, deployed design.

This is why PQXDH retains a discrete-log DH operation for mutual authentication even after adding ML-KEM for confidentiality: post-quantum confidentiality has shipped ahead of post-quantum authentication, and the deniability that DH provides has no drop-in post-quantum replacement yet [@pqxdh].

Constant-time as a moving target

X25519's ladder removes the timing channel at the primitive level, but Raccoon showed the same class of leak can live one layer up, in the protocol, when a premaster secret has its leading zeros stripped before hashing -- and it recurs specifically when a DH key is reused [@raccoon]. Formally-verified constant-time implementations are the structural answer, and they are not yet universal [@rfc7748].

The legacy long tail

This is a deployment problem, not a math problem. 1024-bit FFDHE, shared and custom primes, static DH, and DHE on TLS 1.2 all persist in the field years after the fixes shipped, and Logjam's precomputation economics make those shared 1024-bit primes a standing mass-decryption risk [@weakdh]. RFC 7919's named groups, the TLS 1.3 ephemeral mandate, and browsers dropping DHE have shrunk the surface, but the migration is incomplete [@rfc7919], [@rfc8446].

Hybrid-combiner and downgrade robustness

The entire value of X25519MLKEM768 is "secure if either half holds," and that guarantee is only as good as the combiner and the negotiation around it: a flawed combiner, or an attacker who forces a downgrade to the weaker half, voids it. RFC 9794 standardizes the post-quantum/traditional vocabulary and OpenSSL's group-tuple syntax guards the ordering, but end-to-end downgrade-resistance proofs for the deployed handshakes are still being worked out [@ecdhe-mlkem], [@rfc9794], [@openssl-groups].

Flagged as reasoning, not documented fact: by analogy to every generation before it, the next wave of post-quantum breaks will most likely again be interface leaks -- bad randomness, unvalidated ciphertexts, non-constant-time decapsulation -- rather than breaks of the lattice problem itself. No source documents a failure that has not yet happened; this is an inference from the pattern of the previous five generations.

If the open problems are mostly about the interface, then so is the guidance. Here is fifty years of failure compressed into rules you can apply in a design review on Monday.

13. The Practical Guide, and the One Sentence Underneath

Every rule below is one row of the failure catalog, inverted. If you remember nothing else, remember the diagnostic question, and the four conditions it audits.

Decision rules: use X, with parameters Y, in case Z

New key agreement, general purpose: X25519, ephemeral [@rfc7748], [@ecdh].
Post-quantum-hardened (the recommended default going forward): X25519MLKEM768 [@ecdhe-mlkem], [@openssl35-release], [@openssh99].
FIPS, regulated, or HSM-interop: ECDHE with P-256 or P-384, ephemeral, with full public-key validation; or the FIPS hybrid SecP256r1MLKEM768 [@sp80056a], [@ecdhe-mlkem].
Legacy finite-field interop only: FFDHE ffdhe3072 or larger, ephemeral, with the subgroup check -- never a custom prime and never one below 2048 bits [@rfc7919], [@weakdh].
Always, regardless of primitive: ephemeral and destroyed after use; validate the peer value appropriately (clamping is not validation); run the shared secret through HKDF; authenticate the exchange; never static DH; never reuse the ephemeral [@rfc5869], [@rfc8446].

Situation	Use	Key parameters	Because (named break avoided)
New design, general	X25519, ephemeral	32-byte keys, HKDF the secret	Validation and timing are structural
PQ-hardened default	X25519MLKEM768	Hybrid, both secrets into the KDF	Shor and harvest-now-decrypt-later
FIPS / HSM / interop	ECDHE P-256 or P-384, ephemeral	Full public-key validation	Invalid-curve attacks
Legacy FF interop only	FFDHE `ffdhe3072`+, ephemeral	Subgroup check, never below 2048-bit	Logjam shared-prime precomputation
Always	Ephemeral, validate, KDF, authenticate	Destroy the ephemeral after use	Reuse, man-in-the-middle, CDH-DDH gap

Note: X25519 for greenfield. X25519MLKEM768 when you need post-quantum hardening. ECDHE-P256 or FFDHE-3072-plus only where FIPS or interop forces your hand. And always: ephemeral, validated, KDF'd, and authenticated.

For TLS, run `openssl s_client -connect example.com:443 -tls1_3` and read the "Negotiated TLS1.3 group" line in the output; it names the group actually chosen, such as `X25519` or `X25519MLKEM768`. To offer only the hybrid and confirm the server supports it, add `-groups X25519MLKEM768` (OpenSSL 3.5 or later). For SSH, `ssh -Q kex` lists the key-exchange methods your client supports, and connecting with `ssh -vv` prints the method actually negotiated, for example `mlkem768x25519-sha256`. flowchart TD A{"New greenfield design?"} -- no --> F{"FIPS or HSM interop required?"} A -- yes --> Q{"Post-quantum sensitive?"} Q -- yes --> H["X25519MLKEM768, ephemeral"] Q -- no --> X["X25519, ephemeral"] F -- yes --> P["ECDHE P-256 or P-384, ephemeral, full validation"] F -- no --> L["FFDHE ffdhe3072 or larger, ephemeral, subgroup check"]

The misuse catalog

Each common mistake maps to exactly one named break. Static DH gives no forward secrecy. A reused or cached "ephemeral" collapses forward secrecy and enables Raccoon. A custom, shared, or sub-2048-bit prime invites Logjam, and a non-prime "prime" you cannot audit invites the socat failure. Skipping the on-curve and subgroup check for P-256 ECDH builds an invalid-curve oracle. Treating clamping as validation misreads why X25519 is safe. Using the raw x-coordinate or $g^{ab}$ as a key ignores the CDH-DDH gap. And anonymous DH with no authentication is a standing man-in-the-middle.

The failure catalog

This is the thesis as evidence: every world-scale Diffie-Hellman break, in one grid, each a violated condition with the math intact underneath. The rows were derived across earlier sections -- Logjam [@weakdh] and its RSA twin FREAK [@freak], small-subgroup confinement [@limlee1997] and CVE-2016-0701 [@openssl-cve], the non-prime socat modulus [@socat-secadv7], the invalid-curve class [@bmm2000] and its practical TLS demonstration [@jss2015], Raccoon [@raccoon], and the KDF requirement [@rfc5869].

Break	Year	Condition violated	Root cause	Foreclosed by
Anonymous-DH MITM	design gap	#4 Authentication	DH values bound to no identity	STS/SIGMA; TLS 1.3 signatures
Logjam shared-prime	2015	#2 Strong, unshared group	NFS first step precomputed per shared prime, about 45M core-years at 1024-bit	RFC 7919 named safe primes
Logjam 512-bit export	2015	#2 Strong group (active)	Downgrade to breakable export parameters	TLS 1.3 removes export and static DH
FREAK (Logjam's RSA twin)	2015	#2 Strong group (active)	Downgrade to 512-bit export RSA key transport, then factor it	TLS 1.3 removes export and static RSA
Small-subgroup (Lim-Lee); CVE-2016-0701	1997; 2016	#3 Validation and #1 reuse	Low-order element vs a reused exponent, no subgroup check	Safe primes plus the subgroup check
Non-prime modulus (socat)	2016	#2 Strong, unshared group	A shipped custom 1024-bit "prime" that was composite, possibly backdoored	RFC 7919 named safe primes; never bring your own
Invalid-curve (BMM); JSS TLS-ECDH	2000; 2015	#3 Validation	Off-curve point, formulas ignore b, reused key	Mandatory on-curve and subgroup check; X25519 twist
Raccoon	2020	#1 Freshness (reuse)	Leading-zero stripping plus a reused DH key gives an MSB timing oracle	No key reuse; TLS 1.3 key schedule
Ephemeral reuse or caching	ongoing	#1 Freshness	A cached "ephemeral" is a static key; forward secrecy collapses	Fresh-per-session and destroyed; TLS 1.3
Static DH	ongoing	#1 Freshness	No ephemeral, so no forward secrecy	TLS 1.3 ephemeral mandate
Raw x-coordinate as key	ongoing	#5 KDF discipline	CDH-DDH gap leaves the shared value biased	HKDF extract, per RFC 5869

No, and this is the field's most-corrected misconception. Forward secrecy is a *conjunction*: the ephemeral must be freshly generated per session, never reused, destroyed after use, and run over a strong unshared group. Reuse a performance-optimized ephemeral and you have a static key; run over a shared 1024-bit prime and a precomputation reads you anyway, which is why the foundational paper is titled *Imperfect Forward Secrecy*. "Ephemeral, therefore forward-secret" is false without those conditions. No. Clamping constrains *your own* secret scalar so your side of X25519 avoids cofactor and small-order pitfalls; it does nothing to check or reject a malicious peer value. X25519 is safe against off-curve inputs because of *twist security* -- the curve and its twist are both hard -- not because of clamping. That distinction trips up a lot of otherwise-correct code. No. A Diffie-Hellman value is a structured group element, not a uniformly random string, and there are groups where it is even distinguishable from random -- the CDH-versus-DDH gap. Always run it through a KDF such as HKDF. Skipping the extract step is exactly what RFC 5869 warns against. No. Fresh per-connection keys do not help against Logjam's central result: the number-field-sieve's expensive step depends only on the *prime*, so a shared or common 1024-bit prime can be precomputed once and then used to passively decrypt every connection on it. Use ffdhe3072 or larger, or move to a curve. Below 2048 bits is indefensible. X25519 for anything greenfield: 32-byte keys, a single constant-time Montgomery ladder, and twist security that makes the two hardest ECDH mistakes -- timing leaks and missing point validation -- structurally unreachable. Use P-256 where FIPS, an HSM, or an existing deployment compels it, and then with full public-key validation and a constant-time library. No. ML-KEM is a key encapsulation mechanism, which is inherently interactive and has no non-interactive static-static agreement, so it cannot express the asynchronous and static-static designs DH enables. That is why the post-quantum state of the art is a *hybrid* -- X25519MLKEM768, with X25519's secret combined with ML-KEM's -- rather than a like-for-like swap. The closest true post-quantum analog of Diffie-Hellman, the isogeny scheme CSIDH, does keep the non-interactive interface with 64-byte keys, but its quantum security is contested and reaching the intended level forces much larger, slower parameters. No. Authentication defeats the *active* man-in-the-middle and nothing else -- it is orthogonal to the group's hardness and to input validation. A signed handshake over a shared 1024-bit prime is still Logjam-exposed, and a signed exchange that skips the on-curve check is still an invalid-curve oracle. Authentication is one of four conditions, not a substitute for the others.

The one sentence underneath

Return to Logjam. Across fifty years, almost no deployed break ever solved a discrete logarithm. Attackers precomputed a shared prime, confined a reused key to a small subgroup, fed an off-curve point to a validation-free server, read the clock while a server stripped leading zeros, shipped a modulus that was not even prime, or simply sat in the middle of an unauthenticated exchange. "Using Diffie-Hellman safely" is not one choice but a stack: a strong unshared group, a validated input, a genuinely fresh-and-destroyed ephemeral, an authenticated exchange, and a KDF beneath them. The fifty-year story is the field pulling each of those off the caller's shoulders and into the primitive, the named group, and the protocol, so that the simple thing to do became the safe thing to do. Remove any layer and the matching failure class reopens.

The math held; the deployment handed the secret away.

There is one exception, and it is the horizon. Shor's algorithm is the single break in this whole story that touches the math itself, which is why today's ECDH is being wrapped, not replaced, and why the destination is a hybrid rather than a bigger prime. So audit the exchange the way the whole article has: when a public value you did not generate arrives, and you combine it with a secret you must have freshly generated and will immediately destroy -- is the group strong and unshared, is that incoming value validated, is the exchange authenticated, and does the raw result go through a KDF before anyone calls it a key? Using Diffie-Hellman safely is the discipline of never handing an attacker a condition your protocol forgot to own.

The Math Held; The Interface Leaked: A Field Guide to Digital Signatures

noreply@paragmali.com (Parag Mali) — Sun, 12 Jul 2026 17:41:17 GMT

**A signature scheme's hardness assumption almost never breaks in the field.** The elliptic-curve discrete logarithm and RSA have held for decades. What breaks instead are the three things the scheme hands the caller: the per-signature **nonce**, the signature's **canonical form**, and the **verifier's acceptance rule**.

ECDSA outsources all three, and the same untouched-math scheme produced four categorically different disasters: a reused nonce that gave up the PlayStation 3 master key from two signatures [@ps3-27c3-2010], a biased nonce that recovered a PuTTY key in about sixty (a statistical attack: one few-bit-biased signature does not leak your key) [@cve-2024-31497], a malleable (r, n-s) that let anyone alter a Bitcoin transaction id [@bip-340], and a verifier that accepted (0, 0) for any message ("Psychic Signatures") [@cve-2022-21449].

The history since ECDSA is one sustained move to pull each surface back inside the scheme: RFC 6979 derives the nonce deterministically, hedged signing mixes a little randomness back in to survive fault attacks, and Ed25519 and BIP-340 Schnorr make deterministic nonces, a fixed encoding, and a byte-exact verifier native by design -- the clean scheme that sat patent-frozen from 1989 until 2008.

RSA-PSS answers the same "form" question on the RSA track with the tightest proof of any deployed signature. In 2026 these coexist by niche, and every one of them falls to Shor. Signing safely is a stack: the right scheme, a nonce you need not trust the RNG for, a canonical form, and a strict verifier. The math held; the interface leaked.

1. Two Signatures, One Nonce, and the Master Key Fell Out

In December 2010, at the 27th Chaos Communication Congress, a group called fail0verflow put a single slide on the screen that ended the security of the PlayStation 3 [@ps3-27c3-2010]. Sony had signed every piece of PS3 firmware with ECDSA, a scheme whose security rests on a discrete logarithm no one has ever solved for a 256-bit curve.

Sony's engineers did not need to fail. They reused the same secret random number -- the per-signature nonce -- for every signature, and any two signatures that share a nonce give up the private key by grade-school algebra. The curve was pristine. The master key fell out of two subtractions.

Twelve years later, the failure jumped one layer up the stack. Java shipped an ECDSA verifier that accepted the signature (r, s) = (0, 0) for any message and any public key -- "Psychic Signatures," CVE-2022-21449, a CVSS 7.5 flaw that forged TLS handshakes, signed JWTs, SAML assertions, and WebAuthn responses alike [@cve-2022-21449], [@neil-madden-2022]. One break leaked a private key through the nonce; the other waved a forgery through at the verifier. Neither touched the underlying hardness assumption. Nobody solved a discrete log. Nobody factored a modulus.

That is the pattern this article is about, worth stating as a question you can carry through every section: when a signer or verifier meets a value it did not create, what secret does it leak, or what forgery does it wave through? The answer is almost never "the math was weak." A signature scheme is only as safe as three things it hands the caller and refuses to decide for you: the per-signature nonce, the signature's canonical form, and the verifier's acceptance rule. ECDSA outsources all three at once.

That is why one scheme -- its elliptic-curve discrete logarithm fully intact -- spawned four categorically different disasters: the reused nonce of the PS3, the drained Android Bitcoin wallets, the malleable transactions that unsettled Bitcoin, and the psychic verifier in Java. You half-know these headlines already. This is the one sentence underneath all of them.

The math held; the interface leaked.

The rest of signature history since ECDSA is one effort to pull those three surfaces back inside the scheme, so the caller can no longer get them wrong. To see why a shared random number hands over a private key -- and why "just use Ed25519" is a real answer but not the whole one -- you first have to be precise about what a signature promises, and the three jobs ECDSA quietly delegates to you.

2. What a Signature Actually Is (and the Three Things ECDSA Hands You)

Everyone knows the one-liner: sign with the private key, verify with the public key. That one-liner hides three decisions the scheme refuses to make for you. Start with the part that is simple.

A digital signature scheme is three algorithms. KeyGen -> (pk, sk) produces a public verification key and a private signing key. Sign(sk, m) -> sigma turns a message and the private key into a signature. Verify(pk, m, sigma) -> {accept, reject} lets anyone holding the public key check the pair. The word anyone is the whole point.

A message authentication code also proves a message is authentic, but it verifies with the same secret used to produce it, so only insiders can check it. A signature is publicly verifiable and therefore non-repudiable: the entire world can confirm it, and only the holder of sk could have produced it.

A triple of algorithms: KeyGen produces a public key and a private key; Sign maps a private key and a message to a signature; Verify maps a public key, a message, and a signature to accept or reject. Unlike a message authentication code, it is publicly verifiable and non-repudiable, since verification needs only the public key.

Public verifiability is what makes a signature strictly stronger than a MAC, and strictly slower. A MAC can rest on a single shared secret and a fast keyed hash. A signature cannot rest on any shared secret, because the verifier must be the whole world, so it has to be built on a public-key hardness assumption -- and pays for it in both compute and signature size.

The bar every scheme is measured against

"Secure" needs a precise meaning, and cryptographers settled on one in 1988. Goldwasser, Micali, and Rivest defined existential unforgeability under adaptive chosen-message attack, or EUF-CMA: an adversary who may ask you to sign any messages it likes, adaptively, still cannot produce a valid signature on a single new message it never queried [@gmr-1988]. That is a demanding bar, and every scheme in this article is designed to clear it.

But notice the exact word: a new message. EUF-CMA says nothing about whether there might be more than one valid signature on a message you did sign. Tighten the game by one word and you get strong unforgeability, SUF-CMA: the adversary cannot produce any new valid signature, even on an already-signed message. The gap between "new message" and "new signature" looks pedantic. It is the entire malleability story that unravels a Bitcoin transaction in a later section, so it is worth planting the seed now.

The standard security goal for signatures: after obtaining signatures on messages of its choice, an adversary still cannot produce a valid signature on any message it did not query. It forbids forging a new message but says nothing about producing a second signature on an old one. A stronger goal that forbids even a new signature on an already-signed message. The difference from EUF-CMA is one word -- new message versus new signature -- and it is exactly the property that signature malleability violates.

Key idea: EUF-CMA does not promise there is only one valid signature per message. A scheme can be perfectly unforgeable in the textbook sense and still let anyone manufacture a second, different signature on a message you already signed. Whether that matters depends on whether your protocol ever treats the signature itself as an identifier.

The thesis, stated as the scheme's interface

Here is the organizing idea of the field guide. A signature scheme hands the caller three things and quietly makes each one the caller's responsibility. ECDSA is the worked example that outsources all three.

Surface 1, the nonce. To sign, ECDSA needs a fresh secret random number for every signature. Supply a bad one and you leak the private key. Surface 2, the canonical form. A valid ECDSA signature (r, s) has an evil twin (r, n-s) that also verifies, so the caller must decide whether those are "the same" signature. Surface 3, the verifier. ECDSA's acceptance rule involves modular inversions and range checks, and the specification left enough slack that two conformant-looking verifiers can disagree about which signatures are valid. Every deployed break in this article lives in one of these three surfaces.

flowchart TD KG[KeyGen] --> SK[Private key sk] KG --> PK[Public key pk] M[Message] --> SG[Sign] SK --> SG N["Surface 1: the per-signature nonce you must supply"] --> SG SG --> SIG["Signature r and s"] SIG --> CF["Surface 2: is r and s the same as r and n minus s?"] CF --> VF[Verify] PK --> VF M --> VF VR["Surface 3: the byte-exact acceptance rule"] --> VF VF --> OUT{"Accept or reject"}

The academic description of ECDSA that stands in for the paywalled ANSI standard spells out this division of labor: the scheme defines the arithmetic and leaves nonce generation, encoding choices, and verifier strictness to the implementer [@johnson-menezes-vanstone-2001], [@ansi-x962-1999]. The cleaner schemes we reach later pull these surfaces back inside the definition. BIP-340 Schnorr, for instance, specifies its verifier down to the byte precisely so the caller cannot get surface 3 wrong [@bip-340].

The table below is the entire article compressed into one grid: read across each row to watch a surface move from "the caller's problem" to "the scheme's problem."

Surface	Plain ECDSA	RFC 6979 / hedged ECDSA	Ed25519	BIP-340 Schnorr
1. Nonce	Caller supplies from an RNG	Pulled inside: derived from key and message	Pulled inside: native deterministic	Pulled inside: synthetic and hedged
2. Canonical form	Caller must enforce low-`s`	Caller must enforce low-`s`	Pulled inside: strict `S < L` on decode	Pulled inside: fixed 64-byte form
3. Verifier rule	Caller implements, easy to botch	Caller implements, easy to botch	Mostly specified (RFC 8032)	Pulled inside: specified at the byte level

Two of those three surfaces were invented before ECDSA existed: the nonce in 1985, the "form" problem in 1978. To understand why a modern scheme still hands them to you, we have to go back to the moment signatures were only an idea.

3. Where Signatures Came From (1976 to 1996)

In 1976, two Stanford researchers could describe exactly what a signature must do and had no way to build one. Whitfield Diffie and Martin Hellman's "New Directions in Cryptography" named the requirement -- a private operation that anyone can verify with public information, giving the world a way to hold a signer to their message -- but it offered no algorithm to realize it [@diffie-hellman-1976]. The problem was posed before it was solved, and the two ways it eventually got solved are the two tracks this whole article runs along.

Track A: the trapdoor line

Two years later, RSA gave the first concrete construction. Signing is the private-key operation, verifying is the public-key operation, and the trapdoor -- factoring is hard, so inverting the public operation without the private exponent is infeasible -- makes the asymmetry real [@rsa-1978].

It worked, and it immediately exhibited surface 2. Textbook RSA, which signs the bare message as $\sigma = m^d \bmod N$, is existentially forgeable: pick any $\sigma$, compute $m = \sigma^e \bmod N$, and you have a valid pair for a message you did not choose but that is still "signed."

It is also multiplicatively malleable, because $\sigma_1 \cdot \sigma_2 \bmod N$ is a valid signature on $m_1 \cdot m_2 \bmod N$. The raw trapdoor has no notion of which messages are legitimately structured, so the caller must impose that structure. Getting that structure right, with a proof, took fifteen years and is the subject of a later section.

Track B: the discrete-log line

In 1985, Taher ElGamal built the first signature whose hardness rests on the discrete logarithm problem rather than factoring [@elgamal-1985]. It also introduced the load-bearing footgun that defines the rest of this article: a fresh secret nonce drawn for every single signature. Learn that nonce, or reuse it, and the private key follows. This is where surface 1 enters the lineage, and DSA and ECDSA inherit it unchanged.

The engine and the yardstick

Two ideas from the late 1980s frame everything after. In 1986, Fiat and Shamir showed how to turn an interactive identification protocol -- prove you know a secret without revealing it -- into a non-interactive signature, by replacing the verifier's random challenge with a hash of the commitment and the message [@fiat-shamir-1986]. That transform is the engine underneath Schnorr, DSA, and EdDSA, and it comes with a catch we return to in the limits section: its security is argued in the random-oracle model, treating the hash as an ideal random function.

Then, in 1988, GMR gave the field its yardstick, EUF-CMA, the definition every later scheme is measured against [@gmr-1988].

A method that converts an interactive public-coin identification protocol into a non-interactive signature by replacing the verifier's random challenge with a hash of the commitment and the message. Its security is argued in the random-oracle model, which treats the hash function as an ideal random function.

The clean template, and the patent that froze it

In 1989, Claus-Peter Schnorr published the scheme that, in hindsight, got everything right. The signer commits to a random point $R = kG$, computes a challenge $e = H(R \mathbin{|} m)$ by hashing the commitment and message, and answers with $s = k + ex$, where $x$ is the private key. Verification is a single linear check. That linearity is not a detail; it is the reason the scheme is provable, batchable, aggregatable, and non-malleable, and it is the direct ancestor of EdDSA and BIP-340 [@schnorr-1989-1991].

There was one problem, and it bent the entire history. Schnorr patented the scheme (US 4,995,082, invented 1989, granted 1991, expired 2008) [@patent-us4995082]. NIST, which needed a royalty-free national standard, could not use it. So the best-behaved discrete-log signature ever designed sat frozen for two decades while the government standardized around something else. The clean scheme was invented first and deployed last.

Schnorr's work appears twice in the literature: at CRYPTO '89 as "Efficient Identification and Signatures for Smart Cards," and in the Journal of Cryptology in 1991 as "Efficient Signature Generation by Smart Cards." They are the same line of work at two venues; cite either, and do not treat one date as a correction of the other.

timeline title From an idea to a standard, 1976 to 2020 1976 : Diffie and Hellman name the requirement, no algorithm 1978 : RSA, the first construction, Track A 1985 : ElGamal introduces the per-signature nonce, Track B 1986 : Fiat-Shamir turns identification into signatures 1989 : Schnorr, clean and linear, but patented until 2008 1994 : DSA standardized as FIPS 186 1999 : ECDSA moves DSA onto elliptic curves 2011 : Ed25519 bakes determinism in 2013 : RFC 6979 retrofits deterministic nonces 2020 : BIP-340 Schnorr, byte-exact

With the clean scheme locked behind a patent, the U.S. government needed a signature standard of its own. The choices it made under patent pressure are the reason the worse-behaved scheme shipped to the entire internet first.

4. Padding, DSA, and the Fateful Nonce

Two tracks, two ways to get it wrong first. RSA had to learn not to sign the bare message. The discrete-log camp had to standardize around a secret it could not afford to leak.

Track A cleans up its form

The fix for textbook RSA's forgeability is a rule you will hear again: never sign the bare message. Hash it first, then pad the hash into a full-width, unpredictable representative that fills the modulus. Done right, this kills existential forgery -- an attacker can no longer start from a random $\sigma$ and back out a validly structured message -- and it destroys the multiplicative structure that let signatures be multiplied together.

The workhorse encoding is PKCS#1 v1.5, a deterministic padding standardized in the early 1990s and still ubiquitous [@rfc-8017]. Full-Domain Hash was the conceptual bridge: hash the message across the entire width of the modulus. And in 1996, Bellare and Rogaway's Probabilistic Signature Scheme (PSS) added a random salt and, with it, the tightest security proof any deployed signature would ever get [@bellare-rogaway-pss-1996], [@rsa-1978].

We give PSS its own section later; for now, note only that Track A's first footgun was the form of what it signed, and Track A fixed it by structuring that form.

Track B standardizes, under patent pressure

The discrete-log camp went a different way, and politics shaped the outcome. In 1991 NIST proposed the Digital Signature Algorithm, and in 1994 it became FIPS 186 [@wikipedia-dsa]. DSA is an ElGamal and Schnorr relative, deliberately structured to sidestep both the RSA patent and Schnorr's patent, credited to David Kravitz and assigned to the NSA [@patent-us5231668]. In dodging the clean patented scheme, DSA inherited ElGamal's design decision without modification: a fresh secret nonce for every signature.

Keep the DSA dates distinct: proposed in 1991, standardized as FIPS 186 in 1994. The DSA patent (Kravitz, US 5,231,668) was filed in 1991 and granted in 1993. These four dates get conflated constantly.

Track B goes to curves

Through the late 1990s, Certicom and the ANSI X9F1 committee moved DSA onto elliptic curves. The elliptic-curve discrete logarithm is harder per bit than the finite-field version, so ECDSA gets equivalent security from much smaller keys -- 256-bit curves for 128-bit security, against 3072-bit fields for RSA and finite-field DSA. The template was otherwise unchanged, and so was the fateful decision: the same fresh secret nonce, once per signature [@johnson-menezes-vanstone-2001].

The normative ECDSA standard, ANSI X9.62-1999, is paywalled, so this article cites the open Johnson-Menezes-Vanstone description as its stand-in and never quotes X9.62 text directly. One more scoping note that matters for later decision rules: ECDSA lives on two different families of curves in practice -- the NIST P-curves used across web PKI, and secp256k1, the curve Bitcoin and Ethereum chose. The nonce discipline is identical on both.

By the end of the 1990s, an enormous share of the world's authentication rested on Track B: DSA and ECDSA, both carrying a per-signature secret that had to be unique, unpredictable, and never leaked -- and that the caller, not the scheme, was responsible for generating perfectly, every single time. Surface 1 was loaded and handed over to millions of implementers.

Note: RSA's first footgun was the form of what it signed -- the bare message was forgeable and malleable, and the cure was to structure it. The discrete-log camp's first footgun was the secret it signed with -- a per-signature nonce the scheme trusted the caller to get right. Hold both in mind: the rest of the story is these two footguns being pulled, one at a time, back inside the scheme.

The discrete-log camp had built its entire deployed base on a secret the caller had to generate perfectly, every single time. It took exactly one game console, one wallet app, and one SSH client to prove how badly that goes.

5. The Nonce Catastrophe: ECDSA in Full

Here is the mechanism underneath the PS3 break, the drained Bitcoin wallets, and the 2024 PuTTY disclosure. And here is the precision almost every retelling gets wrong.

ECDSA, mechanically

Key generation is a private scalar $d$ and its public point $Q = [d]G$, where $G$ generates a group of prime order $n$. To sign a message $m$, ECDSA draws a nonce $k$, forms $R = [k]G$, sets $r = x(R) \bmod n$, and computes

$$s = k^{-1}\bigl(H(m) + r,d\bigr) \bmod n.$$

Verification recovers $u_1 = H(m),s^{-1}$ and $u_2 = r,s^{-1}$ and accepts exactly when $x\bigl([u_1]G + [u_2]Q\bigr) \bmod n = r$ [@johnson-menezes-vanstone-2001].Throughout, $H(m)$ is the hash truncated to the leftmost $\text{bitlen}(n)$ bits, not the full digest -- which matters whenever the hash is wider than the order $n$, as with the P-521 curve behind the PuTTY story below. Stare at the signing equation and the danger is obvious.

The nonce $k$ is the load-bearing secret, and $s$ mixes $k$, the message hash, and the private key $d$ into one linear relation. Anything you learn about $k$ becomes a linear equation in $d$.

The fresh secret integer that a discrete-log signature draws for each signature, used to form the commitment point. It is the load-bearing secret: because the signature value mixes the nonce with the message hash and the private key, anything an attacker learns about the nonce turns into a linear equation in the private key.

The repeat case: certain from two signatures

Sign two different messages with the same nonce and the two signatures share the same $r$ (because $r$ depends only on $k$). Subtract the two signing equations and $k$ cancels, leaving a single linear equation you can solve directly for the private key:

$$d = \frac{s_2,H(m_1) - s_1,H(m_2)}{r,(s_1 - s_2)} \bmod n.$$

This is not statistical. It is exact, from two signatures. It is the PlayStation 3: Sony used a constant $k$, so every firmware signature shared an $r$, and two of them were enough [@ps3-27c3-2010]. And it is the 2013 Android Bitcoin thefts: a flaw in the Java SecureRandom implementation caused the nonce to repeat across signatures generated on affected devices, and real coins drained from wallets whose signing keys were exposed by the collision [@bitcoin-android-2013], [@klyubin-android-2013].

Keep the PS3 disclosures distinct: fail0verflow presented the constant-nonce ECDSA break at 27C3 in December 2010; George Hotz released a separate PS3 root key in January 2011 [@ps3-jailbreak-wikipedia]. They are related events, not the same one.

The widely cited figure of roughly 55 BTC lost in the 2013 Android incident comes from secondary reporting; the bitcoin.org primary alert states no loss total. Treat the number as reported, not official.

The recovery really is grade-school algebra. The demonstration below signs two messages under a reused nonce on toy-sized parameters, then plays the attacker: given only the public signature values, it recovers the nonce and then the private key.

{// Illustrative toy sizes, NOT secure. Here r stands in for x([k]G) mod n; // a real attacker reads r straight off the two signatures. const n = 1000003n; // small prime "group order" const mod = (a, m) => ((a % m) + m) % m; function inv(a, m) { // modular inverse, extended Euclid let [or, r] = [mod(a, m), m], [os, s] = [1n, 0n]; while (r !== 0n) { const q = or / r; [or, r] = [r, or - q * r]; [os, s] = [s, os - q * s]; } return mod(os, m); } // Signer's secrets (the attacker never sees these): const d = 123456n; // private key const k = 654321n; // per-signature nonce -- REUSED: the whole bug const r = 42n; // r = x([k]G) mod n; same k gives the same r const h1 = 111111n, h2 = 222222n; // two message hashes // ECDSA signing: s = k^{-1} (h + r*d) mod n const s1 = mod(inv(k, n) * mod(h1 + r * d, n), n); const s2 = mod(inv(k, n) * mod(h2 + r * d, n), n); // Attacker sees only (r, s1, s2, h1, h2). Recover k, then d: const kRec = mod(mod(h1 - h2, n) * inv(mod(s1 - s2, n), n), n); const dRec = mod(mod(s1 * kRec - h1, n) * inv(r, n), n); console.log('recovered nonce k = ' + kRec + ' (true ' + k + ')'); console.log('recovered key d = ' + dRec + ' (true ' + d + ')'); console.log(dRec === d ? 'PRIVATE KEY RECOVERED from two signatures' : 'mismatch');}

Two subtractions and two inversions. No discrete logarithm was harmed.

The bias case: statistical, and it needs many

Now change the failure. Instead of repeating the nonce, suppose the nonce generator is subtly biased -- say the top few bits are always zero. No two signatures share an $r$, so the two-subtraction trick does not apply, and no single signature hands you the key.

What you have instead is a stream of signatures that each leak a little information about a secret, which is precisely the Hidden Number Problem: recover a hidden value from many samples that each reveal a few high-order bits. Reformulate the biased signatures as an HNP instance, run lattice reduction, and the private key falls out -- but only after you collect many signatures.

The problem of recovering a hidden integer from many samples that each leak only a few of its high-order bits. A stream of biased-nonce signatures is an HNP instance, and lattice reduction solves it. This is why a biased nonce is a statistical attack that needs many signatures rather than a single one.

The canonical modern proof is PuTTY. In April 2024, CVE-2024-31497 disclosed that PuTTY's ECDSA nonce generator for the NIST P-521 curve was biased so that the first 9 bits of each nonce are zero, enabling full secret key recovery in roughly 60 signatures [@cve-2024-31497]. Sixty. Not two.

The rigor behind that number is a quarter-century of work: Boneh and Venkatesan introduced the Hidden Number Problem in 1996 [@boneh-venkatesan-1996]; Howgrave-Graham and Smart turned it into the first practical lattice recovery of DSA keys from partial nonces in 2001 [@howgrave-graham-smart-2001]; Nguyen and Shparlinski made ECDSA key recovery from a few leaked nonce bits rigorous in 2003 [@nguyen-shparlinski-2003]; and by 2020, LadderLeak recovered keys with under one bit of nonce leakage per signature [@ladderleak-2020].

Bleichenbacher's much-cited biased-nonce DSA attack has no standalone peer-reviewed paper; it circulated through talks and other authors' write-ups. Cite the mechanism through the Nguyen-Shparlinski and LadderLeak lattice literature, not a fabricated Bleichenbacher reference.

The precision that almost every retelling gets wrong

Key idea: A repeated nonce leaks the private key with certainty from just two signatures. A few-bit-biased nonce is different: a statistical lattice attack that needs many signatures, not one. Conflating repeat with bias is the single most corrected misconception in this corner of cryptography.

Attack	Signatures needed	Method	Certainty	Canonical example
Repeated nonce	Two	Elementary algebra, two linear equations	Exact, deterministic	PS3 (2010); Android wallets (2013)
Biased nonce	Many (tens to thousands)	Hidden Number Problem, lattice reduction	Statistical	PuTTY (roughly 60); LadderLeak (under one bit)

Note: A reused nonce is certain from two signatures, and the fix is to never reuse it. A biased nonce is a statistical HNP or lattice attack that needs many signatures, and the fix is unbiased generation or a constant-time implementation. They have different signature counts, different math, and different cures. Do not conflate them, and do not claim that a single few-bit-biased signature leaks the key.

Each biased signature gives a linear relation of the form $k_i \equiv a_i + b_i\,d \pmod n$ where the unknown nonce $k_i$ is known to be small (its top bits are zero). Stack these relations as rows of a lattice whose short vectors correspond to the small nonces. Running LLL or BKZ finds a short vector, which reveals the nonces and hence $d$. The fewer bits each signature leaks, the more signatures and the more lattice-reduction effort you need -- hence "statistical, and it needs many."

Two root causes hide under the word "bias"

There is a second fork that decides which cure you need. A generation failure -- a bad RNG that repeats or biases the nonce -- is cured by not depending on the RNG at all, which is the deterministic signing of the next section. But a usage failure is different: even a perfectly generated nonce leaks if the code that consumes it, the scalar multiplication $[k]G$, runs in time or draws power that depends on the nonce's bit-length. That is a side channel, and no amount of clever nonce generation closes it.

In 2019, both Minerva and TPM-Fail recovered ECDSA keys from exactly this timing leak, then finished the job with the same lattice machinery as the bias case [@minerva-2019], [@tpm-fail-2019]. The cure there is constant-time scalar multiplication.

flowchart TD K["The per-signature nonce k"] --> G["Generation failure: how k is produced"] K --> U["Usage failure: how k is consumed"] G --> R["Repeat: same k twice, key from two signatures"] G --> B["Bias: predictable bits, key from many signatures"] U --> T["Timing or EM leak of k during scalar multiplication"] R --> C1["Cure: deterministic nonce, RFC 6979"] B --> C1 T --> C2["Cure: constant-time scalar multiplication"]

So the random-number generator was the enemy in three of these four stories. The obvious fix is a better RNG. The actual fix, the one the whole field converged on, was to use no randomness at all.

6. The Fix for a Randomness Disaster Was Less Randomness

The counterintuitive move: if a bad random nonce is what kills you, stop rolling the dice. Derive the nonce deterministically from the one thing that is already secret and the one thing that is already unique.

RFC 6979, the retrofit

Thomas Pornin's RFC 6979 does exactly that. It derives the nonce as a deterministic function of the private key and the message hash, driving an HMAC-based deterministic random bit generator so that signing consumes no runtime entropy. A broken system RNG can no longer cause a repeat or a generation bias, because there is no RNG in the signing path [@rfc-6979].

The elegance is that nothing downstream changes: a deterministic signature verifies under the same verifier as a randomized one, so this is a drop-in with zero protocol change. The RFC states the goal plainly -- signers "do not need access to a source of high-quality randomness" -- and warns that even slight biases in nonce generation may be turned into attacks, exactly the HNP lineage from the previous section [@rfc-6979].

Deriving the per-signature nonce as a deterministic function of the private key and the message, via an HMAC-based deterministic random bit generator, so that signing needs no runtime entropy and a failing RNG can neither repeat nor bias the nonce. The resulting signature verifies under the ordinary verifier.

Note: The whole class of PS3 and Android failures came from trusting a random-number generator to produce a perfect nonce. Deterministic signing removes the generator from the signing path entirely. Then the field adds a measured dose of randomness back for a different reason.

But determinism is not unconditionally safer

Determinism trades one threat model for another. A fully deterministic signer is a fixed, replayable target. Feed it the same key and message twice and it does exactly the same computation, which is the ideal setup for a differential fault attack: run the signer twice, glitch one of the two runs with a voltage or clock disturbance, and diff the correct output against the faulted one to recover the key.

Poddebniak and colleagues demonstrated this against deterministic signature schemes in 2018 [@poddebniak-fault-2018]. NIST took the point: FIPS 186-5 approves deterministic ECDSA yet explicitly flags deterministic schemes as being of particular concern for fault attacks [@fips-186-5].

Randomized signing is vulnerable to RNG failure but resists differential fault attacks, because two signings of the same message differ. Deterministic signing is immune to RNG failure but hands an attacker a replayable target. Neither extreme dominates. Poddebniak et al. showed the fault channel is real, and the FIPS 186-5 warning is the standards body saying so out loud. The resolution is not to pick a side but to combine them.

The resolution: hedged signing

Hedged signing mixes a little fresh randomness back in on top of the deterministic derivation. RFC 6979's own section on variants sanctions adding extra input to the nonce derivation, noting that the result is then no longer deterministic (EC)DSA [@rfc-6979]. The payoff is that a hedged signer survives both failure modes: if the RNG dies, the deterministic core still produces a safe, distinct nonce; if the RNG works, the fresh entropy makes each signing unrepeatable and defeats the differential fault attack.

This is where modern libraries land by default. Go's crypto/ecdsa hedges: its documentation says signatures "are not deterministic, but entropy is mixed with the private key and the message," and that "if rand is nil, Sign will produce a deterministic signature according to RFC 6979" [@go-crypto-ecdsa]. BoringSSL mixes in additional entropy the same way [@boringssl-ecdsa], and the security analysis of hedged Fiat-Shamir signatures under faults gives this posture a formal footing [@aranha-hedged-fs-2019].

Mixing a small amount of fresh randomness into the deterministic nonce derivation, so the signer survives both RNG failure and single fault attacks. It is explicitly not deterministic (EC)DSA, and it is the default in libraries such as Go crypto/ecdsa and BoringSSL.

One caution: it fixes only the generation surface. The usage leak from the previous section -- the timing side channel during scalar multiplication -- is untouched by how you derive the nonce, and its cure remains constant-time code. Three cures, three distinct threats.

Nonce strategy	Nonce source	Survives RNG failure?	Resists the fault attack?	Where it lands
Randomized (classic ECDSA)	Fresh RNG each signature	No, the RNG is the whole risk	Yes, runs differ	The PS3 and Android failure mode
Deterministic (RFC 6979)	HMAC-DRBG of key and message	Yes	No, a replayable target	Verifies under the ordinary verifier
Hedged (RFC 6979 variants)	Deterministic core plus fresh entropy	Yes	Yes	Go and BoringSSL default

RFC 6979 pulled surface 1 back inside the scheme, but only surface 1, and only as a patch bolted onto a design that still leaks the other two. A cleaner scheme would make deterministic nonces, a fixed encoding, and a byte-exact verifier all native. That scheme already existed. It had been waiting on a patent to expire for two decades.

7. The Breakthrough: Schnorr Done Right

Everything RFC 6979 retrofitted, one scheme had baked in from birth. It was invented in 1989 and could not be deployed until 2008, because it was patented.

Why Schnorr is clean

Recall Schnorr's response, $s = k + ex$. That single linear equation is what makes the scheme provable, batchable, and aggregatable. Provable, because the linear structure is what the Pointcheval-Stern forking lemma needs to reduce forgery to the discrete-log problem in the random-oracle model [@pointcheval-stern-1996-2000]. Batchable, because a linear verification equation lets you check many signatures as one combined equation, the subject of a later section. Aggregatable, because linear things add, which is why threshold and multi-signature schemes like FROST and MuSig2 build directly on Schnorr.

The Fiat-Shamir transform is still the engine turning the underlying identification protocol into a signature; Schnorr is simply the cleanest identification protocol to feed it.

EdDSA and Ed25519 pull all three surfaces inside

In 2011, Bernstein, Duif, Lange, Schwabe, and Yang published EdDSA, and its edwards25519 instantiation, Ed25519, is the scheme that pulls all three surfaces inside the definition by design [@bernstein-eddsa-2011].

Surface 1, the nonce, is native and deterministic. Ed25519 computes the nonce as $r = H(\text{prefix} \mathbin{|} M) \bmod L$, hashing a secret key-derived prefix together with the message. There is no runtime RNG in the signing path, so the entire PS3 and Android failure class is structurally impossible rather than merely discouraged. The designers said so directly, and named the incident:

Signatures are generated deterministically ... directly relevant to the recent collapse of the Sony PlayStation 3 security system.

Surface 2, the encoding, is fixed. An Ed25519 signature is a fixed 64-byte string, a point encoding followed by a scalar, and RFC 8032 mandates a strict range check that the scalar satisfy $0 \le S < L$ on decode. Enforcing that check makes a strict Ed25519 implementation strongly unforgeable, closing the malleability surface at the encoding level [@rfc-8032].

Surface 3, the arithmetic, is safe by construction. edwards25519 uses a complete twisted-Edwards addition law with no exceptional cases, so the constant-time implementation is the natural one, and there are no special points where a naive verifier misbehaves [@bernstein-eddsa-2011]. Determinism here is structural, not a patch bolted on afterward, which is the real reason "why determinism helps" is a property of the design rather than a library option.

A Schnorr signature instantiated on a complete twisted-Edwards curve with a key-prefixed deterministic nonce and a fixed encoding. Ed25519 is its edwards25519 instantiation: constant-time by construction, misuse-resistant by design, and strongly unforgeable under a strict verifier. A nonce derived by hashing a secret key-derived prefix together with the message, so it is deterministic, unique per message, and never drawn from a runtime random-number generator. Hashing the public key into the challenge additionally hardens the scheme in multi-user and related-key settings.

Key-prefixing -- folding the public key into the hash that produces the challenge -- is a small design choice with outsized value: it hardens the scheme against multi-user and related-key attacks that a bare Schnorr challenge would leave open.

flowchart TD M[Message] --> SCH SK[Private key] --> SCH subgraph SCH["Inside EdDSA and BIP-340"] N["Surface 1: deterministic key-prefixed nonce"] F["Surface 2: fixed canonical encoding"] V["Surface 3: byte-exact verifier"] end SCH --> SIG["One valid signature, no caller choices"] SIG --> OK{"Accept or reject, deterministically"}

BIP-340: Schnorr over secp256k1, with the verifier pulled inside

Bitcoin needed Schnorr on its own curve, secp256k1, and in 2020 BIP-340 delivered it: 64-byte signatures, 32-byte x-only public keys, and, most importantly for surface 3, a verifier specified down to the byte. The specification states that the verification algorithm is "completely specified ... at the byte level," the guarantee whose full payoff we reach when we return to verifier strictness later [@bip-340].

BIP-340 also adopts the hedged idea natively: it derives a synthetic nonce and notes that the added randomness is only supplemental to security, so a faulty RNG cannot break it. Bitcoin's Taproot upgrade deployed it in November 2021 [@optech-taproot-2021].

Now the punchline. The scheme that makes deterministic nonces, a fixed encoding, and a byte-exact verifier native is Schnorr, invented in 1989. It could not be deployed until its patent expired in 2008, which is exactly why the worse-behaved ECDSA shipped to the entire internet two decades before its clean successor. Ed25519 arrived in 2011, BIP-340 in 2020. The best-behaved design was first on paper and last in production.

The two-decade gap between the clean scheme's invention (US 4,995,082, 1989) and its deployment -- Ed25519 in 2011, BIP-340 in 2020 -- is a patent artifact, not a technical one.

One honest caveat keeps this from being a silver bullet. Plain Ed25519 has no hedging knob: it is purely deterministic, so on hardware where an attacker can inject faults it is the harder-hit fault target, and hardened embedded implementations re-inject randomness exactly as the hedged-signing analysis recommends [@poddebniak-fault-2018], [@aranha-hedged-fs-2019]. This is coexistence, not a coronation. Ed25519 and BIP-340 close all three surfaces on the discrete-log track. But the other 1976 track, RSA, had its own reckoning with the form of a signature, and its answer came with something the discrete-log schemes still lack: a tight proof.

8. RSA-PSS: Probabilistic Signing with a Tight Proof

On the discrete-log track, determinism was the virtue. On the RSA track, the opposite move -- adding randomness -- bought the best security proof any deployed signature has.

PSS, mechanically

Never sign the bare message. The EMSA-PSS encoding hashes the message, mixes in a fresh random salt, and builds a full-width encoded message before the RSA private-key operation. Concretely, it forms $H = \mathrm{Hash}(\text{padding} \mathbin{|} \mathrm{Hash}(M) \mathbin{|} \text{salt})$, masks the salt-bearing data block against a mask generated from $H$, and assembles the encoded message as maskedDB followed by $H$ followed by the trailer byte 0xbc. The signature is that encoded message raised to the private exponent modulo $n$ [@rfc-8017], [@bellare-rogaway-pss-1996]. Verification recomputes the whole structure and checks it exactly, rejecting anything that does not fit the padding shape.

flowchart LR M[Message] --> MH["mHash = Hash of M"] SALT["Random salt"] --> H2["H = Hash of padding, mHash, salt"] MH --> H2 H2 --> DB["maskedDB = data block xor mask from H"] DB --> EM["EM = maskedDB then H then 0xbc"] H2 --> EM EM --> SIG["Signature = EM to the d mod n"]

Why probabilistic buys a tight proof

Bellare and Rogaway proved PSS existentially unforgeable with a tight reduction to the RSA problem in the random-oracle model: a forger against PSS can be turned into an algorithm that inverts RSA with essentially the same success probability and running time [@bellare-rogaway-pss-1996].

That word tight is the whole prize. A tight reduction loses almost nothing between the scheme's security and the underlying hard problem, so a concrete key size gives you concrete confidence; a loose reduction loses a large factor, so you either accept a weaker guarantee or inflate parameters to compensate. By contrast, PKCS#1 v1.5's signature padding has only a heuristic argument, not a tight proof. Among deployed signatures, PSS's guarantee is the gold standard, and it sets up the proof-quality spectrum we return to in the limits section.

The Probabilistic Signature Scheme encoding for RSA, which injects a fresh random salt into the padding before the RSA private-key operation and has the verifier recompute and check that structure exactly. The salt is what makes its security reduction to the RSA problem tight, unlike the deterministic PKCS#1 v1.5 padding.

Full-Domain Hash was PSS's provable but loose predecessor: hash the message across the full width of the modulus, then apply the private operation. PSS keeps that idea and adds the random salt, and the salt is precisely what upgrades the reduction from loose to tight.

The deployment reality, stated exactly

"Is RSA-PSS widely deployed?" has a split answer, and blurring the split is a common error. TLS 1.3 makes RSA-PSS mandatory to implement for the online handshake signature: the CertificateVerify message uses the rsa_pss_rsae_* schemes, and a compliant implementation must support them [@rfc-8446]. That is a specification requirement.

Separately, as a matter of deployment, most Web-PKI certificates -- the long-lived signatures a certificate authority puts on the certs in the chain -- are still PKCS#1 v1.5. So the handshake signature your browser makes live and the certificate-chain signatures it validates are different signatures with different padding, and the honest answer separates them.

Note: The handshake CertificateVerify signature is RSA-PSS by TLS 1.3 requirement. The certificate-chain signatures a certificate authority produces are, in practice, still mostly PKCS#1 v1.5. Same connection, two padding schemes, two different signatures. Do not answer "does the web use PSS?" without saying which of the two you mean.

This is the boundary never to blur. PKCS#1 v1.5 *encryption* padding is the classic Bleichenbacher decryption oracle, a genuine break of the encryption scheme, and a companion part in this series on RSA owns that story. PKCS#1 v1.5 *signatures* are a different animal: the scheme is not broken.

Its real-world failures were implementation bugs in lax verifiers -- most famously the low-exponent forgery that a sloppy parser allows when the public exponent is small, and its 2014 re-occurrence known as BERserk. Those are parsing bugs, not a break of the signature math. That low-exponent forgery has no standalone peer-reviewed paper, so cite the mechanism and the secondary write-ups, never a fabricated primary. The practical rule stands regardless: use RSA-PSS for new designs [@rfc-8017].

The cost profile that decides where PSS fits

RSA's economics are lopsided and they determine where it belongs. Verification uses a tiny public exponent, usually 65537, so it is very fast. Signing uses the full-width private exponent, so it is slow, and the signatures are large: 256 bytes at RSA-2048, 384 bytes at RSA-3072, an order of magnitude bigger than a 64-byte elliptic-curve signature. That makes RSA-PSS excellent where you verify far more often than you sign, such as certificates and firmware images, and painful where you sign constantly. We put real numbers on this in the state-of-the-art section.

PSS pinned RSA's form with a proof. But the discrete-log schemes still had an unresolved question about form, one that let anyone alter a signature without the key and briefly threatened to break Bitcoin.

9. Malleability, Canonicalization, and Strong Unforgeability

Recall the promise EUF-CMA does not make: that there is only one valid signature per message. For ECDSA there are always at least two, and the caller was left to notice.

Surface 2: ECDSA malleability, mechanically

Take any valid ECDSA signature (r, s). Now negate the scalar: (r, n - s) also verifies, and it is a different signature on the same message, producible by anyone with no private key at all. The reason is visible in the verification equation. Replacing $s$ with $n - s$ is replacing it with $-s \bmod n$, which flips the signs of both $u_1$ and $u_2$, negating the recovered point to $-R$. On these curves a point and its negation share an $x$-coordinate, so $x(-R) \bmod n = x(R) \bmod n = r$, and the check still passes.

BIP-340 states the fact plainly: "If (r,s) is a valid ECDSA signature then (r,n-s) is also valid" [@bip-340]. So standard ECDSA meets EUF-CMA but not SUF-CMA -- exactly why that one-word distinction was worth planting seven sections ago.

The existence of a second valid signature on an already-signed message that anyone can derive without the private key. For ECDSA, whenever (r, s) is valid so is (r, n - s). It is the reason plain ECDSA satisfies EUF-CMA but fails SUF-CMA.

The protocol fallout: Bitcoin transaction malleability

Malleability is harmless until a protocol treats the signature as an identifier. Bitcoin did. A transaction id is a hash computed over the whole transaction including its signatures, so flipping s to n - s changes the txid without changing what the transaction does. Anything that referenced an unconfirmed transaction by its id -- a chain of dependent spends, a naive payment tracker -- could be broken by a third party rewriting the signature in flight [@decker-wattenhofer-2014].

The honest Mt. Gox nuance: transaction malleability was real and was publicly blamed for the 2014 collapse, but Decker and Wattenhofer measured the malleability activity on the network and found it insufficient to account for the losses, and later analysis attributes most of the missing coins to a wallet compromise around 2011. Real and blamed, but not the sole cause.

The fix: low-s canonicalization restores SUF-CMA

If two signatures differ only by $s$ versus $n - s$, pick one. The low-s rule requires $s \le n/2$, so exactly one of the twins is standard and the other is rejected on sight. Bitcoin moved this from proposal to policy across BIP-62 and BIP-146, and it restores strong unforgeability in the system that needed it most [@bip-62], [@bip-146]. Ed25519's analogue is the strict S < L range check that RFC 8032 mandates on decode, which is what makes a strict Ed25519 implementation SUF-CMA [@rfc-8032].

Note: If any part of your protocol hashes over or deduplicates by the signature itself -- transaction ids, caches, idempotency keys, replay checks -- plain EUF-CMA lets an attacker manufacture a second valid signature and break your assumption. Enforce low-s or a strict S < L range check, or choose a SUF-CMA-by-design scheme such as BIP-340.

Surface 3: the verifier, which closed last and most embarrassingly

The third surface failed in two distinct shapes. The first is a permissive verifier. In 2022, Java's ECDSA verification accepted the all-zero signature (r, s) = (0, 0) for any message and any key, because it skipped the range check that forbids zero. "Psychic Signatures," CVE-2022-21449, scored CVSS 7.5 and forged anything built on Java's signatures: TLS, signed JWTs, SAML assertions, OIDC id tokens, and WebAuthn responses [@cve-2022-21449], [@neil-madden-2022].

The second shape is more subtle: honest disagreement. "Taming the Many EdDSAs" showed that verifiers all conforming to RFC 8032 nonetheless split on adversarial inputs -- cofactored versus cofactorless verification equations, non-canonical point encodings, small-order points -- so the same signature could be valid to one library and invalid to another [@chalkias-many-eddsas-2020]. For a consensus system that is a disaster, and ZIP-215 answers it by pinning a byte-exact validation profile on which single and batch verification always agree [@zip-215].

Edwards curves have a cofactor, so the verification equation can be written to multiply through by it (cofactored) or not (cofactorless). RFC 8032 notes the cofactorless check is sufficient but not required, and that latitude is exactly what let conformant verifiers disagree until ZIP-215 fixed a single byte-exact rule.

BIP-340 treats this surface as an engineering requirement rather than an afterthought, which is why its verifier is specified to the byte:

The verification algorithm must be completely specified at the byte level. This guarantees that nobody can construct a signature that is valid to some verifiers but not all.

The evidence, in one table

Every named break in this article splits cleanly across the three surfaces. Read the "Surface" column and the thesis stops being a slogan and becomes a body count.

Break	Year	Surface	Sub-type	Root cause	The rule it teaches
PlayStation 3	2010	1. Nonce	Repeat	One constant nonce across all signatures	Never reuse a nonce; two signatures leak the key
Android Bitcoin wallets	2013	1. Nonce	Generation	Broken SecureRandom repeated the nonce	Do not trust the RNG; derive the nonce
Bitcoin txid, Mt. Gox era	2014	2. Canonical form	Malleability	(r, n - s) is a free second signature	Enforce low-s; treat SUF-CMA as the bar
Minerva, TPM-Fail	2019	1. Nonce	Usage, timing	Nonce bit-length leaked during scalar mult	Constant-time scalar multiplication
Many EdDSAs, ZIP-215	2020	3. Verifier	Disagreement	Conformant verifiers split on edge cases	Specify the verifier at the byte level
Psychic Signatures	2022	3. Verifier	Permissive	Verifier accepted (0, 0) for any message	Range-check the signature; reject zero
PuTTY	2024	1. Nonce	Bias	Top 9 bits of each P-521 nonce were zero	Unbiased or deterministic nonce generation

Notice what is missing from the root-cause column: not once did anyone solve an elliptic-curve discrete logarithm or factor a modulus. That is Aha 1, complete. Fix the nonce, fix the form, fix the verifier, and you have a signature you can trust one at a time. But the modern world verifies millions at once, and the obvious way to batch them is quietly forgeable.

10. Batch Verification, and the One Condition That Makes It Safe

Schnorr's linearity offers a tempting shortcut: instead of checking a thousand signatures one by one, check their sum once. Done naively, that shortcut lets an attacker slip forgeries past you.

The technique

Because Schnorr, EdDSA, and BIP-340 verification is a single linear equation, a batch of $u$ signatures can be checked as one random linear combination of those equations. That collapses $u$ separate double-scalar multiplications into a single large multi-scalar multiplication, whose cost is sublinear in the number of distinct bases, which is where the throughput win comes from. The original Ed25519 paper reports a batch of 64 signatures verifying at under 134,000 cycles per signature, against 273,364 cycles for a single verification, roughly a twofold speedup [@bernstein-eddsa-2011].

Checking many signatures with one combined equation instead of one equation each, by exploiting the linearity of Schnorr-type verification. It is safe only when each signature's contribution is weighted by an independent random coefficient; the naive equal-weight sum is forgeable.

The non-negotiable safety condition

The coefficients must be randomized. Here is why the naive all-ones sum fails. A signature verifies exactly when its verification residual is zero. If you add the residuals with equal weight, an attacker can submit two invalid signatures whose residuals are equal and opposite, so they cancel in the sum: the batch equation holds even though neither signature is valid.

Weight each residual by an independent random coefficient and the errors no longer cancel except with negligible probability. BIP-340 derives its batch coefficients from a CSPRNG seeded over all of the inputs, so an attacker cannot predict the weights and engineer a set of residuals that sums to zero without each signature being individually valid [@bip-340].

{// Illustrative model. A signature is VALID iff its verification residual is 0. const n = 1000003n; const mod = (a, m) => ((a % m) + m) % m; // Attacker forges two invalid signatures whose residuals are equal and opposite. const D = 777n; // the cancelling error the attacker chooses const e1 = mod(D, n); // residual of forged signature 1 (nonzero: invalid) const e2 = mod(-D, n); // residual of forged signature 2 (nonzero: invalid) console.log('signature 1 valid on its own? ' + (e1 === 0n)); // false console.log('signature 2 valid on its own? ' + (e2 === 0n)); // false // Naive all-ones batch: accept iff the plain sum of residuals is 0 const allOnes = mod(e1 + e2, n); console.log('all-ones batch residual = ' + allOnes + (allOnes === 0n ? ' -> ACCEPTED, forgery slips through' : ' -> rejected')); // Randomized batch: weight each residual by a per-input random coefficient const a1 = 31337n, a2 = 424242n; // seeded by a CSPRNG over ALL inputs const rand = mod(a1 * e1 + a2 * e2, n); console.log('randomized batch residual = ' + rand + (rand === 0n ? ' -> accepted' : ' -> REJECTED, forgery caught'));}

flowchart TD IN["u signatures to verify"] --> SEED["Seed a CSPRNG over all u inputs"] SEED --> COEF["Random coefficients a1 through au"] COEF --> MSM["One multi-scalar multiplication"] MSM --> OK{"Batch equation holds?"} OK -->|yes| ACC["Accept all"] OK -->|no| FALL["Fall back to individual verification"] IN --> BAD["Naive all-ones coefficients"] BAD --> CANCEL["Attacker makes two residuals cancel"] CANCEL --> FORGE["Invalid batch accepted, a forgery"]

Note: Naive all-ones batch verification is forgeable, because an attacker can craft invalid signatures whose verification errors cancel in the sum. Randomize the coefficients, seeded by a CSPRNG over every input, or do not batch at all.

What batching does not give you, and where it is not used

A failed batch tells you only that some signature in it is bad, not which one, so you must fall back to individual verification to find the culprit -- a real cost when batches are large. Batching is also a Schnorr-family privilege: BIP-340 notes that standardized ECDSA "cannot be verified more efficiently in batch," so the ECDSA deployments that dominate the web get no speedup here [@bip-340]. And adoption is narrower than the technique's fame suggests.

Bitcoin Core does not batch-verify signatures in consensus validation as of 2024 through 2026: the determinism and simplicity of validating each signature independently outweigh the throughput gain. Batch verification is a real optimization for specific high-volume verifiers, not a universal default.

That is the deployed toolkit: ECDSA, Ed25519, BIP-340 Schnorr, and RSA-PSS, each with its surfaces pulled in to a different degree. Which one should you reach for in 2026? The honest answer is not a single winner.

11. State of the Art and Competing Approaches (2024 to 2026)

There is no single best signature scheme to crown. The state of the art is four deployed constructions, each dominant in a niche, plus a set of signposts pointing at the frontier. This is coexistence with selective supersession, not a ladder where each rung replaces the last.

The four, per niche

Deterministic or hedged ECDSA leads by raw deployment. It is what FIPS, TLS, and the secp256k1 chains run, and modern libraries hedge it by default [@rfc-6979], [@go-crypto-ecdsa]. Ed25519 is the misuse-resistant greenfield default: deterministic nonce, fixed encoding, constant-time by construction [@rfc-8032], [@bernstein-eddsa-2011]. BIP-340 Schnorr leads for consensus and threshold work, being byte-exact, SUF-CMA by design, and linear enough to aggregate [@bip-340]. RSA-PSS leads verify-heavy and interoperability-bound settings with its tight proof and cheap verification [@rfc-8017], [@rfc-8446].

Of the classical schemes, only finite-field DSA is withdrawn for new signatures by FIPS 186-5; deterministic ECDSA is approved, which is not the same as mandated [@fips-186-5].

FIPS 186-5 (2023) approves ECDSA, EdDSA, and deterministic ECDSA, and withdraws finite-field DSA for new signatures. Two practical consequences follow. First, "approved" is not "mandated": you may hedge, and the default posture in compliant libraries is randomized or hedged signing rather than bare determinism. Second, EdDSA is now a first-class FIPS citizen, so choosing Ed25519 no longer forces you outside the compliance boundary the way it once did [@fips-186-5].

Property	ECDSA (hedged)	Ed25519	BIP-340 Schnorr	RSA-PSS
Hardness	ECDLP (P-256, secp256k1)	ECDLP (edwards25519)	ECDLP (secp256k1)	RSA, related to factoring
Signature size	64 to 72 bytes (DER)	64 bytes	64 bytes	256 bytes at 2048, 384 at 3072
Public key size	33 bytes compressed	32 bytes	32 bytes, x-only	256 to 384 bytes
Nonce or randomness	Hedged or RFC 6979	Native deterministic	Synthetic, hedged	Fresh random salt
Malleable by default?	Yes, enforce low-s	No under strict S range	No, byte-exact	No
Batch verification	No	Yes	Yes	No
Proof quality	Idealized only	Loose, forking lemma	Loose, forking lemma	Tight
Sign versus verify	Balanced	Balanced	Balanced	Slow sign, fast verify
Best for	FIPS, secp256k1 chains	Greenfield default	Consensus, threshold	RSA interop, verify-heavy

Benchmarks, read correctly

The reference measurements everyone quotes for Ed25519 come from the eBATS and eBACS public benchmarking effort [@bernstein-eddsa-2011], [@ebacs-bench].

Ed25519 operation	Cost, eBATS reference measurement
Sign one message	87,548 cycles
Verify one signature	273,364 cycles
Verify in a batch of 64	under 134,000 cycles per signature

Note: Two facts most first courses skip. RSA verifies far faster than it signs, often by more than an order of magnitude, because the public exponent is tiny and the private exponent is full width, whereas elliptic-curve schemes are balanced between the two operations. And rankings flip with the implementation: a build with hand-tuned P-256 assembly can make ECDSA out-run Ed25519, the reverse of the eBATS ordering. Never quote a cross-scheme benchmark as if it were a property of the mathematics. Measure the library you ship, on the CPU you run.

Signposts at the frontier

Four constructions sit just outside the four-scheme core and deserve a pointer rather than a section. XEdDSA lets you sign with an X25519 Diffie-Hellman key, a specialized key-reuse trick that is deployed at scale in the Signal protocol and the applications built on it, so it is peripheral by role but anything but academic [@xeddsa-spec].

MuSig2 and FROST are two-round multi-signature and threshold-Schnorr protocols that produce an ordinary 64-byte signature indistinguishable from a single-signer one [@musig2-2020], [@rfc-9591]. BLS offers pairing-based aggregation and anchors Ethereum consensus; a companion part in this series on hashing to curves covers it.

And the horizon is post-quantum: NIST finalized ML-DSA in FIPS 204 and SLH-DSA in FIPS 205 in August 2024, both with signatures ranging from kilobytes to tens of kilobytes, an order of magnitude or more larger than anything in the table above [@fips-204], [@fips-205].

XEdDSA is de-randomized like plain EdDSA, so absent hedging it inherits the same differential-fault exposure, which the hedged Fiat-Shamir analysis addresses [@aranha-hedged-fs-2019].

Every row of those tables assumes the hardness holds. That assumption has a precise shape, and a known expiration date.

12. Theoretical Limits

Here is what a first course skips: none of these schemes has an unconditional proof, their proofs differ enormously in quality, and every one of them dies to the same quantum algorithm.

Conditional security, three assumptions

Every guarantee in this article is conditional. The elliptic-curve schemes rest on the hardness of the elliptic-curve discrete logarithm; RSA-PSS rests on the RSA problem, which is no harder than factoring; and all of the Fiat-Shamir schemes rest additionally on the random-oracle model, a heuristic that proves security while treating the hash function as an ideal random function that no real hash exactly is. "Provably secure" here always means "secure if these assumptions hold." That is not a weakness peculiar to signatures; it is the honest status of essentially all deployed public-key cryptography.

The tightness spectrum

The interesting frontier is not whether the proofs exist but how much they lose. A reduction converts an attacker on the scheme into an algorithm against the hard problem, and the tightness of that conversion is how much security leaks in the translation.

Scheme	Reduction (random-oracle model)	Tightness	What it means
RSA-PSS	To the RSA problem, Bellare-Rogaway	Tight	The best guarantee of any deployed signature
Schnorr, EdDSA	To discrete log via the forking lemma, Pointcheval-Stern	Loose, quadratic loss	Seurin showed the loss is essentially optimal, a barrier not a gap
(EC)DSA	Only with an idealized conversion function, Fersch-Kiltz-Poettering	Idealized	Secure because ECDLP holds, not because the scheme is clean

RSA-PSS has a tight reduction to RSA [@bellare-rogaway-pss-1996]. Schnorr and EdDSA have only a loose, quadratic-loss reduction through the forking lemma [@pointcheval-stern-1996-2000], and Seurin proved that this loss is essentially the best any algebraic reduction can do, so it is a genuine barrier rather than a proof waiting to be improved [@seurin-2012]. Later work made the multi-user Schnorr reduction explicit, with a loss on the order of the number of hash queries [@kiltz-masny-pan-2016]. And (EC)DSA has the weakest guarantee of all: the only known proofs must model its conversion function as an idealized, bijective random oracle [@fersch-kiltz-poettering-2016].

Key idea: Deployment and proof quality are inversely correlated here. The most-deployed signature in the world, ECDSA, has the least satisfying proof: it is secure because the elliptic-curve discrete logarithm is hard, not because the scheme is clean. And tight proof or loose, every scheme in this article falls to Shor's algorithm. The destination is post-quantum, not a better curve.

The classical hardness bounds are near-optimal but unproven. Elliptic-curve discrete log costs about 2^128 operations for a 256-bit curve by Pollard's rho, which is optimal in the generic-group model, while factoring costs sub-exponential effort by the general number field sieve. Neither hardness is proven, and RSA inversion is not even proven equivalent to factoring.

The one bound that is not conjectural

There is exactly one limit in this section that is not an open question. Shor's algorithm solves both factoring and the discrete logarithm in quantum polynomial time, so a cryptographically relevant quantum computer breaks every scheme in this article at once [@shor-1997]. No curve size and no modulus length helps, because Shor's cost is polynomial in the key size. The elliptic-curve schemes and RSA-PSS fall together.

For encryption, an adversary can record ciphertext today and decrypt it after building a quantum computer, so the harvest-now-decrypt-later threat is urgent. Signatures authenticate in real time, so a signature forged after a quantum computer exists cannot retroactively forge yesterday's authenticated session. The exception is long-lived keys: code-signing roots, firmware, and certificate roots that must stay valid for decades have to outlive Shor, which is exactly where post-quantum signatures are being adopted first [@shor-1997].

If the theory is this settled -- conditional proofs of differing quality and a known quantum cliff -- then everything still up to you lives in the code. Here is that part, compressed into rules you can apply on Monday.

13. The Practical Guide: Decision Rules

Every rule below is one row of the failure catalog, inverted. If you remember nothing else, remember the decision tree.

flowchart TD START{"What constrains you?"} --> G["Greenfield, no constraint"] START --> F["FIPS or compliance"] START --> R["Must interoperate with RSA"] START --> B["Bitcoin or byte-exact consensus"] START --> T["Threshold or multisig"] G --> GE["Ed25519 with a strict verifier"] F --> FE["Hedged ECDSA P-256 or FIPS-approved Ed25519, enforce low-s"] R --> RE["RSA-PSS as PS256, at least RSA-3072"] B --> BE["BIP-340 Schnorr, or low-s strict-DER ECDSA if legacy"] T --> TE["FROST or MuSig2, never deterministic nonces"]

Use X, with these parameters, in this case

Greenfield, no FIPS constraint. Reach for Ed25519, verified with a strict profile that enforces the canonical S < L range and a ZIP-215-style byte-exact rule [@rfc-8032], [@zip-215]. You get deterministic nonces, a fixed encoding, and constant-time arithmetic for free.

FIPS-constrained. Use deterministic or hedged ECDSA on P-256 following RFC 6979 and its hedged variant, or FIPS 186-5-approved Ed25519. Enforce low-s, and insist on a constant-time library so you do not reopen the Minerva timing surface [@rfc-6979], [@fips-186-5].

RSA interoperability for a new design. Use RSA-PSS (the PS256 scheme), never PKCS#1 v1.5 for new work. Set the salt length equal to the hash length, use a matching MGF hash, and choose at least RSA-3072 for 128-bit security [@rfc-8017].

Bitcoin or byte-exact consensus. Use BIP-340 Schnorr, as deployed in Taproot. If you are stuck with legacy ECDSA, enforce low-s and strict DER encoding so transactions are not malleable [@bip-340], [@bip-146].

Threshold or multisig. Use FROST or MuSig2 over BIP-340, and never use deterministic nonces in multi-party signing -- a deterministic nonce in an interactive protocol can be replayed across differing co-signer inputs and leak the share [@rfc-9591], [@musig2-2020], [@bip-340].

Always. Hedge the nonce whenever the platform can face fault injection. Domain-separate your keys per scheme: BIP-340 warns that reusing one key across RFC-6979 ECDSA and BIP-340 can leak it through cross-scheme nonce coincidences [@bip-340]. Verify strictly, and when any part of your protocol hashes over signatures, treat SUF-CMA, not merely EUF-CMA, as the bar.

Constraint	Use	Key parameters	Because of (named break)
Greenfield, no FIPS	Ed25519	Strict `S < L`, ZIP-215 verifier	Psychic Signatures, Many EdDSAs
FIPS-constrained	Hedged ECDSA P-256, or FIPS Ed25519	RFC 6979 or hedged, low-s, constant-time	PS3, PuTTY, Minerva
RSA interop, new design	RSA-PSS as PS256	Salt length equals hash length, RSA-3072 or larger	v1.5 low-exponent forgeries
Bitcoin or consensus	BIP-340 Schnorr (Taproot)	Or low-s and strict DER if legacy ECDSA	Bitcoin txid malleability
Threshold or multisig	FROST or MuSig2	Never deterministic nonces in multi-party	Reused-nonce key recovery

Note: Ed25519 for anything greenfield. Hedged ECDSA on P-256 for FIPS and secp256k1 chains. RSA-PSS for RSA interoperability. BIP-340 Schnorr for consensus and threshold. Whatever you pick, verify strictly and hedge the nonce wherever faults are possible.

Generate an Ed25519 keypair, then sign and verify a message, with OpenSSL 3.x:

Create the private key: openssl genpkey -algorithm ed25519 -out sk.pem
Extract the public key: openssl pkey -in sk.pem -pubout -out pk.pem
Sign: openssl pkeyutl -sign -inkey sk.pem -rawin -in message.txt -out sig.bin
Verify: openssl pkeyutl -verify -pubin -inkey pk.pem -rawin -in message.txt -sigfile sig.bin

To force RSA-PSS rather than PKCS#1 v1.5 when signing with an RSA key: openssl dgst -sha256 -sign rsa.pem -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -out sig.bin message.txt

Turn each of those rules around and you get the antipatterns that keep showing up in real code review: the misuse catalog.

14. Common Misuse, and the One Sentence Underneath

Every break in this article was a violated rule from the last section. Here they are as the antipatterns a reviewer actually finds in real code.

A caller-supplied nonce from a weak or predictable RNG. This is the Android wallets and, in its biased form, PuTTY [@bitcoin-android-2013], [@cve-2024-31497].
A nonce reused across messages or keys. This is the PS3, and it is certain from two signatures [@ps3-27c3-2010].
Non-constant-time scalar multiplication. This is Minerva and TPM-Fail: even a perfect nonce leaks through timing [@minerva-2019], [@tpm-fail-2019].
Skipping the range, subgroup, or canonical-s checks. This reopens malleability, and for a permissive verifier it reopens forgery [@bip-340].
Treating EUF-CMA as if it meant non-malleability. This is Bitcoin transaction-id malleability: the scheme was unforgeable and the protocol still broke [@bip-340].
Naive all-ones batch coefficients. A forgeable batch, defeated by errors that cancel in the sum [@bip-340].
A lax or permissive verifier. This is Psychic Signatures, which accepted the all-zero signature for any message [@cve-2022-21449].
PKCS#1 v1.5 for new designs, or blurring the encryption-versus-signature split. Use RSA-PSS, and never confuse the v1.5 decryption oracle with the unbroken v1.5 signature scheme [@rfc-8017].
Signing without domain separation. One key across two schemes can leak through cross-scheme nonce coincidences [@bip-340].

Note: First, an ECDSA nonce drawn from a dubious or caller-supplied RNG instead of a derived or hedged one. Second, a missing low-s or strict S < L check, so signatures are malleable. Third, a verifier that does not range-check the signature values and will therefore accept degenerate inputs. Those three account for most of the named breaks in this article.

No, and this is the most repeated mistake in the area. A *repeated* nonce leaks the key with certainty from just two signatures -- that is the PS3 break. A *biased* nonce, where the top few bits are always zero, is a statistical attack: it takes many signatures and lattice reduction, framed as the Hidden Number Problem, to recover the key, as PuTTY's roughly-sixty-signature break showed. "One few-bit-biased signature leaks your key" is false. No. Deriving the nonce deterministically removes the RNG-failure disaster, but it turns the signer into a fixed, replayable target for fault attacks: glitch one of two identical signings and diff the outputs. The resolution is hedged signing -- mix a little fresh randomness back in -- which most modern libraries do by default. No, but it is nuanced. Standard Ed25519 can be malleable at the group level unless the verifier enforces the canonical `S < L` range check that RFC 8032 mandates on decode. Enforce it, as strict libraries do, and Ed25519 is strongly unforgeable; skip it and you inherit a malleability surface. Encryption. The v1.5 *encryption* padding is the classic Bleichenbacher decryption oracle. The v1.5 *signature* scheme is not broken; its real failures were implementation bugs in lax verifiers, such as low-exponent forgeries, not a break of the signature math. Even so, use RSA-PSS for new designs. Ed25519 for anything greenfield: deterministic nonces, a fixed 64-byte encoding, constant-time arithmetic, and strong unforgeability under strict verification. Use ECDSA only where a standard or platform compels it, such as FIPS environments or secp256k1 chains, and then with RFC 6979 or hedged nonces, low-s enforcement, and a constant-time library. If any part of your protocol hashes over or deduplicates by the signature itself -- transaction ids, caches, replay checks -- you need SUF-CMA, because plain EUF-CMA lets an attacker produce a different valid signature on the same message. Enforce low-s or a strict `S < L` range, or choose a SUF-CMA-by-design scheme like BIP-340. No. Shor's algorithm breaks both discrete log and factoring in polynomial time, so ECDSA, EdDSA, BIP-340, and RSA-PSS all fall to a cryptographically relevant quantum computer, and no curve or modulus size helps. The destination is post-quantum signatures such as ML-DSA and SLH-DSA, finalized in August 2024 and being adopted first for long-lived keys.

The one sentence underneath

Return to that slide at 27C3. Sony's curve was pristine; two signatures gave up the master key [@ps3-27c3-2010]. Now line up the rest of the body count. Across three decades, almost no deployed break ever solved an elliptic-curve discrete logarithm or factored a modulus. Attackers reused a nonce, biased one [@cve-2024-31497], leaked one through timing [@minerva-2019], flipped a signature's s to n - s, or accepted the all-zero signature for any message [@cve-2022-21449]. The hardness assumption sat untouched through all of it.

That is why "signing done right" is not a single choice but a stack: the right scheme for the job, a nonce you do not have to trust the RNG for, a canonical form, and a byte-exact verifier -- and, the deepest lesson the evolution kept teaching, a preference for schemes where the dangerous case simply cannot be reached, which is what Ed25519 and BIP-340 buy you. Remove any one layer and a surface reopens.

The horizon does not change the shape of the problem, only its arithmetic: post-quantum signatures like ML-DSA and SLH-DSA will hand the same three surfaces to the same callers, at ten to a hundred times the signature size [@fips-204]. The nonce, the form, and the verifier will still be where the security lives.

A signature is only as safe as the nonce, the form, and the verifier the scheme hands you. The math held; the interface leaked.

Using a signature safely, in the end, is the discipline of never handing an attacker a value your scheme forgot to own.

The Doorway Into the Group: A Field Guide to Hashing a String Onto an Elliptic Curve (RFC 9380)

noreply@paragmali.com (Parag Mali) — Sun, 12 Jul 2026 15:35:47 GMT

**Turning a string into an elliptic-curve point is a two-decade research problem hiding behind a one-line API.** A safe map must produce a point whose discrete log is unknown, in time that does not depend on the input, with a distribution a proof can treat as a random oracle. The two obvious shortcuts each fail one of those tests: $H(m)\cdot G$ publishes the discrete log (forge BLS, break OPRFs), and try-and-increment leaks the input through timing (the Dragonblood attack read WPA3 passwords) [@rfc9380] [@vr20]. RFC 9380 (2023) finally delivers all three at once by hashing to two field elements, mapping each with a per-curve function, and adding the points [@rfc9380]. The standard is now quietly load-bearing under BLS on Ethereum, every elliptic-curve OPRF, OPAQUE, Privacy Pass, and WPA3's constant-time fix. This guide shows you why the naive maps fail, how the standard works, and exactly which suite to use for your curve and protocol.

1. They Timed the Handshake and Read the Wi-Fi Password

In April 2019, two researchers announced they could recover your Wi-Fi password without guessing it, phishing it, or breaking any cipher. They simply timed how long WPA3 took to turn the password into a point on an elliptic curve, and the clock spelled out the secret [@vr20] [@dragonblood-site]. The flaw was not in the encryption or the key exchange. It lived in a single, overlooked step that nearly every discrete-log protocol quietly leans on: the map that carries an arbitrary string into the group.

Get that one step wrong in the other obvious way, computing the point as $H(m)\cdot G$, and you leak a different secret entirely. Now anyone can forge the signatures that map was supposed to protect [@rfc9380] [@bls01]. Two shortcuts, two spilled secrets, one primitive. This is the field guide to getting it right.

A function that maps an arbitrary byte string to a point on an elliptic curve, such that the point's discrete logarithm (its relationship to the group generator) is unknown even to the party computing it. RFC 9380 standardizes the interface as `hash_to_curve` and `encode_to_curve` [@rfc9380].

The primitive looks trivial. Hash the input, get a point, move on. But "get a point" hides a genuine algorithm, because on an elliptic curve only about half of the field elements are valid $x$-coordinates. The two things every engineer tries first are the two things you must never ship, and each betrays a different secret. So before you write a single line, ask three questions.

Key idea: When you turn a string into a curve point, ask: (1) do you know the point's discrete log; (2) does your running time depend on the input; and (3) is the output distribution one your protocol's proof can treat as a random oracle? Every safe construction answers no, no, yes. Every named failure in this article is a wrong answer to exactly one of these.

WPA3-Personal answered the second question wrong. Its handshake, Simultaneous Authentication of Equals (SAE), derives a "password element" by hunt-and-peck: hash the password with a counter, test whether the result is a valid coordinate, and increment until it is [@rfc7664]. The number of iterations depends on the password, so the time the handshake takes is a function of the secret. Mathy Vanhoef and Eyal Ronen showed that an attacker within range can measure that timing and mount an offline dictionary attack, recovering the password without ever seeing a valid login [@vr20] [@dragonblood-site].

sequenceDiagram participant STA as Client (victim) participant AP as Access point participant Atk as Attacker (passive) Note over STA,AP: WPA3-SAE, password shared by both sides STA->>STA: Derive password element by hunt-and-peck Note over STA: Iteration count depends on the password STA->>AP: Send SAE Commit after the loop finishes Atk-->>STA: Measure time from trigger to Commit Note over Atk: Timing reveals the iteration count Atk->>Atk: Offline dictionary attack narrows candidates

RFC 9380, the standard that finally settled this primitive, is blunt about why try-and-increment is unsafe.

"The running time of this method, which is generally referred to as a probabilistic try-and-increment algorithm, depends on the input string. As such, it is not safe to use in protocols sensitive to timing side channels, as was exemplified by the Dragonblood attack." -- RFC 9380, Appendix A [@rfc9380]

The story got worse before it got better. The Wi-Fi Alliance's first attempt at a fix switched SAE to Brainpool curves, which reintroduced a leak: the encoder now had to loop to find a hash smaller than the prime, and the number of iterations depended on the password and the client's MAC address [@dragonblood-site].This second act is the whole lesson in miniature: constant time cannot be patched in curve by curve. The disclosure site concludes that "implementing Dragonfly and WPA3 without side-channel leaks is surprisingly hard" [@dragonblood-site]. Two rounds of the same class of bug, in the handshake now standard across WPA3 Wi-Fi.

Now hold that failure next to its mirror image. Suppose BLS signatures had mapped a message the naive way, $H(m) = H_n(m)\cdot G$ for a public hash $H_n$. A BLS signature is $\sigma = [sk]\cdot H(m)$, so it would equal $[sk]\cdot H_n(m)\cdot G = H_n(m)\cdot([sk]\cdot G) = H_n(m)\cdot PK$, where $PK$ is the public key [@rfc9380] [@bls01]. Every term on the right is public. Anyone forges any signature with no secret key at all. One shortcut leaked a low-entropy password through the clock; the other would hand out the signing key's power to the entire world. The primitive that sits between them is where the security of BLS, every elliptic-curve OPRF, and every password PAKE is actually won or lost.

This is the sibling of a claim Part 15 made about the curve itself: the elliptic-curve discrete-log problem was hard; the danger was the soft gap around it. Here the gap has a name and a shape. It is the doorway, the map that carries untrusted strings into the group. Get the doorway wrong and you publish the discrete log, leak the password through the clock, or hand your proof a distribution it never agreed to. To see why "hash to a point" is an algorithm and not a one-liner, we have to go back to the moment it stopped being free.

2. When Hashing to a Group Stopped Being Free

For a generation of cryptographers, "hash into the group" was not a problem worth naming. In a prime field $\mathbb{F}_p^$, the discrete-log setting Diffie and Hellman worked in, you hash your input to an integer and reduce it modulo $p$, and you are already inside the group. Every nonzero residue is a group element, so the map is a single modular reduction with nothing to prove.This is the free lunch elliptic curves took away. In $\mathbb{F}_p^$ the group is the set of nonzero field elements, so any nonzero hash output is already a valid element. On a curve, the group is a sparse subset of the plane, and most field elements are not on it.

Elliptic curves broke the deal. When Victor Miller and Neal Koblitz independently proposed elliptic-curve cryptography in 1985 and 1987, they replaced $\mathbb{F}_p^*$ with the points of a curve $y^2 = x^3 + ax + b$ [@miller85] [@koblitz87]. That change bought smaller keys and harder discrete logs, but it quietly created a new problem: for a random field element $x$, the quantity $x^3 + ax + b$ is a perfect square only about half the time, and only then is $x$ the coordinate of an actual point. "Hash to a group element" stopped being a reduction and became a search.

Nobody worried, because for a decade nobody needed to hash to a point. Ordinary ECDSA and ECDH hash to a scalar and multiply a fixed generator, which never leaves the safe world of exponents.Hashing to a scalar is trivial and always safe: any integer modulo the group order is a valid exponent. Hashing to a point is the hard problem, because most field elements are not on the curve.

Then, around the turn of the millennium, the demand arrived from two directions at once, and both directions needed a point.

A signature scheme by Boneh, Lynn, and Shacham built on a bilinear pairing. The signature on a message $m$ is $\sigma = [sk]\cdot H(m)$, where $H(m)$ is the message hashed *to a curve point* and $sk$ is the secret key. Verification checks a pairing equation. Security requires that $H(m)$ have unknown discrete log, or signatures become forgeable [@bls01].

The first direction was pairing-based cryptography. In 2001 Boneh, Lynn, and Shacham introduced BLS short signatures, whose entire construction rests on hashing the message to a curve point of unknown discrete log, because the signature is that point scaled by the secret key [@bls01]. The same year, Boneh and Franklin built identity-based encryption on the identical primitive, hashing an identity string to a point [@bf01]. BLS shipped with the first answer anyone reaches for: MapToGroup, a try-and-increment loop [@bls01]. Boneh and Franklin instead gave a deterministic map, but only for certain supersingular curves over fields of characteristic $p \equiv 2 \pmod 3$, too narrow a class to serve as general infrastructure [@bf01].

The second direction was password-authenticated key exchange. In 2008 Dan Harkins proposed Simultaneous Authentication of Equals, a balanced PAKE that maps a password to a group element and then runs a zero-knowledge proof of it [@sae08]. To do the mapping it introduced hunt-and-peck, the same try-and-increment idea aimed at a secret input. SAE was standardized as RFC 7664 in 2015, including its "Hunting and Pecking with ECC Groups" derivation, and then shipped to consumers in 2018 as the core of WPA3-Personal [@rfc7664] [@vr20].

Note: Pairing signatures and password PAKEs look unrelated, but both need the same thing: a hash whose output is a curve point of unknown discrete log. Pairing crypto needs it so a signature can be that point; a PAKE needs it so the password element hides the password. The primitive was born load-bearing under two families of protocols in the same decade [@bls01] [@sae08].

A scoping note worth keeping honest: not every PAKE maps a password to a curve point. SPAKE2 uses fixed, independently generated group elements $M$ and $N$ rather than hashing the password onto the curve, and SRP works in an integer group, not on an elliptic curve at all. The "everything depends on hash-to-curve" claim is precise, not universal, and it applies to the named constructions in this article: SAE, CPace, OPAQUE's OPRF, the RFC 9497 OPRFs, and BLS.The lesson for a field guide: SPAKE2 and SRP are the counterexamples that keep "all PAKEs need hash-to-curve" from being true, and they stop you hunting for the primitive where it is not.

So by 2008 the problem was named and both naive answers were on the table, shipped in real systems: $H(m)\cdot G$ carried the finite-field intuition onto curves, and try-and-increment searched for a valid coordinate. Each was correct in the sense that it produced a usable point. Each was also a disaster, and for opposite reasons. To see exactly what each one leaks, we have to look at the two shortcuts side by side.

3. Two Shortcuts, Two Spilled Secrets

Both obvious maps are correct in the ordinary sense: they produce a point on the curve, quickly, with a few lines of code. That is exactly what makes them dangerous. Each satisfies most of the three properties and fails precisely one, and the property each fails is different, which is why they leak different secrets.

Shortcut A: compute the point as a known multiple of the generator

Hash the message to an integer $H_n(m)$ and scalar-multiply the generator: $P = H_n(m)\cdot G$. The output is uniform on the group, costs one scalar multiplication, and is trivially constant-time. It assumes nothing about the curve. It is the finite-field intuition carried straight onto the curve, and it violates Property 1: whoever computed $P$ knows its discrete log, because they chose it.

RFC 9380 does not hedge about the consequence.

"The resulting point has a known discrete log relationship to $P$. Thus, except in cases where this method is specified by the protocol, it must not be used; doing so risks catastrophic security failures." -- RFC 9380, Appendix A [@rfc9380]

"Catastrophic" is exact, not rhetorical. Take BLS. The public key is $PK = [sk]\cdot G$, and a signature is $\sigma = [sk]\cdot H(m)$. If the map were $H(m) = [H_n(m)]\cdot G$ for a public hash $H_n$, then

$$\sigma = [sk]\cdot H(m) = [sk]\cdot[H_n(m)]\cdot G = [H_n(m)]\cdot([sk]\cdot G) = [H_n(m)]\cdot PK.$$

Every quantity on the right -- $H_n(m)$ from the public message, $PK$ from the wire -- is public. Anyone computes the valid signature on any message directly, with no access to $sk$ [@rfc9380] [@bls01]. This is not an edge case or a timing quirk. It is total, structural forgeability on every input. The same knife cuts elsewhere: for an OPRF, knowing the discrete log of the mapped input breaks obliviousness; for a PAKE, a password element with known discrete log no longer hides the password.

{` // ILLUSTRATION, NOT AN IMPLEMENTATION. // We model the group as integers mod a small prime order n, with // "scalar mult" [k]*P written as (k * P) mod n. Discrete log is trivial // here on purpose -- the point is the STRUCTURE of the forgery, not hardness. function scalarMul(k, P, n) { return ((k % n) * (P % n)) % n; }

const n = 97; // toy prime group order const G = 1; // generator const sk = 42; // signer's SECRET key const PK = scalarMul(sk, G, n); // public key = [sk]*G

// The UNSAFE map: H(m) = [Hn(m)]*G, so its discrete log Hn(m) is public. const Hn_m = 7; // public hash of the message const Hm = scalarMul(Hn_m, G, n); // the mapped point (known DL!)

// Honest signer computes sigma = [sk]*H(m) const legitSig = scalarMul(sk, Hm, n);

// Attacker never sees sk. Uses only Hn(m) and PK: const forgedSig = scalarMul(Hn_m, PK, n); // [Hn(m)]*PK

console.log("legit signature :", legitSig); console.log("forged signature:", forgedSig, "(computed with NO secret key)"); console.log("forgery succeeds:", legitSig === forgedSig); `}

Shortcut B: search for a valid coordinate

The other reflex fixes Property 1 and breaks Property 2. Hash the input together with a counter, test whether the candidate is a valid $x$-coordinate, and loop until it is.

A map that hashes the input with a counter to a candidate $x$, tests whether $x^3 + ax + b$ is a quadratic residue (a valid coordinate), and increments the counter until it is. A random $x$ works about half the time, so the expected number of tries is two [@rfc9380]. The output has *unknown* discrete log, but the number of iterations depends on the input.

Because nobody chose the output as a multiple of $G$, its discrete log is genuinely unknown, so Property 1 holds. That is exactly why BLS and SAE could ship with it and be functionally correct. The flaw is Property 2: the loop count is a function of the input, so the running time is too. When the input is a secret -- a password, a pre-shared key -- the clock becomes an oracle. That is Dragonblood [@vr20] [@rfc9380].

{` // ILLUSTRATION, NOT AN IMPLEMENTATION. // Count hunt-and-peck iterations for several secret inputs over a toy field. // A constant-time map would take the SAME work for every input; this does not. const p = 103; // toy prime, p = 3 mod 4 const a = 2, b = 3; // toy curve y^2 = x^3 + a x + b

// A stand-in for H(input || ctr): deterministic, input-dependent pseudo-x. function candidateX(seed, ctr, p) { return ((seed * 131 + ctr * 419 + 17) % p + p) % p; }

function iterations(seed) { for (let ctr = 1; ctr < 200; ctr++) { const x = candidateX(seed, ctr, p); const g = (((x * x % p) * x % p) + a * x + b) % p; if (isSquare(g, p)) return ctr; } return -1; }

for (const pw of [11, 23, 37, 58, 71, 90]) { console.log("secret input " + pw + " -> " + iterations(pw) + " iterations"); } console.log("Different inputs, different work: that difference is the leak."); `}

Here is the table that makes the whole article click. Two maps, three properties, and each map fails a different one.

Map	Unknown discrete log (Prop 1)	Constant time (Prop 2)	Right distribution (Prop 3)	The secret it spills
$H(m)\cdot G$	Violated (DL is $H_n(m)$)	Satisfied	Satisfied (uniform)	The signing key's power -- universal forgery [@rfc9380] [@bls01]
Try-and-increment	Satisfied	Violated (loop depends on input)	Roughly satisfied	The secret input -- via handshake timing [@rfc9380] [@vr20]

Try-and-increment is only a *vulnerability* when the mapped input is secret. A BLS signature over a *public* message runs the same variable-time loop, but the message is public, so the timing reveals nothing an attacker did not already have. The Dragonblood break works because a Wi-Fi password is a low-entropy secret. This is why the accuracy-critical phrasing is "variable time is dangerous when the input is secret," never "variable time is always dangerous" [@rfc9380] [@vr20].

Note: If the input to your map is a password, a PSK, or any low-entropy secret, a data-dependent loop over it is a timing oracle. Use a constant-time map. RFC 9380 declines to even specify try-and-increment "because the goal is to specify algorithms that can plausibly be computed in constant time" [@rfc9380].

So the requirement is now sharp, and it looks almost contradictory. You need a map whose output has unknown discrete log, like try-and-increment, but whose running time is fixed, unlike try-and-increment. You need the safety of the search without the search. For a few years it was not obvious such a thing existed. The story of how the field built it, one property at a time, is the evolution of the primitive itself.

4. Moving Each Property Into the Curve

The next thirteen years read like a chain of defenses in which the defenders keep advancing. Each generation takes one property that used to depend on the implementer's care and locks it into the construction, where it holds by mathematics rather than by discipline. Three of those steps are true property wins. The rest are the engineering that made the winning map cheap enough and general enough to run on every curve real protocols use. Being honest about which is which is the difference between a clean story and a true one.

The first constant-time map: Shallue-van de Woestijne (2006)

The contradiction from the last section -- unknown discrete log without a search -- was resolved in 2006 by Andrew Shallue and Christiaan van de Woestijne. Their map is the "SW" in "SSWU." Given a field element $u$, algebraic identities guarantee that among a small fixed set of candidate $x$-values derived from $u$, at least one is a valid coordinate [@sw06]. So you compute the candidates, test them in a fixed order with a constant number of operations, and select a valid one. There is no rejection loop, which means no data-dependent branch, which means constant time by construction [@sw06] [@rfc9380]. The discrete log stays unknown. For the first time, a single map was simultaneously unknown-DL and constant-time on almost any curve with characteristic other than 2 or 3.

The idea had roots. Schinzel and Skalba had constructed curve points deterministically for a restricted class of curves in 2004, and Skalba generalized it in 2005; Shallue and van de Woestijne further generalized and simplified that line [@ss04] [@s05] [@rfc9380]. Boneh and Franklin's 2001 supersingular map was a deterministic co-origin, but too narrow to be general infrastructure [@bf01]. SvdW survives today in RFC 9380 as the universal fallback for any curve the specialized maps do not cover [@rfc9380].

But SvdW carried a flaw that every deterministic map after it would share, and it is the one no amount of cleverness in a single map can dodge.

The wall every single map hits

RFC 9380 states the obstacle plainly: the deterministic maps "map to some fraction of the points" on the curve, "not the entire curve," which "means that they cannot be used directly to construct a random oracle that outputs points on the curve" [@rfc9380]. A single SvdW map hits only a constant fraction of the points and is statistically distinguishable from uniform. Any proof that models the map as a random oracle -- BLS's unforgeability, an OPRF's pseudorandomness, a password mapping -- does not go through. This is Property 3, and it is missing.

A formal criterion, defined by Maurer, Renner, and Holenstein in 2004, for when a constructed function can safely stand in for an ideal object such as a random oracle inside a security proof. A hash-to-curve construction is "indifferentiable from a random oracle" when no efficient distinguisher can separate it from a true random oracle, even given access to the underlying building blocks [@mrh04].

Before that wall was scaled, the field spent a few years making the map cheaper.

Cheaper maps: Ulas, Icart, Simplified SWU (2007-2010)

Ulas simplified SvdW in 2007 (the "U" in SWU) [@u07]. Icart gave an independent closed-form map in 2009 for fields with $p \equiv 2 \pmod 3$, built on a single cube root [@icart09]. Brier and coauthors published the "Simplified SWU" map in 2010 for fields with $p \equiv 3 \pmod 4$ [@bcimrt10]. Each is a fixed rational formula costing roughly one field exponentiation plus a few multiplications and one inversion -- cheaper and easier to implement in constant time than SvdW's heavier parameterization.

A closed-form rational map from a field element to a short-Weierstrass curve point $y^2 = x^3 + Ax + B$, descended from the Shallue-van de Woestijne map via Ulas and Brier et al. In its base form it applies when $p \equiv 3 \pmod 4$ and $A\cdot B \neq 0$, and costs about one square root plus constant-time conditional selects [@u07] [@bcimrt10] [@rfc9380].

This was progress, but not on Property 3. Cheaper did not mean uniform: each remained a single map hitting a fraction of the curve, still unusable as a random oracle [@rfc9380]. And the modular conditions left gaps. Icart needs $p \equiv 2 \pmod 3$; Simplified SWU needs $p \equiv 3 \pmod 4$; neither covers curves with $A = 0$ such as secp256k1 and BLS12-381. The deeper question was sharper and universal: could you build something provably indifferentiable out of these non-uniform pieces? The answer arrived, remarkably, in the very same 2010 paper that gave Simplified SWU.

The breakthrough: add two maps (2010)

Here is the shift the whole standard turns on. A single deterministic map cannot be a random oracle. But the sum of two independent maps can be. Hash the message to two field elements, map each to a point, and add the points:

$$H(m) = f\big(H_0(m)\big) + f\big(H_1(m)\big).$$

Brier, Coron, Icart, Madore, Randriam, and Tibouchi proved this construction indifferentiable from a random oracle, and it recovers Property 3 while keeping Properties 1 and 2 [@bcimrt10] [@mrh04]. All three properties, at last, guaranteed at once.

Key idea: A single deterministic map can never be a random oracle: it reaches only a fraction of the curve and is distinguishable from uniform. The fix is structural, not a cleverer single map. Hash to two field elements, map each, and add the two points. The sum is indifferentiable from a random oracle. This is exactly why hash_to_curve (the _RO_ suites) costs twice what encode_to_curve (the _NU_ suites) costs: the second map is not waste, it is what buys the distribution [@bcimrt10] [@ffstv13] [@rfc9380].

flowchart TD M["msg and DST"] --> HF["hash_to_field to two elements"] HF --> Q0["map_to_curve of u0 gives Q0"] HF --> Q1["map_to_curve of u1 gives Q1"] Q0 --> ADD["add points: Q0 plus Q1"] Q1 --> ADD ADD --> RO["sum is indifferentiable from a random oracle"] RO --> CC["clear_cofactor"] CC --> P["point in the prime-order subgroup"]

Precision matters here, because it is easy to over-credit. Brier et al. proved the sum-of-two construction indifferentiable specifically for Icart's map. The general result -- that summing two copies of essentially any deterministic map yields a random oracle -- is due to Farashahi, Fouque, Shparlinski, Tibouchi, and Voloch in 2013.Attribute the general theorem to FFSTV13, not to BCIMRT. RFC 9380's Appendix A is careful about this: BCIMRT proved it for Icart's method; FFSTV13 "improve the analysis, showing that it applies to essentially all deterministic mappings" [@rfc9380] [@ffstv13]. After 2010, nothing about Property 3 was left to invent. What remained was coverage: the sum-of-two construction needs a concrete constant-time inner map for your curve family, and two families were still uncovered.

Covering the remaining curves: Elligator 2 (2013) and isogeny SSWU (2019)

The Montgomery and twisted-Edwards curves -- Curve25519, Curve448, and the Edwards forms behind modern signature and OPRF suites -- needed their own map, because SSWU targets short-Weierstrass curves. Elligator 2 supplied it.

An efficient, invertible map from a field element to a point on any curve that has a point of order 2 (Montgomery or twisted-Edwards form), introduced by Bernstein, Hamburg, Krasnova, and Lange in 2013. RFC 9380 uses it for the Curve25519/448 and edwards25519/448 families and the ristretto255/decaf448 groups [@bhkl13] [@rfc9380].

Elligator's origin is a small marvel. It was built for censorship resistance, a way to make a public key look like uniform random bytes so a network censor cannot even tell a key exchange is happening.The elligator.org site records the pivot in its own words: a number of protocols "need a way to map an arbitrary input to an elliptic curve point whose factorisation is unknown ('hashing to an elliptic curve'), such as verifiable random functions (VRFs) and oblivious pseudorandom functions (OPRFs)." A steganography tool became hash-to-curve infrastructure [@elligator-site]. The tool built to hide a key exchange became the constant-time inner map for an entire curve family's hashing.

The last gap was the pairing-friendly curve BLS needs. BLS12-381 has $A = 0$, exactly where Simplified SWU does not apply. In 2019 Riad Wahby and Dan Boneh closed it: map with SSWU onto an isogenous curve where the parameters are admissible, then push the point back to the target curve through a fixed isogeny of small degree [@wb19] [@rfc9380]. The result is constant-time and works in any characteristic, and it is what made constant-time hashing to BLS12-381 practical at blockchain scale. Their headline measurement is the number that ends an old excuse: the constant-time functions run "within 9% of the fastest, non-constant-time alternatives" [@wb19].

Generation	Year	Key idea	Time	What it secured	Status
$H(m)\cdot G$	~2001	scalar-mult the generator	constant	nothing (DL is known)	never safe as a map [@rfc9380]
Try-and-increment	2001-2008	search for a valid coordinate	variable	unknown discrete log	deprecated for secret inputs [@bls01] [@rfc9380]
SvdW	2006	rational parameterization, no loop	constant	constant time by construction	general fallback [@sw06]
Ulas / Icart / Simplified SWU	2007-2010	cheaper closed-form maps	constant	efficiency, per-curve forms	inner maps [@u07] [@icart09] [@bcimrt10]
BCIMRT sum-of-two	2010	add two mapped points	constant	random-oracle distribution	the `_RO_` core [@bcimrt10] [@ffstv13]
Elligator 2	2013	invertible map for order-2 curves	constant	Montgomery/Edwards coverage	active [@bhkl13]
Isogeny SSWU	2019	map via an isogenous curve	constant	$A=0$ pairing-curve coverage	active [@wb19]

Every curve family that deployed protocols use now has a constant-time, unknown-DL inner map, and the sum-of-two construction wraps any of them into a random oracle. All three properties are achievable everywhere. But an implementer still has to assemble the hashing, the map, the cofactor clearing, the domain separation, and the choice of variant, by hand, and any one of those is a place to slip. What packages the whole lineage so you cannot get it wrong?

5. RFC 9380: All Three Properties at Once

In August 2023 the CFRG published RFC 9380, "Hashing to Elliptic Curves," by Armando Faz-Hernandez, Sam Scott, Nick Sullivan, Riad Wahby, and Christopher Wood [@rfc9380]. It invents no new map. It does something more useful: it folds two decades of maps into a single interface an ordinary engineer cannot easily misuse, and it makes every security-relevant choice legible in the name.

The pipeline has three stages [@rfc9380]:

hash_to_field turns the input and a domain tag into one or more field elements, using expand_message_xmd for fixed-output hashes like SHA-256 and expand_message_xof for extendable-output functions like SHAKE.
map_to_curve sends each field element to a curve point using the map that fits the curve: Simplified SWU for short-Weierstrass curves, isogeny-SSWU for the $A = 0$ pairing curves, Elligator 2 for Montgomery and Edwards curves, and SvdW as the general fallback.
clear_cofactor forces the result into the prime-order subgroup.

flowchart TD IN["input msg and DST"] --> HF["hash_to_field"] HF --> BR{"which variant"} BR -->|"_RO_ hash_to_curve"| TWO["two field elements, two maps, add"] BR -->|"_NU_ encode_to_curve"| ONE["one field element, one map"] TWO --> CC["clear_cofactor"] ONE --> CC CC --> OUT["point in the prime-order subgroup"]

The two-element margin is a proven bound

The first stage hides a subtlety worth pausing on. Reducing a uniformly random integer modulo $p$ is not exactly uniform, because the top of the range wraps around. RFC 9380 fixes this by oversampling.

The first pipeline stage: expand the input and the domain tag into field elements. It draws $L = \lceil(\lceil\log_2 p\rceil + k)/8\rceil$ bytes per element and reduces modulo $p$, so each element is within statistical distance $2^{-k}$ of uniform. The standard suites set $k = 128$, giving a $2^{-128}$ margin [@rfc9380].

That $2^{-128}$ is a proven statistical bound, not a hopeful heuristic, and it is exactly why the byte-length formula is what it is [@rfc9380].

The variant is in the name

The choice between a uniform, proof-usable point and a cheaper nonuniform one is encoded directly in the suite name. hash_to_curve, the _RO_ family, runs the two-element sum-of-two construction and is indifferentiable from a random oracle. encode_to_curve, the _NU_ family, runs a single map at about half the cost and produces a nonuniform distribution [@rfc9380]. The trailing tag is not decoration; it tells you whether the output is safe to feed a proof that assumes a random oracle.

RFC 9380 suite identifiers follow the pattern CURVE_XMD:HASH_MAP_VARIANT_. Once you can read one, you can read them all.

Field	In `BLS12381G2_XMD:SHA-256_SSWU_RO_`	What it fixes
Curve / group	`BLS12381G2`	the target group: BLS12-381, subgroup G2
Expander	`XMD`	`expand_message_xmd` (a fixed-output hash)
Hash	`SHA-256`	the underlying hash function
Inner map	`SSWU`	Simplified SWU (isogeny variant here, since $A=0$)
Variant	`RO`	`_RO_`: uniform, indifferentiable from a random oracle

Read edwards25519_XMD:SHA-512_ELL2_RO_ the same way and it decodes itself: the edwards25519 group, XMD expansion over SHA-512, the Elligator 2 map, uniform variant [@rfc9380]. The name is the spec.

{` // Decompose an RFC 9380 suite ID: CURVE_EXPANDER:HASH_MAP_VARIANT_ function parseSuite(id) { const [left, right] = id.split(":"); const leftParts = left.split(""); const expander = leftParts.pop(); // XMD or XOF const curve = leftParts.join(""); // e.g. BLS12381G2, edwards25519 const rp = right.split("").filter(function (s) { return s.length > 0; }); const hash = rp[0]; // SHA-256, SHA-512, SHAKE256 const variant = rp[rp.length - 1]; // RO or NU const map = rp.slice(1, rp.length - 1).join(""); // SSWU, ELL2, SVDW return { curve: curve, expander: expander, hash: hash, map: map, variant: variant, uniform: variant === "RO" ? "uniform (random oracle)" : "nonuniform" }; }

["BLS12381G2_XMD:SHA-256_SSWU_RO_", "edwards25519_XMD:SHA-512_ELL2_RO_", "P256_XMD:SHA-256_SSWU_NU_"].forEach(function (id) { console.log(id); console.log(" ", JSON.stringify(parseSuite(id))); }); `}

The two steps you must not skip

Two pipeline stages are easy to forget and dangerous to omit. The first is the domain tag.

A byte string, unique to a protocol and its version, mixed into `hash_to_field` so the same input maps to different points in different protocols. RFC 9380 makes it mandatory; DSTs longer than 255 bytes are themselves hashed before use. Without a unique DST, two protocols that share a curve can be driven to collide [@rfc9380].

The second is cofactor clearing, the final stage that lands the point where the group arithmetic is safe.

The final pipeline stage: multiply the mapped point (or apply a faster endomorphism-based equivalent) by the curve's cofactor to force it into the prime-order subgroup, closing off small-subgroup attacks. edwards25519 has cofactor 8; BLS12-381's G1 and G2 have large cofactors with fast clearing methods [@rfc9380].

Note: The everyday act is now tiny: name a suite. The suite ID fixes the curve, the expander, the hash, the inner map, and the distribution variant, and a vetted library does the rest. You are not choosing a construction. You are declaring your curve and your distribution need, and RFC 9380 has already made the safe choice for each [@rfc9380].

Step back and the three properties resolve into one sentence. The mapped point had to be a stranger -- its discrete log unknown, or forgery follows. It had to arrive on time -- in work independent of the input, or the clock leaks the secret. And it had to look the part -- drawn from a distribution the proof can treat as a random oracle, or the proof does not hold.

The point had to be a stranger, arrive on time, and look the part. RFC 9380 is the standard that guarantees all three, and encodes which ones you asked for in a single suite name.

That framing also closes the loop this article opened. WPA3's answer to Dragonblood is Hash-to-Element, or H2E.

After Dragonblood, WPA3-Personal replaced hunt-and-peck with H2E, a constant-time, SSWU-based derivation of the SAE password element, and the Wi-Fi standard was updated to compute the password element "in constant time" [@dragonblood-site]. RFC 9380 supplies the governing principle: a constant-time map is required when the input is secret [@rfc9380]. The hostap and w1.fi advisories track the corresponding constant-time SAE patches shipped to real deployments [@w1fi-security]. The precise IEEE 802.11 clause is not reproduced here; the point is that the fix is exactly the constant-time approach this standard consolidates.

The doorway is finally built to spec. The natural next question is who is actually walking through it, because the answer is: far more of your daily software than you would guess.

6. Who Quietly Depends on This Today

You have almost certainly relied on hash-to-curve this week without noticing. It sits under signatures that finalize blockchains, under the private authentication that hides your browsing from a token issuer, under the password login protocol that never sees your password, and under WPA3. None of these advertise the primitive. All of them break if it is wrong.

flowchart TD BLS["BLS signatures on Ethereum"] --> H2C["RFC 9380 hash-to-curve"] OPRF["OPRF and VOPRF (RFC 9497)"] -->|"one HashToGroup call"| H2C OPAQUE["OPAQUE aPAKE (RFC 9807)"] -->|"indirect"| OPRF PP["Privacy Pass (RFC 9578)"] --> OPRF CPace["CPace PAKE (draft)"] --> H2C SAE["WPA3-Personal SAE"] -->|"H2E"| H2C

BLS signatures and Ethereum consensus

The highest-volume deployment, by signature-verification throughput, is BLS. Every BLS signature hashes its message to a point in G2 and scales it by the secret key, so hash_to_point is not an accessory but the heart of the scheme [@draft-bls-signature-07]. Ethereum's consensus specification fixes this for validator signatures: it adopts the suite BLS12381G2_XMD:SHA-256_SSWU_RO_ -- the isogeny-SSWU map wrapped in the _RO_ construction -- so every consensus client hashes exactly this way [@eth-consensus-specs] [@draft-bls-signature-07] [@rfc9380]. The _RO_ variant is not optional here: the unforgeability proof models the map as a random oracle, so a nonuniform _NU_ map would void the guarantee. Notably, the scheme the entire Ethereum validator set relies on is still specified only by an Internet-Draft, which the consensus spec references directly [@eth-consensus-specs] [@draft-bls-signature-07].Unlike ECDSA, BLS is deterministic and uses no per-signature nonce, so it sidesteps the nonce-reuse failures that have repeatedly broken ECDSA deployments (Part 8). Its one hash-to-curve call becomes the security-critical primitive in place of the nonce [@draft-bls-signature-07].

OPRFs, and everything built on them

The second large consumer is the oblivious pseudorandom function and its verifiable and partially-oblivious variants, standardized in RFC 9497 [@rfc9497].

A two-party protocol in which a client learns $F(k, x)$ for its input $x$ and the server's key $k$, while the server learns neither $x$ nor the result. The client's first step hashes $x$ to a curve point and blinds it; that single `HashToGroup` call is the protocol's entire hash-to-curve surface [@rfc9497].

The dependency is remarkably concentrated. The client's Blind(input) computes HashToGroup(input) and then blinds the point; every other message is scalar arithmetic and proofs. So the whole hash-to-curve exposure of an OPRF is one call, instantiated with Elligator 2 for ristretto255 and SSWU for the P-256, P-384, and P-521 suites [@rfc9497].

Note: For an OPRF, the entire hash-to-curve attack surface is the single HashToGroup call in the client's Blind step. Get that one call right -- correct suite, correct DST, identity element rejected on the wire -- and the primitive is handled. Get it wrong and obliviousness or correctness collapses [@rfc9497].

Because OPRFs are a building block, their hash-to-curve dependency propagates. OPAQUE, the augmented PAKE standardized as RFC 9807 in July 2025, lets a server authenticate a user by password without ever learning the password, and it does so by running an OPRF; its hash-to-curve dependency is indirect but real, flowing through RFC 9497 [@rfc9807] [@rfc9497]. Privacy Pass, standardized across RFC 9576, RFC 9577, and RFC 9578 in June 2024, issues anonymous authentication tokens; its privately verifiable issuance is "based on the Oblivious Pseudorandom Function," with token type 0x0001 being a VOPRF over P-384, which is the SSWU path [@rfc9578] [@rfc9576] [@rfc9577].

PAKEs and Wi-Fi

Two more consumers map a secret rather than a public message. CPace, a balanced PAKE still in CFRG draft with datatracker status "Waiting for IRTF Chair," maps the password to a session generator using SSWU or Elligator 2 depending on the group [@draft-cpace]. And WPA3-Personal's SAE now derives its password element with the constant-time H2E map, the fix that closed Dragonblood [@dragonblood-site].

The whole deployed picture fits in one adoption map.

Consumer	Where hash-to-curve is called	Suite / map	Standard status
BLS / Ethereum consensus	`hash_to_point(msg)` into G2	`BLS12381G2_XMD:SHA-256_SSWU_RO_` [@draft-bls-signature-07]	Internet-Draft [@draft-bls-signature-07]
OPRF / VOPRF / POPRF	`Blind` calls `HashToGroup`	ristretto255 to Elligator 2; P-curves to SSWU [@rfc9497]	RFC 9497 (2023) [@rfc9497]
OPAQUE (aPAKE)	indirect, via its OPRF	inherits the OPRF suite [@rfc9807]	RFC 9807 (2025) [@rfc9807]
Privacy Pass	VOPRF token issuance	P-384 VOPRF to SSWU [@rfc9578]	RFC 9576/9577/9578 (2024) [@rfc9578]
CPace (balanced PAKE)	password to session generator	SSWU or Elligator 2 by group [@draft-cpace]	draft, "Waiting for IRTF Chair" [@draft-cpace]
WPA3-Personal (SAE)	password element (PWE)	H2E, constant-time SSWU [@dragonblood-site]	IEEE 802.11

What actually ships

Under all of this sit a few load-bearing libraries. The blst library implements BLS12-381 hash-to-curve in C and assembly, is compliant with RFC 9380, was audited by NCC Group in January 2021, and is the subject of an ongoing Galois formal-verification effort [@blst] [@blst-verification]. Cloudflare's CIRCL provides RFC 9380 hash-to-curve and RFC 9497 OPRFs in Go, though it labels itself experimental and cautions against production use of parts of it [@circl]. In Rust, curve25519-dalek exposes an Elligator-based one-way map through RistrettoPoint::from_uniform_bytes [@curve25519-dalek].

The headline is quiet by design: the primitive is standardized and load-bearing under a stack of shipping protocols, and almost nobody who depends on it has to think about it. That is what a well-built doorway looks like. Which raises a fair question: if every consumer just names a suite, is there ever a real choice to make, or has the standard decided everything for you?

7. The Decision Matrix

There is no "best" hash-to-curve, and that is a feature. Almost every knob is set for you by something structural: the curve you already committed to, or the distribution your proof already assumes. The value of a field guide is naming which axis governs which choice, and where a wrong turn is a vulnerability rather than a mere preference.

Distribution need decides the variant

The _RO_ versus _NU_ choice is governed by one question: does your proof need a random oracle? For BLS, OPRFs, and password mapping, it does, so _RO_ is the default. The _NU_ variant is an optimization you reach for only with a specific proof that non-uniformity is acceptable, because its output is measurably not uniform.

Dimension	`hash_to_curve` (`_RO_`)	`encode_to_curve` (`_NU_`)
Field elements and maps	2 elements, 2 maps, 1 addition	1 element, 1 map
Relative cost	about twice `_NU_`	baseline, about half of `_RO_`
Output distribution	indifferentiable from a random oracle [@rfc9380]	nonuniform: at least one eighth of G, weights vary by up to a factor of four [@rfc9380]
Proof compatibility	usable where a proof assumes a random oracle	only where non-uniformity is provably sufficient
Decision rule	default for BLS, OPRF, PAKE mapping	optimization only, with a proof

Getting this backward is a shippable misuse: a _NU_ map where the proof assumed _RO_ produces a point from a distribution the proof never agreed to, which is the third of our three failures made concrete [@rfc9380].

Curve family decides the map

The choice among the inner maps is not a matter of taste at all. It is fixed by the shape of your curve.

flowchart TD Q{"what curve do you have"} Q -->|"short Weierstrass, p is 3 mod 4"| SSWU["Simplified SWU"] Q -->|"A is zero: secp256k1, BLS12-381"| ISO["Isogeny SSWU"] Q -->|"Montgomery or Edwards"| ELL["Elligator 2"] Q -->|"none of the above"| SVDW["SvdW fallback"]

Map	Curve condition	Curves in RFC 9380	Dominant cost
Simplified SWU	short-Weierstrass, $p \equiv 3 \pmod 4$, $AB \neq 0$	P-256, P-384, P-521	about one square root plus selects
Isogeny SSWU	$A = 0$ Weierstrass	secp256k1, BLS12-381 G1/G2	SSWU on $E'$ plus one isogeny evaluation
Elligator 2	curve with a point of order 2	curve25519/448, edwards25519/448, ristretto255/decaf448	about one exponentiation
SvdW	any curve, characteristic other than 2 or 3	fallback when none of the above fit	a few exponentiations and inversions

Note: If you find yourself deliberating between SSWU and Elligator 2, stop: your curve has already decided. SSWU serves short-Weierstrass curves, isogeny-SSWU serves the $A=0$ pairing curves, and Elligator 2 serves Montgomery and Edwards curves. The only choice genuinely left to you is the variant, _RO_ or _NU_, and the default is _RO_ [@rfc9380].

The two knobs that are real, but small

Two remaining choices are genuine but low-stakes. The expander, expand_message_xmd versus expand_message_xof, follows your hash family: XMD for SHA-2, XOF for SHAKE, and neither dominates on security, so you pick the hash you already have [@rfc9380]. And within BLS, you can hash to either pairing group.

BLS can hash to either pairing group. Hashing to G2 is more expensive, because G2 lives over the larger field $\mathbb{F}_{p^2}$ and has a larger cofactor, but the minimal-pubkey-size setting Ethereum uses signs in G2. Minimal-signature-size deployments hash to the cheaper G1 instead. This is a real cost-versus-size trade-off, not a security decision, and it is why fast G2 hashing has its own literature [@rfc9380] [@sbcdk09].

The number that ends an old argument

For years, the excuse for variable-time hunt-and-peck was that constant-time maps were too slow. Wahby and Boneh's nine-percent measurement retired it: constant time is essentially free here, and simpler to implement correctly, which is why RFC 9380 can require it without asking anyone to pay for the safety [@wb19].

It is worth remembering how strange this matrix would look to a cryptographer of 1995. In a finite field, hashing into the group is a single modular reduction with no decision matrix at all [@miller85]. Elliptic curves bought hard discrete logs and small keys, and the bill came due as this exact table: a real algorithm, with real parameters, that you must get right. Two maps, an oversampling margin, a distribution that is deliberately "almost uniform": none of it is arbitrary. The next section proves they are the right choices, not just good ones.

8. Why Two Maps, and How Uniform Is "Uniform"

The two-map construction can feel like a clever hack. It is not. It is the provable optimum, and the theory that says so also explains the oversampling margin and the one caveat you must respect.

The lower bound that forces two maps

Start with what a single map cannot do. No efficient deterministic map to a general elliptic curve is both surjective and uniform. Its image is only a constant fraction of the curve, with some points weighted more heavily than others. RFC 9380 quantifies the residual for the single-map case exactly: the output distribution "includes at least one eighth of the points in G," and the probability of points "varies by at most a factor of four" [@rfc9380]. Those bounds descend from the analyses of the individual maps [@sw06] [@bhkl13] [@ft12]. A distribution that misses seven-eighths of the group in the worst case and re-weights the rest simply cannot be indifferentiable from a uniform random oracle. One map is not enough, and this is why.

The matching upper bound

Now the other side. Hashing to two field elements, mapping each, and adding the points is indifferentiable from a random oracle, proven for Icart's map by Brier et al. and generalized to essentially all maps by Farashahi et al. [@bcimrt10] [@ffstv13] [@mrh04]. The lower bound says one map cannot suffice; the upper bound says two do. The gap is closed. The _RO_ construction is the tight, achievable answer to "produce a uniform, proof-usable point," and RFC 9380 standardizes precisely that optimum, no more and no less [@rfc9380].

Why the oversampling margin is what it is

The same "prove it, do not hope it" discipline governs hash_to_field. Reducing a uniform $L$-byte integer modulo $p$ leaves a small bias from the wrap-around at the top of the range. Oversampling by $k$ bits, with

$$L = \left\lceil \frac{\lceil \log_2 p \rceil + k}{8} \right\rceil,$$

drives the statistical distance from uniform below $2^{-k}$. RFC 9380 fixes $k = 128$, so every field element is within $2^{-128}$ of uniform [@rfc9380]. That is a proven bound, which is why truncating the byte length is not a harmless optimization but a real weakening of the margin.

The one caveat that survives all of this

Here the humility arrives. "Indifferentiable from a random oracle" does not mean "safe to drop into any random-oracle proof."

RFC 9380 cites Ristenpart, Shacham, and Shrimpton for the sharpest limit on the whole approach: "not all security proofs that rely on random oracles continue to hold when those oracles are replaced by indifferentiable functionalities" [@rfc9380]. So `_RO_` is the right default, but a protocol's proof still has to be the composable kind that survives the substitution. Indifferentiability is a strong guarantee, not an unconditional one.

There is a smaller but instructive caveat in the same spirit. Either encoding can, with probability roughly $1/r$ for a random input, output the identity element.That probability is negligible and it is infeasible to find such an input deliberately, so RFC 9380 says protocols "SHOULD NOT add a special case to detect and 'fix' the identity element." The special case is itself a bug surface, more likely to introduce a flaw than to prevent one [@rfc9380]. The right move is to do nothing special, which is counterintuitive enough that people get it wrong by trying too hard.

Note: For classical elliptic curves the core problem is solved in the strongest sense available: a single map provably cannot be a uniform random oracle, the two-map sum provably is, and RFC 9380 standardizes exactly that optimum. What is left is not the classical mathematics. It is implementation assurance and the post-quantum question [@rfc9380] [@ffstv13].

One boundary is worth stating precisely, because it sets up everything still open. The "unknown discrete log" property adds no new hardness assumption: it inherits the elliptic-curve discrete-log hardness of the target group, the same assumption Part 15 examined [@rfc9380]. And so, like all elliptic-curve cryptography, it falls to Shor's algorithm on a fault-tolerant quantum computer. Post-quantum signatures sidestep the primitive entirely, because they use no curve. But the consumers that specifically need a point of unknown discrete log -- OPRFs and VRFs -- have no such escape. That is where the frontier actually is.

9. Where It Is Still Moving

A solved primitive with live frontiers sounds like a contradiction, but hash-to-curve is exactly that. The classical mathematics is settled; the engineering, the assurance, and the post-quantum future are not.

Constant time, all the way down

RFC 9380 guarantees constant time at the interface: no data-dependent loop. The guarantee has to hold much further down, though, through the field primitives the map calls -- the square root, the sign function sgn0, the zero-guarded inversion inv0, general inversion, and the conditional selects -- each of which can leak through timing, cache, or micro-architecture if written naively [@rfc9380]. And the map runs on secrets: passwords in SAE and CPace, blinded inputs in OPRFs. A leak in one field routine re-opens the exact class of attack Dragonblood exploited.

Note: Constant time must be structural in the map and then verified down to the field primitives, not patched in after the fact. Dragonblood's second act (Section 1) is the standing proof: fixing timing curve by curve is how you get burned twice [@rfc9380] [@dragonblood-site].

The expensive curves

Hashing to G2 of BLS12-381, over the field $\mathbb{F}_{p^2}$ and with a large cofactor, is markedly costlier than hashing to G1. Faster maps and faster cofactor clearing are open engineering targets, and they matter because Ethereum verifies huge volumes of signatures, so every millisecond of G2 hashing multiplies across the network. Wahby and Boneh brought constant-time hashing within nine percent of variable-time, and the fast-G2 cofactor-clearing line descends from work like Scott and coauthors' "Fast Hashing to G2," but G2 remains the expensive path, and each new pairing-friendly curve keeps the problem alive [@wb19] [@sbcdk09].

Machine-checked implementations

Because constant time is hard to get right, the field increasingly wants implementations proven correct by machine, not just audited by eye. The leading edge is visible in blst, the highest-volume hash-to-G2 implementation: it was audited by NCC Group in January 2021, and its formal verification by Galois is, in the project's own words, ongoing [@blst] [@blst-verification]. That is the honest status: verification of a deployed hash-to-curve path is in progress, not finished. Full end-to-end machine-checked hash-to-curve -- map, cofactor clearing, and constant time, across curve families -- remains open.

The genuinely open frontier: post-quantum

The deepest gap is post-quantum. Signatures have a post-quantum future without curves, so they abandon hash-to-curve. The unknown-DL consumers do not have that luxury. OPRFs and VRFs -- and therefore OPAQUE, Privacy Pass, and anonymous-credential systems built on them -- rest on the hardness of the elliptic-curve discrete log, and a quantum adversary breaks that foundation.Lattice- and isogeny-based OPRF and VRF constructions exist in the research literature, but none is competitive with the elliptic-curve versions in bandwidth or round complexity, and none is standardized. This guide states the frontier without pinning a specific paper, holding to the same citation standard as the rest of the series. There is no drop-in "post-quantum hash-to-curve," because the whole unknown-DL abstraction has to be rebuilt on different assumptions, and the competitive replacement does not yet exist.

Steady-state engineering

Finally, the everyday work: as deployments adopt new pairing-friendly curves (BLS12-377 and BW6 are recent examples) and new prime-order abstractions, each needs its admissible map and cofactor-clearing constants worked out and folded into the suite registry [@rfc9380]. This keeps the "which map for which curve" table a living document rather than a finished one. Given all of it, the practical question remains: what should you actually do on Monday?

10. The Decision Rules, Made Operational

Here is the spine turned into something you can apply to a code review this afternoon. The rules are short because the standard did the hard part; your job is to name the right suite and not undo its work.

Use a vetted suite, never a hand-rolled map. Always call a RFC 9380 suite from a constant-time library for your curve. Never compute $H(m)\cdot G$, and never write your own map. The two most catastrophic errors in this whole area are the two shortcuts from Section 3 [@rfc9380].

Default to _RO_. Use hash_to_curve (_RO_) unless you hold a specific proof that non-uniformity is acceptable, in which case encode_to_curve (_NU_) halves the cost [@rfc9380].

Always set a unique DST. Make the domain-separation tag specific to your protocol and version, hash it if it exceeds 255 bytes, and never share one across protocols. This is domain separation, the same discipline Part 13 develops for keys [@rfc9380].

Map by curve, not by taste. Short-Weierstrass with $p \equiv 3 \pmod 4$ uses SSWU; $A = 0$ uses isogeny-SSWU; Montgomery and Edwards use Elligator 2; anything else uses SvdW [@rfc9380].

Then the per-protocol specifics:

BLS: use BLS12381G2_XMD:SHA-256_SSWU_RO_ for minimal-pubkey-size, or the G1 suite for minimal-signature-size; _RO_ is required for the unforgeability proof; clear the cofactor [@draft-bls-signature-07].
OPRF, VOPRF, OPAQUE: call RFC 9497's HashToGroup and HashToScalar at the ciphersuite's suite; do not substitute your own map; reject the identity element on the wire [@rfc9497].
PAKE mapping a password (SAE, CPace): use a constant-time map only, never hunt-and-peck a secret, and pre-hash the password with a slow KDF such as PBKDF2, scrypt, or Argon2 before mapping, so an adversary who sees an intermediate value still faces a slow dictionary attack [@rfc9380]. Part 12's password-entropy argument is exactly why this matters.

Key idea: Pick the suite for your curve, default to _RO_, set a unique DST, and clear the cofactor. Four decisions, each with a safe default, and RFC 9380 has already made three of them for you the moment you name the suite [@rfc9380].

The mirror image of the rules is the list of things that never belong in a diff. Each is a real, shippable bug that maps to exactly one of the three properties, or to the two guardrails around them.

Misuse	What it violates	The fix
$P = H(m)\cdot G$ instead of a map	Unknown discrete log	use a real RFC 9380 map [@rfc9380]
Hunt-and-peck over a secret	Constant time	constant-time suite; H2E for SAE [@rfc9380] [@dragonblood-site]
`_NU_` where the proof needs `_RO_`	Random-oracle distribution	default to `_RO_` [@rfc9380]
Missing or shared DST	Domain separation	unique per-protocol, per-version DST [@rfc9380]
Skipped cofactor clearing	Subgroup safety	clear the cofactor; check the subgroup [@rfc9380]
Truncated `hash_to_field` output	Uniformity margin	use the full $L$ bytes [@rfc9380]
Raw password without a slow KDF	Offline-guessing resistance	PBKDF2, scrypt, or Argon2 first [@rfc9380]

Note: Vetted RFC 9380 suite for the curve. _RO_ unless a proof licenses _NU_. Unique, versioned DST, hashed if over 255 bytes. Cofactor cleared and subgroup confirmed. Full L-byte hash_to_field. For secrets, a constant-time map and a slow KDF first. For OPRFs, identity rejected on the wire. Anything else is a finding [@rfc9380] [@rfc9497].

RFC 9380 ships official test vectors for every suite. Before you trust an implementation, feed it the standard's sample inputs and DSTs and confirm the output points match byte for byte. A library that reproduces the vectors implements the pipeline correctly; one that does not should never touch a secret [@rfc9380].

A checklist answers "what to do." The last thing to handle is the set of "but why can't I just..." questions that every reviewer and implementer keeps asking, because the answers are where the understanding finally sets.

11. FAQ and Common Misuse

Because its discrete log is $H_n(m)$, which is public, so the point is not a stranger to anyone. In BLS this is fatal: a signature $[sk]\cdot H(m)$ becomes $[H_n(m)]\cdot PK$, which anyone computes from the public message and public key with no secret key. The same known-discrete-log flaw breaks OPRF obliviousness and a PAKE's password element [@rfc9380] [@bls01]. Only when the input is public and you need neither constant time nor a random-oracle distribution. Over a secret input the loop count is a function of the secret, so the running time leaks it. That is precisely the Dragonblood attack that recovered WPA3 passwords, and it is why RFC 9380 declines to specify the method [@rfc9380] [@vr20]. Yes. The `_NU_` output is provably not a random oracle: it covers at least one eighth of the group with weights varying by up to a factor of four [@rfc9380]. If your proof assumes a random oracle -- as the BLS, OPRF, and PAKE proofs do -- you need `_RO_`. Use `_NU_` only with a specific proof that non-uniformity is acceptable [@rfc9380]. Always. The DST is domain separation, not decoration: without a unique, protocol-and-version-specific tag, two protocols sharing a curve can be driven to collide. RFC 9380 makes it mandatory and hashes tags longer than 255 bytes [@rfc9380]. You don't; the curve decides. Curve25519 and edwards25519 use Elligator 2, because they have a point of order 2. BLS12-381 uses isogeny-SSWU, because $A = 0$. The map is a structural consequence of the curve, not a preference [@rfc9380]. No. It is bound to elliptic-curve cryptography and falls to Shor's algorithm. Post-quantum signatures avoid the primitive entirely because they use no curve, but the consumers that need a point of unknown discrete log -- OPRFs and VRFs, and the systems built on them -- have no settled, efficient post-quantum replacement yet [@rfc9380]. No. `encode_to_curve` (`_NU_`) runs one map and produces a nonuniform point at about half the cost. `hash_to_curve` (`_RO_`) hashes to two field elements, maps each, and adds the points, which is indifferentiable from a random oracle. The extra map is what buys the distribution [@rfc9380].

The Point Had to Be a Stranger, Arrive on Time, and Look the Part

Look back at the failures with the whole picture in hand, and they line up with eerie precision. Each was a wrong answer to exactly one of the three questions. Computing $H(m)\cdot G$ published the discrete log: the point was not a stranger, and universal forgery followed [@rfc9380] [@bls01]. Hunt-and-peck leaked the password through the clock: the point did not arrive on time, and Dragonblood read it off the handshake [@vr20]. Reaching for _NU_ where a proof needs _RO_ handed the proof a distribution it never agreed to: the point did not look the part [@rfc9380]. Three questions, three ways to fail, three real systems that failed them.

That is the sibling claim to the one Part 15 made about the curve itself. The elliptic-curve arithmetic was sound; the danger was the soft gap around it. Here the gap has a name and a shape. It is the doorway, the single step that carries an untrusted string into the group, and getting it right means answering all three questions at once: unknown discrete log, constant time, and a proof-usable distribution. RFC 9380 is the standard that guarantees all three and encodes which ones you asked for in a suite name you can read at a glance.

For classical curves, the doorway is finally built to spec. The two-map construction is the provable optimum, the oversampling margin is a proven bound, and a handful of audited libraries carry the whole load quietly under BLS, OPRFs, OPAQUE, Privacy Pass, and WPA3. The open frontier has moved elsewhere: to machine-checked implementations of what already ships, to the stubbornly expensive pairing curves, and above all to a post-quantum world where the unknown-DL consumers have no settled replacement.

So the next time you call a one-line hash-to-curve API, you will know to ask the three questions it answers for you. Do you know the point's discrete log? Does your running time depend on the input? Can your proof model the output as a random oracle? For any construction worth shipping, the answers are the same three words this whole field spent two decades earning: no, no, yes.

The Curve Was Hard; The Gap Was Soft: A Field Guide to Using Elliptic Curves Safely

noreply@paragmali.com (Parag Mali) — Sat, 11 Jul 2026 16:08:14 GMT

**The elliptic-curve discrete-log problem was never the weak point.** On a well-chosen curve, ECDLP is the hardest discrete log we know how to build -- which is why a 256-bit curve buys what a 3072-bit RSA modulus does -- and yet across three decades of deployment, almost no real-world break ever solved one. They did not have to. Every named disaster lives in the gap between a hard curve and safe code: the server that fed an attacker's off-curve point into scalar multiplication and leaked its private key (invalid-curve attacks); the low-order point that clamping never filtered (the Monero cofactor bug); the certificate whose *base point* was never checked (Curveball, CVE-2020-0601); the microsecond of branch timing in a non-constant-time ladder (Brumley-Tuveri). The evolution of curves -- NIST short-Weierstrass (P-256, secp256k1), then Curve25519/X25519, then Ed25519, then the Ristretto255 prime-order abstraction -- is one sustained move: take the safety burden off the implementer and bake it into the curve, the group, and the encoding, so the simple implementation becomes the secure one. Using elliptic curves safely is a stack: the right curve for the job (X25519 for key agreement, Ed25519 for new signatures, P-256 only where FIPS compels it, secp256k1 only for blockchain), validate every untrusted point, multiply in constant time, and prefer constructions where the dangerous case cannot be reached. Remove any layer and the gap reopens.

1. Two Lines of Code

In January 2020, Microsoft patched a vulnerability the U.S. National Security Agency had quietly reported to it -- an unusually public gesture from an agency that normally hoards such findings. The bug was almost embarrassing in its simplicity. Windows would check a TLS certificate's elliptic-curve signature against a trusted root, and it would never check the curve's base point. If you supplied a certificate that carried its own curve parameters -- with a generator you had chosen yourself -- Windows would trust the forged chain as though a real certificate authority had signed it [@cve-2020-0601]. Nobody factored a modulus. Nobody solved a discrete logarithm. The math of P-256 stood untouched. The attacker handed the code one parameter it forgot to validate, and the industry named the result Curveball.

Now go one layer down and back five years. In 2015, researchers recovered a TLS server's ECDH private key -- the actual secret -- by feeding it points that were not on the intended curve at all. The server's own scalar-multiplication routine did the leaking, one small residue at a time, until the whole key fell out [@jager-2015]. Two breaks, at two different layers of the same stack, with exactly one shape. In neither case did anyone attack the problem the textbooks call hard.

That is the claim this article proves. ECDLP-hardness is not the same thing as ECC-security, and every deployed elliptic-curve break lives in the gap between them. The discrete logarithm on a good curve is the strongest one-way function in wide use. The security of a system built on it is a different property entirely -- one that depends on what your code does with a point before it has decided the point is legitimate, and whether it does the arithmetic in a time an attacker cannot measure.

The private curve stayed hard; the code around it confessed.

So here is the diagnostic question the rest of this piece hands you, the one to ask of any elliptic-curve code you own: when a point you did not generate arrives, what does your code do with it before it has checked that the point is on the intended curve and in the right prime-order subgroup -- and does it do that arithmetic in constant time? Hold that question. The names you already fear -- invalid-curve attacks, Curveball, twist attacks, small-subgroup leaks, the cofactor, timing side channels -- are all one sentence underneath it. One mechanism explains the entire catalog, and the thirty-year evolution of curves is a single sustained campaign to put that mechanism out of reach.

To see how an unchecked point hands over a private key -- and why "just use Curve25519" is a real answer but not the whole one -- you first have to see what an elliptic curve actually is, and what makes its discrete log hard.

2. What an Elliptic Curve Actually Is, and Where It Came From

In 1976, Whitfield Diffie and Martin Hellman handed the world a template rather than a cryptosystem [@diffie-hellman-1976]. Pick a mathematical group in which one operation is easy to compute and its inverse is hard. Publish the result of applying that operation $k$ times to a fixed element -- call it $[k]G$. Keep $k$ secret. Anyone can combine public values to agree on a shared secret; no eavesdropper can run the operation backward. The template was sound. The group they had, the integers modulo a large prime, was not thrifty: because sub-exponential algorithms (index calculus) attack discrete logs in that group, safety demanded enormous keys. The obvious question hung in the air for nine years. What if you kept the template and changed the group?

Two inventors, no shared citation

The answer arrived twice, independently, from two people who did not cite each other. Victor Miller proposed it at CRYPTO '85 [@miller-1985]; Neal Koblitz published the same idea in Mathematics of Computation in 1987 [@koblitz-1987]. Both suggested using the group of points on an elliptic curve over a finite field. The decisive property: for a well-chosen curve, the best algorithm anyone knows is generic -- it works in any group and costs about $\sqrt{n}$ operations for a group of size $n$ -- rather than the sub-exponential index calculus that makes ordinary discrete logs cheap. No shortcut specific to the curve exists. That single absence is the whole reason elliptic-curve cryptography exists.Miller (1985) and Koblitz (1987) proposed elliptic-curve cryptography independently and neither cited the other. It is one of cryptography's cleaner cases of simultaneous invention -- the idea was ready to be found.

The primitive, kept tight

Here is the object itself, because this is the primitive and everything later depends on reading it correctly. Take the equation

$$y^2 = x^3 + ax + b$$

over a finite field $\mathbb{F}_p$ (the integers modulo a prime $p$). The set of solutions $(x, y)$, together with one extra "point at infinity" that acts as a zero, forms a finite abelian group. The group operation is a geometric rule called chord-and-tangent addition: to add two points, draw the line through them, find the third place it meets the curve, and reflect over the $x$-axis. To double a point, use the tangent line instead. The rule looks whimsical and turns out to be associative -- the proof runs through divisors and Riemann-Roch, which we cheerfully defer to Silverman.

Repeated point addition: adding a base point G to itself k times, written [k]G. Computed in about log k doublings and additions by the double-and-add method, it is the easy, forward direction of the elliptic-curve trapdoor -- fast to compute, believed hard to reverse.

Scalar multiplication is that forward direction: $[k]G = G + G + \dots + G$, $k$ times. Because you can double repeatedly, computing $[k]G$ for a 256-bit $k$ takes a few hundred operations, not $2^{256}$ of them. Reversing it -- recovering $k$ from $G$ and $Q = [k]G$ -- is the hard problem the whole field rests on.

Given a base point G on an elliptic curve and a second point Q = [k]G, recover the scalar k. This is the one-way assumption elliptic-curve cryptography depends on: easy to compute [k]G in the forward direction, believed infeasible to invert for a well-chosen curve.

Feel the asymmetry directly. The demonstration below computes $[13]G$ on a tiny curve in a handful of steps, then tries to walk it back. At these toy sizes the reversal is trivial; at 256 bits the same reversal is roughly $2^{128}$ operations, which is why the forward arrow is a trapdoor and the backward arrow is a wall.

{` // Toy curve y^2 = x^3 + 2x + 2 over F_17. Illustrative sizes, NOT secure. const p = 17n, a = 2n; function mod(x){ return ((x % p) + p) % p; } function inv(x){ // modular inverse via Fermat: x^(p-2) mod p let r = 1n, b = mod(x), e = p - 2n; while (e > 0n){ if (e & 1n) r = mod(rb); b = mod(bb); e >>= 1n; } return r; } function add(P, Q){ if (P === null) return Q; if (Q === null) return P; const [x1,y1] = P, [x2,y2] = Q; if (x1 === x2 && mod(y1 + y2) === 0n) return null; // P + (-P) = O let s; if (x1 === x2 && y1 === y2) s = mod((3nx1x1 + a) * inv(2ny1)); // tangent (double) else s = mod((y2 - y1) * inv(x2 - x1)); // chord (add) const x3 = mod(ss - x1 - x2); const y3 = mod(s*(x1 - x3) - y1); return [x3, y3]; } function mul(k, G){ // double-and-add: the FORWARD (easy) direction let R = null, base = G; while (k > 0n){ if (k & 1n) R = add(R, base); base = add(base, base); k >>= 1n; } return R; } const G = [5n, 1n]; // generator; this subgroup has prime order 19 const secret = 13n; // the "private key" const Q = mul(secret, G); // the "public key" [13]G, computed in ~5 steps console.log('public point [13]G =', Q[0].toString() + ',' + Q[1].toString());

// Now INVERT it: recover the secret from G and Q. The only general method on a // real curve is search -- trivial here, about 2^128 work at 256 bits. let found = null, R = null; for (let k = 1n; k <= 19n; k++){ R = add(R, G); if (R && R[0] === Q[0] && R[1] === Q[1]){ found = k; break; } } console.log('recovered secret by brute force =', found.toString()); `}

Why the keys shrink

That missing shortcut is not an academic footnote; it is the entire economic case for the primitive. For discrete logs modulo a prime, and for RSA, index-calculus methods run in sub-exponential time, so defenders must inflate the modulus to stay ahead -- 3072 bits to reach roughly 128-bit security. On a well-chosen prime-field curve, no such method is known, so the attacker is stuck paying the generic $\sqrt{n}$ price. A 256-bit curve therefore delivers about 128 bits of security [@miller-1985], [@safecurves-rho]. Same protection, an order of magnitude smaller keys, faster operations. Hold onto one detail for later: this economic win, the thing everyone remembers about elliptic curves, is the one property that has never been what broke.The absence of a sub-exponential attack on prime-field curves is the whole argument. It is why 256-bit ECC stands in for a 3072-bit RSA key -- and it is a statement about what we do not know how to do, not a theorem.

One clean curve, and the seeds of trouble

Standardization made the primitive interoperable. ECDSA was standardized as a digital-signature algorithm (ANSI X9.62) and folded into NIST's FIPS 186 series [@fips-186-5]; the P-curve we still call P-256 dates to FIPS 186-2 in 2000. The Standards for Efficient Cryptography Group published the matching secp parameter sets -- including secp256r1, which is P-256, and its efficiency-minded sibling secp256k1 [@sec2-v2]. The result was a single, clean, interoperable curve with prime order and a genuine strength most of its successors would come to envy. Which raises the puzzle that drives the next two sections: if P-256 is that good, why do the very people who designed its replacement consider "just implement P-256" to be dangerous advice?

3. The NIST Curves, and What They Cost

P-256 did one big thing exactly right, and three smaller things in a way that made the simple implementation insecure. Both halves are true at once, and holding them together is the beginning of expertise here.

What went right

P-256 is the curve $y^2 = x^3 - 3x + b$ over the special prime $p = 2^{256} - 2^{224} + 2^{192} + 2^{96} - 1$, and its single most underrated property is boring: its group has prime order, cofactor 1 [@sec2-v2]. There is no smaller subgroup for an attacker to trap you in, no cofactor to clear, no torsion points hiding in the corners. Add to that an enormous interoperable deployment -- P-256 is the elliptic curve behind most of the web's ECC certificates [@ssl-pulse] -- and you have a curve whose group-level security is excellent. Remember this when Curve25519 arrives two sections from now carrying a cofactor of 8.

The ratio of a curve's full group order to the prime order of the subgroup used for cryptography. P-256 has cofactor 1 (its whole group has prime order), while edwards25519 has cofactor 8, meaning small-order "torsion" points exist alongside the useful prime-order subgroup and must be handled with care.

Note: Cofactor 1 means P-256 has no small-order subgroups at all. That single fact rules out an entire family of attacks at the group level. Keep it in mind when the next generation trades it away for other advantages -- prime order is exactly what Curve25519 gives up and what Ristretto255 later works hard to rebuild.

Cost one: nobody can explain the seed

The coefficient $b$ and the base point were produced from a seed run through SHA-1 -- "verifiably random," in the standard's language, meaning anyone can rerun the hash and confirm the parameters were not obviously cooked. The catch is that the seed itself is an unexplained string of bytes. Nobody outside its authors can say why that seed and not another. SafeCurves calls this failure rigidity: a curve is rigid only when its parameters follow from stated principles with no unexplained freedom, and the NIST P-curves are not [@safecurves-index]. This is a trust concern, not a break -- but trust is a load-bearing beam, as the next beat shows.The P-256 parameters are "verifiably random" from a seed, yet the seed is never justified. That gap -- reproducible but unexplained -- is exactly what rigidity objects to. You can check the arithmetic and still not know why these constants.

Cost two: the addition law has holes

The classic short-Weierstrass addition formulas are not total. They have exceptional cases -- adding a point to its own negation, doubling a point whose $y$-coordinate is zero, anything involving the point at infinity -- where the generic slope formula divides by zero and returns nonsense. A working implementation must branch around those cases. And a branch whose direction depends on secret data is either wrong on the rare input that hits it, or observable through timing, or both. This incomplete addition law is the seed of the timing break we meet in the next section.

Cost three: the formula never looks at b

Here is the sharp one. Look back at the addition code in Section 2 and notice what is missing: the coefficient $b$ never appears. The short-Weierstrass scalar-multiplication formulas depend on $a$, on the coordinates, on the prime -- but not on $b$. So if you hand the routine a point that satisfies some other curve $y^2 = x^3 + ax + c$ with a different constant term, the arithmetic proceeds without complaint, silently computing on that other, possibly far weaker, curve [@safecurves-twist]. The curve has no twist security to catch the mistake. This omission is the seed of the invalid-curve break, and we detonate it in Section 4.

The Dual EC shadow

There is one more reason the community distrusts NIST-supplied constants, and precision matters because it is widely misremembered. In 2007, Dan Shumow and Niels Ferguson showed that the Dual EC DRBG random-number generator could conceal a back door: whoever chose its two special points might hold a secret relationship between them that predicts the generator's output [@shumow-ferguson-2007]. Edward Snowden's 2013 disclosures and, in 2015, a backdoor found inside Juniper's ScreenOS gave that theoretical worry a concrete afterlife.

But be exact about what this does and does not implicate. No weakness has ever been demonstrated in the P-256 curve itself. Dual EC poisoned trust in NIST-supplied constants as a category -- which is precisely why so many engineers now prefer rigid curves -- but it is a trust-and-rigidity argument, not a proven backdoor in the P-curves.Dual EC DRBG is a random-number-generator story, not a P-curve story. It shows NIST-chosen constants could be poisoned; it does not show that P-256's were. Conflating the two overstates the case and is a common error. And notice the shape already forming: prime-order, FIPS-mandated, mathematically strong -- yet the simple implementation is the insecure one.

The efficiency sibling

One more curve belongs on the table. secp256k1 is the SECG's efficiency variant: the curve $y^2 = x^3 + 7$ over $p = 2^{256} - 2^{32} - 977$, with $j$-invariant 0 and an endomorphism (the GLV method) that speeds scalar multiplication [@sec2-v2]. It is the curve Bitcoin chose, and later the base for Taproot's BIP-340 Schnorr signatures [@bip-340]. Keep its domain straight throughout this article: secp256k1 owns the blockchain world and is essentially absent from web PKI and TLS -- a blockchain-interoperability answer, never a general-purpose competitor to the curves that carry your HTTPS traffic.

Two of those three costs -- the unused $b$ and the branchy ladder -- are not abstractions. Each became a remote private-key recovery against real servers. To see how, you need the one mechanism that sits underneath the entire failure catalog.

4. The Failure Catalog: One Atom, Many Surfaces

Here is the mechanism the whole catalog reuses. Call it the atom: a point you did not generate, fed into scalar multiplication before you validate it -- or multiplied in variable time -- is a key-recovery oracle. The discrete logarithm is never solved. The implementation simply computes on the attacker's terms and leaks the answer, one small piece at a time.

Key idea: The atom of every deployed elliptic-curve break: a point you did not generate, used before it is validated -- or a secret scalar multiplied in a time an attacker can measure -- turns your own scalar-multiplication routine into an oracle that hands back the private key. ECDLP stays intact; the code around it does the leaking.

Deriving the atom: invalid-curve attacks

Recall Cost three from the last section: the short-Weierstrass formulas never read $b$. Ingrid Biehl, Bernd Meyer, and Volker Muller turned that omission into an attack at CRYPTO 2000 [@biehl-meyer-muller-2000]. Send the victim a point that satisfies a different curve $y^2 = x^3 + ax + c$ -- one you chose specifically because its group has small-order subgroups. The victim's routine, blind to the constant term, computes your scalar multiple on that weak curve. Its response now lives in a small subgroup, so it reveals the secret scalar modulo a small number. SafeCurves states the mechanism about as plainly as it can be stated:

The standard formulas for scalar multiplication on short Weierstrass curves do not involve the constant coefficient b, so they automatically also work for y^2 = x^3 + ax + c ... Bob will successfully compute n(x,y) without realizing that anything is amiss. Feeding an implementation a point that is not on the intended curve into scalar-multiplication formulas that ignore the coefficient b. The operation runs on a different, attacker-chosen curve whose group has small subgroups, so the response leaks the private scalar modulo small numbers -- with the intended curve's discrete-log hardness never touched. flowchart TD A["Attacker sends a point not on the intended curve"] --> B["Scalar-multiply formula ignores coefficient b"] B --> C["Arithmetic runs on a different, weaker curve"] C --> D["That curve has small-order subgroups"] D --> E["Response reveals the secret modulo a small number"] E --> F["Repeat with several coprime small orders"] F --> G["Chinese Remainder Theorem reassembles the whole key"]

The demonstration below runs the leak on a toy curve whose order factors as $105 = 3 \times 5 \times 7$. The attacker sends three low-order points, harvests three residues, and the Chinese Remainder Theorem stitches them into the secret. Nothing here is a discrete-log computation; it is bookkeeping.

{` // Toy curve y^2 = x^3 + x + 1 over F_101. Group order 105 = 357. // Illustrative ONLY -- a real curve has PRIME order and ~2^256 points. const p = 101n, a = 1n; const mod = x => ((x % p) + p) % p; function inv(x){ let r=1n,b=mod(x),e=p-2n; while(e>0n){ if(e&1n) r=mod(rb); b=mod(bb); e>>=1n;} return r; } function add(P,Q){ if(P===null) return Q; if(Q===null) return P; const [x1,y1]=P,[x2,y2]=Q; if(x1===x2 && mod(y1+y2)===0n) return null; // P + (-P) = O const s = (x1===x2 && y1===y2) ? mod((3nx1x1+a)inv(2ny1)) // tangent (double) : mod((y2-y1)inv(x2-x1)); // chord (add) const x3 = mod(ss-x1-x2); return [x3, mod(s*(x1-x3)-y1)]; } function mul(k,P){ let R=null,b=P; while(k>0n){ if(k&1n) R=add(R,b); b=add(b,b); k>>=1n;} return R; }

const k = 47n; // the victim's PRIVATE scalar // Attacker-crafted low-order points (orders 3, 5, 7). None is the real base point. const traps = [ {P:[28n,8n], t:3n}, {P:[86n,67n], t:5n}, {P:[72n,5n], t:7n} ];

// Chinese Remainder Theorem stitches the residues into k mod 105. let M=1n; for(const m of moduli) M*=m; let x=0n; for(let i=0;i<residues.length;i++){ const Mi=M/moduli[i]; let mi=1n; for(let j=1n;j<moduli[i];j++){ if((Mi*j)%moduli[i]===1n){ mi=j; break; } } x += residues[i]Mimi; } console.log('CRT-reassembled secret =', (((x%M)+M)%M).toString(), ' (true k = '+k+')'); `}

Surface one: small-subgroup attacks

You do not even need to leave the intended curve to run the atom. Chae Hoon Lim and Pil Joong Lee showed this at CRYPTO '97 [@lim-lee-1997]. If the curve's group is not of prime order -- if it has small-order subgroups -- then an attacker submits an element of small order $t$, and the victim's response is forced into that tiny subgroup, revealing the secret modulo $t$. Collect a few coprime $t$ values, apply the Chinese Remainder Theorem, and the full scalar falls out. This is the conceptual root of the rule "validate the subgroup, not just the curve," and it is exactly why prime order mattered so much back in Section 3. It also plants a flag we return to: any curve with a cofactor greater than 1 has these small subgroups by construction.

Submitting a group element of small order so that the victim's scalar multiplication lands in a tiny subgroup, making the response reveal the secret scalar modulo that small order. Repeated across several coprime small orders, the Chinese Remainder Theorem reassembles the full private key. It is the reason implementations must check that inputs lie in the correct prime-order subgroup.

Surface two: the atom made remote

Theory became a remote server compromise in 2015, when Tibor Jager, Jorg Schwenk, and Juraj Somorovsky recovered real TLS servers' ECDH private keys [@jager-2015]. The conditions were mundane: a server that reused a single ECDH key pair across handshakes and did not validate the incoming peer point. Each handshake fed the server another attacker-chosen invalid point; each response leaked another small residue; a few thousand handshakes later, the CRT delivered the private key. No discrete logarithm was ever computed. This is the atom of Section 4, wearing a TLS badge.The practical TLS result is Jager, Schwenk, and Somorovsky's ESORICS 2015 paper, "Practical Invalid Curve Attacks on TLS-ECDH" -- distinct from the same trio's separate USENIX Security 2015 work. The theoretical root is Biehl-Meyer-Muller (2000). Getting the citation right matters, because the two 2015 papers are often conflated.

Surface three: the cofactor, in production

Now the promised return to cofactors. The curve edwards25519 (the signing sibling of Curve25519) has cofactor 8, which means low-order "torsion" points exist. Monero's ring-signature scheme used key images to prevent double-spending: each spent output should map to exactly one key image. The bug was that the code never required a key image to lie in the prime-order subgroup, so an attacker could add a low-order torsion point to a legitimate key image, producing a different but still valid-looking image for the same output -- and spend it twice [@loup-vaillant-cofactor]. The fix was to demand that every key image $I$ satisfy $[\ell]I = \mathcal{O}$, where $\ell$ is the prime subgroup order: multiply by the group order and you must land on the identity, proving no torsion component is hiding inside.

The precise lesson here is the one most often gotten wrong, so it gets its own box.

Note: Curve25519 secret scalars are "clamped" -- specific bits are forced -- and engineers routinely mistake this for input validation. It is not. Clamping constrains your own secret scalar so that it is a multiple of the cofactor and sits in a fixed range. It does nothing to a malicious low-order input point someone else hands you. The Monero cofactor bug is exactly this confusion. If a protocol needs a prime-order group, validate or clear the cofactor on inputs -- or use a construction that removes the choice entirely.

Surface four: Curveball, one layer up

The atom does not care which layer it runs on. In Curveball (CVE-2020-0601), the mishandled point was not a peer's ephemeral key but the curve's own base point. Windows CryptoAPI accepted certificates carrying explicit curve parameters and trusted the generator inside them without checking it against the named curve's real generator. Choose your own generator with a known relationship to a trusted CA's public key, and you forge a signing chain [@cve-2020-0601]. Same disease, higher in the stack: a parameter that should have been validated -- here, the generator itself -- was taken on faith.

Surface five: the clock, with no malicious point at all

The final surface uses no bad input whatsoever. In 2011, Billy Bob Brumley and Nicola Tuveri recovered an OpenSSL ECDSA private key from a remote server purely by timing its scalar multiplications [@brumley-tuveri-2011]. OpenSSL's ladder for binary-field curves ran in data-dependent time; the timing leaked the bit-length of each per-signature nonce, and a lattice computation turned a few hundred such leaks into the key. The underlying flaw earned CVE-2011-1945, whose description is blunt: the implementation "does not properly implement curves over binary fields, which makes it easier for ... attackers to determine private keys via a timing attack and a lattice calculation" [@cve-2011-1945]. This is the atom's other face: arithmetic on a secret, performed in observable time, is an oracle even when every input is honest.

The one family this article does not re-derive

There is a whole lineage of ECDSA breaks -- fail0verflow's PlayStation 3 key recovery, biased-nonce lattice attacks, LadderLeak, Minerva, TPM-Fail -- that shares the atom's DNA but belongs to a sibling story about randomness and leakage rather than point validation [@minerva-2020], [@ladderleak-2020]. The result is quick to state: a repeated or biased per-signature nonce leaks the ECDSA private key through a lattice, deterministic nonces (RFC 6979) fix the generation half, and constant-time arithmetic fixes the leakage half [@rfc6979]. The lattice machinery is owned by this series' dedicated nonce-reuse post, so this article signposts it and moves on.

The pattern, stated once

Read the six surfaces together and one sentence covers all of them: the implementation touched a value it should have rejected, or acted in a time it should have hidden. The leaked bit is never "the discrete logarithm." It is "was this point on the curve," or "was this element in the right subgroup," or "how long did that multiply take," or "do I trust this generator." The table below is the evidence for the entire thesis. Every row is the same atom on a different surface.

Attack	Year	Surface	Channel	Root cause	The rule it teaches
Small-subgroup, Lim-Lee [@lim-lee-1997]	1997	Subgroup	Response residue	Non-prime-order group element accepted	Validate subgroup membership
Invalid-curve, Biehl-Meyer-Muller [@biehl-meyer-muller-2000]	2000	Off-curve point	Response residue	Formula ignores the coefficient b	Check the point is on the intended curve
Remote timing, Brumley-Tuveri [@brumley-tuveri-2011], [@cve-2011-1945]	2011	Scalar-mult timing	Wall-clock time	Non-constant-time ladder	Multiply in constant time
Practical invalid-curve TLS, Jager et al. [@jager-2015]	2015	Off-curve point, reused key	Response residue	No peer-point validation	Validate points, do not reuse ECDH keys
Cofactor key image, Monero [@loup-vaillant-cofactor]	2017	Cofactor torsion	Forged key image	Key image not in prime-order subgroup	Enforce prime order, clamping is not validation
Curveball [@cve-2020-0601]	2020	PKI base point	Forged certificate chain	Explicit generator trusted	Use named curves, never trust explicit parameters
ECDSA-nonce family [@minerva-2020], [@ladderleak-2020]	2020	Nonce leakage	Timing then lattice	Biased or leaked nonce	Deterministic nonces plus constant time

Six surfaces, one disease. And notice what the cure was not: it was not a warning label stapled to P-256. It was a decade-long engineering program to build curves where the dangerous case cannot be reached at all -- and that program, not the catalog, is the most instructive part of the story.

5. The Evolution: Moving Safety Off the Implementer

Read the last section again as a to-do list for curve designers. Each generation that follows crosses off one entry -- and here is the pattern that makes the story an argument rather than a parade of curves: they do it not by warning implementers to be careful, but by making the mistake unreachable. It is an attacker's kill-chain run in reverse. Where an intruder normally advances stage by stage, here the defenders advance, retiring one failure class per curve generation until the dangerous case no longer exists to reach.

flowchart LR G1["Gen 1 NIST short-Weierstrass"] --> F1["Leaves timing and invalid-curve to the implementer"] F1 --> G2["Gen 2 Curve25519 ladder"] G2 --> R2["Retires branch timing and invalid-curve for DH"] R2 --> G3["Gen 3 Ed25519 Edwards law"] G3 --> R3["Retires incomplete addition and the nonce disaster"] R3 --> G4["Gen 4 Ristretto255 encoding"] G4 --> R4["Retires the cofactor class at the type level"]

The chronology underneath that progression spans half a century, from a template to a post-quantum hybrid.

flowchart LR A["1976 Diffie-Hellman template"] --> B["1985-87 Miller and Koblitz propose ECC"] B --> C["2000 NIST P-curves standardized"] C --> D["2006 Curve25519 constant-time ladder"] D --> E["2011 Ed25519 complete Edwards law"] E --> F["2015-2023 Ristretto255 prime-order abstraction"] F --> G["2024-2026 X25519 plus ML-KEM hybrid"]

Generation one, and its own rescue

The NIST and SECG short-Weierstrass curves gave the world prime order, which we have already granted is a real strength. But their exceptional-case addition law and their branchy, easily-non-constant-time ladders left the two most dangerous jobs -- constant-time arithmetic and on-curve validation -- squarely with the implementer. Section 4 is the proof of what happens when an implementer drops them.

The honest rebuttal belongs right here, though, because it is the reason "P-256 is broken" is the wrong sentence. In 2016, Joost Renes, Craig Costello, and Lejla Batina published efficient complete addition formulas that work for every prime-order short-Weierstrass curve, with no exceptional cases to branch around [@rcb-2016]. Given those formulas and a vetted constant-time field implementation, P-256 is perfectly safe -- harder to implement safely, not broken. Its danger is that the naive implementation is the insecure one, and for two decades the naive implementation is what shipped.

Generation two: the ladder that is constant-time by construction

In 2006, Daniel J. Bernstein introduced Curve25519, the Montgomery-form curve $y^2 = x^3 + 486662x^2 + x$ over the prime $2^{255} - 19$ [@curve25519-2006]. The form and its ladder trace back to Peter Montgomery's 1987 work on factorization, revived here for an entirely new purpose [@montgomery-1987]. Two design choices retire whole rows of the failure catalog for free.

The first is the Montgomery ladder: a scalar-multiplication method whose sequence of field operations is identical regardless of the secret bits. There is no secret-dependent branch to time, so the timing surface -- Brumley-Tuveri's surface -- closes by construction rather than by careful coding. Key agreement uses only the $x$-coordinate, so a peer sends 32 bytes and the $y$-coordinate never enters the arithmetic.

A scalar-multiplication algorithm that performs the same sequence of field operations for every bit of the secret -- one conditional swap plus a combined add-and-double per bit -- so its running time and memory-access pattern are independent of the scalar. It is constant-time by construction rather than by defensive programming.

The second is twist security. When someone hands you an $x$-coordinate, it corresponds either to a point on Curve25519 or to a point on its quadratic twist. Bernstein chose the parameters so that both the curve and its twist have near-prime order [@curve25519-2006], [@safecurves-twist]. A hostile off-curve input therefore still lands you in a group where the discrete log is hard, and the invalid-curve attack of Section 4 is defused -- without any explicit on-curve check in the fast path.

A property of a curve whose quadratic twist also has near-prime order. Because a hostile input maps to a point on either the curve or its twist, and both groups are cryptographically hard, an off-curve input cannot be steered into a weak subgroup -- so invalid-curve attacks are defeated without an explicit on-curve validation step.

The new cost is the seed of the next two generations: Curve25519 does not have prime order. Its group has cofactor 8, so low-order points exist.Bernstein states the orders exactly: the curve over its base field has $8 \times \ell$ points, where $\ell$ is the prime $2^{252} + 27742317\ldots$, and its twist has $4 \times \ell'$ points for a different prime $\ell'$ [@curve25519-2006]. That factor of 8 is the cofactor the next two generations spend their effort taming. To keep secret scalars inside the safe range, X25519 clamps them. And clamping is the single most misunderstood step in the whole subject.

Forcing specific bits of an X25519 secret scalar: clearing the low three bits so the scalar is a multiple of the cofactor 8, clearing the high bit, and setting the second-highest bit so every scalar has the same length. Clamping constrains your own secret to sidestep small-subgroup and timing issues on your side. It is emphatically not validation of an input point someone else sends you.

Clamping clears the low three bits (making your scalar a multiple of 8, so multiplying a low-order input contributes nothing on your side), clears the top bit, and sets the second-highest bit so the ladder runs a fixed number of steps [@rfc7748].The X25519 clamp: scalar[0] &= 248 clears the low three bits, scalar[31] &= 127 clears bit 255, and scalar[31] |= 64 sets bit 254 [@rfc7748]. It protects you, the holder of the secret. It does nothing about a malicious low-order point arriving from outside -- which is why the Monero bug happened on a curve whose users clamped diligently.

Generation three: a complete addition law, and deterministic nonces

Curve25519 fixed key agreement, but signatures still lived on the branchy Weierstrass world or on Curve25519's cofactor. Ed25519 (2011) moved signing onto edwards25519, the twisted-Edwards curve birationally equivalent to Curve25519 [@ed25519-2011]. The payoff is Harold Edwards' 2007 normal form: a single addition formula with no exceptional cases at all [@edwards-2007]. Doubling, adding, the identity -- one formula handles them, so the code has no secret-dependent branch and is naturally constant-time at the group-law level, retiring the incomplete-addition surface. Ed25519 also pairs the curve with deterministic nonces, derived by hashing the message and a secret, which kill the catastrophic ECDSA per-signature-randomness disaster by construction (the nonce can no longer be repeated or biased by a weak RNG).

The cofactor did not vanish, though. edwards25519 still has cofactor 8, which is why signature malleability, non-canonical encodings, and disagreements about whether verification should be "cofactored" are real concerns -- and precisely why RFC 8032 specifies canonical-encoding checks, bounds on the signature scalar, and cofactor handling [@rfc8032]. The nonce disaster itself is this series' nonce-reuse post's territory; here it is enough that determinism closes the generation half of it.

Generation four: quotient the cofactor out of existence

The fourth generation asks a sharper question. Instead of patching every protocol to handle cofactor 8, what if the cofactor simply did not exist at the level you program against? Mike Hamburg's Decaf (2015) showed how: build a genuine prime-order group in the encoding on top of a cofactor curve, so that the torsion points are quotiented away and never appear as elements [@decaf-2015]. Ristretto255 is the cofactor-8 instantiation for edwards25519, standardized in RFC 9496 in 2023 [@rfc9496]. The consequence is the cleanest safety property in the whole article: decoding is validation. A sequence of bytes either decodes to a unique, legitimate prime-order element or it is rejected. There is no low-order point to submit, because low-order points are not representable.

flowchart TD A["Wire bytes for a ristretto255 element"] --> B{"Valid canonical encoding?"} B -->|No| C["Reject: not a group element"] B -->|Yes| D["Decode yields a unique prime-order point"] D --> E["No cofactor, no torsion, no small subgroup"] E --> F["Element is safe to use directly"] A group of prime order with no cofactor, constructed over the cofactor-8 curve edwards25519 by encoding equivalence classes of points rather than raw points. Because only valid prime-order elements have encodings, decoding a byte string is itself the validation step, and the entire cofactor and small-subgroup failure class disappears at the type level.

Curve448 and its prime-order sibling decaf448 play the conservative role: a larger, roughly 224-bit-security option standardized in the same family but far more thinly deployed than the 25519 curves [@ed448-goldilocks-2015], [@rfc9496]. Treat it as the extra-margin choice, not a rival to X25519's ubiquity.

The through-line

These four generations are not a strict succession where each kills the last. They coexist: P-256 remains FIPS-mandated, X25519 and Ed25519 split the key-agreement and signature jobs between them, and ristretto255 sits underneath new protocols that need a clean prime-order group. What each generation truly superseded was not its predecessor curve but the burden on the person writing the code.

Note: Each generation took a rule off the implementer and baked it into the curve, the group, or the encoding. Constant-time became the ladder's default. On-curve validation became twist security. Exception-free addition became the Edwards law. Cofactor handling became the ristretto255 encoding. The simple implementation kept becoming the secure one.

Curve	Form	Field prime	Cofactor	Constant-time	Twist secure	Coordinates	Best for
P-256 (secp256r1)	Short Weierstrass	2^256 - 2^224 + 2^192 + 2^96 - 1	1	With care (complete formulas)	No	Full (x, y)	FIPS certificates, ECDSA
secp256k1	Short Weierstrass	2^256 - 2^32 - 977	1	With care	No	Full (x, y)	Blockchain (Bitcoin, BIP-340)
Curve25519 / X25519	Montgomery	2^255 - 19	8	By construction	Yes	x-only (32 bytes)	Key agreement
edwards25519 / Ed25519	Twisted Edwards	2^255 - 19	8	By construction	n/a	Compressed point	Signatures
ristretto255	Prime-order over edwards25519	2^255 - 19	1 (effective)	By construction	Yes	32-byte encoding	New prime-order protocols

Four generations, each erasing one line of the failure catalog. Someone still had to name the law they were all obeying -- and to say out loud that a hard curve and a safe cryptosystem were never the same thing.

6. ECDLP-Hardness Is Not ECC-Security

In 2013, Daniel J. Bernstein and Tanja Lange put the entire argument of this article onto one website, with a claim engineered to provoke: if you implement the standard curves, chances are you are doing it wrong [@safecurves-index]. The website was SafeCurves, and it is the moment the field said the quiet part out loud.

There is a gap between ECDLP difficulty and ECC security ... if you implement the standard curves, chances are you're doing it wrong.

SafeCurves names four concrete places that gap opens: implementations that return wrong results on rare curve points, that leak secrets when fed off-curve inputs, that branch in secret-dependent time, and that leak through cache-timing [@safecurves-index]. Every one of those is a surface from Section 4. And the criteria SafeCurves proposes -- rigidity, twist security, complete addition, ladder support, a handled cofactor -- are not a beauty contest. They are a single design program stated as a checklist: choose the curve so that the easy implementation is the safe one. That is the whole of Section 5 compressed into one sentence.

Which lets us state the thesis in its most general form.

Key idea: Security is a property of the curve, the group, and the implementation together -- not of ECDLP alone. A hard discrete logarithm is necessary and nowhere near sufficient. Every generation in the previous section is this one sentence, applied to a different part of the stack.

The honest tension, handled carefully

It would be easy to read SafeCurves as a verdict that the NIST curves are unsafe. That reading is wrong, and getting it right is part of being an expert here rather than a partisan.

SafeCurves is a strong, deliberately opinionated argument from around 2013 to 2015, not a neutral consensus standard. NIST P-256 and P-384 do fail several of its criteria -- rigidity, twist security, complete addition in the naive formulas -- and yet they remain FIPS-mandated and, implemented with care, secure. The Renes-Costello-Batina complete formulas close the completeness gap in software [@rcb-2016], and vetted constant-time libraries close the rest. The defensible position is precise: P-256 is *harder to implement safely* than Curve25519, not *broken*. "Prefer 25519 for new designs" and "P-256 is unsafe" are different claims, and only the first is true [@safecurves-index].

The scorecard below is the honest version. Read a "No" or a "With care" as "the naive implementation needs help here," never as "this curve is broken." Prime order is where the NIST curves quietly win, and it is exactly what the 25519 curves trade away.

Curve	Rigidity	Twist security	Complete addition	Ladder-friendly	Prime order
P-256	No	No	With care	No	Yes
secp256k1	Partly	No	With care	No	Yes
Curve25519 / X25519	Yes	Yes	Yes	Yes	No (cofactor 8)
Ed25519	Yes	Yes	Yes	n/a	No (cofactor 8)
ristretto255	Yes	Yes	Yes	Yes	Yes

The footnote that keeps this table honest: every "No" for the NIST curves is a statement about the default implementation, not the mathematics. secp256k1 is marked "Partly" rigid because its parameters follow a stated efficiency rationale rather than an unexplained seed. Give P-256 complete formulas and a constant-time field, and its row stops mattering. That is why the state of the art is not a single crowned curve but a matter of knowing which curve already does which job for you -- and in 2026, that picture is unusually settled.

7. State of the Art (2024-2026): Which Curve Does Which Job

There is no single best curve to crown. The state of the art is a small, stable set, each member owning a job -- plus one transition quietly rewriting every handshake on the internet.

X25519 owns key agreement

For ephemeral key agreement, X25519 is the default answer. It is constant-time by construction, twist-secure, and moves 32 bytes per side; it is a standardized TLS 1.3 named group [@rfc8446], the Diffie-Hellman step in WireGuard's Noise handshake [@wireguard-protocol], and the key exchange in Signal, all built on the same Curve25519 arithmetic [@rfc7748]. For plain Diffie-Hellman you need no explicit point-validation code, because twist security already handles hostile inputs -- the operational rules reduce to rejecting the all-zero output and running the shared secret through a KDF rather than using it raw.

P-256 ECDSA owns FIPS certificates

P-256 is co-dominant, and for a specific reason: it is FIPS-mandatory for certificates [@fips-186-5]. Among the curves used in web certificates it is overwhelmingly the one [@ssl-pulse], now implemented with complete formulas or vetted constant-time assembly plus point validation on untrusted inputs [@sp-800-56a-r3]. Where a compliance regime names the curve, P-256 done carefully is exactly right.

Ed25519 is now FIPS-approved too

The important recent shift for signatures is regulatory. FIPS 186-5, published February 3, 2023, approves EdDSA alongside ECDSA [@fips-186-5], so Ed25519 is now both the technically cleaner option and a compliant one. It is standard in OpenSSH, in Git commit signing, and in package-signing tools such as minisign and OpenBSD's signify. Ed25519 needs about 87,500 CPU cycles to sign and about 273,000 to verify -- near 134,000 per signature when many are checked in one batch [@ed25519-2011] -- with the eBACS suite serving as the live cross-CPU benchmark authority [@bench-cr-yp-to]. The state-of-the-art guidance: Ed25519 for new designs, ECDSA where compliance compels it [@rfc8032].

secp256k1 owns blockchain, and only blockchain

secp256k1 carries an enormous deployment -- Bitcoin, Ethereum, and their descendants -- through ECDSA and, since Taproot, BIP-340 Schnorr signatures [@bip-340]. It is nonetheless essentially absent from web PKI and TLS. On any decision about a certificate or a general key exchange, secp256k1 is not a candidate; it is the answer only when the job is blockchain interoperability.

Ristretto255 is the prime-order substrate for new protocols

When a protocol needs a clean prime-order group -- password-authenticated key exchange, verifiable random functions, oblivious pseudorandom functions, threshold signatures, zero-knowledge systems -- ristretto255 (RFC 9496) is the current substrate [@rfc9496]. It is the recommended ciphersuite of the FROST threshold-Schnorr standard, which is defined generically over any prime-order group and also specifies Ed25519, Ed448, P-256, and secp256k1 suites [@rfc9591]; and it ships in production inside Signal's libsignal, whose zkgroup and zkcredential modules build on ristretto255 points [@libsignal].

The post-quantum overlay: ECC wrapped, not replaced

The headline of 2024 to 2026 is not a new curve; it is a wrapper. The hybrid X25519 plus ML-KEM-768 combines the classical curve with the NIST-standardized lattice KEM, and it is now shipping by default in Chrome and across Cloudflare's network [@cloudflare-pq-2024], and available in OpenSSH 9.9, released September 2024 [@openssh-9.9]. The motive is "harvest now, decrypt later": an adversary can record today's X25519 traffic and wait for a quantum computer, so the hybrid forces them to break both the curve and the lattice.Post-quantum-encrypted traffic on Cloudflare's network nearly doubled across 2025, from about 29% of human-generated web requests at the start of the year to roughly 52% by early December [@cloudflare-radar-2025], with a live tracker following the trend [@cloudflare-radar-pq-tracker].

The enabling techniques are themselves state of the art. The Montgomery ladder, the Renes-Costello-Batina complete formulas [@rcb-2016], and machine-checked, correct-by-construction field arithmetic (fiat-crypto, whose code for Curve25519 and P-256 ships inside BoringSSL) are what make these curves safe in practice rather than merely on paper [@fiat-crypto].

Note: It is not a single best curve. It is "the right curve for the job" -- X25519 for key agreement, Ed25519 for new signatures, P-256 where FIPS compels it, secp256k1 only for blockchain, ristretto255 for new prime-order protocols -- plus the hybrid transition that wraps the classical curve in a post-quantum KEM.

FIPS 186-5 approves EdDSA, so Ed25519 is now a compliant signature choice [@fips-186-5]; ECDSA over P-256 remains mandated for certificates; and SP 800-56A Rev. 3 requires point and subgroup validation for untrusted inputs in key establishment [@sp-800-56a-r3]. Compliance and good engineering point the same way more often than they used to.

"Which curve owns which job" is a summary, not a decision procedure. When the jobs overlap -- when a compliance regime, a blockchain, and a threshold protocol pull in three directions at once -- you need the head-to-head.

8. The Decision Matrix

The deployed curves do not all compete for the same job, so the useful comparison is per task -- and it hinges on one axis the benchmarks never advertise: how much of the failure catalog you must defend against by hand.

Key agreement: X25519 unless something forces your hand

Default to X25519. Its twist security and constant-time ladder mean the dangerous inputs are already handled, so your implementation burden is low: reject the all-zero shared secret and run the result through a KDF. Choose P-256 ECDH only where a compliance regime mandates it -- and then take on the full burden the curve leaves you: validate the peer point is on the curve and in the prime-order subgroup, use complete formulas or vetted constant-time code, and never feed the raw x-coordinate to anything but a KDF [@sp-800-56a-r3], [@rcb-2016]. Reach for X448 only when a policy demands a larger margin.

| Curve | Form | Security | Cofactor | Constant-time | Twist secure | Element | Safety burden | Compliance | | --- | --- | --- | --- | --- | --- | --- | --- | | X25519 | Montgomery | ~128-bit | 8 | By construction | Yes | 32 B | Low: reject zero, KDF the output | Not required for certificates | | P-256 ECDH | Short Weierstrass | ~128-bit | 1 | With care | No | 33 B compressed | High: validate point and subgroup, constant-time | FIPS-mandated | | X448 | Montgomery | ~224-bit | 4 | By construction | Yes | 56 B | Low | Extra-margin option |

The one rule that survives every row: never use the raw shared x-coordinate as a key. Run it through a KDF, as this series' key-derivation installment argues.

Signatures: Ed25519 for new work, ECDSA where mandated

For new designs, Ed25519 wins on the axis that matters -- it removes the per-signature randomness that has destroyed more ECDSA keys than any cryptanalysis, and its complete addition law removes the exceptional cases [@rfc8032]. Use ECDSA over P-256 only where a standard names it, and then defend it deliberately: RFC 6979 deterministic nonces, constant-time scalar multiplication, and point validation [@rfc6979]. BIP-340 Schnorr is the right choice for new Bitcoin work and nowhere else [@bip-340].

Scheme	Nonce	Nonce-failure blast radius	Addition law	Malleability rule	Sizes	Batch verify	Compliance
Ed25519	Deterministic (hash-based)	None by construction	Complete Edwards	Enforce canonical S and cofactor checks	64 B sig, 32 B key	Yes	FIPS 186-5 approved
ECDSA P-256	Random, or RFC 6979 deterministic	Catastrophic: a reused or biased nonce leaks the key	Incomplete Weierstrass	Low-S recommended	~64-72 B sig, 33 B key	Limited	FIPS-mandated
BIP-340 Schnorr	Deterministic with auxiliary randomness	Reduced versus ECDSA	secp256k1	Fixed by encoding	64 B sig, 32 B x-only key	Yes	Bitcoin only

Group abstraction: ristretto255 over hand-rolled cofactor clearing

If you are building a new prime-order protocol -- a PAKE, a VRF, an OPRF, a threshold scheme, a zero-knowledge system -- build it on ristretto255 rather than hand-rolling cofactor clearing over raw edwards25519 [@rfc9496]. Decode-is-validation removes the last implementation choice that could go wrong, which is the entire point of Generation four.

The cross-primitive view

Two comparisons sit outside this table but belong in your head. Against RSA, elliptic curves do the same public-key jobs with smaller keys and faster operations, and they share the reframe this series keeps returning to: the math was never the thing that broke. Against post-quantum schemes, ECC is not yet replaced -- the classical curve is the defense-in-depth half of a hybrid, present precisely because a lattice KEM is young and a recorded handshake is forever. Neither comparison changes a single row above; both change how long these rows stay valid.

Every row here assumes the curve's discrete log is genuinely hard. That assumption has a precise boundary -- and a hard expiration date.

9. Where the Hardness Lives, and Its Ceiling

Here is the uncomfortable pair of facts a first course tends to skip. For a well-chosen curve, the security level is exactly the generic bound -- there is no hidden margin above it, but nothing better is known either. And that entire edifice falls to a quantum computer in polynomial time.

The generic bound, matched from both sides

The best classical attacks on a well-chosen curve are generic -- they work in any group and know nothing special about elliptic curves. Pollard rho finds a discrete log in about $\sqrt{n}$ group operations for a group of order $n$, using parallel collision search with distinguished points and almost no memory [@safecurves-rho]. Pohlig-Hellman reduces the problem to the largest prime factor of the group order, which is the deep reason prime order matters: a smooth order is a soft target. Against those upper bounds sits a matching lower bound -- Shoup (1997) and Nechaev (1994) proved that any generic algorithm needs $\Omega(\sqrt{n})$ operations. Upper and lower bounds meet.

Note: For a well-chosen curve, the security level is the generic square-root bound itself: about $\sqrt{n}$ work, matched by a proven lower bound. That is solid -- but it means there is no safety margin hiding above the number, and, as we are about to see, a quantum cliff waiting below it.

That closed gap is genuinely unusual among cryptographic assumptions, and it is the whole reason a 256-bit curve is trusted at the 128-bit level.Most hardness assumptions leave a gap between the best known attack and the best provable lower bound. For generic ECDLP on a well-chosen curve, the two coincide at $\sqrt{n}$. You are trusting a bound that is tight, not merely unbroken. The reason no better attack exists is the absence we opened with: no index-calculus method is known that beats $\sqrt{n}$ on prime-field curves. Semaev's summation polynomials and the programs built on them have not produced one. The economic case of Section 2 is really this limit, seen from the other side.

Symmetric security	Elliptic-curve size	RSA / finite-field DH modulus
112-bit	224-bit	2048-bit
128-bit	256-bit	3072-bit
192-bit	384-bit	7680-bit
256-bit	512-bit	15360-bit

The table is the standard security-level mapping, and its middle row is the sentence everyone quotes: 256-bit ECC stands in for 3072-bit RSA at roughly 128-bit strength, because the curve attacker pays $\sqrt{n}$ while the RSA attacker enjoys sub-exponential index calculus [@safecurves-rho], [@miller-1985].

What "well-chosen" rules out

The generic bound holds only for curves that avoid a short list of special structures, and each item on that list is a curve that looks fine but collapses. The MOV reduction (Menezes, Okamoto, and Vanstone, 1993) maps a curve with low embedding degree -- supersingular curves are the extreme case -- into a finite field where index calculus applies, collapsing its ECDLP [@mov-1993]. Anomalous, or trace-one, curves, where the number of points equals the field prime, admit an additive transfer that solves the discrete log in polynomial time (Smart; independently Satoh-Araki and Semaev) [@smart-1999]. And the small-subgroup and twist conditions of Section 4 round out the list. "Well-chosen" is exactly "avoids these."

The one bound that is not the square root

Every limit so far is classical. The exception is the one that sets the deadline. Shor's algorithm (1997) solves both factoring and discrete logarithms -- including ECDLP -- in polynomial time on a fault-tolerant quantum computer [@shor-1997]. Resource estimates make it concrete: at equal classical security ECC is the easier quantum target -- breaking a 256-bit curve needs roughly 2,330 logical qubits and about $1.3 \times 10^{11}$ Toffoli gates, versus roughly 6,100 logical qubits for RSA-3072, so ECC's smaller keys are the smaller quantum target and fall first [@roetteler-2017]. No choice of curve escapes this; it is a property of the group structure Shor exploits, not of any parameter.

This is why today's ECDH traffic is already on a clock, before any quantum computer exists. An adversary records your X25519 handshakes now and decrypts them the day a machine can run Shor at scale. The recorded session is not refreshed; its forward secrecy is only as good as the assumption that the curve stays hard forever -- which Shor says it will not [@shor-1997]. That single fact, not any classical advance, is what put the hybrid transition of Section 7 on every roadmap.

If the mathematics is this settled -- a closed classical bound and a known quantum ceiling -- then everything still open lives in the code, the trust, and the migration. That is where the real frontier is.

10. What Remains Genuinely Unsettled

The core science is settled, and the failure catalog and its defenses are not in dispute. What is open sits at the edges -- and two of these are live in your dependency tree right now.

Constant-time is a perpetually moving target

Montgomery ladders and complete formulas close the obvious channels, and still the leaks keep reappearing at finer grain. Minerva recovered secp256r1 private keys from about 2,100 signatures by measuring how the nonce's bit-length shifted the signing time [@minerva-2020]. TPM-Fail pulled 256-bit ECDSA keys out of an Intel firmware TPM and an STMicroelectronics TPM despite their certifications [@tpm-fail-2020]. LadderLeak broke ECDSA with less than one bit of nonce leakage [@ladderleak-2020]. There is no general, machine-checkable guarantee of constant-time behavior that survives arbitrary compilers and microarchitectures. The lattice step that turns these leaks into keys belongs to this series' nonce-reuse post; the open problem here is the leakage itself.

The NIST-curve trust question is sociological, not solved

No weakness has ever been demonstrated in P-256 or P-384, and they remain FIPS-mandated [@safecurves-index]. Yet Dual EC DRBG proved that NIST-supplied constants could be poisoned, which is why a large part of the community prefers rigid curves for new work [@shumow-ferguson-2007]. This is not the kind of question a proof can close -- you cannot prove the absence of a motive. The practical resolution is a posture, not a theorem: prefer 25519 for new designs, use P-256 with care where compliance compels it.

The cofactor long tail

RFC 9496 eliminates the cofactor failure class -- for the protocols that adopt it [@rfc9496]. The long tail of deployed EdDSA and X25519 protocols that predate or simply ignore ristretto255 remains a live source of subgroup bugs, with Monero the canonical example [@loup-vaillant-cofactor]. This is a migration problem, not a research problem, which somehow makes it more stubborn rather than less.

The post-quantum migration, past the easy part

Hybrid key agreement is deploying fast [@cloudflare-radar-2025], but the interesting questions start after the first hybrid ships: how long ECC's small keys keep it useful inside hybrids, whether and when to drop the classical half, which combiner and negotiation are right, and the genuinely harder migration on the signature side, where the post-quantum replacements are large and awkward.

Formal verification at scale

fiat-crypto ships proven-correct field arithmetic for Curve25519 and P-256 inside BoringSSL, which is a real milestone [@fiat-crypto]. But end-to-end verification -- field, then group law, then protocol, then constant-time behavior on the actual target instruction set -- is not yet routine. The field-arithmetic layer is solved; the stack above it is not.

There is also a structural non-problem worth naming, so you can stop worrying about it: no one has found a faster-than-square-root attack on prime-field ECDLP, and the summation-polynomial work that raised hopes lives in binary and composite fields -- one more reason NIST now steers new work to prime-field curves: SP 800-186 deprecates the binary and Koblitz curves, retaining them only for legacy interoperability, and FIPS 186-5 dropped them from approval for new signatures [@sp-800-186], [@fips-186-5].

Note: Of the five open fronts, two are the ones likely to be in your stack today: a non-constant-time scalar multiplication somewhere in a dependency, and a protocol built on raw edwards25519 that never adopted ristretto255. The other three are the industry's problem. These two are yours.

Enough theory and frontier. Here is the entire article compressed into rules you can apply on Monday -- each one a named break from the catalog, turned inside out.

11. The Practical Guide and the Misuse Catalog

Every rule below is one row of the failure catalog, inverted. If you remember nothing else from this article, remember the decision tree.

flowchart TD A{"What is the job?"} --> B["Key agreement"] A --> C["Signature"] A --> D["Prime-order protocol"] A --> E["Long-lived confidentiality"] B --> B1{"FIPS-mandated curve?"} B1 -->|No| B2["X25519, reject zero output, KDF the result"] B1 -->|Yes| B3["P-256 ECDH, validate point and subgroup, constant-time, KDF"] C --> C1{"New design?"} C1 -->|Yes| C2["Ed25519, enforce canonical S, strict library"] C1 -->|Mandated| C3["ECDSA P-256, RFC 6979 nonce, constant-time, validate points"] D --> D1["Build on ristretto255 or decaf448"] E --> E1["Wrap X25519 in the ML-KEM-768 hybrid"]

The decision rules, in words

For key agreement, reach for X25519 by default: constant-time by construction, twist security handling hostile inputs, and one operational rule -- run the shared secret through a KDF and reject the all-zero output, never using the raw x-coordinate as a key [@rfc7748]. Use P-256 ECDH only where FIPS compels it, and then validate the peer point on the curve and in the prime-order subgroup, use complete formulas or a vetted constant-time library, and KDF the result [@sp-800-56a-r3], [@rcb-2016].

For signatures, choose Ed25519 for new designs -- deterministic nonces, a complete addition law, and a strict library that enforces the canonical-S check [@rfc8032]. Use ECDSA over P-256 only where mandated, with RFC 6979 deterministic nonces, constant-time scalar multiplication, and point validation [@rfc6979].

For point handling, always validate untrusted points -- on the curve and in the right subgroup -- remember that clamping is not validation, and never accept explicit curve parameters from a certificate or message; use named curves only, which is the whole lesson of Curveball [@cve-2020-0601]. For new prime-order protocols, build on ristretto255 or decaf448 instead of hand-rolling cofactor clearing [@rfc9496]. For long-lived confidentiality, deploy the X25519 plus ML-KEM-768 hybrid now rather than waiting for a pure post-quantum replacement.

Job	Use	Key parameters	Because (the named break)
Key agreement (default)	X25519	Reject zero output, KDF the secret	Raw-x misuse, twist security handles inputs [@rfc7748]
Key agreement (FIPS)	P-256 ECDH	Validate point and subgroup, constant-time	Invalid-curve TLS recovery [@jager-2015], timing [@brumley-tuveri-2011]
Signature (new)	Ed25519	Canonical S, strict library	Nonce disaster and malleability [@rfc8032]
Signature (mandated)	ECDSA P-256	RFC 6979 nonce, constant-time, validate	Nonce reuse or bias [@rfc6979]
Prime-order protocol	ristretto255 / decaf448	Decode is validation	Cofactor double-spend [@loup-vaillant-cofactor]
Certificates	Named curves only	Reject explicit parameters	Curveball [@cve-2020-0601]
Long-lived secrecy	X25519 + ML-KEM-768	Hybrid combiner now	Harvest-now-decrypt-later [@shor-1997]

Why "clamping is not validation" is a rule, not a slogan

The single most common confusion deserves a demonstration rather than an assertion. On a curve with a cofactor, an attacker can add a low-order point to a legitimate one. Clamping your secret scalar hides that tampering inside a Diffie-Hellman product -- so clamping alone cannot even detect it -- while an explicit prime-order subgroup check catches it immediately.

{` // Toy curve y^2 = x^3 + x over F_53. Group order 68 = 4 * 17 (cofactor 4, prime l = 17). const p = 53n, a = 1n, ELL = 17n; const mod = x => ((x % p) + p) % p; function inv(x){ let r=1n,b=mod(x),e=p-2n; while(e>0n){ if(e&1n) r=mod(rb); b=mod(bb); e>>=1n;} return r; } const eq = (P,Q) => (P===null&&Q===null) || (P&&Q&&P[0]===Q[0]&&P[1]===Q[1]); function add(P,Q){ if(P===null) return Q; if(Q===null) return P; const [x1,y1]=P,[x2,y2]=Q; if(x1===x2 && mod(y1+y2)===0n) return null; const s=(x1===x2&&y1===y2)? mod((3nx1x1+a)inv(2ny1)) : mod((y2-y1)inv(x2-x1)); const x3=mod(ss-x1-x2); return [x3, mod(s*(x1-x3)-y1)]; } function mul(k,P){ let R=null,b=P; while(k>0n){ if(k&1n) R=add(R,b); b=add(b,b); k>>=1n;} return R; }

const P = [6n,13n]; // legitimate prime-order point (order 17) const Low = [0n,0n]; // low-order point (order 2, divides the cofactor 4) const tampered = add(P, Low); // attacker adds a low-order component console.log('P and P+Low are distinct encodings?', !eq(P, tampered));

// Clamping forces the secret to a multiple of the cofactor, so it HIDES the tamper: const kClamped = 4n * 5n; // 20, a multiple of the cofactor 4 console.log('clamped [k]P equals clamped k?', eq(mul(kClamped,P), mul(kClamped,tampered)));

// The real defence is an explicit prime-order subgroup check [l]X == identity: console.log('subgroup check -- [l]P is identity?', mul(ELL,P) === null); console.log('subgroup check -- l is identity?', mul(ELL,tampered) === null); `}

The clamped multiplication reports the tampered point as identical, and only the $[\ell]X = \mathcal{O}$ subgroup check flags it. That is the Monero bug in nine lines: the code that needed to reject the tampered element was not the clamp, it was the subgroup test.

On most systems you can see the real curves in seconds. `ssh -Q key` and `ssh -Q kex` list your SSH key and key-exchange algorithms -- look for `ssh-ed25519` and a hybrid like `mlkem768x25519-sha256`. For TLS, `openssl s_client -connect example.com:443 -tls1_3` reports the negotiated group, often `X25519` or the hybrid `X25519MLKEM768`. Versions differ, but these tell you which curve and which post-quantum wrapper your traffic actually uses today, rather than which one you assume it does.

The misuse catalog

Each antipattern below is exactly one violated rule, and each maps to a named break you have now met:

Using the raw ECDH x-coordinate as a key. Run it through a KDF instead; the shared secret is an input to key derivation, not a key.
Treating clamping as validation. Clamping constrains your secret; it does not filter a hostile input point (Monero [@loup-vaillant-cofactor]).
Skipping point validation on Weierstrass curves. The formula ignores $b$, so an off-curve point is processed on a weaker curve (invalid-curve, Jager et al. [@jager-2015]).
Accepting explicit curve parameters. Trusting a generator you did not choose forges chains (Curveball [@cve-2020-0601]).
Non-constant-time or unvalidated P-256. Data-dependent timing is an oracle even with honest inputs (Brumley-Tuveri [@brumley-tuveri-2011]).
Ignoring the cofactor or non-canonical encodings in edwards25519 protocols. Low-order points and duplicate encodings break uniqueness (Monero [@loup-vaillant-cofactor]).
Rolling your own curve arithmetic. Use fiat-crypto-backed libraries; hand-written field code is where constant-time dies [@fiat-crypto].
Reusing or biasing ECDSA nonces. A repeated or leaky nonce hands over the key through a lattice (RFC 6979 fixes the generation half [@rfc6979]).

Note: If your time is limited, hunt these first: the raw ECDH x-coordinate used directly as a key; a P-256 ECDH path that neither validates the peer point nor runs in constant time; and any code path that accepts explicit curve parameters instead of a named curve. Those three account for most real elliptic-curve findings in production code.

Read the whole failure catalog again as this checklist, and one sentence remains true of every row.

12. The Curve Was Hard; The Gap Was Soft

Return to where we started. The NSA phoned Microsoft, and nobody solved a discrete logarithm [@cve-2020-0601]. That is not an anomaly; it is the whole pattern in one image. Across three decades of deployed elliptic-curve cryptography, almost no real break ever inverted the function the textbooks call hard. Attackers fed the implementation a point it should have rejected -- off the curve, or of small order, or a certificate's own untrusted generator (invalid-curve, small-subgroup, Curveball) [@jager-2015]. They added a low-order component that clamping never filtered (Monero) [@loup-vaillant-cofactor]. Or they simply read the clock while the code multiplied a secret (Brumley-Tuveri, and the Minerva-TPM-Fail-LadderLeak lineage) [@brumley-tuveri-2011].

Key idea: The curve was hard; the gap was soft. ECDLP-hardness bought the small keys and the speed, and it held every single time. The breaks all lived in the code, the trust, and the encoding around the curve -- the space between a hard problem and a safe system.

So "using elliptic curves safely" is not one decision but a stack: the right curve for the job, point and subgroup validation on every untrusted input, constant-time scalar multiplication, and -- the deepest lesson the evolution kept teaching -- a preference for constructions where the dangerous case cannot be reached at all, as ristretto255 makes literal [@rfc9496]. Remove any single layer and the gap reopens. And the horizon is honest about its own limit: the destination is not a better curve, because no curve survives Shor [@shor-1997]. It is wrapping the curve in a post-quantum hybrid and buying time.

The private curve stayed hard; the code around it confessed.

Which makes the diagnostic question we opened with finally answerable. When a point you did not generate arrives, what does your code reveal about it -- through its result, its timing, or the parameters it trusts? Using elliptic curves safely is the discipline of making that answer nothing an attacker can measure.

No weakness has ever been shown in P-256 or P-384. The real concern is *rigidity*: their generator and coefficient come from an unexplained seed, and the Dual EC DRBG affair proved that NIST-supplied constants *could* be poisoned in some other primitive. That history is why many engineers prefer rigid curves like Curve25519 for new designs. But "prefer 25519" is a different claim from "P-256 is broken," and only the first one is supported. No -- it is *harder to implement safely*, not broken. Its naive addition formulas have exceptional cases and it lacks twist security, so a careless implementation can leak. Complete addition formulas (Renes-Costello-Batina) and vetted constant-time libraries close those gaps. Implemented carefully, P-256 is secure and remains FIPS-mandated for certificates. For plain Diffie-Hellman, no on-curve check is required: Curve25519's twist security means a hostile input still lands in a hard group, so you just reject the all-zero output and run the result through a KDF. But low-order points still matter for protocols that need a genuine prime-order group. Validate or clear the cofactor there, or better, build on ristretto255 and let decoding do the validation. No. Clamping constrains *your own* secret scalar so a low-order component contributes nothing to your Diffie-Hellman product. It does nothing to validate a malicious input point, and it does not make a tampered point canonical. The Monero cofactor bug is exactly this confusion -- the fix was an explicit prime-order subgroup check, not clamping. Only if you are doing blockchain interoperability. secp256k1 is the Bitcoin and Ethereum curve; it is essentially absent from web PKI and TLS. For general key agreement use X25519, and for certificates use P-256. Reaching for secp256k1 outside the blockchain context is almost always a sign something is being reinvented. Use Ed25519 for new work: deterministic nonces remove the per-signature randomness that has destroyed more ECDSA keys than any cryptanalysis, the addition law is complete and constant-time, and EdDSA has been FIPS-approved since 186-5. Use ECDSA over P-256 only where a standard mandates it, and then with RFC 6979 deterministic nonces, constant-time scalar multiplication, and point validation. No. Shor's algorithm breaks the elliptic-curve discrete log in polynomial time on a fault-tolerant quantum computer, and at equal classical security ECC is the easier quantum target -- a 256-bit curve needs about 2,330 logical qubits versus roughly 6,100 for RSA-3072, so it falls first. Because recorded traffic can be decrypted later, deploy the X25519 plus ML-KEM-768 hybrid now rather than waiting for a pure post-quantum replacement.

RSA Is a Trapdoor, Not a Cryptosystem: OAEP, PSS, and the 25-Year Padding-Oracle Lineage

noreply@paragmali.com (Parag Mali) — Sat, 11 Jul 2026 09:07:03 GMT

**RSA is a trapdoor permutation, not a cryptosystem** -- and almost every real-world RSA break came from treating the bare trapdoor as if it were already secure. Textbook RSA is deterministic and malleable. PKCS#1 v1.5 *encryption* turns "is this padding valid?" into a decryption oracle that leaked through ever-quieter channels for 25 years, from Bleichenbacher (1998) [@bleichenbacher98] through DROWN [@drown16] and ROBOT [@robot18] to Marvin's pure-timing attack (2023) [@marvin23]. "Done right" is a stack that must all hold at once: the right scheme (**OAEP** for encryption, **PSS** for new signatures), the right parameters (2048-bit floor, `e = 65537`, CSPRNG primes), and a **constant-time, fault-checked implementation** -- because security is a property of the scheme *and* its implementation, not of the math. And keep one split absolute: v1.5 *signatures* are unbroken and still dominant (verify strictly), while v1.5 *encryption* must be retired.

1. The Modulus Was Never Factored

In 2018, three researchers signed a message with the private key behind facebook.com's TLS certificate [@robot18]. They never factored Facebook's 2,048-bit modulus. Nobody has ever publicly factored a 2,048-bit modulus -- the public record stands at 829 bits, and that took roughly 2,700 core-years [@rsa250].

Instead, they asked one of Facebook's front-end servers the same yes-or-no question a few hundred thousand times -- is this padding valid? -- and let the pattern of answers spell out the secret [@robot18]. The attack they used was already nineteen years old [@robotsite]. Five years later, a Red Hat engineer reproduced the same break against software that was supposed to be immune, using nothing but a stopwatch, and called it Marvin [@marvin23].

Notice what did not happen. No prime was recovered. No number was factored. The RSA problem -- invert c = m^e mod N without the private key -- stood exactly as hard the day after as the day before. What broke was the server's reaction: the way it answered a question about a ciphertext it did not create.

That is the whole subject in one sentence. Textbook RSA -- the bare m^e mod N you meet in a first course -- is a trapdoor permutation, not a cryptosystem. It is a beautiful one-way function with a secret shortcut, and nothing more.

Everything that turns it into secure encryption or a secure signature lives around the permutation, in the padding, the parameters, and the code that runs the operation. Almost every famous RSA disaster of the last three decades is a variation on one theme: someone used the bare trapdoor as if it were already a cryptosystem, or let the machinery meant to secure it confess -- through an error message, a network timeout, a microsecond of timing, or an injected fault -- whether a check had passed.

The math was never broken. The padding check told the attacker whether it passed.

So here is the diagnostic question this article keeps asking, the one that unlocks the entire failure catalog: when a ciphertext or signature it did not create arrives, what does the receiver reveal about it -- through its answer, its timing, or its faults -- before and after it has checked the padding? Four names you may know as headlines -- Bleichenbacher, DROWN, ROBOT, Marvin -- are four answers to that question, each leaking the same single bit through a quieter channel than the last. To see why one yes-or-no question about padding is enough to reconstruct a session key, you first have to see what RSA actually is, and what it is not.

2. A Trapdoor for Diffie-Hellman's Challenge

In 1976, Whitfield Diffie and Martin Hellman wrote down the shape of a future that did not yet have an engine. Their paper New Directions in Cryptography described public-key encryption and, remarkably, the idea of a digital signature -- a value only you can produce but anyone can check [@dh76]. What they could not supply was a concrete trapdoor: a function easy to compute in one direction, hard to invert, yet effortless to invert if you hold a secret. Their paper is a challenge with a hole in it, and the hole is shaped exactly like RSA.

A year later, Ron Rivest, Adi Shamir, and Leonard Adleman filled it. Pick two large primes $p$ and $q$ and set $N = pq$. Choose a public exponent $e$, and compute a private exponent $d$ with $ed \equiv 1 \pmod{\lambda(N)}$. Publish $(N, e)$; keep $d$, $p$, and $q$ secret. To encrypt a message represented as a number $m < N$, raise it to the public exponent; to decrypt, raise the result to the private one [@rsa78]:

$$c = m^e \bmod N, \qquad m = c^d \bmod N.$$

Why does the second operation undo the first? Because of Euler's theorem (1763): for any $m$ coprime to $N$, $m^{\phi(N)} \equiv 1 \pmod N$, with the totient $\phi(N) = (p-1)(q-1)$. Its Carmichael-function refinement (Carmichael, 1910) gives the same identity for the smaller $\lambda(N) = \mathrm{lcm}(p-1, q-1)$. Since $ed \equiv 1 \pmod{\lambda(N)}$, we have $ed = 1 + k\lambda(N)$ for some integer $k$, so $c^d = m^{ed} = m \cdot (m^{\lambda(N)})^k \equiv m \pmod N$. The exponents cancel, and the plaintext falls out -- but only for someone who could compute $d$, and computing $d$ requires $\lambda(N)$, which requires the factors of $N$. That is the trapdoor: the factorization of $N$ is the secret that inverts the permutation.The original 1978 paper used $\phi(N) = (p-1)(q-1)$, Euler's totient. Modern standards use the smaller Carmichael function $\lambda(N) = \mathrm{lcm}(p-1, q-1)$, which yields a smaller valid $d$ and the same correctness. Either works; $\lambda(N)$ is now the convention in FIPS 186-5.

A function that is easy to evaluate in the forward direction, computationally hard to invert without a secret, and easy to invert with it. RSA's forward map is x maps to x raised to the e, modulo N; the trapdoor that inverts it is the factorization of N. A trapdoor permutation is a primitive, not a complete encryption scheme -- it says nothing about hiding partial information or resisting tampering.

The elegance that made RSA famous is that the same operation both encrypts and signs. Swap the roles of the exponents: to sign a message, raise it to the private exponent, $s = m^d \bmod N$, an act only the key holder can perform; to verify, raise the signature to the public exponent and check that $s^e \equiv m \pmod N$, which anyone can do [@rsa78]. One modular exponentiation, read in two directions, is both a lock only you can open and a seal only you can stamp. Boneh's survey calls this duality the source of both RSA's reach and its long catalogue of misuse [@boneh99].

Two engineering choices from this era matter later, because each returns as an attack surface. The first is the public exponent. Almost every deployed RSA key uses $e = 65537$.65537 is the Fermat prime $F_4 = 2^{16} + 1$. Written in binary it is 1 followed by fifteen 0s and a final 1 -- a Hamming weight of just 2 -- so public-key operations cost only sixteen squarings and one multiplication. Small enough to be fast, large enough to dodge the low-exponent traps of $e = 3$. The second is how the private operation is computed. Rather than one exponentiation modulo $N$, implementations work modulo $p$ and modulo $q$ separately and recombine, using the Chinese Remainder Theorem for roughly a fourfold speedup.

A technique for computing the private operation c raised to the d, modulo N, by working separately modulo p and modulo q on half-size numbers and then recombining the two halves. It cuts the cost of decryption and signing by about four times. Because it splits the secret operation across the two prime factors, it also opens a physical-fault attack surface that the single-modulus form does not have.

Hold those two choices in mind: the small public exponent and the CRT shortcut will each come back to bite an implementation that treats them casually. But the deepest seed was planted in the 1978 paper itself, and it was invisible at the time.

Rivest, Shamir, and Adleman demonstrated the trapdoor on raw message numbers. That bare application is deterministic, algebraically malleable, and carries structure an attacker can exploit -- three properties that, in 1978, looked like harmless features of a clean mathematical object. One operation, elegant enough to both lock and sign. So why is calling that operation directly -- what cryptographers now dismiss as "textbook RSA" -- treated as a bug in every serious codebase?

3. Why Textbook RSA Is Not a Cryptosystem

Encryption has a minimum bar -- lower than most people think -- and textbook RSA fails to clear it. The bar is called IND-CPA: an adversary who picks two plaintexts and receives the encryption of one cannot tell which, better than a coin flip. (Part 1 of this series builds that definition carefully; here we only need its consequences.) Bare RSA loses this game three separate ways, each a direct consequence of the naked permutation.

It is deterministic. The same plaintext always encrypts to the same ciphertext, because $m^e \bmod N$ is a function with no randomness in it. That is fatal whenever the message space is small or guessable. Suppose a server encrypts a one-word trading instruction, either BUY or SELL, under a known public key. An eavesdropper does not need to decrypt anything: they encrypt both candidate words, compare against the captured ciphertext, and read the plaintext with zero queries. Determinism can never be semantic security -- this is structural, not a missing optimization you can bolt on later.

It is malleable. RSA is multiplicatively homomorphic: multiply a ciphertext by $k^e$ and the plaintext is silently multiplied by $k$.

A cipher is malleable when an attacker can transform a ciphertext into another valid ciphertext whose plaintext is a predictable function of the original -- without decrypting anything. RSA satisfies the multiplicative relation: the encryption of m times the encryption-form of k decrypts to m times k, modulo N. Useful for some protocols, catastrophic for confidentiality, and the exact algebraic lever Bleichenbacher later pulls.

That relation is not a curiosity; it is the crowbar behind the entire padding-oracle lineage. The demonstration below uses toy primes so the arithmetic is legible in a browser, but the algebra is identical at 2,048 bits.

{` // Toy RSA: p=61, q=53, N=3233, e=17, d=2753. NOT secure sizes -- for illustration. function modpow(base, exp, mod) { base = base % mod; let r = 1n; while (exp > 0n) { if (exp & 1n) r = (r * base) % mod; base = (base * base) % mod; exp >>= 1n; } return r; } const N = 3233n, e = 17n, d = 2753n; const m = 42n;

// 1) Deterministic: encrypting the same message twice gives the same ciphertext. const c1 = modpow(m, e, N), c2 = modpow(m, e, N); console.log('encrypt 42 twice ->', c1.toString(), c2.toString(), '| identical?', c1 === c2);

// 2) Malleable: multiply the ciphertext by k^e and the plaintext scales by k. const k = 2n; const cForged = (c1 * modpow(k, e, N)) % N; // attacker never decrypts const mForged = modpow(cForged, d, N); // what the victim would recover console.log('tampered ct decrypts to', mForged.toString(), '= (42*2) mod N =', ((m * k) % N).toString()); `}

It leaks structure for small exponents and short messages. If a message is short enough that $m^e < N$, no modular reduction ever happens, so recovering $m$ is a plain integer $e$-th root -- no factoring required.

Johan Hastad showed in 1988 that if the same message is broadcast to $e$ recipients under $e = 3$, the plaintext falls out by the Chinese Remainder Theorem [@hastad88]. Don Coppersmith sharpened this in 1996 into a lattice method that recovers a message whenever the unknown part is smaller than $N^{1/e}$, breaking "stereotyped" messages with a small hidden field [@coppersmith97]. Every one of these is an attack on textbook RSA, not on the RSA problem itself -- the factoring assumption stays intact while the plaintext walks out the front door [@boneh99].

So a trapdoor permutation is not a cryptosystem. To become one, it needs a transformation applied to the message before the permutation -- padding -- and that padding has to do three separable jobs.

Key idea: A trapdoor becomes a cryptosystem only when the padding does three separable jobs: (1) add randomness, so equal plaintexts produce different ciphertexts; (2) add checkable redundancy, so the receiver can detect a tampered or malformed ciphertext; and (3) -- the job the field kept forgetting -- be implemented so that no observable behaviour reveals whether that check passed.

The first two jobs are about the scheme. The third is about the implementation, and the entire history of RSA breaks is the field learning, over and over, that the third job is not optional. Hold that third clause; it is the trap the next section springs.

Note: If your code invokes a modular-exponentiation "textbook RSA" function directly on a message -- no OAEP, no PSS, no padding layer -- it is broken before it ships. Every serious library hides the bare permutation precisely so that no application accidentally uses it as encryption or a signature.

The first widely deployed answer to those three jobs was PKCS#1 version 1.5, specified by RSA Laboratories in the early 1990s and republished as RFC 2313 in 1998 [@rfc2313]. For encryption, it wraps the message as 0x00 || 0x02 || PS || 0x00 || M, where PS is at least eight nonzero random bytes -- that random padding supplies job one [@rfc8017]. For signatures it uses a different, fixed frame, 0x00 || 0x01 || 0xFF...0xFF || 0x00 || DigestInfo, whose rigid structure supplies job two [@rfc8017].

On paper it fixed textbook RSA's two headline problems: the random bytes killed determinism, and the fixed structure gave the receiver something to check. For five years, it looked as if PKCS#1 v1.5 had turned the trapdoor into a cryptosystem. The structure was supposed to add security. In 1998, Daniel Bleichenbacher showed that the structure was the vulnerability.

4. The Atom and Its Echoes: The Encryption Padding-Oracle Lineage

Before the detailed walk, here is the whole story on one timeline -- two intertwined tracks, the schemes on one side and the breaks on the other, with each break forcing the next response.

timeline title RSA schemes and their breaks, 1976 to 2026 1977 : RSA trapdoor answers the Diffie-Hellman challenge 1990 : Wiener breaks small private exponents 1993 : PKCS#1 v1.5 padding deployed 1997 : Bellcore CRT fault factors N from one signature 1998 : Bleichenbacher padding oracle : OAEP standardized in response 2001 : Manger reopens the oracle inside OAEP decoders 2003 : Remote timing attacks proven practical : PSS standardized 2016 : DROWN revives the oracle through SSLv2 2017 : ROCA factors structured Infineon keys 2018 : ROBOT signs with the facebook.com key : TLS 1.3 deletes RSA key exchange 2023 : Marvin recovers keys by timing alone 2024 : NIST IR 8547 sets the quantum retirement clock 2025 : CFRG draft moves to deprecate v1.5 encryption

Bleichenbacher's insight was not about RSA the mathematics at all. It was about a server's manners. A TLS server in the 1990s received an RSA-encrypted pre-master secret, decrypted it, and checked whether the result had valid PKCS#1 v1.5 padding -- did it start with the bytes 0x00 0x02, with a nonzero pad and a delimiter in the right place? If not, it returned an error. That error, however polite, answered a question the attacker was not supposed to be able to ask: was this the encryption of a well-formed message? And because RSA is malleable, the attacker could turn that one bit of feedback into a full decryption.

Here is the mechanism. The attacker holds a target ciphertext $c$ that encrypts an unknown $m$ -- say, a captured TLS session key. They pick a value $s$, compute $c \cdot s^e \bmod N$, and send it to the server. By the homomorphic property, the server is really decrypting $m \cdot s \bmod N$. If the server reports valid padding, the attacker learns that $m \cdot s \bmod N$ begins with 0x00 0x02 -- which means it lies in the interval $[2B, 3B)$, where $B = 2^{8(k-2)}$ for a $k$-byte modulus.

Each conforming $s$ is a linear inequality on the secret $m$. Collect enough of them, adaptively choosing each new $s$ to bisect the surviving range, and the interval of possible $m$ collapses to a single value. Bleichenbacher's 1998 paper showed this takes on the order of $2^{20}$ -- about a million -- queries for a 1,024-bit key, and the private key is never touched [@bleichenbacher98].

sequenceDiagram participant A as Attacker participant S as Decrypting server Note over A: Holds target ciphertext c, wants secret m A->>S: Submit c times s to the e, for chosen s S->>S: Decrypt and check PKCS#1 v1.5 padding alt Decrypted value starts with 00 02 S-->>A: Conforming, no error or fast reply Note over A,S: Attacker learns m times s mod N is in 2B to 3B else Padding invalid S-->>A: Non-conforming, error, alert, or slow reply end Note over A: Narrow the possible range of m, choose next s A->>S: Repeat, roughly a million queries in total Note over A: Range collapses to one value, plaintext m recovered

A generation of engineers filed this under "performance" or "obscure edge case." It is neither. It is a correctness and security failure of the construction: the receiver's validity check, exposed through any observable side effect, is a decryption oracle for chosen ciphertexts. That is the symmetric-side padding-oracle attack of Part 6, transplanted to public-key land -- the same disease, a different organ.

Any observable behaviour -- an error message, a network reset, an injected fault, or a difference in response time -- that reveals whether a decrypted ciphertext had valid padding. Because the attacker chooses the ciphertexts, a padding-validity signal becomes a decryption oracle: enough yes-or-no answers reconstruct the plaintext without ever recovering the private key.

The break violates the strongest standard security goal for encryption, IND-CCA2, in which the adversary may submit chosen ciphertexts to a decryption oracle and still must not learn anything about a challenge plaintext.

The security goal a padding oracle destroys: an adversary who can submit arbitrary ciphertexts for decryption still cannot distinguish which of two chosen plaintexts a challenge ciphertext encrypts. PKCS#1 v1.5 encryption fails this because its validity check leaks. Part 1 of this series develops the full definition.

The scheme-level fix arrived almost immediately. In direct response to Bleichenbacher, RSA Laboratories standardized a new encryption padding, RSAES-OAEP, in PKCS#1 v2.0 (RFC 2437) later in 1998 [@rfc2437]. OAEP's design goal, spelled out in Section 6, is that any tampered ciphertext decodes to unstructured noise, so there is no "conformant" interval left to test -- the oracle has nothing to grade. It fixed the scheme. What it did not, and could not, fix by itself was the decoder.

James Manger proved that in 2001. OAEP's decoding has two distinct early failure modes: the recovered integer can be too large to fit the byte block, or it can fit but fail the padding check. A decoder that distinguishes those two cases through different errors or different timing hands the attacker a new oracle -- and Manger's variant is dramatically cheaper than Bleichenbacher's, needing on the order of $\log_2 N$ queries, roughly 2,048 for a 2,048-bit key, instead of $2^{20}$ [@manger01]. OAEP, the provably secure scheme, re-opened the identical class of attack through an imperfect implementation. The lesson the field wrote down, and then kept having to re-learn, was not "use OAEP." It was "use OAEP and decode it in constant time."

If Bleichenbacher's attack is the atom, the next quarter-century is three quieter echoes of it -- the same leaked bit, reached through channels that get progressively harder to see.

DROWN (2016): a cross-protocol channel. By 2016, TLS servers had long since patched the obvious padding-error message. But many still supported SSLv2, an obsolete protocol from the 1990s, often on a different service such as a mail server -- and often with the same RSA key. DROWN showed that an attacker could use the weak, deliberately export-crippled SSLv2 endpoint as the padding oracle, then use its answers to decrypt a modern TLS session that shared the key.The DROWN team measured that roughly 33 percent of all HTTPS servers were vulnerable at disclosure in March 2016, because key reuse across a TLS service and a forgotten SSLv2 service was rampant [@drownsite]. The math of the oracle was 1998's; only the delivery was new [@drown16].

ROBOT (2018): the same oracle, nineteen years on. Hanno Bock, Juraj Somorovsky, and Craig Young revisited the original attack and found it alive across the modern internet. The padding-error message was gone, but the oracle now leaked through subtler tells: a TCP reset here, a connection timeout there, a duplicated alert message somewhere else -- any behaviour that differed between conforming and non-conforming padding. It affected almost a third of the top 100 domains, including Facebook and PayPal, and traced to products from F5, Citrix, Radware, Palo Alto Networks, IBM, and Cisco [@robot18].To make the point unmissable, the ROBOT authors used the recovered oracle to sign a message with the private key behind facebook.com's certificate -- a decryption oracle repurposed to forge a signature, without ever touching the factorization [@robotsite].

Marvin (2023): pure timing, no error at all. The quietest channel yet removes the error entirely. Hubert Kario's Marvin attack observes only the time the decryption operation takes, since a conforming and a non-conforming padding often run through subtly different code paths. No alert, no reset, no message -- just a stopwatch. Marvin re-found exploitable leaks in implementations that had been declared immune after ROBOT, and it reaches well beyond TLS into S/MIME, JSON Web Tokens, and hardware tokens such as HSMs and smartcards [@marvin23]. The peer-reviewed write-up appeared at ESORICS 2023 under the fitting title Everlasting ROBOT [@eprint2023]. Its recommendation was blunt: stop using PKCS#1 v1.5 encryption.

Note: Every member of this lineage leaks the same single bit -- was the padding valid? -- through a different channel. You cannot patch your way to safety one channel at a time, because the next channel is always quieter than the last. The only durable fixes are structural: remove the mode (as TLS 1.3 did) or forbid it (as FIPS did). If a standard forces RSA encryption, use OAEP with constant-time decoding, never v1.5.

Read the four together and the pattern is impossible to miss. The channel gets quieter each time -- an error string in 1998, a cross-protocol zombie in 2016, a TCP quirk in 2018, a bare microsecond in 2023 -- but the leaked bit never changes. This is why "add more padding" was never the answer and constant-time, uniform-failure decoding was. The catalog below is the evidence for this article's whole argument: every row is some party using the bare trapdoor as if it were a scheme, or letting a check confess.

Attack	Year	Front	Leaked channel	Root cause	The rule it teaches
Textbook RSA [@rsa78]	1978	Encryption	None needed	Deterministic, malleable bare permutation	Never encrypt with the raw primitive
Hastad / Coppersmith [@hastad88, @coppersmith97]	1988 / 96	Params	Algebraic structure	Small e, short or related messages	Pad first, use e equals 65537
Bleichenbacher [@bleichenbacher98]	1998	Encryption	Padding-error message	v1.5 validity check is a decryption oracle	Uniform, unobservable failure
Manger [@manger01]	2001	Implementation	Error-type or timing split	Non-constant-time OAEP decode	OAEP AND constant-time decoding
DROWN [@drown16]	2016	Encryption	SSLv2 cross-protocol	Same RSA key on a weak zombie protocol	Never reuse keys, kill SSLv2
ROBOT [@robot18]	2018	Encryption	TCP reset, timeout, alert	v1.5 oracle still live behind subtle tells	Remove v1.5 key exchange
Marvin [@marvin23]	2023	Implementation	Decryption timing only	Non-constant-time v1.5 depadding	Retire v1.5 encryption entirely
e equals 3 forgery [@cve2006]	2006	Signature	None, forged offline	Lax verifier ignores trailing bytes	Verify strictly, parse it all
BERserk [@cve2014]	2014	Signature	None, forged offline	NSS mis-parses ASN.1 lengths	Verify strictly, re-encode and compare
Bellcore fault [@bdl97]	1997	Implementation	One faulty signature	CRT fault reveals a prime by gcd	Verify signature before release
Kocher, Brumley-Boneh [@kocher96, @brumleyboneh03]	1996 / 2003	Implementation	Exponentiation timing	Secret-dependent modexp time	Base blinding, constant time
Wiener [@wiener90]	1990	Params	Public key structure	Private exponent d too small	Never shrink d for speed
Mining Ps and Qs [@miningpq12]	2012	Params	Shared prime factors	Low boot-time entropy at keygen	Seed the CSPRNG properly
ROCA [@roca17]	2017	Params	Public modulus structure	Infineon's structured primes	Generate primes uniformly

Fourteen rows, one disease. But the encryption oracle is only one of four ways the bare trapdoor betrays its owner. The other three -- the lazy verifier, the glitched chip, and the badly chosen numbers -- are just as instructive, and one of them factors your modulus from a single fault.

5. The Rest of the Catalog: Signatures, Faults, and Bad Primes

If the encryption oracle is the trapdoor leaking through the decryptor's answer, the remaining breaks are the trapdoor leaking through the verifier's laziness, the hardware's faults, and the numbers you fed it. Three fronts, and the encryption-versus-signature split stays absolute across all of them.

Front A: the signature track, and the precision that governs it

Start with a fact that surprises people who lump all of v1.5 together: RFC 8017 records no known attack against the RSASSA-PKCS1-v1_5 signature scheme itself [@rfc8017]. Unlike v1.5 encryption, the signature padding has never been broken as a construction. It rests on a heuristic argument rather than a tight proof, which is why cryptographers reach for PSS in new designs, but it is legacy-not-broken. Where it is fragile is at the verifier -- and that fragility has bitten twice.

The $e = 3$ forgery (2006, CVE-2006-4339). Daniel Bleichenbacher -- the same name, a different result eight years later -- described a forgery that needs no private key at all. A v1.5 signature block ends with the ASN.1 DigestInfo structure, and a lazy verifier checks that the high-order bytes look right without confirming that the DigestInfo sits flush against the end with no trailing data.

With a small public exponent such as $e = 3$, an attacker can construct a number whose cube reproduces the correct prefix and hash in the leading bytes, then pack arbitrary garbage into the trailing bytes the verifier never inspects. Because cubing is cheap and the low bytes are free, the forged "signature" is just a carefully chosen cube root [@cve2006]. OpenSSL's advisory of 5 September 2006 traced it to exactly that omission: "not checking for excess data" after the exponentiation [@openssl2006].

BERserk (2014, CVE-2014-1568). Eight years on, the same antipattern returned in a different guise. Mozilla's NSS library mis-parsed ASN.1 length fields during signature verification, letting an attacker smuggle forged content past the check and produce signatures that NSS -- and therefore Firefox and Chrome builds of the era -- accepted as valid, including for TLS certificates [@cve2014]. Once again, no scheme was broken; a verifier was.

The recurring lesson is one line: verify strictly. Parse the entire structure, reject any trailing data, re-encode the expected block and compare it byte-for-byte, and never take an $e = 3$ shortcut. The provable alternative, PSS, removes the temptation to hand-roll a lenient check by making the encoding randomized and its verification total; its mechanism is in the next section [@pss96]. These verifier bugs live where v1.5 signatures live -- inside the X.509 and PKI machinery that Part 4 of this series covers.

"Still-deployed PKCS#1 v1.5" hides two opposite truths, and conflating them is the single most common error in RSA writing. v1.5 *encryption* (RSAES-PKCS1-v1_5) is fundamentally dangerous padding: its validity check is a decryption oracle, and it should be retired. v1.5 *signatures* (RSASSA-PKCS1-v1_5) are dominant across Web PKI, FIPS-approved in FIPS 186-5, and unbroken as a scheme -- keep them where mandated and verify strictly [@rfc8017], [@fips1865]. One phrase must never carry both meanings. When you read "RSA v1.5 is broken," ask which one, because the honest answer is "the encryption, not the signatures."

Front B: the physical layer

Here the secret is the factorization, so anything the hardware leaks about the private operation leaks the factors directly.

The CRT fault (Boneh, DeMillo, and Lipton, 1997). Recall that fast RSA signing computes the result modulo $p$ and modulo $q$ separately, then recombines. Suppose a single bit flips during the modulo-$p$ half -- from a voltage glitch, a clock fault, cosmic radiation, or a deliberate injection. The faulty signature $s'$ is now correct modulo $q$ but wrong modulo $p$. Then $s'^e - m$ is divisible by $q$ but not by $p$, so a single $\gcd(s'^e - m, N)$ hands you $q$ -- and the modulus is factored from one faulty signature [@bdl97]. The demonstration below does exactly that with toy primes.

{` // Bellcore fault: a fault in one CRT half leaks a prime of N. Toy primes for clarity. function modpow(base, exp, mod) { base = ((base % mod) + mod) % mod; let r = 1n; while (exp > 0n) { if (exp & 1n) r = (r * base) % mod; base = (base * base) % mod; exp >>= 1n; } return r; } function gcd(a, b) { while (b) { const t = a % b; a = b; b = t; } return a; } const p = 61n, q = 53n, N = p * q, e = 17n, d = 2753n; const m = 123n % N; // message representative to sign const dp = d % (p - 1n), dq = d % (q - 1n); const qInv = modpow(q, p - 2n, p);

// Correct CRT signature (Garner recombination): const sp = modpow(m, dp, p), sq = modpow(m, dq, q); const s = (sq + q * (((qInv * (sp - sq)) % p + p) % p)) % N; console.log('correct signature verifies?', modpow(s, e, N) === m);

// A fault corrupts ONLY the mod-p half: const spBad = (sp + 1n) % p; const sBad = (sq + q * (((qInv * (spBad - sq)) % p + p) % p)) % N; console.log('faulty signature verifies?', modpow(sBad, e, N) === m);

// One gcd factors the modulus: const diff = ((modpow(sBad, e, N) - m) % N + N) % N; console.log('gcd(sBad^e - m, N) =', gcd(diff, N).toString(), '-> a secret prime of N (61 or 53)'); `}

The fix is a single line of discipline: verify before release. Recompute $s^e \bmod N$ and confirm it equals $m$ before the signature ever leaves the chip. A faulty signature never escapes, and the gcd trick has nothing to work with.

Timing (Kocher, 1996; Brumley and Boneh, 2003). Paul Kocher observed in 1996 that the time to compute $c^d \bmod N$ depends on the secret bits of $d$, because square-and-multiply does extra work on 1 bits [@kocher96]. Folklore held that timing attacks needed local access. David Brumley and Dan Boneh demolished that in 2003 by recovering an OpenSSL server's private key over a network, measuring only response times [@brumleyboneh03].

The fix is base blinding: multiply the input by $r^e$ for a fresh random $r$ before exponentiating, then divide the result by $r$ afterward, so the timing depends on a value the attacker cannot predict. OpenSSL turned blinding on by default in 2003. Marvin, from the previous section, is this front's 2023 revival -- the timing channel never truly closed.

Front C: the parameters

The trapdoor is only as strong as the numbers behind it.

Wiener (1990). If you shrink the private exponent for speed so that $d < \tfrac{1}{3} N^{1/4}$, a continued-fraction expansion of $e/N$ recovers $d$ outright [@wiener90]. You cannot buy decryption speed by making the secret small.
Coppersmith (1996 to 1997). His lattice method sets the barrier $|x| < N^{1/e}$ that both enables low-exponent attacks on short or stereotyped messages and bounds them -- the same result that returns, weaponized, as ROCA [@coppersmith97].
Mining Your Ps and Qs (2012). Heninger and colleagues scanned the internet's public keys and computed pairwise gcds. Devices that generated keys at first boot, before they had gathered entropy, sometimes shared a prime -- and a shared prime means a free gcd factors both moduli. They computed private keys for about 0.50 percent of TLS hosts and 0.03 percent of SSH hosts this way [@miningpq12].
ROCA (2017, CVE-2017-15361). Infineon's key-generation library built primes with a special structure to speed things up. That structure let a Coppersmith attack factor the public modulus for a practical cost, and the affected chips were everywhere: TPMs, YubiKey 4 tokens, national electronic ID cards, and BitLocker deployments [@roca17], [@cve2017]. Coppersmith's 1996 barrier, patient for twenty years, collected its debt.

The root cause of the last two is not RSA math at all -- it is the CSPRNG that generates the primes, the subject of Part 2 of this series. Feed RSA weak randomness and no scheme, no padding, and no proof can save it.

Note: Turn the diagnostic question on each front and it answers itself. The verifier confesses through a check it skipped. The chip confesses through a fault it did not catch. The clock confesses through a branch it did not equalize. The keygen confesses through primes it did not draw uniformly. In every case the attacker reads the private key off the receiver's behaviour, not off the mathematics.

Keep the empirical yardstick in view: the largest RSA modulus ever publicly factored is RSA-250, at 829 bits.RSA-250 was factored on 28 February 2020 using the Number Field Sieve, at a cost of roughly 2,700 core-years of computation [@rsa250]. That is the real, measured cost of breaking RSA by its front door -- which is exactly why nobody attacks that door when a padding check will open the side gate for a million cheap queries. Four fronts, one disease. The cure is not a cleverer patch on any single front. It is a change in what we mean when we say a scheme is secure.

6. Security Is a Property of the Scheme AND the Implementation

There are two famous ideas behind "RSA done right," and a third, deeper one that the first two kept teaching the hard way.

Idea 1: OAEP, the all-or-nothing randomized transform

Optimal Asymmetric Encryption Padding, designed by Mihir Bellare and Phillip Rogaway in 1994, encodes the message before the RSA permutation so that the encoded block has no gradable structure for an attacker to probe [@oaep94]. Its construction is a two-round Feistel network with a hash-based mask-generation function, MGF1, as the round function -- the hashing machinery that Part 10 of this series develops. Writing $\Vert$ for concatenation and $\oplus$ for XOR, and starting from a random $\mathrm{seed}$:

$$\mathrm{DB} = \mathrm{lHash} \Vert \mathrm{PS} \Vert \texttt{0x01} \Vert M$$ $$\mathrm{maskedDB} = \mathrm{DB} \oplus \mathrm{MGF1}(\mathrm{seed}), \qquad \mathrm{maskedSeed} = \mathrm{seed} \oplus \mathrm{MGF1}(\mathrm{maskedDB})$$ $$\mathrm{EM} = \texttt{0x00} \Vert \mathrm{maskedSeed} \Vert \mathrm{maskedDB}$$

Then the RSA permutation is applied to $\mathrm{EM}$ [@rfc8017]. The two Feistel rounds make every bit of the encoded block depend on every bit of the message and the random seed. That is the property that kills the padding oracle: flip a single bit of an OAEP ciphertext and the decoded block is randomized wholesale, so a tampered ciphertext decodes to noise with overwhelming probability. There is no conformant interval to bisect, no structured remnant to grade.

With the randomness supplying non-determinism and the all-or-nothing mixing supplying tamper-detection, OAEP is not only semantically secure but also non-malleable and secure against chosen-ciphertext attack -- IND-CCA2 in the random-oracle model [@oaep94abs].

Optimal Asymmetric Encryption Padding: a randomized, all-or-nothing transform applied to a message before the RSA permutation. It mixes the message with a random seed through two mask-generation passes -- a two-round Feistel network -- so that every bit of the encoded block depends on every bit of the input. Flipping any ciphertext bit randomizes the whole decoded block, which destroys malleability and yields chosen-ciphertext security in the random-oracle model. OAEP's history includes a moment cryptography should be proud of. In 2001, Victor Shoup showed that the celebrated 1994 security proof did not go through for an arbitrary trapdoor permutation -- it quietly assumed more than the definition guarantees [@shoup01]. The scheme was not broken; the *argument* was. Later that year, Fujisaki, Okamoto, Pointcheval, and Stern repaired it, proving RSA-OAEP is chosen-ciphertext secure in the random-oracle model under the RSA assumption, by leaning on a property called partial-domain one-wayness -- though the resulting reduction is non-tight [@fops04]. A public "our proof, not our scheme, was wrong," followed by a public fix, is exactly how a mature field is supposed to work.

Idea 2: PSS, the salted encoding with a tight proof

For signatures, the Probabilistic Signature Scheme, also from Bellare and Rogaway, plays the analogous role [@pss96]. To sign, it hashes the message to $\mathrm{mHash}$, draws a random $\mathrm{salt}$, and forms $M' = \texttt{padding} \Vert \mathrm{mHash} \Vert \mathrm{salt}$; sets $H = \mathrm{Hash}(M')$; builds $\mathrm{DB} = \mathrm{PS} \Vert \texttt{0x01} \Vert \mathrm{salt}$; masks it as $\mathrm{maskedDB} = \mathrm{DB} \oplus \mathrm{MGF1}(H)$; and assembles $\mathrm{EM} = \mathrm{maskedDB} \Vert H \Vert \texttt{0xbc}$ before applying the RSA private operation [@rfc8017].

The randomization and MGF mixing buy something v1.5 signatures never had: a tight exact-security reduction. A forger against PSS implies an algorithm that inverts RSA at almost the same cost -- so breaking the signatures is essentially as hard as the RSA problem itself, not merely conjectured to be [@pss96]. That is strictly stronger than v1.5's heuristic argument. PSS entered the standard in PKCS#1 v2.1 (RFC 3447) in 2003 and remains in the current v2.2 [@rfc3447]. Yet the split holds: PSS is the provable upgrade, but v1.5 signatures are not superseded in deployment -- they remain the Web PKI default.Beyond these two deployed schemes, the literature holds academic-only refinements -- SAEP/SAEP+, OAEP+, three-round OAEP, tight RSA-KEM variants, and message-recovery PSS-R -- which tighten proofs or trim assumptions but never displaced OAEP and PSS in practice. A pointer, not a section.

Probabilistic Signature Scheme: a randomized, salted RSA signature encoding. A fresh random salt and MGF1 mixing make every signature of the same message different, and the scheme comes with a tight security reduction -- a forger would yield an almost-equally-efficient algorithm for inverting RSA, the strongest guarantee any RSA scheme offers.

Idea 3: the one the first two kept teaching

Here is where the whole article turns. OAEP is provably secure, yet Manger re-opened the identical oracle inside a non-constant-time decoder. PSS is provably secure, yet a fault or a timing leak in the signer factors the key. The scheme was never the whole story.

Key idea: Security is a property of the scheme AND its implementation, not of the math. A perfect scheme run by a decoder that leaks whether padding verified is exactly as broken as no scheme at all. The implementation disciplines below are part of the construction, not optional hygiene layered on top.

Three disciplines close the three channels the catalog exposed:

Constant-time depadding with implicit rejection. On any decryption failure, do not branch or return a distinguishable error; substitute a deterministic pseudo-random value and continue, so whether the padding verified stays unobservable. OpenSSL 3.2 ships this by default and aligned its behaviour with NSS so neither library can be an oracle for the other [@openssl32], [@gorsa].
CRT verify-before-release. Recompute and check the signature before it leaves the device, defeating Bellcore faults [@bdl97].
Base blinding. Randomize the input before exponentiation so timing reveals nothing about the secret [@kocher96], [@brumleyboneh03].

A decryption strategy in which a padding failure does not produce a distinguishable outcome. Instead of branching or returning a specific error, the code substitutes a deterministic pseudo-random value derived from the private key and ciphertext and proceeds. Success and failure become indistinguishable through content, error type, or timing, so there is no oracle left to query. flowchart TD M[Message or ciphertext arrives] --> SCH{"Scheme layer"} SCH -->|Encrypt| OAEP[RSAES-OAEP, randomized all-or-nothing] SCH -->|Sign| PSS[RSASSA-PSS, salted with tight proof] OAEP --> IMPL{"Implementation layer"} PSS --> IMPL IMPL --> CT[Constant-time depad, implicit rejection] IMPL --> VBR[CRT verify-before-release] IMPL --> BL[Base blinding] CT --> PAR{"Parameter layer"} VBR --> PAR BL --> PAR PAR --> P1[Modulus at least 2048 bits, e equals 65537, CSPRNG primes] P1 --> GATE{"Does any behaviour reveal the padding verdict?"} GATE -->|No| SAFE[RSA done right] GATE -->|Yes| BROKEN[Oracle reopens, the catalog repeats]

The libraries now say this in plain language. Go's standard library deprecated its v1.5 decryption function outright, with a warning that doubles as the thesis of this article:

"PKCS #1 v1.5 encryption is dangerous and should not be used ... whether this function returns an error or not discloses secret information." -- Go crypto/rsa package documentation [@gorsa]

That is the whole discipline in one sentence: done right closes the padding-oracle class Part 6 traces across all of cryptography, and it does so at the implementation layer, not the math. So what does the whole stack look like in 2024 to 2026 -- and which parts are quietly being switched off?

7. State of the Art (2024 to 2026): Done Right Is a Stack, Not a Method

Most primitives have a single "best" option you can name. RSA does not. "Done right" in 2026 is a conjunction of independent choices that must all hold at once. Remove any one and the 25-year catalog reopens at that layer. Here is the stack, layer by layer.

Mode	Job	What it adds	Security status	Still requires
v1.5 encryption	Encrypt	Randomness only	Broken as a target -- padding oracle	Retire it; no safe deployment
RSAES-OAEP	Encrypt	Randomness plus all-or-nothing mixing	IND-CCA2 in the random-oracle model	Constant-time, implicit-rejection decode
v1.5 signature	Sign	Fixed checkable structure	Unbroken scheme, heuristic argument	Strict verification, no e equals 3 shortcut
RSASSA-PSS	Sign	Salt plus MGF, tight proof	Provably as hard as the RSA problem	CRT verify-before-release, blinding

Encryption scheme. When a standard genuinely mandates RSA encryption of a small payload, the answer is RSAES-OAEP with SHA-256 and MGF1-SHA-256, decoded in constant time [@rfc8017]. But note the capacity ceiling: a 2,048-bit OAEP-SHA-256 ciphertext carries at most about 190 bytes of plaintext [@sp80056b]. "Encrypt a file with RSA-OAEP" is therefore not just risky but physically wrong -- RSA encryption is for keys, never bulk data.

Signatures. New designs use RSASSA-PSS, which TLS 1.3 makes mandatory for handshake signatures [@pss96], [@rfc8446]. Yet RSASSA-PKCS1-v1_5 signatures remain dominant and FIPS-approved across X.509 and TLS certificates [@fips1865]. This is not a contradiction: keep v1.5 signatures where they are mandated, and verify them strictly [@rfc8017].

Implementation hardening. This is the layer that actually stops the attacks. OpenSSL 3.2 enables constant-time depadding with implicit rejection by default [@openssl32], Go has deprecated its v1.5 decryption entry point [@gorsa], and the CFRG's 2025 implementation-guidance draft folds all of this into formal advice amending RFC 8017 [@cfrgdraft].

Parameters. A 2,048-bit modulus is the floor for 112-bit security; use 3,072 or 4,096 bits for long-term secrets; keep $e = 65537$; and draw primes from a properly seeded CSPRNG [@sp80057], [@fips1865].

Deployment and policy. The most durable fixes were structural, not cryptographic. TLS 1.3 deleted RSA key exchange entirely, which killed the Bleichenbacher-on-TLS class at the protocol level rather than patching each oracle [@rfc8446]. FIPS policy disallows RSA v1.5 key transport after 31 December 2023 [@sp800131a], [@sp80056b]. And NIST IR 8547 puts a clock on RSA itself, deprecating 112-bit strength after 2030 and disallowing it after 2035 [@ir8547].

If you operate under FIPS 140-3, the encryption side of this is not advisory. NIST SP 800-131A Rev. 2 and SP 800-56B Rev. 2 together disallow RSAES-PKCS1-v1_5 key transport after 31 December 2023; approved RSA key establishment must use OAEP [@sp800131a], [@sp80056b]. The signature side is the opposite: RSASSA-PKCS1-v1_5 signatures remain FIPS-approved under FIPS 186-5 [@fips1865]. Same version number, opposite compliance verdicts -- which is exactly why the encryption-signature split is not pedantry.

Note: Right scheme (OAEP to encrypt, PSS for new signatures), right implementation (constant-time depad with implicit rejection, CRT verify-before-release, base blinding), right parameters (2,048-bit floor, e equals 65537, CSPRNG primes), inside the right protocol (TLS 1.3, no RSA key exchange). Every deployed disaster in the catalog removed exactly one of these. None of it is new in 2026; the interesting movement is not toward a better RSA but away from RSA entirely.

RSA done right is stable. The genuinely current story is mostly about what to switch off and what is migrating away -- because for both of RSA's jobs, something smaller and faster is already taking over.

8. What RSA Is Losing To: ECDH, KEM-DEM, and the Post-Quantum Elephant

RSA is not being fixed into the future; it is being replaced out of it -- and the reasons have nothing to do with padding.

The cleanest of those reasons is a design pattern that removes the attacker's target entirely. Never RSA-encrypt data. Instead, encapsulate a fresh random symmetric key with the public key, and let an authenticated cipher carry the bulk. The public-key operation then transports only a uniform random key -- there is no attacker-chosen, structured plaintext for any oracle to grade. The entire Bleichenbacher-to-Marvin family loses its object, because the thing being decrypted is indistinguishable from random by construction [@rfc8446].

A two-part construction for hybrid encryption. A Key Encapsulation Mechanism (KEM) uses the public key to transport a freshly generated random symmetric key; a Data Encapsulation Mechanism (DEM) then encrypts the actual data with that key under an authenticated cipher. Because the public-key step only ever carries a uniform random key -- never a chosen, structured message -- there is no padding structure for an oracle to probe, so the padding-oracle class simply does not apply. sequenceDiagram participant S as Sender participant R as Recipient Note over R: Publishes a public key S->>S: Encapsulate, generate a random key K for the recipient S->>S: Stretch K with a KDF, AEAD-encrypt the data S->>R: Send the encapsulation and the AEAD ciphertext R->>R: Decapsulate to recover K, run the same KDF R->>R: AEAD-decrypt and verify the data Note over R,S: The public-key part carried only a random K, nothing to grade

This is why elliptic-curve key agreement (ECDH, with Ed25519 and ECDSA for signatures) displaced RSA key transport: it does the same jobs with far smaller keys, faster operations, and forward secrecy -- a property RSA key transport never had, and the direct reason TLS 1.3 could delete RSA key exchange rather than merely patch it [@rfc8446]. The KEM-DEM composition is developed in Part 11 of this series, the KDF step in Part 13, and the AEAD that carries the payload in Part 7.

Then there is the elephant. Peter Shor's algorithm factors integers in polynomial time on a large fault-tolerant quantum computer, which makes all factoring-based cryptography -- every RSA key length, done right or not -- eventually breakable. NIST has already named the destination: ML-KEM (FIPS 203) for key encapsulation, and ML-DSA and SLH-DSA (FIPS 204 and 205) for signatures [@fips203], [@fips204], [@fips205].

The migration is lopsided. Key encapsulation is moving fast -- hybrid X25519 plus ML-KEM-768 already carries a double-digit percentage of Cloudflare's TLS 1.3 traffic -- while post-quantum signatures lag badly, because they run 10 to 200 times larger than an RSA signature and no public post-quantum certificate infrastructure existed before roughly 2026 [@cloudflare24], [@ir8547]. The lattice and hash-based internals belong to a later part of this series; here they matter only as RSA's replacement, not its repair.

Quantum risk is not only a future problem. An adversary can record RSA-encrypted traffic today and decrypt it once a capable quantum computer exists. For any secret that must stay confidential past roughly 2030, the exposure is *present tense* -- which is why NIST IR 8547 frames the transition as urgent rather than eventual, and why hybrid key exchange is being deployed now rather than when the machine arrives [@ir8547], [@cloudflare24].

Laid side by side, the trade-offs explain why key transport migrated first and signatures are dragging. For moving a key:

Key transport	Forward secrecy	Padding-oracle exposure	Relative cost	Quantum status
RSAES-OAEP [@rfc8017]	No	Yes, if decode is not constant-time	Large ciphertext, slow keygen	Broken by Shor
ECDH (X25519) [@rfc8446]	Yes	None, no RSA decryption	Small and fast	Broken by Shor
ML-KEM-768 [@fips203]	Yes (ephemeral)	None	Kilobyte-scale, fast	Resistant
Hybrid X25519 + ML-KEM-768 [@cloudflare24]	Yes	None	Sum of both, still practical	Resistant if either holds

And for signing:

Signature	Provable security	Approx. signature size	Main failure mode	Quantum status
RSASSA-PSS [@pss96]	Yes, tight in ROM	256 bytes at 2048-bit	Fault or timing in the signer	Broken by Shor
RSASSA-PKCS1-v1_5 [@rfc8017]	No, heuristic only	256 bytes at 2048-bit	Lax-verifier forgery	Broken by Shor
ECDSA / Ed25519 [@rfc8446]	Yes	64 bytes	Nonce reuse (ECDSA)	Broken by Shor
ML-DSA-65 [@fips204]	Yes, lattice	About 3.3 kilobytes	Implementation bugs	Resistant
SLH-DSA [@fips205]	Yes, hash-based	Many kilobytes	Large and slow	Resistant

The signature column is the migration's hard problem, which is why post-quantum certificates lag key exchange by years [@ir8547]. If RSA is on the clock, it is worth asking where its security actually came from in the first place -- and why nobody has ever proved it was there.

9. Theoretical Limits: Where the Security Comes From, and Its Ceiling

Here is the uncomfortable truth a first course skips: there is no proof that RSA is secure.

Start with the assumption itself. The RSA problem is to compute $e$-th roots modulo $N$ without the factorization. We know this is no harder than factoring -- if you can factor $N$, you can compute $d$ and invert everything, so RSA $\le$ factoring. What nobody has shown is the reverse. It is an open question whether breaking RSA is equivalent to factoring, and Boneh and Venkatesan gave evidence that a broad class of algebraic reductions from factoring to low-exponent RSA is unlikely to exist -- suggesting the two problems may not be equivalent at all [@bv98].

Worse, factoring itself is not known to be hard in any proven sense: it sits in NP intersect co-NP, which is evidence it is probably not NP-complete, and no one has proved a super-polynomial lower bound for it. RSA's security is heuristic and empirical -- it has survived decades of attack, and that is the entire argument [@boneh99].

The empirical ceiling is concrete. The best classical algorithm, the General Number Field Sieve, runs in sub-exponential time, and the public record is RSA-250 at 829 bits, factored in February 2020 for roughly 2,700 core-years [@rsa250]. RSA-250 was one of the RSA Factoring Challenge moduli that RSA Laboratories first published in 1991 as public benchmarks for exactly this kind of progress [@rsanumbers]. That record sets the practical floors:

RSA modulus	Symmetric-equivalent strength	Status
1024-bit	About 80 bits	Broken within reach, disallowed
2048-bit	112 bits	Current minimum, deprecated after 2030
3072-bit	128 bits	Long-term use
4096-bit	About 140 bits (interpolated)	High assurance
7680-bit	192 bits	Next tabulated SP 800-57 step
15360-bit	256 bits	Shows how badly RSA scales

Those equivalences come from NIST SP 800-57, except the 4,096-bit figure -- a common interpolation, since the standard steps directly from 3,072 bits (128) to 7,680 bits (192). The last row is the quiet punchline: matching a 256-bit symmetric key needs a 15,360-bit RSA modulus, because RSA strength grows only sub-exponentially in key length while the cost of using it grows with the cube [@sp80057]. RSA scales badly, and that alone pushes new systems toward elliptic curves.

Then the cliff. On a large fault-tolerant quantum computer, Shor's algorithm factors in polynomial time -- not faster, but categorically faster, collapsing the whole assumption. The only question is engineering, and the estimates are falling fast. In 2019, Gidney and Ekera estimated 20 million noisy qubits and 8 hours to factor a 2,048-bit modulus [@gidney19]; by 2025, Gidney had cut the qubit estimate to under a million [@gidney25].A 20-fold reduction in the resource estimate in six years, with no fundamental barrier in sight, is precisely the trend that drove NIST to put firm dates -- 2030 and 2035 -- on RSA's retirement in IR 8547. The deadline is set by the slope, not by any single machine.

Three impossibility results frame the whole subject, each already seen in this article. First, determinism can never be IND-CPA -- that is structural, not fixable, which is why padding must randomize. Second, there is no known standard-model proof of IND-CCA2 for RSA-OAEP under the plain RSA assumption; the guarantee is random-oracle-model only, and even there non-tight [@shoup01], [@fops04]. Third, Coppersmith's barrier $|x| < N^{1/e}$ both enables low-exponent attacks and bounds them, a hard mathematical edge that no parameter choice moves [@coppersmith97].

Key idea: RSA's safety lives in an unproven gap -- breaking it is no harder than factoring, but maybe strictly easier, and factoring is not even proven hard -- and that gap sits on the near side of a quantum cliff with a falling date on it. "Done right" buys you security against every known classical attack. It does not buy you a proof, and it cannot buy you time past Shor.

If the ground under RSA is this uncertain, what exactly is still unsettled -- and which of those open problems can bite you today?

10. Open Problems: What Remains Genuinely Unsettled

Some of these are for theorists. Others are live in your dependency tree right now.

Is the RSA problem equivalent to factoring? We have only one direction, RSA $\le$ factoring; the reverse is open, with evidence it may fail [@bv98]. It matters because RSA's whole security story rests on a hardness we have never actually pinned down.
A deployed, standard-model IND-CCA2 RSA scheme. Constructions that avoid the random-oracle model exist on paper, but none is a shipping default; practice sidesteps the gap entirely by using KEM-DEM instead of RSA encryption [@fops04].
Constant-time RSA is a perpetually moving target. Marvin re-found timing leaks in implementations previously believed immune, and only a handful -- such as BearSSL and BoringSSL -- passed its tests; leaks hide in general-purpose bignum code and even in error-logging paths, which is why the CFRG draft concludes the only safe path is to deprecate v1.5 encryption outright rather than keep hardening it [@marvin23], [@cfrgdraft].
The long tail of v1.5 encryption. HSMs, PKCS#11 tokens, S/MIME, and JWE still use it in places that cannot simply be switched off -- exactly where Marvin keeps finding live oracles [@marvin23], [@gorsa].
Generation-time entropy and structured primes. Mining Your Ps and Qs and ROCA are operational failures with no clean universal fix: you cannot prove every device in the world seeded its CSPRNG correctly [@miningpq12], [@roca17].
The post-quantum migration timeline. Key exchange is migrating; signatures are years behind, and harvest-now-decrypt-later keeps the pressure on [@cloudflare24], [@ir8547].

Note: Most of this list is someone else's research agenda. Two entries are not: the v1.5 encryption long tail and weak key-generation entropy are the ones most likely to be sitting in your own stack right now, in an HSM integration or an embedded device you inherited. Audit those two first.

Enough theory. Here is the whole argument compressed into rules you can apply on Monday.

11. The Practical Guide and the Misuse Catalog

Every rule below is one named break from the catalog, turned inside out. If you remember nothing else, remember the decision tree.

flowchart TD START{"What do you need?"} -->|Confidentiality| ENC{"Can you avoid RSA encryption?"} START -->|Authenticity| SIG{"New design or legacy mandate?"} ENC -->|Yes| KEM[Use ECDH or hybrid X25519 plus ML-KEM-768] ENC -->|No, a standard forces RSA| OAEP[RSAES-OAEP, SHA-256, constant-time decode, at least 2048-bit] SIG -->|New design| PSS[RSASSA-PSS with SHA-256] SIG -->|Legacy mandate| V15[v1.5 signature, verify strictly, no e equals 3 shortcut] KEM --> KEYS[Keys: e equals 65537, CSPRNG primes, verify-before-release, blinding] OAEP --> KEYS PSS --> KEYS V15 --> KEYS KEYS --> PQ{"Secret must survive past 2030?"} PQ -->|Yes| HY[Deploy hybrid post-quantum now] PQ -->|No| DONE[Ship it]

Spelled out as decision rules:

Encryption and key transport. Prefer ECDH or a KEM, migrating to hybrid X25519 plus ML-KEM-768. If a standard forces RSA encryption, use RSAES-OAEP with SHA-256 and MGF1-SHA-256, a modulus of at least 2,048 bits, and a constant-time decoder with implicit rejection -- never v1.5 encryption [@rfc8017], [@sp80056b]. Never RSA-encrypt a payload; encapsulate a symmetric key and let an AEAD carry the data.
Signatures. Use RSASSA-PSS for new designs [@pss96], [@fips1865]. Use v1.5 signatures only where mandated, and then verify strictly: full parse, re-encode and compare, reject trailing data, no $e = 3$ shortcut [@rfc8017].
Keys and parameters. A 2,048-bit floor, 3,072 or 4,096 bits for long-term secrets; $e = 65537$; independent CSPRNG-drawn primes per key; CRT with verify-before-release; base blinding [@sp80057], [@fips1865].
Migration. Deploy hybrid post-quantum key agreement now, and inventory every RSA usage against the IR 8547 2030 and 2035 clock [@ir8547].

The same rules as a lookup table:

Task	Use	Key parameters	Because (which break)
Key transport, preferred	ECDH or hybrid X25519 + ML-KEM-768	Ephemeral keys	Forward secrecy, no oracle, quantum hedge
Key transport, if RSA forced	RSAES-OAEP, constant-time decode	SHA-256, MGF1-SHA-256, at least 2048-bit	Bleichenbacher, Manger, Marvin
Signature, new design	RSASSA-PSS	SHA-256, random salt	v1.5 hand-wave, e equals 3 forgery
Signature, legacy mandate	v1.5, verified strictly	Full parse, reject trailing bytes	e equals 3 forgery, BERserk
Key generation	e equals 65537, CSPRNG primes	2048 floor, 3072+ long-term	Wiener, ROCA, Mining Ps and Qs
Private operation	CRT verify-before-release, base blinding	Recompute before output	Bellcore fault, Kocher timing
Long-term secrets	Hybrid post-quantum now	Inventory to IR 8547 clock	Shor, harvest-now-decrypt-later

Now the misuse catalog. Each antipattern maps to exactly one rule it violates -- the findings that recur in real code review:

v1.5 encryption "kept for compatibility." The single most dangerous line in an RSA integration. Violates: retire v1.5 encryption.
Non-constant-time OAEP or v1.5 decode. A decoder that branches or times differently on padding failure. Violates: constant-time, implicit-rejection decoding (Manger, Marvin).
Distinct decryption error messages, or branch timing. Any observable difference between "bad padding" and "other error." Violates: uniform, unobservable failure (Bleichenbacher).
Missing CRT verify-before-release. Signing without recomputing the result first. Violates: verify before release (Bellcore fault).
Unblinded exponentiation. A modexp whose time depends on the secret. Violates: base blinding (Kocher, Brumley-Boneh).
An $e = 3$ shortcut in a verifier. Checking the prefix without rejecting trailing data. Violates: verify strictly (e-equals-3 forgery, BERserk).
Textbook RSA copied from a tutorial. The raw permutation on a message. Violates: never call the raw primitive.
RSA-encrypting a large payload. Treating RSA as a bulk cipher. Violates: encapsulate a key, never encrypt data.
Weak, shared, or reused-modulus keys. Primes drawn from a cold CSPRNG or a structured library. Violates: seed the CSPRNG, draw primes uniformly (Mining Ps and Qs, ROCA).
OAEP hash or label mismatch, or MGF1 silently defaulting to SHA-1. A quiet interoperability and downgrade trap. Violates: pin the hash and MGF explicitly.
Treating a "small" side channel as unexploitable. Marvin demolished that folklore -- a few microseconds recovered keys [@marvin23]. Violates: assume every observable leaks.

Grep for the dangerous entry points and confirm every RSA decryption path uses OAEP with a constant-time decoder:

Go: search for DecryptPKCS1v15 and EncryptPKCS1v15; move to DecryptOAEP and EncryptOAEP. The v1.5 decryptor is deprecated for the reason quoted earlier [@gorsa].
Java: flag Cipher.getInstance("RSA/ECB/PKCS1Padding"); require RSA/ECB/OAEPWithSHA-256AndMGF1Padding.
OpenSSL CLI: openssl pkeyutl -encrypt -pubin -inkey pub.pem -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256.

If a dependency genuinely still needs v1.5 decryption, confirm it runs a constant-time, implicit-rejection decoder -- OpenSSL 3.2 does so by default [@openssl32].

Note: In practice, three violations dominate real audits: RSA v1.5 encryption still enabled "for a legacy client," a decryption path that is not verifiably constant-time, and RSA being used to encrypt bulk data instead of a symmetric key. Fix those three and you have closed most of the catalog [@marvin23], [@gorsa], [@openssl32].

Read the whole failure catalog again as this checklist, and one pattern remains.

12. Trapdoor, Not Cryptosystem

Return to where we began. The researchers who signed with facebook.com's key never factored its modulus, and neither did anyone in the twenty-five years of breaks between Bleichenbacher and Marvin. The factoring problem stood untouched the whole time. What fell, every single time, was something else: the decryptor's error handling, the signer's fault behaviour, the implementation's clock, or the generator's entropy. The attacker's real move is never to solve the hard math -- it is to turn the receiver's own reaction into the private-key operation.

The attacker does not factor the modulus. They turn the decryptor's reaction into the private-key operation -- and "done right" is the discipline of leaving that reaction with nothing to say.

That is why "RSA done right" is a stack and not a setting. The right scheme, OAEP to encrypt and PSS for new signatures, does two of the three jobs. The right parameters -- a 2,048-bit floor, $e = 65537$, primes from a real CSPRNG -- keep the trapdoor strong. And the constant-time, fault-checked, blinded implementation does the third job, the one the field kept forgetting: it leaves no observable behaviour that answers the attacker's one question.

Every deployed disaster in this article is a stack with exactly one layer missing. Textbook RSA is broken not because the math is weak but because it is only the trapdoor, with none of the layers that make a trapdoor into a cryptosystem.

The forward horizon makes the discipline sharper, not softer. RSA done right is stable against every known classical attack, and it still has an expiration date, because the one gap RSA can never close is the quantum one. The destination is not a better RSA. It is KEM-DEM composition and post-quantum algorithms -- constructions where the public-key operation carries only a uniform random key, and where Shor has no polynomial-time shortcut to wait for.

So ask the diagnostic question one final time, now that it is answerable. When a ciphertext or signature it did not create arrives, what does the receiver reveal about it -- through its answer, its timing, or its faults? Done right, the answer is nothing at all.

No, but the word "RSA" hides three different answers. Textbook RSA is broken. PKCS#1 v1.5 *encryption* is dangerous and should be retired [@marvin23]. Done-right RSA -- OAEP or PSS, constant-time, at least 2,048 bits, public exponent 65537 -- is fine against every known classical attack, right up until a large quantum computer exists, which is why you should be planning migration in parallel [@ir8547]. No. Encapsulate a fresh random symmetric key with the public key and let an authenticated cipher encrypt the file. RSA-OAEP at 2,048 bits carries only about 190 bytes of plaintext anyway, so it was never meant for bulk data -- it moves keys, not files [@sp80056b]. No. OAEP fixes the *scheme*, not the decoder. James Manger showed that a non-constant-time OAEP decoder re-opens the very same padding oracle, and more cheaply than the original Bleichenbacher attack [@manger01]. The rule is "OAEP AND constant-time decoding," never OAEP alone. Not as a scheme -- there is no known attack against the RSASSA-PKCS1-v1_5 signature construction itself [@rfc8017]. What breaks is lax *verifiers*: the 2006 e-equals-3 forgery and the 2014 BERserk bug both forged signatures past sloppy verification, not by breaking the scheme [@cve2006], [@cve2014]. Verify strictly -- parse the entire structure and reject any trailing data. Only with correct padding and a strict verifier, and it is not worth the risk. A small exponent has repeatedly enabled Hastad's broadcast attack, Coppersmith's low-exponent attacks, and signature forgeries [@hastad88], [@coppersmith97]. Use 65537: it is fast and dodges every low-exponent trap. Yes for today -- a 2,048-bit modulus gives about 112 bits of security [@sp80057]. Use 3,072 bits or more for anything that must stay secret long-term, and start post-quantum planning: NIST deprecates 112-bit RSA after 2030 and disallows it after 2035 [@ir8547]. For most new work, yes. ECDH and Ed25519 give smaller, faster keys with the forward secrecy RSA key transport never had [@rfc8446], and hybrid X25519 plus ML-KEM-768 is already carrying real TLS traffic for the quantum transition [@cloudflare24]. RSA's destination is retirement, not repair.

The One Job of a Password Hash: Why Fast Is the Enemy, and Memory Is the Frontier

noreply@paragmali.com (Parag Mali) — Sat, 11 Jul 2026 02:05:14 GMT

**A password hash has exactly one job:** make each attacker guess as expensive as possible on the attacker's *cheapest* hardware, while staying cheap enough on your own server. That is why a fast, strong hash like SHA-256 is the *wrong* tool. A single gaming GPU tries tens of billions of guesses per second, so a stolen fast-hash dump is a race you have already lost -- as LinkedIn learned in 2012. The fix was never a *stronger* hash but a deliberately, tunably *slower* one, and since 2009 a deliberately *memory-hungry* one, because memory is the resource an attacker cannot buy asymmetrically cheaply. This field guide traces the whole deployed lineage -- DES `crypt` (1979), md5crypt, the twin branch of bcrypt (1999) and PBKDF2 (2000), and the memory-hard generation of scrypt (2009), Argon2 (the 2015 competition winner), and yescrypt (now the Linux `/etc/shadow` default) -- alongside the failure catalog that proves the point (LinkedIn, Adobe, Ashley Madison, Facebook). It ends with exact 2024-2026 parameters, a decision tree, and the one rule beneath them all: your security is the weakest function that ever touches the password. Memory-hardness is the recommended default, not a universal mandate -- the right choice is an economic decision bounded by your own budget.

1. Your Database Leaked, and the Race Is Already Over

In June 2012, roughly six million LinkedIn password hashes appeared on a Russian forum, and within days most were plaintext. Not because anyone had broken SHA-1, and not because there was a key to steal, but because the passwords had been stored with a hash whose whole design goal is to be fast [@troyhunt].

A single modern gaming GPU tries tens of billions of SHA-1 guesses per second, so a stolen dump of a fast hash is not ciphertext an attacker must break. It is a race the defender has already lost, for every password common enough to sit in a wordlist. The uncomfortable lesson: the property that makes SHA-1 a good cryptographic hash -- its blazing speed -- is exactly what makes it a catastrophic way to store a password.

Fix that number in your head, because it governs everything that follows: on one NVIDIA RTX 4090, the hashcat benchmark clocks SHA-1 at about 50.6 billion guesses per second and MD5 at 164.1 billion [@hashcat-4090]. Nobody attacked the mathematics. The attacker simply guessed, very fast, against a list of hashes that were never designed to make guessing expensive.

A value derived from a password by a deliberately slow, salted, one-way function and stored so that a leaked database does not directly reveal the passwords. It is distinct from a password-derived *encryption key*, which is computed and used but never stored (LUKS, VeraCrypt, password managers -- the subject of Part 9). The attack this whole primitive exists to raise the cost of: the adversary holds the stolen verifiers and guesses locally at full hardware speed, with no server and no rate limiter in the loop. Nothing throttles them except the per-guess cost you baked into the hash. This is the offline-attacker threat model the series formalizes in Part 1, "Secure Against Whom?"

So here is the thesis, in your hands from the first page: a password hash has exactly one job -- make each attacker guess as expensive as possible on the attacker's cheapest hardware, while staying cheap enough on your server. The variable that decided LinkedIn was never SHA-1's strength. It was SHA-1's speed, compounded by a missing salt that let identical passwords share a hash and let precomputed tables apply.

Carry one diagnostic sentence through every scheme in this article: on the cheapest hardware an attacker can rent, how much does one guess at one password cost, and how far above your own per-login cost can you push it? By the end, you will drop bcrypt, Argon2id, yescrypt, and tomorrow's breach into that one economic question on sight.

Note: A fast hash's cryptographic strength -- collision resistance, preimage resistance -- is almost irrelevant to password storage. The only property that matters is cost per guess. That is why one of the best general-purpose hashes ever designed is one of the worst possible password hashes.

One boundary, stated once. This article is about the stored-verifier case: turning a low-entropy password into a value you store so that a stolen database does not directly yield the passwords. The same primitives -- PBKDF2, scrypt, Argon2 -- also serve as key-derivation functions whose output becomes an encryption key and where nothing is stored (full-disk encryption, password managers). That dual use is Part 9's territory; here it is a pointer, not a detour.

This is also the handoff Part 10, "A Field Guide to Cryptographic Hashes," pointed at: a general-purpose hash such as BLAKE3 is engineered to be fast, which makes it exactly wrong for storing a password. Use a purpose-built password KDF instead. This is Part 12 of 25, and it is that return.

flowchart TD A["Stolen dump: one salt and one hash per user"] --> B["Load into hashcat or John the Ripper on a rented GPU rig"] B --> C["Take the next guess from a wordlist or a brute-force mask"] C --> D["Compute the hash of the guess combined with the stored salt"] D --> E{"Equals a stored hash?"} E -->|"No"| C E -->|"Yes"| F["Password recovered, record it and move on"] F --> C G["The salt forces a per-user recompute and blocks precomputed tables. It imposes no speed limit. A fast hash runs this entire loop billions of times per second."] -.-> D

To make the economics visceral, run the numbers yourself. The rates below are the real measured throughputs of one rented RTX 4090, and the wordlist is the published rockyou.txt -- the first list every attacker loads, whose 14,344,391 entries are reproducible with wc -l [@seclists-rockyou].

{` // Real measured throughputs on one NVIDIA RTX 4090 (hashcat v6.2.6). var guessesPerSecond = { "SHA-1 (fast hash)": 50600000000, // 50.6 GH/s "PBKDF2-HMAC-SHA256, 999 it.": 8865700, // 8.87 MH/s "bcrypt (cost 5)": 184000, // 184 kH/s "scrypt (N=16384)": 7126 // 7,126 H/s };

var wordlist = 14344391; // rockyou.txt: 14,344,391 entries

function human(seconds) { if (seconds < 1) return (seconds * 1000).toFixed(1) + " ms"; if (seconds < 60) return seconds.toFixed(2) + " s"; if (seconds < 3600) return (seconds / 60).toFixed(1) + " min"; if (seconds < 86400) return (seconds / 3600).toFixed(1) + " hours"; return (seconds / 86400).toFixed(1) + " days"; }

for (var scheme in guessesPerSecond) { var seconds = wordlist / guessesPerSecond[scheme]; console.log(scheme.padEnd(30) + human(seconds)); }

console.log(""); console.log("Now add a per-password salt."); console.log("It blocks precomputed tables and stops two users sharing a hash."); console.log("Effect on the throughputs above: exactly 0 percent. Same rates, same wall clock."); `}

The fast hash exhausts all fourteen million guesses in well under a second. The memory-hard end of the table takes hours to do the same work on the same silicon -- a gap of roughly seven orders of magnitude, bought entirely by making each guess cost more. And notice what the salt does to those rates: nothing. It is a necessary defense against precomputation and hash-sharing, and it is useless against speed. Speed is the whole vulnerability.

A password hash has one job: make every guess cost as much as possible on the attacker's cheapest hardware, while staying cheap on yours. A fast hash gets that job exactly backwards.

If the cipher was never the weak link and the key was never touched, then the defense was lost the moment someone chose a fast function to store a guessable secret. That trade-off -- store something guessable, so make guessing expensive -- was understood with total clarity in 1979, by two people who assumed from the start that the attacker would read the password file.

2. 1979: The Three Rules That Still Govern Everything

Bell Labs, 1979. Robert Morris and Ken Thompson publish "Password Security: A Case History" and reason like economists rather than cryptographers [@morris1979]. Their founding assumption is the one every subsequent scheme inherits: assume the attacker can read the password file, because on a shared UNIX system they can. Everything else follows from that single concession.

Three rules fall out of it, and every scheme in this article -- forty-seven years later -- still obeys all three.

A per-password random value stored in the clear alongside the hash, so that identical passwords do not produce identical hashes and precomputed tables cannot be reused. A salt is *public*, not secret; its job is to make each stored verifier unique, not to hide anything. Generating salts correctly is the domain of a CSPRNG, covered in Part 2, "Predictable or Repeated."

Rule one: never store the password -- store a one-way function of it. A stolen file should be a file of hashes, not a file of passwords. Rule two: salt it. Attach a per-password random value -- twelve bits in the original DES crypt -- so identical passwords do not collide and an attacker cannot precompute one giant table and apply it to every account at once. Rule three: make it deliberately slow. Calibrate the cost so that a legitimate single login is trivial but bulk guessing is not.

The mechanism was concrete, and you can see all three rules in it without touching DES's internals. The password was truncated to eight characters and turned into a 56-bit DES key; that key enciphered an all-zero block twenty-five times over; and the twelve-bit salt perturbed DES's expansion permutation so that off-the-shelf DES hardware could not be pointed straight at the problem [@morris1979]. One-way, because you cannot run DES backward without the key that is the password. Salted, because of those twelve bits. Slow, because of those twenty-five iterations.

That salt design is more pointed than it looks. Morris and Thompson perturbed the DES expansion permutation specifically so an attacker could not use stock DES chips to attack the hashes -- the same "make the attacker's silicon useless" instinct that, thirty years later, becomes memory-hardness [@morris1979].

The economic point was already fully formed in 1979: the openly readable password file was a design choice, so the hash itself had to survive offline guessing. That is the offline-attacker model the whole series leans on, and it is why the strength of the underlying cipher was never the interesting question. The interesting question was cost per guess.

timeline title Four decades of stored verifiers 1979 : Morris and Thompson, DES crypt 1995 : md5crypt and the Modular Crypt Format 1999 : bcrypt from the systems world 2000 : PBKDF2 from the standards world 2007 : sha256crypt and sha512crypt 2009 : scrypt, the memory-hard inflection 2015 : Argon2 wins the PHC, yescrypt recognized 2016 : Alwen and Blocki bound data-independent MHFs 2021 : RFC 9106 standardizes Argon2 : yescrypt becomes the Linux shadow default

Founding rule (1979)	What it defeats	How every modern scheme still does it
Store a one-way function, never the password	A stolen file is not a stolen password list	bcrypt, Argon2, and yescrypt are all one-way; you verify by recomputing and comparing
Salt it (per-password random value)	Precomputation and identical-password collisions	a 16-byte CSPRNG salt embedded directly in the stored hash string
Make it deliberately slow	Bulk offline guessing	a tunable work factor, iteration count, or memory cost

Two neighbors are easy to confuse with this one, so an expert reader does not think they were missed. Password-authenticated key exchange -- SRP, OPAQUE -- avoids *sending* the password to the server at all; it is a different primitive with a different goal, and it still needs a stored verifier underneath. Passkeys and WebAuthn go the other direction and eliminate the shared secret entirely. Both are pointers here, not sections. And the *same* functions in this article (PBKDF2, scrypt, Argon2) are also used to derive encryption keys for disks and vaults, where nothing is stored -- that is [Part 9](/blog/no-room-for-a-nonce-disk-and-length-preserving-encryption-fr), not this one.

Three rules, 1979: one-way, salted, slow. They were right, and they are still the entire skeleton. But two of the three had hard numbers baked in -- an eight-character limit, a twelve-bit salt, twenty-five fixed iterations -- and numbers calibrated to 1979 hardware do not stay calibrated. The next fifteen years are the story of those ceilings collapsing.

3. The Antipattern That Would Not Die

A design calibrated to a single moment ages at the speed of Moore's law. DES crypt was tuned for 1979; by the mid-1990s the same twenty-five iterations were a rounding error, and its eight-character truncation threw away everything a longer password could have bought. The fixes that followed solved real problems -- and, in the same breath, set a trap the industry still steps in today.

Removing the ceilings, and inventing migration

Poul-Henning Kamp's md5crypt ( $1$ ), created in 1995 in his own words, removed the eight-character and twelve-bit-salt limits with roughly a thousand iterations of MD5 and a larger salt [@phk-eol]. Its most durable gift, though, was not the hash. It was the format: the Modular Crypt Format (MCF), a self-describing string of the shape $<id>$<params>$<salt>$<hash> that lets one password file hold several algorithms at once and migrate between them over time.

A self-describing hash string of the form `$$$$`. The leading `` names the algorithm, so a verifier can look at a stored value, dispatch to the right function, recompute with the stored parameters, and -- when those parameters fall below today's target -- transparently rehash. It is the reason a single verifier column can hold `$1$`, `$2b$`, `$6$`, and `$argon2id$` values side by side and upgrade in place.

That registry of <id> tags is worth memorizing, because it is a map of this entire article:

MCF prefix	Scheme	Status in 2026
$1$	md5crypt	legacy, retired by its author [@phk-eol]
$2a$ / $2y$ / $2b$	bcrypt	active [@cryptbf]
$5$	sha256crypt	active (older Linux default) [@drepper-shacrypt]
$6$	sha512crypt	active (older Linux default) [@drepper-shacrypt]
$7$	scrypt	active [@libxcrypt]
$y$	yescrypt	active (current Linux default) [@libxcrypt]
$argon2id$	Argon2id	active (cross-platform recommendation) [@rfc9106; @argon2-ref]

{` // A Modular Crypt Format value is self-describing: $$$$. // That is exactly what makes in-place migration possible. function decodeMCF(s) { return s.split("$").filter(function (p) { return p.length > 0; }); }

var argon = "$argon2id$v=19$m=65536,t=3,p=4$c2FsdHNhbHRzYWx0$aGFzaGhhc2hoYXNoaGFzaA"; var bcrypt = "$2b$12$Ro0CUfOqk6cXEKf3dyaM7OhSCvnwM9s4wIX9JeLapehKK5YdLxKcm";

console.log("argon2id ->", JSON.stringify(decodeMCF(argon))); // ["argon2id", "v=19", "m=65536,t=3,p=4", "", ""]

console.log("bcrypt ->", JSON.stringify(decodeMCF(bcrypt))); // ["2b", "12", "<22-char salt and 31-char hash, concatenated>"]

console.log(""); console.log("The first token names the algorithm. A verifier dispatches on it,"); console.log("recomputes with the stored parameters, and rehashes if they are stale."); `}

The other move of this era was the SHA-2 bridge. Ulrich Drepper's sha256crypt and sha512crypt (2007, $5$ and $6$ ) iterated a NIST-approved hash a default of 5,000 rounds, tunable up to just under a billion, with a salt up to sixteen characters [@drepper-shacrypt]. The motive was compliance as much as cryptography: security teams wanted a NIST-tested primitive, which ruled out Blowfish-based bcrypt and made sha512crypt the long-time Linux /etc/shadow default. Same instinct as md5crypt -- make the cost tunable -- with a FIPS-friendly hash underneath.

The antipattern, named and buried

Now the recurring villain of the whole story: the fast, unsalted, general-purpose hash -- raw MD5, SHA-1, or SHA-256 -- pressed into service as a password store. It fails on both axes at once. Unsalted means precomputed rainbow tables apply and identical passwords share a hash. Fast means billions of guesses per second.

This is precisely the LinkedIn disaster from Section 1 -- unsalted SHA-1, plaintext in days [@troyhunt] -- and, one rung worse, RockYou in 2009, which stored roughly 32.6 million passwords in plaintext, no hash at all [@imperva-rockyou]. That dump became rockyou.txt, the canonical cracking wordlist now itself part of the arms race.

The unsettling corollary for defenders: if your password is memorable, it is almost certainly already a line in that cracking wordlist, tried in the first seconds of any offline attack.

Key idea: The fast-hash trap: the property that makes SHA-256 a good general-purpose hash -- blazing speed -- is exactly what makes it a catastrophic password hash. A salt defeats precomputation and hash-sharing, but it changes the attacker's throughput by exactly zero. The attacker never breaks the hash; a stolen fast-hash dump is a race already lost.

Note: Never store a bare fast hash of a password -- not MD5, not SHA-1, not SHA-256, salted or not. A salt is necessary and it stops precomputation, but it does nothing about speed, and speed is the entire vulnerability. This single antipattern is the root cause of most of the failure catalog below.

Kamp himself buried md5crypt in 2012, and his exact words matter because the field has misquoted them ever since. His notice is titled "Md5crypt Password scrambler is no longer considered safe by author," and it reads, plainly, "I implore everybody to migrate to a stronger password scrambler without undue delay" [@phk-eol]. The widely repeated phrase "unfit for use" is a paraphrase, not his language. What he did write is startling in hindsight: the replacement should "soak up area in hardware based attack implementations" -- a demand for area-cost hardening, seven years before scrypt would make it a formal property.

Kamp's 2012 retirement of md5crypt cited about one million guesses per second on a commodity 2012 GPU and the vulnerability CVE-2012-3287. Read his phrasing carefully -- "soak up area in hardware based attack implementations" is a plea for area-time cost, the exact idea memory-hardness would formalize [@phk-eol].

The successor should "soak up area in hardware based attack implementations." -- Poul-Henning Kamp, retiring md5crypt in 2012, describing memory-hardness before it had a name.

Keep this master catalog in view; the article returns to each entry and unpacks the interesting ones in Sections 9 through 11. Every row is the same lesson from a different angle.

Incident	Year	What was stored	Root cause	The lesson
RockYou	2009	Plaintext	No hash at all	The floor of the failure catalog [@imperva-rockyou]
LinkedIn	2012	Unsalted SHA-1	Fast and unsalted	Speed is the vulnerability [@troyhunt]
Adobe	2013	Reversible 3DES in ECB mode	Encryption, not hashing	Hash, never encrypt, a password [@adobe-ars2013]
Ashley Madison	2015	bcrypt cost-12, plus an MD5 token	A strong hash undone by a parallel fast path	You are only as strong as the weakest function [@cynosure-am]
Dropbox	2016	SHA-512, then bcrypt-10, then AES-256	A good example, shown for contrast	Layer defenses, keep a separate pepper [@dropbox]
Facebook	2019	Plaintext, in log files	Plaintext logging beside a good hash	Audit the whole path, not the column [@krebs-fb2019]
Password shucking	2020	`bcrypt(md5(pw))`, unsalted inner	A naive pre-hash	Never feed an unsalted fast hash into bcrypt [@scottbrady-shucking]
crypt_blowfish bug	2011	bcrypt with an 8-bit character flaw	Sign-extension truncation	Why bcrypt has $2x$ and $2y$ tags [@cryptbf]

md5crypt and sha512crypt fixed 1979's ceilings but kept its third rule frozen in place: a fixed iteration count chosen once and forgotten. The next idea was to stop freezing the cost and make it a knob the defender could turn as hardware improved. That idea arrived twice, independently, in two different communities within about a year -- and the blind spot those twins shared is where the modern story actually begins.

4. A Branch, Not a Line: bcrypt and PBKDF2 Are Twins

The textbook story is a clean relay race: each generation hands the baton to a stronger successor that retires it. For password hashing, that story is wrong, and getting it right is the single most useful correction an expert reader can take from this article. Around 1999 and 2000, two communities answered the same question -- "make the cost a tunable knob" -- independently, and neither replaced the other. They are twins from different families, and both are still alive in 2026.

bcrypt, from the systems world (1999)

Niels Provos and David Mazieres built bcrypt for OpenBSD on top of Eksblowfish, an "expensive key schedule" variant of the Blowfish cipher [@bcrypt1999]. The construction repeats Blowfish's key setup $2^{\text{cost}}$ times, then enciphers the constant string "OrpheanBeholderScryDoubt" sixty-four times, salted with 128 bits and encoded as $2a$<cost>$.... The innovation is the adaptive cost factor: raise cost as hardware improves, without changing the algorithm or invalidating a single stored hash. In the authors' own framing, "the computational cost of any secure password scheme must increase as hardware improves" [@bcrypt1999].

A tunable parameter -- bcrypt's `cost`, PBKDF2's iteration count, Argon2's time and memory knobs -- that raises the per-guess cost so a defender can track hardware improvements over time without changing the algorithm or re-hashing existing verifiers with a new scheme. Turning it up is how you "restore the cost ratio" the spine keeps referring to.

bcrypt was also, almost by accident, the first widely deployed scheme to touch memory in a way that mattered. Blowfish's key schedule builds about 4 KiB of S-boxes, and that small working set already made bcrypt meaningfully harder to parallelize on a GPU than a bare hash -- a fact scrypt's paper would later quantify [@scrypt-paper]. Two structural limits recur later: bcrypt silently truncates its input at 72 bytes, and its $2a$ / $2x$ / $2y$ / $2b$ version tags are a bug-fix lineage, not a version-number vanity.

bcrypt's prefix zoo is a repair history. The $2x$ and $2y$ tags came out of a 2011 fix to an 8-bit-character sign-extension flaw in crypt_blowfish (CVE-2011-2483) that could truncate the input; $2b$ matches the behavior of OpenBSD 5.5 and later [@cryptbf].

PBKDF2, from the standards world (2000)

Burt Kaliski's PBKDF2 came out of RSA Laboratories as part of PKCS #5 v2.0, later republished as RFC 2898 and updated in RFC 8018 [@rfc2898; @rfc8018]. Its design is deliberately plain: iterate a pseudorandom function -- in practice HMAC over an approved hash -- a count c times over the password and salt, concatenating output blocks. Because it is built entirely from NIST-approved hashes, it is the option a FIPS-constrained system can actually deploy, which Blowfish-based bcrypt is not. In 2026 it remains the mandated path wherever FIPS applies.

Its weakness has to be stated precisely, because the folklore version is wrong. PBKDF2 is not weak because "GPUs are fast." It is weak because it has negligible, constant memory and is otherwise a bare loop of HMAC, which makes it the most parallelization-friendly of every deployed scheme. The crux is low, constant memory implies cheap, massive parallelism on GPU, FPGA, and ASIC -- an attacker fits thousands of independent PBKDF2 evaluations onto one chip because each one needs almost no RAM.

A nuance the OWASP floors encode: PBKDF2-HMAC-SHA-512 does 64-bit-word arithmetic, which is somewhat less GPU-friendly than SHA-256's 32-bit words. That is exactly why OWASP's SHA-512 iteration floor (210,000) sits below its SHA-256 floor (600,000) -- fewer iterations buy the same attacker cost [@owasp].

bcrypt (1999) and PBKDF2 (2000) are *contemporaries* from different communities, not a before-and-after. sha256crypt and sha512crypt (2007) are a third sibling. And **no earlier scheme was ever cleanly retired**: Linux ran DES `crypt`, then md5crypt, then sha512crypt for decades, and PBKDF2 is still the FIPS default today. Read the "generations" of password hashing as geological strata that *accreted*, not as a relay race where each runner sits down. The only true inflection point in the whole history is memory-hardness (scrypt, 2009).

Here is the master head-to-head. The rightmost columns are the ones that decide real deployments; the article earns each judgment in the sections that follow.

Scheme	Year	New cost lever	Memory-hard?	FIPS lane?	Side-channel posture	Status in 2026
Raw fast hash (MD5, SHA-1, SHA-256)	--	none; general-purpose speed	No	primitive only	n/a	never use for passwords
DES `crypt(3)`	1979	one-way, salt, 25 iterations [@morris1979]	No	No	data-independent	historic
md5crypt ( $1$ )	1995	~1000 iterations, MCF [@phk-eol]	No	No	data-independent	retired by author
bcrypt	1999	adaptive cost, expensive key schedule [@bcrypt1999]	No (touches ~4 KiB)	No	n/a (not an MHF)	active
PBKDF2	2000	iterated HMAC, tunable count [@rfc2898]	No (negligible memory)	Yes	data-independent	active (FIPS)
sha256crypt / sha512crypt	2007	tunable SHA-2 rounds [@drepper-shacrypt]	No	Yes	data-independent	active (legacy default)
scrypt	2009	sequential memory-hardness [@scrypt-paper]	Yes (data-dependent)	No	data-dependent	active
Argon2i	2015	data-independent MHF [@rfc9106]	Yes (provably suboptimal)	No	data-independent	superseded by id
Argon2id	2015	hybrid i-then-d MHF [@rfc9106]	Yes	No	hybrid	recommended default
yescrypt	2015	scrypt plus pwxform, optional ROM [@yescrypt]	Yes (data-dependent)	No	data-dependent	Linux shadow default

The blind spot all three twins shared

Every one of the 1999-2007 schemes stretches time only, with a small fixed memory footprint. Time-hardness has a fatal asymmetry: a function that is slow but tiny in memory is cheap to parallelize. The defender runs one core per login; the attacker runs thousands of small cores on a GPU or millions in an ASIC. "Slower per guess" is a race the defender loses at scale.

One benchmark makes the asymmetry impossible to unsee -- a single RTX 4090 under hashcat v6.2.6 [@hashcat-4090]:

Hash mode (one RTX 4090)	Guesses per second	Slowdown vs MD5
MD5	164.1 billion	1x
SHA-1	50.6 billion	about 3x
PBKDF2-HMAC-SHA256, 999 iterations	8.87 million	about 18,000x
bcrypt, cost 5	184,000	about 890,000x

flowchart LR A["Lever 1: iteration TIME
DES crypt 1979, md5crypt 1995"] -->|"eroded by faster CPUs"| B["Lever 2: expensive KEY SCHEDULE
bcrypt 1999"] B -->|"eroded by cheap parallel GPU, FPGA, ASIC"| C["Lever 3: MEMORY
scrypt 2009, Argon2 2015"] C -->|"eroded by botnets and cheap RAM"| D["Lever 4: MEMORY plus a site ROM
yescrypt"]

Look at the last two rows. PBKDF2 with a thousand iterations barely dents the attacker's throughput, because it is almost pure compute. bcrypt at a low cost of 5 -- with nothing but its 4 KiB of S-boxes -- already costs roughly five orders of magnitude more per guess than a raw hash, purely because it touches memory and cannot be trivially unrolled. That gap is the entire clue.

If a mere 4 KiB of forced memory buys bcrypt that five-orders-of-magnitude edge over a raw hash, the lever is obvious in hindsight: stop making a guess take longer and start making it take more room. CPU cycles and logic gates are cheap and endlessly parallel. Memory bandwidth and capacity are not. That inversion -- from time to memory -- is the one genuine conceptual leap in this entire history.

5. The Breakthrough: Make Memory the Bottleneck

In 2009, Colin Percival stated the idea that splits the timeline in two: force the computation to hold a large array in RAM for a long time, so that computing the hash cheaply requires buying memory -- the one resource an attacker with GPUs, FPGAs, and ASICs cannot parallelize away [@scrypt-paper]. This is not a faster hash. It is a differently shaped one.

scrypt and sequential memory-hardness

The core of scrypt is a routine called ROMix, and you can understand it without touching the Salsa20 mixing inside. It runs in two phases. In the fill phase, it writes an array V[0..N-1] into memory, where each block is the previous block run through a mixing function -- a strictly sequential chain. In the mix phase, it takes N more steps, and on each step it reads V[j] at an index j that depends on the data computed so far, mixing that block back into a running state [@scrypt-paper; @rfc7914].

That second phase is the whole trick. Because the read indices depend on the data, you cannot know in advance which blocks you will need, so computing scrypt with fewer than N blocks resident forces you to recompute missing blocks on demand -- and that recomputation cascades.

To keep the work down, you must hold about N blocks for about N steps: an area times time cost of roughly $AT \approx \Theta(N^2)$. Halving the memory you hold roughly doubles the time, so the AT product stays near $\Theta(N^2)$. Double the cost parameter N instead and both time and memory double, so the attacker's hardware cost rises four-fold [@scrypt-paper].

A function deliberately designed so that computing it cheaply *requires* holding a large amount of memory for a large fraction of its running time. The point is not that it uses memory, but that no attacker strategy avoids the memory without paying more somewhere else -- imposing a cost that cheap, massively parallel hardware cannot dodge. The attacker's real cost metric for a memory-hard function: the memory it occupies multiplied by the time it is held, a proxy for the silicon area and energy one guess consumes. Time-hardness alone raises only the time term; memory-hardness raises the product. Maximizing AT-cost per guess is the entire objective of the memory-hard program. flowchart TD A["Phase 1, FILL. Write N blocks into RAM, each derived from the previous block"] --> B["The whole array of N blocks now sits resident in memory"] B --> C["Phase 2, MIX. Repeat N times"] C --> D{"Read block j, where j depends on the data so far"} D --> E["Mix that block into the running state"] E --> C D -.->|"Hold fewer than N blocks and you must recompute them"| F["Less memory forces far more work, so AT stays high"]

The payoff shows up on the same RTX 4090 that cracked SHA-1 at 50.6 billion guesses per second. scrypt tuned to N of 16384 runs at just 7,126 guesses per second on that hardware [@hashcat-4090] -- about 25 times slower than bcrypt at cost 5, and roughly seven orders of magnitude below raw MD5, because the GPU's thousands of cores now fight over scarce fast memory instead of streaming through cheap arithmetic.

Key idea: Time-hardness is cheap to parallelize: the defender runs one core, the attacker runs thousands, so making a guess merely slower loses the cost race at scale. Making a guess occupy memory for a long time -- an area-times-time cost -- targets the one resource an attacker cannot buy asymmetrically cheaply. That inversion, from time to memory, is the modern era of password hashing.

Institutionalizing it: the PHC and Argon2

Faced with a hard design problem, cryptography did what it does best -- it ran an open competition. The Password Hashing Competition ran from 2013 to 2015, initiated by Jean-Philippe Aumasson, drew 24 submissions, and was judged by a panel in the same spirit as NIST's AES and SHA-3 contests [@phc]. It named one winner, Argon2, and gave special recognition to four finalists.

The PHC's four special-recognition finalists were Catena, Lyra2, Makwa, and yescrypt -- the last of which went on to become the Linux /etc/shadow default. The competition format, modeled on NIST's, is why the field trusts Argon2: it survived years of public cryptanalysis, not a vendor's say-so [@phc].

Argon2 is the purpose-built memory-hard function. It exposes three independent knobs -- time t (passes), memory m (in KiB), and parallelism p -- fills and mixes a large array using a BLAKE2b-based compression function, and encodes its output in MCF as $argon2id$v=19$m=...,t=...,p=...$salt$hash, where v=19 is version 0x13. The reference defaults are t=3, m=4 MiB (that is $2^{12}$ KiB), and p=1 [@argon2-ref]. Crucially, Argon2 comes in three variants, and choosing among them is a threat-model decision, not a performance tweak.

Whether the memory addresses a function reads depend on the password. **Data-independent** (an iMHF) is cache-timing-safe, because the access pattern reveals nothing about the secret -- but, as the next section proves, it cannot be optimally memory-hard. **Data-dependent** (a dMHF) can reach the ideal hardness bound but leaks information through cache-timing side channels. This tension is the reason Argon2id exists. flowchart TD P["Password plus salt"] --> ID{"Which Argon2 variant?"} ID -->|"Argon2d"| D["Data-dependent indexing. Maximum GPU and trade-off resistance, but the access pattern leaks through cache timing"] ID -->|"Argon2i"| I["Data-independent indexing. Cache-timing safe, but provably attackable, needs more passes"] ID -->|"Argon2id"| H["Hybrid. Data-independent for the first half-pass, then data-dependent"] H --> A["Answers both threat models, so RFC 9106 makes it must-support"]

Argon2d uses data-dependent indexing for maximum resistance to GPU and trade-off attacks, at the cost of a cache-timing side channel. Argon2i uses data-independent indexing, which is cache-timing-safe but needs more passes and is provably weaker. Argon2id is the hybrid: data-independent addressing for the first half-pass, where the side channel matters, then data-dependent addressing for the rest. RFC 9106 makes Argon2id the MUST-support variant [@rfc9106], for reasons the theory section will make exact.

Memory-hardness works. It is the first idea in thirty years that changes the shape of the attacker's cost, not merely its size. So the modern era ought to be simple -- pick the memory-hard winner, turn the memory knob to maximum, and go home. It is not simple, and the two reasons why -- one you meet in production, one buried in complexity theory -- are the rest of this article.

6. State of the Art (2024-2026): Two Defaults, Not One

Ask a working engineer "what should I use in 2026?" and the honest answer is two names, not one -- and which one depends on what you are protecting and where it runs.

Argon2id: the recommended cross-platform default

For greenfield application code, Argon2id is the choice. It is the RFC 9106 MUST-support hybrid and the OWASP recommendation, and it meets NIST SP 800-63B-4's generic requirements -- though NIST itself names no specific memory-hard scheme, and its only approved-scheme pointer is SP 800-132 (PBKDF2) [@nist-63b]. Its one algorithm answers both threat models at once: the data-independent first half-pass closes the cache-timing side channel, and the data-dependent tail resists GPU and ASIC trade-offs. You get independent time, memory, and parallelism knobs, plus a self-describing MCF encoding that makes migration a matter of reading the stored parameters [@rfc9106; @argon2-ref].

Two authoritative parameter sets anchor it. RFC 9106's first recommended option is t=1, p=4, m=2 GiB, a default suitable for all environments; its memory-constrained second option is t=3, p=4, m=64 MiB [@rfc9106]. OWASP publishes a much lower floor -- m=19 MiB, t=2, p=1 -- and it is exactly that: a bare minimum to clear, not a target to aim at [@owasp].

Where Argon2id struggles is the mirror image of its strength. The defender pays the full memory cost on every login, so a 2 GiB-per-hash setting is a genuine RAM and denial-of-service hazard -- which is precisely why the deployed floors sit so far below the theoretical ideal. Some language bindings also default to the weaker Argon2i or to under-powered parameters, so "we use Argon2" is not the same claim as "we use Argon2id at sound parameters."

yescrypt: the Linux `/etc/shadow` default

Alexander Peslyak's yescrypt, from Openwall, is what actually hashes the login password on most modern Linux systems. It builds on scrypt's sequential memory-hardness and adds pwxform -- S-box lookups interleaved with integer multiplication that annoy GPUs, FPGAs, and ASICs even at low memory settings -- plus an optional large, site-specific ROM of tens of gigabytes that throttles "attackers' use of pre-existing hardware such as botnet nodes" [@yescrypt].

It is engineered so its effective memory cost cannot be driven "unreasonably low" the way plain scrypt's and Argon2's can be at high throughput and low latency. That is exactly why it dominates the high-request-rate regime an OS login endpoint lives in [@yescrypt].

yescrypt is the default /etc/shadow scheme on ALT Linux, Arch, Debian 11 and later (it became the default in 11, not 12), Fedora 35 and later, Kali 2021.1 and later, and Ubuntu 22.04 and later; it is also supported but not default on Fedora 29+, RHEL 9+, and Ubuntu 20.04+. The selection mechanism is libxcrypt together with login.defs [@yescrypt; @libxcrypt].

Where yescrypt struggles is breadth: it is Linux-centric, comparatively young, and thin on non-Linux bindings, and its ROM feature demands real operational machinery -- the ROM must never be lost, because losing it invalidates every hash computed against it.

Beware a naming collision before you grep a Windows codebase: the BCrypt* functions in Windows CNG (Cryptography API: Next Generation) are not the bcrypt password hash -- they are the platform's general crypto API [@microsoft-cng]. See the CNG architecture post for what those calls actually do.

The honest status line

scrypt (still widely deployed, standardized as RFC 7914), bcrypt (ubiquitous and battle-tested), and PBKDF2 (the FIPS lane) are all active, not retired. The branch-not-line reality of Section 4 persists straight into the present: there is no single winner that made the others obsolete, only a recommended default and a set of live alternatives chosen by constraint.

Note: Argon2id at RFC 9106 parameters for greenfield application code. Leave yescrypt alone on modern Linux -- it is already the /etc/shadow default, so do not reinvent it. bcrypt at work factor 10 or higher for small, legacy, or simplicity-first services. PBKDF2-HMAC-SHA256 at 600,000 iterations only when FIPS forces your hand.

"Use Argon2id, or leave yescrypt alone" is the entire answer for new code on a known platform. But engineers inherit constraints -- a regulator that forbids the primitive, a runtime with no good binding, a login endpoint one traffic spike away from falling over. So the real question is not "what is best" in the abstract. It is "what are my options, ranked, and exactly when does each one apply?"

7. The Decision Matrix: Use X With These Parameters in Case Y

Five deployed schemes, one page, ranked -- not by fashion but by the economic yardstick this whole article is built on: how much does one attacker guess cost, and what does imposing that cost do to you? The master matrix in Section 4 lays out the axes that decide it -- memory-hardness, standardization and FIPS status, side-channel posture (data-dependent versus data-independent), library breadth, and tunability. What follows turns those axes into rules you can apply on sight.

Your binding constraint	Use this	At these parameters
Greenfield app, good library	Argon2id [@rfc9106]	`2 GiB, t=1, p=4` if affordable; else `64 MiB, t=3, p=4`; never below the `19 MiB, t=2, p=1` floor [@owasp]
Linux OS login	yescrypt [@yescrypt]	the distro default; do not hand-roll it [@libxcrypt]
Huge single-operator, can run a ROM	yescrypt with ROM [@yescrypt]	raise the attacker's capital cost in scarce large RAM
Need memory-hardness, no Argon2id	scrypt [@owasp]	`N=2^17, r=8, p=1` (about 128 MiB)
FIPS-140 or regulatory constraint	PBKDF2-HMAC-SHA256 [@owasp]	at least `600,000` iterations
Legacy, small, or simplicity-first	bcrypt [@owasp]	work factor 10 or higher; enforce the 72-byte cap

Read the axes together and the rules explain themselves. If you need cache-timing safety on a shared or multi-tenant host, you want a data-independent or hybrid scheme, which points at Argon2id over Argon2d or scrypt. If a regulator dictates approved primitives, memory-hardness is off the table entirely and PBKDF2 is the only compliant answer, whatever its per-guess weakness.

If your constraint is a login endpoint hammered at high request rates, yescrypt's low-memory hardening earns its keep. And if your real constraint is a small team that needs one dependency-light library that has existed for two decades, bcrypt is a defensible engineering choice, not a mistake.

That last point is the one that makes this section trustworthy, so state it plainly: the ranking is not absolute. A well-tuned bcrypt on a small service can out-secure an under-provisioned Argon2id that a denial-of-service-averse operator was forced to dial down to the 19 MiB floor. The most credible voice on this is not a bcrypt partisan -- it is the author of yescrypt, the very scheme that sits at the top of the memory-hard heap.

"For smaller deployments, bcrypt with its simplicity and existing library support is a reasonable short-term choice." -- Alexander Peslyak, author of yescrypt [@yescrypt]

FIPS deserves one more word, because it does something the rest of the matrix does not: it collapses the whole decision to a single cell. Under a FIPS-140 constraint, none of the memory-hard reasoning applies -- Argon2, scrypt, and yescrypt are simply not approved -- and you deploy PBKDF2-HMAC-SHA256 at a high iteration count and accept its parallelism weakness as the price of compliance [@owasp]. The economic yardstick still governs; the regulator has just fixed one of its variables for you.

The matrix ranks the schemes, but notice what it quietly concedes. Every "best" here is bounded by what the defender can afford to run on every single login, and every entry leans on assumptions about attacker hardware that no one can hold fixed across a decade. Pull on that thread and you arrive at the theory -- where memory-hardness turns out to carry a hard, proven ceiling that no amount of engineering can lift.

8. Theoretical Limits: Memory-Hard by Design Is Not Optimally Memory-Hard

Password hashing is one of the rare corners of applied cryptography with real complexity theory bolted to it -- and the theory delivers two humbling results the marketing never mentions.

The ideal, and the impossibility

The best conceivable attacker cost for a function that touches n blocks over n steps is $AT = \Theta(n^2)$: you pay the full memory-time area even with unlimited parallel cores. scrypt's data-dependent ROMix is designed to reach it. But you cannot have all three of $\Theta(n^2)$ hardness, cache-timing safety, and free parameter choice at once -- and that is a theorem, not a limitation of any particular design.

Joel Alwen and Jeremiah Blocki proved that data-independent memory-hard functions -- the class you are forced into for cache-timing safety, because their access pattern must not depend on the password -- provably cannot reach the ideal bound. Any iMHF can be computed at $O(n^2/\log^{1-\varepsilon} n)$, and Argon2i specifically at $O(n^{7/4}\log n)$ [@alwen-blocki2016]. That sublinear shortfall is small, but it is real, and it is unavoidable for the whole data-independent class.

This is the single most consequential theorem in the area, because it is the actual reason RFC 9106 prefers the Argon2id hybrid over the safer-looking pure-data-independent Argon2i. Recall the addressing diagram from Section 5: Argon2id uses data-independent addressing only for the first half-pass, where the side channel would otherwise leak, then switches to data-dependent addressing, which escapes the iMHF impossibility.

The RFC bakes this in directly. It makes Argon2id MUST-support, notes that Argon2i "makes more passes over the memory to protect from trade-off attacks," and leans on the Alwen and Blocki attack -- its reference tag [AB16] -- to conclude that three passes is "almost optimal" for Argon2i [@rfc9106; @alwen-blocki2016].

The follow-on results fill in the picture, and they are worth one breath each so the section stays proportionate. Blocki and Zhou tightened the Argon2i bound to $O(n^{1.768})$ from above and $\Omega(n^{1.75})$ from below [@blocki-zhou2017]. DRSample is the best practical iMHF construction known, built for high depth-robustness [@drsample2017]. Balloon was the first practical iMHF to ship with a memory-hardness proof [@balloon2016].

On the data-dependent side, the attacks are only constant-factor trade-offs found by real cryptanalysis rather than asymptotic breaks -- Biryukov and Khovratovich, for instance, cut Catena's AT-cost by a factor of about 25 using $M^{4/5}$ memory [@biryukov-khovratovich2015]. The map is detailed, but the headline is simple: the class you need for side-channel safety is the class that cannot be optimal, so the field hybridizes.

The second, mundane impossibility: no free lunch

The other limit needs no complexity theory at all. The defender pays the same per-evaluation cost as one attacker guess. So the hardness you can impose is capped by your own budget: the RAM you can spare per concurrent login, the hash latency your users will tolerate (roughly one second at the very most), and the denial-of-service surface you expose when an attacker hammers your login endpoint to make you do the expensive work.

RFC 9106 frames the rational defender not as someone climbing an unbounded hardness curve but as someone maximizing attacker cost for a fixed defender time budget -- an optimization with a ceiling, not a dial that goes to infinity [@rfc9106].

Note: You cannot make a password hash arbitrarily strong. You can only make it as strong as you can afford to run on every login, because your per-evaluation cost equals one attacker guess. Hardness is bounded by your RAM times concurrency, your latency tolerance, and your denial-of-service budget -- not by the algorithm.

Quantum computing does not change this calculus either. The Section 11 FAQ carries the full argument; in short, Grover's square-root speedup is no Shor-style break against a one-way hash and is swamped by the per-guess costs a memory-hard scheme already imposes, so post-quantum worry belongs to your key exchange, not your password store [@nist-ir-8105].

Key idea: The humbling synthesis: "memory-hard by design" is not "optimally memory-hard" -- Alwen and Blocki prove the data-independent class cannot reach the ideal AT bound, which is why Argon2id is a hybrid. The defender pays the memory too, so hardness is bounded by your own budget. And the hash is rarely the weakest link anyway. The verdict is not "use the strongest function" but "price a guess correctly, on both sides of the ledger" -- an economic decision, not a single winner.

So the design problem has a proven best answer -- the id hybrid -- and a proven ceiling -- your own budget. Yet scanners still find broken password storage in production every year, and the field still argues, in public, about whether any of this memory-hardness is worth paying for. Both of those live on the open frontier.

9. Open Problems: The Arms Race, and the Argument About Whether to Fight It

The construction-level question is largely settled -- use the id hybrid, and the theory tells you why. The deployment question is wide open, and one of its honest sub-questions is whether the field's own default is worth its price.

Principled parameter selection under real budgets. The question every engineer actually has -- given RAM times concurrency, a latency target, login queries per second, and a denial-of-service tolerance, output the AT-optimal (m, t, p) -- has no closed-form answer. RFC 9106 offers two fixed configurations; OWASP offers equivalence classes; neither is a portable optimizer you can point at your server and trust [@rfc9106; @owasp]. In practice you measure, pick a latency target, and revisit it, which is craft, not calculation.

The moving hardware economics. Every parameter floor silently assumes an attacker hardware model, and that model drifts. When a new GPU generation gains memory bandwidth, yesterday's "19 MiB minimum" quietly weakens without a line of your code changing. The RTX 4090 benchmark in this article is one snapshot of a gradient that shifts every hardware generation [@hashcat-4090]. yescrypt's optional ROM is one structural answer: raise the attacker's capital cost -- the price of enough scarce large RAM -- rather than only the marginal cost per guess [@yescrypt].

Peppering and secret-salt key management. A server-held secret mixed into the hash makes a stolen database uncrackable without also stealing a separately stored key -- a large practical win. But get the construction wrong and you cannot ever rotate that secret.

A *secret*, server-held value -- kept in an HSM, KMS, or TEE, separate from the database -- mixed into the hash as defense-in-depth, so that a stolen database *alone* cannot be cracked. It is distinct from the public per-password salt: the salt makes verifiers unique and is stored beside them; the pepper is a secret and must never be.

The engineering answer is to encrypt the hash under the pepper (which is rotatable, the design Dropbox shipped) rather than to HMAC it into a one-way step (which is not). NIST SP 800-63B-4 recommends exactly this: an additional keyed hashing or encryption operation with the key held in a hardware-protected area such as an HSM or TPM [@dropbox; @nist-63b]. The how-to is in Section 10.

Present both sides, because reasonable experts disagree. **For:** the AT-cost theory and the benchmark gradient are real -- scrypt costs an attacker orders of magnitude more per guess than PBKDF2, and bcrypt sits in between [@scrypt-paper; @hashcat-4090]. **Against, for small deployments:** yescrypt's *own author* endorses bcrypt as a reasonable short-term choice, and Dropbox shipped plain bcrypt at cost 10 plus a global pepper and called it among the strongest approaches in production [@yescrypt; @dropbox]. And the sharpest point of all: the failure catalog shows the *usual* weakest link is almost never the hash's memory parameter. Ashley Madison's genuinely strong bcrypt at cost 12 fell because the same database also stored a fast MD5 token, `md5(lc($username)."::".lc($pass))`, that CynoSure Prime cracked in hours and then case-corrected against the bcrypt values [@cynosure-am]. Facebook logged between 200 and 600 million passwords in *plaintext*, searchable by some 20,000 employees [@krebs-fb2019]. Adobe *encrypted* about 130 million passwords with reversible 3DES in ECB mode instead of hashing them [@adobe-ars2013]. And password shucking turns a naive `bcrypt(md5($pw))` into a fast-hash problem: attacking the bcrypt directly runs near 2,000 guesses per second, but the shucked MD5 runs near 64 *billion* [@scottbrady-shucking]. In every case, memory-hardness would have changed nothing. The correct framing is economic, not dogmatic: memory-hardness is the recommended *default*, not a universal mandate.

Every open problem here reduces to the sentence this article began with -- maximize attacker cost per guess without bankrupting the defender -- now carrying one humbling clause: the hash is only one term in a much larger security budget, and often not the term that decides the outcome. Which means the practical guide is not a victory lap. It is that sentence, made operational.

10. The Practical Guide: Decision Rules, Parameters, and Shipping Code

Everything above collapses into five short things: a decision procedure, a parameter table, the concrete library call for each scheme, a list of patterns to never ship, and the login-handler code the whole article has been building toward. None of it requires re-deriving the theory. It requires knowing your one binding constraint and reading down from there.

Start with the constraint, because it -- not fashion -- picks the scheme. Walk this tree top to bottom and stop at the first branch that matches your situation.

flowchart TD S["Start: choosing a password hash"] --> F{"FIPS or regulatory constraint on primitives?"} F -->|"Yes"| FP["PBKDF2-HMAC-SHA256, at least 600,000 iterations"] F -->|"No"| L{"Hashing the Linux /etc/shadow login?"} L -->|"Yes"| LY["yescrypt, the distro default via libxcrypt. Do not reinvent it"] L -->|"No"| G{"Greenfield app with a good library?"} G -->|"Yes"| GA["Argon2id. Target 2 GiB, t=1, p=4. Else 64 MiB, t=3, p=4. Never below 19 MiB, t=2, p=1"] G -->|"No"| H{"Huge single-operator deployment that can run a multi-GB ROM?"} H -->|"Yes"| HR["yescrypt plus a site ROM, to raise the attacker's capital cost"] H -->|"No"| M{"Need memory-hardness but Argon2id is unavailable?"} M -->|"Yes"| MS["scrypt, N of 2^17, r=8, p=1, about 128 MiB"] M -->|"No"| BC["bcrypt, work factor 10 or higher, 72-byte cap enforced"]

The tree points at a scheme; the table below pins the numbers. Every figure here is drawn from RFC 9106, the OWASP Password Storage Cheat Sheet, yescrypt's documentation, or NIST SP 800-63B-4 [@rfc9106; @owasp; @yescrypt; @nist-63b]. One trap to name explicitly: OWASP's iteration counts drift upward over time, so this table prints the values from the dated 2025-06-02 snapshot, not the live page -- if you copy a number, copy it from a dated source so your reviewer can reproduce it.

Scheme	Parameters to start from (2024-2026)	Salt	Notes
Argon2id	RFC 9106 target `t=1, p=4, m=2 GiB`; memory-constrained `t=3, p=4, m=64 MiB`; OWASP floor `t=2, p=1, m=19 MiB` [@rfc9106; @owasp]	128-bit	Pin the id variant; never ship bare Argon2i as the default
yescrypt	The distro default settings; add a multi-GB ROM at scale [@yescrypt]	library-managed	Linux `/etc/shadow`; $y$ MCF
scrypt	`N` of `2^17`, `r=8`, `p=1` (about 128 MiB) [@owasp]	128-bit	Or an equivalent point on the p-for-RAM curve
bcrypt	Work factor 10 or higher; enforce the 72-byte cap [@owasp]	128-bit	Safe pre-hash only as `bcrypt(base64(hmac-sha384($pw, key=pepper)))`
PBKDF2 (FIPS lane)	HMAC-SHA256 600,000; HMAC-SHA512 210,000; HMAC-SHA1 1,300,000 (legacy) [@owasp]	32 bits or more; use 16 bytes	These are the dated-snapshot figures; the live page has since drifted up

From a scheme name to a real API call

Naming the scheme is not the same as shipping it, and the gap between them is where a surprising amount of security is lost. Here is the bridge from each scheme to the concrete binding a protocol designer actually calls.

Scheme	Where to call it, in real code
Argon2id	libsodium `crypto_pwhash`, the PHC reference library, `argon2-cffi` (Python), PHP `PASSWORD_ARGON2ID`, `golang.org/x/crypto/argon2`, Rust `argon2`, `node-argon2`, and .NET wrappers [@argon2-ref]
yescrypt	`libxcrypt` through `crypt(3)`, plus Openwall's reference code [@libxcrypt]
scrypt	OpenSSL, libsodium, Python `hashlib.scrypt`, Node `crypto.scrypt` [@rfc7914]
bcrypt	PHP `PASSWORD_BCRYPT`, Spring Security, Django, and `bcrypt` libraries for Ruby, Node, and Python [@cryptbf]
PBKDF2	Every standard library: .NET `Rfc2898DeriveBytes`, Python `hashlib.pbkdf2_hmac`, Django's default hasher [@rfc8018]

Note: The load-bearing warning of this whole section: several Argon2 bindings default to the weaker Argon2i, or to under-powered memory and time settings. "We use Argon2" is not the claim you want -- "we use Argon2id at t, m, p we chose on purpose" is. Read your library's default constructor, pin the variant and every parameter explicitly, and write a test that fails if the stored cost drops below your target [@argon2-ref].

The four things you always do

Whatever the tree selected, four habits are non-negotiable. NIST SP 800-63B-4 section 3.1.1.2 requires (SHALL) two of them -- salted and hashed storage with an approved one-way function, and verification of the entire password with no truncation -- and recommends (SHOULD) a third, a stored algorithm-and-cost reference so you can migrate; it also calls for throttling on repeated failures [@nist-63b].

First, generate a 16-byte per-password salt from a CSPRNG. NIST's floor is a mere 32 bits; treat that as a floor to clear by a wide margin, not a target, and generate it the way Part 2, "Predictable or Repeated," insists [@nist-63b]. Second, store the algorithm and its parameters in MCF form, so a verifier can dispatch on the scheme and upgrade in place. Third, compare verifiers in constant time, so you never leak how many bytes an attacker has guessed.

Fourth, rehash on login: when the stored cost sits below today's target, transparently recompute the verifier from the plaintext you just checked, and write the stronger value back. That last habit turns a parameter choice from a one-time decision into a living one -- and it is short enough to paste in.

{` // Mock only -- this illustrates control flow, not real crypto. // A real system calls its library's verify(), needsRehash(), and hash().

var TODAYS_TARGET_COST = 12; // e.g. a bcrypt work factor, or an Argon2 (m,t,p) tuple

// Constant-time comparison: never return early on the first differing byte, // or the timing leaks how much of the verifier the attacker has matched. function constantTimeEquals(a, b) { if (a.length !== b.length) return false; var diff = 0; for (var i = 0; i < a.length; i++) { diff |= a.charCodeAt(i) ^ b.charCodeAt(i); } return diff === 0; }

function mockHash(password, cost) { // Stand-in for bcrypt or Argon2id. Real code never rolls its own. return "cost=" + cost + ":" + password.split("").reverse().join(""); } function costOf(verifier) { return parseInt(verifier.split(":")[0].split("=")[1], 10); }

var stored = mockHash("correct horse battery staple", 10); // hashed years ago at cost 10 console.log(login("correct horse battery staple", stored, function (h) { console.log(" stored verifier upgraded ->", h.slice(0, 16) + "..."); })); console.log(login("wrong password", stored, function () {})); `}

Peppering, and server-cost budgeting

Two operational topics finish the guide. The first is peppering -- powerful, and easy to get wrong in a way you cannot undo.

A pepper is a *secret* mixed into the hash as defense-in-depth, and three things decide whether it helps or hurts. **One: keep the secret out of the database.** It belongs in an HSM, a KMS, or a TEE, never in the same store as the verifiers it is meant to protect -- a pepper sitting next to the hash is not a pepper, it is decoration. **Two: encrypt the hash under the pepper, do not HMAC it in.** Dropbox's design is the reference: `SHA512(pw)`, then bcrypt at cost 10 with a per-user salt, then `AES-256` encryption of that output under a global pepper held separately [@dropbox]. Because encryption is reversible with the key, you can *rotate* the pepper; an HMAC folded into the one-way step cannot be rotated without every user's plaintext, which you do not have. **Three: treat it as a bonus layer, never a substitute** for a real password hash. NIST SP 800-63B-4 recommends exactly this shape -- an additional keyed hashing or encryption operation with the key in a hardware-protected area [@nist-63b]. And if you pepper to work around bcrypt's 72-byte cap, the safe pre-hash is `bcrypt(base64(hmac-sha384($pw, key=pepper)))`: the keyed HMAC defeats password shucking, and the base64 encoding defeats the `$2a$` NUL-truncation bug [@owasp; @scottbrady-shucking].

The second is the budget the whole design lives inside. Memory multiplied by concurrency is the real constraint, not CPU: Argon2id at 1 GiB per hash across 200 simultaneous logins wants roughly 200 GiB of transient RAM. That is the concrete reason the deployed floors of 19 to 64 MiB sit so far below RFC 9106's 2 GiB ideal [@rfc9106; @owasp].

Target somewhere between half a second and one second per hash, measured on your own hardware. Rate-limit the login endpoint, because an attacker who floods it makes you pay the expensive hashing cost -- a denial-of-service lever you handed them. And where the traffic justifies it, put a cheap pre-filter in front of the expensive hash. The parameters are not a trophy you set once; they are a running lease on your own server budget.

The parameter tables above are starting points, not answers -- the only number that finally matters is how long a hash takes on *your* hardware. The Argon2 reference CLI measures it directly. This runs Argon2id at 64 MiB, three passes, and four lanes, and prints the elapsed time:

echo -n "correct horse battery staple" | argon2 mysalt1234 -id -t 3 -m 16 -p 4

Here -id selects the Argon2id variant and -m 16 means 2^16 KiB, which is 64 MiB. Raise -m until the reported time approaches your latency budget -- roughly half a second to a second -- then hold there, and re-measure on the hardware you will actually deploy on, not your laptop [@argon2-ref].

The checklist is short because the lesson is one sentence. But checklists are not what fail in production -- confident, wrong sentences are, the kind that survive code review because they sound obviously true. Clear those before the conclusion, and the bug that has haunted this entire article finally has nowhere left to live.

11. FAQ and Common Misuse: The Confident, Wrong Sentences

Broken password storage survives mostly because a handful of plausible-sounding claims keep getting said in design meetings, and a handful of code patterns keep getting shipped past review. Named and corrected, they lose their power. Here is the consolidated catalog of what goes wrong in real code, each pattern mapped to the named breach it reproduces and the one change that fixes it.

Misuse pattern in the code	The break it reproduces	The fix
A bare fast hash (MD5, SHA-1, SHA-256) as the password store	LinkedIn, RockYou	Use a deliberately slow, ideally memory-hard KDF [@troyhunt]
bcrypt silently truncating input at 72 bytes	Long passphrases lose entropy past byte 72	Enforce the 72-byte cap explicitly, or pre-hash correctly [@cryptbf; @owasp]
Naive unsalted pre-hash `bcrypt(md5($pw))`, and the $2a$ NUL bug	Password shucking	Salt or HMAC the inner step and base64-encode it [@scottbrady-shucking]
Argon2d (data-dependent) on shared or multi-tenant hosts	Cache-timing exposure of the access pattern	Prefer Argon2id, whose first half-pass is data-independent [@rfc9106]
A global salt, a reused salt, or no salt	LinkedIn: rainbow tables and cross-account hash sharing	A unique per-password CSPRNG salt, stored in the MCF string [@troyhunt]
A parallel weak code path that also touches the password	Ashley Madison's fast MD5 `$loginkey` beside strong bcrypt	Audit every function that sees the password, not just the verifier [@cynosure-am]
Encrypting the password instead of hashing it	Adobe's reversible 3DES in ECB mode	Hash, never encrypt, a stored password [@adobe-ars2013]
Storing the pepper next to the hash it protects	Defeats the entire point of a secret	Keep the pepper in a separate HSM, KMS, or TEE [@nist-63b]
Parameters chosen once and never raised	A 2019 cost is weaker every year after	Rehash on login and lift the floor as hardware improves
No maximum input length feeding PBKDF2	Django's 2013 long-password DoS (CVE-2013-1443)	Cap the accepted input length; Django's fix rejected inputs over 4096 bytes [@django-cve]
Verifier comparison that short-circuits on the first byte	A timing side channel on the compare	Constant-time equality, plus throttling of failed attempts [@nist-63b]
"Memory-hardness is always mandatory"	Over-provisioned Argon2id dialed to a DoS-safe floor	Treat memory-hardness as the default, not a universal law [@yescrypt]

The Django entry is worth one sentence, because it is the misuse people least expect. Django set no maximum on password length, so an attacker could submit a roughly one-megabyte password -- about a minute of PBKDF2 work -- purely to exhaust server CPU, a self-inflicted denial of service the fix closed by rejecting any input over 4096 bytes [@django-cve].

Note: Your password security is the weakest function that ever touches the password. A strong bcrypt sitting beside a fast MD5 token (Ashley Madison), a plaintext line in a log file (Facebook), or a naive bcrypt(md5($pw)) (shucking) erases every bit of the verifier's strength. Audit the whole path, not just the column labeled "password hash."

With the code patterns cleared, here are the sentences themselves -- the confident, wrong claims, and the short answers that retire them.

Because its strength is not the property that matters, and its speed is the flaw. A password hash exists to make each guess expensive; SHA-256 is engineered to make each evaluation cheap, so one GPU tries tens of billions of guesses per second and a stolen dump falls to a wordlist in minutes [@hashcat-4090]. Use a deliberately slow, ideally memory-hard function -- Argon2id, scrypt, bcrypt, or PBKDF2 -- not a general-purpose hash. They already salt for you: bcrypt and Argon2id generate a per-password salt and embed it in the MCF string. What you must never do is reuse a salt, use a single global salt, or omit it -- that is precisely the LinkedIn failure, where identical passwords collided and precomputed tables applied [@troyhunt]. The salt is public and stored beside the hash; it is not the secret, and it changes attacker throughput by zero. No. bcrypt (1999) and PBKDF2 (2000) were never superseded, only joined by the memory-hard generation. bcrypt at work factor 10 or higher is a defensible choice for many deployments -- yescrypt's own author says as much for smaller ones -- and a FIPS constraint forces PBKDF2 regardless [@yescrypt]. Memory-hardness is the recommended default, not a universal mandate. A pepper is a *secret*, server-held value -- kept in an HSM or KMS, separate from the database -- mixed into the hash so a stolen database alone cannot be cracked [@nist-63b]. It is defense-in-depth only, never a substitute for a real password hash. If you use one, *encrypt* the hash under it (rotatable, the design Dropbox shipped) rather than HMAC it into the one-way step (not rotatable), and never store it next to the hash [@dropbox]. Only if you do it the careful way. A naive `bcrypt(md5($pw))` with an unsalted inner hash enables password shucking, and a NUL byte in the pre-hash can truncate the input [@scottbrady-shucking]. Salt or HMAC the inner step and base64-encode it -- `bcrypt(base64(hmac-sha384($pw, key=pepper)))` -- or enforce the 72-byte cap and skip pre-hashing entirely. No. Grover's algorithm offers only a square-root speedup on guessing, which roughly halves effective bit-strength, and that gain is swamped by the memory-bandwidth and per-guess costs a good password hash already imposes. No Shor-style structural break applies to a one-way hash [@nist-ir-8105]. Your multi-factor and password-reuse posture matter far more to your real risk. There is no closed-form answer, which is why this is still an open problem. Target roughly half a second to one second per hash, measured on your own hardware, within your RAM-times-peak-concurrency and denial-of-service budget. Start from RFC 9106's two configurations or OWASP's floors, and raise them over time with rehash-on-login as your hardware and your traffic allow [@rfc9106; @owasp].

Every correction here points at the same root. The strength of the hash was never the variable that decided the outcome. The cost per guess -- on the attacker's cheapest hardware, measured against your own server's budget -- always was. That is the sentence the whole article was built to earn, so it is time to say it in full.

One Job, Priced Two Ways

Return to the LinkedIn dump from the first page: a strong hash, no key to break, plaintext in days -- because the function was fast, and fast is the one thing a password hash must never be. Everything between here and there was the field learning, and relearning, that the hash's cryptographic strength was never the variable. The cost of one guess was.

Read as one arc, forty-seven years of password hashing is a single economic move, repeated. Morris and Thompson's 1979 rules -- one-way, salted, slow -- set the terms. Every advance since has restored the same eroding ratio: the cost of one attacker guess divided by the cost of one honest login.

Faster CPUs eroded fixed iteration counts, so bcrypt made the count a knob and added an expensive key schedule. Cheap parallel hardware -- GPUs, FPGAs, ASICs -- eroded time-hardness itself, so scrypt and Argon2 changed the shape of the cost from time to memory, the one resource an attacker cannot buy asymmetrically cheaply. Botnets and cheap RAM erode even that, so yescrypt adds a site-specific ROM. Iteration time, then an expensive key schedule, then memory, then memory plus a ROM: four levers, one idea.

The failure catalog proves the spine from the other side, because every entry is the ratio collapsing somewhere the verifier column could not see. Ashley Madison ran genuinely strong bcrypt and lost anyway, to a fast MD5 token stored beside it [@cynosure-am]. Facebook's at-rest hashing was undone by plaintext in a log file [@krebs-fb2019]. Adobe encrypted where it should have hashed [@adobe-ars2013]. Password shucking turned a careless pre-hash back into a fast-hash problem [@scottbrady-shucking]. In none of these did memory-hardness matter, because the weakest function that touched the password was never the one on the spec sheet.

So the resolution the field converged on is not a single winner. It is a priced menu: Argon2id and yescrypt where you can afford memory-hardness, bcrypt and PBKDF2 where you cannot or where a regulator forbids it -- each choice an economic decision bounded by your own server's budget, exactly as Part 1's attacker model and Part 10's "a fast hash is the wrong tool" both predicted. Keep the one-question test where you can reach it.

On the cheapest hardware an attacker can rent, how much does one guess at one password cost -- and how far above your own per-login cost have you pushed it?

The hash was never the weak link, and no future hash will be, because the weak link is economic, not cryptographic. That is why this is Part 12 of a field guide to protocol design rather than a catalog of algorithms. The job was never to pick the strongest function. It was to price a guess correctly, on both sides of the ledger.

The Tag Verified, the Cipher Held, the Forgery Went Through: A Field Guide to Message Authentication and Safe Composition

noreply@paragmali.com (Parag Mali) — Sat, 11 Jul 2026 01:12:01 GMT

A MAC makes forgery hard; three deployment assumptions keep it hard, and every famous break -- Flickr [@flickr-advisory], Lucky Thirteen [@cve-2013-0169], POODLE [@cve-2014-3566], the SSH plaintext-recovery attack [@apw-2009], and 184 nonce-repeating HTTPS servers [@woot16-bock] -- broke one of those assumptions, not the math. Pick the right family (PRF-based HMAC and KMAC take a reusable key and need no nonce; Wegman-Carter GMAC and Poly1305 run at line rate but demand a one-time key [@wc-1981]), verify in constant time, and always Encrypt-then-MAC over the ciphertext with an independent key and a strongly unforgeable tag [@bn-2000]. This guide traces the primitive from Carter-Wegman universal hashing in 1979 [@cw-1979] to committing AEAD in 2025 [@rfc-9771], names every failure by root cause, and ends with a decision tree and a paste-in review checklist. The one rule underneath all of it: authenticate the ciphertext, the length, and the context -- not just the message.

1. A Forgery Without a Broken Cipher

In 2009, two researchers forged valid Flickr API signatures without knowing Flickr's secret key, without finding a single MD5 collision, and without touching a cipher [@flickr-advisory]. Flickr had done what feels obviously correct -- prepend a secret to each request and hash the result -- and that one instinct silently handed signing power to anyone who had ever seen a single signed URL. The hash worked exactly as designed. The way Flickr used it was the vulnerability.

That gap -- between a primitive that is sound and a deployment that is not -- is the subject of this article, and it is worth stating as a claim you can test against every page that follows.

Key idea: A message authentication code makes forgery hard; keeping it hard is the surrounding protocol's job. Every deployed MAC ships a proof, and every proof quietly assumes three things the deployment must supply: (1) the right key discipline for its family, (2) a constant-time verify, and (3) composition over the right bytes, in the right order, binding the right context. Read that way, the entire failure catalogue is one sentence repeated -- the math never broke; a deployment broke one of these three assumptions.

Before we can prove that thesis, we need the object it is about. Informally, a MAC is a small tag you attach to a message so the receiver can tell the message came from someone who holds the shared secret key, and that nobody altered it in transit.

A MAC is a keyed algorithm that takes a secret key and a message and produces a short fixed-size tag. Anyone holding the same key can recompute the tag to verify that the message is authentic and unmodified. Unlike a plain hash, a MAC is keyed, so an attacker without the key cannot produce a valid tag; unlike a digital signature, the same secret both makes and checks the tag, so there is no public verifiability and no non-repudiation.

Now pair Flickr with a second break that fails a different assumption. In 2013, Lucky Thirteen recovered plaintext from TLS connections that used a perfectly correct HMAC -- the attack timed how the MAC was checked, not whether the MAC was strong [@cve-2013-0169]. Two production systems, two respected primitives, two failures -- and in neither case did the cryptography break. Flickr fed the hash the wrong bytes in the wrong construction; TLS checked a correct tag on a timing-leaky path.

Note: MD5 is badly broken for collision resistance, yet the Flickr forgery did not use a collision. HMAC was and is unbroken, yet Lucky Thirteen still bled plaintext. In both cases the primitive met its specification and the deployment violated an assumption the specification silently depended on. Hold onto that pattern -- it is the whole catalogue in miniature.

By the end of this field guide you will have a decision procedure and a review checklist that catch every failure in this article before it ships.The original Flickr advisory by Duong and Rizzo is no longer served from its source host; the citation here points to a 2009 Wayback Machine snapshot, which is the durable primary record [@flickr-advisory]. But a checklist without the underlying model is just cargo-culting. So start with the model. If the hash was not broken, what exactly is a MAC supposed to promise, and against whom?

2. What a MAC Actually Is

Strip away the mystique and a MAC is three algorithms and one game.

The three algorithms are KeyGen, Tag, and Verify. KeyGen samples a secret key $k$. Tag maps a key and a message $m$ to a tag $t$. Verify takes $k$, $m$, and a candidate $t$ and returns accept or reject. The whole design question is: how hard is it for someone without $k$ to make Verify say accept on a message the key-holder never authorized?

"How hard" was made precise in 1988. Shafi Goldwasser, Silvio Micali, and Ronald Rivest gave the first rigorous adversarial definition of authentication -- existential unforgeability under adaptive chosen-message attack -- while analyzing digital signatures, and MACs inherit it directly [@gmr-1988]. The game is deliberately generous to the attacker.

The adversary may ask for tags on any messages it likes, adaptively, as many as it wants. It wins if it then outputs a valid tag on any *new* message it never queried. A MAC is EUF-CMA secure if every efficient adversary wins only with negligible probability. The name is exact: "existential" means the forgery can be on any message the attacker chooses, and "adaptive chosen-message" means it sees each answer before picking its next query.

EUF-CMA is the usual bar, but composition needs a slightly stronger one, and the difference will decide whether a whole protocol stands or falls in Section 6.

SUF-CMA forbids even a *new tag on an old message*. The adversary wins if it outputs any message-tag pair that verifies and was not itself the result of a tag query -- including a second, different valid tag on a message it already asked about. Every SUF-CMA MAC is EUF-CMA, but not conversely: a MAC with malleable tags can be existentially unforgeable yet still let an attacker maul a valid tag into another valid tag.

Formally, if we write the tag oracle as $\mathsf{Tag}_k(\cdot)$ and let $Q$ be the set of message-tag pairs it returned, the strong-forgery advantage is

$$\mathsf{Adv}^{\text{suf-cma}}_{\mathsf{MAC}}(\mathcal{A}) ;=; \Pr!\left[,(m,t) \leftarrow \mathcal{A}^{\mathsf{Tag}_k(\cdot)} : \mathsf{Verify}_k(m,t)=\text{accept} ;\wedge; (m,t)\notin Q,\right].$$

A MAC is good when this probability stays negligible for every efficient $\mathcal{A}$. That single inequality is the promise Flickr's construction could not keep -- an attacker produced an accepted tag on a message that was never in $Q$.

Four primitives live near this definition, and confusing them is the source of real bugs. An unkeyed hash has no $k$, so anyone can recompute it -- it detects accidental corruption, never a motivated forger; this series covers hashes and their length-extension quirk in Part 10.

A MAC is symmetric: the same secret makes and checks the tag. A digital signature is asymmetric -- a private key signs, a public key verifies -- which buys public verifiability and non-repudiation that a MAC deliberately does not provide (Part 3 develops what the authenticated bytes must actually cover). An AEAD bundles a MAC together with encryption so one call delivers secrecy and integrity at once; Part 7 is the decision matrix for choosing among AEADs, and this article is the authentication half underneath it.

Note: Because a MAC key both signs and verifies, a valid tag proves only that someone holding the shared key produced it -- it cannot tell you which party, and it cannot be shown to a third party as proof. If you need "Alice, and only Alice, provably sent this," you need a signature, not a MAC. Reaching for a MAC where you needed non-repudiation is a design error no amount of key length fixes.

The definitions say a good MAC is unforgeable. Yet the very first keyed MACs people built were forgeable in ways the definition had not yet been written to outlaw. Stranger still, the airtight, provably unforgeable answer arrived before the practical, broken ones. Where did the idea come from?

3. Origins: Universal Hashing and One-Time Authentication

Here is the twist that reorders the whole story. The provably secure MAC was invented first, in 1979 and 1981, and then the field spent two decades shipping forgeable ones anyway. Keep that inversion in mind as the arc unfolds.

timeline title Message authentication, 1979 to 2025 1979 : Carter-Wegman universal hashing 1981 : One-time authentication, tag is hash plus one-time pad 1985 : CBC-MAC standardized in FIPS 113 1996 : HMAC nests the hash to stop length extension 2000 : Encrypt-then-MAC formalized by Bellare and Namprempre 2001 : Krawczyk indicts the SSL and SSH orderings 2004 : GMAC revives Wegman-Carter at line speed 2005 : Poly1305 brings it to fast software 2006 : HMAC proven a PRF without collision resistance 2009 : Flickr forgery and the SSH plaintext recovery 2013 : Lucky Thirteen timing oracle 2016 : KMAC keyed sponge and 184 nonce-repeating servers 2018 : TLS 1.3 record layer becomes AEAD-only 2023 : Terrapin breaks a correct channel 2025 : RFC 9771 standardizes commitment vocabulary

The starting point is a purely combinatorial object. Larry Carter and Mark Wegman, at IBM Research, asked what happens if you pick a hash function at random from a carefully built family, rather than fixing one function everyone knows [@cw-1979].

A universal family of hash functions has the property that, for any two fixed distinct inputs, a function drawn uniformly at random from the family maps them to the same output with only small, bounded probability. The bound comes from counting the functions in the family, not from any assumption about how much computing power an attacker has. It is combinatorial security, not cryptographic security -- which is exactly why it needs no unproven hardness conjecture.

Two years later, Wegman and Carter turned that object into an authenticator [@wc-1981]. The construction is startlingly small: draw a secret hash $h$ from a strongly universal family, draw a secret one-time pad $s$, and send

$$t ;=; h(m) ,\oplus, s.$$

The pad $s$ hides the value $h(m)$ completely, like a one-time pad hides a plaintext, and the universal property bounds an attacker's chance of guessing a valid tag for a different message. The result is unconditional: it holds against an adversary with unlimited computing power, assuming nothing about factoring, discrete logs, or the strength of any cipher.

Authenticate a message as the exclusive-or of a universal hash of the message and a fresh secret pad: the tag reveals nothing about the hash key, and forging a tag on a new message requires guessing a value the universal bound keeps small. The security is information-theoretic and tight -- but the pad must be used exactly once. Authenticate a second message under the same pad and the guarantee evaporates.

Take the smallest instance. Fix a prime $p$, keep a secret pair $(a, b)$, and tag a message $m$ as $t = (a\cdot m + b) \bmod p$. An attacker who sees one pair $(m_1, t_1)$ learns one linear equation in two unknowns; every candidate slope $a$ has exactly one consistent $b$, so the tag on any different $m_2$ stays uniform over all $p$ values. The best forgery succeeds with probability $1/p$. Beautiful, and useless twice: a second pair $(m_2, t_2)$ gives a second equation, the line is pinned, and every future tag is computable.

Note: "One-time" is not a restriction bolted onto Wegman-Carter authentication -- it is the security proof. The pad's single use is what buys the information-theoretic guarantee. That makes it a feature in 1981 and a landmine in 2004, when the same construction is revived at network line speed and a fast deployment reuses the pad. Every GMAC and Poly1305 nonce rule you will ever read is this 1981 constraint wearing modern clothes.

So the airtight answer existed. Yet it authenticates only one message per key, which is impractical for a general-purpose MAC.The two ideas are separate papers: the 1979 work introduced universal classes of hash functions as a combinatorial tool [@cw-1979]; the 1981 follow-up applied them to authentication and set equality [@wc-1981]. Universal hashing came first; authentication was the application. If the unconditional construction was this good but this restrictive, what did engineers reach for instead -- and why did the "obvious" keyed hashes they built break?

4. Early Approaches and Why They Broke

When engineers needed a keyed MAC in the 1990s, most reached for the hash they already had and glued a secret to it. The most natural glue is to put the key in front: compute H(secret || message) and ship the digest as the tag. It looks unforgeable -- you cannot compute the hash without the secret. It is also exactly what Flickr shipped, and it is broken by a property of the hash that has nothing to do with its collision resistance.

The property is length extension, and it is a direct consequence of how Merkle-Damgard hashes -- MD5, SHA-1, and most of SHA-2 -- are built [@length-ext-wiki]. Such a hash absorbs the message block by block into an internal state, and it emits that entire internal state as the digest. The digest is therefore a complete, resumable snapshot: given H(secret || message), you can set a fresh hash's state to that value and keep absorbing more bytes, as if you had known the secret all along.

Given the digest of `secret || message` and the length of the secret, an attacker can compute the digest of `secret || message || padding || extra` for attacker-chosen `extra`, without knowing the secret. The attack works because a Merkle-Damgard digest is the hash's full internal state, so it can be resumed. The only wrinkle is the glue `padding` the original hash appended internally, whose bytes are fully determined by the known lengths.

The forgery is mechanical once you see the state as resumable.

sequenceDiagram participant A as Attacker participant V as Verifier A->>V: Observe one signed request, message m and tag t Note over A: The tag t is the full hash state after absorbing key and m A->>A: Resume from t, append glue padding then chosen extra bytes A->>A: Keep hashing to get t2 for the extended message, no key used A->>V: Submit extended message with tag t2 Note over V: Verifier recomputes over key plus the extended message V-->>A: State matches, forged request accepted

The demonstration below builds a toy Merkle-Damgard hash whose digest is its internal state -- the essential property -- and forges a secret-prefix tag with no secret, then shows that nesting the hash (the HMAC idea) shuts the door.

Toy Merkle-Damgard hash: absorb one byte at a time; the digest IS the state. (The mixer is illustrative, not secure -- the STRUCTURE is the point.)

def compress(state, byte): return ((state * 1000003) ^ byte) & 0xFFFFFFFFFFFF # 48-bit toy state

def toy_hash(data, iv=0xA5A5A5A5A5A5): state = iv for byte in data: state = compress(state, byte) return state # digest == resumable internal state

secret = b"opensesame" # attacker does NOT know this... def prefix_mac(msg): # ...but does know len(secret) == 10 return toy_hash(secret + msg)

1) Attacker observes ONE legitimate (message, tag) pair:

seen = b"amount=10" tag = prefix_mac(seen)

2) Attacker resumes from the tag and appends -- the secret is never touched:

extra = b"&amount=1000000" forged = tag for byte in extra: forged = compress(forged, byte)

3) The verifier recomputes over secret + (seen + extra):

print("secret-prefix forgery accepted:", forged == prefix_mac(seen + extra))

The HMAC-style fix: nest, so the outer pass re-absorbs the inner digest.

def nested_mac(msg): inner = toy_hash(secret + msg) return toy_hash(secret + inner.to_bytes(6, "big"))

resumed = nested_mac(seen) for byte in extra: resumed = compress(resumed, byte) print("nested (HMAC-style) forgery accepted:", resumed == nested_mac(seen + extra)) `}

The first line prints True, the second False. That single flip is the difference between the Flickr construction and HMAC, and it is Aha number one for this article.

Note: H(secret || message) over any Merkle-Damgard hash is not a MAC. It authenticates nothing an attacker who has seen one tag cannot forge, regardless of how strong the underlying hash is. MD5's collision weakness is irrelevant here -- the Flickr forgery used no collision at all [@flickr-advisory]. If you find md5(secret + data) or sha256(secret + data) in a code review, you have found a length-extension forgery waiting to happen.

The mirror construction, secret-suffix H(message || secret), resists length extension because the secret is absorbed last.Secret-suffix is not safe either: since an attacker who finds two messages colliding under the raw hash gets a tag collision for free, H(m || k) reduces to the hash's collision resistance, so it dies with MD5 and SHA-1 -- the opposite failure mode from secret-prefix, same verdict [@length-ext-wiki]. Neither ad-hoc glue is a MAC.

The block-cipher world made a parallel mistake. CBC-MAC, standardized as FIPS 113 in 1985, runs a block cipher in CBC mode with a zero IV and keeps the final ciphertext block as the tag [@fips-113]. It carries an unstated assumption: every message is the same fixed length. Bellare, Kilian, and Rogaway turned the failure of that assumption into a theorem in 1994 -- secure for fixed-length messages, trivially forgeable once lengths vary [@bkr-1994].

The splice is mechanical. Query the tag of a one-block message $m_1$ to get $t_1 = E_k(m_1)$; then the two-block message $m_1 ,\Vert, (m_1 \oplus t_1)$ has tag $E_k\big((m_1 \oplus t_1)\oplus t_1\big) = E_k(m_1) = t_1$ -- a valid tag on a message never queried, produced with no key.

Both failures teach the same lesson, and it organizes the rest of this guide: a raw hash is not a MAC, and raw CBC-MAC is not a MAC for variable-length messages. Both fixes point the same direction -- stop exposing resumable internal state, and make the tag depend on length. From here the field split into two lineages. Which one wins?

5. The Evolution: Two Lineages Diverge

The field did not pick one winner. It split into two families that both survive today, and choosing between them is still the single most consequential decision a practitioner makes about authentication. One family keeps a computational assumption and buys a reusable key; the other keeps the 1981 one-time pad and buys blazing speed.

The PRF lineage: HMAC and CMAC

Bellare, Canetti, and Krawczyk fixed the secret-prefix disaster in 1996 by nesting the hash so the attacker never sees a resumable inner state [@bck-1996]. Standardized a year later as RFC 2104, HMAC computes two keyed passes [@rfc-2104]:

$$\mathsf{HMAC}(k, m) ;=; H\big((k \oplus \text{opad}) ,\Vert, H((k \oplus \text{ipad}) ,\Vert, m)\big).$$

The outer hash re-absorbs the inner digest, so a resumed inner state is useless -- length extension cannot reach the output.

RFC 2104 also fixes how the key becomes block-sized before those two XORs: a key longer than the hash's block size is first hashed down to a digest, and a shorter key is zero-padded up to the block size [@rfc-2104]. That is why HMAC accepts a key of any length -- it normalizes every key to exactly one block before mixing in `ipad` and `opad`. flowchart TD K[Secret key k] --> IP[k XOR ipad] M[Message m] --> INNER[Inner hash of ipad key then message] IP --> INNER INNER --> ID[Inner digest, never emitted directly] K --> OP[k XOR opad] ID --> OUTER[Outer hash of opad key then inner digest] OP --> OUTER OUTER --> T[HMAC tag]

The deepest fact about HMAC is also the most misunderstood, and it took another decade to prove. In 2006 Bellare showed HMAC is a pseudorandom function under the sole assumption that the hash's compression function is a PRF -- collision resistance is not required at all [@bellare-2006].

A keyed function is a PRF if no efficient adversary, allowed to query it on inputs of its choice, can distinguish its outputs from those of a truly random function. A PRF makes an immediate MAC: to forge a tag on a fresh message is to predict a value that, by assumption, looks uniformly random, which succeeds only with negligible probability. Unforgeability falls out of pseudorandomness.

This is why the collision breaks of MD5 and SHA-1 never touched HMAC-MD5 or HMAC-SHA-1. Collisions attack collision resistance; HMAC never needed it.

Note: The folklore "HMAC needs a collision-resistant hash" is false, and it has been false since 2006 [@bellare-2006]. HMAC needs the compression function to behave like a PRF -- a different, and for MD5 and SHA-1 still-standing, assumption. This is not a licence to use MD5 in new systems, but it is why the practical world did not catch fire when SHA-1 collisions arrived.

The block-cipher branch got the same repair. CMAC, from Iwata and Kurosawa's OMAC and standardized as NIST SP 800-38B, derives two subkeys from the block cipher and mixes one into the final block, killing the variable-length splice that sank raw CBC-MAC while keeping a single key [@omac-2003] [@sp-800-38b]. It is the block-cipher PRF-MAC, useful where a device already has AES hardware but no hash.

The survival of HMAC-SHA-1 is the cleanest real-world confirmation of the 2006 proof: SHA-1's collision resistance collapsed publicly, yet no HMAC-SHA-1 forgery followed, because the security reduction never invoked collision resistance in the first place [@bellare-2006].

The sponge refinement: KMAC

SHA-3 changed the substrate. A Keccak sponge splits its state into a public rate and a secret capacity, and it never emits the capacity [@fips-202]. That single property makes the HMAC nesting trick unnecessary: you can key a sponge by simply absorbing the key first.

A sponge function absorbs input into a large internal state divided into a public "rate" (the bytes that interact with input and output) and a secret "capacity" (bytes never revealed). Because the capacity is never output, prepending a key and absorbing it yields a keyed authenticator with no length-extension surface and no double hashing -- the key is protected by the same hidden capacity that protects the hash.

NIST SP 800-185 builds KMAC on cSHAKE this way: prepend the key, absorb, squeeze [@sp-800-185]. Keying is native, domain separation is built in through customization strings, and the tag length is a free parameter because the sponge is an extendable-output function. It is, conceptually, the cleanest MAC in the standard catalogue -- but conceptual cleanliness is not deployment.

KMAC is a real, standardized primitive [@sp-800-185], implemented in OpenSSL 3.0 and later (as the `KMAC-128`/`KMAC-256` EVP_MAC) [@openssl-kmac], as well as BouncyCastle [@bouncycastle-kmac] and Botan [@botan-kmac]. It is not academic-only. But its production footprint is thin next to HMAC, GMAC, and Poly1305, for reasons that are historical inertia rather than cryptography: SHA-2 and HMAC are welded into TLS, IPsec, JWT, HKDF, and hardware everywhere, and SHA-3 adoption trails. This guide gives KMAC full conceptual treatment and makes no claim about its market share -- no reliable quantitative figure exists in a primary source.

Both HMAC and KMAC share a decisive operational property: a reusable key, no nonce, an unbounded stream of messages under one secret. That is the easy side of assumption one. The other lineage trades it away for speed.

The Wegman-Carter lineage: GMAC and Poly1305

In 2004 and 2005, two designs dragged the 1981 one-time construction into high-speed networking. Both keep the pattern tag = universal_hash(m) + one_time_pad, and both solve the "fresh pad per message" problem the same way -- derive the pad from a cipher keystream instead of shipping new randomness.

GHASH, the core of GCM, is a polynomial universal hash over the finite field $\mathrm{GF}(2^{128})$: it evaluates $\sum_i m_i \cdot H^{,L-i+1}$ at a secret point $H = E_k(0)$, then masks the result with an AES-encrypted counter block [@mcgrew-viega-2004]. Run GCM with an empty plaintext and you get GMAC, an authentication-only MAC over associated data, standardized in NIST SP 800-38D [@sp-800-38d]. Poly1305 evaluates a message polynomial modulo the prime $2^{130}-5$ under a secret $r$, then adds a one-time pad $s$; deployed as ChaCha20-Poly1305, it draws a fresh $(r, s)$ from the cipher keystream for every message [@rfc-8439].

flowchart TD N[Nonce and key] --> KS[Cipher keystream block] KS --> R[One-time hash key r] KS --> S[One-time pad s] M[Message m] --> P[Polynomial hash of m under r] R --> P P --> ADD[Add the one-time pad s] S --> ADD ADD --> T[Authentication tag] T -.reuse the same nonce.-> LEAK[Two tags cancel the pad, leaking hash of m1 minus hash of m2]

This is where the 1981 constraint re-enters as a deployment contract. The pad is safe only if the $(key, nonce)$ pair never repeats. Reuse it and the pad cancels out of the difference of two tags, exactly as the toy line did in Section 3 -- except now the "hash" is a polynomial an attacker can then start solving.

A one-time Wegman-Carter MAC: tag = poly_hash(m, r) + s (mod p). Use (r, s) ONCE.

p = (1 << 130) - 5 # the Poly1305 prime, 2**130 - 5

def poly_hash(msg, r): acc = 0 for i in range(0, len(msg), 16): chunk = msg[i:i+16] limb = int.from_bytes(chunk, "little") + (1 << (8 * len(chunk))) acc = ((acc + limb) * r) % p return acc

In ChaCha20-Poly1305 both r and s come fresh from the keystream, per message.

r = 0x1bf54941aff6bf4afdb1f2a3c4d5e6f7 s = 0x0102030405060708090a0b0c0d0e0f10

def one_time_tag(msg): return (poly_hash(msg, r) + s) % p

THE MISTAKE: two messages under the SAME (r, s).

m1, m2 = b"transfer 10 dollars now", b"transfer 9999 bucks now" t1, t2 = one_time_tag(m1), one_time_tag(m2)

The pad s cancels in the difference -- a relation that does not depend on s at all:

lhs = (t1 - t2) % p rhs = (poly_hash(m1, r) - poly_hash(m2, r)) % p print("pad cancelled, tag difference is key-independent:", lhs == rhs) `}

It prints True: subtract two tags produced under the same one-time key and the secret pad vanishes, handing the attacker a clean algebraic relation in the hash key. That is the first step of the "forbidden attack" the failure catalogue will name -- and it is why the distinction below is not pedantry.

The master key versus one-time key split is the heart of the fork. HMAC and KMAC take a master key and authenticate forever under it. GMAC and Poly1305 take a one-time key per message; the ChaCha20-Poly1305 keystream manufactures that one-time key so the master key stays put, but the per-message pad is still strictly single-use. Confuse the two and you get a catastrophe, not a slow degradation.

The whole construction catalogue, side by side:

Table 1 -- MAC constructions, and what each one assumes.

Construction	Year	Core idea	Key model	Strength	Weakness	Status
Carter-Wegman one-time [@wc-1981]	1981	$\mathsf{UH}(m) \oplus \text{pad}$	One-time pad per message	Information-theoretic, no hardness assumption	Pad is strictly single-use	Foundational, revived
Secret-prefix $H(k \Vert m)$ [@length-ext-wiki]	1990s	Prepend key, hash	Reusable	Trivial to build	Length-extension forgeable	Superseded, insecure
Secret-suffix $H(m \Vert k)$ [@length-ext-wiki]	1990s	Append key, hash	Reusable	No length extension	Reduces to a hash collision	Superseded, insecure
CBC-MAC [@fips-113]	1985	CBC chain, keep last block	Reusable	First standardized block-cipher MAC	Variable-length splicing forgery	Historical
HMAC [@rfc-2104]	1996	Nested keyed hash	Reusable, no nonce	PRF not collision resistance, length-ext immune	Serial, no line-rate shortcut	Active, default PRF-MAC
CMAC [@sp-800-38b]	2005	CBC-MAC plus final subkey	Reusable, no nonce	Fixes CBC-MAC with one key	Block-cipher birthday ceiling	Active, niche
GMAC / GHASH [@sp-800-38d]	2004	Polynomial hash over $\mathrm{GF}(2^{128})$	One-time pad per message	Carry-less-multiply hardware (PCLMULQDQ/PMULL)	Nonce reuse is fatal	Active, inside GCM
Poly1305 [@rfc-8439]	2005	Polynomial hash mod $2^{130}-5$	One-time key per message	Fast in pure software, cipher-agnostic	One-time key per message	Active, inside ChaCha20-Poly1305
KMAC [@sp-800-185]	2016	Keyed sponge, cSHAKE	Reusable, no nonce	Native keying, domain separation, XOF tag	Lightly deployed	Active, niche

You can now build an unforgeable tag two ways: a reusable-key PRF or a one-time-key universal hash. But a perfect tag attached in the wrong place, in the wrong order, or over the wrong bytes is still useless. How do you combine a MAC with a cipher without reopening the door you just closed?

6. Composition Done Right, and the Verification Contract

SSL chose one order for combining its cipher and its MAC. SSH chose another. Both were quietly indicted by a theorem years before either was broken in the field. This section is where the second aha moment lands: strength was never the variable -- discipline and order were.

Bellare and Namprempre formalized the question in 2000 by splitting integrity into two notions [@bn-2000]. Integrity of plaintexts (INT-PTXT) says an attacker cannot make the receiver accept a plaintext the sender never sent. Integrity of ciphertexts (INT-CTXT) is stronger: the attacker cannot even produce a new ciphertext the receiver will accept. Their result is the load-bearing one: applying the MAC to the ciphertext -- Encrypt-then-MAC -- yields both IND-CCA confidentiality and INT-CTXT, and it is the only one of the three generic orders that reaches those strong notions.

Encrypt the plaintext, then compute the MAC over the resulting *ciphertext* (plus any associated data and length), and send ciphertext and tag together. On receipt, verify the tag *before* touching the ciphertext; if it fails, discard without decrypting. EtM is the only generic composition order that provably delivers integrity of ciphertexts, and it does so only under two preconditions: the MAC must be strongly unforgeable and the encryption and MAC keys must be independent.

Those two preconditions are not decoration. If the MAC is only EUF-CMA rather than SUF-CMA, the guarantee weakens from INT-CTXT down to INT-PTXT, because a malleable tag lets an attacker produce a different valid ciphertext for the same plaintext [@bn-2000]. And the keys must be independent -- reusing one key for both encryption and authentication voids the proof.

A year later Krawczyk sharpened the verdict [@krawczyk-2001]. He proved that any secure-channel protocol required to work with any secure cipher and any secure MAC must use Encrypt-then-MAC, and he built a perfectly secret cipher that becomes totally insecure under MAC-then-Encrypt -- SSL's order -- and under Encrypt-and-MAC -- SSH's order.

"[A]ny secure channels protocol designed to work with any combination of secure encryption ... and secure MAC must use the encrypt-then-authenticate method." -- Hugo Krawczyk, The Order of Encryption and Authentication for Protecting Communications, 2001 [@krawczyk-2001]

Here is the precision that separates an expert from a slogan-repeater: Krawczyk also proved that MAC-then-Encrypt is secure for specific cipher modes, namely CBC and CTR [@krawczyk-2001]. MtE is therefore fragile and not generically secure -- it holds only when you can prove something extra about the exact cipher -- but it was never unconditionally "broken." Say "MtE is broken" in an interview and you are wrong; say "MtE is not generically secure, so do not use it in new designs" and you are exactly right.

flowchart TD subgraph EtM["Encrypt-then-MAC, generically secure"] E1[Plaintext] --> E2[Encrypt] E2 --> E3[Ciphertext] --> E4[MAC the ciphertext] E4 --> E5[Receiver verifies tag first, then decrypts] end subgraph MtE["MAC-then-Encrypt, fragile"] M1[Plaintext] --> M2[MAC the plaintext] M2 --> M3[Encrypt plaintext and tag together] M3 --> M4[Receiver decrypts first, only then verifies] end subgraph EM["Encrypt-and-MAC, fragile"] A1[Plaintext] --> A2[Encrypt] A1 --> A3[MAC the plaintext, tag sent in the clear] A2 --> A4[Receiver decrypts, and the tag leaks plaintext] end

The diagram shows the mechanism behind every padding-oracle and SSH break at once: in the two fragile orders, the receiver must decrypt or parse before it can verify, so the very act of processing attacker-chosen bytes leaks information. Encrypt-then-MAC verifies first and refuses to decrypt a single byte of an unauthenticated ciphertext.

Key idea: Authenticate the ciphertext, not the plaintext. Encrypt-then-MAC over the ciphertext -- with an independent key, a strongly unforgeable tag, and coverage of the associated data and length -- is the only generic order that lets the receiver reject a forgery before it decrypts anything. Every composition break in the catalogue is a system that decrypted or parsed before it verified.

Table 2 -- Composition orders, ranked by what they generically guarantee.

Order	Idea	Generic security	Weakness	Status
MAC-then-Encrypt [@krawczyk-2001]	MAC the plaintext, then encrypt both	Not generic, secure only for CBC or CTR	Decrypt-before-verify padding and timing oracles	Superseded, fragile
Encrypt-and-MAC [@apw-2009]	Encrypt, and MAC the plaintext separately	Not generic	Tag over plaintext leaks, MAC-covered length field abused	Superseded, fragile
Encrypt-then-MAC [@bn-2000]	MAC the ciphertext	IND-CCA plus INT-CTXT	Needs SUF-CMA, independent keys, constant-time verify	Active, canonical
AEAD [@rfc-8446]	EtM internalized plus nonce management	IND-CCA plus INT-CTXT by construction	Key-commitment gap	Active, default
Committing AEAD [@rfc-9771]	AEAD plus key and context commitment	Adds commitment	Still standardizing, cost profiling	Frontier

The order is only half of correct composition. The other half arrived not from a theorem but from a field disclosure. In 2009 Nate Lawson found that Google's Keyczar library compared HMAC tags byte by byte and returned as soon as two bytes differed [@keyczar-2009]. That early exit turns verification time into a side channel: a forged tag that matches more leading bytes takes measurably longer to reject, so an attacker can recover the correct tag one byte at a time by timing responses.

Q0 -- Yes --> ASCON[Ascon, SP 800-232]
Q0 -- No --> Q1&#123;"Is there a ciphertext to authenticate?"&#125;
Q1 -- No --> HMAC[HMAC-SHA-256, reusable key, no nonce]
Q1 -- Yes --> Q2&#123;"Can you guarantee a unique nonce per key?"&#125;
Q2 -- No --> SIV[AES-GCM-SIV, misuse-resistant]
Q2 -- Yes --> Q3&#123;"Is AES-NI hardware available?"&#125;
Q3 -- Yes --> GCM[AES-GCM, rekey before the per-key ceiling]
Q3 -- No --> CHACHA[ChaCha20-Poly1305]

Read the tree as the three assumptions in order. If there is no ciphertext to bind -- key derivation, a token, a one-time password -- use HMAC-SHA-256 with a key of at least 128 bits, a full-length tag, and a constant-time verify; there is no nonce to mismanage, so assumption one is nearly free. This is the right tool for HKDF [@rfc-5869], JWT's HS256 [@rfc-7518], TOTP, HOTP, and PBKDF2.

If there is a ciphertext, do not hand-roll a MAC plus cipher -- reach for an AEAD, which internalizes Encrypt-then-MAC so you cannot mis-order it. Pick ChaCha20-Poly1305 when you lack AES hardware acceleration, especially on mobile and embedded targets, where its per-key budget is effectively unbounded [@rfc-8439]. Pick AES-GCM when you have AES-NI, and rekey before the $2^{23}$-packet ceiling on high-volume links [@rfc-9001]. If you cannot guarantee nonce uniqueness at scale, step down to AES-GCM-SIV, which degrades gracefully instead of catastrophically on reuse [@rfc-8452].

Two niche branches sit off the main tree. Use GMAC for authentication-only data, such as MACsec framing, with a guaranteed-unique nonce [@sp-800-38d] [@ieee-802-1ae]. Use KMAC when you are already in a Keccak or SHA-3 stack, or when you need domain separation or variable-length tags [@sp-800-185]. And on constrained or side-channel-sensitive hardware, use Ascon [@sp-800-232].

Then there are the rules that apply always, regardless of which box the tree lands in: Encrypt-then-MAC or an AEAD that internalizes it; independent encryption and MAC keys; a constant-time compare; a tag of at least 128 bits with documented truncation floors [@rfc-4868]; authentication that covers the associated data, the length, the version, and the session context; a committing AEAD when one ciphertext might be opened under multiple keys [@bellare-hoang-2022]; and a misuse-resistant AEAD when nonce uniqueness cannot be guaranteed [@rfc-8452].

Key idea: Review authentication code against the checklist and its negation. The checklist tells you what a correct design does; the misuse list tells you what every named CVE did instead. Run any authentication path past both -- if it matches even one line of the misuse list, you have found your next vulnerability before an attacker does.

The negation is the misuse catalogue -- each row is a line of the checklist inverted, and it maps one-to-one onto a named break from Section 7.

Table 6 -- Common misuse, the break it produced, and the fix.

Misuse in real code	Named break it caused	Fix
Secret-prefix $H(k \Vert m)$ MAC	Flickr length extension [@flickr-advisory]	Use HMAC or KMAC
Reusing a GCM nonce or Poly1305 one-time key	Joux, Nonce-Disrespecting [@woot16-bock]	Guarantee uniqueness or use AES-GCM-SIV
`==` or `memcmp` tag comparison	Keyczar timing leak [@keyczar-2009]	Use a constant-time compare
MAC-then-Encrypt or Encrypt-and-MAC in new code	Lucky Thirteen, POODLE, SSH recovery [@cve-2013-0169]	Use an AEAD (Encrypt-then-MAC)
Over-truncated tags	Forgery near the $2^{-t}$ floor [@rfc-4868]	Keep the tag at least 128 bits
One shared key for encryption and MAC	Voids the Encrypt-then-MAC proof [@bn-2000]	Derive independent keys
MACing the plaintext instead of the ciphertext	Padding oracles [@vaudenay-2002]	MAC the ciphertext
Forgetting AD, length, version, context	Terrapin [@terrapin-site]	Bind the full context
Treating "MAC verified" as "key committed"	Invisible salamanders [@bellare-hoang-2022]	Use a committing AEAD

Note: Run every authentication path past these ten lines: - Is the tag over the ciphertext (Encrypt-then-MAC), never the plaintext? - Are the encryption and MAC keys independent? - Is the MAC strongly unforgeable (SUF-CMA), not merely EUF-CMA? - Is the tag comparison constant-time (compare_digest, timingSafeEqual, ConstantTimeCompare)? - Is the whole decrypt-and-verify path constant-time, not just the compare? - Is the tag at least 128 bits, with any truncation documented against the birthday bound? - For a one-time-key MAC (GMAC, Poly1305), is nonce uniqueness guaranteed, or are you on a misuse-resistant mode? - Does the authenticated data cover length, version, ordering, and session context? - Is there any H(secret || message) secret-prefix construction? (Replace with HMAC or KMAC.) - Could one ciphertext open under two keys? If so, use a committing AEAD. Any single failure here is a plausible next CVE.

That checklist and its negation are the working payoff of the whole field guide. Before the conclusion ties them back to the thesis, seven specific beliefs produce most of the remaining real-world authentication bugs -- and each is a corollary of the same one sentence.

13. Frequently Corrected Misconceptions

No. As Section 5 established, Bellare's 2006 proof rests HMAC's security on a single property of the underlying compression function -- that it is itself a PRF -- and not on collision resistance [@bellare-2006]. Since a collision attack targets a property the proof never relies on, HMAC-SHA-1 and HMAC-MD5 kept standing when the SHA-1 and MD5 collision breaks landed. Use HMAC-SHA-256 in new systems for margin, but understand that the reason is prudence, not a collision threat. No, and the difference is the most dangerous one to miss. HMAC takes a reusable key with no nonce. GMAC and Poly1305 are Wegman-Carter one-time-key MACs [@wc-1981]: they need a fresh, unique per-message key, and reusing it is catastrophic, not a gentle degradation. The 184 HTTPS servers caught repeating GCM nonces were forgeable precisely because a one-time construction was treated like a reusable one [@woot16-bock]. Same tag length, completely different key contract. No. HMAC exists to defeat length extension on Merkle-Damgard hashes, but a Keccak sponge never emits its capacity, so it has no length-extension surface and keys natively -- just use KMAC [@sp-800-185]. In fact you cannot build HMAC on a SHAKE extendable-output function anyway; Python's `hmac` module notes that XOFs such as SHAKE-128 and SHAKE-256 cannot be used with HMAC [@python-hmac]. KMAC is the SHA-3-native answer. No -- it is fragile, which is a different and more precise claim. Krawczyk proved MAC-then-Encrypt is *not generically secure*, yet also proved it *is* secure for CBC and CTR modes specifically [@krawczyk-2001]. So it was never unconditionally broken; it is simply deprecated for new designs, where a modern AEAD or Encrypt-then-MAC removes the cipher-specific precondition it depends on. Reach for one of those instead, but do not call MtE "broken." Not by itself, and the overstatement matters. A single $(key, nonce)$ reuse immediately leaks a tag relation and the XOR of the two plaintexts [@joux-2006]. Recovering the subkey $H$ outright is a taller order: a single reuse leaves a root-finding equation with several possible solutions for $H$, so narrowing them to the one true value takes at least a second collision, and usually more. So "one reuse recovers the key" is wrong; "one reuse is already a break, and more reuse recovers the key" is right. No -- there is a real-world counterexample. In 2009 Google's Keyczar library did a byte-wise tag comparison with an early exit, and that timing difference let an attacker recover valid tags one byte at a time [@keyczar-2009]. The comparison time leaked the secret the MAC existed to protect. Constant-time compare is a correctness requirement, not an optimization, and every major platform ships a named primitive for it. No. AEAD handles the encrypted channel, but plenty of jobs have no ciphertext to bind and still need authentication. HKDF key derivation [@rfc-5869], JWT's `HS256` [@rfc-7518], TOTP and HOTP one-time passwords, and PBKDF2 are all standalone HMAC. The standalone MAC did not disappear with AEAD; it moved to the places where you are authenticating keys, tokens, or messages rather than protecting a ciphertext.

Every myth here is a corollary of one sentence, which the conclusion now restates with the whole catalogue behind it.

14. The Math Never Broke

Return to Flickr, now fully understood. Two researchers forged valid API signatures with no key, no collision, and no cipher work, because Flickr's secret-prefix H(secret || params) construction handed out its resumable hash state as the tag [@flickr-advisory]. The hash met its specification exactly. The deployment broke assumption three -- the wrong construction over the wrong bytes -- and that was enough.

Line up the rest of the catalogue and the pattern is total. Lucky Thirteen and POODLE broke order and timing, decrypting before verifying [@cve-2013-0169] [@cve-2014-3566]. The SSH plaintext-recovery attack broke order, authenticating the plaintext and acting on a MAC-covered length field [@apw-2009]. The 184 nonce-repeating GCM servers broke key discipline, reusing a one-time pad [@woot16-bock]. Terrapin broke context, leaving the transcript and sequence numbers unbound even though the cipher, the MAC, and the order were all correct [@terrapin-site]. Not one of them broke AES, SHA-2, or a polynomial hash. Every single failure broke the contract around the primitive.

That is the whole thesis, proven by the evidence rather than asserted: a MAC makes forgery hard, and three deployment assumptions -- the right key discipline, a constant-time verify, and composition over the right bytes in the right order binding the right context -- are what keep it hard. The decision tree and the checklist in Section 12 are simply those three assumptions made executable. Carry them, and their negation, into every review.

The frontier moves the bar again: soon "the tag verified" should also mean "this ciphertext commits to one key and one context," the committing-AEAD work this series develops further in Part 7. And the deeper machinery lives in the siblings -- the security definitions of Part 1, the one-time-key generation of Part 2, the canonicalization discipline of Part 3, the block modes of Part 5, the padding oracles of Part 6, and the AEAD decision matrix of Part 7. This part is the authentication spine that runs through all of them.

The math never broke; a deployment broke one of three assumptions. Authenticate the ciphertext, the length, and the context -- not just the message.

The Fingerprint Two Files Shared: A Field Guide to Cryptographic Hashes

noreply@paragmali.com (Parag Mali) — Fri, 10 Jul 2026 16:04:13 GMT

A cryptographic hash makes one promise: a short fingerprint that stands in, unforgeably and unambiguously, for any input. MD5 and SHA-1 are dead for collision-dependent uses. Their broken collision resistance was weaponized into forged certificates (the 2008 rogue certificate authority [@rogue-ca], Flame in 2012 [@stevens-counter-cryptanalysis-crypto13]) and forged documents and keys (SHAttered in 2017 [@shattered], Shambles in 2020 [@shambles]). Yet HMAC-SHA-1 is not broken [@rfc-2104], and "dead" never meant "extinct." SHA-2 survived by widening the same Merkle-Damgard margins, while SHA-3's sponge and the BLAKE3 tree diversified the toolbox rather than replacing the survivor [@fips-202, @blake3-repo]. Correct usage is clause-matching: pick the function and the mode (bare hash, HMAC, KMAC or cSHAKE, an extendable-output function, or a password KDF) that keeps the exact clause your protocol leans on. This guide gives you the promise, the constructions, every named break with its root cause, and a decision tree you can apply the same afternoon.

1. The Fingerprint Two Files Shared

In February 2017, researchers at Google and CWI Amsterdam published two PDF files that looked nothing alike, with different visible content and different bytes, yet when a computer fingerprinted each of them with SHA-1 it returned the very same 40-hex-digit answer: 38762cf7f55934b34d179ae6a4c80cadccbb7f0a [@shattered]. Two different files, one identical fingerprint. Five years earlier, malware named Flame had used the same species of trick against the older MD5 hash to forge a code-signing certificate that chained up to Microsoft and rode Windows Update onto target machines as if Microsoft itself had signed it [@stevens-counter-cryptanalysis-crypto13].

Neither of those is a stunt. They are the failure mode of one of the quietest, most load-bearing primitives in computing. A cryptographic hash function takes an input of any size and returns a short, fixed-length string of bits, its digest or fingerprint, and it promises something enormous: that fingerprint stands in for the whole input unforgeably and unambiguously.

Almost everything you trust online leans on that promise. A certificate authority does not sign your certificate; it signs a hash of it. Code signing signs a hash of the program. A TLS 1.3 handshake authenticates a hash of the transcript. Every Git commit names its parent by hash, and every blockchain block commits to its transactions through a tree of hashes. When the promise holds, all of that machinery works. When it breaks, it breaks silently and all at once.

Key idea: A cryptographic hash is infrastructure hiding in plain sight. Because signatures, certificates, and integrity checks all sign the digest rather than the data, a break in the hash silently breaks every signature built on top of it. The attacker never has to touch your private key.

That is what makes the two colliding PDFs unsettling rather than merely clever. If an adversary can produce two inputs with the same fingerprint, then any signature over one of them is also a valid signature over the other, and the entire chain of trust resting on that hash quietly inverts. This guide answers four questions in order: what exactly is the promise a hash makes, how is the fingerprint built, how has it broken in the real world, and how do you choose one today that will not break under you.

flowchart TD A["One promise: a short, unforgeable, unambiguous fingerprint"] --> B["How it is built: the Merkle-Damgard chain and the sponge"] B --> C["How it broke: MD5 and SHA-1 collisions weaponized into forgery"] C --> D["What survived: SHA-2 widened the same margins"] D --> E["What diversified: the SHA-3 sponge, the BLAKE3 tree, KangarooTwelve"] E --> F["How to choose: match the function and mode to the clause you rely on"]

A preview of the plot, because it corrects the single most common misconception in the whole subject. Some hashes died and were superseded: MD5 gave way to SHA-1, which gave way to SHA-2. But SHA-2 did not then give way to SHA-3. Instead the field forked. One arc is supersession by failure; the other is deliberate diversification, where several unbroken constructions coexist on purpose so the internet's trust machinery never again rests on a single design. Telling those two arcs apart is most of what it means to understand this topic.

Before we can understand how the fingerprint was forged, we have to say, precisely, what it was ever supposed to guarantee.

2. What a Hash Actually Promises

Ask three engineers what a "secure hash" means and you will get three answers: it is one-way, it has no collisions, it looks random. Cryptographers are more disciplined. They name exactly three properties, plus one more that everyone wishes were true and no real function actually delivers. Getting these four straight is the whole vocabulary, because every break later in this guide is best described as one specific property falling while the others stand.

Start with the object itself. A hash function $H$ maps an input of essentially unbounded length to an output of some fixed length $n$ bits: 256 bits for SHA-256, 160 for SHA-1, 128 for MD5. Because the input space is infinite and the output space has only $2^n$ points, collisions must exist by counting alone. Security is never about avoiding collisions in principle; it is about making them, and two related feats, computationally unreachable.

Phillip Rogaway and Thomas Shrimpton gave the field its rigorous, separated definitions of these properties in 2004, and it is worth stating them the way they did, because the separations matter [@rogaway-shrimpton-fse04].

A deterministic function that maps an input of arbitrary length to a fixed-length digest, engineered so that the digest behaves like a compact, tamper-evident fingerprint of the input: cheap to compute forwards, and infeasible to invert, to steer to a chosen value, or to make two inputs share, except by effort that grows exponentially in the digest length. Given only a target digest $y$, it is computationally infeasible to find any input $m$ with $H(m) = y$. This is the "one-way" property. For an ideal $n$-bit hash the generic cost is about $2^{n}$ evaluations, because you have no better strategy than trying inputs until one lands on $y$. Given a specific input $m_1$, it is infeasible to find a different input $m_2 \ne m_1$ with $H(m_2) = H(m_1)$. The target is fixed and not of your choosing. The generic cost is again about $2^{n}$. It is infeasible to find any two distinct inputs $m_1 \ne m_2$ with $H(m_1) = H(m_2)$. Here you are free to choose both inputs, which is exactly why it is easier to attack. The generic cost is only about $2^{n/2}$, by the birthday bound.

The gap between $2^{n}$ and $2^{n/2}$ is the single most important number in this article. Preimage and second-preimage pit you against a fixed target, so you are searching a space of size $2^n$. Collision lets you range freely over pairs, and pairs accumulate quadratically.

In a space of $N = 2^{n}$ equally likely values, a randomly drawn set is more likely than not to contain a repeat after only about $\sqrt{N} = 2^{n/2}$ draws, because the number of candidate pairs among $k$ items grows like $k^2/2$. Formally the collision probability after $k$ draws is about $1 - e^{-k(k-1)/(2N)}$.

The name comes from the party trick: in a room of just 23 people, two of them probably share a birthday, even though there are 365 candidate days, because 23 people form 253 pairs. For a 256-bit hash the same arithmetic says a collision surfaces after about $2^{128}$ digests rather than $2^{256}$, so a 256-bit hash buys you only 128 bits of collision resistance. This is why collision is the weakest joint, and, as the failure catalog will show, always the first to fall.

Clause	What the attacker must find	Generic cost	Who leans on it	First to fall?
Preimage resistance	Any input hashing to a given digest	$2^{n}$	Password hiding, commitments, key derivation	Last
Second-preimage resistance	A different input matching one fixed input	$2^{n}$	Integrity of a known, published file	Middle
Collision resistance	Any two distinct inputs that agree	$2^{n/2}$	Signatures, certificates, deduplication	First

Second-preimage feels like it should be as easy as collision, but it is not. A collision lets you shop for any lucky pair anywhere in the space. A second preimage nails one endpoint to a value someone else chose and dares you to hit it from the other side, which is a $2^n$ problem, not a $2^{n/2}$ one. Attackers who can only find collisions therefore need to control both documents, which is precisely the constraint the forged-certificate attacks had to engineer around.

The following simulator makes the gap tangible. It uses a toy 24-bit hash so both effects fit in a browser tab: watch how few inputs it takes to stumble into a collision, and how hopeless it is to hit one fixed target by search.

{` // A toy 24-bit "hash": mix the input, then keep the low 24 bits (N = 2^24). const N = 1 << 24; // 16,777,216 possible digests function h(x) { let v = Math.imul(x ^ 0x9e3779b1, 2654435761) >>> 0; v ^= v >>> 15; // fold high bits down so the low bits mix v = Math.imul(v, 0x85ebca6b) >>> 0; v ^= v >>> 13; return v & (N - 1); }

// Collision search: hash 1, 2, 3, ... and stop at the first repeat. const seen = new Map(); let collisionAt = null; for (let i = 1; i < 300000 && collisionAt === null; i++) { const d = h(i); if (seen.has(d)) collisionAt = i; else seen.set(d, i); } console.log("birthday bound, sqrt(N), is about", Math.round(Math.sqrt(N))); console.log("first collision appeared after only", collisionAt, "inputs");

// Preimage search: how many tries to hit ONE fixed target digest? const target = h(42), budget = 500000; let tries = 0, hit = false; for (let x = 5000000; x < 5000000 + budget; x++) { tries++; if (h(x) === target) { hit = true; break; } } console.log("preimage search used", tries, "tries and", hit ? "got lucky" : "found nothing"); console.log("average preimage cost would be about N/2 =", N / 2, "tries"); `}

Collisions surface after only a couple of thousand inputs, while the fixed-target search burns through half a million tries and, on average, would need eight million. That is the $2^{n/2}$ versus $2^{n}$ split in miniature.

There is a fourth property people silently assume: that a good hash behaves like a random oracle, a magic box that returns a fresh uniformly random answer for every distinct input and is otherwise consistent. It is a wonderful modeling fiction, and much of applied cryptography is proved secure by pretending it is real.

But no fixed, finite, publicly specified function can actually be a random oracle, and Canetti, Goldreich, and Halevi proved the gap is not merely cosmetic: there exist schemes secure with an ideal oracle yet insecure under every concrete instantiation [@cgh-jacm-2004]. We make that precise in Section 10. For now, hold the three real clauses in mind, plus the knowledge that the fourth is a heuristic.

Two consequences follow immediately and set up everything after. First, you never sign a document; you sign its digest, so a signature is only as unforgeable as the hash is collision resistant. The formal security goals for signatures and message authentication codes are the subject of Part 1 of this series, the security-definitions field guide; here we only need the consequence. Second, the weakest clause is collision resistance, sitting at $2^{n/2}$. To see how that clause snapped in practice, we first have to see how the fingerprint is assembled, because the assembly leaks in ways the three clauses alone do not reveal.

3. How the Fingerprint Was Built: The Merkle-Damgard Era

The hash function became load-bearing infrastructure almost by accident. When Whitfield Diffie and Martin Hellman introduced public-key cryptography in 1976, their signatures could only operate on short numbers, roughly the size of the key [@diffie-hellman-1976]. But people wanted to sign whole documents, contracts, and programs. The fix was quiet and consequential: do not sign the document, sign a short fingerprint of it. That single move loaded the entire weight of a signature onto the hash. If two documents can share a fingerprint, one signature covers both, so the signature scheme is only as sound as the hash beneath it.

Ralph Merkle's 1979 Stanford thesis formalized one-way hash functions and, in its "Tree Authentication" section, introduced the hash tree that now underlies Git, blockchains, and verified downloads [@merkle-thesis-1979].

The next question was how to build such a fingerprint for an input of any length out of a small, analyzable building block. In 1989, Ralph Merkle and, independently, Ivan Damgard supplied the answer that defined the next two decades [@merkle-crypto89, @damgard-crypto89].

An iterated hash built by padding the message, splitting it into fixed-size blocks, and folding the blocks one at a time through a compression function, starting from a fixed initial value called the IV. Merkle and Damgard proved that if the compression function is collision resistant and the final block encodes the message length, a trick called MD strengthening, then the whole hash inherits collision resistance from the small compression function.

That theorem is the reason the 1990s could build hashes with confidence: analyze one small function, get a hash for arbitrary inputs for free. The digest is simply the chaining state after the last block has been absorbed.

flowchart LR IV["IV: fixed start state"] --> C1["compress"] M1["block m1"] --> C1 C1 --> C2["compress"] M2["block m2"] --> C2 C2 --> C3["compress"] MP["final block: message plus length padding"] --> C3 C3 --> D["digest: the final chaining state"]

Ronald Rivest built the first widely used family on this shape. MD4 arrived in 1990 and the strengthened MD5 in 1992, both producing 128-bit digests from an add-rotate-xor compression function [@rivest-md4-crypto90, @rfc-1320]. RFC 1321 states MD5's purpose plainly: it compresses a message "in a secure manner before being signed with a private key" [@rfc-1321]. The design DNA, a message schedule feeding an ARX round function, propagated forward. In 1993 the U.S. government standardized a 160-bit hash of the same lineage in FIPS 180, retroactively called SHA-0, and in 1995 replaced it with SHA-1, a one-bit tweak of the same design [@nist-hash-functions]. SHA-1's specification survives in the current Secure Hash Standard [@fips-180-4], and it went on to hash most of the internet for roughly twenty years.

The only visible difference between SHA-0 and SHA-1 is a single one-bit rotation added to the message schedule in 1995, a small and initially unexplained change later understood to improve collision resistance. The hedge held for a while, but SHA-0 was later shown to be genuinely fragile: Xiaoyun Wang and colleagues found efficient full collisions for SHA-0 well under the birthday bound in 2005 [@wang-yu-yin-sha0-crypto05]. The quietly patched flaw foreshadowed the cryptanalysis that would soon reach SHA-1 itself.

This detour, hashing before signing, is so routine that engineers forget it is a security assumption. A signature over `H(m)` is meant to bind you to `m`, but it truly binds you only to `H(m)`. If an adversary finds a second message with the same digest, your signature covers that message too, and no key was ever stolen. Every certificate, code signature, and signed software update in this article inherits this assumption, which is why a collision in the hash is not an academic curiosity but a forged signature waiting to happen.

The Merkle-Damgard theorem promised something precise and limited: the whole hash is as collision resistant as its little compression function. It never promised that the shape wrapped around that function, the chaining, the padding, the exposed final state, was itself sound. That distinction is exactly where the first cracks opened, and they opened even for a hypothetically perfect compression function.

4. The Structural Cracks: Why Merkle-Damgard Leaks

Here is the unsettling part. You can hand the Merkle-Damgard construction a perfect compression function, one with no weakness whatsoever, and the assembled hash still leaks. It leaks four separate ways, and none of them is a bug in any particular function. They are properties of the shape.

The first leak is the most practical, and it follows directly from the diagram in the previous section. The digest is the final chaining state. Nothing hides it. So if you know a digest and the length of the message that produced it, you can pick up the computation exactly where it left off.

Given a valid digest $H(m)$ and the length of $m$, but not $m$ itself, an attacker can compute $H(m \,\|\, \text{pad} \,\|\, s)$ for a suffix $s$ of their choosing. It works because a Merkle-Damgard digest is literally the internal chaining state after the last block, so the attacker resumes from that state and folds in more blocks, without ever knowing $m$.

The mechanism has exactly three moving parts, and it needs neither the message nor any secret. Publishing $H(m)$ publishes a fully loaded, resumable hash engine. From the length of $m$ the attacker rebuilds the glue padding, the 0x80 byte, the run of zeros, and the encoded bit-length that Merkle-Damgard appends before the final block [@fips-180-4]. The attacker reloads $H(m)$ as the IV, folds in the glue and then an arbitrary suffix, and reads off a valid digest for the longer message. When the server later hashes secret || m || glue || suffix it walks the identical chain, passes through the intermediate state $H(m)$, folds in the same suffix, and confirms the forgery.

flowchart LR PUB["Public: digest H(m) and length of m"] --> REBUILD["Rebuild glue padding from the length alone"] REBUILD --> RELOAD["Reload H(m) as the compression IV"] RELOAD --> FOLD["Fold in glue, then attacker suffix"] FOLD --> FORGE["Forged digest for m plus glue plus suffix"] SERVER["Server hashes secret plus m plus glue plus suffix"] -.walks the identical chain.-> FORGE

This turns a natural-looking construction into a trap. Many engineers reach for H(secret || message) as a homemade message authentication code, reasoning that you cannot forge a tag without knowing the secret. Length extension refutes that: given a valid tag for one message, an attacker appends chosen data and computes a valid tag for the longer message, secret still unknown. The toy hash below is deliberately tiny and insecure, but it has the one property that matters, its digest is its whole internal state, so it forges end to end exactly as MD5 or SHA-256 would.

{` // A toy Merkle-Damgard hash. Like MD5 and SHA-256, its DIGEST IS ITS FINAL // INTERNAL STATE -- that single fact is the entire length-extension bug. const BLK = 8; const IV = 0x1234abcd >>> 0; const enc = s => [...s].map(c => c.charCodeAt(0));

// The server publishes (message, tag) where tag = H(secret || message). const secret = enc("SECRET"); // 6 bytes the attacker never sees const message = enc("user=guest"); const tag = toyHash(secret.concat(message));

// The attacker knows tag, message, and only the LENGTH of the secret (6). const secretLen = 6; const glue = pad(secretLen + message.length); // rebuilt from lengths, no secret const suffix = enc("&role=admin"); const total = secretLen + message.length + glue.length + suffix.length; const forgedTag = fold(tag, suffix.concat(pad(total))); // resume FROM the tag const forgedMessage = message.concat(glue).concat(suffix);

// The server verifies the forged message with its real secret. const check = toyHash(secret.concat(forgedMessage)); console.log("forged tag (secret never used):", forgedTag.toString(16)); console.log("server recomputation :", check.toString(16)); console.log("forgery accepted?", forgedTag === check); console.log("smuggled 'role=admin' present?", String.fromCharCode(...forgedMessage).includes("role=admin")); `}

Real tools do this against real hashes. The canonical hash_extender demonstrates it on MD5: publish signature = MD5(secret || "data") = 6036708eba0d11f6ef52ad44e8b74d5b, and knowing only data, the algorithm, and that signature, an attacker loads the signature back into MD5's state and continues hashing a chosen suffix to forge a valid tag for secret || data || glue || suffix, never learning the secret. Its own summary is the whole lesson: "100% of the state needed to continue a hash is in the output of most hashing algorithms" [@hash-extender].

Note: On any full-width Merkle-Damgard hash, including MD5, SHA-1, SHA-256, and SHA-512, the prefix-secret construction H(secret || message) is forgeable by length extension. Use HMAC (Section 9) or a length-extension-immune hash instead. This footgun is common enough that it has its own family of real exploits in serialized-signing and token systems, examined in the serialization and canonicalization field guide; here we own the mechanism, that guide owns the exploitation.

A precise note you should carry forward, because it is the sharpest discriminator between functions in this whole article. Length extension afflicts hashes whose digest exposes the full final state: MD5, SHA-1, and the full-width SHA-256 and SHA-512. Truncation can close the leak, but only if it hides enough of the state. The truncated SHA-2 variants that hide at least 128 bits (SHA-384, SHA-512/224, SHA-512/256), the SHA-3 family and SHAKE, HMAC, and the BLAKE hashes are all immune. The dangerous middle case is SHA-224, which discards only 32 bits and is therefore still vulnerable at about $2^{32}$ work, a distinction Section 6 makes exact. Defer the full table there; just remember that "truncated" does not automatically mean "safe."

The other three leaks are subtler but drove the field toward new designs. In 2004, Antoine Joux showed that in any iterated hash, finding $2^k$ messages that all share one digest costs only about $k$ times the work of a single collision, not the vastly larger amount naive counting suggests [@joux-crypto04]. The construction is almost embarrassingly direct: from the starting state, one birthday search finds a colliding pair of blocks that both drive the chain to the same next state; repeat from that state $k$ times; then every one of the $2^k$ ways to pick one block or the other at each step traces the identical chain to the identical final digest.

These multicollisions demolish a piece of folklore that had reassured a generation of engineers: that concatenating two independent hashes, $H_1(m) ,|, H_2(m)$, multiplies their strength. It does not. Build a large multicollision in the weaker half -- many messages that all share one digest under it, which is cheap for a Merkle-Damgard hash -- then birthday-search that set for a pair that also collides under the stronger half. The combined collision resistance is essentially that of the stronger single hash, not the sum. Hashing twice buys almost nothing.

Joux's multicollisions were also the engine for two deeper results. In 2005, John Kelsey and Bruce Schneier used expandable messages built from multicollisions to find second preimages on Merkle-Damgard hashes for far less than the ideal $2^n$: for a target of $2^k$ blocks, the cost falls to roughly $2^{n-k}$ plus overhead [@kelsey-schneier-eurocrypt05]. That matters because second-preimage resistance was supposed to be the sturdy $2^n$ clause, yet the shape leaks it for long messages. In 2006, Kelsey and Tadayoshi Kohno went further with the herding, or Nostradamus, attack: an attacker precomputes a branching diamond structure, publicly commits to a digest as if it were a prediction, and later herds almost any chosen prefix into that exact digest [@kelsey-kohno-eurocrypt06]. It weaponizes the construction into proof-of-prediction fraud.

Key idea: Length extension, Joux multicollisions, long-message second preimages, and herding are flaws in the shape, not in the function. A stronger or longer compression function cannot fix them, because they do not depend on any weakness in that function. Only a different construction can. Hold that thought: it is exactly why, after the failures, the field did not just build a bigger Merkle-Damgard hash. It built a different machine.

Those four are leaks in an idealized world, where the compression function is flawless. They were serious enough to motivate truncated designs and, eventually, entirely new constructions. But they were still theoretical. What turned this subject from a seminar into a decade of emergency migrations was the other kind of failure: the compression function itself fell, and someone chose to weaponize it.

5. The Failure Catalog: Every Notable Break, Named

A broken hash is a lab curiosity until someone forges a certificate with it. Twice, someone did, and once it was a nation-state. This section is the heart of the guide, because the two failure arcs, MD5 and SHA-1, are the evidence for everything the rest of it recommends. Each arc runs the same course: a theoretical crack, a practical collision, a technique that makes the collision meaningful, and finally a real forged trust anchor.

The MD5 arc opens with an early warning that the field under-heeded. In 1996, Hans Dobbertin found practical full collisions for MD4 and collisions in MD5's compression function under a chosen initial value, not yet the full hash but a clear structural crack [@dobbertin-md4]. MD5 stayed deployed anyway.

The warning became undeniable in August 2004, when Xiaoyun Wang, Dengguo Feng, Xuejia Lai, and Hongbo Yu announced full MD5 collisions computable in minutes, alongside collisions for MD4, HAVAL-128, and RIPEMD [@wang-md5-eprint-2004-199]. The following year, Wang and Yu published the peer-reviewed differential method behind it [@wang-yu-break-md5-eurocrypt05]. That is the theoretical death of MD5. Everything after is engineering.

The engineering that mattered was learning to choose the colliding inputs.

A collision between two inputs that begin with two different, attacker-chosen prefixes. The attacker fixes meaningful, distinct starting content for each side, two different names in a certificate, say, and then computes collision blocks that drive both sides to the same digest. This is strictly harder than an identical-prefix collision, where the two inputs share a common start, and strictly more dangerous, because it forges a relationship between two meaningful documents rather than two random blobs.

In 2007, Marc Stevens, Arjen Lenstra, and Benne de Weger built exactly that for MD5, and used it to construct two X.509 certificates with different identities but the same MD5 digest [@stevens-chosen-prefix-md5-eurocrypt07].

A year later the attack left the lab. At the 2008 Chaos Communication Congress, a seven-person team used a chosen-prefix MD5 collision, about $2^{49}$ MD5 compressions on a cluster of roughly 200 PlayStation 3 consoles, to obtain a genuine commercial certificate-authority signature over a benign certificate that simultaneously validated a forged intermediate certificate authority [@rogue-ca]. The forged certificate carried the basic-constraints CA flag set to TRUE, which meant every browser on earth would trust certificates it issued.

flowchart TD P1["Benign prefix: an ordinary certificate request"] --> COL["Collision blocks appended to both sides"] P2["Malicious prefix: a rogue CA certificate"] --> COL COL --> SAME["Both certificates share one MD5 digest"] SAME --> SIG["Commercial CA signs the benign certificate"] SIG --> VALID["The same signature validates the rogue CA:TRUE certificate"]

To make two certificates collide, the attacker needs somewhere to park the roughly 1632-bit collision block without it looking suspicious. In the 2008 rogue CA, that space lived inside a "Netscape Comment" extension, a field parsers ignore, which the researchers wryly described as hiding the collision like a tumor inside an otherwise healthy certificate [@rogue-ca]. The lesson generalizes: any format with an ignorable, attacker-controlled field gives a collision attack room to breathe.

The arc peaked in June 2012 with Flame, espionage malware that carried a code-signing certificate chaining to Microsoft. Flame's authors mounted a new chosen-prefix MD5 collision against a Microsoft sub-authority still using MD5 in its licensing service, letting them sign malware as "Microsoft" and distribute it through Windows Update. Marc Stevens later reconstructed the previously unknown collision variant from the certificate itself [@stevens-counter-cryptanalysis-crypto13].

A nation-state had used a hash collision to subvert the update trust root of a billion machines. After Flame, MD5 in signatures was not deprecated as a matter of hygiene; it was radioactive.

SHA-1 traveled the same road a decade behind. In 2005, Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu published the first collision attack on the full SHA-1, at about $2^{69}$ operations, comfortably under the $2^{80}$ birthday bound its 160-bit output was supposed to guarantee [@wang-yin-yu-full-sha1-crypto05]. That theoretical break is what prompted NIST to open the SHA-3 competition. The practical break took twelve more years of hardware progress. In February 2017, a team from CWI Amsterdam and Google announced SHAttered: the first practical identical-prefix SHA-1 collision, at roughly $2^{63.1}$ SHA-1 computations, about nine quintillion of them, costing the equivalent of 6,500 CPU-years plus 110 GPU-years [@shattered]. The proof was the pair of PDFs from this article's opening, sharing the digest 38762cf7f55934b34d179ae6a4c80cadccbb7f0a.

In January 2020, Gaetan Leurent and Thomas Peyrin closed the arc with "SHA-1 is a Shambles," the first practical chosen-prefix SHA-1 collision at about $2^{63.4}$, which along the way cut the identical-prefix cost to about $2^{61.2}$ [@shambles, @shambles-eprint-2020-014]. To prove it mattered, they forged a PGP/GnuPG key certification, tracked as CVE-2019-14855, building on their own EUROCRYPT 2019 precursor that first pushed collisions toward chosen prefixes [@leurent-peyrin-cp-eurocrypt19].

Note: SHAttered (2017) was an identical-prefix collision: both files began the same way. Shambles (2020) was a chosen-prefix collision: the two inputs started with different, meaningful content, which is what enables impersonation and forged certificates. Different authors, different years, strictly different strength. Conflating them is one of the most common errors in write-ups of SHA-1's demise.

"All attacks that are practical on MD5 are now also practical on SHA-1." That is the Shambles team's own summary of what a chosen-prefix collision means: SHA-1 has caught down to MD5 [@shambles]. timeline title Two failure arcs, MD5 and SHA-1 1996 : Dobbertin cracks MD4, dents the MD5 compression function 2004 : Wang team announces practical full MD5 collisions 2005 : Wang, Yin, Yu attack full SHA-1 near 2 to the 69 2007 : Stevens builds chosen-prefix MD5 collisions 2008 : Rogue certificate authority forged near 2 to the 49 2012 : Flame forges a Microsoft-chained code-signing certificate 2017 : SHAttered, first identical-prefix SHA-1 collision 2020 : Shambles, first chosen-prefix SHA-1 collision

The attack costs are worth tabulating, because they show how a break moves from theory to a weekend on rented hardware.

Break	Year	Attack type	Approx. cost	Real-world impact
Rogue CA	2008	Chosen-prefix MD5	$2^{49}$	Forged intermediate CA trusted by browsers [@rogue-ca]
Flame	2012	Chosen-prefix MD5	new variant	Forged Microsoft-chained code-signing cert [@stevens-counter-cryptanalysis-crypto13]
SHAttered	2017	Identical-prefix SHA-1	$2^{63.1}$	Two PDFs, one SHA-1 digest [@shattered]
Shambles	2020	Chosen-prefix SHA-1	$2^{63.4}$	Forged PGP/GnuPG key certification [@shambles]

There is a discipline the headlines miss, and it protects you from over-correcting. "MD5 and SHA-1 are dead" is true only for uses that depend on collision resistance.

Every break above attacks collision resistance. None of them touches HMAC-SHA-1 or HMAC-MD5, because HMAC does not rely on collision resistance at all, a point RFC 2104 made in 1997 and Section 9 unpacks [@rfc-2104]. So the correct scoping is narrow: MD5 and SHA-1 are dead where an adversary can influence the hashed input and profit from a collision, above all in signatures and certificates. They are not magically extinct. Old certificate chains, legacy protocols, and Git's historical use of SHA-1 all persist, which is why NIST set a firm calendar date, the end of 2030, to withdraw SHA-1 from all U.S. government use rather than pretending it vanished overnight [@nist-retires-sha1]. The recurring lesson of both arcs: deprecating an algorithm in new code is not the same as revoking it from live trust anchors.

Every hash in this catalog shared one design shape, the Merkle-Damgard chain of Section 3. The obvious question, and the one the field actually asked, was whether anything built the same way survived. One did, and understanding why is the turn from despair to design.

6. The Survivor: SHA-2, and Why It Did Not Fall

Here is a paradox worth sitting with. SHA-2 is built the same way as SHA-1: the same Merkle-Damgard skeleton, the same add-rotate-xor compression style, designed by the same institution. By the logic of the last two sections it should have fallen too. Instead, in 2026, SHA-256 is the hash under most of the internet. What did the survivors do differently?

The family, standardized in FIPS 180-4, has six members: SHA-224, SHA-256, SHA-384, and SHA-512, plus the truncated SHA-512/224 and SHA-512/256 [@fips-180-4]. And the answer to the paradox is almost anticlimactic. SHA-2 did not adopt a new idea. It widened the margins of the old one.

Larger chaining state, a longer and more diffusing message schedule, more rounds, and distinct per-round constants together mean that the differential paths Wang's team rode through MD5 and SHA-1 do not propagate far enough to reach the full function. The evidence is two decades of cryptanalysis that keeps stalling short: the best published collision attacks reach only 31 of SHA-256's 64 rounds in practice [@mendel-nad-schlaffer-eurocrypt13], and 39 rounds in the weaker semi-free-start setting [@li-liu-wang-sha2-2024]. A 2024 effort using programmatic SAT solvers likewise reaches only reduced-round or modified-initial-value variants, never the full function [@sha256-sat-2024]. That wide margin, 33 rounds to the deepest true collision and 25 even against the weaker semi-free-start attack, is the entire reason to trust it.

There is one wrinkle the designers half-anticipated, and it is the single most useful discriminator you can memorize. Full-width SHA-256 and SHA-512 are length-extension-vulnerable, exactly as Section 4 warned, because their digest is the whole final state. Most of the truncated members are not. But one truncated member still is, and it is the one people most often get wrong.

A hash whose internal chaining state is wider than its emitted output, so the digest reveals only part of the final state. SHA-384 and SHA-512/256 run the full SHA-512 engine, a 512-bit state, but output only 384 or 256 bits. Because an attacker never sees the whole state, length extension has nothing to resume from, provided enough of the state stays hidden.

The catch is in that last clause. Truncation defeats length extension only when the hidden part of the state is too large to guess. SHA-384 hides 128 bits and SHA-512/256 hides 256; recovering either by brute force is infeasible. SHA-224, however, is SHA-256 with a different initial value and the 256-bit result cut to 224 bits, so it discards only one 32-bit word of state [@rfc-3874].

An attacker brute-forces that missing 32 bits in about $2^{32}$ compression calls, seconds on commodity hardware, recovers the full internal state, and then length-extends exactly as if nothing were hidden. SHA-224 is therefore length-extension-vulnerable with only a 32-bit margin, categorically different from the truncated variants that hide 128 bits or more.

Note: The move that saves SHA-384 and SHA-512/256 is almost embarrassingly simple: hide part of the final state. But the amount hidden is the whole game. Hide 128 or 256 bits and length extension has nothing feasible to resume from. Hide only 32 bits, as SHA-224 does, and an attacker recovers the state in about $2^{32}$ work and extends normally. "Truncated" is not a synonym for "safe"; "truncated by enough" is. The sponge, next section, takes this instinct to its logical end by never exposing the state at all.

The table below is the sharpest single reference in this guide. When someone asks whether a hash is safe to use as H(secret || message), this is the answer.

Length-extension vulnerable	Length-extension immune
MD5, SHA-1	SHA-384
SHA-256, SHA-512 (full-width)	SHA-512/224, SHA-512/256
SHA-224 (only $2^{32}$ margin)	SHA-3 family and SHAKE
	HMAC (any hash), BLAKE2, BLAKE3, KangarooTwelve

SHA-NI, the x86 hash instructions Intel and AMD ship, accelerate SHA-1 and SHA-256 but not SHA-512 [@intel-sha-extensions]. That inverts an old piece of folklore. For years SHA-512 was "the fast one" on 64-bit CPUs because it processes 64-bit words; on a 2023 AMD Zen 4, SHA-256 now runs at about 2.03 cycles per byte while SHA-512 trails at about 4.19, because only SHA-256 gets the hardware boost [@ebash-zen4]. On CPUs without SHA-NI, the old ordering returns and SHA-512 and SHA-512/256 pull ahead again.

That hardware acceleration, combined with universal library support and FIPS validation, is why SHA-256 is the correct default almost everywhere its deployment footprint shows: TLS 1.3 computes its handshake transcript hash with the negotiated SHA-256 or SHA-384 [@rfc-8446], Bitcoin hashes every block header with double SHA-256 [@bitcoin-block-hashing], and Git's next-generation object format is built on SHA-256 [@git-sha256-transition], alongside code-signing digests and certificate fingerprints. It earned that position not by being newest but by being unbroken and fast where it counts.

Keep one framing straight, because the whole next act depends on it. SHA-2 is a survivor of the supersession arc that ran MD5 to SHA-1 to SHA-2. It is not something a later hash replaced. But a healthy survivor carrying most of the world's traffic is still a monoculture, and the field had just learned, viscerally, what happens when everything rests on one design. So even though SHA-2 was fine, cryptographers deliberately built something that shared none of its DNA.

7. The Different Shape: The Sponge and SHA-3

Imagine a hash with no exposed chaining state to steal, that hands you arbitrary-length output for free, and that shares no mathematical DNA with SHA-2. That is not a patch on Merkle-Damgard. It is a different machine.

NIST went looking for that machine deliberately. In November 2007, stung by the 2004 and 2005 collision attacks, it opened a public SHA-3 competition, explicitly asking for a hash built on a construction unlike SHA-2 so that a future break of one would not endanger the other [@nist-sha3-project]. Five finalists emerged from dozens of submissions: BLAKE, Grostl, JH, Keccak, and Skein. In October 2012, NIST selected Keccak, designed by Guido Bertoni, Joan Daemen, Michael Peeters, and Gilles Van Assche, and standardized it as FIPS 202 in August 2015 [@nist-sha3-project, @fips-202]. Its engine is the sponge.

A hash built around a single fixed permutation applied to a state of $b$ bits, split into a rate of $r$ bits and a capacity of $c$ bits, with $b = r + c$. In the absorb phase, message blocks are XORed into the rate portion and the whole state is stirred by the permutation. In the squeeze phase, output bytes are read from the rate, with the permutation applied between reads. The capacity is never read from or written to directly. SHA-3 uses the Keccak permutation on a 1600-bit state over 24 rounds.

The capacity is the whole trick. Because those $c$ bits never touch the input or the output, an attacker who sees the digest learns nothing about them, and cannot reconstruct the internal state.

flowchart LR M["Message blocks"] --> ABS["Absorb: XOR into rate, then permute"] ABS --> STATE["State: rate lane plus capacity lane"] STATE --> SQ["Squeeze: read the rate, permute, repeat"] SQ --> OUT["Output: fixed digest or an XOF stream"] CAP["Capacity lane, never read or written directly"] -.-> STATE

Two payoffs fall out of that single design decision. First, the sponge is length-extension-immune by construction, not by truncation: there is no exposed final state to resume from, so H(secret || message) is not forgeable the way it is on full-width SHA-2. Second, the squeeze phase can simply keep going, so the sponge is natively an extendable-output function.

A hash-like primitive that emits output of any length you ask for from a single input, rather than a fixed-size digest. You request as many bytes as you need and keep squeezing. SHAKE128 and SHAKE256, standardized alongside SHA-3, are the workhorse examples, and they are what modern post-quantum schemes call to expand seeds into keys.

The security is not folklore. Bertoni, Daemen, Peeters, and Van Assche proved the sponge is indifferentiable from a random oracle up to the capacity bound, which is the strongest positive statement you can make about a concrete construction [@bertoni-sponge-indiff-eurocrypt08, @keccak-sponge-duplex]. In two decades of analysis, practical collisions have reached only about 5 of Keccak's 24 rounds, leaving an enormous margin [@keccak-third-party].

Now the load-bearing correction this whole guide exists to make. SHA-3 is not the successor to SHA-2 the way SHA-2 succeeded SHA-1. FIPS 202 chooses its words carefully.

FIPS 202 states that the four SHA-3 functions "supplement the hash functions ... in FIPS 180-4: SHA-1 and the SHA-2 family," and that together the two standards "provide resilience against future advances" because they "rely on fundamentally different design principles" [@fips-202].

The word is supplement, not supersede, and NIST is even blunter elsewhere: its Policy on Hash Functions says there is "no need to transition applications from SHA-2 to SHA-3" [@nist-policy-hash-functions], and the superseded 2012 version of that policy put it more strongly still, with "no need or plan to transition" [@nist-policy-hash-functions-2012-archived].

Key idea: SHA-3 is not the replacement for SHA-2. It is the insurance policy. After watching one construction, Merkle-Damgard, carry MD5, SHA-1, and SHA-2 all at once, the field deliberately funded a second, unrelated construction so that the internet's trust machinery would never again rest on a single design. SHA-2 and SHA-3 coexist on purpose. That is diversification, not supersession, and it is a different axis from the MD5-to-SHA-1-to-SHA-2 ladder entirely.

Ethereum uses "Keccak-256," which is the original competition-era Keccak with its pre-standard padding, not the FIPS 202 SHA3-256 that NIST later finalized with an extra domain-separation suffix. The two give different digests for the same input, a frequent source of confusion for developers who assume "Keccak" and "SHA-3" are interchangeable. Solidity's built-in is spelled keccak256 precisely to signal the original variant [@solidity-globals].

The sponge bought genuine algorithmic diversity and length-extension immunity for free. What it did not buy, at least in plain software, was raw speed: on a Zen 4 core without special instructions, SHA3-256 runs at about 6.8 cycles per byte, roughly three times slower than hardware-accelerated SHA-256 [@ebash-zen4]. That gap set the agenda for the last decade. Could you keep the security and the length-extension immunity and still go an order of magnitude faster? Two answers arrived, from two different shapes.

8. The Fast Hashes: BLAKE3, the Tree, and Where K12 Sits

The command b3sum can hash a large file roughly ten times faster than sha256sum, and it does not cheat to get there [@blake3-repo]. It uses every CPU core and every SIMD lane at once. Understanding how it does that, and when you should let it, means separating two ideas people constantly conflate.

There are two axes here, and they are independent. The construction axis says how a hash is built: Merkle-Damgard (SHA-2), sponge (SHA-3, SHAKE, and, importantly, KangarooTwelve), and the BLAKE tree family derived from the ChaCha stream cipher. The use-case axis says what job you are hiring it for, and "modern high-throughput hash" is a job, currently filled by BLAKE3 and KangarooTwelve. Notice that those two fast hashes sit on different construction axes: one is a tree over an ARX core, the other is a sponge. Speed is not a construction.

The BLAKE line began as a SHA-3 finalist, a ChaCha-derived design [@hash-function-blake-book]. It became practical with BLAKE2, specified in RFC 7693, which is directly keyable: feed it a key and it is a message authentication code with no HMAC wrapper needed [@rfc-7693]. Then in 2020, Jack O'Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O'Hearn released BLAKE3, which took the decisive structural step: it arranges a round-reduced BLAKE2-style compression function inside a binary Merkle tree [@blake3-repo].

flowchart TD ROOT["Root chaining value, extensible to an XOF"] --> P0["Parent node"] ROOT --> P1["Parent node"] P0 --> L0["Chunk 0, 1 KiB"] P0 --> L1["Chunk 1, 1 KiB"] P1 --> L2["Chunk 2, 1 KiB"] P1 --> L3["Chunk 3, 1 KiB"]

The tree is what unlocks the speed. Because each 1 KiB chunk hashes independently, any number of SIMD lanes and threads can work in parallel, and the results combine afterward. The same structure gives BLAKE3 incremental updates and verified streaming, and it collapses five roles into one function: plain hash, keyed MAC, pseudorandom function, key-derivation function via its derive_key mode, and extendable output. On a Zen 4 core it runs at about 0.60 cycles per byte single-threaded, the fastest secure hash in the eBASH benchmarks, before you even add threads [@ebash-zen4].

BLAKE3's own documentation states it plainly: it is "secure against length extension, unlike SHA-2" [@blake3-repo].

The 1 KiB chunk size is not arbitrary. Because every chunk is an independent leaf, a verifier holding the root can check that one specific 1 KiB slice of a huge file is authentic by recomputing only the $O(\log n)$ nodes along its path, never re-hashing the whole file. That is the basis of verified streaming: download a movie and verify each piece as it arrives, aborting instantly if any block is corrupted or tampered [@blake3-repo].

Note: BLAKE3 is engineered to be fast, which makes it exactly wrong for hashing passwords, where you want to be deliberately slow so an attacker cannot try billions of guesses per second. Its own README says as much and points you to Argon2 [@blake3-repo]. Use a purpose-built password KDF, Argon2id for new systems, or scrypt, bcrypt, or PBKDF2, never a bare fast hash. Section 11 returns to this.

The other fast hash keeps you on the sponge. KangarooTwelve, with its TurboSHAKE core, is a tree hash over a round-reduced Keccak permutation, Keccak-p with 12 rounds instead of 24, published as the Informational RFC 9861 in October 2025 [@rfc-9861, @k12-acns18]. It is genuinely fast: the vendor reports about 0.51 cycles per byte using AVX-512 and 0.75 on an Apple M1, though a generic build measures closer to 3.5 cycles per byte on Zen 4 [@keccak-kangarootwelve, @ebash-zen4]. The point to hold onto, and the reason it gets a signpost rather than a section, is that KangarooTwelve is not a third construction family. It is the sponge again, tuned for throughput. It is standardized but still niche, and, like BLAKE3, not FIPS-approved.

On a machine with both tools installed, `time b3sum bigfile.iso` against `time sha256sum bigfile.iso` on a multi-gigabyte file shows the order-of-magnitude gap the BLAKE3 documentation advertises, most of it from multithreading. If you only have OpenSSL, `openssl dgst -sha3-256 bigfile.iso` will show you the software sponge penalty on hardware without SHA-3 instructions.

Everything else is a one-line signpost. RIPEMD-160, Whirlpool, China's SM3, and Russia's Streebog are real and occasionally mandated regionally, but they are not the default anywhere the reader is likely to design for. The losing SHA-3 finalists, Grostl, JH, and Skein, are well-analyzed but standardized nowhere. None of them changes the decision you are about to make.

Fast, parallel, and diverse, yes. But so far we have only hashed raw data. Real protocols almost never use a bare hash. They key it, they separate its domains, and they stretch its output, and that is precisely where most real bugs live.

9. Beyond the Bare Hash: Keyed, Domain-Separated, and XOF Modes

The most common hash bug in production is not a broken algorithm. It is a correct algorithm used in the wrong mode, and the specific mistake, again and again, is pressing H(secret || message) into service as a message authentication code. Section 4 showed why that is forgeable on full-width SHA-2. Here is what to use instead, and why it holds.

The canonical fix is HMAC, from Mihir Bellare, Ran Canetti, and Hugo Krawczyk in 1996 and RFC 2104 in 1997 [@bellare-canetti-krawczyk-crypto96, @rfc-2104].

A message authentication code built from any hash $H$ and a secret key $K$ by nesting two hash calls: $$\mathrm{HMAC}(K, m) = H\big((K \oplus opad) \,\|\, H((K \oplus ipad) \,\|\, m)\big)$$ where $opad$ and $ipad$ are fixed padding constants. Bellare, Canetti, and Krawczyk proved it is a secure pseudorandom function under assumptions weaker than full collision resistance of $H$.

Two things make HMAC the right tool. First, the outer hash of an inner hash structurally defeats length extension: the attacker never sees a resumable state. Second, and this is the payoff of the scoping discipline from Section 5, HMAC does not depend on the collision resistance of its hash. RFC 2104 states directly that the known collision weaknesses of MD5 "do not compromise the use of MD5 within HMAC" [@rfc-2104]. That is why HMAC-SHA-1 and HMAC-MD5 remain unbroken even though bare SHA-1 and MD5 are dead for signatures.

The keys HMAC needs have to come from somewhere with real entropy, which is the subject of Part 2 of this series, the field guide to cryptographic randomness.

HMAC is also the foundation of the most widely deployed key-derivation function, HKDF, which follows an extract-then-expand pattern: an extract step distills a uniform key from messy input keying material, and an expand step stretches it to any needed length [@rfc-5869]. HKDF is worth naming precisely because it is a mode built on a hash by way of HMAC, not a hash itself; when you need to turn a shared secret into keys, reach for it rather than hashing by hand.

Note: If you need to authenticate a message with a key, reach for HMAC-SHA-256 or KMAC. Do not invent H(key || message), H(message || key), or any variation: the first is length-extension-forgeable on full-width SHA-2, and the others have their own pitfalls that decades of analysis went into avoiding. The whole point of HMAC and KMAC is that someone already did the hard work and proved it correct.

The sponge offers a native alternative that needs no nesting, because it already resists length extension. NIST SP 800-185 defines four such functions: cSHAKE, KMAC, TupleHash, and ParallelHash [@sp-800-185]. KMAC is a keyed MAC that simply absorbs the key into the sponge. The other three are about domain separation.

Deliberately making the hash of the same bytes come out different depending on the context in which they appear, by mixing in a distinct label or customization string per context. It stops an input crafted for one purpose from being replayed as a valid input to another. Consider hashing a pair of fields, a username and a message. Concatenating them and hashing, `H(user || message)`, is ambiguous: the pair (`alice`, `bob-says-hi`) and the pair (`aliceb`, `ob-says-hi`) serialize to the same bytes and therefore the same digest. An attacker who controls where the boundary falls can forge equivalences. TupleHash removes the ambiguity by length-encoding each element of the sequence, so the structure, not just the bytes, is bound into the digest. cSHAKE does the same across protocol contexts: a customization string like the ASCII label for "email-signature" versus "file-checksum" guarantees the two contexts can never produce the same output for the same input. Any time distinct inputs could serialize to identical byte streams, you need this, and rolling it yourself is where subtle bugs breed [@sp-800-185].

XOFs change how you get short outputs, too. If you need a 20-byte value at a 128-bit security level, do not hash and chop arbitrary bytes off SHA-256; ask SHAKE128 or a BLAKE3 XOF for exactly 20 bytes. The security level is governed by the capacity, not by how many bytes you happen to read, so requesting the exact length is both cleaner and safer than ad hoc truncation. NIST is even revising SP 800-185 to add streaming specifications for SHAKE, a sign of how central XOFs have become [@nist-fips202-sp800185-update].

Finally, the oldest mode of all scales integrity to enormous data: the Merkle tree from Ralph Merkle's 1979 thesis [@merkle-thesis-1979]. Hash the leaves, hash pairs of hashes upward to a single root, and you can prove any one leaf belongs to the root with a path of only $O(\log n)$ hashes. Git object trees, blockchain state commitments, Certificate Transparency logs, and BLAKE3's verified streaming are all this idea.

The one rule that separates a correct Merkle tree from a broken one is to domain-separate leaf nodes from internal nodes, so an attacker cannot pass an internal node off as a leaf, exactly the flag-bit discipline BLAKE3 bakes in. The sponge and BLAKE families also power keyed authenticated-encryption modes, which belong to the block-cipher and nonce-reuse parts of this series and are out of scope here.

We now have functions and modes that keep every clause of the promise. But every one of them still bows to a few hard mathematical ceilings, and to one machine that does not yet exist. How much should you actually worry?

10. Theoretical Limits: Bounds, the Random Oracle, and Quantum

There is a best-possible hash, and you cannot build it. Understanding exactly why is what separates fear from calibrated caution, especially about quantum computers.

Start with the ceilings. For an ideal $n$-bit hash, collision resistance cannot exceed about $2^{n/2}$ work and preimage resistance cannot exceed about $2^{n}$. Those are not merely the best known attacks; they are the best any hash of that output length can offer, because the birthday and search arguments apply to a perfect random function. The good news, easy to miss, is that the deployed survivors meet those bounds exactly. The best published attack on full SHA-256, SHA-512, SHA3-256, BLAKE2, and BLAKE3 is the generic one, so there is no gap between what an attacker can do and the theoretical floor.

When cryptographers talk about a function's "security margin," they mean something narrower and attack-specific: the unbroken rounds between the deepest cryptanalysis of a given kind and the full round count. For SHA-256 collisions, published attacks reach 31 of 64 rounds (39 in the weaker semi-free-start setting), leaving a wide margin [@mendel-nad-schlaffer-eurocrypt13, @li-liu-wang-sha2-2024]; for Keccak collisions, only about 5 of 24 rounds fall [@keccak-third-party]. Wide margins are why these functions are trusted, not a gap in their strength.

The one clause where an ideal is genuinely out of reach is the random oracle. A fixed, finite, publicly specified function cannot actually behave like a fresh random answer for every input, for the plain reason that its code is right there to analyze. Ran Canetti, Oded Goldreich, and Shai Halevi made this precise: there exist signature and encryption schemes provably secure in the random-oracle model for which every concrete instantiation of the oracle by a real hash yields an insecure scheme [@cgh-jacm-2004, @cgh-stoc-1998]. So the random-oracle model is a proof heuristic, not a guarantee.

The reachable substitute is indifferentiability: the sponge is provably indifferentiable from a random oracle up to its capacity bound, which is the strongest honest claim a real construction can make [@bertoni-sponge-indiff-eurocrypt08]. And a related limit from Section 4 still bites: Antoine Joux's multicollisions mean concatenating two hashes does not multiply their strength, so there is no cheap "hash twice" route to a stronger oracle [@joux-crypto04].

Now quantum, where the folklore is loudest and least accurate. Grover's algorithm gives a square-root speedup for search, so a quantum preimage on an $n$-bit hash costs about $2^{n/2}$ instead of $2^n$ [@grover-1996]. That halves the preimage exponent, which sounds alarming until you plug in numbers: SHA-256 keeps about 128 bits of preimage resistance against an ideal quantum adversary, still far beyond reach.

Collisions are the subtler story. Brassard, Hoyer, and Tapp gave a quantum collision algorithm at about $2^{n/3}$ queries, but it demands a comparable amount of quantum-accessible memory [@brassard-hoyer-tapp-latin98]. Daniel Bernstein's cost analysis shows that once you charge honestly for that memory, plain parallel classical collision search at $2^{n/2}$ with cheap hardware wins, so the practical quantum speedup for collisions is marginal [@bernstein-sharcs09]. SHA-256 therefore keeps roughly 128-bit collision resistance in practice, quantum or not.

Function	Classical preimage	Classical collision	Quantum preimage (Grover)	Quantum collision (practical)
SHA-256	$2^{256}$	$2^{128}$	$2^{128}$	about $2^{128}$
SHA-384	$2^{384}$	$2^{192}$	$2^{192}$	about $2^{192}$
SHA-512	$2^{512}$	$2^{256}$	$2^{256}$	about $2^{256}$

The claim "quantum halves every security level" is right for preimage and wrong for collisions. Halving the collision exponent from $2^{n/2}$ to $2^{n/3}$ would matter only if the required quantum memory were free, and it is not. Charge for it, as Bernstein does, and classical machines are more cost-effective at finding collisions, so the honest planning figure for SHA-256 collision resistance stays near 128 bits [@bernstein-sharcs09].

Note: If you want more headroom against a future quantum adversary, you do not need a different construction. You need more output bits. That is the entire reason NIST keeps SHA-384, SHA-512, and the 256-bit-security XOF SHAKE256 in the toolbox: they push both preimage and collision margins up by widening the digest. KT256, defined in the Informational IRTF RFC 9861 rather than a FIPS standard, offers the same raised margin outside the FIPS boundary [@rfc-9861]. The migration story for the rest of cryptography, where the fix really is new algorithms, is covered separately in the post-quantum migration guide.

Two frontiers sit just outside this guide's scope but deserve a pointer, because they are where hashes are heading. The first is arithmetization-oriented hashes such as Poseidon [@poseidon-usenix-sec21] and the Rescue and Vision family [@rescue-tosc-2020], designed to minimize arithmetic-circuit cost inside zero-knowledge proof systems rather than CPU cycles. They are younger, with far less mature cryptanalysis than SHA-2 or SHA-3, so they are a higher-risk choice outside their niche.

The second runs the other direction: hash-based signatures such as LMS [@rfc-8554], XMSS [@rfc-8391], and the standardized stateless SLH-DSA [@fips-205] build post-quantum signatures purely from hash calls, so they consume this primitive rather than being one. Both belong to the post-quantum posts in this series, not here.

So the mathematics delivers a calm verdict. The survivors are as strong as any hash of their output length can be, and quantum computers sharpen the analysis rather than overturning it. That means the last question is not "which hash is strongest," because among the survivors none is meaningfully stronger than another. The question is which function, in which mode, for your specific job.

11. The Practical Guide: Choose X with These Params in Case Y

Everything so far collapses into one habit. Name the clause your protocol actually leans on, then pick the function and mode that keeps it. Here is that decision on a single page.

flowchart TD START["What is the job?"] --> Q0{"Passwords or low-entropy secrets?"} Q0 -->|Yes| ARGON["Argon2id, or scrypt, bcrypt, PBKDF2"] Q0 -->|No| Q1{"Keyed authentication?"} Q1 -->|Yes| HMAC["HMAC-SHA-256, or KMAC for the sponge"] Q1 -->|No| Q2{"FIPS approval required?"} Q2 -->|No| BLAKE["BLAKE3 for throughput, streaming, and KDF"] Q2 -->|Yes| Q3{"Length-extension safety needed?"} Q3 -->|Yes| TRUNC["SHA-512/256, or SHA-3 and SHAKE"] Q3 -->|No| SHA256["SHA-256, the default"]

The head-to-head matrix fills in the details behind those leaves. Speeds are single-thread cycles per byte on a 2023 AMD Zen 4, from the eBASH measurements [@ebash-zen4]; every full function listed is unbroken, so the choice is driven by length-extension behavior, speed, FIPS status, and parallelism, not by any strength gap.

Function	Construction	Output / security	LE-safe	Speed (cyc/B)	FIPS	Parallel	Best for
SHA-256	Merkle-Damgard	256 / 128 coll	No	2.03	Yes	No	Default integrity, signatures, interop
SHA-512	Merkle-Damgard	512 / 256 coll	No	4.19	Yes	No	High margin on 64-bit software
SHA-512/256	Merkle-Damgard (trunc)	256 / 128 coll	Yes	about 4.2	Yes	No	LE-immunity plus FIPS in software
SHA3-256	Sponge	256 / 128 coll	Yes	6.81	Yes	No	Construction diversity, LE-immune
SHAKE128	Sponge XOF	any / 128 ceiling	Yes	5.43	Yes	No	Variable-length output, PQC expansion
BLAKE2b	ARX (HAIFA)	512 / 256 coll	Yes	3.54	No	Tree mode	Fast keyed software hash
BLAKE3	ARX Merkle tree	256, XOF / 128	Yes	0.60	No	Yes	High-throughput integrity, KDF, streaming
KT128	Sponge tree	any / 128	Yes	about 3.5 (0.51 with AVX-512)	No	Yes	High throughput on the Keccak basis
HMAC-SHA-256	Nested keyed MD	MAC / PRF	Yes	about 2x SHA-256	Yes	No	Keyed authentication
KMAC256	Keyed sponge	any / 256	Yes	about SHAKE256	Yes	No	Keyed plus domain-separated auth

And the same advice as a quick "use / never" card, the version you tape above your desk.

Use this	For this job	Never use here
SHA-256	Default integrity, signatures, interop	MD5, SHA-1
SHA-512/256	Length-extension immunity plus FIPS in software	full-width `H(secret\|\|msg)`
SHA3-256 or SHAKE256	Construction diversity, XOF, PQC seed expansion	naive truncation of a fixed hash
BLAKE3	High-throughput, streaming, KDF via `derive_key` (non-FIPS)	passwords
HMAC-SHA-256 or KMAC	Keyed authentication	`H(key\|\|msg)`
Argon2id	Passwords, low-entropy secrets	any bare fast hash
MD5, CRC	Non-security checksums only	signatures, certs, deduplication

A few sizing rules turn those tables into correct code. For a target security level of $s$ bits against collisions, make the digest at least $2s$ bits, because collision resistance is only half the output length; 128-bit collision security therefore needs a 256-bit hash.

When you need a non-standard output length, request it from an XOF (SHAKE, KMAC, or a BLAKE3 XOF) rather than truncating a fixed digest by hand, and remember that reading more XOF bytes never raises you above the capacity ceiling. Add margin, SHA-384, SHA-512, SHAKE256, or KT256, when you want a quantum hedge, per Section 10. And keep the deprecation discipline precise: MD5 and CRC are fine as non-security checksums against accidental corruption, and nowhere an adversary can influence the input.

On Windows the same primitives appear under specific names. The CNG layer exposes hash and MAC providers through BCrypt, detailed in [the CNG architecture guide](/blog/cng-architecture-bcrypt-ncrypt-ksps); Authenticode computes a hash of a portable-executable file (skipping the fields that signing itself modifies) and signs that digest, which is the exact construction Flame subverted, covered in [the Authenticode and catalog-files guide](/blog/authenticode-and-catalog-files-the-crypto-foundation-under-w); and SChannel, the Windows TLS stack, hashes handshake transcripts and has spent two decades retiring SHA-1 across cipher suites, told in [the SChannel algorithm-agility story](/blog/rotating-every-cipher-schannel-and-the-twenty-year-algorithm). The algorithm choices in this guide map onto those APIs directly.

One operational detail sinks more code than any exotic attack: comparing digests or tags with an ordinary equality check. A loop that returns the moment two bytes differ leaks, through its timing, how many leading bytes matched, which is enough for an attacker to recover a secret tag byte by byte. The fix is a constant-time comparison that always scans the whole buffer. This adjacency between hashing and timing side channels is the same family of bug explored in the padding-oracle field guide.

{` // Comparing a secret tag with early-exit leaks how many bytes matched. function naiveEqual(a, b) { if (a.length !== b.length) return false; for (let i = 0; i < a.length; i++) { if (a[i] !== b[i]) return false; // returns as soon as bytes differ } return true; }

// Constant-time: always scan the whole array, OR the differences together. function constantTimeEqual(a, b) { if (a.length !== b.length) return false; let diff = 0; for (let i = 0; i < a.length; i++) { diff |= a[i] ^ b[i]; // no branch on the secret bytes } return diff === 0; }

const tag = [0x9f, 0x1c, 0x00, 0x42, 0xab]; const guess = [0x9f, 0x1c, 0xff, 0x00, 0x00]; console.log("naive:", naiveEqual(tag, guess)); console.log("constant-time:", constantTimeEqual(tag, guess)); console.log("In production call a vetted primitive: crypto.timingSafeEqual, CRYPTO_memcmp, or sodium_memcmp."); `}

Two rules cover most real disasters, and they have nothing to do with picking the "strongest" hash. Never use a bare fast hash for passwords, and never use a bare hash where you meant HMAC. The rest of choosing is about axes that are not strength at all: length-extension behavior, speed, FIPS status, parallelism, and construction diversity. What remains is to clear up the myths that cause the disasters those two rules do not.

12. Misconceptions and Real-Code Smells

The field's folklore is mostly half-right, which is worse than fully wrong, because half-right beliefs survive code review. Here are the questions that actually decide those reviews.

No, and this is the single most common misconception in the subject. FIPS 202 says its SHA-3 functions supplement the SHA-1 and SHA-2 families rather than replacing them [@fips-202], and NIST's Policy on Hash Functions states plainly that there is "no need to transition applications from SHA-2 to SHA-3" [@nist-policy-hash-functions]. SHA-2 and SHA-3 coexist deliberately, on a diversification axis, so the internet's trust machinery does not rest on one construction. The supersession ladder ran MD5 to SHA-1 to SHA-2 and ended there. Yes. Full-width SHA-256 and SHA-512 are length-extension-vulnerable, because their digest is the whole final chaining state. So is SHA-224, despite being truncated: it hides only 32 bits of state, which an attacker recovers in about $2^{32}$ work before extending normally [@rfc-3874]. The genuinely immune options are SHA-384, SHA-512/224, SHA-512/256, the SHA-3 family and SHAKE, HMAC, KMAC, and the BLAKE hashes. If you need `H(secret || message)` behavior, use HMAC or one of those instead. No. Every collision attack on SHA-1 and MD5 attacks collision resistance, and HMAC does not rely on collision resistance. RFC 2104 states directly that MD5's weaknesses do not compromise its use within HMAC [@rfc-2104]. HMAC-SHA-1 and HMAC-MD5 remain unbroken, though new systems should still prefer HMAC-SHA-256 for margin. Only as a non-security checksum guarding against accidental corruption, where no adversary can influence the input. The moment an attacker can choose inputs, MD5 collisions become forged certificates, as the 2008 rogue certificate authority and 2012 Flame both showed [@rogue-ca, @stevens-counter-cryptanalysis-crypto13]. Never use MD5 for signatures, certificates, or deduplication of adversarial data. No. Grover's algorithm halves the preimage exponent, leaving SHA-256 with about 128-bit preimage resistance, still infeasible [@grover-1996]. For collisions the quantum speedup is marginal once you charge for quantum memory, so SHA-256 keeps roughly 128-bit collision resistance in practice [@bernstein-sharcs09]. The hedge, if you want more margin, is a longer output such as SHA-384 or SHA-512, not a new construction. SHA-256 for general integrity, signatures, and interoperability, especially where FIPS validation matters [@fips-180-4]. BLAKE3 when you want maximum throughput, streaming, or a KDF and do not need FIPS [@blake3-repo]. SHA-512/256 when you want length-extension immunity plus FIPS on 64-bit software. No. Both are built to be fast, which is exactly what you do not want for passwords, where speed helps the attacker guess. Use a purpose-built slow function: Argon2id for new systems, or scrypt, bcrypt, or PBKDF2. BLAKE3's own README says the same and points to Argon2 [@blake3-repo].

The misuse catalog is the FAQ's mirror image: the same mistakes, seen from the code side.

Note: H(secret || message) pressed into service as a MAC (use HMAC or KMAC). Passwords fed to a bare or unsalted fast hash (use Argon2id). Truncating a fixed digest to an odd length instead of asking an XOF (use SHAKE or a BLAKE3 XOF). Comparing tags with == or an early-exit loop instead of a constant-time check.These last two, ad hoc truncation and non-constant-time comparison, are the ones that pass functional tests perfectly and fail only under an adversary, which is why they survive so long in production. Rolling your own Merkle tree without domain-separating leaf and internal nodes. Treating a hash as if it provided encryption. And assuming a real hash is a perfect random oracle rather than an approximation of one.

Return, finally, to the two files from the opening, the ones that shared the SHA-1 digest 38762cf7f55934b34d179ae6a4c80cadccbb7f0a [@shattered]. You can now read that string precisely. It is not evidence that hashing is broken, or that longer is always safer, or that SHA-3 rode in to replace the fallen. It is a single clause, collision resistance, failing on a single aging construction, and being weaponized because signatures sign the digest, not the file. The answer the field actually gave was not one replacement but two moves at once: widen the margins of the survivor, SHA-2, and diversify into unrelated constructions, the SHA-3 sponge and the BLAKE3 tree, so no future break can take everything down together. A cryptographic hash makes one promise. Mastery is knowing exactly which clause of that promise your protocol leans on, and choosing the function and the mode that keeps it.

No Room for a Nonce: Disk and Length-Preserving Encryption, from XTS to FPE

noreply@paragmali.com (Parag Mali) — Fri, 10 Jul 2026 11:17:52 GMT

When ciphertext must occupy exactly the plaintext's space -- a 512-byte disk sector, a 16-digit card number -- there is no room for a fresh nonce and no room for an authentication tag, so encryption collapses back to a deterministic, unauthenticated keyed permutation whose only achievable goal is to look like a random permutation. Its quality is set by two knobs: **block width** (how far one changed bit spreads -- narrow AES-XTS versus wide Adiantum and HCTR2) and **domain size** (how many values it hides among -- huge for a disk, fatally small for format-preserving encryption). Every real-world break sorts onto one knob: XTS malleability and replay come from too-narrow width; the FF3 and FF3-1 message-recovery attacks come from too-small a domain, where recovering the whole codebook costs work merely *polynomial* in the domain rather than the 2^128 the cipher seems to promise. And because a length-preserving cipher can never reject a forgery, integrity must always come from another layer -- dm-integrity, dm-verity, fscrypt's per-file nonce, or an AEAD wherever you are actually allowed to spend the extra bytes.

1. Two Failures, One Cause

A security team did everything the checklist demanded. They encrypted a server's disk with AES-XTS, the mode NIST approved for storage devices [@nist-sp800-38e], and confirmed the volume was unreadable without the key. Months later an attacker with write access to that volume rolled the machine back to a vulnerable kernel, flipped a policy byte, and corrupted a specific database page -- never learning a single byte of plaintext, never touching the key, and never tripping an alert. The disk was still, technically, encrypted the entire time.

Two floors down, a different team faced a legacy analytics pipeline that would only accept six-digit codes -- a short, PIN-length field of exactly the kind format-preserving encryption was built to protect. They enciphered each real code into a different but still-valid six-digit code and moved on.

An analyst with query access to the encryptor then rebuilt the entire mapping from fewer query pairs than there are codes -- a few hundred thousand, an afternoon's compute -- and read every value behind it [@durak-vaudenay2017][@hoang-miller-trieu2019]. That is the same weakness that broke the FF3 standard outright, and Section 8 will put exact numbers on it.

Two very different systems, two very different failures. They share one root cause, and naming it is the whole point of this article.

Encryption whose ciphertext occupies exactly the same space as its plaintext, leaving no room for a fresh nonce and no room for an appended authentication tag.

Both teams were forced into length preservation. A disk sector is a fixed 512 or 4096 bytes with no spare room; a code field is exactly six digits wide.

When you cannot add even one byte, two tools that every modern cipher leans on vanish at once. You lose the nonce, the fresh random input that makes encrypting the same message twice produce different ciphertext. And you lose the tag, the short checksum an authenticated cipher appends so the receiver can reject anything tampered with. Remove both, and encryption collapses back to its oldest form: a single keyed permutation on a fixed set of values.

Key idea: Length preservation forces two properties at once. The cipher is deterministic -- identical plaintext at the identical position under the identical key always yields identical ciphertext -- so the best security you can even ask for is that it look like a random permutation, never the semantic secrecy of a randomized cipher. And it is unauthenticated -- with no tag there is nothing to reject, so it is malleable by construction.

The property that an attacker can transform a ciphertext into another valid ciphertext, changing the decrypted plaintext in a related or randomized way, with nothing in the cipher able to detect that the change happened.

That is why the first team's disk could be rewritten by someone who could not read it. There was no tag to say "no." And it hints at why the second team's codes fell: a permutation is only as good as the space it hides in is large. Everything that follows lives on exactly two knobs.

flowchart TD Root["Every length-preserving cipher is one point in width by domain space"] Root --> Disk["Large domain: a whole disk or volume"] Root --> Field["Small domain: a fixed-format field"] Disk --> XTS["Narrow width, AES-XTS, 16-byte diffusion"] Disk --> Wide["Wide width, Adiantum and HCTR2, whole-unit diffusion"] Field --> FPE["Format-preserving, FF1 and FF3-1, domain is the ceiling"]

Block width is how far a single changed input bit diffuses through the output. Domain size is how many possible values the permutation is defined over. Narrow width and a large domain give you AES-XTS; wide width and a large domain give you Adiantum and HCTR2; a deliberately tiny domain, at any width, gives you format-preserving encryption.

Format-preserving encryption was designed for exactly the small field the second team reached for -- a PIN, a short code, the six middle digits of a card number a processor still has to parse -- which is why that column fell so cheaply. A full sixteen-digit card number, it turns out, is far more expensive to recover, because the cost scales with the size of the domain, not with the format; Section 8 quantifies both.

This is Part 9 of the series. It assumes you take AES as a pseudorandom permutation for granted, that you met authenticated encryption in Part 7, and that you have the security definitions from Part 1. To see why the whole field reduces to these two knobs, start where both problems were born: in a disk sector and a database column that both refused to grow.

2. Two Problems With the Same Shape

Rewind to the turn of the millennium. Two engineers, in two industries that never spoke to each other, hit the same wall.

The first was a storage engineer. A disk is an array of fixed sectors, and the encryption layer sits underneath the filesystem, transparent to it. Sector number 8,675,309 must encrypt to exactly 512 bytes and decrypt back in place, with no per-sector header to stash an initialization vector. So where does the IV go?

The naive answer, Electronic Codebook mode, encrypts each 16-byte block independently under one key. It needs no IV at all -- and it leaks catastrophically. Identical plaintext blocks produce identical ciphertext blocks everywhere on the disk, so the structure of files, repeated records, and zeroed regions survives encryption in plain sight.This is the famous "ECB penguin": encrypt a bitmap of the Linux mascot Tux in ECB mode and the penguin is still perfectly visible in the ciphertext, because the cipher preserves every repeated block [@wikipedia-ecb-penguin].

Cipher Block Chaining with a per-sector IV helps, but if that IV is predictable an attacker can mount watermarking attacks -- planting files whose ciphertext betrays their presence -- and CBC on a disk stays malleable end to end. Clemens Fruhwirth's 2005 survey of hard-disk encryption catalogs exactly these failure modes [@fruhwirth2005].

The second engineer worked in payments. A legacy mainframe validates that a primary account number is sixteen digits and passes a checksum; it will reject anything else. You must encrypt that field so downstream systems still see a well-formed account number. Practitioners called this "datatype-preserving encryption" and traded folklore recipes through the 1990s.

In 2002 John Black and Phillip Rogaway gave the problem a real security model in a paper titled Ciphers with Arbitrary Finite Domains [@black-rogaway2002]. They contributed two reusable techniques that every later scheme inherits: cycle-walking (encipher a value under a larger cipher and re-encrypt until the result lands back in the target set) and a Feistel network over a radix (build a permutation on the integers modulo N from keyed round functions).

A keyed permutation that no efficient adversary can distinguish from a truly random permutation of the same set. It is a *strong* PRP if that indistinguishability survives even when the adversary may also query the inverse -- the decryption -- direction, not just encryption.

Here is the observation that unifies the two stories. When you cannot spend a single byte, you are not choosing between encryption modes; you are choosing a permutation on exactly this space. And the theory for that had already been written.

In 1999 Moni Naor and Omer Reingold, revisiting the classic Luby-Rackoff construction, named the right target -- a strong pseudorandom permutation -- and showed that a wide one, enciphering a whole block at once, could be built cheaply from simpler parts [@naor-reingold1999]. That result sat mostly unused for two decades. Both the disk world and the payments world were, without knowing it, asking Naor and Reingold's question.

A cipher whose output has the same format as its input, so a sixteen-digit account number enciphers to another sixteen-digit account number, and a value drawn from a set of size N enciphers to another value in that same set. timeline title The two rails of length-preserving encryption, 1999 to 2025 1999 : Naor-Reingold, a wide strong PRP can be built cheaply 2002 : Black-Rogaway formalize FPE : LRW names the tweak 2004 : XEX fixes the tweak for disks 2007 : XTS-AES standardized for storage 2016 : NIST approves FF1 and FF3 2017 : FF3 broken by Durak-Vaudenay 2018 : Adiantum published, wide-block from hash-encrypt-hash 2021 : HCTR2 published : FF3-1 broken by Beyne 2025 : NIST removes FF3 and FF3-1

The nonce that length preservation takes away is the subject of Part 2; here it is simply gone. Both camps reached for the same fix at almost the same moment -- a tweak, a public per-position input that re-keys the permutation cheaply for each sector or each field. It worked. Then it failed, in a way that taught the field its first hard lesson about where key material is allowed to appear.

3. The First Answers, and Why They Broke

The first generation of real answers shared a blind spot: they confused diffusion with security, and format with strength. Three named near-misses show the confusion, and each taught a lesson the survivors still carry.

ESSIV: a real fix that fixed the wrong thing

Fruhwirth's ESSIV -- Encrypted Salt-Sector IV -- attacks the predictable-IV problem head on. Instead of deriving the CBC initialization vector from the visible sector number, it computes $\mathrm{IV} = E_{\mathrm{salt}}(\text{sector})$ with $\mathrm{salt} = H(K)$, so the IV is a secret function of the key and an attacker can no longer predict it [@fruhwirth2005]. Watermarking dies.

But ESSIV is a fix to confidentiality leakage, not to tampering: the underlying CBC is still malleable, and its diffusion runs only forward, so a flipped ciphertext bit corrupts one block and predictably flips the corresponding plaintext bit in the next. ESSIV made the disk harder to read. It did nothing to make it harder to rewrite.

The Elephant diffuser: a diffuser is not a MAC

Microsoft's first BitLocker, shipped in Windows Vista, saw the malleability problem and answered it with engineering muscle. Niels Ferguson's 2006 design bolts an unkeyed, invertible diffuser -- nicknamed Elephant -- onto AES-CBC, explicitly "to improve security against manipulation attacks" [@ferguson2006]. The diffuser spreads any single changed bit across the entire sector, so tampering yields sector-wide noise instead of a clean bit-flip. That feels like integrity. It is not.

An unkeyed diffuser randomizes the plaintext when the ciphertext is altered, but it authenticates nothing: every altered sector still decrypts to *some* 512 bytes, and there is no tag to reject them. It converts an attacker's chosen edit into unpredictable garbage -- a real downgrade in attacker control -- but it can never say "this sector was tampered with." Microsoft eventually accepted the verdict and moved BitLocker to XTS-AES, retiring the Elephant diffuser entirely [@microsoft-bitlocker]. The deployment history is told in full in the [BitLocker architecture post](/blog/bitlocker-on-windows-architecture-attacks-and-the-limits-of-/).

LRW: a cipher defeated by encrypting its own key

The line that actually led somewhere abandoned chaining for a tweak. In 2002 Moses Liskov, Ronald Rivest, and David Wagner formalized the idea [@liskov-rivest-wagner2002].

A block cipher with an extra public input, the *tweak*, that cheaply selects a different permutation for each value of the tweak. On a disk the tweak is the position, so every sector or block is enciphered under its own effectively independent permutation without rekeying.

Their LRW mode computes $C = E_{K_1}(P \oplus \Delta) \oplus \Delta$ with the mask $\Delta = K_2 \otimes i$, where $K_2$ is a second key, $i$ is the block index, and $\otimes$ is multiplication in the finite field $\mathrm{GF}(2^{128})$. Each block gets its own location-bound permutation, fully parallel and random-access. It was the right shape -- and it carried a fatal detail.

The mask is built from raw key material. On a disk that is not hypothetical: a full-disk-encryption key can be paged or hibernated onto the very disk it protects, so a plaintext block can equal the tweak key $K_2$ itself. And because the mask is a linear function of $K_2$, a handful of such known blocks are enough to recover it.

It is an unusually instructive failure: the defect is structural, not a tuning mistake. No larger key and no extra round removes it, because the weakness lives in *where the mask comes from* rather than in how strong the underlying cipher is. A construction that a swap file can quietly undo cannot anchor full-disk encryption, so LRW was superseded rather than patched.

FPE's first generation: format is not strength

On the payments rail, Black and Rogaway's cycle-walking and Feistel-over-radix were the first rigorous constructions [@black-rogaway2002]. They are elegant and correct -- and they quietly encode the domain problem that will dominate the rest of this article.

Encipher a value from a small set by encrypting it under a larger cipher and re-encrypting the result until it lands back inside the target set. Each re-encryption is a "walk," and because the larger cipher is a permutation, the composed map is a permutation on the small set.

The demo below enciphers four-digit values into four-digit values by cycle-walking a toy 16-bit permutation down into the range 0 to 9,999. Notice two things: the output is always a valid four-digit number, and the same input always maps to the same output. Format is preserved, determinism is total -- and the whole security rests on how large that range is.

{` // Enciphering 4-digit values using a bigger permutation, then walking back into range. // The toy 16-bit map is a Feistel network, which is a bijection for ANY round function. function roundF(half, k) { return ((half * 167) ^ (k * 91) ^ ((half << 3) & 0xFF)) & 0xFF; } function feistel16(x, keys) { let L = (x >> 8) & 0xFF, R = x & 0xFF; for (let r = 0; r < 4; r++) { const nL = R, nR = L ^ roundF(R, keys[r]); L = nL; R = nR; } return ((L << 8) | R) & 0xFFFF; } const KEYS = [23, 57, 131, 200]; const N = 10000; // the 4-digit domain: values 0000..9999

function fpeEncrypt(x) { let y = feistel16(x, KEYS), walks = 0; while (y >= N) { y = feistel16(y, KEYS); walks++; } return { y: y, walks: walks }; }

for (const v of [1, 42, 1234, 9999]) { const r = fpeEncrypt(v); console.log(String(v).padStart(4,'0') + ' -> ' + String(r.y).padStart(4,'0') + ' (walks: ' + r.walks + ')'); } console.log('Deterministic: encrypting 1234 twice gives ' + fpeEncrypt(1234).y + ' both times.'); `}

LRW's lesson was precise: never let key material into the mask; derive the offset from the cipher itself. That one idea, written $\Delta = E_K(i) \cdot \alpha^j$, is the hinge on which the entire disk rail now turns.

4. The Climb, Generation by Generation

Now the climb begins. Two rails, and every generation is motivated by a concrete, named failure of the one before it.

Approach	Year	Key idea	Status
ECB per sector	~1998	no IV, blocks enciphered independently	broken, leaks patterns
CBC with ESSIV	2005	secret sector IV kills watermarking	superseded
Elephant diffuser	2006	unkeyed whole-sector diffusion on CBC	removed for XTS
LRW	2002	tweak offset from a second key	superseded, key leak
XEX	2004	offset from the cipher, powers of alpha	superseded by XTS
XTS-AES	2007	XEX plus two keys plus ciphertext stealing	the disk default
CMC / EME / EME2	2003-2004	wide-block SPRP, about two passes	niche, academic
Adiantum	2018	hash-encrypt-hash, no AES needed	deployed
HCTR2	2021	hash-encrypt-hash, AES-accelerated	deployed for filenames
Black-Rogaway FPE	2002	cycle-walking, Feistel over a radix	foundation
FF1 (from FFX)	2010 / 2016	10-round Feistel, large tweak	approved, sole survivor
FF3 (from BPS)	2010 / 2016	8-round Feistel, 64-bit tweak	broken 2017, removed 2025
FF3-1	2019 draft	FF3 with tweak shrunk to 56 bits	broken 2021, removed 2025

From XEX to XTS: the offset that carries no key

Phillip Rogaway's 2004 XEX construction applies LRW's lesson exactly [@rogaway-xex2004]. It uses a single key and derives the per-block offset from the cipher: encipher the data-unit number once, then step the offset across block positions by repeatedly multiplying by the field element alpha. No key material ever enters the mask.

flowchart LR I["Data-unit number i"] --> E2["Encrypt i under key K2"] E2 --> A["Multiply by alpha, once per block position j"] A --> D["Offset for block j"] P["Plaintext block Pj"] --> X1["XOR with the offset"] D --> X1 X1 --> E1["Encrypt under key K1"] E1 --> X2["XOR with the offset again"] D --> X2 X2 --> C["Ciphertext block Cj"]

The multiply-by-alpha step is why XTS is fast and parallel.Multiplying by alpha in this field is just a one-bit left shift of the 128-bit offset followed by a conditional XOR with the reduction constant 0x87 when the top bit overflows. No general multiplier is needed, and every block's offset can be computed independently. The simulator below advances the offset exactly this way and enciphers the same plaintext block at four consecutive positions. The ciphertext differs every time -- position binding -- while re-running the demo reproduces it exactly -- determinism.

{` // Advance the per-block offset by multiply-by-alpha in GF(2^128), // then encipher each block as C = E(P XOR offset) XOR offset. function mulAlpha(t) { // t: 16-byte little-endian offset let carry = 0; for (let i = 0; i < 16; i++) { const b = t[i]; t[i] = ((b << 1) | carry) & 0xFF; carry = b >>> 7; } if (carry) t[0] ^= 0x87; // reduction for the field GF(2^128) return t; } function toyCipher(block, key) { // a stand-in permutation with diffusion, NOT real AES const out = Uint8Array.from(block); for (let r = 0; r < 3; r++) { let acc = (key + r * 97 + 1) & 0xFF; for (let i = 0; i < 16; i++) { acc = (acc + out[i] + key + i * 31) & 0xFF; out[i] = (out[i] ^ acc ^ 0x5A) & 0xFF; } for (let i = 1; i < 16; i++) out[i] = (out[i] + out[i - 1]) & 0xFF; } return out; } function hex(a) { return Array.from(a, b => b.toString(16).padStart(2, '0')).join(''); }

const seed = new Uint8Array(16); seed[0] = 0x01; // stands in for Encrypt(i) under K2 let offset = Uint8Array.from(seed); const plaintext = new Uint8Array(16).fill(0xAA); // the SAME plaintext at every position

for (let j = 0; j < 4; j++) { const masked = new Uint8Array(16); for (let i = 0; i < 16; i++) masked[i] = plaintext[i] ^ offset[i]; const enc = toyCipher(masked, 0x3C); const ct = new Uint8Array(16); for (let i = 0; i < 16; i++) ct[i] = enc[i] ^ offset[i]; console.log('position ' + j + ' offset ' + hex(offset).slice(0,12) + '.. ciphertext ' + hex(ct).slice(0,12) + '..'); mulAlpha(offset); // step to the next position } console.log('Identical plaintext, different position, different ciphertext.'); `}

XTS-AES, standardized as IEEE 1619-2007 and adopted by NIST as SP 800-38E, turns XEX into a shippable disk mode [@ieee1619-2007][@nist-sp800-38e]. It uses two independent keys -- one to encipher, one to derive the tweak from the data-unit number -- and adds a trick for data units whose length is not a whole number of blocks.

A technique that lets a block cipher encrypt a message whose length is not a multiple of the block size without expanding it, by borrowing ciphertext bytes from the second-to-last block to pad the final short block. The output length exactly equals the input length.

NIST caps a single XTS data unit at $2^{20}$ AES blocks, or 16 MiB [@nist-sp800-38e]. And an "AES-256-XTS" key is therefore two 256-bit keys, 512 bits in total -- a fact that trips up more implementers than any other on this list. XTS is excellent at what it does, but three residual limits, all inherent to a narrow block, drive everything after it: it diffuses across only one 16-byte block, it is unauthenticated, and it leaks equality of identical blocks at identical positions.

Note: An attacker with write access to an XTS volume cannot paste your plaintext around or flip chosen plaintext bits the way a CTR or CBC gadget allows. What they can do, all undetected because there is no tag: overwrite a 16-byte block to produce fresh random garbage on decryption, replay an older ciphertext image of the same block at the same position to roll it back, or randomize any block at 16-byte granularity. It is destruction and rollback, not surgical editing -- but it is entirely unauthenticated.

The wide-block ideal, briefly

If narrow diffusion is the problem, the obvious cure is to encipher the entire data unit as one wide block, so that flipping any single bit re-randomizes the whole sector. Shai Halevi and Phillip Rogaway built exactly this: CMC in 2003 and the parallelizable EME in 2004 are wide-block strong PRPs over a whole variable-length unit [@halevi-rogaway-cmc2003][@halevi-rogaway-eme2004]. They are the ideal that XTS only approximates.

They also cost roughly two block-cipher passes over every byte, with no hardware primitive to accelerate the whole construction the way AES-NI accelerates XTS, and they carried patent clouds. So they never became the disk default, surviving only as a niche (EME2 lives in IEEE Std 1619.2 [@ieee1619-2-2010]).

The wide ideal was known for fifteen years before it was cheap enough to ship -- and the thing that made it cheap was the 1999 Naor-Reingold result waiting in the wings [@naor-reingold1999]. Hold that thought; it is the next section.

The format-preserving rail: FFX, BPS, and the NIST methods

On the payments side, the community turned Black-Rogaway's Feistel-over-radix into concrete, submittable modes with public tweaks and fixed round counts. Two proposals mattered. FFX, from Mihir Bellare, Phillip Rogaway, and Terence Spies, uses a larger tweak and about ten rounds -- the conservative choice [@ffx-spec2010]. BPS, from Eric Brier, Thomas Peyrin, and Jacques Stern, uses a compact per-message tweak and fewer rounds for speed [@bps-spec].

Naming trap: BPS is Brier-Peyrin-Stern, the basis of FF3. It is not "Bellare-Rogaway-Spies" -- that trio wrote FFX, the basis of FF1. The initials collide; the papers do not.

A third submission, VAES3, became the draft method FF2 but was withdrawn before finalization over a security concern.The withdrawal is documented in a 2015 analysis, "Analysis of VAES3 (FF2)" [@dworkin-perlner-vaes3-2015]; when NIST published its standard the next year, only FF1 and FF3 remained. When NIST finalized SP 800-38G in 2016, it approved exactly two methods: FF1, from FFX, with ten Feistel rounds and a large tweak; and FF3, from BPS, with eight rounds and a 64-bit tweak [@nist-sp800-38g-2016]. Both are AES-backed unbalanced Feistel networks over the message radix, and the standard required a minimum domain of at least 100 values, recommending at least one million.

flowchart TD Split["Digit string split into halves A and B"] --> Round["Round function: AES-based PRF over round number, tweak, and half B"] Round --> Combine["Add the PRF output into half A, modulo radix to the m, the half's range"] Combine --> Swap["Swap the halves and repeat for every round"] Swap --> Out["Recombine into a same-length digit string"]

The difference that matters downstream is precisely FF3's smaller tweak and fewer rounds versus FF1's larger tweak and ten rounds [@nist-sp800-38g-2016]. FF3 bought speed by spending security margin. Within a year, someone came to collect.

Each rail had reached a plateau. XTS was narrow; FF3 was fast and fragile. What the field lacked was not another cipher but a reframing -- one that would explain both plateaus at once and, in the same move, make the wide permutation cheap enough to deploy.

5. The Right Target Was a Permutation All Along

Step back from the individual ciphers and ask what you are actually allowed to want. Not the secrecy of a randomized message -- there is no randomness. Not integrity -- there is no tag. What you can ask for is the best keyed permutation over this exact space, one that behaves like a randomly chosen permutation even against an adversary who can query it forward and backward.

That is a tweakable strong pseudorandom permutation, and once you name it, the whole field snaps into focus. XTS is a narrow approximation of it. CMC and EME are wide but expensive versions of it. Every FF method is a version of it on a tiny domain.

Whether one changed input bit diffuses across the entire data unit (wide) or only within its own 16-byte block (narrow). A wide-block cipher enciphers the whole sector as a single indivisible permutation.

Two ideas define the modern state of the art, one for each rail. The first is how to make the wide permutation cheap. Naor and Reingold's 1999 insight was that you do not need two full cipher passes to get a wide strong PRP; you can sandwich a small amount of real cipher work between two passes of fast keyed hashing [@naor-reingold1999]. This is the hash-encrypt-hash shape, and it is the skeleton of both deployed wide-block modes.

flowchart LR P["Whole data unit, plaintext"] --> H1["Keyed universal hash over the whole unit"] H1 --> Mid["One narrow primitive call on a short middle block"] Mid --> H2["The same keyed hash, applied again"] H2 --> C["Whole data unit, ciphertext"]

Adiantum instantiates it with NH and Poly1305 hashing wrapped around a single AES call and an XChaCha12 stream cipher [@adiantum2018]. HCTR2 instantiates it with POLYVAL hashing around an XCTR-over-AES core [@hctr2-2021]. The payoff is the second surprise of this article: the wide ideal is not intrinsically expensive. Adiantum runs at 10.6 cycles per byte on an ARM Cortex-A7, over five times faster than AES-256-XTS implemented in software on the same chip [@adiantum2018]. The mode that gives you whole-sector diffusion is faster than narrow XTS on hardware that lacks an AES instruction.

Why one narrow call is enough, and why it is strong

It is easy to assert that Adiantum and HCTR2 are strong pseudorandom permutations. It is more useful to see why, because the reason is the entire point of the hash-encrypt-hash shape. Take Adiantum concretely. Split the data unit into a long left part $P_L$ (the bulk of the sector) and a single block-sized right part $P_R$. Four steps encipher it:

$$P_M = P_R \boxplus H_k(T, P_L), \qquad C_M = \mathrm{AES}(P_M),$$

$$C_L = P_L \oplus \mathrm{stream}(C_M), \qquad C_R = C_M \boxminus H_k(T, C_L).$$

Here $H_k$ is a fast keyed almost-universal hash (NH plus Poly1305 in Adiantum), $T$ is the tweak, $\boxplus$ and $\boxminus$ are addition and subtraction on one block, and $\mathrm{stream}$ is a stream cipher (XChaCha12) seeded by the single enciphered block. Read it as three beats.

The first hash layer buys unpredictability. Because $H_k$ is keyed and depends on the entire left bulk, the input $P_M$ to the one real cipher call is a secret function of the whole message. An adversary cannot steer that middle input: change any part of $P_L$ and $P_M$ jumps unpredictably, so two distinct queries land on essentially independent middle inputs.

The middle is why one narrow pass suffices. The hash has already done the wide mixing, so the cryptographic core only has to supply pseudorandomness on a single block. That is why one AES call can stand in for a full wide-block cipher pass -- the expensive part was never the enciphering, it was the diffusion, and cheap keyed hashing does the diffusion.

The second hash layer spreads the result back out. Running the stream cipher over the bulk and folding the enciphered block back through the hash makes one changed input bit rewrite the entire unit -- wide-block avalanche, built from one block-cipher call and two linear-time hash passes.

Now the part that makes it a strong PRP, secure even against an adversary who queries decryption. The construction is symmetric and invertible, so the identical "you cannot steer the middle" argument runs backward for inverse queries: a decryption query is just as unable to force a chosen middle block as an encryption query is.

An adversary mixing adaptive forward and inverse queries can win only by making two queries collide on the same middle block $P_M$ -- and because $H_k$ randomizes that block, a collision is a birthday event of probability about $q^2 / 2^b$ for $q$ queries and a $b$-bit block. That two-directional birthday bound is strong-PRP security; it is Adiantum's "provable quadratic advantage bound" [@adiantum2018].

This is Naor and Reingold's insight in operational form: two universal-hash layers do the work of the outer rounds of a Feistel network, so a single cryptographic core reaches the strong PRP that Luby-Rackoff needs four Feistel rounds -- and CMC or EME need two full cipher passes -- to reach [@naor-reingold1999].

"HCTR2 is a tweakable super-pseudorandom permutation: any change to the plaintext will result in an unrecognizably different ciphertext." -- the google/hctr2 project README [@google-hctr2-repo]

The second idea is the hard one, and it lives on the format-preserving rail. There is no reframing that saves small-domain FPE, because the enemy is not the cipher's construction; it is the size of the set.

The number of possible inputs a permutation is defined over. For format-preserving encryption it is a hard security parameter, not a tuning knob: below a floor, the permutation can be reconstructed by enumeration no matter how sound the round function is.

So the community's real progress on FPE was an act of acceptance: minimum-domain floors are not conservative style, they are load-bearing defenses, and a field too small to defend should not be format-preserved at all. That verdict is the subject of Section 8. For now, hold both realizations together.

Key idea: Wide-block enciphering is the disk field finally reaching the ideal permutation that XTS only approximated -- and doing it cheaply. But reaching the ideal permutation changes nothing about integrity: even a perfect tweakable SPRP has no tag, so tampering still produces plaintext the layer above will consume. Wide-block fixes diffusion. It does not, and cannot, add authentication.

So the wide permutation shipped. Which raises the obvious question, and its genuinely surprising answer: if wide-block is provably better and can run faster, why is the disk in front of you -- and the phone in your pocket -- still enciphering its contents with narrow XTS?

6. What Actually Ships in 2026

Here is the deployed map, and its most honest surprise: the better cipher shipped, but not where you would guess. The wide-block permutation went to phones -- for filenames and for chips without an AES instruction -- while the contents of nearly every encrypted volume on Earth are still enciphered with narrow XTS.

AES-XTS -- the mode, not a fixed key size -- is the universal full-volume default. BitLocker has defaulted to XTS-AES since Windows 10 version 1511 [@microsoft-bitlocker]. Apple states plainly that "FileVault uses the AES-XTS data encryption algorithm to protect full volumes on internal and removable storage devices" [@apple-filevault]. On Linux, LUKS and dm-crypt default to aes-xts-plain64 [@luks-cryptsetup-faq]. Three operating systems, three vendors, one narrow-block mode.

What is not universal is the key size, and conflating the two is a frequent error: BitLocker and FileVault default to AES-128-XTS -- Microsoft's own menu lists "XTS-AES 128-bit (default)" [@microsoft-bitlocker-autopilot], and FileVault's often-quoted "256-bit key" is the two-key XTS total of two 128-bit keys -- while LUKS and dm-crypt default to AES-256-XTS, a 512-bit volume key, because an XTS key size is twice the underlying AES key [@cryptsetup-luksformat-man]. The BitLocker architecture post covers the Windows side of this in depth.

File-based encryption is where wide-block actually ships, and its details are the single most mis-stated fact in this area. In Android and Linux fscrypt, on hardware with AES acceleration, file contents use aes-256-xts and filenames use aes-256-hctr2; the AOSP documentation says HCTR2 has been the "preferred mode of filenames encryption for devices with accelerated cryptography instructions" since Android 14, and is planned to become the default [@aosp-fbe]. On hardware without AES acceleration, Adiantum encrypts both contents and filenames [@aosp-fbe][@aosp-adiantum].The Adiantum announcement is often mis-cited. Its actual title is "Introducing Adiantum: Encryption for the Next Billion Users" [@google-adiantum-blog]; the "entry-level processors" phrasing comes from the research paper's subtitle, not the blog post.

Note: Read that deployment carefully. HCTR2's shipped niche is filename encryption, where its whole-unit diffusion hides the length and structure of names cheaply. It does not encrypt production disk-sector contents on AES hardware -- XTS does. Wide-block did not replace XTS for contents anywhere in mainstream deployment as of 2026.

On the payments rail, the map collapsed to a single survivor. NIST's second public draft of SP 800-38G Revision 1, dated February 2025, states that "the encryption method FF3 is no longer specified," leaving FF1 as the sole NIST-approved format-preserving method [@nist-sp800-38gr1-2pd2025].

Integrity is a first-class part of the picture, not a footnote

Because none of these permutations authenticate anything, real systems bolt integrity on at an adjacent layer, and that layer is genuinely part of the state of the art:

dm-verity builds a Merkle hash tree over a read-only volume and verifies every block on read against a trusted root hash. It is how Android Verified Boot detects any modification of a system image [@kernel-dm-verity].
dm-integrity keeps a per-sector integrity journal for read-write volumes; combined with dm-crypt it detects modification -- alter the device and the read returns an I/O error instead of forged plaintext -- at a storage and write cost [@kernel-dm-integrity]. It does not stop rollback or replay: the per-sector tag has no freshness anchor, so restoring a sector's own earlier authentic ciphertext-and-tag pair still verifies. Anti-rollback needs an externally-anchored root, whose options Section 9 spells out in full.
fscrypt's per-file nonce is a random 16-byte value per file, fed through HKDF-SHA512 to derive a per-file key, so identical plaintext in two different files does not produce identical ciphertext [@kernel-fscrypt]. It restores the per-unit uniqueness that a bare deterministic cipher throws away.

flowchart TD Data["Data at rest"] --> Conf["Confidentiality: XTS, Adiantum, or HCTR2, with no tag"] Conf --> Q{"Is tamper in scope?"} Q -->|"read-only image"| Verity["dm-verity: read-only Merkle root, resists rollback"] Q -->|"read-write volume"| Integ["dm-integrity: modification only, no freshness anchor"] Q -->|"per-file uniqueness"| Nonce["fscrypt 16-byte nonce"] Q -->|"room to expand"| AEAD["Use an AEAD one layer up"]

The engineering even shows in the key management: an Adiantum key carries more per-key overhead than an AES-256-XTS key, which is why fscrypt offers a DIRECT_KEY option to amortize it.This is the kind of detail that reveals which mode a platform expects to be common: the optimization exists because Adiantum keys are the expensive case [@kernel-fscrypt]. The map now shows three disk permutations and one surviving FPE method, each optimizing something different. To choose among them you need the head-to-head -- and one decisive question that sits above all four.

7. Narrow, Wide, or Authenticated

The question everyone asks is "which length-preserving cipher should I use?" The better question sits one layer up. Start with the disk contenders head to head.

Dimension	AES-XTS	Adiantum	HCTR2	dm-integrity plus dm-crypt, or an AEAD
Block width	narrow, 16 bytes	wide, whole unit	wide, whole unit	not applicable
Security target	narrow tweakable SPRP	wide tweakable SPRP	wide tweakable SPRP	authenticated encryption
Tamper behavior	position-bound garbage or rollback	whole-unit garbage	whole-unit garbage	modification detected and rejected
Rollback or replay	undetected	undetected	undetected	still undetected -- needs an externally-anchored root
Same-offset leakage	leaks equal 16-byte blocks	none within a unit	none within a unit	none
Hardware acceleration	AES-NI, ARMv8 CE	none required	AES-NI, CLMUL, ARMv8 CE	depends
Ciphertext expansion	zero	zero	zero	nonzero, a tag or journal
Deployed status	universal disk default	non-AES hardware	filenames, Android 14+	integrity layer

The table has a punchline. On AES hardware, XTS and HCTR2 are close in speed, and HCTR2's only advantage is diffusion -- better tamper behavior and no same-offset leak -- never authentication [@hctr2-2021][@nist-sp800-38e]. If tampering is in your threat model, none of the three columns on the left answers it. Only the right-hand column does.

Key idea: "Which length-preserving cipher?" is downstream of a prior question: am I actually forced to be length-preserving, or can I add a tag one layer up? If you can spend even a few bytes, the honest answer is not a better permutation but authenticated encryption -- the subject of Part 7 of this series. Length-preserving modes are what you reach for only when expansion is genuinely impossible.

That framing is exactly why 2026 deployment keeps XTS for contents (fast, accelerated, random-access) and adds wide-block only where it cheaply helps -- HCTR2 for filenames, Adiantum where there is no AES instruction -- while putting authentication in dm-verity, dm-integrity, or a per-file nonce [@kernel-dm-integrity].

FF1 versus FF3-1, calibrated

On the format-preserving rail the head-to-head is narrower, and it is easy to read it wrong.

Dimension	FF1	FF3-1
Feistel rounds	10	8
Tweak	large	56-bit, reduced from FF3's 64
Structural weakness	small-domain recovery, shared by all FPE [@hoang-tessaro-trieu2018]	a linear distinguisher on the tweak schedule [@beyne2021], plus small-domain recovery
NIST status, Feb 2025	approved, sole survivor [@nist-sp800-38gr1-2pd2025]	removed [@nist-sp800-38gr1-2pd2025]
Verdict	most defensible FPE, still not for small fields	migrate off

Do not read this as "FF1 secure, FF3-1 broken, done." All small-domain format-preserving encryption leaks; FF1's larger tweak and ten rounds simply give it a wider margin, not immunity. FF1 is the more defensible choice, not a substitute for a genuinely large-domain permutation. The break that removed FF3-1 was structural to the FF3 family, which is why no tweak-size tuning saved it.

For a field small enough that any permutation is enumerable -- a PIN, a short code, a low-cardinality category -- the honest engineering answer is not a better cipher but a **tokenization vault**: replace the sensitive value with a random surrogate and keep the mapping in a secured lookup table. A vault sidesteps the domain wall entirely, because a random surrogate reveals nothing about its input. Under compliance regimes like PCI DSS, a vault is frequently the safer and simpler choice than any format-preserving cipher, at the cost of maintaining the lookup [@pci-ssc-tokenization2011].

Every trade-off in both tables bottoms out in the same two limits: block width and domain size. It is time to show that these are not engineering accidents that a cleverer design will someday fix. They are theorems -- and one of them has a measurable height.

8. What Length Preservation Makes Impossible

Everything so far could be read as "we just need a better cipher." We do not. Three of these limits are theorems, and the fourth is a wall no cipher climbs -- but it is a wall you can measure. The security definitions behind them come from Part 1.

First theorem: no expansion means no randomness, so the cipher is deterministic. Semantic security against a chosen-plaintext attacker -- the IND-CPA notion -- provably requires that encrypting the same message twice can produce different ciphertexts. A length-preserving cipher has nowhere to put the randomness that would make that happen, so IND-CPA is unreachable by construction. The best achievable target is indistinguishability from a random permutation, an SPRP. This is not a weakness of XTS; it is the ceiling for anything that refuses to expand.

Second theorem: no expansion means no tag, so there is no integrity. With zero expansion there is nothing to carry a MAC, so every ciphertext decrypts to some plaintext and the cipher can never reject a forgery. A length-preserving cipher can only randomize on tamper (wide-block) or permit position-bound edits (narrow-block). It can never say "no." Authentication is provably outside the primitive and must live at another layer.

Third theorem: a per-position tweak leaks equality at its own granularity -- but be precise about which positions. A deterministic tweakable cipher with a public per-position tweak necessarily reveals when two positions hold identical plaintext. XTS leaks equality of identical 16-byte blocks at the same key, data unit, and block index. It does not leak equality of two identical sectors at different offsets: every data-unit index feeds its own tweak, so the same 512 bytes at sector 10 and sector 20 encipher to completely different ciphertext.

The one place byte-identical plaintext does collide is file-relative. In file-based encryption such as fscrypt, where each file's data-unit index restarts at zero, two byte-identical files would run through the same tweak sequence and encipher identically -- which is exactly why fscrypt derives a distinct per-file key from a random 16-byte nonce, so even identical files no longer collide [@kernel-fscrypt].

This is not a whole-volume claim: under whole-disk dm-crypt with absolute-sector tweaks, two identical files at different offsets get different data-unit numbers and therefore different tweaks, and do not collide -- the only equality leak there is between data units that share the same key and tweak, such as one logical block rewritten in place. Wide-block modes do not remove this addressing-model leak; they only push its boundary out from a 16-byte block to the whole unit.

The fourth limit: the small-domain wall, measured

The deepest cut is on the format-preserving rail, and it is not a theorem about any cipher's construction. It is a theorem about the domain. For a permutation over a set of size $N$, message recovery and codebook reconstruction are achievable in work that is polynomial in $N$. Security does not fall off a cliff at some threshold; it degrades smoothly toward zero as the field shrinks.

The attack line is a decade of steadily sharper results: Bellare, Hoang, and Tessaro gave the first message-recovery attacks that exploit the small domain rather than any flaw in the round function [@bellare-hoang-tessaro2016]; Hoang, Tessaro, and Trieu generalized them in a paper aptly titled The Curse of Small Domains [@hoang-tessaro-trieu2018]; and Hoang, Miller, and Trieu pushed the FF3 attack out to large domains [@hoang-miller-trieu2019].

Put numbers on it, because the numbers are the payoff. Durak and Vaudenay's 2017 break of FF3 -- the one that broke a deployed NIST standard outright -- runs in time about $\mathrm{radix}^{5n/2}$, which for an $n$-digit field of size $D = \mathrm{radix}^{n}$ is $D^{2.5}$, using about $\mathrm{radix}^{11n/12} = D^{11/12}$ adaptively chosen plaintext-ciphertext pairs under two related tweaks [@nist-ff3-news2017][@durak-vaudenay2017].

Two features make this devastating rather than academic. First, the data cost $D^{11/12}$ is sublinear in the domain: you need fewer query pairs than there are values in the field. Second, the attack recovers the round functions themselves, so it yields "any plaintext under any tweak" -- a key-equivalent break, not the exposure of a single record [@nist-ff3-news2017]. NIST's own note put the cost at roughly $2^{75}$ steps for a nine-digit field such as a Social Security number [@nist-ff3-news2017].

Two years later Hoang, Miller, and Trieu improved the running time from $O(N^5)$ to $O(N^{17/6})$ -- on the order of $D^{17/12}$, about $D^{1.42}$ -- and, as their title Attacks Only Get Better promises, extended the break to large domains rather than only tiny ones [@hoang-miller-trieu2019].

The table makes the scaling visceral, and settles one honesty debt the cold open owed you.

Field (domain D)	Adaptive pairs, about D^(11/12)	Total work, Durak-Vaudenay, about D^(2.5)	Total work, Hoang-Miller-Trieu, about D^(17/12)
Six-digit PIN or card-middle, D = 10^6	about 3.2 x 10^5	about 2^50	about 2^30
Nine-digit SSN, D = 10^9	about 1.8 x 10^8	about 2^75	about 2^50
Sixteen-digit PAN, D = 10^16	about 4.6 x 10^14	about 2^133 (exceeds 2^128)	about 2^75

A note on the last column: the $2^{50}$ entry for the nine-digit SSN is Hoang, Miller, and Trieu's concrete reported cost for that imbalanced domain (an uneven split, with constant factors folded in), which sits above the symmetric leading term $D^{17/12} \approx 2^{42}$ at $D = 10^9$. Recomputing the header formula alone therefore gives the smaller exponent; no figure in the table changes [@hoang-miller-trieu2019].

Read the rows in order. A six-digit field -- a PIN, or the six middle digits of a card, exactly the size FF3 was designed for -- has a million points, and recovering its whole codebook takes a few hundred thousand adaptive pairs and on the order of $2^{30}$ operations, an afternoon of compute. That is the cold open's second failure, now with numbers.

But look at the last row: a full sixteen-digit card number lives in a domain near $10^{16}$, and the same polynomial attack costs about $2^{75}$ by the best known method and about $2^{133}$ by Durak-Vaudenay's $D^{2.5}$ -- which actually exceeds a $2^{128}$ key search. The break tracks the field size, not the cipher and not the format. It is the small field that dies, which is exactly why the honest claim is "a six-digit column falls cheaply," never "a whole card column falls to a few thousand queries."

The crudest version of the attack needs no cleverness at all. If the domain is small, you can simply query every point and write down the codebook.

{` // A permutation is a lock only if its domain is astronomically large. // Here N = 1,000,000 (a six-digit field). We recover the ENTIRE codebook // by brute enumeration -- the worst case for the attacker, and it still wins. function makeSecretPermutation(n, seed) { const p = new Int32Array(n); for (let i = 0; i < n; i++) p[i] = i; let s = seed >>> 0; for (let i = n - 1; i > 0; i--) { // Fisher-Yates shuffle, toy PRNG s = (Math.imul(s, 1103515245) + 12345) & 0x7fffffff; const j = s % (i + 1); const t = p[i]; p[i] = p[j]; p[j] = t; } return p; // secret to the attacker } const N = 1000000; // the six-digit domain const secret = makeSecretPermutation(N, 20260709); function oracle(x) { return secret[x]; } // attacker gets only oracle access

const recovered = new Int32Array(N); let queries = 0; for (let x = 0; x < N; x++) { recovered[x] = oracle(x); queries++; }

let correct = 0; for (let x = 0; x < N; x++) if (recovered[x] === secret[x]) correct++; console.log('domain size N = ' + N.toLocaleString()); console.log('queries used (brute) = ' + queries.toLocaleString()); console.log('codebook entries learned = ' + correct.toLocaleString() + ' of ' + N.toLocaleString()); console.log('A 128-bit block hides among 2^128 points. This field hides among ' + N.toLocaleString() + '.'); `}

The demo is deliberately a worst case for the attacker: brute enumeration spends the full $N$ queries and learns the codebook one point at a time. The real attacks in the table do strictly better on all three axes at once. They use sublinear data -- about $D^{11/12}$ pairs, fewer than $N$; they recover the round functions rather than individual points, so the whole codebook and every tweak fall together; and they extend to domains far too large to enumerate by hand. Enumeration is only the ceiling. The cryptanalysis lives well beneath it. Either way, the verdict is identical.

A permutation over a million points is not a lock. It is a puzzle -- and the attacker has all the pieces.

Why FF3-1 could not be patched a second time

NIST had already conceded the problem in public. Its April 2017 note stated that FF3 "is no longer suitable as a general-purpose FPE method" and floated reducing the tweak from eight bytes to six [@nist-ff3-news2017]. NIST's fix took the same reduce-the-tweak approach but settled on a more modest cut, and it lived only in a draft: the 2019 initial public draft of SP 800-38G Revision 1 defined FF3-1, shrinking the tweak from 64 to 56 bits -- seven bytes, not the six the researchers had floated -- and mandating a million-value minimum domain [@nist-sp800-38gr1-ipd2019]. The only final NIST format-preserving standard remained the 2016 document, with FF1 and FF3.

Then Tim Beyne broke FF3-1 again in 2021 with a linear-cryptanalysis distinguisher that exploits the FF3 tweak schedule -- each tweak half influences only alternating rounds -- an attack that hits FF3 and FF3-1 but not FF1, whose larger tweak and ten rounds do not share the structure [@beyne2021].

The weakness is structural to the FF3 family, so no third round of tweak-size tuning could close it. NIST removed the method entirely rather than shrink the tweak a third time [@nist-sp800-38gr1-2pd2025].

Break	Year	Knob	Root cause	Lesson
ECB penguin	1990s	width (none)	deterministic independent blocks [@fruhwirth2005]	diffusion is necessary but not sufficient
CBC watermarking	2000s	width	predictable per-sector IV [@fruhwirth2005]	derive the IV secretly, as ESSIV does
LRW key leak	2002	key handling	tweak mask built from raw key material [@liskov-rivest-wagner2002]	never let key material into the mask
Elephant diffuser retired	2006	integrity	a diffuser randomizes but cannot reject [@ferguson2006]	a diffuser is not a MAC
XTS malleability and replay	2007	narrow width, no tag	16-byte diffusion, no authentication [@nist-sp800-38e]	integrity is a separate layer
FF3 broken	2017	small domain	polynomial codebook recovery, 64-bit tweak [@durak-vaudenay2017]	domain size is a security parameter
FF3-1 broken	2021	structure / tweak schedule	linear weakness in the tweak schedule [@beyne2021]	no tweak tuning fixes a structural break

Every entry tagged "width" or "small domain" is one of the two knobs, and the two tagged "key handling" and "integrity" are the lessons that shaped the survivors. FF3-1 sits apart: its 2021 break is structural -- a linear weakness in the tweak schedule, not a knob you can turn. The disk rail carries an upper-bound limit of its own, from the birthday paradox rather than the domain -- and it is widely misdescribed.Two different numbers get conflated here. XTS, Adiantum, and HCTR2 are birthday-bounded: their distinguishing advantage grows like $q^2 / 2^{128}$ for $q$ enciphered blocks under one key, so collisions become a concern only around $2^{64}$ blocks, and that is a per-key threshold. NIST's cap of $2^{20}$ AES blocks on a single XTS data unit is a separate, far more conservative per-data-unit ceiling that SP 800-38E attributes to IEEE 1619 section 5.1, with no birthday rationale attached [@nist-sp800-38e]. The cap is not "where XTS breaks," and the birthday bound is not "why the cap is $2^{20}$"; conflating them invites the false idea that XTS fails at 16 MiB. The format-preserving rail, by contrast, ended exactly where the theorems said it would.

"The encryption method FF3 is no longer specified." -- NIST SP 800-38G Revision 1, Second Public Draft, February 2025 [@nist-sp800-38gr1-2pd2025]

If the limits are theorems and the wall has a measured height, then the open questions are not "can we fix these?" They are "given that we cannot, what should we build next, and what does the honest alternative cost?"

9. Where This Still Bites

The science here is unusually settled, which makes the genuinely open questions sharp and few.

Why is XTS still the disk-contents default? Wide-block SPRPs are provably better at diffusion and ship in production, yet no mainstream operating system defaults contents to a wide-block mode on AES hardware. The reasons are performance and inertia: XTS is single-pass, parallel, random-access, and accelerated by AES-NI on x86 and the ARMv8 Cryptography Extensions, so at line rate it is very hard to beat.

HCTR2's two POLYVAL passes make it modestly heavier than single-pass XTS on the same accelerated chip, and there is no equally standardized wide-block-for-contents mode for general operating-system use. Android shipped wide-block only where it is cheap and clearly helps -- HCTR2 for filenames, Adiantum for chips without AES -- while keeping XTS for contents [@aosp-fbe]. When, if ever, contents should move to wide-block on accelerated hardware is a real and unresolved deployment question.

Can authenticated storage be cheap enough to default? Integrity is opt-in today, and it helps to size why. For read-write volumes, dm-crypt plus dm-integrity attaches a per-sector integrity tag; the kernel's own example uses a 4-byte crc32c, but tamper detection against an active attacker needs a keyed MAC such as a 32-byte hmac(sha256) [@kernel-dm-integrity].

And note what even a cryptographic tag does and does not buy: dm-integrity detects modification only. As Section 6 noted, it has no freshness anchor, so an old authentic pair still verifies; rollback resistance is therefore a separate requirement met only by an externally-anchored root: dm-verity's read-only Merkle root, a TPM or Secure-Boot-measured counter, or per-sector version counters. So the open problem is really two: cheap modification detection and a cheap freshness anchor for read-write data.

The tag is not the expensive part. The expensive part is atomicity: a sector and its tag must be updated together or not at all, and dm-integrity's default journaled mode guarantees that by writing everything twice. The documentation says plainly that it "degrades write throughput twice because the data have to be written twice" [@kernel-dm-integrity] -- roughly a 2x write amplification, and the real reason authenticated read-write storage stays an expert opt-in. A faster bitmap mode skips the double write, but the same documentation warns it "is also less reliable" [@kernel-dm-integrity].

Contrast dm-verity: it protects read-only data with a Merkle tree computed once at build time and pinned to an externally-held root -- the freshness anchor dm-integrity structurally lacks -- then merely verified, and cached, on read, with no write path, no journal, no atomic tag update, and optional forward-error-correction adding only about 0.8% space [@kernel-dm-verity]. That asymmetry is exactly why verified boot and system images enable dm-verity by default while mutable user data sits on bare dm-crypt.

Whether tamper detection can be made cheap enough to default for read-write data -- rather than paying a doubled write for it -- is the open engineering question. Where expansion is allowed at all, the clean answer stays an AEAD.

Is any small-domain FPE safe, or is tokenization the only honest answer? The decade-long attack line points one way: shrink the domain and you enumerate the permutation in polynomial work. For genuinely small fields, the consistent verdict is that a tokenization vault, not a cleverer cipher, is the defensible design.

Mostly no, and it is worth puncturing the "quantum breaks all cryptography" over-generalization. XTS, Adiantum, and HCTR2 are built from symmetric permutations, not from the factoring and discrete-log problems a quantum computer would break. The only quantum lever against them is Grover's search, which square-roots the effort of a brute-force key hunt -- and a 256-bit key already answers it, leaving an effective 128-bit security level [@nist-ir8105]. AES-256-XTS and 256-bit HCTR2 need no algorithm swap. There is no length-preserving equivalent of the wrenching public-key transition described in the [post-quantum migration post](/blog/post-quantum-cryptography-on-windows-the-thirty-year-migrati/).

Side channels in the radix arithmetic. FPE's Feistel rounds do modular arithmetic over the message radix, and getting that arithmetic both correct and constant-time is subtle. NIST went so far as to ban floating-point arithmetic in FF1 implementations after a rounding bug in Bouncy Castle's FF1 code -- reported by Daniel Bleichenbacher and fixed in a commit titled "Fix rounding issue with FF1" -- showed how easily a floating-point shortcut corrupts the cipher [@bouncycastle-ff1-commit][@nist-sp800-38gr1-2pd2025].

Constant-time radix conversion remains an under-examined implementation area. This kind of slow, standards-driven hardening -- ban the dangerous construct, wait for implementations to catch up -- is the same pattern that governs cipher agility elsewhere.

Open problems belong to the field. You, though, have a disk to encrypt and a column to tokenize today. So here are the rules.

10. A Field Guide: Which Cipher, Which Parameters, Which Layer

Every rule below is the thesis made operational: pick the widest permitted permutation for your domain, and name the layer that provides integrity.

flowchart TD Start["What are you encrypting?"] --> Expand{"Can you spend extra bytes?"} Expand -->|"yes"| AEAD["Use an AEAD, not a length-preserving mode"] Expand -->|"no"| Kind{"Volume, files, or a field?"} Kind -->|"full volume"| XTS["AES-XTS-256, plus an integrity layer if tamper matters"] Kind -->|"file-based"| FBE["fscrypt: XTS plus HCTR2 on AES hardware, Adiantum without"] Kind -->|"structured field"| Field{"Domain at least a million?"} Field -->|"yes"| FF1["FF1 with a per-context tweak"] Field -->|"no"| Vault["Tokenization vault, not FPE"]

Full volume or block device. Configure AES-256-XTS -- the strongest widely-supported choice. Do not mistake that for the default, which varies by product: BitLocker and FileVault ship AES-128-XTS, while LUKS ships AES-256-XTS (Section 6), so name the mode, not a universal key size. Respect the 16 MiB data-unit cap of $2^{20}$ AES blocks [@nist-sp800-38e].

If tampering is in your threat model, add integrity at another layer, and be precise about what each layer buys: dm-integrity detects modification only -- it has no freshness anchor, so it does not resist rollback or replay -- while rollback resistance needs an externally-anchored root (see Section 9). Budget dm-integrity's roughly 2x write amplification for read-write volumes. Key management and evil-maid defenses are a separate topic, covered in the BitLocker architecture post; this cipher-level guidance deliberately defers them.

Note: The single most common configuration error in this whole area. "AES-256-XTS" means two independent 256-bit keys -- one to encipher, one to derive the tweak -- for 512 bits total. Linux fscrypt makes this concrete by requiring a 64-byte master key for a v1 AES-256-XTS policy [@kernel-fscrypt]. If your key material is 256 bits, you have half a key.

File-based encryption on Android or Linux fscrypt. On hardware with AES acceleration, use aes-256-xts for contents and aes-256-hctr2 for filenames. On hardware without AES acceleration, use adiantum for both. Never hand-roll it; drive it through the platform's file-based-encryption configuration [@aosp-fbe].

Can you expand at all? Then stop -- do not use a length-preserving mode. The designers of HCTR2 say so themselves, in the plainest terms.

"Usually, the right mode to use for encryption is a mode like AES-GCM or AES-GCM-SIV ... However there are some applications, such as disk encryption, where the ciphertext must be the same size as the plaintext and there is no room for such information." -- the google/hctr2 README [@google-hctr2-repo]

Note: Length preservation is a painful constraint you accept only when it is forced on you. The moment your format allows a nonce and a tag -- a new database column, a wrapping record, a protocol field -- switch to an AEAD and get authentication for free. That is the whole message of Part 7.

Structured field that must keep its format. Use FF1, not FF3-1. Require a domain of at least one million values. Use a maximal-entropy, per-context tweak. Never use it for small fields such as PINs or short codes, and prefer a tokenization vault whenever you can add a lookup [@nist-sp800-38gr1-2pd2025].

The common-misuse mirror

Each row below is a misconception mapped to the failure it causes.

Misconception	Reality	Failure it mirrors
"XTS detects tampering"	No -- XTS is malleable; add dm-integrity or dm-verity [@nist-sp800-38e]	XTS malleability
"dm-integrity stops rollback and replay"	No -- it detects modification only, with no freshness anchor; rollback needs an externally-anchored root [@kernel-dm-integrity]	XTS rollback
"An AES-256-XTS key is 256 bits"	No -- two keys, 512 bits; fscrypt v1 wants 64 bytes [@kernel-fscrypt]	key-size confusion
"AES-256-XTS is the default everywhere"	No -- the AES-XTS mode is universal, but BitLocker and FileVault default to AES-128-XTS; only LUKS defaults to AES-256-XTS [@microsoft-bitlocker-autopilot][@cryptsetup-luksformat-man]	mode vs key size
"Wide-block means authenticated"	No -- Adiantum and HCTR2 fix diffusion, not integrity [@hctr2-2021]	Elephant diffuser
"FF3 was fixed as FF3-1"	No -- FF3-1 was broken again and removed in 2025 [@nist-sp800-38gr1-2pd2025]	FF3-1 broken
"FPE on a PIN is fine"	No -- a small domain is enumerable in polynomial work; use a vault [@hoang-tessaro-trieu2018]	FF3 broken
"FPE may use floating-point radix math"	No -- banned in FF1 after the Bouncy Castle bug [@bouncycastle-ff1-commit]	radix side channel
"Store the disk key on its own disk"	No -- that is the LRW self-encryption trap [@liskov-rivest-wagner2002]	LRW key leak

On Linux, inspect a LUKS device with `sudo cryptsetup luksDump /dev/sdX` and look at the Cipher line -- you will typically see `aes-xts-plain64`. On Windows, run `manage-bde -status C:` in an elevated prompt and read the Encryption Method field, which reports XTS-AES on any recent install.

The mirror above names the misconceptions. Two practical questions sit outside it -- when to reach for an AEAD instead, and exactly how far XTS malleability reaches -- and the FAQ takes those on.

11. Frequently Asked Questions

Because the nonce and the authentication tag do not fit. A sector is a fixed size with no spare bytes, and that is the entire reason length-preserving modes exist. Where the extra bytes *do* fit, an AEAD like AES-GCM or AES-GCM-SIV is exactly what you should use instead [@google-hctr2-repo]. No, not in the way a CTR or CBC bit-flipping gadget can. XTS malleability is position-bound: an attacker can overwrite a 16-byte block into random garbage, or replay an old image of the same block at the same position, but cannot make a chosen, surgical edit to your plaintext. It is destruction and rollback, not controlled rewriting [@nist-sp800-38e].

The two knobs, one last time

Return to the two teams from the opening. The first team's disk was rewritten by someone who could not read it because XTS has no tag and diffuses across only 16 bytes -- too narrow, and unauthenticated by construction. The second team's six-digit codes were recovered because a million-value field is a domain small enough that rebuilding its whole permutation costs about $2^{30}$ operations and fewer adaptive queries than there are codes -- too small, and deterministic by construction. Neither failure was a bug. Both were the direct, predictable consequence of forcing a cipher to preserve length.

That is the whole field in one sentence. When ciphertext must occupy exactly the plaintext's space, encryption becomes a deterministic, unauthenticated keyed permutation, and its quality is set by two knobs: how wide its diffusion is, and how large its domain is. Every real break in this article sorts onto one of them, or onto the handful of hard-won lessons the failure catalog logs beside the knobs -- leaky key handling, missing integrity, and a structural tweak-schedule flaw. And the small-domain break, notably, costs work merely polynomial in the domain, not the exponential figure the format advertises.

Because the primitive has no tag, integrity always lives one layer up: in a Merkle tree, an integrity journal, a per-file nonce, or an AEAD wherever you are finally allowed to spend the bytes. Part 10 turns from confidentiality on stored data to the machinery that negotiates it in motion.

You can now read any storage or tokenization design at a glance. Ask three questions: how wide is its permutation, how large is its domain, and -- because it has no tag -- what layer above is keeping it from being rewritten?

The AEAD Decision Matrix: Seven Ciphers, Three Edges, One Choice

noreply@paragmali.com (Parag Mali) — Thu, 09 Jul 2026 15:04:00 GMT

Every modern AEAD makes the same three promises, and they differ only at the sharp edges where those promises stop. Three edges decide a real deployment: the **nonce contract** (catastrophic on reuse, or misuse-resistant), the **performance and hardware profile** (AES-NI plus CLMUL, constant-time software, or a few thousand gates), and **commitment** (does one ciphertext bind to exactly one key?). "Which AEAD should I use?" is not a ranking from worst to best. It is a function: default to AES-GCM or ChaCha20-Poly1305, and escalate to AES-GCM-SIV, AES-CCM, Ascon, OCB3, or AEGIS only when a specific edge demands it. Every famous break -- invisible salamanders, nonce reuse on live TLS, `CCM_8` forgeries -- was a deployment stepping on an edge, not a broken cipher.

1. Two Failures That Should Have Been Impossible

In 2018, a team of cryptographers handed Facebook a single encrypted file that decrypted to two completely different, perfectly meaningful images, and whose authentication tag verified correctly for both, under two different keys [@dodis2018-salamander]. The cipher was AES-GCM, the internet's default authenticated encryption. Nothing about it was broken. That is the unsettling part: AES-GCM did exactly what it promised. It had just never promised the thing everyone assumed it did.

The result defeated Facebook Messenger's abuse-reporting scheme, which relied on the encrypted attachment being bound to the key that produced it. The authors called their construction "invisible salamanders" [@dodis2018-salamander]. Your reflex on reading this -- "but it is authenticated, so how did the tag pass?" -- is the entire subject of this article. The word "authenticated" carries less than you think.

Now the second failure, because one break only shows one edge. In 2016, researchers scanning the live internet found 184 HTTPS servers reusing a (key, nonce) pair under AES-GCM [@bock2016]. A repeated nonce in GCM is not a hygiene slip that costs you a little margin. It instantly leaks the XOR of the two plaintexts, and -- via an attack Antoine Joux had described to NIST a decade earlier, in 2006 -- it lets an attacker forge arbitrary messages [@joux2006]. The confidentiality half of that failure is easy to see for yourself.

{` // A stream-cipher-style AEAD (GCM, ChaCha20-Poly1305) turns the key+nonce // into a keystream, then XORs it with the plaintext. Reuse the nonce and // you reuse the keystream -- so it cancels out of the XOR of two ciphertexts. const keystream = [0x9e, 0x37, 0xb1, 0xf2, 0x4a]; // same (key, nonce) => same keystream const xor = (a, b) => a.map((x, i) => x ^ b[i]);

const p1 = [...'HELLO'].map(c => c.charCodeAt(0)); const p2 = [...'WORLD'].map(c => c.charCodeAt(0));

const c1 = xor(p1, keystream); // ciphertext 1 const c2 = xor(p2, keystream); // ciphertext 2, SAME nonce

// Attacker sees only c1 and c2, never the key or keystream: const leaked = xor(c1, c2); // == p1 XOR p2, the keystream is gone const recovered = xor(leaked, p2); // and if p2 is ever guessed, p1 falls out console.log('c1 XOR c2 leaks P1 XOR P2:', leaked.join(',')); console.log('recovered P1:', String.fromCharCode(...recovered)); `}

Neither of these was a broken cipher. AES-GCM met its specification in both cases. The salamander bound nothing to a key because AES-GCM was never designed to; the TLS servers leaked plaintext because they violated the one contract GCM cannot survive. Both failures were the same shape: a deployment reached for an AEAD and then stepped on the one sharp edge its designers had moved somewhere the deployment could not avoid.

That is the thesis of this field guide. Authenticated encryption with associated data (AEAD) fused confidentiality and integrity so completely that composing them by hand became a solved problem. The remaining hard problem is choosing among AEADs, and that choice turns on three edges: the nonce contract, the performance and hardware profile, and the commitment axis.

Part 1 of this series defined an AEAD's guarantees as IND-CPA plus INT-CTXT; commitment is a robustness property beyond both. Part 6 showed the padding-oracle cost of the botched integrity AEAD was built to remove.The nonce-reuse breaks are widely mis-cited to CVE-2016-0270. That CVE actually names IBM Domino, and the NVD entry itself warns it has been "incorrectly used for GCM nonce reuse issues in other products" [@cve-2016-0270].

Key idea: "Which AEAD should I use?" is not a ranking. It is a function over three edges: pick the construction whose failure mode your deployment can guarantee it will never trigger. The salamander triggered the commitment edge; the TLS servers triggered the nonce edge. Same lesson, two edges.

If "authenticated" does not mean "bound to one key," and "encrypted" does not survive a repeated nonce, then what exactly does an AEAD promise, and where does each promise stop? To answer that, we have to go back to the moment when "encrypt" and "authenticate" were still two separate calls.

2. From Two Calls to One

Here is a question that sounds trivial and is not: you have a cipher and you have a message authentication code (MAC). In which order do you apply them? Most engineers guess. The wrong guess shipped in SSL, IPsec, and SSH, and it took the field a decade of theory to sort out which guess was which.

There are three orderings. Encrypt-and-MAC authenticates the plaintext and appends the tag beside the ciphertext (SSH). MAC-then-Encrypt authenticates the plaintext, then encrypts plaintext and tag together (SSL and, for years, TLS). Encrypt-then-MAC encrypts, then authenticates the ciphertext (IPsec).

In 2000, Mihir Bellare and Chanathip Namprempre proved these are not interchangeable: only Encrypt-then-MAC generically preserves both privacy and integrity for any secure cipher and any secure MAC [@bn2000]. A year later, Hugo Krawczyk showed the authenticate-then-encrypt method used in SSL is not generically secure while Encrypt-then-MAC is [@krawczyk2001]. Part 6 is the sequel: MAC-then-encrypt is exactly the door a padding oracle walks through.

One keyed call that provides confidentiality of the plaintext, integrity of the plaintext, and integrity (but not secrecy) of an extra header called the associated data. The uniform interface was standardized in RFC 5116 [@rfc5116].

The lesson cut deeper than "pick Encrypt-then-MAC." If two correct primitives can combine into an insecure whole, hand-composition is a footgun no matter how good the parts are. The fix was to make a single primitive own both goals, so no protocol designer could botch the join.

Charanjit Jutla showed the way in 2001 with IAPM and IACBC, modes that delivered message integrity in essentially one pass, almost for free [@jutla2001]. The same year, Phillip Rogaway and coauthors published OCB, a one-pass block-cipher mode built for exactly this [@ocb2001]. Then, in 2002, Rogaway named the primitive: real messages carry a header -- routing bytes, version numbers, sequence counters -- that must be authenticated but not encrypted, because intermediaries need to read it. His "Authenticated-Encryption with Associated-Data" formalized that extra input [@rogaway2002-ad].

The header bytes an AEAD authenticates but does not encrypt: routing information, protocol versions, sequence numbers. Tampering with the associated data makes tag verification fail, but the associated data itself travels in the clear [@rogaway2002-ad].

By 2008, David McGrew closed the loop with RFC 5116, which defined the universal interface every construction in this article implements: AEAD-Encrypt(key, nonce, associated_data, plaintext) returns ciphertext || tag, and the matching decrypt either returns the plaintext or a single, uninformative failure [@rfc5116]. One call in, one call out, no way to reverse the order of two primitives because there is only one primitive.

A value that must be unique for every encryption under a given key. The mode, not the label, sets the exact requirement: some AEADs tolerate only uniqueness, others tolerate randomness, and a few tolerate repetition. Nonce generation is the subject of Part 2 of this series. timeline title The road to the AEAD portfolio 2000 : Encrypt-then-MAC proven the safe order 2001 : IAPM and OCB, single-pass native AE 2002 : Rogaway names AEAD and adds associated data 2003 : CCM standardized in RFC 3610 2004 : GCM published by McGrew and Viega 2006 : Joux forbidden attack, SIV misuse-resistance 2008 : RFC 5116 fixes the universal interface 2013 : CAESAR opens, AEGIS announced 2016 : Nonce reuse found on live TLS servers 2018 : Invisible salamanders break message franking 2019 : CAESAR portfolio chosen, OCB2 broken 2021 : Partitioning oracles weaponize non-commitment 2025 : NIST standardizes Ascon-AEAD128

The interface was settled by 2008. But an interface is a promise about shape, not about sharp edges. Two constructions that satisfy AEAD-Encrypt to the letter can fail in completely different ways, and the very first AEADs to ship proved it. The one that reached the most devices did not even win on technical merit. It won because of a patent.

3. The Welded Modes: CCM and GCM

The first AEAD to reach a billion devices was not chosen by a cryptographer. It was chosen, in effect, by a patent lawyer. When IEEE 802.11i (the security amendment that became WPA2) needed an authenticated mode, the elegant candidate was OCB -- but OCB was patent-encumbered, and the working group would not build a wireless standard on it. So 802.11i standardized the plodding, unpatented alternative: AES-CCM. By device count, that decision made CCM one of the most widely deployed AEADs in the world.

AES-CCM = Counter mode + CBC-MAC. Doug Whiting, Russ Housley, and Niels Ferguson specified it in RFC 3610 in 2003, and NIST blessed it as SP 800-38C the next year [@rfc3610][@sp80038c]. It runs the message through CBC-MAC for the tag, then encrypts with counter (CTR) mode: two passes, a strictly serial MAC, and the message length needed up front, which makes streaming awkward. What that price buys is frugality -- nothing but AES, a tiny code footprint, no second primitive to implement.

That profile is why CCM owns the constrained niche -- Bluetooth Low Energy [@bluetooth-core], Zigbee [@ieee802154], and the TLS CCM_8 ciphersuites. IEEE 802.11i also made CCMP the mandatory data-confidentiality protocol for WPA2 [@ieee80211i], and CCM rode WPA2/CCMP into the overwhelming majority of Wi-Fi hardware over the following decade.Zigbee and 802.15.4 use a variant called CCM*, which additionally permits integrity-only or encryption-only operation, unlike the strict RFC 3610 CCM that always does both. So the same plodding, all-AES mode anchors two very different worlds: the smallest constrained devices and the Wi-Fi layer that WPA2 secures. These are the building blocks from Part 5, welded to a MAC.

AES-GCM = Counter mode + GHASH. David McGrew and John Viega published Galois/Counter Mode in 2004, and NIST standardized it as SP 800-38D in 2007 [@mcgrew2004-gcm][@sp80038d]. GCM is everything CCM is not: one pass, fully parallelizable, and, on any CPU with AES-NI and CLMUL, blisteringly fast. It is the internet default, and AES-128-GCM is the single mandatory-to-implement (MUST) AEAD cipher suite in TLS 1.3, per RFC 8446 Section 9.1 -- a status the spec grants to no other AEAD [@rfc8446]. To understand every later failure, you need one fact about how its tag works.

An authenticator of the form $\text{tag} = H_k(\text{message}) + s$, where $H_k$ is a fast keyed universal hash and $s$ is a **one-time** secret mask. It is unforgeable only if $s$ is never reused. Repeat the mask and the algebraic structure of $H_k$ leaks. This is the shared root of both GHASH (AES-GCM) and Poly1305 (ChaCha20-Poly1305) [@bernstein-poly1305].

GCM's tag is a one-time Wegman-Carter MAC. GHASH evaluates a polynomial over the finite field $\mathrm{GF}(2^{128})$, with the ciphertext and associated-data blocks as coefficients, at a single secret point $H = \operatorname{AES}_k(0^{128})$. It then masks the result with a per-nonce keystream block:

$$T = \operatorname{GHASH}_H(A, C) \oplus \operatorname{AES}_k(J_0), \qquad H = \operatorname{AES}_k(0^{128})$$

where $J_0$ is the initial counter block derived from the nonce. The mask $\operatorname{AES}_k(J_0)$ is the one-time secret $s$. It is one-time only because the nonce is unique. Repeat the nonce and you repeat $J_0$, so you repeat the mask, and now two tags share it -- which turns GHASH's linear structure into a system of polynomial equations an attacker can solve.

The universal hash inside AES-GCM: a polynomial in the secret point $H = \operatorname{AES}_k(0^{128})$ evaluated over $\mathrm{GF}(2^{128})$. Because the polynomial is linear in its coefficients, recovering $H$ lets an attacker forge tags for chosen messages [@joux2006]. flowchart LR N["Nonce and counter"] --> CTR["AES counter mode"] K["Key"] --> CTR CTR --> KS["Keystream"] P["Plaintext"] --> X(("XOR")) KS --> X X --> C["Ciphertext"] C --> GH["GHASH polynomial at secret point H"] AD["Associated data"] --> GH K --> H["H is AES of the zero block"] H --> GH GH --> M(("XOR one-time mask")) K --> J["AES of the first counter block"] J --> M M --> T["Authentication tag"]

That single design choice hands GCM two sharp edges the rest of this article pays for. The first is the nonce contract: reuse is not a weakness, it is a detonator.

Note: For every nonce-respecting AEAD -- AES-GCM, ChaCha20-Poly1305, AES-CCM, OCB3, AEGIS, and Ascon -- one repeated (key, nonce) pair is an immediate, total break of confidentiality for those two messages, and, with a second collision, of authenticity for the whole key [@joux2006].One nonce collision leaks $P_1 \oplus P_2$ right away. Uniquely pinning the secret point $H$ for universal forgery generally needs at least two collisions, since a single pair of equations leaves multiple candidate roots. "One repeat is game over for those messages" is correct; "one repeat exposes $H$" is not quite [@bock2016].

The second edge is quieter and just as dangerous, because it is two different numbers people constantly merge into one.

Note: SP 800-38D fixes a per-message plaintext limit of $2^{39}-256$ bits (about 64 GiB), set by 32-bit counter-mode block-space exhaustion, and, separately, a per-key invocation cap of roughly $2^{32}$ messages when nonces are random 96-bit values, driven by the birthday bound on nonce collision [@sp80038d]. These are two different mechanisms, not two readings of one bound (the full derivation is in Section 8). The multi-user bounds that justify TLS 1.3's nonce randomization are tight [@htt2018-gcm]. Enforce both limits. Conflating them is the single most common accuracy error on this primitive.

GCM gave the internet a one-pass, hardware-fast AEAD, and in the same stroke handed it a mode where one repeated nonce is not a bug but a catastrophe. This is the same GCM negotiated inside shipping Windows protocols such as Schannel TLS and SMB 3 encryption [@ms-schannel-tls][@ms-smb-encryption]. Antoine Joux saw the danger in 2006 [@joux2006]. It took the rest of the world ten years, and a scan of the live internet, to believe him.

4. A Trunk With Branches

The family tree of AEAD is not a ladder from worst to best. Draw it that way and none of the modern constructions make sense, because none of them strictly dominates GCM. Draw it correctly and the whole field snaps into focus: it is a trunk with branches. The trunk is linear -- generic composition, then single-pass native AE, then the welded deployed modes CCM and GCM. Then, at GCM, the tree splits, because GCM's edges each forced a separate successor. No later generation wins. That is the entire point.

flowchart TD G0["Generic composition, Encrypt-then-MAC"] --> G1["Single-pass native AE, IAPM and OCB"] G1 --> G2["Welded deployed modes, CCM and GCM"] G2 --> E1["Nonce edge"] G2 --> E2["Hardware edge"] G2 --> E3["Commitment edge"] E1 --> S1["SIV and AES-GCM-SIV, misuse-resistant"] E2 --> S2["ChaCha20-Poly1305, constant-time in software"] E2 --> S4["AEGIS and Ascon, hardware and footprint co-design"] E3 --> S3["Committing transforms, added on top"]

The hardware edge: ChaCha20-Poly1305

GHASH is hard to make constant-time without the CLMUL instruction, and table-based AES is both slow and timing-leaky without AES-NI. Around 2013, that was the reality on most phones and ARM chips: no AES hardware, so GCM was either slow or a cache-timing side channel. Daniel Bernstein's answer kept GCM's exact nonce-respecting contract but threw out the parts that needed special silicon.

ChaCha20 is an ARX cipher -- addition, rotation, XOR -- with no S-box tables to leak timing, so it is constant-time by construction [@bernstein-chacha]. Poly1305 is a one-time Wegman-Carter MAC over a prime field rather than $\mathrm{GF}(2^{128})$, and its one-time key is derived from a ChaCha keystream block [@bernstein-poly1305].Poly1305's one-time key comes from a ChaCha-derived block, not from AES. ChaCha20-Poly1305 never calls AES at all -- a common misconception, since GCM's mask does use AES.

The IETF standardized the pair as an AEAD in RFC 8439 [@rfc8439], and it now protects TLS 1.3, WireGuard, OpenSSH, and the age file-encryption tool [@wireguard-protocol][@openssh-chacha][@age-spec]. Its normative status is widely overstated: in TLS 1.3, ChaCha20-Poly1305 is a recommended (SHOULD) cipher suite, not mandatory-to-implement, per RFC 8446 Section 9.1 -- a default by adoption, not by requirement [@rfc8446]. It is not weaker than GCM; it is the same nonce contract on a different, table-free engine.XChaCha20-Poly1305 extends the nonce to 192 bits, large enough that random nonces essentially never collide. That is a mitigation of nonce-collision anxiety, not misuse-resistance -- reuse the full 192-bit nonce and it fails exactly like GCM.

The nonce edge: misuse-resistance and SIV

Every mode so far detonates on a repeated nonce, and reuse keeps happening in the real world: cloned virtual machines that resume with identical RNG state, counters reset by a crash, embedded devices with weak entropy at first boot. If the operator cannot guarantee uniqueness, the fix is a mode that survives the operator's mistake. In 2006, Phillip Rogaway and Thomas Shrimpton formalized exactly that goal and built the first construction for it [@rs2006].

An AEAD whose failure under a repeated nonce is graceful, not catastrophic. Repeating a `(key, nonce, associated_data, plaintext)` tuple leaks only whether two encryptions were of the *same* message -- message equality -- and nothing else. It never leaks a plaintext XOR and never hands over the authentication subkey [@rs2006][@rfc8452].

The mechanism is a single clean idea: derive the initialization vector synthetically, as a pseudorandom function of the entire message, rather than accepting it from the caller. The tag is the IV.

A construction in which the initialization vector is computed as a MAC over the associated data and the full plaintext, then reused as the authentication tag. Two different messages under a repeated nonce produce different synthetic IVs, so their keystreams differ and nothing leaks [@rs2006]. flowchart TD K["Key"] --> MAC["MAC over associated data and full plaintext"] AD["Associated data"] --> MAC N["Nonce"] --> MAC P["Plaintext"] --> MAC MAC --> IV["Synthetic IV, which is also the tag"] IV --> CTR["Counter-mode encryption"] K --> CTR P --> CTR CTR --> C["Ciphertext"] IV --> T["Tag sent alongside ciphertext"]

That picture also explains the price, which returns as a theorem in Section 8: to compute the IV you must read the whole message first, so SIV cannot emit its first ciphertext byte until it has seen the last plaintext byte. It is inherently two-pass.

Two deployable forms exist. AES-SIV, standardized by Dan Harkins in RFC 5297, targets deterministic authenticated encryption and key wrapping [@rfc5297]. AES-GCM-SIV, from Shay Gueron, Adam Langley, and Yehuda Lindell in RFC 8452, keeps AES-NI and CLMUL speed while adding misuse-resistance, building on the Gueron-Lindell GCM-SIV design [@rfc8452][@gl2015-gcmsiv].AES-GCM-SIV hashes with POLYVAL, a little-endian sibling of GHASH, and derives a fresh message-authentication and encryption key per nonce -- a structure whose tight multi-user bounds were later established by Bose, Hoang, and Tessaro [@bht2018].

"...two authenticated encryption algorithms that are nonce misuse resistant -- that is, they do not fail catastrophically if a nonce is repeated." -- RFC 8452 [@rfc8452]

Read that phrasing carefully, because it is the most misquoted sentence about SIV. "Do not fail catastrophically" is not "do not fail." A repeated nonce in AES-GCM-SIV still leaks message equality: an attacker learns that two ciphertexts encrypt the same plaintext, which is a real and sometimes serious leak. SIV makes reuse survivable, never free.

The elegance detour: OCB3 and the patent that chose a standard

Two edges now had answers, each on its own branch. A third construction sits off to the side -- not because it is worse, but because history was unkind to it. OCB3, finalized by Ted Krovetz and Phillip Rogaway in 2011 and standardized as RFC 7253, is arguably the most elegant AEAD ever standardized: one block-cipher key, one pass, fully parallel, roughly one cipher call per block [@kr2011-ocb3][@rfc7253].

Its mechanism is where the elegance lives. Each block is masked by a key-derived offset -- $\text{Offset}i = \text{Offset}{i-1} \oplus L_{\text{ntz}(i)}$, a Gray-code walk over precomputed $L$ values -- and wrapped as $C_i = \text{Offset}_i \oplus \operatorname{ENCIPHER}(K, P_i \oplus \text{Offset}_i)$, while a single running plaintext checksum, $\text{Checksum} = P_1 \oplus P_2 \oplus \cdots$, produces the tag in the same pass [@rfc7253]. No second key, no separate MAC. It matches or beats GCM in software. And it lost anyway -- not for a technical reason.A widely repeated claim is that OCB3 is inverse-free. It is not. OCB-DECRYPT recovers each block as $P_i = \text{Offset}_i \oplus \operatorname{DECIPHER}(K, C_i \oplus \text{Offset}_i)$, calling the block-cipher inverse $E_K^{-1}$ (RFC 7253 Section 4.3), so a hardware implementation needs both the AES encrypt and decrypt circuits [@rfc7253]. Inverse-free is the property of the CTR, stream, and sponge modes -- AES-GCM, AES-CCM, ChaCha20-Poly1305, AEGIS, and Ascon -- and OCB is precisely the lineage that trades it away for single-primitive elegance.

OCB was patented from birth. When the 802.11i working group picked WPA2's cipher in 2003, it passed over OCB precisely because of those patents and chose the unencumbered CCM instead. Rogaway later granted free licenses for open-source and non-military use, but the damage was done: every major protocol had already standardized on GCM and ChaCha20-Poly1305 in the years the patents were live. In 2021 Rogaway released OCB into the public domain outright -- too late to matter [@kr2021-ocb]. The honest answer to "when should I reach for OCB3 in 2026?" is "almost never, and the reason is a patent grave, not a design flaw."

The dead end: OCB2 and the limits of "provably secure"

There is one more branch, and it is a warning. Between OCB1 and OCB3 sat OCB2, an ISO-standardized refinement with a security proof. In 2019, Akiko Inoue, Tetsu Iwata, Kazuhiko Minematsu, and Bertram Poettering broke it outright: universal forgery and full plaintext recovery, because OCB2 used the XEX* tweakable cipher outside the regime its proof actually covered [@inoue2019-ocb2].

The OCB2 break is the cautionary counterpoint to every "but it has a proof" argument. OCB2 was standardized by ISO/IEC and carried a published security proof, and it was still broken end to end. A proof secures a *model*; if the construction steps outside the model's assumptions, the proof guarantees nothing. Tellingly, the same attack left OCB1 and OCB3 untouched [@inoue2019-ocb2] -- a dead-end twig on a branch whose siblings survived.

Two of GCM's three edges now had dedicated successors, and the family tree looked complete. But every construction so far -- GCM, ChaCha20-Poly1305, even the misuse-resistant SIV family -- quietly shared a third assumption nobody had thought to check: that "authenticated" meant "bound to one key." In 2018, that assumption broke in production, on the desk of a Facebook security engineer holding a single file with two faces.

5. The Commitment Reckoning and the CAESAR Crucible

Now we can explain the salamanders. Recall the mechanism from Section 3: a GCM tag is a one-time Wegman-Carter value, $\operatorname{GHASH}_H(A,C) \oplus \operatorname{AES}_k(J_0)$, and GHASH is linear in its coefficients. An attacker who gets to pick two keys, $k_1$ and $k_2$, can treat "make the tag verify under both" as a system of linear equations over $\mathrm{GF}(2^{128})$ and simply solve it, producing one ciphertext that decrypts to two chosen, meaningful plaintexts and passes verification under each key [@dodis2018-salamander].

Nothing is broken. GCM guarantees confidentiality and integrity; it never promised that a ciphertext binds to a single key. That promise has a name, and until 2018 almost nobody was asking for it.

A graded robustness property beyond confidentiality and integrity. CMT-1 means a ciphertext binds to exactly one *key*; CMT-4 means it binds to the full *context* -- key, nonce, associated data, and message. Bellare and Hoang proved that CMT-1 does not imply CMT-4: committing to the key is strictly weaker than committing to everything [@bh2022-commit]. RFC 9771 standardized the vocabulary [@rfc9771].

Key idea: "Authenticated" never meant "committed." An AEAD's guarantees -- IND-CPA plus INT-CTXT -- say nothing about binding a ciphertext to one key. Commitment is a third, orthogonal axis. That is why every default AEAD was non-committing and nobody noticed for fifteen years: the property was simply outside the definition everyone was proving.

Two ideas define the modern state of the art, and both come from refusing to treat "cipher plus separate hash" as the end of the design space.

The first is robustness as an explicit goal -- against the operator and against the adversary who supplies the key. The operator side is MRAE from Section 4, elevated from a mode into a principle. The adversary side is committing AEAD, and its timeline is a tidy example of a theoretical curiosity becoming an operational threat.

Invisible salamanders (2018) showed a non-committing AEAD could defeat message franking [@dodis2018-salamander]. Then, in 2021, Julia Len, Paul Grubbs, and Thomas Ristenpart turned it into a weapon: partitioning oracle attacks use the missing commitment to recover passwords and other low-entropy keys, crafting one ciphertext that decrypts under many candidate keys and using each success or failure to bisect the key space [@len2021-partition].

Mihir Bellare and Viet Tung Hoang then built the graded CMT-1 through CMT-4 theory and cheap transforms that add full commitment with no ciphertext-size increase [@bh2022-commit]; Ange Albertini and coauthors independently catalogued the abuses and a padding-based fix [@albertini2022], with further theory from John Chan and Phillip Rogaway [@chan2022]. Sanketh Menda and coauthors then showed the gap is portfolio-wide, with context-discovery attacks against CCM, EAX, SIV, GCM, and OCB3 [@menda2023]. RFC 9771 (2025) finally fixed the vocabulary the field had been improvising [@rfc9771].

A vetted AEAD gives you confidentiality and integrity and stops there. Commitment is a third axis nobody put in the definition -- which is exactly why one ciphertext could wear two faces.

The second idea is native co-design: stop bolting a cipher to a hash and instead build the AEAD around the hardware or footprint you actually have. Hongjun Wu and Bart Preneel's AEGIS (2013) is the AES-hardware extreme [@wu2013-aegis].

It keeps a large internal state -- eight 128-bit words in AEGIS-128L -- seeded directly from the key and nonce; the AES round function updates that state as it absorbs associated data and plaintext, encryption squeezes a keystream out of the state, and the tag comes from the finalized state. One primitive, one pass, no separate hash, and the IETF draft records that all variants are inverse-free and built from the AES encryption round [@aegis-draft18].

That structure is why AEGIS outruns GCM on AES silicon, and why nonce reuse is catastrophic: the whole state is a function of (key, nonce), so a repeat lets an attacker unwind it. Its commitment story is a nuance, not a headline -- a 128-bit AEGIS tag delivers only about 64-bit committing security, while a 256-bit tag pushes cross-key collisions out of reach, so AEGIS is partially committing by tag length, neither fully committing nor flatly broken [@aegis-draft18].

At the other extreme, Ascon -- by Christoph Dobraunig, Maria Eichlseder, Florian Mendel, and Martin Schläffer -- uses a single lightweight permutation in a sponge, giving a tiny gate count and a side-channel-friendly structure for constrained hardware [@ascon2021].

flowchart LR KN["Key and nonce"] --> ST["Large internal state, eight 128-bit words"] ST --> UP["AES round function update"] AD["Associated data"] --> UP P["Plaintext"] --> UP UP --> KS["Squeezed keystream"] P --> X(("XOR")) KS --> X X --> C["Ciphertext"] UP --> FIN["Finalized state"] FIN --> T["Authentication tag"] A mode built from one public permutation whose state is split into a "rate" (absorbs input and is squeezed for output) and a "capacity" (the hidden security margin). A single permutation absorbs the key, nonce, associated data, and plaintext, then squeezes keystream and a tag -- no separate cipher and hash. Ascon is a sponge [@ascon2021].

What forced this whole portfolio into existence was a contest. The CAESAR competition (2013-2019), organized by Daniel Bernstein, was a public, multi-year, break-it-in-the-open bake-off [@caesar-home]. Its most important decision was structural: rather than crown a single winner, it selected a portfolio across three use cases -- itself the thesis of this article. The winners were Ascon and ACORN for lightweight use, AEGIS-128 and OCB for high-performance use, and Deoxys-II (first choice) and COLM for defense in depth [@caesar-portfolio].Deoxys-II is built on the TWEAKEY framework for tweakable block ciphers, by Jérémy Jean, Ivica Nikolić, Thomas Peyrin, and Yannick Seurin [@caesar-portfolio].

Note carefully: Deoxys-II is a CAESAR winner, not an also-ran -- it simply was never deployed at scale. The real also-rans are the candidates eliminated during the competition, including MORUS, which took a certificational break from Tomer Ashur, Maria Eichlseder, and coauthors and did not make the final portfolio [@morus2018].

The competition ended in 2019 with no single champion, on purpose. So where does that leave a working engineer in 2026, staring at seven names in a crypto library's documentation? To choose well, you need the current map: exactly what each construction is, where it wins, and where it bites.

6. The 2026 Portfolio, Precisely

The current portfolio has a stable shape: two defaults and five specialists. By deployment, the two internet defaults are AES-GCM and ChaCha20-Poly1305 -- but normatively they are not equals. In TLS 1.3 only AES-128-GCM is mandatory-to-implement (MUST); AES-256-GCM and ChaCha20-Poly1305 are both recommended (SHOULD), the same normative level as each other, per RFC 8446 Section 9.1 [@rfc8446]. Everything else is a deliberate escalation. Here is each construction on its edges.

AES-GCM. Counter mode plus a one-time GHASH tag; one pass, parallel, O(1) online state [@mcgrew2004-gcm]. Excels on any CPU with AES-NI and CLMUL, which is every server and most modern clients. Struggles on hardware without those instructions and, above all, on nonce discipline: reuse is fatal and the two ceilings are easy to breach [@sp80038d]. Adoption: the internet default.

ChaCha20-Poly1305. ARX stream cipher plus a prime-field one-time MAC; constant-time in pure software with no lookup tables [@rfc8439]. Excels precisely where GCM struggles -- phones, embedded ARM, any target without AES hardware. Same nonce contract as GCM, so it struggles on the exact same reuse cliff. Adoption: TLS 1.3, WireGuard, OpenSSH [@wireguard-protocol][@openssh-chacha].

AES-CCM. Counter mode plus CBC-MAC; two passes, serial, needs the length up front, but all-AES and tiny [@rfc3610]. Excels in constrained stacks (BLE [@bluetooth-core], Zigbee [@ieee802154]) and inside WPA2 [@ieee80211i]. Struggles on throughput and streaming, and the truncated CCM_8 tag trades a real forgery budget for eight bytes [@rfc3610]. Adoption: enormous by device count.

AES-GCM-SIV and AES-SIV. Synthetic-IV misuse-resistant modes; two-pass by necessity [@rfc8452][@rfc5297]. Excel when nonce uniqueness cannot be guaranteed: reuse degrades to leaking message equality rather than catastrophe. Struggle on streaming (they must buffer) and still leak equality, so they are survivable, not free. AES-GCM-SIV runs near AES-GCM speed on server hardware, around 0.92 cycles per byte on Broadwell in the original design [@gl2015-gcmsiv]. Adoption: growing where reuse risk is real.

OCB3. One block-cipher key, one pass, fully parallel, roughly one cipher call per block -- technically superb [@kr2011-ocb3][@rfc7253]. Excels on elegance and software speed. Struggles on adoption, for the patent reasons in Section 4; it is nonce-respecting like GCM; and, unlike the CTR and stream modes, it is not inverse-free -- decryption calls the AES inverse circuit (RFC 7253 Section 4.3), a genuine hardware cost [@rfc7253]. Adoption: minimal, despite standardization.

AEGIS. Keystream and authentication driven straight from the AES round function over a large nonce-seeded state; the throughput frontier, around 0.48 cycles per byte for AEGIS-128L on 4 KB messages, faster than CCM, GCM, and OCB [@wu2013-aegis][@aegis-v11]. Excels on raw speed when you control both endpoints. Struggles on maturity and contract: nonce-respecting (catastrophic on reuse, since its whole state is nonce-seeded) and only partially committing -- about 64-bit committing security at a 128-bit tag, stronger at a 256-bit tag [@aegis-draft18].

Critically, AEGIS is still an active Internet-Draft, draft-irtf-cfrg-aegis-aead-18 from October 2025, not an RFC -- it states plainly that it "is not an IETF product and is not a standard," and it expires on 8 April 2026 [@aegis-draft18].

Ascon-AEAD128. A single 320-bit lightweight permutation in a sponge; tiny state, side-channel-friendly [@ascon2021]. Excels on constrained hardware and gate count. Struggles on server throughput, where AEGIS and GCM win. It is nonce-respecting, but -- unlike the CTR and stream modes -- it commits to its inputs without any added transform: the tag is squeezed from the sponge's large hidden capacity after that capacity has already absorbed the key, so one ciphertext binds natively to the key it was produced under. RFC 9771 lists Ascon-AEAD128 as both key-committing and full-committing as-is, not merely as a candidate for a bolt-on fix [@rfc9771]. The currency fact matters here: NIST SP 800-232 is final (August 2025) and standardizes Ascon-AEAD128, a tweaked variant of CAESAR Ascon-128 using a larger data-absorption rate -- widely reported as 128-bit, up from the CAESAR design's 64-bit, with the exact parameter in the SP 800-232 body [@sp800232][@nist-ascon-2023]. It is not byte-identical to CAESAR Ascon [@ascon-site].

A profile only helps if you can actually call the construction, so here is where each one ships as of mid-2026, with each cell cited to that project's official documentation.

Construction	Where it ships (libraries and frameworks)
AES-GCM	OpenSSL/BoringSSL [@openssl-aes][@boringssl-aead], Go `crypto/cipher` [@go-cipher], Java JCA [@java-jca], .NET CNG [@dotnet-aesgcm], WebCrypto (the only AEAD in the browser) [@webcrypto], libsodium (hardware-gated) [@libsodium-aegis]
ChaCha20-Poly1305 / XChaCha20	OpenSSL/BoringSSL [@openssl-chacha][@boringssl-aead], libsodium (default for random-nonce APIs) [@libsodium-aegis], Go `x/crypto` [@go-chacha], WireGuard [@wireguard-protocol], OpenSSH [@openssh-chacha], `age` [@age-spec]
AES-CCM	mbedTLS [@mbedtls-ccm], wolfSSL [@wolfssl-ccm], Wi-Fi [@ieee80211i] / BLE [@bluetooth-core] / Zigbee [@ieee802154] firmware
AES-GCM-SIV	BoringSSL/OpenSSL [@boringssl-aead][@openssl-aes] and several bindings; less universal than the defaults
AES-SIV	`miscreant`-lineage (deterministic / key-wrap) libraries [@miscreant]
OCB3	some libraries (public domain since 2021 [@kr2021-ocb]); rarely a default
AEGIS	libsodium (`crypto_aead_aegis128l` / `aegis256`) [@libsodium-aegis] and a growing set of high-performance libraries; not a TLS suite
Ascon-AEAD128	reference and third-party implementations [@ascon-site]; adoption ramping since SP 800-232 [@sp800232]

Two entries on that map decide real architectures. In the browser, AES-GCM is the only AEAD you can call.WebCrypto's SubtleCrypto.encrypt recognizes RSA-OAEP, AES-CTR, AES-CBC, and AES-GCM, and AES-GCM is the only one of those that authenticates. Browser JavaScript that needs an AEAD without bundling a crypto library therefore has exactly one option [@webcrypto]. And AEGIS, still pre-RFC, is nonetheless callable in production today through libsodium's crypto_aead_aegis128l and crypto_aead_aegis256 APIs -- deployability and standardization are not the same axis [@libsodium-aegis].

Note: Eight constructions -- the seven cipher families, with the SIV line counted in its two deployable forms (AES-GCM-SIV and AES-SIV) -- all linear-time, all vetted, and not one strictly dominates the others. CAESAR chose a portfolio across three use cases on purpose, and NIST added Ascon for the constrained end. There is no "best AEAD," and any article or vendor that names one is hiding an assumption about your deployment [@caesar-portfolio].

Two more items complete the 2026 map. The CAESAR defense-in-depth winners, Deoxys-II and COLM, remain excellent and essentially undeployed [@caesar-portfolio]. And committing transforms -- the fix for the commitment edge -- exist and are cheap, but as of mid-2026 none is mandated in a shipping standard; RFC 9771 supplies vocabulary, not a required construction [@rfc9771][@bh2022-commit].

Laid side by side, those eight look interchangeable. They are not, and the tool that makes the difference legible is a single table built on the three edges.

7. The Decision Matrix

This is the one screen to memorize. Everything in the previous six sections collapses into two tables and a flowchart, all built on the same three edges.

The first table answers the only question that matters at selection time: for each construction, what happens on the first repeated nonce, what hardware does it want, and does one ciphertext bind to one key?

Construction	First repeated nonce	Hardware profile	Commitment
AES-GCM	Catastrophic: leaks P1 XOR P2, enables forgery	AES-NI + CLMUL fast	Not committing
ChaCha20-Poly1305	Catastrophic: same one-time-MAC cliff	Constant-time in software	Not committing
AES-CCM	Catastrophic	All-AES, tiny footprint	Not committing
AES-GCM-SIV	Graceful: leaks message equality only	AES-NI + CLMUL	Not committing
AES-SIV	Graceful: leaks message equality only	All-AES, two-pass	Not committing
OCB3	Catastrophic	Fast one-pass software	Not committing
AEGIS	Catastrophic	AES round function, fastest	Not fully committing
Ascon-AEAD128	Catastrophic	A few thousand gates	Committing from the sponge, no transform

Read down the commitment column and the reckoning of Section 5 gets sharper than a flat "nobody commits." Every deployed CTR, stream, and welded construction -- AES-GCM, ChaCha20-Poly1305, AES-CCM, the SIV family, and OCB3 -- is non-committing by default, which is exactly what the salamander and the portfolio-wide context-discovery attacks exploit [@dodis2018-salamander][@menda2023]. Two constructions break the pattern, and it is no accident that both fuse the tag with the cipher state instead of bolting on a separate hash: AEGIS partially commits through tag length, and Ascon commits natively from its sponge [@aegis-draft18][@rfc9771]. So commitment is not a uniform failure across the portfolio -- the commitment edge is a real axis of variation, with Ascon at the strong end, AEGIS in between, and the CTR, stream, and welded modes needing a bolt-on transform. Read the nonce column and only the SIV family survives a repeat, and only by leaking equality [@rfc8452]. The second table adds the structural and performance facts you need once the edges are settled.

Construction	Passes / parallel	Online state	Nonce	Tag	Reported speed	Inverse-free?	Best suited for
AES-GCM	1 pass, parallel	`O(1)`	96-bit	128-bit	~1 cpb with AES-NI	Yes	The default, server and client
ChaCha20-Poly1305	1 pass	`O(1)`	96-bit	128-bit	Fast in pure software	Yes	Targets without AES hardware
AES-CCM	2 pass, serial	`O(1)`	7 to 13 byte	128-bit (`CCM_8`: 64)	Modest	Yes	Constrained stacks, WPA2
AES-GCM-SIV	2 pass	`O(n)` buffered	96-bit	128-bit	~0.92 cpb on Broadwell	Yes	Nonce-reuse risk
AES-SIV	2 pass	`O(n)` buffered	None or supplied	128-bit	Modest	Yes	Key wrap, deterministic AE
OCB3	1 pass, parallel	`O(1)`	up to 120-bit	up to 128-bit	Fast in software	No (needs AES decrypt)	Elegance (now patent-clear)
AEGIS	1 pass, parallel	`O(1)`	128 or 256-bit	128 or 256-bit	~0.48 cpb (AEGIS-128L)	Yes	Maximum throughput, both endpoints
Ascon-AEAD128	1 pass	`O(1)`	128-bit	128-bit	Small, not server-fast	Yes	New constrained designs

Reported speeds are platform-dependent and should be read against live cross-platform benchmarks rather than a single headline number; the AEGIS and GCM-SIV figures come from their design papers, and the ECRYPT benchmarking project tracks the rest across many CPUs [@wu2013-aegis][@gl2015-gcmsiv][@bench-aead]. One column earns a second look: OCB3 is the only full-treatment construction that is not inverse-free, so it alone needs the AES decrypt circuit in hardware -- the honest cost behind its elegance [@rfc7253].

The tables classify. To choose, turn them into a procedure. The flowchart below is the thesis rendered as a decision tree: branch on the nonce guarantee first (it is the edge that fails hardest), then on hardware, and treat commitment as an orthogonal final step because no default provides it.

flowchart TD Q1{"Can you guarantee a unique nonce per key?"} Q1 -->|No| SIV["AES-GCM-SIV: move on the nonce edge"] Q1 -->|Yes| Q2{"AES-NI and CLMUL on both endpoints?"} Q2 -->|Yes| GCM["AES-GCM"] Q2 -->|No| Q2b{"Server or mobile software, or a constrained device?"} Q2b -->|Software| CC["ChaCha20-Poly1305"] Q2b -->|Constrained| ASC["Ascon-AEAD128"] SIV --> Q3{"Must one ciphertext bind to exactly one key?"} GCM --> Q3 CC --> Q3 ASC --> Q3 Q3 -->|Yes| ADD["Add a key-commitment transform, unless it already commits like Ascon"] Q3 -->|No| DONE["Ship it"]

The argument is now explicit and testable. "Which AEAD?" reduces to "which edge can your deployment guarantee it never triggers?" If you cannot guarantee nonce uniqueness, move right on the nonce edge to the SIV family. If you have no AES hardware, move on the hardware edge to ChaCha20-Poly1305 or, for a constrained device, Ascon. If a ciphertext must bind to exactly one key, you must add something on the commitment edge, because none of the defaults gives it to you. You can even write the function down.

{` // "Which AEAD?" is a function of three edges, not a ranking. function chooseAEAD({ nonceUnique, hasAesHardware, constrained, mustCommitToKey }) { let pick; if (!nonceUnique) { pick = 'AES-GCM-SIV'; // edge 1: cannot guarantee unique nonces } else if (hasAesHardware) { pick = 'AES-GCM'; // the internet default when AES-NI + CLMUL are present } else if (constrained) { pick = 'Ascon-AEAD128'; // edge 2: no AES hardware, only a few thousand gates } else { pick = 'ChaCha20-Poly1305'; // edge 2: no AES hardware, constant-time in software } // edge 3 is orthogonal: most picks do not commit, so you must ADD a transform -- // except Ascon, which already commits natively from its sponge (RFC 9771). const alreadyCommits = pick === 'Ascon-AEAD128'; return (mustCommitToKey && !alreadyCommits) ? pick + ' + key-commitment transform (this construction does not bind to one key)' : pick; }

console.log(chooseAEAD({ nonceUnique: true, hasAesHardware: true, constrained: false, mustCommitToKey: false })); console.log(chooseAEAD({ nonceUnique: false, hasAesHardware: true, constrained: false, mustCommitToKey: false })); console.log(chooseAEAD({ nonceUnique: true, hasAesHardware: false, constrained: false, mustCommitToKey: false })); console.log(chooseAEAD({ nonceUnique: true, hasAesHardware: true, constrained: false, mustCommitToKey: true })); `}

"Which AEAD should I use?" is not a ranking from worst to best. It is a function: pick the one whose failure mode your deployment can guarantee it will never trigger.

The table tells you which edge to move. It does not tell you why you sometimes cannot move all of them at once -- why there is no row that is graceful on reuse and single-pass and fully committing and fastest. That absence is not a gap in the engineering. It is a theorem.

8. The Limits Are Theorems

Everything so far has been a design choice: move an edge here, pay for it there. Now the hard walls -- the places where no cleverness helps, because a proof says so. There are four, and each maps to an edge; the standard graduate references develop these bounds in full [@boneh-shoup][@katz-lindell].

GCM's two ceilings, correctly attributed (the nonce edge). GCM's two limits from SP 800-38D are not conservative engineering guesses; they are where the security proof runs out -- and they come from two different mechanisms people constantly merge into one.

The per-message limit of $2^{39}-256$ bits (about 64 GiB) is 32-bit counter-mode block-space exhaustion, not a GHASH property: with a 96-bit nonce GCM forms the initial counter block $J_0 = \mathrm{IV} \parallel 0^{31} \parallel 1$, the increment function advances only the low 32 bits, so counter mode emits at most $2^{32}-2$ keystream blocks, and $(2^{32}-2)\times 128$ bits $= 2^{39}-256$ bits; exceed it and the counter wraps, repeating a keystream block -- a two-time pad (SP 800-38D Section 5.2.1.1) [@sp80038d].

The per-key cap of roughly $2^{32}$ messages under random 96-bit nonces is a different quantity entirely: the birthday bound on nonce collision, where with $q$ messages the probability of a repeat scales as $q^2 / 2^{96}$, so $q \approx 2^{32}$ keeps it negligible (SP 800-38D Section 8.3) [@sp80038d]. Hoang, Tessaro, and Thiruvengadam proved these multi-user bounds tight, validating the nonce-randomization mechanism TLS 1.3 uses [@htt2018-gcm].

Tag length obeys its own bound: for a one-time Wegman-Carter MAC, forgery probability grows with the number of attempts and shrinks with tag length -- exactly why a 64-bit CCM_8 tag is a genuine, quantifiable forgery budget rather than a free optimization [@rfc3610].

Misuse-resistance cannot be online (the nonce edge, again). This is the deepest limit in the article, and it explains a design decision that otherwise looks like laziness. Rogaway and Shrimpton proved that a deterministic, nonce-misuse-resistant AEAD cannot be online or single-pass [@rs2006]. The reason is forced by the definition: to be misuse-resistant, the ciphertext must depend on the entire plaintext (otherwise two messages sharing a prefix under a repeated nonce would leak that prefix), so the encryptor cannot emit even the first ciphertext byte until it has read the last plaintext byte. There is no implementation trick around it.

Key idea: You cannot have both streaming and misuse-resistance. It is a theorem, not a missing feature: a misuse-resistant AEAD must read the whole message before it can output any ciphertext. That single impossibility is why AES-GCM-SIV is two-pass, and why "just make GCM misuse-resistant without slowing it down" is a request for something that provably does not exist [@rs2006].

Commitment is an orthogonal axis (the commitment edge). A conventional CTR-based AEAD like GCM is provably not key-committing -- the property is simply absent from IND-CPA and INT-CTXT, so no amount of using GCM "correctly" produces it. Worse, commitment is not one property but a lattice: Bellare and Hoang proved CMT-1 (bind to the key) does not imply CMT-4 (bind to the full context), a separation RFC 9771 later codified [@bh2022-commit][@rfc9771].

And the attacks are cheaper than the naive birthday intuition suggests: Menda and coauthors found context-discovery attacks against CCM, EAX, SIV, GCM, and OCB3, including an $O(2^{n/3})$ attack on SIV using Wagner's k-tree algorithm -- well below the $2^{n/2}$ a birthday bound would suggest [@menda2023].

Releasing plaintext early is a strictly weaker world (all edges). The fourth limit is about what your decrypt function is allowed to do before it finishes.

What an implementation exposes if it emits decrypted plaintext before checking the authentication tag. The formal integrity notion for this setting, INT-RUP, is codified in RFC 9771 Section 4.3.10; security that survives releasing unverified plaintext is strictly stronger than standard AEAD security, and most fast one-pass modes satisfy it only weakly -- which is why the safe API contract is verify-then-release: never act on plaintext until the tag checks out [@rfc9771].

Now assemble the thought experiment. Imagine the ideal AEAD: graceful on nonce reuse and fully committing and online/single-pass and fastest at both the server and constrained extremes. The second limit rules out "misuse-resistant and online" together. The third makes commitment an add-on that the CTR, stream, and welded defaults lack -- Ascon, uniquely, commits natively. The performance extremes pull in opposite directions -- AES round function for servers, a few thousand gates for sensors. You cannot have all of it in one construction, and this is not because nobody has been clever enough. It is because the properties provably conflict.

Note: No single construction closes all three edges at once. That is not a temporary state of the art -- it is a set of theorems about online-ness, commitment, and hardware. The portfolio is not a failure to converge on a winner; it is the mathematically necessary shape of the solution [@rs2006][@bh2022-commit].

So that is settled, provably. Which turns the interesting question inside out. Not "which mode wins?" but "can we make the robustness the default instead of the expert's opt-in?" That is exactly where the field is fighting right now.

9. Where AEAD Still Bites

The classical problem is closed. For confidentiality and integrity, the bounds are tight and the constructions are vetted. The entire live frontier is about robustness: turning the properties nobody used to ask for into defaults nobody has to remember. Six problems are open, and every one of them has the same shape.

Committing AEAD by default. Salamanders and partitioning oracles still bite wherever a committing transform is omitted, which today means almost everywhere [@dodis2018-salamander][@len2021-partition]. The frustrating part is that the fix is cheap: Bellare and Hoang give transforms that add full commitment with no ciphertext-size increase, and Albertini and coauthors give a padding-based alternative [@bh2022-commit][@albertini2022]. Yet as of mid-2026, none is mandated by a shipping standard. RFC 9771 gives the field a shared vocabulary for the property but does not require any construction to have it [@rfc9771]. The open question is not "how?" but "why is it still opt-in?"

Misuse-resistance by default. If nonce reuse is a recurring operational reality, why is the misuse-resistant mode the escalation rather than the default? The answer is the theorem from Section 8: defaulting to SIV means defaulting to two-pass, non-streaming encryption, and a great deal of infrastructure assumes it can encrypt a stream as it arrives [@rs2006]. Whether that trade is worth making by default is an active argument, not a settled one.

The release-of-unverified-plaintext trade space. Between "buffer everything and verify first" and "stream plaintext out immediately" is a design space that streaming media, large-file transfer, and constrained receivers all care about, and the security definitions there -- INT-RUP and its relatives -- are still maturing [@rfc9771].

Hardware co-design versus conservative margin. AEGIS already beats AES-GCM on AES-NI hardware, and the appetite for line-rate encryption keeps growing [@wu2013-aegis]. Newer AES-round designs push the throughput frontier even higher, but they trade cryptanalytic maturity for speed, and their standardization status is unsettled -- AEGIS itself is still only an Internet-Draft, not an RFC [@aegis-draft18]. How much margin to spend for how much throughput is an open judgment call.

Post-quantum reality, minus the myth. This one is mostly a matter of correcting a widespread misconception.

For symmetric authenticated encryption the honest answer is: no, and you mostly need to change nothing. Grover's algorithm offers at most a square-root speedup on key search, which halves the effective key length [@grover1996], so a 128-bit key gives about 64 bits of margin against a hypothetical quantum attacker. The remedy is simply to prefer 256-bit keys -- ChaCha20 already uses one, and AES-256 is a configuration flag -- and to size tags with a margin. The post-quantum upheaval lives in key exchange and signatures, not the AEAD record layer, as developed in the post-quantum-cryptography-on-Windows post in this collection. Your record encryption survives the transition largely intact.

Constant-time without special hardware. GHASH is hard to implement in constant time without CLMUL, and table-based AES leaks timing without AES-NI -- the exact side-channel pressure that motivated both ChaCha20-Poly1305 and Ascon [@bernstein-chacha][@ascon2021]. On the smallest devices, where neither special instruction exists and power analysis is a live threat, side-channel-resistant AEAD is still an open engineering problem, and it is much of the reason NIST ran a lightweight competition at all.

None of these has a finished answer. But notice the shape of every one: they are all attempts to move an edge to a place where the deployer cannot step on it by mistake -- misuse-resistance you do not have to remember, commitment you do not have to add, side-channel safety you do not have to hand-tune. Which brings us back to the only rule that actually travels.

10. The Field Guide: Rules and Parameters

Everything above collapses into one default and a short list of deliberate escalations. Follow the ladder and you will never again pick an AEAD by folklore.

Note: Default to AES-GCM with a 96-bit nonce that is unique per key (prefer a counter over a random value), a hard cap well under $2^{32}$ messages per key, at most 64 GiB per message, and a full 128-bit tag. Use ChaCha20-Poly1305 under the same contract when you lack AES hardware. Leave the default only when a specific edge forces you, and only in the direction that edge points [@sp80038d][@rfc8439].

The escalation ladder, each rung mapped to the edge it moves:

Cannot guarantee a unique nonce? Escalate on the nonce edge to AES-GCM-SIV [@rfc8452]. Cloned VMs, counters that reset on crash, weak first-boot entropy -- if any of these is in your threat model, the graceful failure is worth the second pass.
A stack mandates it, or the device is tiny? Use AES-CCM where WPA2, BLE, or Zigbee require it [@rfc3610], and Ascon-AEAD128 for new constrained designs, now that it is a NIST standard [@sp800232].
Deterministic encryption or key wrapping? Use AES-SIV, which needs no nonce at all and is built for exactly this [@rfc5297].
You want one-pass, single-primitive elegance? OCB3 is now patent-clear, though rarely the right call given how entrenched the defaults are [@rfc7253][@kr2021-ocb].
You control both endpoints and need maximum throughput? AEGIS is the fastest option, if you can accept a pre-RFC specification [@aegis-draft18][@wu2013-aegis].
Must a ciphertext bind to exactly one key? Add a key-commitment transform -- for password-based encryption, key rotation, message franking, or multi-recipient encryption -- because no default provides it [@bh2022-commit].

Nonce generation itself is the subject of Part 2 of this series; do not re-derive it here, but do treat "unique per key" as a hard contract, not a hope. On Windows, the CNG architecture post in this collection shows AES-GCM exposed through the BCryptEncrypt API with an authenticated-cipher-mode information structure -- a concrete example of the parameters below appearing in a real API.

Pair the ladder with the support map from Section 6 when you check feasibility: browser JavaScript means AES-GCM through WebCrypto, and a target with no AES hardware means ChaCha20-Poly1305 or, if it is also tiny, Ascon-AEAD128 [@webcrypto].

The exact operational contract, per construction:

Construction	Nonce	Tag	Per-key limit	Per-message limit	On nonce reuse
AES-GCM	96-bit, unique	128-bit	~`2^32` msgs (random nonce)	`2^39 - 256` bits, ~64 GiB	Catastrophic
ChaCha20-Poly1305	96-bit, unique	128-bit	~`2^32` msgs (random nonce)	~256 GiB	Catastrophic
AES-CCM	7 to 13 byte	up to 128-bit (`CCM_8`: 64)	Set by nonce length	Set by length field	Catastrophic
AES-GCM-SIV	96-bit	128-bit	Improved (per-nonce derived keys)	~64 GiB	Graceful: message equality
AES-SIV	None or supplied	128-bit	Large	Large	Graceful: message equality
OCB3	up to 120-bit	up to 128-bit	Large	Large	Catastrophic
AEGIS	128 or 256-bit	128 or 256-bit	Large (wide nonce)	Large	Catastrophic
Ascon-AEAD128	128-bit	128-bit	Per SP 800-232	Per SP 800-232	Catastrophic

The only two numeric limits you must hard-code are GCM's, because they are the ones people breach silently: cap messages per key well under $2^{32}$ and messages under 64 GiB each [@sp80038d]. ChaCha20-Poly1305's own ~256 GiB per-message ceiling comes the same way, from its 32-bit block counter over 64-byte blocks ($2^{32}\times 64$ bytes $= 2^{38}$ bytes), per RFC 8439 [@rfc8439]. AES-GCM-SIV's per-nonce key derivation buys better multi-user bounds, which is the whole reason RFC 8452 derives fresh keys rather than reusing one [@bht2018][@rfc8452].

Every rung of that ladder has a mirror image: a misuse pattern that punishes the deployment for stepping on the edge it ignored. Each row below maps a real-code mistake to the named break that catches it.

Misuse seen in real code	Edge it triggers	Named break
Reusing a `(key, nonce)` in GCM or AEGIS	Nonce	Nonce-Disrespecting Adversaries, 2016
Assuming an AEAD binds a ciphertext to one key	Commitment	Invisible salamanders; partitioning oracles
Truncating the tag to save bytes (`CCM_8`)	Integrity budget	Wegman-Carter forgery bound
Treating AES-GCM-SIV reuse as "free"	Nonce	SIV leaks message equality
Hand-rolling Encrypt-and-MAC	Composition	Bellare-Namprempre; Krawczyk
Conflating GCM's two ceilings	Nonce	The two SP 800-38D limits

The evidence for each is in the sections above: nonce reuse on live servers [@bock2016], the salamander and its weaponization [@dodis2018-salamander][@len2021-partition], the tag-length forgery budget [@rfc3610], the SIV equality leak [@rfc8452], the composition theorems [@bn2000][@krawczyk2001], and the two ceilings [@sp80038d]. One extra caution on attribution: the live-TLS nonce-reuse break is repeatedly mis-cited to CVE-2016-0270, which actually names IBM Domino and which the NVD itself flags as commonly misapplied to other products [@cve-2016-0270].

Search your codebase for any AEAD decryption that runs under an attacker-influenced key: password-based encryption, key wrapping, token decryption, or multi-recipient envelopes. If a decrypt path lets the caller supply or guess the key and then branches on whether decryption succeeded, you are exposed to a partitioning oracle and need a key-commitment transform on top of the AEAD [@len2021-partition][@bh2022-commit].

That ladder is your inoculation against folklore. But folklore is sticky, so let us take the most common misconceptions head-on.

11. Seven Beliefs That Get Deployments Broken

No. For every nonce-respecting mode -- AES-GCM, ChaCha20-Poly1305, AES-CCM, OCB3, AEGIS, and Ascon -- a single repeated `(key, nonce)` is an immediate, total break: it leaks the XOR of the two plaintexts, and a second collision hands over the authentication subkey for universal forgery [@joux2006][@bock2016]. Only the SIV family degrades gracefully, and only to leaking message equality [@rfc8452]. No. It makes reuse *survivable*, not free. A repeated nonce still leaks message equality -- an attacker learns that two ciphertexts encrypt the same plaintext -- which can matter a great deal in the right context. Use it as a safety net, not a license to stop managing nonces [@rfc8452][@rs2006]. No to both misreadings. It is a different trade-off on the hardware edge, not a weaker cipher: constant-time in pure software with no lookup tables, which makes it the *stronger* choice on hardware without AES-NI, where table-based AES leaks timing. And it is not mandatory-to-implement in TLS 1.3 -- it is a *recommended* (SHOULD) cipher suite, while only AES-128-GCM is the MUST, per RFC 8446 Section 9.1 [@rfc8446]. Recommended, not weaker, not mandatory. No. Random 96-bit nonces are collision-safe only up to roughly $2^{32}$ messages per key, by the birthday bound. That per-key invocation cap is a different number from the per-message limit of about 64 GiB; enforce both, and prefer a counter over a random nonce when you can [@sp80038d][@htt2018-gcm]. No. Most AEADs are not key-committing. One AES-GCM ciphertext can be built to decrypt to two different meaningful plaintexts under two different keys, with the tag valid both times -- the invisible-salamanders result. If a ciphertext must bind to one key, add a commitment transform [@dodis2018-salamander][@bh2022-commit]. No. AEGIS leads on raw throughput, but it is nonce-respecting (catastrophic on reuse), not fully key-committing, and still an active Internet-Draft rather than an RFC -- it states plainly that it "is not a standard" and expires on 8 April 2026. Fastest is not the same as best for your deployment [@aegis-draft18][@wu2013-aegis]. No. Symmetric authenticated encryption survives the quantum transition. Grover's algorithm only halves effective key length [@grover1996], so prefer 256-bit keys and size tags with margin. The post-quantum upheaval is in key exchange and signatures, not the AEAD record layer. Separately, note that NIST's Ascon-AEAD128 is a tweaked variant of the CAESAR Ascon-128, not byte-identical [@sp800232].

The Function, Not the Ranking

Seven ciphers, one interface, three edges. Every break here was a deployment triggering an edge its designers deliberately moved elsewhere. The invisible salamanders triggered the commitment edge that AEAD never promised to close [@dodis2018-salamander]. The 184 TLS servers triggered the nonce edge that GHASH cannot survive [@bock2016]. The CCM_8 forgery budget is the tag-length edge priced in bits [@rfc3610]. The SIV equality leak is the residue of the one edge SIV does close -- even the fix has a seam [@rfc8452]. And OCB3's near-total absence is the patent grave, an edge that was never technical at all [@kr2021-ocb].

Put them together and the thesis is earned rather than asserted. "Which AEAD should I use?" is not a ranking from worst to best, because Section 8 proved there is no best -- no single construction can be misuse-resistant and single-pass and fully committing and fastest at once. It is a function: pick the construction whose failure mode your deployment can guarantee it will never trigger.

Master the three edges and you can place any construction on the map, predict any misuse before it ships, and evaluate whatever the next competition invents -- key-derivation-bound AEADs, the TLS record layer's next mode, or a lightweight winner that does not exist yet. The names will change. The edges will not.

Predictable or Repeated: The Only Two Ways Cryptographic Randomness Betrays You

noreply@paragmali.com (Parag Mali) — Wed, 08 Jul 2026 03:29:38 GMT

Every cryptographic key, nonce, IV, and salt is only as safe as two invisible properties: was it *unpredictable* when it had to be, and *unique* when it had to be? A cryptographically secure pseudorandom number generator (CSPRNG) manufactures unpredictability by stretching a little true entropy into unlimited output, and fails when the seed is guessable, the state cloned, the design back-doored, or the draw biased. But that is only half the story: the four roles each demand a different mix of unique, unpredictable, and secret, so a perfect generator still gets you owned if you reuse a GCM nonce or predict a CBC IV. A field guide to both halves, through every famous break.

1. Two Lines of Code

In 1996, two Berkeley graduate students needed only seconds to reconstruct the secret keys protecting Netscape's SSL. The RC4 and RSA math was flawless. The browser had simply seeded its random number generator from the time of day and a couple of process IDs, and there were only so many of those [@src-netscape1996]. In 2008, a one-line change to Debian's OpenSSL quietly collapsed the space of every key it generated, for SSH, SSL, and OpenVPN, down to about 32,768 possibilities, small enough to enumerate on a laptop over lunch [@src-debian2008].

In both cases the cipher was flawless and the keys were guessable. That is the whole problem with cryptographic randomness: it is the one primitive whose failures are invisible, and there are only two of them.

Look closely at Netscape first. Ian Goldberg and David Wagner reverse-engineered the Navigator's seeding code and found that the entire unpredictable input to the session key came from the current time, the process ID, and the parent process ID [@src-netscape1996]. An attacker who knew roughly when you connected, and process IDs live in a small range, could reproduce the seed and therefore the key. The seed was the secret, and the secret was enumerable. Nothing about RC4 or RSA had to break; the attacker simply recomputed what the generator would produce.

Debian, twelve years later, is the same mistake wearing a different costume. A developer commented out a line in OpenSSL that fed unpredictable data into the seed, silencing a memory-analysis warning about reading uninitialized bytes.The specific tool that flagged the warning is often reported as Valgrind, sometimes as Purify; the Debian advisory itself names neither, so we will not assert one. The load-bearing fact is the collapsed keyspace, which is not in dispute [@src-debian2008]. With that input gone, the only entropy left was the process ID, so the effective keyspace for roughly two years of Debian-generated keys shrank to about $2^{15}$ values, and CVE-2008-0166 was born [@src-debian2008, @src-cve2008]. Every SSH host key, TLS certificate, and OpenVPN key produced on an affected system could be pre-computed and looked up in a table.

A deterministic algorithm that stretches a short, truly-random *seed* into an arbitrarily long output stream that no efficient adversary can distinguish from uniform random bits, even after observing part of the stream. The word *cryptographically* is what separates it from an ordinary statistical PRNG: the guarantee is about unpredictability to a computationally bounded attacker, not merely about passing statistical tests.

Neither of these was a broken algorithm. Both were broken randomness, and the reason they feel like the same story is that they are the same story. A weak or misused random value emits bytes that look perfect, pass every statistical test, and are catastrophically broken anyway. The generator hums along; the ciphertext validates; the signature verifies. The failure is silent until someone else computes what you computed.

Note: A broken cipher raises an error. A bad MAC rejects. A weak or reused random value does neither: it produces output that is statistically indistinguishable from good randomness and passes every test you can run against it, while being fully predictable or fatally repeated to an attacker who knows where to look. You cannot detect the failure by inspecting the output. This is why randomness bugs survive code review, ship to production, and are found years later by the people exploiting them.

Once you have seen these two breaks side by side, the organizing claim of this article is hard to unsee. A random value can betray you in exactly one of two ways, and the failure can happen in exactly one of two places.

Key idea: There are only two ways a random value fails: it is predictable when it needed to be unpredictable, or repeated when it needed to be unique. And each failure happens in exactly one of two places: the generator that manufactures the bits, or the role that consumes them (key, nonce, IV, salt). Two knobs, two places. Every famous break in this article is a single knob turned the wrong way in a single place.

This is Part 2 of a field guide for protocol designers. Part 1 argued that the standard security definition, IND-CPA, mandates randomized encryption: a scheme that encrypts the same message the same way twice cannot hide whether two ciphertexts carry the same plaintext, so a nonce or IV is the randomness the definition demands. This part supplies, and disciplines, that randomness. By the end you should be able to look at any randomness bug, Netscape or Debian or the ones still shipping today, and name on sight which knob turned and which place failed.

Both breaks were the generator half failing on the predictability knob. To see why that failure was almost inevitable, we have to go back to the moment cryptography decided to put all of its secrecy into a string of unguessable bits.

2. Why Randomness Became a Primitive

In 1883, Auguste Kerckhoffs told cryptographers something that sounds obvious now and was radical then: assume the enemy knows your algorithm [@src-kerckhoffs1883]. Do not build your security on the secrecy of the machine, because the machine will be captured, reverse-engineered, or sold. Build it on the key. That single move relocated all of a system's secrecy into one place, a string of bits, and quietly made the unpredictability of that string the foundation of everything, without ever saying where the string should come from.

Sixty-six years later, Claude Shannon made the requirement mathematical. In Communication Theory of Secrecy Systems (1949), he proved that the one-time pad achieves perfect secrecy: a ciphertext reveals nothing whatsoever about the plaintext, not even to an adversary with unlimited computing power [@src-shannon1949].

But the proof comes with a receipt. Perfect secrecy demands a key that is perfectly random and at least as long as the message, used once. Randomness stopped being a nice-to-have and became a first-class, provable requirement, the raw material perfect secrecy is quite literally made of. Hold onto the price tag on that key; it comes back in Section 8 as the gap no engineering closes.

*Entropy* measures how much true unpredictability a source contains. For cryptography the conservative measure is *min-entropy*, defined for a distribution $X$ as $H_\infty(X) = -\log_2 \max_x \Pr[X = x]$. It is governed by the single most likely outcome, that is, by the attacker's best single guess. A source with $n$ bits of min-entropy resists guessing about as well as $n$ uniformly random bits. This worst-case framing is deliberate: an average-case measure like Shannon entropy can look healthy while one dominant value makes the source easy to guess.

But where do perfectly random bits come from? In 1951 John von Neumann issued the warning that still hangs over the whole field. Studying ways to generate random digits on the first computers, he tried the middle-square method, squaring a number and taking the middle digits as the next value, and watched it collapse into short repeating cycles or fall to zero. His verdict was characteristically sharp.

"Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin." -- John von Neumann, 1951 [@src-vonneumann1951]

The point was not that deterministic recipes are useless, but that arithmetic alone only simulates randomness; it cannot manufacture the genuine article [@src-vonneumann1951]. A formula has no unpredictability of its own. Whatever you get out was already determined by what you put in.

So for a while, randomness was treated as a scarce physical resource you harvested and rationed. In 1955 the RAND Corporation published A Million Random Digits with 100,000 Normal Deviates, an entire book whose content was a table of random numbers generated from electronic noise, sold so that scientists could look up randomness the way they looked up logarithms [@src-rand1955].Two years later the UK built ERNIE (Electronic Random Number Indicator Equipment) to draw Premium Bond winners from thermal noise. It is a lovely piece of history, but no contemporaneous engineering primary survives that we could verify, so treat it as color, not evidence [@src-ernie1957]. Randomness you could buy in a bookstore is a charming idea and a useless one for cryptography, because a published table is, by definition, known to your adversary, and it cannot produce a fresh, secret, per-session value on demand.

Von Neumann's ban is on arithmetic *manufacturing* randomness from nothing. It says nothing against *extracting* clean randomness from a messy physical source that already contains some, which is exactly what a modern entropy source does, and what a [fuzzy extractor](/blog/fuzzy-extractors-windows-hello/) does when it turns a noisy biometric into a stable key. Extraction concentrates existing unpredictability; it never conjures it. Garbage in, garbage out remains the law.

The conceptual breakthrough that made a deterministic generator respectable arrived in the early 1980s. Andrew Yao (1982) and Manuel Blum with Silvio Micali (1984) redefined what "random enough" should mean for cryptography [@src-yao1982, @src-blummicali1984]. Their answer was not statistical but computational: a sequence is cryptographically strong if no efficient algorithm, given all the output bits so far, can predict the next bit with probability meaningfully better than one-half.

A pseudorandom generator passes the *next-bit test* if no probabilistic polynomial-time algorithm, shown any prefix of the output sequence, can guess the following bit with advantage non-negligibly above $1/2$. Yao proved this test is equivalent to indistinguishability from uniform: pass the next-bit test and you pass *every* efficient statistical test at once [@src-yao1982, @src-blummicali1984, @src-katzlindell]. This is unpredictability made rigorous, and it is a yardstick no finite battery of statistical tests can ever meet.

This is knob number one, unpredictability, turned into a precise definition. Notice what it does not promise: it says nothing about where the seed comes from, and it holds only against a bounded adversary. Both caveats will return to bite. But the intellectual foundation was now in place. Shannon and Kerckhoffs had made unguessable bits the foundation of everything; von Neumann had warned that no formula can produce them from thin air. So engineers reached for the nearest deterministic recipe that merely looked random, and walked straight into the first disaster.

3. Statistical Randomness That Was Not Unpredictable

Here is a fact that should bother you. The Mersenne Twister, the default random generator in Python, Ruby, PHP, R, MATLAB, and countless other systems [@src-python-random], passes essentially every statistical randomness test ever devised. Its output is beautifully uniform, its period is astronomically long, and it is completely predictable. Observe 624 of its 32-bit outputs and you can reconstruct its entire internal state and compute every value it will ever emit, forward and backward [@src-mersenne1998]. That gap, between "looks random" and "is unguessable," is the mistake that organizes the rest of this article.

Start with the simplest offender. A linear congruential generator (LCG) produces its next value with $x_{n+1} = (a \cdot x_n + c) \bmod m$. It is fast, it is fine for shuffling a deck in a video game, and it is fatal for cryptography, because given a handful of outputs you can solve for the hidden constants and then predict everything. There is no unpredictability here at all, only arithmetic dressed up to look busy.

The Mersenne Twister (MT19937, 1998) is vastly more sophisticated, with 19,937 bits of state and excellent equidistribution, but it shares the same fatal property: it is linear over the field $\mathbb{F}_2$ [@src-mersenne1998]. Every output bit is a fixed linear function of the state bits. Collect enough outputs and you have a system of linear equations whose solution is the state. Sophistication did not buy unpredictability, because unpredictability was never what the design optimized for.

A *statistical PRNG* is built to produce output with good distributional properties: uniformity, long period, no detectable correlations. A *cryptographically secure* generator is built to a strictly harder standard: no efficient adversary who has seen past output can predict future output, the next-bit test of Section 2. The two goals are independent. The Mersenne Twister maximizes the first and fails the second completely. Passing statistical batteries is necessary but nowhere near sufficient for cryptography.

The first genuinely cryptographic generators came from the banking world. The ANSI X9.17 and later X9.31 designs used a block cipher (two-key Triple-DES in X9.17; Triple-DES or AES in X9.31) rather than a linear recurrence [@src-hac, @src-ansix931], so predicting the output required breaking the cipher itself, not solving a linear system. This was real progress.

But it moved the whole burden onto a single secret: the seed key $K$ baked into the construction. If $K$ leaks, or worse, if it is hard-coded into shipped firmware, the generator becomes fully predictable to anyone who reads the binary. That is not a hypothetical. It is exactly the DUHK attack we will meet in Section 4, where hard-coded X9.31 keys let researchers decrypt live VPN traffic [@src-duhk2018].

And then there is the naive-seeding family, the Netscape lineage: take a strong-enough algorithm and feed it a weak seed built from the clock, the process ID, and the parent process ID [@src-netscape1996]. The algorithm is irrelevant if the seed is enumerable. In 2005 the IETF wrote this lesson into a Best Current Practice, RFC 4086, whose Section 3 explicitly warns against seeding from clocks and serial numbers because they are low-entropy and guessable [@src-rfc4086]. The rule was written in blood.

{` // The cipher is never touched. Only the seed is guessed. // We model a small seed space, like Netscape's time+PID+PPID // or Debian's collapsed ~2^15..2^18 keyspace.

function prng(seed) { // a toy stand-in for the real generator let s = seed >>> 0; s = (Math.imul(1103515245, s) + 12345) >>> 0; // classic LCG constants return s; // the victim's "key material" }

const SEED_BITS = 18; // pretend this is all the entropy there was const seedSpace = 1 << SEED_BITS; // 262144 possibilities const secretSeed = Math.floor(Math.random() * seedSpace); const victimKey = prng(secretSeed); // a "256-bit key" -- but only 18 bits of real entropy

// Attacker knows the seed came from a small space and enumerates it: let tries = 0, recovered = -1; for (let guess = 0; guess < seedSpace; guess++) { tries++; if (prng(guess) === victimKey) { recovered = guess; break; } }

console.log('seed space:', seedSpace, 'possibilities'); console.log('attacker tries needed:', tries); console.log('key recovered:', recovered === secretSeed); console.log('note: a 256-bit key with 18 bits of seed entropy is an 18-bit key'); `}

Note: The clock, the process ID, the MAC address, a counter: all are low-entropy and often knowable to an attacker. A generator seeded from them has only as much unpredictability as the seed, no matter how strong the algorithm downstream. RFC 4086 codifies this as a rule: do not seed from clocks or serial numbers [@src-rfc4086]. In practice you should never seed a CSPRNG by hand at all; ask the operating system, which is the entire lesson of Sections 5 and 10.

Note: This is the first place your mental model has to shift. Statistical quality and cryptographic unpredictability are different properties, and the second does not follow from the first. The Mersenne Twister is the proof: it sails through the test batteries and collapses after 624 outputs [@src-mersenne1998]. Unpredictability is a computational claim about what an adversary can compute, not a statistical claim about how the bits are distributed. Tests can falsify a generator; they can never certify one, a limit we make precise in Section 8.

The theorists had, in fact, already named the generator that gets this right.

Blum, Blum and Shub (1986) built a generator whose unpredictability *provably* reduces to the hardness of factoring large integers: break the generator and you have factored the modulus [@src-bbs1986]. It is the gold standard on paper. It is also almost never deployed, because it computes only a few bits per modular squaring, orders of magnitude too slow for a system handing out billions of random bytes a day. It is a deliberate non-deployment: the theoretically cleanest answer, benched for being impractical, while the field builds fast generators on other assumptions.

The Mersenne Twister is not a villain, incidentally.For Monte-Carlo simulation, numerical integration, and procedural graphics, the Mersenne Twister is excellent and appropriate. It is disqualified only for cryptography, where its predictability is disqualifying. "Wrong for crypto" is not "wrong for everything." The lesson is narrower and sharper: statistical excellence is not unpredictability, and unpredictability is the only thing cryptography cares about. Turning that computational ideal into something an operating system can hand out billions of times a day took four decades and a string of named catastrophes. Here is how the generator evolved.

4. CSPRNG Design, Generation by Generation

No single insight fixed cryptographic randomness. Instead there was a ratchet. Each generation of generator worked until a named disaster exposed the one thing it got wrong, and that disaster defined the next generation. Walk the ladder and you will notice every rung is one of the two knobs failing in the generator half of the problem. The consumer-half failures, PS3 and BEAST and the rest, are on the ladder too, because they drove design changes, but their real home is Section 6.

NIST's term for the deterministic-expansion core of a modern generator: the algorithm that takes a seed and produces output bits by repeatedly applying a cryptographic function (a block cipher, a hash, or an HMAC). "DRBG" names the *expansion* half specifically; it does not include the entropy source that supplies the seed. A DRBG never creates unpredictability, it only stretches whatever the seed already contained. The short, high-entropy input a DRBG expands. Its length is set by the target security level (for example, 256 bits of seed for a 256-bit security level). Everything the generator's output can resist is bounded by the min-entropy of this seed: expand a 15-bit seed into a gigabyte of output and you have a gigabyte with 15 bits of unpredictability, which is exactly what happened to Debian.

Generation 0, physical and tabular (1949 to 1957). Harvest physical noise; if you must reuse it, print it in a book [@src-rand1955]. Superseded because a static table cannot deliver a fresh, secret, per-session value on demand, and von Neumann's middle-square degenerated to short cycles [@src-vonneumann1951].

Generation 1, statistical PRNGs with ad-hoc seeding (1960s to 1998). LCGs and the Mersenne Twister, seeded from the clock and the PID. Two knob-one failures stacked on top of each other: the generator is linearly invertible (624 outputs recover a Mersenne Twister) and the seed is enumerable (Netscape) [@src-mersenne1998, @src-netscape1996].

Generation 2, block-cipher PRNGs (ANSI X9.17 and X9.31, 1985 to 1998). A cipher instead of a linear recurrence, so the output is not linearly invertible. It failed because all security rested on a secret seed key $K$, and real products hard-coded $K$ into firmware. In 2017 the DUHK attack recovered full generator state and decrypted VPN traffic on devices such as FortiOS by exploiting a hard-coded X9.31 key, with no entropy input, no reseed, and no forward secrecy to save them [@src-duhk2018].

Generation 3, OS entropy pools (Linux /dev/random, Ts'o 1994). The operating system harvests timing from interrupts, keystrokes, and disk activity into a hashed pool, estimates how much entropy it holds, and blocks when the estimate runs low [@src-random-c, @src-random4man]. This was the first mass-deployed OS CSPRNG and a real advance, but it had three weaknesses: entropy estimation is guesswork; blocking starves fresh systems at boot and spawned years of /dev/random-versus-/dev/urandom confusion; and low boot entropy breaks real keys at scale. In 2012, Mining Your Ps and Qs scanned the internet and found that 5.57% of TLS hosts and 9.60% of SSH hosts shared keys, with thousands of RSA moduli factorable because devices generated keys before their pools had any entropy [@src-heninger2012, @src-factorable]. The independent Ron was wrong, Whit is right study found shared RSA prime factors across millions of moduli the same year [@src-lenstra2012], and in 2013 researchers factored 184 RSA keys from Taiwan's government-certified Citizen Digital Certificate smartcards, hardware that had passed FIPS and Common Criteria evaluation [@src-smartfacts, @src-coppersmith2013]. The Linux RNG's internal state had already been shown recoverable by cryptanalysis in 2006 [@src-gutterman2006], and the Windows generator was analyzed the following year [@src-dorrendorf2007].

Generation 4, standardized DRBGs (NIST SP 800-90A, 2006 onward). Hash_DRBG, HMAC_DRBG, and CTR_DRBG: a seed plus a cryptographic function stretched deterministically, with a defined reseed limit (at most $2^{48}$ generate requests) and a backtracking-resistant update step [@src-sp80090a]. And, shipped in the same document, the field's most instructive cautionary tale: Dual_EC_DRBG. It had a documented output bias, distinguishable from uniform independently of any back door (Schoenmakers and Sidorenko, 2006) [@src-schoenmakers2006], and, far worse, a trapdoor: whoever chose its elliptic-curve points $P$ and $Q$ could recover the internal state from about 32 bytes of output and predict all future output (Shumow and Ferguson, 2007) [@src-shumow2007]. It was reported in 2013 as a deliberate program [@src-reuters2013] and caught weaponized in Juniper's ScreenOS in 2015 [@src-checkoway2016]. A generator can pass every statistical test and still be owned by whoever chose its constants.

flowchart TD G0["Gen 0: physical and tabular (1949 to 1957)"] -->|"a table makes no fresh secret"| G1["Gen 1: statistical PRNGs, ad-hoc seeds"] G1 -->|"MT 624 outputs, Netscape seed"| G2["Gen 2: block-cipher PRNGs (X9.17, X9.31)"] G2 -->|"DUHK hard-coded key"| G3["Gen 3: OS entropy pools (Linux, 1994)"] G3 -->|"boot starvation, Mining Ps and Qs"| G4["Gen 4: standardized DRBGs (SP 800-90A)"] G4 -->|"Dual EC back door"| G5["Gen 5: accumulation, ChaCha20, getrandom"] G5 -->|"fork or VM-snapshot clone"| G6["Gen 6: converged validated architecture (90B, 90A, 90C)"]

Generation 5, entropy accumulation, stream-cipher cores, and non-blocking syscalls (1999 to 2022). Yarrow (1999) [@src-yarrow1999] led to Fortuna (2003), whose 32 staggered pools abolish entropy estimation altogether [@src-fortuna2003, @src-freebsd-random]. The output core moved to ChaCha20, Bernstein's refinement of his earlier Salsa20 cipher [@src-chacha, @src-snuffle]: OpenBSD re-cored arc4random on it in 2014 [@src-arc4random], and Linux replaced its non-blocking pool with a ChaCha20 CRNG in kernel 4.8, October 2016 [@src-linux48].Pin this milestone at Linux 4.8 (October 2016), commit e192be9d9a30555. A "5.6 / 2020" figure sometimes cited refers to an unrelated change in how /dev/random handles O_NONBLOCK, not the ChaCha20 output core [@src-linux48]. Jason Donenfeld's 5.17 to 5.18 rewrite in 2022 added BLAKE2s extraction, per-CPU fast key erasure, and unified /dev/random with /dev/urandom [@src-donenfeld2022]. The getrandom(2) syscall (2014) blocks once at boot until the pool is seeded, then never again [@src-getrandom]. One hazard the generator alone cannot fix remained: fork() and VM-snapshot cloning duplicate the entire generator state and re-emit identical values, a uniqueness failure measured in the wild by Ristenpart and Yilek (2010) and Everspaugh et al. (2014) [@src-ristenpart2010, @src-everspaugh2014].OpenBSD kept the name arc4random for source compatibility even though the "RC4" it originally stood for is long gone; since 2014 the bytes come from ChaCha20 [@src-arc4random].

Generation 6, the converged validated architecture (2018 to 2026, current state of the art). A validated entropy source (SP 800-90B) [@src-sp80090b] seeds a computational DRBG (SP 800-90A) [@src-sp80090a], and the two are tied together by SP 800-90C [@src-sp80090c], with continuous reseeding, forward secrecy, clone defense, and no single component trusted on its own. We draw the whole pipeline in Section 5.

flowchart TD K["State: Key and counter block V"] --> G["Generate: AES-CTR keystream from Key and V"] G --> O["Hand the requested bytes to the caller"] G --> U["Update: re-derive a fresh Key and V"] U --> K2["New state replaces the old one, so prior output cannot be recomputed (backtracking resistance)"] K2 --> R["After up to 2 to the 48 requests, reseed from the entropy source"] R --> K

Now put every break on one page. This is the spine of the article: read down the two middle columns and each disaster resolves to a single knob turned the wrong way in a single place.

Table 1: The Failure Catalog.

Break	Year	Knob broken	Place	Root cause	Lesson
Netscape SSL [@src-netscape1996]	1996	predictable	generator	Seed was time + PID + PPID	Never seed from the clock or a PID
Debian OpenSSL [@src-debian2008]	2008	predictable	generator	Entropy input removed, keyspace ~2^15	Never delete an entropy source
PS3 signing key [@src-fail0verflow2010]	2010	repeated	role (nonce)	Same ECDSA nonce k on every signature	A signature nonce must be unique
Android Bitcoin [@src-android2013]	2013	repeated	role (nonce)	SecureRandom mis-init repeated k	Seed the RNG before drawing a nonce
Mining Ps and Qs [@src-heninger2012]	2012	predictable	generator	Keys generated before pool seeded	Do not make keys at cold boot
Dual_EC_DRBG [@src-shumow2007]	2007-15	predictable	generator	Trapdoor in chosen points P, Q	Choose constants verifiably; never trust one source
VM-snapshot reset [@src-ristenpart2010]	2010-14	repeated	generator	Snapshot clones generator state	Force a reseed after any clone
DUHK [@src-duhk2018]	2017	predictable	generator	Hard-coded X9.31 seed key	Never hard-code the seed key
BEAST [@src-cve2011]	2011	predictable	role (CBC IV)	TLS 1.0 chained the previous ciphertext block as IV	A CBC IV must be unpredictable
GCM nonce reuse in TLS [@src-cve2016]	2016	repeated	role (nonce)	Repeated 96-bit nonce under one key	Never reuse a GCM nonce per key
CTR_DRBG cache attack [@src-cohney2020]	2020	predictable	generator	AES-table cache timing leaks state	Prefer a constant-time core
PuTTY P-521 [@src-putty2024]	2024	predictable	role (nonce)	Biased k, first 9 bits always zero [@src-putty-oss]	A nonce must be uniform or derived

Two columns, two values each. Every row is one knob times one place. Hold this table in mind; Section 10 hands it back to you as the Common Misuse catalog, the same evidence seen from the developer's chair.

Dual_EC did more than embarrass a standard; it changed how the field designs generators. Before it, the working assumption was "trust the standardized construction." After it, the assumption became "mix multiple sources and trust no single one, and choose your constants so anyone can verify there is no hidden structure." That is why modern kernels fold hardware RNG output *into* a pool rather than using it raw, why NIST's newer curves come with published nothing-up-my-sleeve derivations, and why SP 800-90C validates the *whole* pipeline instead of asking you to trust the source and the DRBG separately [@src-sp80090c].

The lesson generalizes: a generator that can be back-doored by whoever picks its constants is not secure just because it passes tests. This philosophy shapes Sections 5 and 7.

The most surprising fix of all runs the other direction. After the PlayStation 3, the Bitcoin wallets, and eventually PuTTY, all showed that a random per-signature nonce is catastrophic when it repeats or leaks,Recovery differs by failure mode. A repeated nonce means two signatures share $k$, so simple algebra recovers the private key, as on the PlayStation 3. A biased nonce, like PuTTY's nine always-zero leading bits, falls to a lattice attack on the Hidden Number Problem: each signature leaks a few key bits, and lattice reduction solves for the key once about 60 accumulate [@src-putty-oss]. RFC 6979 (2013) responded by removing the randomness: it derives the ECDSA nonce deterministically from the private key and the message, so that signers, in the RFC's own words,

"...do not need access to a source of high-quality randomness." -- RFC 6979, on deterministic ECDSA nonces [@src-rfc6979]

The fix for a randomness disaster was less randomness. That inversion is the clue that the generator is only half the story, and often not the interesting half. Seven generations and a dozen named disasters later, the modern architecture is strikingly simple once you see what every break was really teaching. Strip away the history and one design remains, built from two layers most engineers never separate.

5. The Modern Architecture, and the Two Layers Underneath It

Every fix in Section 4 was pushing toward one shape, and here it is. A true entropy source seeds a computational DRBG that stretches a little unpredictability into unlimited output, with continuous reseeding, forward secrecy, and clone defense, and with distrust of any single component built in. Nothing exotic. Just forty years of scar tissue drawn as a single pipeline.

flowchart LR E["Entropy sources: interrupts, RDSEED, TPM"] --> A["Accumulate and condition"] A --> D["DRBG expansion: AES-CTR or ChaCha20"] D --> I["OS interface: getrandom, arc4random, BCryptGenRandom"] I --> R["Consumer roles: key, nonce, IV, salt"] D -.->|"reseed and fast key erasure"| A

Read it left to right. Physical noise (interrupt timing, the RDSEED instruction, a TPM) is accumulated and conditioned into a seed. A DRBG expands that seed into as many bytes as anyone asks for. An operating-system interface hands those bytes to your program, which pours them into one of four roles. The dotted return edge carries the two properties that keep the whole thing honest: reseeding, which periodically mixes fresh entropy back in, and forward secrecy, which we define in a moment.

The distinction the whole subject turns on is the one from Sections 2 and 3, now with a job. Computational unpredictability is about what an adversary can compute, and the DRBG delivers it by construction, provided the seed carried real entropy. A DRBG does not create entropy; it stretches it. Garbage seed in, garbage out, which is why the entire left side of the diagram exists.

A generator has *forward secrecy* if an attacker who compromises its state at some moment still cannot reconstruct output it produced *earlier*. ChaCha20-based generators achieve it by *fast key erasure*: after generating a block, the first bytes of fresh keystream overwrite the generator's own key, and only the remaining bytes are handed out [@src-donenfeld2022]. The key that produced earlier output no longer exists anywhere, so a later memory capture cannot roll time backward. Periodically folding fresh entropy into the generator's state, on a schedule (SP 800-90A mandates a reseed within $2^{48}$ requests) or on an event (after a `fork()`, after a VM resumes from a snapshot) [@src-sp80090a]. Reseeding gives *backward* secrecy: even if the state was compromised, once enough new entropy is mixed in, future output becomes unpredictable again. sequenceDiagram participant S as Generator state participant O as Output S->>O: emit one block of keystream S->>S: overwrite the key with fresh keystream Note over S: the old key is gone, so earlier output cannot be recomputed

Now the correction that clears away half the folklore in this field: the difference between an OS interface and a CPU instruction. These are two different layers, and conflating them is the source of endless bad advice.

An OS-provided interface is what your code should call: getrandom(2) or /dev/urandom on Linux, getentropy(3) or arc4random(3) on BSD and macOS, BCryptGenRandom or ProcessPrng on Windows [@src-getrandom, @src-arc4random]. On Linux, getrandom blocks once until the pool is first seeded and then never again, which is what finally ended the /dev/random-versus-/dev/urandom dilemma at the interface level [@src-getrandom, @src-lwn2014].

A CPU instruction is what the operating system consumes, not what you call. Intel's RDRAND returns the output of an on-die hardware DRBG; RDSEED returns near-raw, seed-grade entropy [@src-intel-drng]. The kernel does not hand these to you and does not trust them on their own. It mixes them into its pool alongside other sources, precisely the post-Dual_EC discipline: never let one opaque component be the whole answer [@src-donenfeld2022].

Note: The rule is a single sentence: applications call the OS interface; only the kernel touches the CPU instruction. If you find yourself calling RDRAND directly in application code, you have skipped the layer that mixes, reseeds, and forward-secures the output, and you are trusting one piece of opaque silicon with the whole job, the exact mistake Dual_EC taught the field never to make [@src-shumow2007]. Call getrandom, getentropy, or BCryptGenRandom, and let the kernel decide how much to trust the hardware [@src-getrandom, @src-donenfeld2022].

flowchart TD APP["Your application"] -->|"calls this"| OSI["OS interface: getrandom, arc4random, BCryptGenRandom"] OSI --> POOL["Kernel entropy pool and DRBG"] POOL --> OSI RDRAND["CPU RDRAND: on-die DRBG output"] -.->|"mixed in, never trusted alone"| POOL RDSEED["CPU RDSEED: seed-grade entropy"] -.->|"mixed in, never trusted alone"| POOL So why not skip the pool and read `RDRAND` directly? Because mixing is what turns "trust no single source" from a slogan into a working defense. If the kernel folds `RDRAND` into a pool that also holds interrupt timing and a saved seed, then even a compromised `RDRAND` cannot by itself make the output predictable, because it is only one of several inputs to the mix. Trust is not placed in any one component; it is *diluted* across all of them. We return to the hardware-versus-software question in Section 7.

Seen this way, the generator's entire job is to manufacture knob number one, unpredictability, and to keep manufacturing it reliably. Everything else in the pipeline, reseeding, forward secrecy, clone defense, architected distrust, exists to stop that one property from silently failing. The four decades of history collapse into a single sentence: keep the seed unguessable, keep the state fresh, and never let one component be the whole answer.

Windows implements this same pipeline with a CTR_DRBG tree behind BCryptGenRandom and ProcessPrng; the platform-specific internals, including how the kernel seeds per-processor generators and defends against VM cloning, are the subject of a dedicated companion article on the Windows CSPRNG, and the CNG API surface itself lives in a companion post. We will not re-derive them here.

That is the generator half, essentially solved as engineering. But a flawless generator is only half the battle, because the same perfect byte stream is catastrophic if you pour it into the wrong role. What, exactly, does each role demand?

6. What Ships in 2026, and the Four Roles

By 2026 the generator architecture is settled engineering, and most real-world breaks happen after the generator, when good bytes are used wrong. So we cover what ships in a paragraph, then spend the rest of the section on the practitioner's heart of the matter: the four roles and the exact property mix each one demands.

What ships: the SP 800-90A trio (CTR_DRBG, HMAC_DRBG, Hash_DRBG) [@src-sp80090a]; Fortuna on FreeBSD [@src-freebsd-random]; and ChaCha20-based generators reachable through Linux getrandom and OpenBSD or macOS arc4random [@src-donenfeld2022, @src-arc4random].

The 2025 capstone is SP 800-90C, which stops treating "the entropy source" and "the DRBG" as separate acts of faith and validates the whole pipeline as one of four RBG construction classes: RBG1 (seeded once, no ongoing source, for constrained devices), RBG2 (continuous access to a validated SP 800-90B source, the common server case), RBG3 (an enhanced, full-entropy construction combining a physical source directly with a DRBG), and RBGC (chained constructions built from approved components) [@src-sp80090c].

A pre-draft SP 800-90A Rev. 2 adds an XOF_DRBG built on the SHAKE functions from FIPS 202, forward-looking but not yet final [@src-sp80090a-r2].

Post-quantum cryptography makes randomness more foundational, not less. The 2024 standards each draw fresh CSPRNG output per operation: ML-KEM consumes random seeds during key generation and encapsulation [@src-fips203], ML-DSA draws a fresh value per signature [@src-fips204], and SLH-DSA pulls several seeds per operation [@src-fips205]. More draws mean more surface for a weak generator to ruin, which is why the post-quantum migration guides for these algorithms treat the CSPRNG as a first-class dependency.

Now the centerpiece. There are exactly four roles a random value plays in cryptography, and each demands a different combination of three properties: uniqueness, unpredictability, and secrecy. Getting the combination wrong is how a perfect generator still gets you owned.

Table 2: The four roles and what each one owes.

Role	Unique?	Unpredictable?	Secret?	Canonical failure if violated
Key	yes	yes	yes	Guessable key: Netscape 1996, Debian 2008 [@src-debian2008]
CBC IV	yes	yes	no	Predictable IV enabled BEAST [@src-cve2011]
CTR/GCM nonce	yes	no	no	Repeated nonce leaks the auth key: GCM in TLS 2016 [@src-cve2016]
Salt	yes	no (but random)	no	Reused or global salt enables shared precomputed tables

Read the differences, because they are the whole point.

A *nonce* is a "number used once." Its one absolute requirement is *uniqueness* per key: no two operations under the same key may ever use the same nonce. A nonce does *not* have to be unpredictable, and it does *not* have to be secret. A simple monotonic counter is a perfectly good nonce, and often the safest one, because a counter cannot accidentally repeat the way random draws can. An *IV* is the per-message starting value for a block-cipher mode, and its requirement depends entirely on the mode. In CBC, the IV must be both *unpredictable* and *unique*. In CTR and GCM, the value the spec labels "IV" is really a *nonce*, and needs only to be *unique*. The word "IV" therefore does not name a fixed requirement; the *mode* names the requirement. This is the single most common source of confusion in applied randomness.

The nonce-versus-IV correction deserves stating precisely, because folklore gets it backwards. The same value slot is called an "IV" or a "nonce" depending on the mode, and the mode dictates the requirement.

A CBC IV must be unpredictable and unique. TLS 1.0 chained CBC records by using the previous ciphertext block as the next IV, which is predictable, and that predictability is exactly what the BEAST attack exploited in 2011 (CVE-2011-3389). TLS 1.1 restored a fresh random IV, and the modern answer is to move to authenticated encryption entirely [@src-cve2011, @src-imperialviolet].

A CTR or GCM nonce, by contrast, needs only to be unique; unpredictability buys nothing. But uniqueness is absolute: reuse a GCM nonce under one key even once and you immediately leak the XOR of the two plaintexts, and the reused authentication tags give a polynomial equation whose roots recover the GHASH subkey $H$ (Joux's forbidden attack), enabling universal forgery. That failure was measured across live TLS servers in 2016 under the name "Nonce-Disrespecting Adversaries" and captured for IBM Domino as CVE-2016-0270 [@src-cve2016].

Note: GCM's security collapses the instant a nonce repeats under a given key. Two messages encrypted with the same key and nonce leak the XOR of their plaintexts and, worse, give up the polynomial equation that recovers the GHASH subkey $H$, after which an attacker can forge arbitrary messages that pass authentication [@src-cve2016]. There is no partial failure here and no recovery: one repeat is game over for that key. Use a counter, or if you cannot guarantee uniqueness across senders, use a nonce-misuse-resistant mode (Section 7).

flowchart TD V["A value in the IV or nonce slot"] --> Q{"Which mode?"} Q -->|"CBC"| CBC["Must be unpredictable AND unique. A predictable IV enabled BEAST"] Q -->|"CTR or GCM"| CTR["Must be unique only. Reuse leaks the authentication key"]

The last role, the salt, is where a different myth does the damage.

A *salt* is a unique, random value added to a password before hashing, and it is *not secret*: it is stored in plaintext right next to the resulting hash. Its only job is uniqueness, so that identical passwords hash to different values and one precomputed table cannot attack many accounts at once. A salt needs to be unique and random; it does not need to be unpredictable or hidden.

Believing a salt must be secret, or reusing one global salt across every password, is a common and wrong model. A per-password random salt of 16 bytes, stored openly, does its entire job; hiding it adds nothing, and sharing it across users throws the job away, because then one precomputed table works against every account again [@src-nist-63b].The same reuse math retired WEP: a short per-packet IV space made keystream reuse inevitable, which broke the confidentiality of the whole scheme [@src-bgw2001]. That story belongs to cipher agility and is told in the SChannel post.

{` const crypto = require('crypto');

// KEY: secret + unpredictable + high-entropy. Straight from the OS CSPRNG. const key = crypto.randomBytes(32); // 256 bits, for AES-256 or ChaCha20

// GCM NONCE: unique only. A monotonic counter guarantees uniqueness by construction. let counter = 0n; function nextNonce() { const n = Buffer.alloc(12); // 96-bit GCM nonce n.writeBigUInt64BE(counter, 4); // 64-bit counter in the low bytes counter += 1n; // never repeats under this key return n; }

// SALT: unique + random, stored in the clear next to the password hash. const salt = crypto.randomBytes(16); console.log('salt is PUBLIC, stored with the hash:', salt.toString('hex'));

// The WRONG lines, for contrast: // const nonce = Buffer.alloc(12); // reused -> repeated knob, in a role // const salt = Buffer.from(String(Math.random())); // Math.random is not a CSPRNG -> predictable

console.log('key bytes:', key.length, 'nonce bytes:', nextNonce().length); console.log('same generator, three different disciplines'); `}

Key idea: A perfect generator poured into the wrong role is as fatal as a broken generator. The key must be secret, unpredictable, and high-entropy; the CBC IV unpredictable and unique; the CTR or GCM nonce unique and nothing more; the salt unique, random, and public. Four roles, one generator, four different disciplines. Learn which mix each role owes and most of the Failure Catalog stops being possible.

Now the reader knows what each role demands. But which generator should manufacture the bits, and is a hardware RNG better than a software one? The honest answer overturns the question.

7. CTR_DRBG, Fortuna, and ChaCha, and Hardware Versus Software

"Which cipher makes the best RNG?" is the wrong question. The real axis is architecture: how you source entropy and how you expand it. Get that framing right and the "competitors" turn out to be solving different halves of the same problem.

Start with a category correction. Fortuna is largely an accumulator, the sourcing half: its contribution is how it collects and schedules entropy. CTR_DRBG and ChaCha20 are expanders, the expansion half: their contribution is how they stretch a seed. A real system runs an accumulator front-end feeding an expander core, so "CTR_DRBG versus Fortuna" is partly a comparison between a fuel tank and an engine.

Table 3: Deployed generators, head to head.

Property	CTR_DRBG	HMAC_DRBG	Fortuna	ChaCha20
Output cost	~1 AES per 128 bits	~1 HMAC per block	cipher core per block	1 ChaCha permutation per 512 bits
State size	Key + 128-bit V	two hash-length values	32 pools + generator key	256-bit key, per CPU
Forward secrecy	Update re-derives (Key, V)	Update re-derives (Key, V)	frequent generator rekey	fast key erasure
Side channel	AES-table lookups can leak [@src-cohney2020]	HMAC, low risk	depends on core	constant-time by design [@src-chacha]
Entropy philosophy	assumes a 90B seed	assumes a 90B seed	accumulates, no estimate	assumes a pool seed
Reseed	up to 2^48 requests	up to 2^48 requests	32-pool staggered	continuous, per CPU
FIPS 140	yes, validated	yes	no	not yet
Concurrency	per instance	per instance	single system RNG	per CPU since Linux 2022
Best suited for	FIPS boundaries, AES-NI	deterministic signatures	accumulator front-end	modern software default

Two 2026 differences decide most real choices. First, ChaCha20 is constant-time by design, with no secret-dependent branches or table lookups,Constant-time means the code's execution path and memory-access pattern do not depend on secret values, so an attacker watching timing or cache behavior learns nothing about the state. so it sidesteps the cache-timing state-recovery that Cohney et al. demonstrated against real CTR_DRBG deployments in 2020 [@src-cohney2020, @src-chacha]. Second, CTR_DRBG holds FIPS 140 validation, which is exactly why it still dominates inside compliance boundaries and on AES-NI hardware despite the side-channel result [@src-sp80090a]. Linux's 2022 move to per-CPU ChaCha instances closed the historic concurrency gap that once favored other designs [@src-donenfeld2022].

Two honest philosophies for the sourcing half. *Estimation* (NIST SP 800-90B) models the noise source, computes a min-entropy estimate, and runs health tests to catch degradation [@src-sp80090b]. *Accumulation* (Fortuna) refuses to estimate at all: it spreads incoming events across many pools and drains them on a schedule that guarantees at least one pool eventually holds enough entropy, whatever the true rate [@src-fortuna2003, @src-cryptoeng]. One measures; the other structures itself so that measurement is unnecessary. flowchart TD EV["Entropy events arrive"] --> RR["Round-robin into pools P0 through P31"] RR --> P0["P0 drains at every reseed"] RR --> PI["Pi drains only when 2 to the i divides the reseed number"] RR --> P31["P31 fills rarely but very deeply"] P0 --> RK["Reseed the generator key"] PI --> RK P31 --> RK

The hardware-versus-software debate resolves the same way: not by choosing, but by mixing.

Table 4: Hardware RNG versus software DRBG.

	Hardware (RDRAND/RDSEED, TPM)	Software DRBG (CTR/ChaCha)
Source of unpredictability	physical noise on the die	deterministic expansion of a seed
Auditability	opaque, you cannot inspect silicon	open, testable, formally analyzable
Failure mode	silent bias or back door (the Dual_EC lesson) [@src-shumow2007]	bad seed, state clone, side channel
Forward secrecy	vendor-dependent	explicit (key erasure or Update)
Verdict	seed-grade input only	auditable expansion + forward secrecy

Hardware supplies physical unpredictability but is a black box you cannot audit; software supplies auditable, forward-secret expansion but only stretches a seed. The field's resolution is to fold hardware output into the pool rather than trust it directly [@src-donenfeld2022]. "Trust RDRAND alone" fell out of favor precisely because Dual_EC proved silicon and standards can both be compromised [@src-shumow2007].

Note: This article cross-references the Windows CSPRNG rather than duplicating it. How CNG assembles a per-processor CTR_DRBG tree, seeds it, and defends against VM cloning is covered in full there.

On the consumer side, the same "which construction" question has its own honest table, this time about how much uniqueness burden you are willing to carry.

Table 5: Plain GCM versus GCM-SIV versus deterministic.

	AES-GCM (counter nonce)	AES-GCM (random nonce)	AES-GCM-SIV	RFC 6979 (signatures)
Uniqueness burden	on the caller (counter)	birthday cap ~2^32 per key	tolerates repeats	none (deterministic)
Cost vs GCM	1x	1x	~2x	not applicable
Catastrophic on reuse?	yes	yes	no	not applicable
Use when	you control a counter	low message volume	uniqueness not guaranteed	ECDSA/DSA nonces

AES-GCM-SIV (RFC 8452) derives its internal inputs from a synthetic value computed over the nonce and the plaintext, so a repeated nonce with different messages no longer leaks the authentication subkey [@src-rfc8452]. In the RFC's own words, such schemes are built so that they

"...do not fail catastrophically if a nonce is repeated." -- RFC 8452, on AES-GCM-SIV [@src-rfc8452]

It costs roughly twice a plain GCM encryption and is the right answer when you cannot guarantee unique nonces, for example across many independent senders. RFC 6979 is the other escape hatch, for signatures: derive the nonce deterministically and the uniqueness problem disappears entirely [@src-rfc6979].

It is tempting to declare a winner between NIST's estimation and Fortuna's accumulation, but both ship in 2026 and neither is wrong. Estimation gives you a number you can put in a validation report and a health test that fires when the source degrades [@src-sp80090b]. Accumulation gives you robustness without ever having to trust a number, at the cost of absorbing a burst of fresh entropy more slowly, since high-index pools reseed rarely [@src-fortuna2003]. The mature view is that they are different risk trades over the same unmeasurable quantity, which is the subject of the next section.

The engineering looks won: mix a validated source into a constant-time, forward-secret DRBG behind an un-misusable syscall. So why does the field still call randomness an open problem? Because underneath the engineering sit limits no code can cross.

8. What You Cannot Know or Prove

Everything so far was engineering, and engineering can be improved. This section is about the floor beneath it: things you fundamentally cannot do, no matter how good your code is. This is where confidence turns to humility.

Min-entropy is unmeasurable. You cannot measure the min-entropy of an unknown physical source from its output; you can only estimate it under a model of how the source behaves. SP 800-90B's entire apparatus, its estimators and its health tests, is a disciplined estimate, not a proof [@src-sp80090b]. The whole tower we built in Sections 5 through 7 rests on this one estimate, and if the model is wrong, everything above it is quietly weaker than its paperwork claims.

Statistical tests can falsify, never certify. SP 800-22 is explicitly hypothesis testing with P-values: a suite can reject a generator, but it can never prove unpredictability [@src-sp80022]. The proof by counterexample is the Mersenne Twister from Section 3: it passes the standard batteries and is fully broken by 624 outputs [@src-mersenne1998].The named suites, SP 800-22, Dieharder, TestU01, are useful for catching gross defects and nothing more. Running more of them does not raise your security; it only lowers the chance you shipped something obviously broken. "More tests" is not "more secure."

Every deployed computational CSPRNG rests on an unproven assumption. The best a deterministic generator achieves is the next-bit guarantee, but that guarantee is conditional: cryptographically secure generators exist if and only if one-way functions exist, an equivalence whose hard direction (any one-way function yields a generator) was proved by Hastad, Impagliazzo, Levin, and Luby [@src-hill1999, @src-katzlindell]. The existence of one-way functions is itself unproven, since it would imply that $P \neq NP$. There is no unconditionally secure pseudorandom generator. Every CSPRNG you have ever used is secure only relative to a hardness assumption no one has proved.

For an $m$-bit random value drawn repeatedly, collisions become likely after roughly $2^{m/2}$ draws, not $2^m$, because the chance is dominated by pairs. For a 96-bit GCM nonce, the 50% collision point sits near $2^{48}$ draws. This is a *provable ceiling*, not an implementation defect. It is why SP 800-38D caps random 96-bit nonces well below that point, at about $2^{32}$ messages per key, to keep the repeat probability negligibly small [@src-sp80038d].

The birthday bound is a hard uniqueness ceiling. Randomness cannot buy its way past its own birthday bound. The only escapes are to stop drawing randomly, using a counter whose uniqueness is structural, or to tolerate repeats with a nonce-misuse-resistant mode. There is no third option in which random draws stay unique forever.

{` // Birthday bound: for an m-bit value, collisions get likely near 2^(m/2). const m = 96; // GCM nonce size in bits

const fiftyPercentPoint = Math.pow(2, m / 2); // ~2^48 draws for 96 bits console.log('nonce size:', m, 'bits'); console.log('~50% collision near 2^' + (m / 2) + ' =', fiftyPercentPoint.toExponential(3), 'draws');

// Birthday approximation for n draws into a 2^nBits space: function collisionProb(nBits, n) { const space = Math.pow(2, nBits); return 1 - Math.exp(-(n * (n - 1)) / (2 * space)); }

// SP 800-38D caps random 96-bit nonces at ~2^32 messages per key. // Why 2^32 and not 2^48? Because the cap targets a TINY probability, not 50%: const cap = Math.pow(2, 32); console.log('at 2^32 draws, P(collision) =', collisionProb(96, cap).toExponential(3)); console.log('that is the safety margin SP 800-38D buys by capping at 2^32'); `}

Back doors are undetectable by construction. Dual_EC is the proof once more: a standardized generator can pass every statistical test and still be fully predictable to whoever chose its constants $P$ and $Q$ [@src-shumow2007]. You cannot test your way out of this, which is exactly why the field moved to verifiable constants and architected mixing rather than after-the-fact detection.

Clone defense is mitigation, not elimination. fork() and VM-snapshot cloning duplicate a good generator [@src-ristenpart2010, @src-everspaugh2014]. VM Generation ID counters, library fork detection, and forced reseeds shrink the window but do not close it; Ferguson's 2019 Windows RNG whitepaper is explicit that a guest cannot fully solve the snapshot problem on its own, because it may not even know it was cloned until it has already emitted duplicate output [@src-ferguson2019win].

And beneath all of it, the permanent gap Shannon left us in Section 2. His one-time pad is unconditionally secure but unscalable, a fresh truly-random key as long as every message. The CSPRNG is scalable but assumption-dependent. No cryptography closes that gap; we simply chose the scalable side and spend the rest of our effort defending it.

flowchart TD START["A randomness bug"] --> K{"Which property failed?"} K -->|"predictable"| P{"Where did it fail?"} K -->|"repeated"| R{"Where did it fail?"} P -->|"generator"| PG["Netscape, Debian, Dual EC, DUHK, CTR_DRBG cache"] P -->|"role"| PR["BEAST (CBC IV), PuTTY (biased nonce)"] R -->|"generator"| RG["VM-snapshot clone"] R -->|"role"| RR["PS3, Bitcoin, GCM nonce reuse"]

Key idea: You cannot certify unpredictability; you can only fail to falsify it. Min-entropy is estimated, not measured; statistical tests reject but never prove; the whole construction rests on an unproven hardness assumption; and back doors leave no trace in the output. The entire discipline of cryptographic randomness is the management of a risk you cannot directly measure. The surprise is not that randomness sometimes fails. It is that it works as well as it does.

If entropy is unmeasurable and unpredictability uncertifiable, where is randomness still actively failing today?

9. Where Randomness Is Still Failing

The architecture is settled; the frontier is not. Here are six places where cryptographic randomness is still an open engineering or standards problem in 2026, honestly contested rather than adjudicated.

Early-boot entropy on embedded devices, IoT, and VMs. A device that must generate a key seconds after first power-on, before its pool has filled, is the exact condition Mining Your Ps and Qs measured in 2012, and it still recurs in modern IoT field scans [@src-factorable]. Partial fixes exist: getrandom blocking once until seeded [@src-getrandom], saved-seed files such as FreeBSD's /boot/entropy [@src-freebsd-random], and mixing in RDSEED or jitter-based entropy at boot. None fully solves a cold device with no stored seed and no timing history.

Validating entropy sources at fleet scale. SP 800-90B health tests run per device, but proving that a noise-source model holds across millions of heterogeneous units, over years, across temperature swings and silicon aging, is unsolved [@src-sp80090b]. Fortuna's accumulation sidesteps estimation but not validation: you still have to argue the events carry entropy at all.

Making nonce-misuse resistance the default. Plain AES-GCM fails catastrophically on a single nonce repeat [@src-rfc8452], yet plain AES-GCM, not AES-GCM-SIV, remains the default AEAD offered by most general-purpose libraries. This is an adoption and standards gap, not a research one. TLS 1.3's fixed-IV construction and the pending SP 800-38D revision help at the protocol layer [@src-sp80038d-news], but general-purpose libraries still hand developers the sharp tool by default.

Post-quantum's larger draws and wider side-channel surface. ML-DSA and its siblings consume more randomness and add rejection-sampling and secret-dependent code paths, which means more surface for a weak generator or a timing leak to do damage [@src-fips204]. The post-quantum migration is an opportunity to get seeding right and a risk if it is not.

Trusting hardware RNGs after Dual_EC. "Never trust a single hardware source" is settled practice, but a general, auditable way to earn trust in opaque silicon is open [@src-shumow2007]. Cohney's cache attack showed that even a correct-on-paper CTR_DRBG can leak through microarchitectural side channels, so "the math is right" is not the end of the story [@src-cohney2020].

Formally verifying RNG code end to end. Proofs typically assume "the implementation does not leak," but real code leaks through caches, forks, and uninitialized memory. Individual primitives have been verified, but a whole-pipeline proof under a realistic leakage model, from entropy source through DRBG to syscall, is not yet routine.

Note: Notice the pattern across all six. The expansion problem, how to stretch a good seed into unlimited unpredictable output, is essentially solved: constant-time cores, forward secrecy, validated constructions. What remains open is sourcing (getting and validating real entropy, especially at boot and at fleet scale) and trust (how much to believe an opaque hardware source, and how to prove a whole implementation does not leak). The hard part was never the arithmetic. It is the physics and the epistemics around it.

Physical noise is quietly back in fashion, incidentally.RDSEED, on-chip ring-oscillator sources, quantum RNGs, and TPMs are all seeing renewed use as entropy inputs. The difference from the pre-Dual_EC era is that they are mixed in, never trusted raw. The physics returned; the blind trust did not. These are the field's problems. But you have to ship code tomorrow. What are the rules, concretely, per role, that keep you out of the Failure Catalog?

10. The Field Guide, Made Operational

Everything collapses to a short set of rules of the form "use X with these parameters in case Y." This is the two-knobs thesis turned into a checklist, and its mirror, the Common Misuse catalog, is simply the Failure Catalog seen from the developer's chair.

Rule zero for the generator: never roll your own, and never hand-seed. Always call the operating system's CSPRNG.

Table 6: Generator, call this, not this.

Platform	Call this	Never this
Linux	`getrandom(2)`, `/dev/urandom` [@src-getrandom]	a hand-seeded userspace PRNG, blocking on `/dev/random`
BSD, macOS	`getentropy(3)`, `arc4random(3)` [@src-arc4random]	`rand()`, `random()`
Windows	`BCryptGenRandom`, `ProcessPrng`	anything seeded by hand (see the CNG post)
Language stdlib	Node `crypto.randomBytes`, Python `secrets` / `os.urandom`, Go `crypto/rand`, Java `SecureRandom`	`Math.random()`, Python `random`, Mersenne Twister, `java.util.Random`, `System.Random`

Then, per role, the decision rules:

Key. Draw from the OS CSPRNG at the primitive's exact key length: 32 bytes for AES-256 or ChaCha20. Nothing else.
CTR/GCM nonce. Prefer a 96-bit monotonic counter (uniqueness by construction). If you must draw randomly, cap usage at fewer than $2^{32}$ messages per key. Never reuse a nonce under one key. If uniqueness cannot be guaranteed across distributed senders, use AES-GCM-SIV [@src-rfc8452].
CBC IV. A fresh unpredictable value per message from the CSPRNG, never a counter and never the previous ciphertext block, or better, migrate to authenticated encryption.
Salt. At least 16 bytes from the CSPRNG, unique per password, stored in plaintext. Never a global salt.
Signature nonce (k). Use RFC 6979 deterministic k, or a hedged scheme; never a bare draw whose bias or repetition leaks the private key [@src-rfc6979].
Fork or VM clone. Rely on kernel reseed (VM Generation ID) plus library fork detection, and force a reseed after any clone or snapshot before emitting security-critical output [@src-ferguson2019win].

Table 7: The Common Misuse catalog (the mirror of Table 1).

Misuse	Knob and place broken	Named break
Seeding from `time()` or a PID	predictable, generator	Netscape [@src-netscape1996]
Removing or starving entropy	predictable, generator	Debian, Ps and Qs [@src-debian2008]
Reusing a GCM nonce under one key	repeated, role	GCM in TLS 2016 [@src-cve2016]
Predictable or chained CBC IV	predictable, role	BEAST [@src-cve2011]
Static or biased signature k	repeated or predictable, role	PS3, PuTTY [@src-putty2024]
Secret, short, or global salt	repeated, role	shared precomputed tables
Hand-seeding a userspace PRNG	predictable, generator	interface misuse
Trusting `RDRAND` alone	predictable, generator	the Dual_EC lesson [@src-shumow2007]
Mersenne Twister or `Math.random` for tokens	predictable, generator	statistical PRNG used for crypto [@src-mersenne1998]
No reseed after `fork()` or snapshot	repeated, generator	VM-reset [@src-ristenpart2010]
UUIDv4 as a security token	predictable, role	guessable token [@src-rfc9562]

The meta-lesson runs through every row: historically the interface, not the algorithm, is the bug. Blocking /dev/random, seeding a userspace PRNG, forgetting to reseed after a fork; these are misuses of how you reach randomness, not weaknesses in the math. That is exactly why getrandom and getentropy were designed to be un-misusable.

Note: If you are managing entropy in application code, mixing your own sources, maintaining a userspace pool, reseeding on a timer, you have almost certainly introduced a bug the OS already solved. The correct amount of RNG plumbing in an application is zero. Call getrandom, getentropy, BCryptGenRandom, or your language's CSPRNG wrapper, and stop [@src-getrandom, @src-arc4random].

Note: A version-4 UUID is 122 random bits formatted for uniqueness, not unpredictability, and its specification never promised it would be unguessable or drawn from a CSPRNG [@src-rfc9562]. Using one as a password-reset token, session ID, or API key is betting your security on an implementation detail the standard does not guarantee. For anything security-sensitive, draw CSPRNG bytes directly [@src-rfc4086].

Draw every random value from the OS CSPRNG (`getrandom` / `getentropy` / `BCryptGenRandom` / `secrets` / `crypto/rand`), never a language `random()` or Mersenne Twister. Keys: exact key length from the CSPRNG. GCM nonces: 96-bit counter, or random with a hard cap below $2^{32}$ per key, and if senders are distributed use AES-GCM-SIV. CBC IVs: fresh unpredictable per message, or move to AEAD. Salts: at least 16 bytes, unique per password, stored in the clear. Signature nonces: RFC 6979 deterministic. After any `fork()` or VM snapshot, force a reseed before emitting keys or nonces. Reject any hand-rolled entropy pool.

The rules are short because the thesis is short: for every value you draw, name the property the role owes, and confirm the bytes came from the OS CSPRNG. Answer both and you are out of the catalog. Before the closing, here are the misconceptions that send good engineers back into it.

11. Frequently Asked Questions

The most expensive randomness bugs start as reasonable-sounding beliefs. Here are eight, corrected.

No. After the CSPRNG is seeded once at first boot, both devices draw from the same ChaCha-based generator and are cryptographically identical in the quality of their output; the only remaining difference is that `/dev/random` blocks until that first seeding while `/dev/urandom` does not [@src-donenfeld2022]. The old advice to prefer `/dev/random` for "more secure" keys was based on an entropy-depletion model that does not hold for a modern CSPRNG: a properly seeded generator does not run out of unpredictability by producing output. Use `getrandom()` and stop worrying about which device node to open [@src-random4man]. No, only once. `getrandom()` blocks a single time at first boot until the pool is initially seeded, and then never blocks again for the life of the system [@src-getrandom]. Code that polls an entropy estimate before every draw, or loops waiting for `/dev/random`, is solving a problem that the blocking-once design already solved at the interface. It depends on the mode, and this is the most common confusion in the field. A CTR or GCM nonce must only be *unique*; unpredictability buys nothing there, and a counter is a fine, often safer, choice [@src-sp80038d]. The role that must be *unpredictable* is the CBC IV, whose predictability is exactly what BEAST exploited [@src-cve2011]. Same-looking value, different requirement, set by the mode. No. A salt is unique and random but stored in plaintext right next to the password hash. Its only job is to make identical passwords hash differently so one precomputed table cannot attack many accounts. Treating it as a secret, or reusing one global salt, misunderstands the role: uniqueness is what matters, not secrecy. No. Random 96-bit nonces are bounded by the birthday problem, so a single key should encrypt fewer than about $2^{32}$ messages to keep the repeat probability negligible [@src-sp80038d]. Beyond that, use a monotonic counter, which cannot collide, or AES-GCM-SIV, which tolerates a repeat without catastrophic failure [@src-rfc8452]. Neither extreme is right. Do not trust `RDRAND` *alone*, because it is opaque silicon you cannot audit, but do not refuse it either. The correct posture, the one modern kernels take, is to mix its output into a pool alongside other sources so that no single component is the whole answer [@src-intel-drng]. That is the discipline Dual_EC taught the field [@src-shumow2007]. No. A version-4 UUID is built for uniqueness, and its specification does not require the bits to be unguessable or drawn from a CSPRNG [@src-rfc9562]. For session IDs, reset tokens, and API keys, draw CSPRNG bytes directly instead of relying on a UUID's format [@src-rfc4086]. No. Statistical suites can falsify a generator but never certify one. The Mersenne Twister passes the standard batteries and is completely broken after 624 outputs [@src-mersenne1998]. Test suites catch gross defects; they say nothing about whether an adversary can predict the next bit, which is the only property that matters for cryptography [@src-sp80022].

12. Two Knobs, Two Places

Go back to where we started. Netscape, 1996: the RC4 and RSA math was flawless, and the keys were guessable because the seed was the clock and a couple of process IDs [@src-netscape1996]. Debian, 2008: the algorithm was fine, and roughly two years of keys collapsed to about 32,768 possibilities because one entropy input was removed [@src-debian2008]. You can now name both on sight: predictable, in the generator. That is the whole diagnosis, and it is the same diagnosis for every break in this article.

By now the two knobs and two places should lock together into a single picture. Every value you draw owes one of two properties, unpredictability or uniqueness, and either can be squandered in one of two places: the generator that mints the bits, or the role that spends them (key, nonce, IV, salt). That product, two properties across two places, is the entire fault space, and nothing in this article fell outside it.

What makes randomness uniquely treacherous is that none of those faults trips an alarm. Hand a cipher the wrong key and it errors; forge a MAC and the receiver rejects it. A predictable or repeated value sets off nothing at all: it clears every statistical test and validates cleanly, yet it is already broken for anyone who thought to recompute it. Each disaster we catalogued was one of those four faults, found only in hindsight.

Snap the catalog into the grid one last time. Netscape and Debian: predictable, generator. The PS3's static nonce and the Bitcoin wallets' repeated nonce: repeated, role [@src-fail0verflow2010, @src-android2013]. Dual_EC: predictable, generator, by design [@src-shumow2007]. GCM reuse in TLS: repeated, role [@src-cve2016]. PuTTY's biased P-521 nonce: predictable, role [@src-putty2024]. Four boxes, and every disaster in four decades fits in one of them.

Mastering randomness is not about finding a better source of magic. It is about knowing, for every value you draw, which of the two properties it owes and which of the two places can betray it.

That is the discipline. For every value you draw, name the property the role owes (unique? unpredictable? secret?) and confirm the generator is the OS CSPRNG. The generator half is essentially solved engineering; the failures that remain live in sourcing, in trust, and above all in the roles, where a perfect byte stream still gets you owned if you pour it into the wrong one.

This was Part 2 of the field guide. Part 1 established what "secure" means and why the definitions demand randomized encryption in the first place; later parts take the disciplined bytes you now know how to produce and build the modes, key-encapsulation mechanisms, and signatures that consume them. For the platform-specific instance of everything here, how Windows assembles its CSPRNG and defends it against cloning, the Windows CSPRNG companion article is waiting. Every one of those constructions will hand you a value to generate. Now you know the only two questions to ask of it.