Parag Mali - tag: hash-functions

How SHA-2 and SHA-3 Would Break: Merkle-Damgard Collisions, Length Extension, and the Sponge's Algebraic Frontier

noreply@paragmali.com (Parag Mali) — Fri, 17 Jul 2026 00:00:00 GMT

SHA-256 and SHA3-256 make the same promise -- about $2^{128}$ work to force a collision -- yet they share almost none of their internal machinery, and that difference decides how each one would die. SHA-2 (Merkle-Damgard) would fall to a Wang-style differential collision threaded through its compression function, and it already carries the length-extension property by design. SHA-3 (the Keccak sponge) has neither weakness; it relocates the fight onto the algebra of a single degree-2 step, chi, where the leading avenues are cube-style algebraic attacks and internal-differential collisions. The best published attacks reach only about 39 of SHA-256's 64 steps and 6 of Keccak's 24 rounds, so both margins are very large today. This is a structural "how it would break," not a report that either has.

1. Two Hashes, Two Deaths

Here is something you can say about a machine before it has ever failed: which way it will fall. A structural engineer knows a bridge sways before it buckles and buckles before it snaps, because the failure mode is written into the truss, not painted onto the surface. Cryptographic hash functions are the same. We can already describe, mechanically and in detail, how SHA-2 and SHA-3 would break -- even though nobody has ever broken either one, and the two descriptions have almost nothing in common.

The two functions NIST standardized as the Secure Hash Standard and the SHA-3 Standard [@fips-180-4, @fips-202] sit underneath an enormous amount of modern security: certificate chains, signed software updates, commitment schemes, the proof-of-work in a blockchain. They advertise the same guarantee. For a 256-bit digest, finding any two inputs that hash to the same value should cost about $2^{128}$ attempts.That number is easy to say and hard to feel. $2^{128} \approx 3.4 \times 10^{38}$, a 39-digit count -- far more than every grain of sand on Earth, yet far fewer than the atoms that make up the Earth. It is an enormous human-scale number that is still dwarfed by atomic-scale ones.

A hash function is collision resistant if it is computationally infeasible to find two distinct inputs $m_1 \neq m_2$ with $H(m_1) = H(m_2)$. It is the strongest of the standard hash guarantees, and the one that differential cryptanalysis attacks head-on.

That $2^{128}$ figure is not a design target so much as a ceiling imposed by arithmetic. Feed roughly $2^{n/2}$ random messages into any $n$-bit hash and, by the birthday paradox, two of them almost certainly collide. So $2^{128}$ is simply the generic cost of a birthday search against a 256-bit digest -- the number every genuine attack has to beat to count as a break [@fips-180-4].

For an ideal $n$-bit hash, a collision can always be found in about $2^{n/2}$ evaluations by the birthday paradox, and a preimage in about $2^{n}$. These generic costs -- $2^{128}$ and $2^{256}$ for a 256-bit digest -- are the definitional security targets any structural attack must beat.

Now the strange part. SHA-256 and SHA3-256 keep that identical promise using almost none of the same parts. SHA-2 iterates a small compression function from a fixed starting value, folding the message in one block at a time -- the Merkle-Damgard design. SHA-3 does something else entirely: it stirs the message into a large public permutation and squeezes the digest back out -- the sponge. Two constructions, one guarantee.

And because the constructions differ, their deaths differ. A Merkle-Damgard hash does not die at its digest size; it dies when a carefully chosen difference in the input threads through the compression function and cancels at the output -- the method that felled MD5 and SHA-1. It also leaks: given a digest, you can often extend the message without knowing it. The sponge has neither trait. It hands an attacker a different target: the algebra of its one nonlinear step, chi, which has algebraic degree just two.

Note: Construction, not digest size, decides the failure mode. SHA-2 would die by a differential collision on its compression function and carries length extension by design; SHA-3 exposes an algebraic frontier against its degree-2 chi and has no length extension. Same promise, different battlefield.

That is the whole argument in one sentence, and the rest of this article is its proof. One boundary before we begin.

Note: This is structural cryptanalysis: it studies attacks on the mathematics of the algorithms themselves. Side-channel leaks, fault and power attacks, buggy implementations, weak random number generators, and protocol misuse are out of scope. They are real, and they are the subject of the companion article, "How the Hash Functions Broke in Real Life: MD5, Flame, SHATTERED, and the Long Death of SHA-1." When the practical SHA-1 collisions appear below, they appear only as proof that a method works end to end, never as deployment history.

To see why each death is pre-written, we have to go back to the 1989 bargain that made iterated hashing trustworthy in the first place -- and to the two debts it quietly signed on SHA-2's behalf.

2. The Merkle-Damgard Bargain of 1989

In 1989 two people, an ocean apart and unaware of each other, answered the same question the same way -- and in doing so pre-wrote how SHA-2 would one day die. Ralph Merkle and Ivan Damgard each presented the result at CRYPTO 1989 [@merkle-1989, @damgard-1989].

The question was practical and deep at once: how do you hash a message of arbitrary length using only a fixed-size building block, and then prove the result is as hard to collide as that block? Their answer is the scaffolding under nearly every hash you have ever used.

A method for turning a fixed-input-length compression function $f$ into a hash on arbitrary-length messages. Pad the message and split it into blocks $M_1, M_2, \ldots, M_t$; starting from a fixed initialization value $H_0 = IV$, compute $H_i = f(H_{i-1}, M_i)$; output the final $H_t$ as the digest. Merkle and Damgard each proved that if $f$ is collision resistant, so is the whole hash.

The shape of the compression function they had in mind, and the one every hash in the SHA line uses, is Davies-Meyer.

A way to build a compression function from a block cipher $E$: treat the message block as the key and the chaining value as the plaintext, then feed the input back in, $H_i = E_{M_i}(H_{i-1}) \oplus H_{i-1}$. The final XOR -- the "feed-forward" -- is what makes the function hard to invert; without it, an attacker could run the cipher backward from a chaining value. flowchart LR IV["IV (fixed start)"] --> F1["f"] B1["Message block 1"] --> F1 F1 -->|"chaining value H1"| F2["f"] B2["Message block 2"] --> F2 F2 -->|"chaining value H2"| F3["f"] B3["Final block + length pad"] --> F3 F3 --> D["Digest = final chaining value"]

The proof is the reason the world trusted this design for two decades, and it is worth seeing rather than taking on faith -- because both of SHA-2's structural debts fall directly out of it.

Suppose an attacker finds two different messages $M \neq M'$ with $H(M) = H(M')$; the two chains therefore end on the same value, $H_t = H'_{t'}$. If the messages have different lengths, Merkle-Damgard strengthening (appending the encoded length) forces the final blocks to differ, so two *different* last-step inputs map under $f$ to the *same* output -- a collision in $f$. If the messages have the same length, walk backward from the output, comparing the inputs fed into each $f$-call. The outputs match at the last step; if the inputs there differ, that is your $f$-collision, and if they match, the previous chaining values must match too, so you recurse one step earlier. Because $M \neq M'$, some block differs, so the backward march cannot stay equal all the way down -- at the first step (scanning from the end) where the two $f$-inputs diverge, two distinct inputs produce one output. Every collision of $H$ yields a collision of $f$. Contrapositively, if $f$ is collision resistant, so is $H$: the whole hash is at least as strong as its compression function [@merkle-1989, @hac-chap9].

That reduction is a genuine gift -- it lets designers focus all their effort on one small function -- but it comes with two debts written into the same clause.

The first debt: all of the hash's collision resistance now concentrates in $f$. An attacker cannot collide SHA-256 without colliding its 64-step compression function, so that function is the entire battlefield. Everything a would-be attacker does, and everything the last thirty-five years of cryptanalysis has done, targets exactly that object [@hac-chap9].

The second debt is subtler and, for a practitioner, more immediately dangerous. Look again at the chain: the digest that comes out is the final chaining value -- the complete internal state at the end of the computation. Nothing is hidden.

Given only $H(m)$ and the length of $m$ -- but not $m$ itself -- an attacker can compute $H(m \,\|\, \text{pad} \,\|\, y)$ for a suffix $y$ of their choosing. Because the digest is the full internal state, the attacker simply resumes the iteration from it, appends the "glue" padding, and continues hashing. This works against any plain Merkle-Damgard hash, including SHA-256.

The strengthening step -- appending the message length before the final block -- is what makes the collision proof go through, closing the length-mismatch case above.The Davies-Meyer feed-forward and the length padding are two separate load-bearing details. The feed-forward stops an attacker running the block cipher backward from a chaining value; the length padding stops trivial collisions between messages of different lengths. Remove either and the reduction fails. But it does nothing to hide the state -- so it fixes the first debt's proof without touching the second debt at all.

Merkle-Damgard signs two debts at once: all collision resistance concentrates in one small function, and the digest is the whole internal state. The first is where the hash can be collided; the second is why it leaks.

A construction that concentrates all its collision resistance in one small function is only ever as safe as that function is hard to collide. For fifteen years, nobody could touch it. Then the mathematics of differences arrived, and MD5 fell in minutes.

3. How a Merkle-Damgard Hash Actually Dies

The break, when it came, did not come from brute force. It came from the arithmetic of differences -- and once you see it, you cannot un-see why SHA-2 is built the way it is.

The idea starts with a question no birthday search ever asks: what if we do not treat the hash as a black box, but push a specific, carefully chosen difference between two messages through its internals and watch where the difference travels? If we can arrange for the difference to appear, ripple through a few steps, and then cancel itself out exactly at the output, the two messages collide -- and we never paid the $2^{n/2}$ birthday toll.

Florian Chabaud and Antoine Joux built the first working version of this against SHA-0 at CRYPTO 1998 [@chabaud-joux-1998]. Their tools were two ideas that still organize the whole subject.

A low-weight input difference that creates a small perturbation over a handful of steps and then cancels itself, leaving the internal state unchanged afterward. Chaining local collisions along a message is how an attacker assembles a full collision out of small, controllable pieces. A step-by-step trail specifying the exact difference between two computations at every point in the compression function, chosen so the trail begins with a message difference and ends in zero difference at the output. The probability that a random conforming message pair follows the trail governs the attack's cost.

By chaining local collisions along a differential characteristic so the total difference vanished at the output, Chabaud and Joux drove the estimated cost of a SHA-0 collision down to roughly $2^{61}$ -- comfortably below the $2^{80}$ birthday wall for a 160-bit digest [@chabaud-joux-1998]. It was an estimate, not a produced collision, but it was the first real crack in the black box.

Then Xiaoyun Wang turned the estimate into a scalpel. In 2004 and 2005, Wang, with Hongbo Yu and Yiqun Lisa Yin, announced results that stunned the field: full, practical collisions for MD5 -- Rivest's 1992 design [@rfc-1321] -- and a collision attack on the full SHA-1 [@rfc-3174], well below its birthday bound [@wang-yu-2005, @wang-yin-yu-2005]. The engine had two new parts. First, a hand-built nonlinear path through the hard early steps of the compression function, spliced onto a cheap linear trail for the rest. Second, and decisive:

A technique that satisfies the early, high-probability conditions of a differential characteristic *deterministically*, by directly adjusting message words rather than hoping a random pair happens to comply. It moves the expensive part of the search from the beginning of the trail to its cheaper tail, collapsing the total cost. flowchart TD A["Choose a message difference"] --> B["Build a differential characteristic ending in zero output difference"] B --> C["Derive per-step sufficient conditions on state bits"] C --> D["Message-modify the early steps to force cheap conditions deterministically"] D --> E["Search the residual probability for the hard late conditions"] E --> F["Output a conforming colliding message pair"]

Why does modifying the message help so much? The answer is specific enough to work through, and it is exactly the mechanism the earlier hashes handed the attacker for free.

In a Davies-Meyer compression function the first sixteen steps consume the sixteen message words of the block essentially one per step: at step $i$ (for $i = 0..15$) the expanded word is just $W_i = m_i$, mixed straight into the state. So each early-step word is a *free variable the attacker fully controls*, entering the computation at exactly one known step.

Now, a differential characteristic holds only if certain state bits take certain values at each step -- Wang's "sufficient conditions." Left to chance, each condition holds with probability about $1/2$, so a path with $k$ conditions costs about $2^k$ to satisfy by random trial. That is the naive price.

Here is the trick. Because the attacker controls the very word $m_i$ feeding step $i$, they do not gamble on the step-$i$ conditions -- they solve for them. Given the desired output bits of step $i$, invert the round update (its additions, rotations, and Boolean function) for the single unknown $m_i$ and set it to the one value that forces those conditions with probability $1$. Every condition in the first sixteen steps is paid off deterministically, at unit cost, instead of probabilistically at $2^k$.

Past step sixteen the message words re-enter through the schedule and no longer offer a fresh free variable per step, so the later conditions cannot all be solved directly. Wang's multi-message modification clears some of them with coordinated corrective tweaks to several early words, but the residual conditions in the "tail" stay probabilistic. The total cost therefore collapses from the full characteristic probability (all conditions) to just the tail probability. That collapse is why full MD5 fell in seconds to minutes and full SHA-1 fell to roughly $2^{69}$, far below its $2^{80}$ birthday wall [@wang-yu-2005, @wang-yin-yu-2005].

The consequence reset every intuition about hash security. Full MD5, whose 128-bit digest nominally promised $2^{64}$ collision work, collapsed to seconds on a PC [@wang-yu-2005]. Full SHA-1, nominally $2^{80}$, fell to an estimated $2^{69}$ -- a real break of the design, years before anyone spent the compute to exhibit an actual colliding pair [@wang-yin-yu-2005].

Key idea: A Merkle-Damgard hash does not die at its digest size. It dies when a differential characteristic plus message modification threads its compression function -- deterministically through the early steps, cheaply through the tail. Collision resistance was never really about the number of output bits; it was always about whether a path exists through $f$.

Those hand-built attacks were later paid out in full, practical collisions -- SHAttered in 2017 and SHAmbles in 2020 -- but those belong to the deployment story, not this one.SHAttered (Stevens, Karpman, Bursztein, Albertini, and Markov) produced the first identical-prefix SHA-1 collision at about $2^{63.1}$ [@shattered]; SHAmbles (Gaetan Leurent and Thomas Peyrin, correcting a common misattribution to Marc Stevens) produced the first chosen-prefix SHA-1 collision at about $2^{63.4}$ and forged a PGP key [@leurent-peyrin-2020, @sha-mbles-site]. Both are evidence that the differential method pays out end to end; the empirical sibling article tells their story.

This is the pivotal fact of the whole subject. So the real question becomes: can that same scalpel reach SHA-2? The answer forced cryptography down two roads at once.

4. Two Lineages Diverge

Start with the cleanest evidence in the whole story that patching a weak compression function only buys time. SHA-0, standardized in 1993, had a linear message expansion; it was withdrawn in 1995 and replaced by SHA-1, whose entire difference from SHA-0 was a single one-bit rotation added to that expansion. That one rotation held for about a decade. Then a Wang-style path walked straight through it [@wang-yin-yu-2005]. A one-operation fix delayed the break; it did not prevent it.

This is the pattern to internalize: "just add a round" or "just add a rotation" hardens a Davies-Meyer compression function against yesterday's specific path, not against the method. Incremental patches to a lightly-mixed function buy years, not safety -- which is exactly why SHA-2 needed a qualitatively heavier schedule and SHA-3 needed a different construction altogether.

The full six-generation lineage reads as a slow-motion argument, each break motivating the next design.

Hash family	Year	Key design idea	Best collision result	Status
MD4	1990	Merkle-Damgard + Davies-Meyer, 128-bit, 48 steps	full collisions (Dobbertin 1996 [@dobbertin-1996]; Wang 2005)	historical [@wang-yu-2005]
MD5	1992	MD4 hardened: 64 steps, per-step constants	practical full collisions (Wang-Yu 2005)	historical [@rfc-1321, @wang-yu-2005]
SHA-0	1993	160-bit, linear message expansion (no rotation)	$\approx 2^{61}$ (Chabaud-Joux 1998)	withdrawn 1995 [@chabaud-joux-1998]
SHA-1	1995	SHA-0 plus one-bit rotation, 80 steps	$\approx 2^{69}$ (Wang 2005); demonstrated 2017	superseded [@wang-yin-yu-2005, @rfc-3174]
SHA-2	2001	bigger state, nonlinear carry-laden schedule, 64/80 steps	39/64 steps (reduced-round), full unbroken	active [@fips-180-4, @li-liu-wang-2024]
SHA-3 / Keccak	2012/2015	sponge over Keccak-f[1600], 24 rounds, hidden capacity	6/24 rounds (reduced-round), full unbroken	active [@fips-202, @keccak-team]

gantt title Two lineages, 1989 to 2024 dateFormat YYYY axisFormat %Y section Construction Merkle-Damgard bargain :1989, 1990 MD5 (RFC 1321) :1992, 1993 SHA-1 (FIPS 180-1) :1995, 1996 SHA-2 standardized :2001, 2002 SHA-3 competition :2007, 2012 SHA-3 standard (FIPS 202) :2015, 2016 section Cryptanalysis Chabaud-Joux on SHA-0 :1998, 1999 Wang breaks MD5 and SHA-1 :2004, 2006 Zero-sum full Keccak-f :2011, 2012 SHA-256 reduced-round records :2013, 2024 SHA-3 internal-differential records :2013, 2024

By 2005 the field had a decision to make, and it split into two philosophies. The first response was to harden the same construction. SHA-2, standardized in 2001 and still the workhorse, kept Merkle-Damgard and Davies-Meyer but gave the compression function a much larger state, a heavier message schedule, and modular-addition mixing designed to starve differential paths [@fips-180-4]. That is the road we return to in the next section, because it worked.

The second response was more radical. Stung by the 2004-2005 breaks, NIST announced the SHA-3 competition on November 2, 2007, and ran it until October 2, 2012, explicitly seeking a hash that would fail differently from the Merkle-Damgard family -- so that a single new cryptanalytic idea could not bring down every standardized hash at once [@nist-sha3-project]. The winner was Keccak, a sponge built on a large public permutation, designed from the start so that its hidden capacity removes length extension by construction [@keccak-team].

The subtle and load-bearing point is what NIST did not do. It did not retire SHA-2.

The MD5-to-SHA-1-to-SHA-2 progression was a ladder of *replacements* -- each rung was climbed because the one below it broke. SHA-2 and SHA-3 are not that. FIPS 202 states in its own opening that the SHA-3 functions "supplement" SHA-1 and the SHA-2 family rather than replace them [@fips-202], and NIST's hash-function policy says plainly that "currently there is no need to transition applications from SHA-2 to SHA-3" [@nist-hash-policy]. The two standards coexist on purpose. The value is *diversification*: if a structural idea ever threatens one construction, the other rests on entirely different mathematics and is untouched. SHA-3 is an insurance policy, not an eviction notice.

Two lineages, two answers to the same failure. But an answer is only as good as the attack it survives -- so why has SHA-2 held for more than two decades, and what exactly did the sponge change?

5. Why SHA-2 Held, and What the Sponge Changed

Two structural facts define the present. The first is that the same message-modification machinery that broke MD5 and SHA-1 stalls on SHA-2 -- and the reason is one design choice. The second is that the sponge removed the debts a differential attack needs, and put something entirely different in their place.

Why the scalpel stalls on SHA-2

Recall the mechanism from the last section: message modification is cheap only because, in the early steps, each message word is a free variable entering the state at exactly one place. Break that "one free word per early step" structure and the trick loses its edge. SHA-2 breaks it on purpose.

In MD5 and SHA-1 the message schedule is essentially a permutation (and, later, a linear expansion) of the input words -- clean and separable. SHA-2's schedule is neither. From step 16 onward, each expanded word is built by

$$W_t = \sigma_1(W_{t-2}) + W_{t-7} + \sigma_0(W_{t-15}) + W_{t-16} \pmod{2^{32}}$$

where $\sigma_0$ and $\sigma_1$ are XORs of rotations and a shift, and the additions are modular [@fips-180-4]. Those modular additions couple the words through carries: changing one bit of one word can ripple through carry chains into many others. The clean, separable structure that let an attacker solve each early condition by inverting a single word update is gone. Now solving one sufficient condition perturbs others, and the conditions can no longer be discharged by independent single-word inversion.

That carry-laden coupling is the structural reason the hand construction that felled MD5 and SHA-1 cannot be driven through SHA-2 by hand at all -- which is why the modern realization of the attack is an automated SAT/SMT solver managing the entangled condition system, and why even that tooling, in the record-holders' own words, "has reached the bottleneck" around 39 of 64 steps [@li-liu-wang-2024]. We return to that number in the next section. For now the point is qualitative: SHA-2 held because it made the compression function's internals too coupled for the differential scalpel to carve cleanly.

What the sponge deleted

Keccak does not try to win the differential arms race. It changes the board.

A hash built from a single large public permutation $f$ over a state of $b = r + c$ bits. To *absorb*, split the padded message into $r$-bit blocks and, for each, XOR it into the first $r$ bits of the state and apply $f$. To *squeeze*, read out $r$ bits at a time, applying $f$ between reads, until enough digest bits are produced. Keccak uses $f = $ Keccak-f[1600], a 1600-bit permutation of 24 rounds. The sponge state of $b$ bits is split into a *rate* of $r$ bits and a *capacity* of $c$ bits, with $b = r + c$. The message is XORed only into the rate, and the digest is read only from the rate. The capacity is never touched directly by input or output -- it is the hidden reservoir that carries security across the permutation calls. flowchart LR MSG["Message blocks"] -->|"XOR into rate"| RATE["Rate (r bits)"] RATE --> F["Keccak-f[1600], 24 rounds"] CAP["Capacity (c bits): never absorbed or squeezed"] --> F F -->|"squeeze"| DIG["Digest, read from the rate only"]

Two debts vanish at once. Because the capacity is never emitted, the digest is not the full internal state -- so no attacker can resume the computation from it. Length extension is gone by construction, not by patch [@keccak-sponge-duplex]. And because there is no keyed compression function -- just one fixed public permutation -- there is no keyed object to push a differential characteristic through to force a hash collision directly the way Wang did. The entire Merkle-Damgard attack surface is simply absent.

So what is left to attack? The permutation's algebra. And that turns on the single nonlinear step inside Keccak-f.

flowchart LR S["State (1600 bits)"] --> T["theta: linear column mixing (degree 1)"] T --> R["rho: bit rotations (degree 1)"] R --> P["pi: lane permutation (degree 1)"] P --> C["chi: the only nonlinear step (degree 2)"] C --> I["iota: add round constant (degree 1)"] I --> O["Next-round state"] Every Boolean function can be written uniquely as a sum (XOR) of monomials over GF(2) -- its algebraic normal form. Its *algebraic degree* is the size of the largest monomial. An affine map (only single-variable terms) has degree 1; an AND of two input bits has degree 2. Low degree is a weakness: it makes a function vulnerable to higher-order-differential and cube techniques. Chi is the single nonlinear step of Keccak-f. Along each 5-bit row it computes $y_i = x_i \oplus ((\lnot\, x_{i+1}) \land x_{i+2})$. Each output bit contains exactly one AND of two state bits, so chi has algebraic degree exactly 2. The other four steps of a round -- theta, rho, pi, iota -- are all linear or affine.

Here is the load-bearing fact of the sponge's exposure, and it is worth deriving rather than asserting, because one word in it does real work.

One round is the composition $R = \iota \circ \chi \circ \pi \circ \rho \circ \theta$. Four of those five steps are affine, degree 1; only chi is nonlinear, degree 2. Composing a degree-2 map with affine maps leaves the degree at 2, so a single round has $\deg(R) = 2$.

Now use the composition bound for Boolean functions, $\deg(f \circ g) \le \deg(f) \cdot \deg(g)$, across $r$ rounds: $\deg(R^2) \le 2 \cdot 2 = 4$, $\deg(R^3) \le 2 \cdot 4 = 8$, and in general each extra round multiplies the bound by at most 2. The degree doubles per round:

$$1 ;\to; 2 ;\to; 4 ;\to; 8 ;\to; \cdots ;\to; 2^r, \qquad \text{so} \qquad \deg(\text{Keccak-}f^{,r}) \le 2^r.$$

Note: The degree doubles each round, giving $\deg \le 2^r$. It does not square each round -- squaring would compound as $2, 4, 16, 256, \ldots \approx 2^{(2^r)}$, a doubly-exponential growth that would make the bound useless within a few rounds and contradict the $2^r$ figure itself. The composition law multiplies the degree by at most $\deg(R) = 2$ each round, which is doubling. Getting this verb right is the difference between a bound that explains the whole algebraic frontier and one that is internally inconsistent.

You can watch chi's degree-2 structure force a higher-order derivative to vanish directly. A degree-$d$ function summed (XORed) over any affine cube of dimension $d+1$ gives zero, so summing degree-2 chi over any 3-dimensional cube must vanish.

{` // chi on a 5-bit row: y[i] = x[i] XOR ((NOT x[i+1]) AND x[i+2]), indices mod 5. function chi(x) { let y = 0; for (let i = 0; i < 5; i++) { const xi = (x >> i) & 1; const xi1 = (x >> ((i + 1) % 5)) & 1; const xi2 = (x >> ((i + 2) % 5)) & 1; const bit = xi ^ ((xi1 ^ 1) & xi2); // one AND term => algebraic degree 2 y |= (bit << i); } return y; }

// A degree-d function has a vanishing (d+1)-th order derivative: // summing it over ANY affine cube of dimension d+1 gives 0. // chi has degree 2, so summing over ANY 3-dimensional cube must XOR to 0. function sumOverCube(base, d1, d2, d3) { let acc = 0; for (let m = 0; m < 8; m++) { let x = base; if (m & 1) x ^= d1; if (m & 2) x ^= d2; if (m & 4) x ^= d3; acc ^= chi(x); } return acc; }

console.log("chi(0b10110) =", chi(0b10110)); console.log("3-cube sum (should 0) =", sumOverCube(0b00000, 0b00001, 0b00010, 0b00100)); console.log("3-cube sum (should 0) =", sumOverCube(0b10101, 0b00011, 0b01100, 0b10000)); `}

One honesty note keeps this from overclaiming.The bound $\deg \le 2^r$ is the elementary degree-doubling law, and it is only informative at low round counts. Keccak-f acts on 1600 bits, where any permutation has degree at most 1599; $2^r$ already exceeds 1599 at $r = 11$, so the elementary bound says nothing about the full 24 rounds. The full-round result in Section 8 -- a zero-sum partition of size $2^{1590}$ -- rests instead on the tighter degree bound of Boura, Canteaut, and De Canniere, which stays below the state size even at 24 rounds [@boura-canteaut-decanniere-2011]. The elementary law is what caps the cube attack at a handful of rounds; the refined bound is what lets the zero-sum reach all 24. Keep them separate and the whole SHA-3 frontier stays legible.

Now the two constructions can be laid side by side.

Dimension	SHA-2 (SHA-256)	SHA-3 (SHA3-256 / SHAKE256)
Construction	Merkle-Damgard + Davies-Meyer	Keccak sponge over Keccak-f[1600]
Full-function rounds/steps	64 steps	24 rounds
Leading structural avenue	differential collision search (single)	algebraic frontier + internal-differential (two)
Best structural collision reach	39/64 SFS; $\approx$ 31/64 from-IV	6/24 (SHAKE256); 5/24 (SHA3-384)
Length extension	yes (structural; digest is the full state)	no (capacity hidden)
Generic collision target	$2^{128}$	$2^{128}$

Key idea: SHA-2 keeps the differential attack surface and the length-extension debt. The sponge trades both away for an algebraic surface whose degree grows only as $2^r$. The two functions do not sit on one strength axis, a bigger lock on the same door -- they expose different attack surfaces, so they would die different deaths. Construction, not digest size, selects the failure mode.

So the two constructions hand an attacker two different problems: thread a carry-laden differential path through 64 steps, or break the low-degree algebra of 24 rounds. How far has anyone actually gotten on each?

6. The Best Published Attacks, With Margins

Here is the scoreboard. After thirty-five years of cryptanalysis, where exactly does the deepest published structural attack stop on each function, and how far is that from the full thing? This is the article's quantitative core, and it comes with a reading rule that matters more than any single number.

The yardstick never changes. A structural break must beat the generic birthday cost of $2^{n/2}$ on the full function. On full SHA-256 and full SHA3-256, no published attack does; the best-known collision attack on either full function is still the generic one at $2^{128}$ [@li-liu-wang-2024, @dinur-etal-2015]. Every record below is progress on the method, measured against a full function that remains untouched.

The SHA-2 record board

Against SHA-2, one avenue has all the reach: automated differential characteristic search -- the mechanized descendant of Wang's hand construction, driven by SAT and SMT solvers and steered by mixed-integer linear programming. Reading its records needs one distinction.

Attack models differ in how much freedom the attacker has over the chaining value. A *from-IV* collision uses the real fixed initialization value and is a true hash collision. A *semi-free-start* (SFS) collision lets the attacker choose the single chaining value both messages share. A *free-start* (FS) collision lets the attacker also choose a difference in the chaining value. SFS and FS are weaker attacker goals -- progress markers on the compression function, not full-hash collisions.

The current records, all practical (a real colliding pair verified on a PC, not merely a sub-birthday estimate), come from Mendel, Nad, and Schlaffer in 2013 and from Li, Liu, and Wang in 2024 [@mendel-nad-schlaffer-2013, @li-liu-wang-2024]:

Function	Model	Best reach	Practical?	Untouched margin	Source
SHA-256	from-IV	31 / 64 steps	yes	33 steps	Mendel-Nad-Schlaffer 2013 [@mendel-nad-schlaffer-2013]
SHA-256	semi-free-start	39 / 64 steps	yes	25 steps	Li-Liu-Wang 2024 [@li-liu-wang-2024]
SHA-224	free-start	40 / 64 steps	yes (prior best $2^{110}$)	24 steps	Li-Liu-Wang 2024 [@li-liu-wang-2024]
SHA-512	semi-free-start	28 practical / 31 theoretic of 80	yes	49+ steps	Li-Liu-Wang 2024 [@li-liu-wang-2024]
SHA3-384	collision	5 / 24 rounds	yes	19 rounds	Zhang-Hou-Liu 2024 [@zhang-hou-liu-2024]
SHAKE256	collision	6 / 24 rounds	yes	18 rounds	Zhang-Hou-Liu 2024 [@zhang-hou-liu-2024]
Keccak (keyed)	key recovery	9 / 24 rounds	6-round practical	large	Dinur et al. 2015 [@dinur-etal-2015]
Keccak-f	zero-sum distinguisher	24 / 24 rounds	structural set	none, but no break	Boura-Canteaut-De Canniere 2011 [@boura-canteaut-decanniere-2011]

The 2024 paper set three firsts at once: the first practical semi-free-start collision for 39-step SHA-256 (improving the decade-old 38-step SFS record), the first practical free-start collision for 40-step SHA-224 where the prior best was a theoretic attack at time complexity $2^{110}$, and practical and theoretic collisions for reduced SHA-512 [@li-liu-wang-2024]. Impressive -- and then the same authors say where the road ends.

"The current advanced tool to search for SHA-2 characteristics has reached the bottleneck." -- Li, Liu, and Wang, on why the attack cannot climb past about 39 of 64 steps [@li-liu-wang-2024]

The modular-addition diffusion from Section 5 is exactly what makes longer characteristics effectively unfindable with today's encodings.The three record numbers for the same function are not a contradiction -- they are three different attacker games. From-IV is a true collision on the real hash. Semi-free-start hands the attacker the shared chaining value. Free-start additionally hands them a chosen difference in it. Each added freedom buys a few more steps of reach, which is why SHA-256 shows 31 (from-IV), 39 (SFS), and its sibling SHA-224 shows 40 (FS). The from-IV number is the one that measures the real hash.

The SHA-3 record board

The sponge side has three attacks rather than one, and the next section is devoted to why they disagree about what "progress" means.

For the scoreboard, the reduced-round facts are these. Cube and cube-like algebraic attacks reach up to 9 of the 24 rounds in keyed Keccak modes, with the 6-round attacks fully practical on a desktop [@dinur-etal-2015]. Internal-differential collision cryptanalysis holds the actual collision records: 5 of 24 rounds for SHA3-384 and 6 of 24 for SHAKE256 [@zhang-hou-liu-2024]. And a zero-sum distinguisher reaches all 24 rounds of the permutation while producing no collision at all [@boura-canteaut-decanniere-2011] -- the row we unpack next.

The team that set the deepest keyed reach stated the bottom line directly.

"The security margin of Keccak remains large." -- Dinur, Morawiecki, Pieprzyk, Srebrny, and Straus, whose cube-attack-like cryptanalysis reached the deepest keyed rounds [@dinur-etal-2015]

Now state the margins plainly, because they are the number a practitioner actually needs. For SHA-256, the deepest collision result leaves roughly 25 of 64 steps untouched in the semi-free-start model and about 33 untouched from the real IV. For a Keccak collision, roughly 18 of 24 rounds remain untouched. Those are not narrow escapes; they are the bulk of each function.

{` // A toy keyed function of public bits p0,p1 and secret key bits k0,k1. // Algebraic degree 2 -- the product (p0 AND p1) is the only quadratic term. function f(p0, p1, k0, k1) { return ((p0 & p1) ^ (p1 & k0) ^ (p0 & k1) ^ k0) & 1; }

// Cube over the single public variable p0: XOR f over p0 in {0,1}. // The resulting "superpoly" is LINEAR in the key -- it equals p1 XOR k1. function cubeSumOverP0(p1, k0, k1) { return f(0, p1, k0, k1) ^ f(1, p1, k0, k1); }

// The attacker cannot see the key. Say the hidden key is: const K0 = 1, K1 = 0;

// Query with p1 = 0 and sum the cube: the quadratic term dies and k1 falls out. const recovered = cubeSumOverP0(0, K0, K1); console.log("cube superpoly at p1=0 recovers k1 =", recovered, "(true k1 =", K1, ")");

// The same reading discipline for any reduced-round headline: function margin(reached, total, unit) { return "reached " + reached + "/" + total + " " + unit + " -> " + (total - reached) + " " + unit + " still untouched"; } console.log(margin(39, 64, "SHA-256 steps")); console.log(margin(6, 24, "SHAKE256 rounds")); `}

Note: A headline of "39 of 64 steps" or "6 of 24 rounds" is not "almost broken." A reduced-round record measures distance to a break, not nearness to one. The record climbs by one or two steps every few years, against a function whose full round count has never been threatened. Treat every such headline as a ruler laid against the margin, and the panic evaporates.

One number on the SHA-3 side hides an argument, though. The deepest result of all -- the zero-sum -- reaches every one of Keccak's 24 rounds, and still breaks nothing. To see why, look at the three rival attacks on the sponge side by side.

7. The SHA-3 Fork the Thesis Holds Open

SHA-2's story has one protagonist: differential collision search, with no rival of comparable reach. SHA-3's story has three, and they disagree about what the word "reach" even means. Pretending that one of them is the SHA-3 break would misrepresent the field, so this section keeps the fork open.

Method A: cube and cube-like attacks

The cube attack, introduced by Itai Dinur and Adi Shamir in 2009, is the avenue the title of this article names the algebraic frontier, because it exploits the low degree that chi keeps small at low round counts [@dinur-shamir-2009].

An algebraic attack that treats each output bit as an unknown low-degree polynomial over GF(2) in the public and secret input bits. Summing the output over a *cube* -- all $2^{k}$ assignments of $k$ chosen public variables -- cancels every term that does not contain all $k$ cube variables, leaving a lower-degree "superpoly." Choose the cube so the superpoly is linear in the secret bits, collect enough such equations, and solve for the key.

The method is provably successful on random low-degree polynomials whenever the number of public variables exceeds $d + \log_d n$, and it runs in about $2^{d-1} n + n^2$ bit operations for degree $d$ over $n$ secret variables [@dinur-shamir-2009].

Against Keccak, Dinur, Morawiecki, Pieprzyk, Srebrny, and Straus reached up to 9 of the 24 rounds faster than exhaustive search, with the 6-round attacks fully practical [@dinur-etal-2015]. But note precisely what it threatens: key recovery or forgery in keyed Keccak modes -- not a plain hash collision.The cube round counts come from keyed Keccak variants such as message authentication codes and the Keyak authenticated-encryption scheme, whose nominal round count is 12. In those modes a secret key exists to recover. Standard SHA3-256 hashing has no key, so the cube attack does not by itself produce a hash collision -- it is the deepest algebraic reach, in a different attack model. And because the degree doubles each round, the cube needed grows astronomically past a handful of rounds, which is exactly why it stalls at 9.

Method B: zero-sum distinguishers

Take the same low-degree property but drop the key, and you get a statement about the raw permutation. Christina Boura, Anne Canteaut, and Christophe De Canniere used a refined bound on the degree of iterated permutations to build zero-sum partitions of size $2^{1590}$ for the full 24-round Keccak-f [@boura-canteaut-decanniere-2011].

This is the deepest structural reach anyone has into Keccak -- it touches every round -- and it is also the least usable result in the subject. It distinguishes the permutation from an ideal one; it yields no collision, no preimage, nothing against the hash. Hold that paradox; the next section is built on it.

Method C: internal-differential collisions

The record-holders work differently again -- and they are the avenue that produces the property that actually matters for a hash.

Instead of a difference between two separate messages, an internal-differential attack exploits a difference *within a single state* -- for example, between translated copies of the state under Keccak's internal symmetry. Inputs that respect the symmetry are squeezed into a small subset of possible outputs, so a birthday search confined to that small subset finds collisions far below the usual $2^{n/2}$ cost.

The line runs from Thomas Peyrin's generalized internal differentials in 2010, through Dinur, Dunkelman, and Shamir -- who produced actual collisions for 3-round Keccak, a 4-round Keccak-384 attack $2^{45}$ times faster than birthday, and collisions on 5 rounds of Keccak-256 [@dinur-dunkelman-shamir-2013] -- to Zhongyi Zhang, Chengan Hou, and Meicheng Liu, whose "probabilistic linearization" extended the records to 5 rounds of SHA3-384 and 6 of SHAKE256 [@zhang-hou-liu-2024]. This is the only avenue that delivers real collisions on the standardized instances.

Now put the three side by side. The decisive column is not "how many rounds" but "what property."

Dimension	Cube / cube-like	Zero-sum higher-order diff.	Internal-differential collisions
Property threatened	keyed-mode key recovery / forgery	permutation distinguisher (not a break)	actual hash collisions
Best round reach	9 / 24 (keyed)	24 / 24 (full permutation)	6 / 24 (SHAKE256), 5 / 24 (SHA3-384)
Practical demo	yes, 6-round	structural ($2^{1590}$ sets)	yes, actual collisions to 3-5 rounds
Model	keyed (MAC / AE)	keyless permutation	keyless hash / XOF
What limits it	degree doubles per round ($\deg \le 2^r$)	capacity hides the permutation output	characteristic decays with rounds
Structural motivation	direct: low-degree chi	direct: the degree bound	indirect: Keccak internal symmetry
Source	[@dinur-shamir-2009, @dinur-etal-2015]	[@boura-canteaut-decanniere-2011]	[@dinur-dunkelman-shamir-2013, @zhang-hou-liu-2024]

Note: This is the honest balance the whole SHA-3 story turns on. The zero-sum method reaches all 24 rounds but yields no break; the cube method reaches 9 rounds but only in keyed modes; the internal-differential method reaches just 6 rounds but is the sole avenue that produces real collisions. The algebraic / cube line is the distinctive, structurally-motivated frontier -- it follows directly from the degree-2 chi -- and the internal-differential line holds the collision records. Both are leading; neither out-reaches the other on the property that matters to it. Contrast SHA-2, where differential collision search is the single leading avenue with no competitor of comparable reach.

Third-party cryptanalysis of all three families is tracked publicly by the designers [@keccak-third-party]. That middle row -- 24 rounds reached, nothing broken -- is not an accident of today's tooling. It is a provable wall, and understanding why forces us to the theoretical limits.

8. The Ideal, the Bounds, and the Distinguisher Trap

Begin with the middle row from the last table, because it is the strangest fact in the subject. A zero-sum partition of size $2^{1590}$ reaches all 24 rounds of Keccak-f -- the deepest structural property anyone has ever proved about it -- and it yields no collision, no preimage, nothing usable against the hash [@boura-canteaut-decanniere-2011]. How can the deepest result reach the whole function and still break nothing?

First, the wall that neither function's attackers have crossed. On the full SHA-256 and full SHA3-256, no published structural attack beats $2^{128}$ for a collision or $2^{256}$ for a preimage. Best-known equals generic on both, so the "beat the birthday bound" gap is not merely large -- it is open and uncrossed. Nobody has closed any of it on either full function [@fips-180-4, @fips-202]. Everything in Section 6 is reduced-round evidence about how far away the break is.

Now the answer to the zero-sum paradox, which is a proof, not a hope.

A structured set of inputs whose images under the permutation XOR to zero and whose preimages also XOR to zero. Because Keccak-f has bounded algebraic degree forward and a low-degree inverse backward, one can build such a set straight through the permutation. It distinguishes Keccak-f from an ideal random permutation, and does nothing else. A construction built on an ideal primitive is *indifferentiable* from a random oracle if no efficient distinguisher can tell them apart, even when the distinguisher is allowed to query the underlying primitive directly. It is the strong security notion for hash constructions: it certifies that the construction adds no exploitable structure of its own on top of the primitive.

Bertoni, Daemen, Peeters, and Van Assche proved in 2008 that a sponge calling a random permutation is indifferentiable from a random oracle: the success probability of any generic attack is at most its success probability against a true random oracle plus $N^2/2^{c-1}$, where $N$ is the number of queries to the permutation and $c$ is the capacity [@sponge-indiff, @keccak-sponge-duplex].

The strength of the claim is the clause "even when the distinguisher can query the permutation directly" -- a simulator can answer those permutation queries consistently with random-oracle outputs up to about $2^{c/2}$ work. Below $2^{c/2}$ in the hidden capacity, nothing separates the sponge from an ideal object.

That is what disarms the zero-sum. A zero-sum partition is a structural distinguisher of the permutation f alone -- a global property of how f maps XOR structure. The indifferentiability theorem runs in exactly one direction: any attack on the sponge F implies a distinguisher on f, but not the converse. Having a distinguisher on f does not hand you an attack on F. In the designers' own words:

"The existence of a structural distinguisher for f does not necessarily imply an attack or weakness in F." -- Bertoni, Daemen, Peeters, and Van Assche, on why a permutation property need not be a hash break [@keccak-sponge-duplex]

Concretely, a collision requires steering the rate bits of two different messages so that their squeezed outputs coincide across the hidden capacity. The zero-sum gives an attacker no such steering: it certifies a global XOR property of the permutation, not a way to aim two chosen inputs at one output through the capacity. So it lies outside the class of exploitable structure that indifferentiability rules out, and a distinguisher whose success probability is zero below $2^{c/2}$, in the designers' phrase, "forms no threat" [@keccak-sponge-duplex].

Key idea: A permutation distinguisher is not a hash break. "Reaches all 24 rounds" and "produces a collision" are provably different statements, separated by the capacity's $2^{c/2}$ indifferentiability margin -- not merely by the current state of tooling. Depth of reach into the permutation is not nearness to a break of the hash. The capacity stands between them by design, and the standing between is a theorem.

Note: When you read a headline that an attack "reaches the full permutation" or "distinguishes all rounds," resist the reflex to read it as near-catastrophe. For a sponge, a full-permutation distinguisher and a hash collision are separated by a proof. The distinguisher can be the deepest result in the field and still be the least dangerous one.

Two more limits complete the picture, one on each construction. On the sponge side, the full-round zero-sum exists only because of the refined degree bound, not the elementary one.Recall from Section 5 that the elementary law $\deg \le 2^r$ goes vacuous by round 11 against a 1600-bit state. The 24-round zero-sum instead rests on Boura, Canteaut, and De Canniere's tighter bound for permutations whose nonlinear layer is parallel small S-boxes, which stays below the state size all the way to round 24 [@boura-canteaut-decanniere-2011]. The elementary law caps the cube attack; the refined law powers the distinguisher. On the Merkle-Damgard side, length extension is not a bug that a better implementation could remove -- it is a provable inevitability of the untruncated construction. Because the SHA-2 digest is the whole final chaining value, anyone with $H(m)$ and $|m|$ can resume the iteration, full stop [@hac-chap9]. The only cures are to stop emitting the whole state (the sponge), hide enough of it (sufficient truncation), or wrap the hash (HMAC).

So the limits cut both ways. The sponge is closer to the ideal object -- indifferentiable up to the capacity, no length extension -- while SHA-2 is closer to the ideal margin in practice, with the bulk of its steps untouched by any differential path. Neither is anywhere near falling. One scope marker remains, because readers always ask about it.

Quantum computing changes the constants, not the constructions. Grover's algorithm lowers a *preimage* search from $2^n$ to $2^{n/2}$, but it does *not* beat the classical $2^{n/2}$ birthday bound for *collisions* in realistic bounded-memory models, and it exploits no internal structure of either hash -- it is a generic square-root speedup that would apply equally to an ideal random oracle. So it is generic, not structural, and the correct hedge is simply *more output bits* (SHA-384, SHA-512, SHAKE256 at a longer output), not a new design [@nist-hash-policy]. Shor's algorithm, which does break things structurally, is irrelevant here: it factors integers and computes discrete logarithms, threatening asymmetric primitives such as RSA and elliptic-curve signatures, not symmetric hashes. In this series, Shor counts as structural math for those primitives; for SHA-2 and SHA-3 it simply does not apply.

The limits leave exactly one question standing: if neither function is close to falling, where would the next round of progress even come from?

9. Where the Next Round Would Come From

If either function is ever broken, the break will arrive through one of a small, nameable set of open problems. Naming them turns you from spectator into forecaster.

flowchart TD Q["Where would a break come from?"] --> A["SHA-2: differential search"] Q --> B["SHA-3: three avenues"] A --> A1["OP-1: past ~40 of 64 steps? Stalled at 39"] B --> B1["Cube: 9 of 24, keyed only"] B --> B2["Internal-differential: 6 of 24 collisions"] B --> B3["Zero-sum: 24 of 24, distinguisher only"] B2 --> OP2["OP-2: close the ~18-round gap?"] B3 --> OP4["OP-4: cross the capacity barrier?"] A1 --> OP3["OP-3: beat 2^128 on a full function, THE break, none known"] OP2 --> OP3 OP4 --> OP3

OP-1: Can differential characteristic search push SHA-256 past about 40 steps? The full-function margin depends entirely on how far this single leading avenue can climb, and the 2024 record paper reports the tooling "has reached the bottleneck" -- modular-addition diffusion makes longer characteristics effectively unfindable with current encodings. Whether that is a fundamental barrier or an artifact of today's solvers is itself open [@li-liu-wang-2024].

OP-2: Can any SHA-3 avenue close the roughly 18-round gap to a full-Keccak collision? The best collision reach is 6 of 24 rounds, and the two leading avenues advance a round or two per decade, neither clearly out-reaching the other on its target property [@zhang-hou-liu-2024, @dinur-etal-2015].

OP-3: Is there any structural attack that beats the birthday bound on either full family? This is the break -- and today the answer is a clean no. Best-known equals generic on both full functions; everything else in this article is reduced-round evidence about how far away this remains [@fips-180-4, @fips-202].

OP-4: Why is the permutation-distinguisher / hash-collision divide so hard to cross? The deepest reach into Keccak-f, the 24-round zero-sum, currently yields nothing usable against the hash. Section 8 argued the reason is structural: indifferentiability makes the capacity a genuine barrier, not merely a reduction nobody has found yet [@boura-canteaut-decanniere-2011, @sponge-indiff].

Note: Only OP-3 is a break. OP-1, OP-2, and OP-4 are all measurements of distance to OP-3. That reframing is the whole discipline of reading this field calmly: a reduced-round record moves a ruler, it does not open a door.

None of this is the same as a proof of security. What cryptographers have proved are reductions -- collision resistance reduces to the compression function, sponge security reduces to the capacity -- plus the absence of any known attack that beats generic. "No known structural attack" is a strong empirical statement after thirty-five years of trying, but it is not "proven unbreakable." Honesty about that gap is part of taking the subject seriously.

None of these problems is close to solved -- which is exactly why the practical answer to "what should I ship?" is calm and specific.

10. What the Margins Mean on Monday Morning

So: should you change anything you shipped last week? Almost certainly not -- but there is exactly one structural gotcha worth ten minutes of your attention.

Start with the reassurance. Both full functions are sound for their advertised properties. Every record in Section 6 is reduced-round; 39 of 64 steps of SHA-256 and 6 of 24 rounds of SHAKE256 do not threaten the full functions, which remain safe for collision, preimage, and second-preimage resistance [@li-liu-wang-2024, @zhang-hou-liu-2024]. Nothing in this article is a reason to rip out SHA-256.

The one gotcha is length extension, and it is a property of the construction, not a defect to be patched. If you ever compute a keyed digest as $H(\text{secret} ,|, \text{message})$ with a plain Merkle-Damgard hash and treat it as a MAC, an attacker who sees that digest can forge a valid one for a longer message without knowing the secret.

A construction that turns any iterated hash into a keyed message authentication code by nesting two hash calls, roughly $H((k \oplus opad) \,\|\, H((k \oplus ipad) \,\|\, m))$. The outer keyed hash means an attacker never sees a raw $H(\text{secret} \,\|\, \cdots)$ digest to extend, which is why HMAC defeats length extension even over SHA-256.

The decision guide fits in one table.

Your need	Use	Why it works
Keyed integrity / a MAC	HMAC over SHA-2	the outer keyed hash hides the extensible state [@rfc-2104]
A drop-in SHA-2 that resists extension	SHA-512/256	truncation hides 256 state bits, more than enough [@fips-180-4]
Length-extension immunity by construction	SHA-3 or SHAKE	the capacity is never emitted, so there is no state to resume [@fips-202]
What to avoid	plain SHA-256(secret then message) as a MAC	the digest is the full state, so it is forgeable

Note: Need a keyed digest? Reach for HMAC or a SHA-3 / KMAC construction. Need a plain hash that will not leak its state? Use SHA-512/256 or any SHA-3 function. Do not hand-roll $H(\text{secret} ,|, m)$.

The mechanism is worth seeing once, because it explains why "just hash the secret with the message" fails.

sequenceDiagram participant D as Defender participant A as Attacker D->>D: hash secret then message into digest t D->>A: publish digest t and the message length Note over A: attacker never learns the secret A->>A: resume internal state from t A->>A: append glue padding then an evil suffix A->>D: send forged message plus a new digest Note over D: the new digest verifies as a valid hash

{` // A toy Merkle-Damgard hash where the state IS the digest. // This shows the STRUCTURE of length extension, not real SHA-256 output. function compress(state, ch) { // stand-in mixing; a real f is Davies-Meyer over a block cipher return (((state << 5) ^ (state >>> 2) ^ ch) >>> 0); } function hash(state, message) { for (let i = 0; i < message.length; i++) { state = compress(state, message.charCodeAt(i)); } return state >>> 0; }

const IV = 0x9e3779b9;

// Defender hashes a secret-prefixed message and publishes only the digest. const secretPrefixed = "SECRET-user=guest"; const digest = hash(IV, secretPrefixed);

// Attacker never learns the secret, but the digest IS the state, // so they resume from it and append a chosen suffix. const suffix = "&admin=true"; const forged = hash(digest, suffix);

// In a real hash the attacker also splices in the glue padding, // producing H(m || pad || suffix). Here (no padding) it lands exactly: const legitimate = hash(IV, secretPrefixed + suffix);

console.log("forged =", forged); console.log("expected =", legitimate); console.log("match =", forged === legitimate); `}

A worked forgery for SHA-256 and SHA-512 is exactly what the hash_extender tool automates [@hash-extender]. But truncation is where the naive intuition "any truncation stops extension" quietly fails, and SHA-224 is the trap.

Note: SHA-224 is SHA-256 truncated to 224 bits, so it emits seven of the eight 32-bit words of the internal state and hides only one -- about 32 bits. An attacker who wants to resume the iteration is missing just those $2^{32}$ possibilities, which is a brute-force search a laptop finishes quickly, after which extension proceeds normally. So SHA-224 is not immune to length extension; it merely raises the bar by a weak 32-bit margin. That point is a structural word-count argument, not something the standard tool shows: hash_extender declines to implement SHA-224 precisely because the naive extension does not directly apply, so it lists SHA-224 as "not" vulnerable [@hash-extender]. Do not read that "not" as safety -- read it as "32 bits of margin." True immunity needs a truncation that hides enough state, which is why SHA-512/256, hiding 256 bits, genuinely resists extension and SHA-224 does not [@fips-180-4].

Two smaller notes round out the practical picture. First, a naming hazard that bites developers.Ethereum's "Keccak-256" is not SHA3-256. Ethereum froze the original Keccak submission before FIPS 202 added its domain-separation padding, so the two functions append different bits before permuting and therefore produce different digests on the same input [@fips-202]. Code that assumes they are interchangeable will compute the wrong hash. This is a padding difference, not a security difference -- but it is a real bug source. Second, the quantum question again: if you are hedging against a future quantum adversary, the move is a longer digest -- SHA-384, SHA-512, or SHAKE256 at a longer output -- not a migration to a different construction, because Grover is a generic speedup that a bigger output absorbs [@nist-hash-policy].

Every one of these recommendations is a corollary of a single idea -- the construction decides the failure mode. A few common misconceptions fall out of the same idea, and they are worth clearing up directly.

11. Misconceptions the Structural View Corrects

Six questions whose honest answers are all the same idea in disguise: the construction, not the digest size, decides the failure mode.

No -- it *supplements* the SHA-2 family rather than replacing it, and NIST sees no need to migrate; the two coexist as deliberate design diversity, so a break in one construction leaves the other standing. See Section 4 [@fips-202, @nist-hash-policy]. Only for preimages, and only generically: Grover square-roots a preimage search but gives no real edge on collisions in bounded-memory models, so the hedge is a longer digest, not a new construction. See Section 8 [@nist-hash-policy]. No -- only a truncation that hides *enough* state. SHA-512/256 hides 256 bits and resists it; SHA-224 hides only about 32, a weak margin rather than true immunity. See the SHA-224 truncation trap in Section 10 [@fips-180-4]. No. A reduced-round record measures distance to a break, not nearness to one: the deepest results reach only about 39 of SHA-256's 64 steps and 6 of Keccak's 24 rounds. See Section 6 [@li-liu-wang-2024, @zhang-hou-liu-2024]. No. Ethereum froze the pre-FIPS-202 Keccak padding, so Keccak-256 and SHA3-256 return different digests on the same input -- same permutation, different padding. See Section 10 [@fips-202]. No -- it is a provable structural property of any untruncated Merkle-Damgard hash, removed (not patched) by HMAC, sufficient truncation, or the sponge. See Sections 2 and 10 [@rfc-2104, @fips-202].

Every answer traces back to the same sentence. It is worth stating that sentence one last time, with all the evidence now behind it.

12. The Construction Writes the Obituary

We opened with a claim that should have sounded like a bluff: that we could describe, mechanically, how each of these functions would fall before either has ever fallen. Now the description is complete, and the two obituaries share nothing but a date that has not arrived.

SHA-2 would die the way MD5 and SHA-1 died -- a Wang-style differential collision threaded through its compression function, deterministic in the early steps and cheap through the tail. It survives today only because its carry-laden schedule keeps that path unfindable past about 39 of 64 steps, and it carries length extension as a structural debt from the 1989 bargain [@li-liu-wang-2024].

SHA-3 would die a different death entirely, on the algebra of a single degree-2 step: the cube frontier that stalls at 9 keyed rounds, and the internal-differential collisions that hold the records at 6 [@zhang-hou-liu-2024]. Its deepest known property reaches all 24 rounds and still breaks nothing, because the capacity provably stands between a permutation distinguisher and a hash collision.

And the margins are the payoff. Roughly 25 to 33 of SHA-256's 64 steps, and roughly 18 of Keccak's 24 rounds, remain entirely untouched after thirty-five years of the best cryptanalysis in the world [@li-liu-wang-2024, @zhang-hou-liu-2024, @dinur-etal-2015]. The construction writes the obituary; the cryptanalysis only fills in the date, and the date is nowhere close.

The construction writes the obituary. A hash does not die at its digest size -- it dies where its construction is thinnest, and no two constructions are thin in the same place.

That is the through-line, and it hands off cleanly. How these functions have actually failed people in the field -- Flame's rogue certificate, the long death of SHA-1 in the browser -- is a different kind of story, told by the empirical sibling to this piece, "How the Hash Functions Broke in Real Life."

And it points forward: if a hash's construction predicts its structural death, the natural next question for this series is whether the same is true for the ciphers and signature schemes still to come -- whether, for every primitive, the shape of the thing already names the way it would break.

How the Hash Functions Broke in Real Life: MD5, Flame, SHATTERED, and the Long Death of SHA-1

noreply@paragmali.com (Parag Mali) — Fri, 17 Jul 2026 00:00:00 GMT

**MD5 and SHA-1 -- the two hashes that once fingerprinted the entire internet -- are genuinely, mathematically broken.** Their collision resistance fell to cryptanalysis between 2004 and 2020. But every real breach they enabled -- the 2008 rogue certificate authority, 2012's Flame espionage malware, 2017's SHATTERED, 2020's Shambles -- needed a *second* failure beside the broken math: a deployment that had not yet stopped trusting the dead algorithm. That empirical record is the base rate that justifies migrating to SHA-2 and SHA-3, and it is why "broken" is narrower than you think: HMAC-SHA-1 is still fine, and preimage resistance held. Here is exactly what fell, what did not, and how to audit and migrate your own systems.

1. The Forged Certificate That Rode Windows Update

In the spring of 2012, espionage malware called Flame spread through the Middle East wearing a disguise almost nothing could see through: it arrived signed by Microsoft, and reached some targets through Windows Update itself [@kaspersky-flame-gadget]. To a defending machine, it looked like a legitimate patch from the one vendor it trusted most. To forge that signature, its operators did something the open cryptographic literature had not yet published. They engineered a chosen-prefix collision in MD5 -- a hash academics had called broken since 2004 [@wang2004-md5-eprint] -- and counterfeited a certificate that chained up to Microsoft's own root of trust [@stevens-counter-crypto-c13].

The certificate they forged was not a plainly-issued code-signing certificate. It was a Terminal Server Licensing Service certificate living inside Microsoft's Enforced Licensing public-key infrastructure -- an obscure corner of Microsoft's PKI that the company was, in 2012, still signing with MD5. When Microsoft published Security Advisory 2718704 on 3 June 2012, the fix was to revoke exactly those certificates: the "Microsoft Enforced Licensing Intermediate PCA" and the "Microsoft Enforced Licensing Registration Authority CA" [@ms-advisory-2718704].

A deterministic function that compresses a message of any length into a short, fixed-length string called a digest (or hash). The same input always yields the same digest, a different input almost always yields a different one, and the function is designed to be one-way: easy to compute forwards, infeasible to invert.

The unsettling part of Flame is not that the math failed. It is that the math had failed nearly a decade earlier, in a Chinese cryptographer's paper read at a 2004 conference, and Microsoft's licensing servers were still signing with it eight years later. A broken primitive sat, patiently, underneath a live production certificate authority until someone with a nation-state budget decided to point a collision at it.

Now hold one more artifact in your head, because you will learn to read it by the end of this article. It is a SHA-1 digest:

38762cf7f55934b34d179ae6a4c80cadccbb7f0a

In February 2017, researchers at CWI Amsterdam and Google published two different PDF files that both hash to exactly that value [@shattered-2017]. Two distinct documents, one fingerprint -- a thing that was supposed to be computationally impossible for the hash guarding TLS certificates, Git commits, and software signatures across the internet.

Note: Every break in this story needed two failures at once: a hash whose collision resistance had already fallen, and a deployment that had not yet stopped trusting it. Cryptanalysis was necessary. It was never, by itself, sufficient.

That diagnostic sentence is the spine of everything that follows. It is also why this article is the honest exception in a series otherwise arguing that the deployment breaks, not the math. Here the math genuinely broke. And even here, on the one topic where the primitive itself fell, no attacker ever won on cryptanalysis alone. They won where broken math met a system still trusting it.

So if MD5 was known-broken years before Flame, why was anything still signing with it? To answer that, we have to be precise about what a hash actually promises -- because only one of its promises is fragile, and it is the exact one a certificate leans on.

2. What a Hash Promises, and Which Clause Is Fragile

A cryptographic hash function makes three separate promises. They sound similar, they are routinely confused, and telling them apart is the difference between understanding this whole story and mis-stating it. Only one of the three is fragile, and it is the one certificates depend on.

Given a target digest `h`, it is infeasible to find any message `m` such that `hash(m) = h`. This is the "one-way" promise: you cannot run the function backwards from an output to an input. Given a specific message `m1`, it is infeasible to find a *different* message `m2` with the same digest. The attacker is handed one input and must match its fingerprint with another. It is infeasible to find *any* two messages `m1` and `m2`, both of the attacker's choosing, that produce the same digest. Nobody hands the attacker a target. They get to pick both sides. This is the weakest promise to keep, and the one every certificate and signature quietly relies on.

The gap between those last two properties is not cosmetic; it is exponential, and it has a name.

For an `n`-bit digest, finding a collision takes only about `2^(n/2)` work, not `2^n`. The name comes from the birthday paradox: in a room of just 23 people, two probably share a birthday, because what matters is the number of *pairs*, which grows quadratically. Collisions among many hashes are similarly cheap.

That square-root haircut is brutal. By that bound, a 128-bit MD5 digest [@rfc1321-md5] offers only about $2^{64}$ collision security, and a 160-bit SHA-1 digest only about $2^{80}$. Preimage and second-preimage attacks still cost close to the full $2^{128}$ or $2^{160}$, which is why -- spoiler for Section 8 -- those properties survived while collision resistance did not. When you hear "MD5 is broken," the precise statement is "MD5's roughly $2^{64}$ collision barrier fell far below $2^{64}$." It is not "you can invert an MD5 hash."

Property	Informal promise	Generic attack cost	What it protects
Preimage resistance	Cannot find any message for a given digest	about `2^n`	Password storage, commitments
Second-preimage resistance	Cannot match a given message's digest with another	about `2^n`	Integrity of one known file
Collision resistance	Cannot find any two messages sharing a digest	about `2^(n/2)`	Certificates, signatures, content addressing

One construction under all of them

MD5, SHA-1, and SHA-2 are not three unrelated designs. They are three members of one family, built from the same skeleton.

A design that turns a fixed-size compression function into a hash for arbitrary-length messages. You start from a fixed initial value (the IV), split the padded message into blocks, and feed each block plus the running state through the compression function. The final state is the digest. flowchart LR IV["Fixed IV"] --> F1["Compress"] M1["Block 1"] --> F1 F1 --> F2["Compress"] M2["Block 2"] --> F2 F2 --> F3["Compress"] M3["Block 3 plus length padding"] --> F3 F3 --> D["Digest"]

Ronald Rivest published MD5 in April 1992 as RFC 1321, producing a 128-bit digest [@rfc1321-md5].Rivest is the "R" in RSA. MD5 was his hardened successor to MD4, and RFC 1321 says it "backs off" from MD4's speed for a "greater likelihood of ultimate security." The US government's SHA lineage followed a parallel path: SHA (retroactively SHA-0) arrived in FIPS 180 in 1993, and SHA-1 replaced it in FIPS 180-1 on 17 April 1995 with a 160-bit digest [@fips-180-1].SHA-0 was withdrawn within about two years for an undisclosed weakness. SHA-1 differs from it by a single added one-bit rotation in the message schedule -- a fix whose importance Section 3 makes concrete.

Why a collision is a forgery

Here is the load-bearing fact that ties the fragile clause to the real world. A digital signature does not sign your whole message. It signs the hash of your message, because the message may be gigabytes and the signing math is slow. The signer computes a digest, then encrypts that short digest with a private key.

flowchart TD M1["Message A, benign"] --> H["Hash function"] M2["Message B, malicious"] --> H H --> D["One shared digest"] D --> S["Signer signs the digest"] S --> Sig["A single signature"] Sig --> Cover["Valid for A and for B"]

Follow the consequence. If an attacker finds two messages with the same digest, and persuades someone to sign the harmless one, that same signature is automatically valid on the malicious one. The certificate authority signs a benign-looking certificate request; the attacker walks away with a valid signature on a forged certificate that shares its hash. Collision resistance is the only thing standing between "sign this document" and "you just signed a document you never saw."

const safe = ["hmac", "checksum", "legacy-verify", "kdf"];

if (collisionDependent.includes(use)) { return "MIGRATE -- collision-dependent, move to SHA-2 or SHA-3"; } if (safe.includes(use)) { return "OK -- collision resistance is not relied on"; } return "REVIEW -- ask: can an adversary supply both inputs and profit?"; }

["certificate", "hmac", "git-object", "checksum", "code-signing"].forEach(function (u) { console.log(u, "->", auditHashUse(u)); }); `}

Choosing a target

Once you know a use must move, pick the destination by requirement, not by fashion. SHA-256 is the FIPS default and the safe first answer: NIST tells designers to implement it at a minimum, and it is hardware-accelerated on modern CPUs [@fips-180-4-sha2, @nist-policy-hash]. Want length-extension resistance for free? SHA-512/256 truncates a wider computation and never leaks its full state [@fips-180-4-sha2]. Want structural diversity or an extendable-output function? Reach for SHA-3 or SHAKE [@fips-202-sha3].

If raw speed dominates and a FIPS stamp is not required, BLAKE3 is the fastest of the group and length-extension-immune [@blake3-repo]. And if you are stuck on a SHA-1 you cannot rip out this quarter, deploy sha1dc collision detection the way Git did, then schedule the real migration behind it [@sha1dc-stevens-shumow-2017].

The mistakes, mapped to the breaks

Every common hashing mistake is a small echo of one of the four incidents. Naming the echo makes the fix obvious.

The mistake	The break it echoes	The fix
MD5 or SHA-1 for certificates or code signing	Rogue CA (2008), Flame (2012)	SHA-256 signatures [@fips-180-4-sha2]
SHA-1 as a content-address or dedup key	SHATTERED (2017)	SHA-256 objects, `sha1dc` as a stopgap [@sha1dc-stevens-shumow-2017]
SHA-1 for identity or Web-of-Trust signatures	Shambles (2020), CVE-2019-14855	Modern GnuPG on SHA-256 [@shambles-2020]
`H(secret \|\| message)` used as a MAC	Length extension	HMAC, or KMAC on SHA-3 [@rfc2104-hmac]
Assuming a collision break means a preimage break	--	Keep salted preimage and HMAC uses; migrate only collision-dependent ones [@nist-policy-hash]

Two pitfalls deserve their own warning because they masquerade as fixes. Swapping Ethereum's Keccak-256 for standardized SHA3-256 silently changes your digests [@fips-202-sha3]. And treating sha1dc as a migration rather than a stopgap leaves you permanently one clever attack away from trouble; detection buys time, it does not end the dependency.

Note: Whatever legacy you must tolerate, draw one hard line: never generate a new MD5 or SHA-1 signature, certificate, or code-signing object. Every incident in this article began where a system was still willing to sign with a dead hash.

Internalize that audit question and these habits, and you have absorbed the base rate this whole history was written to establish. It remains only to name it.

11. The Base Rate

Return to the digest from the opening: 38762cf7f55934b34d179ae6a4c80cadccbb7f0a. You can read it now. It is a SHA-1 value shared by two different PDFs -- an identical-prefix collision from SHATTERED, which means it proved SHA-1 dead for content addressing but could not, on its own, forge a certificate for a chosen identity [@shattered-2017]. That one artifact contains the whole argument in miniature: a real mathematical break, precisely scoped, whose damage depended entirely on what still trusted it.

Stretch that artifact across nearly four decades. From MD5's publication in 1992 to the federally mandated retirement of its successor SHA-1 in 2030 is a thirty-eight-year empirical record [@rfc1321-md5, @nist-sha1-retire-2022]. Inside it sit four undisputed, reproducible, weaponized breaks -- the 2008 rogue CA, 2012's Flame, 2017's SHATTERED, 2020's Shambles. Four in twelve years, on two primitives that fell the same way, is not a scare story. It is a measured base rate, and it is the entire justification for migrating to SHA-2 and SHA-3.

Every break in this story needed two failures at once: a hash whose math had fallen, and a deployment that had not yet stopped trusting it.

That is how this article honors the series' thesis without lying about the mathematics. This is the one topic where the primitive genuinely fell -- and even here, cryptanalysis was necessary and never sufficient. RapidSSL had to still be signing with MD5. Microsoft's licensing PKI had to still be signing with MD5. GnuPG had to still default to SHA-1. The math was the loaded chamber; the still-live deployment pulled the trigger.

And keep the boundary honest, because the slogan overreaches in both directions. "Broken" is scoped to collision-dependent uses: HMAC-SHA-1 survives on a proof, and preimage resistance held [@rfc2104-hmac, @nist-policy-hash]. The successors do not stand in a line of succession at all -- SHA-2 and SHA-3 coexist by diversification, one construction covering the other's structural blind spot.

For the mechanisms of how the successors themselves could someday be attacked -- Merkle-Damgard collisions, length extension, and the sponge's algebraic frontier -- see the structural sibling to this piece, How SHA-2 and SHA-3 Would Break, along with the series' dedicated hash and X.509/PKI field guides. The migration itself is not a debate. It is an argument the evidence already won. The only open question left is how fast the still-live tail catches up before the next drop in attack cost reaches the systems that forgot to.

12. Frequently Asked Questions

Yes, for collision-dependent uses such as signatures and certificates, where its collision resistance has fallen far below the birthday bound. No, for its other promises: SHA-1 preimage resistance remains intact, and HMAC-SHA-1 is not broken by any collision attack [@rfc2104-hmac, @nist-policy-hash]. "Broken" is a statement about one property, not the whole function. No, and the difference is the whole game. SHATTERED (2017) is an identical-prefix collision: it proved SHA-1 dead for content addressing but cannot forge a chosen identity [@shattered-2017]. Shambles (2020) is a chosen-prefix collision: strictly more powerful, and the capability that actually forges certificates and impersonates identities [@shambles-2020]. No. Git added `sha1dc` collision detection and an opt-in SHA-256 object format, but SHA-1 is still the default object format for most repositories [@sha1dc-stevens-shumow-2017, @git-hash-transition]. The transition is staged and largely unfinished; do not assume your repository has migrated. Yes. HMAC's security reduces to the compression function acting as a pseudorandom function, not to collision resistance, so the collision attacks in this article do not apply. RFC 2104 stated this in 1997, and later proofs formalized it [@rfc2104-hmac]. You do not need to rip HMAC-SHA-1 out on account of SHATTERED or Shambles. No. Antoine Joux showed in 2004 that multicollisions in iterated hashes make a concatenation only about as strong as its stronger component, not the sum of the two [@joux-multicollisions-c04]. Cascading broken hashes is a dead end; migrate to a single strong hash instead. No. Ethereum uses the original Keccak padding, finalized before FIPS 202 changed the padding rule for standardized SHA-3. The two produce different digests for the same input, so the libraries are not interchangeable [@fips-202-sha3]. Yes, but narrowly. Non-adversarial checksums that detect accidental corruption, HMAC, key derivation, and verification of already-trusted legacy data are all acceptable, because none relies on an adversary being unable to find a collision [@nist-policy-hash]. Never use either for a new signature, certificate, or code-signing operation.

The Fingerprint Two Files Shared: A Field Guide to Cryptographic Hashes

noreply@paragmali.com (Parag Mali) — Fri, 10 Jul 2026 16:04:13 GMT

A cryptographic hash makes one promise: a short fingerprint that stands in, unforgeably and unambiguously, for any input. MD5 and SHA-1 are dead for collision-dependent uses. Their broken collision resistance was weaponized into forged certificates (the 2008 rogue certificate authority [@rogue-ca], Flame in 2012 [@stevens-counter-cryptanalysis-crypto13]) and forged documents and keys (SHAttered in 2017 [@shattered], Shambles in 2020 [@shambles]). Yet HMAC-SHA-1 is not broken [@rfc-2104], and "dead" never meant "extinct." SHA-2 survived by widening the same Merkle-Damgard margins, while SHA-3's sponge and the BLAKE3 tree diversified the toolbox rather than replacing the survivor [@fips-202, @blake3-repo]. Correct usage is clause-matching: pick the function and the mode (bare hash, HMAC, KMAC or cSHAKE, an extendable-output function, or a password KDF) that keeps the exact clause your protocol leans on. This guide gives you the promise, the constructions, every named break with its root cause, and a decision tree you can apply the same afternoon.

1. The Fingerprint Two Files Shared

In February 2017, researchers at Google and CWI Amsterdam published two PDF files that looked nothing alike, with different visible content and different bytes, yet when a computer fingerprinted each of them with SHA-1 it returned the very same 40-hex-digit answer: 38762cf7f55934b34d179ae6a4c80cadccbb7f0a [@shattered]. Two different files, one identical fingerprint. Five years earlier, malware named Flame had used the same species of trick against the older MD5 hash to forge a code-signing certificate that chained up to Microsoft and rode Windows Update onto target machines as if Microsoft itself had signed it [@stevens-counter-cryptanalysis-crypto13].

Neither of those is a stunt. They are the failure mode of one of the quietest, most load-bearing primitives in computing. A cryptographic hash function takes an input of any size and returns a short, fixed-length string of bits, its digest or fingerprint, and it promises something enormous: that fingerprint stands in for the whole input unforgeably and unambiguously.

Almost everything you trust online leans on that promise. A certificate authority does not sign your certificate; it signs a hash of it. Code signing signs a hash of the program. A TLS 1.3 handshake authenticates a hash of the transcript. Every Git commit names its parent by hash, and every blockchain block commits to its transactions through a tree of hashes. When the promise holds, all of that machinery works. When it breaks, it breaks silently and all at once.

Key idea: A cryptographic hash is infrastructure hiding in plain sight. Because signatures, certificates, and integrity checks all sign the digest rather than the data, a break in the hash silently breaks every signature built on top of it. The attacker never has to touch your private key.

That is what makes the two colliding PDFs unsettling rather than merely clever. If an adversary can produce two inputs with the same fingerprint, then any signature over one of them is also a valid signature over the other, and the entire chain of trust resting on that hash quietly inverts. This guide answers four questions in order: what exactly is the promise a hash makes, how is the fingerprint built, how has it broken in the real world, and how do you choose one today that will not break under you.

flowchart TD A["One promise: a short, unforgeable, unambiguous fingerprint"] --> B["How it is built: the Merkle-Damgard chain and the sponge"] B --> C["How it broke: MD5 and SHA-1 collisions weaponized into forgery"] C --> D["What survived: SHA-2 widened the same margins"] D --> E["What diversified: the SHA-3 sponge, the BLAKE3 tree, KangarooTwelve"] E --> F["How to choose: match the function and mode to the clause you rely on"]

A preview of the plot, because it corrects the single most common misconception in the whole subject. Some hashes died and were superseded: MD5 gave way to SHA-1, which gave way to SHA-2. But SHA-2 did not then give way to SHA-3. Instead the field forked. One arc is supersession by failure; the other is deliberate diversification, where several unbroken constructions coexist on purpose so the internet's trust machinery never again rests on a single design. Telling those two arcs apart is most of what it means to understand this topic.

Before we can understand how the fingerprint was forged, we have to say, precisely, what it was ever supposed to guarantee.

2. What a Hash Actually Promises

Ask three engineers what a "secure hash" means and you will get three answers: it is one-way, it has no collisions, it looks random. Cryptographers are more disciplined. They name exactly three properties, plus one more that everyone wishes were true and no real function actually delivers. Getting these four straight is the whole vocabulary, because every break later in this guide is best described as one specific property falling while the others stand.

Start with the object itself. A hash function $H$ maps an input of essentially unbounded length to an output of some fixed length $n$ bits: 256 bits for SHA-256, 160 for SHA-1, 128 for MD5. Because the input space is infinite and the output space has only $2^n$ points, collisions must exist by counting alone. Security is never about avoiding collisions in principle; it is about making them, and two related feats, computationally unreachable.

Phillip Rogaway and Thomas Shrimpton gave the field its rigorous, separated definitions of these properties in 2004, and it is worth stating them the way they did, because the separations matter [@rogaway-shrimpton-fse04].

A deterministic function that maps an input of arbitrary length to a fixed-length digest, engineered so that the digest behaves like a compact, tamper-evident fingerprint of the input: cheap to compute forwards, and infeasible to invert, to steer to a chosen value, or to make two inputs share, except by effort that grows exponentially in the digest length. Given only a target digest $y$, it is computationally infeasible to find any input $m$ with $H(m) = y$. This is the "one-way" property. For an ideal $n$-bit hash the generic cost is about $2^{n}$ evaluations, because you have no better strategy than trying inputs until one lands on $y$. Given a specific input $m_1$, it is infeasible to find a different input $m_2 \ne m_1$ with $H(m_2) = H(m_1)$. The target is fixed and not of your choosing. The generic cost is again about $2^{n}$. It is infeasible to find any two distinct inputs $m_1 \ne m_2$ with $H(m_1) = H(m_2)$. Here you are free to choose both inputs, which is exactly why it is easier to attack. The generic cost is only about $2^{n/2}$, by the birthday bound.

The gap between $2^{n}$ and $2^{n/2}$ is the single most important number in this article. Preimage and second-preimage pit you against a fixed target, so you are searching a space of size $2^n$. Collision lets you range freely over pairs, and pairs accumulate quadratically.

In a space of $N = 2^{n}$ equally likely values, a randomly drawn set is more likely than not to contain a repeat after only about $\sqrt{N} = 2^{n/2}$ draws, because the number of candidate pairs among $k$ items grows like $k^2/2$. Formally the collision probability after $k$ draws is about $1 - e^{-k(k-1)/(2N)}$.

The name comes from the party trick: in a room of just 23 people, two of them probably share a birthday, even though there are 365 candidate days, because 23 people form 253 pairs. For a 256-bit hash the same arithmetic says a collision surfaces after about $2^{128}$ digests rather than $2^{256}$, so a 256-bit hash buys you only 128 bits of collision resistance. This is why collision is the weakest joint, and, as the failure catalog will show, always the first to fall.

Clause	What the attacker must find	Generic cost	Who leans on it	First to fall?
Preimage resistance	Any input hashing to a given digest	$2^{n}$	Password hiding, commitments, key derivation	Last
Second-preimage resistance	A different input matching one fixed input	$2^{n}$	Integrity of a known, published file	Middle
Collision resistance	Any two distinct inputs that agree	$2^{n/2}$	Signatures, certificates, deduplication	First

Second-preimage feels like it should be as easy as collision, but it is not. A collision lets you shop for any lucky pair anywhere in the space. A second preimage nails one endpoint to a value someone else chose and dares you to hit it from the other side, which is a $2^n$ problem, not a $2^{n/2}$ one. Attackers who can only find collisions therefore need to control both documents, which is precisely the constraint the forged-certificate attacks had to engineer around.

The following simulator makes the gap tangible. It uses a toy 24-bit hash so both effects fit in a browser tab: watch how few inputs it takes to stumble into a collision, and how hopeless it is to hit one fixed target by search.

{` // A toy 24-bit "hash": mix the input, then keep the low 24 bits (N = 2^24). const N = 1 << 24; // 16,777,216 possible digests function h(x) { let v = Math.imul(x ^ 0x9e3779b1, 2654435761) >>> 0; v ^= v >>> 15; // fold high bits down so the low bits mix v = Math.imul(v, 0x85ebca6b) >>> 0; v ^= v >>> 13; return v & (N - 1); }

// Collision search: hash 1, 2, 3, ... and stop at the first repeat. const seen = new Map(); let collisionAt = null; for (let i = 1; i < 300000 && collisionAt === null; i++) { const d = h(i); if (seen.has(d)) collisionAt = i; else seen.set(d, i); } console.log("birthday bound, sqrt(N), is about", Math.round(Math.sqrt(N))); console.log("first collision appeared after only", collisionAt, "inputs");

// Preimage search: how many tries to hit ONE fixed target digest? const target = h(42), budget = 500000; let tries = 0, hit = false; for (let x = 5000000; x < 5000000 + budget; x++) { tries++; if (h(x) === target) { hit = true; break; } } console.log("preimage search used", tries, "tries and", hit ? "got lucky" : "found nothing"); console.log("average preimage cost would be about N/2 =", N / 2, "tries"); `}

Collisions surface after only a couple of thousand inputs, while the fixed-target search burns through half a million tries and, on average, would need eight million. That is the $2^{n/2}$ versus $2^{n}$ split in miniature.

There is a fourth property people silently assume: that a good hash behaves like a random oracle, a magic box that returns a fresh uniformly random answer for every distinct input and is otherwise consistent. It is a wonderful modeling fiction, and much of applied cryptography is proved secure by pretending it is real.

But no fixed, finite, publicly specified function can actually be a random oracle, and Canetti, Goldreich, and Halevi proved the gap is not merely cosmetic: there exist schemes secure with an ideal oracle yet insecure under every concrete instantiation [@cgh-jacm-2004]. We make that precise in Section 10. For now, hold the three real clauses in mind, plus the knowledge that the fourth is a heuristic.

Two consequences follow immediately and set up everything after. First, you never sign a document; you sign its digest, so a signature is only as unforgeable as the hash is collision resistant. The formal security goals for signatures and message authentication codes are the subject of Part 1 of this series, the security-definitions field guide; here we only need the consequence. Second, the weakest clause is collision resistance, sitting at $2^{n/2}$. To see how that clause snapped in practice, we first have to see how the fingerprint is assembled, because the assembly leaks in ways the three clauses alone do not reveal.

3. How the Fingerprint Was Built: The Merkle-Damgard Era

The hash function became load-bearing infrastructure almost by accident. When Whitfield Diffie and Martin Hellman introduced public-key cryptography in 1976, their signatures could only operate on short numbers, roughly the size of the key [@diffie-hellman-1976]. But people wanted to sign whole documents, contracts, and programs. The fix was quiet and consequential: do not sign the document, sign a short fingerprint of it. That single move loaded the entire weight of a signature onto the hash. If two documents can share a fingerprint, one signature covers both, so the signature scheme is only as sound as the hash beneath it.

Ralph Merkle's 1979 Stanford thesis formalized one-way hash functions and, in its "Tree Authentication" section, introduced the hash tree that now underlies Git, blockchains, and verified downloads [@merkle-thesis-1979].

The next question was how to build such a fingerprint for an input of any length out of a small, analyzable building block. In 1989, Ralph Merkle and, independently, Ivan Damgard supplied the answer that defined the next two decades [@merkle-crypto89, @damgard-crypto89].

An iterated hash built by padding the message, splitting it into fixed-size blocks, and folding the blocks one at a time through a compression function, starting from a fixed initial value called the IV. Merkle and Damgard proved that if the compression function is collision resistant and the final block encodes the message length, a trick called MD strengthening, then the whole hash inherits collision resistance from the small compression function.

That theorem is the reason the 1990s could build hashes with confidence: analyze one small function, get a hash for arbitrary inputs for free. The digest is simply the chaining state after the last block has been absorbed.

flowchart LR IV["IV: fixed start state"] --> C1["compress"] M1["block m1"] --> C1 C1 --> C2["compress"] M2["block m2"] --> C2 C2 --> C3["compress"] MP["final block: message plus length padding"] --> C3 C3 --> D["digest: the final chaining state"]

Ronald Rivest built the first widely used family on this shape. MD4 arrived in 1990 and the strengthened MD5 in 1992, both producing 128-bit digests from an add-rotate-xor compression function [@rivest-md4-crypto90, @rfc-1320]. RFC 1321 states MD5's purpose plainly: it compresses a message "in a secure manner before being signed with a private key" [@rfc-1321]. The design DNA, a message schedule feeding an ARX round function, propagated forward. In 1993 the U.S. government standardized a 160-bit hash of the same lineage in FIPS 180, retroactively called SHA-0, and in 1995 replaced it with SHA-1, a one-bit tweak of the same design [@nist-hash-functions]. SHA-1's specification survives in the current Secure Hash Standard [@fips-180-4], and it went on to hash most of the internet for roughly twenty years.

The only visible difference between SHA-0 and SHA-1 is a single one-bit rotation added to the message schedule in 1995, a small and initially unexplained change later understood to improve collision resistance. The hedge held for a while, but SHA-0 was later shown to be genuinely fragile: Xiaoyun Wang and colleagues found efficient full collisions for SHA-0 well under the birthday bound in 2005 [@wang-yu-yin-sha0-crypto05]. The quietly patched flaw foreshadowed the cryptanalysis that would soon reach SHA-1 itself.

This detour, hashing before signing, is so routine that engineers forget it is a security assumption. A signature over `H(m)` is meant to bind you to `m`, but it truly binds you only to `H(m)`. If an adversary finds a second message with the same digest, your signature covers that message too, and no key was ever stolen. Every certificate, code signature, and signed software update in this article inherits this assumption, which is why a collision in the hash is not an academic curiosity but a forged signature waiting to happen.

The Merkle-Damgard theorem promised something precise and limited: the whole hash is as collision resistant as its little compression function. It never promised that the shape wrapped around that function, the chaining, the padding, the exposed final state, was itself sound. That distinction is exactly where the first cracks opened, and they opened even for a hypothetically perfect compression function.

4. The Structural Cracks: Why Merkle-Damgard Leaks

Here is the unsettling part. You can hand the Merkle-Damgard construction a perfect compression function, one with no weakness whatsoever, and the assembled hash still leaks. It leaks four separate ways, and none of them is a bug in any particular function. They are properties of the shape.

The first leak is the most practical, and it follows directly from the diagram in the previous section. The digest is the final chaining state. Nothing hides it. So if you know a digest and the length of the message that produced it, you can pick up the computation exactly where it left off.

Given a valid digest $H(m)$ and the length of $m$, but not $m$ itself, an attacker can compute $H(m \,\|\, \text{pad} \,\|\, s)$ for a suffix $s$ of their choosing. It works because a Merkle-Damgard digest is literally the internal chaining state after the last block, so the attacker resumes from that state and folds in more blocks, without ever knowing $m$.

The mechanism has exactly three moving parts, and it needs neither the message nor any secret. Publishing $H(m)$ publishes a fully loaded, resumable hash engine. From the length of $m$ the attacker rebuilds the glue padding, the 0x80 byte, the run of zeros, and the encoded bit-length that Merkle-Damgard appends before the final block [@fips-180-4]. The attacker reloads $H(m)$ as the IV, folds in the glue and then an arbitrary suffix, and reads off a valid digest for the longer message. When the server later hashes secret || m || glue || suffix it walks the identical chain, passes through the intermediate state $H(m)$, folds in the same suffix, and confirms the forgery.

flowchart LR PUB["Public: digest H(m) and length of m"] --> REBUILD["Rebuild glue padding from the length alone"] REBUILD --> RELOAD["Reload H(m) as the compression IV"] RELOAD --> FOLD["Fold in glue, then attacker suffix"] FOLD --> FORGE["Forged digest for m plus glue plus suffix"] SERVER["Server hashes secret plus m plus glue plus suffix"] -.walks the identical chain.-> FORGE

This turns a natural-looking construction into a trap. Many engineers reach for H(secret || message) as a homemade message authentication code, reasoning that you cannot forge a tag without knowing the secret. Length extension refutes that: given a valid tag for one message, an attacker appends chosen data and computes a valid tag for the longer message, secret still unknown. The toy hash below is deliberately tiny and insecure, but it has the one property that matters, its digest is its whole internal state, so it forges end to end exactly as MD5 or SHA-256 would.

{` // A toy Merkle-Damgard hash. Like MD5 and SHA-256, its DIGEST IS ITS FINAL // INTERNAL STATE -- that single fact is the entire length-extension bug. const BLK = 8; const IV = 0x1234abcd >>> 0; const enc = s => [...s].map(c => c.charCodeAt(0));

// The server publishes (message, tag) where tag = H(secret || message). const secret = enc("SECRET"); // 6 bytes the attacker never sees const message = enc("user=guest"); const tag = toyHash(secret.concat(message));

// The attacker knows tag, message, and only the LENGTH of the secret (6). const secretLen = 6; const glue = pad(secretLen + message.length); // rebuilt from lengths, no secret const suffix = enc("&role=admin"); const total = secretLen + message.length + glue.length + suffix.length; const forgedTag = fold(tag, suffix.concat(pad(total))); // resume FROM the tag const forgedMessage = message.concat(glue).concat(suffix);

// The server verifies the forged message with its real secret. const check = toyHash(secret.concat(forgedMessage)); console.log("forged tag (secret never used):", forgedTag.toString(16)); console.log("server recomputation :", check.toString(16)); console.log("forgery accepted?", forgedTag === check); console.log("smuggled 'role=admin' present?", String.fromCharCode(...forgedMessage).includes("role=admin")); `}

Real tools do this against real hashes. The canonical hash_extender demonstrates it on MD5: publish signature = MD5(secret || "data") = 6036708eba0d11f6ef52ad44e8b74d5b, and knowing only data, the algorithm, and that signature, an attacker loads the signature back into MD5's state and continues hashing a chosen suffix to forge a valid tag for secret || data || glue || suffix, never learning the secret. Its own summary is the whole lesson: "100% of the state needed to continue a hash is in the output of most hashing algorithms" [@hash-extender].

Note: On any full-width Merkle-Damgard hash, including MD5, SHA-1, SHA-256, and SHA-512, the prefix-secret construction H(secret || message) is forgeable by length extension. Use HMAC (Section 9) or a length-extension-immune hash instead. This footgun is common enough that it has its own family of real exploits in serialized-signing and token systems, examined in the serialization and canonicalization field guide; here we own the mechanism, that guide owns the exploitation.

A precise note you should carry forward, because it is the sharpest discriminator between functions in this whole article. Length extension afflicts hashes whose digest exposes the full final state: MD5, SHA-1, and the full-width SHA-256 and SHA-512. Truncation can close the leak, but only if it hides enough of the state. The truncated SHA-2 variants that hide at least 128 bits (SHA-384, SHA-512/224, SHA-512/256), the SHA-3 family and SHAKE, HMAC, and the BLAKE hashes are all immune. The dangerous middle case is SHA-224, which discards only 32 bits and is therefore still vulnerable at about $2^{32}$ work, a distinction Section 6 makes exact. Defer the full table there; just remember that "truncated" does not automatically mean "safe."

The other three leaks are subtler but drove the field toward new designs. In 2004, Antoine Joux showed that in any iterated hash, finding $2^k$ messages that all share one digest costs only about $k$ times the work of a single collision, not the vastly larger amount naive counting suggests [@joux-crypto04]. The construction is almost embarrassingly direct: from the starting state, one birthday search finds a colliding pair of blocks that both drive the chain to the same next state; repeat from that state $k$ times; then every one of the $2^k$ ways to pick one block or the other at each step traces the identical chain to the identical final digest.

These multicollisions demolish a piece of folklore that had reassured a generation of engineers: that concatenating two independent hashes, $H_1(m) ,|, H_2(m)$, multiplies their strength. It does not. Build a large multicollision in the weaker half -- many messages that all share one digest under it, which is cheap for a Merkle-Damgard hash -- then birthday-search that set for a pair that also collides under the stronger half. The combined collision resistance is essentially that of the stronger single hash, not the sum. Hashing twice buys almost nothing.

Joux's multicollisions were also the engine for two deeper results. In 2005, John Kelsey and Bruce Schneier used expandable messages built from multicollisions to find second preimages on Merkle-Damgard hashes for far less than the ideal $2^n$: for a target of $2^k$ blocks, the cost falls to roughly $2^{n-k}$ plus overhead [@kelsey-schneier-eurocrypt05]. That matters because second-preimage resistance was supposed to be the sturdy $2^n$ clause, yet the shape leaks it for long messages. In 2006, Kelsey and Tadayoshi Kohno went further with the herding, or Nostradamus, attack: an attacker precomputes a branching diamond structure, publicly commits to a digest as if it were a prediction, and later herds almost any chosen prefix into that exact digest [@kelsey-kohno-eurocrypt06]. It weaponizes the construction into proof-of-prediction fraud.

Key idea: Length extension, Joux multicollisions, long-message second preimages, and herding are flaws in the shape, not in the function. A stronger or longer compression function cannot fix them, because they do not depend on any weakness in that function. Only a different construction can. Hold that thought: it is exactly why, after the failures, the field did not just build a bigger Merkle-Damgard hash. It built a different machine.

Those four are leaks in an idealized world, where the compression function is flawless. They were serious enough to motivate truncated designs and, eventually, entirely new constructions. But they were still theoretical. What turned this subject from a seminar into a decade of emergency migrations was the other kind of failure: the compression function itself fell, and someone chose to weaponize it.

5. The Failure Catalog: Every Notable Break, Named

A broken hash is a lab curiosity until someone forges a certificate with it. Twice, someone did, and once it was a nation-state. This section is the heart of the guide, because the two failure arcs, MD5 and SHA-1, are the evidence for everything the rest of it recommends. Each arc runs the same course: a theoretical crack, a practical collision, a technique that makes the collision meaningful, and finally a real forged trust anchor.

The MD5 arc opens with an early warning that the field under-heeded. In 1996, Hans Dobbertin found practical full collisions for MD4 and collisions in MD5's compression function under a chosen initial value, not yet the full hash but a clear structural crack [@dobbertin-md4]. MD5 stayed deployed anyway.

The warning became undeniable in August 2004, when Xiaoyun Wang, Dengguo Feng, Xuejia Lai, and Hongbo Yu announced full MD5 collisions computable in minutes, alongside collisions for MD4, HAVAL-128, and RIPEMD [@wang-md5-eprint-2004-199]. The following year, Wang and Yu published the peer-reviewed differential method behind it [@wang-yu-break-md5-eurocrypt05]. That is the theoretical death of MD5. Everything after is engineering.

The engineering that mattered was learning to choose the colliding inputs.

A collision between two inputs that begin with two different, attacker-chosen prefixes. The attacker fixes meaningful, distinct starting content for each side, two different names in a certificate, say, and then computes collision blocks that drive both sides to the same digest. This is strictly harder than an identical-prefix collision, where the two inputs share a common start, and strictly more dangerous, because it forges a relationship between two meaningful documents rather than two random blobs.

In 2007, Marc Stevens, Arjen Lenstra, and Benne de Weger built exactly that for MD5, and used it to construct two X.509 certificates with different identities but the same MD5 digest [@stevens-chosen-prefix-md5-eurocrypt07].

A year later the attack left the lab. At the 2008 Chaos Communication Congress, a seven-person team used a chosen-prefix MD5 collision, about $2^{49}$ MD5 compressions on a cluster of roughly 200 PlayStation 3 consoles, to obtain a genuine commercial certificate-authority signature over a benign certificate that simultaneously validated a forged intermediate certificate authority [@rogue-ca]. The forged certificate carried the basic-constraints CA flag set to TRUE, which meant every browser on earth would trust certificates it issued.

flowchart TD P1["Benign prefix: an ordinary certificate request"] --> COL["Collision blocks appended to both sides"] P2["Malicious prefix: a rogue CA certificate"] --> COL COL --> SAME["Both certificates share one MD5 digest"] SAME --> SIG["Commercial CA signs the benign certificate"] SIG --> VALID["The same signature validates the rogue CA:TRUE certificate"]

To make two certificates collide, the attacker needs somewhere to park the roughly 1632-bit collision block without it looking suspicious. In the 2008 rogue CA, that space lived inside a "Netscape Comment" extension, a field parsers ignore, which the researchers wryly described as hiding the collision like a tumor inside an otherwise healthy certificate [@rogue-ca]. The lesson generalizes: any format with an ignorable, attacker-controlled field gives a collision attack room to breathe.

The arc peaked in June 2012 with Flame, espionage malware that carried a code-signing certificate chaining to Microsoft. Flame's authors mounted a new chosen-prefix MD5 collision against a Microsoft sub-authority still using MD5 in its licensing service, letting them sign malware as "Microsoft" and distribute it through Windows Update. Marc Stevens later reconstructed the previously unknown collision variant from the certificate itself [@stevens-counter-cryptanalysis-crypto13].

A nation-state had used a hash collision to subvert the update trust root of a billion machines. After Flame, MD5 in signatures was not deprecated as a matter of hygiene; it was radioactive.

SHA-1 traveled the same road a decade behind. In 2005, Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu published the first collision attack on the full SHA-1, at about $2^{69}$ operations, comfortably under the $2^{80}$ birthday bound its 160-bit output was supposed to guarantee [@wang-yin-yu-full-sha1-crypto05]. That theoretical break is what prompted NIST to open the SHA-3 competition. The practical break took twelve more years of hardware progress. In February 2017, a team from CWI Amsterdam and Google announced SHAttered: the first practical identical-prefix SHA-1 collision, at roughly $2^{63.1}$ SHA-1 computations, about nine quintillion of them, costing the equivalent of 6,500 CPU-years plus 110 GPU-years [@shattered]. The proof was the pair of PDFs from this article's opening, sharing the digest 38762cf7f55934b34d179ae6a4c80cadccbb7f0a.

In January 2020, Gaetan Leurent and Thomas Peyrin closed the arc with "SHA-1 is a Shambles," the first practical chosen-prefix SHA-1 collision at about $2^{63.4}$, which along the way cut the identical-prefix cost to about $2^{61.2}$ [@shambles, @shambles-eprint-2020-014]. To prove it mattered, they forged a PGP/GnuPG key certification, tracked as CVE-2019-14855, building on their own EUROCRYPT 2019 precursor that first pushed collisions toward chosen prefixes [@leurent-peyrin-cp-eurocrypt19].

Note: SHAttered (2017) was an identical-prefix collision: both files began the same way. Shambles (2020) was a chosen-prefix collision: the two inputs started with different, meaningful content, which is what enables impersonation and forged certificates. Different authors, different years, strictly different strength. Conflating them is one of the most common errors in write-ups of SHA-1's demise.

"All attacks that are practical on MD5 are now also practical on SHA-1." That is the Shambles team's own summary of what a chosen-prefix collision means: SHA-1 has caught down to MD5 [@shambles]. timeline title Two failure arcs, MD5 and SHA-1 1996 : Dobbertin cracks MD4, dents the MD5 compression function 2004 : Wang team announces practical full MD5 collisions 2005 : Wang, Yin, Yu attack full SHA-1 near 2 to the 69 2007 : Stevens builds chosen-prefix MD5 collisions 2008 : Rogue certificate authority forged near 2 to the 49 2012 : Flame forges a Microsoft-chained code-signing certificate 2017 : SHAttered, first identical-prefix SHA-1 collision 2020 : Shambles, first chosen-prefix SHA-1 collision

The attack costs are worth tabulating, because they show how a break moves from theory to a weekend on rented hardware.

Break	Year	Attack type	Approx. cost	Real-world impact
Rogue CA	2008	Chosen-prefix MD5	$2^{49}$	Forged intermediate CA trusted by browsers [@rogue-ca]
Flame	2012	Chosen-prefix MD5	new variant	Forged Microsoft-chained code-signing cert [@stevens-counter-cryptanalysis-crypto13]
SHAttered	2017	Identical-prefix SHA-1	$2^{63.1}$	Two PDFs, one SHA-1 digest [@shattered]
Shambles	2020	Chosen-prefix SHA-1	$2^{63.4}$	Forged PGP/GnuPG key certification [@shambles]

There is a discipline the headlines miss, and it protects you from over-correcting. "MD5 and SHA-1 are dead" is true only for uses that depend on collision resistance.

Every break above attacks collision resistance. None of them touches HMAC-SHA-1 or HMAC-MD5, because HMAC does not rely on collision resistance at all, a point RFC 2104 made in 1997 and Section 9 unpacks [@rfc-2104]. So the correct scoping is narrow: MD5 and SHA-1 are dead where an adversary can influence the hashed input and profit from a collision, above all in signatures and certificates. They are not magically extinct. Old certificate chains, legacy protocols, and Git's historical use of SHA-1 all persist, which is why NIST set a firm calendar date, the end of 2030, to withdraw SHA-1 from all U.S. government use rather than pretending it vanished overnight [@nist-retires-sha1]. The recurring lesson of both arcs: deprecating an algorithm in new code is not the same as revoking it from live trust anchors.

Every hash in this catalog shared one design shape, the Merkle-Damgard chain of Section 3. The obvious question, and the one the field actually asked, was whether anything built the same way survived. One did, and understanding why is the turn from despair to design.

6. The Survivor: SHA-2, and Why It Did Not Fall

Here is a paradox worth sitting with. SHA-2 is built the same way as SHA-1: the same Merkle-Damgard skeleton, the same add-rotate-xor compression style, designed by the same institution. By the logic of the last two sections it should have fallen too. Instead, in 2026, SHA-256 is the hash under most of the internet. What did the survivors do differently?

The family, standardized in FIPS 180-4, has six members: SHA-224, SHA-256, SHA-384, and SHA-512, plus the truncated SHA-512/224 and SHA-512/256 [@fips-180-4]. And the answer to the paradox is almost anticlimactic. SHA-2 did not adopt a new idea. It widened the margins of the old one.

Larger chaining state, a longer and more diffusing message schedule, more rounds, and distinct per-round constants together mean that the differential paths Wang's team rode through MD5 and SHA-1 do not propagate far enough to reach the full function. The evidence is two decades of cryptanalysis that keeps stalling short: the best published collision attacks reach only 31 of SHA-256's 64 rounds in practice [@mendel-nad-schlaffer-eurocrypt13], and 39 rounds in the weaker semi-free-start setting [@li-liu-wang-sha2-2024]. A 2024 effort using programmatic SAT solvers likewise reaches only reduced-round or modified-initial-value variants, never the full function [@sha256-sat-2024]. That wide margin, 33 rounds to the deepest true collision and 25 even against the weaker semi-free-start attack, is the entire reason to trust it.

There is one wrinkle the designers half-anticipated, and it is the single most useful discriminator you can memorize. Full-width SHA-256 and SHA-512 are length-extension-vulnerable, exactly as Section 4 warned, because their digest is the whole final state. Most of the truncated members are not. But one truncated member still is, and it is the one people most often get wrong.

A hash whose internal chaining state is wider than its emitted output, so the digest reveals only part of the final state. SHA-384 and SHA-512/256 run the full SHA-512 engine, a 512-bit state, but output only 384 or 256 bits. Because an attacker never sees the whole state, length extension has nothing to resume from, provided enough of the state stays hidden.

The catch is in that last clause. Truncation defeats length extension only when the hidden part of the state is too large to guess. SHA-384 hides 128 bits and SHA-512/256 hides 256; recovering either by brute force is infeasible. SHA-224, however, is SHA-256 with a different initial value and the 256-bit result cut to 224 bits, so it discards only one 32-bit word of state [@rfc-3874].

An attacker brute-forces that missing 32 bits in about $2^{32}$ compression calls, seconds on commodity hardware, recovers the full internal state, and then length-extends exactly as if nothing were hidden. SHA-224 is therefore length-extension-vulnerable with only a 32-bit margin, categorically different from the truncated variants that hide 128 bits or more.

Note: The move that saves SHA-384 and SHA-512/256 is almost embarrassingly simple: hide part of the final state. But the amount hidden is the whole game. Hide 128 or 256 bits and length extension has nothing feasible to resume from. Hide only 32 bits, as SHA-224 does, and an attacker recovers the state in about $2^{32}$ work and extends normally. "Truncated" is not a synonym for "safe"; "truncated by enough" is. The sponge, next section, takes this instinct to its logical end by never exposing the state at all.

The table below is the sharpest single reference in this guide. When someone asks whether a hash is safe to use as H(secret || message), this is the answer.

Length-extension vulnerable	Length-extension immune
MD5, SHA-1	SHA-384
SHA-256, SHA-512 (full-width)	SHA-512/224, SHA-512/256
SHA-224 (only $2^{32}$ margin)	SHA-3 family and SHAKE
	HMAC (any hash), BLAKE2, BLAKE3, KangarooTwelve

SHA-NI, the x86 hash instructions Intel and AMD ship, accelerate SHA-1 and SHA-256 but not SHA-512 [@intel-sha-extensions]. That inverts an old piece of folklore. For years SHA-512 was "the fast one" on 64-bit CPUs because it processes 64-bit words; on a 2023 AMD Zen 4, SHA-256 now runs at about 2.03 cycles per byte while SHA-512 trails at about 4.19, because only SHA-256 gets the hardware boost [@ebash-zen4]. On CPUs without SHA-NI, the old ordering returns and SHA-512 and SHA-512/256 pull ahead again.

That hardware acceleration, combined with universal library support and FIPS validation, is why SHA-256 is the correct default almost everywhere its deployment footprint shows: TLS 1.3 computes its handshake transcript hash with the negotiated SHA-256 or SHA-384 [@rfc-8446], Bitcoin hashes every block header with double SHA-256 [@bitcoin-block-hashing], and Git's next-generation object format is built on SHA-256 [@git-sha256-transition], alongside code-signing digests and certificate fingerprints. It earned that position not by being newest but by being unbroken and fast where it counts.

Keep one framing straight, because the whole next act depends on it. SHA-2 is a survivor of the supersession arc that ran MD5 to SHA-1 to SHA-2. It is not something a later hash replaced. But a healthy survivor carrying most of the world's traffic is still a monoculture, and the field had just learned, viscerally, what happens when everything rests on one design. So even though SHA-2 was fine, cryptographers deliberately built something that shared none of its DNA.

7. The Different Shape: The Sponge and SHA-3

Imagine a hash with no exposed chaining state to steal, that hands you arbitrary-length output for free, and that shares no mathematical DNA with SHA-2. That is not a patch on Merkle-Damgard. It is a different machine.

NIST went looking for that machine deliberately. In November 2007, stung by the 2004 and 2005 collision attacks, it opened a public SHA-3 competition, explicitly asking for a hash built on a construction unlike SHA-2 so that a future break of one would not endanger the other [@nist-sha3-project]. Five finalists emerged from dozens of submissions: BLAKE, Grostl, JH, Keccak, and Skein. In October 2012, NIST selected Keccak, designed by Guido Bertoni, Joan Daemen, Michael Peeters, and Gilles Van Assche, and standardized it as FIPS 202 in August 2015 [@nist-sha3-project, @fips-202]. Its engine is the sponge.

A hash built around a single fixed permutation applied to a state of $b$ bits, split into a rate of $r$ bits and a capacity of $c$ bits, with $b = r + c$. In the absorb phase, message blocks are XORed into the rate portion and the whole state is stirred by the permutation. In the squeeze phase, output bytes are read from the rate, with the permutation applied between reads. The capacity is never read from or written to directly. SHA-3 uses the Keccak permutation on a 1600-bit state over 24 rounds.

The capacity is the whole trick. Because those $c$ bits never touch the input or the output, an attacker who sees the digest learns nothing about them, and cannot reconstruct the internal state.

flowchart LR M["Message blocks"] --> ABS["Absorb: XOR into rate, then permute"] ABS --> STATE["State: rate lane plus capacity lane"] STATE --> SQ["Squeeze: read the rate, permute, repeat"] SQ --> OUT["Output: fixed digest or an XOF stream"] CAP["Capacity lane, never read or written directly"] -.-> STATE

Two payoffs fall out of that single design decision. First, the sponge is length-extension-immune by construction, not by truncation: there is no exposed final state to resume from, so H(secret || message) is not forgeable the way it is on full-width SHA-2. Second, the squeeze phase can simply keep going, so the sponge is natively an extendable-output function.

A hash-like primitive that emits output of any length you ask for from a single input, rather than a fixed-size digest. You request as many bytes as you need and keep squeezing. SHAKE128 and SHAKE256, standardized alongside SHA-3, are the workhorse examples, and they are what modern post-quantum schemes call to expand seeds into keys.

The security is not folklore. Bertoni, Daemen, Peeters, and Van Assche proved the sponge is indifferentiable from a random oracle up to the capacity bound, which is the strongest positive statement you can make about a concrete construction [@bertoni-sponge-indiff-eurocrypt08, @keccak-sponge-duplex]. In two decades of analysis, practical collisions have reached only about 5 of Keccak's 24 rounds, leaving an enormous margin [@keccak-third-party].

Now the load-bearing correction this whole guide exists to make. SHA-3 is not the successor to SHA-2 the way SHA-2 succeeded SHA-1. FIPS 202 chooses its words carefully.

FIPS 202 states that the four SHA-3 functions "supplement the hash functions ... in FIPS 180-4: SHA-1 and the SHA-2 family," and that together the two standards "provide resilience against future advances" because they "rely on fundamentally different design principles" [@fips-202].

The word is supplement, not supersede, and NIST is even blunter elsewhere: its Policy on Hash Functions says there is "no need to transition applications from SHA-2 to SHA-3" [@nist-policy-hash-functions], and the superseded 2012 version of that policy put it more strongly still, with "no need or plan to transition" [@nist-policy-hash-functions-2012-archived].

Key idea: SHA-3 is not the replacement for SHA-2. It is the insurance policy. After watching one construction, Merkle-Damgard, carry MD5, SHA-1, and SHA-2 all at once, the field deliberately funded a second, unrelated construction so that the internet's trust machinery would never again rest on a single design. SHA-2 and SHA-3 coexist on purpose. That is diversification, not supersession, and it is a different axis from the MD5-to-SHA-1-to-SHA-2 ladder entirely.

Ethereum uses "Keccak-256," which is the original competition-era Keccak with its pre-standard padding, not the FIPS 202 SHA3-256 that NIST later finalized with an extra domain-separation suffix. The two give different digests for the same input, a frequent source of confusion for developers who assume "Keccak" and "SHA-3" are interchangeable. Solidity's built-in is spelled keccak256 precisely to signal the original variant [@solidity-globals].

The sponge bought genuine algorithmic diversity and length-extension immunity for free. What it did not buy, at least in plain software, was raw speed: on a Zen 4 core without special instructions, SHA3-256 runs at about 6.8 cycles per byte, roughly three times slower than hardware-accelerated SHA-256 [@ebash-zen4]. That gap set the agenda for the last decade. Could you keep the security and the length-extension immunity and still go an order of magnitude faster? Two answers arrived, from two different shapes.

8. The Fast Hashes: BLAKE3, the Tree, and Where K12 Sits

The command b3sum can hash a large file roughly ten times faster than sha256sum, and it does not cheat to get there [@blake3-repo]. It uses every CPU core and every SIMD lane at once. Understanding how it does that, and when you should let it, means separating two ideas people constantly conflate.

There are two axes here, and they are independent. The construction axis says how a hash is built: Merkle-Damgard (SHA-2), sponge (SHA-3, SHAKE, and, importantly, KangarooTwelve), and the BLAKE tree family derived from the ChaCha stream cipher. The use-case axis says what job you are hiring it for, and "modern high-throughput hash" is a job, currently filled by BLAKE3 and KangarooTwelve. Notice that those two fast hashes sit on different construction axes: one is a tree over an ARX core, the other is a sponge. Speed is not a construction.

The BLAKE line began as a SHA-3 finalist, a ChaCha-derived design [@hash-function-blake-book]. It became practical with BLAKE2, specified in RFC 7693, which is directly keyable: feed it a key and it is a message authentication code with no HMAC wrapper needed [@rfc-7693]. Then in 2020, Jack O'Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O'Hearn released BLAKE3, which took the decisive structural step: it arranges a round-reduced BLAKE2-style compression function inside a binary Merkle tree [@blake3-repo].

flowchart TD ROOT["Root chaining value, extensible to an XOF"] --> P0["Parent node"] ROOT --> P1["Parent node"] P0 --> L0["Chunk 0, 1 KiB"] P0 --> L1["Chunk 1, 1 KiB"] P1 --> L2["Chunk 2, 1 KiB"] P1 --> L3["Chunk 3, 1 KiB"]

The tree is what unlocks the speed. Because each 1 KiB chunk hashes independently, any number of SIMD lanes and threads can work in parallel, and the results combine afterward. The same structure gives BLAKE3 incremental updates and verified streaming, and it collapses five roles into one function: plain hash, keyed MAC, pseudorandom function, key-derivation function via its derive_key mode, and extendable output. On a Zen 4 core it runs at about 0.60 cycles per byte single-threaded, the fastest secure hash in the eBASH benchmarks, before you even add threads [@ebash-zen4].

BLAKE3's own documentation states it plainly: it is "secure against length extension, unlike SHA-2" [@blake3-repo].

The 1 KiB chunk size is not arbitrary. Because every chunk is an independent leaf, a verifier holding the root can check that one specific 1 KiB slice of a huge file is authentic by recomputing only the $O(\log n)$ nodes along its path, never re-hashing the whole file. That is the basis of verified streaming: download a movie and verify each piece as it arrives, aborting instantly if any block is corrupted or tampered [@blake3-repo].

Note: BLAKE3 is engineered to be fast, which makes it exactly wrong for hashing passwords, where you want to be deliberately slow so an attacker cannot try billions of guesses per second. Its own README says as much and points you to Argon2 [@blake3-repo]. Use a purpose-built password KDF, Argon2id for new systems, or scrypt, bcrypt, or PBKDF2, never a bare fast hash. Section 11 returns to this.

The other fast hash keeps you on the sponge. KangarooTwelve, with its TurboSHAKE core, is a tree hash over a round-reduced Keccak permutation, Keccak-p with 12 rounds instead of 24, published as the Informational RFC 9861 in October 2025 [@rfc-9861, @k12-acns18]. It is genuinely fast: the vendor reports about 0.51 cycles per byte using AVX-512 and 0.75 on an Apple M1, though a generic build measures closer to 3.5 cycles per byte on Zen 4 [@keccak-kangarootwelve, @ebash-zen4]. The point to hold onto, and the reason it gets a signpost rather than a section, is that KangarooTwelve is not a third construction family. It is the sponge again, tuned for throughput. It is standardized but still niche, and, like BLAKE3, not FIPS-approved.

On a machine with both tools installed, `time b3sum bigfile.iso` against `time sha256sum bigfile.iso` on a multi-gigabyte file shows the order-of-magnitude gap the BLAKE3 documentation advertises, most of it from multithreading. If you only have OpenSSL, `openssl dgst -sha3-256 bigfile.iso` will show you the software sponge penalty on hardware without SHA-3 instructions.

Everything else is a one-line signpost. RIPEMD-160, Whirlpool, China's SM3, and Russia's Streebog are real and occasionally mandated regionally, but they are not the default anywhere the reader is likely to design for. The losing SHA-3 finalists, Grostl, JH, and Skein, are well-analyzed but standardized nowhere. None of them changes the decision you are about to make.

Fast, parallel, and diverse, yes. But so far we have only hashed raw data. Real protocols almost never use a bare hash. They key it, they separate its domains, and they stretch its output, and that is precisely where most real bugs live.

9. Beyond the Bare Hash: Keyed, Domain-Separated, and XOF Modes

The most common hash bug in production is not a broken algorithm. It is a correct algorithm used in the wrong mode, and the specific mistake, again and again, is pressing H(secret || message) into service as a message authentication code. Section 4 showed why that is forgeable on full-width SHA-2. Here is what to use instead, and why it holds.

The canonical fix is HMAC, from Mihir Bellare, Ran Canetti, and Hugo Krawczyk in 1996 and RFC 2104 in 1997 [@bellare-canetti-krawczyk-crypto96, @rfc-2104].

A message authentication code built from any hash $H$ and a secret key $K$ by nesting two hash calls: $$\mathrm{HMAC}(K, m) = H\big((K \oplus opad) \,\|\, H((K \oplus ipad) \,\|\, m)\big)$$ where $opad$ and $ipad$ are fixed padding constants. Bellare, Canetti, and Krawczyk proved it is a secure pseudorandom function under assumptions weaker than full collision resistance of $H$.

Two things make HMAC the right tool. First, the outer hash of an inner hash structurally defeats length extension: the attacker never sees a resumable state. Second, and this is the payoff of the scoping discipline from Section 5, HMAC does not depend on the collision resistance of its hash. RFC 2104 states directly that the known collision weaknesses of MD5 "do not compromise the use of MD5 within HMAC" [@rfc-2104]. That is why HMAC-SHA-1 and HMAC-MD5 remain unbroken even though bare SHA-1 and MD5 are dead for signatures.

The keys HMAC needs have to come from somewhere with real entropy, which is the subject of Part 2 of this series, the field guide to cryptographic randomness.

HMAC is also the foundation of the most widely deployed key-derivation function, HKDF, which follows an extract-then-expand pattern: an extract step distills a uniform key from messy input keying material, and an expand step stretches it to any needed length [@rfc-5869]. HKDF is worth naming precisely because it is a mode built on a hash by way of HMAC, not a hash itself; when you need to turn a shared secret into keys, reach for it rather than hashing by hand.

Note: If you need to authenticate a message with a key, reach for HMAC-SHA-256 or KMAC. Do not invent H(key || message), H(message || key), or any variation: the first is length-extension-forgeable on full-width SHA-2, and the others have their own pitfalls that decades of analysis went into avoiding. The whole point of HMAC and KMAC is that someone already did the hard work and proved it correct.

The sponge offers a native alternative that needs no nesting, because it already resists length extension. NIST SP 800-185 defines four such functions: cSHAKE, KMAC, TupleHash, and ParallelHash [@sp-800-185]. KMAC is a keyed MAC that simply absorbs the key into the sponge. The other three are about domain separation.

Deliberately making the hash of the same bytes come out different depending on the context in which they appear, by mixing in a distinct label or customization string per context. It stops an input crafted for one purpose from being replayed as a valid input to another. Consider hashing a pair of fields, a username and a message. Concatenating them and hashing, `H(user || message)`, is ambiguous: the pair (`alice`, `bob-says-hi`) and the pair (`aliceb`, `ob-says-hi`) serialize to the same bytes and therefore the same digest. An attacker who controls where the boundary falls can forge equivalences. TupleHash removes the ambiguity by length-encoding each element of the sequence, so the structure, not just the bytes, is bound into the digest. cSHAKE does the same across protocol contexts: a customization string like the ASCII label for "email-signature" versus "file-checksum" guarantees the two contexts can never produce the same output for the same input. Any time distinct inputs could serialize to identical byte streams, you need this, and rolling it yourself is where subtle bugs breed [@sp-800-185].

XOFs change how you get short outputs, too. If you need a 20-byte value at a 128-bit security level, do not hash and chop arbitrary bytes off SHA-256; ask SHAKE128 or a BLAKE3 XOF for exactly 20 bytes. The security level is governed by the capacity, not by how many bytes you happen to read, so requesting the exact length is both cleaner and safer than ad hoc truncation. NIST is even revising SP 800-185 to add streaming specifications for SHAKE, a sign of how central XOFs have become [@nist-fips202-sp800185-update].

Finally, the oldest mode of all scales integrity to enormous data: the Merkle tree from Ralph Merkle's 1979 thesis [@merkle-thesis-1979]. Hash the leaves, hash pairs of hashes upward to a single root, and you can prove any one leaf belongs to the root with a path of only $O(\log n)$ hashes. Git object trees, blockchain state commitments, Certificate Transparency logs, and BLAKE3's verified streaming are all this idea.

The one rule that separates a correct Merkle tree from a broken one is to domain-separate leaf nodes from internal nodes, so an attacker cannot pass an internal node off as a leaf, exactly the flag-bit discipline BLAKE3 bakes in. The sponge and BLAKE families also power keyed authenticated-encryption modes, which belong to the block-cipher and nonce-reuse parts of this series and are out of scope here.

We now have functions and modes that keep every clause of the promise. But every one of them still bows to a few hard mathematical ceilings, and to one machine that does not yet exist. How much should you actually worry?

10. Theoretical Limits: Bounds, the Random Oracle, and Quantum

There is a best-possible hash, and you cannot build it. Understanding exactly why is what separates fear from calibrated caution, especially about quantum computers.

Start with the ceilings. For an ideal $n$-bit hash, collision resistance cannot exceed about $2^{n/2}$ work and preimage resistance cannot exceed about $2^{n}$. Those are not merely the best known attacks; they are the best any hash of that output length can offer, because the birthday and search arguments apply to a perfect random function. The good news, easy to miss, is that the deployed survivors meet those bounds exactly. The best published attack on full SHA-256, SHA-512, SHA3-256, BLAKE2, and BLAKE3 is the generic one, so there is no gap between what an attacker can do and the theoretical floor.

When cryptographers talk about a function's "security margin," they mean something narrower and attack-specific: the unbroken rounds between the deepest cryptanalysis of a given kind and the full round count. For SHA-256 collisions, published attacks reach 31 of 64 rounds (39 in the weaker semi-free-start setting), leaving a wide margin [@mendel-nad-schlaffer-eurocrypt13, @li-liu-wang-sha2-2024]; for Keccak collisions, only about 5 of 24 rounds fall [@keccak-third-party]. Wide margins are why these functions are trusted, not a gap in their strength.

The one clause where an ideal is genuinely out of reach is the random oracle. A fixed, finite, publicly specified function cannot actually behave like a fresh random answer for every input, for the plain reason that its code is right there to analyze. Ran Canetti, Oded Goldreich, and Shai Halevi made this precise: there exist signature and encryption schemes provably secure in the random-oracle model for which every concrete instantiation of the oracle by a real hash yields an insecure scheme [@cgh-jacm-2004, @cgh-stoc-1998]. So the random-oracle model is a proof heuristic, not a guarantee.

The reachable substitute is indifferentiability: the sponge is provably indifferentiable from a random oracle up to its capacity bound, which is the strongest honest claim a real construction can make [@bertoni-sponge-indiff-eurocrypt08]. And a related limit from Section 4 still bites: Antoine Joux's multicollisions mean concatenating two hashes does not multiply their strength, so there is no cheap "hash twice" route to a stronger oracle [@joux-crypto04].

Now quantum, where the folklore is loudest and least accurate. Grover's algorithm gives a square-root speedup for search, so a quantum preimage on an $n$-bit hash costs about $2^{n/2}$ instead of $2^n$ [@grover-1996]. That halves the preimage exponent, which sounds alarming until you plug in numbers: SHA-256 keeps about 128 bits of preimage resistance against an ideal quantum adversary, still far beyond reach.

Collisions are the subtler story. Brassard, Hoyer, and Tapp gave a quantum collision algorithm at about $2^{n/3}$ queries, but it demands a comparable amount of quantum-accessible memory [@brassard-hoyer-tapp-latin98]. Daniel Bernstein's cost analysis shows that once you charge honestly for that memory, plain parallel classical collision search at $2^{n/2}$ with cheap hardware wins, so the practical quantum speedup for collisions is marginal [@bernstein-sharcs09]. SHA-256 therefore keeps roughly 128-bit collision resistance in practice, quantum or not.

Function	Classical preimage	Classical collision	Quantum preimage (Grover)	Quantum collision (practical)
SHA-256	$2^{256}$	$2^{128}$	$2^{128}$	about $2^{128}$
SHA-384	$2^{384}$	$2^{192}$	$2^{192}$	about $2^{192}$
SHA-512	$2^{512}$	$2^{256}$	$2^{256}$	about $2^{256}$

The claim "quantum halves every security level" is right for preimage and wrong for collisions. Halving the collision exponent from $2^{n/2}$ to $2^{n/3}$ would matter only if the required quantum memory were free, and it is not. Charge for it, as Bernstein does, and classical machines are more cost-effective at finding collisions, so the honest planning figure for SHA-256 collision resistance stays near 128 bits [@bernstein-sharcs09].

Note: If you want more headroom against a future quantum adversary, you do not need a different construction. You need more output bits. That is the entire reason NIST keeps SHA-384, SHA-512, and the 256-bit-security XOF SHAKE256 in the toolbox: they push both preimage and collision margins up by widening the digest. KT256, defined in the Informational IRTF RFC 9861 rather than a FIPS standard, offers the same raised margin outside the FIPS boundary [@rfc-9861]. The migration story for the rest of cryptography, where the fix really is new algorithms, is covered separately in the post-quantum migration guide.

Two frontiers sit just outside this guide's scope but deserve a pointer, because they are where hashes are heading. The first is arithmetization-oriented hashes such as Poseidon [@poseidon-usenix-sec21] and the Rescue and Vision family [@rescue-tosc-2020], designed to minimize arithmetic-circuit cost inside zero-knowledge proof systems rather than CPU cycles. They are younger, with far less mature cryptanalysis than SHA-2 or SHA-3, so they are a higher-risk choice outside their niche.

The second runs the other direction: hash-based signatures such as LMS [@rfc-8554], XMSS [@rfc-8391], and the standardized stateless SLH-DSA [@fips-205] build post-quantum signatures purely from hash calls, so they consume this primitive rather than being one. Both belong to the post-quantum posts in this series, not here.

So the mathematics delivers a calm verdict. The survivors are as strong as any hash of their output length can be, and quantum computers sharpen the analysis rather than overturning it. That means the last question is not "which hash is strongest," because among the survivors none is meaningfully stronger than another. The question is which function, in which mode, for your specific job.

11. The Practical Guide: Choose X with These Params in Case Y

Everything so far collapses into one habit. Name the clause your protocol actually leans on, then pick the function and mode that keeps it. Here is that decision on a single page.

flowchart TD START["What is the job?"] --> Q0{"Passwords or low-entropy secrets?"} Q0 -->|Yes| ARGON["Argon2id, or scrypt, bcrypt, PBKDF2"] Q0 -->|No| Q1{"Keyed authentication?"} Q1 -->|Yes| HMAC["HMAC-SHA-256, or KMAC for the sponge"] Q1 -->|No| Q2{"FIPS approval required?"} Q2 -->|No| BLAKE["BLAKE3 for throughput, streaming, and KDF"] Q2 -->|Yes| Q3{"Length-extension safety needed?"} Q3 -->|Yes| TRUNC["SHA-512/256, or SHA-3 and SHAKE"] Q3 -->|No| SHA256["SHA-256, the default"]

The head-to-head matrix fills in the details behind those leaves. Speeds are single-thread cycles per byte on a 2023 AMD Zen 4, from the eBASH measurements [@ebash-zen4]; every full function listed is unbroken, so the choice is driven by length-extension behavior, speed, FIPS status, and parallelism, not by any strength gap.

Function	Construction	Output / security	LE-safe	Speed (cyc/B)	FIPS	Parallel	Best for
SHA-256	Merkle-Damgard	256 / 128 coll	No	2.03	Yes	No	Default integrity, signatures, interop
SHA-512	Merkle-Damgard	512 / 256 coll	No	4.19	Yes	No	High margin on 64-bit software
SHA-512/256	Merkle-Damgard (trunc)	256 / 128 coll	Yes	about 4.2	Yes	No	LE-immunity plus FIPS in software
SHA3-256	Sponge	256 / 128 coll	Yes	6.81	Yes	No	Construction diversity, LE-immune
SHAKE128	Sponge XOF	any / 128 ceiling	Yes	5.43	Yes	No	Variable-length output, PQC expansion
BLAKE2b	ARX (HAIFA)	512 / 256 coll	Yes	3.54	No	Tree mode	Fast keyed software hash
BLAKE3	ARX Merkle tree	256, XOF / 128	Yes	0.60	No	Yes	High-throughput integrity, KDF, streaming
KT128	Sponge tree	any / 128	Yes	about 3.5 (0.51 with AVX-512)	No	Yes	High throughput on the Keccak basis
HMAC-SHA-256	Nested keyed MD	MAC / PRF	Yes	about 2x SHA-256	Yes	No	Keyed authentication
KMAC256	Keyed sponge	any / 256	Yes	about SHAKE256	Yes	No	Keyed plus domain-separated auth

And the same advice as a quick "use / never" card, the version you tape above your desk.

Use this	For this job	Never use here
SHA-256	Default integrity, signatures, interop	MD5, SHA-1
SHA-512/256	Length-extension immunity plus FIPS in software	full-width `H(secret\|\|msg)`
SHA3-256 or SHAKE256	Construction diversity, XOF, PQC seed expansion	naive truncation of a fixed hash
BLAKE3	High-throughput, streaming, KDF via `derive_key` (non-FIPS)	passwords
HMAC-SHA-256 or KMAC	Keyed authentication	`H(key\|\|msg)`
Argon2id	Passwords, low-entropy secrets	any bare fast hash
MD5, CRC	Non-security checksums only	signatures, certs, deduplication

A few sizing rules turn those tables into correct code. For a target security level of $s$ bits against collisions, make the digest at least $2s$ bits, because collision resistance is only half the output length; 128-bit collision security therefore needs a 256-bit hash.

When you need a non-standard output length, request it from an XOF (SHAKE, KMAC, or a BLAKE3 XOF) rather than truncating a fixed digest by hand, and remember that reading more XOF bytes never raises you above the capacity ceiling. Add margin, SHA-384, SHA-512, SHAKE256, or KT256, when you want a quantum hedge, per Section 10. And keep the deprecation discipline precise: MD5 and CRC are fine as non-security checksums against accidental corruption, and nowhere an adversary can influence the input.

On Windows the same primitives appear under specific names. The CNG layer exposes hash and MAC providers through BCrypt, detailed in [the CNG architecture guide](/blog/cng-architecture-bcrypt-ncrypt-ksps); Authenticode computes a hash of a portable-executable file (skipping the fields that signing itself modifies) and signs that digest, which is the exact construction Flame subverted, covered in [the Authenticode and catalog-files guide](/blog/authenticode-and-catalog-files-the-crypto-foundation-under-w); and SChannel, the Windows TLS stack, hashes handshake transcripts and has spent two decades retiring SHA-1 across cipher suites, told in [the SChannel algorithm-agility story](/blog/rotating-every-cipher-schannel-and-the-twenty-year-algorithm). The algorithm choices in this guide map onto those APIs directly.

One operational detail sinks more code than any exotic attack: comparing digests or tags with an ordinary equality check. A loop that returns the moment two bytes differ leaks, through its timing, how many leading bytes matched, which is enough for an attacker to recover a secret tag byte by byte. The fix is a constant-time comparison that always scans the whole buffer. This adjacency between hashing and timing side channels is the same family of bug explored in the padding-oracle field guide.

{` // Comparing a secret tag with early-exit leaks how many bytes matched. function naiveEqual(a, b) { if (a.length !== b.length) return false; for (let i = 0; i < a.length; i++) { if (a[i] !== b[i]) return false; // returns as soon as bytes differ } return true; }

// Constant-time: always scan the whole array, OR the differences together. function constantTimeEqual(a, b) { if (a.length !== b.length) return false; let diff = 0; for (let i = 0; i < a.length; i++) { diff |= a[i] ^ b[i]; // no branch on the secret bytes } return diff === 0; }

const tag = [0x9f, 0x1c, 0x00, 0x42, 0xab]; const guess = [0x9f, 0x1c, 0xff, 0x00, 0x00]; console.log("naive:", naiveEqual(tag, guess)); console.log("constant-time:", constantTimeEqual(tag, guess)); console.log("In production call a vetted primitive: crypto.timingSafeEqual, CRYPTO_memcmp, or sodium_memcmp."); `}

Two rules cover most real disasters, and they have nothing to do with picking the "strongest" hash. Never use a bare fast hash for passwords, and never use a bare hash where you meant HMAC. The rest of choosing is about axes that are not strength at all: length-extension behavior, speed, FIPS status, parallelism, and construction diversity. What remains is to clear up the myths that cause the disasters those two rules do not.

12. Misconceptions and Real-Code Smells

The field's folklore is mostly half-right, which is worse than fully wrong, because half-right beliefs survive code review. Here are the questions that actually decide those reviews.

No, and this is the single most common misconception in the subject. FIPS 202 says its SHA-3 functions supplement the SHA-1 and SHA-2 families rather than replacing them [@fips-202], and NIST's Policy on Hash Functions states plainly that there is "no need to transition applications from SHA-2 to SHA-3" [@nist-policy-hash-functions]. SHA-2 and SHA-3 coexist deliberately, on a diversification axis, so the internet's trust machinery does not rest on one construction. The supersession ladder ran MD5 to SHA-1 to SHA-2 and ended there. Yes. Full-width SHA-256 and SHA-512 are length-extension-vulnerable, because their digest is the whole final chaining state. So is SHA-224, despite being truncated: it hides only 32 bits of state, which an attacker recovers in about $2^{32}$ work before extending normally [@rfc-3874]. The genuinely immune options are SHA-384, SHA-512/224, SHA-512/256, the SHA-3 family and SHAKE, HMAC, KMAC, and the BLAKE hashes. If you need `H(secret || message)` behavior, use HMAC or one of those instead. No. Every collision attack on SHA-1 and MD5 attacks collision resistance, and HMAC does not rely on collision resistance. RFC 2104 states directly that MD5's weaknesses do not compromise its use within HMAC [@rfc-2104]. HMAC-SHA-1 and HMAC-MD5 remain unbroken, though new systems should still prefer HMAC-SHA-256 for margin. Only as a non-security checksum guarding against accidental corruption, where no adversary can influence the input. The moment an attacker can choose inputs, MD5 collisions become forged certificates, as the 2008 rogue certificate authority and 2012 Flame both showed [@rogue-ca, @stevens-counter-cryptanalysis-crypto13]. Never use MD5 for signatures, certificates, or deduplication of adversarial data. No. Grover's algorithm halves the preimage exponent, leaving SHA-256 with about 128-bit preimage resistance, still infeasible [@grover-1996]. For collisions the quantum speedup is marginal once you charge for quantum memory, so SHA-256 keeps roughly 128-bit collision resistance in practice [@bernstein-sharcs09]. The hedge, if you want more margin, is a longer output such as SHA-384 or SHA-512, not a new construction. SHA-256 for general integrity, signatures, and interoperability, especially where FIPS validation matters [@fips-180-4]. BLAKE3 when you want maximum throughput, streaming, or a KDF and do not need FIPS [@blake3-repo]. SHA-512/256 when you want length-extension immunity plus FIPS on 64-bit software. No. Both are built to be fast, which is exactly what you do not want for passwords, where speed helps the attacker guess. Use a purpose-built slow function: Argon2id for new systems, or scrypt, bcrypt, or PBKDF2. BLAKE3's own README says the same and points to Argon2 [@blake3-repo].

The misuse catalog is the FAQ's mirror image: the same mistakes, seen from the code side.

Note: H(secret || message) pressed into service as a MAC (use HMAC or KMAC). Passwords fed to a bare or unsalted fast hash (use Argon2id). Truncating a fixed digest to an odd length instead of asking an XOF (use SHAKE or a BLAKE3 XOF). Comparing tags with == or an early-exit loop instead of a constant-time check.These last two, ad hoc truncation and non-constant-time comparison, are the ones that pass functional tests perfectly and fail only under an adversary, which is why they survive so long in production. Rolling your own Merkle tree without domain-separating leaf and internal nodes. Treating a hash as if it provided encryption. And assuming a real hash is a perfect random oracle rather than an approximation of one.

Return, finally, to the two files from the opening, the ones that shared the SHA-1 digest 38762cf7f55934b34d179ae6a4c80cadccbb7f0a [@shattered]. You can now read that string precisely. It is not evidence that hashing is broken, or that longer is always safer, or that SHA-3 rode in to replace the fallen. It is a single clause, collision resistance, failing on a single aging construction, and being weaponized because signatures sign the digest, not the file. The answer the field actually gave was not one replacement but two moves at once: widen the margins of the survivor, SHA-2, and diversify into unrelated constructions, the SHA-3 sponge and the BLAKE3 tree, so no future break can take everything down together. A cryptographic hash makes one promise. Mastery is knowing exactly which clause of that promise your protocol leans on, and choosing the function and the mode that keeps it.