# The Fingerprint Two Files Shared: A Field Guide to Cryptographic Hashes > A field guide to cryptographic hashes: the one promise behind SHA-2, SHA-3, and BLAKE3, why MD5 and SHA-1 died, and how to choose one that keeps its promise. *Published: 2026-07-09* *Canonical: https://paragmali.com/blog/the-fingerprint-two-files-shared-a-field-guide-to-cryptograp* *© Parag Mali. All rights reserved.* --- A cryptographic hash makes one promise: a short fingerprint that stands in, unforgeably and unambiguously, for any input. MD5 and SHA-1 are dead for collision-dependent uses. Their broken collision resistance was weaponized into forged certificates (the 2008 rogue certificate authority [@rogue-ca], Flame in 2012 [@stevens-counter-cryptanalysis-crypto13]) and forged documents and keys (SHAttered in 2017 [@shattered], Shambles in 2020 [@shambles]). Yet HMAC-SHA-1 is not broken [@rfc-2104], and "dead" never meant "extinct." SHA-2 survived by widening the same Merkle-Damgard margins, while SHA-3's sponge and the BLAKE3 tree diversified the toolbox rather than replacing the survivor [@fips-202, @blake3-repo]. Correct usage is clause-matching: pick the function and the mode (bare hash, HMAC, KMAC or cSHAKE, an extendable-output function, or a password KDF) that keeps the exact clause your protocol leans on. This guide gives you the promise, the constructions, every named break with its root cause, and a decision tree you can apply the same afternoon. ## 1. The Fingerprint Two Files Shared In February 2017, researchers at Google and CWI Amsterdam published two PDF files that looked nothing alike, with different visible content and different bytes, yet when a computer fingerprinted each of them with SHA-1 it returned the very same 40-hex-digit answer: `38762cf7f55934b34d179ae6a4c80cadccbb7f0a` [@shattered]. Two different files, one identical fingerprint. Five years earlier, malware named Flame had used the same species of trick against the older MD5 hash to forge a code-signing certificate that chained up to Microsoft and rode Windows Update onto target machines as if Microsoft itself had signed it [@stevens-counter-cryptanalysis-crypto13]. Neither of those is a stunt. They are the failure mode of one of the quietest, most load-bearing primitives in computing. A cryptographic hash function takes an input of any size and returns a short, fixed-length string of bits, its digest or fingerprint, and it promises something enormous: that fingerprint stands in for the whole input unforgeably and unambiguously. Almost everything you trust online leans on that promise. A certificate authority does not sign your certificate; it signs a hash of it. Code signing signs a hash of the program. A TLS 1.3 handshake authenticates a hash of the transcript. Every Git commit names its parent by hash, and every blockchain block commits to its transactions through a tree of hashes. When the promise holds, all of that machinery works. When it breaks, it breaks silently and all at once. > **Key idea:** A cryptographic hash is infrastructure hiding in plain sight. Because signatures, certificates, and integrity checks all sign the digest rather than the data, a break in the hash silently breaks every signature built on top of it. The attacker never has to touch your private key. That is what makes the two colliding PDFs unsettling rather than merely clever. If an adversary can produce two inputs with the same fingerprint, then any signature over one of them is also a valid signature over the other, and the entire chain of trust resting on that hash quietly inverts. This guide answers four questions in order: what exactly is the promise a hash makes, how is the fingerprint built, how has it broken in the real world, and how do you choose one today that will not break under you. flowchart TD A["One promise: a short, unforgeable, unambiguous fingerprint"] --> B["How it is built: the Merkle-Damgard chain and the sponge"] B --> C["How it broke: MD5 and SHA-1 collisions weaponized into forgery"] C --> D["What survived: SHA-2 widened the same margins"] D --> E["What diversified: the SHA-3 sponge, the BLAKE3 tree, KangarooTwelve"] E --> F["How to choose: match the function and mode to the clause you rely on"] A preview of the plot, because it corrects the single most common misconception in the whole subject. Some hashes died and were superseded: MD5 gave way to SHA-1, which gave way to SHA-2. But SHA-2 did not then give way to SHA-3. Instead the field forked. One arc is supersession by failure; the other is deliberate diversification, where several unbroken constructions coexist on purpose so the internet's trust machinery never again rests on a single design. Telling those two arcs apart is most of what it means to understand this topic. Before we can understand how the fingerprint was forged, we have to say, precisely, what it was ever supposed to guarantee. ## 2. What a Hash Actually Promises Ask three engineers what a "secure hash" means and you will get three answers: it is one-way, it has no collisions, it looks random. Cryptographers are more disciplined. They name exactly three properties, plus one more that everyone wishes were true and no real function actually delivers. Getting these four straight is the whole vocabulary, because every break later in this guide is best described as one specific property falling while the others stand. Start with the object itself. A hash function $H$ maps an input of essentially unbounded length to an output of some fixed length $n$ bits: 256 bits for SHA-256, 160 for SHA-1, 128 for MD5. Because the input space is infinite and the output space has only $2^n$ points, collisions must exist by counting alone. Security is never about avoiding collisions in principle; it is about making them, and two related feats, computationally unreachable. Phillip Rogaway and Thomas Shrimpton gave the field its rigorous, separated definitions of these properties in 2004, and it is worth stating them the way they did, because the separations matter [@rogaway-shrimpton-fse04]. A deterministic function that maps an input of arbitrary length to a fixed-length digest, engineered so that the digest behaves like a compact, tamper-evident fingerprint of the input: cheap to compute forwards, and infeasible to invert, to steer to a chosen value, or to make two inputs share, except by effort that grows exponentially in the digest length. Given only a target digest $y$, it is computationally infeasible to find any input $m$ with $H(m) = y$. This is the "one-way" property. For an ideal $n$-bit hash the generic cost is about $2^{n}$ evaluations, because you have no better strategy than trying inputs until one lands on $y$. Given a specific input $m_1$, it is infeasible to find a different input $m_2 \ne m_1$ with $H(m_2) = H(m_1)$. The target is fixed and not of your choosing. The generic cost is again about $2^{n}$. It is infeasible to find any two distinct inputs $m_1 \ne m_2$ with $H(m_1) = H(m_2)$. Here you are free to choose both inputs, which is exactly why it is easier to attack. The generic cost is only about $2^{n/2}$, by the birthday bound. The gap between $2^{n}$ and $2^{n/2}$ is the single most important number in this article. Preimage and second-preimage pit you against a fixed target, so you are searching a space of size $2^n$. Collision lets you range freely over pairs, and pairs accumulate quadratically. In a space of $N = 2^{n}$ equally likely values, a randomly drawn set is more likely than not to contain a repeat after only about $\sqrt{N} = 2^{n/2}$ draws, because the number of candidate pairs among $k$ items grows like $k^2/2$. Formally the collision probability after $k$ draws is about $1 - e^{-k(k-1)/(2N)}$. The name comes from the party trick: in a room of just 23 people, two of them probably share a birthday, even though there are 365 candidate days, because 23 people form 253 pairs. For a 256-bit hash the same arithmetic says a collision surfaces after about $2^{128}$ digests rather than $2^{256}$, so a 256-bit hash buys you only 128 bits of collision resistance. This is why collision is the weakest joint, and, as the failure catalog will show, always the first to fall. | Clause | What the attacker must find | Generic cost | Who leans on it | First to fall? | |---|---|---|---|---| | Preimage resistance | Any input hashing to a given digest | $2^{n}$ | Password hiding, commitments, key derivation | Last | | Second-preimage resistance | A different input matching one fixed input | $2^{n}$ | Integrity of a known, published file | Middle | | Collision resistance | Any two distinct inputs that agree | $2^{n/2}$ | Signatures, certificates, deduplication | First | Second-preimage feels like it should be as easy as collision, but it is not. A collision lets you shop for any lucky pair anywhere in the space. A second preimage nails one endpoint to a value someone else chose and dares you to hit it from the other side, which is a $2^n$ problem, not a $2^{n/2}$ one. Attackers who can only find collisions therefore need to control both documents, which is precisely the constraint the forged-certificate attacks had to engineer around. The following simulator makes the gap tangible. It uses a toy 24-bit hash so both effects fit in a browser tab: watch how few inputs it takes to stumble into a collision, and how hopeless it is to hit one fixed target by search. {` // A toy 24-bit "hash": mix the input, then keep the low 24 bits (N = 2^24). const N = 1 << 24; // 16,777,216 possible digests function h(x) { let v = Math.imul(x ^ 0x9e3779b1, 2654435761) >>> 0; v ^= v >>> 15; // fold high bits down so the low bits mix v = Math.imul(v, 0x85ebca6b) >>> 0; v ^= v >>> 13; return v & (N - 1); } // Collision search: hash 1, 2, 3, ... and stop at the first repeat. const seen = new Map(); let collisionAt = null; for (let i = 1; i < 300000 && collisionAt === null; i++) { const d = h(i); if (seen.has(d)) collisionAt = i; else seen.set(d, i); } console.log("birthday bound, sqrt(N), is about", Math.round(Math.sqrt(N))); console.log("first collision appeared after only", collisionAt, "inputs"); // Preimage search: how many tries to hit ONE fixed target digest? const target = h(42), budget = 500000; let tries = 0, hit = false; for (let x = 5000000; x < 5000000 + budget; x++) { tries++; if (h(x) === target) { hit = true; break; } } console.log("preimage search used", tries, "tries and", hit ? "got lucky" : "found nothing"); console.log("average preimage cost would be about N/2 =", N / 2, "tries"); `} Collisions surface after only a couple of thousand inputs, while the fixed-target search burns through half a million tries and, on average, would need eight million. That is the $2^{n/2}$ versus $2^{n}$ split in miniature. There is a fourth property people silently assume: that a good hash behaves like a random oracle, a magic box that returns a fresh uniformly random answer for every distinct input and is otherwise consistent. It is a wonderful modeling fiction, and much of applied cryptography is proved secure by pretending it is real. But no fixed, finite, publicly specified function can actually be a random oracle, and Canetti, Goldreich, and Halevi proved the gap is not merely cosmetic: there exist schemes secure with an ideal oracle yet insecure under every concrete instantiation [@cgh-jacm-2004]. We make that precise in Section 10. For now, hold the three real clauses in mind, plus the knowledge that the fourth is a heuristic. Two consequences follow immediately and set up everything after. First, you never sign a document; you sign its digest, so a signature is only as unforgeable as the hash is collision resistant. The formal security goals for signatures and message authentication codes are the subject of Part 1 of this series, [the security-definitions field guide](/blog/secure-against-whom-the-security-definitions-every-protocol-); here we only need the consequence. Second, the weakest clause is collision resistance, sitting at $2^{n/2}$. To see how that clause snapped in practice, we first have to see how the fingerprint is assembled, because the assembly leaks in ways the three clauses alone do not reveal. ## 3. How the Fingerprint Was Built: The Merkle-Damgard Era The hash function became load-bearing infrastructure almost by accident. When Whitfield Diffie and Martin Hellman introduced public-key cryptography in 1976, their signatures could only operate on short numbers, roughly the size of the key [@diffie-hellman-1976]. But people wanted to sign whole documents, contracts, and programs. The fix was quiet and consequential: do not sign the document, sign a short fingerprint of it. That single move loaded the entire weight of a signature onto the hash. If two documents can share a fingerprint, one signature covers both, so the signature scheme is only as sound as the hash beneath it. Ralph Merkle's 1979 Stanford thesis formalized one-way hash functions and, in its "Tree Authentication" section, introduced the hash tree that now underlies Git, blockchains, and verified downloads [@merkle-thesis-1979]. The next question was how to build such a fingerprint for an input of any length out of a small, analyzable building block. In 1989, Ralph Merkle and, independently, Ivan Damgard supplied the answer that defined the next two decades [@merkle-crypto89, @damgard-crypto89]. An iterated hash built by padding the message, splitting it into fixed-size blocks, and folding the blocks one at a time through a compression function, starting from a fixed initial value called the IV. Merkle and Damgard proved that if the compression function is collision resistant and the final block encodes the message length, a trick called MD strengthening, then the whole hash inherits collision resistance from the small compression function. That theorem is the reason the 1990s could build hashes with confidence: analyze one small function, get a hash for arbitrary inputs for free. The digest is simply the chaining state after the last block has been absorbed. flowchart LR IV["IV: fixed start state"] --> C1["compress"] M1["block m1"] --> C1 C1 --> C2["compress"] M2["block m2"] --> C2 C2 --> C3["compress"] MP["final block: message plus length padding"] --> C3 C3 --> D["digest: the final chaining state"] Ronald Rivest built the first widely used family on this shape. MD4 arrived in 1990 and the strengthened MD5 in 1992, both producing 128-bit digests from an add-rotate-xor compression function [@rivest-md4-crypto90, @rfc-1320]. RFC 1321 states MD5's purpose plainly: it compresses a message "in a secure manner before being signed with a private key" [@rfc-1321]. The design DNA, a message schedule feeding an ARX round function, propagated forward. In 1993 the U.S. government standardized a 160-bit hash of the same lineage in FIPS 180, retroactively called SHA-0, and in 1995 replaced it with SHA-1, a one-bit tweak of the same design [@nist-hash-functions]. SHA-1's specification survives in the current Secure Hash Standard [@fips-180-4], and it went on to hash most of the internet for roughly twenty years. The only visible difference between SHA-0 and SHA-1 is a single one-bit rotation added to the message schedule in 1995, a small and initially unexplained change later understood to improve collision resistance. The hedge held for a while, but SHA-0 was later shown to be genuinely fragile: Xiaoyun Wang and colleagues found efficient full collisions for SHA-0 well under the birthday bound in 2005 [@wang-yu-yin-sha0-crypto05]. The quietly patched flaw foreshadowed the cryptanalysis that would soon reach SHA-1 itself. The Merkle-Damgard theorem promised something precise and limited: the whole hash is as collision resistant as its little compression function. It never promised that the shape wrapped around that function, the chaining, the padding, the exposed final state, was itself sound. That distinction is exactly where the first cracks opened, and they opened even for a hypothetically perfect compression function. ## 4. The Structural Cracks: Why Merkle-Damgard Leaks Here is the unsettling part. You can hand the Merkle-Damgard construction a perfect compression function, one with no weakness whatsoever, and the assembled hash still leaks. It leaks four separate ways, and none of them is a bug in any particular function. They are properties of the shape. The first leak is the most practical, and it follows directly from the diagram in the previous section. The digest is the final chaining state. Nothing hides it. So if you know a digest and the length of the message that produced it, you can pick up the computation exactly where it left off. Given a valid digest $H(m)$ and the length of $m$, but not $m$ itself, an attacker can compute $H(m \,\|\, \text{pad} \,\|\, s)$ for a suffix $s$ of their choosing. It works because a Merkle-Damgard digest is literally the internal chaining state after the last block, so the attacker resumes from that state and folds in more blocks, without ever knowing $m$. The mechanism has exactly three moving parts, and it needs neither the message nor any secret. Publishing $H(m)$ publishes a fully loaded, resumable hash engine. From the length of $m$ the attacker rebuilds the glue padding, the `0x80` byte, the run of zeros, and the encoded bit-length that Merkle-Damgard appends before the final block [@fips-180-4]. The attacker reloads $H(m)$ as the IV, folds in the glue and then an arbitrary suffix, and reads off a valid digest for the longer message. When the server later hashes `secret || m || glue || suffix` it walks the identical chain, passes through the intermediate state $H(m)$, folds in the same suffix, and confirms the forgery. flowchart LR PUB["Public: digest H(m) and length of m"] --> REBUILD["Rebuild glue padding from the length alone"] REBUILD --> RELOAD["Reload H(m) as the compression IV"] RELOAD --> FOLD["Fold in glue, then attacker suffix"] FOLD --> FORGE["Forged digest for m plus glue plus suffix"] SERVER["Server hashes secret plus m plus glue plus suffix"] -.walks the identical chain.-> FORGE This turns a natural-looking construction into a trap. Many engineers reach for `H(secret || message)` as a homemade message authentication code, reasoning that you cannot forge a tag without knowing the secret. Length extension refutes that: given a valid tag for one message, an attacker appends chosen data and computes a valid tag for the longer message, secret still unknown. The toy hash below is deliberately tiny and insecure, but it has the one property that matters, its digest is its whole internal state, so it forges end to end exactly as MD5 or SHA-256 would. {` // A toy Merkle-Damgard hash. Like MD5 and SHA-256, its DIGEST IS ITS FINAL // INTERNAL STATE -- that single fact is the entire length-extension bug. const BLK = 8; const IV = 0x1234abcd >>> 0; const enc = s => [...s].map(c => c.charCodeAt(0)); function compress(state, byte) { // one toy ARX step (not secure) let s = (state + byte) >>> 0; s = ((s << 7) | (s >>> 25)) >>> 0; return Math.imul(s ^ 0x9e3779b1, 2654435761) >>> 0; } function fold(state, bytes) { // raw fold, adds no padding let s = state >>> 0; for (const b of bytes) s = compress(s, b); return s >>> 0; } function pad(len) { // MD strengthening from the LENGTH ALONE const p = [0x80]; while ((len + p.length + 1) % BLK !== 0) p.push(0x00); p.push(len & 0xff); // 1-byte toy length encoding return p; } function toyHash(bytes) { // full hash = fold over message + padding return fold(IV, bytes.concat(pad(bytes.length))); } // The server publishes (message, tag) where tag = H(secret || message). const secret = enc("SECRET"); // 6 bytes the attacker never sees const message = enc("user=guest"); const tag = toyHash(secret.concat(message)); // The attacker knows tag, message, and only the LENGTH of the secret (6). const secretLen = 6; const glue = pad(secretLen + message.length); // rebuilt from lengths, no secret const suffix = enc("&role=admin"); const total = secretLen + message.length + glue.length + suffix.length; const forgedTag = fold(tag, suffix.concat(pad(total))); // resume FROM the tag const forgedMessage = message.concat(glue).concat(suffix); // The server verifies the forged message with its real secret. const check = toyHash(secret.concat(forgedMessage)); console.log("forged tag (secret never used):", forgedTag.toString(16)); console.log("server recomputation :", check.toString(16)); console.log("forgery accepted?", forgedTag === check); console.log("smuggled 'role=admin' present?", String.fromCharCode(...forgedMessage).includes("role=admin")); `} Real tools do this against real hashes. The canonical `hash_extender` demonstrates it on MD5: publish `signature = MD5(secret || "data") = 6036708eba0d11f6ef52ad44e8b74d5b`, and knowing only `data`, the algorithm, and that signature, an attacker loads the signature back into MD5's state and continues hashing a chosen suffix to forge a valid tag for `secret || data || glue || suffix`, never learning the secret. Its own summary is the whole lesson: "100% of the state needed to continue a hash is in the output of most hashing algorithms" [@hash-extender]. > **Note:** On any full-width Merkle-Damgard hash, including MD5, SHA-1, SHA-256, and SHA-512, the prefix-secret construction `H(secret || message)` is forgeable by length extension. Use HMAC (Section 9) or a length-extension-immune hash instead. This footgun is common enough that it has its own family of real exploits in serialized-signing and token systems, examined in [the serialization and canonicalization field guide](/blog/the-signature-was-valid-the-message-was-forged-a-field-guide); here we own the mechanism, that guide owns the exploitation. A precise note you should carry forward, because it is the sharpest discriminator between functions in this whole article. Length extension afflicts hashes whose digest exposes the full final state: MD5, SHA-1, and the full-width SHA-256 and SHA-512. Truncation can close the leak, but only if it hides enough of the state. The truncated SHA-2 variants that hide at least 128 bits (SHA-384, SHA-512/224, SHA-512/256), the SHA-3 family and SHAKE, HMAC, and the BLAKE hashes are all immune. The dangerous middle case is SHA-224, which discards only 32 bits and is therefore still vulnerable at about $2^{32}$ work, a distinction Section 6 makes exact. Defer the full table there; just remember that "truncated" does not automatically mean "safe." The other three leaks are subtler but drove the field toward new designs. In 2004, Antoine Joux showed that in any iterated hash, finding $2^k$ messages that all share one digest costs only about $k$ times the work of a single collision, not the vastly larger amount naive counting suggests [@joux-crypto04]. The construction is almost embarrassingly direct: from the starting state, one birthday search finds a colliding pair of blocks that both drive the chain to the same next state; repeat from that state $k$ times; then every one of the $2^k$ ways to pick one block or the other at each step traces the identical chain to the identical final digest. These multicollisions demolish a piece of folklore that had reassured a generation of engineers: that concatenating two independent hashes, $H_1(m) \,\|\, H_2(m)$, multiplies their strength. It does not. Build a large multicollision in the weaker half -- many messages that all share one digest under it, which is cheap for a Merkle-Damgard hash -- then birthday-search that set for a pair that also collides under the stronger half. The combined collision resistance is essentially that of the stronger single hash, not the sum. Hashing twice buys almost nothing. Joux's multicollisions were also the engine for two deeper results. In 2005, John Kelsey and Bruce Schneier used expandable messages built from multicollisions to find second preimages on Merkle-Damgard hashes for far less than the ideal $2^n$: for a target of $2^k$ blocks, the cost falls to roughly $2^{n-k}$ plus overhead [@kelsey-schneier-eurocrypt05]. That matters because second-preimage resistance was supposed to be the sturdy $2^n$ clause, yet the shape leaks it for long messages. In 2006, Kelsey and Tadayoshi Kohno went further with the herding, or Nostradamus, attack: an attacker precomputes a branching diamond structure, publicly commits to a digest as if it were a prediction, and later herds almost any chosen prefix into that exact digest [@kelsey-kohno-eurocrypt06]. It weaponizes the construction into proof-of-prediction fraud. > **Key idea:** Length extension, Joux multicollisions, long-message second preimages, and herding are flaws in the shape, not in the function. A stronger or longer compression function cannot fix them, because they do not depend on any weakness in that function. Only a different construction can. Hold that thought: it is exactly why, after the failures, the field did not just build a bigger Merkle-Damgard hash. It built a different machine. Those four are leaks in an idealized world, where the compression function is flawless. They were serious enough to motivate truncated designs and, eventually, entirely new constructions. But they were still theoretical. What turned this subject from a seminar into a decade of emergency migrations was the other kind of failure: the compression function itself fell, and someone chose to weaponize it. ## 5. The Failure Catalog: Every Notable Break, Named A broken hash is a lab curiosity until someone forges a certificate with it. Twice, someone did, and once it was a nation-state. This section is the heart of the guide, because the two failure arcs, MD5 and SHA-1, are the evidence for everything the rest of it recommends. Each arc runs the same course: a theoretical crack, a practical collision, a technique that makes the collision meaningful, and finally a real forged trust anchor. The MD5 arc opens with an early warning that the field under-heeded. In 1996, Hans Dobbertin found practical full collisions for MD4 and collisions in MD5's compression function under a chosen initial value, not yet the full hash but a clear structural crack [@dobbertin-md4]. MD5 stayed deployed anyway. The warning became undeniable in August 2004, when Xiaoyun Wang, Dengguo Feng, Xuejia Lai, and Hongbo Yu announced full MD5 collisions computable in minutes, alongside collisions for MD4, HAVAL-128, and RIPEMD [@wang-md5-eprint-2004-199]. The following year, Wang and Yu published the peer-reviewed differential method behind it [@wang-yu-break-md5-eurocrypt05]. That is the theoretical death of MD5. Everything after is engineering. The engineering that mattered was learning to choose the colliding inputs. A collision between two inputs that begin with two different, attacker-chosen prefixes. The attacker fixes meaningful, distinct starting content for each side, two different names in a certificate, say, and then computes collision blocks that drive both sides to the same digest. This is strictly harder than an identical-prefix collision, where the two inputs share a common start, and strictly more dangerous, because it forges a relationship between two meaningful documents rather than two random blobs. In 2007, Marc Stevens, Arjen Lenstra, and Benne de Weger built exactly that for MD5, and used it to construct two X.509 certificates with different identities but the same MD5 digest [@stevens-chosen-prefix-md5-eurocrypt07]. A year later the attack left the lab. At the 2008 Chaos Communication Congress, a seven-person team used a chosen-prefix MD5 collision, about $2^{49}$ MD5 compressions on a cluster of roughly 200 PlayStation 3 consoles, to obtain a genuine commercial certificate-authority signature over a benign certificate that simultaneously validated a forged intermediate certificate authority [@rogue-ca]. The forged certificate carried the basic-constraints CA flag set to TRUE, which meant every browser on earth would trust certificates it issued. flowchart TD P1["Benign prefix: an ordinary certificate request"] --> COL["Collision blocks appended to both sides"] P2["Malicious prefix: a rogue CA certificate"] --> COL COL --> SAME["Both certificates share one MD5 digest"] SAME --> SIG["Commercial CA signs the benign certificate"] SIG --> VALID["The same signature validates the rogue CA:TRUE certificate"] To make two certificates collide, the attacker needs somewhere to park the roughly 1632-bit collision block without it looking suspicious. In the 2008 rogue CA, that space lived inside a "Netscape Comment" extension, a field parsers ignore, which the researchers wryly described as hiding the collision like a tumor inside an otherwise healthy certificate [@rogue-ca]. The lesson generalizes: any format with an ignorable, attacker-controlled field gives a collision attack room to breathe. The arc peaked in June 2012 with Flame, espionage malware that carried a code-signing certificate chaining to Microsoft. Flame's authors mounted a new chosen-prefix MD5 collision against a Microsoft sub-authority still using MD5 in its licensing service, letting them sign malware as "Microsoft" and distribute it through Windows Update. Marc Stevens later reconstructed the previously unknown collision variant from the certificate itself [@stevens-counter-cryptanalysis-crypto13]. A nation-state had used a hash collision to subvert the update trust root of a billion machines. After Flame, MD5 in signatures was not deprecated as a matter of hygiene; it was radioactive. SHA-1 traveled the same road a decade behind. In 2005, Xiaoyun Wang, Yiqun Lisa Yin, and Hongbo Yu published the first collision attack on the full SHA-1, at about $2^{69}$ operations, comfortably under the $2^{80}$ birthday bound its 160-bit output was supposed to guarantee [@wang-yin-yu-full-sha1-crypto05]. That theoretical break is what prompted NIST to open the SHA-3 competition. The practical break took twelve more years of hardware progress. In February 2017, a team from CWI Amsterdam and Google announced SHAttered: the first practical identical-prefix SHA-1 collision, at roughly $2^{63.1}$ SHA-1 computations, about nine quintillion of them, costing the equivalent of 6,500 CPU-years plus 110 GPU-years [@shattered]. The proof was the pair of PDFs from this article's opening, sharing the digest `38762cf7f55934b34d179ae6a4c80cadccbb7f0a`. In January 2020, Gaetan Leurent and Thomas Peyrin closed the arc with "SHA-1 is a Shambles," the first practical chosen-prefix SHA-1 collision at about $2^{63.4}$, which along the way cut the identical-prefix cost to about $2^{61.2}$ [@shambles, @shambles-eprint-2020-014]. To prove it mattered, they forged a PGP/GnuPG key certification, tracked as CVE-2019-14855, building on their own EUROCRYPT 2019 precursor that first pushed collisions toward chosen prefixes [@leurent-peyrin-cp-eurocrypt19]. > **Note:** SHAttered (2017) was an identical-prefix collision: both files began the same way. Shambles (2020) was a chosen-prefix collision: the two inputs started with different, meaningful content, which is what enables impersonation and forged certificates. Different authors, different years, strictly different strength. Conflating them is one of the most common errors in write-ups of SHA-1's demise. "All attacks that are practical on MD5 are now also practical on SHA-1." That is the Shambles team's own summary of what a chosen-prefix collision means: SHA-1 has caught down to MD5 [@shambles]. timeline title Two failure arcs, MD5 and SHA-1 1996 : Dobbertin cracks MD4, dents the MD5 compression function 2004 : Wang team announces practical full MD5 collisions 2005 : Wang, Yin, Yu attack full SHA-1 near 2 to the 69 2007 : Stevens builds chosen-prefix MD5 collisions 2008 : Rogue certificate authority forged near 2 to the 49 2012 : Flame forges a Microsoft-chained code-signing certificate 2017 : SHAttered, first identical-prefix SHA-1 collision 2020 : Shambles, first chosen-prefix SHA-1 collision The attack costs are worth tabulating, because they show how a break moves from theory to a weekend on rented hardware. | Break | Year | Attack type | Approx. cost | Real-world impact | |---|---|---|---|---| | Rogue CA | 2008 | Chosen-prefix MD5 | $2^{49}$ | Forged intermediate CA trusted by browsers [@rogue-ca] | | Flame | 2012 | Chosen-prefix MD5 | new variant | Forged Microsoft-chained code-signing cert [@stevens-counter-cryptanalysis-crypto13] | | SHAttered | 2017 | Identical-prefix SHA-1 | $2^{63.1}$ | Two PDFs, one SHA-1 digest [@shattered] | | Shambles | 2020 | Chosen-prefix SHA-1 | $2^{63.4}$ | Forged PGP/GnuPG key certification [@shambles] | There is a discipline the headlines miss, and it protects you from over-correcting. "MD5 and SHA-1 are dead" is true only for uses that depend on collision resistance. Every hash in this catalog shared one design shape, the Merkle-Damgard chain of Section 3. The obvious question, and the one the field actually asked, was whether anything built the same way survived. One did, and understanding why is the turn from despair to design. ## 6. The Survivor: SHA-2, and Why It Did Not Fall Here is a paradox worth sitting with. SHA-2 is built the same way as SHA-1: the same Merkle-Damgard skeleton, the same add-rotate-xor compression style, designed by the same institution. By the logic of the last two sections it should have fallen too. Instead, in 2026, SHA-256 is the hash under most of the internet. What did the survivors do differently? The family, standardized in FIPS 180-4, has six members: SHA-224, SHA-256, SHA-384, and SHA-512, plus the truncated SHA-512/224 and SHA-512/256 [@fips-180-4]. And the answer to the paradox is almost anticlimactic. SHA-2 did not adopt a new idea. It widened the margins of the old one. Larger chaining state, a longer and more diffusing message schedule, more rounds, and distinct per-round constants together mean that the differential paths Wang's team rode through MD5 and SHA-1 do not propagate far enough to reach the full function. The evidence is two decades of cryptanalysis that keeps stalling short: the best published collision attacks reach only 31 of SHA-256's 64 rounds in practice [@mendel-nad-schlaffer-eurocrypt13], and 39 rounds in the weaker semi-free-start setting [@li-liu-wang-sha2-2024]. A 2024 effort using programmatic SAT solvers likewise reaches only reduced-round or modified-initial-value variants, never the full function [@sha256-sat-2024]. That wide margin, 33 rounds to the deepest true collision and 25 even against the weaker semi-free-start attack, is the entire reason to trust it. There is one wrinkle the designers half-anticipated, and it is the single most useful discriminator you can memorize. Full-width SHA-256 and SHA-512 are length-extension-vulnerable, exactly as Section 4 warned, because their digest is the whole final state. Most of the truncated members are not. But one truncated member still is, and it is the one people most often get wrong. A hash whose internal chaining state is wider than its emitted output, so the digest reveals only part of the final state. SHA-384 and SHA-512/256 run the full SHA-512 engine, a 512-bit state, but output only 384 or 256 bits. Because an attacker never sees the whole state, length extension has nothing to resume from, provided enough of the state stays hidden. The catch is in that last clause. Truncation defeats length extension only when the hidden part of the state is too large to guess. SHA-384 hides 128 bits and SHA-512/256 hides 256; recovering either by brute force is infeasible. SHA-224, however, is SHA-256 with a different initial value and the 256-bit result cut to 224 bits, so it discards only one 32-bit word of state [@rfc-3874]. An attacker brute-forces that missing 32 bits in about $2^{32}$ compression calls, seconds on commodity hardware, recovers the full internal state, and then length-extends exactly as if nothing were hidden. SHA-224 is therefore length-extension-vulnerable with only a 32-bit margin, categorically different from the truncated variants that hide 128 bits or more. > **Note:** The move that saves SHA-384 and SHA-512/256 is almost embarrassingly simple: hide part of the final state. But the amount hidden is the whole game. Hide 128 or 256 bits and length extension has nothing feasible to resume from. Hide only 32 bits, as SHA-224 does, and an attacker recovers the state in about $2^{32}$ work and extends normally. "Truncated" is not a synonym for "safe"; "truncated by enough" is. The sponge, next section, takes this instinct to its logical end by never exposing the state at all. The table below is the sharpest single reference in this guide. When someone asks whether a hash is safe to use as `H(secret || message)`, this is the answer. | Length-extension vulnerable | Length-extension immune | |---|---| | MD5, SHA-1 | SHA-384 | | SHA-256, SHA-512 (full-width) | SHA-512/224, SHA-512/256 | | SHA-224 (only $2^{32}$ margin) | SHA-3 family and SHAKE | | | HMAC (any hash), BLAKE2, BLAKE3, KangarooTwelve | SHA-NI, the x86 hash instructions Intel and AMD ship, accelerate SHA-1 and SHA-256 but not SHA-512 [@intel-sha-extensions]. That inverts an old piece of folklore. For years SHA-512 was "the fast one" on 64-bit CPUs because it processes 64-bit words; on a 2023 AMD Zen 4, SHA-256 now runs at about 2.03 cycles per byte while SHA-512 trails at about 4.19, because only SHA-256 gets the hardware boost [@ebash-zen4]. On CPUs without SHA-NI, the old ordering returns and SHA-512 and SHA-512/256 pull ahead again. That hardware acceleration, combined with universal library support and FIPS validation, is why SHA-256 is the correct default almost everywhere its deployment footprint shows: TLS 1.3 computes its handshake transcript hash with the negotiated SHA-256 or SHA-384 [@rfc-8446], Bitcoin hashes every block header with double SHA-256 [@bitcoin-block-hashing], and Git's next-generation object format is built on SHA-256 [@git-sha256-transition], alongside code-signing digests and certificate fingerprints. It earned that position not by being newest but by being unbroken and fast where it counts. Keep one framing straight, because the whole next act depends on it. SHA-2 is a survivor of the supersession arc that ran MD5 to SHA-1 to SHA-2. It is not something a later hash replaced. But a healthy survivor carrying most of the world's traffic is still a monoculture, and the field had just learned, viscerally, what happens when everything rests on one design. So even though SHA-2 was fine, cryptographers deliberately built something that shared none of its DNA. ## 7. The Different Shape: The Sponge and SHA-3 Imagine a hash with no exposed chaining state to steal, that hands you arbitrary-length output for free, and that shares no mathematical DNA with SHA-2. That is not a patch on Merkle-Damgard. It is a different machine. NIST went looking for that machine deliberately. In November 2007, stung by the 2004 and 2005 collision attacks, it opened a public SHA-3 competition, explicitly asking for a hash built on a construction unlike SHA-2 so that a future break of one would not endanger the other [@nist-sha3-project]. Five finalists emerged from dozens of submissions: BLAKE, Grostl, JH, Keccak, and Skein. In October 2012, NIST selected Keccak, designed by Guido Bertoni, Joan Daemen, Michael Peeters, and Gilles Van Assche, and standardized it as FIPS 202 in August 2015 [@nist-sha3-project, @fips-202]. Its engine is the sponge. A hash built around a single fixed permutation applied to a state of $b$ bits, split into a rate of $r$ bits and a capacity of $c$ bits, with $b = r + c$. In the absorb phase, message blocks are XORed into the rate portion and the whole state is stirred by the permutation. In the squeeze phase, output bytes are read from the rate, with the permutation applied between reads. The capacity is never read from or written to directly. SHA-3 uses the Keccak permutation on a 1600-bit state over 24 rounds. The capacity is the whole trick. Because those $c$ bits never touch the input or the output, an attacker who sees the digest learns nothing about them, and cannot reconstruct the internal state. flowchart LR M["Message blocks"] --> ABS["Absorb: XOR into rate, then permute"] ABS --> STATE["State: rate lane plus capacity lane"] STATE --> SQ["Squeeze: read the rate, permute, repeat"] SQ --> OUT["Output: fixed digest or an XOF stream"] CAP["Capacity lane, never read or written directly"] -.-> STATE Two payoffs fall out of that single design decision. First, the sponge is length-extension-immune by construction, not by truncation: there is no exposed final state to resume from, so `H(secret || message)` is not forgeable the way it is on full-width SHA-2. Second, the squeeze phase can simply keep going, so the sponge is natively an extendable-output function. A hash-like primitive that emits output of any length you ask for from a single input, rather than a fixed-size digest. You request as many bytes as you need and keep squeezing. SHAKE128 and SHAKE256, standardized alongside SHA-3, are the workhorse examples, and they are what modern post-quantum schemes call to expand seeds into keys. The security is not folklore. Bertoni, Daemen, Peeters, and Van Assche proved the sponge is indifferentiable from a random oracle up to the capacity bound, which is the strongest positive statement you can make about a concrete construction [@bertoni-sponge-indiff-eurocrypt08, @keccak-sponge-duplex]. In two decades of analysis, practical collisions have reached only about 5 of Keccak's 24 rounds, leaving an enormous margin [@keccak-third-party]. Now the load-bearing correction this whole guide exists to make. SHA-3 is not the successor to SHA-2 the way SHA-2 succeeded SHA-1. FIPS 202 chooses its words carefully. FIPS 202 states that the four SHA-3 functions "supplement the hash functions ... in FIPS 180-4: SHA-1 and the SHA-2 family," and that together the two standards "provide resilience against future advances" because they "rely on fundamentally different design principles" [@fips-202]. The word is supplement, not supersede, and NIST is even blunter elsewhere: its Policy on Hash Functions says there is "no need to transition applications from SHA-2 to SHA-3" [@nist-policy-hash-functions], and the superseded 2012 version of that policy put it more strongly still, with "no need or plan to transition" [@nist-policy-hash-functions-2012-archived]. > **Key idea:** SHA-3 is not the replacement for SHA-2. It is the insurance policy. After watching one construction, Merkle-Damgard, carry MD5, SHA-1, and SHA-2 all at once, the field deliberately funded a second, unrelated construction so that the internet's trust machinery would never again rest on a single design. SHA-2 and SHA-3 coexist on purpose. That is diversification, not supersession, and it is a different axis from the MD5-to-SHA-1-to-SHA-2 ladder entirely. Ethereum uses "Keccak-256," which is the original competition-era Keccak with its pre-standard padding, not the FIPS 202 SHA3-256 that NIST later finalized with an extra domain-separation suffix. The two give different digests for the same input, a frequent source of confusion for developers who assume "Keccak" and "SHA-3" are interchangeable. Solidity's built-in is spelled `keccak256` precisely to signal the original variant [@solidity-globals]. The sponge bought genuine algorithmic diversity and length-extension immunity for free. What it did not buy, at least in plain software, was raw speed: on a Zen 4 core without special instructions, SHA3-256 runs at about 6.8 cycles per byte, roughly three times slower than hardware-accelerated SHA-256 [@ebash-zen4]. That gap set the agenda for the last decade. Could you keep the security and the length-extension immunity and still go an order of magnitude faster? Two answers arrived, from two different shapes. ## 8. The Fast Hashes: BLAKE3, the Tree, and Where K12 Sits The command `b3sum` can hash a large file roughly ten times faster than `sha256sum`, and it does not cheat to get there [@blake3-repo]. It uses every CPU core and every SIMD lane at once. Understanding how it does that, and when you should let it, means separating two ideas people constantly conflate. There are two axes here, and they are independent. The construction axis says how a hash is built: Merkle-Damgard (SHA-2), sponge (SHA-3, SHAKE, and, importantly, KangarooTwelve), and the BLAKE tree family derived from the ChaCha stream cipher. The use-case axis says what job you are hiring it for, and "modern high-throughput hash" is a job, currently filled by BLAKE3 and KangarooTwelve. Notice that those two fast hashes sit on different construction axes: one is a tree over an ARX core, the other is a sponge. Speed is not a construction. The BLAKE line began as a SHA-3 finalist, a ChaCha-derived design [@hash-function-blake-book]. It became practical with BLAKE2, specified in RFC 7693, which is directly keyable: feed it a key and it is a message authentication code with no HMAC wrapper needed [@rfc-7693]. Then in 2020, Jack O'Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O'Hearn released BLAKE3, which took the decisive structural step: it arranges a round-reduced BLAKE2-style compression function inside a binary Merkle tree [@blake3-repo]. flowchart TD ROOT["Root chaining value, extensible to an XOF"] --> P0["Parent node"] ROOT --> P1["Parent node"] P0 --> L0["Chunk 0, 1 KiB"] P0 --> L1["Chunk 1, 1 KiB"] P1 --> L2["Chunk 2, 1 KiB"] P1 --> L3["Chunk 3, 1 KiB"] The tree is what unlocks the speed. Because each 1 KiB chunk hashes independently, any number of SIMD lanes and threads can work in parallel, and the results combine afterward. The same structure gives BLAKE3 incremental updates and verified streaming, and it collapses five roles into one function: plain hash, keyed MAC, pseudorandom function, key-derivation function via its `derive_key` mode, and extendable output. On a Zen 4 core it runs at about 0.60 cycles per byte single-threaded, the fastest secure hash in the eBASH benchmarks, before you even add threads [@ebash-zen4]. BLAKE3's own documentation states it plainly: it is "secure against length extension, unlike SHA-2" [@blake3-repo]. The 1 KiB chunk size is not arbitrary. Because every chunk is an independent leaf, a verifier holding the root can check that one specific 1 KiB slice of a huge file is authentic by recomputing only the $O(\log n)$ nodes along its path, never re-hashing the whole file. That is the basis of verified streaming: download a movie and verify each piece as it arrives, aborting instantly if any block is corrupted or tampered [@blake3-repo]. > **Note:** BLAKE3 is engineered to be fast, which makes it exactly wrong for hashing passwords, where you want to be deliberately slow so an attacker cannot try billions of guesses per second. Its own README says as much and points you to Argon2 [@blake3-repo]. Use a purpose-built password KDF, Argon2id for new systems, or scrypt, bcrypt, or PBKDF2, never a bare fast hash. Section 11 returns to this. The other fast hash keeps you on the sponge. KangarooTwelve, with its TurboSHAKE core, is a tree hash over a round-reduced Keccak permutation, Keccak-p with 12 rounds instead of 24, published as the Informational RFC 9861 in October 2025 [@rfc-9861, @k12-acns18]. It is genuinely fast: the vendor reports about 0.51 cycles per byte using AVX-512 and 0.75 on an Apple M1, though a generic build measures closer to 3.5 cycles per byte on Zen 4 [@keccak-kangarootwelve, @ebash-zen4]. The point to hold onto, and the reason it gets a signpost rather than a section, is that KangarooTwelve is not a third construction family. It is the sponge again, tuned for throughput. It is standardized but still niche, and, like BLAKE3, not FIPS-approved. On a machine with both tools installed, `time b3sum bigfile.iso` against `time sha256sum bigfile.iso` on a multi-gigabyte file shows the order-of-magnitude gap the BLAKE3 documentation advertises, most of it from multithreading. If you only have OpenSSL, `openssl dgst -sha3-256 bigfile.iso` will show you the software sponge penalty on hardware without SHA-3 instructions. Everything else is a one-line signpost. RIPEMD-160, Whirlpool, China's SM3, and Russia's Streebog are real and occasionally mandated regionally, but they are not the default anywhere the reader is likely to design for. The losing SHA-3 finalists, Grostl, JH, and Skein, are well-analyzed but standardized nowhere. None of them changes the decision you are about to make. Fast, parallel, and diverse, yes. But so far we have only hashed raw data. Real protocols almost never use a bare hash. They key it, they separate its domains, and they stretch its output, and that is precisely where most real bugs live. ## 9. Beyond the Bare Hash: Keyed, Domain-Separated, and XOF Modes The most common hash bug in production is not a broken algorithm. It is a correct algorithm used in the wrong mode, and the specific mistake, again and again, is pressing `H(secret || message)` into service as a message authentication code. Section 4 showed why that is forgeable on full-width SHA-2. Here is what to use instead, and why it holds. The canonical fix is HMAC, from Mihir Bellare, Ran Canetti, and Hugo Krawczyk in 1996 and RFC 2104 in 1997 [@bellare-canetti-krawczyk-crypto96, @rfc-2104]. A message authentication code built from any hash $H$ and a secret key $K$ by nesting two hash calls: $$\mathrm{HMAC}(K, m) = H\big((K \oplus opad) \,\|\, H((K \oplus ipad) \,\|\, m)\big)$$ where $opad$ and $ipad$ are fixed padding constants. Bellare, Canetti, and Krawczyk proved it is a secure pseudorandom function under assumptions weaker than full collision resistance of $H$. Two things make HMAC the right tool. First, the outer hash of an inner hash structurally defeats length extension: the attacker never sees a resumable state. Second, and this is the payoff of the scoping discipline from Section 5, HMAC does not depend on the collision resistance of its hash. RFC 2104 states directly that the known collision weaknesses of MD5 "do not compromise the use of MD5 within HMAC" [@rfc-2104]. That is why HMAC-SHA-1 and HMAC-MD5 remain unbroken even though bare SHA-1 and MD5 are dead for signatures. The keys HMAC needs have to come from somewhere with real entropy, which is the subject of Part 2 of this series, [the field guide to cryptographic randomness](/blog/predictable-or-repeated-the-only-two-ways-cryptographic-rand). HMAC is also the foundation of the most widely deployed key-derivation function, HKDF, which follows an extract-then-expand pattern: an extract step distills a uniform key from messy input keying material, and an expand step stretches it to any needed length [@rfc-5869]. HKDF is worth naming precisely because it is a mode built on a hash by way of HMAC, not a hash itself; when you need to turn a shared secret into keys, reach for it rather than hashing by hand. > **Note:** If you need to authenticate a message with a key, reach for `HMAC-SHA-256` or `KMAC`. Do not invent `H(key || message)`, `H(message || key)`, or any variation: the first is length-extension-forgeable on full-width SHA-2, and the others have their own pitfalls that decades of analysis went into avoiding. The whole point of HMAC and KMAC is that someone already did the hard work and proved it correct. The sponge offers a native alternative that needs no nesting, because it already resists length extension. NIST SP 800-185 defines four such functions: cSHAKE, KMAC, TupleHash, and ParallelHash [@sp-800-185]. KMAC is a keyed MAC that simply absorbs the key into the sponge. The other three are about domain separation. Deliberately making the hash of the same bytes come out different depending on the context in which they appear, by mixing in a distinct label or customization string per context. It stops an input crafted for one purpose from being replayed as a valid input to another. XOFs change how you get short outputs, too. If you need a 20-byte value at a 128-bit security level, do not hash and chop arbitrary bytes off SHA-256; ask SHAKE128 or a BLAKE3 XOF for exactly 20 bytes. The security level is governed by the capacity, not by how many bytes you happen to read, so requesting the exact length is both cleaner and safer than ad hoc truncation. NIST is even revising SP 800-185 to add streaming specifications for SHAKE, a sign of how central XOFs have become [@nist-fips202-sp800185-update]. Finally, the oldest mode of all scales integrity to enormous data: the Merkle tree from Ralph Merkle's 1979 thesis [@merkle-thesis-1979]. Hash the leaves, hash pairs of hashes upward to a single root, and you can prove any one leaf belongs to the root with a path of only $O(\log n)$ hashes. Git object trees, blockchain state commitments, Certificate Transparency logs, and BLAKE3's verified streaming are all this idea. The one rule that separates a correct Merkle tree from a broken one is to domain-separate leaf nodes from internal nodes, so an attacker cannot pass an internal node off as a leaf, exactly the flag-bit discipline BLAKE3 bakes in. The sponge and BLAKE families also power keyed authenticated-encryption modes, which belong to the block-cipher and nonce-reuse parts of this series and are out of scope here. We now have functions and modes that keep every clause of the promise. But every one of them still bows to a few hard mathematical ceilings, and to one machine that does not yet exist. How much should you actually worry? ## 10. Theoretical Limits: Bounds, the Random Oracle, and Quantum There is a best-possible hash, and you cannot build it. Understanding exactly why is what separates fear from calibrated caution, especially about quantum computers. Start with the ceilings. For an ideal $n$-bit hash, collision resistance cannot exceed about $2^{n/2}$ work and preimage resistance cannot exceed about $2^{n}$. Those are not merely the best known attacks; they are the best any hash of that output length can offer, because the birthday and search arguments apply to a perfect random function. The good news, easy to miss, is that the deployed survivors meet those bounds exactly. The best published attack on full SHA-256, SHA-512, SHA3-256, BLAKE2, and BLAKE3 is the generic one, so there is no gap between what an attacker can do and the theoretical floor. When cryptographers talk about a function's "security margin," they mean something narrower and attack-specific: the unbroken rounds between the deepest cryptanalysis of a given kind and the full round count. For SHA-256 collisions, published attacks reach 31 of 64 rounds (39 in the weaker semi-free-start setting), leaving a wide margin [@mendel-nad-schlaffer-eurocrypt13, @li-liu-wang-sha2-2024]; for Keccak collisions, only about 5 of 24 rounds fall [@keccak-third-party]. Wide margins are why these functions are trusted, not a gap in their strength. The one clause where an ideal is genuinely out of reach is the random oracle. A fixed, finite, publicly specified function cannot actually behave like a fresh random answer for every input, for the plain reason that its code is right there to analyze. Ran Canetti, Oded Goldreich, and Shai Halevi made this precise: there exist signature and encryption schemes provably secure in the random-oracle model for which every concrete instantiation of the oracle by a real hash yields an insecure scheme [@cgh-jacm-2004, @cgh-stoc-1998]. So the random-oracle model is a proof heuristic, not a guarantee. The reachable substitute is indifferentiability: the sponge is provably indifferentiable from a random oracle up to its capacity bound, which is the strongest honest claim a real construction can make [@bertoni-sponge-indiff-eurocrypt08]. And a related limit from Section 4 still bites: Antoine Joux's multicollisions mean concatenating two hashes does not multiply their strength, so there is no cheap "hash twice" route to a stronger oracle [@joux-crypto04]. Now quantum, where the folklore is loudest and least accurate. Grover's algorithm gives a square-root speedup for search, so a quantum preimage on an $n$-bit hash costs about $2^{n/2}$ instead of $2^n$ [@grover-1996]. That halves the preimage exponent, which sounds alarming until you plug in numbers: SHA-256 keeps about 128 bits of preimage resistance against an ideal quantum adversary, still far beyond reach. Collisions are the subtler story. Brassard, Hoyer, and Tapp gave a quantum collision algorithm at about $2^{n/3}$ queries, but it demands a comparable amount of quantum-accessible memory [@brassard-hoyer-tapp-latin98]. Daniel Bernstein's cost analysis shows that once you charge honestly for that memory, plain parallel classical collision search at $2^{n/2}$ with cheap hardware wins, so the practical quantum speedup for collisions is marginal [@bernstein-sharcs09]. SHA-256 therefore keeps roughly 128-bit collision resistance in practice, quantum or not. | Function | Classical preimage | Classical collision | Quantum preimage (Grover) | Quantum collision (practical) | |---|---|---|---|---| | SHA-256 | $2^{256}$ | $2^{128}$ | $2^{128}$ | about $2^{128}$ | | SHA-384 | $2^{384}$ | $2^{192}$ | $2^{192}$ | about $2^{192}$ | | SHA-512 | $2^{512}$ | $2^{256}$ | $2^{256}$ | about $2^{256}$ | The claim "quantum halves every security level" is right for preimage and wrong for collisions. Halving the collision exponent from $2^{n/2}$ to $2^{n/3}$ would matter only if the required quantum memory were free, and it is not. Charge for it, as Bernstein does, and classical machines are more cost-effective at finding collisions, so the honest planning figure for SHA-256 collision resistance stays near 128 bits [@bernstein-sharcs09]. > **Note:** If you want more headroom against a future quantum adversary, you do not need a different construction. You need more output bits. That is the entire reason NIST keeps SHA-384, SHA-512, and the 256-bit-security XOF SHAKE256 in the toolbox: they push both preimage and collision margins up by widening the digest. KT256, defined in the Informational IRTF RFC 9861 rather than a FIPS standard, offers the same raised margin outside the FIPS boundary [@rfc-9861]. The migration story for the rest of cryptography, where the fix really is new algorithms, is covered separately in [the post-quantum migration guide](/blog/the-thirty-year-migration-ships-in-a-pip-install-how-post-qu). Two frontiers sit just outside this guide's scope but deserve a pointer, because they are where hashes are heading. The first is arithmetization-oriented hashes such as Poseidon [@poseidon-usenix-sec21] and the Rescue and Vision family [@rescue-tosc-2020], designed to minimize arithmetic-circuit cost inside zero-knowledge proof systems rather than CPU cycles. They are younger, with far less mature cryptanalysis than SHA-2 or SHA-3, so they are a higher-risk choice outside their niche. The second runs the other direction: hash-based signatures such as LMS [@rfc-8554], XMSS [@rfc-8391], and the standardized stateless SLH-DSA [@fips-205] build post-quantum signatures purely from hash calls, so they consume this primitive rather than being one. Both belong to the post-quantum posts in this series, not here. So the mathematics delivers a calm verdict. The survivors are as strong as any hash of their output length can be, and quantum computers sharpen the analysis rather than overturning it. That means the last question is not "which hash is strongest," because among the survivors none is meaningfully stronger than another. The question is which function, in which mode, for your specific job. ## 11. The Practical Guide: Choose X with These Params in Case Y Everything so far collapses into one habit. Name the clause your protocol actually leans on, then pick the function and mode that keeps it. Here is that decision on a single page. flowchart TD START["What is the job?"] --> Q0{"Passwords or low-entropy secrets?"} Q0 -->|Yes| ARGON["Argon2id, or scrypt, bcrypt, PBKDF2"] Q0 -->|No| Q1{"Keyed authentication?"} Q1 -->|Yes| HMAC["HMAC-SHA-256, or KMAC for the sponge"] Q1 -->|No| Q2{"FIPS approval required?"} Q2 -->|No| BLAKE["BLAKE3 for throughput, streaming, and KDF"] Q2 -->|Yes| Q3{"Length-extension safety needed?"} Q3 -->|Yes| TRUNC["SHA-512/256, or SHA-3 and SHAKE"] Q3 -->|No| SHA256["SHA-256, the default"] The head-to-head matrix fills in the details behind those leaves. Speeds are single-thread cycles per byte on a 2023 AMD Zen 4, from the eBASH measurements [@ebash-zen4]; every full function listed is unbroken, so the choice is driven by length-extension behavior, speed, FIPS status, and parallelism, not by any strength gap. | Function | Construction | Output / security | LE-safe | Speed (cyc/B) | FIPS | Parallel | Best for | |---|---|---|---|---|---|---|---| | SHA-256 | Merkle-Damgard | 256 / 128 coll | No | 2.03 | Yes | No | Default integrity, signatures, interop | | SHA-512 | Merkle-Damgard | 512 / 256 coll | No | 4.19 | Yes | No | High margin on 64-bit software | | SHA-512/256 | Merkle-Damgard (trunc) | 256 / 128 coll | Yes | about 4.2 | Yes | No | LE-immunity plus FIPS in software | | SHA3-256 | Sponge | 256 / 128 coll | Yes | 6.81 | Yes | No | Construction diversity, LE-immune | | SHAKE128 | Sponge XOF | any / 128 ceiling | Yes | 5.43 | Yes | No | Variable-length output, PQC expansion | | BLAKE2b | ARX (HAIFA) | 512 / 256 coll | Yes | 3.54 | No | Tree mode | Fast keyed software hash | | BLAKE3 | ARX Merkle tree | 256, XOF / 128 | Yes | 0.60 | No | Yes | High-throughput integrity, KDF, streaming | | KT128 | Sponge tree | any / 128 | Yes | about 3.5 (0.51 with AVX-512) | No | Yes | High throughput on the Keccak basis | | HMAC-SHA-256 | Nested keyed MD | MAC / PRF | Yes | about 2x SHA-256 | Yes | No | Keyed authentication | | KMAC256 | Keyed sponge | any / 256 | Yes | about SHAKE256 | Yes | No | Keyed plus domain-separated auth | And the same advice as a quick "use / never" card, the version you tape above your desk. | Use this | For this job | Never use here | |---|---|---| | SHA-256 | Default integrity, signatures, interop | MD5, SHA-1 | | SHA-512/256 | Length-extension immunity plus FIPS in software | full-width `H(secret\|\|msg)` | | SHA3-256 or SHAKE256 | Construction diversity, XOF, PQC seed expansion | naive truncation of a fixed hash | | BLAKE3 | High-throughput, streaming, KDF via `derive_key` (non-FIPS) | passwords | | HMAC-SHA-256 or KMAC | Keyed authentication | `H(key\|\|msg)` | | Argon2id | Passwords, low-entropy secrets | any bare fast hash | | MD5, CRC | Non-security checksums only | signatures, certs, deduplication | A few sizing rules turn those tables into correct code. For a target security level of $s$ bits against collisions, make the digest at least $2s$ bits, because collision resistance is only half the output length; 128-bit collision security therefore needs a 256-bit hash. When you need a non-standard output length, request it from an XOF (SHAKE, KMAC, or a BLAKE3 XOF) rather than truncating a fixed digest by hand, and remember that reading more XOF bytes never raises you above the capacity ceiling. Add margin, SHA-384, SHA-512, SHAKE256, or KT256, when you want a quantum hedge, per Section 10. And keep the deprecation discipline precise: MD5 and CRC are fine as non-security checksums against accidental corruption, and nowhere an adversary can influence the input. One operational detail sinks more code than any exotic attack: comparing digests or tags with an ordinary equality check. A loop that returns the moment two bytes differ leaks, through its timing, how many leading bytes matched, which is enough for an attacker to recover a secret tag byte by byte. The fix is a constant-time comparison that always scans the whole buffer. This adjacency between hashing and timing side channels is the same family of bug explored in [the padding-oracle field guide](/blog/they-read-your-plaintext-without-breaking-your-cipher-a-fiel). {` // Comparing a secret tag with early-exit leaks how many bytes matched. function naiveEqual(a, b) { if (a.length !== b.length) return false; for (let i = 0; i < a.length; i++) { if (a[i] !== b[i]) return false; // returns as soon as bytes differ } return true; } // Constant-time: always scan the whole array, OR the differences together. function constantTimeEqual(a, b) { if (a.length !== b.length) return false; let diff = 0; for (let i = 0; i < a.length; i++) { diff |= a[i] ^ b[i]; // no branch on the secret bytes } return diff === 0; } const tag = [0x9f, 0x1c, 0x00, 0x42, 0xab]; const guess = [0x9f, 0x1c, 0xff, 0x00, 0x00]; console.log("naive:", naiveEqual(tag, guess)); console.log("constant-time:", constantTimeEqual(tag, guess)); console.log("In production call a vetted primitive: crypto.timingSafeEqual, CRYPTO_memcmp, or sodium_memcmp."); `} Two rules cover most real disasters, and they have nothing to do with picking the "strongest" hash. Never use a bare fast hash for passwords, and never use a bare hash where you meant HMAC. The rest of choosing is about axes that are not strength at all: length-extension behavior, speed, FIPS status, parallelism, and construction diversity. What remains is to clear up the myths that cause the disasters those two rules do not. ## 12. Misconceptions and Real-Code Smells The field's folklore is mostly half-right, which is worse than fully wrong, because half-right beliefs survive code review. Here are the questions that actually decide those reviews. No, and this is the single most common misconception in the subject. FIPS 202 says its SHA-3 functions supplement the SHA-1 and SHA-2 families rather than replacing them [@fips-202], and NIST's Policy on Hash Functions states plainly that there is "no need to transition applications from SHA-2 to SHA-3" [@nist-policy-hash-functions]. SHA-2 and SHA-3 coexist deliberately, on a diversification axis, so the internet's trust machinery does not rest on one construction. The supersession ladder ran MD5 to SHA-1 to SHA-2 and ended there. Yes. Full-width SHA-256 and SHA-512 are length-extension-vulnerable, because their digest is the whole final chaining state. So is SHA-224, despite being truncated: it hides only 32 bits of state, which an attacker recovers in about $2^{32}$ work before extending normally [@rfc-3874]. The genuinely immune options are SHA-384, SHA-512/224, SHA-512/256, the SHA-3 family and SHAKE, HMAC, KMAC, and the BLAKE hashes. If you need `H(secret || message)` behavior, use HMAC or one of those instead. No. Every collision attack on SHA-1 and MD5 attacks collision resistance, and HMAC does not rely on collision resistance. RFC 2104 states directly that MD5's weaknesses do not compromise its use within HMAC [@rfc-2104]. HMAC-SHA-1 and HMAC-MD5 remain unbroken, though new systems should still prefer HMAC-SHA-256 for margin. Only as a non-security checksum guarding against accidental corruption, where no adversary can influence the input. The moment an attacker can choose inputs, MD5 collisions become forged certificates, as the 2008 rogue certificate authority and 2012 Flame both showed [@rogue-ca, @stevens-counter-cryptanalysis-crypto13]. Never use MD5 for signatures, certificates, or deduplication of adversarial data. No. Grover's algorithm halves the preimage exponent, leaving SHA-256 with about 128-bit preimage resistance, still infeasible [@grover-1996]. For collisions the quantum speedup is marginal once you charge for quantum memory, so SHA-256 keeps roughly 128-bit collision resistance in practice [@bernstein-sharcs09]. The hedge, if you want more margin, is a longer output such as SHA-384 or SHA-512, not a new construction. SHA-256 for general integrity, signatures, and interoperability, especially where FIPS validation matters [@fips-180-4]. BLAKE3 when you want maximum throughput, streaming, or a KDF and do not need FIPS [@blake3-repo]. SHA-512/256 when you want length-extension immunity plus FIPS on 64-bit software. No. Both are built to be fast, which is exactly what you do not want for passwords, where speed helps the attacker guess. Use a purpose-built slow function: Argon2id for new systems, or scrypt, bcrypt, or PBKDF2. BLAKE3's own README says the same and points to Argon2 [@blake3-repo]. The misuse catalog is the FAQ's mirror image: the same mistakes, seen from the code side. > **Note:** `H(secret || message)` pressed into service as a MAC (use HMAC or KMAC). Passwords fed to a bare or unsalted fast hash (use Argon2id). Truncating a fixed digest to an odd length instead of asking an XOF (use SHAKE or a BLAKE3 XOF). Comparing tags with `==` or an early-exit loop instead of a constant-time check.These last two, ad hoc truncation and non-constant-time comparison, are the ones that pass functional tests perfectly and fail only under an adversary, which is why they survive so long in production. Rolling your own Merkle tree without domain-separating leaf and internal nodes. Treating a hash as if it provided encryption. And assuming a real hash is a perfect random oracle rather than an approximation of one. Return, finally, to the two files from the opening, the ones that shared the SHA-1 digest `38762cf7f55934b34d179ae6a4c80cadccbb7f0a` [@shattered]. You can now read that string precisely. It is not evidence that hashing is broken, or that longer is always safer, or that SHA-3 rode in to replace the fallen. It is a single clause, collision resistance, failing on a single aging construction, and being weaponized because signatures sign the digest, not the file. The answer the field actually gave was not one replacement but two moves at once: widen the margins of the survivor, SHA-2, and diversify into unrelated constructions, the SHA-3 sponge and the BLAKE3 tree, so no future break can take everything down together. A cryptographic hash makes one promise. Mastery is knowing exactly which clause of that promise your protocol leans on, and choosing the function and the mode that keeps it.