Skip to main content
Security & Privacy

Why AES-256 Decryption Fails When the Password Is Right

A ciphertext does not record its mode, its IV or how the password became a key. What an AES decrypt needs besides the password, and how each wrong guess fails.

By 9 min read
An open padlock standing on a laptop keyboard in dark blue light, with a green reflection running along the shackle.

You have a ciphertext, the password and a decrypt that returns bad decrypt or a line of garbage. The password is rarely the problem. A ciphertext is a run of bytes, and nothing in the bytes says which mode produced them, how the password turned into a key or where the IV went.

A decrypt needs all of that. The tool that encrypted made those choices, and it either stored them in a layout only it understands or stored them nowhere.

What a decrypt needs besides the password

Six inputs, and the password is only the first part of one of them.

Key size and mode. AES works on 128-bit blocks and takes a 128, 192 or 256-bit key6. "AES-256" names the key and says nothing about the mode. CBC, CTR and GCM turn the same cipher into three different formats, and a tool that assumes the wrong one produces noise or an error.

How the password becomes a key. AES wants exactly 16, 24 or 32 key bytes, and a password is none of those. Tools run it through a key derivation function, and every setting of that function belongs to the recipe: the hash, the salt, the iteration count. OpenSSL's enc command derives the key with MD5 or SHA-256 in a single round unless -pbkdf2 or -iter is passed, and -pbkdf2 alone means PBKDF2 at 10,000 iterations12. Same password, different settings, different key. A 64-character hex string typed into a password box is a password to the tool, not the 32-byte key it spells.

The salt. Random bytes mixed into that derivation so the same password gives a different key each time. It is stored beside the ciphertext, which means the decrypting tool has to know where.

The IV or nonce. Not secret, but it has to match. CBC and CTR use 16 bytes. GCM uses 12, because NIST recommends implementations restrict themselves to 96-bit IVs5.

Padding or tag. CBC pads the plaintext up to a multiple of 16 bytes, and always pads, even when the input already fits7. GCM appends an authentication tag, 128 bits in the usual case, and returns no plaintext at all when the tag does not verify5.

Encoding. The bytes usually travel as base64 or hex, sometimes wrapped across lines. The tool has to decode them before the cipher sees anything.

Where the recipe lives: three layouts

Three byte layouts side by side. OpenSSL and CryptoJS passphrase mode: the text Salted__, an 8-byte salt, then ciphertext. This tool: a 3-byte header, a 16-byte salt, an IV, then ciphertext with a tag. Raw key and IV: ciphertext only, with key and IV supplied separately.
The same cipher, three different answers to the question of where the key, the salt and the IV go.

OpenSSL enc and CryptoJS passphrase mode write the ASCII text Salted__, then 8 salt bytes, then the ciphertext13. Key and IV are derived together from passphrase and salt, so neither is stored. In base64 the first ten characters read U2FsdGVkX1, which is the quickest tell there is.

CryptoJS derives with MD5 in one round, a 256-bit AES key, CBC and PKCS#7 padding34. A string made by CryptoJS 4.2.0 therefore decrypts with openssl enc -d -aes-256-cbc -md md5 -a -A, and fails with the same command minus -md md5. OpenSSL's own default digest has been SHA-256 since version 1.1.02, so ciphertext made with the openssl command may need -md sha256 instead, or -pbkdf2 if it was encrypted that way. Which one applies depends on the command line that made the file, and the file does not say.

This tool writes a three-byte header (version, mode, key size), a 16-byte salt, the IV and the ciphertext, with the 16-byte tag at the end in GCM mode. The key comes from PBKDF2 with SHA-256 at 600,000 iterations. Ciphertext written by earlier versions of the page used 100,000 iterations and, before the mode selector existed, no header. In base64 the default AES-256-GCM output starts with AQAC.

Raw key and IV is what most library code and APIs use. The ciphertext arrives alone, and the key and IV are passed to the decrypt call separately, often as hex or base64 strings. OpenSSL takes them with -K and -iv, both as hex2. Nothing in these bytes names the mode, so somebody has to tell you.

What each wrong guess looks like

Each wrong input fails differently, and the symptom points at the cause.

ModeWrong key or derivationWrong IV or nonce
GCMFailsFails
CBCFails, rarely garbageFirst block wrong
CTRGarbageGarbage

"Fails" means an error and no plaintext at all. "Garbage" means bytes of the right length that look like noise, with no error.

GCM is the easy case. It checks a tag before it releases anything, so a wrong key, a wrong nonce or a damaged byte all end the same way: no output and an error5. In a browser, Web Crypto rejects with an OperationError8, and it does not say which input was wrong.

CTR is the hard case. It has no check of any kind, so every wrong key decrypts to bytes of the right length that look like noise, and nothing reports a problem.

CBC sits between them, and its padding check is a weak password test, not a real one2. After decryption the last block must end in valid padding, and a wrong key almost never produces that. Almost. Node's crypto module, given one AES-256-CBC ciphertext and 300,000 random 256-bit keys, accepted the padding for 1,152 of them, 0.38%. That matches the arithmetic: a random last block is accepted when it ends in a single 01 byte, or 02 02, and so on, which adds up to about 1 in 255. Roughly one wrong key in 255 therefore yields a clean-looking "success" and a screen of garbage. A decrypt that returns something is not proof that the key was right.

The wrong-IV column follows from how CBC chains. It XORs each decrypted block with the previous ciphertext block, and the IV only plays that part for the first block. Get the key right and the IV wrong and the first 16 bytes of the plaintext are wrong while everything after them decrypts perfectly. If you see readable text with a broken opening, check the IV before you doubt the password.

A two-minute look at a ciphertext you did not make

  1. Decode the base64 and look at the start. Salted__ (U2FsdGVkX1 in base64) means OpenSSL or CryptoJS passphrase format. The key and IV are derived, so you need the passphrase and the digest.
  2. Check the length. After removing any header, a CBC ciphertext is a multiple of 16 bytes, always, since padding is always added. Anything else points at GCM (plaintext length plus a 16-byte tag, in the usual case) or CTR (exactly the plaintext length).
  3. Look for where the IV went. Without a Salted__ header, many producers prepend the IV to the blob (16 bytes, or 12 for GCM) and others send it through a separate field. The code or the API documentation behind the data says which.
  4. Ask what turned the password into a key. If the producer was passing the password straight to a library call with a 32-character string, the string may be the raw key, not a password at all.
  5. Check the encoding. Wrapped lines, a trailing newline, URL-safe base64 with - and _: any of these makes a decoder fail before AES gets involved.

What this tool reads, and what it does not

The AES Encrypt/Decrypt decrypts what it writes, and since this guide was first published it reads the two layouts above as well. Under Decrypt you say which one you hold.

This tool's own data needs only the password: its current layout, the 100,000-iteration layout and the original headerless AES-256-GCM one. It detects mode and key size from the header, so you never need to remember them.

OpenSSL / CryptoJS (Salted__) needs the passphrase and the settings the file does not record. Choose EVP_BytesToKey with MD5 for CryptoJS and openssl enc -md md5, EVP_BytesToKey with SHA-256 for openssl enc from 1.1.0 on without -pbkdf2, or PBKDF2 with SHA-256 for -pbkdf2 and the -iter count you used (10000 if none). Then pick the key size and CBC or CTR. It reads that text as Base64 (U2FsdGVkX1) or hex. When it fails, change the derivation before you doubt the passphrase.

Raw key + IV takes the ciphertext, the key and the IV, written as hex, Base64 or plain text, and a mode: CBC, CTR or GCM. The key length sets the AES size, and a GCM ciphertext needs its 16-byte tag at the end. The mode is not in the data, so if one fails, try another.

It does not cover everything. A Salted__ file made with another digest, such as -md sha512, a GCM tag stored apart from the ciphertext with extra authenticated data, and CBC without padding all need the openssl command on your own machine, where the mode, the digest and the key derivation are arguments you can set.

The errors follow the table above. A CBC decrypt that fails the padding check points at the key or the derivation, not the IV. Bytes that are not valid text are shown as hex with a warning, because CTR cannot fail and about one wrong CBC key in 255 passes the padding check.

If you are choosing a mode for something new, AES-GCM vs AES-CBC covers that decision, and the Base64 tool is the quick way to look at the first bytes of a ciphertext before deciding which layout you have.

Sources

Every number in this article traces to a source below. Where a claim could not be sourced, it was cut rather than softened.

  1. Primary sourceOpenSSL Project

    That the enc command writes the magic string "Salted__" followed by the salt before the ciphertext, derives key and IV together from the passphrase and salt, uses EVP_BytesToKey with one iteration unless -pbkdf2 or -iter is given and warns that this derivation is deprecated, and defaults to PBKDF2 with 10000 iterations when -pbkdf2 is set.

  2. Primary sourceOpenSSL Project

    That the default digest is SHA-256 and was MD5 before OpenSSL 1.1.0, that -pbkdf2 defaults to 10000 iterations, that -K and -iv take hex strings, that block ciphers use PKCS#7 padding by default, and that padding allows only "a rudimentary integrity or password check".

  3. Primary sourceCryptoJS

    That passphrase-mode ciphertext is serialised as the two words of "Salted__" plus an 8-byte salt plus the ciphertext, that key and IV are derived together with EvpKDF, and that block ciphers default to CBC mode with PKCS#7 padding.

  4. Primary sourceCryptoJS

    That the EvpKDF default hasher is MD5 with one iteration, and that AES has a key size of 256 bits by default.

  5. Primary sourceNIST

    That implementations are recommended to restrict IV support to 96 bits, that the tag length is normally 128, 120, 112, 104 or 96 bits, and that authenticated decryption returns FAIL, not plaintext, when the tag does not verify (section 7.2).

  6. Primary sourceNIST

    That AES has a 128-bit block in every variant and key lengths of 128, 192 and 256 bits.

  7. Primary sourceIETF

    The padding rule behind PKCS#7 block padding, that k-(lth mod k) octets each of value k-(lth mod k) are appended and that all input is padded, including input that is already a multiple of the block size.

  8. Peer-reviewedMDN Web Docs

    That the algorithm parameters passed to decrypt must match those used for encryption, and that the call rejects with OperationError when there was an error decrypting the ciphertext.

Topics

  • AES
  • Encryption
  • Decryption
  • Openssl
  • Cryptojs

Tools mentioned in this article

  • AES-256 Encrypt / Decrypt Online - Free, In-Browser - Encrypt and decrypt text with AES-128, AES-192, or AES-256 in GCM, CBC, or CTR mode. Also decrypts OpenSSL and CryptoJS Salted__ output and raw key + IV ciphertext, entirely in your browser.
  • Base64 Encoder & Decoder - Encode UTF-8 text to Base64 online or decode Base64 back to UTF-8 and plain text. Runs in your browser with no upload.
  • PBKDF2 Hash Generator - Derive cryptographic keys from passwords using PBKDF2 with configurable iterations, salt and hash function.

Related articles

A pink sticky note handwritten with the word password and the digits 123456, stuck to a black laptop keyboard against a blue background
Security & Privacy

Password Strength: What Length and Character Sets Buy You

Entropy done honestly, NIST's current 15-character floor, and measured GPU cracking speeds showing the hash function matters more than the extra characters.