Decrypting Salted Aes File Generated on Command Line with Ruby

Decrypting salted AES file generated on command line with Ruby

OpenSSL uses a custom header and key derivation routine. Security.SE has a good description of the header and the docs for EVP_BytesToKey describe the key derivation.

We can modify your code to use this weird and somewhat broken key derivation as follows:

encoded_and_encrypted_text = File.read my_file_path
encrypted_text = Base64.decode64 encoded_and_encrypted_text.to_s.strip

header = encrypted_text[0,8]
salt = encrypted_text[8,8]
payload = encrypted_text[16..-1]

decipher = OpenSSL::Cipher::AES.new 256, :CBC
decipher.decrypt

D_1 = OpenSSL::Digest::MD5.new(my_password + salt).digest
D_2 = OpenSSL::Digest::MD5.new(D_1 + my_password + salt).digest
D_3 = OpenSSL::Digest::MD5.new(D_2 + my_password + salt).digest

decipher.key = (D_1 + D_2)
decipher.iv = D_3

plain_text = decipher.update(payload) + decipher.final

Why can't I make OpenSSL with Ruby and Command line OpenSSL interoperable?

You need to use the -K (upper case) and -iv options on the command line to specify key and IV explicitly as a string of hex digits. If you use -k (lower case), OpenSSL will derive key and IV from the password using a key derivation function. When OpenSSL derives a key, it will also use a "salted" ciphertext format which is incompatible with the plain blockwise CBC you are expecting.

Note that in your Ruby code, you are using the first 256 bits (32 bytes) of an ASCII string directly as a key, which is almost certainly not what you want for a real world application where security is an issue. You should use a (randomly generated) binary key, or derive a key from a password using a key derivation function such as PBKDF2, bcrypt or scrypt.

Trying to decrypt a string using openssl/golang which has been encrypted in rails

Trying to decrypt data encrypted in a different system will not work unless you are aware and deal with the many intricate details of how both systems do the cryptography. Although both Rails and the openssl command line tool use the OpenSSL libraries under the hood for their crypto operations, they both use it in their own distinct ways that are not directly interoperable.

If you look close to the two systems, you'll see that for example:

• Rails message encryptor not only encrypts the message but also signs it
• Rails encryptor uses Marshal to serialize the input data
• the openssl enc tool expects the encrypted data in a distinct file format with a Salted__<salt> header (this is why you get the bad magic number message from openssl)
• the openssl tool must be properly configured to use the same ciphers as Rails encryptor and key generator, as openssl defaults are different from Rails defaults
• the default ciphers configuration changed significantly since Rails 5.2.

With this general info, we can have a look at a a practical example. It is tested in Rails 4.2 but should work equally up to Rails 5.1.

Anatomy of a Rails-encrypted message

Let me start with a slightly amended code that you presented. The only changes there are to preset the password and salt to static values and print a lot of debug info:

def encrypt_text(text_to_encrypt)
  password = "password" # the password to derive the key
  salt = "saltsalt" # salt must be 8 bytes

  key = ActiveSupport::KeyGenerator.new(password).generate_key(salt, 32)

  puts "salt (hexa) = #{salt.unpack('H*').first}" # print the saltin HEX
  puts "key (hexa) = #{key.unpack('H*').first}" # print the generated key in HEX

  crypt = ActiveSupport::MessageEncryptor.new(key)
  output = crypt.encrypt_and_sign(text_to_encrypt)
  puts "output (base64) = #{output}"
  output
end

encrypt_text("secret text")

When you run this, you'll get something like the following output:

salt (hexa) = 73616c7473616c74
key (hexa) = 196827b250431e911310f5dbc82d395782837b7ae56230dce24e497cf07b6518
output (base64) = SGRTUXYxRys1N1haVWNpVWxxWTdCMHlyMk15SnQ0dWFBOCt3Z0djWVdBZz0tLTkrd1hBNWJMVm9HcnptZ3loOG1mNHc9PQ==--80d091e8799776113b2c0efd1bf75b344bf39994

The last line (output of the encrypt_and_sign method) is a combination of two parts separated by -- (see source):

1. the encrypted message (Base64-encoded) and
2. the message signature (Base64-encoded).

The signature is not important for encryption so let's take a look in the first part - let's decode it in Rails console:

> Base64.strict_decode64("SGRTUXYxRys1N1haVWNpVWxxWTdCMHlyMk15SnQ0dWFBOCt3Z0djWVdBZz0tLTkrd1hBNWJMVm9HcnptZ3loOG1mNHc9PQ==")
=> "HdSQv1G+57XZUciUlqY7B0yr2MyJt4uaA8+wgGcYWAg=--9+wXA5bLVoGrzmgyh8mf4w=="

You can see that the decoded message again consists of two Base64-encoded parts separated by -- (see source):

1. the encrypted message itself
2. the initialization vector used in the encryption

Rails message encryptor uses the aes-256-cbc cipher by default (note that this has changed since Rails 5.2). This cipher needs an initialization vector, which is randomly generated by Rails and must be present in the encrypted output so that we can use it together with the key to decipher the message.

Moreover, Rails does not encrypt the input data as a simple plain text, but rather a serialized version of the data, using the Marshal serializer by default (source). If we decrypted such serialized value with openssl, we would still get a slightly garbled (serialized) version of the initial plain text data. That's why it will be more appropriate to disable serialization while encrypting the data in Rails. This can be done by passing a parameter to the encryption method:

  # crypt = ActiveSupport::MessageEncryptor.new(key)
  crypt = ActiveSupport::MessageEncryptor.new(key, serializer: ActiveSupport::MessageEncryptor::NullSerializer)

A re-run of the code yields output that is slightly shorter than the previous version, because the encrypted data has not been serialized now:

salt (hexa) = 73616c7473616c74
key (hexa) = 196827b250431e911310f5dbc82d395782837b7ae56230dce24e497cf07b6518
output (base64) = SUlIWFBjSXRUc0JodEMzLzhXckJzUT09LS1oZGtPV1ZRc2I5Wi8zOG01dFNOdVdBPT0=--58bbaf983fd20459062df8b6c59eb470311cbca9

Finally, we must find out some info about the encryption key derivation procedure. The source tells us that the KeyGenerator uses the pbkdf2_hmac_sha1 algorithm with 2**16 = 65536 iterations to derive the key from the password / secret.

Anatomy of an openssl encrypted message

Now, a similar investigation is needed on the openssl side to learn the details of its decryption process. First, if you encrypt anything using the openssl enc tool, you will find out that the output has a distinct format:

Salted__<salt><encrypted_message>

It begins with the Salted__ magic string, then followed by the salt (in hex form) and finally followed by the encrypted data. To be able to decrypt any data using this tool, we must get our encrypted data into the same format.

The openssl tool uses the EVP_BytesToKey (see source) to derive the key by default but can be configured to use the pbkdf2_hmac_sha1 algorithm using the -pbkdf2 and -md sha1 options. The number of iterations can be set using the -iter option.

How to decrypt Rails-encrypted message in openssl

So, finally we have enough information to actually try to decrypt a Rails-encrypted message in openssl.

First we must decode the first part of the Rails-encrypted output again to get the encrypted data and the initialization vector:

> Base64.strict_decode64("SUlIWFBjSXRUc0JodEMzLzhXckJzUT09LS1oZGtPV1ZRc2I5Wi8zOG01dFNOdVdBPT0=")
=> "IIHXPcItTsBhtC3/8WrBsQ==--hdkOWVQsb9Z/38m5tSNuWA=="

Now let's take the IV (the second part) and convert it to a hexa string form, as that is the form that openssl needs:

> Base64.strict_decode64("hdkOWVQsb9Z/38m5tSNuWA==").unpack("H*").first
=> "85d90e59542c6fd67fdfc9b9b5236e58"  # the initialization vector in hex form

Now we need to take the Rails-encrypted data and convert it to the format that openssl will recognize, i.e. prepend it with the magic string and salt and Base64-encode it again:

> Base64.strict_encode64("Salted__" + "saltsalt" + Base64.strict_decode64("IIHXPcItTsBhtC3/8WrBsQ=="))
=> "U2FsdGVkX19zYWx0c2FsdCCB1z3CLU7AYbQt//FqwbE=" # encrypted data suitable for openssl

Finally, we can construct the openssl command to decrypt the data:

$ echo  "U2FsdGVkX19zYWx0c2FsdCCB1z3CLU7AYbQt//FqwbE=" | 
> openssl enc -aes-256-cbc -d -iv 85d90e59542c6fd67fdfc9b9b5236e58 \
>   -pass pass:password -pbkdf2 -iter 65536 -md sha1 -a
secret text

And voilá, we successfully decrypted the initial message!

The openssl parameters are as follows:

• -aes-256-cbc sets the same cipher as Rails uses for encryption
• -d stands for decryption
• -iv passes the initialization vector in the hex string form
• -pass pass:password sets the password used to derive the encryption key to "password"
• -pbkdf2 and -md sha1 set the same key derivation algorithm as is used by Rails (pbkdf2_hmac_sha1)
• -iter 65536 sets the same number of iterations for key derivation as was done in Rails
• -a allows to work with Base64-encoded encrypted data - no need to handle raw bytes in files

By default openssl reads from STDIN, so we simply pass the encrypted data (in proper format) to openssl using echo.

debugging

In case you hit any problems when decrypting with openssl, it is useful to add the -P parameter to the command line, which outputs debugging info about the cipher / key parameters:

$ echo ... | openssl ... -P
salt=73616C7473616C74
key=196827B250431E911310F5DBC82D395782837B7AE56230DCE24E497CF07B6518
iv =85D90E59542C6FD67FDFC9B9B5236E58

The salt, key, and iv values must correspond to the debugging values printed by the original code in the encrypt_text method printed above. If they are different, you know you are doing something wrong...

Now, I guess you can expect similar problems when trying to decrypt the message in go but I think you have some good pointers now to start.

AES-256-CBC with Digest from Ruby to NodeJS

The critical missing piece is the IV, which is required when encryption/decryption is to be made across language boundaries as apparently the encrypter will generate a random IV (or something like that - still don't understand how Ruby decrypts the string without an IV.... but then what do I know....), if one is not provided.

The following snippets show how to encrypt a string in Ruby and decrypt in NodeJS.

#!/usr/bin/env ruby

require 'openssl'
require 'base64'
require 'openssl/cipher'
require 'openssl/digest'

aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
aes.encrypt
aes.key = Digest::SHA256.digest('IHazSekretKey') 
aes.iv  = '1234567890123456'

p Base64.encode64( aes.update('text to be encrypted') << aes.final )

The above prints: "eiLbdhFSFrDqvUJmjbUgwD8REjBRoRWWwHHImmMLNZA=\n"

#!/usr/bin/env node

var crypto = require('crypto'); 

function decrypto(toDecryptStr) {
  var result,
    encoded   = new Buffer(toDecryptStr, 'base64'),
    decodeKey = crypto.createHash('sha256').update('IHazSekretKey', 'ascii').digest(),
    decipher  = crypto.createDecipheriv('aes-256-cbc', decodeKey, '1234567890123456');

  result = decipher.update(encoded);
  result += decipher.final();

  return result;
}

console.log(decrypto('eiLbdhFSFrDqvUJmjbUgwD8REjBRoRWWwHHImmMLNZA=\n'))

The JS script now properly decrypts the string.

One unfortunate side effect is that existing encrypted data will need to be decrypted and then re-encrypted with an IV that is then used in the decrypting implementation.

A PITA but nonetheless a working solution.

AES equivalent in Ruby openssl?

Digging into Gibberish code... provides the clues to the answers. and why the traditional mechanism does not work.

dec = function(string, pass) {
    // string, password in plaintext
    var cryptArr = Base64.decode(string),
    salt = cryptArr.slice(8, 16),
    pbe = openSSLKey(s2a(pass), salt),
    key = pbe.key,
    iv = pbe.iv;
    cryptArr = cryptArr.slice(16, cryptArr.length);
    // Take off the Salted__ffeeddcc
    string = rawDecrypt(cryptArr, key, iv);
    return string;
},

Converting to ruby is now fairly trivial.. noting it down for my personal future reference.

require 'base64'
require 'openssl'

def decode(k,t)
  cryptArr = Base64.decode64(t)
  salt     = cryptArr[8..15]
  data     = cryptArr[16..-1] 

  aes = OpenSSL::Cipher::Cipher.new('AES-256-CBC').decrypt
  aes.pkcs5_keyivgen(k, salt, 1)
  s = aes.update(data) + aes.final
end

orig = "Made with Gibberish\n"
cipr = "U2FsdGVkX1+21O5RB08bavFTq7Yq/gChmXrO3f00tvJaT55A5pPvqw0zFVnHSW1o"
pass = "password"

puts decode(pass, cipr)

Openssl aes-256-cbc encryption from command prompt and decryption in PHP (and vice versa)

The OpenSSL statement generates a random 8 bytes salt during encryption, which is used together with the password to derive a 32 bytes key and a 16 bytes IV with the OpenSSL function EVP_BytesToKey().

With key and IV the encryption is performed with AES-256 in CBC mode. The result consists of the concatenation of the ASCII encoding of Salted__, followed by the salt and the actual ciphertext, all Base64 encoded.

The decryption in PHP/OpenSSL must be implemented as follows:

• Determination of salt and actual ciphertext.
• Using salt, password and EVP_BytesToKey() to get key and IV.
• Using key and IV to perform decryption with AES-256 in CBC mode.

One possible implementation is:

<?php
function EVP_BytesToKey($salt, $password) {
    $bytes = '';
    $last = '';
    while(strlen($bytes) < 48) {
        $last = hash('md5', $last . $password . $salt, true);
        $bytes.= $last;
    }
    return $bytes;
} 
    
$saltCiphertext = base64_decode('U2FsdGVkX18ruQUgA9LEOOvdOUQXv/o8z6ZNO820MKzSIbMjFcyfNo1efQwAOINxMY9+UxZjxaT+JEWmlUyYQw==');
$salt = substr($saltCiphertext, 8, 8);
$ciphertext = substr($saltCiphertext, 16);
$keyIv = EVP_BytesToKey($salt, 'sw8/M!CLl:=cmgtHts?v/Wb7C$Vk9Sy-{go.*+E;[GAg~KQi*rI!1#z;x/KT');
$key = substr($keyIv, 0, 32);
$iv = substr($keyIv, 32);
echo openssl_decrypt($ciphertext, 'aes-256-cbc', $key, OPENSSL_RAW_DATA, $iv); // {un:est@test.com,upass:klkKJS*dfd!j@d76w}
?>

In earlier versions OpenSSL used MD5 as digest in EVP_BytesToKey() by default, from version V1.1.0 SHA256. In the posted example, decryption with MD5 is successful, so obviously MD5 was used in encryption.

Note that key derivation with EVP_BytesToKey() is deemed insecure nowadays.

Decrypting a PHP generated OpenSSL string in Terminal

Just like your last question, you are struggling with encodings a bit. If you carefully consult the documentation for openssl_encrypt, you'll note that both the key and IV should be passed as raw values, not hex.

You did this correctly in your code with the IV, but not the key. You passed the key as a hex value, which means it was twice as long as it needed to be. Just the first 256-bits of the key are used, in this case, B374A26A71490437AA024E4FADD5B497, since you passed 512-bits of key material in total.

So we know that our raw key, when ASCII encoded, is B374A26A71490437AA024E4FADD5B497, which is exactly 256-bits. However, the OpenSSL -K flag that I discussed in your first question requires the key to be passed hex encoded, which means we need to hex encode our key. So we hex encode B374A26A71490437AA024E4FADD5B497 to get 4233373441323641373134393034333741413032344534464144443542343937, which is the actual hex encoded encryption key.

So, in summary, the final command is this, which gives an output of just the byte 0x70, which I assume is correct:

openssl enc -d -K 4233373441323641373134393034333741413032344534464144443542343937 -iv 61736466673534336173646667353433 -in input.bin -out out.bin

This assumes that input.bin is the base64 decoded binary of the base64 ciphertext you provided.

OpenSSL Encryption Key and IV differs from that generated from Java program

In addition to removing the CRLF from the password value as per comment by reinier:

You are apparently using OpenSSL 1.1.0 or higher, which changed the hash used by default in enc PBKDF/PBE from MD5 to SHA256. See man enc (may require a special section like 1ssl) on a Unixy system which yours apparently isn't or on the web under HISTORY, and maybe Password to key function compatible with OpenSSL commands? . Note with SHA256 you need only 2 chunks not 4 in the PBKDF.

FYI you don't need to copy the array slices; both SecretKeySpec and IvParameterSpec have constructor overloads that use a slice of the byte array rather than the whole array. And AES-256 key is 32 bytes (at least in Java), which is what you coded even though you commented differently.

---

Related Topics