C# version of OpenSSL EVP_BytesToKey method?
I found this pseudo-code explanation of the EVP_BytesToKey method (in /doc/ssleay.txt of the openssl source):
/* M[] is an array of message digests
* MD() is the message digest function */
M[0]=MD(data . salt);
for (i=1; i<count; i++) M[0]=MD(M[0]);
i=1
while (data still needed for key and iv)
{
M[i]=MD(M[i-1] . data . salt);
for (i=1; i<count; i++) M[i]=MD(M[i]);
i++;
}
If the salt is NULL, it is not used.
The digests are concatenated together.
M = M[0] . M[1] . M[2] .......
So based on that I was able to come up with this C# method (which seems to work for my purposes and assumes 32-byte key and 16-byte iv):
private static void DeriveKeyAndIV(byte[] data, byte[] salt, int count, out byte[] key, out byte[] iv)
{
List<byte> hashList = new List<byte>();
byte[] currentHash = new byte[0];
int preHashLength = data.Length + ((salt != null) ? salt.Length : 0);
byte[] preHash = new byte[preHashLength];
System.Buffer.BlockCopy(data, 0, preHash, 0, data.Length);
if (salt != null)
System.Buffer.BlockCopy(salt, 0, preHash, data.Length, salt.Length);
MD5 hash = MD5.Create();
currentHash = hash.ComputeHash(preHash);
for (int i = 1; i < count; i++)
{
currentHash = hash.ComputeHash(currentHash);
}
hashList.AddRange(currentHash);
while (hashList.Count < 48) // for 32-byte key and 16-byte iv
{
preHashLength = currentHash.Length + data.Length + ((salt != null) ? salt.Length : 0);
preHash = new byte[preHashLength];
System.Buffer.BlockCopy(currentHash, 0, preHash, 0, currentHash.Length);
System.Buffer.BlockCopy(data, 0, preHash, currentHash.Length, data.Length);
if (salt != null)
System.Buffer.BlockCopy(salt, 0, preHash, currentHash.Length + data.Length, salt.Length);
currentHash = hash.ComputeHash(preHash);
for (int i = 1; i < count; i++)
{
currentHash = hash.ComputeHash(currentHash);
}
hashList.AddRange(currentHash);
}
hash.Clear();
key = new byte[32];
iv = new byte[16];
hashList.CopyTo(0, key, 0, 32);
hashList.CopyTo(32, iv, 0, 16);
}
UPDATE: Here's more/less the same implementation but uses the .NET DeriveBytes interface: https://gist.github.com/1339719
OpenSSL 1.1.0c changed the digest algorithm used in some internal components. Formerly, MD5 was used, and 1.1.0 switched to SHA256. Be careful the change is not affecting you in both EVP_BytesToKey and commands like openssl enc.
C# AES-256-CFB like PHP OpenSSL encryption
I think that the OpenSSL CFB mode isn't compatible with .NET: there is a difference in the last block. OpenSSL CFB simply truncates it if it isn't full, while .NET will pad it. This code WILL work if you set the Mode = CipherMode.CBC, but will produce different result if you use the Mode = CipherMode.CFB (you can see that the last characters generated by GenerateSecret() are different). I'm giving a GenerateSecret() and a ExtractSecret() written in C#.
If someone with more experience than me can explain this difference (I've seen the explanation here, but it isn't clear how I should "solve" it).
public static string GenerateSecret(string username, string token)
{
byte[] key;
using (var md5 = MD5.Create())
{
key = md5.ComputeHash(Encoding.UTF8.GetBytes(username));
}
key = BytesToLowerHexBytes(key);
var iv = Convert.FromBase64String(token);
if (iv.Length != 16)
{
Array.Resize(ref iv, 16);
}
byte[] encrypted;
using (var rijndael = new RijndaelManaged())
{
rijndael.Mode = CipherMode.CFB;
rijndael.Padding = PaddingMode.PKCS7;
rijndael.KeySize = 256;
using (var msEncrypt = new MemoryStream())
{
using (ICryptoTransform encryptor = rijndael.CreateEncryptor(key, iv))
using (var csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
{
var buffer = Encoding.UTF8.GetBytes(token);
csEncrypt.Write(buffer, 0, buffer.Length);
}
encrypted = msEncrypt.ToArray();
}
}
var buffer2 = Encoding.UTF8.GetBytes(Convert.ToBase64String(encrypted));
using (var ms = new MemoryStream(iv.Length + buffer2.Length))
{
ms.Write(iv, 0, iv.Length);
ms.Write(buffer2, 0, buffer2.Length);
return Convert.ToBase64String(ms.GetBuffer(), 0, (int)ms.Length);
}
}
public static string ExtractSecret(string username, string encrypted)
{
byte[] key;
using (var md5 = MD5.Create())
{
key = md5.ComputeHash(Encoding.UTF8.GetBytes(username));
}
key = BytesToLowerHexBytes(key);
byte[] bytes = Convert.FromBase64String(encrypted);
byte[] iv = bytes;
Array.Resize(ref iv, 16);
byte[] decrypted;
using (var rijndael = new RijndaelManaged())
{
rijndael.Mode = CipherMode.CFB;
rijndael.Padding = PaddingMode.PKCS7;
rijndael.KeySize = 256;
using (var msDecrypt = new MemoryStream())
{
using (ICryptoTransform decryptor = rijndael.CreateDecryptor(key, iv))
using (var csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Write))
{
var buffer = Convert.FromBase64String(Encoding.UTF8.GetString(bytes, 16, bytes.Length - 16));
csDecrypt.Write(buffer, 0, buffer.Length);
}
decrypted = msDecrypt.ToArray();
}
}
return Encoding.UTF8.GetString(decrypted);
}
private static byte[] BytesToLowerHexBytes(byte[] bytes)
{
// The hash is a hex string
var bytes2 = new byte[bytes.Length * 2];
for (int i = 0, j = 0; i < bytes.Length; i++)
{
byte b1 = (byte)(bytes[i] >> 4);
bytes2[j] = (byte)(b1 <= 9 ? '0' + b1 : 'a' + b1 - 10);
j++;
byte b2 = (byte)(bytes[i] & 15);
bytes2[j] = (byte)(b2 <= 9 ? '0' + b2 : 'a' + b2 - 10);
j++;
}
return bytes2;
}
Potentially working
I'm stripping the extra bytes generated by .NET for CFB mode. The BytesToLowerHexBytes is from the previous sample. Note that I'm not expert enough in cryptography to guarantee that what I'm doing is correct.
public static string GenerateSecret(string username, string token)
{
byte[] key;
using (var md5 = MD5.Create())
{
key = md5.ComputeHash(Encoding.UTF8.GetBytes(username));
}
key = BytesToLowerHexBytes(key);
var iv = Convert.FromBase64String(token);
if (iv.Length != 16)
{
Array.Resize(ref iv, 16);
}
byte[] encrypted;
int encryptedLength;
using (var rijndael = new RijndaelManaged())
{
rijndael.Mode = CipherMode.CFB;
rijndael.Padding = PaddingMode.Zeros;
rijndael.KeySize = 256;
using (var msEncrypt = new MemoryStream())
{
var buffer = Encoding.UTF8.GetBytes(token);
using (ICryptoTransform encryptor = rijndael.CreateEncryptor(key, iv))
using (var csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
{
csEncrypt.Write(buffer, 0, buffer.Length);
}
// CFB is a stream cipher, where the length of the encrypted text should be
// equal to the length of the original text... So we strip the last bytes
encrypted = msEncrypt.GetBuffer();
encryptedLength = buffer.Length;
}
}
var buffer2 = Encoding.UTF8.GetBytes(Convert.ToBase64String(encrypted, 0, encryptedLength));
using (var ms = new MemoryStream(iv.Length + buffer2.Length))
{
ms.Write(iv, 0, iv.Length);
ms.Write(buffer2, 0, buffer2.Length);
return Convert.ToBase64String(ms.GetBuffer(), 0, (int)ms.Length);
}
}
public static string ExtractSecret(string username, string encrypted)
{
byte[] key;
using (var md5 = MD5.Create())
{
key = md5.ComputeHash(Encoding.UTF8.GetBytes(username));
}
key = BytesToLowerHexBytes(key);
byte[] bytes = Convert.FromBase64String(encrypted);
byte[] iv = bytes;
Array.Resize(ref iv, 16);
byte[] decrypted;
int decryptedLength;
using (var rijndael = new RijndaelManaged())
{
rijndael.Mode = CipherMode.CFB;
rijndael.Padding = PaddingMode.Zeros;
rijndael.KeySize = 256;
using (var msDecrypt = new MemoryStream())
{
var buffer = Convert.FromBase64String(Encoding.UTF8.GetString(bytes, 16, bytes.Length - 16));
using (ICryptoTransform decryptor = rijndael.CreateDecryptor(key, iv))
using (var csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Write))
{
csDecrypt.Write(buffer, 0, buffer.Length);
// We have to add the remaining bytes of the block. They aren't
// important, so we add zeroes
int remaining = 16 - (buffer.Length % 16);
if (remaining != 0)
{
csDecrypt.Write(new byte[remaining], 0, remaining);
}
}
// CFB is a stream cipher, where the length of the encrypted text should be
// equal to the length of the original text... So we strip the last bytes
decrypted = msDecrypt.GetBuffer();
decryptedLength = buffer.Length;
}
}
return Encoding.UTF8.GetString(decrypted, 0, decryptedLength);
}
C# & OpenSSL: Differing Output for AES Encryption
If you specify a key and IV directly, then the salt does not even get into play. The salt is needed to convert the pass phrase into a key using a key derivation function (the proprietary EVP_BytesToKey in the case of OpenSSL). Hence you probably get inconsequential output for the salt.
Now the first output of OpenSSL in 1) contains a header (check the ASCII values :), the salt and then the cipher text, this is the base 64 string in hexadecimals:
53616C7465645F5F CC77E2A591358A1C EB67B757ED1D22B25A2CFA1B190155C9
Your application 3) and the openssl command in 2) both output
EB67B757ED1D22B25A2CFA1B190155C9
so every little thing seems to be alright.
Equivalent to PasswordDeriveBytes in openssl
This shows how to implement PBKDF1 with OpenSSL, which according to the documentation is the algorithm used by PasswordDeriveBytes.
#include <string.h>
#include <stdlib.h>
#include <openssl/sha.h>
void pbkdf1(const char *password, const char *salt, long iter, unsigned char dk[SHA_DIGEST_LENGTH])
{
size_t pwlen = strlen(password);
size_t dlen = pwlen + 8;
unsigned char *buf;
if (dlen > SHA_DIGEST_LENGTH)
buf = malloc(dlen);
else
buf = malloc(SHA_DIGEST_LENGTH);
memcpy(buf, password, pwlen);
strncpy((char *)buf + pwlen, salt, 8);
while (iter-- > 0)
{
SHA1(buf, dlen, buf);
dlen = SHA_DIGEST_LENGTH;
}
memcpy(dk, buf, SHA_DIGEST_LENGTH);
free(buf);
}
OpenSSL encryption using .NET classes
Finally figured this one out. In the event someone needs to integrate openssl and .NET without using the openssl wrappers, I'll share the results here.
1) The main issue with my original code (as in the question) is that you must initialize the BlockSize and KeySize on your RijndaelManaged instance BEFORE setting the key or IV.
2) I also had BlockSize set to 256 when it should only be 128
3) The remainder of my issue came to the fact that openssl puts and expects "Salted__" onto the front of the salt before appending the encrypted string and then base64 encoding it. (I saw this initially in the openssl documentation with respect to file encryption but didn't think it did it when doing it directly through commandline - Apparently I was wrong!! Note also the capitalization of the S in Salted!)
With that all in mind, here is my "fixed" code:
public class Protection
{
public string OpenSSLEncrypt(string plainText, string passphrase)
{
// generate salt
byte[] key, iv;
byte[] salt = new byte[8];
RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
rng.GetNonZeroBytes(salt);
DeriveKeyAndIV(passphrase, salt, out key, out iv);
// encrypt bytes
byte[] encryptedBytes = EncryptStringToBytesAes(plainText, key, iv);
// add salt as first 8 bytes
byte[] encryptedBytesWithSalt = new byte[salt.Length + encryptedBytes.Length + 8];
Buffer.BlockCopy(Encoding.ASCII.GetBytes("Salted__"), 0, encryptedBytesWithSalt, 0, 8);
Buffer.BlockCopy(salt, 0, encryptedBytesWithSalt, 8, salt.Length);
Buffer.BlockCopy(encryptedBytes, 0, encryptedBytesWithSalt, salt.Length + 8, encryptedBytes.Length);
// base64 encode
return Convert.ToBase64String(encryptedBytesWithSalt);
}
public string OpenSSLDecrypt(string encrypted, string passphrase)
{
// base 64 decode
byte[] encryptedBytesWithSalt = Convert.FromBase64String(encrypted);
// extract salt (first 8 bytes of encrypted)
byte[] salt = new byte[8];
byte[] encryptedBytes = new byte[encryptedBytesWithSalt.Length - salt.Length - 8];
Buffer.BlockCopy(encryptedBytesWithSalt, 8, salt, 0, salt.Length);
Buffer.BlockCopy(encryptedBytesWithSalt, salt.Length + 8, encryptedBytes, 0, encryptedBytes.Length);
// get key and iv
byte[] key, iv;
DeriveKeyAndIV(passphrase, salt, out key, out iv);
return DecryptStringFromBytesAes(encryptedBytes, key, iv);
}
private static void DeriveKeyAndIV(string passphrase, byte[] salt, out byte[] key, out byte[] iv)
{
// generate key and iv
List<byte> concatenatedHashes = new List<byte>(48);
byte[] password = Encoding.UTF8.GetBytes(passphrase);
byte[] currentHash = new byte[0];
MD5 md5 = MD5.Create();
bool enoughBytesForKey = false;
// See http://www.openssl.org/docs/crypto/EVP_BytesToKey.html#KEY_DERIVATION_ALGORITHM
while (!enoughBytesForKey)
{
int preHashLength = currentHash.Length + password.Length + salt.Length;
byte[] preHash = new byte[preHashLength];
Buffer.BlockCopy(currentHash, 0, preHash, 0, currentHash.Length);
Buffer.BlockCopy(password, 0, preHash, currentHash.Length, password.Length);
Buffer.BlockCopy(salt, 0, preHash, currentHash.Length + password.Length, salt.Length);
currentHash = md5.ComputeHash(preHash);
concatenatedHashes.AddRange(currentHash);
if (concatenatedHashes.Count >= 48)
enoughBytesForKey = true;
}
key = new byte[32];
iv = new byte[16];
concatenatedHashes.CopyTo(0, key, 0, 32);
concatenatedHashes.CopyTo(32, iv, 0, 16);
md5.Clear();
md5 = null;
}
static byte[] EncryptStringToBytesAes(string plainText, byte[] key, byte[] iv)
{
// Check arguments.
if (plainText == null || plainText.Length <= 0)
throw new ArgumentNullException("plainText");
if (key == null || key.Length <= 0)
throw new ArgumentNullException("key");
if (iv == null || iv.Length <= 0)
throw new ArgumentNullException("iv");
// Declare the stream used to encrypt to an in memory
// array of bytes.
MemoryStream msEncrypt;
// Declare the RijndaelManaged object
// used to encrypt the data.
RijndaelManaged aesAlg = null;
try
{
// Create a RijndaelManaged object
// with the specified key and IV.
aesAlg = new RijndaelManaged { Mode = CipherMode.CBC, KeySize = 256, BlockSize = 128, Key = key, IV = iv };
// Create an encryptor to perform the stream transform.
ICryptoTransform encryptor = aesAlg.CreateEncryptor(aesAlg.Key, aesAlg.IV);
// Create the streams used for encryption.
msEncrypt = new MemoryStream();
using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
{
using (StreamWriter swEncrypt = new StreamWriter(csEncrypt))
{
//Write all data to the stream.
swEncrypt.Write(plainText);
swEncrypt.Flush();
swEncrypt.Close();
}
}
}
finally
{
// Clear the RijndaelManaged object.
if (aesAlg != null)
aesAlg.Clear();
}
// Return the encrypted bytes from the memory stream.
return msEncrypt.ToArray();
}
static string DecryptStringFromBytesAes(byte[] cipherText, byte[] key, byte[] iv)
{
// Check arguments.
if (cipherText == null || cipherText.Length <= 0)
throw new ArgumentNullException("cipherText");
if (key == null || key.Length <= 0)
throw new ArgumentNullException("key");
if (iv == null || iv.Length <= 0)
throw new ArgumentNullException("iv");
// Declare the RijndaelManaged object
// used to decrypt the data.
RijndaelManaged aesAlg = null;
// Declare the string used to hold
// the decrypted text.
string plaintext;
try
{
// Create a RijndaelManaged object
// with the specified key and IV.
aesAlg = new RijndaelManaged {Mode = CipherMode.CBC, KeySize = 256, BlockSize = 128, Key = key, IV = iv};
// Create a decrytor to perform the stream transform.
ICryptoTransform decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV);
// Create the streams used for decryption.
using (MemoryStream msDecrypt = new MemoryStream(cipherText))
{
using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
{
using (StreamReader srDecrypt = new StreamReader(csDecrypt))
{
// Read the decrypted bytes from the decrypting stream
// and place them in a string.
plaintext = srDecrypt.ReadToEnd();
srDecrypt.Close();
}
}
}
}
finally
{
// Clear the RijndaelManaged object.
if (aesAlg != null)
aesAlg.Clear();
}
return plaintext;
}
}
OpenSSL encryption failing to decrypt C#
You openssl command is wrong, I assume it should be:
openssl enc -base64 -aes-256-cbc -k 12345678901234567890123456789012 -in test.txt -out cbc64.txt
reading your code I see you made a few wrong assumptions:
- You are using the password as a key and as IV (where did you get that?)
Password is not key - you've assumed the output is pure ciphertext
try to add -p parameter to the openssl enc command and it will display as well salt and derived key and iv
So lets fix that:
- OpenSSL generates random 8 bytes salt
- OpenSSL uses its "magic" function EVP_BytesToKey to derive the key and IV from the passsword and salt, example impl in c#
- OpenSSL output looks like: base64(Salted_{salt}{ciphertext})
Related Topics
Invalid Length for a Base-64 Char Array
Insert Entire Datatable into Database at Once Instead of Row by Row
Difference Between String and Stringbuilder in C#
Getting the Current Tab's Url from Google Chrome Using C#
Usercontrol' Constructor with Parameters in C#
Is It Safe to Check Floating Point Values for Equality to 0
Getting File Names Without Extensions
Test If a Property Is Available on a Dynamic Variable
A Method to Count Occurrences in a List
How to Hide a Process in Task Manager in C#
C#: List All Classes in Assembly
Nullable Type Issue with : Conditional Operator