.Net Core MAChine Key Alternative for Webfarm

.Net Core Machine Key alternative for webfarm

I've made some tests to back up my comment about copying keys. First I created simple console application with the following code:

var serviceCollection = new ServiceCollection();
serviceCollection.AddDataProtection()
    .SetApplicationName("my-app")
    .PersistKeysToFileSystem(new DirectoryInfo(@"G:\tmp\so\keys"));
var services = serviceCollection.BuildServiceProvider();
var provider = services.GetService<IDataProtectionProvider>();
var protector = provider.CreateProtector("some_purpose");                
Console.WriteLine(Convert.ToBase64String(protector.Protect(Encoding.UTF8.GetBytes("hello world"))));

So, just create DI container, register data protection there with specific folder for keys, resolve and protect something.

This generated the following key file in target folder:

<?xml version="1.0" encoding="utf-8"?>
<key id="e6cbce11-9afd-43e6-94be-3f6057cb8a87" version="1">
  <creationDate>2017-04-10T15:28:18.0565235Z</creationDate>
  <activationDate>2017-04-10T15:28:18.0144946Z</activationDate>
  <expirationDate>2017-07-09T15:28:18.0144946Z</expirationDate>
  <descriptor deserializerType="Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption.ConfigurationModel.AuthenticatedEncryptorDescriptorDeserializer, Microsoft.AspNetCore.DataProtection, Version=1.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60">
    <descriptor>
      <encryption algorithm="AES_256_CBC" />
      <validation algorithm="HMACSHA256" />
      <masterKey p4:requiresEncryption="true" xmlns:p4="http://schemas.asp.net/2015/03/dataProtection">
        <!-- Warning: the key below is in an unencrypted form. -->
        <value>rVDib1M1BjbCqGctcP+N25zb+Xli9VWX46Y7+9tsoGywGnIg4p9K5QTM+c388i0mC0JBSLaFS2pZBRdR49hsLQ==</value>
      </masterKey>
    </descriptor>
  </descriptor>
</key>

As you see, file is relatively simple. It states creation, activation, expiration dates, algorithms used, reference to deserializer class and of course key itself.

Now I configured asp.net application (so, another application, not that console one) like this:

services.AddDataProtection()
    .SetApplicationName("my-app")
    .PersistKeysToFileSystem(new DirectoryInfo(@"G:\tmp\so\keys-asp"))
    .DisableAutomaticKeyGeneration();

If you now try to run application and do something that requires protection - it will fail, because there no keys and automatic key generation is disabled. However, if I copy keys generated by console app to the target folder - it will
happily use them.

So pay attention to the usual security concerns with copying keys, to expiration time of those keys (configurable with SetDefaultKeyLifetime) and using the same version of Microsoft.AspNetCore.DataProtection in all applications you share keys with (because it's version is specified in key xml file) - and you should be fine. It's better to generate your shared keys in one place and in all other places set DisableAutomaticKeyGeneration.

Adding machineKey to web.config on web-farm sites

This should answer:

How To: Configure MachineKey in ASP.NET 2.0 - Web Farm Deployment Considerations

Web Farm Deployment Considerations

If you deploy your application in a Web farm, you must ensure that the
configuration files on each server share the same value for
validationKey and decryptionKey, which are used for hashing and
decryption respectively. This is required because you cannot guarantee
which server will handle successive requests.

With manually generated key values, the settings should
be similar to the following example.

<machineKey  
validationKey="21F090935F6E49C2C797F69BBAAD8402ABD2EE0B667A8B44EA7DD4374267A75D7
               AD972A119482D15A4127461DB1DC347C1A63AE5F1CCFAACFF1B72A7F0A281B"       

decryptionKey="ABAA84D7EC4BB56D75D217CECFFB9628809BDB8BF91CFCD64568A145BE59719F"
validation="SHA1"
decryption="AES"
/>

If you want to isolate your application from other applications on the
same server, place the in the Web.config file for each
application on each server in the farm. Ensure that you use separate
key values for each application, but duplicate each application's keys
across all servers in the farm.

In short, to set up the machine key refer the following link:
Setting Up a Machine Key - Orchard Documentation.

Setting Up the Machine Key Using IIS Manager

If you have access to the IIS management console for the server where
Orchard is installed, it is the easiest way to set-up a machine key.

Start the management console and then select the web site. Open the
machine key configuration:

The machine key control panel has the following settings:

Uncheck "Automatically generate at runtime" for both the validation
key and the decryption key.

Click "Generate Keys" under "Actions" on the right side of the panel.

Click "Apply".

and add the following line to the web.config file in all the webservers under system.web tag if it does not exist.

<machineKey  
    validationKey="21F0SAMPLEKEY9C2C797F69BBAAD8402ABD2EE0B667A8B44EA7DD4374267A75D7
                   AD972A119482D15A4127461DB1DC347C1A63AE5F1CCFAACFF1B72A7F0A281B"           
    decryptionKey="ABAASAMPLEKEY56D75D217CECFFB9628809BDB8BF91CFCD64568A145BE59719F"
    validation="SHA1"
    decryption="AES"
/>

Please make sure that you have a permanent backup of the machine keys and web.config file

Using machine keys for IDataProtector - ASP.NET CORE

I only needed the MachineKey.UnProtect function. I could not get anything to work with the APIs from ASP.NET CORE so I had no choice but to stitch up the source code from the .net Framework. The following code ended up working for me to unprotect something.

public static class MachineKey
    {
        private static readonly UTF8Encoding SecureUTF8Encoding = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true);
        public static byte[] Unprotect(byte[] protectedData, string validationKey, string encKey, params string[] specificPurposes)
        {
            // The entire operation is wrapped in a 'checked' block because any overflows should be treated as failures.
            checked
            {
                using (SymmetricAlgorithm decryptionAlgorithm = new AesCryptoServiceProvider())
                {

                    decryptionAlgorithm.Key = SP800_108.DeriveKey(HexToBinary(encKey), "User.MachineKey.Protect", specificPurposes);

                    // These KeyedHashAlgorithm instances are single-use; we wrap it in a 'using' block.
                    using (KeyedHashAlgorithm validationAlgorithm = new HMACSHA256())
                    {
                        validationAlgorithm.Key = SP800_108.DeriveKey(HexToBinary(validationKey), "User.MachineKey.Protect", specificPurposes);

                        int ivByteCount = decryptionAlgorithm.BlockSize / 8; 
                        int signatureByteCount = validationAlgorithm.HashSize / 8;
                        int encryptedPayloadByteCount = protectedData.Length - ivByteCount - signatureByteCount;
                        if (encryptedPayloadByteCount <= 0)
                        {
                            return null;
                        }

                        byte[] computedSignature = validationAlgorithm.ComputeHash(protectedData, 0, ivByteCount + encryptedPayloadByteCount);

                        if (!BuffersAreEqual(
                            buffer1: protectedData, buffer1Offset: ivByteCount + encryptedPayloadByteCount, buffer1Count: signatureByteCount,
                            buffer2: computedSignature, buffer2Offset: 0, buffer2Count: computedSignature.Length))
                        {

                            return null;
                        }

                        byte[] iv = new byte[ivByteCount];
                        Buffer.BlockCopy(protectedData, 0, iv, 0, iv.Length);
                        decryptionAlgorithm.IV = iv;

                        using (MemoryStream memStream = new MemoryStream())
                        {
                            using (ICryptoTransform decryptor = decryptionAlgorithm.CreateDecryptor())
                            {
                                using (CryptoStream cryptoStream = new CryptoStream(memStream, decryptor, CryptoStreamMode.Write))
                                {
                                    cryptoStream.Write(protectedData, ivByteCount, encryptedPayloadByteCount);
                                    cryptoStream.FlushFinalBlock();

                                    byte[] clearData = memStream.ToArray();

                                    return clearData;
                                }
                            }
                        }
                    }
                }
            }
        }

        private static bool BuffersAreEqual(byte[] buffer1, int buffer1Offset, int buffer1Count, byte[] buffer2, int buffer2Offset, int buffer2Count)
        {
            bool success = (buffer1Count == buffer2Count); // can't possibly be successful if the buffers are of different lengths
            for (int i = 0; i < buffer1Count; i++)
            {
                success &= (buffer1[buffer1Offset + i] == buffer2[buffer2Offset + (i % buffer2Count)]);
            }
            return success;
        }

        private static class SP800_108
        {

            public static byte[] DeriveKey(byte[] keyDerivationKey, string primaryPurpose, params string[] specificPurposes)
            {
                using (HMACSHA512 hmac = new HMACSHA512(keyDerivationKey))
                {

                    GetKeyDerivationParameters(out byte[] label, out byte[] context, primaryPurpose, specificPurposes);

                    byte[] derivedKey = DeriveKeyImpl(hmac, label, context, keyDerivationKey.Length * 8);

                    return derivedKey;
                }
            }

            private static byte[] DeriveKeyImpl(HMAC hmac, byte[] label, byte[] context, int keyLengthInBits)
            {
                checked
                {
                    int labelLength = (label != null) ? label.Length : 0;
                    int contextLength = (context != null) ? context.Length : 0;
                    byte[] buffer = new byte[4 /* [i]_2 */ + labelLength /* label */ + 1 /* 0x00 */ + contextLength /* context */ + 4 /* [L]_2 */];

                    if (labelLength != 0)
                    {
                        Buffer.BlockCopy(label, 0, buffer, 4, labelLength); // the 4 accounts for the [i]_2 length
                    }
                    if (contextLength != 0)
                    {
                        Buffer.BlockCopy(context, 0, buffer, 5 + labelLength, contextLength); // the '5 +' accounts for the [i]_2 length, the label, and the 0x00 byte
                    }
                    WriteUInt32ToByteArrayBigEndian((uint)keyLengthInBits, buffer, 5 + labelLength + contextLength); // the '5 +' accounts for the [i]_2 length, the label, the 0x00 byte, and the context

                    int numBytesWritten = 0;
                    int numBytesRemaining = keyLengthInBits / 8;
                    byte[] output = new byte[numBytesRemaining];

                    for (uint i = 1; numBytesRemaining > 0; i++)
                    {
                        WriteUInt32ToByteArrayBigEndian(i, buffer, 0); // set the first 32 bits of the buffer to be the current iteration value
                        byte[] K_i = hmac.ComputeHash(buffer);

                        // copy the leftmost bits of K_i into the output buffer
                        int numBytesToCopy = Math.Min(numBytesRemaining, K_i.Length);
                        Buffer.BlockCopy(K_i, 0, output, numBytesWritten, numBytesToCopy);
                        numBytesWritten += numBytesToCopy;
                        numBytesRemaining -= numBytesToCopy;
                    }

                    // finished
                    return output;
                }
            }

            private static void WriteUInt32ToByteArrayBigEndian(uint value, byte[] buffer, int offset)
            {
                buffer[offset + 0] = (byte)(value >> 24);
                buffer[offset + 1] = (byte)(value >> 16);
                buffer[offset + 2] = (byte)(value >> 8);
                buffer[offset + 3] = (byte)(value);
            }

        }

        private static void GetKeyDerivationParameters(out byte[] label, out byte[] context, string primaryPurpose, params string[] specificPurposes)
        {
            label = SecureUTF8Encoding.GetBytes(primaryPurpose);

                using (MemoryStream stream = new MemoryStream())
                using (BinaryWriter writer = new BinaryWriter(stream, SecureUTF8Encoding))
                {
                    foreach (string specificPurpose in specificPurposes)
                    {
                        writer.Write(specificPurpose);
                    }
                    context = stream.ToArray();
                }
        }

        private static byte[] HexToBinary(string data)
        {
            if (data == null || data.Length % 2 != 0)
            {
                // input string length is not evenly divisible by 2
                return null;
            }

            byte[] binary = new byte[data.Length / 2];

            for (int i = 0; i < binary.Length; i++)
            {
                int highNibble = HexToInt(data[2 * i]);
                int lowNibble = HexToInt(data[2 * i + 1]);

                if (highNibble == -1 || lowNibble == -1)
                {
                    return null; // bad hex data
                }
                binary[i] = (byte)((highNibble << 4) | lowNibble);
            }

            int HexToInt(char h)
            {
                return (h >= '0' && h <= '9') ? h - '0' :
                (h >= 'a' && h <= 'f') ? h - 'a' + 10 :
                (h >= 'A' && h <= 'F') ? h - 'A' + 10 :
                -1;
            }
            return binary;
        }

    } 

[EXAMPLE]

var message = "My secret message";

var encodedMessage = Encoding.ASCII.GetBytes(message);

var protectedMessage = MachineKey.Protect(encodedMessage, "My Purpose");

var protectedMessageAsBase64 = Convert.ToBase64String(protectedMessage);

// Now make sure you reverse the process 

var convertFromBase64 = Convert.FromBase64String(protectedMessageAsBase64);

var unProtectedMessage = MachineKey.Unprotect(convertFromBase64, "Your validation key", "Your encryption key", "My Purpose");

var decodedMessage = Encoding.ASCII.GetString(unProtectedMessage);

This is just a simple example. First, make sure you have the correct validation and encryption keys from IIS. This may seem like an obvious point but it drove me mad because I was using the wrong keys. Next, make sure you know what purpose the message was enrypted with. In my Example, the purpose is "My purpose". If the message was encrypted without a purpose, just leave the purpose paramter out when you unprotect something. Finally, you have to know how your encrypted message has been presented to you. Is it base64 encoded, for example, you need to know this so you can do the reverse.

How does one distribute Data Protection keys with a .NET Core web app?

Answering my own question. The following seems to work across a web farm. Call the method below from Startup.ConfigureServices. It assumes that the key (that was generated on a dev machine) resides in the Keys folder off the root of the project.

public Startup(IHostingEnvironment env)
{
    /* skipping boilerplate setup code */

    Environment = env;
}

private IHostingEnvironment Environment { get; set; }

private void ConfigureDataProtection(IServiceCollection services) {
    // get the file from the Keys directory
    string keysFile = string.Empty;
    string keysPath = Path.Combine(Environment.ContentRootPath, "Keys");
    if (!Directory.Exists(keysPath)) {
        Log.Add($"Keys directory {keysPath} doesn't exist");
        return;
    }

    string[] files = Directory.GetFiles(keysPath);
    if (files.Length == 0) {
        LLog.Add($"No keys found in directory {keysPath}");
        return;
    } else {
        keysFile = files[0];

        if (files.Length >= 2) {
            LLog.Add($"Warning: More than 1 key found in directory {keysPath}.  Using first one.");
        }
    }

    // find and optionally create the path for the key storage
    var appDataPath = Path.Combine(
        System.Environment.GetFolderPath(System.Environment.SpecialFolder.CommonApplicationData),
        "companyName",
        "productName"
    );

    if (!Directory.Exists(appDataPath)) {
        try {
            Directory.CreateDirectory(appDataPath);
        } catch (Exception ex) {
            Log.Add($"Error creating key storage folder at {appDataPath}.  Error: {ex.Message}");
            return;
        }
    }

    // delete any keys from the storage directory
    try {
        foreach (var file in new DirectoryInfo(appDataPath).GetFiles()) file.Delete();
    } catch (Exception ex) {
        Log.Add($"Error deleting keys from {appDataPath}.  Error: {ex.Message}");
        return;
    }

    try {
        string targetPath = Path.Combine(appDataPath, new FileInfo(keysFile).Name);
        File.Copy(keysFile, targetPath, true);
    } catch (Exception ex) {
        Log.Add($"Error copying key file {keysFile} to {appDataPath}.  Error: {ex.Message}");
        return;
    }

    // everything is in place
    services.AddDataProtection()
        .PersistKeysToFileSystem(new DirectoryInfo(appDataPath))
        .DisableAutomaticKeyGeneration();
}

ASP.NET machinekey set keys in code

There is no way to set this programmatically once the web application starts. However, it is still possible to accomplish your goal.

If each application is running in its own application pool, and if each application pool has its own identity, then check out the CLRConfigFile switch in applicationHost.config. You can use this per-application pool to inject a new level of configuration. See http://weblogs.asp.net/owscott/archive/2011/12/01/setting-an-aspnet-config-file-per-application-pool.aspx for an example of how to use this. You could set an explicit and unique <system.web/machineKey> element in each application pool's custom CLR config file.

This is the same mechanism used by Azure Web Sites, GoDaddy, and other hosters that need to set default explicit machine keys on a per-application basis. Remember to ACL each target .config file appropriately for the application pool which will be accessing it.

MVC 6 WebFarm: The antiforgery token could not be decrypted

Explanation

You'll need to reuse the same key.

If you are on Azure, the keys are synced by NAS-type storage on %HOME%\ASP.NET\DataProtection-Keys.

For locally run application, they are stored in the %LOCALAPPDATA%\ASP.NET\DataProtection-Keys of the user running the application or stored in the registry if it's being executed in IIS.

If none of the above match, the key is generated for the lifetime of the process.

Solution

So the first option is not available (Azure only). However, you could sync the keys from %LOCALAPPDATA%\ASP.NET\DataProtection-Keys of the user running your application on each machine running your application.

But even better, you could just point it to a network share like this:

sc.ConfigureDataProtection(configure =>
{
    // persist keys to a specific directory
    configure.PersistKeysToFileSystem(new DirectoryInfo(@"Z:\temp-keys\"));
});

This will allow you to scale while keeping your security.

Important: Your keys will expire every 90 days. It will be important to regenerate them frequently.

You can change it using this bit of code but the shorter, the safer you are.

services.ConfigureDataProtection(configure =>
{
    // use 14-day lifetime instead of 90-day lifetime
    configure.SetDefaultKeyLifetime(TimeSpan.FromDays(14));
});

Source

• Key Encryption at Rest
• Default Settings

---

Related Topics