How to Re-Order Http Headers

Does the order of headers in an HTTP response ever matter?

No, it does not matter for headers with different names. See RFC 2616, section 4.2:

The order in which header fields with differing field names are
received is not significant. However, it is "good practice" to send
general-header fields first, followed by request-header or response-
header fields, and ending with the entity-header fields.

It DOES matter, however, for multiple headers with the same name:

Multiple message-header fields with the same field-name MAY be
present in a message if and only if the entire field-value for that
header field is defined as a comma-separated list [i.e., #(values)].
It MUST be possible to combine the multiple header fields into one
"field-name: field-value" pair, without changing the semantics of the
message, by appending each subsequent field-value to the first, each
separated by a comma. The order in which header fields with the same
field-name are received is therefore significant to the
interpretation of the combined field value, and thus a proxy MUST NOT
change the order of these field values when a message is forwarded.

Strict ordering of HTTP headers in HttpWebrequest

.Net Core

If you set the headers yourself, you can specify the order. When the common headers are added it will find the existing headers instead of appending them:

using System.Net;

namespace ConsoleApp2
{
    class Program
    {
        static void Main(string[] args)
        {
            var request = WebRequest.Create("http://www.google.com");
            request.Headers.Add("Host", "www.google.com");
            // this will be set within GetResponse.
            request.Headers.Add("Connection", "");
            request.Headers.Add("Accept", "*/*");
            request.Headers.Add("User-Agent", "Mozilla/5.0 etc");
            request.GetResponse();
        }
    }
}

Sample Image

Here is an example with HttpClient:

using System.Net.Http;
using System.Threading.Tasks;

namespace ConsoleApp3
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var client = new HttpClient();
            client.DefaultRequestHeaders.Add("Host", "www.google.com");
            client.DefaultRequestHeaders.Add("Connection", "keep-alive");
            client.DefaultRequestHeaders.Add("Accept", "*/*");
            client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 etc");
            await client.GetAsync("http://www.google.com");
            await client.PostAsync("http://www.google.com", new StringContent(""));
        }
    }
}

GET with ordered headers
POST with ordered headers

Edit
The above code did not work on .Net Framework only .Net Core

.Net Framework

On .Net Framework the headers are reserved so they cannot be set like this, see Cannot set some HTTP headers when using System.Net.WebRequest.

One work around is to use reflection to modify the behavior of the framework class, but be warned this could break if the libraries are updated so it's not recommended!.

Essentially, HttpWebRequest calls ToString on WebHeaderCollection to serialize.
See https://referencesource.microsoft.com/#System/net/System/Net/HttpWebRequest.cs,5079

So a custom class can be made to override ToString. Unfortunately reflection is needed to set the headers as WebRequest copies the collection on assignment to Headers, instead of taking the new reference.

WARNING, THE FOLLOWING CODE CAN BREAK IF FRAMEWORK CHANGES

If you use this, write some unit tests that verify the behavior still stays consistent after updates to .NET Framework

using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Reflection;

namespace ConsoleApp2
{
    class Program
    {
        static void Main(string[] args)
        {
            // WARNING, CODE CAN BREAK IF FRAMEWORK CHANGES
            // If you use this, write some unit tests that verify the behavior still stays consistent after updates to .NET Framework
            var request = (HttpWebRequest)WebRequest.Create("http://www.google.com");
            var field = typeof(HttpWebRequest).GetField("_HttpRequestHeaders", BindingFlags.Instance | BindingFlags.NonPublic);
            var headers = new CustomWebHeaderCollection(new Dictionary<string, string>
            {
                ["Host"] = "www.google.com",
                ["Connection"] = "keep-alive",
                ["Accept"] = "*/*",
                ["User-Agent"] = "Mozilla/5.0 etc"
            });
            field.SetValue(request, headers);
            request.GetResponse();
        }
    }

    internal class CustomWebHeaderCollection : WebHeaderCollection
    {
        private readonly Dictionary<string, string> _customHeaders;

        public CustomWebHeaderCollection(Dictionary<string, string> customHeaders)
        {
            _customHeaders = customHeaders;
        }

        public override string ToString()
        {
            // Could call base.ToString() split on Newline and sort as needed

            var lines = _customHeaders
                .Select(kvp => $"{kvp.Key}: {kvp.Value}")
                // These two new lines are needed after the HTTP header
                .Concat(new [] { string.Empty, string.Empty });

            var headers = string.Join("\r\n", lines);

            return headers;
        }
    }
}

Sample Image

Is the order of webrequest Headers important?

The order of HTTP Headers doesn't matter for headers with different names. If there are multiple headers with the same name, however, the order is important.

See RFC 2616

The order in which header fields with differing field names are
received is not significant. However, it is "good practice" to send
general-header fields first, followed by request-header or response-
header fields, and ending with the entity-header fields.

Multiple message-header fields with the same field-name MAY be
present in a message if and only if the entire field-value for that
header field is defined as a comma-separated list [i.e., #(values)].
It MUST be possible to combine the multiple header fields into one
"field-name: field-value" pair, without changing the semantics of the
message, by appending each subsequent field-value to the first, each
separated by a comma. The order in which header fields with the same
field-name are received is therefore significant to the
interpretation of the combined field value, and thus a proxy MUST NOT
change the order of these field values when a message is forwarded.

Execution order of Http Response headers?

You're right to be skeptical.

There's no requirement that a client wait until the response body is complete to evaluate the Set-Cookie header that preceded the body, and there's in fact good reason to believe that most browsers will set the cookie before the body is complete (since many web pages will look at document.cookie in JavaScript inside a HTML page).

In fact, I tested this (using a MeddlerScript you can see here: http://pastebin.com/SUwCFyxS) and found that IE, Chrome and Firefox all set the cookie before the download completes, and set the cookie even if the user hits "Cancel" on the download.

The HTTP specification includes the notion of a Trailer (which is a header that appears after the response body) but these are little used and not supported in many clients (e.g. WinINET/IE). If the client did support Trailers, the server could send the Set-Cookie header after the body which would mean that the client couldn't see it until the body finished downloading.

How to order the HTTP headers sent by HttpClient or WebRequest?

Here is my temporary workaround:

Create a local proxy and pass the HttpClient request through it. Inside the proxy I reorder the headers as I want them.

The local proxy I am using is the Open Source C# Titanium Web Proxy, which works just fine for the purpose. It supports .NET Standard 2.0, so it is cross platform.

Below are code examples of a test-proxy setup and a client that uses the proxy. You'll have to do your own implementation of the event handlers.

Client:

using System.Net;
using System.Net.Http;
using System.Threading.Tasks;

namespace TestClient
{
    class Program
    {
        static void Main(string[] args)
        {
            var handler = new HttpClientHandler
            {
                Proxy = new WebProxy($"http://localhost:8000", false),
                UseProxy = true,
            };

            var client = new System.Net.Http.HttpClient(handler);

            var t  = Task.Run(() => GetDataAsync(client, "http://google.com/"));
            t.Wait();
            var result = t.Result;
        }

        static async Task<string> GetDataAsync(System.Net.Http.HttpClient client, string path)
        {
            HttpResponseMessage response = await client.GetAsync(path);
            if (response.IsSuccessStatusCode)
            {
                return await response.Content.ReadAsStringAsync();
            }
            else
            {
                return null;
            }
        }
    }
}

Proxy server setup:

using System;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Titanium.Web.Proxy;
using Titanium.Web.Proxy.EventArguments;
using Titanium.Web.Proxy.Models;

namespace ClassLibrary6
{
    public class ProxySetup
    {
        public ProxySetup()
        {
            var proxyServer = new ProxyServer();

            //locally trust root certificate used by this proxy 
            proxyServer.TrustRootCertificate = true;

            //optionally set the Certificate Engine
            //Under Mono only BouncyCastle will be supported
            //proxyServer.CertificateEngine = Network.CertificateEngine.BouncyCastle;

            proxyServer.BeforeRequest += OnRequest;
            proxyServer.BeforeResponse += OnResponse;
            proxyServer.ServerCertificateValidationCallback += OnCertificateValidation;
            proxyServer.ClientCertificateSelectionCallback += OnCertificateSelection;

            var explicitEndPoint = new ExplicitProxyEndPoint(IPAddress.Any, 8000, true)
            {
                //Exclude HTTPS addresses you don't want to proxy
                //Useful for clients that use certificate pinning
                //for example dropbox.com
                // ExcludedHttpsHostNameRegex = new List<string>() { "google.com", "dropbox.com" }

                //Use self-issued generic certificate on all HTTPS requests
                //Optimizes performance by not creating a certificate for each HTTPS-enabled domain
                //Useful when certificate trust is not required by proxy clients
                // GenericCertificate = new X509Certificate2(Path.Combine(System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location), "genericcert.pfx"), "password")
            };

            //An explicit endpoint is where the client knows about the existence of a proxy
            //So client sends request in a proxy friendly manner
            proxyServer.AddEndPoint(explicitEndPoint);
            proxyServer.Start();

            //Transparent endpoint is useful for reverse proxy (client is not aware of the existence of proxy)
            //A transparent endpoint usually requires a network router port forwarding HTTP(S) packets or DNS
            //to send data to this endPoint
            var transparentEndPoint = new TransparentProxyEndPoint(IPAddress.Any, 8001, true)
            {
                //Generic Certificate hostname to use
                //when SNI is disabled by client
                GenericCertificateName = "google.com"
            };

            proxyServer.AddEndPoint(transparentEndPoint);

            //proxyServer.UpStreamHttpProxy = new ExternalProxy() { HostName = "localhost", Port = 8888 };
            //proxyServer.UpStreamHttpsProxy = new ExternalProxy() { HostName = "localhost", Port = 8888 };

            foreach (var endPoint in proxyServer.ProxyEndPoints)
                Console.WriteLine("Listening on '{0}' endpoint at Ip {1} and port: {2} ",
                    endPoint.GetType().Name, endPoint.IpAddress, endPoint.Port);

            //Only explicit proxies can be set as system proxy!
            //proxyServer.SetAsSystemHttpProxy(explicitEndPoint);
            //proxyServer.SetAsSystemHttpsProxy(explicitEndPoint);

            //wait here (You can use something else as a wait function, I am using this as a demo)
            Console.Read();

            //Unsubscribe & Quit
            proxyServer.BeforeRequest -= OnRequest;
            proxyServer.BeforeResponse -= OnResponse;
            proxyServer.ServerCertificateValidationCallback -= OnCertificateValidation;
            proxyServer.ClientCertificateSelectionCallback -= OnCertificateSelection;

            proxyServer.Stop();
        }

        public async Task OnRequest(object sender, SessionEventArgs e)
        {
            e.WebSession.Request.RequestHeaders.AddHeader("Ab", "Moo");

            var headers = e.WebSession.Request.RequestHeaders.GetAllHeaders();

            var ordered = headers.OrderBy(x => x.Name);

            e.WebSession.Request.RequestHeaders.Clear();

            foreach (var httpHeader in ordered)
            {
                e.WebSession.Request.RequestHeaders.AddHeader(httpHeader);
            }

        }

        public async Task OnResponse(object sender, SessionEventArgs e)
        {
        }

        public Task OnCertificateValidation(object sender, CertificateValidationEventArgs e)
        {
            return null;
        }

        public Task OnCertificateSelection(object sender, CertificateSelectionEventArgs e)
        {
            return null;
        }
    }
}