Encoding Parameters for a Url

How to encode URL parameters?

With PHP

echo urlencode("http://www.image.com/?username=unknown&password=unknown");

Result

http%3A%2F%2Fwww.image.com%2F%3Fusername%3Dunknown%26password%3Dunknown

With Javascript:

var myUrl = "http://www.image.com/?username=unknown&password=unknown";
var encodedURL= "http://www.foobar.com/foo?imageurl=" + encodeURIComponent(myUrl);

DEMO: http://jsfiddle.net/Lpv53/

Encoding parameters for a URL

I would recommend Uri.EscapeDataString instead of using HttpUtility functions. See discussion in Server.UrlEncode vs. HttpUtility.UrlEncode.

How can I correctly encode query parameters in a url?

When dealing with HttpClient related tasks, A better approch is to use some handly library like RestSharp.

Using RestSharp NuGet library (https://www.nuget.org/packages/RestSharp) - You don't need to worry about URL encoding, JSON parsing and a lot more

var client = new RestClient("https://countries.com");
var request = new RestRequest("GetCountry", Method.GET);
// As you mentioned, If you do no need to encode URL parameters. Add the encoding off option
request.AddQueryParameter("countryName", "India", ParameterType.QueryStringWithoutEncode);  
request.AddQueryParameter("cityName", "Kochi", ParameterType.QueryStringWithoutEncode);  
var response = client.Execute(request);

This will give

https://countries.com/GetCountry?countryName=India&cityName=Kochi

URL encoding is automaticaly handled. You don't need to worry about it

Encoding URL query parameters in Java

java.net.URLEncoder.encode(String s, String encoding) can help too. It follows the HTML form encoding application/x-www-form-urlencoded.

URLEncoder.encode(query, "UTF-8");

On the other hand, Percent-encoding (also known as URL encoding) encodes space with %20. Colon is a reserved character, so : will still remain a colon, after encoding.

Should I url encode a query string parameter that's a URL?

According to RFC 3986:

The query component is indicated by the first question mark ("?")
character and terminated by a number sign ("#") character or by the
end of the URI.

So the following URI is valid:

http://www.example.com?next=http://www.example.com

The following excerpt from the RFC makes this clear:

... as query components are often used to carry identifying
information in the form of "key=value" pairs and one frequently used
value is a reference to another URI, it is sometimes better for
usability to avoid percent-encoding those characters.

It is worth noting that RFC 3986 makes RFC 2396 obsolete.

Java URL encoding of query string parameters

URLEncoder is the way to go. You only need to keep in mind to encode only the individual query string parameter name and/or value, not the entire URL, for sure not the query string parameter separator character & nor the parameter name-value separator character =.

String q = "random word £500 bank $";
String url = "https://example.com?q=" + URLEncoder.encode(q, StandardCharsets.UTF_8);

When you're still not on Java 10 or newer, then use StandardCharsets.UTF_8.toString() as charset argument, or when you're still not on Java 7 or newer, then use "UTF-8".

Note that spaces in query parameters are represented by +, not %20, which is legitimately valid. The %20 is usually to be used to represent spaces in URI itself (the part before the URI-query string separator character ?), not in query string (the part after ?).

Also note that there are three encode() methods. One without Charset as second argument and another with String as second argument which throws a checked exception. The one without Charset argument is deprecated. Never use it and always specify the Charset argument. The javadoc even explicitly recommends to use the UTF-8 encoding, as mandated by RFC3986 and W3C.

All other characters are unsafe and are first converted into one or more bytes using some encoding scheme. Then each byte is represented by the 3-character string "%xy", where xy is the two-digit hexadecimal representation of the byte. The recommended encoding scheme to use is UTF-8. However, for compatibility reasons, if an encoding is not specified, then the default encoding of the platform is used.

Encoding a URL Query Parameter so it can have a '+'

Found what I believe to be a decent solution. It turns out that a large part of the problem is actually the "exchange" function, which takes a string for a URL, but then re-encodes that URL for reasons I cannot fathom. However, the exchange function can be sent a java.net.URI instead. In this case, it does not try to interpolate anything, as it is already a URI. I then use java.net.URLEncoder.encode() to encode the pieces. I still have no idea why this isn't standard in Spring, but this should work.

    private String mapToQueryString(Map<String, String> query) {
        List<String> entries = new LinkedList<String>();
        for (Map.Entry<String, String> entry : query.entrySet()) {
            try {
                entries.add(URLEncoder.encode(entry.getKey(), "UTF-8") + "=" + URLEncoder.encode(entry.getValue(), "UTF-8"));
            } catch(Exception e) {
                log.error("Unable to encode string for URL: " + entry.getKey() + " / " + entry.getValue(), e);
            }
        }
        return String.join("&", entries);
    }

    /* Later in the code */
    String endpoint = "https://baseurl.example.com/blah";
    String finalUrl = query.isEmpty() ? endpoint : endpoint + "?" + mapToQueryString(query);
    URI uri;
    try {
        uri = new URI(finalUrl);
    } catch(URISyntaxException e) {
        log.error("Bad URL // " + finalUrl, e);
            return null;
        }
    }
    /* ... */
    HttpEntity<TheResponse> resp = myRestTemplate.exchange(uri, ...)

How to get a parameter from a URL encoded?

The encoding is happening in your first line already:

string queryString = new Uri(URL).Query;

Presumably you want to avoid writing your own code to extract the Query part from a URL. Which is sensible. You can still rely on the Uri class to do the parsing for you:

var uri=new Uri(URL);
var queryString= URL.Replace(uri.GetLeftPart(UriPartial.Path), "").TrimStart('?');

———————————————————————————————————

(

var URL="http://a/b/?d= @£$%% sdf we  456 7 5?367";
var uri=new Uri(URL);
Console.WriteLine(uri.Query); // Already url-encoded by the Uri constructor
var queryString = URL.Replace(uri.GetLeftPart(UriPartial.Path), "").TrimStart('?');
Console.WriteLine(System.Web.HttpUtility.ParseQueryString(queryString)); //Not encoded!

)

(As an aside, LinqPad helps)

Encode both parameter and value or just value in URL for http post?

URLEncoder.encode is called to ensure that there are no unsafe chars in your URI. an unsafe char is nearly every char which is not a letter, a digit and some special chars.

From the java-doc of the URLEncoder

The alphanumeric characters "a" through "z", "A" through "Z" and "0" through >"9" remain the same.
The special characters ".", "-", "*", and "_" remain the same.
The space character " " is converted into a plus sign "+".
All other characters are unsafe and are first converted into one or more bytes >using some encoding scheme. Then each byte is represented by the 3-character >string "%xy", where xy is the two-digit hexadecimal representation of the byte. >The recommended encoding scheme to use is UTF-8. However, for compatibility >reasons, if an encoding is not specified, then the default encoding of the >platform is used.

Example:

String query = "foo=abc&bar=def";

so. if you encode the whole query it will result in

foo%3Dabc%26bar%3Defg

In this case you also encoded the = and & which are needed to seperate the parts of the query.

You have to encode the names and value of your query to ensure that they do not contain unsafe chars. e.g. &, = and any non printable/special char. if you know, that your name only contains safe chars, you don't have to encode the name.

String param1 = URLEncoder.encode("abc", "utf-8");
String param2 = URLEncoder.encode("a&b", "utf-8");
String query = "foo=" + param1 + "&bar=" + param2;

results in

foo=abc&bar=a%26b

which should be the query you need. please note the & in param2 which will be encoded, cause it is "unsafe" for an valid url!