Url Encoding a String

Java URL encoding of query string parameters

URLEncoder is the way to go. You only need to keep in mind to encode only the individual query string parameter name and/or value, not the entire URL, for sure not the query string parameter separator character & nor the parameter name-value separator character =.

String q = "random word £500 bank $";
String url = "https://example.com?q=" + URLEncoder.encode(q, StandardCharsets.UTF_8);

When you're still not on Java 10 or newer, then use StandardCharsets.UTF_8.toString() as charset argument, or when you're still not on Java 7 or newer, then use "UTF-8".

---

Note that spaces in query parameters are represented by +, not %20, which is legitimately valid. The %20 is usually to be used to represent spaces in URI itself (the part before the URI-query string separator character ?), not in query string (the part after ?).

Also note that there are three encode() methods. One without Charset as second argument and another with String as second argument which throws a checked exception. The one without Charset argument is deprecated. Never use it and always specify the Charset argument. The javadoc even explicitly recommends to use the UTF-8 encoding, as mandated by RFC3986 and W3C.

All other characters are unsafe and are first converted into one or more bytes using some encoding scheme. Then each byte is represented by the 3-character string "%xy", where xy is the two-digit hexadecimal representation of the byte. The recommended encoding scheme to use is UTF-8. However, for compatibility reasons, if an encoding is not specified, then the default encoding of the platform is used.

See also:

• What every web developer must know about URL encoding

How to know whether a string is url encoded in iOS?

Check the accepted answer from:
How to find out if string has already been URL encoded?

It says:
Decode, compare to original. If it does differ, original is encoded. If it doesn't differ, original isn't encoded. But still it says nothing about whether the newly decoded version isn't still encoded. A good task for recursion.

Since, you are working on your own API response and the url string will be either encoded or plain text; you can just decode once and compare with the original string.

Straight-forward:
Decode and check if if it matches with original string.

1. match - use the original string to check for valid url using regexp
2. not a match - use the decoded string for valid url using regexp.

How to URL encode strings in C#

According to RFC 1738:

Thus, only alphanumerics, the special characters "$-_.+!*'(),", and
reserved characters used for their reserved purposes may be used
unencoded within a URL.

Neither HttpUtility.UrlEncode nor WebUtility.UrlEncode will encode those characters since the standard says the parentheses () can be used unencoded.

I don't know why the URL Encoder / Decoder you linked encodes them since it also lists them as as a character that can be used in a URL.

Swift - encode URL

Swift 3

In Swift 3 there is addingPercentEncoding

let originalString = "test/test"
let escapedString = originalString.addingPercentEncoding(withAllowedCharacters: .urlHostAllowed)
print(escapedString!)

Output:

test%2Ftest

Swift 1

In iOS 7 and above there is stringByAddingPercentEncodingWithAllowedCharacters

var originalString = "test/test"
var escapedString = originalString.stringByAddingPercentEncodingWithAllowedCharacters(.URLHostAllowedCharacterSet())
println("escapedString: \(escapedString)")

Output:

test%2Ftest

The following are useful (inverted) character sets:

URLFragmentAllowedCharacterSet  "#%<>[\]^`{|}
URLHostAllowedCharacterSet      "#%/<>?@\^`{|}
URLPasswordAllowedCharacterSet  "#%/:<>?@[\]^`{|}
URLPathAllowedCharacterSet      "#%;<>?[\]^`{|}
URLQueryAllowedCharacterSet     "#%<>[\]^`{|}
URLUserAllowedCharacterSet      "#%/:<>?@[\]^`

If you want a different set of characters to be escaped create a set:

Example with added "=" character:

var originalString = "test/test=42"
var customAllowedSet =  NSCharacterSet(charactersInString:"=\"#%/<>?@\\^`{|}").invertedSet
var escapedString = originalString.stringByAddingPercentEncodingWithAllowedCharacters(customAllowedSet)
println("escapedString: \(escapedString)")

Output:

test%2Ftest%3D42

Example to verify ascii characters not in the set:

func printCharactersInSet(set: NSCharacterSet) {
    var characters = ""
    let iSet = set.invertedSet
    for i: UInt32 in 32..<127 {
        let c = Character(UnicodeScalar(i))
        if iSet.longCharacterIsMember(i) {
            characters = characters + String(c)
        }
    }
    print("characters not in set: \'\(characters)\'")
}

Should I url encode a query string parameter that's a URL?

According to RFC 3986:

The query component is indicated by the first question mark ("?")
character and terminated by a number sign ("#") character or by the
end of the URI.

So the following URI is valid:

http://www.example.com?next=http://www.example.com

The following excerpt from the RFC makes this clear:

... as query components are often used to carry identifying
information in the form of "key=value" pairs and one frequently used
value is a reference to another URI, it is sometimes better for
usability to avoid percent-encoding those characters.

It is worth noting that RFC 3986 makes RFC 2396 obsolete.

How to urlencode data for curl command?

Use curl --data-urlencode; from man curl:

This posts data, similar to the other --data options with the exception that this performs URL-encoding. To be CGI-compliant, the <data> part should begin with a name followed by a separator and a content specification.

Example usage:

curl \
    --data-urlencode "paramName=value" \
    --data-urlencode "secondParam=value" \
    http://example.com

See the man page for more info.

This requires curl 7.18.0 or newer (released January 2008). Use curl -V to check which version you have.

You can as well encode the query string:

curl --get \
    --data-urlencode "p1=value 1" \
    --data-urlencode "p2=value 2" \
    http://example.com
    # http://example.com?p1=value%201&p2=value%202

URL Encoding of String accepting all special characters in swift

Thank you @MartinR and @benleggiero for : How do I URL encode a string

It helped a lot.

It was not including all special characters.

Checked one by one and added those which were missing as below:

extension CharacterSet {

    public static let urlQueryParameterAllowed = CharacterSet.urlQueryAllowed.subtracting(CharacterSet(charactersIn: "&?~!$*(.,)_-+':"))

    public static let urlQueryDenied           = CharacterSet.urlQueryAllowed.inverted()
    public static let urlQueryKeyValueDenied   = CharacterSet.urlQueryParameterAllowed.inverted()
    public static let urlPathDenied            = CharacterSet.urlPathAllowed.inverted()
    public static let urlFragmentDenied        = CharacterSet.urlFragmentAllowed.inverted()
    public static let urlHostDenied            = CharacterSet.urlHostAllowed.inverted()

    public static let urlDenied                = CharacterSet.urlQueryDenied
        .union(.urlQueryKeyValueDenied)
        .union(.urlPathDenied)
        .union(.urlFragmentDenied)
        .union(.urlHostDenied)

    public func inverted() -> CharacterSet {
        var copy = self
        copy.invert()
        return copy
    }
}

URL Encoding—Ampersand Problem

It looks like the field is being encoded twice. First pass will result in & changed into %26, then urlencoding %26 will result in %2526, since the encoding for % itself is %25.

---

Related Topics