Objective-C and Swift Url Encoding

Objective-C and Swift URL encoding

To escape the characters you want is a little more work.

Example code

iOS7 and above:

NSString *unescaped = @"http://www";
NSString *escapedString = [unescaped stringByAddingPercentEncodingWithAllowedCharacters:[NSCharacterSet URLHostAllowedCharacterSet]];
NSLog(@"escapedString: %@", escapedString);

NSLog output:

escapedString: http%3A%2F%2Fwww

The following are useful URL encoding character sets:

URLFragmentAllowedCharacterSet  "#%<>[\]^`{|}
URLHostAllowedCharacterSet      "#%/<>?@\^`{|}
URLPasswordAllowedCharacterSet  "#%/:<>?@[\]^`{|}
URLPathAllowedCharacterSet      "#%;<>?[\]^`{|}
URLQueryAllowedCharacterSet     "#%<>[\]^`{|}
URLUserAllowedCharacterSet      "#%/:<>?@[\]^`

Creating a characterset combining all of the above:

NSCharacterSet *URLCombinedCharacterSet = [[NSCharacterSet characterSetWithCharactersInString:@" \"#%/:<>?@[\\]^`{|}"] invertedSet];

Creating a Base64

In the case of Base64 characterset:

NSCharacterSet *URLBase64CharacterSet = [[NSCharacterSet characterSetWithCharactersInString:@"/+=\n"] invertedSet];

For Swift 3.0:

var escapedString = originalString.addingPercentEncoding(withAllowedCharacters:.urlHostAllowed)

For Swift 2.x:

var escapedString = originalString.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.URLHostAllowedCharacterSet())

Note: stringByAddingPercentEncodingWithAllowedCharacters will also encode UTF-8 characters needing encoding.

Pre iOS7 use Core Foundation

Using Core Foundation With ARC:

NSString *escapedString = (NSString *)CFBridgingRelease(CFURLCreateStringByAddingPercentEscapes(
    NULL,
   (__bridge CFStringRef) unescaped,
    NULL,
    CFSTR("!*'();:@&=+$,/?%#[]\" "),
    kCFStringEncodingUTF8));

Using Core Foundation Without ARC:

NSString *escapedString = (NSString *)CFURLCreateStringByAddingPercentEscapes(
    NULL,
   (CFStringRef)unescaped,
    NULL,
    CFSTR("!*'();:@&=+$,/?%#[]\" "),
    kCFStringEncodingUTF8);

Note: -stringByAddingPercentEscapesUsingEncoding will not produce the correct encoding, in this case it will not encode anything returning the same string.

stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding encodes 14 characrters:

`#%^{}[]|\"<> plus the space character as percent escaped.

testString:

" `~!@#$%^&*()_+-={}[]|\\:;\"'<,>.?/AZaz"  

encodedString:

"%20%60~!@%23$%25%5E&*()_+-=%7B%7D%5B%5D%7C%5C:;%22'%3C,%3E.?/AZaz"  

Note: consider if this set of characters meet your needs, if not change them as needed.

RFC 3986 characters requiring encoding (% added since it is the encoding prefix character):

"!#$&'()*+,/:;=?@[]%"

Some "unreserved characters" are additionally encoded:

"\n\r \"%-.<>\^_`{|}~"

Inconsistencies in URL encoding methods across Objective-C and Swift

The issue actually rests in the difference between NSString method stringByAddingPercentEncodingWithAllowedCharacters and String method addingPercentEncoding(withAllowedCharacters:). And this behavior has been changing from version to version. (It looks like the latest beta of iOS 11 now restores this behavior we used to see.)

I believe the root of the issue rests in the particulars of how paths are percent encoded. Section 3.3 of RFC 3986 says that colons are permitted in paths except in the first segment of a relative path.

The NSString method captures this notion, e.g. imagine a path whose first directory was foo: (with a colon) and a subdirectory of bar: (also with a colon):

NSString *string = @"foo:/bar:";
NSCharacterSet *cs = [NSCharacterSet URLPathAllowedCharacterSet];
NSLog(@"%@", [string stringByAddingPercentEncodingWithAllowedCharacters:cs]);

That results in:

foo%3A/bar:

The : in the first segment of the page is percent encoded, but the : in subsequent segments are not. This captures the logic of how to handle colons in relative paths per RFC 3986.

The String method addingPercentEncoding(withAllowedCharacters:), however, does not do this:

let string = "foo:/bar:"
os_log("%@", string.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed)!)

Yields:

foo:/bar:

Clearly, the String method does not attempt that position-sensitive logic. This implementation is more in keeping with the name of the method (it considers solely what characters are "allowed" with no special logic that tries to guess, based upon where the allowed character appears, whether it's truly allowed or not.)

---

I gather that you are saddled with the code supplied in the question, but we should note that this behavior of percent escaping colons in relative paths, while interesting to explain what you experienced, is not really relevant to your immediate problem. The code you have been provided is simply incorrect. It is attempting to percent encode a URL as if it was just a path. But, it’s not a path; it’s a URL, which is a different thing with its own rules.

The deeper insight in percent encoding URLs is to acknowledge that different components of a URL allow different sets of characters, i.e. they require different percent encoding. That’s why NSCharacterSet has so many different URL-related character sets.

You really should percent encode the individual components, percent encoding each with the character set allowed for that type of component. Only when the individual components are percent encoded should they then be concatenated together to form the whole the URL.

Alternatively, NSURLComponents is designed precisely for this purpose, getting you out of the weeds of percent-encoding the individual components yourself. For example:

var components = URLComponents(string: "http://httpbin.org/post")!
let foo = URLQueryItem(name: "foo", value: "bar & baz")
let qux = URLQueryItem(name: "qux", value: "42")
components.queryItems = [foo, qux]

let url = components.url!

That yields the following, with the & and the two spaces properly percent escaped within the foo value, but it correctly left the & in-between foo and qux:

http://httpbin.org/post?foo=bar%20%26%20baz&qux=42

It’s worth noting, though, that NSURLComponents has a small, yet fairly fundamental flaw: Specifically, if you have query values, NSURLQueryItem, that could have + characters, most web services need that percent escaped, but NSURLComponents won’t. If your URL has query components and if those query values might include + characters, I’d advise against NSURLComponents and would instead advise percent encoding the individual components of a URL yourself.

How do I URL encode a string

Unfortunately, stringByAddingPercentEscapesUsingEncoding doesn't always work 100%. It encodes non-URL characters but leaves the reserved characters (like slash / and ampersand &) alone. Apparently this is a bug that Apple is aware of, but since they have not fixed it yet, I have been using this category to url-encode a string:

@implementation NSString (NSString_Extended)

- (NSString *)urlencode {
    NSMutableString *output = [NSMutableString string];
    const unsigned char *source = (const unsigned char *)[self UTF8String];
    int sourceLen = strlen((const char *)source);
    for (int i = 0; i < sourceLen; ++i) {
        const unsigned char thisChar = source[i];
        if (thisChar == ' '){
            [output appendString:@"+"];
        } else if (thisChar == '.' || thisChar == '-' || thisChar == '_' || thisChar == '~' || 
                   (thisChar >= 'a' && thisChar <= 'z') ||
                   (thisChar >= 'A' && thisChar <= 'Z') ||
                   (thisChar >= '0' && thisChar <= '9')) {
            [output appendFormat:@"%c", thisChar];
        } else {
            [output appendFormat:@"%%%02X", thisChar];
        }
    }
    return output;
}

Used like this:

NSString *urlEncodedString = [@"SOME_URL_GOES_HERE" urlencode];

// Or, with an already existing string:
NSString *someUrlString = @"someURL";
NSString *encodedUrlStr = [someUrlString urlencode];

---

This also works:

NSString *encodedString = (NSString *)CFURLCreateStringByAddingPercentEscapes(
                            NULL,
                            (CFStringRef)unencodedString,
                            NULL,
                            (CFStringRef)@"!*'();:@&=+$,/?%#[]",
                            kCFStringEncodingUTF8 );

---

Some good reading about the subject:

Objective-c iPhone percent encode a string?

Objective-C and Swift URL encoding

http://cybersam.com/programming/proper-url-percent-encoding-in-ios

https://devforums.apple.com/message/15674#15674
http://simonwoodside.com/weblog/2009/4/22/how_to_really_url_encode/

Urlencode in Objective-C

You don't show us how you created the MyData variable or what type it is, but I'm guessing it's a NSString that you might have constructed something like the following:

NSString *myString = [NSString stringWithFormat:@"%@=%@", parameterKey, parameterValue];
NSData   *postData = [myString dataUsingEncoding:NSUTF8StringEncoding];

What you need to do is to "percent escape" any characters that are defined as reserved per RFC 3986. Thus, you'd replace the above with:

NSString *myString = [NSString stringWithFormat:@"%@=%@", parameterKey, [self percentEScapeString:parameterValue]];
NSData   *postData = [myString dataUsingEncoding:NSUTF8StringEncoding];

where

- (NSString *)percentEscapeString:(NSString *)string
{ 
    return CFBridgingRelease(CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault,
                                                                     (CFStringRef)string,
                                                                     NULL,
                                                                     (CFStringRef)@":/?@!$&'()*+,;=",
                                                                     kCFStringEncodingUTF8));
}

Technically, per the W3C specs for application/x-www-form-urlencoded, you should replace spaces with + characters, thus:

- (NSString *)percentEscapeString:(NSString *)string
{ 
    NSString *result = CFBridgingRelease(CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault,
                                                                                 (CFStringRef)string,
                                                                                 (CFStringRef)@" ",
                                                                                 (CFStringRef)@":/?@!$&'()*+,;=",
                                                                                 kCFStringEncodingUTF8));
    return [result stringByReplacingOccurrencesOfString:@" " withString:@"+"];
}

Personally, I put these sorts of methods in a NSString category, rather than in the current class, but hopefully this illustrates the idea.

Regardless, do not be tempted to use stringByAddingPercentEscapesUsingEncoding, because that doesn't give you the control you need. You really need to use CFURLCreateStringByAddingPercentEscapes, as shown above.

---

You asked about the emoticons. The above percent escaping works fine with the emoticons. For example, consider:

NSString *string1 = @"Text !@# & ( | 'l;,. quot;;
NSString *string2 = [self percentEscapeString:string1];
NSString *string3 = [string2 stringByReplacingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
NSLog(@"%@", string1);
NSLog(@"%@", string2);
NSLog(@"%@", string3);

2013-12-25 18:20:16.391 PercentEncodeTest[67199:70b] Text !@# & ( | 'l;,. br>2013-12-25 18:20:16.397 PercentEncodeTest[67199:70b] Text%20%21%40%23%20%26%20%28%20%7C%20%27l%3B%2C.%20%F0%9F%98%92%F0%9F%98%9A%F0%9F%98%9C
2013-12-25 18:20:16.401 PercentEncodeTest[67199:70b] Text !@# & ( | 'l;,. br>

As you can see, string2 is entirely percent escaped and should be correctly transmitted. And when we convert back to NSUTF8StringEncoding, we get our emoticons back fine.

I suspect that the problem now is not in the fact that the emoticons are correctly percent escaped, but rather your use of them by the destination.

Swift - encode URL

Swift 3

In Swift 3 there is addingPercentEncoding

let originalString = "test/test"
let escapedString = originalString.addingPercentEncoding(withAllowedCharacters: .urlHostAllowed)
print(escapedString!)

Output:

test%2Ftest

Swift 1

In iOS 7 and above there is stringByAddingPercentEncodingWithAllowedCharacters

var originalString = "test/test"
var escapedString = originalString.stringByAddingPercentEncodingWithAllowedCharacters(.URLHostAllowedCharacterSet())
println("escapedString: \(escapedString)")

Output:

test%2Ftest

The following are useful (inverted) character sets:

URLFragmentAllowedCharacterSet  "#%<>[\]^`{|}
URLHostAllowedCharacterSet      "#%/<>?@\^`{|}
URLPasswordAllowedCharacterSet  "#%/:<>?@[\]^`{|}
URLPathAllowedCharacterSet      "#%;<>?[\]^`{|}
URLQueryAllowedCharacterSet     "#%<>[\]^`{|}
URLUserAllowedCharacterSet      "#%/:<>?@[\]^`

If you want a different set of characters to be escaped create a set:

Example with added "=" character:

var originalString = "test/test=42"
var customAllowedSet =  NSCharacterSet(charactersInString:"=\"#%/<>?@\\^`{|}").invertedSet
var escapedString = originalString.stringByAddingPercentEncodingWithAllowedCharacters(customAllowedSet)
println("escapedString: \(escapedString)")

Output:

test%2Ftest%3D42

Example to verify ascii characters not in the set:

func printCharactersInSet(set: NSCharacterSet) {
    var characters = ""
    let iSet = set.invertedSet
    for i: UInt32 in 32..<127 {
        let c = Character(UnicodeScalar(i))
        if iSet.longCharacterIsMember(i) {
            characters = characters + String(c)
        }
    }
    print("characters not in set: \'\(characters)\'")
}

String won't url encode in iOS

For future reference, this is what I found to work (i.e. encode everything properly)

+ (NSString*)encodeURL:(NSString *)string
{
    NSString *newString = (__bridge_transfer NSString *)CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault, (__bridge CFStringRef)string, NULL, CFSTR(":/?#[]@!$ &'()*+,;=\"<>%{}|\\^~`"), CFStringConvertNSStringEncodingToEncoding(NSUTF8StringEncoding));

    if (newString) 
    {
        return newString;
    }

    return @"";
}

URLencoding in swift

Thank you, Rob, that answers helped me.

request.setValue("application/x-www-form-urlencoded", forHTTPHeaderField: "Content-Type")
request.setValue("application/json", forHTTPHeaderField: "Accept")
request.httpBody = postParameters.map { key, value in
   let keyString = key.addingPercentEncoding(withAllowedCharacters: .urlQueryValueAllowed)!
   let valueString = (value as! String).addingPercentEncoding(withAllowedCharacters: .urlQueryValueAllowed)!
   return keyString + "=" + valueString
}.joined(separator: "&").data(using: .utf8)

and extension

extension CharacterSet {

static var urlQueryValueAllowed: CharacterSet = {
    let generalDelimitersToEncode = ":#[]@" // does not include "?" or "/" due to RFC 3986 - Section 3.4
    let subDelimitersToEncode = "!$&'()*+,;="

    var allowed = CharacterSet.urlQueryAllowed
    allowed.remove(charactersIn: generalDelimitersToEncode + subDelimitersToEncode)

    return allowed
}()
}

---

Related Topics