Swift 4 Base64 String to Data Not Working Due to String Containing "Incomplete" Emoji

Swift 4 base64 String to Data not working due to String containing incomplete emoji

(Some of this is out of comments, but trying to bring it together and describe solutions.)

First, your strings are not UTF-8. They're UTF-16 or malformed UTF-16. Sometimes UTF-16 happens to be interpretable as UTF-8, but when it is, there will be NULL characters scattered through the string. In your "working" example, it's not really working.

let toApostrophe = "ACAAKgBVAFMAQQAqACAn"                 // *USA* '//
if let textData = Data(base64Encoded: toApostrophe) {
    if let decodedString = String(data: textData, encoding: .utf8) {
        print(decodedString)
        print(decodedString.count)
        print(decodedString.map { $0.unicodeScalars.map { $0.value } } )
    } else {
        print("DID NOT DECODE UTF8")
    }
} else {
    print("DID NOT DECODE BASE64")
}

Prints:

 *USA* '
15
[[0], [32], [0], [42], [0], [85], [0], [83], [0], [65], [0], [42], [0], [32], [39]]

Note that the length of string is 15 characters, not 8 like you were probably expecting. That's because it includes an extra invisible NULL (0) between most characters.

toEndBracket doesn't happen to be legal UTF-8, however. Here are its bytes:

["00", "20", "00", "2a", "00", "55", "00", "53", "00", "41", "00", "2a", "00", "20", "27", "96", "00", "20", "d8", "3c", "00", "22", "00", "7d", "00", "5d", "00"]

This is ok until it gets to 0xd8. That starts with the bits 110, which indicates that it's the start of a two byte sequence. But the next byte is 0x3c, which is not a valid second byte of a multi-byte sequence (it should start with 10, but it starts with 00). So we can't decode this as UTF-8. Even using decodeCString(_:as:repairingInvalidCodeUnits) can't decode this string because it's filled with embedded NULLs. You've got to decode it using at least the right encoding.

But let's do that. Decode as UTF-16. At least that's close, even though it's slightly invalid UTF-16.

let toEndBracket16 = String(data: toEndBracketData, encoding: .utf16)
// " *USA* ➖ �"}]"

Now we can at least work with this. It's invalid JSON, though. So we can strip that by filtering it:

let legalJSON = String(toEndBracket16.filter { $0 != "\u{FFFD}" })
// " *USA* ➖ "}]"

I don't really recommend this approach. It's incredibly fragile and based on broken input. Fix the input. But in a world where you're trying to parse broken input, these are the tools.

How can I encode a string to Base64 in Swift?

I don’t have 6.2 installed but I don’t think 6.3 is any different in this regard:

dataUsingEncoding returns an optional, so you need to unwrap that.

NSDataBase64EncodingOptions.fromRaw has been replaced with NSDataBase64EncodingOptions(rawValue:). Slightly surprisingly, this is not a failable initializer so you don’t need to unwrap it.

But since NSData(base64EncodedString:) is a failable initializer, you need to unwrap that.

Btw, all these changes were suggested by Xcode migrator (click the error message in the gutter and it has a “fix-it” suggestion).

Final code, rewritten to avoid force-unwraps, looks like this:

import Foundation

let str = "iOS Developer Tips encoded in Base64"
println("Original: \(str)")

let utf8str = str.dataUsingEncoding(NSUTF8StringEncoding)

if let base64Encoded = utf8str?.base64EncodedStringWithOptions(NSDataBase64EncodingOptions(rawValue: 0)) 
{

    println("Encoded:  \(base64Encoded)")

    if let base64Decoded = NSData(base64EncodedString: base64Encoded, options:   NSDataBase64DecodingOptions(rawValue: 0))
                          .map({ NSString(data: $0, encoding: NSUTF8StringEncoding) })
    {
        // Convert back to a string
        println("Decoded:  \(base64Decoded)")
    }
}

(if using Swift 1.2 you could use multiple if-lets instead of the map)

Swift 5 Update:

import Foundation

let str = "iOS Developer Tips encoded in Base64"
print("Original: \(str)")

let utf8str = str.data(using: .utf8)

if let base64Encoded = utf8str?.base64EncodedString(options: Data.Base64EncodingOptions(rawValue: 0)) {
    print("Encoded: \(base64Encoded)")

    if let base64Decoded = Data(base64Encoded: base64Encoded, options: Data.Base64DecodingOptions(rawValue: 0))
    .map({ String(data: $0, encoding: .utf8) }) {
        // Convert back to a string
        print("Decoded: \(base64Decoded ?? "")")
    }
}

JavaScript substring without splitting emoji

So this isn't really an easy thing to do, and I'm inclined to tell you that you shouldn't write this on your own. You should use a library like runes.

Just a simple npm i runes, then:

const runes = require('runes');
const usaText = "AZ";
runes.substr(usaText, 0, 2); // "Abr>

How to list (almost) all emojis in Swift for iOS 8 without using any form of lookup tables?

You can loop over those hex values with a Range: 0x1F601...0x1F64F and then create the Strings using a UnicodeScalar:

for i in 0x1F601...0x1F64F {
    guard let scalar = UnicodeScalar(i) else { continue }
    let c = String(scalar)
    print(c)
}

Outputs:

/p>

If you want all the emoji, just add another loop over an array of ranges:

// NOTE: These ranges are still just a subset of all the emoji characters;
//       they seem to be all over the place...
let emojiRanges = [
    0x1F601...0x1F64F,
    0x2702...0x27B0,
    0x1F680...0x1F6C0,
    0x1F170...0x1F251
]

for range in emojiRanges {
    for i in range {
        guard let scalar = UnicodeScalar(i) else { continue }
        let c = String(scalar)
        print(c)
    }
}

---

For those asking, the full list of available emojis can be found here: https://www.unicode.org/emoji/charts/full-emoji-list.html

A parsable list of unicode sequences for all emojis can be found in the emoji-sequences.txt file under the directory for the version you're interested in here: http://unicode.org/Public/emoji/

As of 9/15/2021 the latest version of the emoji standard available on Apple devices is 13.1.

Facebook/messenger archive contains emoji that I am unable to parse

.encode('latin1').decode('utf8) is correct - it results in the codepoint U+fe33a("). This codepoint is in a Private Use Area (PUA) (specifically Supplemental Private Use Area-A), so everyone can assign his own meaning to that codepoint (Maybe facebook wanted to use a crying face, when there wasn't yet one in Unicode, so they used PUA?).

Googling for that char (https://www.google.com/search?q=) makes google autocorrect it to U+1f62d (") - sadly I have no idea how google maps U+fe33a to U+1f62d.

Googling for U+fe33a site:unicode.org gives https://unicode.org/L2/L2010/10132-emojidata.pdf, which lists U+1F62D as proposed official codepoint.

As that document from unicode lists U+fe33a as a codepoint used by google, I searched for android old emoji codepoints pua. Among other stuff two actually usable results:

1. How to get Android emoji code point - the question links to :
   
   
   • https://unicodey.com/emoji-data/table.htm - a html table, that seems to be acceptably parsable
   • and even better: https://github.com/google/mozc/blob/master/src/data/emoji/emoji_data.tsv - a tab sepperated list, that maps modern codepoints to legacy PUA codepoints and other information like this:
     
     1F62D FE33A E72D E411[...]
2. https://github.com/googlei18n/noto-emoji/issues/115 - this thread links to:
   
   
   • https://github.com/Crissov/noto-emoji/blob/legacy-pua/emoji_aliases.txt - a machine readable document, that translates legacy PUA codepoints to modern codepoints like this:
     
     FE33A;1F62D # Google

I included my search queries in the answer, because non of the results I found are in any way authoritative - but it should be enough, to get your tool working :-)

How do I decode HTML entities in Swift?

This answer was last revised for Swift 5.2 and iOS 13.4 SDK.

---

There's no straightforward way to do that, but you can use NSAttributedString magic to make this process as painless as possible (be warned that this method will strip all HTML tags as well).

Remember to initialize NSAttributedString from main thread only. It uses WebKit to parse HTML underneath, thus the requirement.

// This is a[0]["title"] in your case
let htmlEncodedString = "The Weeknd <em>‘King Of The Fall’</em>"

guard let data = htmlEncodedString.data(using: .utf8) else {
    return
}

let options: [NSAttributedString.DocumentReadingOptionKey: Any] = [
    .documentType: NSAttributedString.DocumentType.html,
    .characterEncoding: String.Encoding.utf8.rawValue
]

guard let attributedString = try? NSAttributedString(data: data, options: options, documentAttributes: nil) else {
    return
}

// The Weeknd ‘King Of The Fall’
let decodedString = attributedString.string

extension String {

    init?(htmlEncodedString: String) {

        guard let data = htmlEncodedString.data(using: .utf8) else {
            return nil
        }

        let options: [NSAttributedString.DocumentReadingOptionKey: Any] = [
            .documentType: NSAttributedString.DocumentType.html,
            .characterEncoding: String.Encoding.utf8.rawValue
        ]

        guard let attributedString = try? NSAttributedString(data: data, options: options, documentAttributes: nil) else {
            return nil
        }

        self.init(attributedString.string)

    }

}

let encodedString = "The Weeknd <em>‘King Of The Fall’</em>"
let decodedString = String(htmlEncodedString: encodedString)

Not able to retrieve data from database in ios

Create the database at runtime, and use the below logic,

to write the data in sqlite.

    sqlite3 *pDb; 
char *databaseName; 

databaseName = @"TempDatabase.sql";

NSArray *documentPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDir = [documentPaths objectAtIndex:0];
//databasePath = [documentsDir stringByAppendingPathComponent:databaseName];

databasePath =[[[NSBundle mainBundle] resourcePath] stringByAppendingPathComponent:@"TempDatabase.sql"];

//[self copyDatabaseIfNeeded];

if(sqlite3_open([databasePath UTF8String], &pDb) == SQLITE_OK) 
{

    const char *sqlStatement = "INSERT INTO tablename (name) VALUES(?)";

    sqlite3_stmt *compiledStatement;
    if(sqlite3_prepare_v2(pDb, sqlStatement, -1, &compiledStatement, NULL) == SQLITE_OK) 
    {

         sqlite3_bind_text( compiledStatement, 1, [strTxtFldValue UTF8String], -1, SQLITE_TRANSIENT);
    }
    if(sqlite3_step(compiledStatement) != SQLITE_DONE ) 
    {
         NSLog( @"~~~~~~~~~~~ 1 Error: %s", sqlite3_errmsg(pDb) );
UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Error" message:@"Name name is not added." delegate:self cancelButtonTitle:@"OK" otherButtonTitles:nil];
        [alert show];
        [alert release];

    } else {

        UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Confirmation" message:@"name is added successfully." delegate:self cancelButtonTitle:@"OK" otherButtonTitles:nil];
        [alert show];
        [alert release];

            }
    sqlite3_finalize(compiledStatement);
}

sqlite3_close(pDb);

To read data from sqlite

     sqlite3 *database;
const char *dbpath = [appDel.databasePath UTF8String];
sqlite3_stmt *statement;

NSMutableArray *names = [[NSMutableArray alloc]init];

if (sqlite3_open(dbpath, &database) == SQLITE_OK) 
{

    const char *sqlStatement = "select name from tablename";

    if (sqlite3_prepare_v2(database, sqlStatement, -1, &statement, NULL) == SQLITE_OK) {
        while (sqlite3_step(statement) == SQLITE_ROW) 
        {
            NSString *cName = [NSString stringWithUTF8String:(char *)sqlite3_column_text(statement,0)];
            //NSLog(@"cName = %@",cName);

            Names *name = [[Names alloc] initWithName:cName];
            [names addObject:name];
            [name release];
        }

    }
    sqlite3_finalize(statement);

}
sqlite3_close(database);

Try out this and vote me if it works fine for you.

Best way to encode text data for XML in Java?

Very simply: use an XML library. That way it will actually be right instead of requiring detailed knowledge of bits of the XML spec.

---

Related Topics