Unexpectedly Large Realm File Size

Unexpectedly large Realm file size

The large file size when adding all of the objects in a single transaction is due to an unfortunate interaction between Realm's transaction log subsystem and Realm's memory allocation algorithm for large blobs. Realm's memory layout algorithm requires that the file size be at least 8x the size of the largest single blob stored in the Realm file. Transaction log entries, summarizing the modifications made during a single transaction, are stored as blobs within the Realm file.

When you add 40,000 objects in one transaction, you end up with a single transaction log entry that's around 5MB in size. This means that the file has to be at least 40MB in size in order to store it. (I'm not quite sure how it ends up being nearly twice that size again. It might be that the blob size is rounded up to a power of two somewhere along the line…)

When you add one object in 40,000 transactions, you still end up with a single transaction log entry only this time it's on a hundred or so bytes in size. This happens because when Realm commits a transaction, it attempts to first reclaim unused transaction log entries before allocating space for new entries. Since the Realm file is not open elsewhere, the previous entry can be reclaimed as each new commit is performed.

realm/realm-core#2343 tracks improving how Realm stores transaction log entries to avoid the significant overallocation you're seeing.

For now my suggestion would be to split the difference between the two approaches and add groups of objects per write transaction. This will trade off a little performance by increasing the number of commits but will reduce the impact of the memory layout algorithm by reducing the size of the largest transaction log entry you create. From a quick test, committing every 2,000 objects results in a file size of around 4MB, while being significantly quicker than adding each object in a separate write transaction.

Realm file size is too large

as bcamur sad,

the Realm file will maintain its size on disk to efficiently reuse
that space for future objects

but there is also written

The extra space will eventually be reused by future writes, or may be
compacted — for example by calling
Realm().writeCopyToPath(_:encryptionKey:).

and

call invalidate to tell Realm that you no longer need any of the
objects that you’ve read from the Realm so far, which frees us from
tracking intermediate versions of those objects. The Realm will update
to the latest version the next time it is accessed

Realm large db size due to frequent updates

I had an issue where my realm file size was increasing at an alarming rate, and it was an issue of not calling close() when the app closed unexpectedly during development. As such my database file (only had about 1k items in it) was at 10MB. Properly closing out my realm instance solved the problem and reduced my database file size to ~300KB. Really, it's worth checking your entire codebase to make sure you're actually closing out all realm instances. It's annoying, but way better than having users complain about running out of storage ;)

Based on your example above, I'd recommend also using the executeTransaction method (instead of beginning/committing transactions) provided by realm:

drealm.executeTransaction(new Realm.Transaction() {
    @Override
    public void execute(Realm realm) {
        dlr.setEarnings(dlr.getEarnings() + dlr.getPerSecond());
    }
});

drealm.close();

Realm file size is too large

as bcamur sad,

the Realm file will maintain its size on disk to efficiently reuse
that space for future objects

but there is also written

The extra space will eventually be reused by future writes, or may be
compacted — for example by calling
Realm().writeCopyToPath(_:encryptionKey:).

and

call invalidate to tell Realm that you no longer need any of the
objects that you’ve read from the Realm so far, which frees us from
tracking intermediate versions of those objects. The Realm will update
to the latest version the next time it is accessed

Minimizing Realm File Size: Practical approach

It's likely you're holding a reference to an older version of the data, preventing it from being reclaimed. Take a look at the File size & tracking of intermediate versions section of Realm's documentation for more information.

150k words text file is 0.8mb where realm db size is 18mb

You could try the following, for science:

private void insertWord() {
    long time = System.currentTimeMillis();
    StringBuilder builder=new StringBuilder();

    RealmConf conf = RealmConf.getInstance(true);
    int i = 0;
    int batchCount = 0;


    while (i < 150000) {
        if(batchCount == 0) {
            RealmUtils.startTransaction(conf);
        }
        batchCount++
        i++;
        String word = "Word:" + i;
        EB eb = new EB(word);
        builder.append(word+"\n");

        RealmUtils.saveWord(eb, conf);
        Log.i("word check" + i++, "seelog:" + word);
        if(batchCount == 3000) {
            RealmUtils.commitTransaction(conf);
            batchCount = 0;
        }
    }
    if(batchCount != 0) {
        RealmUtils.commitTransaction(conf);
    }
    writeStringIntoFile(builder.toString(),0);
}

Realm slow on updating multiple objects

Try putting the for loop inside the write block:

try! self.realm.write {
    for contact in self.selectedContacts {
        contact.isSelected = false;
        self.realm.add(contact, update: true)
    }
}

---

Related Topics