Can Mongodb and Its Drivers Preserve the Ordering of Document Elements

Can MongoDB and its drivers preserve the ordering of document elements

From Version 2.6 on, MongoDB preserves the order of fields where possible. However, the _id field always comes first an renaming fields can lead to re-ordering. However, I'd generally try not to rely on details like this. As the original question mentions, there are also additional layers to consider which each must provide some sort of guarantee for the stability of the order...

Original Answer:

No, MongoDB does not make guarantees about the ordering of fields:

"There is no guarantee that the field order will be consistent, or the same, after an update."

In particular, in-place updates that change the document size will usually change the ordering of fields. For example, if you $set a field whose old value was of type number and the new value is NumberLong, fields usually get re-ordered.

However, arrays preserve ordering correctly:

[ {'key1' : 'value1'}, {'key2' : 'value2'}, ... ]

I don't see why this is "ugly" and "bloated" at all. Storing a list of complex objects couldn't be easier. However, abusing objects as lists is definitely ugly: Objects have associative array semantics (i.e. there can only be one field of a given name), while lists/arrays don't:

// not ok:
db.foo2.insert({"foo" : "bar", "foo" : "lala" });
db.foo2.find();
{ "_id" : ObjectId("4ef09cd9b37bc3cdb0e7fb26"), "foo" : "lala" }

// a list can do that
db.foo2.insert({ 'array' : [ {'foo' : 'bar'}, { 'foo' : 'lala' } ]});
db.foo2.find();
{ "_id" : ObjectId("4ef09e01b37bc3cdb0e7fb27"), "array" : 
      [ { "foo" : "bar" }, { "foo" : "lala" } ] }

Keep in mind that MongoDB is an object database, not a key/value store.

Do arrays stored in MongoDB keep their order?

yep MongoDB keeps the order of the array.. just like Javascript engines..

Mongodb document format consistency

1- Does mongodb always return a consistent DBObject format which will always convert to the same json so that the hash would always be the same. - No Mongo does not guarantee the order so the json can be different based on what kind of updates were done on the document. There is no guarantee that the field order will be consistent, or the same, after an update. If no such order changing updates were done on it then the order should be preserved MongoDB update on Field Order .
But when you serialize the json into an object using Jackson or something else it will serialize to the same object and should have the same hash.

2 - Would such an implementation even be viable? As in storing the hash with the object itself, essentially changing the object (thus making the hash invalid) but getting around by not retrieving that field in the response.
Looks like from this answer you can use Jakson or Gson to hash the json object, even though it is not ordered.
excluding a field should not be a problem.
If you store the hash as a field in the object itself all the write queries that save ( which is an overwrite of the entire document ) will have to write the hash into it. If any of them fail to do so the hash will be lost.
An update query will have another problem since along with changing the data it also has to update the hash of the document. So this will have to involve reading the object, modifying it, computing the hash and storing it back. You will not be able to use the primitive update queries.

If you make the hash as the primary key which is _id field that would mitigate this problem although you probably need it for something else.

3- The simplest way would be to store the _id of the document to be hashed into another collection along with the hash as the _id of the new collection.

{
    "_id":<hash code of docuemnt>,
    "refer":<_id of the document to be hashed>
}

This would involve multiple read writes which will hurt performance and depending on your use case it

Mongo according to me is a simplistic database designed to store and retrieve objects. If you have the need to do something complicated with it other than retrieving fast and writing its probably not fit for the task.

Does mongo db ListField maintain ordering?

Yes they do, the driver is faithful to the underlying data and mongoDb guarantees this (Do arrays stored in MongoDB keep their order?)

Modify the order in which properties are displayed in MongoDB

The above question and answer are quite old. Anyhow, if somebody visits this I feel like I should add:

This answer is completely wrong. Actually in Mongo Documents ARE ordered key-value pairs. However when using pymongo it will use python dicts for documents which indeed are not ordered (as of cpython 3.6 python dicts retain order, however this is considered an implementation detail). But this is a limitation of the pymongo driver.

Be aware, that this limitation actually impacts the usability. If you query the db for a subdocument it will only match if the order of the key-values pairs is correct.

Just try the following code yourself:

from pymongo import MongoClient
db = MongoClient().testdb
col = db.testcol
subdoc = {
   'field1': 1,
   'field2': 2,
   'filed3': 3
}
document = {
   'subdoc': subdoc
}
col.insert_one(document)
print(col.find({'subdoc': subdoc}).count())

Each time this code gets executed the 'same' document is added to the collection. Thus, each time we run this code snippet the printed value 'should' increase by one. It does not because find only maches subdocuemnts with the correct ordering but python dicts just insert the subdoc in arbitrary order.

see the following answer how to use ordered dict to overcome this: https://stackoverflow.com/a/30787769/4273834

Mongodb find's returning document order

Documents are stored in natural order

The documents are stored in descending order based on date. So the collection has 20140731 as the first document.

Unless you are using a capped collection, there is no guarantee for the ordering of documents on disk (also referred to as natural order).

Document deletions and moves (when a document outgrows its allocated record space) create space on the free list which will be reused.

Here's a quick example which should demonstrate this in the mongo shell:

// Start with an empty database & collection
use demodb; db.dropDatabase(); db.order.drop()

// Add some test data
for (i=0; i<1000; i++) {
    db.order.insert({'i': i})
}

// Looks like insertion order! (0..9)
db.order.find({}).limit(10);

// Pause 5s for effect :)
sleep(5000);

// Remove half the entries
db.order.remove({ i: { $lt: 500 }})

// Re-add the missing entries
for (i=0; i<500; i++) {
    db.order.insert({'i': i})
}

// Not the entries you expected .. space from deleted records was reused
db.order.find({}).limit(10)

// Clean up demodb
db.dropDatabase()

Order of results

When I use the find command with filters {$gte : 20140720, $lte : 20140731}, mongodb returns back the query in ascending order of "date" field.

If an index is used for a query, the documents are returned in the order they are found in the index. You should take advantage of this when constructing your indexes for common queries (see: Use Indexes to Sort Query Results).

FYI, a simple index (eg. on {date:1}) can be used to return the results sorted in either ascending or descending order.

Sort by ObjectID

If you are using MongoDB's default ObjectIDs for _id, you can sort by { _id: 1 } to approximate insertion order since the first 4 bytes of the ObjectID incorporate a timestamp. If you wanted to use this for sorting a query based on date and approximate insertion order you would ensure an index on {date:1, _id:1}.

Note that ObjectIDs are typically generated by the client driver, so if you have clock drift on your app servers (or the _id is created some time before the document is inserted) the ObjectIDs may not strictly reflect "insertion order" as seen by the server. If accuracy of insertion order is highly important, it is generally possible to generate the _id on the server-side (approach varies depending on the driver).

MongoDB go driver maintain sort order

A map does not have any ordering guarantees for its elements. Use a bson.D for documents where element ordering is important:

SetSort({{"timestamp",-1},{"_id":-1}})

Mongodb document format consistency

1- Does mongodb always return a consistent DBObject format which will always convert to the same json so that the hash would always be the same. - No Mongo does not guarantee the order so the json can be different based on what kind of updates were done on the document. There is no guarantee that the field order will be consistent, or the same, after an update. If no such order changing updates were done on it then the order should be preserved MongoDB update on Field Order .
But when you serialize the json into an object using Jackson or something else it will serialize to the same object and should have the same hash.

2 - Would such an implementation even be viable? As in storing the hash with the object itself, essentially changing the object (thus making the hash invalid) but getting around by not retrieving that field in the response.
Looks like from this answer you can use Jakson or Gson to hash the json object, even though it is not ordered.
excluding a field should not be a problem.
If you store the hash as a field in the object itself all the write queries that save ( which is an overwrite of the entire document ) will have to write the hash into it. If any of them fail to do so the hash will be lost.
An update query will have another problem since along with changing the data it also has to update the hash of the document. So this will have to involve reading the object, modifying it, computing the hash and storing it back. You will not be able to use the primitive update queries.

If you make the hash as the primary key which is _id field that would mitigate this problem although you probably need it for something else.

3- The simplest way would be to store the _id of the document to be hashed into another collection along with the hash as the _id of the new collection.

{
    "_id":<hash code of docuemnt>,
    "refer":<_id of the document to be hashed>
}

This would involve multiple read writes which will hurt performance and depending on your use case it

Mongo according to me is a simplistic database designed to store and retrieve objects. If you have the need to do something complicated with it other than retrieving fast and writing its probably not fit for the task.

---

Related Topics