How to Fire Raw Mongodb Queries Directly in Ruby

How to fire raw MongoDB queries directly in Ruby

Here's a (possibly) better mini-tutorial on how to get directly into the guts of your MongoDB. This might not solve your specific problem but it should get you as far as the MongoDB version of SELECT * FROM table.

---

First of all, you'll want a Mongo::Connection object. If
you're using MongoMapper then you can call the connection
class method on any of your MongoMapper models to get a connection
or ask MongoMapper for it directly:

connection = YourMongoModel.connection
connection = MongoMapper.connection

Otherwise I guess you'd use the from_uri constructor to build
your own connection.

Then you need to get your hands on a database, you can do this
using the array access notation, the db method, or get
the current one straight from MongoMapper:

db = connection['database_name']    # This does not support options.
db = connection.db('database_name') # This does support options.
db = MongoMapper.database           # This should be configured like
                                    # the rest of your app.

Now you have a nice shiny Mongo::DB instance in your hands.
But, you probably want a Collection to do anything interesting
and you can get that using either array access notation or the
collection method:

collection = db['collection_name']
collection = db.collection('collection_name')

Now you have something that behaves sort of like an SQL table so
you can count how many things it has or query it using find:

cursor = collection.find(:key => 'value')
cursor = collection.find({:key => 'value'}, :fields => ['just', 'these', 'fields'])
# etc.

And now you have what you're really after: a hot out of the oven Mongo::Cursor
that points at the data you're interested in. Mongo::Cursor is
an Enumerable so you have access to all your usual iterating
friends such as each, first, map, and one of my personal
favorites, each_with_object:

a = cursor.each_with_object([]) { |x, a| a.push(mangle(x)) }

There are also command and eval methods on Mongo::DB that might do what you want.

Executing raw MongoDB queries in Rails controller

I found a way to do it without have to establish a connection with Mongo console. Using mongoid, you can just use the update_all to generate the same query.

How to query MongoDB directly from Ruby instead of using Mongoid?

If you're using Mongoid 3, it provides easy access to its MongoDB driver: Moped. Here's an example of accessing some raw data without using Models to access the data:

db = Mongoid::Sessions.default

# inserting a new document
collection = db[:collection_name]
collection.insert(name: 'my new document')

# finding a document
doc = collection.find(name: 'my new document').first

# iterating over all documents in a collection
collection.find.each do |document|
  puts document.inspect
end

find_by_sql equivalent for mongoid?

Mongoid wraps the Collection object to return objects of the proper class.

So, if User is a Mongoid model:

cursor = User.collection.find({}, {}) # Just like the Ruby driver...
records = cursor.to_a # An array of User objects

Edit to add: It actually wraps Mongo's Cursor class too. See here:

def each
  @cursor.each do |document|
    yield Mongoid::Factory.build(@klass, document)
  end
end

MongoDB, Ruby do not display id field?

That should almost work. The error you're getting is something like this:

TypeError: keys must be strings or symbols

right? Just use a Hash for fields instead of an Array:

db.collection("unicorns").find(nil, :fields => { :name => true, :_id => false })

So it is pretty much a straight transliteration of the JavaScript version.

Sorry, I don't know of any tutorials on this stuff let alone good ones. I've figured it out myself through guess work and extrapolating the JavaScript docs.

Finding the size of a Mongo::Collection::View

collection.find({ foo: 'bar' }).count()

should solve your problem. There is no size method available in mongo but there is count.

Ruby GridFS - Search on filename

First get a connection to the database, we'll call this db. Then you can connect to your GridFS as a Mongo::Grid or Mongo::GridFileSystem instance:

fs = Mongo::Grid.new(db)
fs = Mongo::GridFileSystem.new(db)

Now you can use the Mongo::GridExt::InstanceMethods methods on fs. In particular, you can use exist?:

f = fs.exist? :filename => 'pancakes.png'

The exist? method is poorly named as it gives you a Hash if it finds something and a nil if it doesn't.

That's not terribly useful if you're searching for, say, all filenames that match /pancakes/. However, GridFS is just a pair of normal MongoDB collections:

• fs.files: the file metadata.
• fs.chunks: the file data (in chunks).

If you want to do arbitrary metadata searches then you just need to get your hands on fs.files and have your way with it:

fs     = db['fs.files']
cursor = fs.find(:filename => /pancakes/)
# Now iterate through cursor or .count it or ...

The fs above will be a Mongo::Collection so its find method accepts all the usual query options.

How to log all request to rails console in Mongoid 5?

You can customize Mongoid logging level as described in the documentation. You can also configure it in the main Rails application.

module MyApplication
  class Application < Rails::Application
    config.mongoid.logger = Logger.new($stdout, :warn)
  end
end

If you want to reuse the same Rails logger, simply assign Rails.logger (just make sure to assign it after the Rails.logger is initialized.

Rails association with two foreign keys?

Do you mean, that it should not be firing a new SELECT query with each pass?

You can optimize this with the includes() directive like this:

events = Event.includes(:payload).where('signature LIKE ?', 'example_signature')
events.each do |event|
  puts e.payload.data_payload
end

That would load all your data in 2 queries, one for all the events, and another for all the payloads that have a foreign key back to your events:

Event Load (0.2ms)  SELECT "event".* FROM "event"  WHERE (signature LIKE 'example_signature')
Payload Load (0.2ms)  SELECT "data".* FROM "data"  WHERE "data"."cid" IN (2,4,5,6,7)

Edit:

Your comments seem to indicate that performance is a top concern for this (probably because the tables are huge), so I think you were pretty close already. I would suggest selecting just the columns you need (since the events table has many of them), and using and SQL JOIN to add the other table's contents to query directly (since there is a one-to-one mapping of event to payload)

This will do that:

Event.
  select('event.cid, event.sid, event.signature, data.cid, data.sid, data.data_payload').
  joins(:payload).
  where('signature LIKE ?', 'example_signature').each do |event|
  puts event.data_payload
end
# SELECT event.cid, event.sid, event.signature, data.cid, data.sid, data.data_payload FROM "event" INNER JOIN "data" ON "data"."cid" = "event"."cid" WHERE (signature LIKE 'example_signature')

This will also preserve your Event object, so you can call other methods on it, but ONLY if they don't depend on attributes you excluded in the select() clause.

If you don't need any other methods from your Event object, and just want results as quickly as possible, then you could run a raw query with:

sanitized_str = ActiveRecord::Base.connection.quote('example_signature')
result = ActiveRecord::Base.connection.execute(
  "SELECT event.cid, event.sid, event.signature, data.cid, data.sid, data.data_payload
   FROM event
   INNER JOIN data ON data.cid = event.cid
   WHERE (signature LIKE #{sanitized_str});"
)
# => [{"cid"=>2, "sid"=>1, "signature"=>"example_signature", "data_payload"=>"Hello world!", 0=>2, 1=>1, 2=>"example_signature", 3=>2, 4=>1, 5=>"Hello world!"}]

This will return an array of records with each record as a plain-old ruby hash. Event objects won't be built in-memory, saving a lot of overhead.

---

Related Topics