Rails: Faster Way to Perform Updates on Many Records

Rails: Faster way to perform updates on many records

Try wrapping your entire code into a single database transaction. Since you're on Heroku it'll be a Postgres bottom-end. With that many update statements, you can probably benefit greatly by transacting them all at once, so your code executes quicker and basically just leaves a "queue" of 6500 statements to run on Postgres side as the server is able to dequeue them. Depending on the bottom end, you might have to transact into smaller chunks - but even transacting 100 at a time (and then close and re-open the transaction) would greatly improve throughput into Pg.

http://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html
http://www.postgresql.org/docs/9.2/static/sql-set-transaction.html

So before line 2 you'd add something like:

def add_details(shop, shopify_orders)
  Order.transaction do
    shopify_orders.each do |shopify_order|

And then at the very end of your method add another end:

      if !payment_details.blank?
        PaymentDetail.add_details(order, payment_details)
      end
    end //shopify_orders.each..
  end //Order.transaction..
end //method

Updating Lots of Records at Once in Rails

You can do something like this:

collection2 = collection.map { |c| [c[:external_id], c.except(:external_id)]}.to_h

def update
  ChildModel.where(external_id: collection2.keys).each |cm| do
    ext_id = cm.external_id
    cm.assign_attributes collection2[ext_id]
    cm.save if cm.changed?
    collection2.delete(ext_id)
  end
  if collection2.present?
    new_ids = collection2.keys
    new = collection.select { |c| new_ids.include? c[:external_id] }
    ChildModel.create(new)
  end
end

Better because

• fetches all required records all at once
• creates all new records at once

You can use update_columns if you don't need callbacks/validations
Only drawback, more ruby code manipulation which I think is a good tradeoff for db queries..

How to update thousands of records

Yes, update_all is the right method but no, you can't do it like this. Your some_method will only get called once to set up a database call (I assume you're persisting to a database). You'll then get an error because dob won't be recognised in the scope of the User class.

You'll need to translate your date logic to SQL functions.

Something like (for mysql):

User.update_all("age = year(now()) - 
                year(dob) - 
                (DATE_FORMAT(now(), '%m%d') < DATE_FORMAT(dob, '%m%d'))")

(NB. the date_format stuff is so that you get the right age for people who's birthdays are later in the year than the current date - see this question for more details)

Updating several records at once in rails

It sounds like you are looking for ActiveRecord::Base.update_all - from the documentation:

Updates all records with details given if they match a set of conditions supplied, limits and order can also be supplied. This method constructs a single SQL UPDATE statement and sends it straight to the database. It does not instantiate the involved models and it does not trigger Active Record callbacks or validations.

Product.last_day_of_freshness.update_all(:stale => true)

Actually, since this is rails 2.x (You didn't specify) - the named_scope chaining may not work, you might need to pass the conditions for your named scope as the second parameter to update_all instead of chaining it onto the end of the Product scope.

rails find_or_initialize to update or create large amount of records

This is the code we ended up using, perhaps not the most efficient but it works:

def add_details(shopify_orders)
    tax_lines = []
    shopify_orders.each do |shopify_order|
      shopify_order.tax_lines.each do |shopify_tax_line|
        tax_line = Taxline.where(:order_id => shopify_order.id).first_or_initialize
        tax_line.price = shopify_tax_line.price
        tax_line.rate = shopify_tax_line.rate
        tax_line.title = shopify_tax_line.title
        tax_lines << tax_line
      end
    end
    TaxLine.import tax_lines, :validate => false
end

Mass/bulk update in rails without using update_all with a single query?

Mass update without using update_all can be achievable using activerecord-import gem.
Please refer to this gem for more information.
Methods with detail.

Example:
Lets say there is a table named "Services" having a "booked" column. We want to update its value using the gem outside the loop.

services.each do |service| 
 service.booked = false 
 service.updated_at = DateTime.current if service.changed?
end

ProvidedService.import services.to_ary, on_duplicate_key_update: { columns: %i[booked updated_at] }

active-record import by default does not update the "updated_at" column. So we've to explicitly update it.

Rails, using collection model to update multiple records at once on a single form

Based on Sward's comment, a new model wasn't needed. Since each product already has a parent (store), I can use the Store model with accepts_nested_attributes_for :products.

I just created a separate collections controller for products (to handle the incoming form submission and redirect) and a view with a form with fields_for to update multiple records at once.

Update Multiple Records with a single command in Ruby

ActiveRecord Create can take an array of hashes and create multiple records simultaneously.

However, ActiveRecord Update cannot.

You could potentially create an "update_batch" method on your model that allows for an array of hashes. You would have to send an array of hashes and each hash would have to include the id of the record you are updating (and allow that in your strong parameters definition). Then in your update_batch method you would have to grab the id from each hash and update each:

class Attendance < ActiveRecord
  def update_batch(attendance_records)
    attendance_records.each do |record|
      Attendance.find(record[:id]).update(record.except(:id))
    end
  end
end

---

Related Topics