Ruby on Rails Multiple Http Request at the Same Time

Ruby on Rails Multiple HTTP request at the same time?

Yes! For a proof-of-concept, try

require 'thread'

client = Instagram.client(:access_token => session[:access_token])
@user = client.user
@recent_media_items = client.user_recent_media

threads = []
threads << Thread.new { @lv = client.tag_recent_media('lv', options = {:count => 60}) }
threads << Thread.new { @lv1 = client.tag_recent_media('lv1', options = {:count => 60}) }
threads << Thread.new { @lv2 = client.tag_recent_media('lv2', options = {:count => 60}) }
threads << Thread.new { @lv3 = client.tag_recent_media('lv3', options = {:count => 60}) }
threads.each(&:join) # this waits for all the threads to finish before proceeding
puts [@lv, @lv1, @lv2, @lv3]

In practice, you will want to set up some error handling and retry settings within the threads. Also, you may run into issues with thread-safety in the Instagram gem. If you are doing this on a large scale with hundreds or thousands of requests, you may want to try a concurrent HTTP client like Typhoeus or an evented HTTP client like EM-HTTP-Request. For these, you would have to manually implement the tag_recent_media method that is in the Instagram gem.

Ruby library to make multiple HTTP requests simultaneously

You are probably looking for Typhoeus.

Typhoeus runs HTTP requests in parallel while cleanly encapsulating handling logic

https://github.com/typhoeus/typhoeus

Duplicate entry error with multiple request at exact same time

The issue is here:

if !Playtime.where(local_id: params[:local_id]).exists?
  # ⇒ SWITCH OF THE CONTEXT, RECORD CREATED BY ANOTHER PROCESS
  @playtime = Playtime.create!(playtime_params)

Basically, the check passes and then before create gets to the database the other process that handled the simultaneous request has created the record in the database.

The most common railish approach would be to use Optimistic Locking and rescue StaleObjectError, but here you already have a constraint on the field, so it’d be even easier:

def create
  # no need to begin here, you might rescue the whole function body
  @playtime = Playtime.create!(playtime_params)
  json_response(@playtime.local_id, true)
rescue ActiveRecord::RecordNotUnique
    # record is already there, ok
    json_response(params[:local_id], true)
rescue ActiveRecord::RecordInvalid => invalid
    # params are invalid, report
    json_response(invalid.record.errors.full_messages.first, false)
end

Sidenote: we usually supply an HTTP error code with error reporting, your code currently likely serves 200 OK within “invalid” response.

Listen for multiple response produced by one HTTP request

We ended up building a solution on top of eventmachine. EM allow us to receive events every time a piece of data is received on the connection. Then data is passed to http_parser.rb to parse incoming http messages and finally we enqueue the message on sidekiq.

Is there a better way to make multiple HTTP requests asynchronously in Ruby?

Instead of the while clause you used, you can call Thread#join to make the main thread wait for other threads.

threads = []
urls.each_value do |thing|
    threads << Thread.new do
        result = Net::HTTP.get(URI.parse(thing))
        json_stuff = JSON::parse(result)
        info = json["person"]["bio"]["info"]

        thing["name"] = info
    end
end

# Wait until threads are done.
threads.each { |aThread|  aThread.join }

---

Related Topics