How to Run Two Methods in Parallel Ruby

How to run two methods in parallel ruby

Use thread.

t1 = Thread.new do
  first_method
end
second_method
t1.join

How do I run two threads in Ruby at the same time?

http://ruby-doc.org/core/classes/Thread.html

x = 2
Thread.new do
    x = 3
end
x = 4

For true concurrency having more then 2 cores or 2 processors is required - but it may not work if implementation is single-threaded (such as the MRI).

Parallelizing methods in Rails

Ruby has the excellent promise gem. Your example would look like:

require 'future'

def method_one
...
def method_nth

def summary
  result1 = future { method_one }
  ......
  resultn = future { method_nth }
  collect_results result1, ..., resultn
end

Simple, isn't it? But let's get to more details. This is a future object:

result1 = future { method_one }

It means, the result1 is getting evaluated in the background. You can pass it around to other methods. But result1 doesn't have any result yet, it is still processing in the background. Think of passing around a Thread. But the major difference is - the moment you try to read it, instead of passing it around, it blocks and waits for the result at that point. So in the above example, all the result1 .. resultn variables will keep getting evaluated in the background, but when the time comes to collect the results, and when you try to actually read these values, the reads will wait for the queries to finish at that point.

Install the promise gem and try the below in Ruby console:

require 'future'
x = future { sleep 20; puts 'x calculated'; 10 }; nil
# adding a nil to the end so that x is not immediately tried to print in the console
y = future { sleep 25; puts 'y calculated'; 20 }; nil

# At this point, you'll still be using the console!
# The sleeps are happening in the background

# Now do:
x + y
# At this point, the program actually waits for the x & y future blocks to complete

Edit: Typo in result, should have been result1, change echo to puts

How to make multiple parallel concurrent requests with Rails and Heroku

Don't use Resque. Use Sidekiq instead.

Resque runs in a single-threaded process, meaning the workers run synchronously, while Sidekiq runs in a multithreaded process, meaning the workers run asynchronously/simutaneously in different threads.

Make sure you assign a URL to scrape per worker. It's no use if one worker scrape multiple URLs.

With Sidekiq, you can pass the link to a worker, e.g.

LINKS = [...]
LINKS.each do |link|
  ScrapeWoker.perform_async(link)
end

The perform_async doesn't actually execute the job right away. Instead, the link is just put in a queue in redis along with the worker class, and so on, and later (could be milliseconds later) workers are assigned to execute each job in queue in its own thread by running the perform instance method in ScrapeWorker. Sidekiq will make sure to retry again if exception occur during execution of a worker.

PS: You don't have pass a link to the worker. You can store the links to a table and then pass the ids of the records to workers.

More info about sidekiq

Running multiple background parallel jobs with Rails

Some thoughts...

• Just because you need to read 50 sites and naturally want some parallel work does not mean that you need 50 processes or threads. You need to balance the slowdown and overhead. How about having 10 or 20 processes each read a few sites?

• Depending on which Ruby you are using, be careful about the green threads, you may not get the parallel result you want

• You might want to structure it like a reverse, client-side inetd, and use connect_nonblock and IO.select to get the parallel connections you want by making all the servers respond in parallel. You don't really need parallel processing of the results, you just need to get in line at all the servers in parallel, because that is where the latency really is.

So, something like this from the socket library...extend it for multiple outstanding connections...

require 'socket'
include Socket::Constants
socket = Socket.new(AF_INET, SOCK_STREAM, 0)
sockaddr = Socket.sockaddr_in(80, 'www.google.com')
begin
  socket.connect_nonblock(sockaddr)
  rescue Errno::EINPROGRESS
  IO.select(nil, [socket])
  begin
    socket.connect_nonblock(sockaddr)
    rescue Errno::EISCONN
  end
end
socket.write("GET / HTTP/1.0\r\n\r\n")
# here perhaps insert IO.select. You may not need multiple threads OR multiple
# processes with this technique, but if you do insert them here
results = socket.read

How to perform parallel processing on a single rails API in server side?

Note that there is a huge difference between event based servers and background jobs.

An event based server often runs on a single thread and uses callbacks for non-blocking IO. The most famous example is Node.js. For ruby there is the eventmachine library and various frameworks and simple HTTP servers.

An evented server can start processing one request and then switch to another request while the first is waiting for a database call for example.

Note that even if you have a event based server you can't be slow at processing requests. The user experience will suffer and clients will drop the connection.

Thats where background jobs (workers) come in, they let your web process finish fast so that it can send the response and start dealing with the next request. Slow processes like sending out emails or cleanup that don't have does not require user feedback or concurrency are farmed out to workers.

So in conclusion - if your application is slow then using parallel processing is not going to save you. Its not a silver bullet. Instead you should invest in optimising database queries and leveraging caching so that responses are fast.

While you could potentially run database queries or other operations in parallel in Rails, the greatly added complexity is probably not worth the performance gains.

What I mean here is that with what you are usually doing in Rails concurrency is not really applicable - you're fetching something from the DB and using it to make JSON or HTML. You can't really start rendering until you have the results back anyways. While you could potentially do something like fetch data and use it to render partials concurrently Rails does not support this out of the box since it would greatly increase the complexity while not offering much to the majority of the users of the framework.

As always - don't optimise prematurely.

Is there any way to call all the methods inside the class with the single line code in Ruby?

is it possible to call all the methods by a single line code

Yes, that is possible.

Personally, I don't get the obsession with squeezing everything into a single line. It does not make code smaller, it does not make code better, it does not make code easier to read. But it is technically possible.

In Ruby, line breaks are always optional. They can always be replaced with something else. If the line break is used as an expression separator, it can be replaced with a semicolon (;), which is also an expression separator. If the line break is used to terminate some syntactic element, it can be replaced with either a semicolon (;), a keyword (for example then in if and unless, when in case, do in for, while, and until, and so on), and sometimes just whitespace.

So, you could write your code in a single line like this:

object = Sample.new(par1, par2); object.method1; object.method2; object.method3; object.method4; # … etc …

calling the methods one by one using the object.method_name(parameter) is really hard and taking very long space and time.

Whether you write the code on one line or multiple lines has no effect on the space or time requirements of your program.

If you execute the methods sequentially, the space requirement will be the maximum of the space requirements of all the methods and the time requirement will be the sum of the time requirements of all the methods.

You can execute the methods in parallel. In that case, the space requirement will be the sum of the space requirements of all the methods and the time requirement will be the maximum of the time requirements of all the methods plus any time needed to coordinate the parallel execution and merge the results back together. Also, executing the methods in parallel will change the result of the program!

Either way, you can only improve either space or time, not both.

---

Related Topics