Ruby: Wait for All Threads Completed Using Join and Threadswait.All_Waits - What the Difference

Ruby: Wait for all threads completed using join and ThreadsWait.all_waits - what the difference?

The documentation clearly states that all_waits will execute any passed block after each thread's execution; join doesn't offer anything like this.

require "thwait"

threads = [Thread.new { 1 }, Thread.new { 2 }]

ThreadsWait.all_waits(threads) do |t|
  puts "#{t} complete."
end # will return nil

# output:
# #<Thread:0x00000002773268> complete.
# #<Thread:0x00000002772ea8> complete.

To accomplish the same with join, I imagine you would have to do this:

threads.each do |t|
  t.join
  puts "#{t} complete."
end # will return threads

Apart from this, the all_waits methods eventually calls the join_nowait method which processes each thread by calling join on it.

Without any block, I would imagine that directly using join would be faster since you would cut back on all ThreadsWait methods leading up to it. So I gave it a shot:

require "thwait"
require "benchmark"

loops = 100_000
Benchmark.bm do |x|
  x.report do
    loops.times do
      threads = [Thread.new { 2 * 1000 }, Thread.new { 4 * 2000 }]
      threads.each(&:join)
    end
  end

  x.report do
    loops.times do
      threads = [Thread.new { 2 * 1000 }, Thread.new { 4 * 2000 }]
      ThreadsWait.all_waits(threads)
    end
  end
end

# results:
# user       system     total       real
# 4.030000   5.750000   9.780000  ( 5.929623 )
# 12.810000  17.060000  29.870000 ( 17.807242 )

How do I manage ruby threads so they finish all their work?

If you modify spawn_thread_for to save a reference to your created Thread, then you can call Thread#join on the thread to wait for completion:

x = Thread.new { sleep 0.1; print "x"; print "y"; print "z" }
a = Thread.new { print "a"; print "b"; sleep 0.2; print "c" }
x.join # Let the threads finish before
a.join # main thread exits...

produces:

abxyzc

(Stolen from the ri Thread.new documentation. See the ri Thread.join documentation for some more details.)

So, if you amend spawn_thread_for to save the Thread references, you can join on them all:

(Untested, but ought to give the flavor)

# main thread
work_units = Queue.new # and fill the queue...

threads = []
10.downto(1) do
  threads << Thread.new do
    loop do
      w = work_units.pop
      Thread::exit() if w.nil?
      do_some_work(w)
    end
  end
end

# main thread continues while work threads devour work

threads.each(&:join)

Ruby multi threading #join and simultaneous execution

What SHOULD I be doing here?

You should be joining your threads. Otherwise when the main thread (your script) exits, it takes all unfinished threads with it. The reason why execution is serial in your first case is that you wait for a thread to finish right after you start it (and before you start the next one). First create all threads, then wait on them.

i = 4
threads = i.times.map do |n|
  Thread.new {
    puts "opening thread for #{n} degree"
    myFunction(n)
  }
end

threads.each(&:join)
# or
require 'thwait'
ThreadsWait.all_waits(*threads)

You will see further improvements in threading performance if you run the code on JRuby or Rubinius, as their threads are not crippled in any way by some global lock.

Add multithreads/concurency in script

Instead of puts write output to a variable (hash).
If you wand to wait for all threads to finish their job before showing the output, use ThreadsWait class.

require 'thwait'
file = File.read('./services.json')
data_hash = JSON.parse(file)

h = {}
threads = []
service = data_hash.keys
service.each do |microservice|
threads << Thread.new do
  thread_id = Thread.current.object_id.to_s(36)
  begin
  h[thread_id] = "Microservice: #{microservice}"
  port = data_hash["#{microservice}"]['port']
  h[thread_id] << "Port: #{port}"

  nodes = "knife search 'chef_environment:#{env} AND recipe:#{microservice}' -i"
  node = %x[ #{nodes} ].split
    node.each do |n|
    h[thread_id]<< "Node: #{n}"
    uri = URI("http://#{n}:#{port}/healthcheck?count=10")
    res = Net::HTTP.get_response(uri)
    status = Net::HTTP.get(uri)
    h[thread_id] << res.code
    h[thread_id] << status
    h[thread_id] << res.message
  end

rescue Net::ReadTimeout
  h[thread_id] << "ReadTimeout Error"
  next
end
end
end
threads.each do |thread|
  thread.join
end

# wait untill all threads finish their job
ThreadsWait.all_waits(*threads)

p h

[edit]

ThreadsWait.all_waits(*threads) is redundant in above code and can be omitted, since line treads.each do |thread| thread.join end does exactely the same thing.

Can ruby generate non-daemonic threads?

Ruby Threads exhibit the Python daemonic behaviour by default, but really have no such built in concept.

Your example, without the while (or a join/value) will exit when the main program reaches the end.

For Threads to take the Python non daemonic behaviour, you have to specifically wait for them.

require 'thwait' #stdlib

wait_threads = []

wait_threads.push( Thread.new() do
  1000.times do |value|
    printf "%s ", value
  end
end )

Thread.new() do
  sleep 1
  puts "I am like a python daemon"
end

ThreadsWait.all_waits( *wait_threads )
puts 'I am the main thread'

---

Related Topics