Resque VS Sidekiq

Resque vs Sidekiq?

Resque:

Pros:

  • does not require thread safety (works with pretty much any gem out there);
  • has no interpreter preference (you can use any ruby);

    Resque currently supports MRI 2.3.0 or later
  • loads of plugins.

Cons

  • runs a process per worker (uses more memory);
  • does not retry jobs (out of the box, anyway).

Sidekiq:

Pros

  • runs thread per worker (uses much less memory);
  • less forking (works faster);
  • more options out of the box.

Cons

  • [huge] requires thread-safety of your code and all dependencies. If you run thread-unsafe code with threads, you're asking for trouble;
  • works on some rubies better than others (jruby is recommended, efficiency on MRI is decreased due to GVL (global VM lock)).

Is it possible to use resque/Sidekiq /rails-jobs without redis (using MySql)

Delayed::Job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background.

...

Active Job is a framework for declaring jobs and making them run on a variety of queuing backends

...

Active Job has built-in adapters for multiple queuing backends (Sidekiq, Resque, Delayed Job, and others). To get an up-to-date list of the adapters see the API Documentation for ActiveJob::QueueAdapters.

...

Delayed Job uses your SQL database for storage and processes jobs in a single-threaded process. It's simple to set up but the performance and scalability aren't great. (c) Sidekiq FAQ

P.S. I prefer to find place for Redis.

delayed_jobs vs resque vs beanstalkd?

For my projects I will feel very comfortbale with collectiveidea/delayed_job in rails2 and 3.
I don't know beanstalkd, but i will try it soon :-).
I have followed the suggestions in the resque documentation.
I will report it.

Resque vs DelayedJob

How does Resque compare to DelayedJob, and why would you choose one over the other?

  • Resque supports multiple queues
  • DelayedJob supports finer grained priorities
  • Resque workers are resilient to memory leaks / bloat
  • DelayedJob workers are extremely simple and easy to modify
  • Resque requires Redis
  • DelayedJob requires ActiveRecord
  • Resque can only place JSONable Ruby objects on a queue as arguments
  • DelayedJob can place any Ruby object on its queue as arguments
  • Resque includes a Sinatra app for monitoring what's going on
  • DelayedJob can be queried from within your Rails app if you want to add an interface

If you're doing Rails development, you already have a database and ActiveRecord. DelayedJob is super easy to setup and works great. GitHub used it for many months to process almost 200 million jobs.

Choose Resque if:

  • You need multiple queues
  • You don't care / dislike numeric priorities
  • You don't need to persist every Ruby object ever
  • You have potentially huge queues
  • You want to see what's going on
  • You expect a lot of failure / chaos
  • You can setup Redis
  • You're not running short on RAM

Choose DelayedJob if:

  • You like numeric priorities
  • You're not doing a gigantic amount of jobs each day
  • Your queue stays small and nimble
  • There is not a lot failure / chaos
  • You want to easily throw anything on the queue
  • You don't want to setup Redis

Choose Beanstalkd if:

  • You like numeric priorities
  • You want extremely fast queue
  • You don't want to waste you RAM
  • You want to serve high number of jobs
  • You're fine with JSONable Ruby objects on a queue as arguments
  • You need multiple queues

In no way is Resque a "better" DelayedJob, so make sure you pick the tool that's best for your app.

A nice comparison of queueing backend speed:

                 enqueue                work
-------------------------------------------------
delayed job | 200 jobs/sec 120 jobs/sec
resque | 3800 jobs/sec 300 jobs/sec
rabbitmq | 2500 jobs/sec 1300 jobs/sec
beanstalk | 9000 jobs/sec 5200 jobs/sec

Have a nice day!

P.S. There is a RailsCast about resque, Delayed Job (revised version) and Beanstakld. Have a look!

P.P.S. My favourite choiche is now Sidekiq ( very Simple, Fast and efficient for simple jobs ), have a look at this page for comparison.

Moving web crawler into background: Resque or Sidekiq

Sounds like you did enough digging to end up in the right direction! I'd factor that out into a separate background worker system too.

Sidekiq is better-maintained these days, and the multithreading is very useful for your use case, so I'd pick that. Good starting points are the Sidekiq homepage and this Railscast, both of which give you lots of information to hit the ground running.

Celery like software for Ruby?

There are a lot of options for queues in ruby.

Sidekiq -> https://github.com/mperham/sidekiq

Resque, you got the link

DelayedJob -> http://blog.leetsoft.com/delayed_job/

All of them are pretty much the same. So you just need to use the one you are more confortable working with the examples.

In my projects i ended up using Sidekiq, the reviews about it are pretty awesome.



Related Topics



Leave a reply



Submit