Run Delayed Jobs and Sidekiq at the Same Time

Run Delayed Jobs and Sidekiq at the same time

For Sidekiq 2.17.1 and later, somewhere in the Rails initializers, call the following:

Sidekiq.hook_rails!
Sidekiq.remove_delay!

and you will have only prefixed sidekiq_delay methods and so on.

(official document)

---

For older versions of Sidekiq:

Put the following in config/initializers/sidekiq.rb

module Sidekiq::Extensions::Klass
  alias :sidekiq_delay :delay
  remove_method :delay
  alias :sidekiq_delay_for :delay_for
  remove_method :delay_for
  alias :sidekiq_delay_until :delay_until
  remove_method :delay_until
end

module Sidekiq::Extensions::ActiveRecord
  alias :sidekiq_delay :delay
  remove_method :delay
  alias :sidekiq_delay_for :delay_for
  remove_method :delay_for
  alias :sidekiq_delay_until :delay_until
  remove_method :delay_until
end

module Sidekiq::Extensions::ActionMailer
  alias :sidekiq_delay :delay
  remove_method :delay
  alias :sidekiq_delay_for :delay_for
  remove_method :delay_for
  alias :sidekiq_delay_until :delay_until
  remove_method :delay_until
end

And then you can use sidekiq_delay to queue in Sidekiq, and call delay to queue in Delayed Job.

Heroku Delayed Job and Sidekiq in the Same app

It is possible.

First, You should solve similar interfaces conflict problem like .delay. here: Run Delayed Jobs and Sidekiq at the same time

Second, you should set multiple active_job adapter configuration. here: http://edgeguides.rubyonrails.org/active_job_basics.html#backends

About Procfile, You can register multiple worker
For example)

web: bundle exec puma -t 5:5 -p ${PORT:-3000} -C config/puma.rb
worker: bundle exec sidekiq
delayedjobworker: bundle exec rake jobs:work

Check https://devcenter.heroku.com/articles/procfile#more-process-type-examples

How can I use Sidekiq delay with a worker

With SideKiq you can scheduled when a job will be executed with a friendly API :

MyWorker.perform_in(3.hours, 'mike', 1) # Expect a duration
MyWorker.perform_at(3.hours.from_now, 'mike', 1) # Expect a date

Check it out : Scheduled Jobs

conflict delayed_job / sidekiq

As I mention in the comment In order to get it working you have to redefine/basically monkey patch the handle_asynchronously method something like this

Anywhere you like (but make sure it loaded )

in your config/initializers/patch.rb the code look like this

module Patch
  def handle_asynchronously(method, opts = {})
    aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1
    with_method, without_method = "#{aliased_method}_with_delay#{punctuation}", "#{aliased_method}_without_delay#{punctuation}"
    define_method(with_method) do |*args|
      curr_opts = opts.clone
      curr_opts.each_key do |key|
        if (val = curr_opts[key]).is_a?(Proc)
          curr_opts[key] = if val.arity == 1
            val.call(self)
          else
            val.call
          end
        end
      end
      ## Replace this with other syntax
      # delay(curr_opts).__send__(without_method, *args)
      __delay__(curr_opts).__send__(without_method, *args)
    end
    alias_method_chain method, :delay
  end
end

Module.send(:include,Patch)

And I believe rest all will follow then they way it should :)

Reason:

Delayed::Job include delay method on Object and Sidekiq include it delay method over ActiveRecord
Hence when the class try to invoke delay it look up it ancestors class (including the Eigen Class)
and it find the method define or included in ActiveRecord::Base class (which is sidekiq delay)

why does __delay__ work because alias define the copy of the existing method which is delay method of DelayedJob , hence when you invoke the __delay__ method it invoke delay method define DelayedJob
include to Object

Note:

Although the solution is bit patch but the it works . Keeping in mind that every direct .delay methid invocation is invoking delay method of the SideKiq and not DelayedJob to invoke the DelayedJob delay method you always has call it this way __delay__

Suggestion :

Monkey Patching is just a bad practice on my personal note I would rather not use 2 entirely different background processing library for a single application to achieve the same task. If the task is process thing in background why cant it be done with a single library either delayed_job or sidekiq (why it is that you required both of them )

So the point and to simply thing make your background processing an ease with respect to future I sincerely advice you take any one of the two library for background processing and I feel that would the valid answer for your question instead of monkey patching an doing other crazy stuff

Hope this help

Sidekiq enqueue job after a wait time

You could do a few things:

1. Implement a check in the worker to make sure the object exists; otherwise, re-enqueue the job. Probably want to think about this to make sure you don't accidentally re-enqueue bad jobs forever, but this seems like a good sanity check for you.

2. Introduce a delay. In particular, sidekiq can wait to pull jobs from the queue until a specified time.

   "Sidekiq allows you to schedule the time when a job will be executed. You use perform_in(interval, *args) or perform_at(timestamp, *args) rather than the standard perform_async(*args):

   MyWorker.perform_in(3.hours, 'mike', 1)
MyWorker.perform_at(3.hours.from_now, 'mike', 1)"

   See https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs for more details on this option.

Personally I would go for #1 but #2 might be a quicker fix if you're desperate.

---

Related Topics