Invoke Delayed_Job Capistrano Tasks Only on Specific Servers

Invoke delayed_job capistrano tasks only on specific servers

When you define a task in Capistrano you can restrict the execution of the task to specific role(s). The way you do this is by passing the :role option.

It seems the default delayed_job Capistrano recipe does this.

desc "Stop the delayed_job process"
task :stop, :roles => lambda { roles } do
  run "cd #{current_path};#{rails_env} script/delayed_job stop"
end

According to the source code, the task fetches the list of roles from the :delayed_job_server_role configuration variable.

Back to your problem, to narrow the execution of the tasks to a specific group of servers, define a new role (for example worker) in your deploy.rb

role :worker, "192.168.1.1" # Assign the IP of your machine

Then set the :delayed_job_server_role to that name

set :delayed_job_server_role, :worker

That's all. Now the tasks will be executed, but only to the servers listed in the :worker role.

Delayed Job Capistrano task without restarting each time

I'm also not sure what you're looking for but maybe a simple

cap delayed_job:stop

from the command line will do? You can see all available tasks at

cap -T

UPDATE

I would argue that it is a best practice to restart your Delayed Job workers every time you deploy. In the end, the workers execute your code, and that tends to change between deploys. Now, if the code your workers run rarely changes (that includes the rails boot process, environment files, initializers, settings, models that you use, etc.) and you want to take care of this yourself, then simply remove the hooks such as

after "deploy:stop",    "delayed_job:stop"

from the deploy.rb and you're fine: the dj tasks will still be at your disposal, but they will not be triggered during deploy.

Rails 4, capistrano 3, delayed_job - can't find bin/delayed_job on one of the servers

Check if you can see delayed_job on your staging server in #{deploy_to}/shared/bin/

If it's not there, copy it there from your project's bin folder.

Starting delayed_job at startup

You should create one recipe with the restart command.

namespace :delayed_job do 
    desc "Restart the delayed_job process"
    task :restart, :roles => :app do
        run "cd #{current_path}; RAILS_ENV=#{rails_env} script/delayed_job restart"
    end
end

Then you add it to be executed at the end of your deployment.

after "deploy:update_code", "delayed_job:restart"

delayed_job: One job per tenant at a time?

Your best bet is probably to spin a custom solution that implements a distributed lock - essentially, the workers all run normally and pull from the usual queues, but before performing work check with another system (Redis, RDBMS, API, whatever) to verify that no other worker is yet performing a job for that tenant. If that tenant is not being worked, then set the lock for the tenant in question and work the job. If the tenant is locked, don't perform the work. It's your call on a lot of the implementation details like whether to move on to try another job, re-enqueue the job at the back of the queue, whether to consider it a failure and bind it to your retry limits, or do something else entirely. This is pretty open-ended, so I'll leave the details to you, but here are some tips:

• Inheritance will be your friend; define this behavior on a base job and inherit from it on the jobs you expect your workers to run. This also allows you to customize the behavior if you have "special" cases for certain jobs that come up without breaking everything else.
• Assuming you're not running through ActiveJob (since it wasn't mentioned), read up on delayed_job hooks: https://github.com/collectiveidea/delayed_job/#hooks - they may be an appropriate and/or useful tool
• Get familiar with some of the differences and tradeoffs in Pessimistic and Optimistic locking strategies - this answer is a good starting point: Optimistic vs. Pessimistic locking
• Read up on general practices surrounding the concept of distributed locks so you can choose the best tools and strategies for yourself (it doesn't have to be a crazy complicated solution, a simple table in the database that stores the tenant identifier is sufficient, but you'll want to consider the failure cases - how to you manage locks that are abandoned, for example)

Seriously consider not doing this; is it really strictly required for the system to operate properly? If so, it's probably indicative in an underlying flaw in your data model or how you've structured transformations around that data. Strive for ACIDity in your application when thinking about operations on the data and you can avoid a lot of these problems. There's a reason it's not a commonly available "out of the box" feature on background job runners. If there is an underlying flaw, it won't just bite you on this problem but on something else - guaranteed!

---

Related Topics