How to Ensure a Rake Task Only Running a Process at a Time

How to ensure a rake task only running a process at a time

You could use a lock file for this. When the task runs, try to grab the lock and run the rake task if you get the lock. If you don't get the lock, then don't run rake; you might want to log an error or warning somewhere too or you can end up with your rake task not doing anything for weeks or months before you know about it. When rake exits, unlock the lock file.

Something like RAA might help but I haven't used it so maybe not.

You could also use a PID file. You'd have a file somewhere that holds the rake processes process ID. Before starting rake, you read the PID from that file and see if the process is running; if it isn't then start up rake and write its PID to the PID file. When rake exists, delete the PID file. You'd want to combine this with locking on the PID file if you want to be really strict but this depends on your particular situation.

How to make sure only one rake task is running at a time?

Unix solved this issue with pid files. Pid files are located in /var/run and contain the programs process Id. Here is the man page: http://fuse4bsd.creo.hu/localcgi/man-cgi.cgi?pidfile+3

You might find it a bit old fashioned (and I agree), but it's an often used and proven method.

Are rake tasks suitable for long running processes in production?

You can try delayed job with this extension.

class MyJob

  include Delayed::ScheduledJob

  run_every 1.day

  def display_name
    "MyJob"
  end

  def perform
    # code to run ...
  end
end

Or manually enqueue another job with Time.now + 5.minutes for example after current job is finished inside perform method.

Run rake script for specific time range

I think you should first consider what is making this script use up so much memory.

(one thing would be loading up lots of records from the database, and appending them to an array)

But assuming you have already done everything you can,
I'd do something like this.

LIVE_FOR = 1.hour

def run!
  finish_before = LIVE_FOR.from_now

  array = get_the_array # some big collection to operate on

  array.each do |object|
    while Time.now < finish_before
      ...
    end
  end
end

But really, i'd first try to tackle why you have a memory leak.

rake check if already running

I use lockrun to prevent cron tasks from running multiple times (this only works when invoking the command through the same lockrun invocation, so if you need to protect from various invocation paths, then you'll need to look for other methods).

In your crontab, you invoke it like this:

*/5 * * * * /usr/local/bin/lockrun --lockfile=/var/run/this_task.lockrun -- cd /my/path && RAILS_ENV=production bundle exec rake this:task

Dealing with very long running rake task

Personally I love Resque, you can use the resque-scheduler gem for dealing with long running or periodic tasks.

If you don't have to run your task very often, you can demonize the the rake task to make sure it keeps running if your SSH session dies or something.

Try something like this:

nohup rake my:task &

nohup will send the output to nohup.out in the directory you run the task in, and will also let leave your ssh session without the process dying, secondly the & will run it as a deamon.

---

Related Topics