Scheduling a Regular Event: Cron/Cron Alternatives (Including Celery)

Scheduling a regular event: Cron/Cron alternatives (including Celery)

A simple, non-Celery way to approach things would be to create custom django-admin commands to perform your asynchronous or scheduled tasks.

Then, on Windows, you use the at command to schedule these tasks. On Linux, you use cron.

I'd also strongly recommend ditching Windows if you can for a development environment. Your life will be so much better on Linux or even Mac OSX. Re-purpose a spare or old machine with Ubuntu for example, or run Ubuntu in a VM on your Windows box.

Scheduling a regular event: Cron/Cron alternatives (including Celery)

A simple, non-Celery way to approach things would be to create custom django-admin commands to perform your asynchronous or scheduled tasks.

Then, on Windows, you use the at command to schedule these tasks. On Linux, you use cron.

I'd also strongly recommend ditching Windows if you can for a development environment. Your life will be so much better on Linux or even Mac OSX. Re-purpose a spare or old machine with Ubuntu for example, or run Ubuntu in a VM on your Windows box.

Why would running scheduled tasks with Celery be preferable over crontab?

I've been using cron for a production website, and have switched to celery on a current project.
I'm far more into celery than cron, here is why:

• Celery + Celerybeat has finer granularity than cron. Cron cannot run more than once a minute, while celery can (I have a task run every 90 seconds which checks an email queue to send messages, and another which cleans the online users list).
• A cron line has to call a script or a unique command, with absolute path and user info. Celery calls python functions, no need to write more than code.
• With celery, to deploy to another machine, you generally just have to pull/copy your code, which is generally in one place. Deploying with cron would need more work (you can automate it but...)
• I really find celery better suited than cron for routine cleaning (cache, database), and in general, for short tasks. Dumping a database is more a work for cron, however, because you don't want clutter the event queue with too long tasks.
• Not the least, Celery is easily distributed across machines.

How do I get a Cron like scheduler in Python?

If you're looking for something lightweight checkout schedule:

import schedule
import time

def job():
    print("I'm working...")

schedule.every(10).minutes.do(job)
schedule.every().hour.do(job)
schedule.every().day.at("10:30").do(job)

while 1:
    schedule.run_pending()
    time.sleep(1)

Disclosure: I'm the author of that library.

Schedule Django Celery periodic tasks to run from time to other time

celery.py

from celery.schedule import crontab
app.conf.beat_schedule = {
    'compute-every-minute-mon-through-friday-9-to-5': {
         'task': 'sum',
         'schedule': crontab(minute='*/1',
hour='9-17', day_of_week='mon,tue,wed,thu,fri'),
     },
  }

minute='*/1' - runs every minute

hour='9-17' - runs 9am to 5pm

day_of_week='mon,tue,wed,thu,fri' - Monday through Friday

Most of these are available on the documentation page, check it out!

Celery - schedule periodic tasks starting at a specific time

Celery seems like a good solution for your scheduling problem: Celery's PeriodicTasks have run time resolution in seconds.

You're using an appropriate tool here, but the crontab entry is not what you want. You want to use python's datetime.timedelta object; the crontab scheduler in celery.schedules has only minute resolution, but using timedelta's to configure the PeriodicTask interval provides strictly more functionality, in this case, per second resolution.

e.g. from the Celery docs

>>> from celery.task import tasks, PeriodicTask
>>> from datetime import timedelta
>>> class EveryThirtySecondsTask(PeriodicTask):
...     run_every = timedelta(seconds=30)
...
...     def run(self, **kwargs):
...         logger = self.get_logger(**kwargs)
...         logger.info("Execute every 30 seconds")

http://ask.github.com/celery/reference/celery.task.base.html#celery.task.base.PeriodicTask

class datetime.timedelta(days=0, seconds=0, microseconds=0, milliseconds=0, minutes=0, hours=0, weeks=0)

The only challenge here is that you have to describe the frequency with which you want this task to run rather than at what clock time you want it to run; however, I would suggest you check out the Advanced Python Scheduler http://packages.python.org/APScheduler/

It looks like Advanced Python Scheduler could easily be used to launch normal (i.e. non Periodic) Celery tasks at any schedule of your choosing using it's own scheduling functionality.

Set up a scheduled job?

One solution that I have employed is to do this:

1) Create a custom management command, e.g.

python manage.py my_cool_command

2) Use cron (on Linux) or at (on Windows) to run my command at the required times.

This is a simple solution that doesn't require installing a heavy AMQP stack. However there are nice advantages to using something like Celery, mentioned in the other answers. In particular, with Celery it is nice to not have to spread your application logic out into crontab files. However the cron solution works quite nicely for a small to medium sized application and where you don't want a lot of external dependencies.

EDIT:

In later version of windows the at command is deprecated for Windows 8, Server 2012 and above. You can use schtasks.exe for same use.

**** UPDATE ****
This the new link of django doc for writing the custom management command

---

Related Topics