Cancel an Already Executing Task with Celery

Cancel an already executing task with Celery?

revoke cancels the task execution. If a task is revoked, the workers ignore the task and do not execute it. If you don't use persistent revokes your task can be executed after worker's restart.

https://docs.celeryq.dev/en/stable/userguide/workers.html#worker-persistent-revokes

revoke has an terminate option which is False by default. If you need to kill the executing task you need to set terminate to True.

>>> from celery.task.control import revoke
>>> revoke(task_id, terminate=True)

https://docs.celeryq.dev/en/stable/userguide/workers.html#revoke-revoking-tasks

Which is the best way to programatically terminate (cancel) a celery task

If you really need to kill the running task, using terminate=True in revoke() method is the proper way and your best option. The problem is that doing this has no guaranteed effect or, better said, can have negative side effects. To understand why, it's necessary to know how revoking works and what terminate=True does. When you call revoke(), you are sending broadcast message to your workers via the broker. That means that before the time you realize you want/need to to kill the task, sending the revoke message and receiving the control command by the worker, the worker might have already finished the task, grabbed another task from the queue and started working on it. When it finally receives the command, it just kills the worker process, which, by that time, might already be working on different task.

If you know beforehand you might need to abort your tasks and can adjust the tasks code, you can take a look at abortable tasks.

How to kill a running task in celery?

Why are you doing data.AsyncResult? You can try doing something like this.

from celery.task.control import revoke
revoke(task_id, terminate=True)

Also, further details can be found here:

http://docs.celeryproject.org/en/latest/userguide/workers.html#revoke-revoking-tasks

Is there any way to non-violently stop particular task of celery worker?

A running task is a running subprocess of the worker (when using prefork), this means that the only way to abort a task is to kill the subprocess that is running it.

You may try to experiment your own implementation of revoke event handling trying to figure out the subprocess ID and kill only that one, but honestly don't know if is worth and if it can really work.

I think that short answer is you can't.

Anyway killing the worker is needed sometimes, especially in initial phases of projects where you still need to dimension correctly the resources, just make sure you log somewhere the running tasks so you can reschedule them or just use CELERY_ACKS_LATE

How to inspect and cancel Celery tasks by task name

# Retrieve tasks
# Reference: http://docs.celeryproject.org/en/latest/reference/celery.events.state.html
query = celery.events.state.tasks_by_type(your_task_name)

# Kill tasks
# Reference: http://docs.celeryproject.org/en/latest/userguide/workers.html#revoking-tasks
for uuid, task in query:
    celery.control.revoke(uuid, terminate=True)

---

Related Topics