How to Run Celery Workers by Superuser

How to run celery workers by superuser?

OK I found the solution!

In celery 3.1 and above, workers with pickle serialization will crash as mentioned in documentation:

Worker will now crash if running as the root user with pickle enabled.

So to use sudo, you need to disable pickle serialization in celery configs.I did it by using json:

app.conf.update(
     CELERY_ACCEPT_CONTENT = ['json'],
     CELERY_TASK_SERIALIZER = 'json',
     CELERY_RESULT_SERIALIZER = 'json',
)

and then if you run using sudo, it will work!

RuntimeWarning: You're running the worker with superuser privileges: this is absolutely not recommended

Gotcha after hours of headbanging.The error ImportError('The curl client requires the pycurl library.') was resolved with the help of this github comment.As Safwan Samsudeen pointed out the warning RuntimeWarning: You're running the worker with superuser privileges: this is absolutely not recommended
is not an issue.So my .conf files are the same and I uninstalled pycurl and again reinstalled it using the command pip install pycurl --compile --global-option="--with-openssl" --no-cache-dir'

Running Celery as root

You can set it to true like this:

# export C_FORCE_ROOT="true"

Then make sure it is set as an env. variable

# echo $C_FORCE_ROOT
true

But make sure to make it permanent, as this will vanish with the next restart

Have fun :) !!

celery 3.1+ doesn't allow you to run workers as superuser specifically when pickling strings

pickle simply isn't secure:

The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.

As such the Celery team has decided that its users need to be explicit that they wish to accept pickled data.

Why the supervisor make the celery worker changing form running to starting all the time?

Finally, I solve this problem yesterday night.

about the reason

• I make the project could success running at a windows 10 system, but did no check when I change the project to centos7.+. The command:env/bin/python project/manage.py celeryd could not run success. So the supervisor would start a process which will failed soon.

• Why the command could not success? I had pip installed all the package need. But it show err below:

  Running a worker with superuser privileges when the worker accepts messages serialized with pickle is a very bad idea!

  If you really want to continue then you have to set the C_FORCE_ROOT
  environment variable (but please think about this before you do).

  User information: uid=0 euid=0 gid=0 egid=0

  I try to search some blog about this error, and get the anser:

  export C_FORCE_ROOT='true' # at the centos enviroument

action to solve(after meeting error like this)

• add export C_FORCE_ROOT='true' to centos's enviroment file and source it.

• check command 'env/bin/python project/manage.py celeryd ',did it run successful.

• restart the supervisord. Attention please! not supervisorctl reload,it just reload the .conf file,not the environment file. Try kill the process supervisord -c xx.conf(ps aux | grep supervisord and kill -9 process_number,be careful).

some url about the blog

• the error when just run celeryd not sucess in chinese

---

Related Topics