How to Create a Daemon in Python

How do you create a daemon in Python?

Current solution

A reference implementation of PEP 3143 (Standard daemon process library) is now available as python-daemon.

Historical answer

Sander Marechal's code sample is superior to the original, which was originally posted in 2004. I once contributed a daemonizer for Pyro, but would probably use Sander's code if I had to do it over.

How to make a Python script run like a service or daemon in Linux

You have two options here.

1. Make a proper cron job that calls your script. Cron is a common name for a GNU/Linux daemon that periodically launches scripts according to a schedule you set. You add your script into a crontab or place a symlink to it into a special directory and the daemon handles the job of launching it in the background. You can read more at Wikipedia. There is a variety of different cron daemons, but your GNU/Linux system should have it already installed.

2. Use some kind of python approach (a library, for example) for your script to be able to daemonize itself. Yes, it will require a simple event loop (where your events are timer triggering, possibly, provided by sleep function).

I wouldn't recommend you to choose 2., because you would be, in fact, repeating cron functionality. The Linux system paradigm is to let multiple simple tools interact and solve your problems. Unless there are additional reasons why you should make a daemon (in addition to trigger periodically), choose the other approach.

Also, if you use daemonize with a loop and a crash happens, no one will check the mail after that (as pointed out by Ivan Nevostruev in comments to this answer). While if the script is added as a cron job, it will just trigger again.

How to implement a daemon process in Python?

You create a process that will run forever and join() to it. The second process will never get created, because the join() will stall your main process forever.

If this is how you want to proceed, then you should for example first create both processes and then join them:

if __name__ == '__main__':
    p1 = Process(target=ticking, args=())
    p1.start()

    p2 = Process(target=echo, args=())
    p2.start()

    p1.join()
    p2.join()

Understanding Python code for creating daemon process

A process being part of same session means when bash receives SIGHUP when you disconnect over ssh, it terminates all the processes in the session. Default SIGHUP handler also terminates process. To avoid this you want to create a new session.

Additionally, TTY is a somewhat old concept, which through the years has picked up a huge history and loads of corner cases intended for interactivity, but are counterproductive when running daemons, because you do not want the kernel to send random signals to your non-interactive daemon process when something TTY related happens.

---

So ... you do not want to be a member of TTY session and you do not want to be a session leader, because those get signalled.

That's why you start a daemon: you run python1, this process forks, you get python1 and python2. Python2 starts new session and forks, so you get python3.

Setsid in python2 ensures python3 is not connected to python1 session, and fork in python2 is needed because python2 is still a session leader.

Python3 will then not receive any TTY related signals and won't be killed by SIGHUP when you disconnect or whatever TTY related stuff happens on the box.

python-daemon to start an independent process but let the main application continue?

From multiple partial tips i got, finally managed to get it working...

def start(command, working_directory):
    pid = os.fork()
    if pid == 0:
        os.setsid()
        os.umask(0) # I'm not sure about this, not on my notebook at the moment
        os.execv(command[0], command) # This was strange as i needed to use the name of the shell script twice: command argv[0] [args]. Upon using ksh as command i got a nice error...
    else:
        with open(os.path.join(working_directory, 'application.pid'), 'w') as pf:
            pf.write(str(pid))

        return pid

That together solved the issue. The started process is not a child process of the running python script and won't stop when the script terminates.

Efficient Python Daemon

I think your idea is pretty much exactly what you want. For example:

import time

def do_something():
    with open("/tmp/current_time.txt", "w") as f:
        f.write("The time is now " + time.ctime())

def run():
    while True:
        time.sleep(60)
        do_something()

if __name__ == "__main__":
    run()

The call to time.sleep(60) will put your program to sleep for 60 seconds. When that time is up, the OS will wake up your program and run the do_something() function, then put it back to sleep. While your program is sleeping, it is doing nothing very efficiently. This is a general pattern for writing background services.

To actually run this from the command line, you can use &:

$ python background_test.py &

When doing this, any output from the script will go to the same terminal as the one you started it from. You can redirect output to avoid this:

$ python background_test.py >stdout.txt 2>stderr.txt &

How can I keep my python-daemon process running or restart it on fail?

If you really want to run a script 24/7 in background, the cleanest and easiest way to do it would surely be to create a systemd service.

There are already many descriptions of how to do that, for example here.

One of the advantages of systemd, in addition to being able to launch a service at startup, is to be able to restart it after failure.

Restart=on-failure

---

If all you want to do is automatically restart the program after a crash, the easiest method would probably be to use a bash script.

You can use the until loop, which is used to execute a given set of commands as long as the given condition evaluates to false.

#!/bin/bash

until python /path/to/script.py; do
    echo "The program crashed at `date +%H:%M:%S`. Restarting the script..."
done

If the command returns a non zero exit-status, then the script is restarted.

How to start daemon process from python on windows?

Using the answer Janne Karila pointed out this is how you can run a process that doen't die when its parent dies, no need to use the win32process module.

DETACHED_PROCESS = 8
subprocess.Popen(executable, creationflags=DETACHED_PROCESS, close_fds=True)

DETACHED_PROCESS is a Process Creation Flag that is passed to the underlying CreateProcess function.

---

Related Topics