Run Python Script Only If It's Not Running

crontab to run python file if not running already

pgrep -f lists itself as a false match when run from cron

I did the test with a script.py running an infinite loop. Then

pgrep -f script.py

...from the terminal, gave one pid, 13132 , while running from cron:

pgrep -f script.py > /path/to/out.txt

outputs two pids, 13132 and 13635.

We can therefore conclude that the command pgrep -f script.py lists itself as a match, when run from cron. Not sure how and why, but most likely, this is indirectly caused by the fact that cron runs with a quite limited set of environment variables (HOME, LOGNAME, and SHELL).

The solution

Running pgrep -f from a (wrapper) script makes the command not list itself, even when run from cron. Subsequently, run the wrapper from cron:

#!/bin/bash

if ! pgrep -f 'test.py'
then
    nohup python /home/dp/script/test.py & > /var/tmp/test.out
# run the test, remove the two lines below afterwards
else
    echo "running" > ~/out_test.txt
fi

Running python script with cron only if not running

The only suggestion I would make is to make your exception handling a little more specific. You don't want to accidentally delete the fcntl import one day and hide the NameError that results. Always try to catch the most specific exception you want to handle. In this case, I suggest something like:

import errno

try:
    fcntl.lock(...)
except IOError, e:
    if e.errno == errno.EAGAIN:
        sys.stderr.write(...)
        sys.exit(-1)
    raise

This way, any other cause of the lock being unobtainable shows up (probably in your email since you're using cron) and you can decide if it's something for an administrator to look at, another case for the program to handle, or something else.

How to constantly run Python script in the background on Windows?

On Windows, you can use pythonw.exe in order to run a python script as a background process:

Python scripts (files with the extension .py) will be executed by
python.exe by default. This executable opens a terminal, which stays
open even if the program uses a GUI. If you do not want this to
happen, use the extension .pyw which will cause the script to be
executed by pythonw.exe by default (both executables are located in
the top-level of your Python installation directory). This suppresses
the terminal window on startup.

For example,

C:\ThanosDodd\Python3.6\pythonw.exe C:\\Python\Scripts\moveDLs.py

In order to make your script run continuously, you can use sched for event scheduling:

The sched module defines a class which implements a general purpose
event scheduler

import sched
import time

event_schedule = sched.scheduler(time.time, time.sleep)

def do_something():
    print("Hello, World!")
    event_schedule.enter(30, 1, do_something, (sc,))

event_schedule.enter(30, 1, do_something, (s,))
event_schedule.run()

Now in order to kill a background process on Windows, you simply need to run:

taskkill /pid processId /f

Where processId is the ID of the process you want to kill.

Make sure only a single instance of a program is running

The following code should do the job, it is cross-platform and runs on Python 2.4-3.2. I tested it on Windows, OS X and Linux.

from tendo import singleton
me = singleton.SingleInstance() # will sys.exit(-1) if other instance is running

The latest code version is available singleton.py. Please file bugs here.

You can install tend using one of the following methods:

easy_install tendo
pip install tendo
manually by getting it from http://pypi.python.org/pypi/tendo

How to run Python script only during certain hours of the day?

You should use cron jobs (if you are running Linux).

Eg: To execute your python script everyday between 7 am and 9 am.

0 7 * * * /bin/execute/this/script.py

minute: 0
of hour: 7
of day of month: * (every day of month)
of month: * (every month)
and week: * (All)

Now say you want to exit the program at 9 am .

You can implement your python code like this so that it gets terminated automatically after 2 hours.

import time

start = time.time()

PERIOD_OF_TIME = 7200 # 120 min

while True :
    ... do something

    if time.time() > start + PERIOD_OF_TIME : break

Linux - Check if python script is running in screen and run if not

ensure that flaky script keeps running

Your proposed approach could be made to work.
But a nanny script would be much simpler.
Call it e.g. nanny.sh.

#! /usr/bin/env bash

while true
do
    script.py
    sleep 1  # pause, so if script.py immediately dies we don't burn a core
done

Now we have replaced your "sometimes we randomly find script.py no longer running"
situation with one where we're confident that the nanny
is always running.

Diagnosing / fixing script.py is left as an exercise
for the reader. Fortunately it now is a less urgent matter.