Running Multiple Commands Simultaneously from Python

Running multiple commands simultaneously from python

You could use the subprocess module and have all three running independently: use subprocess.Popen. Take care in setting the "shell" parameter correctly.

Use the wait() or poll() method to determine when the subprocesses are finished.

How do I run multiple commands with python subprocess( ) without waiting for the end of each command?

Switch out your subprocess.run(command_lst) with Popen(command_lst, shell=True) in each of your scripts and and loop through the command list like the example below to run the processes in parallel.

This is how you implement Popen to run processes in parallel using arbitrary commands for simplicity.

from subprocess import Popen

commands = ['ls -l', 'date', 'which python']

processes = [Popen(cmd, shell=True) for cmd in commands]

Run multiple bash commands simultaneously in Python

you're just running one bash with 3 commands in it.

If the commands aren't setting variables or depending from each other (else you would not be able to parallelize them), maybe you could create 3 subprocess.Popen instances instead:

commands = '''
bashcmd1
bashcmd2
bashcmd3
'''

for process in [subprocess.Popen(['/bin/bash', '-c', line], stdout=subprocess.PIPE) 
                for line in commands.split("\n") if line]:  # filter out blank lines
    out, err = process.communicate() # or just rc = process.wait()
    # print out & err

that command first create a list comprehension of Popen objects (list not generator so the processes start immediately), then perform a communicate to wait for completion (but other processes are running in the meanwhile)

The upside is that you can apply this technique to any script containing commands, and you don't need to use the shell & capability (more portable, including Windows provided you're using ["cmd","/c" prefix instead of bash)

Python - run two commands at the same time

This can be achieved by using the multiprocessing module in python, please find the code below

#!/usr/bin/python
from multiprocessing import Process,Queue
import random
import time

def printrand():
   #Checks whether Queue is empty and runs
   while q.empty():
      rand = random.choice(range(1,100))
      time.sleep(1)
      print rand


if __name__ == "__main__":
   #Queue is a data structure used to communicate between process 
   q = Queue()
   #creating the process
   p = Process(target=printrand)
   #starting the process
   p.start()
   while True:
      ip = raw_input("Write something: ")
      #if user enters stop the while loop breaks
      if ip=="stop":
         #Populating the queue so that printramd can read and quit the loop
         q.put(ip)
         break
   #Block the calling thread until the process whose join() 
   #method is called terminates or until the optional timeout occurs.
   p.join()

---

Related Topics