How to Run an External Command Asynchronously from Python

How can I run an external command asynchronously from Python?

subprocess.Popen does exactly what you want.

from subprocess import Popen
p = Popen(['watch', 'ls']) # something long running
# ... do other stuff while subprocess is running
p.terminate()

(Edit to complete the answer from comments)

The Popen instance can do various other things like you can poll() it to see if it is still running, and you can communicate() with it to send it data on stdin, and wait for it to terminate.

Using asyncio to run command in a time interval and terminate it afterwards

os.system is a blocking operation, so it doesn't play well with asyncio. Whenever you use asyncio, all "longer" operations need to be asynchronous, or you lose all benefits of going with asyncio in the first place.

import asyncio

async def cmd():
    try:
        proc = await asyncio.create_subprocess_shell(
            "sleep 2 && date > my.txt",
            shell=True,
            stdout=asyncio.subprocess.DEVNULL,
            stderr=asyncio.subprocess.DEVNULL)
        await proc.communicate()

    except asyncio.CancelledError:
        proc.terminate()
        print("X")

async def run():
    try:
        await asyncio.wait_for(cmd(), 1)
        print("A")

    except asyncio.TimeoutError:
        print("B")

asyncio.run(run())

As @dim mentions in comments, asyncio.create_subprocess_exec will launch an external command asynchronously. wait_for will actually raise two exceptions: TimeoutError in the awaiting code, and CancelledError in the awaited code. We can use the latter to realise the operation was cancelled, and to terminate or kill our subprocess (as this is not done automatically).

How can I run a terminal command asynchronously using Python?

It looks like you are trying to make an RFCOMM connection.

Both rfcomm and hcitool have been deprecated by the BlueZ project back in 2017 so I wouldn't recommend starting a project using these.

BlueZ has APIs that are designed to be accessed from languages like Python and these are documented at: https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/doc and the official examples are at: https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/test

However, RFCOMM (or Serial Port Profile) connections can be done with the socket library in the standard Python releases. There are examples at https://stackoverflow.com/a/63382835/7721752

There are also examples in the above answer of using the Bluedot library to do the same.

How to asynchronously call a shell script from Python?

subprocess.call() and subprocess.run() creates a process, waits for it to finish, and returns a CompletedProcess object.

subprocess.Popen() creates a process and returns it. It is used under the hood of the previous functions. You can then wait for the process to finish, send it messages, or whatever else you want to do with it. The arguments are mostly the same as to call or run.

https://docs.python.org/3/library/subprocess.html

As a bit of elaboration, Popen is the python implementation of using the os to start a new process. os.fork() is a lower level that doesn't actually do what we want here, that would spawn another instance of the python interpreter with the same memory state as the current one. If you wanted to use the lower level syscall, os.spawn is closer to subprocess.run than os.fork.

To verify that Popen is doing what you want, this test program will pring "returncode = None", then wait 5 seconds, and print "returncode = 0"

from subprocess import Popen

p = Popen(["sleep", "5"])
print("started the proc")  # this will print immediately
p.poll()  # this checks if the process is done but does not block
print(f"p returncode = {p.returncode}")
p.wait()  # this blocks until the process exits
print(f"p returncode = {p.returncode}")

How can I call an async function inside a command in discord python?

The answer to this question is to add an await. before your async function.

@client.command()
async def hello(ctx):
   await ctx.send('hello')
   await facts(ctx)

async def facts(ctx):
   await ctx.send('facts')

And there you'd be able to call your async function.

Asynchronously receive output from long running shell commands with asyncio (Python 3.4+)?

Use get_lines() coroutines, to get shell commands output asynchronously and pass the coroutines to asyncio.as_completed(), to get the results in the order they finish:

#!/usr/bin/env python3.5
import asyncio
import sys
from asyncio.subprocess import PIPE, STDOUT

async def get_lines(shell_command):
    p = await asyncio.create_subprocess_shell(shell_command,
            stdin=PIPE, stdout=PIPE, stderr=STDOUT)
    return (await p.communicate())[0].splitlines()

async def main():
    # get commands output concurrently
    coros = [get_lines('"{e}" -c "print({i:d}); import time; time.sleep({i:d})"'
                       .format(i=i, e=sys.executable))
             for i in reversed(range(5))]
    for f in asyncio.as_completed(coros): # print in the order they finish
        print(await f)

if sys.platform.startswith('win'):
    loop = asyncio.ProactorEventLoop() # for subprocess' pipes on Windows
    asyncio.set_event_loop(loop)
else:
    loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

---

Related Topics