Oserror 38 [Errno 38] with Multiprocessing

[ERROR] OSError: [Errno 38] Function not implemented - Accessing trend deepsecurity.ComputersApi via Lambda

Resolved the issue by changing the python version from 3.8 to 3.7 in the lambda. The script now runs successfully

Multiprocessing + Flask OSError: [Errno 48] Address already in use / OSError: [Errno 38] Socket operation on non-socket

Replace

app.run(port=8000)

with

if __name__ == '__main__':
    app.run(port=8000)

When using multiprocessing, anything that is "top level" gets run in every thread. So both your main thread and the thread you're creating try to open port 8000

Python multiprocessing: Permission denied

For POSIX semaphores to work, the users need r/w access to shared memory (/dev/shm).

Check the permissions to /dev/shm. On my laptop (Ubuntu) it looks like this:

$ ls -ld /dev/shm
drwxrwxrwt 2 root root          40 2010-01-05 20:34 shm

To permanently set the correct permissions (even after a reboot), add the following to your /etc/fstab:

none /dev/shm tmpfs rw,nosuid,nodev,noexec 0 0

Haven't tried this, just copied from a forum post.

Multiprocess and Multiprocessing with no file IO: OSError: [Errno 24] Too many open files

The way to avoid changing the ulimit is to make sure that your process pool size does not increase beyond 1024. That's why 1000 works and 10000 fails.

Here's an example of managing the processes with a Pool which will ensure you don't go above the ceiling of your ulimit value:

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))

https://docs.python.org/3/library/multiprocessing.html#introduction

other threads recommend closing any opened files but I do not access
files in my code

You don't have files open, but your processes are opening file descriptors which is what the OS is seeing here.

---

Related Topics