Python App Does Not Print Anything When Running Detached in Docker

Python app does not print anything when running detached in docker

Finally I found a solution to see Python output when running daemonized in Docker, thanks to @ahmetalpbalkan over at GitHub. Answering it here myself for further reference :

Using unbuffered output with

CMD ["python","-u","main.py"]

instead of

CMD ["python","main.py"]

solves the problem; you can see the output (both, stderr and stdout) via

docker logs myapp

now!

Python 3 app running in docker-compose will not print using print('text', end='\r')

The output of docker-compose up is aggregated ¹ and effectively line-buffered ² (see Sample log), meaning it only prints when a \n is seen. There are other anecdotes ^3,4 of this behaviour with \r.

You can see that it is actually printed by running docker-compose run app ⁵.

References:

1. https://docs.docker.com/compose/reference/up/
2. https://eklitzke.org/stdout-buffering
3. https://github.com/docker/compose/issues/1549#issuecomment-745086053
4. Output of pv on docker-compose startup not working as expected
5. https://docs.docker.com/compose/faq/#whats-the-difference-between-up-run-and-start

Workarounds

1. Run a single app directly

Run docker-compose run app, as mentioned above.

The output of docker-compose run goes directly to the stdout of your terminal.

2. Duplicate the output to a volume-mounted file and watch it

tee the output in CMD to a file in a mounted volume and then tail -F that file.

touch app.log
docker-compose up & tail -F app.log

(Ctrl+C to exit from tail -F app.log.)

Sample files

app.py:

from time import sleep
  
for i in range(10):
    print('text: {}'.format(i), end='\r')
    sleep(0.3)

Dockerfile:

FROM python:3.7-alpine
WORKDIR /app

COPY . .
# CMD [ "python3", "-u", "app.py" ]
CMD python3 -u app.py > app.log

docker-compose.yml:

version: "3.9"
services:
  app:
    build: .
    volumes:
      - ./:/app

---

Sample log

From docker-compose up without workaround:

==> /var/lib/docker/containers/xxx/xxx-json.log <==
{"log":"text: 0\rtext: 1\rtext: 2\rtext: 3\rtext: 4\rtext: 5\rtext: 6\rtext: 7\rtext: 8\rtext: 9\r","stream":"stdout","time":"xxx"}

Thus, print('text: 0\rtext: 1\rtext: 2\rtext: 3\rtext: 4\rtext: 5\rtext: 6\rtext: 7\rtext: 8\rtext: 9\r') effectively prints text: 9\r.

How to view logs in macOS

1. Open a shell in the VM using nc -U ~/Library/Containers/com.docker.docker/Data/debug-shell.sock.
   
   Ref: https://docs.docker.com/desktop/mac/release-notes/2.x/#docker-desktop-community-2400
2. Run tail /var/lib/docker/containers/*/*.log.

python script error in docker detached mode, but executes correctly in docker attached mode

What seemed to have fixed the problem was installing sudo in the docker image:

RUN apt-get install -y sudo

Strange, since I was executing docker exec as root already ... (?)

docker log don't show python print output

After a long search I found this https://serverfault.com/a/940357

Add flush=True

print(datetime.now(), flush=True)

Or add PYTHONUNBUFFERED: 1 to docker-compose.yml which is added by PyCharm by default

version: '3.6'

services:

  test:
    ....
    environment:
      PYTHONUNBUFFERED: 1  # <---
    ....

---

Related Topics