How to Resize /Dev/Shm

How to resize /dev/shm?

1. Edit file /etc/fstab (with sudo if needed).
2. In this file, try to locate a line like this one : none /dev/shm tmpfs defaults,size=4G 0 0.

Case 1 - This line exists in your /etc/fstab file:

• Modify the text after size=. For example if you want an 8G size, replace size=4G by size=8G.
• Exit your text editor, then run (with sudo if needed) $ mount -o remount /dev/shm.

Case 2 - This line does NOT exists in your /etc/fstab file:

• Append at the end of the file the line none /dev/shm tmpfs defaults,size=4G 0 0, and modify the text after size=. For example if you want an 8G size, replace size=4G by size=8G.
• Exit your text editor, then run (with sudo if needed) $ mount /dev/shm.

docker SHM_SIZE /dev/shm: resizing shared memory

You set shm_size in build, this will just affect build, you need to set it in service level, like next:

docker-compose.yaml:

version: "3.6"

services:

 #other services go here..
 postgres:
    restart: always
    image: postgres:10
    hostname: postgres
    container_name: fiware-postgres
    expose:
      - "5432"
    ports:
      - "5432:5432"
    networks:
      - default
    environment:
      - "POSTGRES_PASSWORD=password"
      - "POSTGRES_USER=postgres"
      - "POSTGRES_DB=postgres"
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    build:
      context: .
      shm_size: 256mb
    shm_size: 512mb

Dockerfile:

FROM postgres:10

RUN df -h | grep shm

Then, docker-compose up -d --build to start it and check:

shubuntu1@shubuntu1:~/66$ docker-compose --version
docker-compose version 1.24.0, build 0aa59064
shubuntu1@shubuntu1:~/66$ docker-compose up -d --build
Building postgres
Step 1/2 : FROM postgres:10
 ---> 0959974989f8
Step 2/2 : RUN df -h | grep shm
 ---> Running in 25d341cfde9c
shm             256M     0  256M   0% /dev/shm
Removing intermediate container 25d341cfde9c
 ---> 1637f1afcb81

Successfully built 1637f1afcb81
Successfully tagged postgres:10
Recreating fiware-postgres ... done
shubuntu1@shubuntu1:~/66$ docker exec -it fiware-postgres df -h | grep shm
shm             512M  8.0K  512M   1% /dev/shm

You can see in build time it shows 256m, but the runtime container it shows 512m.

Increase the size of /dev/shm in Azure ML Studio

The /dev/shm is a virtual filesystem for passing data between programs that implementation of traditional shared memory on Linux.

So you could not increase it via set up some options on Application Layout.

But for example, you can remount /dev/shm with 8G size in Linux Shell with administrator permission like root as follows.

mount -o remount,size=8G /dev/shm

However, it seems that Azure ML studio not support remote access via SSH protocol, so the feasible plan is upgrade the standard tier if using free tier at present.

pq: could not resize shared memory segment. No space left on device

This is because docker by-default restrict size of shared memory to 64MB.

You can override this default value by using --shm-size option in docker run.

docker run -itd --shm-size=1g postgres

or in docker-compose:

db:
  image: "postgres:11.3-alpine"
  shm_size: 1g

Check this out. More info here.

Hope this helps.

Docker - increase size of the /dev/shm

Just in case somebody stumbles upon this - it is now available as a CLI option to 'docker run':

  --shm-size                      Size of /dev/shm, default value is 64MB

How and when to use /dev/shm for efficiency?

You don't use /dev/shm. It exists so that the POSIX C library can provide shared memory support via the POSIX API. Not so you can poke at stuff in there.

If you want an in-memory filesystem of your very own, you can mount one wherever you want it.

mount -t tmpfs tmpfs /mnt/tmp, for example.

A Linux tmpfs is a temporary filesystem that only exists in RAM. It is implemented by having a file cache without any disk storage behind it. It will write its contents into the swap file under memory pressure. If you didn't want the swapfile you can use a ramfs.

I don't know where you got the idea of using /dev/shm for efficiency in reading files, because that isn't what it does at all.

Maybe you were thinking of using memory mapping, via the mmap system call?

Read this answer here: https://superuser.com/a/1030777/4642 it covers a lot of tmpfs information.

---

Related Topics