Creating a symbolic in shared volume of docker and accessing it in host machine
You can create a symbolic link with any path name you want. When you access this, it's used as a normal filesystem path in its own context; if it's a relative path, it's accessed relative to the location of the link. If you have the same filesystem in multiple contexts (a bind-mounted Docker directory in both the host and a container; a remote filesystem) it's possible a symlink will resolve correctly in one context but not the other.
In your example:
- The symlink points at the absolute path
/workspace/log/project_info_hostinfo_timeinfo_exe_param.log
- Inside the container, the
/workspace
directory is the mounted host directory, so it works - Outside the container, there is no
/workspace
directory, so it doesn't work
Also in your example, the link and its target are in the same directory. This means that if the link target is just a filename, it will be looked up in the same directory as the link. That avoids the problem of the absolute paths being different.
import os
# If the link target is in the same directory, just use a filename,
# not an absolute path
fname = 'project_info_hostinfo_timeinfo_exe_param.log'
symlink_name = '/workspace/log/project_info.log'
os.symlink(fname, symlink_name)
It's often helpful to create a symlink as a relative path to avoid problems with relocating directory trees; ln -s ../assets/index.html .
, for example, will still point at the same place even if it's in a container context or your colleague has a different directory structure on their workstation.
Mount host directory with a symbolic link inside in docker container
Symlinks are a big challenge inside docker. In your case you can mount both directories:
-v /home/test/:/home/test -v /mnt/mountedfile:/mnt/mountedfile
For symbolic links to work both inside and outside the container, they have to be absolute paths and use exactly the same names.
In general, symlinks do not work inside docker. I found this the hard way.
Symbolic Link Host to Docker Container
For further investigation about this question. I would like to notify that I've "solved" my issue with the same approach than @Kai Hofstetter in the following post: How to mount a directory in the docker container to the host?
How can I make symlinks made from inside docker linux containers to be seen from a windows host (maybe involving samba, if needed)
TL;DR
- Run git-bash as administrator.
- Issue
export MSYS=winsymlinks:nativestrict
in git-bash. - From there on,
ln -s
works in windows. - Links are seen from inside the docker.
Details
We'll walk thru these steps:
- Preparation: Prepare a temporary dir with some files within the
abc
directory. - See it fail: We'll try to make a symlink and see it fail.
- Create symlink: We'll create the symlink in windows and see it. We'll point
xyz
toabc
. - Run docker: We'll then run docker with
ubuntu
and change contents inxyz
. - Check in ubuntu container: We'll see the changes also in
abc
from within the docker. - Check in windows host: Well check both
abc
andxyz
from ouside the container.
1. Preparation
- In a git-bash go to
/c
and create a temporary dirtmp
. - Inside it, create an
abc
dir and throw some contents there.
cd /c
mkdir tmp
cd tmp/
mkdir abc
cd abc/
echo 1111 > old_1
echo 2222 > old_2
echo 3333 > old_3
Here's a sample session:
2. See it fail
First let's try the "normal" way and see it fail.
- In a git-bash, navigate to
/c/tmp
- Then do a symlink making
xyz
to point toabc
:ln -s abc xyz
- See it fails, by ls-ing the
tmp
and seexyz
is a regular dir. - To be sure, create new content in
xyz
and see it's not there inabc
.
Try to create the link. It will not become a symlink, but rather create a copy of the directory.
cd /c/tmp/
ln -s abc xyz
Create new_bad
in xyz
and don't see it in abc
.
cd xyz/
touch new_bad
cd ../abc/
ls -l
Clear the wrong xyz
rm -Rf xyz/
Here's a sample session:
3. Create symlink
Here it comes the real stuff. The inspiration comes from @Slayvin's answer here, as well as here Git Bash shell fails to create symbolic links and the official git-for-windows repo here https://github.com/git-for-windows/git/pull/156
- First open a new git-bash in Administrator mode. The reason is that only admins can create links in windows.
- Once you are a CLI admin, navigate to the destination and set this evironment variable:
export MSYS=winsymlinks:nativestrict
This will tell the runtime subsytem of git-bash to actually use the symlinks feature. As we are admins we'll succeed.
- The do just "normal symlinks" as you would expect:
ln -s abc xyz
It works!!! Now next move is to test within docker!
NOTE: As per Sebastian's answer here https://stackoverflow.com/a/40914277/1315009 you DON'T need to be administrator to create symlinks in git-bash if you enabled the developer tools. In the search-bar write for developers
and enable it:
4. Run docker + 5. Check in docker
- The bash with admin privileges is no longer needed. So we'll close it and re-instantiate a "normal" bash.
- In it, run an ubuntu continainer with docker. Use
-it
to interact with the ubuntu's bash. Usewinpty
to allow-it
to work. - Bind-mount the
/c/tmp
directory so bothabc
andxyz
are reachable. I chose to mount it to/files
. - From inside,
cd /files
and see thatxyz
is actually a symlink. - Create some new content in
xyz
Run and see:
winpty docker run -it --rm --mount type=bind,source="c:\tmp",target=/files --name ubuntu-link ubuntu
cd /files/
ls -l
Create content:
cd xyz
echo "yeaaahh" > new_good
Check it's really a symlink by going to abc
:
cd ..
cd abc/
cat new_good
Sample session:
6. Check in windows host
- Step out from the docker. Stay in the git-bash.
- Again: This git-bash does not need to be privileged. The only moment we had to be admin was to "create" the symlink in windows.
- From the unprivileged bash, explore
abc
as well asxyz
and see that there's the content we created from inside the docker, appearing in both the original directory and in the symlink.
Sample session:
Final check
We can finally go to a classical CMD
to see how it looks like. We can see it's clearly indicated that it's a symlink for a directory and we also see the target there:
Golden touch
If you have the "developer tools" activates as stated above, the only missing thing is the ENV VAR.
We can set this by editing the .bashrc
at your windows home:
By doing this we can just use git-bash completely normally and start creating the symlinks from windows without any overload.
Caution
The symlinks created this way work from windows and are seen from inside docker. But not the oposite. If you create symlinks inside the container they don't get created in windows.
Therefore, in mounted volumes, setup the symlinks always from git-bash and consume them from the container. If you create them from the container, they still can be consumed from the container. But won't be usable from windows.
Conclussion
It can be done fully from the linux flavour commands via git-bash. Only that you need to be admin to create the links and tell the git-bash runtime to use that feature. And that the link needs to be done from windows, instead from inside the ubuntu.
docker container can't open symlinked file in volume mount during startup
A symlink points to a file in the file system. So the symbolic link in the docker host's file system points to a file in that file system.
If you mount that symbolic link into the container it is interpreted within file system of the container. This means that it still points to /opt/my-app/test1.cfg
. And this file does not exist in the container.
You can mount the original directory, can't you?
- /opt/my-app/:/opt/my-app
I guess you don't want to mount the whole directory. You might want to exclude some files and that's why you created a temp directory with symbolic links.
In this case it might be possible to create the temp directory and use hard links, because they are just another file entry which point to the same files. But this is only possible if the hard link is created on the same partition.
See What is the difference between a hard link and a symbolic link
Maybe docker's bind-propagation capapilities can help you too.
Why are symlinks behaving like this in docker?
It is not a docker issue. Symbolic links just behave that way.
Also take a look at Mount host directory with a symbolic link inside in docker container
How does volume mount from container to host and vice versa work?
Volumes are used for persistent storage and the volumes persists independent of the lifecycle of the container.
We can go through a demo to understand it clearly.
First, let's create a container using the named volumes approach as:
docker run -ti --rm -v DataVolume3:/var ubuntu
This will create a docker volume named DataVolume3
and it can be viewed in the output of docker volume ls
:
docker volume ls
DRIVER VOLUME NAME
local DataVolume3
Docker stores the information about these named volumes in the directory /var/lib/docker/volumes/
(*):
ls /var/lib/docker/volumes/
1617af4bce3a647a0b93ed980d64d97746878564b141f30b6110d0818bf32b76 DataVolume3
Next, let's write some data from the ubuntu container at the mounted path var
:
echo "hello" > var/file1
root@2b67a89a0050:/# cat /var/file1
hello
We can see this data with cat
even after deleting the container:
cat /var/lib/docker/volumes/DataVolume3/_data/file1
hello
Note: Although, we are able to access the volumes like shown above but it not a recommended practice to access volumes data like this.
Now, next time when another container uses the same volume then the data from the volume gets mounted at the container directory specified as part of -v
flag.
(*) The location may vary based on OS as pointed by David and probably can be seen by the docker volume inspect
command.
Related Topics
How to Send Http Requests to Flask Server
What Does "The Following Packages Will Be Superseded by a Higher Priority Channel" Mean
Security of Python's Eval() on Untrusted Strings
Using Quotation Marks Inside Quotation Marks
Determine Whether Integer Is Between Two Other Integers
How to Print the Value of a Tensor Object in Tensorflow
How to Parse a Yaml File in Python
How to Find the Mime Type of a File in Python
How to Use 'Return' to Get Back Multiple Values from a for Loop? How to Put Them in a List
Running Unittest with Typical Test Directory Structure
How to Open Process Again in Linux Terminal
Pytest Running with Another Version of Python
Match a Whole Word in a String Using Dynamic Regex
How to Write to a Python Subprocess' Stdin
What Does the _File_ Variable Mean/Do
Beautifulsoup Grab Visible Webpage Text