Deleting Tmp Files

Deleting tmp files

You can get the temp directory for the current R session. It does not change when called several times

tmp_dir <- tempdir()
tmp_dir
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh"
tempdir()
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh"

The temp directory contains the temp files and directory for the current
R session

list.files(tmp_dir)
#> [1] "file16dc20539ab"  "file16dc4ad71f"   "file16dc5bab1716"
#> [4] "file16dc74d65663"

The session temp directory is in the temp directory of the system. You can use this path if you want to delete all in the temp directory of the system (not recommended though because it is for all the system, not just R temp files)

dirname(tmp_dir)
#> [1] "C:/Users/chris/AppData/Local/Temp"

This path is also contains in an environnement variable for the OS. (Obviously, I am on windows)

Sys.getenv("TEMP")
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp"
shell("echo %TMP%", intern = T) # command line from R on windows
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp"

tempfile() gives the path of a possible temporary file, in the
tempdir() directory by default, with no file extension. The file is
not created and tempfile gives different values when calls several
times

tmp_file <- tempfile()
tmp_file
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh\\file16dc202636f"
file.exists(tmp_file)
#> [1] FALSE

tempfile() # new file path when called again
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh\\file16dc72594e58"

We can write something to tmp_file.

# file is created by writeLines if it does not exist (this is the case here)
writeLines("This is a temp file", con = tmp_file)
file.exists(tmp_file)
#> [1] TRUE

We can read from this file

readLines(tmp_file)
#> [1] "This is a temp file"

Now if you want to delete this file

file.remove(tmp_file)
#> [1] TRUE
file.exists(tmp_file)
#> [1] FALSE

If you want to delete all files in the R session temp folder, you can use
file.remove on a list of files. For this example purpose, I deleted all
temp file beginning with "file" ("^file" is a regex for that pattern). There are more than I created - R session seems to create some temp file along the way.

files <- list.files(tmp_dir, full.names = T, pattern = "^file")
files
#>  [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc1a6a6e15"
#>  [2] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc1ff572fc"
#>  [3] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc20539ab" 
#>  [4] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc2e2227b8"
#>  [5] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc4ad71f"  
#>  [6] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc513c35b6"
#>  [7] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc570a473f"
#>  [8] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc5bab1716"
#>  [9] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc6e102bd4"
#> [10] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc6f253f90"
#> [11] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/file16dc74d65663"
file.remove(files)
#> Warning in file.remove(files): impossible d'effacer le fichier 'C:
#> \Users\chris\AppData\Local\Temp\RtmpmusYkh/file16dc1ff572fc', à cause de
#> 'Permission denied'
#>  [1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE

I get a warning because there is a file I can't delete (probably in use by R right now)

If you want to remove a folder you can use unlink too

# create a new directory under tempdir
dir.create(dir1 <- file.path(tempdir(), "testdir"))
# create 2 file under this new directory
file.create(file1 <- tempfile(tmpdir = dir1))
#> [1] TRUE
file.create(file2 <- tempfile(tmpdir = dir1))
#> [1] TRUE
file1
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/testdir\\file16dc26b5cb7"
file2
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/testdir\\file16dc2b0816fe"
list.files(dir1, full.names = T)
#> [1] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/testdir/file16dc26b5cb7" 
#> [2] "C:\\Users\\chris\\AppData\\Local\\Temp\\RtmpmusYkh/testdir/file16dc2b0816fe"

# we can delete the all directory with `unlink`. It deletes also the directory
unlink(dir1, recursive = T)
dir.exists(dir1)
#> [1] FALSE

How does Linux manage deleting temporary files?

There's no bit that marks a file as a temporary file.

Every inode has a link count field, which is the number of directory entries that refer to the file. Every time you make a hard link to a file this count is increased, and when you remove a name it's decreased; when the count goes to zero, the file is deleted (the inode is marked as available, and all the data blocks are put on the free list).

When a file is opened in a process, a copy of the inode is kept in the kernel's file table, and the number of file handles that refer to it are added into the link count in this copy. When a process closes its file descriptor, the link count is decremented. The file isn't actually removed until this in-memory link count drops to zero. This is what keeps a file on disk while it's open, even if all the names are removed.

So when you create a temporary file, it performs the following steps:

1. Creates the file. The on-disk inode link count = 1.
2. Opens the file. The kernel inode link count = 2.
3. Removes the filename. The kernel inode link count = 1.

At this point, the process can keep using the temporary file, but it can't be opened by another process because it has no name.

When the process closes the file handle, the link count goes to 0, and the file is deleted.

Recent versions of Linux have an O_TMPFILE flag to open(2) that automates this. Instead of specifying a filename, you just specify the directory, which is just used to find a filesystem to hold the file data. When this is used, it effectively does all 3 steps above in one call, but it never actually creates the filename anywhere (so race conditions and name conflicts are avoided).

Are files from /tmp automatically deleted by AWS Lambda?

No, files from /tmp are not automatically deleted.

---

The AWS Lambda FAQs state:

To improve performance, AWS Lambda may choose to retain an instance of your function and reuse it to serve a subsequent request, rather than creating a new copy. To learn more about how Lambda reuses function instances, visit our documentation. Your code should not assume that this will always happen.

As per the above doc and experience, you may find an empty or "pre-used" /tmp directory depending on if AWS Lambda has reused a previous Lambda environment for your current request.

This may, or may not be suitable depending on the use case & there's no guarantees so if you need to ensure a clean /tmp directory on every function invocation, clear the /tmp directory yourself.

---

Is there a flush() sort of function?

No, AWS does not (& shouldn't) offer a way programmatically via their SDK as this is related to file I/O.

How to delete all files inside the /tmp directory will be dependent on the Lambda function runtime.

For Python, try:

from subprocess import call
...
call('rm -rf /tmp/*', shell=True)

how to delete Temp files in robot framework?

rasjani's comment actually got me thinking - try the Run Process call with shell=${True}.

Run Process    del    %TEMP%/*.*    /s    /f    /q     shell=${True}

The comment has reminded me something from the official docs of subprocess:

On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.

---

Related Topics