Raster Package Taking All Hard Drive

Weird R problem: Internet keeps disconnecting whenever I run any raster calculation using the raster package

The raster package does not use your network for normal computations (it only uses it to download data when you use the getData function). So it is directly related to that. It has to be related to what you computer does when files are created.

All I can think of is that you have a system that automatically copies your data to the cloud. So if you create a bunch of big files, that would slow down the Internet.

As for the filling up of your disk; that is because you are using very large files with functions that then need save these to disk (in a temp file). You can use other functions such as calc to use filenames. There would still be files, of course, but you may cut out intermediate files; if there are any. See raster::removeTmpFiles for removing them without exiting R.

writing a raster stack on disk with writeRaster in R changes the range of the values for each layer

I'd try to run something like

cellStats(rast.stack, mean)
cellStats(rast.stack, min)
cellStats(rast.stack, max)

on the original and reloaded stack.

This because I suspect that the differences you are reporting are not "real". In particular, it seems suspect to me that in the original raster all the bands cover the full 0 to 65535 range: that is difficult to be happening (would mean a complete "saturation" of the possible DN range in all bands).

What's in my opinion happening here is that the "ranges" of your "original" stack are only showing the "possible" min-max range given the data type. writeRaster is saving the computed statistics in the XML file associated to the tiff (see R: how to write a raster to disk without auxiliary file?), so that when you read it in again the "ranges" values are "correct".

HTH.

Failure during raster IO possibly related to raster size?

The issue was solved by changing my folders (and the rasters saved in the folders) from a D drive (external 2 TB solid state hard drive) to my computer's C drive.
I can still run codes working with smaller raster sizes directly on my D drive (which, in this case, actually still had plenty of memory left on it).

Can't write compressed geotiffs to disk using terra package

If you find a bug in a package, then the best place to report it would be on github or such (here for the terra package.

I only saw your comment after writing the answer; so for posterity, below I show that compression works; at least in the current version on CRAN and in the development version.

library(terra)
#terra version 1.1.4

r <- rast(ncol=1000, nrow=1000, xmin=0, xmax=10, ymin=0, ymax=10)
values(r) <- sqrt(1:ncell(r))

ff <- replicate(4, paste0(tempfile(), ".tif"))

writeRaster(r, ff[1], overwrite=T) #complex raster no compression
writeRaster(r, ff[2],  gdal=c("COMPRESS=LZW"))
writeRaster(r>100, ff[3]) #simple raster no compression
writeRaster(r>100, ff[4],   gdal=c("COMPRESS=LZW"))

sapply(ff, file.size, USE.NAMES=FALSE)
#[1] 4003716 2886677 4003712  146519
 

---

Related Topics