Downloading Files from Ftp with R

Downloading files from ftp with R

Use the curl library to extract the directory listing

> library(curl)
> url = "ftp://ftp.pride.ebi.ac.uk/pride/data/archive/2015/11/PXD000299/"
> h = new_handle(dirlistonly=TRUE)
> con = curl(url, "r", h)
> tbl = read.table(con, stringsAsFactors=TRUE, fill=TRUE)
> close(con)
> head(tbl)
                                                 V1
1  12-0210_Druart_Uterus_J0N-Co_1a_ORBI856.raw.mzML
2  12-0210_Druart_Uterus_J0N-Co_2a_ORBI857.raw.mzML
3  12-0210_Druart_Uterus_J0N-Co_3a_ORBI858.raw.mzML
4 12-0210_Druart_Uterus_J10N-Co_1a_ORBI859.raw.mzML
5 12-0210_Druart_Uterus_J10N-Co_2a_ORBI860.raw.mzML
6 12-0210_Druart_Uterus_J10N-Co_3a_ORBI861.raw.mzML

Paste the relevant ones on to the url and use

urls <- paste0(url, tbl[1:5,1])
fls = basename(urls)
curl_fetch_disk(urls[1], fls[1])

Import File from FTP to R

I think you get this error message because function download.file() has no argument named credentials.

I would try to pass the credentials as discussed here:

url = "ftp://username:password@ftppath/www_logs/testfolder/test.csv"
download.file(url, destfile = "test.csv")

If you want to load the file into an R data.frame, you could try something like this:

library(RCurl) 
url <- "ftp://ftppath/www_logs/testfolder/test.csv"
text_data <- getURL(url, userpwd = "username:password", connecttimeout = 60)
df <- read.csv(text = text_data)

Using R to download newest files from ftp-server

This should work

library(RCurl)
url <- "ftp://yourServer"
userpwd <- "yourUser:yourPass"
filenames <- getURL(url, userpwd = userpwd,
             ftp.use.epsv = FALSE,dirlistonly = TRUE) 

-

times<-lapply(strsplit(filenames,"[-.]"),function(x){
  time<-paste(c(substr(x[1], nchar(x[1])-3, nchar(x[1])),x[2:6]),
        collapse="-")
  time<-as.POSIXct(time, "%Y-%m-%d-%H-%M-%S", tz="GMT")
})
ind <- which.max(times)
dat <- try(getURL(paste(url,filenames[ind],sep=""), userpwd = userpwd))

So datis now containing the newest file

To make it reproduceable: all others can use this instead of the upper part use

filenames<-c("FileA2014-03-05-10-24-12.csv","FileB2014-03-06-10-25-12.csv") 

Download .RData and .csv files from FTP using RCurl (or any other method)

You can try breaking it into two steps: first download the file, then load it.

download.file(downloadURL, "temp.rData")
load("temp.rData")

or sticking with rCurl you can try:

bin = getBinaryURL(downloadURL, ...yourOtherParams...) 
writeBin(bin, "temp.rData")  
load("temp.rData")

Download zipped files from ftp via RCurl

Here is my solution:

library(RCurl)

url<- "ftp://adress/"
filenames <- getURL(url, userpwd="USER:PASSWORD", ftp.use.epsv = FALSE, dirlistonly = TRUE) #reading filenames from ftp-server
destnames <- filenames <-  strsplit(filenames, "\r*\n")[[1]] # destfiles = origin file names
con <-  getCurlHandle( ftp.use.epsv = FALSE, userpwd="USER:PASSWORD")
mapply(function(x,y) writeBin(getBinaryURL(x, curl = con, dirlistonly = FALSE), y), x = filenames, y = paste("C:\\temp\\",destnames, sep = "")) #writing all zipped files in one directory

Hopefully for anybody it's usefull!
Regards,
Florian

---

Related Topics