What's Wrong With My Function to Load Multiple .Csv Files into Single Dataframe in R Using Rbind

What's wrong with my function to load multiple .csv files into single dataframe in R using rbind?

There's a lot of unnecessary code in your function. You can simplify it to:

load_data <- function(path) { 
  files <- dir(path, pattern = '\\.csv', full.names = TRUE)
  tables <- lapply(files, read.csv)
  do.call(rbind, tables)
}

pollutantmean <- load_data("specdata")

Be aware that do.call + rbind is relatively slow. You might find dplyr::bind_rows or data.table::rbindlist to be substantially faster.

Read and rbind multiple csv files

Find files (list.files) and read the files in a loop (lapply), then call (do.call) row bind (rbind) to put all files together by rows.

myMergedData <- 
  do.call(rbind,
          lapply(list.files(path = "N:/Ring data by cruise"), read.csv))

Update: There is a vroom package, according to the manuals it is much faster than data.table::fread and base read.csv. The syntax looks neat, too:

library(vroom)
myMergedData <- vroom(files)

Reading Multiple CSV files as data frames in R

There are so many ways!!

setwd("C:/your_path_here")
fnames <- list.files()
csv <- lapply(fnames, read.csv)
result <- do.call(rbind, csv)


********  ********  ********  ********  ********  ********  ********  ********  ********  ********  


filedir <- setwd("C:/your_path_here")
file_names <- dir(filedir)
your_data_frame <- do.call(rbind,lapply(file_names,read.csv))


********  ********  ********  ********  ********  ********  ********  ********  ********  ********  


filedir <- setwd("C:/your_path_here")
file_names <- dir(filedir)
your_data_frame <- do.call(rbind, lapply(file_names, read.csv, skip = 1, header = FALSE))


********  ********  ********  ********  ********  ********  ********  ********  ********  ********  


filedir <- setwd("C:/your_path_here")
file_names <- dir(filedir)
your_data_frame <- do.call(rbind, lapply(file_names, read.csv, header = FALSE))


********  ********  ********  ********  ********  ********  ********  ********  ********  ******** 


# 
temp <- setwd("C:/your_path_here")
temp = list.files(pattern="*.csv")
myfiles = lapply(temp, read.delim)


********  ********  ********  ********  ********  ********  ********  ********  ********  ******** 


# Here is another options to convert the .csv files into one data.frame. Using R base functions. 
# This is order of magnitude slower than the options below.

files <- setwd("C:/your_path_here")
# Get the files names
files = list.files(pattern="*.csv")
# First apply read.csv, then rbind
myfiles = do.call(rbind, lapply(files, function(x) read.csv(x, stringsAsFactors = FALSE)))

library(readr)
library(dplyr)
tbl = lapply(files, read_csv) %>% bind_rows()


********  ********  ********  ********  ********  ********  ********  ********  ********  ******** 


# LIST OF FILE PATHS

library(readr)
library(stringr)

List_of_file_paths <- list.files(path ="C:/your_path_here/", pattern = ".csv", all.files = TRUE, full.names = TRUE)


********  ********  ********  ********  ********  ********  ********  ********  ********  ********


# LIST OF FILES IN FOLDER
xlist<-list.files(pattern = "*.csv")

for(i in xlist) { 
  x <- read.csv((i))
  assign(i, x)
    }


********  ********  ********  ********  ********  ********  ********  ********  ********  ********

Using a loop inside a function to load dataframes from CSV files

library(dplyr)
library(readr)
library(purrr)

trading.opencsv <- function(rd) {
  asset.directory <- paste0(rd, "list.csv")
  assetlist <- read_csv(asset.directory)
  print(assetlist$Market)
  map(assetlist$Market, ~ read_csv(paste0(rd, .x, ".csv"))) %>%
    bind_rows()
}

How to import multiple .csv files at once?

Something like the following should result in each data frame as a separate element in a single list:

temp = list.files(pattern="*.csv")
myfiles = lapply(temp, read.delim)

This assumes that you have those CSVs in a single directory--your current working directory--and that all of them have the lower-case extension .csv.

If you then want to combine those data frames into a single data frame, see the solutions in other answers using things like do.call(rbind,...), dplyr::bind_rows() or data.table::rbindlist().

If you really want each data frame in a separate object, even though that's often inadvisable, you could do the following with assign:

temp = list.files(pattern="*.csv")
for (i in 1:length(temp)) assign(temp[i], read.csv(temp[i]))

Or, without assign, and to demonstrate (1) how the file name can be cleaned up and (2) show how to use list2env, you can try the following:

temp = list.files(pattern="*.csv")
list2env(
  lapply(setNames(temp, make.names(gsub("*.csv$", "", temp))), 
         read.csv), envir = .GlobalEnv)

But again, it's often better to leave them in a single list.

Import and rbind multiple csv files with common name in R

Do you look for something like this?

do.call(rbind, lapply(list.files(path=".", pattern="AM-25"), read.table, header=TRUE, sep=","))

This would rbind together the matrices read from your csv files which contain the characters "AM-25".
The arguments for read.table could be different, depending on your csv files.

---

EDIT

I hope this works for the case that you do not know all possible five-letter prefixes of filenames in your directory:

##Get all different first five letter strings for all cvs files in directory "."                                                                                                                                                                                           
file.prefixes <- unique(sapply(list.files(path=".", pattern="*.csv"), substr, 1,5))

##Group all matching file names according to file.prefixes into a list                                                                                                                                                                                                     
file.list <- lapply(file.prefixes, function(x)list.files(pattern=paste("^",x,".*.csv",sep=""), path="."))
names(file.list) <- file.prefixes ##just for convenience                                                                                                                                                                                                                   

##parse all csv files in file.list, create a list of lists containing all tables for each prefix                                                                                                                                                                           
tables <- lapply(file.list, function(filenames)lapply(filenames, function(file)read.table(file, header=TRUE)))

##for each prefix, rbind the tables. Result is a list of length being length(file.prefixes)                                                                                                                                                                                
##  each containing a matrix with the combined data parsed from the files that match the prefix                                                                                                                                                                            
joined.tables <- lapply(tables, function(t)do.call(rbind, t))

##Save tables to files                                                                                                                                                                                                                                                     
for (prefix in names(joined.tables))write.table(joined.tables[[prefix]], paste(prefix, ".csv", sep=""))

---

Related Topics