Function not found in R doParallel 'foreach' - Error in { : task 1 failed - could not find function raster
In the vignette of foreach and the help page of foreach, the argument .packages
is pointed out as necessary to provide when using parallel computation with functions that are not loaded by default. So your code in the first example should be:
ptime<-system.time({
r <- foreach(z = 1:length(files),
.combine=cbind,
.packages='raster') %dopar% {
# some code
# and more code
}
})
Some more explanation
The foreach
package does a lot of setting up behind the scenes. What happens is the following (in principle, technical details are a tad more complicated):
foreach
sets up a system of "workers" that you can see as separate R sessions that are each committed to a different core in a cluster.The function that needs to be carried out is loaded into each "worker" session, together with the objects needed to carry out the function
each worker calculates the result for a subset of the data
The results of the calculation on the different workers is put together and reported in the "master" R session.
As the workers can be seen as separate R sessions, packages from the "master" session are not automatically loaded. You have to specify which packages should be loaded in those worker sessions, and that's what the .package
argument of foreach
is used for.
Note that when you use other packages (e.g. parallel
or snowfall
), you'll have to set up these workers explicitly, and also take care of passing objects and loading packages on the worker sessions.
permutation error with parallel and foreach package task 1 failed - could not find error
The code below should work. The reason you are getting this error is that the foreach function creates new workers in new sessions, and in those sessions functions and packages in the main session aren't exported. So note the .export
argument, and the .packages
argument.
# Permutations function ---------------------------------------------------
doNullPermutations <- function(nit, snps, estuaries) {
x <- foreach(iteration = 1:3, # do 100 iterations
.combine = rbind,
.export = ls(globalenv()),
.packages = 'tidyverse') %dopar% one_iteration(snps, estuaries)
return(x)
}
perms_res_3est_10kperm <- doNullPermutations(1, snps = 20083, estuaries = 3)
R Parallel Programming: Error in { : task 1 failed - could not find function % %
you have to use .packages = c("magrittr", ...)
and include all the packages, which are necessary to run the code within the foreach
loop. However, .packages = "foreach"
is not helping.
See, you can imagine that all the packages you define in .packages
are forwareded / loaded in each parallel worker.
Error in { : task 1 failed - could not find function init_status.icm
I changed the numbers of cores (ncores) to 1 in both functions (simulate and baseline) and succeed it.
control.icm actually set the core to 1 as you can see it in the image below:
https://i.stack.imgur.com/xnx69.png
Related Topics
Converting Geo Coordinates from Degree to Decimal
Error in If/While (Condition) {:Argument Is of Length Zero
Convert Currency with Commas into Numeric
Sum All Values in Every Column of a Data.Frame in R
Adding Text to a Grid.Table Plot
Ggplot2: Curly Braces on an Axis
Convert Seconds to Days: Hours:Minutes:Seconds
Creating a Summary Statistical Table from a Data Frame
Plot Polynomial Regression Curve in R
Categorize Numeric Variable with Mutate
Cannot Install an R Package from Github
Remove Everything After Space in String
R Split Numeric Vector at Position
Gsub() in R Is Not Replacing '.' (Dot)
Missing Legend with Ggplot2 and Geom_Line
How to Increase Font Size in a Plot in R
Fill and Border Colour in Geom_Point (Scale_Colour_Manual) in Ggplot