Mclapply Returns Null Randomly

R, the environment of mclapply and removing variables

You should call the gc function after removing the variable so that the memory associated with the object is freed by the garbage collector sooner rather than later. The rm function only removes the reference to the data, while the actual object may continue to exist until the garbage collector eventually runs.

You may also want to call gc before the first mclapply to make testing easier:

gc()
opt.Models = mclapply(1:100, mc.cores=20, function(i){
    res = loadResult(reg, id=i)    
    return(post.Process(res))
  })

# presumably do something with opt.Models...

rm(opt.Models)
gc()  # free up memory before forking

opt.Models = mclapply(1:100, mc.cores=20, function(i){
    res = loadResult(reg, id=i)    
    return(post.Process(res))
  })

Is mclapply() with mc.cores = 1 the same as lapply()?

The source code of parallel::mclapply contains this bit of code:

 ... 
 if (cores < 2L) 
        return(lapply(X = X, FUN = FUN, ...))
 ... 

So I believe the answer is yes, you should get the same results as using lapply directly, but there is also some additional overhead. I doubt that this will affect the runtime very significantly.

The documentation also states that:

Details

mclapply is a parallelized version of lapply, provided mc.cores > 1:
for mc.cores == 1 it simply calls lapply.

Related Topics