Differencebetween Parent.Frame() and Parent.Env() in R; How Do They Differ in Call by Reference

What is the difference between parent.frame() and parent.env() in R; how do they differ in call by reference?

parent.env is the environment in which a closure (e.g., function) is defined. parent.frame is the environment from which the closure was invoked.

f = function() 
     c(f=environment(), defined_in=parent.env(environment()),  
       called_from=parent.frame())
g = function() 
     c(g=environment(), f())

and then

> g()
$g
<environment: 0x14060e8>

$f
<environment: 0x1405f28>

$defined_in
<environment: R_GlobalEnv>

$called_from
<environment: 0x14060e8>

I'm not sure when a mere mortal would ever really want to use them, but the concepts are useful in understanding lexical scope here

> f = function() x
> g = function() { x = 2; f() }
> h = function() { x = 3; function() x }
> x = 1
> f()
[1] 1
> g()
[1] 1
> h()()
[1] 3

or in the enigmatic 'bank account' example in the Introduction to R. The first paragraph of the Details section of ?parent.frame might clarify things.

Environments are pervasive in R, e.g., the search() path is (approximately) environments chained together in a sibling -> parent relationship. One sometimes sees env = new.env(parent=emptyenv()) to circumvent symbol look-up -- normally env[["x"]] would look first in env, and then in env's parent if not found. Likewise, <<- looks for assignment starting in the parent.env. The relatively new reference class implementation in R relies on these ideas to define an instance-specific environment in which symbols (instance fields and methods) can be found.

Variable scope or parent environment

Why does f2() consider f1() its parent environment

Because it is defined inside f1.

but f3() does not consider f2() its parent environment?

Because it is not defined inside f2.

You need to distinguish between containing environment, and parent frame. f2 is the parent frame of f3 in your call. But f1 is its containing environment regardless.

See also What is the difference between parent.frame() and parent.env() in R; how do they differ in call by reference?, and Hadley’s introduction into environments.

How to use `parent.env -` in R?

Is this what you wanted to achieve?

e1 <- new.env()
e2 <- new.env()
parent.env(e1 <- e2
parent.env(e1)
#<environment: 0x10dfaba70>

Purpose of this R idiom (match.call followed by eval(parent.frame())

I think this should answer everything, explanations are in the code :

# for later
FOO <- function(x) 1000 * x
y <- 1

foo <- function(...) {
  cl = match.call()
  message("cl")
  print(cl)
  message("as.list(cl)")
  print(as.list(cl))
  message("class(cl)")
  print(class(cl))
  # we can modify the call is if it were a list
  cl[[1]] <- quote(FOO)
  message("modified call")
  print(cl)
  y <- 2
  # now I want to call it,  if I call it here or in the parent.frame might
  # give a different output
  message("evaluate it locally")
  print(eval(cl))
  message("evaluate it in the parent environment")
  print(eval(cl, parent.frame()))
  message("eval.parent is equivalent and more idiomatic")
  print(eval.parent(cl))
  invisible(NULL)
}
foo(y)

# cl
# foo(y)
# as.list(cl)
# [[1]]
# foo
# 
# [[2]]
# y
# 
# class(cl)
# [1] "call"
# modified call
# FOO(y)
# evaluate it locally
# [1] 2000
# evaluate it in the parent environment
# [1] 1000
# eval.parent is equivalent and more idiomatic
# [1] 1000

eval: why does enclos = parent.frame() make a difference?

So, different parents. In the example that doesn't work parent.frame is looking up from inside eval into the internal environment of subset2. In the working example, parent.frame is looking up from inside subset3, likely to your global where your df sits.

Example:

tester <- function() {
  print(parent.frame())
} 

tester() # Global

(function() {
  tester()
})() # Anonymous function's internal env

Return list vs environment from an R function

Although similars, there're differences in return a list and a enviroment.
From Advanced R:

Generally, an environment is similar to a list, with four important exceptions:
Every name in an environment is unique.
The names in an environment are not ordered (i.e., it doesn’t make sense to ask what the first element of an environment is).
An environment has a parent.
Environments have reference semantics.
More technically, an environment is made up of two components, the frame, which contains the name-object bindings (and behaves much like a named list), and the parent environment. Unfortunately “frame” is used inconsistently in R. For example, parent.frame() doesn’t give you the parent frame of an environment. Instead, it gives you the calling environment. This is discussed in more detail in calling environments.

From the help:

help(new.env)

Environments consist of a frame, or collection of named objects, and a pointer to an enclosing environment. The most common example is the frame of variables local to a function call; its enclosure is the environment where the function was defined (unless changed subsequently). The enclosing environment is distinguished from the parent frame: the latter (returned by parent.frame) refers to the environment of the caller of a function. Since confusion is so easy, it is best never to use ‘parent’ in connection with an environment (despite the presence of the function parent.env).

from the function's documentation:

e1 <- new.env(parent = baseenv())  # this one has enclosure package:base.
e2 <- new.env(parent = e1)
assign("a", 3, envir = e1)
ls(e1)
#[1] "a"

However ls will gives the environments created:

ls()
#[1] "e1" "e2"

And you can access your enviroment objects just like a list:

e1$a
#[1] 3

Playing with your functions:

f1 <- function(x) {
   ret <- new.env()
   ret$x <- x
   ret$y <- x^2
   return(ret)
}

res <- f1(2)
res
#<environment: 0x0000021d55a8a3e8>

res$y
#[1] 4

f2 <- function(x) {
   ret <- list()
   ret$x <- x
   ret$y <- x^2
   return(ret)

res2 <- f(2)
res2
#$x
#[1] 2

#$y
#[1] 4

res2$y
#[1] 4

Their performance is quite similar, according to microbenchmarking:

microbenchmark::microbenchmark(
   function(x) {
      ret <- new.env()
      ret$x <- x
      ret$y <- x^2
      return(ret)
   },
   function(x) {
      ret <- list()
      ret$x <- x
      ret$y <- x^2
      return(ret)
   },
   times = 500L
)

#Unit: nanoseconds
#                                                                                 #expr
# function(x) {     ret <- new.env()     ret$x <- x     ret$y <- x^2     #return(ret) }
#    function(x) {     ret <- list()     ret$x <- x     ret$y <- x^2     #return(ret) }
# min lq   mean median  uq  max neval
#   0  1 31.802      1 100  801   500
#   0  1 37.802      1 100 2902   500

and they return objects with same sizes:

object.size(res)
#464 bytes

object.size(res2)
#464 bytes

and you can always generate a list from an enviroment (list2env) and the inverse too (as.list):

L <- list(a = 1, b = 2:4, p = pi, ff = gl(3, 4, labels = LETTERS[1:3]))
e <- list2env(L)
e$ff
# [1] A A A A B B B B C C C C
#Levels: A B C

as.list(e)
#$ff
# [1] A A A A B B B B C C C C
#Levels: A B C
#
#$p
#[1] 3.141593
#
#$b
#[1] 2 3 4
#
#$a
#[1] 1

When/how/where is parent.frame in a default argument interpreted?

Default arguments are evaluated within the evaluation frame of the function call, from which place parent.frame() is the calling environment. envir's value will thus be a pointer to the environment from which fn was called.

Also, just try it out to see for yourself:

debug(fn)
fn()
# debugging in: fn()
# debug at #2: {
# }
Browse[2]> envir
# <environment: R_GlobalEnv>

R: Substitute variables bound in all parent environments

Building on @MikkoMarttila's answer, I think the following does what I requested

do_something <- function(todo) {

  # A helper to substitute() in a pre-quoted expression
  substitute_q <- function(expr, env) {
    eval(substitute(substitute(x, env), list(x = expr)))
  }

  substitute_parents <- function(expr) {
    expr <- substitute(expr)

    # list all parent envs
    envs <- list()
    env <- environment()
    while (!identical(env, globalenv())) {
      envs <- c(envs, env)
      env <- parent.env(env)
    }
    # substitute in all parent envs
    for (e in envs) {
      expr <- substitute_q(expr, e)
    }

    # previously did not include globalenv() and
    # substitute() doesnt "substitute" there
    e <- as.list(globalenv())
    substitute_q(expr, e)
  }

  cat(
    paste(
      deparse(substitute_parents(todo)),
      collapse = "\n"
    )
  )
}

This gives

nested_do <- function() {
  var_2 <- "not_this"

  do_something({
    print(var_1)
    Sys.sleep(100)
    print("world")
    print(var_2)
  })
}

var_1 <- "hello"
var_2 <- "goodbye"

do_something({
  print(var_1)
  Sys.sleep(100)
  print("world")
  print(var_2)
})
#> {
#>     print("hello")
#>     Sys.sleep(100)
#>     print("world")
#>     print("goodbye")
#> }
nested_do()
#> {
#>     print("hello")
#>     Sys.sleep(100)
#>     print("world")
#>     print("goodbye")
#> }

Why parent.frame(5)?

Rather than using relative parent frames, it's much more direct to just capture the frame as the start of the function then pass that directly. (You don't have control over how many frames tryCatch created when it runs).

And while I think allowing either a string or a symbol name is really just a dangerous mess, R does allow it. You can inspect the type of the promise passed to the function and deparse it if it's not a string. It's would be better to have two different parameters. One if you want to pass a symbol, and a different one if you want to pass a character.

loa <- function(x, dir='./dados/') { 
      toenv <- parent.frame()
      xn <- substitute(x)
      if (!is.character(xn)) xn <- deparse(xn)
      if (right(dir,1) != '/') dir <- paste0(dir, '/')
      path <- paste0(dir,gsub('\"', '', xn), '.rda')
      tryCatch(load(path, envir = toenv), 
          error = function(e) print(e)
      )
}

Why doesn't R look up for the specified object in the provided envrionment parents tree?

So, this is an unusually nuanced issue. There are two relevant types of environments that you need to think about here, the binding environment, or the environment that has a binding to your function, and the enclosing environment, or the environment where your function was created. In this case the binding environment is e10, but the enclosing environment is the global environment. From Hadley Wickham's Advanced R:

The enclosing environment belongs to the function, and never changes, even if the function is moved to a different environment. The enclosing environment determines how the function finds values; the binding environments determine how we find the function.

Consider the following (executed after executing your supplied code) that demonstrates this:

eval(expression(testfun()), envir = e10)
# [1] "e10"
# Error in testfun() : object 'testvar' not found
testvar <- 600
eval(expression(testfun()), envir = e10)
# [1] "e10"
# [1] 600

Moreover, now consider:

eval(envir = e10, expr = expression(
    testfun2 <- function(x) {
        print(envnames::environment_name(caller_env()))
        return (testvar)
    }
))
eval(expression(testfun2()), envir = e10)
# [1] "e10"
# [1] 1200

I hope this clarifies the issue.

Update: Determining the Enclosing and Binding Environments

So how can we determine the binding and enclosing environments for functions such as testfun()?

As G. Grothendieck's answer shows, the environment() function gives you the enclosing environment for a function:

environment(e10$testfun)
# <environment: R_GlobalEnv>

To my knowledge, there isn't a simple function in base R to give you a function's binding environments. If the function you're looking for is in a parent environment, you can use pryr::where():

pryr::where("mean")
# <environment: base>

(There is a base function to see if a function is in an environment, exists(), and pryr::where() uses it. But, it doesn't recurse through parent environments like where().)

However, if you're having to search through child environments, to my knowledge there isn't a function for that. But, seems pretty simple to mock one up:

get_binding_environments <- function(fname) {
    ## First we need to find all the child environments to search through.
    ## We don't want to start from the execution environment;
    ## we probably want to start from the calling environment,
    ## but you may want to change this to the global environment.
    e <- parent.frame()
    ## We want to get all of the environments we've created
    objects <- mget(ls(envir = e), envir = e)
    environments <- objects[sapply(objects, is.environment)]
    ## Then we use exists() to see if the function has a binding in any of them
    contains_f <- sapply(environments, function(E) exists(fname, where = E))
    return(unique(environments[contains_f]))
}

get_binding_environments("testfun")
# [[1]]
# <environment: 0x55f865406518>

e10
# <environment: 0x55f865406518>