Lambda Variable Capture in Loop - What Happens Here

Lambda variable capture in loop - what happens here?

In this line

 listActions.Add(() => Console.WriteLine(i));

the variable i, is captured, or if you wish, created a pointer to the memory location of that variable. That means that every delegate got a pointer to that memory location. After this loop execution:

foreach (int i in Enumerable.Range(1, 10))
{
    listActions.Add(() => Console.WriteLine(i));
}

for obvious reasons i is 10, so the memory content that all pointers present in Action(s) are pointing, becomes 10.

In other words, i is captured.

By the way, should note, that according to Eric Lippert, this "strange" behaviour would be resolved in C# 5.0.

So in the C# 5.0 your program would print as expected:

1,2,3,4,5...10

EDIT:

Can not find Eric Lippert's post on subject, but here is another one:

Closure in a Loop Revisited

What do lambda function closures capture?

What do the closures capture exactly?

Closures in Python use lexical scoping: they remember the name and scope of the closed-over variable where it is created. However, they are still late binding: the name is looked up when the code in the closure is used, not when the closure is created. Since all the functions in your example are created in the same scope and use the same variable name, they always refer to the same variable.

There are at least two ways to get early binding instead:

The most concise, but not strictly equivalent way is the one recommended by Adrien Plisson. Create a lambda with an extra argument, and set the extra argument's default value to the object you want preserved.
More verbosely but also more robustly, we can create a new scope for each created lambda:
```
>>> adders = [0,1,2,3]
>>> for i in [0,1,2,3]:
...     adders[i] = (lambda b: lambda a: b + a)(i)
...     
>>> adders[1](3)
4
>>> adders[2](3)
5
```
The scope here is created using a new function (another lambda, for brevity), which binds its argument, and passing the value you want to bind as the argument. In real code, though, you most likely will have an ordinary function instead of the lambda to create the new scope:
```
def createAdder(x):
    return lambda y: y + x
adders = [createAdder(i) for i in range(4)]
```

Lambda capture problem with iterators?

You asked for a reference to the specification; the relevant location is section 8.8.4, which states that a "foreach" loop is equivalent to:

    V v;
    while (e.MoveNext()) {
        v = (V)(T)e.Current;
        embedded-statement
    }

Note that the value v is declared outside the while loop, and therefore there is a single loop variable. That is then closed over by the lambda.

UPDATE

Because so many people run into this problem the C# design and compiler team changed C# 5 to have these semantics:

    while (e.MoveNext()) {
        V v = (V)(T)e.Current;
        embedded-statement
    }

Which then has the expected behaviour -- you close over a different variable every time. Technically that is a breaking change, but the number of people who depend on the weird behaviour you are experiencing is hopefully very small.

Be aware that C# 2, 3, and 4 are now incompatible with C# 5 in this regard. Also note that the change only applies to foreach, not to for loops.

See http://ericlippert.com/2009/11/12/closing-over-the-loop-variable-considered-harmful-part-one/ for details.

Commenter abergmeier states:

C# is the only language that has this strange behavior.

This statement is categorically false. Consider the following JavaScript:

var funcs = [];
var results = [];
for(prop in { a : 10, b : 20 })
{
  funcs.push(function() { return prop; });
  results.push(funcs[0]());
}

abergmeier, would you care to take a guess as to what are the contents of results?

Captured variable in a loop in C#

Yes - take a copy of the variable inside the loop:

while (variable < 5)
{
    int copy = variable;
    actions.Add(() => copy * 2);
    ++ variable;
}

You can think of it as if the C# compiler creates a "new" local variable every time it hits the variable declaration. In fact it'll create appropriate new closure objects, and it gets complicated (in terms of implementation) if you refer to variables in multiple scopes, but it works :)

Note that a more common occurrence of this problem is using for or foreach:

for (int i=0; i < 10; i++) // Just one variable
foreach (string x in foo) // And again, despite how it reads out loud

See section 7.14.4.2 of the C# 3.0 spec for more details of this, and my article on closures has more examples too.

Note that as of the C# 5 compiler and beyond (even when specifying an earlier version of C#), the behavior of foreach changed so you no longer need to make local copy. See this answer for more details.

Python create function in a loop capturing the loop variable

lambdas in python are closures.... the arguments you give it aren't going to be evaluated until the lambda is evaluated. At that time, i=9 regardless, because your iteration is finished.

The behavior you're looking for can be achieved with functools.partial

import functools

def f(a,b):
    return a*b

funcs = []

for i in range(0,10):
    funcs.append(functools.partial(f,i))

Lambda inside loop

When you specify the capture, you can choose between capture by value and capture by reference. You have chosen to capture by reference. Capturing by reference means that the variable inside the lambda function is referring to the same object. The implication is that any changes to this variable will be shared and you also need to make sure that the referenced object stays around for the life-time of the lambda function.

You probably meant to capture by values. To do this, you can either replace the capture specification to become [=] or to become [x]. The latter makes sure that only x can be accessed while the former would allow other variables to be accessible.

BTW, I'd recommend not using lock() and unlock() explicitly but rather use one of the lock guards. With this, the body of your loop would look something like this:

vec.push_back(std::thread{[x](){
    std::lock_guard<std::mutex> kerberos(m);
    std::cout << x << "\n";
}});

Why is it bad to use an iteration variable in a lambda expression

Consider this code:

List<Action> actions = new List<Action>();

for (int i = 0; i < 10; i++)
{
    actions.Add(() => Console.WriteLine(i));
}

foreach (Action action in actions)
{
    action();
}

What would you expect this to print? The obvious answer is 0...9 - but actually it prints 10, ten times. It's because there's just one variable which is captured by all the delegates. It's this kind of behaviour which is unexpected.

EDIT: I've just seen that you're talking about VB.NET rather than C#. I believe VB.NET has even more complicated rules, due to the way variables maintain their values across iterations. This post by Jared Parsons gives some information about the kind of difficulties involved - although it's back from 2007, so the actual behaviour may have changed since then.

How to tell a lambda function to capture a copy instead of a reference in C#?

What the compiler is doing is pulling your lambda and any variables captured by the lambda into a compiler generated nested class.

After compilation your example looks a lot like this:

class Program
{
        delegate void Action();
        static void Main(string[] args)
        {
                List<Action> actions = new List<Action>();

                DisplayClass1 displayClass1 = new DisplayClass1();
                for (displayClass1.i = 0; displayClass1.i < 10; ++displayClass1.i )
                        actions.Add(new Action(displayClass1.Lambda));

                foreach (Action a in actions)
                        a();
        }

        class DisplayClass1
        {
                int i;
                void Lambda()
                {
                        Console.WriteLine(i);
                }
        }
}

By making a copy within the for loop, the compiler generates new objects in each iteration, like so:

for (int i = 0; i < 10; ++i)
{
    DisplayClass1 displayClass1 = new DisplayClass1();
    displayClass1.i = i;
    actions.Add(new Action(displayClass1.Lambda));
}

Detailed Explanation of Variable Capture in Closures

Is tricky. Will come onto it in a minute.
There's no difference - in both cases, it's the variable itself which is captured.
Nope, no boxing occurs.

It's probably easiest to demonstrate how the capturing works via an example...

Here's some code using a lambda expression which captures a single variable:

using System;

class Test
{
    static void Main()
    {
        Action action = CreateShowAndIncrementAction();
        action();
        action();
    }

    static Action CreateShowAndIncrementAction()
    {
        Random rng = new Random();
        int counter = rng.Next(10);
        Console.WriteLine("Initial value for counter: {0}", counter);
        return () =>
        {
            Console.WriteLine(counter);
            counter++;
        };
    }
}

Now here's what the compiler's doing for you - except that it would use "unspeakable" names which couldn't really occur in C#.

using System;

class Test
{
    static void Main()
    {
        Action action = CreateShowAndIncrementAction();
        action();
        action();
    }

    static Action CreateShowAndIncrementAction()
    {
        ActionHelper helper = new ActionHelper();        
        Random rng = new Random();
        helper.counter = rng.Next(10);
        Console.WriteLine("Initial value for counter: {0}", helper.counter);

        // Converts method group to a delegate, whose target will be a
        // reference to the instance of ActionHelper
        return helper.DoAction;
    }

    class ActionHelper
    {
        // Just for simplicity, make it public. I don't know if the
        // C# compiler really does.
        public int counter;

        public void DoAction()
        {
            Console.WriteLine(counter);
            counter++;
        }
    }
}

If you capture variables declared in a loop, you'd end up with a new instance of ActionHelper for each iteration of the loop - so you'd effectively capture different "instances" of the variables.

It gets more complicated when you capture variables from different scopes... let me know if you really want that sort of level of detail, or you could just write some code, decompile it in Reflector and follow it through :)

Note how:

There's no boxing involved
There are no pointers involved, or any other unsafe code

EDIT: Here's an example of two delegates sharing a variable. One delegate shows the current value of counter, the other increments it:

using System;

class Program
{
    static void Main(string[] args)
    {
        var tuple = CreateShowAndIncrementActions();
        var show = tuple.Item1;
        var increment = tuple.Item2;

        show(); // Prints 0
        show(); // Still prints 0
        increment();
        show(); // Now prints 1
    }

    static Tuple<Action, Action> CreateShowAndIncrementActions()
    {
        int counter = 0;
        Action show = () => { Console.WriteLine(counter); };
        Action increment = () => { counter++; };
        return Tuple.Create(show, increment);
    }
}

... and the expansion:

using System;

class Program
{
    static void Main(string[] args)
    {
        var tuple = CreateShowAndIncrementActions();
        var show = tuple.Item1;
        var increment = tuple.Item2;

        show(); // Prints 0
        show(); // Still prints 0
        increment();
        show(); // Now prints 1
    }

    static Tuple<Action, Action> CreateShowAndIncrementActions()
    {
        ActionHelper helper = new ActionHelper();
        helper.counter = 0;
        Action show = helper.Show;
        Action increment = helper.Increment;
        return Tuple.Create(show, increment);
    }

    class ActionHelper
    {
        public int counter;

        public void Show()
        {
            Console.WriteLine(counter);
        }

        public void Increment()
        {
            counter++;
        }
    }
}

lambda in for loop only takes last value

Please read about minimal examples. Without reading your code, I believe you have run into a well known issue addressed in previous questions and answers that needs 2 lines to illustrate. Names in function bodies are evaluated when the function is executed.

funcs = [lambda: i for i in range(3)]
for f in funcs: print(f())

prints '2' 3 times because the 3 functions are identical and the 'i' in each is not evaluated until the call, when i == 2. However,

funcs = [lambda i=i:i for i in range(3)]
for f in funcs: print(f())

makes three different functions, each with a different captured value, so 0, 1, and 2 are printed. In your statement

__cMenu.add_command(label="{}".format(option),
    command=lambda: self.filter_records(column, option))

add option=option before : to capture the different values of option. You might want to rewrite as

lambda opt=option: self.filter_records(column, opt)

to differentiate the loop variable from the function parameter. If column changed within the loop, it would need the same treatment.

Lambda Variable Capture in Loop - What Happens Here