Why How to Not Edit a Method That Contains an Anonymous Method in the Debugger

Why can I not edit a method that contains an anonymous method in the debugger?

Yes there is a very good reason for why you cannot do this. The simple reason is cost. The cost of enabling this feature in C# (or VB) is extremely high.

Editing a lambda function is a specific case of a class of ENC issues that are very difficult to solve with the current ENC (Edit'n'Continue) architecture. Namely, it's very difficult to ENC any method which where the ENC does one of the following:-

1. Generates Metadata in the form of a class
2. Edits or generates a generic method

The first issue is more of a logic constraint but it also bumps into a couple of limitations in the ENC architecture. Namely the problem is generating the first class isn't terribly difficult. What's bothersome is generating the class after the second edit. The ENC engine must start tracking the symbol table for not only the live code, but the generated classes as well. Normally this is not so bad, but this becomes increasingly difficult when the shape of a generated class is based on the context in which it is used (as is the case with lambdas because of closures). More importantly, how do you resolve the differences against instances of the classes that are already alive in the process?

The second issue is a strict limitation in the CLR ENC architecture. There is nothing that C# (or VB) can do to work around this.

Lambdas unfortunately hit both of these issues dead on. The short version is that ENC'ing a lambda involves lots of mutations on existing classes (which may or may not have been generated from other ENC's). The big problem comes in resolving the differences between the new code and the existing closure instances alive in the current process space. Also, lambdas tend to use generics a lot more than other code and hit issue #2.

The details are pretty hairy and a bit too involved for a normal SO answer. I have considered writing a lengthy blog post on the subject. If I get around to it I'll link it back into this particular answer.

How to continue debugging after editing a method containing a lambda expression?

UPDATE: The desired feature was added in Visual Studio 2015, after many requests from users for this feature. This answer, and the question, are now out of date.

---

Is there a way to avoid this error?

Yes. Remove the lambda from the method. Or, don't edit the method.

Is there a way to avoid this error without removing the lambda from the method and still editing the method?

No. The error message is not lying to you.

The reason for this, if you're curious, is because lambdas are compiled as methods of a nested class, and local variables that the lambda closes over become fields of that class. The edit-and-continue feature rewrites the current method on-the-fly as you edit it, but even simple edits can result in complex changes to those nested classes. Rather than spend an enormous amount of effort on making E&C work for this scenario, and thereby steal resources from other, more valuable features, the debugger team simply made it illegal.

Why can't you edit and continue debugging when there's a Lambda expression in the method?

Edit and continue is able to change method implementations "live", but not what fields are in types.

Lambda expressions (and anonymous methods) can end up creating their own private types when they capture variables. Changing the lambda expression can change the types involved, which would break edit and continue.

It sounds like it should be possible to make changes to the code which don't have this impact, but I suspect it's simply easier to prevent it entirely - which also means you don't start making changes and then find that you're prevented half way through your change.

(Personally I'm not a fan of E&C in the first place, so I've never noticed it.)

Avoid or embrace C# constructs which break edit-and-continue?

Without wanting to sound trite - it is good practice to write unit/integration tests rather than rely on Edit-Continue.

That way, you expend the effort once, and every other time is 'free'...

Now I'm not suggesting you retrospectively write units for all your code; rather, each time you have to fix a bug, start by writing a test (or more commonly multiple tests) that proves the fix.

As @Dave Swersky mentions in the comments, Mchael Feathers' book, Working Effectively with Legacy Code is a good resource (It's legacy 5 minutes after you wrote it, right?)

So Yes, I think it's a mistake to avoid new C# contructs in favor of allowing for edit and continue; BUT I also think it's a mistake to embrace new constructs just for the sake of it, and especially if they lead to harder to understand code.

Edit and Continue quit working for me at some point

"Edit and Continue is not supported when you start debugging using Attach to Process. Edit and Continue is not supported for mixed-mode, combined managed and native, debugging, SQL debugging, Compact Framework (Smart Device) projects, debugging on Windows 98, or 64-bit debugging."

http://msdn.microsoft.com/en-us/library/ba77s56w.aspx

edit -- That link says VB, but I'm sure I've had the same problem with mixed native and managed code and 64 bit debugging when using c#.

Visual Studio 2010: Why can't I resume an application when I made changes in a delegate?

Edit and Continue doesn't support all possible code changes. It is well documented in the MSDN library, I'll repro the list here:

• Changes to the current statement or any other active statement.
• Changes to global symbols, including the following:
  
  • Adding new types.
  • Adding methods to a type.
  • Changing the signature of a type.
  • Adding fields, events, or properties to a type.
• Editing an anonymous method or any method that contains an anonymous method.
• Adding a new anonymous method.
• Adding, removing, or changing attributes.
• Adding, removing, or changing using directives.
• Removing or changing local variables. Adding local variables is allowed.
• Adding a foreach, using, or lock around the active statement.
• Modifying a method that contains a yield return or yield break statement.
• Changing a constructor with a field that is initialized by an anonymous method.

You are running afoul of the "Changing the signature of a type" limitation.

Example and draw back of using Anonymous Methods

I find it very usefull to use anonymous methods to avoid global variables

Without anonymous methods:

private static Dictionary<Binding, ErrorProvider> dict = 
    new  Dictionary<Binding, ErrorProvider>();

public static void ParseBinding(Binding binding)
{

     var errorProvider = new ErrorProvider();

     dict.Add(binding, errorProvider);

     binding.Parse += new ConvertEventHandler(binding_Parse);

}

static void binding_Parse(object sender, ConvertEventArgs e)
{
     var binding = sender as Binding;
     var errorProvider = dict[binding];

     try
     {
          // some validation form e.Value
          // throws exception if not valid
     }
     catch (Exception ex)
     {
         errorProvider.SetError(binding.Control, ex.Message);
     }
}

This is really dangerous, since I need to take care for myself to remove the entries from the dictionary if not used anymore, otherwise I have a memory leak since the garbage collector will never dispose the binding or the error provider.

Now the much simpler implementation with anonymous methods:

public static void ParseBinding(Binding binding)
{
    var errorProvider = new ErrorProvider();

    binding.Parse += (sender, e) => 
        {
           try
           {
                // some validation form e.Value
                // throws exception if not valid
           }
           catch (Exception ex)
           {
               errorProvider.SetError(binding.Control, ex.Message);
           }
        };
}

C# Debugging functions that contain lambda expressions

It isn't that it would be impossible to achieve in all cases (I don't think). It would be a monster feature to develop, though.

When you've got LINQ syntax in your method, generally that involves some anonymous method either behind-the-scenes:

// This LINQ query...
var fields = from field in data select field;

// ...is equivalent to this:
var fields = data.Select(f => f);

...or just flat-out in front of the scenes (as in your example):

( from field in data select field ).Max( f => f.Occurrences ) // <- lambda

An anonymous method in turn gets compiled into a type with instance methods to support the code you've written.

In the example above, consider the f => f.Occurrences lambda. This gets compiled into a type with a single instance field whose type is that of the local f in that lambda; this type contains a method that returns f.Occurrences.

So when the code ultimately enumerates over the result of your LINQ query, what's happening is that an instance of this compiler-generated type is being constructed for every field in data and that type's single method which has been generated to support the f => f.Occurrences lambda expression is being called to calculate Max.

The issue with edit-and-continue is that if there's any change to the lambda expressions in the method being edited, this necessitates changing the types generated, which is not an option. One would think this could still be done in the case where nothing is altered about the lambda expressions themselves; as long as the same locals are captured and the anonymous methods are unchanged, it should be feasible to modify a method with these characteristics while debugging just as it is for "normal" methods in VS.

But as you can see, the type generation used to support anonymous methods in general and therefore LINQ queries specifically adds a great deal of complexity to the edit-and-continue process, and in many cases makes it impossible (since it requires changing generated types completely).

I think it was just decided that it wasn't worth the development cost to even bother trying to support this behavior in the limited scenarios where it could hypothetically work.

Scope of anonymous methods

Var and out parameters and the Result variable cannot be captured because the safety of this operation cannot be statically verified. When the Result variable is of a managed type, such as a string or an interface, the storage is actually allocated by the caller and a reference to this storage is passed as an implicit parameter; in other words, the Result variable, depending on its type, is just like an out parameter.

The safety cannot be verified for the reason Jon mentioned. The closure created by an anonymous method can outlive the method activation where it was created, and can similarly outlive the activation of the method that called the method where it was created. Thus, any var or out parameters or Result variables captured could end up orphaned, and any writes to them from inside the closure in the future would corrupt the stack.

Of course, Delphi does not run in a managed environment, and it doesn't have the same safety restrictions as e.g. C#. The language could let you do what you want. However, it would result in hard to diagnose bugs in situations where it went wrong. The bad behaviour would manifest itself as local variables in a routine changing value with no visible proximate cause; it would be even worse if the method reference were called from another thread.

This would be fairly hard to debug. Even hardware memory breakpoints would be a relatively poor tool, as the stack is modified frequently. One would need to turn on the hardware memory breakpoints conditionally upon hitting another breakpoint (e.g. upon method entry). The Delphi debugger can do this, but I would hazard a guess that most people don't know about the technique.

Update: With respect to the additions to your question, the semantics of passing instance references by value is little different between methods that contain a closure (and capture the paramete0 and methods that don't contain a closure. Either method may retain a reference to the argument passed by value; methods not capturing the parameter may simply add the reference to a list, or store it in a private field.

The situation is different with parameters passed by reference because the expectations of the caller are different. A programmer doing this:

procedure GetSomeString(out s: string);
// ...
GetSomeString(s);

would be extremely surprised if GetSomeString were to keep a reference to the s variable passed in. On the other hand:

procedure AddObject(obj: TObject);
// ...
AddObject(TObject.Create);

It is not surprising that AddObject keeps a reference, since the very name implies that it's adding the parameter to some stateful store. Whether that stateful store is in the form of a closure or not is an implementation detail of the AddObject method.

---

Related Topics