Yield Return Statement Inside a Using() { } Block Disposes Before Executing

yield return statement inside a using() { } block Disposes before executing

When you call GetAllAnimals it doesn't actually execute any code until you enumerate the returned IEnumerable in a foreach loop.

The dataContext is being disposed as soon as the wrapper method returns, before you enumerate the IEnumerable.

The simplest solution would be to make the wrapper method an iterator as well, like this:

public static IEnumerable<Animal> GetAllAnimals() {
    using (AnimalDataContext dataContext = new AnimalDataContext()) {
        foreach (var animalName in dataContext.GetAllAnimals()) {
            yield return GetAnimal(animalName);
        }
    }
}

This way, the using statement will be compiled in the outer iterator, and it will only be disposed when the outer iterator is disposed.

Another solution would be to enumerate the IEnumerable in the wrapper. The simplest way to do that would be to return a List<Animal>, like this:

public static IEnumerable<Animal> GetAllAnimals() {
    using (AnimalDataContext dataContext = new AnimalDataContext()) {
        return new List<Animal>(dataContext.GetAllAnimals());
    }
}

Note that this loses the benefit of deferred execution, so it will get all of the animals even if you don't need them.

Can I use using in a yield-return-method?

IEnumerator<T> implements IDisposable, and foreach loops will dispose the thing they're enumerating over when they're finished (this includes linq methods which use a foreach loop, such as .ToArray()).

It turns out that the compiler-generated state machine for generator methods implements Dispose in a smart way: if the state machine is in a state which is "inside" a using block, then calling Dispose() on the state machine will dispose the thing protected by the using statement.

Let's take an example:

public IEnumerable<string> M() {
    yield return "1";
    using (var ms = new MemoryStream())
    {
        yield return "2";  
        yield return "3";
    }
    yield return "4";
}

I'm not going to paste the entire generated state machine, as it's very large. You can see it on SharpLab here.

The core of the state machine is the following switch statement, which tracks our progress past each of the yield return statements:

switch (<>1__state)
{
    default:
        return false;
    case 0:
        <>1__state = -1;
        <>2__current = "1";
        <>1__state = 1;
        return true;
    case 1:
        <>1__state = -1;
        <ms>5__1 = new MemoryStream();
        <>1__state = -3;
        <>2__current = "2";
        <>1__state = 2;
        return true;
    case 2:
        <>1__state = -3;
        <>2__current = "3";
        <>1__state = 3;
        return true;
    case 3:
        <>1__state = -3;
        <>m__Finally1();
        <ms>5__1 = null;
        <>2__current = "4";
        <>1__state = 4;
        return true;
    case 4:
        <>1__state = -1;
        return false;
}

You can see that we create the MemoryStream as we enter state 2, and dispose it (by calling <>m__Finally1()) as we exit state 3.

Here's the Dispose method:

void IDisposable.Dispose()
{
    int num = <>1__state;
    if (num == -3 || (uint)(num - 2) <= 1u)
    {
        try
        {
        }
        finally
        {
            <>m__Finally1();
        }
    }
}

If we're in states -3, 2, or 3, then we'll call <>m__Finally1();. States 2 and 3 are those inside the using block.

(State -3 seems to be a guard in case we wrote yield return Foo() and Foo() threw an exception: in this case we would stay in state -3 and would be unable to iterate any further. However we are still allowed to dispose the MemoryStream in this case).

Just for completeness, <>m__Finally1 is defined as:

private void <>m__Finally1()
{
    <>1__state = -1;
    if (<ms>5__1 != null)
    {
        ((IDisposable)<ms>5__1).Dispose();
    }
}

You can find the specification for this in the C# Language Specification, section 10.14.4.3:

If the state of the enumerator object is suspended, invoking Dispose:

Changes the state to running.

Executes any finally blocks as if the last executed yield return statement were a yield break statement. If this causes an exception to be thrown and propagated out of the iterator body, the state of the enumerator object is set to after and the exception is propagated to the caller of the Dispose method.

Changes the state to after.

When using yield within a using statement, when does Dispose occur?

So we have several separate issues going on here.

First off, dealing with the using in the iterator block. IEnumerator extends IDisposable. The code that generates iterator blocks is actually robust enough that any try/finally blocks (a using results in a try/finally block being created) results in the contents of the finally block being called in the Dispose method of the enumerator, if it wasn't already called. So as long as the enumerator is disposed, it won't leak the StreamReader.

So now we ask ourselves if the enumerator is disposed. All foreach statements will call Dispose on the enumerator (should it implement IDisposable). They do so even if you exit using a break or return statement, as well as when it finishes normally.

So you can be sure that under all circumstances the resource won't be leaked, barring the cases where nothing can be prevented from leaking (i.e. someone unpugging the machine).

Is dispose called down to the bottom via yield return?

The C# compiler takes care of a lot for you when it turns your iterator into the real code. For instance, here's the MoveNext which contains the implementation of your second example¹:

private bool MoveNext()
{
    try
    {
        switch (this.<>1__state)
        {
            case 0:
                this.<>1__state = -1;
                this.<sillier>5__1 = this.source.GetEnumerator();
                this.<>1__state = -3;
                while (this.<sillier>5__1.MoveNext())
                {
                    this.<>2__current = this.<sillier>5__1.Current;
                    this.<>1__state = 1;
                    return true;
                Label_005A:
                    this.<>1__state = -3;
                }
                this.<>m__Finally1();
                this.<sillier>5__1 = null;
                return false;

            case 1:
                goto Label_005A;
        }
        return false;
    }
    fault
    {
        this.System.IDisposable.Dispose();
    }
}

So, you'll notice that the finally clause from your using isn't there at all, and it's a state machine² that relies on being in certain good (>= 0) states in order to make further progress forwards. (It's also illegal C#, but hey ho).

Now lets look at its Dispose:

[DebuggerHidden]
void IDisposable.Dispose()
{
    switch (this.<>1__state)
    {
        case -3:
        case 1:
            try
            {
            }
            finally
            {
                this.<>m__Finally1();
            }
            break;
    }
}

So we can see the <>m__Finally1 is called here (as well as due to exiting the while loop in MoveNext.

And <>m__Finally1:

private void <>m__Finally1()
{
    this.<>1__state = -1;
    if (this.<sillier>5__1 != null)
    {
        this.<sillier>5__1.Dispose();
    }
}

So, we can see that sillier was disposed and we moved into a negative state which means that MoveNext doesn't have to do any special work to handle the "we've already been disposed state".

So,

An Idea: there can be a if(disposed) yield break; after every yield return. now dispose method of silly enumerator will just have to set disposed = true and move the enumerator once to dispose all the required stuff.

Is completely unnecessary. Trust the compiler to transform the code so that it does all of the logical things it should - it just runs it's finally clause once, when it's either exhausted the iterator logic or when it's explicitly disposed.

¹All code samples produced by .NET Reflector. But it's too good at decompiling these constructs these days so if you go and look at the Silly method itself:

[IteratorStateMachine(typeof(<Silly>d__1)), Extension]
private static IEnumerable<T> Silly<T>(this IEnumerable<T> source)
{
    IEnumerator<T> <sillier>5__1;
    using (<sillier>5__1 = source.GetEnumerator())
    {
        while (<sillier>5__1.MoveNext())
        {
            yield return <sillier>5__1.Current;
        }
    }
    <sillier>5__1 = null;
}

It's managed to hide most details about that state machine away again. You need to chase the type referenced by the IteratorStateMachine attribute to see all of the gritty bits shown above.

²Please also note that the compiler is under no obligations to produce a state machine to allow iterators to work. It's an implementation detail of the current C# compilers. The C# Specification places no restriction on how the compiler transforms the iterator, just on what the effects should be.

Return a Disposable object for use in a using block

You've got the right approach, but seem a bit lost as to how it's right.

Consider the code that you (correctly) say can't work:

DBHandle GetDB()
{
    using( var db = DatabaseObj.GetHandle() )
    {
        db.Open();
        return db;
    }
}

This code is pretty much equivalent to:

DBHandle GetDB()
{
    var db = DatabaseObj.GetHandle();
    try
    {
      db.Open();
      return db;
    }
    finally
    {
        if(db != null)//not included if db is a value-type
          ((IDisposable)db).Dispose();
    }
}

A few things of note here include that the try doesn't happen until after the assignment (the same is true of using - it doesn't save you from exceptions prior to the assignment in the using) and that db is cast to IDisposable meaning both that it can't compile if that assignment isn't valid, and also that Dispose() can be either implicitly or explicitly implemented and this will work either way.

Now of course, finally blocks will execute whether an exception occurs or not. You can't use using because it's equivalent to a finally and you want to Dispose() in your method only if an exception occurs. Hence you take the finally and turn it into a catch:

DBHandle GetDB()
{
    var db = DatabaseObj.GetHandle();
    try
    {
      db.Open();
      return db;
    }
    catch
    {
        if(db != null)
          ((IDisposable)db).Dispose();
        throw;
    }
}

This is pretty much the same as you have, except for the addition of a null check (maybe you can rule out the need for it) and that I'm using the bare throw (it's generally a good idea when you are going to re-throw an exception without altering or examining it. In some cases throwing a new exception is better, in which case you should include the original as the InnerException property of that new exception, so as to provide further information to someone debugging).

So all in all, you were on the right track. Hopefully I've helped explain why.

a concern about yield return and breaking from a foreach

Excellent question. You do not need to worry about this; the compiler takes care of it for you. Basically, what we do is we put the cleanup code for the finally blocks into a special cleanup method on the generated iterator. When control leaves the caller's foreach block, the compiler generates code which calls the cleanup code on the iterator.

A simplified example:

static IEnumerable<int> GetInts()
{
    try { yield return 1; yield return 2;} 
    finally { Cleanup(); }
}

Your question is basically "Is Cleanup() called in this scenario?"

foreach(int i in GetInts()) { break; }

Yes. The iterator block is generated as a class with a Dispose method that calls Cleanup, and then the foreach loop is generated as something similar to:

{
  IEnumerator<int> enumtor = GetInts().GetEnumerator();
  try
  {
    while(enumtor.MoveNext())
    {
      i = enumtor.Current;
      break;
    }
  }
  finally
  {
    enumtor.Dispose();
  }
}

So when the break happens, the finally takes over and the disposer is called.

See my recent series of articles if you want more information about some of the weird corner cases we considered in the design of this feature.

http://blogs.msdn.com/ericlippert/archive/tags/Iterators/default.aspx

Will using destroy data if return will be inside

using is a Syntactic sugar, it need to contain an object which implements IDisposable interface when you leave the using scope .net will call the IDisposable.Dispose method Automatically.

Here is a sample c# online

when the program leave Test method. .net will call IDisposable.Dispose method Automatically

Yield return inside usings

See this post for a good explanation of the issues with using and yield. Because you return in enumerator, the using block will already have destroyed the context before anything is accessed. The answers have good solutions, basically, either make the wrapper method an enumerator, or build a list instead.

Also it's usually more practical to have using around the reader, not the connection, and use CommandBehavior.CloseConnection to ensure resources are released when the reader's done. Though it doesn't really matter in your situation, if you ever return a data reader from a method, this will ensure the connection is closed properly when the reader is disposed.

   using(SqlDataReader reader = 
             command.ExecuteReader(CommandBehavior.CloseConnection)) {
        while (reader.Read())
        {
            yield reader[0];
        }
   }

Yield Return Statement Inside a Using() { } Block Disposes Before Executing