Trouble Yielding Inside a Block/Lambda

Trouble yielding inside a block/lambda

Lambdas don't implicitly accept blocks like regular methods do, so your func1 can't yield. Do this instead:

func1 = lambda do |x, &blk|
  for i in 1 .. 5
    blk.call(x * i)
  end
end

Specifically, I believe this is because yield would send control back to the caller's block, which would not include lambda invocations. So the following code works like you "expect":

def foo
  (lambda { |n| yield(n) }).call(5)
end
foo { |f| puts f }  # prints 5

Ruby: Yielding from an inner block does not work

What is going wrong?

yield transfers control to the block that was passed to the method in whose definition the yield appears. In your case, the yield doesn't appear in a method definition, ergo, there is nowhere to yield to.

Declaring a Ruby lambda with a yield

You cannot yield from a lambda directly to a nonexplicit block passed to it as you are doing now (See comments below for source provided by @cremno showing that yield may only be used in method and singleton class definitions)

Source Extraction:

if ((type != ISEQ_TYPE_METHOD && type != ISEQ_TYPE_CLASS) || block == 0) {
  rb_vm_localjump_error("no block given (yield)", Qnil, 0);
}

Primary Solution:

You could do this with an explicit block like so

f = lambda do |data,&block| 
      data.select do |m| 
        block.call(m)
      end.map do |m| 
        [m, block.call(m)]
      end
    end
d = ["string",1,2,3,4,"string2"]
f.call(d) { |n| n.is_a?(String) }
#=>[["string", true], ["string2", true]]

Secondary Solution:

However you can also yield inside the lambda if the block is passed to the method instead e.g.

def some_method(data)
  #extended syntax for readability but works fine as 
  #f = lambda{|data,&block| data.select{|m| block.call(m) }.map { |m| [m, block.call(m)]}}
  f = lambda do |data| 
        data.select do |m| 
          yield(m)
        end.map do |m| 
          [m, yield(m)]
        end
      end
  f.call(data)
end
some_method(["string",1,2,3,4,"string2"]) { |s| s.is_a?(String) } 
#=> [["string", true], ["string2", true]]

Tertiary Soultion: (A spawn of the secondary solution that more closely matches your question)

Define a secondary method

def some_method(some_data)
   x = filter(some_data) {|m| m.is_a?(String)}
   y = filter(some_data) {|m| m.is_a?(Fixnum)}
   [x, y]
end  
def filter(data) 
  data.select do |m| 
    yield(m)
  end.map do |m| 
    [m, yield(m)]
  end
end
some_method(["string",1,2,3,4,"string2"])
#=>[
    [["string", true], ["string2", true]], 
    [[1, true], [2, true], [3, true], [4, true]]
   ]

Quaternary Solution:

There is technically a fourth option and I am posting it just for your sake because it represents your original code as closely as possible. It is by far the strangest pattern (almost as strange as the word quaternary) of the 4 but I like to be thorough:

def some_method(some_data)
    f = lambda{|data| data.select{|m| yield(m) }.map { |m| [m, yield(m)]}}
    if block_given?
      f.call(some_data)
    else  
      x = some_method(some_data) {|m| m.is_a?(String)}
      y = some_method(some_data) {|m| m.is_a?(Fixnum)}
      [x,y] 
    end
end
some_method(["string",1,2,3,4,"string2"])
#=>[
    [["string", true], ["string2", true]], 
    [[1, true], [2, true], [3, true], [4, true]]
   ]

Why can you not use yield in a lambda, when you can use await in a lambda?

According to Eric Lippert, anonymous iterators were not added to the language because it would be overly complicated to implement it.

That is not precisely what I intended to convey. The relevant cost is implementation cost, yes, but it is implementation cost in an existing compiler which was not set up architecturally to implement that complex feature.

The compiler has to do the same thing for async methods as it has to for iterators (convert them into state machines), so I am very confused why anonymous iterators are not allowed as well, when anonymous async methods are.

A brief history is relevant. C# first had anonymous methods and iterator blocks in C# 2.0. When I added lambdas in C# 3.0 it was a major cost to refactor all of the existing anonymous method code so that it could handle all the new features of lambdas. That made it even more complicated and expensive to modify. Making iterator block lambdas was judged too costly for the benefits that would be accrued; it would have been a large percentage of the total cost. We could not afford it. If you added up every team in Developer Division's work schedule, the team with the "longest pole" was the C# 3.0 compiler team, and my work on the semantic analyzer was IIRC the longest pole on the compiler team. Every day we might have slipped C# 3.0, that would have been a day that Visual Studio would have slipped. Therefore anything that didn't make LINQ better was cut, and that included iterator lambdas.

In C# 4, iterator lambdas were one feature of many that were considered. We had a list of potential good features literally longer than your arm and we could afford to do less than a tenth of them.

In C# 5 the team added async methods. The design and implementation teams tried for a long time to come up with an underlying abstraction that was common to both the iterator block and await rewrites; they are obviously similar, as you note. But ultimately, the cost of finding the general solution did not pay for itself. Generality is surprisingly expensive, and finding a generality that by design unifies only two things is silly if it is not cheap.

Therefore the decision was made to implement the await rewriter as its own thing. Given that the team was going to take on this large cost, and given that the original transformation of async methods was going to be into a lambda form anyway, the decision was made to invest in the full feature: async methods containing lambdas, async lambdas containing lambdas, the whole deal. The cost of that feature was a small fraction of the cost of the whole feature, which was extremely expensive.

And again, we have a problem with long poles. Any work on the lambda engine that could potentially have destabilized await is to be avoided, and that includes trying to make them work with iterator blocks.

Now compare Visual Basic. VB for a long time had no iterator blocks at all. When they were added, there was no existing infrastructure to keep working! The whole thing could be built from the ground up to handle iterator blocks containing lambdas and lambdas containing iterator blocks, and so that was done.

The C# compiler has been thoroughly rearchitected and rewritten via the Roslyn project. I am hoping that this will lower the cost of implementing iterator block lambdas in a hypothetical future version of C#. We shall see!

Why does yielding to lambda splat array arguments in Ruby?

I'm answering my own question here, because this is a known bug:

https://bugs.ruby-lang.org/issues/12705

And it was fixed in Ruby 2.4.1 (thanks @ndn)

yield return in the lambda expression?

C# doesn't support anonymous iterator blocks, so you'll simply need to use a named method instead of an anonymous method.

public static IEnumerable<IEnumerable<string>> Split(IEnumerable<string> tokens)
{
    using(var iterator = tokens.GetEnumerator())
        while(iterator.MoveNext())
            if(iterator.Current == "[")
                yield return SplitGroup(iterator);
}

public static IEnumerable<string> SplitGroup(
    IEnumerator<string> iterator)
{
    while(iterator.MoveNext() && iterator.Current != "]")
        yield return iterator.Current;
}

In C#, why can't an anonymous method contain a yield statement?

Eric Lippert recently wrote a series of blog posts about why yield is not allowed in some cases.

• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6

EDIT2:

• Part 7 (this one was posted later and specifically addresses this question)

You will probably find the answer there...

---

EDIT1: this is explained in the comments of Part 5, in Eric's answer to Abhijeet Patel's comment:

Q :

Eric,

Can you also provide some insight into
why "yields" are not allowed inside an
anonymous method or lambda expression

A :

Good question. I would love to have
anonymous iterator blocks. It would be
totally awesome to be able to build
yourself a little sequence generator
in-place that closed over local
variables. The reason why not is
straightforward: the benefits don't
outweigh the costs. The awesomeness of
making sequence generators in-place is
actually pretty small in the grand
scheme of things and nominal methods
do the job well enough in most
scenarios. So the benefits are not
that compelling.

The costs are large. Iterator
rewriting is the most complicated
transformation in the compiler, and
anonymous method rewriting is the
second most complicated. Anonymous
methods can be inside other anonymous
methods, and anonymous methods can be
inside iterator blocks. Therefore,
what we do is first we rewrite all
anonymous methods so that they become
methods of a closure class. This is
the second-last thing the compiler
does before emitting IL for a method.
Once that step is done, the iterator
rewriter can assume that there are no
anonymous methods in the iterator
block; they've all be rewritten
already. Therefore the iterator
rewriter can just concentrate on
rewriting the iterator, without
worrying that there might be an
unrealized anonymous method in there.

Also, iterator blocks never "nest",
unlike anonymous methods. The iterator
rewriter can assume that all iterator
blocks are "top level".

If anonymous methods are allowed to
contain iterator blocks, then both
those assumptions go out the window.
You can have an iterator block that
contains an anonymous method that
contains an anonymous method that
contains an iterator block that
contains an anonymous method, and...
yuck. Now we have to write a rewriting
pass that can handle nested iterator
blocks and nested anonymous methods at
the same time, merging our two most
complicated algorithms into one far
more complicated algorithm. It would
be really hard to design, implement,
and test. We are smart enough to do
so, I'm sure. We've got a smart team
here. But we don't want to take on
that large burden for a "nice to have
but not necessary" feature. -- Eric

Return statements inside procs, lambdas, and blocks

As one answer in the linked question shows:

The return keyword always returns from the method or lambda in the current context. In blocks, it will return from the method in which the closure was defined. It cannot be made to return from the calling method or lambda.

Your first example was successful because you defined victor in the same function you wanted to return from, so a return was legal in that context. In your second example, victor was defined in the top-level. The effect of that return, then, would not be to return from batman_yield (the calling method), but [if it were valid] to return from the top-level itself (where the Proc was defined).

Clarification: while you can access the return value of a block (i.e. "The value of the last expression evaluated in the block is passed back to the method as the value of the yield" - as per your comment), you can't use the return keyword, for the reason stated above. Example:

def batman_yield
    value = yield
    return value
    "Iron man will win!"
end

victor = Proc.new { return "Batman will win!" }
victor2 = Proc.new { "Batman will win!" }

#batman_yield(&victor) === This code throws an error.
puts batman_yield(&victor2) # This code works fine.

Python Try Catch Block inside lambda

Nope. A Python lambda can only be a single expression. Use a named function.

It is convenient to write a generic function for converting types:

def tryconvert(value, default, *types):
    for t in types:
        try:
            return t(value)
        except (ValueError, TypeError):
            continue
    return default

Then you can write your lambda:

lambda v: tryconvert(v, 0, int)

You could also write tryconvert() so it returns a function that takes the value to be converted; then you don't need the lambda:

def tryconvert(default, *types):
    def convert(value):
        for t in types:
            try:
                return t(value)
            except (ValueError, TypeError):
                continue
        return default
    # set name of conversion function to something more useful
    namext = ("_%s_" % default) + "_".join(t.__name__ for t in types)
    if hasattr(convert, "__qualname__"): convert.__qualname__ += namext
    convert.__name__ += namext
    return convert

Now tryconvert(0, int) returns a function convert_0_int that takes a value and converts it to an integer, and returns 0 if this can't be done. You can use this function right away (not saving a copy):

mynumber = tryconert(0, int)(value)

Or save it to call it later:

intconvert = tryconvert(0, int)
# later...
mynumber = intconvert(value)

---

Related Topics