Reflection Type Inference on Java 8 Lambdas

Reflection type inference on Java 8 Lambdas

I have found a way of doing it for serializable lambdas. All my lambdas are serializable, to that works.

Thanks, Holger, for pointing me to the SerializedLambda.

The generic parameters are captured in the lambda's synthetic static method and can be retrieved from there. Finding the static method that implements the lambda is possible with the information from the SerializedLambda

The steps are as follows:

1. Get the SerializedLambda via the write replacement method that is auto-generated for all serializable lambdas
2. Find the class that contains the lambda implementation (as a synthetic static method)
3. Get the java.lang.reflect.Method for the synthetic static method
4. Get generic types from that Method

---

UPDATE: Apparently, this does not work with all compilers. I have tried it with the compiler of Eclipse Luna (works) and the Oracle javac (does not work).

---

// sample how to use
public static interface SomeFunction<I, O> extends java.io.Serializable {

    List<O> applyTheFunction(Set<I> value);
}

public static void main(String[] args) throws Exception {

    SomeFunction<Double, Long> lambda = (set) -> Collections.singletonList(set.iterator().next().longValue());

    SerializedLambda sl = getSerializedLambda(lambda);      
    Method m = getLambdaMethod(sl);

    System.out.println(m);
    System.out.println(m.getGenericReturnType());
    for (Type t : m.getGenericParameterTypes()) {
        System.out.println(t);
    }

    // prints the following
    // (the method) private static java.util.List test.ClassWithLambdas.lambda$0(java.util.Set)
    // (the return type, including *Long* as the generic list type) java.util.List<java.lang.Long>
    // (the parameter, including *Double* as the generic set type) java.util.Set<java.lang.Double>

---

// getting the SerializedLambda
public static SerializedLambda getSerializedLambda(Object function) {
    if (function == null || !(function instanceof java.io.Serializable)) {
        throw new IllegalArgumentException();
    }

    for (Class<?> clazz = function.getClass(); clazz != null; clazz = clazz.getSuperclass()) {
        try {
            Method replaceMethod = clazz.getDeclaredMethod("writeReplace");
            replaceMethod.setAccessible(true);
            Object serializedForm = replaceMethod.invoke(function);

            if (serializedForm instanceof SerializedLambda) {
                return (SerializedLambda) serializedForm;
            }
        }
        catch (NoSuchMethodError e) {
            // fall through the loop and try the next class
        }
        catch (Throwable t) {
            throw new RuntimeException("Error while extracting serialized lambda", t);
        }
    }

    throw new Exception("writeReplace method not found");
}

---

// getting the synthetic static lambda method
public static Method getLambdaMethod(SerializedLambda lambda) throws Exception {
    String implClassName = lambda.getImplClass().replace('/', '.');
    Class<?> implClass = Class.forName(implClassName);

    String lambdaName = lambda.getImplMethodName();

    for (Method m : implClass.getDeclaredMethods()) {
        if (m.getName().equals(lambdaName)) {
            return m;
        }
    }

    throw new Exception("Lambda Method not found");
}

Why Java Lambda Type Inference deals only with method with unique parameter ?

It seems you are not defining this correctly:

(str, str2) -> str.length()

You should also receive a compile time error for the second example.

Type inference on lambdas

A simple fix is to use a target type:

return Optional.<Integer> empty();

Also I note that you use Integer.parseInt which returns an int, so you could also use an OptionalInt which will solve your problem and save a boxing operation:

try {
  return OptionalInt.of(Integer.parseInt(line));
} catch (NumberFormatException xep) {
  return OptionalInt.empty();
}

Why Doesn't Java 8 Type Inference Consider Exceptions Thrown by Lambdas in Overload Selection?

If it makes you feel any better, this topic was indeed carefully considered during the JSR-335 design process.

The question is not "why isn't it able", but "why did we choose not to." When we find multiple potentially applicable overloads, we certainly could have chosen to speculatively attribute the lambda body under each set of signatures, and prune those candidates for which the lambda body failed to type-check.

However, we concluded that doing so would likely do more harm than good; it means, for example, that small changes to the method body, under this rule, could cause some method overload selection decisions to silently change without the user intending to do so. In the end, we concluded that using the presence of errors in the method body to discard a potentially applicable candidate would cause more confusion than benefit, especially given that there is a simple and safe workaround -- provide a target-type. We felt that reliability and predictability here outweighed optimal concision.

Java lambda type with generics

        new Adapter<Double>() {
            @Override
            public Double adapt(String s) {
                return Double.valueOf(s);
            }
        }

For the above code, it will create an implementation class for Adapter interface, after you compiled you can see: A$1.class is under your directory, this is an imeplementated class for new Adapter. and you can use javap -c A\$1 to check the bytecode, like:

Compiled from "A.java"
final class A$1 implements A$Adapter<java.lang.Double> {
  A$1();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public java.lang.Double adapt(java.lang.String);
    Code:
       0: aload_1
       1: invokestatic  #2                  // Method java/lang/Double.valueOf:(Ljava/lang/String;)Ljava/lang/Double;
       4: areturn

  public java.lang.Object adapt(java.lang.String);
    Code:
       0: aload_0
       1: aload_1
       2: invokevirtual #3                  // Method adapt:(Ljava/lang/String;)Ljava/lang/Double;
       5: areturn
}

as you can see the A$1.adapt class has inferred the Double type by compiler.

and for Lambda expression, the compiler just deal it as invokedynamic #2, 0 // InvokeDynamic #0:adapt:()LA$Adapter;, so caused by type erasure, this will be Object type for this.

Different generic behaviour when using lambda instead of explicit anonymous inner class

tldr:

1. There is a bug in javac that records the wrong enclosing method for lambda-embedded inner classes. As a result, type variables on the actual enclosing method cannot be resolved by those inner classes.
2. There are arguably two sets of bugs in the java.lang.reflect API implementation:
   
   
   • Some methods are documented as throwing exceptions when nonexistent types are encountered, but they never do. Instead, they allow null references to propagate.
   • The various Type::toString() overrides currently throw or propagate a NullPointerException when a type cannot be resolved.

The answer has to do with the generic signatures that usually get emitted in class files that make use of generics.

Typically, when you write a class that has one or more generic supertypes, the Java compiler will emit a Signature attribute containing the fully parameterized generic signature(s) of the class's supertype(s). I've written about these before, but the short explanation is this: without them, it would not be possible to consume generic types as generic types unless you happened to have the source code. Due to type erasure, information about type variables gets lost at compilation time. If that information were not included as extra metadata, neither the IDE nor your compiler would know that a type was generic, and you could not use it as such. Nor could the compiler emit the necessary runtime checks to enforce type safety.

javac will emit generic signature metadata for any type or method whose signature contains type variables or a parameterized type, which is why you are able to obtain the original generic supertype information for your anonymous types. For example, the anonymous type created here:

TypeToken<?> token = new TypeToken<List<? extends CharSequence>>() {};

...contains this Signature:

LTypeToken<Ljava/util/List<+Ljava/lang/CharSequence;>;>;

From this, the java.lang.reflection APIs can parse the generic supertype information about your (anonymous) class.

But we already know that this works just fine when the TypeToken is parameterized with concrete types. Let's look at a more relevant example, where its type parameter includes a type variable:

static <F> void test() {
    TypeToken sup = new TypeToken<F[]>() {};
}

Here, we get the following signature:

LTypeToken<[TF;>;

Makes sense, right? Now, let's look at how the java.lang.reflect APIs are able to extract generic supertype information from these signatures. If we peer into Class::getGenericSuperclass(), we see that the first thing it does is call getGenericInfo(). If we haven't called into this method before, a ClassRepository gets instantiated:

private ClassRepository getGenericInfo() {
    ClassRepository genericInfo = this.genericInfo;
    if (genericInfo == null) {
        String signature = getGenericSignature0();
        if (signature == null) {
            genericInfo = ClassRepository.NONE;
        } else {
            // !!!  RELEVANT LINE HERE:  !!!
            genericInfo = ClassRepository.make(signature, getFactory());
        }
        this.genericInfo = genericInfo;
    }
    return (genericInfo != ClassRepository.NONE) ? genericInfo : null;
}

The critical piece here is the call to getFactory(), which expands to:

CoreReflectionFactory.make(this, ClassScope.make(this))

ClassScope is the bit we care about: this provides a resolution scope for type variables. Given a type variable name, the scope gets searched for a matching type variable. If one is not found, the 'outer' or enclosing scope is searched:

public TypeVariable<?> lookup(String name) {
    TypeVariable<?>[] tas = getRecvr().getTypeParameters();
    for (TypeVariable<?> tv : tas) {
        if (tv.getName().equals(name)) {return tv;}
    }
    return getEnclosingScope().lookup(name);
}

And, finally, the key to it all (from ClassScope):

protected Scope computeEnclosingScope() {
    Class<?> receiver = getRecvr();

    Method m = receiver.getEnclosingMethod();
    if (m != null)
        // Receiver is a local or anonymous class enclosed in a method.
        return MethodScope.make(m);

    // ...
}

If a type variable (e.g., F) is not found on the class itself (e.g., the anonymous TypeToken<F[]>), then the next step is to search the enclosing method. If we look at the disassembled anonymous class, we see this attribute:

EnclosingMethod: LambdaTest.test()V

The presence of this attribute means that computeEnclosingScope will produce a MethodScope for the generic method static <F> void test(). Since test declares the type variable W, we find it when we search the enclosing scope.

So, why doesn't it work inside a lambda?

To answer this, we must understand how lambdas get compiled. The body of the lambda gets moved into a synthetic static method. At the point where we declare our lambda, an invokedynamic instruction gets emitted, which causes a TypeToken implementation class to be generated the first time we hit that instruction.

In this example, the static method generated for the lambda body would look something like this (if decompiled):

private static /* synthetic */ Object lambda$test$0() {
    return new LambdaTest$1();
}

...where LambdaTest$1 is your anonymous class. Let's dissassemble that and inspect our attributes:

Signature: LTypeToken<TW;>;
EnclosingMethod: LambdaTest.lambda$test$0()Ljava/lang/Object;

Just like the case where we instantiated an anonymous type outside of a lambda, the signature contains the type variable W. But EnclosingMethod refers to the synthetic method.

The synthetic method lambda$test$0() does not declare type variable W. Moreover, lambda$test$0() is not enclosed by test(), so the declaration of W is not visible inside it. Your anonymous class has a supertype containing a type variable that your the class doesn’t know about because it’s out of scope.

When we call getGenericSuperclass(), the scope hierarchy for LambdaTest$1 does not contain W, so the parser cannot resolve it. Due to how the code is written, this unresolved type variable results in null getting placed in the type parameters of the generic supertype.

Note that, had your lambda had instantiated a type that did not refer to any type variables (e.g., TypeToken<String>) then you would not run into this problem.

Conclusions

(i) There is a bug in javac. The Java Virtual Machine Specification §4.7.7 ("The EnclosingMethod Attribute") states:

It is the responsibility of a Java compiler to ensure that the method identified via the method_index is indeed the closest lexically enclosing method of the class that contains this EnclosingMethod attribute. (emphasis mine)

Currently, javac seems to determine the enclosing method after the lambda rewriter runs its course, and as a result, the EnclosingMethod attribute refers to a method that never even existed in the lexical scope. If EnclosingMethod reported the actual lexically enclosing method, the type variables on that method could be resolved by the lambda-embedded classes, and your code would produce the expected results.

It is arguably also a bug that the signature parser/reifier silently allows a null type argument to be propagated into a ParameterizedType (which, as @tom-hawtin-tackline points out, has ancillary effects like toString() throwing a NPE).

My bug report for the EnclosingMethod issue is now online.

(ii) There are arguably multiple bugs in java.lang.reflect and its supporting APIs.

The method ParameterizedType::getActualTypeArguments() is documented as throwing a TypeNotPresentException when "any of the actual type arguments refers to a non-existent type declaration". That description arguably covers the case where a type variable is not in scope. GenericArrayType::getGenericComponentType() should throw a similar exception when "the underlying array type's type refers to a non-existent type declaration". Currently, neither appears to throw a TypeNotPresentException under any circumstances.

I would also argue that the various Type::toString overrides should merely fill in the canonical name of any unresolved types rather than throwing a NPE or any other exception.

I have submitted a bug report for these reflection-related issues, and I will post the link once it is publicly visible.

Workarounds?

If you need to be able to reference a type variable declared by the enclosing method, then you can't do that with a lambda; you'll have to fall back to the longer anonymous type syntax. However, the lambda version should work in most other cases. You should even be able to reference type variables declared by the enclosing class. For example, these should always work:

class Test<X> {
    void test() {
        Supplier<TypeToken<X>> s1 = () -> new TypeToken<X>() {};
        Supplier<TypeToken<String>> s2 = () -> new TypeToken<String>() {};
        Supplier<TypeToken<List<String>>> s3 = () -> new TypeToken<List<String>>() {};
    }
}

Unfortunately, given that this bug has apparently existed since lambdas were first introduced, and it has not been fixed in the most recent LTS release, you may have to assume the bug remains in your clients’ JDKs long after it gets fixed, assuming it gets fixed at all.

Why didn't this java 8 example using type inference compile in Eclipse?

First, correcting your terminology: when you say syntax sugar, what you really are asking about is type inference, that when asked to infer a type for j in the inner lambda, that the compiler fails to come up with the right type.

Second, correcting your data: The error messages you cite are not coming from the JDK compiler; they're coming from Eclipse.

This is just an Eclipse bug. The reference compiler (javac from Oracle JDK) handles your first example just fine.

Is it possible (how) to get the name of a method reference at Runtime Java?

As you're saying that you only need this for debugging purposes, here is a trick (i.e. a dirty hack) that will allow you to do what you want.

First of all, your functional interface must be Serializable:

@FunctionalInterface
public interface FooInterface extends Serializable {

    void method();
}

Now, you can use this undocumented, internal-implementation-dependent and extremely risky code to print some information about the method reference targeted to your FooInterface functional interface:

@FunctionalInterface
public interface FooInterface extends Serializable {

    void method();

    default String getName() {
        try {
            Method writeReplace = this.getClass().getDeclaredMethod("writeReplace");
            writeReplace.setAccessible(true);
            SerializedLambda sl = (SerializedLambda) writeReplace.invoke(this);
            return sl.getImplClass() + "::" + sl.getImplMethodName();
        } catch (Exception e) {
            return null;
        }
    }
}

When you call this method:

doStuff(Foo::aMethodReference);

You'll see the following output:

package/to/the/class/Foo::aMethodReference

Note 1: I've seen this approach in this article by Peter Lawrey.

Note 2: I've tested this with openjdk version "11" 2018-09-25 and also with java version "1.8.0_192".

How JDK 8's type inference works with generic?

Well, there is an entire new chapter, §18. Type Inference, in the language specification, but it’s not an easy read. Even the first section’s summary, addressing exactly your question, is tough:

In comparison to the Java SE 7 Edition of The Java® Language Specification, important changes to inference include:

• Adding support for lambda expressions and method references as method invocation arguments.
• Generalizing to define inference in terms of poly expressions, which may not have well-defined types until after inference is complete. This has the notable effect of improving inference for nested generic method and diamond constructor invocations.
• Describing how inference is used to handle wildcard-parameterized functional interface target types and most specific method analysis.
• Clarifying the distinction between invocation applicability testing (which involves only the invocation arguments) and invocation type inference (which incorporates a target type).
• Delaying resolution of all inference variables, even those with lower bounds, until invocation type inference, in order to get better results.
• Improving inference behavior for interdependent (or self-dependent) variables.
• Eliminating bugs and potential sources of confusion. This revision more carefully and precisely handles the distinction between specific conversion contexts and subtyping, and describes reduction by paralleling the corresponding non-inference relations. Where there are intentional departures from the non-inference relations, these are explicitly identified as such.
• Laying a foundation for future evolution: enhancements to or new applications of inference will be easier to integrate into the specification.

The second bullet has the biggest impact on your code example. You have a nested method invocation of a generic method without specifying explicit type arguments, which makes it a so-called poly expression whose actual type might be inferred from the target type, which is the parameter type of AAA in your case.

So this is a rather easy constellation as AAA isn’t generic and no ambiguity regarding it’s parameter type. It’s always List<A>. There is no search for a “common ancestor class” here, all that has to be checked, is whether the argument expression’s type (B) is compatible with the inferred type (A).

Java 8: generic type inference fails on method reference?

This is an Eclipse bug. I narrowed it down to the use of a nested generic parameter (Collection<R>) as the type of an argument (in BiPredicate) to the referenced method (specified as Collection<KolladaRoot>). It should compile fine in javac.exe.

I'd say stick with explicit type specification until it's fixed in 4.5 M2, it should have less effect (if any) than switching to lambdas.

---

Related Topics