Very Confused by Java 8 Comparator Type Inference

Very confused by Java 8 Comparator type inference

First, all the examples you say cause errors compile fine with the reference implementation (javac from JDK 8.) They also work fine in IntelliJ, so its quite possible the errors you're seeing are Eclipse-specific.

Your underlying question seems to be: "why does it stop working when I start chaining." The reason is, while lambda expressions and generic method invocations are poly expressions (their type is context-sensitive) when they appear as method parameters, when they appear instead as method receiver expressions, they are not.

When you say

Collections.sort(playlist1, comparing(p1 -> p1.getTitle()));

there is enough type information to solve for both the type argument of comparing() and the argument type p1. The comparing() call gets its target type from the signature of Collections.sort, so it is known comparing() must return a Comparator<Song>, and therefore p1 must be Song.

But when you start chaining:

Collections.sort(playlist1,
                 comparing(p1 -> p1.getTitle())
                     .thenComparing(p1 -> p1.getDuration())
                     .thenComparing(p1 -> p1.getArtist()));

now we've got a problem. We know that the compound expression comparing(...).thenComparing(...) has a target type of Comparator<Song>, but because the receiver expression for the chain, comparing(p -> p.getTitle()), is a generic method call, and we can't infer its type parameters from its other arguments, we're kind of out of luck. Since we don't know the type of this expression, we don't know that it has a thenComparing method, etc.

There are several ways to fix this, all of which involve injecting more type information so that the initial object in the chain can be properly typed. Here they are, in rough order of decreasing desirability and increasing intrusiveness:

Use an exact method reference (one with no overloads), like Song::getTitle. This then gives enough type information to infer the type variables for the comparing() call, and therefore give it a type, and therefore continue down the chain.
Use an explicit lambda (as you did in your example).
Provide a type witness for the comparing() call: Comparator.<Song, String>comparing(...).
Provide an explicit target type with a cast, by casting the receiver expression to Comparator<Song>.

Java 8 Comparator comparing doesn't chain

The error seems to be related to Pair's generic parameters. One workaround it to use an explicit type, as you've attempted:

pairList.sort(Comparator.<Pair>comparingInt(Pair::firstValue).thenComparingInt(Pair::secondValue));
//                       ^^^^^^

Note the comparingInt() which reduces the number of parameters you need to specify, and improves performance by avoiding boxing.

Another solution is to parameterize the type reference:

pairList.sort(Comparator.comparingInt(Pair<?,?>::firstValue).thenComparingInt(Pair::secondValue));
//                                        ^^^^^

How does method chaining work in Java 8 Comparator?

As you chain both, the compiler cannot infer the type argument of the returned comparator of comparing()because it depends on the returned comparator of thenComparingInt() that itself cannot be inferred.

Specify the type in the lambda parameter of comparing() (or use a method reference) and it solves the inference issue as the returned type of comparing() could so be inferred. :

    Comparator<Squirrel> c = Comparator.comparing((Squirrel s)  -> s.getSpecies())
                                       .thenComparingInt(s -> s.getWeight());

Note that specifying the type in the lambda parameter of thenComparingInt() (or using a method reference) such as :

    Comparator<Squirrel> c = Comparator.comparing(s -> s.getSpecies())
                                       .thenComparingInt((Squirrel s) -> s.getWeight());

will not work as the receiver (here the return type of the chained method) is not considered in the inference type computation.

This JDK 8 tutorial/documentation explains that very well :

Note: It is important to note that the inference algorithm uses only
invocation arguments, target types, and possibly an obvious expected
return type to infer types. The inference algorithm does not use
results from later in the program.

Java 8 type inference error, assigning lambda expression to a variable of type Object

The type inference for a lambda expression happens from the target type, meaning when you write something like this for example:

 () -> "";

That is indeed a Supplier (to you, not the compiler), but what if I have a type declared like this:

static interface Producer<T> {
    T produce();
}

This means that your lambda could be a Producer or a Supplier. Thus assigning has to be to a @FunctionalInterface (or casting) so that type inference could happen.

In the JLS these are defined as poly expressions (they depend on the context in which they are used - like generics, method reference, ternary operator)

Comparator in collector in stream causes issues with type inference?

Unfortunately, the type inference has a really complex specification, which makes it very hard to decide whether a particular odd behavior is conforming to the specification or just a compiler bug.

There are two well-known deliberate limitations to the type inference.

First, the target type of an expression is not used for receiver expressions, i.e. in a chain of method invocations. So when you have a statement of the form

TargetType x = first.second(…).third(…);

the TargetType will be use to infer the generic type of the third() invocation and its argument expressions, but not for second(…) invocation. So the type inference for second(…) can only use the stand-alone type of first and the argument expressions.

This is not an issue here. Since the stand-alone type of list is well defined as List<String>, there is no problem in inferring the result type Stream<String> for the stream() call and the problematic collect call is the last method invocation of the chain, which can use the target type TreeMap<Integer, List<String>> to infer the type arguments.

The second limitation is about overload resolution. The language designers made a deliberate cut when it comes to a circular dependency between incomplete types of argument expressions which need to know the actual target method and its type, before they could help determine the right method to invoke.

This also doesn’t apply here. While groupingBy is overloaded, these methods differ in the number of parameters, which allows to select the only appropriate method without knowing the argument types. It can also be shown that the compiler’s behavior doesn’t change when we replace groupingBy with a different method which has the intended signature but no overloads.

Your issue can be solved by using, e.g.

TreeMap<Integer, List<String>> res = list.stream()
    .collect(Collectors.groupingBy(
        (String s) -> s.charAt(0) % 3,
        () -> new TreeMap<>(Comparator.reverseOrder()),
        Collectors.toList()
    ));

This uses an explicitly typed lambda expression for the grouping function, which, while not actually contributes to the types of the map’s key, causes the compiler to find the actual types.

While the use of explicitly typed lambda expressions instead of implicitly typed ones can make a difference on method overload resolution, as said above, it shouldn’t apply here, as this specific scenario is not an issue of overloaded methods.

Weirdly enough, even the following change makes the compiler error go away:

static <X> X dummy(X x) { return x; }
…

TreeMap<Integer, List<String>> res = list.stream()
    .collect(Collectors.groupingBy(
        s -> s.charAt(0) % 3,
        dummy(() -> new TreeMap<>(Comparator.reverseOrder())),
        Collectors.toList()
    ));

Here, we’re not helping with any additional explicit type and also not changing the formal nature of the lambda expressions, but still, the compiler suddenly infers all types correctly.

The behavior seems to be connected to the fact that zero parameter lambda expressions are always explicitly typed. Since we can’t change the nature of a zero parameter lambda expression, I created the followed alternative collector method for verification:

public static <T, K, D, A, M extends Map<K, D>>
Collector<T, ?, M> groupingBy(Function<? super T, ? extends K> classifier,
                              Function<Void,M> mapFactory,                                 
                              Collector<? super T, A, D> downstream) {
    return Collectors.groupingBy(classifier, () -> mapFactory.apply(null), downstream);
}

Then, using an implicitly typed lambda expression as map factory compiles without problems:

TreeMap<Integer, List<String>> res = list.stream()
    .collect(groupingBy(
        s -> s.charAt(0) % 3,
        x -> new TreeMap<>(Comparator.reverseOrder()),
        Collectors.toList()
    ));

whereas using an explicitly typed lambda expression causes a compiler error:

TreeMap<Integer, List<String>> res = list.stream()
    .collect(groupingBy(                           // compiler error
        s -> s.charAt(0) % 3,
        (Void x) -> new TreeMap<>(Comparator.reverseOrder()),
        Collectors.toList()
    ));

In my opinion, even if the specification backs up this behavior, it should get corrected, as the implication of providing explicit types should never be that the type inference gets worse than without. That’s especially true for zero argument lambda expressions which we can’t turn into implicitly typed ones.

It also doesn’t explain why turning all arguments into explicitly typed lambda expressions will also eliminate the compiler error.

Making sense of error message related to type inference when using a method reference

The difference between Character::isAlphabetic and c -> Character.isAlphabetic(c) is that since Character.isAlphabetic(int) is not an overloaded method, the former is an exact method reference whereas the latter is an implicitly typed lambda expression.

We can show that an inexact method reference is accepted the same way as an implicitly typed lambda expression:

class SO71643702 {
    public static void main(String[] args) {
        String str = "123abc456def";
        List<Character> l = str.chars()
            .mapToObj(c -> (char) c)
            .filter(Predicate.not(SO71643702::isAlphabetic))
            .toList();
        System.out.println(l);
    }

    public static boolean isAlphabetic(int codePoint) {
        return Character.isAlphabetic(codePoint);
    }

    public static boolean isAlphabetic(Thread t) {
      throw new AssertionError("compiler should never choose this method");
    }
}

This is accepted by the compiler.

However, this doesn’t imply that this behavior is correct. Exact method references may help in overload selection where inexact do not, as specified by §15.12.2.:

Certain argument expressions that contain implicitly typed lambda expressions (§15.27.1) or inexact method references (§15.13.1) are ignored by the applicability tests, because their meaning cannot be determined until the invocation's target type is selected.

In contrast, when it comes to the 15.13.2. Type of a Method Reference, there is no difference between exact and inexact method references mentioned. Only the target type determines the actual type of the method reference (assuming that the target type is a functional interface and the method reference is congruent).

Consequently, the following works without problems:

class SO71643702 {
    public static void main(String[] args) {
        String str = "123abc456def";
        List<Character> l = str.chars()
            .mapToObj(c -> (char) c)
            .filter(Character::isAlphabetic)
            .toList();
        System.out.println(l);
    }
}

^{Of course, that’s not the original program logic}

Here, Character::isAlphabetic still is an exact method reference, but it’s congruent with the target type Predicate<Character>, so it works not different to

Predicate<Character> p = Character::isAlphabetic;

Predicate<Character> p = (Character c) -> Character.isAlphabetic(c);

It’s not as if the insertion of a generic method into nesting of method invocations will stop the type inference from working in general. As discussed in this answer to a similar fragile type inference issue, we can insert a generic method not contributing to the resulting type without problems:

class SO71643702 {
    static <X> X dummy(X x) { return x; }

    public static void main(String[] args) {
        String str = "123abc456def";
        List<Character> l = str.chars()
            .mapToObj(c -> (char) c)
            .filter(dummy(Character::isAlphabetic))
            .toList();
        System.out.println(l);
    }
}

and even “fix” the problem of the original code by inserting the method

class SO71643702 {
    static <X> X dummy(X x) { return x; }

    public static void main(String[] args) {
        String str = "123abc456def";
        List<Character> l = str.chars()
            .mapToObj(c -> (char) c)
            .filter(Predicate.not(dummy(Character::isAlphabetic)))
            .toList();
        System.out.println(l);
    }
}

It’s important that there is no subtype relationship between Predicate<Character> and Predicate<Integer>, so the dummy method can not translate between them. It’s just returning exactly the same type as the compiler inferred for its argument.

I consider the compiler error a bug, but as I said at the other answer, even if the specification backs up this behavior, it should get corrected, in my opinion.

As a side note, for this specific example, I’d use

var l = str.chars()
    .filter(c -> !Character.isAlphabetic(c))
    .mapToObj(c -> (char)c)
    .toList();

anyway, as this way, you’re not boxing int values to Character objects, just to unbox them to int again in the predicate, but rather, only box values after passing the filter.

Very Confused by Java 8 Comparator Type Inference