Curious Null-Coalescing Operator Custom Implicit Conversion Behaviour

Curious null-coalescing operator custom implicit conversion behaviour

Thanks to everyone who contributed to analyzing this issue. It is clearly a compiler bug. It appears to only happen when there is a lifted conversion involving two nullable types on the left-hand side of the coalescing operator.

I have not yet identified where precisely things go wrong, but at some point during the "nullable lowering" phase of compilation -- after initial analysis but before code generation -- we reduce the expression

result = Foo() ?? y;

from the example above to the moral equivalent of:

A? temp = Foo();
result = temp.HasValue ? 
    new int?(A.op_implicit(Foo().Value)) : 
    y;

Clearly that is incorrect; the correct lowering is

result = temp.HasValue ? 
    new int?(A.op_implicit(temp.Value)) : 
    y;

My best guess based on my analysis so far is that the nullable optimizer is going off the rails here. We have a nullable optimizer that looks for situations where we know that a particular expression of nullable type cannot possibly be null. Consider the following naive analysis: we might first say that

result = Foo() ?? y;

is the same as

A? temp = Foo();
result = temp.HasValue ? 
    (int?) temp : 
    y;

and then we might say that

conversionResult = (int?) temp 

is the same as

A? temp2 = temp;
conversionResult = temp2.HasValue ? 
    new int?(op_Implicit(temp2.Value)) : 
    (int?) null

But the optimizer can step in and say "whoa, wait a minute, we already checked that temp is not null; there's no need to check it for null a second time just because we are calling a lifted conversion operator". We'd them optimize it away to just

new int?(op_Implicit(temp2.Value)) 

My guess is that we are somewhere caching the fact that the optimized form of (int?)Foo() is new int?(op_implicit(Foo().Value)) but that is not actually the optimized form we want; we want the optimized form of Foo()-replaced-with-temporary-and-then-converted.

Many bugs in the C# compiler are a result of bad caching decisions. A word to the wise: every time you cache a fact for use later, you are potentially creating an inconsistency should something relevant change. In this case the relevant thing that has changed post initial analysis is that the call to Foo() should always be realized as a fetch of a temporary.

We did a lot of reorganization of the nullable rewriting pass in C# 3.0. The bug reproduces in C# 3.0 and 4.0 but not in C# 2.0, which means that the bug was probably my bad. Sorry!

I'll get a bug entered into the database and we'll see if we can get this fixed up for a future version of the language. Thanks again everyone for your analysis; it was very helpful!

UPDATE: I rewrote the nullable optimizer from scratch for Roslyn; it now does a better job and avoids these sorts of weird errors. For some thoughts on how the optimizer in Roslyn works, see my series of articles which begins here: https://ericlippert.com/2012/12/20/nullable-micro-optimizations-part-one/

Implicit conversion with null-coalescing operator

Well, the specification says (I change to x and y for less confusion here):

• Otherwise, if y has a type Y and an implicit conversion exists from x to Y, the result type is Y. At run-time, x is first evaluated. If x is not null, x is unwrapped to type X0 (if X exists and is nullable) and converted to type Y, and this becomes the result. Otherwise, y is evaluated and becomes the result.

This happens. First, the left-hand side x, which is just a, is checked for null. But it is not null in itself. Then the left-hand side is to be used. The implicit conversion is then run. Its result of type B is ... null.

Note that this is different from:

    A a = new A();
    B b = (B)a ?? new B();

In this case the left operand is an expression (x) which is null in itself, and the result becomes the right-hand side (y).

Maybe implicit conversions between reference types should return null (if and) only if the original is null, as a good practice?

---

I guess the guys who wrote the spec could have done it like this (but did not):

• Otherwise, if y has a type Y and an implicit conversion exists from x to Y, the result type is Y. At run-time, x is first evaluated and converted to type Y. If the output of that conversion is not null, that output becomes the result. Otherwise, y is evaluated and becomes the result.

Maybe that would have been more intuitive? It would have forced the runtime to call your implicit conversion no matter if the input to the conversion were null or not. That should not be too expensive if typical implementations quickly determined that null → null.

Implicit casting of Null-Coalescing operator result

To make the second case work with the ternary operator, you could use the following:

int result = input != null ? input.Value : 10;

The Value property of the Nullable<T> type returns the T value (in this case, the int).

Another option is to use Nullable<T>.HasValue:

int result = input.HasValue ? input.Value : 10;

The myNullableInt != null construct is only syntactic sugar for the above HasValue call.

What is the justification for this NullableT behavior with implicit conversion operators

I said earlier that (1) this is a compiler bug and (2) it is a new one. The first statement was accurate; the second was me getting confused in my haste to get to the bus on time. (The bug I was thinking of that is new to me is a much more complicated bug involving lifted conversions and lifted increment operators.)

This is a known compiler bug of long standing. Jon Skeet first brought it to my attention some time ago and I believe there's a StackOverflow question about it somewhere; I do not recall where offhand. Perhaps Jon does.

So, the bug. Let's define a "lifted" operator. If an operator converts from a non-nullable value type S to a non-nullable value type T then there is also a "lifted" operator that converts from S? to T?, such that a null S? converts to a null T? and a non-null S? converts to T? by unwrapping S? to S, converting S to T, and wrapping T to T?.

The specification says that (1) the only situation in which there is a lifted operator is when S and T are both non-nullable value types, and (2) that the lifted and non-lifted conversion operators are both considered as to whether they are applicable candidates for the conversion and if both applicable, then the source and target types of the applicable conversions, lifted or unlifted, are used to determine the best source type, best target type, and ultimately, best conversion of all the applicable conversions.

Unfortunately, the implementation thoroughly violates all of these rules, and does so in a way that we cannot change without breaking many existing programs.

First off, we violate the rule about the existence of lifted operators. A lifted operator is considered by the implementation to exist if S and T are both non-nullable value types, or if S is a non-nullable value type and T is any type to which a null could be assigned: reference type, nullable value type, or pointer type. In all those cases we produce a lifted operator.

In your particular case, we lift to nullable by saying that we convert a nullable type to the reference type Cat by checking for null. If the source is not null then we convert normally; if it is, then we produce a null Cat.

Second, we violate thoroughly the rule about how to determine the best source and target types of applicable candidates when one of those candidates is a lifted operator, and we also violate the rules about determining which is the best operator.

In short, it is a big mess that cannot be fixed without breaking real customers, and so we will likely enshrine the behaviour in Roslyn. I will consider documenting the exact behaviour of the compiler in my blog at some point, but I would not hold my breath while waiting for that day if I were you.

And of course, many apologies for the errors.

C# type inference (var) assignment from '??' null-coalescing operator

It's not a runtime consideration.

The compile time type of a variable declared using var is the static type of its initializer. The static type of a ?? expression is the common type of the static type of both operands. But the static type of the second operand is the static type of y, which isn't known. Therefore the static type of the whole initializer is unknown, and deduction fails.

It's true that there exist types for which the initialization would be consistent, but they can't be found using the C# inference rules.

How can I use 'null coalescing operator' together with converting values if they exists?

$bookid = (int)($_POST['bookid'] ?? 0);
$userid = (int)($_POST['userid'] ?? 0);
$binding = (int)($_POST['binding'] ?? 0);
$colorpages = (int)($_POST['colorpages'] ?? 0);

So (int) won't have to deal with the null values

not exact 7.13 The null coalescing operator in C# 4.0 spec

in C#3.0 it says:

Otherwise, if an implicit conversion exists from A0 to B, the result
type is B.

in my guess, it may be a typo in C#4.0.

Implicit conversion to struct executes against Nullablestruct

because you've a variable of type MyId, not of Guid? and you're assigning that variable to a Guid? and the MyId type has an IMPLICIT conversion operator. If you made it EXPLICIT then the compiler would probably complain about no conversion. What exactly would you expect to happen by assigning a null Foo class instance to a Guid? variable? Do you really expect that to be meaningful? –

How the right associative of null coalescing operator behaves?

The spec is actually self-contradictory on this one.

Section 7.13 of the C# 4 spec states:

The null coalescing operator is right-associative, meaning that operations are grouped from right to left. For example, an expression of the form a ?? b ?? c is evaluated as a ?? (b ?? c).

On the other hand, as has been pointed out, 7.3.1 claims that:

Except for the assignment operators, all binary operators are left-associative

I entirely agree that for simple cases it doesn't matter how you do the grouping... but there may be cases where it really matters due to implicit type conversions doing interesting things if the operands have different types.

I'll consider it further, ping Mads and Eric, and add an erratum for the relevant section of C# in Depth (which inspired this question).

EDIT: Okay, I've now got an example where it does matter... and the null coalescing operator is definitely right-associative, at least in the MS C# 4 compiler. Code:

using System;

public struct Foo
{
    public static implicit operator Bar(Foo input)
    {
        Console.WriteLine("Foo to Bar");
        return new Bar();
    }

    public static implicit operator Baz(Foo input)
    {
        Console.WriteLine("Foo to Baz");
        return new Baz();
    }
}

public struct Bar
{
    public static implicit operator Baz(Bar input)
    {
        Console.WriteLine("Bar to Baz");
        return new Baz();
    }
}

public struct Baz
{
}

class Test
{
    static void Main()
    {
        Foo? x = new Foo();
        Bar? y = new Bar();
        Baz? z = new Baz();

        Console.WriteLine("Unbracketed:");
        Baz? a = x ?? y ?? z;
        Console.WriteLine("Grouped to the left:");
        Baz? b = (x ?? y) ?? z;
        Console.WriteLine("Grouped to the right:");
        Baz? c = x ?? (y ?? z);
    }
}

Output:

Unbracketed:
Foo to Baz
Grouped to the left:
Foo to Bar
Foo to Bar
Bar to Baz
Grouped to the right:
Foo to Baz

In other words,

x ?? y ?? z

behaves the same as

x ?? (y ?? z)

but not the same as

(x ?? y) ?? z

I'm not currently sure why there are two conversions from Foo to Bar when using (x ?? y) ?? z - I need to check that out more carefully...

EDIT: I now have another question to cover the double conversion...

Problem with implicit conversion and null

I believe that your problem is that both sides of the ternary operator must be of the same or compatible types.

Try writing

if (v == null)
    return null;
else
    return new MyClass(v);

EDIT: I can only reproduce your issue if I make MyClass a struct, in which case your question is impossible; a struct cannot be null.

Please provide more details.

---

Related Topics