Ref and Out Parameters in C# and Cannot Be Marked as Variant

ref and out parameters in C# and cannot be marked as variant

"out" means, roughly speaking, "only appears in output positions".

"in" means, roughly speaking, "only appears in input positions".

The real story is a bit more complicated than that, but the keywords were chosen because most of the time this is the case.

Consider a method of an interface or the method represented by a delegate:

delegate void Foo</*???*/ T>(ref T item);

Does T appear in an input position? Yes. The caller can pass a value of T in via item; the callee Foo can read that. Therefore T cannot be marked "out".

Does T appear in an output position? Yes. The callee can write a new value to item, which the caller can then read. Therefore T cannot be marked "in".

Therefore if T appears in a "ref" formal parameter, T cannot be marked as either in or out.

Let's look at some real examples of how things go wrong. Suppose this were legal:

delegate void X<out T>(ref T item);
...
X<Dog> x1 = (ref Dog d)=>{ d.Bark(); }
X<Animal> x2 = x1; // covariant;
Animal a = new Cat();
x2(ref a);

Well dog my cats, we just made a cat bark. "out" cannot be legal.

What about "in"?

delegate void X<in T>(ref T item);
...
X<Animal> x1 = (ref Animal a)=>{ a = new Cat(); }
X<Dog> x2 = x1; // contravariant;
Dog d = new Dog();
x2(ref d);

And we just put a cat in a variable that can only hold dogs. T cannot be marked "in" either.

What about an out parameter?

delegate void Foo</*???*/T>(out T item);

? Now T only appears in an output position. Should it be legal to make T marked as "out"?

Unfortunately no. "out" actually is not different than "ref" behind the scenes. The only difference between "out" and "ref" is that the compiler forbids reading from an out parameter before it is assigned by the callee, and that the compiler requires assignment before the callee returns normally. Someone who wrote an implementation of this interface in a .NET language other than C# would be able to read from the item before it was initialized, and therefore it could be used as an input. We therefore forbid marking T as "out" in this case. That's regrettable, but nothing we can do about it; we have to obey the type safety rules of the CLR.

Furthermore, the rule of "out" parameters is that they cannot be used for input before they are written to. There is no rule that they cannot be used for input after they are written to. Suppose we allowed

delegate void X<out T>(out T item);
class C
{
Animal a;
void M()
{
X<Dog> x1 = (out Dog d) =>
{
d = null;
N();
if (d != null)
d.Bark();
};
x<Animal> x2 = x1; // Suppose this were legal covariance.
x2(out this.a);
}
void N()
{
if (this.a == null)
this.a = new Cat();
}
}

Once more we have made a cat bark. We cannot allow T to be "out".

It is very foolish to use out parameters for input in this way, but legal.


UPDATE: C# 7 has added in as a formal parameter declaration, which means that we now have both in and out meaning two things; this is going to create some confusion. Let me clear that up:

  • in, out and ref on a formal parameter declaration in a parameter list means "this parameter is an alias to a variable supplied by the caller".
  • ref means "the callee may read or write the aliased variable, and it must be known to be assigned before the call.
  • out means "the callee must write the aliased variable via the alias before it returns normally". It also means that the callee must not read the aliased variable via the alias before it writes it, because the variable might not be definitely assigned.
  • in means "the callee may read the aliased variable but does not write to it via the alias". The purpose of in is to solve a rare performance problem, whereby a large struct must be passed "by value" but it is expensive to do so. As an implementation detail, in parameters are typically passed via a pointer-sized value, which is faster than copying by value, but slower on the dereference.
  • From the CLR's perspective, in, out and ref are all the same thing; the rules about who reads and writes what variables at what times, the CLR does not know or care.
  • Since it is the CLR that enforces rules about variance, rules that apply to ref also apply to in and out parameters.

In contrast, in and out on type parameter declarations mean "this type parameter must not be used in a covariant manner" and "this type parameter must not be used in a contravariant manner", respectively.

As noted above, we chose in and out for those modifiers because if we see IFoo<in T, out U> then T is used in "input" positions and U is used in "output" positions. Though that is not strictly true, it is true enough in the 99.9% use case that it is a helpful mnemonic.

It is unfortunate that interface IFoo<in T, out U> { void Foo(in T t, out U u); } is illegal because it looks like it ought to work. It cannot work because from the CLR verifier's perspective, those are both ref parameters and therefore read-write.

This is just one of those weird, unintended situations where two features that logically ought to work together do not work well together for implementation detail reasons.

Why can't I use output parameters with covariant generic types?

My question actually had an answer already at ref and out parameters in C# and cannot be marked as variant

A relevant bit of Eric Lippert's great answer (but there's more):

Unfortunately no. "out" actually is not different than "ref" behind the scenes. The only difference between "out" and "ref" is that the compiler forbids reading from an out parameter before it is assigned by the callee, and that the compiler requires assignment before the callee returns normally. Someone who wrote an implementation of this interface in a .NET language other than C# would be able to read from the item before it was initialized, and therefore it could be used as an input. We therefore forbid marking T as "out" in this case. That's regrettable, but nothing we can do about it; we have to obey the type safety rules of the CLR.

Why do C# out generic type parameters violate covariance?

The problem is indeed here:

bool TryGetValue( string SUID, out T obj ); // Error here?

You marked obj as out parameter, that still means though that you are passing in obj so it cannot be covariant, since you both pass in an instance of type T as well as return it.

Edit:

Eric Lippert says it better than anyone I refer to his answer to "ref and out parameters in C# and cannot be marked as variant" and quote him in regards to out parameters:

Should it be legal to make T marked as "out"? Unfortunately no. "out"
actually is not different than "ref" behind the scenes. The only
difference between "out" and "ref" is that the compiler forbids
reading from an out parameter before it is assigned by the callee, and
that the compiler requires assignment before the callee returns
normally. Someone who wrote an implementation of this interface in a
.NET language other than C# would be able to read from the item before
it was initialized, and therefore it could be used as an input. We
therefore forbid marking T as "out" in this case
. That's regrettable,
but nothing we can do about it; we have to obey the type safety rules
of the CLR.

Argument not assignable due to contravariance

The root of the problem is that you can get any converter out of your dictionary, and give it your SharedBaseClass instance to convert. Therefore the converters in your dictionary need to be declared as accepting a SharedBaseClass.

If your dictionary accepts converters which take a SharedBaseClass instance, then all converters which go into it must also be able to take a SharedBaseClass instance to convert, as you're technically able to fetch any one of them out of the dictionary, and give it any SharedBaseClass instance.

Any way forwards therefore hinges on us being able to get rid of that dictionary containing IConverter<SharedBaseClass, SharedBaseClass> instances. One possible approach is:

public class ConverterExecutor
{
private readonly Dictionary<Type, Func<SharedBaseClass, SharedBaseClass>> _converters = new();

public void RegisterConverter<TU, T>(IConverter<TU, T> converter) where TU : SharedBaseClass where T : SharedBaseClass
{
_converters[typeof(T)] = x => converter.Convert((T)x);
}

public IEnumerable<SharedBaseClass> ConvertMultiple(IEnumerable<SharedBaseClass> classesToConvert)
{
var converted = new List<SharedBaseClass>();

foreach(var toConvert in classesToConvert)
{
_converters.TryGetValue(toConvert.GetType(), out var converter);

if (converter != null) {
converted.Add(converter(toConvert));
continue;
}

converted.Add(toConvert);
}

return converted;
}
}

Then:

var executor = new ConverterExecutor();
executor.RegisterConverter(new DestinationConverter());

Link.

We've replaced those IConverter<SharedBaseClass, SharedBaseClass> instances with delegates which take a SharedBaseClass and return a SharedBaseClass. Each delegate holds onto a converter, and casts the SharedBaseClass instance into the type which the converter is expecting.

Now if you pass the wrong type to a particular converter you get an InvalidCastException: the problem hasn't really gone away, but we've moved the check from compile-time to runtime.

Can I have an interface parameter, pass by reference?

If you don't use implicit typing, and just define your variable as the interface, it will work:

IMyInterface model = new MyClass():

MyMethod(ref model);

Arguments passed by ref must match the type exactly, as they can be reassigned within the method to another type that matches that contract. In your case, this won't work. Imagine the following:

protected bool MyMethod (ref IMyInterface model) 
{
// This has to be allowed
model = new SomeOtherMyInterface();
}

// Now, in your usage:
var model = new MyClass(); // Exactly the same as MyClass model = new MyClass();

MyMethod(ref model); // Won't compile...

// Here, model would be defined as `MyClass` but have been assigned to a `SomeOtherMyInterface`, hence it's invalid...

How to work around not being able to use refs in a lambda?

When using the ref keyword, the type must also be given. Like this:

(ref object data, ref SDL_Event e) => { ... }

The parameter list of a lambda is like the parameter list of an ordinary named method. The types can be left out in a lambda, however, but only when no parameters have modifyers such as ref, out, params.

T must be contravariantly valid

Consider what would happen if the compiler allowed that:

interface IR<out T>
{
void D(T t);
}

class C : IR<Mammal>
{
public void D(Mammal m)
{
m.GrowHair();
}
}
...
IR<Animal> x = new C();
// legal because T is covariant and Mammal is convertible to Animal
x.D(new Fish()); // legal because IR<Animal>.D takes an Animal

And you just tried to grow hair on a fish.

The "out" means "T is only used in output positions". You are using it in an input position.

Passing properties by reference in C#

Properties cannot be passed by reference. Here are a few ways you can work around this limitation.

1. Return Value

string GetString(string input, string output)
{
if (!string.IsNullOrEmpty(input))
{
return input;
}
return output;
}

void Main()
{
var person = new Person();
person.Name = GetString("test", person.Name);
Debug.Assert(person.Name == "test");
}

2. Delegate

void GetString(string input, Action<string> setOutput)
{
if (!string.IsNullOrEmpty(input))
{
setOutput(input);
}
}

void Main()
{
var person = new Person();
GetString("test", value => person.Name = value);
Debug.Assert(person.Name == "test");
}

3. LINQ Expression

void GetString<T>(string input, T target, Expression<Func<T, string>> outExpr)
{
if (!string.IsNullOrEmpty(input))
{
var expr = (MemberExpression) outExpr.Body;
var prop = (PropertyInfo) expr.Member;
prop.SetValue(target, input, null);
}
}

void Main()
{
var person = new Person();
GetString("test", person, x => x.Name);
Debug.Assert(person.Name == "test");
}

4. Reflection

void GetString(string input, object target, string propertyName)
{
if (!string.IsNullOrEmpty(input))
{
var prop = target.GetType().GetProperty(propertyName);
prop.SetValue(target, input);
}
}

void Main()
{
var person = new Person();
GetString("test", person, nameof(Person.Name));
Debug.Assert(person.Name == "test");
}

ref and out parameters in C# and cannot be marked as variant

"out" means, roughly speaking, "only appears in output positions".

"in" means, roughly speaking, "only appears in input positions".

The real story is a bit more complicated than that, but the keywords were chosen because most of the time this is the case.

Consider a method of an interface or the method represented by a delegate:

delegate void Foo</*???*/ T>(ref T item);

Does T appear in an input position? Yes. The caller can pass a value of T in via item; the callee Foo can read that. Therefore T cannot be marked "out".

Does T appear in an output position? Yes. The callee can write a new value to item, which the caller can then read. Therefore T cannot be marked "in".

Therefore if T appears in a "ref" formal parameter, T cannot be marked as either in or out.

Let's look at some real examples of how things go wrong. Suppose this were legal:

delegate void X<out T>(ref T item);
...
X<Dog> x1 = (ref Dog d)=>{ d.Bark(); }
X<Animal> x2 = x1; // covariant;
Animal a = new Cat();
x2(ref a);

Well dog my cats, we just made a cat bark. "out" cannot be legal.

What about "in"?

delegate void X<in T>(ref T item);
...
X<Animal> x1 = (ref Animal a)=>{ a = new Cat(); }
X<Dog> x2 = x1; // contravariant;
Dog d = new Dog();
x2(ref d);

And we just put a cat in a variable that can only hold dogs. T cannot be marked "in" either.

What about an out parameter?

delegate void Foo</*???*/T>(out T item);

? Now T only appears in an output position. Should it be legal to make T marked as "out"?

Unfortunately no. "out" actually is not different than "ref" behind the scenes. The only difference between "out" and "ref" is that the compiler forbids reading from an out parameter before it is assigned by the callee, and that the compiler requires assignment before the callee returns normally. Someone who wrote an implementation of this interface in a .NET language other than C# would be able to read from the item before it was initialized, and therefore it could be used as an input. We therefore forbid marking T as "out" in this case. That's regrettable, but nothing we can do about it; we have to obey the type safety rules of the CLR.

Furthermore, the rule of "out" parameters is that they cannot be used for input before they are written to. There is no rule that they cannot be used for input after they are written to. Suppose we allowed

delegate void X<out T>(out T item);
class C
{
Animal a;
void M()
{
X<Dog> x1 = (out Dog d) =>
{
d = null;
N();
if (d != null)
d.Bark();
};
x<Animal> x2 = x1; // Suppose this were legal covariance.
x2(out this.a);
}
void N()
{
if (this.a == null)
this.a = new Cat();
}
}

Once more we have made a cat bark. We cannot allow T to be "out".

It is very foolish to use out parameters for input in this way, but legal.


UPDATE: C# 7 has added in as a formal parameter declaration, which means that we now have both in and out meaning two things; this is going to create some confusion. Let me clear that up:

  • in, out and ref on a formal parameter declaration in a parameter list means "this parameter is an alias to a variable supplied by the caller".
  • ref means "the callee may read or write the aliased variable, and it must be known to be assigned before the call.
  • out means "the callee must write the aliased variable via the alias before it returns normally". It also means that the callee must not read the aliased variable via the alias before it writes it, because the variable might not be definitely assigned.
  • in means "the callee may read the aliased variable but does not write to it via the alias". The purpose of in is to solve a rare performance problem, whereby a large struct must be passed "by value" but it is expensive to do so. As an implementation detail, in parameters are typically passed via a pointer-sized value, which is faster than copying by value, but slower on the dereference.
  • From the CLR's perspective, in, out and ref are all the same thing; the rules about who reads and writes what variables at what times, the CLR does not know or care.
  • Since it is the CLR that enforces rules about variance, rules that apply to ref also apply to in and out parameters.

In contrast, in and out on type parameter declarations mean "this type parameter must not be used in a covariant manner" and "this type parameter must not be used in a contravariant manner", respectively.

As noted above, we chose in and out for those modifiers because if we see IFoo<in T, out U> then T is used in "input" positions and U is used in "output" positions. Though that is not strictly true, it is true enough in the 99.9% use case that it is a helpful mnemonic.

It is unfortunate that interface IFoo<in T, out U> { void Foo(in T t, out U u); } is illegal because it looks like it ought to work. It cannot work because from the CLR verifier's perspective, those are both ref parameters and therefore read-write.

This is just one of those weird, unintended situations where two features that logically ought to work together do not work well together for implementation detail reasons.



Related Topics



Leave a reply



Submit