How Are Nullable Types Implemented Under the Hood in .Net

how are nullable types implemented under the hood in .net?

Ultimately, they are just a generic struct with a bool flag - except with special boxing rules. Because structs are (by default) initialized to zero, the bool defaults to false (no value):

public struct Nullable<T> where T : struct {
private readonly T value;
private readonly bool hasValue;
public Nullable(T value) {
this.value = value;
hasValue = true;
}
public T Value {
get {
if(!hasValue) throw some exception ;-p
return value;
}
}
public T GetValueOrDefault() { return value; }
public bool HasValue {get {return hasValue;}}
public static explicit operator T(Nullable<T> value) {
return value.Value; }
public static implicit operator Nullable<T>(T value) {
return new Nullable<T>(value); }
}

Extra differences, though:

  • special boxing rules (you can't normally do this)
  • special C# rules for comparing to null etc
  • "lifted" operators in C# (and in .NET via EqualityComparer<T>, Comparer<T> etc)
  • special rules on generic type constraints (to prevent Nullable<Nullable<T>>)

How does a NullableT type work behind the scenes?

The nullable type is a struct consisting of two fields: a bool and a T. When the value is null, the bool is false and the T has the default value. When the value is not null, the bool is true.

There are two main benefits to using Nullable as compared to implementing the functionality yourself. There's the language support, as described in more detail in ChaosPandion's answer, and there's the fact that boxing (converting to an object) will automatically remove the nullable "wrapper", leaving either a null reference or the plain T object.z

How Does .Net Allow Nullables To Be Set To Null

How did Nullable<T> become an exception to the rule "You can't assign null to a value type?"

By changing the language, basically. The null literal went from being "a null reference" to "the null value of the relevant type".

At execution time, "the null value" for a nullable value type is a value where the HasValue property returns false. So this:

int? x = null;

is equivalent to:

int? x = new int?();

It's worth separating the framework parts of Nullable<T> from the language and CLR aspects. In fact, the CLR itself doesn't need to know much about nullable value types - as far as I'm aware, the only important aspect is that the null value of a nullable value type is boxed to a null reference, and you can unbox a null reference to the null value of any nullable value type. Even that was only introduced just before .NET 2.0's final release.

The language support mostly consists of:

  • Syntactic sugar in the form of ? so int? is equivalent to Nullable<int>
  • Lifted operators
  • The changed meaning of null
  • The null-coalescing operator (??) - which isn't restricted to nullable value types

Where in memory are nullable types stored?

First off, Nullable<int> is just a shorthand for something like:

struct Nullable<T> 
{
bool hasValue;
T value;
}

Plus all the constructors, accessors, and so on. That's all it is -- a nullable int is an ordinary int plus a flag that says whether the int is null or not. All the rest is compiler magic that treats "null" as a valid value; all "null" does with a nullable type is makes you one of those structs with the flag set to false.

So now that we have that out of the way, your question is "where do they go in memory"? They go the same place that any other structs go in memory: where the runtime and compiler believe to be the best place given the lifetime of the memory.

Most structs go on the heap. Anyone who tells you that "structs always go on the stack" doesn't actually know what they are talking about; our documentation does not say that and it is not true. Structs only go on the temporary memory pool, aka "the stack", when they are local variables or temporaries, and the local variables are not closed-over outer variables of an anonymous method or lambda, and the local variables are not in an iterator block. All other structs go on the heap in our implementation.

Note also that there is no requirement whatsoever that an implementation of the CLI use "the stack" to make their temporary pool. The classic JScript memory manager, for example, stores its temporary pool on the heap. (Though of course the JScript runtime engine is not an implementation of the CLI; I'm merely pointing out that one can design a managed runtime engine that puts no user data whatsoever on "the stack".) Logically it is a stack data structure, but that data structure is not store on "the" stack, it's just a stack structure allocated on the heap.

I have to ask: why do you care? The CLR manages memory on your behalf. Why do you care where nullable types go? They go where they live long enough to be useful to you; you don't have to worry about that.

How is it that can I execute method on int? set to null without NullReferenceException?

Because int? is actually a Nullable<Int32> and Nullable<T> is a struct, and a structure cannot be null.

It is just how Nullable types work. They are not reference values, so they can't be null, but they can have a state when they are considered equivalent to null.

You can get more details about Nullable<T> implementation in how are nullable types implemented under the hood in .net? and Nullable<T> implementation


Though as pointed by @JeppeStigNielsen there is one case when you can get a NRE:

However: When boxed to a reference type, special treatment of
Nullable<> ensures we do get a true null reference. So for example
i.GetType() with i as in the question will blow up with the
NullReferenceException. That is because this method is defined on
object and not overridable

How does the compiler recognise NullableT to be a 'special type'?

Based off of the Shared Source CLI 2.0, Nullable<T> is made "special" via the PREDEFTYPEDEF macro, which takes the name "System.Nullable" and maps it to the attribute PT_G_OPTIONAL which is checked throughout the rest of the compiler.

With regards to the aliases of intSystem.Int32 et al., see the "nice name" column.

From sscli20\csharp\inc\predeftype.h:

//         id            full type name       required  simple     numer    AggKind  fund type   elementtype,      nice name,    zero, quasi simple numer, attribute arg size serialization type,  predef attribute, arity, in mscorlib)
PREDEFTYPEDEF(PT_BYTE, "System.Byte", 1, 1, 1, Struct, FT_U1, ELEMENT_TYPE_U1, L"byte", 0, 0, 1, SERIALIZATION_TYPE_U1, PA_COUNT, 0, 1)
PREDEFTYPEDEF(PT_SHORT, "System.Int16", 1, 1, 1, Struct, FT_I2, ELEMENT_TYPE_I2, L"short", 0, 0, 2, SERIALIZATION_TYPE_I2, PA_COUNT, 0, 1)
PREDEFTYPEDEF(PT_INT, "System.Int32", 1, 1, 1, Struct, FT_I4, ELEMENT_TYPE_I4, L"int", 0, 0, 4, SERIALIZATION_TYPE_I4, PA_COUNT, 0, 1)
PREDEFTYPEDEF(PT_LONG, "System.Int64", 1, 1, 1, Struct, FT_I8, ELEMENT_TYPE_I8, L"long", &longZero, 0, 8, SERIALIZATION_TYPE_I8, PA_COUNT, 0, 1)
// ... snip ...
// Nullable<T>
PREDEFTYPEDEF(PT_G_OPTIONAL, "System.Nullable", 0, 0, 0, Struct, FT_STRUCT, ELEMENT_TYPE_END, NULL, 0, 0, 0, 0, PA_COUNT, 1, 1)

Then its used elsewhere like this:

From sscli20\csharp\sccomp\nullable.cpp:

/***************************************************************************************************
Return true iff the method is the nullable ctor taking one parameter.
***************************************************************************************************/
bool FUNCBREC::IsNubCtor(METHSYM * meth)
{
return meth && meth->getClass()->isPredefAgg(PT_G_OPTIONAL) && meth->params->size == 1 &&
meth->params->Item(0)->isTYVARSYM() && meth->isCtor();
}

For those reading this question, you are probably the target audience for Shared Source CLI 2.0 Internals, which was released as a free e-book.

Can Nullable types be sent through Protocol Buffers?

I will try to improve Nick's answer as it hasn't helped me.
grpc compiler claimed that he has no information on google.protobuf.Int32Wrapper type. I have found it is actually called google.protobuf.Int32Value (https://github.com/protocolbuffers/protobuf/blob/48234f5f012582843bb476ee3afef36cda94cb66/src/google/protobuf/wrappers.proto#L88), though google really calls it Int32Wrapper.
So the code that helped me was the following:

...
import "google/protobuf/wrappers.proto";
...
message TestMessage {
...
google.protobuf.Int32Value nullableInt = 5;
}

Other links:

  • C# lib source - https://github.com/protocolbuffers/protobuf/blob/48234f5f012582843bb476ee3afef36cda94cb66/csharp/src/Google.Protobuf/WellKnownTypes/Wrappers.cs#L781
  • C# doc - https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/int32-value

Are nullable value types just wrappers around the regular value type?

Yes, it is a generic struct:

public struct Nullable<T> where T : struct, new()

This is probably more confusing if you've only seen the T? syntax - but that is just syntactic sugar, the compiler is changing it to Nullable<T>.

Source: http://msdn.microsoft.com/en-us/library/b3h38hb0.aspx , http://msdn.microsoft.com/en-us/library/1t3y8s4s.aspx



Related Topics



Leave a reply



Submit