Are P/Invoke [In, Out] Attributes Optional for Marshaling Arrays

Are P/Invoke [In, Out] attributes optional for marshaling arrays?

No, they are not exactly optional. It just happens to work by accident. It is however a very common accident. It works because the array doesn't actually gets marshaled. The pinvoke marshaller sees that the C# array is already compatible with the native array so skips the step to create a copy of it. It merely pins the array and passes the pointer to the native code.

This is of course very efficient and you will inevitably get the results back because the native code is directly writing the array elements. So neither the [In] nor the [Out] attributes matter.

It gets much murkier if the array element type isn't that simple. It isn't that easy to identify an element type that's a struct or class type that isn't blittable or whose layout doesn't match after marshaling so the pinvoke marshaller has to make a copy of the array. Especially the layout incompatibility can be very hard to identify because the managed layout is undiscoverable. And can change depending on the jitter that is used. It may work in x86 but not x64 for example, pretty nasty when AnyCPU is selected. Getting it to copy the modified copy back to the C# array does require [Out].

Not sure what to advice, other than that nobody ever got fired for being explicit in their declarations. Perhaps you should always be explicit when the array element type isn't simple so you'll never have an accident.

What is the difference between [In, Out] and ref when using pinvoke in C#?

The usage of ref or out is not arbitrary. If the native code requires pass-by-reference (a pointer) then you must use those keywords if the parameter type is a value type. So that the jitter knows to generate a pointer to the value. And you must omit them if the parameter type is a reference type (class), objects are already pointers under the hood.

The [In] and [Out] attributes are then necessary to resolve the ambiguity about pointers, they don't specify the data flow. [In] is always implied by the pinvoke marshaller so doesn't have to be stated explicitly. But you must use [Out] if you expect to see any changes made by the native code to a struct or class member back in your code. The pinvoke marshaller avoids copying back automatically to avoid the expense.

A further quirk then is that [Out] is not often necessary. Happens when the value is blittable, an expensive word that means that the managed value or object layout is identical to the native layout. The pinvoke marshaller can then take a shortcut, pinning the object and passing a pointer to managed object storage. You'll inevitably see changes then since the native code is directly modifying the managed object.

Something you in general strongly want to pursue, it is very efficient. You help by giving the type the [StructLayout(LayoutKind.Sequential)] attribute, it suppresses an optimization that the CLR uses to rearrange the fields to get the smallest object. And by using only fields of simple value types or fixed size buffers, albeit that you don't often have that choice. Never use a bool, use byte instead. There is no easy way to find out if a type is blittable, other than it not working correctly or by using the debugger and compare pointer values.

Just be explicit and always use [Out] when you need it. It doesn't cost anything if it turned out not to be necessary. And it is self-documenting. And you can feel good that it still will work if the architecture of the native code changes.

Marshal's [In] [Out] Attributes

From the documentation, with my emphasis:

As an optimization, arrays of blittable types and classes that contain only blittable members are pinned instead of copied during marshaling. These types can appear to be marshaled as In/Out parameters when the caller and callee are in the same apartment. However, these types are actually marshaled as In parameters, and you must apply the InAttribute and OutAttribute attributes if you want to marshal the argument as an In/Out parameter.

Since byte is blittable, your array of byte is pinned rather than copied, and so marshals as In/Out.

When I substitute byte[] with char[] the CLR will follow my In-Out attributes.

The C# type char is not blittable. An array of char is not pinned, and so [In] char[] marshals as an In parameter.

Marshal an array of structs in pInvoke

Your C++ code does not receive an array of structs. It receives an array of pointers to struct. Change the C++ code to be like so:

int SendArray(MyStruct* Arr, int recordsCount);

or perhaps

int SendArray(MyStruct Arr[], int recordsCount);

And then your p/invoke should be

[DllImport(...)]
public static extern int SendArray([In] MyStruct[] arr, int recordsCount);

I am also suspicious of Pack=8. Are you quite sure?

C# Interop Marshaling behaviour for arrays seems inconsistent with documentation

It is not mentioned on the documentation page you linked, but there is another topic dedicated to the marshaling of arrays, where it says:

With pinning optimization, a blittable array can appear to operate as an In/Out parameter when interacting with objects in the same apartment.

Both conditions are met in your case: array of uint is blittable, and there is no machine-to-machine marshaling. It is still a good idea to declare it [Out], so your intention is documented within the code.

P/Invoke bug or am I doing it wrong?

I believe I have figured out what is going on. In my opinion, it is a bug in the P/Invoke marshaler, but I wouldn't be surprised if Microsoft's official line were, "This behaviour is by design."

When you configure an array to be marshaled within a struct, your options are very constrained. From what I can tell, you can use UnmanagedType.SafeArray for any of the VARIANT primitive types, or you can use UnmanagedType.ByValArray, which requires you to specify SizeConst -- that is, the marshaler supports only embedded arrays that are always exactly the same length.

The Marshal.SizeOf function, then, when computing the size of the structure, counts the size of the array as SizeConst * Marshal.SizeOf(arrayElementType). This is always the case, regardless of the actual size of the array an instance points to.

The bug appears to be that the marshaler always copies all elements in the array, even if that number is greater than SizeConst. So, as in my situation, if you set SizeConst to 1, but supply an array with 6 elements, then it allocates memory based on Marshal.SizeOf, which allocates a single slot for array data, and then proceeds to marshal all 6 elements, writing past the end of the buffer and corrupting memory.

The reason the stack slots for parameters were corrupted in this manner can be explained only by the marshaler allocating memory for this serialization on the stack. By overflowing the end of that buffer, it then overwrote data further up the stack, including the spot where, ultimately, the return address gets written and the first two arguments that had already been placed into their slots on the stack. After this marshaling operation, it then wrote a pointer to that stack buffer into the third argument, explaining why the third argument's value was, in fact, a valid pointer to the data structure.

I was lucky, with my particular configuration, that the end of the 6th element occurred before corrupting other elements further up the stack, in that only the first two arguments to ChangeServiceConfig2W were being damaged -- code was able to continue to execute after ChangeServiceConfig2W returned its error about the handle being invalid. With a larger array, or a simpler function where the buffer allocated by the marshaler was closer to the end of the stack frame, it could very well have overwritten important data further up the stack and resulted in an ExecutionEngineException as @GSerg saw.

On 64-bit systems, there must be more free space on the stack -- for one thing, all pointers are now 64 bits wide, so that spaces things out. Writing past the end of the buffer, then, isn't getting as far up the stack and isn't managing to corrupt the first or second argument to ChangeServiceConfig2W. This is how this code worked in initial testing and appeared to be correct.

In my opinion, this is a bug in the marshaler; it has enough information to avoid corrupting memory (just don't marshal more than SizeConst elements, because that's all the memory you have allocated!), but it goes ahead and writes past the end of the allocated buffer anyway. I can see the converse philosophy, which is, "If you've told the marshaler SizeConst is 1, then don't supply an array with more than 1 element." But there is no clear warning that doing so can corrupt the execution environment in any of the documentation that I read. Given the lengths that .NET goes to to avoid that type of corruption, I have to think of this as a bug in the marshaler.

I have gotten my code working by updating the DataLock class, which temporarily prepares the lpsaActions data as a pointer-to-array (which ChangeServiceConfig2W requires and the default P/Invoke marshaler does not appear to support), to stash the real lpsaActions array and replace it with a dummy 1-element array. This prevents the marshaler from then marshaling more than 1 element, and the memory corruption does not occur. When the DataLock object is Disposed, it restores lpsaActions to its previous value.

I have placed this (working) code into a public GitHub repo, along with an essentially identical C++ version that I used to compare the state of registers and the stack as code flow entered the ChangeServiceConfig2W function during my diagnosis:

• https://github.com/logiclrd/TestServiceFailureActionChange

To reproduce the problem I was seeing, comment out these lines from WindowsAPI/SERVICE_FAILURE_ACTIONSW.cs:

            // Replace the lpsaActions array with a dummy that contains only one element, otherwise the P/Invoke marshaller
            // will allocate a buffer of size 1 and then write lpsaActions.Length items to it and corrupt memory.
            _originalActionsArray = data.lpsaActions;

            data.lpsaActions = new SC_ACTION[1];

Then, run the program as a 32-bit process. (If you are on a 64-bit operating system, which is likely, you'll need to adjust the build configuration so that it either specifies "Prefer 32-bit" or targets the "x86" platform directly.)

Microsoft may or may not become aware of and fix this bug. At the very least, it would be good of Microsoft to update the documentation for UnmanagedType.ByValArray to include a warning about this possible scenario -- I'm not sure how to convey that to them. Given that current releases of .NET have it, though, I think it is best to simply avoid supplying arrays whose length is not exactly equal to SizeConst when marshaling structures to unmanaged code. :-)

P/Invoke problem marshalling parameter

The Size attribute you gave in the [StructLayout] attribute is a good hint. Verify that with this snippet:

        int len = Marshal.SizeOf(typeof(Plate));
        System.Diagnostics.Debug.Assert(len == 138);

The only way you are going to get passed this assertion is when you replace "bool" with "byte" (So TBool = 1 byte) and use a packing of 1:

[StructLayout(LayoutKind.Sequential, Pack=1), Serializable]
public struct Plate {
  //...
}

If that still doesn't work then you'll really have to debug the unmanaged code.

---

Related Topics