Stack Capacity in C#

What is the default capacity of Stack object?

If you look at the current source code it says 10 also.

http://referencesource.microsoft.com/#mscorlib/system/collections/stack.cs,7c1d0a7ea96800a3,references

Why is stack size in C# exactly 1 MB?

Sample Image

You are looking at the guy that made that choice. David Cutler and his team selected one megabyte as the default stack size. Nothing to do with .NET or C#, this was nailed down when they created Windows NT. One megabyte is what it picks when the EXE header of a program or the CreateThread() winapi call doesn't specify the stack size explicitly. Which is the normal way, almost any programmer leaves it up the OS to pick the size.

That choice probably pre-dates the Windows NT design, history is way too murky about this. Would be nice if Cutler would write a book about it, but he's never been a writer. He's been extraordinarily influential on the way computers work. His first OS design was RSX-11M, a 16-bit operating system for DEC computers (Digital Equipment Corporation). It heavily influenced Gary Kildall's CP/M, the first decent OS for 8-bit microprocessors. Which heavily influenced MS-DOS.

His next design was VMS, an operating system for 32-bit processors with virtual memory support. Very successful. His next one was cancelled by DEC around the time the company started disintegrating, not being able to compete with cheap PC hardware. Cue Microsoft, they made him a offer he could not refuse. Many of his co-workers joined too. They worked on VMS v2, better known as Windows NT. DEC got upset about it, money changed hands to settle it. Whether VMS already picked one megabyte is something I don't know, I only know RSX-11 well enough. It isn't unlikely.

Enough history. One megabyte is a lot, a real thread rarely consumes more than a couple of handfuls of kilobytes. So a megabyte is actually rather wasteful. It is however the kind of waste you can afford on a demand-paged virtual memory operating system, that megabyte is just virtual memory. Just numbers to the processor, one each for every 4096 bytes. You never actually use the physical memory, the RAM in the machine, until you actually address it.

It is extra excessive in a .NET program because the one megabyte size was originally picked to accommodate native programs. Which tend to create large stack frames, storing strings and buffers (arrays) on the stack as well. Infamous for being a malware attack vector, a buffer overflow can manipulate the program with data. Not the way .NET programs work, strings and arrays are allocated on the GC heap and indexing is checked. The only way to allocate space on the stack with C# is with the unsafe stackalloc keyword.

The only non-trivial usage of the stack in .NET is by the jitter. It uses the stack of your thread to just-in-time compile MSIL to machine code. I've never seen or checked how much space it requires, it rather depends on the nature of the code and whether or not the optimizer is enabled, but a couple of tens of kilobytes is a rough guess. Which is otherwise how this website got its name, a stack overflow in a .NET program is quite fatal. There isn't enough space left (less than 3 kilobytes) to still reliably JIT any code that tries to catch the exception. Kaboom to desktop is the only option.

Last but not least, a .NET program does something pretty unproductive with the stack. The CLR will commit the stack of a thread. That's an expensive word that means that it doesn't just reserve the size of the stack, it also makes sure that space is reserved in the operating system's paging file so the stack can always be swapped out when necessary. Failing to commit is a fatal error and terminates a program unconditionally. That only happens on machine with very little RAM that runs entirely too many processes, such a machine will have turned to molasses before programs start dying. A possible problem 15+ years ago, not today. Programmers that tune their program to act like an F1 race-car use the <disableCommitThreadStack> element in their .config file.

Fwiw, Cutler didn't stop designing operating systems. That photo was made while he worked on Azure.

Update, I noticed that .NET no longer commits the stack. Not exactly sure when or why this happened, it's been too long since I checked. I'm guessing this design change happened somewhere around .NET 4.5. Pretty sensible change.

Is there a way to increase the stack size in c#?

The big bad warning

If you use recursion in a program and reach a point where having a StackOverflowException is an actual threat, please do not consider increasing the stack size as a valid solution.

If you encounter a StackOverflowException you are doing something very wrong; you should instead be using a Stack<T> for depth-first processing, or a Queue<T> for breadth-first processing. Example.

The solution

This can be achieved by using editbin.exe, which is installed with this package;
VC++ tools

Find the location of editbin.exe, mine was located at C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64\editbin.exe, I would suggest using Everything by voidtools in lieu of Microsoft's awful search to find this.

Set the stack size manually

Navigate to your bin folder and execute the following:

"<full path of editbin.exe>" /stack:<stack size in bytes, decimal> <your executable name>

For example I executed this:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64\EDITBIN.EXE" /stack:2097152 ExampleProgram.exe

Which set the stack reserve size to 2MB.

With this I was capable of reaching twice the recursion level; (1MB stack reserve on left, 2MB stack reserve on right).

Recursion level

Set the stack size automatically

Right click on your project and select 'Options', then click on 'Build Events' and add the following to your post-build events:

"<full path of editbin.exe>" /stack:<stack size in bytes, decimal> "$(TargetPath)"

For example I added

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64\EDITBIN.EXE" /stack:2097152 "$(TargetPath)"

This will run editbin.exe every time you build your executable.

Note: You will see a lot lower level of recursion reached when running your program from Visual Studio as you will from running it explicitly via explorer or cmd. You will still however see a 2x increase in the level of recursion met if moving from a 1MB stack reserve to a 2MB stack reserve.

Capacity Property in C# Arraylist

Because you added 3 more elements to the list...

Capacity is the size of the internal array... It starts at 0 then defaults to 4 on first single add. It then goes up by a factor of 2 when it needs more capacity.. This keeps the memory copying at a minimum (at the cost of a small bit of memory).

Note : Don't confuse Capacity with Count though, they are different things

You add 3 elements to your array

ArrayList listInt = new ArrayList(); // capacity = 0 
listInt.Add(9);                      // capacity = 4
listInt.Add(10);                     // capacity = 4
listInt.Add(11);                     // capacity = 4

So when you add your first element to the list, internally Add calls EnsureCapacity. Which looks like this (Full Source Here)

private const int _defaultCapacity = 4;

...
private void EnsureCapacity(int min)
{
   if (_items.Length < min)
   {
      int newCapacity = _items.Length == 0 ? _defaultCapacity : _items.Length * 2;
      ...
   }
}

Take note of _items.Length * 2 ...

listInt.AddRange(list2);             // capacity = 8

You may only have had 3 elements in your list, However the capacity was 4. Now you add 3 more elements, its got no other choice then multiply the capacity by 2

List.Capacity Property

Gets or sets the total number of elements the internal data structure
can hold without resizing.

List.Count Property

Gets the number of elements contained in the List.

Why don't Stack T and Queue T have Capacity property while List T does?

I think that the reason that List has a Capacity property and Stack and Queue do not is that the normal usage of those types is different.

For a List it is fairly common to populate it with a large set of values, even some time after it has been created. Providing the Capacity property (and constructor argument) helps to mitigate the number of reallocations that would be done when adding a large number of items to the list.

Stack and Queue on the other hand do not tend to have large numbers of items added to them at once after they've been created.

Presumably Microsoft decided that it wasn't worth adding the Capacity property because it wouldn't be used very much.

However, do note that Queue does have a constructor that allows you to specify an initial capacity, and so does Stack.

Also note that both classes also have a TrimExcess() method, as mentioned by @drch below.

So Microsoft thought it would be useful at construction time, but not useful later on - so they only added the capacity functionality to the constructors.

(Incidentally I've just had a quick check through our code base, and it seems that the only time we use a capacity for List is in fact at construction time. So maybe if Microsoft were designing List now, they might also omit the Capacity property for List...)

Call Stack limitation in C#

Each thread has a stack size. The predefined stack size for the main thread of a program is fixed in the exe file. Each recursive call you make, you consume a little of this stack. When you finish it, the CLR throws a StackOverflowException. For Console/Graphical programs the default stack size should be 1mb of memory. You can't make this memory "bigger" from inside the program (you can use editbin.exe to change it from "outside" the program). This memory isn't dynamic. It is fixed (technically, the address space reserved for this memory is fixed, the memory is really allocated by the Windows OS on demand, probably 4kb at a time, but always up to the reserved address space). You can create secondary threads with the stack size you want.

Note that the handling of the stack in this way is a limitation of x86/x64 architecture, http://en.wikipedia.org/wiki/Stack-based_memory_allocation:

Some processors families, such as the x86, have special instructions for manipulating the stack of the currently executing thread. Other processor families, including PowerPC and MIPS, do not have explicit stack support, but instead rely on convention and delegate stack management to the operating system's application binary interface (ABI).

Increase stack size of main program or create a new thread with larger stack size for recursive code blocks?

Which solution is the better one?

Which solution is the better one?

The first approach using editbin in a post-build event is failing when using strong name key to sign the assembly. After the assembly was changed using editbin the verification of the signed assembly will fail. sn.exe -v assembly.exe will return Failed to verify assembly -- Strong name validation failed ...

See as well:

Is an assembly signed with a strong name before or after the post-build event?

Using the AfterCompile event and resigning the assembly is a workaround (which I'm using now). The project file should contain the following lines:

  <Target Name="AfterCompile">
    <Exec Command="
"$(DevEnvDir)..\..\VC\bin\editbin.exe" /STACK:16777216 "$(ProjectDir)obj\$(ConfigurationName)\$(TargetFileName)"
"$(FrameworkSDKDir)bin\NETFX 4.5.1 Tools\sn.exe" -Ra "$(ProjectDir)obj\$(ConfigurationName)\$(TargetFileName)" "$(SolutionDir)\STRONGNAME.snk"
" />
  </Target>
  <PropertyGroup>
    <PostBuildEvent>REM "See AfterCompile for stack size and resigning"</PostBuildEvent>
  </PropertyGroup>

I was getting aware of the after compile event when I was reading the following answer: https://stackoverflow.com/a/22617361/7556646

The second approach but for the hole program and not only for the recursive code blocks would look like that:

static class Program
{
    [STAThread]
    static void Main(string[] args)
    {
        Thread thread = new Thread(delegate()
        {
            Main2(args);
        }, 16 * 1024 * 1024);
        thread.SetApartmentState(ApartmentState.STA);
        thread.Start();
        thread.Join();
    }

    static void Main2(string[] args)
    {
        Application.EnableVisualStyles();
        Application.SetCompatibleTextRenderingDefault(false);
        Application.Run(app);
    }
}

The second approach has the drawback that the stack size of a BackgroundWorker DoWork event is still 1 MB (32-bit or any) or 4 MB (64-bit).