Why Is Stack Size in C# Exactly 1 Mb

Why is stack size in C# exactly 1 MB?

You are looking at the guy that made that choice. David Cutler and his team selected one megabyte as the default stack size. Nothing to do with .NET or C#, this was nailed down when they created Windows NT. One megabyte is what it picks when the EXE header of a program or the CreateThread() winapi call doesn't specify the stack size explicitly. Which is the normal way, almost any programmer leaves it up the OS to pick the size.

That choice probably pre-dates the Windows NT design, history is way too murky about this. Would be nice if Cutler would write a book about it, but he's never been a writer. He's been extraordinarily influential on the way computers work. His first OS design was RSX-11M, a 16-bit operating system for DEC computers (Digital Equipment Corporation). It heavily influenced Gary Kildall's CP/M, the first decent OS for 8-bit microprocessors. Which heavily influenced MS-DOS.

His next design was VMS, an operating system for 32-bit processors with virtual memory support. Very successful. His next one was cancelled by DEC around the time the company started disintegrating, not being able to compete with cheap PC hardware. Cue Microsoft, they made him a offer he could not refuse. Many of his co-workers joined too. They worked on VMS v2, better known as Windows NT. DEC got upset about it, money changed hands to settle it. Whether VMS already picked one megabyte is something I don't know, I only know RSX-11 well enough. It isn't unlikely.

Enough history. One megabyte is a lot, a real thread rarely consumes more than a couple of handfuls of kilobytes. So a megabyte is actually rather wasteful. It is however the kind of waste you can afford on a demand-paged virtual memory operating system, that megabyte is just virtual memory. Just numbers to the processor, one each for every 4096 bytes. You never actually use the physical memory, the RAM in the machine, until you actually address it.

It is extra excessive in a .NET program because the one megabyte size was originally picked to accommodate native programs. Which tend to create large stack frames, storing strings and buffers (arrays) on the stack as well. Infamous for being a malware attack vector, a buffer overflow can manipulate the program with data. Not the way .NET programs work, strings and arrays are allocated on the GC heap and indexing is checked. The only way to allocate space on the stack with C# is with the unsafe stackalloc keyword.

The only non-trivial usage of the stack in .NET is by the jitter. It uses the stack of your thread to just-in-time compile MSIL to machine code. I've never seen or checked how much space it requires, it rather depends on the nature of the code and whether or not the optimizer is enabled, but a couple of tens of kilobytes is a rough guess. Which is otherwise how this website got its name, a stack overflow in a .NET program is quite fatal. There isn't enough space left (less than 3 kilobytes) to still reliably JIT any code that tries to catch the exception. Kaboom to desktop is the only option.

Last but not least, a .NET program does something pretty unproductive with the stack. The CLR will commit the stack of a thread. That's an expensive word that means that it doesn't just reserve the size of the stack, it also makes sure that space is reserved in the operating system's paging file so the stack can always be swapped out when necessary. Failing to commit is a fatal error and terminates a program unconditionally. That only happens on machine with very little RAM that runs entirely too many processes, such a machine will have turned to molasses before programs start dying. A possible problem 15+ years ago, not today. Programmers that tune their program to act like an F1 race-car use the <disableCommitThreadStack> element in their .config file.

Fwiw, Cutler didn't stop designing operating systems. That photo was made while he worked on Azure.

---

Update, I noticed that .NET no longer commits the stack. Not exactly sure when or why this happened, it's been too long since I checked. I'm guessing this design change happened somewhere around .NET 4.5. Pretty sensible change.

1 MB stack size limit enforced by the CLR is for a thread or the entire application/process?

1MB is just the default stack size per thread, not for the whole application. Each thread has its own stack. When you specify a different stack size in the thread constructor you are overriding the default for that thread.

If you want to test the limits of the stack size you would need to call a recursive function until the stack is filled up and overflows (hence this website's name). Simply creating more threads will just create more stacks.

Default Stack Size Varies between .NET 3.5 and 4.x?

In the meanwhile, I found the answer.

The stack size is not changes, it is 1 MB across all versions.

What got changed is obviously the way memory is committed. With .NET 4.x, seems only 100K of each stack is actually committed (i.e. consumed, more information can be found in the Tomes of Wisdom). The rest is just reserved in case it's actually needed later. The app still loses 1MB of its virtual memory and crashes at about 1450 threads created, regardless of the version of the Framework.

Thanks to Hans Passant for the tip:
https://stackoverflow.com/a/28658130/383426

Is there a way to increase the stack size in c#?

The big bad warning

If you use recursion in a program and reach a point where having a StackOverflowException is an actual threat, please do not consider increasing the stack size as a valid solution.

If you encounter a StackOverflowException you are doing something very wrong; you should instead be using a Stack<T> for depth-first processing, or a Queue<T> for breadth-first processing. Example.

---

The solution

This can be achieved by using editbin.exe, which is installed with this package;

Find the location of editbin.exe, mine was located at C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64\editbin.exe, I would suggest using Everything by voidtools in lieu of Microsoft's awful search to find this.

Set the stack size manually

Navigate to your bin folder and execute the following:

"<full path of editbin.exe>" /stack:<stack size in bytes, decimal> <your executable name>

For example I executed this:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64\EDITBIN.EXE" /stack:2097152 ExampleProgram.exe

Which set the stack reserve size to 2MB.

With this I was capable of reaching twice the recursion level; (1MB stack reserve on left, 2MB stack reserve on right).

Set the stack size automatically

Right click on your project and select 'Options', then click on 'Build Events' and add the following to your post-build events:

"<full path of editbin.exe>" /stack:<stack size in bytes, decimal> "$(TargetPath)"

For example I added

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64\EDITBIN.EXE" /stack:2097152 "$(TargetPath)"

This will run editbin.exe every time you build your executable.

Note: You will see a lot lower level of recursion reached when running your program from Visual Studio as you will from running it explicitly via explorer or cmd. You will still however see a 2x increase in the level of recursion met if moving from a 1MB stack reserve to a 2MB stack reserve.

How do I avoid changing the Stack Size AND avoid getting a Stack Overflow in C#

A while back I explored this problem in my blog. Or, rather, I explored a related problem: how do you find the depth of a binary tree without using recursion? A recursive tree depth solution is trivial, but blows the stack if the tree is highly imbalanced.

My recommendation is to study ways of solving this simpler problem, and then decide which of them, if any, could be adapted to your slightly more complex algorithm.

Note that in these articles the examples are given entirely in JScript. However, it should not be difficult to adapt them to C#.

Here we start by defining the problem.

http://blogs.msdn.com/ericlippert/archive/2005/07/27/recursion-part-one-recursive-data-structures-and-functions.aspx

The first attempt at a solution is the classic technique that you'll probably adopt: define an explicit stack; use it rather than relying upon the operating system and compiler implementing the stack for you. This is what most people do when faced with this problem.

http://blogs.msdn.com/ericlippert/archive/2005/08/01/recursion-part-two-unrolling-a-recursive-function-with-an-explicit-stack.aspx

The problem with that solution is that it's a bit of a mess. We can go even farther than simply making our own stack. We can make our own little domain-specific virtual machine that has its own heap-allocated stack, and then solve the problem by writing a program that targets that machine! This is actually easier than it sounds; the operations of the machine can be extremely high level.

http://blogs.msdn.com/ericlippert/archive/2005/08/04/recursion-part-three-building-a-dispatch-engine.aspx

And finally, if you are really a glutton for punishment (or a compiler developer) you can rewrite your program in Continuation Passing Style, thereby eliminating the need for a stack at all:

http://blogs.msdn.com/ericlippert/archive/2005/08/08/recursion-part-four-continuation-passing-style.aspx

http://blogs.msdn.com/ericlippert/archive/2005/08/11/recursion-part-five-more-on-cps.aspx

http://blogs.msdn.com/ericlippert/archive/2005/08/15/recursion-part-six-making-cps-work.aspx

CPS is a particularly clever way of moving the implicit stack data structure off the system stack and onto the heap by encoding it in the relationships between a bunch of delegates.

Here are all of my articles on recursion:

http://blogs.msdn.com/ericlippert/archive/tags/Recursion/default.aspx

What is the stack size of a BackgroundWorker DoWork Thread? Is there way to change it?

The stack size inside a BackgroundWorker DoWork event is the same as for the main thread.

Prof:

Set the stack size in a post build event to 8 MB for example:

"$(DevEnvDir)..\..\VC\bin\editbin.exe" /STACK:8388608 "$(TargetPath)"

Then ask for the stack size using the following code:

[DllImport("kernel32.dll")]
internal static extern void GetCurrentThreadStackLimits(out uint lowLimit, out uint highLimit);


public static uint GetStackSize()
{
    uint low, high;
    GetCurrentThreadStackLimits(out low, out high);
    return high - low;
}

Using GetStackSize in the main program and in the DoWork event return in both cases 8 MB or whatever you specified using EDITBIN /STACK.

What are the dangers when creating a thread with a stack size of 50x the default?

Upon comparing test code with Sam, I determined that we are both right!

However, about different things:

• Accessing memory (reading and writing) is just as fast wherever it is - stack, global or heap.
• Allocating it, however, is fastest on stack and slowest on heap.

It goes like this: stack < global < heap. (allocation time)

Technically, stack allocation isn't really an allocation, the runtime just makes sure a part of the stack (frame?) is reserved for the array.

I strongly advise being careful with this, though.

I recommend the following:

1. When you need to create arrays frequently which never leave the function (e.g. by passing its reference), using the stack will be an enormous improvement.
2. If you can recycle an array, do so whenever you can! The heap is the best place for long-term object storage. (polluting global memory isn't nice; stack frames can disappear)

(Note: 1. only applies to value types; reference types will be allocated on the heap and the benefit will be reduced to 0)

To answer the question itself: I have not encountered any problem at all with any large-stack test.

I believe the only possible problems are a stack overflow, if you are not careful with your function calls and running out of memory when creating your thread(s) if the system is running low.

The section below is my initial answer. It is wrong-ish and the tests aren't correct. It is kept only for reference.

---

My test indicates the stack-allocated memory and global memory is at least 15% slower than (takes 120% the time of) heap-allocated memory for usage in arrays!

This is my test code, and this is a sample output:

Stack-allocated array time: 00:00:00.2224429
Globally-allocated array time: 00:00:00.2206767
Heap-allocated array time: 00:00:00.1842670
------------------------------------------
Fastest: Heap.

  |    S    |    G    |    H    |
--+---------+---------+---------+
S |    -    | 100.80 %| 120.72 %|
--+---------+---------+---------+
G |  99.21 %|    -    | 119.76 %|
--+---------+---------+---------+
H |  82.84 %|  83.50 %|    -    |
--+---------+---------+---------+
Rates are calculated by dividing the row's value to the column's.

I tested on Windows 8.1 Pro (with Update 1), using an i7 4700 MQ, under .NET 4.5.1

I tested both with x86 and x64 and the results are identical.

Edit: I increased the stack size of all threads 201 MB, the sample size to 50 million and decreased iterations to 5.

The results are the same as above:

Stack-allocated array time: 00:00:00.4504903
Globally-allocated array time: 00:00:00.4020328
Heap-allocated array time: 00:00:00.3439016
------------------------------------------
Fastest: Heap.

  |    S    |    G    |    H    |
--+---------+---------+---------+
S |    -    | 112.05 %| 130.99 %|
--+---------+---------+---------+
G |  89.24 %|    -    | 116.90 %|
--+---------+---------+---------+
H |  76.34 %|  85.54 %|    -    |
--+---------+---------+---------+
Rates are calculated by dividing the row's value to the column's.

Though, it seems the stack is actually getting slower.

Increase stack size of main program or create a new thread with larger stack size for recursive code blocks?

• Which solution is the better one?
• Which solution is the better one?

The first approach using editbin in a post-build event is failing when using strong name key to sign the assembly. After the assembly was changed using editbin the verification of the signed assembly will fail. sn.exe -v assembly.exe will return Failed to verify assembly -- Strong name validation failed ...

See as well:

• Is an assembly signed with a strong name before or after the post-build event?

Using the AfterCompile event and resigning the assembly is a workaround (which I'm using now). The project file should contain the following lines:

  <Target Name="AfterCompile">
    <Exec Command="
"$(DevEnvDir)..\..\VC\bin\editbin.exe" /STACK:16777216 "$(ProjectDir)obj\$(ConfigurationName)\$(TargetFileName)"
"$(FrameworkSDKDir)bin\NETFX 4.5.1 Tools\sn.exe" -Ra "$(ProjectDir)obj\$(ConfigurationName)\$(TargetFileName)" "$(SolutionDir)\STRONGNAME.snk"
" />
  </Target>
  <PropertyGroup>
    <PostBuildEvent>REM "See AfterCompile for stack size and resigning"</PostBuildEvent>
  </PropertyGroup>

I was getting aware of the after compile event when I was reading the following answer: https://stackoverflow.com/a/22617361/7556646

The second approach but for the hole program and not only for the recursive code blocks would look like that:

static class Program
{
    [STAThread]
    static void Main(string[] args)
    {
        Thread thread = new Thread(delegate()
        {
            Main2(args);
        }, 16 * 1024 * 1024);
        thread.SetApartmentState(ApartmentState.STA);
        thread.Start();
        thread.Join();
    }

    static void Main2(string[] args)
    {
        Application.EnableVisualStyles();
        Application.SetCompatibleTextRenderingDefault(false);
        Application.Run(app);
    }
}

The second approach has the drawback that the stack size of a BackgroundWorker DoWork event is still 1 MB (32-bit or any) or 4 MB (64-bit).

See as well:

• What is the stack size of a BackgroundWorker DoWork Thread? Is there way to change it?

• What is the draw back of creating a new thread with a larger stack size for this calculation?
• What would be a reasonable stack size?

See the comments from Hans Passant.

---

Related Topics