Why am I Getting an Out of Memory Exception in My C# Application

Why am I getting an Out Of Memory Exception in my C# application?

In a normal 32 bit windows app, the process only has 2GB of addressable memory. This is irrelevant to the amount of physical memory that is available.

So 2GB available but 1.5 is the max you can allocate. The key is that your code is not the only code running in the process. The other .5 GB is probably the CLR plus fragmentation in the process.

Update: in .Net 4.5 in 64 bit process you can have large arrays if gcAllowVeryLargeObjects setting is enabled:

On 64-bit platforms, enables arrays that are greater than 2 gigabytes (GB) in total size.
The maximum number of elements in an array is UInt32.MaxValue.

<configuration>
  <runtime>
    <gcAllowVeryLargeObjects enabled="true" />
  </runtime>
</configuration>

C# : Out of Memory exception

Two points:

1. If you are running a 32 bit Windows, you won't have all the 4GB accessible, only 2GB.
2. Don't forget that the underlying implementation of List is an array. If your memory is heavily fragmented, there may not be enough contiguous space to allocate your List, even though in total you have plenty of free memory.

System.OutOfMemoryException' was thrown when there is still plenty of memory free

You may want to read this: "“Out Of Memory” Does Not Refer to Physical Memory" by Eric Lippert.

In short, and very simplified, "Out of memory" does not really mean that the amount of available memory is too small. The most common reason is that within the current address space, there is no contiguous portion of memory that is large enough to serve the wanted allocation. If you have 100 blocks, each 4 MB large, that is not going to help you when you need one 5 MB block.

Key Points:

• the data storage that we call “process memory” is in my opinion best visualized as a massive file on disk.
• RAM can be seen as merely a performance optimization
• Total amount of virtual memory your program consumes is really not hugely relevant to its performance
• "running out of RAM" seldom results in an “out of memory” error. Instead of an error, it results in bad performance because the full cost of the fact that storage is actually on disk suddenly becomes relevant.

out-of-memory exception even though enough memory seems to be available

Using the tool vmmap I found the reason for the problem: The actual memory, which is available for the managed heap, is much less than the 2GB limit. The are several shared libraries loaded for interaction with MS Office tools (~400 MB). There are also native code dlls (~300MB) which also allocate unmanaged heap (~300MB). There is also lot's of other stuff and, in the end, only around 700MB remain for the managed heap.

Since the is much less memory available than I originally thought, the LOH fragmentation may have more impact than I suspected and indeed: vmmap shows, that the largest free block in that memory area becomes smaller over timer, even though the available memory remains the same. I think, this proofs the fragmentation is the cause of the problem. The trigger of the exception is often the binary serialization which we use sometimes for deep copying objects. It seems to cause a peak in memory usage.

So what to do about it? I am considering the following options:

• Switch to x64 (which will happen in the long run anyway)
• Switch to .NET 4.5.1 which allows to defragment the LOH
• Reduce general memory usage: It looks like the defragmentation takes much longer if around 200MB is additionally available. This happens when some big libraries are not loaded. In my experiments I could not trigger an out-of-memory-exception anymore.
• Change the code, which will probably be too time consuming

Best way to avoid out of memory exception in application

Well, according to the topic of the question, best way to avoid out of memory exception would be not to create objects that fill in that memory.

Handling exception is easiest solution, though may bring different difficulties and inconsistencies into the application with time. Another way would be getting available size of memory resources like this:

Process currentProcess = Process.GetCurrentProcess();
long memorySize = currentProcess.PrivateMemorySize64

Then you can calculate the length of your queue based on estimate of one object memory capacity.

Another way would be to check for memory size in each worker thread. Whenever there's no memory, the thread can just finish. This way many threads would be spawning and dying but the application would be at maximum available capacity.

.NET Out Of Memory Exception - Used 1.3GB but have 16GB installed

There is no difference until you compile to same target architecture. I suppose you are compiling for 32 bit architecture in both cases.

It's worth mentioning that OutOfMemoryException can also be raised if you get 2GB of memory allocated by a single collection in CLR (say List<T>) on both architectures 32 and 64 bit.

To be able to benefit from memory goodness on 64 bit architecture, you have to compile your code targeting 64 bit architecture. After that, naturally, your binary will run only on 64 bit, but will benefit from possibility having more space available in RAM.

Out of memory exception even for a very small task

I tried and I can reproduce your problem. It is definitely out of memory and nothing like "It just seems to be" the memory usage increases to about 4 GB and then the error shows up. Console output is just to see what's happening there.

The Image object seems to be not the best way to save the data.

I tried this and got this to work with many many files. Maybe you can change the code to fit your needs:

            String[] allfiles = Directory.GetFiles(imagePath, "*.*", SearchOption.AllDirectories);
            //List<Image> allImages = new List<Image>();
            List<Byte[]> allImagesBytes = new List<Byte[]>();

            foreach (string file in allfiles)
            {
                using (FileStream stream = new FileStream(file, FileMode.Open, FileAccess.Read))
                {
                    if (IsImage(stream))
                    {
                        Console.Clear();
                        Console.Write(allImagesBytes.Count());
                        //allImages.Add(Image.FromStream(stream));
                        //allImages.Add(Image.FromFile(file));
                        allImagesBytes.Add(File.ReadAllBytes(file));

                    }
                }
            }

---

Related Topics