What Is the Max Limit of Data into List<String> in C#

what is the max limit of data into liststring in c#?

The maximum number of elements that can be stored in the current implementation of List<T> is, theoretically, Int32.MaxValue - just over 2 billion.

In the current Microsoft implementation of the CLR there's a 2GB maximum object size limit. (It's possible that other implementations, for example Mono, don't have this restriction.)

Your particular list contains strings, which are reference types. The size of a reference will be 4 or 8 bytes, depending on whether you're running on a 32-bit or 64-bit system. This means that the practical limit to the number of strings you could store will be roughly 536 million on 32-bit or 268 million on 64-bit.

In practice, you'll most likely run out of allocable memory before you reach those limits, especially if you're running on a 32-bit system.

What's the max items in a ListT?

List<T> will be limited to the max of an array, which is 2GB (even in x64). If that isn't enough, you're using the wrong type of data storage. You can save a lot of overhead by starting it the right size, though - by passing an int to the constructor.

Re your edit - with 134217728 x Int32, that is 512MB. Remember that List<T> uses a doubling algorithm; if you are drip-feeding items via Add (without allocating all the space first) it is going to try to double to 1GB (on top of the 512MB you're already holding, the rest of your app, and of course the CLR runtime and libraries). I'm assuming you're on x86, so you already have a 2GB limit per process, and it is likely that you have fragmented your "large object heap" to death while adding items.

Personally, yes, it sounds about right to start getting an out-of-memory at this point.


Edit: in .NET 4.5, arrays larger than 2GB are allowed if the <gcAllowVeryLargeObjects> switch is enabled. The limit then is 2^31 items. This might be useful for arrays of references (8 bytes each in x64), or an array of large structs.

List size limitation in C#

A List<int> is backed by an int[]. You will fail as soon as a larger backing array cannot be allocated - and bear in mind that:

  • There's a 2GB per-object limit in the CLR even in 64 bits (EDIT: as of .NET 4.5, this can be avoided for the 64-bit CLR - see <gcAllowVeryLargeObjects>)
  • The list will try to allocate a backing array which is larger than what it immediately requires, in order to accommodate later Add requests without reallocation.
  • During the reallocation, there has to be enough total memory for both the old and the new arrays.

Setting the Capacity to a value which will put the backing array near the theoretical limit may get you a higher cutoff point than the natural growth, but that limit will certainly come.

I would expect a limit of around 229 elements (536,870,912) - I'm slightly surprised you haven't managed to get beyond 134,217,728. How much memory do you actually have? What version of .NET are you using, and on what architecture? (It's possible that the per-object limit is 1GB for a 32-bit CLR, I can't remember for sure.)

Note that even if the per-object limit wasn't a problem, as soon as you got above 231 elements you'd have problems addressing those elements directly with List<T>, as the indexer takes an int value.

Basically, if you want a collection with more than int.MaxValue elements, you'll need to write your own, probably using multiple backing arrays. You might want to explicitly prohibit removals and arbitrary insertions :)

I hit an OutOfMemoryException with Liststring - is this the limit or am I missing something?

If you're trying to use very large lists in 64 bit environments you need to enable large objects in the application configuration.

http://msdn.microsoft.com/en-us/library/hh285054.aspx

The OOM is likely due to the way Lists/ArrayLists allocate memory, which I believe is each time their boundary is reached, they attempt to double in size. The list cannot double from 2^24. You could theoretically maximize your list size by pre-specifying a size. (I.e. 2GB)

How to get max length record from ListString?

If you want to find all longest strings:

IEnumerable<string> longestStrings = list
.GroupBy(str => str == null ? 0 : str.Length)
.OrderByDescending(g => g.Key) // order by length
.First();

Another approach which might be cheaper in terms of memory consumption:

int maxLength = list.Max(str=> str == null ? 0 : str.Length); 
longestStrings = list
.Where(str => (str == null ? 0 : str.Length) == maxLength);

Is there a limit of elements that could be stored in a List?

The current implementation of List<T> uses Int32 everywhere - to construct its backing array, for its Count property, as an indexer and for all its internal operations - so there's a current theoretical maximum of Int32.MaxValue items (2^31-1 or 2147483647).

But the .NET framework also has a maximum object size limit of 2GB, so you'll only get anywhere near the items limit with lists of single-byte items such as List<byte> or List<bool>.

In practice you'll probably run out of contiguous memory before you hit either of those limits.

Liststring[] determine max length by Linq

One approach:

int maxColumns = sList.Max(arr => arr.Length);
List<int> Result = Enumerable.Range(0, maxColumns)
.Select(i => sList.Max(arr => (arr.ElementAtOrDefault(i) ?? "").Length))
.ToList();

You want the max-length per column. The columns are the array indices. So you need a way to look at the arrays per index. Therefore i've used Enumerable.Range(0, maxColumns). Then i'm using ElementAtOrDefault to handle the case that the array doesn't contain so many "columns"(not needed as i'm explaining below). That returns null for reference types like string. I replace them with the null-coalescing operator with "" which yields 0 as length.

Since you've mentioned that "each row has the same number of columns" you can make it a little bit more readable:

List<int> Result = Enumerable.Range(0, sList.First().Length)
.Select(i => sList.Max(arr => arr[i].Length))
.ToList();

Calculating a safe maximum size for a List

You should be able to get list of items that is backed by byte array of almost 2Gb (if you try to grow the list dynamically it will obviously fail close to half of max size due to reallocation stategy).

If you data is not immutable you may instead consider GC friendlier approach that does not create many objects on LOH. Try targeting all allocations to be under limit of LOH allocation (I think 85Kb) and measure if code behaves better.

How many records a List object can hold at a time in c#

There is a limit of 2GB for storing objects in memory. You also have a limitation over the amount of items that a collection can handle as other already pointed out.

In any case, I'm pretty sure you don't have to test these limit getting million of records from database unless there is no other option, what is not the case for 99.999% of the cases.



Related Topics



Leave a reply



Submit