what is the max limit of data into liststring in c#?
The maximum number of elements that can be stored in the current implementation of List<T>
is, theoretically, Int32.MaxValue
- just over 2 billion.
In the current Microsoft implementation of the CLR there's a 2GB maximum object size limit. (It's possible that other implementations, for example Mono, don't have this restriction.)
Your particular list contains strings, which are reference types. The size of a reference will be 4 or 8 bytes, depending on whether you're running on a 32-bit or 64-bit system. This means that the practical limit to the number of strings you could store will be roughly 536 million on 32-bit or 268 million on 64-bit.
In practice, you'll most likely run out of allocable memory before you reach those limits, especially if you're running on a 32-bit system.
What's the max items in a ListT?
List<T>
will be limited to the max of an array, which is 2GB (even in x64). If that isn't enough, you're using the wrong type of data storage. You can save a lot of overhead by starting it the right size, though - by passing an int
to the constructor.
Re your edit - with 134217728 x Int32, that is 512MB. Remember that List<T>
uses a doubling algorithm; if you are drip-feeding items via Add
(without allocating all the space first) it is going to try to double to 1GB (on top of the 512MB you're already holding, the rest of your app, and of course the CLR runtime and libraries). I'm assuming you're on x86, so you already have a 2GB limit per process, and it is likely that you have fragmented your "large object heap" to death while adding items.
Personally, yes, it sounds about right to start getting an out-of-memory at this point.
Edit: in .NET 4.5, arrays larger than 2GB are allowed if the <gcAllowVeryLargeObjects>
switch is enabled. The limit then is 2^31 items. This might be useful for arrays of references (8 bytes each in x64), or an array of large struct
s.
List size limitation in C#
A List<int>
is backed by an int[]
. You will fail as soon as a larger backing array cannot be allocated - and bear in mind that:
- There's a 2GB per-object limit in the CLR even in 64 bits (EDIT: as of .NET 4.5, this can be avoided for the 64-bit CLR - see
<gcAllowVeryLargeObjects>
) - The list will try to allocate a backing array which is larger than what it immediately requires, in order to accommodate later
Add
requests without reallocation. - During the reallocation, there has to be enough total memory for both the old and the new arrays.
Setting the Capacity
to a value which will put the backing array near the theoretical limit may get you a higher cutoff point than the natural growth, but that limit will certainly come.
I would expect a limit of around 229 elements (536,870,912) - I'm slightly surprised you haven't managed to get beyond 134,217,728. How much memory do you actually have? What version of .NET are you using, and on what architecture? (It's possible that the per-object limit is 1GB for a 32-bit CLR, I can't remember for sure.)
Note that even if the per-object limit wasn't a problem, as soon as you got above 231 elements you'd have problems addressing those elements directly with List<T>
, as the indexer takes an int
value.
Basically, if you want a collection with more than int.MaxValue
elements, you'll need to write your own, probably using multiple backing arrays. You might want to explicitly prohibit removals and arbitrary insertions :)
I hit an OutOfMemoryException with Liststring - is this the limit or am I missing something?
If you're trying to use very large lists in 64 bit environments you need to enable large objects in the application configuration.
http://msdn.microsoft.com/en-us/library/hh285054.aspx
The OOM is likely due to the way Lists/ArrayLists allocate memory, which I believe is each time their boundary is reached, they attempt to double in size. The list cannot double from 2^24. You could theoretically maximize your list size by pre-specifying a size. (I.e. 2GB)
How to get max length record from ListString?
If you want to find all longest strings:
IEnumerable<string> longestStrings = list
.GroupBy(str => str == null ? 0 : str.Length)
.OrderByDescending(g => g.Key) // order by length
.First();
Another approach which might be cheaper in terms of memory consumption:
int maxLength = list.Max(str=> str == null ? 0 : str.Length);
longestStrings = list
.Where(str => (str == null ? 0 : str.Length) == maxLength);
Is there a limit of elements that could be stored in a List?
The current implementation of List<T>
uses Int32
everywhere - to construct its backing array, for its Count
property, as an indexer and for all its internal operations - so there's a current theoretical maximum of Int32.MaxValue
items (2^31-1
or 2147483647
).
But the .NET framework also has a maximum object size limit of 2GB, so you'll only get anywhere near the items limit with lists of single-byte items such as List<byte>
or List<bool>
.
In practice you'll probably run out of contiguous memory before you hit either of those limits.
Liststring[] determine max length by Linq
One approach:
int maxColumns = sList.Max(arr => arr.Length);
List<int> Result = Enumerable.Range(0, maxColumns)
.Select(i => sList.Max(arr => (arr.ElementAtOrDefault(i) ?? "").Length))
.ToList();
You want the max-length per column. The columns are the array indices. So you need a way to look at the arrays per index. Therefore i've used Enumerable.Range(0, maxColumns)
. Then i'm using ElementAtOrDefault
to handle the case that the array doesn't contain so many "columns"(not needed as i'm explaining below). That returns null
for reference types like string
. I replace them with the null-coalescing operator with ""
which yields 0 as length.
Since you've mentioned that "each row has the same number of columns" you can make it a little bit more readable:
List<int> Result = Enumerable.Range(0, sList.First().Length)
.Select(i => sList.Max(arr => arr[i].Length))
.ToList();
Calculating a safe maximum size for a List
You should be able to get list of items that is backed by byte array of almost 2Gb (if you try to grow the list dynamically it will obviously fail close to half of max size due to reallocation stategy).
If you data is not immutable you may instead consider GC friendlier approach that does not create many objects on LOH. Try targeting all allocations to be under limit of LOH allocation (I think 85Kb) and measure if code behaves better.
How many records a List object can hold at a time in c#
There is a limit of 2GB for storing objects in memory. You also have a limitation over the amount of items that a collection can handle as other already pointed out.
In any case, I'm pretty sure you don't have to test these limit getting million of records from database unless there is no other option, what is not the case for 99.999% of the cases.
Related Topics
How to Read a Text File Without Locking It
How to Get a Variable's Name as It Was Physically Typed in Its Declaration
Insert Data Using Entity Framework Model
How to Detect When a Windows Form Is Being Minimized
Calling a Soap Service in .Net Core
How to Get the Word Under the Cursor in Windows
How to Do Network Discovery Using Udp Broadcast
How to Specify My Explicit Type Comparator Inline
What Is the Fastest Way I Can Compare Two Equal-Size Bitmaps to Determine Whether They Are Identical
Wpf - How to Create Menu and Submenus Using Binding
Virtual Webcam Input as Byte Stream
Directory.Getfiles of Certain Extension
What's the Best Strategy for Equals and Gethashcode
Why Are C# Interface Methods Not Declared Abstract or Virtual