Most Efficient Way to Append Arrays in C#

Most efficient way to append arrays in C#?

You can't append to an actual array - the size of an array is fixed at creation time. Instead, use a List<T> which can grow as it needs to.

Alternatively, keep a list of arrays, and concatenate them all only when you've grabbed everything.

See Eric Lippert's blog post on arrays for more detail and insight than I could realistically provide :)

Adding values to a C# array

You can do this way -

int[] terms = new int[400];
for (int runs = 0; runs < 400; runs++)
{
    terms[runs] = value;
}

Alternatively, you can use Lists - the advantage with lists being, you don't need to know the array size when instantiating the list.

List<int> termsList = new List<int>();
for (int runs = 0; runs < 400; runs++)
{
    termsList.Add(value);
}

// You can convert it back to an array if you would like to
int[] terms = termsList.ToArray();

Edit: a) for loops on List<T> are a bit more than 2 times cheaper than foreach loops on List<T>, b) Looping on array is around 2 times cheaper than looping on List<T>, c) looping on array using for is 5 times cheaper than looping on List<T> using foreach (which most of us do).

Best way to push into C# array

array.push is like List<T>.Add. .NET arrays are fixed-size so you can't actually add a new element. All you can do is create a new array that is one element larger than the original and then set that last element, e.g.

Array.Resize(ref myArray, myArray.Length + 1);
myArray[myArray.GetUpperBound(0)] = newValue;

EDIT:

I'm not sure that this answer actually applies given this edit to the question:

The crux of the matter is that the element needs to be added into the
first empty slot in an array, lie a Java push function would do.

The code I provided effectively appends an element. If the aim is to set the first empty element then you could do this:

int index = Array.IndexOf(myArray, null);

if (index != -1)
{
    myArray[index] = newValue;
}

EDIT:

Here's an extension method that encapsulates that logic and returns the index at which the value was placed, or -1 if there was no empty element. Note that this method will work for value types too, treating an element with the default value for that type as empty.

public static class ArrayExtensions
{
    public static int Push<T>(this T[] source, T value)
    {
        var index = Array.IndexOf(source, default(T));

        if (index != -1)
        {
            source[index] = value;
        }

        return index;
    }
}

How do I concatenate two arrays in C#?

var z = new int[x.Length + y.Length];
x.CopyTo(z, 0);
y.CopyTo(z, x.Length);

Fastest way to extend array

Create a new array which is the size you want, then use the static Array.Copy method to copy the original arrays into the new one.

You can't "extend" an array, you can only create a bigger one and copy the original into it.

Also, consider using List<int> or LinkedList<> instead of an array, unless you require extremely fine-grained control over what is in memory.

How to append an element to a C# array for JSON reasons?

You can't resize fixed size array. The array is always has a fixed size

The best solution here is to use List<Channel> instead of Channel[].

Otherwise you can create a new array with the size of the previous plus one and set adding value by the last array index, but my opinion that it would be dirty.

Here are msdn docs about arrays: Array
Here is what the say

Unlike the classes in the System.Collections namespaces, Array has a fixed capacity. To increase the capacity, you must create a new Array object with the required capacity, copy the elements from the old Array object to the new one, and delete the old Array.

Best way to combine two or more byte arrays in C#

For primitive types (including bytes), use System.Buffer.BlockCopy instead of System.Array.Copy. It's faster.

I timed each of the suggested methods in a loop executed 1 million times using 3 arrays of 10 bytes each. Here are the results:

1. New Byte Array using System.Array.Copy - 0.2187556 seconds
2. New Byte Array using System.Buffer.BlockCopy - 0.1406286 seconds
3. IEnumerable<byte> using C# yield operator - 0.0781270 seconds
4. IEnumerable<byte> using LINQ's Concat<> - 0.0781270 seconds

I increased the size of each array to 100 elements and re-ran the test:

1. New Byte Array using System.Array.Copy - 0.2812554 seconds
2. New Byte Array using System.Buffer.BlockCopy - 0.2500048 seconds
3. IEnumerable<byte> using C# yield operator - 0.0625012 seconds
4. IEnumerable<byte> using LINQ's Concat<> - 0.0781265 seconds

I increased the size of each array to 1000 elements and re-ran the test:

1. New Byte Array using System.Array.Copy - 1.0781457 seconds
2. New Byte Array using System.Buffer.BlockCopy - 1.0156445 seconds
3. IEnumerable<byte> using C# yield operator - 0.0625012 seconds
4. IEnumerable<byte> using LINQ's Concat<> - 0.0781265 seconds

Finally, I increased the size of each array to 1 million elements and re-ran the test, executing each loop only 4000 times:

1. New Byte Array using System.Array.Copy - 13.4533833 seconds
2. New Byte Array using System.Buffer.BlockCopy - 13.1096267 seconds
3. IEnumerable<byte> using C# yield operator - 0 seconds
4. IEnumerable<byte> using LINQ's Concat<> - 0 seconds

So, if you need a new byte array, use

byte[] rv = new byte[a1.Length + a2.Length + a3.Length];
System.Buffer.BlockCopy(a1, 0, rv, 0, a1.Length);
System.Buffer.BlockCopy(a2, 0, rv, a1.Length, a2.Length);
System.Buffer.BlockCopy(a3, 0, rv, a1.Length + a2.Length, a3.Length);

But, if you can use an IEnumerable<byte>, DEFINITELY prefer LINQ's Concat<> method. It's only slightly slower than the C# yield operator, but is more concise and more elegant.

IEnumerable<byte> rv = a1.Concat(a2).Concat(a3);

If you have an arbitrary number of arrays and are using .NET 3.5, you can make the System.Buffer.BlockCopy solution more generic like this:

private byte[] Combine(params byte[][] arrays)
{
    byte[] rv = new byte[arrays.Sum(a => a.Length)];
    int offset = 0;
    foreach (byte[] array in arrays) {
        System.Buffer.BlockCopy(array, 0, rv, offset, array.Length);
        offset += array.Length;
    }
    return rv;
}

*Note: The above block requires you adding the following namespace at the the top for it to work.

using System.Linq;

To Jon Skeet's point regarding iteration of the subsequent data structures (byte array vs. IEnumerable<byte>), I re-ran the last timing test (1 million elements, 4000 iterations), adding a loop that iterates over the full array with each pass:

1. New Byte Array using System.Array.Copy - 78.20550510 seconds
2. New Byte Array using System.Buffer.BlockCopy - 77.89261900 seconds
3. IEnumerable<byte> using C# yield operator - 551.7150161 seconds
4. IEnumerable<byte> using LINQ's Concat<> - 448.1804799 seconds

The point is, it is VERY important to understand the efficiency of both the creation and the usage of the resulting data structure. Simply focusing on the efficiency of the creation may overlook the inefficiency associated with the usage. Kudos, Jon.

Prepend to a C# Array

If you are only going to perform this operation once then there isn't a whole lot of choices. The code provided by Monroe's answer should do just fine.

byte[] newValues = new byte[values.Length + 1];
newValues[0] = 0x00;                                // set the prepended value
Array.Copy(values, 0, newValues, 1, values.Length); // copy the old values

If, however, you're going to be performing this operation multiple times you have some more choices. There is a fundamental problem that prepending data to an array isn't an efficient operation, so you could choose to use an alternate data structure.

A LinkedList can efficiently prepend data, but it's less efficient in general for most tasks as it involves a lot more memory allocation/deallocation and also looses memory locallity, so it may not be a net win.

A double ended queue (known as a deque) would be a fantastic data structure for you. You can efficiently add to the start or the end, and efficiently access data anywhere in the structure (but you can't efficiently insert somewhere other than the start or end). The major problem here is that .NET doesn't provide an implementation of a deque. You'd need to find a 3rd party library with an implementation.

You can also save yourself a lot when copying by keeping track of "data that I need to prepend" (using a List/Queue/etc.) and then waiting to actually prepend the data as long as possible, so that you minimize the creation of new arrays as much as possible, as well as limiting the number of copies of existing elements.

You could also consider whether you could adjust the structure so that you're adding to the end, rather than the start (even if you know that you'll need to reverse it later). If you are appending a lot in a short space of time it may be worth storing the data in a List (which can efficiently add to the end) and adding to the end. Depending on your needs, it may even be worth making a class that is a wrapper for a List and that hides the fact that it is reversed. You could make an indexer that maps i to Count-i, etc. so that it appears, from the outside, as though your data is stored normally, even though the internal List actually holds the data backwards.

---

Related Topics