What Is the Use of the Arraysegment<T> Class

How is the new C# SpanT different from ArraySegmentT?

Span<T> does not replace anything. It's value-added. It provides a type-safe view into continuous segments of memory which can be allocated in many different ways: either as a managed array, a stack-based memory or unmanaged memory.

ArraySegment<T> is limited to managed arrays. You can't use it to wrap data allocated on the stack using stackalloc. Span<T> allows you to do that.

ArraySegment<T> also does not provide a read-only view into the underlying array. ReadOnlySpan<T> gives you that.

Span<T> is not supposed to replace arrays. At the end of the day it's just a view into data. That data has to be allocated somehow, and in managed world that allocation, for most cases, will be an array allocation. So you still need arrays.

You should use Span<T> if you want your code to be able to manipulate more than just arrays. E.g. consider a parsing library. Right now, to allow it to work with arrays, stack-allocated memory and unmanaged memory, it has to provide multiple entry points in the API for each of these, and use unsafe code to actually manipulate the data. It also probably would need to expose a string-based API to be used by people who have their data allocated as strings. With Span and ReadOnlySpan you can merge all that logic to a single, Span-based solution which will be applicable in all these scenarios.

Span<T> is definitely not going to be something that's used by everybody and very often. It's a highly specialized part of .NET framework useful mostly to library authors and in very high performance critical scenarios. E.g. Kestrel, the web service behind ASP.NET Core will get a lot of performance benefits from moving to Span<T> because e.g. parsing the request can be done using Span<T> and stack-allocated memory, which puts no pressure on GC. But you, writing websites and services based on ASP.NET Core, don't necessary have to use it.

ArraySegment must be cast to IList to iterate through it with an Index. Is there a way to hide the irrelevant IList methods?

Turns out, in .NET 4.7.2, the ArraySegment<T> doesn't expose an indexer unless if it's cast to the IList<T> interface, but it does in .NET Core 2.1.

You may cast to the IReadOnlyList<T> interface; note that it doesn't prevent you from changing the contained objects themselves if they are mutable:

The IReadOnlyList<T> represents a list in which the number and order of list elements is read-only. The content of list elements is not guaranteed to be read-only.

So, it only guarantees that the collection itself (the container) is immutable (no adds, removes, replacements). You can't replace an element because the indexer doesn't have a setter.

There's also the ReadOnlyCollection<T> wrapper class (add using System.Collections.ObjectModel;). Also no setter on the indexer though.

If none of these work for you, and you want to implement a wrapper, just create a class that takes in an IList<T> in the constructor, and implement your own indexer. I'd also implement IEnumerable<T>, so that you get LINQ support, and it will also make it work with the foreach loop.

// Add these...
using System.Collections;
using System.Collections.Generic;

//...

public class MyWrapper<T> : IEnumerable<T>
{
  private IList<T> _internalList;

  public MyWrapper(IList<T> list)
  {
    _internalList = list;
  }

  public int Count => _internalList.Count;

  // the indexer
  public T this[int index]
  {
    get => _internalList[index];
    set => _internalList[index] = value;
  }

  public IEnumerator<T> GetEnumerator()
  {
    return _internalList.GetEnumerator();
  }

  IEnumerator IEnumerable.GetEnumerator() 
  {
    return this.GetEnumerator();
  }
}

And then just pass the array segment to the constructor (without casting).

P.S. If this is some internal code, used in few places only, writing a wrapper may or may not be worth the trouble; it's OK to use an IList<T> as long as its clear what the intention was.

Also, if you don't mind working with a copy of the array that's only limited to the range defined by the array segment, and if the copy operation cost is not a concern, you can call either the ToArray or ToList extension method on the segment. But I suspect you want the original array to be updated and you just need a "window" into a segment.

ArraySegment - Returning the actual segment C#

Starting from Thomas Levesque's suggestion I've built a simple ArraySegmentWrapper<T> class to use in this way:

static void Main(string[] args)
{
    int[] arr = new int[10];
    for (int i = 0; i < arr.Length; i++)
        arr[i] = i;

    // arr = 0,1,2,3,4,5,6,7,8,9

    var segment = new ArraySegmentWrapper<int>(arr, 2, 7);
    segment[0] = -1;
    segment[6] = -1;
    // now arr = 0,1,-1,3,4,5,6,7,-1,9

    // this prints: -1,3,4,5,6,7,-1
    foreach (var el in segment)
        Console.WriteLine(el);
}

Implementation:

public class ArraySegmentWrapper<T> : IList<T>
{
    private readonly ArraySegment<T> segment;

    public ArraySegmentWrapper(ArraySegment<T> segment)
    {
        this.segment = segment;
    }

    public ArraySegmentWrapper(T[] array, int offset, int count)
        : this(new ArraySegment<T>(array, offset, count))
    {
    }

    public int IndexOf(T item)
    {
        for (int i = segment.Offset; i < segment.Offset + segment.Count; i++)
            if (Equals(segment.Array[i], item))
                return i;
        return -1;
    }

    public void Insert(int index, T item)
    {
        throw new NotSupportedException();
    }

    public void RemoveAt(int index)
    {
        throw new NotSupportedException();
    }

    public T this[int index]
    {
        get
        {
            if (index >= this.Count)
                throw new IndexOutOfRangeException();
            return this.segment.Array[index + this.segment.Offset];
        }
        set
        {
            if (index >= this.Count)
                throw new IndexOutOfRangeException();
            this.segment.Array[index + this.segment.Offset] = value;
        }
    }

    public void Add(T item)
    {
        throw new NotSupportedException();
    }

    public void Clear()
    {
        throw new NotSupportedException();
    }

    public bool Contains(T item)
    {
        return this.IndexOf(item) != -1;
    }

    public void CopyTo(T[] array, int arrayIndex)
    {
        for (int i = segment.Offset; i < segment.Offset + segment.Count; i++)
        {
            array[arrayIndex] = segment.Array[i];
            arrayIndex++;
        }
    }

    public int Count
    {
        get { return this.segment.Count; }
    }

    public bool IsReadOnly
    {
        get { return false; }
    }

    public bool Remove(T item)
    {
        throw new NotSupportedException();
    }

    public IEnumerator<T> GetEnumerator()
    {
        for (int i = segment.Offset; i < segment.Offset + segment.Count; i++)
            yield return segment.Array[i];
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }
}

---

EDIT :

As pointed out by @JeppeStigNielsen in the comments, since .NET 4.5 ArraySegment<T> implements IList<T>

C# access unmanaged array using MemoryT or ArraySegmentT?

Yes for Memory<T>, but you need to create your own MemoryManager<T>. Don't worry - this isn't as scary as it sounds - here's one I wrote earlier...:

/// <summary>
/// A MemoryManager over a raw pointer
/// </summary>
/// <remarks>The pointer is assumed to be fully unmanaged, or externally pinned - no attempt will be made to pin this data</remarks>
public sealed unsafe class UnmanagedMemoryManager<T> : MemoryManager<T>
    where T : unmanaged
{
    private readonly T* _pointer;
    private readonly int _length;

    /// <summary>
    /// Create a new UnmanagedMemoryManager instance at the given pointer and size
    /// </summary>
    /// <remarks>It is assumed that the span provided is already unmanaged or externally pinned</remarks>
    public UnmanagedMemoryManager(Span<T> span)
    {
        fixed (T* ptr = &MemoryMarshal.GetReference(span))
        {
            _pointer = ptr;
            _length = span.Length;
        }
    }
    /// <summary>
    /// Create a new UnmanagedMemoryManager instance at the given pointer and size
    /// </summary>
    public UnmanagedMemoryManager(T* pointer, int length)
    {
        if (length < 0) throw new ArgumentOutOfRangeException(nameof(length));
        _pointer = pointer;
        _length = length;
    }
    /// <summary>
    /// Obtains a span that represents the region
    /// </summary>
    public override Span<T> GetSpan() => new Span<T>(_pointer, _length);

    /// <summary>
    /// Provides access to a pointer that represents the data (note: no actual pin occurs)
    /// </summary>
    public override MemoryHandle Pin(int elementIndex = 0)
    {
        if (elementIndex < 0 || elementIndex >= _length)
            throw new ArgumentOutOfRangeException(nameof(elementIndex));
        return new MemoryHandle(_pointer + elementIndex);
    }
    /// <summary>
    /// Has no effect
    /// </summary>
    public override void Unpin() { }

    /// <summary>
    /// Releases all resources associated with this object
    /// </summary>
    protected override void Dispose(bool disposing) { }
}

Now you can use:

var mgr = new UnmanagedMemoryManager((byte*)ptr + (index * Width), Width);
Memory<byte> memory = mgr.Memory;

and memory can be stored on the heap.

However, to minimize allocations you probably want to create a single UnmanagedMemoryManager<byte> that covers the entire region - once only - and then use .Slice(...) on the .Memory that represents the entire region. That way you have a single object and lots of slices (the slices are structs, not objects).

Note this implementation assumes that you're going to control the lifetime of the memory elsewhere - the Dispose() here does not attempt to release the memory via Marshal etc.

Class to JSON object using ArraySegment

You need to serialize the objectToSerialize into a buffer of bytes first

var data = JsonConvert.SerializeObject(objectToSerialize);
var encoded = Encoding.UTF8.GetBytes(data);
var buffer = new ArraySegment<Byte>(encoded, 0, encoded.Length);
await webSocket.SendAsync.SendAsync(buffer, WebSocketMessageType.Text, true, CancellationToken.None);

How to segmentate an IListT to segments of N size, without creating copies and without memory allocations?

You need to implement an ArraySegment<T> equivalent for IList<T>. See implementation below. For optimal performance, consider using spans instead.

ListSegment<T> Struct

public readonly struct ListSegment<T> : IList<T>
{
    public List<T> Items { get; }
    public int Offset { get; }
    public int Count { get; }

    public ListSegment(List<T> items, int offset, int count)
    {
        Items = items ?? throw new ArgumentNullException(nameof(items));
        Offset = offset;
        Count = count;

        if (items.Count < offset + count)
        {
            throw new ArgumentException("List segment out of range.", nameof(count));
        }
    }

    public void CopyTo(T[] array, int index)
    {
        if (Count > 0)
        {
            Items.CopyTo(Offset, array, index, Count);
        }
    }

    public bool Contains(T item) => IndexOf(item) != -1;

    public int IndexOf(T item)
    {
        for (var i = Offset; i < Offset + Count; i++)
        {
            if (Items[i].Equals(item))
            {
                return i;
            }
        }

        return -1;
    }

    private T ElementAt(int index)
    {
        if (Count > 0)
        {
            return Items[Offset + index];
        }

        throw new ArgumentOutOfRangeException(nameof(index));
    }

    public ListSegmentEnumerator GetEnumerator() => new ListSegmentEnumerator(this);

    #region IEnumerable<T> interface
    IEnumerator<T> IEnumerable<T>.GetEnumerator() => GetEnumerator();
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
    #endregion

    #region ICollection<T> interface
    bool ICollection<T>.IsReadOnly => true;

    void ICollection<T>.Add(T item) => throw new NotImplementedException();
    bool ICollection<T>.Remove(T item) => throw new NotImplementedException();
    void ICollection<T>.Clear() => throw new NotImplementedException();
    #endregion

    #region IList<T> interface
    void IList<T>.Insert(int index, T item) => throw new NotImplementedException();
    void IList<T>.RemoveAt(int index) => throw new NotImplementedException();
    T IList<T>.this[int index]
    {
        get => ElementAt(index);
        set => throw new NotImplementedException();
    }
    #endregion

    public struct ListSegmentEnumerator : IEnumerator<T>
    {
        private readonly List<T> items;
        private readonly int start;
        private readonly int end;
        private int current;

        public ListSegmentEnumerator(ListSegment<T> segment)
        {
            items = segment.Items;
            start = segment.Offset;
            end = start + segment.Count;
            current = start - 1;
        }

        public bool MoveNext()
        {
            if (current < end)
            {
                current++;

                return current < end;
            }
            return false;
        }

        public T Current => items[current];
        object IEnumerator.Current => Current;

        void IEnumerator.Reset() => current = start - 1;
        void IDisposable.Dispose() { }
    }
}

Does protobuf-net support ArraySegment (yet?)

That looks like overly complicated to me. I think that your idea to make Foo generic is pretty good, so you could have:

[ProtoContract]
class Foo<T>
{
    [ProtoMember(1)]
    public T Data { get; set; }
}

And then you would instantiate a new Object with simple code like:

var foo = new Foo<Bar>() { Data = new Bar() { Data = "Some Data" } };

However,based on your additional comments you can't guarantee the object is serializable, or has proto attributes. For example the bar class could look like:

class Bar
{
    public string Data { get; set; }
}

In that case you can use the RuntimeTypeModel class to configure serialization at runtime.

Here is an example of full serialization and deserialization using protobuf with a dynamic runtime configuration for the data object:

using (var serializationBuffer = new MemoryStream())
{
    var foo = new Foo<Bar>() { Data = new Bar() { Data = "Some Data" } };

    RuntimeTypeModel.Default.Add(foo.Data.GetType(), false)
        .Add(foo.GetType().GetProperties().Select(p => p.Name).ToArray());

    Serializer.Serialize(serializationBuffer, foo);

    serializationBuffer.Seek(0, SeekOrigin.Begin);

    var deserialized = Serializer.Deserialize<Foo<Bar>>(serializationBuffer);
}

This way you can use any object as your data object, even if it doesn't have serialization attributes. However, you will have a performance penalty, since reflection is used to discover the type properties. However, you should get the flexibility you need.

The correct way to send this object over the wire using WCF would be to configure a WCF custom behavior using the protobuf ProtoEndpointBehavior and then WCF would automatically serialize and deserialize your Foo objects using protobuf. This means that the client would just make sure it is using objects decorated with the proto attributes, and send them over the wire. WCF would take care of the serialization/deserialization.

Here is a comprehensive example how to consume WCF protobuf services:

http://www.drdobbs.com/windows/working-with-protobuf-wcf-services/240159282?pgno=1

The efficiency of sending data over the wire depends on many factors. If you are sending big objects through the wire you can set WCF TransferMode to use Streaming, and then read the data in byte array chunks on the other end.

However, efficiency is something you should measure, not assume.

---

Related Topics