How to Use the Ellipsis Slicing Syntax in Python

How do you use the ellipsis slicing syntax in Python?

Ellipsis, or ... is not a hidden feature, it's just a constant. It's quite different to, say, javascript ES6 where it's a part of the language syntax. No builtin class or Python language constuct makes use of it.

So the syntax for it depends entirely on you, or someone else, having written code to understand it.

Numpy uses it, as stated in the documentation. Some examples here.

In your own class, you'd use it like this:

>>> class TestEllipsis(object):
...     def __getitem__(self, item):
...         if item is Ellipsis:
...             return "Returning all items"
...         else:
...             return "return %r items" % item
... 
>>> x = TestEllipsis()
>>> print x[2]
return 2 items
>>> print x[...]
Returning all items

Of course, there is the python documentation, and language reference. But those aren't very helpful.

How slicing and ellipsis works in numpy?

Three full stops ... (and not … (U+2026)), refers to the Ellipsis singleton object. It has no built-in special operations but is often used in slicing expressions.

No built-in classes utilise the Ellipsis object however NumPy uses ... as a shortcut when slicing arrays, for example, where x is a 4D array: x[i, ...] is equivalant to x[i, :, :, :].

NumPy - Indexing

What does the Ellipsis object do?

This came up in another question recently. I'll elaborate on my answer from there:

Ellipsis is an object that can appear in slice notation. For example:

myList[1:2, ..., 0]

Its interpretation is purely up to whatever implements the __getitem__ function and sees Ellipsis objects there, but its main (and intended) use is in the numpy third-party library, which adds a multidimensional array type. Since there are more than one dimensions, slicing becomes more complex than just a start and stop index; it is useful to be able to slice in multiple dimensions as well. E.g., given a 4 × 4 array, the top left area would be defined by the slice [:2, :2]:

>>> a
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12],
       [13, 14, 15, 16]])

>>> a[:2, :2]  # top left
array([[1, 2],
       [5, 6]])

Extending this further, Ellipsis is used here to indicate a placeholder for the rest of the array dimensions not specified. Think of it as indicating the full slice [:] for all the dimensions in the gap it is placed, so for a 3d array, a[..., 0] is the same as a[:, :, 0] and for 4d a[:, :, :, 0], similarly, a[0, ..., 0] is a[0, :, :, 0] (with however many colons in the middle make up the full number of dimensions in the array).

Interestingly, in python3, the Ellipsis literal (...) is usable outside the slice syntax, so you can actually write:

>>> ...
Ellipsis

EDIT: Ellipsis is also used in the standard library typing module: e.g. Callable[..., int] to indicate a callable that returns an int without specifying the signature, or tuple[str, ...] to indicate a variable-length homogeneous tuple of strings.

Slice lists and the ellipsis in Python; slicing lists and lists of lists with lists of slices

Slice lists and ellipsis were originally introduced in Python to supply nice syntax sugar for the precedessor of numpy (good old Numeric). If you're using numpy (no reason to go back to any of its predecessors!-) you should of course use them; if for whatever strange reason you're doing your own implementation of super-flexible multi-dimensional arrays, you'll definitely want to study the way numpy uses them and probably imitate it closely (it is pretty well designed after all). I can't think of good uses beyond multi-dimensional arrays.

Understanding slicing

The syntax is:

a[start:stop]  # items start through stop-1
a[start:]      # items start through the rest of the array
a[:stop]       # items from the beginning through stop-1
a[:]           # a copy of the whole array

There is also the step value, which can be used with any of the above:

a[start:stop:step] # start through not past stop, by step

The key point to remember is that the :stop value represents the first value that is not in the selected slice. So, the difference between stop and start is the number of elements selected (if step is 1, the default).

The other feature is that start or stop may be a negative number, which means it counts from the end of the array instead of the beginning. So:

a[-1]    # last item in the array
a[-2:]   # last two items in the array
a[:-2]   # everything except the last two items

Similarly, step may be a negative number:

a[::-1]    # all items in the array, reversed
a[1::-1]   # the first two items, reversed
a[:-3:-1]  # the last two items, reversed
a[-3::-1]  # everything except the last two items, reversed

Python is kind to the programmer if there are fewer items than you ask for. For example, if you ask for a[:-2] and a only contains one element, you get an empty list instead of an error. Sometimes you would prefer the error, so you have to be aware that this may happen.

Relationship with the slice object

A slice object can represent a slicing operation, i.e.:

a[start:stop:step]

is equivalent to:

a[slice(start, stop, step)]

Slice objects also behave slightly differently depending on the number of arguments, similarly to range(), i.e. both slice(stop) and slice(start, stop[, step]) are supported.
To skip specifying a given argument, one might use None, so that e.g. a[start:] is equivalent to a[slice(start, None)] or a[::-1] is equivalent to a[slice(None, None, -1)].

While the :-based notation is very helpful for simple slicing, the explicit use of slice() objects simplifies the programmatic generation of slicing.

Formal syntax of Python's extended slice notation?

Anything is possible, as long as it is a valid Python expression. The object produced by the expression between [...] is passed to the __getitem__ method. That's it.

Commas produce a tuple, : colons in an expression produce a slice() object. Beyond that, use whatever you want.

That's because the grammar allows for any expression_list in the notation. See the reference documentation:

subscription ::=  primary "[" expression_list "]"

Slicing is further specified in the Slicings section:

slicing      ::=  primary "[" slice_list "]"
slice_list   ::=  slice_item ("," slice_item)* [","]
slice_item   ::=  expression | proper_slice
proper_slice ::=  [lower_bound] ":" [upper_bound] [ ":" [stride] ]
lower_bound  ::=  expression
upper_bound  ::=  expression
stride       ::=  expression

So again arbitrary expressions are allowed, and : triggers the proper_slice grammar rule.

Note that the lower_bound, upper_bound and stride expression results are used to construct a slice() object, which can only handle integer values. Anything that can't be converted to an integer will result in a TypeError being raised. That's not the same thing as a syntax error; t[1:...] is syntactically just fine, but ... is not convertable to an integer so you get a runtime TypeError exception. Your two examples using non-integer slice values are not possible on Python versions 2.4 and up at the very least.

Your actual syntax errors all stem from invalid expressions. Apart from the : proper_slice notation, if you can't put the part between [...] on the right-hand side of an assignment, you can't use it in a slice either.

For example, ; can only be used to put multiple simple statements on a single logical line. Statements can contain expressions, but expressions can never contain statements, excluding ; from expressions. (9:5), is not a valid expression (nowhere else in Python could you use a : in parentheses, the parenth_form rule doesn't allow for any such options).

The Python 2 grammar for slicings is a little more elaborate in that ... is a specific notation in the grammar there, and you can't actually use the ... outside of slicings (in Python 3 you can use ... anywhere an expression is valid), which is why t[(...)] is a syntax error in Python 2 but not in Python 3.

What is the difference between the `slice` (:) and the `ellipsis` (...) operators in `numpy`?

The motivations behind the two are completely different.

• The ellipsis means "all the other dimensions (which I can't be bothered to enumerate or am unsure or don't care how many there are) in their entirety"

• the slice means "the subset of the current dimension as specified by the starting and ending indices (and stride)".

Is there a Julia equivalent to NumPy's ellipsis slicing syntax (...)?

Not yet, but you can help yourself if you want.

    import Base.getindex, Base.setindex!
    const .. = Val{:...}

    setindex!{T}(A::AbstractArray{T,1}, x, ::Type{Val{:...}}, n) = A[n] = x
    setindex!{T}(A::AbstractArray{T,2}, x, ::Type{Val{:...}}, n) = A[ :, n] = x
    setindex!{T}(A::AbstractArray{T,3}, x, ::Type{Val{:...}}, n) = A[ :, :, n] =x

    getindex{T}(A::AbstractArray{T,1}, ::Type{Val{:...}}, n) = A[n]
    getindex{T}(A::AbstractArray{T,2}, ::Type{Val{:...}}, n) = A[ :, n]
    getindex{T}(A::AbstractArray{T,3}, ::Type{Val{:...}}, n) = A[ :, :, n]

Then you can write

    > rand(3,3,3)[.., 1]
    3x3 Array{Float64,2}:
     0.0750793  0.490528  0.273044
     0.470398   0.461376  0.01372 
     0.311559   0.879684  0.531157

If you want more elaborate slicing, you need to generate/expand the definition or use staged functions.

Edit: Nowadays, see https://github.com/ChrisRackauckas/EllipsisNotation.jl

---

Related Topics