Why Are Slice and Range Upper-Bound Exclusive

Why are slice and range upper-bound exclusive?

The documentation implies this has a few useful properties:

word[:2]    # The first two characters
word[2:]    # Everything except the first two characters

Here’s a useful invariant of slice operations: s[:i] + s[i:] equals s.

For non-negative indices, the length of a slice is the difference of the indices, if both are within bounds. For example, the length of word[1:3] is 2.

I think we can assume that the range functions act the same for consistency.

C#: Why is the upper bound of ranges exclusive?

There are a lot of design decisions that just won't have satisfying answers. However, this one at least has a paper-trail you can follow in the design notes (in one or more mentions) starting at the following link.

_{Note : This was just an arbitrary decision, the reason they chose exclusive is because they chose exclusive. It could easily not have been the case due to the pros and cons.}

C# Language Design Notes for Jan 22, 2018

...

Conclusion

Let us go with .. means exclusive. Since we've chosen to focus on the
indexing/slicing scenario, this seems the right thing to do:

• It allows a.Length as an endpoint without adding/subtracting 1.
• It lets the end of one range be the beginning of the next without overlap
• It avoids ugly empty ranges of the form x..x-1

And as @vc74 mentions in the comments and also stated in the document supplied, other languages like Python follow this convention, however others don't.

Understanding slicing

The syntax is:

a[start:stop]  # items start through stop-1
a[start:]      # items start through the rest of the array
a[:stop]       # items from the beginning through stop-1
a[:]           # a copy of the whole array

There is also the step value, which can be used with any of the above:

a[start:stop:step] # start through not past stop, by step

The key point to remember is that the :stop value represents the first value that is not in the selected slice. So, the difference between stop and start is the number of elements selected (if step is 1, the default).

The other feature is that start or stop may be a negative number, which means it counts from the end of the array instead of the beginning. So:

a[-1]    # last item in the array
a[-2:]   # last two items in the array
a[:-2]   # everything except the last two items

Similarly, step may be a negative number:

a[::-1]    # all items in the array, reversed
a[1::-1]   # the first two items, reversed
a[:-3:-1]  # the last two items, reversed
a[-3::-1]  # everything except the last two items, reversed

Python is kind to the programmer if there are fewer items than you ask for. For example, if you ask for a[:-2] and a only contains one element, you get an empty list instead of an error. Sometimes you would prefer the error, so you have to be aware that this may happen.

Relationship with the slice object

A slice object can represent a slicing operation, i.e.:

a[start:stop:step]

is equivalent to:

a[slice(start, stop, step)]

Slice objects also behave slightly differently depending on the number of arguments, similarly to range(), i.e. both slice(stop) and slice(start, stop[, step]) are supported.
To skip specifying a given argument, one might use None, so that e.g. a[start:] is equivalent to a[slice(start, None)] or a[::-1] is equivalent to a[slice(None, None, -1)].

While the :-based notation is very helpful for simple slicing, the explicit use of slice() objects simplifies the programmatic generation of slicing.

Why does .loc have inclusive behavior for slices?

Quick answer:

It often makes more sense to do end-inclusive slicing when using labels, because it requires less knowledge about other rows in the DataFrame.

Whenever you care about labels instead of positions, end-exclusive label slicing introduces position-dependence in a way that can be inconvenient.

---

Longer answer:

Any function's behavior is a trade-off: you favor some use cases over others. Ultimately the operation of .iloc is a subjective design decision by the Pandas developers (as the comment by @ALlollz indicates, this behavior is intentional). But to understand why they might have designed it that way, think about what makes label slicing different from positional slicing.

Imagine we have two DataFrames df1 and df2:

df1 = pd.DataFrame(dict(X=range(4)), index=['a','b','c','d'])
df2 = pd.DataFrame(dict(X=range(3)), index=['b','c','z'])

df1 contains:

   X

a  0
b  1
c  2
d  3

df2 contains:

   X

b  0
c  1
z  2

Let's say we have a label-based task to perform: we want to get rows between b and c from both df1 and df2, and we want to do it using the same code for both DataFrames. Because b and c don't have the same positions in both DataFrames, simple positional slicing won't do the trick. So we turn to label-based slicing.

If .loc were end-exclusive, to get rows between b and c we would need to know not only the label of our desired end row, but also the label of the next row after that. As constructed, this next label would be different in each DataFrame.

In this case, we would have two options:

• Use separate code for each DataFrame: df1.loc['b':'d'] and df2.loc['b':'z']. This is inconvenient because it means we need to know extra information beyond just the rows that we want.
• For either dataframe, get the positional index first, add 1, and then use positional slicing: df.iloc[df.index.get_loc('b'):df.index.get_loc('c')+1]. This is just wordy.

But since .loc is end-inclusive, we can just say .loc['b':'c']. Much simpler!

Whenever you care about labels instead of positions, and you're trying to write position-independent code, end-inclusive label slicing re-introduces position-dependence in a way that can be inconvenient.

That said, maybe there are use cases where you really do want end-exclusive label-based slicing. If so, you can use @Willz's answer in this question:

df.loc[start:end].iloc[:-1]

Python range slicing and indexing behavior

To quote Guido van Rossum himself:

[...] I was swayed by the elegance of half-open intervals. Especially the
invariant that when two slices are adjacent, the first slice's end
index is the second slice's start index is just too beautiful to
ignore. For example, suppose you split a string into three parts at
indices i and j -- the parts would be a[:i], a[i:j], and a[j:].

---

Related Topics