Should You Always Favor Xrange() Over Range()

Should you always favor xrange() over range()?

For performance, especially when you're iterating over a large range, xrange() is usually better. However, there are still a few cases why you might prefer range():

• In python 3, range() does what xrange() used to do and xrange() does not exist. If you want to write code that will run on both Python 2 and Python 3, you can't use xrange().

• range() can actually be faster in some cases - eg. if iterating over the same sequence multiple times. xrange() has to reconstruct the integer object every time, but range() will have real integer objects. (It will always perform worse in terms of memory however)

• xrange() isn't usable in all cases where a real list is needed. For instance, it doesn't support slices, or any list methods.

[Edit] There are a couple of posts mentioning how range() will be upgraded by the 2to3 tool. For the record, here's the output of running the tool on some sample usages of range() and xrange()

RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: ws_comma
--- range_test.py (original)
+++ range_test.py (refactored)
@@ -1,7 +1,7 @@

 for x in range(20):
-    a=range(20)
+    a=list(range(20))
     b=list(range(20))
     c=[x for x in range(20)]
     d=(x for x in range(20))
-    e=xrange(20)
+    e=range(20)

As you can see, when used in a for loop or comprehension, or where already wrapped with list(), range is left unchanged.

What is the difference between range and xrange functions in Python 2.X?

In Python 2.x:

• range creates a list, so if you do range(1, 10000000) it creates a list in memory with 9999999 elements.

• xrange is a sequence object that evaluates lazily.

In Python 3:

• range does the equivalent of Python 2's xrange. To get the list, you have to explicitly use list(range(...)).
• xrange no longer exists.

What is the difference between range and xrange functions in Python 2.X?

In Python 2.x:

• range creates a list, so if you do range(1, 10000000) it creates a list in memory with 9999999 elements.

• xrange is a sequence object that evaluates lazily.

In Python 3:

• range does the equivalent of Python 2's xrange. To get the list, you have to explicitly use list(range(...)).
• xrange no longer exists.

What is the difference between range and xrange functions in Python 2.X?

In Python 2.x:

• range creates a list, so if you do range(1, 10000000) it creates a list in memory with 9999999 elements.

• xrange is a sequence object that evaluates lazily.

In Python 3:

• range does the equivalent of Python 2's xrange. To get the list, you have to explicitly use list(range(...)).
• xrange no longer exists.

How is irange() any different from range() or xrange()?

irange() returns a generator type, which can only be iterated over. Nothing else. Once you iterated over it, the generator is exhausted and can not be iterated over again.

The Python 2 xrange() type and Python 3 range() type are sequence types, they support various operations that other sequences support as well, such as reporting on their length, test for containment, and indexing:

>>> xr = xrange(10, 20, 3)
>>> len(xr)
4
>>> 10 in xr
True
>>> xr[0]
10
>>> xr[1]
13

You can iterate over these objects more than once:

>>> for i in xr:
...     print i,
... 
10 13 16 19
>>> for i in xr:
...     print i,
... 
10 13 16 19

You can even use the reversed() function to iterate over them in reverse, efficiently:

>>> for i in reversed(xr):
...     print i,
... 
19 16 13 10

The Python 3 range() type is an improved version of xrange(), in that it supports more sequence operations, is more efficient still, and can handle values beyond sys.maxint (what would be a long integer in Python 2).

It supports slicing, for example, which results in a new range() object for the sliced values:

>>> r = range(10, 20, 3)
>>> r[:2]
range(10, 16, 3)

You can use negative indices just like you can with other Python sequences, to get elements counting from the end:

>>> r[-2]
16
>>> r[-2:]
range(16, 22, 3)

and the type supports testing for equality; two range() instances are equal if they'd yield the same values:

>>> range(10, 20, 3) == range(10, 21, 3)
True

In Python 2, the only advantage the generator irange() might have is that it doesn't suffer from the limitation to non-long integers that xrange() is subjected to:

>>> import sys
>>> xrange(sys.maxint)
xrange(9223372036854775807)
>>> xrange(sys.maxint + 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long

I read xrange is lazy it works at the last moment is it

xrange() produces an object that only tracks the start, stop and step values. Nothing else.

Iterating then makes use of those values; the 5th element is always going to be start + (5 * step) anyway, so it just calculates that, rather than create a list like range() does.

To do this while iterating, a separate rangeiterator is created:

>>> xr = xrange(100)
>>> it = iter(xr)
>>> it
<rangeiterator object at 0x100660450>
>>> next(it)
0
>>> next(it)
1

The range iterator knows what step the iteration is at and asks xrange() to produce the next integer for that step.

xrange as an iterator and chunking

An xrange is not any sort of iterator. People keep calling it a generator, but it's not; an xrange is an immutable sequence, like a tuple:

>>> x = xrange(5)
>>> x[2]
2
>>> for i in x:
...     print i
...
0
1
2
3
4
>>> for i in x:
...     print i
...
0
1
2
3
4

As with any other sequence type, each time you request an iterator from an xrange, you get a new, independent iterator. Thus, when you zip xrange(10) with itself, you get the same output as if you had zipped [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] with itself, rather than if you had zipped iter([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) with itself.

Adjust xrange depending on where signal is

Use argwhere to get the indices where y is above a certain threshold, e.g. np.argwhere(y > 10). In practice you may want to add some margin of say 200 points at both sides:

import numpy as np
import matplotlib.pyplot as plt

def pdf(x, mean, std):
    """Return a PDF."""
    y_out = 1/(std * np.sqrt(2 * np.pi)) * np.exp(-(x - mean)**2 / (2 * std**2))
    return y_out

x = np.linspace(-1000, 1000, 20000)
y = [100 * pdf(x, 0, 1) + np.random.normal(0, 1, 20000),
     100 * pdf(x, -50, 1) + 100 * pdf(x, 0, 1) + 100 * pdf(x, 40, 1) + np.random.normal(0, 1, 20000)]

def get_lims(y, threshold=10, margin=200):
    """Get indices +/- margin where y is above threshold."""
    signal = np.argwhere(y > threshold).flatten()
    return np.clip((signal[0] - margin, signal[-1] + margin), 0, len(y)-1)

fig, ax = plt.subplots(nrows=2)
for i in range(2):
    ax[i].plot(x, y[i])
    ax[i].set_xlim(x[get_lims(y[i])])

---

Related Topics