Xrange(2**100)' -> Overflowerror: Long Int Too Large to Convert to Int

`xrange(2**100)` - OverflowError: long int too large to convert to int

Okay, here's a go at a fuller reimplementation.

class MyXRange(object):
    def __init__(self, a1, a2=None, step=1):
        if step == 0:
            raise ValueError("arg 3 must not be 0")
        if a2 is None:
            a1, a2 = 0, a1
        if (a2 - a1) % step != 0:
            a2 += step - (a2 - a1) % step
        if cmp(a1, a2) != cmp(0, step):
            a2 = a1
        self.start, self.stop, self.step = a1, a2, step

    def __iter__(self):
        n = self.start
        while cmp(n, self.stop) == cmp(0, self.step):
            yield n
            n += self.step

    def __repr__(self):
        return "MyXRange(%d,%d,%d)" % (self.start, self.stop, self.step)

    # NB: len(self) will convert this to an int, and may fail
    def __len__(self):
        return (self.stop - self.start)//(self.step)

    def __getitem__(self, key):
        if key < 0:
            key = self.__len__() + key
            if key < 0:
                raise IndexError("list index out of range")
            return self[key]
        n = self.start + self.step*key
        if cmp(n, self.stop) != cmp(0, self.step):
            raise IndexError("list index out of range")
        return n

    def __reversed__(self):
        return MyXRange(self.stop-self.step, self.start-self.step, -self.step)

    def __contains__(self, val):
        if val == self.start: return cmp(0, self.step) == cmp(self.start, self.stop)
        if cmp(self.start, val) != cmp(0, self.step): return False
        if cmp(val, self.stop) != cmp(0, self.step): return False
        return (val - self.start) % self.step == 0

And some testing:

def testMyXRange(testsize=10):
    def normexcept(f,args):
        try:
            r = [f(args)]
        except Exception, e:
            r = type(e)
        return r

    for i in range(-testsize,testsize+1):
        for j in range(-testsize,testsize+1):
            print i, j
            for k in range(-9, 10, 2):
                r, mr = range(i,j,k), MyXRange(i,j,k)

                if r != list(mr):
                    print "iter fail: %d, %d, %d" % (i,j,k)

                if list(reversed(r)) != list(reversed(mr)):
                    print "reversed fail: %d, %d, %d" % (i,j,k)

                if len(r) != len(mr):
                    print "len fail: %d, %d, %d" % (i,j,k)

                z = [m for m in range(-testsize*2,testsize*2+1)
                      if (m in r) != (m in mr)]
                if z != []:
                    print "contains fail: %d, %d, %d, %s" % (i,j,k,(z+["..."])[:10])

                z = [m for m in range(-testsize*2, testsize*2+1) 
                      if normexcept(r.__getitem__, m) != normexcept(mr.__getitem__, m)]
                if z != []:
                    print "getitem fail: %d, %d, %d, %s" % (i,j,k,(z+["..."])[:10])

Portable, memory efficient range() for Python 2.x and Python 3.x

One alternative is to use the Six module, that provides simple utilities for wrapping over differences between Python 2 and Python 3. It is intended to support codebases that work on both Python 2 and 3 without modification.

Why does Python say Inappropriate Argument Type for xrange?

xrange won't work with large numbers since the code is executed as a C program (https://docs.python.org/2/library/functions.html#xrange).

According to docs -

CPython implementation detail: xrange() is intended to be simple and
fast. Implementations may impose restrictions to achieve this. The C
implementation of Python restricts all arguments to native C longs
(“short” Python integers), and also requires that the number of
elements fit in a native C long. If a larger range is needed, an
alternate version can be crafted using the itertools module:
islice(count(start, step), (stop-start+step-1+2*(step<0))//step).

I am not sure what system specs you got, but generally c int is 2^16-1 and c long is 2^32-1.

OverflowError: normalized days too large to fit in a C int

I've found out that some of Python's built-in functions, such as range or xrange, do not support larger integers, probably because they're implemented in C as an optimization. Take a look at this question for an example.

This could be the case for your code. Does t[0] + t[1] fit in an integer? If not, you'll have to either find a way around it (normalize t[0] + t[1]? Depends on what you want to do, and your snippet doesn't make that clear) or implement your own timedelta.

EDIT:

Taking a look at Python's documentation and running your code on my desktop (WinXP 32bit/Python2.7), I see no reason for an integer overflow. However, you mention that this problem occurs occasionally, so it could be the Amazon instance's times() returning some funky values (yay virtualization ;)).

First, try doing some tests to determine for exactly what ranges of t[0] and t[1] the exception happens. If they actually do have some uncommonly high values (maybe because the instance was paused then resumed, don't know with such little detail), your code test against that.

---

Related Topics