In-Place Type Conversion of a Numpy Array

In-place type conversion of a NumPy array

You can make a view with a different dtype, and then copy in-place into the view:

import numpy as np
x = np.arange(10, dtype='int32')
y = x.view('float32')
y[:] = x

print(y)

yields

array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.], dtype=float32)

To show the conversion was in-place, note that copying from x to y altered x:

print(x)

prints

array([         0, 1065353216, 1073741824, 1077936128, 1082130432,
       1084227584, 1086324736, 1088421888, 1090519040, 1091567616])

In-place type conversion of a Numpy array again?

From the copy argument documentation:

If this is set to false, and the dtype, order, and subok requirements
are satisfied, the input array is returned instead of a copy.

It can't actually convert the type in place. It only works if the input array is already of the dtype you want; otherwise, it has to make a copy anyway.

Numpy inplace dtype conversion

About the copy parameter:

By default, astype always returns a newly allocated array. If this
is set to false, and the dtype, order, and subok
requirements are satisfied, the input array is returned instead
of a copy.

So it's conditional.

In [540]: x=np.arange(10)
In [542]: x.dtype
Out[542]: dtype('int32')
In [543]: z=x.astype('float32',copy=False)
In [544]: z
Out[544]: array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.], dtype=float32)
In [545]: x.__array_interface__
Out[545]: 
{'data': (188221848, False),
 'descr': [('', '<i4')],
 'shape': (10,),
 'strides': None,
 'typestr': '<i4',
 'version': 3}
In [546]: z.__array_interface__
Out[546]: 
{'data': (191273640, False),
 'descr': [('', '<f4')],
 'shape': (10,),
 'strides': None,
 'typestr': '<f4',
 'version': 3}

z has different memory location.

---

The accepted answer in your link appears to work

In [549]: z=x.view('float32')
In [550]: z[:]=x
In [551]: z
Out[551]: array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.], dtype=float32)
In [552]: x
Out[552]: 
array([         0, 1065353216, 1073741824, 1077936128, 1082130432,
       1084227584, 1086324736, 1088421888, 1090519040, 1091567616])
In [553]: z
Out[553]: array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.], dtype=float32)
In [555]: x.__array_interface__
Out[555]: 
{'data': (188221848, False),
 'descr': [('', '<i4')],
 'shape': (10,),
 'strides': None,
 'typestr': '<i4',
 'version': 3}
In [556]: z.__array_interface__
Out[556]: 
{'data': (188221848, False),
 'descr': [('', '<f4')],
 'shape': (10,),
 'strides': None,
 'typestr': '<f4',
 'version': 3}

This works because z shares memory with x, but with a different dtype. When copied from x to z, they are converted to match the new dtype. Memory locations is preserved. However I can't guarantee that there wasn't a temporary buffer.

---

In case it isn't clear, conversion form int32 to float32 requires a change in the underlying bytes. The bit representation of integers is different from that of floats.

In [594]: np.array(1, 'int32').tobytes()
Out[594]: b'\x01\x00\x00\x00'
In [595]: np.array(1, 'float32').tobytes()
Out[595]: b'\x00\x00\x80?'

How can I convert a numpy array of float type into a numpy array of int type?

In [385]: npW_x = np.random.rand(4,5)*10
     ...: npW_round_111 = np.around(npW_x, decimals=0)
     ...: sum_x_111 = np.sum(npW_round_111, axis=1)
     ...: 
In [386]: sum_x_111
Out[386]: array([38., 25., 24., 30.])

save with default fmt (Read and reread the np.savetxt docs!)

In [387]: np.savetxt('test',sum_x_111)
In [388]: cat test
3.800000000000000000e+01
2.500000000000000000e+01
2.400000000000000000e+01
3.000000000000000000e+01

save with int format:

In [389]: np.savetxt('test',sum_x_111, fmt='%10d')
In [390]: cat test
        38
        25
        24
        30

conversion in memory to int:

In [391]: In [391]: sum_x_111.astype(int)
Out[391]: array([38, 25, 24, 30])

Converting numpy dtypes to native python types

Use val.item() to convert most NumPy values to a native Python type:

import numpy as np

# for example, numpy.float32 -> python float
val = np.float32(0)
pyval = val.item()
print(type(pyval))         # <class 'float'>

# and similar...
type(np.float64(0).item()) # <class 'float'>
type(np.uint32(0).item())  # <class 'int'>
type(np.int16(0).item())   # <class 'int'>
type(np.cfloat(0).item())  # <class 'complex'>
type(np.datetime64(0, 'D').item())  # <class 'datetime.date'>
type(np.datetime64('2001-01-01 00:00:00').item())  # <class 'datetime.datetime'>
type(np.timedelta64(0, 'D').item()) # <class 'datetime.timedelta'>
...

(Another method is np.asscalar(val), however it is deprecated since NumPy 1.16).

---

For the curious, to build a table of conversions of NumPy array scalars for your system:

for name in dir(np):
    obj = getattr(np, name)
    if hasattr(obj, 'dtype'):
        try:
            if 'time' in name:
                npn = obj(0, 'D')
            else:
                npn = obj(0)
            nat = npn.item()
            print('{0} ({1!r}) -> {2}'.format(name, npn.dtype.char, type(nat)))
        except:
            pass

There are a few NumPy types that have no native Python equivalent on some systems, including: clongdouble, clongfloat, complex192, complex256, float128, longcomplex, longdouble and longfloat. These need to be converted to their nearest NumPy equivalent before using .item().

Convert numpy array type and values from Float64 to Float32

Actually i tried hard but not able to do as the
'sklearn.tree._tree.Tree' objects is not writable.

It is causing a precision issue while generating a PMML file, so i
raised a bug over there and they gave an updated solution for it by
not converting it in to the Float64 internally.

For more info, you can follow this link:
Precision Issue

Tensorflow - ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float)

TL;DR Several possible errors, most fixed with x = np.asarray(x).astype('float32').

Others may be faulty data preprocessing; ensure everything is properly formatted (categoricals, nans, strings, etc). Below shows what the model expects:

[print(i.shape, i.dtype) for i in model.inputs]
[print(o.shape, o.dtype) for o in model.outputs]
[print(l.name, l.input_shape, l.dtype) for l in model.layers]

---

The problem's rooted in using lists as inputs, as opposed to Numpy arrays; Keras/TF doesn't support former. A simple conversion is: x_array = np.asarray(x_list).

The next step's to ensure data is fed in expected format; for LSTM, that'd be a 3D tensor with dimensions (batch_size, timesteps, features) - or equivalently, (num_samples, timesteps, channels). Lastly, as a debug pro-tip, print ALL the shapes for your data. Code accomplishing all of the above, below:

Sequences = np.asarray(Sequences)
Targets   = np.asarray(Targets)
show_shapes()

Sequences = np.expand_dims(Sequences, -1)
Targets   = np.expand_dims(Targets, -1)
show_shapes()

# OUTPUTS
Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000)
Targets:   (200,)

Expected: (num_samples, timesteps, channels)
Sequences: (200, 1000, 1)
Targets:   (200, 1)

---

As a bonus tip, I notice you're running via main(), so your IDE probably lacks a Jupyter-like cell-based execution; I strongly recommend the Spyder IDE. It's as simple as adding # In[], and pressing Ctrl + Enter below:

---

Function used:

def show_shapes(): # can make yours to take inputs; this'll use local variable values
    print("Expected: (num_samples, timesteps, channels)")
    print("Sequences: {}".format(Sequences.shape))
    print("Targets:   {}".format(Targets.shape))   

---

Related Topics