Error Opening File in H5Py (File Signature Not Found)

OSError: Unable to open file (file signature not found) / End of HDF5 error back trace

I can reproduce the error message with:

In [88]: h5py.File('echo.py','r')                                                              
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-88-4c05cde6b6ff> in <module>
----> 1 h5py.File('echo.py','r')

/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py in __init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, **kwds)
    406                 fid = make_fid(name, mode, userblock_size,
    407                                fapl, fcpl=make_fcpl(track_order=track_order),
--> 408                                swmr=swmr)
    409 
    410             if isinstance(libver, tuple):

/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    171         if swmr and swmr_support:
    172             flags |= h5f.ACC_SWMR_READ
--> 173         fid = h5f.open(name, flags, fapl=fapl)
    174     elif mode == 'r+':
    175         fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5f.pyx in h5py.h5f.open()

OSError: Unable to open file (file signature not found)

With the downloaded link (a 5M file):

1614:~/mypy$ h5debug ../Downloads/data.hdf 
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139633948224384:
  #000: ../../../src/H5F.c line 579 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: ../../../src/H5Fint.c line 1208 in H5F_open(): unable to read superblock
    major: File accessibilty
    minor: Read failed
  #002: ../../../src/H5Fsuper.c line 273 in H5F__super_read(): file signature not found
    major: File accessibilty
    minor: Not an HDF5 file
cannot open file

Looks like the file is HDF4, not 5.

h5fromh4 -v ../Downloads/data.hdf 

makes a data.h5 file with one dataset "data"

In [3]: f = h5py.File('../Downloads/data.h5','r')                                              
In [4]: f                                                                                      
Out[4]: <HDF5 file "data.h5" (mode r+)>
In [5]: list(f.keys())                                                                         
Out[5]: ['data']
In [9]: f['data']                                                                              
Out[9]: <HDF5 dataset "data": shape (680, 451), type "<f8">

In hdfview I see the file is HDFEOS_V2.19

With pyhdf (and relevant HDF4 libraries) I can:

In [3]: from pyhdf.SD import SD, SDC                                                           
In [5]: f = SD('../Downloads/data.hdf', SDC.READ)                                              
In [6]: f.datasets()                                                                           
Out[6]: 
{'Longitude': (('Cell_Along_Swath:mod04', 'Cell_Across_Swath:mod04'),
  (680, 451),
  5,
  0),
...

And other datasets like those listed by HDFView.

HDF5 file created with h5py can't be opened by h5py

Since we resolved the issue in the comments on my question, I'm writing the results out here to mark it as solved.

The main problem was that I forgot to close the file after I created it. There would have been two simple options, either:

import numpy as np
import h5py

f = h5py.File('myfile.hdf5','w')
group = f.create_group('a_group')
group.create_dataset(name='matrix', data=np.zeros((10, 10)), chunks=True, compression='gzip')
f.close()

or, my favourite because the file is closed automatically:

import numpy as np
import h5py

with h5py.File('myfile.hdf5','w') as f:
    group = f.create_group('a_group')
    group.create_dataset(name='matrix', data=np.zeros((10, 10)), chunks=True, compression='gzip')

Related Topics