Link Atlas/Mkl to an Installed Numpy

Link ATLAS/MKL to an installed Numpy

Assuming you're running some flavour of linux, here's one way you could do it:

1. Find out what BLAS library numpy is currently linked against using ldd.

   • For versions of numpy older than v1.10:

     $ ldd /<path_to_site-packages>/numpy/core/_dotblas.so
     

     For example, if I install numpy via apt-get, it links to

     ...
     libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fed81de8000)
     ...
     

     If _dotblas.so doesn't exist, this probably means that numpy failed to detect any BLAS libraries when it was originally installed, in which case it simply doesn't build any of the BLAS-dependent components. This often happens if you install numpy using pip without manually specifying a BLAS library (see below). I'm afraid you'll have no option but to rebuild numpy if you want to link against an external BLAS library.

   ---

   • For numpy v1.10 and newer:

     _dotblas.so has been removed from recent versions of numpy, but you should be able to check the dependencies of multiarray.so instead:

     $ ldd /<path_to_site-packages>/numpy/core/multiarray.so
     

2. Install ATLAS/MKL/OpenBLAS if you haven't already. By the way, I would definitely recommend OpenBLAS over ATLAS - take a look at this answer (although the benchmarking data is now probably a bit out of date).

3. Use update-alternatives to create a symlink to the new BLAS library of your choice. For example, if you installed libopenblas.so into /opt/OpenBLAS/lib, you would do:

   $ sudo update-alternatives --install /usr/lib/libblas.so.3 \
                                        libblas.so.3 \
                                        /opt/OpenBLAS/lib/libopenblas.so \
                                        50
   

   You can have multiple symlinks configured for a single target library, allowing you to manually switch between multiple installed BLAS libraries.

   For example, when I call $ sudo update-alternatives --config libblas.so.3, I can choose between one of 3 libraries:

     Selection    Path                                    Priority   Status
   ------------------------------------------------------------
     0            /opt/OpenBLAS/lib/libopenblas.so         40        auto mode
     1            /opt/OpenBLAS/lib/libopenblas.so         40        manual mode
     2            /usr/lib/atlas-base/atlas/libblas.so.3   35        manual mode
   * 3            /usr/lib/libblas/libblas.so.3            10        manual mode
   

If you really want the "newest" version of numpy, you could also take a look at my answer on compiling numpy from source with OpenBLAS integration.

Installing numpy with BLAS support using pip

As @tndoan mentioned in the comments, it's possible to make pip respect a particular configuration for numpy by placing a config file in ~/.numpy-site.cfg - see this answer for more details.

My personal preference is to configure and build numpy by hand. It's not particularly difficult, and it gives you better control over numpy's configuration.

sudo pip3 install numpy does not respect ~/.numpy-site.cfg

It turns out that my guess was indeed correct. sudo isn't using the right $HOME. sudo echo $HOME worked because bash expanded $HOME before calling sudo to run the command.

The following test did the trick though:

# In test.sh
echo "$HOME"

And now I get

$ sudo bash test.sh
/root

which confirms that $HOME is incorrect. It turns out that a bunch of settings had been set in /etc/sudoers (always_set_home and env_reset), which meant that sudo -E bash test.sh had no effect either.

I finally just installed it with

$ sudo HOME=/path/to/my/home pip3 install numpy

which worked.

Compile numpy WITHOUT Intel MKL/BLAS/ATLAS/LAPACK

According to the official documentation:

Disabling ATLAS and other accelerated libraries

Usage of ATLAS and other accelerated libraries in Numpy can be
disabled via:

  BLAS=None LAPACK=None ATLAS=None python setup.py build

However, this information seems to be out of date, since I found that even with these options numpy v1.9.2 was still automatically finding libopenblas.so:

numpy_source_dir/$ BLAS=None LAPACK=None ATLAS=None python setup.py config
...
openblas_info:
  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/OpenBLAS/lib']
    language = f77

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/OpenBLAS/lib']
    language = f77
...

One workaround is to copy site.cfg.example to site.cfg, then edit it to make the paths to the relevant BLAS/LAPACK libraries invalid:

[openblas]
libraries =
library_dirs =
include_dirs =

When you subsequently call BLAS=None LAPACK=None ATLAS=None python setup.py config you should get an output containing this:

...
openblas_info:
/home/alistair/src/python/numpy/numpy/distutils/system_info.py:594: UserWarning: Specified path  is invalid.
  warnings.warn('Specified path %s is invalid.' % d)
  libraries  not found in []
  NOT AVAILABLE
...

I expect that the same approach will work for ATLAS and MKL, although I don't have these libraries installed in order to do a proper test.

You should, of course, be aware that not having accelerated BLAS/LAPACK libraries will have a big detrimental effect on performance for linear algebra ops.

---

Update

As mentioned in the comments below, you didn't actually "compile" your current version of numpy, but rather installed it from a binary distribution. The approach I gave above would require you to build numpy from source, which is not an easy thing to do in Windows (although there are official instructions here).

A much easier option would be to install one of the unoptimized numpy binaries available from Christoph Gohlke's website here.

---

Related Topics