How to Use Tensorflow Without Cuda on Linux

How can I use TensorFlow without CUDA on Linux?

If you build the binary with --config=cuda (as was done for tensorflow-gpu) then your machine must have GPU drivers. You can install GPU drivers on the machine even if the machine doesn't have GPU which is the common practical solution.

What happens is that --config=cuda sets GOOGLE_CUDA macro in the code which changes behavior during runtime. In particular it causes dso_loader to run which you can see by following line printed

2017-02-16 17:15:08: I tensorflow/stream_executor/dso_loader.cc:125] successfully opened CUDA library libcurand.8.0.dylib locally

A popular approach at Google is to deploy a "fat" binary -- ie binary that bundles drivers and client code for all possible hardware accelerators because that simplifies testing and deployment.

In open-source release, drivers and client code are separate, but this pattern remains -- the GPU-capable binary is expecting to have GPU drivers available.

How can I run tensorflow without GPU?

It should work. It mainly disables the CUDA device. So, the code looks for other sources (CPU) to run the code.

import os
import tensorflow as tf
#os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID" #If the line below doesn't work, uncomment this line (make sure to comment the line below); it should help.
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
#Your Code Here

Setup Tensorflow 2.4 on Ubuntu 20.04 with GPU without sudo

Instructions to setup Tensorflow 2.4.x (tested for 2.4.1) on an Ubuntu 20.04 environment without admin rights. It is assumed that a sysadmin already installed the latest Cuda drivers. It consists of install Cuda 11.0 toolkit + CuDNN 8.2.0.

Instructions below will install CUDA 11.0 (tested to work for Tensorflow 2.4.1) under directory /home/pherath/cuda_toolkits/cuda-11.0 without sudo rights.

Step 1. Download CUDA 11.0

wget http://developer.download.nvidia.com/compute/cuda/11.0.2/local_installers/cuda_11.0.2_450.51.05_linux.run
chmod +x cuda_11.0.2_450.51.05_linux.run

Step 2, Option 1: For a quick automatized form, use the following

./cuda_11.0.2_450.51.05_linux.run --silent --tmpdir=. --toolkit --toolkitpath=/home/pherath/cuda_toolkits/cuda-11.0

Step 2, Option 2: Here is a visual step-by-step guide

./cuda_11.0.2_450.51.05_linux.run

Continue, then accept the EULA.

Leave only Cuda Toolkit checked, uncheck everything else. Then go to Options.

Go into Toolkit Options.

Uncheck everything, then go to Change Toolkit Install Path and replace it with /home/pherath/cuda_toolkits/cuda-11.0 After this step, proceed with Install.

Step 3. Download CUDA 11.0 patch

wget https://developer.download.nvidia.com/compute/cuda/11.0.3/local_installers/cuda_11.0.3_450.51.06_linux.run
chmod +x cuda_11.0.3_450.51.06_linux.run

Step 4. Option 1: Quick and silent mode

./cuda_11.0.3_450.51.06_linux.run --silent --tmpdir=. --toolkit --toolkitpath=/home/pherath/cuda_toolkits/cuda-11.0

Step 4. Option 2: GUI mode
Repeat exact steps of Step 2, Option 2.

Installation might give an error.
When checking the logs, the error I saw suggests that there might be a bug in the installation script. The only offending term is the symbolic link of one file.

[ERROR]: boost::filesystem::create_symlink: File exists: "libcuinj64.so.11.0", "/home/pherath/cuda_toolkits/cuda-11.0/targets/x86_64-linux/lib/libcuinj64.so"

I came across several other single errors in various distribution attempts (e.g., on Ubuntu 16.04):

libcuinj64.so.11.0, libaccinj64.so.11.0, libnvrtc-builtins.so.11.0

This error can be fixed with the following 2 lines

cd /home/pherath/cuda_toolkits/cuda-11.0/targets/x86_64-linux/lib # move to the dir of the offending line
ln -s libaccinj64.so.11.0 libaccinj64.so #reorder such that symbolic link and target are in correct order (we need libaccinj64.so -> libaccinj64.so.11.0)

Step 5. Download CuDNN 8.2.0

cd /home/pherath/cuda_toolkits # move back to the parent of previous dir

You will need to download CuDNN .tgz file from CuDNN archives, I used v8.2.0. This step will require you to create an account at CuDNN and download through web interface. If you don’t have GUI on the machine you are setting up tensorflow, I suggest using "Link Redirect Trace" add-on to track the exact link the file would be downloaded from (here is a google chrome add-on link). You can trace the link using your local computer with GUI, then use wget to download the traced link on the VM. Note that there is a relatively short lifetime of this traced link.

After downloading, the name will be still encrypted, rename it back to .tgz by

mv $some_ambiguous_name cudnn-11.3-linux-x64-v8.2.0.53.tgz

Now we unzip at the parent of the cuda installation dir

tar -xvzf cudnn-11.3-linux-x64-v8.2.0.53.tgz # this will extract things under a dir called 'cuda'

Now we need to copy all lib64 and include to the corresponding dirs under cuda toolkit installation

cp -fv cuda/lib64/*.* cuda-11.0/lib64/.
cp -fv cuda/include/*.* cuda-11.0/include/.

Step 6. Create/append/prepend PATH and LD_LIBRARY_PATH environment variables.

Add the following lines to the end of your ~/.bashrc (otherwise, make sure to extend the corresponding environment vars for each bash you'll run your TF scripts from).

export CUDA11=/home/pherath/cuda_toolkits/cuda-11.0

export PATH=$CUDA11/bin:$PATH

export LD_LIBRARY_PATH=$CUDA11/lib64:$CUDA11/extras/CUPTI/lib64:$LD_LIBRARY_PATH

Start either new terminal or

source ~/.bashrc 

in each existing terminal.

Check if installation worked

You can run the following lines to test if TF 2.4.1 + profiler works:

conda create -n tf python==3.7 -y  # create a python environment
conda activate tf #activate the virtual environment (here conda)
pip install tensorflow==2.4.1 # install tf 2.4.1
python -c "import tensorflow as tf, logging; logging.basicConfig(level=logging.INFO, format='%(asctime)s %(message)s'); tf.config.list_physical_devices('GPU'); tf.profiler.experimental.start('.'); tf.profiler.experimental.stop()" # test to see if TF with GPU works

#########################################################################

If you want to instead install Cuda Toolkit 10.2 on Ubuntu 20.04 LTS, then the single liner installation code changes accordingly (need to add library_path, and override the complaint of mismatching gcc version).

./cuda_10.2.89_440.33.01_linux.run --silent --tmpdir=. --toolkit --toolkitpath=/home/pherath/cuda_toolkits/cuda-10.2 --librarypath=/home/pherath/cuda_toolkits/cuda-10.2 --override

Keep in mind that you need to repeat this process for also the patches of cuda toolkit 10.2. Afterwards, you would need to download corresponding cuDNN and copy lib64 & include into cuda toolkit's directory (same as instructions above).

#########################################################################

If still getting errors, there is a good chance that you don't have the right cuda/nvidia drivers installed. For fixing this, you will need sudo rights!

1.

First, purge all cuda/nvidia content (I cannot add reference due to limited reputation..); basically run the lines below with sudo rights.
apt clean; apt update; apt purge cuda; apt purge nvidia-*; apt autoremove; apt install cuda

2.

Follow instructions from google https://cloud.google.com/compute/docs/gpus/install-drivers-gpu#ubuntu-driver-steps

3.

Reboot the machine.

Why is Tensorflow not recognizing my GPU after conda install?

August 2021 Conda install may be working now, as according to @ComputerScientist in the comments below, conda install tensorflow-gpu==2.4.1 will give cudatoolkit-10.1.243 and cudnn-7.6.5

The following was written in Jan 2021 and is out of date

Currently conda install tensorflow-gpu installs tensorflow v2.3.0 and does NOT install the conda cudnn or cudatoolkit packages. Installing them manually (e.g. with conda install cudatoolkit=10.1) does not seem to fix the problem either.

A solution is to install an earlier version of tensorflow, which does install cudnn and cudatoolkit, then upgrade with pip

conda install tensorflow-gpu=2.1
pip install tensorflow-gpu==2.3.1

(2.4.0 uses cuda 11.0 and cudnn 8.0, however cudnn 8.0 is not in anaconda as of 16/12/2020)

Edit: please also see @GZ0's answer, which links to a github discussion with a one-line solution

How to run Tensorflow on CPU

You can apply device_count parameter per tf.Session:

config = tf.ConfigProto(
        device_count = {'GPU': 0}
    )
sess = tf.Session(config=config)

See also protobuf config file:

tensorflow/core/framework/config.proto

Install Tensorflow 2.x only for CPU using PIP

Issue solved after installing a CPU only version.

I used pin tensorflow-cpu and the version of the release. Somehow the fallback solution for CPU did not work in my setup.

How do I use TensorFlow GPU?

Follow this tutorial Tensorflow GPU I did it and it works perfect.

Attention! - install version 9.0! newer version is not supported by Tensorflow-gpu

Steps:

1. Uninstall your old tensorflow
2. Install tensorflow-gpu pip install tensorflow-gpu
3. Install Nvidia Graphics Card & Drivers (you probably already have)
4. Download & Install CUDA
5. Download & Install cuDNN
6. Verify by simple program

from tensorflow.python.client import device_lib 
print(device_lib.list_local_devices())

---

Related Topics