How to Tell If Tensorflow Is Using Gpu Acceleration from Inside Python Shell

Am I really using GPU for tensorflow?

Check your nvidia-smi (nvidia system management interface) as the program runs, looking for Volatile GPU-util. The task manager is not a good indication of GPU usage (nor very accurate usages of other resources like RAM and temps imo...). The fact that your GPU temp is 71 degrees for a 3080 Ti indicates certainly the GPU is used (unless some other process is using it)

For instance, I am training right now with an RTX 3090 and my smi output from the command line looks like (truncating the processes from the screenshot):

But my task manager looks like (note the gpu usage):

Now if you have some sort of I/O bottleneck, i.e. loading of tensors from the CPU taking too long so the GPU sits idle, well that is a different issue, and can be solved by profiling and other tools for making sure that the loading process is optimized.

how do i find out if tensorflow uses the gpu under windows and python 3.6?

This code will confirm that tensorflow using GPU or CPU

sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

How to force tensorflow to forget about the GPU, after it has been found?

I don't understand why you need to let the TensorFlow forget. You have GPU that doesn't mean you have to use GPU.

You can use tf.device to specify the underlying device.

For example:

# Place tensors on the CPU
with tf.device('/CPU:0'):
  a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
  b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])

c = tf.matmul(a, b)
print(c)

So even though you have GPU, the program will still use CPU.

How to get AWS GPU instance details using Python-tensorflow

Since you said you installed Python yourself, it's likely you didn't start with a deep learning AMI with all the drivers installed, so you'd have to install Nvidia drivers, CUDA, and cudnn. But trying to install Nvidia drivers on an AWS EC2 instance can be tough...

Solution: start with the deep learning AMIs:

https://aws.amazon.com/machine-learning/amis/

---

Related Topics