Difference Between Variable and Get_Variable in Tensorflow

Difference between Variable and get_variable in TensorFlow

I'd recommend to always use tf.get_variable(...) -- it will make it way easier to refactor your code if you need to share variables at any time, e.g. in a multi-gpu setting (see the multi-gpu CIFAR example). There is no downside to it.

Pure tf.Variable is lower-level; at some point tf.get_variable() did not exist so some code still uses the low-level way.

Difference between get_variable and model_variable function?

As far as I know, yes. Looking at the source code for model_variable on github, model_variable wraps variable (defined in the linked module), which itself is a wrapper for get_variable.

This answer seems to confirm that, and this Google Groups discussion gives insight into why tf.contrib has seemingly duplicate functions, and specifically this function.

As for which one to use, probably always tf.get_variable unless you are integrating something with slim or something else that specifically calls for it. I've never used model_variable myself.

EDIT: Clarify that variable is defined in the link, not tf.Variable.

Tensorflow: How does tf.get_variable work?

tf.get_variable(name) creates a new variable called name (or add _ if name already exists in the current scope) in the tensorflow graph.

In your example, you're creating a python variable called var1.

The name of that variable in the tensorflow graph is not ** var1, but is Variable:0.

Every node you define has its own name that you can specify or let tensorflow give a default (and always different) one. You can see the name value accessing the name property of the python variable. (ie print(var1.name)).

On your second line, you're defining a Python variable var2 whose name in the tensorflow graph is var1.

The script

import tensorflow as tf

var1 = tf.Variable(3.,dtype=tf.float64)
print(var1.name)
var2 = tf.get_variable("var1",[],dtype=tf.float64)
print(var2.name)

In fact prints:

Variable:0
var1:0

If you, instead, want to define a variable (node) called var1 in the tensorflow graph and then getting a reference to that node, you cannot simply use tf.get_variable("var1"), because it will create a new different variable valled var1_1.

This script

var1 = tf.Variable(3.,dtype=tf.float64, name="var1")
print(var1.name)
var2 = tf.get_variable("var1",[],dtype=tf.float64)
print(var2.name)

prints:

var1:0
var1_1:0

If you want to create a reference to the node var1, you first:

1. Have to replace tf.Variable with tf.get_variable. The variables created with tf.Variable can't be shared, while the latter can.

2. Know what the scope of the var1 is and allow the reuse of that scope when declaring the reference.

Looking at the code is the better way for understanding

import tensorflow as tf

#var1 = tf.Variable(3.,dtype=tf.float64, name="var1")
var1 = tf.get_variable(initializer=tf.constant_initializer(3.), dtype=tf.float64, name="var1", shape=())
current_scope = tf.contrib.framework.get_name_scope()
print(var1.name)
with tf.variable_scope(current_scope, reuse=True):
    var2 = tf.get_variable("var1",[],dtype=tf.float64)
    print(var2.name)

outputs:

var1:0
var1:0

Different outcomes when using tf.Variable() and tf.get_variable()

tf.Variable accepts an initial value upon creation (a constant), this explains deterministic results when you use it.

tf.get_variable is slightly different: it has an initializer argument, by default None, which is interpreted like this:

If initializer is None (the default), the default initializer passed in the variable scope will be used. If that one is None too, a glorot_uniform_initializer will be used.

Since you didn't pass an initializer, the variable got uniform random initial value.

---

Related Topics