Defining Private Module Functions in Python

Defining private module functions in python

In Python, "privacy" depends on "consenting adults'" levels of agreement - you can't force it (any more than you can in real life;-). A single leading underscore means you're not supposed to access it "from the outside" -- two leading underscores (w/o trailing underscores) carry the message even more forcefully... but, in the end, it still depends on social convention and consensus: Python's introspection is forceful enough that you can't handcuff every other programmer in the world to respect your wishes.

((Btw, though it's a closely held secret, much the same holds for C++: with most compilers, a simple #define private public line before #includeing your .h file is all it takes for wily coders to make hash of your "privacy"...!-))

How do you correctly use private functions in Python?

Python doesn't quite have the concept of private functions. It does, however, treat class attributes whose names start with at least two underbars and end with at most one underbar a little specially - it mangles the names to make them slightly harder to access. In this example, you can see that the function __func2 has had its name mangled. It's still possible to access and call the function - but you have to make a special effort to do it, simply calling o.func2() fails:

james@bodacious:tmp$cat test.py

class myclass:

    def func1(self):
        print "one"

    def __func2(self):
        print "two"

o = myclass()

print dir(o)

o._myclass__func2()
o.func2()
james@bodacious:tmp$python test.py
['__doc__', '__module__', '_myclass__func2', 'func1']
two
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    o.func2()
AttributeError: myclass instance has no attribute 'func2'
james@bodacious:tmp$

So to answer the question you asked:

How do you correctly use private functions in Python?

The answer is: just like any other function, but you have to be aware of the mangled name.

Moving on to the question you wanted to ask:

AttributeError: matrix instance has no attribute '_matrix'

This is coming from line 154:

self._matrix.__loadVec(vec,res)

The error message is telling you that the object called self is an instance of class matrix; but it has no attribute called _matrix. Referring to the __savetoMatrix function above, it looks like the attribute is simply called matrix- so you need to refer to it as self.matrix ("the attribute called matrix of the object called self).

This The __savetoMatrix function references self.matrix rather than self._matrix.

However, there's a deeper problem here. Reading between the lines, it looks as though this code comes from a class called matrix; and instances of the class have an attribute called matrix. When you call self.matrix.__loadVec(), you're calling the function called __loadvec() which is bound to the attribute matrix bound to the object called self.

Even if this is what you wanted to do, this won't work because of name mangling as outlined above - assuming that the attribute called matrix has class inner_matrix, you'd have to call the function as self._matrix._inner_matrix__loadVec()

I think that what you're actually trying to do is call the method called __loadVec() defined in class matrix though. To do this, you just need to call self.__loadVec(). Because it's a call to a function within the same class, you don't even need to worry about the name mangling - these functions are intended to be used inside the class, and the interpreter will handle the mangling for you.

james@bodacious:tmp$cat test.py

class myclass:

    def func1(self):
        print "one"

    def __func2(self):
        print "two"

    def func3(self):
        self.__func2()
        print "three"

o = myclass()
print dir(o)
o.func3()
james@bodacious:tmp$python test.py
['__doc__', '__module__', '_myclass__func2', 'func1', 'func3']
two
three

Python private function coding convention

From Python's Class module documentation:

Private” instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a convention that is followed by most Python code: a name prefixed with an underscore (e.g. _spam) should be treated as a non-public part of the API (whether it is a function, a method or a data member). It should be considered an implementation detail and subject to change without notice.

Since there is a valid use-case for class-private members (namely to avoid name clashes of names with names defined by subclasses), there is limited support for such a mechanism, called name mangling. Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class.

Private (implementation) class in Python

Use a single underscore prefix:

class _Internal:
    ...

This is the official Python convention for 'internal' symbols; "from module import *" does not import underscore-prefixed objects.

Reference to the single underscore convention.

Dividing a module private functions into separate files Python

The short answer is that you cannot implement those methods in another file. You should consider whether or not those functions need to be broken down into smaller component functions if you believe the implementation code is so long that it affects readability and complexity.

If you refactor these long methods and break them down into their component behaviors you will increase readability and decrease complexity. This in turn will make your code more pythonic.

You're shouldn't try to force different language paradigms onto python -- instead, you should learn the pythonic way of handling certain problems/tasks.

Please refer to the zen of python (if you're unfamiliar):

~$ python
Python 2.7.13 (default, Dec 18 2016, 07:03:39)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>>

I'm guessing you're a C or C++ programmer. Python doesn't differentiate definitions from implementations. You should not be attempting to write this type of code in Python.

Python is an interpreted language, so it doesn't go through the compilation process that allows implementations to live in different files.

Compiled programs go through the following processes:

• Preprocessing
• compilation
• assembly
• linking

Interpreted programs are interpreted line by line.

---

An Aside on "Private" Class Members:

Furthermore, python doesn't strictly enforce the "private", "protected", or "public" scope of class attributes/methods. There are conventions for these scopes, however this is merely a convenience and is not strictly enforced.

Public:

• Any attribute that doesn't begin with a leading underscore.

Protected:

• Any attribute that begins with a single leading underscore.

Private:

• Any attribute that begins with two leading underscores.
• These attributes are mangled, so the original attribute name is changed. The attribute is still inherited.

Here's a sample program to demonstrate:

#!/usr/bin/env python
class BaseExample(object):
    def __init__(self):
        print "In base ctor"
        self.val = 1
        print "self.val: ", self.val
        self._val = 2
        print "self._val: ", self._val
        self.__val = 3
        print "self.__val: ", self.__val
        print '-'*50

    def print_private_attr(self):
        print "Printing private attribute: "
        print '\n','+'*50, '\n'
        print self.__val

class ExampleChild(BaseExample):
    def __init__(self):
        #self = Example()
        super(ExampleChild, self).__init__()
        print "In child ctor"
        print "self.val: ", self.val
        print "self._val: ", self._val
        print "self.__val: ", self._BaseExample__val
        print '-'*50

    def change_private_inherited_attr(self, val):
        print "Printing private attribute: ",self._BaseExample__val
        print '-'*50
        self._BaseExample__val = val
        print "Printing private attribute: ", self._BaseExample__val

print 'In global scope:'
print '\n','='*50, '\n'
example = BaseExample()
print '\n','-'*50, '\n'
example2 = ExampleChild()
print '\n','='*50, '\n'
print example.__dict__
print example2.__dict__
print '\n','='*50, '\n'
print 'In global scope:'
print 'example._BaseExample__val: ', example._BaseExample__val
print 'example2._BaseExample__val: ', example2._BaseExample__val
print 'Changing value of private attr:'
example2._BaseExample__val = 10
print 'In global scope:'
print 'example2._BaseExample__val: ', example2._BaseExample__val
example2.change_private_inherited_attr(100)
print 'In global scope:'
print 'example2._BaseExample__val: ', example2._BaseExample__val

Which has the following output:

In global scope:

==================================================

In base ctor
self.val:  1
self._val:  2
self.__val:  3
--------------------------------------------------

--------------------------------------------------

In base ctor
self.val:  1
self._val:  2
self.__val:  3
--------------------------------------------------
In child ctor
self.val:  1
self._val:  2
self.__val:  3
--------------------------------------------------

==================================================

{'_val': 2, '_BaseExample__val': 3, 'val': 1}
{'_val': 2, '_BaseExample__val': 3, 'val': 1}

==================================================

In global scope:
example._BaseExample__val:  3
example2._BaseExample__val:  3
Changing value of private attr:
In global scope:
example2._BaseExample__val:  10
Printing private attribute:  10
--------------------------------------------------
Printing private attribute:  100
In global scope:
example2._BaseExample__val:  100

Python: 'Private' module in a package

The solution I've settled on is to create a sub-package 'private' and place all the modules I wish to hide in there. This way they stay stowed away, leaving mypack's module list cleaner and easier to parse.

To me, this doesn't look unpythonic either.

Private methods in Python

Python doesn't have the concept of private methods or attributes. It's all about how you implement your class. But you can use pseudo-private variables (name mangling); any variable preceded by __(two underscores) becomes a pseudo-private variable.

From the documentation:

Since there is a valid use-case for class-private members (namely to
avoid name clashes of names with names defined by subclasses), there
is limited support for such a mechanism, called name mangling. Any
identifier of the form __spam (at least two leading underscores, at
most one trailing underscore) is textually replaced with
_classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard
to the syntactic position of the identifier, as long as it occurs
within the definition of a class.

class A:
    def __private(self):
       pass

So __private now actually becomes _A__private.

Example of a static method:

>>> class A:
...     @staticmethod         # Not required in Python 3.x
...     def __private():
...         print 'hello'
...
>>> A._A__private()
hello

Does defining a non-class private function means anything in Python?

The single leading underscore is a Python naming convention. A number of tools rely on that convention. For example help() will ignore single underscored names. Using from somemodule import * will also ignore single underscored names.

The double leading underscore triggers name mangling (prefixing the name with the class name and a single leading underscore). It is used to create thread-local references. That allows intra-class calls that won't be accidentally broken by a subclass. There is a nice example of this in the tutorial at http://docs.python.org/tutorial/classes.html#private-variables .

Lastly, there is the special method naming convention of two leading underscores and two trailing underscores. Those names are used by the interpreter to implement operators and to implement a number of standard protocols (i.e. iterators, context managers, pickling, etc). See http://www.rafekettler.com/magicmethods.html for a nice write-up on the special methods.

Why are Python's 'private' methods not actually private?

The name scrambling is used to ensure that subclasses don't accidentally override the private methods and attributes of their superclasses. It's not designed to prevent deliberate access from outside.

For example:

>>> class Foo(object):
...     def __init__(self):
...         self.__baz = 42
...     def foo(self):
...         print self.__baz
...     
>>> class Bar(Foo):
...     def __init__(self):
...         super(Bar, self).__init__()
...         self.__baz = 21
...     def bar(self):
...         print self.__baz
...
>>> x = Bar()
>>> x.foo()
42
>>> x.bar()
21
>>> print x.__dict__
{'_Bar__baz': 21, '_Foo__baz': 42}

Of course, it breaks down if two different classes have the same name.

---

Related Topics