What are metaclasses in Python?
A metaclass is the class of a class. A class defines how an instance of the class (i.e. an object) behaves while a metaclass defines how a class behaves. A class is an instance of a metaclass.
While in Python you can use arbitrary callables for metaclasses (like Jerub shows), the better approach is to make it an actual class itself. type is the usual metaclass in Python. type is itself a class, and it is its own type. You won't be able to recreate something like type purely in Python, but Python cheats a little. To create your own metaclass in Python you really just want to subclass type.
A metaclass is most commonly used as a class-factory. When you create an object by calling the class, Python creates a new class (when it executes the 'class' statement) by calling the metaclass. Combined with the normal __init__ and __new__ methods, metaclasses therefore allow you to do 'extra things' when creating a class, like registering the new class with some registry or replace the class with something else entirely.
When the class statement is executed, Python first executes the body of the class statement as a normal block of code. The resulting namespace (a dict) holds the attributes of the class-to-be. The metaclass is determined by looking at the baseclasses of the class-to-be (metaclasses are inherited), at the __metaclass__ attribute of the class-to-be (if any) or the __metaclass__ global variable. The metaclass is then called with the name, bases and attributes of the class to instantiate it.
However, metaclasses actually define the type of a class, not just a factory for it, so you can do much more with them. You can, for instance, define normal methods on the metaclass. These metaclass-methods are like classmethods in that they can be called on the class without an instance, but they are also not like classmethods in that they cannot be called on an instance of the class. type.__subclasses__() is an example of a method on the type metaclass. You can also define the normal 'magic' methods, like __add__, __iter__ and __getattr__, to implement or change how the class behaves.
Here's an aggregated example of the bits and pieces:
def make_hook(f):
"""Decorator to turn 'foo' method into '__foo__'"""
f.is_hook = 1
return f
class MyType(type):
def __new__(mcls, name, bases, attrs):
if name.startswith('None'):
return None
# Go over attributes and see if they should be renamed.
newattrs = {}
for attrname, attrvalue in attrs.iteritems():
if getattr(attrvalue, 'is_hook', 0):
newattrs['__%s__' % attrname] = attrvalue
else:
newattrs[attrname] = attrvalue
return super(MyType, mcls).__new__(mcls, name, bases, newattrs)
def __init__(self, name, bases, attrs):
super(MyType, self).__init__(name, bases, attrs)
# classregistry.register(self, self.interfaces)
print "Would register class %s now." % self
def __add__(self, other):
class AutoClass(self, other):
pass
return AutoClass
# Alternatively, to autogenerate the classname as well as the class:
# return type(self.__name__ + other.__name__, (self, other), {})
def unregister(self):
# classregistry.unregister(self)
print "Would unregister class %s now." % self
class MyObject:
__metaclass__ = MyType
class NoneSample(MyObject):
pass
# Will print "NoneType None"
print type(NoneSample), repr(NoneSample)
class Example(MyObject):
def __init__(self, value):
self.value = value
@make_hook
def add(self, other):
return self.__class__(self.value + other.value)
# Will unregister the class
Example.unregister()
inst = Example(10)
# Will fail with an AttributeError
#inst.unregister()
print inst + inst
class Sibling(MyObject):
pass
ExampleSibling = Example + Sibling
# ExampleSibling is now a subclass of both Example and Sibling (with no
# content of its own) although it will believe it's called 'AutoClass'
print ExampleSibling
print ExampleSibling.__mro__
What are Python metaclasses useful for?
Metaclasses are indispensable if you want to have class objects (as opposed to instances of class objects) equipped with "special customized behavior", since an object's behavior depends on special methods on the type of the object, and a class object's type is, exactly a synonym for, the metaclass.
For example, if you want a class object X such that "print X" emits "Time is now 8:46am" (at 8:46 am, or, more generally, the current time) this must mean that type(x) (AKA X's metaclass) has a special custom __str__ method -- and similarly (with the various applicable special-methods) if you want to give meaning to expressions such as X + Y where X and Y are both class objects, or X[23] (where X, again, is a class object), and so forth.
Most other customization tasks are now (in Python 2.6 or better) easier to implement with a class decorator, which can alter a class object right after the end of the class statement. There are a few more cases where this is not feasible because the alterations must be made very early on if they are to have any effect (e.g., setting or altering __slots__).
In Python 3, metaclasses gain one extra little bit of usefulness: a metaclass can now optionally specify the mapping object to be populated during the execution of the class statement's body (by default, it's a normal dict). This allows the order of name bindings in the class body to be preserved and used (while the normal dict loses order), which is sometimes nice when the class must have "fields" in a certain specific order (e.g. to map 1:1 onto a C struct, a row in a CSV file or DB table, and the like) -- in Python 2.* this had to be redundantly specified (typically with an extra class attribute that's a sequence and thus does preserve order), and this feature of Python 3 metaclasses allows the redundancy to be removed.
What are the differences between a `classmethod` and a metaclass method?
As classes are instances of a metaclass, it is not unexpected that an "instance method" on the metaclass will behave like a classmethod.
However, yes, there are differences - and some of them are more than semantic:
- The most important difference is that a method in the metaclass is not "visible" from a class instance. That happens because the attribute lookup in Python (in a simplified way - descriptors may take precedence) search for an attribute in the instance - if it is not present in the instance, Python then looks in that instance's class, and then the search continues on the superclasses of the class, but not on the classes of the class. The Python stdlib make use of this feature in the
abc.ABCMeta.registermethod.
That feature can be used for good, as methods related with the class themselves are free to be re-used as instance attributes without any conflict (but a method would still conflict). - Another difference, though obvious, is that a method declared in the metaclass can be available in several classes, not otherwise related - if you have different class hierarchies, not related at all in what they deal with, but want some common functionality for all classes, you'd have to come up with a mixin class, that would have to be included as base in both hierarchies (say for including all classes in an application registry). (NB. the mixin may sometimes be a better call than a metaclass)
- A classmethod is a specialized "classmethod" object, while a method in the metaclass is an ordinary function.
So, it happens that the mechanism that classmethods use is the "descriptor protocol". While normal functions feature a __get__ method that will insert the self argument when they are retrieved from an instance, and leave that argument empty when retrieved from a class, a classmethod object have a different __get__, that will insert the class itself (the "owner") as the first parameter in both situations.
This makes no practical differences most of the time, but if you want access to the method as a function, for purposes of adding dynamically adding decorator to it, or any other, for a method in the metaclass meta.method retrieves the function, ready to be used, while you have to use cls.my_classmethod.__func__ to retrieve it from a classmethod (and then you have to create another classmethod object and assign it back, if you do some wrapping).
Basically, these are the 2 examples:
class M1(type):
def clsmethod1(cls):
pass
class CLS1(metaclass=M1):
pass
def runtime_wrap(cls, method_name, wrapper):
mcls = type(cls)
setattr(mcls, method_name, wrapper(getatttr(mcls, method_name)))
def wrapper(classmethod):
def new_method(cls):
print("wrapper called")
return classmethod(cls)
return new_method
runtime_wrap(cls1, "clsmethod1", wrapper)
class CLS2:
@classmethod
def classmethod2(cls):
pass
def runtime_wrap2(cls, method_name, wrapper):
setattr(cls, method_name, classmethod(
wrapper(getatttr(cls, method_name).__func__)
)
)
runtime_wrap2(cls1, "clsmethod1", wrapper)
In other words: apart from the important difference that a method defined in the metaclass is visible from the instance and a classmethod object do not, the other differences, at runtime will seem obscure and meaningless - but that happens because the language does not need to go out of its way with special rules for classmethods: Both ways of declaring a classmethod are possible, as a consequence from the language design - one, for the fact that a class is itself an object, and another, as a possibility among many, of the use of the descriptor protocol which allows one to specialize attribute access in an instance and in a class:
The classmethod builtin is defined in native code, but it could just be coded in pure python and would work in the exact same way. The 5 line class bellow can be used as a classmethod decorator with no runtime differences to the built-in @classmethod" at all (though distinguishable through introspection such as calls toisinstance, and evenrepr` of course):
class myclassmethod:
def __init__(self, func):
self.__func__ = func
def __get__(self, instance, owner):
return lambda *args, **kw: self.__func__(owner, *args, **kw)
And, beyond methods, it is interesting to keep in mind that specialized attributes such as a @property on the metaclass will work as specialized class attributes, just the same, with no surprising behavior at all.
How Meta classes works in python?
As put in the comments: Python metaclasses are different from django metaclasses: Django just, for historical reasons, use the same terminology for the inner class where one annotates extra parameters about a class, where the primary members of the outer class are meant to correspond to fields in a model or form.
A Python metaclass, on the other hand, are what you are describing in your example, though you have checked some Python 2 documentation. In current Python, the metaclass is determined by passing the keyword argument "metaclas=" in the declaration of a new class, where the base classes go:
class MyClass(Base1, Base2, metaclass=MyMeta):
...
As far as I know it, the Django behavior had origin in which early versions of Django actually used a custom (Python) metaclass to annotate some of the parameters now used in the nested Meta - and in doing so, it took a shortcut of defining the metaclass inline inside the class body: instead of assigning the __metaclass__ name to an externally defined metaclass, as the usual for normal use, it would just define the class inplace: from the point of view of the language runtime, it would find the name __metaclass__ bound to a valid metaclass and use that to build the class.
Later versions, even in Python 2, modified this approach - the inner class was no longer the actual "metaclass" of the Model or Form (as the previous approach was clearly overkill).
MetaClass in Python
Yes - the metaclass's __new__ and __init__ methods are called only when the class is created. After that, in your example, the class will be bound to theMultiply name. In many aspects, it is just an object like any other in Python. When you do objA = Multiply() you are not creating a new instance of type(Multiply), which is the metaclass - you are creating a new instance of Multiply itself: Multiply.__new__ and Multiply.__init__ are called.
Now, there is this: the mechanism in Python which make __new__ and __init__ be called when creating an instance is the code in the metaclass __call__ method. That is, just as when you create any class with a __call__ method and use an instance of it with the calling syntax obj() will invoke type(obj).__call__(obj), when you do Multiply() what is called (in this case) is Singleton.__call__(Multiply).
Since it is not implemented, Singleton's superclass, which is type __call__ method is called instead - and it is in there that Multiply.__new__ and __init__ are called.
That said, there is nothing in the code above that would make your classes behave as "singletons". And more importantly you don't need a metaclass to have a singleton in Python. I don't know who invented this thing, but it keeps circulating around.
First, if you really need a singleton, all you need to do is to write a plain class, no special anything, create your single instance, and document that the instance should be used. Just as people use None - no one keeps getting a reference to Nonetype and keep calling it to get a None reference:
class _Multiply:
...
# document that the code should use this instance:
Multiply = _Multiply()
second Alternatively, if your code have a need, whatsoever, for instantiating the class that should be a singleton where it will be used, you can use the class' __new__ method itself to control instantiation, no need for a metaclass:
class Multiply:
_instance = None
def __new__(cls):
if not cls._instance:
cls._instance = super().__new__(cls)
# insert any code that would go in `__init__` here:
...
...
return cls._instance
Third just for demonstration purposes, please don't use this, the metaclass mechanism to have singletons can be built in the __call__ method:
class Singleton(type):
registry = {}
def __new__(mcls,name,bases,attr):
print(f"name {name}")
print(f"bases {bases}")
print(f"attr {attr}")
print("Class created")
print ("Space Please")
return super(Singleton,mcls).__new__(cls,name,bases,attr)
def __call__(cls, *args, **kw):
registry = type(cls).registry
if cls not in registry:
print(f"{cls.__name__} being instantiated for the first time")
registry[cls] = super().__call__(*args, **kw)
else:
print(f"Attempting to create a new instance of {cls.__name__}. Returning single instance instead")
return registry[cls]
class Multiply(metaclass = Singleton):
pass
Inheritance from metaclasses in Python
When you do TestClass():, the body of the class is run in a namespace which becomes the class __dict__. The metaclass just informs the construction of that namespace via __new__ and __init__. In this case, you have set up the metaclass of TestClass to be type.
When you inherit from TestClass, e. g. with class NewTestClass(TestClass, PropertyConverter):, the version of PropertyConvertMetaclass you wrote operates on the __dict__ of NewTestClass only. TestClass has been created at that point, with no properties, because its metaclass way type, and the child class is empty, so you see no properties.
There are a couple of possible solutions here. The simpler one, but out of reach because of your assignment, is to do class TestClass(metaclass=PropertyConvertMetaclass):. All children of TestClass will have PropertyConvertMetaclass and so all getters will be converted to properties.
The alternative is to look carefully at the arguments of PropertyConvertMetaclass.__new__. Under normal circumstances, you only operate on the future_class_attr attribute. However, you have access to future_class_bases as well. If you want to upgrade the immediate siblings of PropertyConverter, that's all you need:
class PropertyConvertMetaclass(type):
def __new__(mcs, future_class_name, future_class_parents, future_class_attr):
# The loop is the same for each base __dict__ as for future_class_attr,
# so factor it out into a function
def update(d):
for name, value in d.items():
# Don't check for dunders: dunder can't start with `get_`
if name.startswith('get_') and callable(value):
prop = name[4:]
# Getter and setter can't be defined in separate classes
if 'set_' + prop in d and callable(d['set_' + prop]):
setter = d['set_' + prop]
else:
setter = None
if 'del_' + prop in d and callable(d['del_' + prop]):
deleter = d['del_' + prop]
else:
deleter = None
future_class_attr[prop] = property(getter, setter, deleter)
update(future_class_dict)
for base in future_class_parents:
# Won't work well with __slots__ or custom __getattr__
update(base.__dict__)
return super().__new__(mcs, future_class_name, future_class_parents, future_class_attr)
This is probably adequate for your assignment, but lacks a certain amount of finesse. Specifically, there are two deficiencies that I can see:
- There is no lookup beyond the immediate base classes.
- You can't define a getter in one class and a setter in another.
To address the first issue, you will have to traverse the MRO of the class. As @jsbueno suggests, this is easier to do on the fully constructed class using __init__ rather than the pre-class dictionary. I would solve the second issue by making a table of available getters and setters before making any properties. You could also make the properties respect MRO by doing this. The only complication with using __init__ is that you have to call setattr on the class rather than simply updating its future __dict__.
class PropertyConvertMetaclass(type):
def __init__(cls, class_name, class_parents, class_attr):
getters = set()
setters = set()
deleters = set()
for base in cls.__mro__:
for name, value in base.__dict__.items():
if name.startswith('get_') and callable(value):
getters.add(name[4:])
if name.startswith('set_') and callable(value):
setters.add(name[4:])
if name.startswith('del_') and callable(value):
deleters.add(name[4:])
for name in getters:
def getter(self, *args, **kwargs):
return getattr(super(cls, self), 'get_' + name)(*args, **kwargs)
if name in setters:
def setter(self, *args, **kwargs):
return getattr(super(cls, self), 'set_' + name)(*args, **kwargs)
else:
setter = None
if name in deleters:
def deleter(self, *args, **kwargs):
return getattr(super(cls, self), 'del_' + name)(*args, **kwargs)
else:
deleter = None
setattr(cls, name, property(getter, setter, deleter)
Anything that you do in the __init__ of a metaclass can just as easily be done with a class decorator. The main difference is that the metaclass will apply to all child classes, while a decorator only applies where it is used.
How does inheritance work in Python metaclass?
So --- a somewhat confusing question that can be answered,and somethat simplified
by simply running some examples in the interactive mode.
But to start, when you state:
type.__call__(...) in turn run two other methods (a __new__ and a __init__).
It is a simplification of what takes place.
When we create new class, like in resolving a class statement class A:, type.__call__ is invoked alright. But is this call is searched in the class of Meta itself. That is, the "metaclass" of "Meta" - which by default is type.
Bear with me:
When we are talking about an ordinary class E with no custom metaclass, and you create an instance by doing E() - Python searches for the __call__ method in the class of which E is an instance: that is, its metaclass. As it is type, then type.__call__ is called. It is type.__call__ which calls the __new__ and __init__ methods, as you stated, but not only for metaclasses: it orchestrates these calls in any object instantiation - the exact same mechanism is used in any object instantiation in Python: both ordinary objects and classes:
In [178]: class MetaMeta(type):
...: def __call__(metacls, *args, **kw):
...: print("Now at the meta-meta class")
...: return super().__call__(*args, **kw)
...:
In [179]: class EmptyMeta(type, metaclass=MetaMeta):
...: def __call__(cls, *args, **kw):
...: print("At the metaclass __call__")
...: return super().__call__(*args, **kw)
...:
...:
...:
In [180]: class A(metaclass=EmptyMeta):
...: pass
...:
Now at the meta-meta class
In [181]: a = A()
At the metaclass __call__
In [182]: class Direct(metaclass=MetaMeta): pass
In [183]: Direct()
Now at the meta-meta class
Out[183]: <__main__.Direct at 0x7fa66bc72c10>
So, in short: when creating a class A, which is an instance of Meta, the __call__ method of the class of Meta is called. That will call __init__ and __new__ in the metaclass Meta. If those are not defined, ordinary attribute lookup will call these methods in the superclass of Meta, which happens to also be "type".
Now, moving on on your question: when one inherits from a class with a custom metaclass, like your B class, Python takes the most derived metaclass in its superclasses as its own metaclass, not type. No need to explicitly declare a custom metaclass. That, in practical means, is what makes metaclass needed instead of just Class decorators: these affect only the class where they are declared, and have no effect in further subclasses.
In [184]: class B(A): pass
Now at the meta-meta class
In [185]: B()
At the metaclass __call__
Out[185]: <__main__.B at 0x7fa6682ab3a0>
In [186]: B.__class__
Out[186]: __main__.EmptyMeta
Even in an explicit call to type instead of the class statement, the derived class' metaclass will be the metaclass of the superclass. Note, however, that in this case we are hardcoding the call to the "metameta" class to type.__new__ and the "custom metaclass of the metaclass" is ignored:
In [187]: C = type("C", (A, ), {})
In [188]: C()
At the metaclass __call__
Out[188]: <__main__.C at 0x7fa653cb0d60>
If you want to programmaticaly create a class that has a custom "meta meta class" (God forbid one needing this in anything but learning purposes), there is a special call in the types module that does that:
In [192]: import types
In [193]: D = types.new_class("D", (A,), {})
Now at the meta-meta class
In [194]: D()
At the metaclass __call__
Out[194]: <__main__.D at 0x7fa6682959a0>
And to wrap it up, note that if the superclasses of a class have diverging metaclasses, Python will refuse to create a class at all. That is somewhat common in "real world" code, when people try to create Abstract classes (wich usea custom metaclass) with a base class in some framework with an ORM, which typically also have a custom metaclass:
In [203]: class Meta1(type): pass
In [204]: class Meta2(type): pass
In [205]: class A(metaclass=Meta1): pass
In [206]: class B(metaclass=Meta2): pass
In [207]: class C(A, B): pass
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-207-1def53cc27f4> in <module>
----> 1 class C(A, B): pass
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
Which is fixable by producing a derived metaclass that inherits from
the metaclasses in both ancestor branches (this requires that both metaclasses
are well behaved, using the super() instead of hardcoding calls to type - but
that is the case with well maintained and popular frameworks):
In [208]: class Meta3(Meta1, Meta2): pass
In [209]: class C(A, B, metaclass=Meta3): pass
In [210]:
Related Topics
Why Is Reading Lines from Stdin Much Slower in C++ Than Python
How to Get Linux Console Window Width in Python
How to Store the Result of an Executed Shell Command in a Variable in Python
List of Lists Changes Reflected Across Sublists Unexpectedly
Get the Cartesian Product of a Series of Lists
How to Use Threading in Python
Error "Microsoft Visual C++ 14.0 Is Required (Unable to Find Vcvarsall.Bat)"
How to Add Value Labels on a Bar Chart
Is There Go Up Line Character? (Opposite of \N)
Why Does Multiprocessing Use Only a Single Core After I Import Numpy
How to Kill a Python Child Process Created With Subprocess.Check_Output() When the Parent Dies
Call to Operating System to Open Url
Converting String "Jun 1 2005 1:33Pm" into Datetime
Why Does This Unboundlocalerror Occur (Closure)
Understanding Python Subprocess.Check_Output'S First Argument and Shell=True