An actually kind of practical use for metaclass hackery in Python
September 13, 2010
I should begin by saying that my example here isn’t at all sophisticated. In fact it’s quite simple and not even all that metaclass-y. But the result I wanted was simple so it’s nice (and very Pythonic) that I can achieve it in a simple way.
It has been said that 99% of programmers will have no use for metaclasses in Python. Metaclasses have been available in Python since 1998 and easily accessible since 2001. And until now I haven’t felt the need to go anywhere near one. All the examples I had seen seemed either contrived or at least thoroughly unlike anything I was likely to want to do any time soon. A Python class is a splendidly simple thing. A need to mess around with its creation is vanishingly rare.
Let’s begin with what I want to be able to do.
When I’m writing Python programs, I often find myself passing around containers holding a few pieces of related information. For the purpose of this example, let’s assume that the related information we want to pass around is a kind of fruit and its price. But you can think of many more: people and their addresses; dates, times, and the events that take place then; and so on.
There are plenty of ways to group related information in Python. I could put the items in a list:
>>> l=["banana",1.5]
or a tuple:
>>> t=("mango",1.3)
But, conceptually, the two values have names so it seems better to use something that allows me to use their names. Perhaps a dict:
>>> d={"fruit": "pomegranate", "price": 2.0}
or a bare-bones class:
>>> class f1:
... pass
...
>>> o1=f1()
>>> o1.fruit="orange"
>>> o1.price=1.75
The only problem with those last two (and it’s not that big a problem) is that they’re vulnerable to typos:
>>> o2=f1()
>>> o2.friut="lemon"
>>> o2.pirce=1.3
Depending on how good your code tests are and how good your lint-like tools are, errors of that sort may be caught easily or they may not.
If I’m writing a program that’s pretty big and therefore likely to have a typo in it somewhere, I’d be glad to have some sort of protection against that sort of mistake. Computers are supposed to make life more convenient, after all.
And, happily, Python provides that sort of protection in the form of the __slots__ attribute of a slightly less bare-bones class:
>>> class f2(object):
... __slots__=("fruit","price")
...
Python will complain if I try to get at an instance’s attribute that’s not listed in __slots__:
>>> o3=f2()
>>> o3.fruit="papaya"
>>> o3.pirce=1.8
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'f2' object has no attribute 'pirce'
That’s all well and good and I’ve done that sort of thing any number of times. But in a pretty-big program, I’m likely to want two or three classes like that. And I’m likely to want to elaborate the classes at least a bit more. A __str__() function that prints the attributes is helpful in debugging, for example. So the top of my program is going to have several classes that are very similar except in the __slots__ attribute. That’s hardly a big deal, but to my mind it’s not very Pythonic. It’s one of Python’s virtues in my opinion that Python programs look like they’re doing what they are doing. If I wanted to write programs that required a lot of extra machinery, I could write in C.
Unfortunately, assigning to __slots__ from the method __init__() does’t work. It has to be a class attribute and has to be created at class-creation time. Wouldn’t it be convenient if there were a way to gin up nearly identical classes at runtime? Wait, that’s what Python’s metaclass machinery does. And for our purposes it’s really simple. We just need to call type.__new__() with the right parameters. That’s all there is to creating a class.
So here’s a function that creates classes that have custom __slots__ attributes:
def makeClass(slots):
# str() function for the classes created, so that they
# print in a useful way. Doesn't have to be defined here.
def str(self):
contentsStr=""
for slot in self.__slots__:
if not hasattr(self,slot):
contentsStr+=slot+" not set, "
else:
contentsStr+=slot+"="+repr(getattr(self,slot))+", "
contentsStr=contentsStr[:-2]
return self.__class__.__name__+" with slots: "+ \
contentsStr
# This is all there is to it
return type.__new__(type,"auto-generated-class",(),
{"__slots__":slots,"__str__": str})
And if I put some test code:
c1=makeClass(["a","b"])
c2=makeClass(["c","d"])
o1=c1()
print o1
o1.a=42
o1.b=1
print "o1.a",o1.a
print "o1.b",o1.b
print "o1",o1
o2=c2()
o2.c=3
o2.d=4
print "o2",o2
o1.c=3 # Should produce AttributeError
after that function in a file and run it, I get what I’d expect:
$ python test.py
auto-generated-class with slots: a not set, b not set
o1.a 42
o1.b 1
o1 auto-generated-class with slots: a=42, b=1
o2 auto-generated-class with slots: c=3, d=4
Traceback (most recent call last):
File "test.py", line 35, in <module>
o1.c=3 # Should produce AttributeError
AttributeError: 'auto-generated-class' object has no attribute 'c'
Pretty easy once it’s clear what type.__new__() does.