Building a Minimal Plugin Architecture in Python

Building a minimal plugin architecture in Python

Mine is, basically, a directory called "plugins" which the main app can poll and then use imp.load_module to pick up files, look for a well-known entry point possibly with module-level config params, and go from there. I use file-monitoring stuff for a certain amount of dynamism in which plugins are active, but that's a nice-to-have.

Of course, any requirement that comes along saying "I don't need [big, complicated thing] X; I just want something lightweight" runs the risk of re-implementing X one discovered requirement at a time. But that's not to say you can't have some fun doing it anyway :)

Implementing a Plugin System in Python

Having written quite a few different plugin architectures in different platforms and languages, I will say that you are pretty much on track with how most plugins systems are written.

Basically what it boils down to, is your host and your plugin need to have some kind of common contract to work with; in other words, your host needs to know enough about your plugin that it can pass along or share whatever common resources the plugin will need, and the plugin needs to know enough about the host to interact with those resources.

Implementing this using a base class and a derived class as you have done is a very common methodology.

How To Create a Flexible Plug-In Architecture?

This is not an answer as much as a bunch of potentially useful remarks/examples.

  • One effective way to make your application extensible is to expose its internals as a scripting language and write all the top level stuff in that language. This makes it quite modifiable and practically future proof (if your primitives are well chosen and implemented). A success story of this kind of thing is Emacs. I prefer this to the eclipse style plugin system because if I want to extend functionality, I don't have to learn the API and write/compile a separate plugin. I can write a 3 line snippet in the current buffer itself, evaluate it and use it. Very smooth learning curve and very pleasing results.

  • One application which I've extended a little is Trac. It has a component architecture which in this situation means that tasks are delegated to modules that advertise extension points. You can then implement other components which would fit into these points and change the flow. It's a little like Kalkie's suggestion above.

  • Another one that's good is py.test. It follows the "best API is no API" philosophy and relies purely on hooks being called at every level. You can override these hooks in files/functions named according to a convention and alter the behaviour. You can see the list of plugins on the site to see how quickly/easily they can be implemented.

A few general points.

  • Try to keep your non-extensible/non-user-modifiable core as small as possible. Delegate everything you can to a higher layer so that the extensibility increases. Less stuff to correct in the core then in case of bad choices.
  • Related to the above point is that you shouldn't make too many decisions about the direction of your project at the outset. Implement the smallest needed subset and then start writing plugins.
  • If you are embedding a scripting language, make sure it's a full one in which you can write general programs and not a toy language just for your application.
  • Reduce boilerplate as much as you can. Don't bother with subclassing, complex APIs, plugin registration and stuff like that. Try to keep it simple so that it's easy and not just possible to extend. This will let your plugin API be used more and will encourage end users to write plugins. Not just plugin developers. py.test does this well. Eclipse as far as I know, does not.

Plugin architecture - Plugin Manager vs inspecting from plugins import *

The metaclass approach is useful for this issue in Python < 3.6 (see @quasoft's answer for Python 3.6+). It is very simple and acts automatically on any imported module. In addition, complex logic can be applied to plugin registration with very little effort. It requires:

The metaclass approach works like the following:

1) A custom PluginMount metaclass is defined which maintains a list of all plugins

2) A Plugin class is defined which sets PluginMount as its metaclass

3) When an object deriving from Plugin - for instance MyPlugin is imported, it triggers the __init__ method on the metaclass. This registers the plugin and performs any application specific logic and event subscription.

Alternatively if you put the PluginMount.__init__ logic in PluginMount.__new__ it is called whenver a new instance of a Plugin derived class is created.

class PluginMount(type):
"""
A plugin mount point derived from:
http://martyalchin.com/2008/jan/10/simple-plugin-framework/
Acts as a metaclass which creates anything inheriting from Plugin
"""

def __init__(cls, name, bases, attrs):
"""Called when a Plugin derived class is imported"""

if not hasattr(cls, 'plugins'):
# Called when the metaclass is first instantiated
cls.plugins = []
else:
# Called when a plugin class is imported
cls.register_plugin(cls)

def register_plugin(cls, plugin):
"""Add the plugin to the plugin list and perform any registration logic"""

# create a plugin instance and store it
# optionally you could just store the plugin class and lazily instantiate
instance = plugin()

# save the plugin reference
cls.plugins.append(instance)

# apply plugin logic - in this case connect the plugin to blinker signals
# this must be defined in the derived class
instance.register_signals()

Then a base plugin class which looks like:

class Plugin(object):
"""A plugin which must provide a register_signals() method"""
__metaclass__ = PluginMount

Finally, an actual plugin class would look like the following:

class MyPlugin(Plugin):
def register_signals(self):
print "Class created and registering signals"

def other_plugin_stuff(self):
print "I can do other plugin stuff"

Plugins can be accessed from any python module that has imported Plugin:

for plugin in Plugin.plugins:
plugin.other_plugin_stuff()

See the full working example

Seeking advice about plugin architecture frameworks for master's project

For architecture I think you should follow micro-service arch
and yes every language have their limitation then when you create service for high performance then use rust or c++ and when you want to create high performance api then create golang service and deploy them on cloud and kubernate.

For framework, if you still confused what framework is suite you then follow clean architecture that will help you when create service
ex. you create golang service with gin framework you can switch to fiber framework very easy.

ref.
https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html
https://github.com/bxcodec/go-clean-arch



Related Topics



Leave a reply



Submit