Sorrows mudlib: An event notification/subscription model, part 1

One of the most important design decisions for my MUD codebase, is to be as decoupled as possible. The codebase is reasonably clean as a result of this, but there are many systems that need to be revisited. Some are a legacy of earlier versions of the codebase, and others have not had a real use made of them that would drive the evolution of their design.

One of these systems, the one I will examine in this blog post, is a generic system to both subscribe to events, and to broadcast events to existing subscribers. As I am currently at the stage where I am ready to build other systems that make use of this lower-level one, I need to reexamine how it currently works and consider whether it does what is needed, or if I can do better.

The current implementation

Currently, an object that wants to listen to the OnServicesStarted event needs to define specific elements on its class.

class Object:
    __listenevents__ = [ "OnServicesStarted" ]

    def OnServicesStarted(self):
       pass

That is, for each event it wishes to receive, the name has to be present within a magic attribute. And of course, a handler function needs to be defined on the class with the same name as the matching event. Defining these elements does not provide automatic subscription however, the object still has to subscribe for the events it wishes to listen to.

class Object:
    def __init__(self):
        sorrows.services.AddEventListener(self)

The main downside of this approach is the boilerplate involved, where the name of the event needs to be declared and then a function of the same name defined. A lesser downside is that the services service does not seem like the most relevant place to go to for event registration.

This will be referred to as the magic attribute and handler method pattern.

Idea: An event service

Moving this functionality to a custom service seems a little cleaner. Objects would then call in the following way to subscribe:

    sorrows.events.AddListener(self)

But having this functionality in a custom service, means that any other service that wants to use it might need to declare the custom service as a dependency.

class GameLogicService(Service):
    __dependencies__ = [ "events" ]

This adds boilerplate and complication where it can otherwise be avoided. Chances are that most services would end up with a dependency on the events service. One advantage of having this system within the services service, was that it was guaranteed to be available for all services regardless of whether they had started fully yet or not.

This is a dead end.

Idea: An event handler

A way to avoid the dependency problem, is to move this functionality outside of the service infrastructure, perhaps as an independent singleton object. The following code pretty much wrote itself, following the conception of this idea.

class Event:
    registry = weakref.WeakValueDictionary()

    def __init__(self, eventName):
        self.name = eventName

    def Wait(self):
        c = self.registry.get(self.name, None)
        if c is None:
            c = self.registry[self.name] = stackless.channel()
        return c.receive()

This is a break from the magic attribute and handler method pattern. How it would be used highlights this.

class SomeObject:
    def __init__(self):
        # .. do some setup first
        stackless.tasklet(self.HandlePlayerLoginEvent)()

    def HandlePlayerLoginEvent(self):
        while True:
            user = Event("PlayerLogin").Wait()
            # .. do what we wanted to do to them

One of the advantages mooted about Stackless, or coroutines in general, is the ability to lessen boilerplate and to be able to write more readable code. This solution illustrates that advantage, although an obvious next step would be encapsulating the Event class instantiation and Wait method call into a simple function. It might be called WaitForEvent, taking the event name as an argument and blocking until it occurs, returning the relevant parameters.

def WaitForEvent(eventName):
    return Event(eventName).Wait()

At this stage, I am distracting myself with details. The goal is a cleaner event system, rather than the lesser details of how that system would be used. But before I move on and forget, one last thought.. a thought occurs that passing the event name as a string is clunky, and that it might be cleaner to be able to do eventHandler.eventName and have that implicitly do the blocking the Wait method from above does. But these things can be explored when I have decided on the best system to make use of them with.

The current idea is quite appealing, especially because as described above, it moves towards the ideal of more readable code rather than disjoint callback-based boilerplate. But after sleeping on it, I remembered the advantage of having events declared as data, rather than within code. Or what the last idea had over this one. My mudlib uses a code reloading framework, and if things like events are declared within code, the code reloading support cannot be extended to automatically correct things like putting in place new event subscriptions. What if SomeObject was extended to handle a second event?

class SomeObject:
    def __init__(self):
        # .. do some setup first
        stackless.tasklet(self.HandlePlayerLoginEvent)()
        stackless.tasklet(self.HandlePlayerLogoffEvent)()

When the script defining this code is modified, and reloaded, instances of this class may already be out there in use. Their __init__ method will have already run as its old version, and it is unreasonable to expect the code reloading system to compare the two and reconcile the differences. They will inherit the new HandlePlayerLogoffEvent (not shown above of course), but will not be registered for the event.

With the magic attribute and handler method pattern, it is trivial to subscribe and unsubscribe objects for events as entries in the magic method come and go. Maybe it is time to take a step back..

Design requirements

So having examined two approaches, I have two highlighted design requirements. They are listed in order of priority.

1. Integrated with code reloading support.
2. Avoid boilerplate and allow use of synchronous syntax.

Idea: Inline event declaration

The synchronous approach as shown above is very appealing, but it is unreasonable to infer event registrations from within code. The events that need to be subscribed to, or unsubscribed from, have to be declared. If we were to go with the synchronous approach, we need to adapt it in some way to do this.

One solution is wrapping event handlers with a decorator that declares what event it handles.

    @event("PlayerLogin")
    def HandlePlayerLogins(self):
        user = WaitForEvent()

In my experience with other people making use of decorators, they tend to be more of a pollution than a benefit. Of course, with restraint, they are fine used only where really suitable and actually needed.

Considering this solution, a minor downside is the duplication between the event name and the decorated function name. Or, given that the function name is not important, that this is a little clunky in the same way that the magic attribute and handler method pattern is. A cleaner variation to the solution would be to use the function name as the event name, and to have the decorator take no arguments. However, a larger downside is that the behaviour of decorators within the code reloading framework is undefined and as yet, undetermined.

Another solution is encoding a function name to indicate that it handles a given event. In the system currently in use we already have the __listenevents__ magic method, and of course Python itself has support for private methods (def __Func(self), so this is not necessarily a bad approach to go with. Perhaps something like.

    def _EVENT_PlayerLogin(self):

This is a little ugly, but there are a wealth of possibilities: _event_PlayerLogin, event_PlayerLogin, etc.. This is beginning to look acceptable to me.

An additional use case

The last described solution is all very well, but it is intended to provide for the case where desired events are declared and subscription is taken case of automatically. This will be the most common use case. However, it should still be possible to dynamically register for events.

The ability to dynamically register for events, allows flexible subsystems to be written that build on the underlying event system, by being able to register or unregister for arbitrary events as needed. It is easy enough to add an API to allow this, but it is worth considering how the code reloading system factors into this.

As I do not have a need for this functionality at this time, and it will not affect the design chosen to suit my current needs, I will hold off on designing this for now. But I expect it will make use of the following handy idiom:

    def Register(self, eventName, func):
        funcName = func.__name__
        obRef = weakref.ref(func.im_self)
        self.listeners[eventName].append((funcName, obRef))

    def Send(self, eventName, *args, **kwargs):
        for funcName, obRef in self.listeners[eventName]:
            ob = obRef()
            if ob is not None:
                func = getattr(ob, funcName)
                func(*args, **kwargs)

Conclusion

I will most likely go with the final solution I described. It is the simplest and cleanest from a usage point of view, and this is what is important to me. The event system would take care of enough to make a developer's work simpler and more straightforward, without abstracting away functionality needlessly.

Next post: Sorrows mudlib: An event notification/subscription model, part 2