Code reloading and event notifications

A problem with the use of code reloading is reconciling the changes made to code with how existing and new uses of that code are affected. I've been experimenting with removing this problem for users of my code reloading framework, in this case in the implementation of an event subscription and notification system for my MUD framework.

Without code reloading

The use case is notifying instances of a class when an event they are interested in occurs. If code reloading is not in use, the following code might be one way for a user to indicate interest in, and to receive events.

Manual event registration:

class Object(BaseObject):
    __events__ = BaseObject.__events__ + [ "OnSomethingHappened" ]

    def __init__(self):
        BaseObject.__init__(self)
        events.Register(self)

    def OnSomethingHappened(self):
        pass

But in taking this approach, the user needs to keep several things in mind. That they do not clobber the events the base class may register for. That they list the function names of new events in the __events__ list on the class. That the base class may do the registration as well, so either they need to check and only register locally, or just register a second time if they cannot be bothered with it.

With code reloading

If code reloading is used, then the manual registration becomes a potential problem. Should an event registration call get added to the class sometime during the development process, existing instances will not receive the new events. And as changes are made over time and further events are added, which instances receive them will depend on whether they were created after the appropriate change. This is bad because it complicates the model of possible behaviours and problems that the user has to keep in their head. The way it should work, is such that any changes made to code are automatically applied to existing uses of that code, where it is reasonable to do so.

By having the framework react to the live addition of a class in a namespace, further live updates to that class and subsequent instantiations of it, it should be possible to build on these things to make event subscription a lot simpler for the user.

Automatic event registration:

class Object(BaseObject):
    def event_SomethingHappened(self):
        pass

Here the user simply needs to know that if they prefix a function name on their class with event_, any instances whether already existing or yet to be created, will be guaranteed to receive it.

Implementation

The main goal is that event delivery should be direct and not require any additional work to identify valid subscribers. All the work would instead be done when code reloading takes place. This would be achieved by processing classes that are created or updated and determining what events they implicitly require notification of.

Code reloading event registration:

    def OnClassCreation(namespace, className, class_):
        pass

    def OnClassUpdate(class_):
        pass

    cr = reloader.CodeReloader()
    cr.SetClassCreationCallback(OnClassCreation)
    cr.SetClassUpdateCallback(OnClassUpdate)

Existing instances would be unsubscribed from events where methods are removed, and subscribed to events where methods are added. This is a little complicated. It should be a lot easier to react to creation of new instances. But getting notified about the creation of new instances without requiring boilerplate on the part of the user is not straightforward.

One way would be to replace a method involved in the instantiation of instances of the class, like __new__ or __init__. __new__ can be ruled out, as it only works for new-style classes. __init__ seems to be the likeliest choice. With the events notifying of class creation and updates, injecting a method into a given class is a straightforward extension.

Monkey-patched __init__ method:

    def __init__(self, *args, **kwargs):
        self.__real_init__(self, *args, **kwargs)
        events.TrackAndRegisterInstance(self)

But maybe there is an easier way to hook into instance creation that I am not aware of.

Conclusion

It is very possible for the user to change code in a way that is incompatible with existing use of that code. But I think that it is a valuable goal not to try and increase the number of ways that this might happen.

Some of the problems are caused by the way Python is implemented, like where functions in mid-execution (particularly when using Stackless) use the code that was in place when they started executing, and continue using it until they exit. In this case a user needs to add boilerplate to ensure that changed code is adopted.

Loop boilerplate:

    def Loop(self):
        while self.running:
            self._Loop()
            Sleep(1.0)

    def _Loop(self):
        pass # Actual loop logic

Other problems are caused by the user making changes that are incompatible with existing use, like modifying functions on a class to use a new variable that only gets initialised for new instances in the accompanying new version of __init__. To illustrate this, if the counter variable is added to __init__ and _Loop, then existing instances will cause an AttributeError on their next call to _Loop.

Problem case:

class SomeObject(BaseObject):
    def __init__(self):
        self.counter = 1
        ...

    ...

    def _Loop(self):
        self.counter += 1
        ...

I don't have a good solution for this. It is one of those things the user just has to remain aware of. And in around eight years of using another code reloading solution, I cannot say that I really recall making this mistake myself. The only way I can conceive of dealing with it, is by analysing code changes using the AST or some other method of introspection.

In fact, given the direction I am heading, this might be worth exploring. If it can be done in a way where it does not become a further burden on the user, then definitely so.