Saturday 30 January 2010

Tracking class instantiations

One of the experiments I have been doing based on code reloading, is to among other things, add the ability to react to instantiations of arbitrary classes. The way I go about doing this is to react to the definition of a class by giving it a new __init__ method.

If the class came with a custom __init__ method, then this is the easiest case to handle. When my replacement method is called, I can just call the original custom method before I notify whatever is interested.

    def init_wrapper(self, *args, **kwargs):
class_.__real_init__(self, *args, **kwargs)
events.Register(self)
What gets a little more complex, is the case when the class does not have a custom __init__ method defined. In this situation, I still need to notify whatever is interested, but in order to preserve existing behaviour I need to pass the call down to the base classes.
    def init_standin(self, *args, **kwargs):
if type(self) is types.InstanceType:
# Old-style..
for baseClass in class_.__bases__:
if hasattr(baseClass, "__init__"):
baseClass.__init__(self, *args, **kwargs)
else:
# New-style..
super(class_, self).__init__(*args, **kwargs)
events.Register(self)
The thing that really complicates this, is the class_ variable used in the both snippets included above. This needs to refer to the class that the given method being executed is defined on. I cannot infer the class from any of the existing variables, so it needs to be provided in some other way. Because I know it when I install the __init__ method, I define these functions inline so their use of it includes it in their closures.
    def MonitorClassInstantiation(self, class_):
def init_wrapper(self, *args, **kwargs):
class_.__real_init__(self, *args, **kwargs)
events.Register(self)

def init_standin(self, *args, **kwargs):
if type(self) is types.InstanceType:
for baseClass in class_.__bases__:
if hasattr(baseClass, "__init__"):
baseClass.__init__(self, *args, **kwargs)
else:
super(class_, self).__init__(*args, **kwargs)
events.Register(self)

init_real = class_.__dict__.get("__init__", None)
if init_real is None:
class_.__init__ = init_standin
else:
class_.__init__ = init_wrapper
class_.__real_init__ = init_real
With this working, the remaining concern is the effect of a hierarchy of classes modified in this way. Each has one of these two new __init__ methods, and the subclasses call down through them to the base classes. For now, I have a WeakKeyDictionary of instances in the events handler that ignores calls from instances that have already called.

Source code: events.py

Thursday 28 January 2010

Karaka trees

I live nearby Basque park, one of the many small parks littered around Auckland. People usually go there to walk their dogs, water their kids and sunbathe. There's a nice artistic fountain sculpture water feature thing in the center. And there's also this sign.

2010-01-28 - Karaka Trees - 02 - Basque park sign

Interestingly it mentions how the area was known for its karaka trees, which the maori would harvest the fruit from. Sounds quite nice.

2010-01-28 - Karaka Trees - 03 - Basque park sign

Hey, here's a fruit laden karaka tree in front of an apartment.

2010-01-28 - Karaka Trees - 04 - Apartment shrubbery

They're all over the place. There's an old one in a dog exercise area up Mount Eden. They are in front yards of houses.

2010-01-28 - Karaka Trees - 01 - Side street

Very tempting.. but wait!? WTF?

Eating the untreated kernel causes severe muscle cramps that can even rip the muscles off the bone


They don't mention that on the sign, do they? This pdf document gives a pretty good overview about the tree and its fruit.

Tuesday 26 January 2010

Griffin's Choc Krispie

Krispies are one of the better biscuits you can buy in New Zealand. A pleasing combination of sugar, flour and coconut, they are one of the first I go for if someone places an assortment of biscuits in front of me.

An unopened packet:

2010-01-26 - Griffins Choc Krispie - 01 - Unopened

An opened packet:

2010-01-26 - Griffins Choc Krispie - 02 - Opened

If I had to describe these biscuits, I'd have to wonder whether some plonker in an office thought "Krispies are nice, let's dip them in chocolate." I seem to recall reading how junk food companies add new brands and then phase them out on a regular basis as a way of doing business. This brand seems like it was designed for that purpose. Bring it out capitalising on the appeal of Krispies, consumers are tricked into buying and trying it, eventually the consumers have grown wise to the lack of worth and the product is phased out to be replaced. Probably replaced with the next new biscuit of similar worth, and the cycle repeats.

Griffin's Milk Chocolate Afghans

Featured in the standard New Zealand cookbook, the Edmonds cookbook, Afghans are a pretty standard home baked biscuit. I've been leaning towards making them for a while, but decided to buy a packet instead to see if I could save myself the trouble.

An unopened packet:

2010-01-26 - Griffins Milk Chocolate Afghans - 01 - Unopened

The opened packet:

2010-01-26 - Griffins Milk Chocolate Afghans - 02 - Opened

In a homemade afghan, you notice the cornflakes that went into them and the biscuit is made worthwhile by the half walnut that is mounted on top. These biscuits are a shallow approximation and are about as satisfying as a generic chocolate biscuit. If I wanted a decent chocolate biscuit, I'd buy a packet of hobnobs.

Home-made yogurt

One of my regular expenses was for sachets of yogurt. The standard approach to making your own yogurt in New Zealand, is to buy a yogurt maker and then to buy a sachet of yogurt powder for each batch you make from then on. I have the most well known and commonly available brand, which is Easiyo. The price for a sachet is around $3.00NZ.

So reading about people making their own yogurt from milk and a residual amount of their previous batch, sounded like it was worth trying for myself. Milk is quite expensive, reaching around $4.00NZ for 2 liters, but it is possible to find lesser known brands for as little as $2.50NZ.

The process is that the milk is first heated up, then cooled down, then an amount of the existing yogurt is mixed in and finally the mix is left to ferment or yogurtify or whatever the technical term is.

Heating the milk:

2010-01-25 - Home-made yogurt - 01 - Milk heating

The homemade yogurt:

2010-01-25 - Home-made yogurt - 02 - Yogurt

The first time I made it, I followed the process in the given link directly. The resulting yogurt was a consistent texture noticeably thicker than the milk that went into it. It still qualified as runny, being nearer the milk than the thickness that greek yogurt is expected to be. No whey separated either in the resulting batch, or in the yogurt as it sat in the fridge for a week.

I've made it twice since, and reading somewhere else that higher temperatures made for thicker yogurt, I've altered the process somewhat. The difference is that I heat the milk for half an hour longer and let it sit half an hour shorter. The yogurt that results from this is thicker, but is accompanied by a reasonable amount of whey. The yogurt looks almost like a round cheese within the whey.

I'd be worried about the separation but I've also seen it in Easiyo I have made, and from what have I read on the internet it is good for you. As I put the resulting yogurt back in the cleaned milk container, with the older yogurt adding to the volume I end up with a little more than fits in the container. So I discard a small amount of whey to leave me with approximately 2 liters, and then mix the remaining yogurt and whey before pouring it into the container.

It tastes pretty good. The yogurt from a sachet usually tastes slightly bitter, which I understand may be because of the milk powder that goes into it. The sachet yogurt is also made from tap water which can be heavily chlorinated or florinated or something and tastes like it. The home made yogurt is made from whatever milk the cows drunk, which does not.

Making yogurt this way makes twice the yogurt at under half the price. Definitely worth it, even if it does take over 14 hours to do.

Foraging nearby

Ever since I first watched the River Cottage television shows with Hugh Fearnley-Whittingstall, I have liked the idea of foraging for food.

A wild fennel plant growing in my backyard:

2010-01-25 - Foraging - 01 - Backyard

Several plants in an empty lot down the road:

2010-01-25 - Foraging - 02 - Empty yard down the road

In this photo there is a giant fennel plant. And below it some kind of berryfruit plant, probably blackberry. Not sure I would bother with the berries though, as this particlar place is where all the rubbish that blows down hill tends to end up. When the time comes I'm thinking of harvesting the fennel seeds and drying them. There are enough wild fennel plants nearby to make it worth my while, for personal use at least.

Foraging isn't without its dangers though. There's the warnings against roadside foraging, which would be the case for the plants shown in the second photo. This is due to heavy metals from car exhausts and street run-off. Then there is the danger of plant misidentification, for fennel for instance, it is common to read warnings against mistaking it with poisonous hemlock. Although as far as I can tell, that seems to be scaremongering which is copied and pasted as people read it elsewhere and write about it on their own web page or blog post.

Ernest Adams Rich Fruits Christmas Tarts

Christmas food is some of the best seasonal food, between cake and mincemeat tarts. Now that Christmas has passed, some of the leftover stock is being sold at a reduced price. At the supermarket the other week, I resisted buying a decent leftover Ernest Adams cake and instead went for Ernest Adams tarts.

Here's the packet unopened:

2010-01-22 - Ernest Adams Rich Fruits Christmas Tarts - 01 - Unopened

And broken tart innards:

2010-01-22 - Ernest Adams Rich Fruits Christmas Tarts - 02 - Opened

I would not buy them again. The taste was unremarkable, and was just a generic rich flavour. This is the sort of thing that would be best bought at a local non-chain bakery, but there are no decent bakeries around her.. just bakehouses. Whatever, a bakehouse is.

Code reloading and event notifications

A problem with the use of code reloading is reconciling the changes made to code with how existing and new uses of that code are affected. I've been experimenting with removing this problem for users of my code reloading framework, in this case in the implementation of an event subscription and notification system for my MUD framework.

Without code reloading

The use case is notifying instances of a class when an event they are interested in occurs. If code reloading is not in use, the following code might be one way for a user to indicate interest in, and to receive events.

Manual event registration:

class Object(BaseObject):
__events__ = BaseObject.__events__ + [ "OnSomethingHappened" ]

def __init__(self):
BaseObject.__init__(self)
events.Register(self)

def OnSomethingHappened(self):
pass
But in taking this approach, the user needs to keep several things in mind. That they do not clobber the events the base class may register for. That they list the function names of new events in the __events__ list on the class. That the base class may do the registration as well, so either they need to check and only register locally, or just register a second time if they cannot be bothered with it.

With code reloading

If code reloading is used, then the manual registration becomes a potential problem. Should an event registration call get added to the class sometime during the development process, existing instances will not receive the new events. And as changes are made over time and further events are added, which instances receive them will depend on whether they were created after the appropriate change. This is bad because it complicates the model of possible behaviours and problems that the user has to keep in their head. The way it should work, is such that any changes made to code are automatically applied to existing uses of that code, where it is reasonable to do so.

By having the framework react to the live addition of a class in a namespace, further live updates to that class and subsequent instantiations of it, it should be possible to build on these things to make event subscription a lot simpler for the user.

Automatic event registration:
class Object(BaseObject):
def event_SomethingHappened(self):
pass
Here the user simply needs to know that if they prefix a function name on their class with event_, any instances whether already existing or yet to be created, will be guaranteed to receive it.

Implementation

The main goal is that event delivery should be direct and not require any additional work to identify valid subscribers. All the work would instead be done when code reloading takes place. This would be achieved by processing classes that are created or updated and determining what events they implicitly require notification of.

Code reloading event registration:
    def OnClassCreation(namespace, className, class_):
pass

def OnClassUpdate(class_):
pass

cr = reloader.CodeReloader()
cr.SetClassCreationCallback(OnClassCreation)
cr.SetClassUpdateCallback(OnClassUpdate)
Existing instances would be unsubscribed from events where methods are removed, and subscribed to events where methods are added. This is a little complicated. It should be a lot easier to react to creation of new instances. But getting notified about the creation of new instances without requiring boilerplate on the part of the user is not straightforward.

One way would be to replace a method involved in the instantiation of instances of the class, like __new__ or __init__. __new__ can be ruled out, as it only works for new-style classes. __init__ seems to be the likeliest choice. With the events notifying of class creation and updates, injecting a method into a given class is a straightforward extension.

Monkey-patched __init__ method:
    def __init__(self, *args, **kwargs):
self.__real_init__(self, *args, **kwargs)
events.TrackAndRegisterInstance(self)
But maybe there is an easier way to hook into instance creation that I am not aware of.

Conclusion

It is very possible for the user to change code in a way that is incompatible with existing use of that code. But I think that it is a valuable goal not to try and increase the number of ways that this might happen.

Some of the problems are caused by the way Python is implemented, like where functions in mid-execution (particularly when using Stackless) use the code that was in place when they started executing, and continue using it until they exit. In this case a user needs to add boilerplate to ensure that changed code is adopted.

Loop boilerplate:
    def Loop(self):
while self.running:
self._Loop()
Sleep(1.0)

def _Loop(self):
pass # Actual loop logic
Other problems are caused by the user making changes that are incompatible with existing use, like modifying functions on a class to use a new variable that only gets initialised for new instances in the accompanying new version of __init__. To illustrate this, if the counter variable is added to __init__ and _Loop, then existing instances will cause an AttributeError on their next call to _Loop.

Problem case:
class SomeObject(BaseObject):
def __init__(self):
self.counter = 1
...

...

def _Loop(self):
self.counter += 1
...
I don't have a good solution for this. It is one of those things the user just has to remain aware of. And in around eight years of using another code reloading solution, I cannot say that I really recall making this mistake myself. The only way I can conceive of dealing with it, is by analysing code changes using the AST or some other method of introspection.

In fact, given the direction I am heading, this might be worth exploring. If it can be done in a way where it does not become a further burden on the user, then definitely so.