Tuesday 7 April 2009

Python standard library: pickletools

There's a crash bug in the in progress Stackless Python 3.1a2 release, but only on Linux. The particular use case, is where a tasklet is run for a very short time before being interrupted, pickled at its current point of execution, unpickled to a new version of the tasklet and then run. However, there appears to be some corruption in the pickling, which results in a recreated tasklet that appears to loop forever.

Now, the first step is to understand what pickle output. In the past, I've broken down the pickle strings into a readable form by hand. This time, I googled for "disassembling python pickle dump" and one of the first hits was pickletools. Now, this code appears to be somewhere within a Sourceforge repository, and I am too lazy to track it down, so I googled for "pickletools" and found it was now in the standard library for Python 2.6. Bingo!

Trying it out gives a dump which looks like this:

>>> pickletools.dis(s)
0: \x80 PROTO 3
2: c GLOBAL 'stackless tasklet'
21: q BINPUT 0
23: ) EMPTY_TUPLE
24: R REDUCE
25: q BINPUT 1
27: ( MARK
28: K BININT1 8
30: N NONE
31: K BININT1 0
33: ] EMPTY_LIST
34: q BINPUT 2
36: c GLOBAL 'stackless._wrap frame'
59: q BINPUT 3
61: c GLOBAL 'stackless._wrap code'
83: q BINPUT 4
85: ( MARK
...
Disassembling both the Linux and Windows pickle dumps of the tasklet, gives me the point at which they differ.

Windows:
  ...
400: N NONE
401: N NONE
402: J BININT -1
407: K BININT1 4
409: ) EMPTY_TUPLE
...
Linux:
  ...
400: N NONE
401: N NONE
402: K BININT1 31
404: K BININT1 4
406: K BININT1 120
...
Brilliant :-) This is a huge help and will save me a lot of time.

No comments:

Post a Comment