stacklesssocket.py, greenlet and blocking operations

It seems like many years ago now, that I wrote stacklesssocket as a hobby project for whatever reason I have since forgotten. By monkey patching a matching synchronous interface to asynchronous resources (i.e. asyncore) over the socket module, a lot of existing Python code that used the socket module would just run when used in an application that also used Stackless Python's tasklets. Andrew Dalke's example illustrates this.


I now have a number of hobby projects that only get changed when I have the time, energy and interest to work on them. And as such, this module would only get attention when those other projects used it and encountered a problem. This didn't stop other people from using the module. Whether they still use it I do not know, but the other day I saw that a business that at least used to base their service on it got bought by Google. It is also used internally at C.C.P. temporarily in a situation where their Stackless IO library (still pending open source-ification) is not supported.

Recently, I received an offer of sponsorship to do something I have wanted to do for a long time. I won't mention who is sponsoring me, without getting their permission first. Anyway, the central task within the sponsored work was to get the Python standard library unit tests running against it, and to get the tests (at least the non-UDP ones) passing. This is now done, and apart from two particular UDP unit tests, the module passes the tests for Python 2.6 and 2.7. The module is also compatible with Python 2.5, and won't work any worse with that version than it did before. There are at least some API differences (which exceptions are raised) that it does not specifically support when run against Python 2.5. It is interesting to note however, that given there are only two UDP test failures, it is a good sign that generally the new module changes are naturally correct.

The Python unit test suite uses a lot of thread blocking calls, which also block the Stackless scheduler that manages the tasklets running on the given thread. And often, those tasklets need to do something for the thread blocking call to get what it needs. So, this is not Stackless compatible behaviour. The most common of these calls are time.sleep, select.select and calling acquire on a thread lock. In order to get these working, I've implemented a threading-related module to also do monkey-patching to hand off the work into another thread. Something that also required more work that I would have liked, was working around the constraints that Stackless enforces when scheduling in another thread (there is a scheduler per-thread) and that thread is blocked while waiting for a channel operation. There aren't many users of Stackless' scheduler per thread support, and a few of the rare corner cases need some usage to clean up exactly how they should work.

While it has been an additional goal since stacklessocket was first created to eventually implement a matching suite of modules that can be used to turn any thread blocking operation to a tasklet one, unfortunately I've had to spend my time elsewhere. But we are now with the addition of the threading-related module part of the way there, even if only to a need-driven level so that the Python unit tests will run. I've been envious that gevent has a suite of modules like this, although I have no idea of the extent to which they work. My involvement with greenlet has been to write a module that provided a Stackless-like interface for it, when it was first released. Beyond that, my involvement with anything greenlet related has been to sending an email to a tengentially related mailing list, related to noting that the Python license more than likely applies to any source code that was extracted from Stackless Python (and therefore it can't be MIT licensed within greenlet), both demonstrating my lack of knowledge about how it is used and potentially annoying other people with license complications.

Anyway, if you use this module, I suggest giving the latest version a try. Let me know how it works for you.