Reading from non-blocking sockets
Until recently, I had never actually used my Stackless socket module for anything other than my own hobbyist projects. These are done out purely of interest, and the limited networking traffic they receive is solely me testing them. I had also never received any complaints about the throughput of the module. So it was a complete surprise to me when I learnt that I was doing non-blocking IO incorrectly with Python's socket module.
I had always assumed that when a call to select indicated there was data to be read, this meant that it was okay to do a recv call. And for my purposes, this appeared to work fine. But when you are polling select thirty times a second, and the packet sizes that are received are consistently around 16 KB, that limits the throughput to 280 KB/s. For instance, an application on an embedded device might be fetching 115 MB in updates. This would take 4 minutes.
It turns out that when select indicates there is data waiting to be read on a socket, you can call recv until EWOULDBLOCK is raised. Who knew? Probably someone who had read appropriate documentation, this document seems to be pretty pertinent.
With the above implemented, the current implementation reads the 115 MB in 13 seconds. And that is with it limiting the number of recv calls within each poll of asyncore, in order to yield in a timely fashion to the application that is pumping it.
Wow! Didn't know that either, thanks for the clarification.
ReplyDeleteHi Richard!
ReplyDeleteUse Twisted.
Thanks,
-glyph
;-)
More seriously:
ReplyDelete- Just use a larger buffer for recv(). If there's more data waiting, then it will be retrieved in one call rather than two. This is one reason that Twisted starts off with a 64k buffer size for recv().
- Hearing about "packet sizes" in TCP makes me wince. TCP doesn't have packets, at least, not from the perspective of an application receiving data from an API call. It's really hard to dispel the myth of 1 send() == 1 recv(), so let's please not make that worse.
- This is a great example of why you shouldn't be locking your event loop to your frame rate. You should just be reading data as fast as it arrives. If you're blocking on vsync(), then do your drawing in another thread. (It's really too bad that graphics APIs don't give you a file descriptor you can select() on while you're waiting for the vblank interrupt.)
- Really, use Twisted. There's a lot to this problem, and we're still figuring some of it out ourselves. For example: . Our performance isn't optimal, but if you use Twisted, you will hopefully one day inherit a fix that uses SO_SNDBUF and SO_RCVBUF to intelligently and optimally size all buffers for maximum performance, among other fixes. If you have to twiddle your own recv() and send() calls, you're unlikely to get anything for free.
- There are more fun issues to contend with when you start worrying about portability, too. For starters: http://www.advogato.org/article/672.html
Thanks for the advice Glyph, every word you say is very cogent and pertinent. And I can understand why you need to say it.
ReplyDeleteThis is just an off the cuff blog post admitting I had no idea what I was doing, and why that was, in case others benefited from it. It does not need to enforce correct notions of how TCP works - that's what your comments are for! ;-)
While there is a lot of value in your advice to use Twisted, unfortunately Twisted is a large unknown. We are working on a device where both CPU and memory are limited, and need to be budgeted. Using Stackless, Stackless socket and similar home brewed solutions ensure that we know exactly what is happening at every level and have complete control over it.
Unix Network Programming, Volume 1 (the “swoosh book”) is a great place to start if you are putting the select, poll, send, and recv tinkertoys together for the first time. And, actually, it's pretty great if you're doing it for the hundredth time too! :-)
ReplyDeleteHey Richard,
ReplyDeleteThanks for this post, I wasn't aware of this either. As far as I can see, the same is true for send() and accept() - the latter being *really* important when your server gets a large number of connections in a short period of time.
Cheers,
Chris.