Wednesday, 22 July 2009

IOCP based sockets with ctypes in Python: 2

Previous post: IOCP-based sockets with ctypes in Python: 1

My current goal is to get basic asynchronous socket IO working using Winsock via ctypes. So this post documents the steps I have made today, in defining Windows functions in an interactive Python console and using them as I go so that I have a working tested result.

I have a standard set of import statements I have used to get everything I need directly available from ctypes.

from ctypes import windll, pythonapi
from ctypes import c_bool, c_char, c_ubyte, c_int, c_uint, c_short, c_ushort, c_long, c_ulong, c_void_p, byref, c_char_p, Structure, Union, py_object, POINTER, pointer
from ctypes.wintypes import HANDLE, ULONG, DWORD, BOOL, LPCSTR, LPCWSTR, WinError, WORD
Before Winsock can be used, a call to WSAStartup needs to be made to initiate it for the current process.
int WSAStartup(
__in WORD wVersionRequested,
And before this function can be defined, the WSADATA structure it references needs to have already been defined. And the definition of WSADATA requires the values of the constants WSADESCRIPTION_LEN and WSASYS_STATUS_LEN which need to be searched for in Windows header files.

class WSADATA(Structure):
_fields_ = [
("wVersion", WORD),
("wHighVersion", WORD),
("szDescription", c_char * (WSADESCRIPTION_LEN+1)),
("szSystemStatus", c_char * (WSASYS_STATUS_LEN+1)),
("iMaxSockets", c_ushort),
("iMaxUdpDg", c_ushort),
("lpVendorInfo", c_char_p),
At this point, the prerequisites are present so that the WSAStartup function can also be defined.
WSAStartup = windll.Ws2_32.WSAStartup
WSAStartup.argtypes = (WORD, POINTER(WSADATA))
WSAStartup.restype = c_int
With WSAStartup defined, it can now be called.
def MAKEWORD(bLow, bHigh):
return (bHigh << 8) + bLow

wsaData = WSADATA()
ret = WSAStartup(MAKEWORD(2, 2), LP_WSADATA(wsaData))
if ret != 0:
raise WinError(ret)
This works in the console and I can move onto creating sockets.

The first step is to define the WSASocket function.
__in int af,
__in int type,
__in int protocol,
__in LPWSAPROTOCOL_INFO lpProtocolInfo,
__in GROUP g,
__in DWORD dwFlags
This is as straightforward as it gets, once the datatype of GROUP is tracked down in the Windows header files. WSAPROTOCOL can be dismissed with a NULL value. Defining socket would be easier but it doesn't allow me to pass in a flag indicating the created socket should work in an overlapped manner.
GROUP = c_uint
SOCKET = c_uint

WSASocket = windll.Ws2_32.WSASocketA
WSASocket.argtypes = (c_int, c_int, c_int, c_void_p, GROUP, DWORD)
WSASocket.restype = SOCKET
And after locating the constants needed for the arguments from the Windows header files, I can now call the function.

raise WinError()
I'll be reusing 'ret' so I'll store it in another variable.
listenSocket = ret
The next step will be to listen on a socket and accept incoming connections on it, then to read data from other sockets whose connections are accepted. So now I need to call listen on the socket I have just created. So to define it..
int listen(
__in SOCKET s,
__in int backlog
listen = windll.Ws2_32.listen
listen.argtypes = (SOCKET, c_int)
listen.restype = BOOL
And to call it..
SOMAXCONN = 0x7fffffff

ret = listen(listenSocket, SOMAXCONN)
if ret != 0:
raise WinError()
WindowsError: [Error 10022] An invalid argument was supplied.
listenSocket should be a valid socket, so back to the documentation.
A descriptor identifying a bound, unconnected socket.
Oh, that's right, I need to bind it first.
int bind(
__in SOCKET s,
__in const struct sockaddr *name,
__in int namelen
I don't have a definition for sockaddr, so that needs to be made first.
struct in_addr {
union {
struct {
u_char s_b1,s_b2,s_b3,s_b4;
} S_un_b;
struct {
u_short s_w1,s_w2;
} S_un_w;
u_long S_addr;
} S_un;

struct sockaddr_in {
short sin_family;
u_short sin_port;
struct in_addr sin_addr;
char sin_zero[8];
And the corresponding Python definition.
class _UN_b(Structure):
_fields_ = [
("s_b1", c_ubyte),
("s_b2", c_ubyte),
("s_b3", c_ubyte),
("s_b4", c_ubyte),

class _UN_w(Structure):
_fields_ = [
("s_w1", c_ushort),
("s_w2", c_ushort),

class _UN(Structure):
_fields_ = [
("S_un_b", _UN_b),
("S_un_w", _UN_w),
("S_addr", c_ulong),

class in_addr(Union):
_fields_ = [
("S_un", _UN),
_anonymous_ = ("S_un",)

class sockaddr_in(Structure):
_fields_ = [
("sin_family", c_short),
("sin_port", c_ushort),
("sin_addr", in_addr),
("szDescription", c_char * 8),

sockaddr_inp = POINTER(sockaddr_in)
The definition seems to work, now to define bind and try it out.
bind = windll.Ws2_32.bind
bind.argtypes = (SOCKET, POINTER(sockaddr_in), c_int)
bind.restype = c_int
However, the values for sin_addr and sin_port need to be calculated and adjusted to suit the relevant data structures in sockaddr_in.
u_short WSAAPI htons(
__in u_short hostshort

struct hostent* FAR gethostbyname(
__in const char *name

struct hostent {
char FAR * h_name;
char FAR FAR **h_aliases;
short h_addrtype;
short h_length;
char FAR FAR **h_addr_list;
Which require the following definitions.
class hostent(Structure):
_fields_ = [
("h_name", c_charp),
("h_aliases", POINTER(c_charp)),
("h_addrtype", c_short),
("h_length", c_short),
("h_addr_list", POINTER(c_charp)),

hostentp = POINTER(hostent)

gethostbyname = windll.Ws2_32.gethostbyname
gethostbyname.argtypes = (c_char_p,)
gethostbyname.restype = hostentp

inet_addr = windll.Ws2_32.inet_addr
inet_addr.argtypes = (c_char_p,)
inet_addr.restype = c_ulong

inet_ntoa = windll.Ws2_32.inet_ntoa
inet_ntoa.argtypes = (in_addr,)
inet_ntoa.restype = c_char_p

htons = windll.Ws2_32.htons
htons.argtypes = (c_ushort,)
htons.restype = c_ushort
This should cover it, and the calling of bind should now be possible. A sockaddr_in structure needs to be passed to it, so the data needed to populate that structure needs to be located first.
hostdata = gethostbyname("")
ip = inet_ntoa(cast(hostdata.contents.h_addr_list, POINTER(in_addr)).contents)
Something is going wrong here.
ValueError: Procedure probably called with too many arguments (8 bytes in excess)
Some testing..
ia = cast(hostdata.contents.h_addr_list, POINTER(in_addr)).contents
print sizeof(ia)
print sizeof(in_addr)
This gives the following output.
So, whatever is going wrong is not obvious. This is already taking long enough, so I will stick with localhost as the ip address for now.
port = 10101
ip = ""

sa = sockaddr_in()
sa.sin_family = AF_INET
sa.sin_addr.S_addr = inet_addr(ip)
sa.sin_port = htons(port)


ret = bind(listenSocket, sockaddr_inp(sa), sizeof(sa))
if ret == SOCKET_ERROR:
raise WinError()
This works fine, now to try listening again.
ret = listen(listenSocket, SOMAXCONN)
if ret != 0:
raise WinError()
And this now works, Windows pops up the standard dialog window mentioning that Python has been blocked from accepting incoming connections and asks if I want to allow it. After allowing Python to accept incoming connections, port 10101 can now be telneted to, which demonstrates that the socket is now actually be listened to. The next step is to accept connections, which can wait until whenever I get the time to finish that code.

Next post: IOCP-based sockets with ctypes in Python: 3
Script source code: 01 -

No comments:

Post a Comment