08 Jul, 2009, Kelvin wrote in the 1st comment:
Votes: 0
I was curious to see what everyone is using for their MU* projects out there in terms of asnychat and Twisted. I started Evennia on asynchat, but Twisted seemed to be a lot more appealing with its telnet protocol implementation and a few other neat things like deferreds. I imagine I had things setup incorrectly on asynchat, but I also noticed some performance gains from switching to Twisted and wondered if anyone else experienced anything similar.
17 Jul, 2009, donky wrote in the 2nd comment:
Votes: 0
Kelvin said:
I was curious to see what everyone is using for their MU* projects out there in terms of asnychat and Twisted. I started Evennia on asynchat, but Twisted seemed to be a lot more appealing with its telnet protocol implementation and a few other neat things like deferreds. I imagine I had things setup incorrectly on asynchat, but I also noticed some performance gains from switching to Twisted and wondered if anyone else experienced anything similar.


Asyncore is what I use, but if I had the time, I would discard it and use low-level asynchronous operations (select, IOCP via ctypes and so forth) directly. It would take as much time to adopt, there would be less unnecessary levels of abstraction and I would be directly working with functionality that is applicable outside of Python.

This presentation on asynchronous vs threaded python does a good job at covering the cost of using most asynchronous frameworks. Syntax, layers of abstraction, incompatibility with standard modules, etc..

This is why I prefer Stackless Python. I can make replacement IO modules to monkeypatch in place of the the standard ones, and then most of that cost is removed.
25 Aug, 2009, Barm wrote in the 3rd comment:
Votes: 0
Kelvin said:
… but if I had the time, I would discard it and use low-level asynchronous operations (select, IOCP via ctypes and so forth) directly. It would take as much time to adopt, there would be less unnecessary levels of abstraction and I would be directly working with functionality that is applicable outside of Python.


You can use select from within Python. That's what I'm doing in my asynchronous, single-threaded server. I poll select.select() with a 0 timeout during the game loop.
25 Aug, 2009, Kelvin wrote in the 4th comment:
Votes: 0
Is something as simple as a MUD really worth the effort in getting so low level? For an academic exercise, perhaps, but reinventing the wheel for no real tangible advantage sounds like a lot of work for little gain. I don't typically hear people getting too upset about MU* performance anymore.
25 Aug, 2009, Barm wrote in the 5th comment:
Votes: 0
In my opinion, it's very much worth it.

First of all, it's probably way less work than you imagine. My socket module is only 131 lines and half of those are comments and blank lines. My Telnet module is 764 lines, again heavily commented. Those two files are the entirety of my network code. When I wrote them my goal wasn't speed, it was control. I didn't want to rely on third party libraries and, especially with Telnet, there were options I wanted specific control over. For example, if the client resizes his window and supports NAWS, my server-side word wrapping automatically adjusts.

I like Twisted (bought Fettig's book and read Glyph's blog) but trying to shoehorn everything into a deferred callbacks made my head spin. I'm not knocking it if it works for you. It's a great library. I just found it more approachable to write my own.
27 Aug, 2009, donky wrote in the 6th comment:
Votes: 0
Barm said:
You can use select from within Python. That's what I'm doing in my asynchronous, single-threaded server. I poll select.select() with a 0 timeout during the game loop.
The select module has a limited number of sockets that it can handle at a time (at least on Windows), if you have more than this in the dictionary you pass in (or whatever you pass in), then it errors noting this constraint. In any case, the select module is comparable to IOCP with ctypes in terms of the level at which it sits. If I were doing a cross-platform solution, I'd support select as well, although I'd split up the sockets handled by it into bundles of 512.
27 Aug, 2009, donky wrote in the 7th comment:
Votes: 0
Barm said:
In my opinion, it's very much worth it.

First of all, it's probably way less work than you imagine. My socket module is only 131 lines and half of those are comments and blank lines. My Telnet module is 764 lines, again heavily commented. Those two files are the entirety of my network code. When I wrote them my goal wasn't speed, it was control. I didn't want to rely on third party libraries and, especially with Telnet, there were options I wanted specific control over. For example, if the client resizes his window and supports NAWS, my server-side word wrapping automatically adjusts.

I like Twisted (bought Fettig's book and read Glyph's blog) but trying to shoehorn everything into a deferred callbacks made my head spin. I'm not knocking it if it works for you. It's a great library. I just found it more approachable to write my own.
I look back at the work I've put into using asyncore and I wonder why I didn't just use select. Asyncore as a framework, wraps the quirks of select, meaning you have to deal with two levels of problems. When I ditched asyncore for a more low-level solution my code became so much simpler and all the workarounds I had to make to get my solution to be cleanly usable, were unnecessary. For years I swore by asyncore, but now I think I was just going for the route I had decided was easier.
27 Aug, 2009, Barm wrote in the 8th comment:
Votes: 0
donky said:
The select module has a limited number of sockets that it can handle at a time (at least on Windows), if you have more than this in the dictionary you pass in (or whatever you pass in), then it errors noting this constraint.


I did some testing. First on Linux:
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38) 
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import *
>>> from select import select

>>> s = [ socket(AF_INET, SOCK_STREAM) for x in range(1022) ]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/socket.py", line 154, in __init__
socket.error: (24, 'Too many open files')

>>> s = [ socket(AF_INET, SOCK_STREAM) for x in range(1021) ]
>>>

>>> select(s,[],[],0)
([1021 socket objects omitted], [], [])


It looks like I can have 1024 file descriptors open per process (I'm guessing the python shell is eating three via STD_OUT, STD_IN, and STD_ERR).

Then Windows:
Python 2.6.1 (r261:67517, Dec  4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from socket import *
>>> from select import select
>>>
>>> s= [ socket(AF_INET, SOCK_STREAM) for x in range(1024) ]
>>> select(s, [],[],0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: too many file descriptors in select()

>>> del s
>>> s= [ socket(AF_INET, SOCK_STREAM) for x in range(513) ]
>>> select(s, [],[],0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: too many file descriptors in select()

>>> del s
>>> s= [ socket(AF_INET, SOCK_STREAM) for x in range(512) ]
>>> select(s, [],[],0)
([], [], [])
>>>


I tried a list comprehension of a 100,000 sockets and it worked. I didn't test them though. I'm inclined to just set max connections to 512 and call it a day. I haven't figured out why element 0 of the returned tuple (the recv list) was full on Linux and empty on Windows yet.
27 Aug, 2009, Tyche wrote in the 9th comment:
Votes: 0
Barm said:
I haven't figured out why element 0 of the returned tuple (the recv list) was full on Linux and empty on Windows yet.


Perhaps something to do with the underlying select implementation.
An invalid socket on Linux is -1, and an invalid socket on Windows is 0.
Windows sockets may be negative, err …unsigned integers are used.
Also select on Windows only refers to sockets, not open files.
27 Aug, 2009, Barm wrote in the 10th comment:
Votes: 0
I found this page Differences Between Windows and Unix Non... with some info.

So far, my socket module run flawlessly under Linux. Some day I might test it under Windows but that platform often devolves into a support rat hole is not a high priority for me.
0.0/10