27 Apr, 2010, Kardon wrote in the 1st comment:
Votes: 0
Hello all,

If you listen in on the ichat channel on IMC, you may know that I've been working on an IMC client with Python in addition to a MUD built in Python using the Miniboa telnet library. Progress has been good on both of these projects, and as of tonight I've integrated the IMC client into my MUD and ran some successful tests!

After all of this I've come across a few realizations. First, I've been teaching myself the Python language using these as my pet projects to try different things out. The biggest problem I've encountered is circular import problems, which is symptomatic of a bad design. Unfortunately coming from a C# background where everything kind of "plugged together" through meta-data I was not prepared for tackling module design properly. After a bit of hacking here and there to get around these issues I believe it is time to redesign my codebase. My question is this: Does anyone know of good on-line material covering idioms and/or best practices in regards to the Python programming language? Especially in regards to organizing modules and bigger multi-file projects.

Thanks!
27 Apr, 2010, David Haley wrote in the 2nd comment:
Votes: 0
Usually circular dependency is an indication of bad design, but sometimes you truly have no choice. (Logging systems that include util modules for occasional use, which themselves use the logging system, are a good example of this.) You can get around circular dependencies by identifying which module is the "primary" module, and only incidentally makes use of the other module, and then which module is truly dependent on the primary module. In the example above, the logging module is nominally more independent, whereas the util module is a true consumer of logging.

In this case, you would solve it by having a local import where you need it.

# logger.py
def log(*args, **kwargs):
from .util import dict_format
if kwargs:
print("LOG: %s, %s" % (str(args), dict_format(kwargs))
else:
print("LOG: %s" % str(args))

#—————–
# util.py

from logger import log

def my_func1(bla):
log("calling my_func1")
def my_func2(bla, **kwargs):
log("calling my_func1", **kwargs)
def dict_format(d):
# now here, we can't call "log" with kwargs or we get an infinite loop
return str(d)


Well, you get the idea. Use a local import: unlike Perl, and C for that matter, these happen at execution time, not at "compile" time. (The comparison is fishy because in C's case it's a preprocessor include, but yeah.) Perl's 'use' is processed at compile-time, regardless of where it appears in the file. (Contrast with 'require'.) But Python's import is treated as a statement like any other. (And, it's safe to execute several times.)



Here's a useful link for general Python best practices and idioms. It doesn't talk too much about how to organize modules, though.

Maybe if you have a specific example for a module organization that's giving you trouble, we can talk about it and see if it's a case where you have no choice or a case where you can fix it.
27 Apr, 2010, Barm wrote in the 3rd comment:
Votes: 0
One thing I found helpful is to approach __init__.py as a resource for users of the module instead of authors a module. In other words, I use __init__.py to clean up my module's external API but I avoid references to it from any python files within the module itself. Other modules in the same project can (and should) use it – as they are also users of the module.

So while a Miniboa user would import like this (tapping miniboa/__init__.py):

from miniboa import TelnetClient


Inside the module itself I always use the full, actual module path, like this line from miniboa/async.py:

from miniboa.telnet import TelnetClient


I hope that makes sense and that you find Miniboa helpful for your project.
27 Apr, 2010, 3squire wrote in the 4th comment:
Votes: 0
If you're about to rewrite your codebase in python anyway, I've got an excellent stub for a codebase written in Stackless Python that you might want to start from. Stackless Python is a lot easier to work with for this type of project and you'd have the added benefit of having everything built for you except the game code. Let me know.
28 Apr, 2010, Kardon wrote in the 5th comment:
Votes: 0
Quote
Well, you get the idea. Use a local import: unlike Perl, and C for that matter, these happen at execution time, not at "compile" time. (The comparison is fishy because in C's case it's a preprocessor include, but yeah.) Perl's 'use' is processed at compile-time, regardless of where it appears in the file. (Contrast with 'require'.) But Python's import is treated as a statement like any other. (And, it's safe to execute several times.)



Here's a useful link for general Python best practices and idioms. It doesn't talk too much about how to organize modules, though.

Maybe if you have a specific example for a module organization that's giving you trouble, we can talk about it and see if it's a case where you have no choice or a case where you can fix it.


Very useful information, thank you! I think my biggest problem is getting used to how modules interact with each other and trying to remember I'm working with an interpreted language. I'll pull up a more specific example when I'm a little less tired, just got home from work so I'm posting this then going to bed. :lol:

Barm: Also useful information, thank you. =) And Miniboa has been working fantastic for me.

Quote
If you're about to rewrite your codebase in python anyway, I've got an excellent stub for a codebase written in Stackless Python that you might want to start from. Stackless Python is a lot easier to work with for this type of project and you'd have the added benefit of having everything built for you except the game code. Let me know.

I'm not so much rewriting my codebase in Python as I am writing a new codebase in python. But I'm always up for looking at other examples.
28 Apr, 2010, David Haley wrote in the 6th comment:
Votes: 0
Kardon said:
I think my biggest problem is getting used to how modules interact with each other and trying to remember I'm working with an interpreted language.

Yeah, it's quite a shift in how you think about things. In C/C++ for example you can get around this by using forward declarations; headers often need to know that a type or function exists, not how it's defined. And so your headers have forward declarations, and .c/.cpp files include the relevant .h files to get declarations. But in Python (etc.) you don't have this distinction.

Python is different than, e.g., Perl because its 'import' statements are truly statements like any other in the language (similar to Lua). I don't remember what Ruby does here. In Perl's case, IIRC, 'require' is a statement like any other but 'use' is special in that the file is scanned for all 'use' statements which are run in order when the file is imported.

Anyhow I've run into a few cases of necessary circular dependency in modules (like logging and utils which I mentioned) so local imports is a common solution for me, but yeah, usually I try to separate things into modules as is reasonable.

What Barm said works very well for modules facing 'clients', although sometimes internally the module is circularly referential anyhow.
28 Apr, 2010, quixadhal wrote in the 7th comment:
Votes: 0
3squire said:
If you're about to rewrite your codebase in python anyway, I've got an excellent stub for a codebase written in Stackless Python that you might want to start from. Stackless Python is a lot easier to work with for this type of project and you'd have the added benefit of having everything built for you except the game code. Let me know.


I'd be curious to see this, just because I'm looking at rewriting an old turn-based game which used files as a client/server game, preferably with telnet or a web based client, although that's not a requirement. A python version might be interesting. :)
29 Apr, 2010, Davion wrote in the 8th comment:
Votes: 0
3squire said:
If you're about to rewrite your codebase in python anyway, I've got an excellent stub for a codebase written in Stackless Python that you might want to start from. Stackless Python is a lot easier to work with for this type of project and you'd have the added benefit of having everything built for you except the game code. Let me know.


Out of curiosity, why stackless? Under these circumstances, specifically, why does stackless take an edge?
29 Apr, 2010, 3squire wrote in the 9th comment:
Votes: 0
I wrote a huge post while sitting in the Denver Airport and needless to say it's wifi ate it. I will repost my reply it soon.
29 Apr, 2010, Idealiad wrote in the 10th comment:
Votes: 0
donky's Sorrows lib is also Stackless.
30 Apr, 2010, Kelvin wrote in the 11th comment:
Votes: 0
For something as simple as a MUD, Stackless is probably not worth it. Microthreads aren't really necessary, and using Stackless really lessens your potential userbase. It's still anything but mainstream, is hard to install "correctly" on certain operating systems alongside CPython, and is a much better fit for applications that actually need something like microthreads to be responsive (IE: EVE Online, a complex 3D game).

That is, unless you want to write in Stackless as an academic exercise. That's cool too.
22 May, 2010, donky wrote in the 12th comment:
Votes: 0
Kelvin said:
For something as simple as a MUD, Stackless is probably not worth it. Microthreads aren't really necessary, and using Stackless really lessens your potential userbase.


MUDs are not simple. Saying microthreads are not necessary is about as helpful as saying functions are not necessary. And where you say userbase, presumably you mean the developers who would adopt the codebase, rather than any actual userbase who would play it.

Kelvin said:
It's still anything but mainstream, is hard to install "correctly" on certain operating systems alongside CPython, and is a much better fit for applications that actually need something like microthreads to be responsive (IE: EVE Online, a complex 3D game).

That is, unless you want to write in Stackless as an academic exercise. That's cool too.


What is mainstream? Who says that EVE needed microthreads to be responsive?

All of these questions are rhetorical, I am trying to highlight how subjective and unsubstantiated your post was.
0.0/12