29 May, 2014, syn wrote in the 21st comment:
Votes: 0
I think I was standing at the edge and judged the distance shallowly, but nagged that it was deeper..

That makes perfect sense. I can tell you I am not remotely a DBA, clearly, and am just garnering the slight knowledge of how not to completely look like a fool, just mostly look like one. We all learn. At any rate, I think my internal war was/is with the (clearly in context to the systems herein) massive data structures and countless bits and bobs every tom dick and harry decided to tack on over the years. While I mentally would love to segregate a lot of that garbage out (beyond things that I am/will outright remove), having someone point out that, no that ledge isnt so shallow, was what I needed.

I am less concerned with the memory use, or size, specifically, and just more concerned with creating a backend system that actually makes sense from a data perspective. Clearly I am no World of Warcraft, and as you easily pointed out a visual medium allows for more subterfuge regarding loading and how to obfuscate any data call lag times. We obviously do not have that luxury in nearly any case.

I think I've run into shiny toy territory and need to be more objective in my scoping. While the DB is and will be perfect for a lot of things, and warehousing data, it will also often be a square peg to this or that problems round hole. As I learn more about this new toy I am confident I will see that line more easily, but for now should temper my enthusiasm with some cold realism.

Outside of that dose of reality, w.r.t. saving stateful information of 'objects' in memory, how often do you do that? I was contemplating setting up an event queue, actually I inserted one based off of Jobo's socketmud setup with some tweaks, regardless, I was contemplating a staggered event setup with varying intervals depending on importance of the data in question, for items that I retain and manipulate in memory. I suppose mostly the interval times would be dependent on how much of a potential data loss I would be willing to take..

Jees it seems like I am just asking obvious somewhat redundant questions going back to, use common sense and look at your own context. So apologies for that. I think this has helped me think through what my gut has been telling me about many things in relation to all of this, and the responses, even if it is akin to use common sense, are helpful in validating that.

So perhaps I should break with that and simply ask;

Were you to give someone who is relatively new to using a DBMS in this context, or any, advice what would it be?
29 May, 2014, syn wrote in the 22nd comment:
Votes: 0
Ok, here is maybe, hopefully, a better question.

I want to begin the process of making the game world persistent. I am going to start with objects because that would be the most immediate, and easiest to accomplish imo.

To effect this I want to do the following:

1, Create a unique identifier for any object created in the game, hereafter uwid
2, Create a 'Last seen at' table, serving multiple purposes
2a, On a save pass it would only need to record up to 4 bits of information
2a1, Item UWID, to identify which object this is
2a2, Room VNUM, to tell us what container location it is in
2a3, IN_OBJ UWID, populated if it is within another object, and links to that objects UWID
2a4, IN_OBJ as Player, if it was in a player inventory – this would be helpful i imagine for many reasons but in this case to say whether I need the GAME to save the object, or the character record is handling that save
2b, This data would become effectively a master object location index for the game to figure out where an object most recently was. I think this would be useful in many contexts, but lets take an OLC prog for instance.
If I want a prog that can teleport a player to a specific object, well now I can find that very specific object firstly, but secondly if it moves I can attempt to acquire its actual current location, making the program more robust.
This would allow for a builder to have a lot more 'fun' when designing quests, or hunting games, allowing them to actually move an object, and then other entities can track and utilize the data.
Another example might be a murder mystery. Once we know the murder weapon in the game, to track where the weapon goes, I just want some simple information, I dont need access the entire object table for that, just the above information
2c, Updating location, in memory, would be easy, to me, - find the uwid and set the couple of variables you need, then saving this list takes much less time and effort than scanning all items for a location change, saving each object to its new location.
Saving this table would be more trivial, and faster than saving each object back to its table - though I suppose only sending an update for the object which has a new location field would work, I'd rather keep it simple, and this seems simpler to me.

Bottom line is, I dont want the Object to care where it is, I want the game to. Unless this is somehow flawed in its inception?

Now to the SQL specific bits, how often, as I have seen both of the two respondents post regarding working with or having persistent worlds, how often do you save this type data down to the server, and do you find it to be (relatively) an intensive event for the game?

I am hoping that by effectively making a 'lightweight' table of information it will be less taxing for the game, or SQL to utilize as information or for data warehousing purposes.

Is this even an intelligent design in principle, or would it be best to scrap this 'last seen at' table? I know I could find the data in an object, but I just didnt like the idea of having the disparate parts scattered per object and then need to look at the object. The advantage to housing it on the object is obviously if the object is removed, (destroyed, purged, eaten, what have you) the where location would immediately return as null, letting me know the item is gone. The table would add slightly to checks, requiring that I register, and de-register any object to the table, but I think this is not a big deal versus what I see as the benefits..
29 May, 2014, plamzi wrote in the 23rd comment:
Votes: 0
If you're out to make a fully persistent world, it seems to me your work is cut out for you, and I wouldn't even worry about making the kind of index you're describing, which seems like extra work that in no way helps persistence. It may be a good exercise, but I honestly don't see much of a gain from what you're describing. If the goal is to be able to traverse all objects regardless of their location, why not maintain a linked list of all item instances in the game? This will be easier than maintaining a subset of properties, and it will enable you to look up any property, not just "last seen".

As far as persistence goes, having a unique id for every object instance is a very good idea, and pretty much standard nowadays. More interesting would be how and when you're planning your reads and writes to the table or tables storing those instances. I haven't gotten very far in my own project that has a similar design, but I was thinking that inserts and deletes from that table need to be in immediate sync with the game server, while modifications of existing items (such as an item changing a property) can be lower priority. I'm also leaning towards a single db table for all instances, but maybe partitioned based on an item's location / position.

To help optimize things further, I'm also thinking that many types of items are too ephemeral to deserve their own row in a table (e. g. consumables). if someone has 30 healing pots, maybe these can be stored as an array of unique keys + a single reference to the healing potion "prototype object" id.

I'm planning to have prototypes for any entity that may need to exist in more than one copy, but in my experience it seems to me that even dynamically generated instances would benefit from having a "prototype" generated for them. For instance, in my Dikurivative game we started generating random items without prototypes at first, but if these items got lost due to an issue, they were more difficult to restore because there was no redundancy for their exact properties. So maybe what I'm looking at are two tables for all items: one containing "template properties" and one containing instance specifics such as unique id and current location.
30 May, 2014, quixadhal wrote in the 24th comment:
Votes: 0
You might also want to look at some existing persistent world MUD's to see how they do some things. If you grab the Gurba mudlib and the DGD game driver (LPMUD), you'll see one example of a MUD that has fully persistent objects and code which allows you to update such objects on the fly.

The big question there is… do you want to make a GAME, or do you want to make a piece of software? Quite a few people start doing a LOT of work to write their own drivers because they believe they have to in order to make a game that works the way they want it to work. In some cases, that might be true… but usually, it's not. Now, if you want to write the software, that's another story…. just remember to also write a game so you know if the software is flexible enough to actually do what you will want it to do in a few years.
30 May, 2014, syn wrote in the 25th comment:
Votes: 0
Hi Plamzi,

Thanks for the input and general description and internal considerations you are making yourself.

I posited the pseudo list for tracking and traversal so that I wouldn't yet need to seriously re-work the very core of how the items interact, and/or restructure all data at once. Best practice? No, but I figured it would be more of a proof of concept and get me to an initial state of, the world is, for intents and purposes 'persistent' semantics aside. This would just take adding a couple of new elements to existing object structure (and one for char_data to id a mob/player carrying an item) and a new struct to pipe information into and then read at will. Perhaps, last seen was a poor name for it, but it really, effectively, would be an in memory object register with all current tracking data. In this instance, I just need to inject searches and saves based on the table, savable to retain a state-full look at everything's location from disk, and provide data integrity, so as to not rely solely on the object believing it was in X location without any form of checking.

In general, I think that I/anyone would likely want a 2 fold system anyway:

A register and a receipt, I (obj) register at vnum 3001 You (game) say ok good! The game now has a record of where it was told I am, and I have a record of where I said I am. This would allow for validating locations/states and to me help with any odd issues/bugs that may arise, and/or people figuring some method of duplication… hey that's weird The Object™ says its in 6577 but I see it in 3001, with the same uwid..

Likely redundant and I am being pedantic but, that is how I think. I like information, and being able to validate that information more easily, to me, would be a win down the line. As more of the game becomes persistent the Last Seen register would grow to encompass them as well providing some form of location accountability within the system.

Its fine to remember the location, but, was that REALLY the location? How would I know otherwise? Extrapolate with cross-referenced and saved links between all objects creating a mesh information blob, and where would I store that data.. per 'object'?

Is that a flawed view of a persistent system? I just don't see the benefit in not having any means of fact checking, and relying solely on a single point of data failure for this information.

As to retaining prototypes, I think that is a great idea. At best you need it only perhaps on some items initial creation, at worst it fills in any holes or an entire object. It may be at a 'default' state but it exists. The approach has, and will continue to exist. I don't think there is any reason why you wouldn't do this.

For the more destructible of objects, I think it would be safe to do this for any object, specifically, that you cannot modify as a player. EG Once I create a potion of healing, all I could do to it would be to use it, store it, or drop it (insert any other states it can obtain). So I don't need to worry about it suddenly having a name changed, or the like. Even in that instance, I could just glob together all of the 40 healing potions that Mr Man has and separate out an object that is later modified, at that time. At that point its no longer a healing potion anyway, its sort of like a healing potion, but something else entirely as far as the game is concerned. So.. anything that is effectively a copy of a prototype, until X, is a prototype, and gets globed up as 'one' unit of prototyped object Y for instance.

Did that make any sense? It sounded sensical in my mind..

Now..

Quix, thank you, again.

I will definitely check out DGD/LPMUD. I love to look at other work and take inspiration/thought/warnings/back to the drawing board moments. Its very helpful to re-evaluate my own thinking, or ability.

To your rhetorical question.

I think if I am really honest, I am more excited, and more driven about the technology, and understanding how to shape a universe that my game would want to live in. If that makes sense.. I am not trying to re-invent the wheel.. and yet, in ways I am. I get a lot of satisfaction out of seeing a goal, long and short, and overcoming my knowledge gap to understand the goal and make it work. I have a couple friends working on the project with me, and they keep me a bit more short term directed, which is good, and improves their quality of life greatly. Outside of that I like to work on projects that culminate, as possible, to a more major goal.

My first was, actually that other thread, regarding choking out spam sent to a player. While likely needlessly, it led me from the do_drop command all the way down to the socket to take a pandering look at how it operates and I was surprised by what I found.

I had had no idea that these lines of codebases were non-blocking and it was relatively trivial to overload its very small socket buffer, doing bad bad things. This led me to learn quite a bit about the socket layer the game uses, and add in a nice micro-queue to hold onto data if it saturates on the write, or runs into a blocking system. Removing most of the protecting buffers for output was a fun test, and it actually worked quite well.

As un-related as that all is, I hope it illustrates that, in general, I have a sound game objective, or goal, and accomplish it. How I reach that goal sometimes is a bit of a meandering travail, but I am all the better for the knowledge gained.

I am sure it would be easier to abandon ye olde modified in some terrible ways, and some great ways Godwars and run off into the sunset with (some other codebase/driver). I certainly have nothing against other code, or people using them, but I am a little bit of a stick in the MUD as it were.

My first MUD was a ROM, I moved around between them SMAUG and Circle until I found GW, and I just love it. Quirks, flaws, excellent points all. So, I suppose in many ways I am driving into the creating a piece of software sector, and in the end will defer to the limitations I end up with, whatever those are, myself or otherwise.

Such a long winded response for, I believe I do want to write the software, however old C is, and however insignificant the codebase, I am enjoying the journey.
30 May, 2014, plamzi wrote in the 26th comment:
Votes: 0
syn said:
A register and a receipt, I (obj) register at vnum 3001 You (game) say ok good! The game now has a record of where it was told I am, and I have a record of where I said I am.


I'm not sure I understand the use case completely. If you're talking about creating a new item, the way it would usually work is you ask the server / game to create it, and it returns either some error, or some information about the new item, e. g. its unique id. If you're talking about changing the location of an item, again the request represents an attempt to move the item, but it is the server that decides whether movement is possible, and returns the new location on success, or a message on failure.

In either scenario, the important point is that the server is *authoritative*. You should not be assuming that your item has a vnum of 3001 or that its location is inside a backpack until the server tells you it's so.

syn said:
Its fine to remember the location, but, was that REALLY the location? How would I know otherwise? Extrapolate with cross-referenced and saved links between all objects creating a mesh information blob, and where would I store that data.. per 'object'?

Is that a flawed view of a persistent system? I just don't see the benefit in not having any means of fact checking, and relying solely on a single point of data failure for this information.


If you are talking about storing the location of an item in more than one place in memory, then the only reason you would have a discrepancy in a single-server scenario is if your code forgot to update some place when the location changed. In that case, all you will need is to know where to get the authoritative information about an item's location. It should come from the same place that runs all the checks to ensure that moving the item to that location is possible. It should never be a matter of voting (A and B win out over C :)

It's not best practice to store the same information in more than one place in-memory (when you don't have to). Instead, you should be storing references to the single point of authority about an item's location. Anything else is just introducing more bureaucracy and breaking points, not to mention reduced performance. If you are creating an index, make sure it's worth the extra trouble. And never go to the index to tell you where an item REALLY is :)

Now, having redundancy in the persistent storage is quite different from having redundancy in-memory. It's a good thing because it will allow you to reconstruct a consistent state even if, e. g. the runtime was buggy or the tables that the runtime usually writes to got corrupted from hardware errors etc. The normal practice is to take periodic snapshots of the persistent storage that the game writes to. These can be either hot backups, like backup database schemas or slave database servers, or cold backups like database dumps. Ideally, both.

Achieving a conventional persistence model for one game server is really not that hard. While the server is running, it is authoritative. If it crashes, you check your persistent storage. If it seems intact, you restore the server state from it. If it's not intact, you use your persistent storage backups to restore it, and then you read the server state from it. When your game server starts up again, it becomes the single point of authority where all the "facts" come from.

Of course, it would be a lot more complicated if you had multiple game servers trying to all maintain the same state. In that kind of scenario, you could decide that the persistent storage is authoritative. In that case, every time a server needs to move an item, it would lock a row, read its latest location, perform movement checks, write back the new location, unlock the row.
30 May, 2014, syn wrote in the 27th comment:
Votes: 0
I suppose that makes sense, its adding a possible corruption point, or breakage.

I just didnt like the idea of the location being stored solely on the object, though i suppose semantically its not much different either way i would need to loop through some list to find object 31x.

So something like this may be more appropriate?

typedef struct world_location WORLD_LOCATION;

struct world_location
{
WORLD_LOCATION *next;
int *ruwid;
int *vnum;
char *type;
};

and in OBJ_DATA for instance,


WORLD_DATA *location;
int uwid;


as a really simple example, obviously..
31 May, 2014, quixadhal wrote in the 28th comment:
Votes: 0
One question for you syn, are you writing this as an original codebase, or modifying an existing Dikurivative?

The use of "vnum" leads me to think you're modifying an existing system, because otherwise it's a pretty clunky system to stick with. The Diku fans here will always disagree with me on this, but having worked with them for decades, I can tell you that it was a source of constant errors before OLC, and now is mostly a source of annoyance when you have to change them, or look them up.

Generally speaking, you store data in one place, always. You may store REFERENCES to that data elsewhere for ease of searching or indexing, but duplicating data is wasteful and just exposes you to future bugs where you forget to update an instance of it somewhere.

My suggestion, if building stuff from scratch, is to adopt a different concept for identification, namely to use an actually useful and descriptive string as your primary identifier, and have pointers to it wherever you need a reference.

Now, isn't that inefficient, you ask? Sure it is… but again, you're not coding World of Warcraft. There's a tradeoff between simplicity of use, and efficiency. How is having a string identifier any "easier" to use? Here's two examples.

Diku-style:
struct npc {

int vnum;
struct area *areap;
int flags;

}


Here's a pointer, p, to an npc… tell me what NPC it is and what zone it's in?

In that traditional Dikurivative format, I'll have to do a bunch of dereferencing to print that information and see, unless you have it all memorized. p->areap->name for the area, p->short_desc for the name, and then I'd also need to look at p->flags & FLAG_PROTOTYPE to see if it's an instance of an NPC, or the template copy (for those Dikurivatives that have OLC).

Now, if you had used a descriptive string ID, the way an LPMUD does, you'd have seen something like:

p->id == "/d/smurf_village/npc/papasmurf.c#13"

At a glance, you see the npc in question is papa smurf, that he's in the smurf_village (or at least, spawned there), and is the 13th instance to have been created… meaning he's a clone.

This sounds horrible, I know. You have to do all those big long string comparisons all the time, instead of an easy integer comparison. But really, think about the use cases. How often does the game want to "find" things, vs. manipulating things it already has… compared to how often a player/builder wants to "find" things? When the game manipulates things, it doesn't look them up by vnum (usually)… it has a pointer and follows it. The place those vnums or id's get used is when a player types "find sword" or a builder wants to know where the "hatchet of doom" is, so he can load one onto his new NPC "PK Bubba Fett".

If you still aren't convinced…let's go the other way. You have vnum 3001, tell me what it is!

If you're a Diku person, your first problem is… did I mean a room, an npc, or an object? Without knowing, somehow, I can only narrow it down if that number happens to be used in more than one list. But using a pathname style of ID, as above, I can easily see it's an npc.

—-

Now, as to why you want references…

One way is to keep a central list (or tree, or hash, or whatever) and any time you create or destroy an instance of an object/room/npc/player, you update the data structure. This is how DikuMUD does things, and it chooses linked lists to do it. It means that you need to maintain such a list for every structure element of every kind of object you want to be able to walk along.

So, there's a list for npcs. There's a list of npcs in each room. There's a list of npcs in each zone. There's a list of npcs that have red hair… you get the idea. You've seen the code… list pointers all over the place to support this threading.

There are other ways.

You could create what in LPMUD terms would be called a daemon, a singleton object that you can call functions in. In an LPMUD, you don't have messy links of entwined structures. In fact, you can't… because each object is soft-coded in LPC, and can't refer to parts of another object's code. Instead, we use daemons.

You want to keep track of every NPC? You make a daemon, and when you create a new instance of an npc, it registers itself with that daemon. So, when you want to find all the village guards in smurfland, instead of YOU duplicating code that walks the list of all NPC's and compares their id's, you call NPC_D->find("/d/smurf_village/npc/*guard*"), and the daemon code does the comparisons, returning a nice array of matches for you to work with.

You can do this in C too. It's a little more messy (as everything in C is), but the concept is the same. When you clone your new NPC, call a function to register it. You don't need to keep all those pointers INSIDE the structures of the npcs… that's silly. Just keep that in your central location.

I would encourage you to try to keep data as isolated and sandboxed as you can. Don't store the NPC location in your list of NPC's… then you just have to update that data all the time. When you want to know where an NPC is, ask the NPC!

object *result = NPC_D->locate("/d/smurf_village/npc/*guard*");
foreach (object *ob in result) {
write("NPC " + ob->get_name() + " is in " + environment(ob)->get_name() + "\n";
}


You can do the same thing in C, but you have to remember C isn't dynamic… so it'd be more like

struct npc *results = find_npc("/d/smurf_village/npc/*guard*");
struct npc *r = results;
while(r) {
send_to_char(ch, "NPC ");
send_to_char(ch, r->short_desc);
send_to_char(ch, " is in ");
send_to_char(ch, real_roomp(r->in_room)->name);
send_to_char(ch, "\n");
r++;
}
free(results);
r = results = NULL;


Not sure if this gets my point across well or not, but… :)
31 May, 2014, syn wrote in the 29th comment:
Votes: 0
I am unfortunately and intimately familiar with vnums.. I can't imagine any sane person arguing that vnums are an intelligent design in the present day. When they were originally incepted, it may have been an amazing holy cow idea, I have no clue, but now? Its effing terrible. A lot of my thoughts, and complaints about how large the data structures in dikuriv's revolve around the insane linking, and traversal of things you shouldnt have to traverse to find a simple bit of information. (imo)

I actually had debated internally whether any global identifier should be an int, or a string and figured I was just being naive re: strings. I suppose given diku's proclivity for that type of logic, I could formulate a type of..

Model : Serial format and have both a registrant string and a 'serial' int to couple a bit more easily with how the system currently operates, perhaps replacing them over time until the god aweful number system is gone. I wanted to say, earlier, that to me it would be more logical to eschew the idea of a room, mob or object altogether on a fundamental level. In reality they are all objects, as is the game itself really. Each one just shows different aspects of an object, expanding the definition thereof. Typecasting each is very.. I suppose old thinking would be an adequate term. Old as in, in the days of yore, we need to strictly, clearly, and with finality define everything, period.

So, no, I don't think what you've said, or how LP seem to apparently work are horrible at all. They make sense in that, you don't want or need to be in direct control of everything, all the time. That's why we all make utility functions to begin with. I sure as hell didnt want to re-produce my SQL sanitizers, checkers, inserts, etc. so I didn't.

That thinking should be applied more at the fundamental level, but it obviously wasn't.

I had been internally considering, along with the uwid (string or otherwise) beginning to create a prototype object which is any and all objects. It's final form dictated by the characteristics assigned to it. For now/forseeable future it would be a toy and an experiment, but eventually I would be more than happy to entirely remove the standard objects and instead allow the 'builder' to literally shape an object exactly as they wish it to be.

While without first hand experience I do realize some(many) other drivers allow for/and are built on this premise. I also realize that the language I am using now is either not ideal for, or likely outright insane to do this with, but at the end of the day I would rather at least implement things alongside whatever crazy meta project I am working on towards a game I know myself and friends can make. So that may be foolish, but there it is ha.

Getting back to the reality of what I am amidst currently, I don't think I would have a problem in general creating a parser to deal with that sort of string identifier, given it be sanely constructed. A month ago I would have had no idea, but as my meanderings have proved fruitful, there are obviously a plethora of code examples, but most importantly recently I have been working on expanding 'mobprog' strings to parse and recognize a variable construction.

So I could pass the string off to the parser function, and get back a nice clean set of features thereof.

As a question, in LPC, does the object retain all knowledge of where it is, solely, or is that information retained by a third party keeping the 'plane of existence' separate, from the act of existing?
01 Jun, 2014, quixadhal wrote in the 30th comment:
Votes: 0
Bear with me on the long answer here… I'm assuming you have little to no LPMUD knowledge, so I'm giving you the framework to understand my answer. :)

In LPC, everything in the game is an object in the programming sense of the term. That is, the game driver itself mostly handles network I/O, file I/O, and running LPC code. In the early days, the driver maintained a lot of state info about security, containment, lighting, etc… but much of that has gradually been migrated to the LPC mudlibs to give people more direct control over it, and to minimize the need to ever modify or reboot the driver itself.

In a typical LPMUD, the driver tracks the containment relationship of objects. It provides that information via two efuns, environment() and all_inventory(). As you might guess, environment() returns the object that contains the target object (if any), and all_inventory() returns an array of all objects contained by the target object (possibly empty). The driver manages this via a simple move_object() efun.

So, if Fred is holding a bag and wants to give it to Bob, the "give" code boils down to Fred->all_inventory()[bag]->move_object(Bob);
If Fred wanted to drop the bag instead, "drop" would be Fred->all_inventory()[bag]->move_object(environment());
and if Bob wanted to then pick it up, "get" would be environment(Bob)->all_inventory()[bag]->move_object(Bob);

Of course, the details are slightly more messy, since in actual use you'd likely have to find each object to get a reference to it… and you might not want to get the whole inventory list, but that's the basic idea.

So, with that in mind…

Because everything in LPC is softcode, the ID examples I've been using are actually the pathnames of the softcode files themselves. In LPC, every object is identified by the source code used to create it. So, "/d/smurf_village/npc/papa_smurf.c" is actually the code for papa smurf. When the driver sees a reference to such a thing, if it isn't yet loaded it will compile and load that LPC file and return the object reference to it. In the case of singleton objects like daemons or (typically) rooms, you really can do things like "/room/foo.c"->sound_foghorn() in your code.

In the case of npcs and items, you typically don't want a single instance, so the driver provides a clone_object() efun, which creates an instance of that object. That's identified by the same pathanem, but has a "#44" tacked on the end, incrementing the number each time a new clone is created.

So, long story short, it depends on what you mean by "location".

As far as the game driver is concerned, the only location information it cares about is containment. It tracks the inventory/environment relationship of all objects, and that's it. You can even disable that entirely and do it yourself, if you want… which is useful if you want to make a non-room-based game.

OTOH, the "location" of an object in game terms is really given by the pathname of its environment(). So, if you clone an orc, and it wanders around for a while, you might see where it is by something like:

environment(find_object("/d/underdark/npc/orc.c#715"));

It might return "/d/mirkwood/room/forest_path_4.c". So, in game and human terms, the location is a particular section of the map that's a forest path, and maybe that room would have functions to return coordinates, or a distance from your current location, or some other game mechanic that you'd use for positional information.

EDIT: I should also note that the pathnames used above are a convention, and the driver doesn't really care about them. Most mudlibs suggest you split things up so you have areas or zones (called "domains" here) in a hierarchy, and then put rooms/npcs/items/etc in subdirectories… but there's no enforcement of that.

You could have /d/planet/earth/north_america/florida/miami/wharf_district/room/back_alley.c or you could have /room_34738724.c… the driver doesn't care one bit, but for your own sanity, the first one is a bit more useful. :)
01 Jun, 2014, Tyche wrote in the 31st comment:
Votes: 0
syn said:
It had a help integration system, but it did something that to me seemed ludicrous. To find an individual helpfile it did a SELECT * FROM vanda_help

With only it seemed about 10 helpfiles, sure, no big deal, but with even 100, 200, and for well documented MUDs hundreds of helpfiles.. thats insane, to me. To load that amount of data into the mud to then scan through it?

Of course I didnt go with this option, though I did try it to see how badly it would do, mainly it just segfaulted with bad read/write errors/OOB etc. So I went with a targeted where/like search instead.

One needs to be careful with tainted user input when using where/like clauses.
You need to guard against users typing something like the following:

> help "commands';drop table players;–"

So you should always use prepared SQL statements with possibly tainted input, for example:
p_stmt = con->prepareStatement("SELECT level, keyword, text FROM helps WHERE keyword like ?");
p_stmt->setString(1, tainted_string);


It's also good practice to name only the specific columns you are interested in.

This is an SQL example from Murk++ to search help files.
void Character::do_help (std::string argument)
{
sqlite3_stmt *stmt = NULL;

if (argument.empty())
argument = "summary";

char *sql = sqlite3_mprintf("SELECT level, keyword, text FROM helps WHERE level <= %d", level);

if (sqlite3_prepare(g_db->database, sql, -1, &stmt, 0) != SQLITE_OK) {
bug_printf("Could not prepare statement: '%s' Error: %s", sql, sqlite3_errmsg(g_db->database));
sqlite3_free(sql);
return;
}

while (sqlite3_step(stmt) == SQLITE_ROW) {
int lvl = sqlite3_column_int( stmt, 0 );
std::string keyword = (const char*)sqlite3_column_text( stmt, 1 );
std::string text = (const char*)sqlite3_column_text( stmt, 2 );

if (is_name (argument, keyword)) {
if (lvl >= 0 && str_cmp (argument, "imotd")) {
send_to_char (keyword);
send_to_char ("\r\n");
}

send_to_char (text);
sqlite3_finalize(stmt);
sqlite3_free(sql);
return;
}
}

sqlite3_finalize(stmt);
sqlite3_free(sql);
send_to_char ("No help on that word.\r\n");
return;
}


Note that I could have used LIKE, but instead I deferred checking to the original function is_name().
* is_name() might be a candidate for a stored procedure. (Stored procedures aren't supported in SQLite)
* Even better, a redesign of the help table(s) to implement keyword tagging.
01 Jun, 2014, syn wrote in the 32nd comment:
Votes: 0
Those are excellent points, and thankfully I implemented most of that into my help redesign. - I think anyway. Ha

I abstract any help request so the initial help requests hit some checks.

The checks look to see if there are any stray ' in the request, or if the user is looking for a list by category, and from there I do a dump of the name, or name/category if its a categorical search.

Category searches are all hardwired, so while you do help all credits, that just generates an int -> if(f == 3){send_category_list(ch, 3);} which then does a hard SELECT e.g. SELECT entry_num, keywords, category FROM bd_help ORDER BY entry_num WHERE category='credits' for example

This would show an output similar to:

Broken Dreams Help System - Credits:
Entry Number Help Name
[1 Diku][3 Merc]
[24 Mindcloud][25 Godwars]


A regular search I do something similar, but only dumping names, then do an is_name/is_prefix search to look, and then silently send a select based off the name list I obtained, and that way I can do a series of partial matches with a safe name (provided I am not stupid enough to make a help entry drop table (such and such) as a help file..).

In the end the help, after all the initial checks and verifies of whether we want a real help, or a category list, I wound up with the following:


f = sanitize_arg(argument);
// did we find any specials? if so, lets cancel the function and send a message about why..
if(f==1)
{
stc("#WPlease remove any special characters from your search string #0(#RE#rG#0:{4#W!@##$%^&*()_+':><,./?\|{{}[]=-`~#n#0)#W.#n\n\r",ch);
return;
}
// this is where we set the query string with our argument, and do our partial search as well in the form of %s = search %% = wildcard
// eg who or who whois or whois who or whois who whom or whois
//first lets check for the name
passed_arg = str_dup(check_help_arg(argument));
if(!stcmp(passed_arg[0], '\0')) //passed_arg set to null if 0 results - disallow certain bad word combos, drop table, insert / insert into
{
sprintf(buf, "#WNo help entry found for #0'#C%s#0'#W. The Immortals will be notified. You can also search for a category, like help all commands.#n\n", argument);
log_string(LOG_HELP,"Missing help for: %s",passed_arg);
stc(buf,ch);
return;
}
snprintf(query, sizeof(query) +1, "SELECT entry_num, level, category, keywords, text FROM bd_help WHERE keywords=\'%s\' OR keywords LIKE \'%s %%\' OR keywords LIKE \'%% %s\' OR keywords LIKE \'%% %s %%\' OR keywords LIKE \'%% %%%s %%\'OR keywords LIKE \'%% %s%% %%\' OR keywords LIKE \'%s%%\' LIMIT 10", passed_arg, passed_arg, passed_arg, passed_arg, passed_arg, passed_arg, passed_arg);
//sanitize and send the query off to SQL, then set the result set to res
mysql_safe_query(query);
res = mysql_store_result(&db);

while ((row = mysql_fetch_row(res))) // while row (data row from SQL) has a row result from res (result)…
{
int level;
level = atoi(row[1]);// lets do a late check to find level of helpfile to check if the requester can view!
if((ch->level < level) || (ch->trust < level))
{
sprintf(buf, "#WNo help entry found for '%s'#n\n", argument); //sorry bub, no way!
stc(buf,ch);
return;
}
if(e == 0) //are we at base initialization?
{
e = 1; //set our bit to 1 to mark our main result return
stc(" #0-#w=#W[#CB#croke#Cn #CD#cream#Cs #CH#cel#Cp #CS#cyste#Cm#W]#w=#0-#n\n\r",ch);
stc("#C+#0—————————————————————————#C+#n\n\r",ch);
sprintf(buf,"#WH#welp #WF#wile #WF#wound #R- #0[#RE#rntry#0: #R%d#0] [#CK#ceyword#0(#Cs#0): #W%s#0] [#CL#cevel#0: #C%d#0]#n\n\r", atoi(row[0]), row[3], level);
stc(buf,ch);
stc("#C+#0—————————————————————————#C+#n\n\r",ch);
sprintf(buf, "#W%s#n\n\r", row[4]);
stc(buf,ch);
stc("#C+#0—————————————————————————#C+#n\n\r",ch);
extra = TRUE;
}
else //this kicks in after the first loop - if there is one, so following found entries will show up as related helpfiles instead of dumping each helpfile that partially matches search string
{
if(extra == TRUE)
{
sprintf(buf, " #RT#rhe following were also found with partially matching keyword#W(#0s#W)#R:#n\n\r");
stc(buf,ch);
stc("#C+#0—————————————————————————#C+#n\n\r",ch);
extra = FALSE;
}
sprintf(buf, "#0(#GK#geyword #GM#gatch #0- #R%s #0)#R:#n", argument);
stc(buf, ch);
sprintf(buf,"#0[#RE#rntry#0: #R%d#0] [#CK#ceyword#0(#Cs#0): #W%s#0] [#CL#cevel#0: #C%d#0]#n\n\r", atoi(row[0]), row[3], level);
stc(buf,ch);
}
}
mysql_free_result(res); //free the result/memory
return; //done.
}


Given the above, I could probably do this a bit better, but it has seemed to work quite well so far. I tried to comment it, as I made it as thoroughly as possible also..

Holy crap the pasting formatting is horrendous.. its nice and neat in the code, but my keyboard is flipping out so apologies for the horrendous formatting above..
01 Jun, 2014, Tyche wrote in the 33rd comment:
Votes: 0
Methinks you have a memory leak on line 33. You need to call mysql_free_result(res) there as well before returning.
01 Jun, 2014, syn wrote in the 34th comment:
Votes: 0
Ahh thanks!

Much appreciated
01 Jun, 2014, plamzi wrote in the 35th comment:
Votes: 0
Here's a grab bag of utility functions that would be useful:

int query(char *qry) {
mysql_real_query(db, qry, strlen(qry));
if (*mysql_error(db)) {
log_printf("SQL Error: %s", mysql_error(db));
log(qry);
return 0;
}
return 1;
}

void query_printf (char *fmt, …) {
char buf[MAX_STRING_LENGTH];
va_list args;
va_start (args, fmt);
vsprintf (buf, fmt, args);
va_end (args);
query(buf);
}

void get_rows(char *qry, …) {
char buf[MAX_STRING_LENGTH];
va_list args;
va_start (args, qry);
vsprintf (buf, qry, args);
va_end (args);
query(buf);
result = mysql_store_result(db);
}

int get_field(char *field) {
MYSQL_FIELD *f;
if (!result)
return NOTHING;
mysql_field_seek(result, 0);
while ((f = mysql_fetch_field(result)))
if (!strcmp(field, f->name))
return mysql_field_tell(result)-1;
return -1;
}

char *get_first_field(MYSQL_RES *r, char *field) {
mysql_data_seek(r, 0);
row = mysql_fetch_row(r);
result = r;
return row[get_field(field)];
}

/* remember to do you own freeing */
MYSQL_ROW get_row(char *qry) {
query(qry);
result = mysql_store_result(db);
return mysql_fetch_row(result);
}

#define VAL_S(f) row[get_field(f)]
#define VAL_I(f) atoi(row[get_field(f)])
#define VAL_L(f) atol(row[get_field(f)])
#define VAL_F(f) atof(row[get_field(f)])
02 Jun, 2014, syn wrote in the 36th comment:
Votes: 0
Thank's Plamzi,

I hadn't gotten around to making a full utility suite yet, clearly, thats very helpful.
02 Jun, 2014, Pymeus wrote in the 37th comment:
Votes: 0
snprintf(query, sizeof(query) +1, "SELECT entry_num, level, category, keywords, text FROM bd_help WHERE keywords=\'%s\' OR keywords LIKE \'%s %%\' OR keywords LIKE \'%% %s\' OR keywords LIKE \'%% %s %%\' OR keywords LIKE \'%% %%%s %%\'OR keywords LIKE \'%% %s%% %%\' OR keywords LIKE \'%s%%\' LIMIT 10", passed_arg, passed_arg, passed_arg, passed_arg, passed_arg, passed_arg, passed_arg);


The same results could be gotten without so many (relatively) slow wildcard comparisons if expressed as:
"SELECT entry_num, level, category, keywords, text FROM bd_help WHERE keywords LIKE \'%% %s%% %%\' OR keywords LIKE \'%s%%\' LIMIT 10"

It's possible the query optimizer will consolidate the patterns for you, but I wouldn't bet on it. I think it's also a lot easier on the eyes too, but that's just my opinion.

It's worth pointing out that mysql has a mysql_real_escape_string() function that takes care of string escaping issues for you, when used correctly. It even accounts for character encodings other than what you're used to, although we usually ignore such things in mud code, for better or worse.

Also, as a general rule, don't give sizeof()+1 to snprintf. If in your example 'query' is a string pointer, then sizeof() returns the size of the pointer, not the char array it points to, which will truncate the query (probably to "SELE" or "SELECT e"). If it's a statically sized C array instead, then any time the expanded string fills or exceeds the size of the array, 1 byte of memory after the end of the array will be overwritten. In either case, the behavior is probably not what you want.
02 Jun, 2014, Pymeus wrote in the 38th comment:
Votes: 0
On the same subject,
void query_printf (char *fmt, …) {
char buf[MAX_STRING_LENGTH];
va_list args;
va_start (args, fmt);
vsprintf (buf, fmt, args);
va_end (args);
query(buf);
}

might be better written as
void query_printf (char *fmt, …) {
char buf[MAX_STRING_LENGTH];
va_list args;
va_start (args, fmt);
vsnprintf (buf, sizeof(buf), fmt, args);
va_end (args);
query(buf);
}

IF your target environment includes vsnprintf.
02 Jun, 2014, Runter wrote in the 39th comment:
Votes: 0
Just thought I would add that an ORM wrapper gives you this safety out of the box.

this:

Help.all.where(:name.like => "commands';drop table players;")


would be fine.
02 Jun, 2014, Tyche wrote in the 40th comment:
Votes: 0
Runter said:
Just thought I would add that an ORM wrapper gives you this safety out of the box.

this:

Help.all.where(:name.like => "commands';drop table players;")


would be fine.


While that's true, all the non-ORM SQL APIs give you safety out of the box as well.
See my C++ example earlier.

And you have the exact same concerns with ORMs as well

tainted_string = "commands';drop table players;"
Help.where("keywords LIKE '#{tainted_string}'") # subject to injection
Help.where("keywords LIKE ?", tainted_string) # not subject to injection

So just like the C API, you have to actively avoid constructing certain queries that an ORM will also allow.
If you examine the API calls your ORM call makes with your hash syntax you used above above, you will
no doubt find calls to mysql_stmt_prepare() and mysql_stmt_bind_param()
(or whatever appropriate to the database adaptor you're using).

Using prepared queries and binding parameters are the way to go.
There are also significant performance advantages.
…which I didn't take advantage of in the do_help routine I created. ;-)
20.0/47