14 Aug, 2011, Bojack wrote in the 1st comment:
Votes: 0
Hey guys, im trying to get some help with valgrind, ive forgot how to use it well. Our mud always runs at 3.0 cpu percent with 3.2 mem percent and ive always thought that was high but it never increases more then that. We have 12000+ rooms so ive always wondered if it was a leak or just normal. Can anyone give me an example of how to use it correctly?
14 Aug, 2011, Omega wrote in the 2nd comment:
Votes: 0
valgrind -v –tool=memcheck –leak-check=full –show-reachable=yes –demangle=yes –db-attach=yes –num-callers=10 –track-origins=yes –track-fds=yes ./<engine>


Now, with that said; set in your mud's SHUTDOWN to remove all data/objects/etc from memory, parse all your lists and purge the data out.

Then load your mud, let it run for a day or so under valgrind; and then shutdown your mud.

It will give you quite the detailed report on where memory was created (and if it is still reachable) else wise, it'll tell you there are no leaks.

Mind you, thats for if you suspect memory leaks. Ultimately though, you may need to profile your code and see if it is optimized properly. Find out where it is bottle necking; since I don't know what mud-base your running; I cannot tell you, but in rom; aggr_update is about 40% of the overall CPU usage. So look at functions that globally parse; and perhaps dumb them down, or atleast slow them down (frequency) and you'll notice a huge drop in your CPU.

With that said, without seeing the valgrind result, I cannot help with the memory issue; but I know that when I did that with my ROM mud the first time; I damn near died; ROM is a giant memory leak (just my personal opinion) Anyways, yeah, thats your best option; setup a series of functions to close out all memory at shutdown; and you'll find out where things were loaded into memory and not free'd. Or double allocations; or simply forgotten to free. And that may be your memory issue, as for CPU, it can get effected with an extremely large pull from the mem usage.

Anyways; CPU usage can be optimized down; although may require sacrifice. Or use of the dreaded threads :P personal opinion.
14 Aug, 2011, plamzi wrote in the 3rd comment:
Votes: 0
If your mud uses 3% and doesn't run on a fascist machine, I wouldn't go leaping into optimization efforts.

In general, I think memory leaks are a much bigger concern than CPU usage (unless CPU usage spikes up to 100%). Even if you have your own server, bad memory leaks can bring it to its knees, so they're worth plugging. Look at whether your production server's RAM usage grows over time, and if it doesn't, I'd suggest worrying about other things.
14 Aug, 2011, Runter wrote in the 4th comment:
Votes: 0
re using threads for lower cpu

Using threads will not lower CPU usage. It'll use more with the benefit of potentially spreading it across multiple cores and/or concurrency.
14 Aug, 2011, Rarva.Riendf wrote in the 5th comment:
Votes: 0
Quote
Then load your mud, let it run for a day or so under valgrind; and then shutdown your mud.

Err just set the PULSE the lower you can ;p (I pretty much did everything you advice..end ended up rewriting all the memory management to free the memory when needed instead of using the lousy factory pattern included in ROM)


Quote
We have 12000+ rooms so ive always wondered if it was a leak or just normal

My distant ROM based code handle 14k room with 30meg of RAM

an example of what should be done before exiting a ROM based code.

void clear_memory() {
int i;
//clear socials
for (i=0; i< MAX_SOCIALS; i++) {
free_string(&social_table[i].char_no_arg);
free_string(&social_table[i].others_no_arg);
free_string(&social_table[i].char_found);
free_string(&social_table[i].others_found);
free_string(&social_table[i].vict_found);
free_string(&social_table[i].char_not_found);
free_string(&social_table[i].char_auto);
free_string(&social_table[i].others_auto);
}
//clear clans
CLAN_DATA *clan, *clan_next;
for (clan = clan_list;clan ;clan = clan_next) {
clan_next = clan->next;
free_clan(clan);
}

//clear legend and pk board
for (i = 0;i < LEGEND_BOARD_SIZE;i ++)
free_string( &pLeg[i].name);
for (i = 0;i < PK_BOARD_SIZE;i ++) {
free_string( &pPK[i].name);
free_string( &pPK[i].rank_name);
}

//let see if we did our job correctly till yet
//clear the char/mobs loaded
CHAR_DATA *ch, *ch_next;
for (ch = char_list;ch ;ch = ch_next) {
ch_next = ch->next;
extract_char(ch, TRUE, CHAR_RETURN_LIST);
free_char(ch);
}

char_list = NULL;

//clear the loaded objects
OBJ_DATA *obj, *obj_next;
for (obj = object_list;obj ;obj = obj_next) {
obj_next = obj->next;
extract_obj(obj);
free_obj(obj);
}
object_list = NULL;

//clear the object index
int iHash;
OBJ_INDEX_DATA *pObjIndex, *pObjIndex_next;
for (iHash = 0;iHash < MAX_KEY_HASH;iHash ++)
for (pObjIndex = obj_index_hash[iHash];pObjIndex ;pObjIndex = pObjIndex_next) {
pObjIndex_next = pObjIndex->next;
free_obj_index(pObjIndex);
}

DISABLED_DATA *disabledD, *disabledD_next;
for (disabledD = disabled_first;disabledD;disabledD = disabledD_next) {
disabledD_next = disabledD->next;
free_string( &disabledD->disabled_by);
free(disabledD);
}
disabled_first = NULL;

DIS_SPELL_DATA *disabledSpellD, *disabledSpellD_next;
for (disabledSpellD = dis_spell_first;disabledSpellD;disabledSpellD = disabledSpellD_next) {
disabledSpellD_next = disabledSpellD->next;
free_string( &disabledSpellD->disabled_by);
free(disabledSpellD);
}
dis_spell_first = NULL;

HELP_DATA *help, *help_next;
for (help = help_first;help;help = help_next) {
help_next = help->next;
free_help(help);
}
help_first = NULL;
help_last = NULL;

//clear the mob_index
MOB_INDEX_DATA *pMobIndex, *pMobIndex_next;
for (iHash = 0;iHash < MAX_KEY_HASH;iHash ++)
for (pMobIndex = mob_index_hash[iHash];pMobIndex ;pMobIndex = pMobIndex_next) {
pMobIndex_next = pMobIndex->next;
free_mob_index(pMobIndex);
}

//clears shop (should be none left as when you extract a char it extracts the shop as well
SHOP_DATA *shop, *shop_next;
for (shop = shop_last;shop;shop = shop_next) {
shop_next = shop->next;
log_param("Extracting shop from %d", shop->keeper);
extract_shop(shop);
}
shop_first = NULL;
shop_last = NULL;

//clear the room_index
int vnum = 0;
AREA_DATA *pArea, *pArea_next;
for (pArea = area_first;pArea ;pArea = pArea_next) {
pArea_next = pArea->next;
log_param("Unloading area:%s", pArea->name);
for (vnum = pArea->lvnum;vnum <= pArea->uvnum;vnum ++)
extract_room_index(get_room_index(vnum));
free_area(pArea);
}

free(top_string);
free(log_file);
exit(0);
}
14 Aug, 2011, Runter wrote in the 6th comment:
Votes: 0
If you want to lower the CPU usage look into the aggressive_update and fight_update functions. (I think that's what they're called.) The better way for aggressive npcs is just storing a list of only npcs that are aggressive and testing against it. A better way for the fight update is similarly storing only folks who are currently in combat. I seem to remember that these two functions on a developed mud using rom (and probably other codebases) was up to 60% of all execution time when profiling.
14 Aug, 2011, Omega wrote in the 7th comment:
Votes: 0
Yeah, violence_update is a beast, but the aggr_update is the worst, I rewrote both of them on my old rom mud and it saved big-time.

As for Rarva.Riendf, yeah, thats what I did as well with the cleanup_mud function; purged out everything opposed to using the old fashioned free_list b/s. But thats just me.

My recommendation was and always will be, read your code, followed by understand what you read. Then you can write better code.
That is the biggest help when it comes to debugging, optimizing, memory leak prevention, it all adds up together and understanding is at the core of it.
15 Aug, 2011, oenone wrote in the 8th comment:
Votes: 0
You don't have to clean up your memory if the program is shut down afterwards.. It's a good style, but unnecessary, since the OS will give free all memory the process used.
15 Aug, 2011, Omega wrote in the 9th comment:
Votes: 0
Oenone, thats not the point, with VALGRIND, if you clear out the memory; on shutdown valgrind will show you any memory left allocated. IE, it finds your leaks.

Double allocations, etc, all show up because you cleared out everything that is normally allocated.

Example
ch->name = strdup("name");
ch->name = strdup("othername");


on shutdown, when you free ch->name, it will still have an active pointer, pointing to the hanging data.

Ie, the first strdup call is a leak, and it will show it up. Mind you, leaks normally aren't as obvious as the one I generated here; but the point is the same. When you shutdown and you clear out all your linked lists, and their associated data. You will find all your points that are leaking. Which is why I recommended booting it up, and letting it run for a day or two of active play then shutting down, as you will get a nice detailed log of where memory is leaking. Which is part of the initial post; addressing the cpu and memory usage. This may directly show where it is going wrong; if not, it will at least show the potential memory links/loss or a clean shutdown if everything is cleaned out properly.
15 Aug, 2011, RoFAdmin wrote in the 10th comment:
Votes: 0
Oenone,

While what you are saying is true, it doesnt apply well here.

The reason it has been said to clean everything up before the mud shutdowns is for the benefit of valgrind.
If you don't do this valgrind is gonna complain about a lot of things when the server goes down because you have allocated memory you have not explicitly free'd even though it is going to get free'd anyway.
15 Aug, 2011, oenone wrote in the 11th comment:
Votes: 0
Ah, okay.. For this purpose, it really is helpful to free everything.
16 Aug, 2011, David Haley wrote in the 12th comment:
Votes: 0
Well, sort of.

Valgrind will simply tell you you haven't freed those. That is different from a memory leak – Valgrind prints out a different message for that, along the lines of "memory allocated <here> has been permanently lost" – in other words, when you lose the reference to it.

Personally I find it easier to clean up and I think it's good practice to have proper cleanup routines anyhow (and not force yourself to reboot to return to a clean slate). It also makes the output easier to sift through. But the kind of memory leaks you're talking about here are not actual memory leaks, they're just memory that hasn't been freed. A leak is when you lose track of allocated memory.
16 Aug, 2011, Vigud wrote in the 13th comment:
Votes: 0
Quote
Personally I find it easier to clean up and I think it's good practice to have proper cleanup routines anyhow (and not force yourself to reboot to return to a clean slate).
Excuse me this unrelated question, but are you implying that if I keep starting a program that allocates 20 megabytes and exits without freeing that allocated memory, I will eventually run out of virtual memory?
16 Aug, 2011, David Haley wrote in the 14th comment:
Votes: 0
I was talking about rebooting the MUD, not the OS, since that is what this part of the thread has been about…
17 Aug, 2011, Runter wrote in the 15th comment:
Votes: 0
It should be mentioned that using the exotic memory pool allocation scheme with some of these muds that valgrind may be tricked into believing a leak hasn't occurred when in fact it has. Since the entire block of memory is allocated at once and does have a pointer to it. I'd imagine valgrind wouldn't be able to tell what is and isn't free since in this scheme you must manage it yourself.
17 Aug, 2011, David Haley wrote in the 16th comment:
Votes: 0
Good point Runter – since it manages some of its own memory, it might have a leak in its internal reference rather than an actual pointer that Valgrind can see.
17 Aug, 2011, Rarva.Riendf wrote in the 17th comment:
Votes: 0
Runter said:
It should be mentioned that using the exotic memory pool allocation scheme with some of these muds that valgrind may be tricked into believing a leak hasn't occurred when in fact it has. Since the entire block of memory is allocated at once and does have a pointer to it. I'd imagine valgrind wouldn't be able to tell what is and isn't free since in this scheme you must manage it yourself.

Yep in the base ROM code you will probably not detect any leak as it uses a factory pattern. Instead of a leak you have strange behaviours (using a pointer that is marked ->valid == false as en example).

Quote
Valgrind will simply tell you you haven't freed those. That is different from a memory leak – Valgrind prints out a different message for that, along the lines of "memory allocated <here> has been permanently lost" – in other words, when you lose the reference to it.

Nod but thing is you will not be sure that everything that should have been freed is. If you clear everything yourself,and still have stuff that was not freed before exiting, you can spot lists that can still be accessed but actually will not be used ever again. (Imagine some variable only needed at boot that you never free, waste of memory if it can be freed once done (sometimes global can be handy even for a short period of time..even if it is not 'good code')
17 Aug, 2011, David Haley wrote in the 18th comment:
Votes: 0
I'm not sure I'd call it a factory pattern (that is something else entirely really), it's a custom allocation scheme. But yes, we all agree that in this case it does funky things to memory management.

Rarva said:
Nod but thing is you will not be sure that everything that should have been freed is.

That is not the same as a memory leak, though. Memory leaks are pretty universally defined as allocated memory that has been "lost", and cannot be safely recovered or otherwise unallocated.

Rarva said:
(Imagine some variable only needed at boot that you never free, waste of memory if it can be freed once done (sometimes global can be handy even for a short period of time..even if it is not 'good code')

Yes, this is an interesting example, and it is indeed wasted memory, but it is not a memory leak.
17 Aug, 2011, Runter wrote in the 19th comment:
Votes: 0
Generally when we talk about leaks we talk about also something that isn't a one time erroneous expense, but a continual erroneous expense. I mean, I think the term leak means a hole in a bucket that empties the bucket over time. Spilling some from the bucket isn't really a leak.

Well, I think both uses of the term are correct. I just think one is more correct than the other. :)

Also, in garbage collection languages it's common to end up with memory sticking around longer than it is needed because it has a reference still alive to it. Even if it's not global. We don't consider this a memory leak, even if if there's worthless memory usage going on.
17 Aug, 2011, Rarva.Riendf wrote in the 20th comment:
Votes: 0
Quote
Rarva said:
(Imagine some variable only needed at boot that you never free, waste of memory if it can be freed once done (sometimes global can be handy even for a short period of time..even if it is not 'good code')

Yes, this is an interesting example, and it is indeed wasted memory, but it is not a memory leak.

It is a leak as it is memory you won't recover though it wont be used ever again. (You could, but you dont, so it is the same effect than a dereferenced pointer)
And you can have enough of them to go past your allocated memory. That is why your cleaning method should clear everything you know you are actually using.
Everything else that has not been freed is then very suspect. (and as you said yourself it is way easier to sort Valgrind messages then)
and a mud should not have a single warning. I read that something as huge as OpenOffice has very few, if any.
0.0/33