<!-- MHonArc v2.4.4 --> <!--X-Subject: [MUD-Dev] [Tech] socket slowdown solved --> <!--X-From-R13: Quevf Uenl <ptNnzv-pt.UenlEntr.Sqzbagba.OP.QO> --> <!--X-Date: Tue, 13 Oct 1998 22:25:43 -0700 --> <!--X-Message-Id: 199810140526.XAA16572@ami-cg.GraySage.Edmonton.AB.CA --> <!--X-Content-Type: text/plain --> <!--X-Head-End--> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> <html> <head> <title>MUD-Dev message, [MUD-Dev] [Tech] socket slowdown solved</title> <!-- meta name="robots" content="noindex,nofollow" --> <link rev="made" href="mailto:cg#ami-cg,GraySage.Edmonton.AB.CA"> </head> <body background="/backgrounds/paperback.gif" bgcolor="#ffffff" text="#000000" link="#0000FF" alink="#FF0000" vlink="#006000"> <font size="+4" color="#804040"> <strong><em>MUD-Dev<br>mailing list archive</em></strong> </font> <br> [ <a href="../">Other Periods</a> | <a href="../../">Other mailing lists</a> | <a href="/search.php3">Search</a> ] <br clear=all><hr> <!--X-Body-Begin--> <!--X-User-Header--> <!--X-User-Header-End--> <!--X-TopPNI--> Date: [ <a href="msg00195.html">Previous</a> | <a href="msg00197.html">Next</a> ] Thread: [ <a href="msg00274.html">Previous</a> | <a href="msg00216.html">Next</a> ] Index: [ <A HREF="author.html#00196">Author</A> | <A HREF="#00196">Date</A> | <A HREF="thread.html#00196">Thread</A> ] <!--X-TopPNI-End--> <!--X-MsgBody--> <!--X-Subject-Header-Begin--> <H1>[MUD-Dev] [Tech] socket slowdown solved</H1> <HR> <!--X-Subject-Header-End--> <!--X-Head-of-Message--> <UL> <LI><em>To</em>: <A HREF="mailto:mud-dev#kanga,nu">mud-dev#kanga,nu</A></LI> <LI><em>Subject</em>: [MUD-Dev] [Tech] socket slowdown solved</LI> <LI><em>From</em>: Chris Gray <<A HREF="mailto:cg#ami-cg,GraySage.Edmonton.AB.CA">cg#ami-cg,GraySage.Edmonton.AB.CA</A>></LI> <LI><em>Date</em>: Tue, 13 Oct 1998 23:26:06 -0600</LI> <LI><em>Reply-To</em>: <A HREF="mailto:mud-dev#kanga,nu">mud-dev#kanga,nu</A></LI> </UL> <!--X-Head-of-Message-End--> <!--X-Head-Body-Sep-Begin--> <HR> <!--X-Head-Body-Sep-End--> <!--X-Body-of-Message--> <PRE> Back in August (I was checking the archives), under the thread "Ethernet NICS, maximum connections..mud testing", I had indicated a performance problem I was having with communication between my server and client, when running on the same Linux machine. Very little of the CPU time was used, but there was no explanation for the slowdown. Well, yesterday was the day I finally got a small test program to display the same behaviour (it relates to mixing requests with replies with requests without replies). I emailed it to myself at work to try out on some other UNIX variants. Same result on all of them! It turns out that today I was also chasing UDP performance problems for work things, and in my investigation, I came across the TCP_NODELAY option to setsockopt for the TCP protocol layer. Since the work stuff is UDP, not TCP, that was no help, but I decided to try it on my test client and server. Voila! Drastic improvement in performance. I've just put it into the real server and client here at home, and the speedup is around a factor of 10! At last! So, I'm posting this to the list in case anyone else runs into this same problem, and doesn't find the answer in a handy book. Note that the Linux setsockopt man page does mention this, and my Linux at least doesn't have a 'udp' page in section 4p (no 4p at all!). SunOS4 had one dated 1987 that explains this, so its not a new thing. So what is going on? There is a "Nagle algorithm" in the TCP code on (most? all?) UNIX boxes. It is intended to cut down on the sending of small packets on the network, thus reducing network load and system load. It does this by waiting a small amount of time (I'm betting its 20ms on this box) to see if there will be another packet following soon. If there is, it combines them. There appears to be some heuristics in there, since testing with a program that *just* sends small packets sees no slowdown (or perhaps there is one, but because the server and client are not synchronizing, its effect is masked). A program that does only send/reply pairs also is not slowed down, perhaps because the TCP code recognizes the situation and sets some flags or something. However, if your traffic consists of alternating requests with replies and requests without replies, the slowdown shows up. Doing on = 1; setsockopt(socket, IPPROTO_TCP, TCP_NODELAY, &on, sizeof(int)); tells the system to disable that heuristic for that socket. Note that this is not related to things like FIONBIO which make a socket a non-blocking socket. In my simple test program, I only needed to do this in my client, but in the real thing, I needed it in both the client and the server. Since this does put an additional burden on the machine and network, it may be advisable to only do it for local connections (where the client and server are on the same machine), or where the connection is *very* fast. Since I see a delay of 20ms here, and my fastest 'ping' time to another machine is 30ms, I probably should do it only for local connections. Also, in my case it only matters at all when the client is being used to source 30,000 lines of MUD-code to built the world. Now I can get on with real work on my MUD! -- Chris Gray cg#ami-cg,GraySage.Edmonton.AB.CA </PRE> <!--X-Body-of-Message-End--> <!--X-MsgBody-End--> <!--X-Follow-Ups--> <HR> <ul compact><li><strong>Follow-Ups</strong>: <ul> <li><strong><A NAME="00216" HREF="msg00216.html">[MUD-Dev] Re: [Tech] socket slowdown solved</A></strong> <ul compact><li><em>From:</em> J C Lawrence <claw#kanga,nu></li></ul> </UL></LI></UL> <!--X-Follow-Ups-End--> <!--X-References--> <!--X-References-End--> <!--X-BotPNI--> <UL> <LI>Prev by Date: <STRONG><A HREF="msg00195.html">[MUD-Dev] Re: Current Projects</A></STRONG> </LI> <LI>Next by Date: <STRONG><A HREF="msg00197.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World Design)</A></STRONG> </LI> <LI>Prev by thread: <STRONG><A HREF="msg00274.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World D esign)</A></STRONG> </LI> <LI>Next by thread: <STRONG><A HREF="msg00216.html">[MUD-Dev] Re: [Tech] socket slowdown solved</A></STRONG> </LI> <LI>Index(es): <UL> <LI><A HREF="index.html#00196"><STRONG>Date</STRONG></A></LI> <LI><A HREF="thread.html#00196"><STRONG>Thread</STRONG></A></LI> </UL> </LI> </UL> <!--X-BotPNI-End--> <!--X-User-Footer--> <!--X-User-Footer-End--> <ul><li>Thread context: <BLOCKQUOTE><UL> <LI><STRONG>[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World D esign)</STRONG>, <EM>(continued)</EM> <ul compact> <ul compact> <ul compact> <LI><strong><A NAME="00249" HREF="msg00249.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World D esign)</A></strong>, Steve Sparks <a href="mailto:ssparks#enigma,sss.org">ssparks#enigma,sss.org</a>, Mon 19 Oct 1998, 22:36 GMT <UL> <LI><strong><A NAME="00260" HREF="msg00260.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World D esign)</A></strong>, Vadim Tkachenko <a href="mailto:vt#freehold,crocodile.org">vt#freehold,crocodile.org</a>, Tue 20 Oct 1998, 03:31 GMT <UL> <LI><strong><A NAME="00262" HREF="msg00262.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World</A></strong>, Jon A. Lambert <a href="mailto:jlsysinc#ix,netcom.com">jlsysinc#ix,netcom.com</a>, Tue 20 Oct 1998, 04:37 GMT </LI> <LI><strong><A NAME="00274" HREF="msg00274.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World D esign)</A></strong>, Steve Sparks <a href="mailto:ssparks#enigma,sss.org">ssparks#enigma,sss.org</a>, Tue 20 Oct 1998, 17:09 GMT </LI> </UL> </LI> </UL> </LI> </ul> </ul> </ul> </LI> <LI><strong><A NAME="00196" HREF="msg00196.html">[MUD-Dev] [Tech] socket slowdown solved</A></strong>, Chris Gray <a href="mailto:cg#ami-cg,GraySage.Edmonton.AB.CA">cg#ami-cg,GraySage.Edmonton.AB.CA</a>, Wed 14 Oct 1998, 05:25 GMT <UL> <LI><strong><A NAME="00216" HREF="msg00216.html">[MUD-Dev] Re: [Tech] socket slowdown solved</A></strong>, J C Lawrence <a href="mailto:claw#kanga,nu">claw#kanga,nu</a>, Thu 15 Oct 1998, 06:02 GMT </LI> </UL> </LI> <LI><strong><A NAME="00192" HREF="msg00192.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World Design)</A></strong>, Koster, Raph <a href="mailto:rkoster#origin,ea.com">rkoster#origin,ea.com</a>, Tue 13 Oct 1998, 23:14 GMT <UL> <li><Possible follow-up(s)><br> <LI><strong><A NAME="00194" HREF="msg00194.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World Design)</A></strong>, Chris Gray <a href="mailto:cg#ami-cg,GraySage.Edmonton.AB.CA">cg#ami-cg,GraySage.Edmonton.AB.CA</a>, Wed 14 Oct 1998, 01:41 GMT <UL> <LI><strong><A NAME="00202" HREF="msg00202.html">[MUD-Dev] Re: Trusting the Client (Re: Laws of Online World Design)</A></strong>, J C Lawrence <a href="mailto:claw#under,engr.sgi.com">claw#under,engr.sgi.com</a>, Wed 14 Oct 1998, 17:32 GMT </LI> </UL> </LI> </UL> </LI> </UL></BLOCKQUOTE> </ul> <hr> <center> [ <a href="../">Other Periods</a> | <a href="../../">Other mailing lists</a> | <a href="/search.php3">Search</a> ] </center> <hr> </body> </html>