TCP window size (zz) - [Networking]

TCP window size (zz) - [Networking]

How many of you have notice that no matter you have a 100 Mb line you
only get near 2 Mbps, well that is
because the TCP window size is only of 32 kB or 64 kB on Linux (depends
on distribution) and 8 kB on M$ Windows.

The TCP window size is the amount of data that will be send on a
connection before a host stops and waits
for an acknowledgment. This is used by TCP to prevent congestion.
Ideally it should be:

Window size = Bandwidth x round trip time


@@WARNING@@
* If your window size is too small, you won't use the network to it's
full capacity
* If your window size is too big, you risk overloading the network
and creating congestion and packet loss
* On a WAN, setting the TCP window size correctly plays a big part in
getting good performance
(it can easily double performance or more)


The peak bandwidth of the link is typically expressed in Mbit/s. The
round-trip delay for a link can be measured with traceroute, and for
high-speed WAN
links is typically between 10 msec and 100 msec. For a 60 msec, 120 Mbps
path, the bandwidth*delay product would be 7200 kbit, or 900 kByte (kB).

...so here is how to change the
TCP Window size on Linux in order to achieve higher bandwidth.

#cd /proc/sys/net/core
#ls
message_burst netdev_max_backlog rmem_default wmem_default
message_cost optmem_max rmem_max wmem_max
-------
(The secret are on these files)
/proc/sys/net/core/rmem_default - default receive window
/proc/sys/net/core/rmem_max - maximum receive window
/proc/sys/net/core/wmem_default - default send window
/proc/sys/net/core/wmem_max - maximum send window
--------
# cat wmem_default wmem_max rmem_default rmem_max
65535
65535
65535
65535
#
(If you change these numbers you are changing the TCP window)

The theorical values are 65535 on all of them because asume bandwidth of
100
Mbits/s and the round trip time was 5 msec, the TCP window should be

(100x10^6) bytes/sec * (5x10^-3) sec = 65000 bytes or 65 kilobytes

or

500x10^3 bits (65 kilobytes)


But imagine right now we (the UPR) have a DS3 (45Mbit/sec) with Sprint
and the average
round trip is 115 ms (do ping to anywhere outside and you will get
higher numbers)

So the computation will be:

45 Mbit/sec * 115 ms
= 45e6 * 115e-3
= 5,175,000 bits / 8 / 1024
= 631 KBytes

That means that our ideal TCP Window is 631 KBytes.
# cat 646875 > /proc/sys/net/core/wmem_max
# cat 646875 > /proc/sys/net/core/wmem_default
# cat 646875 > /proc/sys/net/core/rmem_max
# cat 646875 > /proc/sys/net/core/rmem_default


Well, hope this works for you. Note that this is not using the Internet2
link yet. Probably will
require a much smaller TCP Window. Please let me know if you find any
difference on performance.

DISCLAIMER: These are teorical numbers and are not guaranty to work for
everyone in the same way.


For M$ Windows 9x users please refer to:
http://moat.nlanr.net/Software/TCPtune/


REFERENCES:
http://dast.nlanr.net/Articles/GettingStarted/TCP_window_size.html
http://ncne.nlanr.net/research/tcp/testrig/
A very useful presentation:
http://ncne.nlanr.net/training/techs/1998/980128/talks/welch

General Info:
http://www.ncsa.uiuc.edu/People/vwelch/net_perf/tcp_windows.html
http://www.psc.edu/networking/perf_tune.html (outdated 1999 but useful)

http://www.ncsa.uiuc.edu/People/vwelch/net_perf_tools.htm