Re: Udp sending performance in Gbit Ethernet

"Stephan Wolf [MVP]" wrote:

> Not sure whether your test program or setup is correct. You should
> better use some well-known test program like TTCP. Various
> implementations of TTCP are available for Windows, see e.g.
>
> "Test TCP (TTCP) Benchmarking Tool for Measuring TCP and UDP
> Performance"
> http://www.pcausa.com/Utilities/pcattcp.htm

Here are the results:

>wsttcp -t -u -l1024 -n1000000 -p1234 192.168.2.201
wsttcp-t: buflen=1024, nbuf=1000000, align=16384/+0, port=1234 udp ->
192.168.2.201
wsttcp-t: socket
wsttcp-t: 1024000000 bytes in 23.22 real sec = 43068.18 KB/sec (352814505.36
bps)
wsttcp-t: 1000006 I/O calls, msec/call = 0.02, calls/sec = 43068.44
1024000000 1137252422.38 1137252445.60 23.22 352814505.36

>wsttcp -t -u -l1025 -n1000000 -p1234 192.168.2.201
wsttcp-t: buflen=1025, nbuf=1000000, align=16384/+0, port=1234 udp ->
192.168.2.201
wsttcp-t: socket
wsttcp-t: 1025000000 bytes in 67.05 real sec = 14929.48 KB/sec (122302265.57
bps)
wsttcp-t: 1000006 I/O calls, msec/call = 0.07, calls/sec = 14915.00
1025000000 1137252450.74 1137252517.78 67.05 122302265.57

>ipconfig /all

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Compaq NC7131 Gigabit Server
Adapter #2
Physical Address. . . . . . . . . :
Dhcp Enabled. . . . . . . . . . . : No
IP Address. . . . . . . . . . . . : 192.168.2.200
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
DNS Servers . . . . . . . . . . . : 195.74.0.47
NetBIOS over Tcpip. . . . . . . . : Disabled

.... So the performance drops about 65%.

> We've been using the UDP tests with WSTTCP for years and IIRC, we got
> it to 100% load our Gigabit network (using cards with the Marvell
> "Yukon" GigE chipset).

Okay, what were the settings? I have achieved wire speed also, BUT with 1024
byte packet size (and faster interface than PCI).

> The performance drop you describe can actually be caused by an
> increased number of interrupts. However, modern NICs usually implement
> "interrupt moderation" so that should not actually be a problem.

I did not understand your point, but I did run performance monitor when
running the tests. The results were:

- With 1024 byte packets, Interrupts/sec was approximately 5450
- With 1024 byte packets, Interrupts/sec was approximately 15100

So,

1) The 1025 byte packets really did increase the # of interrupts by factor
of 3 compared to 1024 byte packets
2) The 1025 byte packets were again sent at 1 packet per interrupt. For 1024
byte packets this ratio was about 8...

I again kindly ask you to read the previous posts and check the previous
threads from google. This phenomenon has been detected in multiple
configurations and multiple windows versions.

Clearly the posts from "m" show that there is a work-around for this, and I
will investigate it when I have the time. But, it still does not explain, why
the naive approach to the UDP sending has so dramatic drop of performance at
1024->1025 byte transition.

Greetings,

Juha

.

Relevant Pages

  • Re: Udp sending performance in Gbit Ethernet
    ... Not sure whether your test program or setup is correct. ... implementations of TTCP are available for Windows, ... This seems to be connected to the NIC interrupts in some way, ... > it seems like that 1024-byte packets can be sent at full rate, ...
    (microsoft.public.development.device.drivers)
  • [PATCH 19-rc1] Fix typos in /Documentation : T
    ... If the request is not yet sent to userspace AND the signal is not ... +NAPI works by shutting down event interrupts when there's work and ... The core layer invokes devices to send packets in a round robin format. ... Correct a reference to free'ed memory during controller ...
    (Linux-Kernel)
  • Re: Handling 100.000 packets/sec or more
    ... |goes as|high as 10 and the interface is losing packets ... the interface won't send interrupts to ...
    (freebsd-net)
  • Re: Handling 100.000 packets/sec or more
    ... Adrian Penisoara writes: ... the interface won't send interrupts to ... total number of mbufs available to the system. ... We got ingress filtering so the packets go no further ...
    (freebsd-net)
  • Re: Polling tuning and performance
    ... I'm not exactly tied to polling. ... Never mind the fact that bgeand the underlying hardware ... sucking in packets as fast as possible with minimal loss. ... Context switches and interrupts are both shown. ...
    (freebsd-performance)