>
> Good news: Disabling "Large Send Offload" fixes the problem.
>
> How does enabled LSO in the GPLPV driver interact with the LSO implemented
> by
> the NIC driver in Linux? The mentioned problem could be reproduced on
> three
> machines with Broadcom NICs (BCM5708 and BCM715), but wasn't triggered on
> a
> system with an onboard Nvidia (MCP51).
>
It's all a bit of a mystery... but I think it goes something like this:
LSO allows the operating system to send a large (>MTU) TCP packet to the
hardware. The hardware then breaks the packet up into header+MSS sized chunks,
recalculates the sequence numbers and checksums, and sends them. Efficiency
gains are to be had in doing this.
In a DomU, if a packet is sent to the virtual adapter, where it flows onto the
Dom0 Bridge interface, and then onto a real hardware interface, then it is also
more efficient to keep the packet 'large' all the way to the physical nic where
the hardware then finally breaks it up and sends it.
If, instead of going to a real hardware interface, the packet goes to another
DomU, then it is also more efficient to keep the packet 'large' all the way to
the DomU, and never bother breaking it up. Also, because the originating DomU
marked the flag as 'checksum correct', the destination DomU never bothers
checking the checksum, so you save doing the original checksum calculation, and
all the inbetween checksum validation.
There's a lot that could go wrong there isn't there? Particularly if you start
doing NAT or something funny on the bridge!
Windows support Large Send Offload in a mostly compatible way, at least under
Windows 2003 server. XP has a known problem with interaction with the firewall
service, but still seems to work okay once that is disabled. Unfortunately
Windows likes to break the packet up into more pieces than the Linux backend
can handle, so in the GPLPV drivers I allocate some memory pages and copy the
packet data into them. This reduces the number of scatter-gather segments to a
maximum of 16, which also reduces the amount of space each packet takes up on
the ring.
What Windows doesn't support though is the Large Receive Offload, which the xen
network backend driver assumes is supported of LSO is advertised as supported.
So I have to fudge this - if Dom0 sends me a 'large' packet, I break it up
before giving it to Windows.
Also, although Windows says that it supports RX checksum offload, where the
network adapter (or xennet in this case) calculates the checksum and reports on
it's correctness, it turns out that it doesn't really. If the checksum is never
calculated because the packet originated on a Linux DomU, and I just tell
Windows 'the checksum is correct, just trust me', it goes and checks anyway and
then drops the packet when it turns out to not be correct.
The above two paragraphs are why the TX speed from the GPLPV drivers is much
faster (~2x) than RX.
So there is plenty of room for me to have made errors there...
Is anything useful reported if you run debugview from sysinternals?
James
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home