> >>> Apart from that, it all looks fine to me.
> >> Thanks for looking at this. It had been missing the gaze of some
> >> networking-savvy eyes.
> > There is one other potential issue which just occurred to me. These
> > patches assign netifs to groups pretty much arbitrarily, beyond trying
> > to keep the groups balanced. It might be better to try to group
> > interfaces so that the tasklet runs on the same VCPU as the interrupt
> > i.e. grouping interfaces according to interrupt affinity. That would
> > have two main benefits:
> >
> > -- Less cross-VCPU traffic, and hence better cache etc. behaviour.
> > -- Potentially better balancing. If you find that you've accidentally
> > assigned two high-traffic interfaces to the same group, irqbalance
> > or whatnot should rebalance the interrupts to different vcpus, but
> > that doesn't automatically do us much good because most of the work
> > is done in the tasklet (which will still only run on one vcpu and
> > hence become a bottleneck). If we rebalanced the netif groups when
> > irqbalance rebalanced the interrupts then we'd bypass the issue.
> >
> > Of course, someone would need to go and implement the
> > rebalance-in-response-to-irqbalance, which would be non-trivial.
> Your idea is workable if the netfront is bound with a single queue
> NIC via a bridge. Hence we know which interrupt is used to serve the
> netfront, and then we can group netfronts according to the interrupt
> affinity. And as you said, the effort is non-trivial.
>
> However if the NIC is multi-queued, which has only one interface but
> multiple interrupt queues. All netfronts are bounded with this interface
> via one bridge. We have no idea which interrupt queue is serving for
> a specified netfront. So the rebalance according to interrupt affinity
> is a challenge. Do you have idea on this point?
Sorry, I should have been clearer here. When I said ``interrupt'' I
meant the event channel interrupt which the netfront instance will use
to notify netback, not the physical hardware interrupt of whatever
physical NIC is ultimately associated with it. We should always know
which event channel a given netfront is using, and hence which
interrupt, and so we should be able to find out its affinity. In
effect, we'd rebalance in response to messages from the guest to
netback, which is at least vaguely reasonable as a proxy for actual
load.
There are at least three relevant contexts here:
-- Interrupts generated by the hardware
-- The netback tasklets
-- Interrupts generated by the guest
As you say, doing anything based on where hardware interrupts are
being delivered is somewhere between hard an impossible, but it might
be possible to do something useful with the interrupts from the guest.
Steven.
signature.asc
Description: Digital signature
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home