RE: [Xen-devel] Directly mapping vifs to physical devices in netback -an alternative to bridge

["Santos, Jose Renato G" <joserenato.santos@xxxxxx>]

Ian,

Thanks for the taking the time to look at this.
Comments are embedded on the text below. 

> -----Original Message-----
> From: Ian Pratt [mailto:m+Ian.Pratt@xxxxxxxxxxxx] 
> Sent: Thursday, August 31, 2006 7:40 AM
> To: Santos, Jose Renato G; Xen Devel
> Cc: Turner, Yoshio; G John Janakiraman; ian.pratt@xxxxxxxxxxxx
> Subject: RE: [Xen-devel] Directly mapping vifs to physical 
> devices in netback -an alternative to bridge
> 
> 
> I'm kinda surprised that it doesn't work better than that. We 
> see bridge fns show up a lot on oprofile results, so I'd have 
  Yes, bridge shows up a lot on oprofile results as I pointed
  out in my presentation in the last Xen summit, but this is 
  significantly reduced by disabling netfilter (see results below)

> expected to see more than 1.5% benefit. How are you measuring 
> CPU utilization? Are the dom0/domU on different CPUs?
> 

Yes, dom0 and domU were running on different CPUs. The reported CPU
utilization was for dom0 CPU.

CPU utilization was computed using the total number of oprofile samples
(for unhalted CLOCK cycles) divided by the max number of possible
samples, during a fixed time interval (based on sample rate and CLOCK
frequency)

> Do you get the downgraded bridging performance simply by 
> having CONFIG_BRIDGE_NETFILTER=y in the compiled kernel, or
> do you need to have modules loaded or rules installed? Does 
> ebtables have the same effect?
> 

Yes, simply having CONFIG_BRIDGE_NETFILTER=y (default for xen0 configs)
with no rules or modules installed significantly increases the CPU
utilization. Look at the oprofile results below comparing the three
approaches (filtered based on functions defined in net/bridge/bridge.o
or on the replacement functions for the alternative non bridge
approach). 
I have not tried ebtables 

regards
Renato

=======================================================
Filtered Oprofiles results for the receive case :

Bridge functions from OProfile results
(CONFIG_BRIDGE_NETFILTER=y)
Samples   %       function
2868    1.9956    br_nf_pre_routing
1768    1.2302    br_nf_post_routing
1713    1.1920    br_handle_frame
1389    0.9665    br_nf_forward_ip
1236    0.8600    br_nf_pre_routing_finish
1141    0.7939    br_fdb_update
1138    0.7919    __br_fdb_get
978     0.6805    ip_sabotage_out
771     0.5365    br_handle_frame_finish
593     0.4126    br_nf_forward_finish
526     0.3660    br_dev_queue_push_xmit
494     0.3437    __br_forward
437     0.3041    br_forward
412     0.2867    ip_sabotage_in
388     0.2700    br_forward_finish
299     0.2081    br_nf_dev_queue_xmit
204     0.1419    setup_pre_routing
4       0.0028    br_fdb_cleanup
16359  11.3830  TOTAL                          
=======================================================
Bridge functions from OProfile results
(# CONFIG_BRIDGE_NETFILTER is not set)
Samples   %       function
1256    1.0052    br_handle_frame
1000    0.8003    br_fdb_update
901     0.7211    __br_fdb_get
380     0.3041    br_handle_frame_finish
283     0.2265    br_dev_queue_push_xmit
146     0.1168    __br_forward
121     0.0968    br_forward
93      0.0744    br_forward_finish
4       0.0032    br_fdb_cleanup
1       0.0008    br_hold_timer_expired
1       0.0008    br_send_config_bpdu
4186    3.3500  TOTAL                          
=======================================================
Alternative forwarding functions replacing bridge:
Samples   %       function
2479    2.0423    vifdevmap_rx_packet
573     0.4721    vifdevmap_guest_packet
372     0.3065    vifdevmap_find
297     0.2447    vifdevmap_tx_packet
159     0.1310    __vifdevmap_find
3880    3.1966  TOTAL                          

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel