On Sat, Jan 29, 2011 at 06:12:34PM +0100, Bart Coninckx wrote:
> On 01/29/11 16:30, Pasi Kärkkäinen wrote:
> > On Sat, Jan 29, 2011 at 04:27:52PM +0100, Bart Coninckx wrote:
> >
> >> On 01/29/11 16:09, Pasi Kärkkäinen wrote:
> >>
> >>> On Thu, Jan 27, 2011 at 09:35:38AM +0100, Adi Kriegisch wrote:
> >>>
> >>>
> >>>> Hi!
> >>>>
> >>>>
> >>>>
> >>>>>> iSCSI tipically has a quite big overhead due to the protocol, FC, SAS,
> >>>>>> native infiniband, AoE have very low overhead.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> For iSCSI vs AoE, that isn't as true as you might think. TCP offload can
> >>>>> take care of a lot of the overhead. Any server class network adapter
> >>>>> these days should allow you to send 60kb packets to the network adapter
> >>>>> and it will take care of the segmentation, while AoE would be limited to
> >>>>> MTU sized packets. With AoE you need to checksum every packet yourself
> >>>>> while with iSCSI it is taken care of by the network adapter.
> >>>>>
> >>>>>
> >>>> What AoE actually does is sending a frame per block. Block size is 4K --
> >>>> so
> >>>> no need for fragmentation. The overhead is pretty low, because we're
> >>>> talking about Ethernet frames.
> >>>> Most iSCSI issues I have seen are with reordering of packages due to
> >>>> transmission across several interfaces. So what most people recommend is
> >>>> to
> >>>> keep the number of interfaces to two. To keep performance up this means
> >>>> you
> >>>> have to use 10G, FC or similar which is quite expensive -- especially if
> >>>> you'd like to have a HA SAN network (HSRP and stuff like that is
> >>>> required).
> >>>>
> >>>> AoE does not suffer from those issues: Using 6 GBit interfaces is no
> >>>> problem at all, load balancing will happen automatically, as the load is
> >>>> distributed equally across all available interfaces. HA is very simple:
> >>>> just use two switches and connect one half of the interfaces to one
> >>>> switch
> >>>> and the other half to the other switch. (It is recommended to use
> >>>> switches
> >>>> that can do jumbo frames and flow control)
> >>>> IMHO most of the current recommendations and practises surrounding iSCSI
> >>>> are there to overcome the shortcomings of the protocol. AoE is way more
> >>>> robust and easier to handle.
> >>>>
> >>>>
> >>>>
> >>> iSCSI does not have problems using multiple gige interfaces.
> >>> Just setup multipathing properly.
> >>>
> >>> -- Pasi
> >>>
> >>>
> >>> _______________________________________________
> >>> Xen-users mailing list
> >>> Xen-users@xxxxxxxxxxxxxxxxxxx
> >>> http://lists.xensource.com/xen-users
> >>>
> >>>
> >> On this subject: am using multipathing to iSCSI too, hoping to have
> >> aggregated speed on top of path redundancy but the speed seems not to
> >> surpass the one of a single interface.
> >>
> >> Is anyone successful at doing this?
> >>
> >>
> > You're benchmarking sequential/linear IO, using big blocksizes, right?
> >
> > Some questions:
> > - Are you using multipath round robin path policy?
> > - After how many IOs do you switch paths? You might need to lower the
> > min_ios.
> >
> >
> > -- Pasi
> >
> >
> Hi Pasi,
>
> the benchmarking was intuitively done, with just dd and bonnie++.
>
with dd, use "bs=1024k" or so to force big blocks.
> It is indeed rr, this is a part of my multipath.conf:
>
> defaults {
> udev_dir /dev
> polling_interval 10
> selector "round-robin 0"
> path_grouping_policy multibus
> getuid_callout "/lib/udev/scsi_id --whitelisted
> --device=/dev/%n"
> prio const
> path_checker directio
> rr_min_io 100
> max_fds 8192
> rr_weight priorities
> failback immediate
> no_path_retry 5
> user_friendly_names no
> }
>
> should the "100" go down a bit?
>
Yeah, for example Equallogic enterprise iSCSI storage
manuals recommends rr_min_io of "3".
That should allow utilizing both paths.
-- Pasi
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home