WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: RE: RE: [Xen-devel] poor domU VBD performance.

To: Ian Pratt <m+Ian.Pratt@xxxxxxxxxxxx>
Subject: Re: RE: RE: [Xen-devel] poor domU VBD performance.
From: Jens Axboe <axboe@xxxxxxx>
Date: Thu, 31 Mar 2005 09:10:43 +0200
Cc: Xen development list <xen-devel@xxxxxxxxxxxxxxxxxxx>, Kurt Garloff <garloff@xxxxxxx>, Vincent Hanquez <tab@xxxxxxxxx>, Christian Limpach <Christian.Limpach@xxxxxxxxxxxx>
Delivery-date: Thu, 31 Mar 2005 07:45:25 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20050331070514.GH9204@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <A95E2296287EAD4EB592B5DEEFCE0E9D1E3930@xxxxxxxxxxxxxxxxxxxxxxxxxxx> <20050331070514.GH9204@xxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Thu, Mar 31 2005, Jens Axboe wrote:
> On Wed, Mar 30 2005, Ian Pratt wrote:
> > > I'll check the xen block driver to see if there's anything 
> > > else that sticks out.
> > >
> > > Jens Axboe
> > 
> > Jens, I'd really appreciate this.
> > 
> > The blkfront/blkback drivers have rather evolved over time, and I don't
> > think any of the core team fully understand the block-layer differences
> > between 2.4 and 2.6. 
> > 
> > There's also some junk left in there from when the backend was in Xen
> > itself back in the days of 1.2, though Vincent has prepared a patch to
> > clean this up and also make 'refreshing' of vbd's work (for size
> > changes), and also allow the blkfront driver to import whole disks
> > rather than paritions. We had this functionality on 2.4, but lost it in
> > the move to 2.6.
> > 
> > My bet is that it's the 2.6 backend that is where the true perofrmance
> > bug lies. Using a 2.6 domU blkfront talking to a 2.4 dom0 blkback seems
> > to give good performance under a wide variety of circumstances. Using a
> > 2.6 dom0 is far more pernickety. I agree with Andrew that I suspect it's
> > the work queue changes are biting us when we don't have many outstanding
> > requests.
> 
> You never schedule the queues you submit the io against for the 2.6
> kernel, you only have a tq_disk run for 2.4 kernels. This basically puts
> you at the mercy of the timeout unplugging, which is really suboptimal
> unless you can keep the io queue of the target busy at all times.
> 
> You need to either mark the last bio going to that device as BIO_SYNC,
> or do a blk_run_queue() on the target queue after having submitted all
> io in this batch for it.

Here is a temporary work-around, this should bring you close to 100%
performance at the cost of some extra unplugs. Uncompiled.

--- blkback.c~  2005-03-31 09:06:16.000000000 +0200
+++ blkback.c   2005-03-31 09:09:27.000000000 +0200
@@ -481,7 +481,6 @@
     for ( i = 0; i < nr_psegs; i++ )
     {
         struct bio *bio;
-        struct bio_vec *bv;
 
         bio = bio_alloc(GFP_ATOMIC, 1);
         if ( unlikely(bio == NULL) )
@@ -494,17 +493,12 @@
         bio->bi_private = pending_req;
         bio->bi_end_io  = end_block_io_op;
         bio->bi_sector  = phys_seg[i].sector_number;
-        bio->bi_rw      = operation;
 
-        bv = bio_iovec_idx(bio, 0);
-        bv->bv_page   = virt_to_page(MMAP_VADDR(pending_idx, i));
-        bv->bv_len    = phys_seg[i].nr_sects << 9;
-        bv->bv_offset = phys_seg[i].buffer & ~PAGE_MASK;
+       bio_add_page(bio, virt_to_page(MMAP_VADDR(pending_idx, i)),
+                       phys_seg[i].nr_sects << 9,
+                       phys_seg[i].buffer & ~PAGE_MASK);
 
-        bio->bi_size    = bv->bv_len;
-        bio->bi_vcnt++;
-
-        submit_bio(operation, bio);
+        submit_bio(operation | (1 << BIO_RW_SYNC), bio);
     }
 #endif
 

-- 
Jens Axboe


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel