WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU

To: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>, Dante Cinco <dantecinco@xxxxxxxxx>, Konrad Wilk <konrad.wilk@xxxxxxxxxx>
Subject: RE: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU kernel with PCI passthrough
From: "Lin, Ray" <Ray.Lin@xxxxxxx>
Date: Thu, 18 Nov 2010 14:39:40 -0700
Accept-language: en-US
Acceptlanguage: en-US
Cc: Jeremy Fitzhardinge <jeremy@xxxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, "mathieu.desnoyers@xxxxxxxxxx" <mathieu.desnoyers@xxxxxxxxxx>, Andrew Thomas <andrew.thomas@xxxxxxxxxx>, "keir.fraser@xxxxxxxxxxxxx" <keir.fraser@xxxxxxxxxxxxx>, Chris Mason <chris.mason@xxxxxxxxxx>
Delivery-date: Thu, 18 Nov 2010 13:40:44 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <c044d15d-9b58-4e73-83eb-b3260a7e0fb5@default>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcuHZtQ4zStQxED7TLSN89VSGA5RAwAAYdZA
Thread-topic: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU kernel with PCI passthrough
 

-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Dan Magenheimer
Sent: Thursday, November 18, 2010 1:21 PM
To: Dante Cinco; Konrad Wilk
Cc: Jeremy Fitzhardinge; Xen-devel; mathieu.desnoyers@xxxxxxxxxx; Andrew 
Thomas; keir.fraser@xxxxxxxxxxxxx; Chris Mason
Subject: RE: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops 
domU kernel with PCI passthrough

In case it is related:
http://lists.xensource.com/archives/html/xen-devel/2010-07/msg01247.html 

Although I never went further on this investigation, it appeared to me that 
pvclock_clocksource_read was getting called at least an order-of-magnitude more 
frequently than expected in some circumstances for some kernels.  And IIRC it 
was scaled by the number of vcpus.

We did suspect it, since our old setting was HZ=1000 and we assigned more than 
10 VCPUs to domU. But we don't see the performance difference with HZ=100.

> -----Original Message-----
> From: Dante Cinco [mailto:dantecinco@xxxxxxxxx]
> Sent: Thursday, November 18, 2010 12:36 PM
> To: Konrad Rzeszutek Wilk
> Cc: Jeremy Fitzhardinge; Xen-devel; mathieu.desnoyers@xxxxxxxxxx; 
> Andrew Thomas; keir.fraser@xxxxxxxxxxxxx; Chris Mason
> Subject: Re: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 
> pvops domU kernel with PCI passthrough
> 
> I mentioned earlier in an previous post to this thread that I'm able 
> to apply Dulloor's xenoprofile patch to the dom0 kernel but not the 
> domU kernel. So I can't do active-domain profiling but I'm able to do 
> passive-domain profiling but I don't know how reliable the results are 
> since it shows pvclock_clocksource_read as the top consumer of CPU 
> cycles at 28%.
> 
> CPU: Intel Architectural Perfmon, speed 2665.98 MHz (estimated) 
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a 
> unit mask of 0x00 (No unit mask) count 100000
> samples  %        image name               app name
> symbol name
> 918089   27.9310
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           pvclock_clocksource_read
> 217811    6.6265  domain1-modules          domain1-modules
> /domain1-modules
> 188327    5.7295  vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> mutex_spin_on_owner
> 186684    5.6795
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __xen_spin_lock
> 149514    4.5487
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __write_lock_failed
> 123278    3.7505
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __kernel_text_address
> 122906    3.7392
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           xen_spin_unlock
> 90903     2.7655
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __spin_time_accum
> 85880     2.6127
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __module_address
> 75223     2.2885
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           print_context_stack
> 66778     2.0316
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           __module_text_address
> 57389     1.7459
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           is_module_text_address
> 47282     1.4385  xen-syms-4.1-unstable    domain1-xen
> syscall_enter
> 47219     1.4365
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           prio_tree_insert
> 46495     1.4145  vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> pvclock_clocksource_read
> 44501     1.3539
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           prio_tree_left
> 32482     0.9882
> vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-5.11.dcinco-debug
> domain1-kernel           native_read_tsc
> 
> I ran oprofile (0.9.5 with xenoprofile patch) for 20 seconds while the 
> I/Os were running. Here's the command I used:
> 
> opcontrol --start --xen=/boot/xen-syms-4.1-unstable 
> --vmlinux=/boot/vmlinux-2.6.32.25-pvops-stable-dom0-5.7.dcinco-debug
> --passive-domains=1
> --passive-images=/boot/vmlinux-2.6.36-rc7-pvops-kpcif-08-2-domu-
> 5.11.dcinco-debug
> 
> I had to remove dom0_max_vcpus=1 (but kept dom0_vcpus_pin=true) in the 
> Xen command line. Otherwise, oprofile only gives the samples from 
> CPU0.
> 
> I'm going to try perf next.
> 
> - Dante
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>