[Xen-devel] ctx suddenly rises in HVM Xen Guest

[Songtao Liu <songtao.liu@xxxxxxxx>]

Hello,

I have tested HVM XEN GUEST performance with LMbench and found a problem with 
the context switch time in guest. Following results are with CREDIT scheduler, 
but SEDF scheduler has similar results.

The context switch time will rise suddenly when the PROCESS NUMBER (in context 
switch microbenchmark) arrives a 'threshold', just like a step-function. 
Normally the 'threshold' of PROCESS NUMBER varies in the range of 35-44 (in UP 
HVM XEN guest) and 52-62 (in SMP HVM XEN guest), which is relevant to UP/SMP 
and size of processes defined by LMbench. Before the 'threshold' context switch 
time keeps a relative stable value, after threshold it also keeps a relative 
stable value. The native Linux has no such problem.

***************************************
Comparison of Test Results
'size' denotes size of processes
ProcNum denotes number of processes
***************************************
(1)
'size=0k'    
HVM XEN UP-GUEST 
3.58(ProcNum=42)
65.36(ProcNum=43) /* 'threshold' 43 */

HVM XEN SMP-Guest
29.71(ProcNum=60)           
116.86(ProcNum=61)  /* 'threshold' 61 */

(2)
'size=32k'
HVM XEN UP-GUEST 
3.01(ProcNum=37)
101.81(ProcNum=38)  /* 'threshold' 38 */

HVM XEN SMP-Guest
31.53(ProcNum=53)           
158.46(ProcNum=54)  /* 'threshold' 54 */

I profiling the TLB misses  with following configuration to see whether it is 
relevant to TLB misses or not. 
1. Count TLB misses in Dom0 when DomU running modified context switch 
microbenchmark 40 times (with only ProcNum=32, which can gurantee it has not 
arrived 'threshold')
2. Count TLB misses in Dom0 when DomU running modified context switch 
microbenchmark 40 times (with only ProcNum=64, which can gurantee it has not 
arrived 'threshold')
3. Count TLB misses in Dom0 when no ctx microbenchmark running in DomU
So the only difference between the three configurations are ProcNum and 
Running/NotRunning ctx microbenchmark in DomU.
But results show that there is no obvious difference between ProcNum=32 and 
ProcNum=64.

****************************************
TLB misses Comparison
****************************************
DTLB-misses (/second)
4387800 (ProcNum=32)  /* before 'threshold' */
4326450 (ProcNum=64)  /* after 'threshold' */
2935300 (no ctx running)

ITLB-misses (/second)
230167 (ProcNum=32)  /* before 'threshold' */
235384 (ProcNum=64)  /* after 'threshold' */
85988 (no ctx running)

****************************************
version
****************************************
Xen:xen-3.0.3-8
Dom0: 2.6.19-1.2911.6.5.fc6xen 
DomU: 2.6.22.9-61.fc6 

Tools:
lmbench-3.0-a8
oprofile 0.9.2

****************************************
CPUINFO
****************************************
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 CPU         T7400  @ 2.16GHz
stepping        : 6
cpu MHz         : 2168.220
cache size      : 4096 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush 
dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc pni monitor ds_cpl 
vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips        : 5422.48

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 CPU         T7400  @ 2.16GHz
stepping        : 6
cpu MHz         : 2168.220
cache size      : 4096 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush 
dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc up pni monitor 
ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
bogomips        : 5422.48

Can anyone explain this problem for me? Any comments/advices will be 
appreciated.
Songtao

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel