[Xen-users] 47s clock skew in domUs every so often, Debian squeeze packaged hypervisor 4.0.1-2

[Andy Smith <andy@xxxxxxxxxxxxxx>]

Hi,

Ever since upgrading a particular machine from Debian lenny with Xen
3.3.x to Debian squeeze with xen-hypervisor-4.0-amd64 4.0.1-2 back
in June, it's been inflicting an intermittent 47s clock skew on
multiple domUs.

All the domUs run ntpd, but only one of them has an ntpd which seems
to detect it. Example:

Sep 20 16:11:39 test0 ntpd[1802]: time reset -46.869813 s
Sep  6 12:58:41 test0 ntpd[1742]: time reset -46.869301 s
Jul 27 11:19:35 test0 ntpd[1742]: time reset -46.869786 s
Jun 29 19:05:10 test0 ntpd[1708]: time reset -46.868480 s

Another one runs heartbeat and it similarly gets upset:

Sep 20 15:54:56 foo2 heartbeat: [1407]: WARN: Late heartbeat: Node foo2: 
interval 47000 ms
Sep  6 12:43:02 foo2 heartbeat: [1407]: WARN: Late heartbeat: Node foo2: 
interval 47590 ms
(logs go back no further; I expect same happened on matching dates)

Its ntpd doesn't seem to complain, though.

There are some customer domUs on this host running various versions
of Debian and CentOS (both 2.6.18 backported Xen kernels and 2.6.36+
pvops), on which dovecot sees the time go backwards and kills
itself, but I don't have direct access to these domUs so can't
report much other than that.

There are other domUs which don't complain but this could be due to
them having no software which cares; I have to investigate more to
see if something is actually happening on them.

Has anyone ever seen this before? I'm thinking this is a bug in Xen
because it never happened in Xen 3.x, has only happened since
upgrade, and the way it happens on the same day in multiple domUs
(of different Linux distributions) suggest dom0 or hypervisor to me.
I also have other hosts of Xen 4.x where this doesn't happen.

dom0 is a Supermicro X7DCL-i, Xeon 5410 running amd64 Debian squeeze
and the distribution Xen packages.

Some info on the various hosts involved:

       Distribution    Kernel
------------------------------------------
dom0   Debian squeeze  2.6.32-5-xen-amd64
foo2   Debian squeeze  2.6.32-5-686-bigmem
test0  Debian lenny    2.6.26-2-686-bigmem
bar0   Debian squeeze  2.6.32-5-686-bigmem

       Current      Available
       clocksource  clocksources
-----------------------------------
dom0   xen          xen
foo2   tsc          xen tsc
test0  xen          xen tsc jiffies
bar0   xen          xen tsc

Any ideas?

Cheers,
Andy

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users