This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Xen 4.0.0x allows for data corruption in Dom0

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Xen 4.0.0x allows for data corruption in Dom0
From: Joanna Rutkowska <joanna@xxxxxxxxxxxxxxxxxxxxxx>
Date: Tue, 09 Mar 2010 00:48:20 +0100
Delivery-date: Mon, 08 Mar 2010 15:48:47 -0800
Dkim-signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=messagingengine.com; h=message-id:date:from:mime-version:to:subject:references:in-reply-to:content-type; s=smtpout; bh=ee0XLxs47ufAYZGaaflBD/fCq/U=; b=C8fu03/bESVdECfV3vDFzT1U+GGyemQQf5bCWbxPWQVhiZAvu1H1D+QOJtwLOi2WT0BQ1L3uMB9gSyid2UL7C+SsDBtVqbGGBanWJoKX8ock7WPb9Obdf+yoHKhj4kV8/3vR1qHPBUvJaJ6qkq6dWqEp1zL5mgzAmMm6tQ2SwYw=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4B958B14.5030805@xxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4B922A89.2060105@xxxxxxxxxxxxxxxxxxxxxx> <4B957914.4050408@xxxxxxxx> <4B957B93.4060401@xxxxxxxxxxxxxxxxxxxxxx> <4B958475.3050407@xxxxxxxx> <4B9586E0.2060005@xxxxxxxxxxxxxxxxxxxxxx> <4B958B14.5030805@xxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20100301 Fedora/3.0.3-1.fc12 Lightning/1.0b2pre Thunderbird/3.0.3
On 03/09/2010 12:41 AM, Jeremy Fitzhardinge wrote:
> On 03/08/2010 03:23 PM, Joanna Rutkowska wrote:
>> But the corruptions always happen in 32-bytes chunks, which might
>> suggest it's not a page-related problem (e.g. wrongly re-used page), as
>> in that case we would be observing (at least sometimes) much bigger
>> chunks of corrupted data, I think.
> Given that the domU doesn't have any devices or much going on, it could
> easily be corrupting memory in only small amounts.
But see, before I tried this with such a small dummy do-nothing DomU
(which I did for the purpose of reporting to xen-devel), I experienced
very similar corruption when running regular VMs, i.e. with normal linux
and all the usual apps inside them. Same pattern of corruption.

>> The reason why I still believe it's a hypervisor related thing, it that
>> I'm currently using the very *same* Dom0 kernel (very recent
>> xen/stable-2.6.31) with Xen 3.4.2 and the system is damn stable. And I
>> really mean extensive use with 5-7 VMs running all the time doing
>> various things from Web browsing to kernel building.
> OK, it's always good to get some positive feedback.

At least one full-time user of the pvops kernel ;)

>> If I was to make an educated guess I would say it's something related to
>> some interrupt handling, i.e. Xen mishandling it, e.g. the handler is
>> writing out-of-buffer somewhere and it just happens to land in the Dom0
>> fs buffer used by e.g. dd operation.
> It would be interesting to see what happens if you write the file with
> the test domain paused (xm pause ...).  If the corruption continues,
> then it is almost certainly Xen.


> If it stops, then it either means the
> corruption was caused by pages inappropriately shared between dom0 and
> domU, or something like vcpu context switch is corrupting memory (which
> would be very sad).

Unfortunately, I cannot do any more tests. We have downgraded all our
test machines to Xen 3.4.2 and are using them for other things now. Sorry.


Attachment: signature.asc
Description: OpenPGP digital signature

Xen-devel mailing list
<Prev in Thread] Current Thread [Next in Thread>