WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] [PATCH] KEXEC: disconnect all PCI devices from the PCI b

To: Jan Beulich <JBeulich@xxxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] KEXEC: disconnect all PCI devices from the PCI bus on crash
From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
Date: Thu, 7 Jul 2011 12:40:29 +0100
Cc: Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>, Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Delivery-date: Thu, 07 Jul 2011 04:36:36 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4E15B2BF020000780004C976@xxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <7ea606c5ce8c8fcfc577.1309955952@xxxxxxxxxxxxxxxxxxxxxxxxxxx> <20110706184233.GA30520@xxxxxxxxxxxx> <1310028827.634.199.camel@xxxxxxxxxxxxxxxxxxxxxx> <4E159421020000780004C8EA@xxxxxxxxxxxxxxxxxxxx> <1310029954.634.204.camel@xxxxxxxxxxxxxxxxxxxxxx> <4E159B94020000780004C91E@xxxxxxxxxxxxxxxxxxxx> <1310032414.634.208.camel@xxxxxxxxxxxxxxxxxxxxxx> <4E15B2BF020000780004C976@xxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Alpine 2.00 (DEB 1167 2008-08-23)
On Thu, 7 Jul 2011, Jan Beulich wrote:
> >>> On 07.07.11 at 11:53, Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx> wrote:
> > On Thu, 2011-07-07 at 10:42 +0100, Jan Beulich wrote:
> >> >>> On 07.07.11 at 11:12, Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx> wrote:
> >> > On Thu, 2011-07-07 at 10:10 +0100, Jan Beulich wrote:
> >> >> >>> On 07.07.11 at 10:53, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> >> >> > On Wed, 2011-07-06 at 19:42 +0100, Konrad Rzeszutek Wilk wrote:
> >> >> >> On Wed, Jul 06, 2011 at 01:39:12PM +0100, Andrew Cooper wrote:
> >> >> >> > +/* Disconnect a PCI device from the PCI bus.  From the PCI spec:
> >> >> >> > + *     "When a 0 is written to [the COMMAND] register, the device 
> >> >> >> > is
> >> >> >> > + *     logically disconnected from the PCI bus for all accesses 
> >> >> >> > except
> >> >> >> > + *     configuration accesses. All devices are required to support
> >> >> >> > + *     this base level of functionality."
> >> >> >> > + */
> >> >> >> > +void disconnect_pci_device(struct pci_dev *pdev)
> >> >> >> > +{
> >> >> >> > +    pci_conf_write16(pdev->bus, PCI_SLOT(pdev->devfn),
> >> >> >> > +                     PCI_FUNC(pdev->devfn), PCI_COMMAND, 0);
> >> >> >> 
> >> >> >> So if you have a PCI serial card (or Intel AMT) and you are using 
> >> >> >> that for
> >> >> >> serial output on the hypervisor line, this will turn it off. There 
> >> >> >> should
> >> >> >> be some whitelist capability to not do it for PCI serial devices 
> >> >> >> that are
> >> >> >> owned (used) by the hypervisor.
> >> >> > 
> >> >> > That would be useful for debugging the kexec process itself but in the
> >> >> > general case there won't be any further output from the hypervisor and
> >> >> > if the kexec'd kernel wants to use the device it is going to have to 
> >> >> > set
> >> >> > it up again anyways.
> >> >> 
> >> >> No, not generally. Just look at Linux' early-printk code: The device
> >> >> is assumed to be enabled (by the BIOS), as the PCI subsystem can't
> >> >> possibly be initialized at this point already.
> >> > 
> >> > That's arguably a debugging facility as well though.
> >> > 
> >> >> This also means that white-listing just devices Xen uses may not be
> >> >> enough: If Xen doesn't use a serial console (or the secondary kernel
> >> >> wants to use some other device Xen doesn't care about - VGA or
> >> >> other kind of console devices come to mind), it must not find it fully
> >> >> disconnected from the bus. Consequently I would think that while
> >> >> interrupt and DMA activity should be forced off, decoding I/O and
> >> >> memory addresses by the devices shouldn't be.
> >> > 
> >> > The problem is that this can't be done without device specific
> >> > knowledge, which the hypervisor generally doesn't have and we can't get
> >> > the device's owning domain to do anything because we are crashing.
> >> 
> >> Why would there be any device specific knowledge needed? It's
> >> all done through the command word, just that writing zero isn't
> >> really appropriate.
> > 
> > So presumably if you disable bus mastering you've effectively disabled
> > DMA but how do you disable interrupts via the command word?
> 
> Interrupts can be disabled equally well in the IO-APIC and by
> disabling MSI for all devices.

Considering that it would be nice to have PCI device reset capabilities
in the hypervisor anyway, in case a device is malfunctioning, and
considering that we all know that PCI devices don't always follow PCI
specs, wouldn't it be safer to just reset all the devices in this case?
At least the reset code is well tested and known to work (or known to
work with some particular devices, and for this reasons there are
quirks). Also resetting devices would better cover the case when a
device is misbehaving.
Of course we still need a whitelist for serial consoles and maybe other
devices.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel