|
xen-devel
Re: [Xen-devel] RFC: MCA/MCE concept
Hi,
On 06/06/07 10:28, Christoph Egger wrote:
On Monday 04 June 2007 18:16:56 Gavin Maltby wrote:
Hi,
On 05/30/07 10:10, Christoph Egger wrote:
On Wednesday 30 May 2007 10:49:40 Jan Beulich wrote:
"Christoph Egger" <Christoph.Egger@xxxxxxx> 30.05.07 09:45 >>>
On Wednesday 30 May 2007 09:19:12 Jan Beulich wrote:
case I) - Xen reveives a MCE from the CPU
1) Xen MCE handler figures out if error is an correctable error (CE)
or uncorrectable error (UE)
2a) error == CE:
Xen notifies Dom0 if Dom0 installed an MCA event handler
for statistical purpose
[rest cut]
For the hypervisor to dom0 communication that 2a) above refers to I think
we need to agree on two aspects: what form the notification event will
take, and what error telemetry data and additional information will
be provided by the hypervisor for dom0 to chew on for statistical
and diagnosis purposes.
Additionally, the hypervisor must be able to notify domU that has
a PV MCA driver.
Yes, forgot that; although I guess I view that most likely as a future phase.
For the first I've assumed so far that an event channel notification
of the MCA event will suffice; as long as the hypervisor only polls
for correctable MCA errors at a low-frequency rate (currently 15s interval)
there is no danger of spamming that single notification.
Why polling?
Polling for correctable errors, but #MC as usual for others. Setting
MCi_CTL bits for correctable errors does not produce a machine check,
so polling is the only approach unless one sets additional (and
undocumented, certainly for AMD chips) config bits. What I was getting
at here is that polling at largish intervals for correctables is
the correct approach - trapping for them or polling at a high-frequency
is bad because in cases where you have some form of solid correctable
error (say a single bad pin in a dimm socket affecting one or two ranks
of that dimm but never able to produce a UE) the trap handling and
diagnosis software consume the machine and things make little useful
forward progress.
On receipt of the notification the event handler will need to suck
some event data out of somewhere - uncertain which somewhere would
be best?
We should standardize both the format and the content of this event
data. The following is just to get the conversation started in this
area.
Content first. Obviously we need the raw MCA register content -
MCi_STATUS, MCi_ADDR, MCi_MISC. We also need know which
MCA detector bank made the observation, so we need to include
some indication of which chip (where I use "chip" to coincide
with "socket"), core on that chip, and MCA bank number
the telemetry came from. I think I am correct in saying that
hyperthreaded CPUs do not have any MCA banks per-thread, but we
may want to allow for that future possibility (I know, for instance,
that some SPARC cpus have error state for each hardware thread).
And we need the domain and the domain's vcpu to identify
who is impacted.
Yes, the domain ID. I'm not sure we need the vcpu id if we instead
present some physical identifiers such as chip, core number etc
(and have the namespaces well-defined). If we don't present those
the vcpu in the payload and some external method to resolve that to physical
components. Since errors correlate to physical components it would, I
think, be nicer to report detector info in some physical sense.
As regards a vcpu to physical translation, I didn't think there was any
fixed mapping (or certainly any mapping that a dom0 should interpret
and rely on). For example if we have two physical cores but choose
to present 32 vcpus to domain I don't believe there is anything to
say that 0-15 map always run on physical core 0?
We should also allow for additional model-specific error telemetry
that may be available and relevant - I know that will be necessary
for some upcoming x86 cpu models. We should probably avoid adding
"cooked" content to this error event payload - such cooking of the
raw data is much more easily performed in dom0 (the example I'm
thinking of here is physical address to memory location translation).
In terms of the form of the error event data, the simplest but also
the dumbest would be a binary structure passed from hypervisor
to dom0:
struct mca_error_data_ver1 {
uint8_t version; /* structure version */
uint64_t mc_status;
uint64_t mc_addr;
uint64_t mc_misc;
uint16_t mc_chip;
uint16_t mc_core;
uint16_t mc_bank;
uint16_t domid;
uint16_t vcpu_id;
...
};
That is easily passed around and can be extended by versioning.
A more self-describing and naturally extensible approach would be
to parcel the error data in some form of name-type-value list.
That's what we do in the corresponding kernel->userland error
code in Solaris; the downside is that the supporting libnvpair
library is not tiny and likely not the sort of footprint to
include in a hypervisor. Perhaps some cut-down form would do.
In the public xen.h header is a VIRQ_DOM_EXC defined, which seems
to be appropriate for an NMI event.
There are two functions to send VIRQs: send_guest_vcpu_virq() and
send_guest_global_virq().
However, VIRQ_DOM_EXC is not properly implemented:
All virtual interrupts are maskable. We definitely need
an event that guarantees to immediately interrupts the guest, no matter
if this is Dom0 or DomU and whatever they are doing.
And VIRQ_DOM_EXC is explicitely reserved for Dom0. Maybe
we should introduce a VIRQ_MCA as a special NMI event for both Dom0 and DomU?
Sounds like it may be necessary. I don't know this mechanism very well
so I'll go and do some reading (after a big long unrelated codereview).
Cheers
Gavin
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home