RE: [Xen-users] (3ware) xen 3.0 amd64 crash... seems to be tied intodisk i/o, > 4 gig ram

["Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>]

> I got a response back from 3ware, as I forwarded Ian's note to them...
> My card(s) is(are) an 8506-4LP

I'm not sure about the newer controllers, but the older generation
certainly couldn't do DMA above 4GB, and relied on bounce buffers or
GART support.

To try and recreate your setup we put an old 3ware controller in a
machine with 6GB and have been hitting it hard all day without problems.
The swiotlb bounce buffer code seems to be working fine. This is on a
Xeon, but since we use swiotlb on AMD64 too there shouldn't be any
difference.

Ian
 
> I don't know much about 'the use of the IOMMU'. I do know 
> that at least one of my kernels was bitching about the bios 
> settings for IOMMU, and it seems to be an active area of 
> Linux development. It seems to be some sort of aperature 
> possibly used as a temporary storage space for information 
> destined for addresses > 4gig? It also appears to tied into 
> the onboard (northbridge) GART stuff. I believe the engineers 
> MCE/memory comments apply to the controller memory, as I 
> would expect a clearer MCE message if we got an ECC fault on 
> main memory.
> 
> If anyone can recommed a good "big linux", or amd64 linux 
> list, I'd be happier posting this stuff there... although I 
> expect there are more "high powered techs" on this list, than 
> most others... of course there are a lot of "very junior" sys 
> admins here too :-)
> 
>    Date: Thu, 8 Dec 2005 14:32:58 -0800
>    From: David Graas <@amcc.com>
>    To: Tom Brown <tbrown@xxxxxxxxxxxxx>
>    Subject: RE: [Xen-users] xen 3.0 amd64 crash... seems to 
> be tied into disk
>        i/o, > 4 gig ram (fwd)
> 
>    Tom, I am not sure if this helps or not but your e-mail 
> did concern me
>    so I checked with some of our engineers on this. Their 
> reply is below...
> 
>    "We have supported > 4GB ram, even with the 3w-xxxx driver and
>    5000/6000/7000/8000 series cards on amd64 through the use 
> of the IOMMU
>    GART (Northbridge memory aperature) before the Opteron 
> even shipped to
>    customers, even though these cards only do 32-bit DMA operations.
> 
>    We have supported > 4GB ram with the 9500-S and 9550-SX 
> since day one.
>    We also support PAE (Physical Address Extensions) mode 
> with the 9500-S
>    and 9550-SX So you can DMA to > 4GB of ram on a 32-bit 
> pentium pro or
>    higher architecture.
> 
>    Also, he is getting MCE error messages (Machine Check 
> Exceptions) from
>    his Northbridge, which could mean bad memory."
> 
>    It might be worth a call to our support group 
> (800-840-6055) to see if
>    there is a fix on this situation.
> 
> 
>    David Graas
>    Corporate Sales Manager
>    3ware - an AMCC company
>    Direct 408-542-8670
>    Mobile 650-269-2972
>    Fax 408-542-8602
>    @amcc.com
>    www.amcc.com
> 
> 
> I've invalidated David's address, but it is first inital, 
> lastname if you want to write to him...
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users