On Mon, Feb 01, 2010 at 09:54:02PM +0100, Ronny.Hegewald@xxxxxxxxx wrote:
> >Well, since you are volunteering. The problem sounds like the sound card
> >allocates a buffer from the region above 4GB and tries to DMA to it.
> >Keep in mind that on most machines, when you have 4GB, 768 MB of it are
> >offset
> >past the 4GB mark. You have these two mega regions: 0-3.3GB, 4GB-4.7GB,
> >the 3.3GB to 4GB is called the PCI hole.
>
> I added some log-messages in the sound-driver and indeed, it allocates memory
> from above 4GB.
>
> On bare metal and with dom0 based on the forward-ported xen-patches from
> gentoo the dma-memory is allocated in the 4GB range.
>
> When i checked the dma-memory allocation in the driver i noticed that
> depending if the sound-card reports that it is 64bit-compatible the driver
> sets the dma_mask to 64, else to 32 (by calling pci_set_consistent_dma_mask).
>
> So i changed the driver that it always sets the dma-mask to 32 - and the
> sound worked without the need to start a domU.
Good detective job. BTW, what is the name of the driver in the source
code?
>
> Which left the question why it didnt worked out of the box on pvops-dom0.
>
> So i followed the code for the dma-allocation from the sound-driver and found
> the problem in the xen-swiotlb code in arch/x86/xen/pci-swiotlb.
>
> Bare metal and the forward-ported dom0 both call dma_alloc_coherent_mask if
> they try to allocate coherent dma-memory - pvops dom0 doesnt. This call sets
> the DMA-mask to 24 bit when __GFP_DMA is set, else to 32-bit.
What is your sound-driver detecting the card as? As 64-bit or 32-bit or
none of those?
On pvops we can get away from calling dma_alloc_coherent b/c we have
this piece of logic to determine where the driver can DMA from:
604 if (hwdev != NULL && hwdev->coherent_dma_mask)
605 mask = hwdev->coherent_dma_mask;
606 else
607 mask = DMA_BIT_MASK(32);
You mentioned that the sound driver calls the pci_set_consistent_dma_mask. That
function is the one that sets the coherent_dma_mask:
2045 int
2046 pci_set_consistent_dma_mask(struct pci_dev *dev, u64 mask)
2047 {
2048 if (!pci_dma_supported(dev, mask))
2049 return -EIO;
2050
2051 dev->dev.coherent_dma_mask = mask;
So the 'xen_swiotlb_alloc_coherent' checks if you have the coherent DMA
mask and if not, assumes you have a driver that can only access up to
4GB. While the bare-metal assumes that if the driver doesn't have that
mask , it checks the gfp_t flag and if it has __GFP_DMA make the mask
24-bit, otherwise 32-bit.
The only difference here is that under pvops we behave badly with
devices that have GFP_DMA set and don't have the coherent_dma_mask
(which it does not seem to be the case?).
So is your sound-driver not detecting the card properly and not setting
the coherent_dma_mask and/or dma_mask?
Can you print out both of those entries when the sound driver
calls the 'xen_swiotlb_alloc_coherent' (without setting the flags to 32
forcefully?)
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home