dma_handle returned by dma_alloc_coherent() not bus address

[kdanderso]

The dma_handle returned by dma_alloc_coherent() in the raspberrypi-kernel_1.20170427-1 release does not appear to be a valid bus address. In previous releases, the returned dma_handle had bits 30 and 31 set to 1. However, in the 20170427 release, bits 30 and 31 are set to 0. This is resulting in data transfer errors in our device driver when transferring from memory to I2S.

We would appreciate assistance in determining how to address this issue. It appears that the device tree dma-ranges value is intended to ensure appropriate address values are returned, but it does not seem to be effective for the dma_alloc_coherent() call.

[P33M]

Can you narrow it down to the specific rpi-update firmware version as this has greater granularity?

https://github.com/Hexxeh/rpi-firmware/commits/master

By doing sudo rpi-update <hash> you will get the shipped kernel from the commit ID in that repository. Start at the Apr 26th version and work backwards until you get a working bus_address.

[popcornmix]

I've had a look at latest 4.9 kernel and printed out the values from:
https://github.com/raspberrypi/linux/blob/rpi-4.9.y/drivers/gpio/gpio-bcm-virt.c#L104

[    0.144123] [brcmvirt_gpio_probe]: alloced ts_base:ae500000 bus_addr:ee500000

The bus_addr has top two bits as expected.

Are you sure you are looking at the bus address, rather than the arm address?
Are you on Pi2/Pi3? (Pi1 will have 0x4 as the top nibble).

[kdanderso]

We are looking at the bus address. Here is the code used to allocate:

RdXferBuffer = dma_zalloc_coherent (&pdev->dev, PAGE_SIZE, &Rd_dma_addr, GFP_KERNEL);

I printed out the values in RdXferBuffer and Rd_dma_addr:

[ 343.326665] RdXferBuffer = 0xBA909000, Rd_dma_addr = 0x3A909000

The bus_addr has top two bits cleared.

This is on a Pi3.

[popcornmix]

Can you try applying this patch:

diff --git a/drivers/gpio/gpio-bcm-virt.c b/drivers/gpio/gpio-bcm-virt.c
index c372554..aaa0f23 100644
--- a/drivers/gpio/gpio-bcm-virt.c
+++ b/drivers/gpio/gpio-bcm-virt.c
@@ -108,6 +108,7 @@ static int brcmvirt_gpio_probe(struct platform_device *pdev)
                err = -ENOMEM;
                goto out;
        }
+pr_err("[%s]: alloced ts_base:%p bus_addr:%p\n", __func__, (void *)ucb->ts_base, (void *)ucb->bus_addr);
 
        gpiovirtbuf = (u32)ucb->bus_addr;
        err = rpi_firmware_property(fw, RPI_FIRMWARE_FRAMEBUFFER_SET_GPIOVIRTBUF,

and confirm if that works or doesn't.
I know if you pass null in as the device parameter to dma_zalloc_coherent you won't get a valid bus address back, so that might be something to check carefully.

[kdanderso]

I think I did not communicate clearly earlier. The addresses printed out for my dma_zalloc_coherent() call were from our device driver, which is targeted toward I2S, not GPIO. I don't think the patch you're suggesting will have any effect on the issue we're seeing.

Your comment about a null value may be relevant, although we don't understand what would have changed between versions to cause this type of issue. We are looking into it.

[popcornmix]

I don't think the patch you're suggesting will have any effect on the issue we're seeing.

If it gives the same correct answer as I see, then it shows the kernel/device tree you are using are set up correctly, and the problem is almost certainly in your driver (perhaps an invalid device being passed in).

If it gives the the same wrong answer you see from your driver, it will prove your driver is not to blame, and the problem lies in the kernel/device tree.

So, it seems a pretty useful test to run.

[kdanderso]

I understand.

It's beginning to look like there is a problem with our overlay now that dma support has been incorporated into the device tree. Once we've had a chance to investigate, I'll post an update, but do not need any additional input for right now.

Thanks for your help so far.

[kdanderso]

We have determined that of_dma_configure() for our device is called before the dma-ranges value has been set. This is probably happening because the target-path value we used in our overlay was "/". Changing the target-path value to "/soc" appears to correct the problem so that the dma_handle is a bus address and dma now works as expected.

Does it make sense that drivers that require dma have to reference the /soc target-path, or is there a more appropriate way of ensuring that dma_alloc_coherent returns bus addresses?

[popcornmix]

@pelwell is this something you know?

[pelwell]

We have determined that of_dma_configure() for our device is called before the dma-ranges value has been set.

Although I'm sure you are seeing a real problem, this statement doesn't make sense. The DT exists and is fully populated before the first driver is loaded.

of_dma_configure calls of_dma_get_range, which searches up the tree looking for a dma-ranges property. The property is set for the /soc node, which is correct because it is supposed to be for a bus. Anything outside /soc must be addressable by some other means.

Can you say a bit more about your module? It might help to choose the right solution.

[kdanderso]

Your description agrees with what we are seeing. I.e. if our overlay sets target-path to "/", the call to of_dma_configure does not find a dma-ranges property, since that property is defined for the /soc node, not for the root node.

Our module provides multi-channel bi-directional audio using the i2s interface. We also use the uart and spi. Data transfers to and from i2s are using dma.

Would it be reasonable to say that the dma controller is under the /soc node and therefore anything using dma needs to target that node?

[pelwell]

Would it be reasonable to say that the dma controller is under the /soc node and therefore anything using dma needs to target that node?

Yes, it seems that is a requirement. If your device were addressable by any other means - I2C, SPI etc. - then /soc would naturally be a parent, but I'm guessing you have a simple device that is completely driven by the I2S signals (and perhaps a few GPIOs). I'm slightly bothered that there will be a node under /soc without a bus address, but it won't be the first.

Why are you driving the DMA yourself and not letting the standard I2S driver do it for you via ALSA?

[kdanderso]

Our system use a 32-channel TDM PCM bus and we need to mux/demux the data so each channel has its own contiguous memory space. Furthermore, we are using a circular buffering scheme for the data. We were not able to figure out how to make the standard I2S driver work for this type of operation.