staging: vchiq_arm: use one DMA device to map and unmap bulk pagelists

[rachmataditiya]

Fixes #7493 — full analysis, reproducer, and oops there.

Since the pagelist code moved into vchiq_core.c, create_pagelist() maps the bulk scatterlist with instance->state->dev while free_pagelist()/cleanup_pagelistinfo() unmap with g_dma_dev (on BCM2711 the brcm,bcm2711-dma device resolved for use_36bit_addrs, with different dma-ranges), and the coherent pagelist buffer is allocated on instance->state->dev but freed on g_dma_dev. This breaks every userspace /dev/vchiq bulk transfer on Pi 4 arm64: the VPU is handed 36-bit pagelist entries computed under the wrong device's dma-ranges (bulk writes arrive as garbage, sustained load wedges the VPU), and the first bulk read oopses in free_pagelist() on the DMA_FROM_DEVICE unmap cache invalidate.

This restores the pairing rpi-6.12.y shipped (map/unmap on g_dma_dev, coherent alloc/free on instance->state->dev), which is the last known-working combination.

Verified on a Pi 4B Rev 1.5 (2GB): on stock 6.18.34+rpt a vc.ril.video_splitter payload loopback from a 64-bit userland oopses as in the issue and vc.ril.video_decode never decodes; on rpi-6.18.y with this patch the loopback round-trips byte-identical and H.264 decode from a 64-bit process runs at full rate (26k frames / 600 s soak, clean dmesg).

Context: 64-bit userland MMAL work discussed in the "Porting Linux VCHIQ Driver To 64 Bit" forum thread.

[popcornmix]

Looks plausible. @pelwell I assume this is still needed after a number of iterations of upstreaming?

[pelwell]

I'm confused - this patch appears to take a mismatch and reverse it, i.e. it is still unbalanced, but the other way round.

[pelwell]

g_dma_dev should be used in both instances.

[6by9]

There is vchiq_test in https://github.com/raspberrypi/userland/blob/master/interface/vchiq_arm/vchiq_test.c which should exercise bulk transfers fairly comprehensively.

I'll try and get it running both with and without this patch on Pi4 (Pi0-3 should be identical as g_dma_device is the same device).

[6by9]

vchiq_test appears to fail generally on Pi4.
I guess upstream still doesn't have the 40bit DMA controller driver, so that probably accounts for not having had the issue reported there.

Testing on a Pi3 with a 64bit OS seems to work fine, so being associated with the dma device would follow.
The V4L2 drivers avoid doing bulk transfers, so that is likely why they don't have issues.

By the same logic it feels like the dma_pool added in 5252444 should probably also be allocated against g_dma_dev rather than always dev, but this is getting a bit beyond my knowledge of vchiq.

[6by9]

Mainline 7.2.0-rc1 also fails on a Pi4.

[pelwell]

#7497 fixes vchiq-test on Pi 4/6.18 for me.

[pelwell]

Mainline 7.2.0-rc1 also fails on a Pi4.

Does it work on other mainline kernel versions?

[6by9]

Comparing against 6.12, I think I agree with this patch.

dma_alloc_coherent and dma_free_coherent use instance->state->dev
dma_map_sg and dma_unmap_sg use g_dma_dev
One of each was wrong on 6.18, hence the slightly odd looking patch.

Pass as to why vchiq_test was failing when I'd applied this before. It's working fine now.

Seeing your new PR, whether dma_alloc_coherent and dma_free_coherent should use instance->state->dev is a different question.

[6by9]

#7497 fixes vchiq-test on Pi 4/6.18 for me.

vchiq_test -f fails immediately for me with that.

Functional test - iters:100
======== iteration 1 ========
vchiq_test: 1540: expected callback reason VCHIQ_BULK_TRANSMIT_DONE, got 5
vchiq_test: 856: func_error != 0

[pelwell]

Seeing your new PR, whether dma_alloc_coherent and dma_free_coherent should use instance->state->dev is a different question.

It's possible the two end up being equivalent if the DMA allocations end up coming from the CMA heap, which we confine to the first 1GB.

The three main vchiq_test modes - ping, bulk and functional - all pass with this patch on a 4GB Pi 4, so I'm happy to merge it and drop mine.

[6by9]

I'm on an 8GB Pi4. Swapping dmam_pool_create to use g_dma_dev (as per your patch and my suggestion) then also fails vchiq_test -f with the same error:

Functional test - iters:100
======== iteration 1 ========
vchiq_test: 1540: expected callback reason VCHIQ_BULK_TRANSMIT_DONE, got 5
vchiq_test: 856: func_error != 0

[pelwell]

I think the changes to the device used for allocation may remove the CMA constraint, even though the VPU needs it.

vchiq_test -f fails immediately for me with that.

I've just found the same thing.

[rachmataditiya]

Thanks both — much appreciated.

I've pushed a small amend for the checkpatch bot: re-wrapped one long line in the commit message and fixed a continuation indent in cleanup_pagelistinfo(). Whitespace only — no functional change from what was tested.

On the allocation side: this patch deliberately keeps dma_alloc_coherent/dma_pool_create on instance->state->dev and only re-pairs map/unmap onto g_dma_dev, restoring the 6.12 pairing — for what I think is the reason @pelwell describes: the VPU-visible allocations stay under the CMA / first-GB constraint. The 8GB -f failure when the pool moves to g_dma_dev looks consistent with that. Whether the alloc device should eventually change too seems like a separate question — happy to help dig into it if useful.

[6by9]

Mainline 7.2.0-rc1 also fails on a Pi4.

Does it work on other mainline kernel versions?

Not tried.
I happened to have the 7.2.0 kernel from my recent upstreaming efforts. Finding an earlier one will be a bit more involved.