dmaengine: dw-edma: Mark emulated IRQ as level-triggered

The interrupt-emulation virtual IRQ uses handle_level_irq(), but the IRQ
descriptor has not been marked with IRQ_LEVEL.

The interrupt emulat

dmaengine: dw-edma: Mark emulated IRQ as level-triggered

The interrupt-emulation virtual IRQ uses handle_level_irq(), but the IRQ
descriptor has not been marked with IRQ_LEVEL.

The interrupt emulation is somewhat unusual: the eDMA interrupt handlers
dispatch the virtual IRQ for every edma_int[] interrupt because software
cannot reliably tell an interrupt-emulation event from one caused solely
by DONE/ABORT status. If an interrupt arrives before the doorbell
handler is registered for the virtual IRQ, the IRQ core marks it
pending. When the IRQ is later started, check_irq_resend() treats it as
non-level and replays the pending interrupt, causing the newly
registered handler to run for a stale event.

Mark the virtual IRQ with IRQ_LEVEL so the stale pending state is
cleared without being replayed. This was observed in pci_endpoint_test
as two doorbell handler calls when the DMA-variant test ran before
DOORBELL_TEST.

Fixes: d9d5e1bdd180 ("dmaengine: dw-edma: Add virtual IRQ for interrupt-emulation doorbells")
Cc: stable@vger.kernel.org
Reported-by: Niklas Cassel <cassel@kernel.org>
Closes: https://lore.kernel.org/r/ampndLtU32ODmncX@ryzen
Tested-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Link: https://patch.msgid.link/20260730160701.3550710-1-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Program endpoint function numbers

The eDMA/HDMA transfers the driver issues carry a requester function
number in their TLPs, but nothing ever programs it: eDMA v0 leaves the
FUNC

dmaengine: dw-edma: Program endpoint function numbers

The eDMA/HDMA transfers the driver issues carry a requester function
number in their TLPs, but nothing ever programs it: eDMA v0 leaves the
FUNC_NUM field of the channel control word zero and HDMA leaves the
per-channel func_num register at its reset value, so every transfer is
attributed to function 0. That is invisible in single-function setups,
but once the DMA block serves a non-zero endpoint function, its
requests must carry that function's number for the host to attribute
and translate them correctly.

Record the function number in the chip data (PCI_FUNC() of the probing
device for dw-edma-pcie) and program it per channel.

Endpoint-local chip instances keep func_no at 0, so transfers issued by
the endpoint-side driver remain PF0-attributed. Delegated channels are
programmed by the host-side dw-edma-pcie instance when it takes over the
channel, using that instance's PCI_FUNC().

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-15-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Add chip flags to match data

Allow PCI ID match data to pass dw_edma_chip flags into dw_edma_probe().
This keeps per-device policy in the match data instead of open-coding i

dmaengine: dw-edma-pcie: Add chip flags to match data

Allow PCI ID match data to pass dw_edma_chip flags into dw_edma_probe().
This keeps per-device policy in the match data instead of open-coding it
in probe().

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-14-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Handle optional data blocks

Skip data block BAR mapping and debug output when a channel has no data
block size. This lets future providers describe channels that only need
d

dmaengine: dw-edma-pcie: Handle optional data blocks

Skip data block BAR mapping and debug output when a channel has no data
block size. This lets future providers describe channels that only need
descriptor memory exposed.

No functional change intended for existing Synopsys EDDA and
AMD (Xilinx) MDB/CPM6 devices. Their static channel descriptions still
provide data block sizes where data block windows are used. A zero-sized
data block now means "not present" for future metadata providers.

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-13-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Factor out descriptor block address lookup

Add an optional physical address override to struct dw_edma_block and
use a helper to compute descriptor block addresses.

No func

dmaengine: dw-edma-pcie: Factor out descriptor block address lookup

Add an optional physical address override to struct dw_edma_block and
use a helper to compute descriptor block addresses.

No functional change intended. Existing Synopsys EDDA and AMD (Xilinx)
MDB/CPM6 block descriptors leave the override unset, so the helper still
returns the same values as before.

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-12-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Add register offset match flag

Add a match-data flag for devices whose DMA register block starts at an
offset inside the mapped BAR. Existing Synopsys EDDA and AMD (Xilinx)

dmaengine: dw-edma-pcie: Add register offset match flag

Add a match-data flag for devices whose DMA register block starts at an
offset inside the mapped BAR. Existing Synopsys EDDA and AMD (Xilinx)
MDB/CPM6 matches keep using the BAR mapping base directly.

The offsets handled here come from static, trusted match data. Later
metadata-derived offsets are validated when the endpoint DMA metadata
path parses them at that trust boundary.

No functional change intended.

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-11-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Add platform ops to match data

Move the platform ops pointer into match data. Existing EDDA/MDB/CPM6
matches keep using dw_edma_pcie_plat_ops.

No functional changes intende

dmaengine: dw-edma-pcie: Add platform ops to match data

Move the platform ops pointer into match data. Existing EDDA/MDB/CPM6
matches keep using dw_edma_pcie_plat_ops.

No functional changes intended.

Suggested-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-10-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Rename vsec_data to dma_data

dw_edma_pcie_probe() now obtains DMA layout data through device-specific
capability callbacks, not only from PCIe Vendor-Specific Extended
Capab

dmaengine: dw-edma-pcie: Rename vsec_data to dma_data

dw_edma_pcie_probe() now obtains DMA layout data through device-specific
capability callbacks, not only from PCIe Vendor-Specific Extended
Capabilities. Rename the local data copy from vsec_data to dma_data
before adding endpoint DMA BAR metadata discovery, which does not rely
on VSEC.

No functional change intended.

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-9-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Add capability match data

Move device-specific capability parsing behind per-device match data.

The existing probe path mixes two decisions: which static template a PCI
ID

dmaengine: dw-edma-pcie: Add capability match data

Move device-specific capability parsing behind per-device match data.

The existing probe path mixes two decisions: which static template a PCI
ID uses, and which device-specific capability parser adjusts that
template. Split those decisions so device-specific discovery can be
added through match data instead of adding more vendor checks to
dw_edma_pcie_probe().

No functional change is intended for the existing Synopsys EDDA and
AMD (Xilinx) MDB/CPM6 matches. They still copy the same static template
data and run the same capability parsing logic before BAR mapping. The
AMD (Xilinx) MDB/CPM6 entries also keep using endpoint memory physical
addresses for descriptor windows through a new match-data flag.

Suggested-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-8-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Track non-LL mode in DMA data

The dw-edma-pcie driver copies static template data into a mutable
dw_edma_pcie_data instance before applying capability-derived updates.
Keep

dmaengine: dw-edma-pcie: Track non-LL mode in DMA data

The dw-edma-pcie driver copies static template data into a mutable
dw_edma_pcie_data instance before applying capability-derived updates.
Keep the derived non-LL mode in that copy as well, instead of only
tracking it in a local variable in dw_edma_pcie_probe().

This prepares for keeping capability parsing behind match data without a
separate non-LL output parameter.

No functional change intended.

Suggested-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-7-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Add partial channel ownership mode

A DesignWare eDMA instance may represent only a subset of channels that
is also initialized by another OS instance, such as an endpoint-side OS

dmaengine: dw-edma: Add partial channel ownership mode

A DesignWare eDMA instance may represent only a subset of channels that
is also initialized by another OS instance, such as an endpoint-side OS.
Add a partial ownership flag for instances that must preserve
controller-wide state owned by that peer.

In partial ownership mode, dw-edma skips the initial core reset and uses
the limited quiesce path in probe() and remove() instead of the full
core-off path. The flag also makes the driver validate the ownership
granularity required by each register layout before registering
channels.

Probe fails if the limited quiesce cannot stop its resources; remove
reports the error after completing the remaining driver teardown.

Partial instances also skip interrupt-emulation doorbell allocation: the
emulated doorbell is a controller-level resource, and a partial owner
must not claim it on behalf of the whole block.

For EDMA_MF_EDMA_UNROLL and EDMA_MF_HDMA_COMPAT, the driver programs
per-direction registers, such as DMA_{WRITE,READ}_INT_MASK_OFF and
DMA_{WRITE,READ}_INT_CLEAR_OFF. These register layouts have at most
EDMA_MAX_{WR,RD}_CH channels per direction, so the capped hardware
channel count still represents the whole direction. A partial instance
can therefore expose write or read channels only if it owns every
channel in that direction; otherwise two OS instances could update the
same direction-wide registers without a shared locking protocol.

In contrast, HDMA native uses per-channel registers, so it can be owned
at channel granularity.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-6-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Initialize IRQ data before requesting IRQs

dw_edma_irq_request() passes struct dw_edma_irq to request_irq() before
dw_edma_channel_setup() fills the back pointer. A shared interr

dmaengine: dw-edma: Initialize IRQ data before requesting IRQs

dw_edma_irq_request() passes struct dw_edma_irq to request_irq() before
dw_edma_channel_setup() fills the back pointer. A shared interrupt can
therefore enter the handler with dw_irq->dw still NULL, leading to a
NULL pointer dereference.

Set the back pointer before installing each handler.

Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Cc: stable@vger.kernel.org
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-5-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Add core quiesce operations

Add core operations that quiesce only the resources represented by a
dw-edma instance, separate from the existing full controller off path.

For unrol

dmaengine: dw-edma: Add core quiesce operations

Add core operations that quiesce only the resources represented by a
dw-edma instance, separate from the existing full controller off path.

For unrolled eDMA, quiesce masks the direction, disables ENGINE_EN,
waits for it to read back clear, then clears pending interrupt status.
If ENGINE_EN does not clear, return the timeout to the caller.
HDMA compatibility mode does not implement ENGINE_EN, so quiesce clears
CHi_PWR_EN for each represented channel instead. Both register maps
share interrupt control per direction, so quiescing one channel
quiesces the whole direction. Callers must own that direction and stop
the peer from programming it first.

HDMA native has per-channel registers and can quiesce the represented
channel directly.

No caller is added yet, so this is a no-functional-change preparation
for delegated channel reclaim and partial-owned remove paths.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-4-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Add per-channel interrupt routing control

DesignWare eDMA can signal completion locally through edma_int[] and
remotely through IMWr/MSI. When channels are delegated to a remote

dmaengine: dw-edma: Add per-channel interrupt routing control

DesignWare eDMA can signal completion locally through edma_int[] and
remotely through IMWr/MSI. When channels are delegated to a remote
frontend, the local endpoint side and the remote host side must not both
service the same DONE/ABORT status.

Add channel interrupt routing state and initialize it from the controller
instance configuration. Update the eDMA and HDMA native paths so
linked-list interrupt generation, HDMA non-linked-list interrupt enables,
and DONE/ABORT masking follow the selected mode. For HDMA native
non-linked-list channels, keep the local stop/abort enables set so status
is latched. In remote mode, also enable remote signaling and mask the local
interrupt pins.

Keep the existing dw-edma-pcie host-side instances in remote interrupt
routing mode so their IMWr/MSI completion model remains unchanged after
local routing becomes the zero value.

Note:
- The routing mode describes where a channel should report completion.
It does not by itself say whether this dw-edma instance owns the
interrupt status. A local instance must ignore remote-only channels,
and a remote instance must ignore local-only channels, even if such
interrupts are unexpectedly delivered. Otherwise the non-owner side
could steal the interrupt from the owner by clearing shared DONE/ABORT
status.

Cc: Devendra K Verma <devendra.verma@amd.com>
Suggested-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-3-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Factor out HDMA interrupt setup helper

The HDMA linked-list and non-linked-list start paths both program the
stop/abort interrupt setup register using the same local/remote enabl

dmaengine: dw-edma: Factor out HDMA interrupt setup helper

The HDMA linked-list and non-linked-list start paths both program the
stop/abort interrupt setup register using the same local/remote enable
policy. Only the interrupt-mask handling differs by transfer mode.

Factor the common setup into dw_hdma_v0_core_int_setup() before adding
per-channel interrupt routing support. No functional change intended.

Suggested-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260721062815.4117887-2-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Defer channel IRQ handling to workqueue

On some SoCs (e.g. R-Car S4) the endpoint-side eDMA raises a single
fixed SPI that is hardwired to CPU0 and covers every read and write
ch

dmaengine: dw-edma: Defer channel IRQ handling to workqueue

On some SoCs (e.g. R-Car S4) the endpoint-side eDMA raises a single
fixed SPI that is hardwired to CPU0 and covers every read and write
channel. Handling channel events directly in that hard IRQ context
serializes the completion processing of all channels on one CPU:
descriptor recycling and refill, client callbacks (the vchan tasklet
runs on the scheduling CPU) and the doorbell writes all funnel through
CPU0, while the handler additionally spins on each channel's vc.lock.
Especially under heavy multichannel load, this contention becomes a
performance bottleneck.

Keep the hard IRQ handler minimal: clear the status, dispatch channel
events, and defer per-channel processing to work items. A work item per
channel preserves ordering while allowing different channels to run in
parallel on any CPU.

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-10-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Snapshot the v0 interrupt status once per handler pass

The v0 interrupt handler reads the interrupt status register twice per
invocation, once through the DONE accessor and once

dmaengine: dw-edma: Snapshot the v0 interrupt status once per handler pass

The v0 interrupt handler reads the interrupt status register twice per
invocation, once through the DONE accessor and once through the ABORT
accessor, although both fields live in the same 32-bit register. On
remote setups (dw-edma-pcie) each read is a non-posted round trip across
the PCIe link costing on the order of a microsecond, and with one
completion interrupt per element the duplicate adds up. As an example,
profiling the R-Car S4 remote path put the handler at ~7us per
invocation, dominated by such reads.

Read the register once and derive the DONE and ABORT views from the
snapshot. No abort is lost to this because the pass only clears status
bits it observed, so an abort raised after the snapshot keeps its status
and its own interrupt delivery brings it to the next pass. An abort on
an observed channel cannot race the clear either. Software can restart
the halted channel only after abort() runs, and abort() is called after
dw_edma_v0_core_clear_abort_int().

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-9-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma-pcie: Drop redundant pci_free_irq_vectors()

dw_edma_pcie enables the PCI device with pcim_enable_device(), so IRQ
vectors allocated by pci_alloc_irq_vectors() are released by
pcim

dmaengine: dw-edma-pcie: Drop redundant pci_free_irq_vectors()

dw_edma_pcie enables the PCI device with pcim_enable_device(), so IRQ
vectors allocated by pci_alloc_irq_vectors() are released by
pcim_msi_release() on device release. The driver should not call
pci_free_irq_vectors() manually.

Drop the redundant remove-time cleanup and rely on the managed PCI
device lifetime instead, as documented by commit 03e4905402ae ("PCI/MSI:
Clarify pci_free_irq_vectors() usage for managed devices").

Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-8-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Clear stale requests on termination

terminate_all() can finish immediately when the channel is unconfigured,
paused, idle, or already stopped in hardware. A pending PAUSE request

dmaengine: dw-edma: Clear stale requests on termination

terminate_all() can finish immediately when the channel is unconfigured,
paused, idle, or already stopped in hardware. A pending PAUSE request can
survive these paths and block issue_pending() even after termination.

Clear the request whenever termination leaves the channel idle. A running
channel keeps its STOP request until the interrupt handler consumes it.

Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-7-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Serialize channel state checks

pause() and resume() read and update channel state without holding vc.lock,
while the interrupt handlers update the same state under it. Take the s

dmaengine: dw-edma: Serialize channel state checks

pause() and resume() read and update channel state without holding vc.lock,
while the interrupt handlers update the same state under it. Take the same
lock around those state checks so that request, status, and configured stay
consistent.

For example, pause() can observe EDMA_ST_BUSY right before the interrupt
handler completes the final descriptor and moves the channel to
EDMA_ST_IDLE, and then record EDMA_REQ_PAUSE on an already idle channel. No
further interrupt will acknowledge the request, and since issue_pending()
requires EDMA_REQ_NONE, the channel is wedged for good: terminate_all()
leaves the stale request behind, so even reconfiguring the channel does not
recover it.

issue_pending() already runs under vc.lock, but it tests configured before
taking it. Move that test under the lock as well, so configured, request,
and status are evaluated as one channel-state snapshot.

Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-6-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Complete descriptors before pausing

If PAUSE is requested while the final burst of a descriptor is in
flight, the DONE interrupt takes the PAUSE path without checking whether
the

dmaengine: dw-edma: Complete descriptors before pausing

If PAUSE is requested while the final burst of a descriptor is in
flight, the DONE interrupt takes the PAUSE path without checking whether
the descriptor has been depleted. The depleted descriptor remains on the
issued list and the channel enters EDMA_ST_PAUSE.

On resume, dw_edma_start_transfer() can select that depleted descriptor
again even though no burst remains, leaving the channel in an invalid
busy state.

Check for descriptor completion before acknowledging PAUSE. If there is
no work to start on resume, leave the channel idle. Also ignore DONE
interrupts while the channel is paused so a stale or repeated interrupt
cannot change its state or start queued work.

Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Cc: stable@vger.kernel.org
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-5-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Serialize abort state updates

dw_edma_abort_interrupt() drops vc.lock before changing request and
status. issue_pending() can acquire the lock in that small window,
observe the o

dmaengine: dw-edma: Serialize abort state updates

dw_edma_abort_interrupt() drops vc.lock before changing request and
status. issue_pending() can acquire the lock in that small window,
observe the old busy state, and skip starting queued descriptors. Then
the abort handler overwrites the channel status as idle, leaving the new
descriptors stranded for good.

Keep descriptor completion and the state transition in the same critical
section.

Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-4-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Terminate all descriptors without callbacks

The DMA Engine client documentation says in the "Terminate APIs" section
of Documentation/driver-api/dmaengine/client.rst:

"No callba

dmaengine: dw-edma: Terminate all descriptors without callbacks

The DMA Engine client documentation says in the "Terminate APIs" section
of Documentation/driver-api/dmaengine/client.rst:

"No callback functions will be called for any incomplete transfers."

dw-edma instead calls vchan_cookie_complete() when a deferred STOP reaches
the interrupt handler. This schedules a callback for the active descriptor
and leaves other issued or submitted descriptors queued. A late callback
after dmaengine_terminate_sync() can dereference client state that has
already been freed, while leftover descriptors may later restart into
reused buffers or leak.

Move all issued and submitted descriptors to the terminated list whenever
termination completes. For a pending STOP, do this from both the DONE and
ABORT paths. Complete their cookies in order without scheduling callbacks.

A STOP can remain pending until the running transfer raises an
interrupt. Make device_synchronize() wait for such a pending STOP to
complete before releasing terminated descriptors. Reuse it from
free_chan_resources(), then release the remaining virt-dma resources.
Sleep instead of busy-polling while waiting, and warn if the existing
timeout expires.

Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-3-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Fix HDMA channel status register access

GET_CH_32() takes the direction before the channel ID, but
dw_hdma_v0_core_ch_status() passed them in the opposite order. This can
make th

dmaengine: dw-edma: Fix HDMA channel status register access

GET_CH_32() takes the direction before the channel ID, but
dw_hdma_v0_core_ch_status() passed them in the opposite order. This can
make the status callback read another HDMA channel status register.

Use the same argument order as the other HDMA register accesses.

Fixes: e74c39573d35 ("dmaengine: dw-edma: Add support for native HDMA")
Cc: stable@vger.kernel.org
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260717180639.2643243-2-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...

dmaengine: dw-edma: Enable HDMA 64R/W Channels

As per 'Designware Cores PCI Express Controller Databook',
Section 7.1 - Overview, HDMA supports 64 Read and 64 Write
channels. Current controller driv

dmaengine: dw-edma: Enable HDMA 64R/W Channels

As per 'Designware Cores PCI Express Controller Databook',
Section 7.1 - Overview, HDMA supports 64 Read and 64 Write
channels. Current controller driver supports up to 8 read and
write channels only. In order to utilize all the channels the
controller driver need to have the channel related structs
and variables as per the number of channels supported by IP.
Following changes are made to enable 64 Read / 64 Write
channel support:

o Defined HDMA specific macros to reflect the channel count.
o The count of ll_regions and dt_regions in dw_edma_chip and
dw_edma_pcie_data shall be in accordance to number of read
and write channels.
o In dw_edma_probe() configure the channels as per the channels
of the IP used.
o Changed mask types to u64 for higher channel counts.

Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713064854.4065262-1-devverma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>

show more ...