The dma_free_coherent() function needs a valid device pointer or it will
crash.
Fixes: 550c591a89a1 ("dmaengine: sh: Add DMAC driver for RZ/G2L SoC")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Link: https://lore.kernel.org/r/20210827085410.GA9183@kili
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Pointer lmdesc is being inintialized with a value that is never read,
it is later being re-assigned a new value. Fix this by initializing
it with the latter value.
Addresses-Coverity: ("Unused value")
Fixes: 550c591a89a1 ("dmaengine: sh: Add DMAC driver for RZ/G2L SoC")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Link: https://lore.kernel.org/r/20210829152811.529766-1-colin.king@canonical.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Building on ARCH=um causes a "redefined" warning, so remove this
PT_OFFSET macro to avoid the warning.
drivers/dma/ptdma/ptdma.h:34: warning: "PT_OFFSET" redefined
34 | #define PT_OFFSET 0x0
|
In file included from ./arch/um/include/asm/thread_info.h:17,
from ./include/linux/thread_info.h:60,
from ./include/asm-generic/preempt.h:5,
from ./arch/um/include/generated/asm/preempt.h:1,
from ./include/linux/preempt.h:78,
from ./include/linux/spinlock.h:55,
from ./include/linux/wait.h:9,
from ./include/linux/wait_bit.h:8,
from ./include/linux/fs.h:6,
from ./include/linux/debugfs.h:15,
from drivers/dma/ptdma/ptdma-debugfs.c:12:
./arch/x86/um/shared/sysdep/ptrace_user.h:4: note: this is the location of the previous definition
4 | #define PT_OFFSET(r) ((r) * sizeof(long))
Reported-by: kernel test robot <lkp@intel.com>
Fixes: 2a99524459ce ("dmaengine: ptdma: Initial driver for the AMD PTDMA")
Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Link: https://lore.kernel.org/r/1630178908-54973-1-git-send-email-Sanju.Mehta@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Expose data about the configuration and operation of the
PTDMA through debugfs entries: device name, capabilities,
configuration, statistics.
Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Link: https://lore.kernel.org/r/1629208559-51964-4-git-send-email-Sanju.Mehta@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add support for AMD PTDMA controller. It performs high-bandwidth
memory to memory and IO copy operation. Device commands are managed
via a circular queue of 'descriptors', each of which specifies source
and destination addresses for copying a single buffer of data.
Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
Link: https://lore.kernel.org/r/1629208559-51964-2-git-send-email-Sanju.Mehta@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The spinlock is not being used in hard interrupt context. There is no need
to disable irq when acquiring the lock. The interrupt thread handler also
is not in bottom half context, therefore we can also remove disabling of
the bh. Convert all dev_lock acquisition to plain spin_lock() calls.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162984026772.1939166.11504067782824765879.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The cmd_lock spinlock is not being used in hard interrupt context. There is
no need to disable irq when acquiring the lock. Convert all cmd_lock
acquisition to plain spin_lock() calls.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162984027930.1939209.15758413737332339204.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
DSA spec says WQ priv bit is 0 if the Privileged Mode Enable field of the
PCI Express PASID capability is 0 and pasid is enabled. Make sure that the
WQCFG priv field is set correctly according to usage type. Reject config if
setting up kernel WQ type and no support. Also add the correct priv setup
for a descriptor.
Fixes: 484f910e93 ("dmaengine: idxd: fix wq config registers offset programming")
Cc: Ramesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162939084657.903168.14160019185148244596.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The xilinx dma driver uses the consistent allocations, so for correct
operation also set the DMA mask for coherent APIs. It fixes the below
kernel crash with dmatest client when DMA IP is configured with 64-bit
address width and linux is booted from high (>4GB) memory.
Call trace:
[ 489.531257] dma_alloc_from_pool+0x8c/0x1c0
[ 489.535431] dma_direct_alloc+0x284/0x330
[ 489.539432] dma_alloc_attrs+0x80/0xf0
[ 489.543174] dma_pool_alloc+0x160/0x2c0
[ 489.547003] xilinx_cdma_prep_memcpy+0xa4/0x180
[ 489.551524] dmatest_func+0x3cc/0x114c
[ 489.555266] kthread+0x124/0x130
[ 489.558486] ret_from_fork+0x10/0x3c
[ 489.562051] ---[ end trace 248625b2d596a90a ]---
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Reviewed-by: Harini Katakam <harini.katakam@xilinx.com>
Link: https://lore.kernel.org/r/1629363528-30347-1-git-send-email-radhey.shyam.pandey@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The CSI2RX subsystem on J721E is serviced by UDMA via PSI-L to transfer
frames to memory. It can have up to 32 threads per instance. J721E has
two instances of the subsystem, so there are 64 threads total. Add them
to the endpoint map.
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Acked-by: Peter Ujfalusi <peter.ujflausi@gmail.com>
Link: https://lore.kernel.org/r/20210819110106.31409-1-p.yadav@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add DMA Controller driver for RZ/G2L SoC.
Based on the work done by Chris Brandt for RZ/A DMA driver.
Signed-off-by: Biju Das <biju.das.jz@bp.renesas.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://lore.kernel.org/r/20210806095322.2326-4-biju.das.jz@bp.renesas.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Now that UML has PCI support, this driver must depend also on
!UML since it pokes at X86_64 architecture internals that don't
exist on ARCH=um.
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20210809112409.a3a0974874d2.I2ffe3d11ed37f735da2f39884a74c953b258b995@changeid
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Since submission is sent to limited portal, the actual wq size for shared
wq is set by the threshold rather than the wq size. When the wq type is
shared, set the allocated descriptors to the threshold.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162827151733.3459223.3829837172226042408.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The submission path for dmaengine API does not do descriptor freeing on
failure. Also, with the abort mechanism, the freeing of descriptor happens
when the abort callback is completed. Therefore free descriptor on all
error paths for submission call to make things consistent. Also remove the
double free that would happen on abort in idxd_dma_tx_submit() call.
Fixes: 6b4b87f2c3 ("dmaengine: idxd: fix submission race window")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162827146072.3459011.10255348500504659810.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The list lock is never acquired in interrupt context. Therefore there is no
need to disable interrupts. Remove interrupt flags for lock operations.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162826417450.3454650.3733188117742416238.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
While IRQ test agaist the returned variable in practice is a good enough
there is still a room for theoretical mistake in case the vIRQ of the
device contains the same error code that acpi_register_gsi() may return.
Due to this, check for error code separately from matching the vIRQs.
Besides that, append documentation to tell why acpi_gsi_to_irq() can't
be used and we call acpi_register_gsi() instead.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://lore.kernel.org/r/20210802175532.54311-1-andriy.shevchenko@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Currently the CRST parsing relies on the fact that on most of x86 devices
the IRQ mapping is 1:1 with Linux vIRQ. However, it may be not true for
some. Fix this by converting GSI to Linux vIRQ before checking it.
Fixes: ee8209fd02 ("dma: acpi-dma: parse CSRT to extract additional resources")
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://lore.kernel.org/r/20210730202715.24375-1-andriy.shevchenko@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The interrupt thread handler currently loops forever to process outstanding
completions. This causes either an "irq X: nobody cared" kernel splat or
the NMI watchdog kicks in due to running too long in the function. The irq
thread handler is expected to run again after exiting if there are
interrupts fired while the thread handler is running. So the handler code
can process all the completed I/O in a single pass and exit without losing
the follow on completed I/O.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162802977005.3084234.11836261157026497585.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The block on fault flag is not cleared when we disable or reset wq. This
causes it to remain set if the user does not clear it on the next
configuration load. Add clear of flag in dxd_wq_disable_cleanup()
routine.
Fixes: da32b28c95 ("dmaengine: idxd: cleanup workqueue config after disabling")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162803023553.3086015.8158952172068868803.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Various DMA users call the dmaengine_slave_config() and expect it to
succeed, but that can only succeed if .device_config is implemented.
Add empty device_config function rather than patching all the places
which use dmaengine_slave_config().
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Kedareswara rao Appana <appana.durga.rao@xilinx.com>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Vinod Koul <vkoul@kernel.org>
Link: https://lore.kernel.org/r/20210804195140.61396-1-marex@denx.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Since we converted internal data types to match DT, there is no need to have
an intermediate conversion layer, hence drop a few conditionals and for loops
for good.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Link: https://lore.kernel.org/r/20210802184355.49879-3-andriy.shevchenko@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Users are a bit frightened of the harmless message that tells that
DT is missed on ACPI-based platforms. Remove it for good, it will
simplify the future conversion to fwnode and device property APIs.
Fixes: a9ddb575d6 ("dmaengine: dw_dmac: Enhance device tree support")
Depends-on: f5e84eae79 ("dmaengine: dw: platform: Split OF helpers to separate module")
BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=199379
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Link: https://lore.kernel.org/r/20210802184355.49879-1-andriy.shevchenko@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
'device_driver_attach()' dereferences its first argument (i.e. 'alt_drv')
so it must not be NULL.
Simplify the error handling logic about NULL 'alt_drv' in order to be
more robust and future-proof.
Fixes: 568b212646 ("dmaengine: idxd: fix uninit var for alt_drv")
Fixes: 6e7f3ee97b ("dmaengine: idxd: move dsa_drv support to compatible mode")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/77f0dc4f3966591d1f0cffb614a94085f8895a85.1627560174.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Commit dea7a9fbb0 ("dmaengine: imx-dma: remove dma_slave_config
direction usage") changes the method from a "configuration when called"
to an "configuration when used". Due to this, only the cyclic DMA type
gets configured correctly, while the generic DMA type is left
non-configured.
Without this additional call, the struct imxdma_channel::word_size member
is stuck at DMA_SLAVE_BUSWIDTH_UNDEFINED and imxdma_prep_slave_sg() always
returns NULL.
Signed-off-by: Juergen Borleis <jbe@pengutronix.de>
Fixes: dea7a9fbb0 ("dmaengine: imx-dma: remove dma_slave_config direction usage")
Link: https://lore.kernel.org/r/20210729071821.9857-1-jbe@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Commit dea7a9f
dmaengine: imx-dma: remove dma_slave_config direction usage
changes the method from a "configuration when called" to an "configuration
when used". Due to this, only the cyclic DMA type gets configured
correctly, while the generic DMA type is left non-configured.
Without this additional call, the struct imxdma_channel::word_size member
is stuck at DMA_SLAVE_BUSWIDTH_UNDEFINED and imxdma_prep_slave_sg() always
returns NULL.
Signed-off-by: Juergen Borleis <jbe@pengutronix.de>
Link: https://lore.kernel.org/r/20210729071821.9857-1-jbe@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Burst length, DMA HW capability set in dt-binding is now used in driver.
Signed-off-by: Pandith N <pandith.n@intel.com>
Tested-by: Pan Kris <kris.pan@intel.com>
Link: https://lore.kernel.org/r/20210802055454.15192-4-pandith.n@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Added support for multiple DMA_MEM_TO_DEV, DMA_DEV_TO_MEM transfers in
parallel. This is required for peripherals using DMA for transmit and
receive operations at the same time. APB slot number needs to be
programmed in channel hardware handshaking interface
Signed-off-by: Pandith N <pandith.n@intel.com>
Tested-by: Pan Kris <kris.pan@intel.com>
Link: https://lore.kernel.org/r/20210802055454.15192-3-pandith.n@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Removed free slot check algorithm in dw_axi_dma_set_hw_channel. For 8
DMA channels, use respective handshake slot in DMA_HS_SEL APB register.
For every channel, an dedicated slot is provided in hardware handshake
register AXIDMA_CTRL_DMA_HS_SEL_n. Peripheral source number is
programmed in respective channel slots.
Signed-off-by: Pandith N <pandith.n@intel.com>
Tested-by: Pan Kris <kris.pan@intel.com>
Link: https://lore.kernel.org/r/20210802055454.15192-2-pandith.n@intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The usb_dmac_get_current_residue function used to
take a signed integer as a pos parameter.
The only callers of this function passes an unsigned integer to it.
Therefore to make it obviously safe, let's just make this an unsgined
integer as this is used in pointer arithmetics.
Signed-off-by: Jordy Zomer <jordy@pwning.systems>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/20210731091939.510816-1-jordy@pwning.systems
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Coverity static analysis of linux-next found issue.
The check (status == IDXD_COMP_DESC_ABORT) is always false since status
was previously masked with 0x7f and IDXD_COMP_DESC_ABORT is 0xff.
Fixes: 6b4b87f2c3 ("dmaengine: idxd: fix submission race window")
Reported-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162698465160.3560828.18173186265683415384.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Enabling device and wq returns standard errno and that does not provide
enough details to indicate what exactly failed. The hardware command status
is only 8bits. Expand the command status to 32bits and use the upper 16
bits to define software errors to provide more details on the exact
failure. Bit 31 will be used to indicate the error is software set as the
driver is using some of the spec defined hardware error as well.
Cc: Ramesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681373579.1968485.5891788397526827892.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The device submission portal is on a 4k page and any of those 64bit aligned
address on the page can be used for descriptor submission. By rotating the
offset through the 4k range and prevent successive writes to the same MMIO
address, performance improvement is observed through testing.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681372446.1968485.10634280461681015569.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The sbitmap wait and allocate routine checks the index that is returned
from sbitmap_queue_get(). It should be idxd >= 0 as 0 is also a valid
index. This fixes issue where submission path hangs when WQ size is 1.
Fixes: 0705107fcc ("dmaengine: idxd: move submission to sbitmap_queue")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162697645067.3478714.506720687816951762.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
0-day detected uninitialized alt_drv variable in the bind_store() function.
The branch can be taken when device is not idxd device or wq 'struct
device'. Init alt_drv to NULL.
Fixes: 6e7f3ee97b ("dmaengine: idxd: move dsa_drv support to compatible mode")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162689250332.2114335.636367120454420852.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Set GRPCFG traffic class to value of 1 for best performance on current
generation of accelerators. Also add override option to allow experimentation.
Sysfs knobs are disabled for DSA/IAX gen1 devices.
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681373005.1968485.3761065664382799202.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When using SCMI clocks, the clocks are probed later than subsys initcall
level. This driver uses platform_driver_probe which is not compatible with
deferred probing and won't be probed again later if probe function fails
due to clocks not being available at that time.
This patch replaces the use of platform_driver_probe with
platform_driver_register which will allow probing the driver later again
when clocks will be available.
Signed-off-by: Clément Léger <clement.leger@bootlin.com>
Link: https://lore.kernel.org/r/20210728094607.50589-1-clement.leger@bootlin.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
If the router_xlate can not find the controller in the available DMA
devices then it should return with -EPORBE_DEFER in a same way as the
of_dma_request_slave_channel() does.
The issue can be reproduced if the event router is registered before the
DMA controller itself and a driver would request for a channel before the
controller is registered.
In of_dma_request_slave_channel():
1. of_dma_find_controller() would find the dma_router
2. ofdma->of_dma_xlate() would fail and returned NULL
3. -ENODEV is returned as error code
with this patch we would return in this case the correct -EPROBE_DEFER and
the client can try to request the channel later.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210717190021.21897-1-peter.ujfalusi@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
use pm_runtime_resume_and_get() to replace pm_runtime_get_sync and
pm_runtime_put_noidle. this change is just to simplify the code,
there is no actual functional change.
Signed-off-by: Zhang Qilong <zhangqilong3@huawei.com>
Link: https://lore.kernel.org/r/20210607064640.121394-4-zhangqilong3@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Fix the checkpatch.pl warning: "Prefer kstrto<type> to single variable sscanf".
Signed-off-by: Salah Triki <salah.triki@gmail.com>
Link: https://lore.kernel.org/r/20210710165432.GA690401@pc
Signed-off-by: Vinod Koul <vkoul@kernel.org>
STM32 USART/UART is not managing correctly the default DMA REQ/ACK protocol
leading to possibly lock the DMA stream.
Default protocol consists in maintaining ACK signal up to the removal of
REQuest and the transfer completion.
In case of alternative REQ/ACK protocol, ACK de-assertion does not wait the
removal of the REQuest, but only the transfer completion.
This patch retrieves the need of the alternative protocol through the
device tree, and sets the protocol accordingly.
It also unwrap STM32_DMA_DIRECT_MODE_GET macro definition for consistency
with new STM32_DMA_ALT_ACK_MODE_GET macro definition.
Signed-off-by: Amelie Delaunay <amelie.delaunay@foss.st.com>
Link: https://lore.kernel.org/r/20210624093959.142265-3-amelie.delaunay@foss.st.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When using 'pcim_enable_device()', 'pci_alloc_irq_vectors()' is
auto-magically a managed function.
It is useless (but harmless) to record an action to explicitly call
'pci_free_irq_vectors()'.
So keep things simple, comment why and how these resources are freed, axe
some useless code and save some memory.
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://lore.kernel.org/r/4f8932e2d0d8d092bf60272511100030e013bc72.1623875508.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
