Implement suspend/resume callbacks to store APB DMA channel's
register on suspend and restore APB DMA channel's register on
resume.

Signed-off-by: Laxman Dewangan <ldewangan@nvidia.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Conflicts:
	include/linux/dmaengine.h

Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Currently the OF DMA code uses a spin lock to protect the of_dma_list from
concurrent access and a per controller reference count to protect the controller
from being freed while a request operation is in progress. If
of_dma_controller_free() is called for a controller who's reference count is not
zero it will return -EBUSY and not remove the controller. This is fine up until
here, but leaves the question what the caller of of_dma_controller_free() is
supposed to do if the controller couldn't be freed.  The only viable solution
for the caller is to spin on of_dma_controller_free() until it returns success.
E.g.

	do {
		ret = of_dma_controller_free(dev->of_node)
	} while (ret != -EBUSY);

This is rather ugly and unnecessary and none of the current users of
of_dma_controller_free() check it's return value anyway. Instead protect the
list by a mutex. The mutex will be held as long as a request operation is in
progress. So if of_dma_controller_free() is called while a request operation is
in progress it will be put to sleep and only wake up once the request operation
has finished.

This means that it is no longer possible to register or unregister OF DMA
controllers from a context where it's not possible to sleep. But I doubt that
we'll ever need this.

Also rename of_dma_get_controller back to of_dma_find_controller.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

of_dma_request_slave_channel() currently does not drop the reference to the
dma_spec of_node if no DMA controller matching the of_node could be found. This
patch fixes it by always calling of_node_put().

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Jon Hunter <jon-hunter@ti.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

if we initilize dma driver by module_init, there are still many devices
which will be initilized earlier than dma. these devices will fail to
get dma channel.
this moves dmaengine earlier than device_initcall and make dma available
for all devices.

Reported-by: Renwei Wu <Renwei.Wu@csr.com>
Signed-off-by: Barry Song <Baohua.Song@csr.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Some Renesas USB modules have SUDMAC. This patch supports it using
the shdma-base driver.

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

This patch adds Kconfig in the drivers/dma/sh. This patch also adds
a new config "SH_DMAE_BASE" and the "config SH_DMAE" depends on it.
Since some drivers (e.g. sh_mmcif.c) depends on shdma-base.c if
CONFIG_DMA_ENGINE=y, the "config SH_DMAE_BASE" is set as "bool".

Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Update at_hdmac driver to support generic DMA device tree binding. Devices
can still request channel with dma_request_channel() then it doesn't break
DMA for non DT boards.

Signed-off-by: Ludovic Desroches <ludovic.desroches@atmel.com>
Acked-by: Nicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

v3.3 provides support for write back descriptor error status. This allows
reporting of errors in a descriptor field. In supporting this, certain
errors such as P/Q validation errors no longer halts the channel. The DMA
engine can continue to execute until the end of the chain and allow software
to report the "errors" up the stack. We are also going to mask those error
interrupts and handle them when the "chain" has completed at the end.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

This workaround checks for channel 2&3 and remove RAID cap.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

v3.3 introduced 16 sources PQ operations. This also introduced super extended
descriptors to support the 16 srcs operations. This patch adds support for
the 16 sources ops and in turn adds the super extended descriptors for those
ops.

5 SED pools are created depending on the descriptor sizes. An SED can be a 64
bytes sized descriptor or larger and must be physically contiguous. A kmem
cache pool is created for allocating the software descriptor that manages the
hardware descriptor. The super extended descriptor will take place of extended
descriptor under certain operations and be "attached" to the op descriptor
during operation. This is a new feature for ioatdma v3.3.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

CB3.2 and earlier hardware has silicon bugs that are no longer needed with
the new hardware. We don't have to use a NULL op to signal interrupt for
RAID ops any longer. This code make sure the legacy workarounds only happen on
legacy hardware.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Since we have proper ACPI DMA helpers implemented, the driver may use it. This
patch introduces custom filter function together with acpi_device_id table.

Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The slave device could be enumerated by ACPI. In that case the
dma_request_slave_channel should use the acpi_dma_request_slave_channel()
helper.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

There is a new generic API to get a DMA channel for a slave device (commit
9a6cecc8 "dmaengine: add helper function to request a slave DMA channel"). In
similar fashion to the DT case (commit aa3da644

 "of: Add generic device tree
DMA helpers") we introduce helpers to the DMAC drivers which are enumerated by
ACPI.

The proposed extension provides the following API calls:
	acpi_dma_controller_register(), devm_acpi_dma_controller_register()
	acpi_dma_controller_free(), devm_acpi_dma_controller_free()
	acpi_dma_simple_xlate()
	acpi_dma_request_slave_chan_by_index()
	acpi_dma_request_slave_chan_by_name()

The first two should be used, for example, at probe() and remove() of the
corresponding DMAC driver. At the register stage the DMAC driver supplies a
custom xlate() function to translate a struct dma_spec into struct dma_chan.

Accordingly to the ACPI Fixed DMA resource specification the only two pieces of
information the slave device has are the channel id and the request line (slave
id). Those two are represented by struct dma_spec. The
acpi_dma_request_slave_chan_by_index() provides access to the specifix FixedDMA
resource by its index. Whereas dma_request_slave_channel() takes a string
parameter to identify the DMA resources required by the slave device. To make a
slave device driver work with both DeviceTree and ACPI enumeration a simple
convention is established: "tx" corresponds to the index 0 and "rx" to the
index 1. In case of robust configuration the slave device driver unfortunately
needs to call acpi_dma_request_slave_chan_by_index() directly.

Additionally the patch provides "managed" version of the register/free pair
i.e. devm_acpi_dma_controller_register() and devm_acpi_dma_controller_free().
Usually, the driver uses only devm_acpi_dma_controller_register().

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

list_for_each_entry is able to handle empty lists just fine, there is no need
to make sure that the list is non empty.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

In of_dma_controller_register() routine we are calling of_get_property() as an
parameter to be32_to_cpup(). In case the property doesn't exist we will get a
crash.

This patch changes this code to check if we got a valid property first and then
runs be32_to_cpup() on it.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

No DMA of-function alters the name, so this patch changes the name arguments
to be constant. Most drivers will probably request DMA channels using a
constant name.

Signed-off-by: Markus Pargmann <mpa@pengutronix.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The alignment workaround is only necessary for cb3.2 or earlier platforms.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The PQ Val ops work on the newer hardware so we should actually provide support
for it and remove the disabling bits.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Make it so only 3.2 and earlier platform need the PCI config register
clearings since this implementation does not have the registers.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The Intel Atom S1200 family ioatdma changed the channel reset behavior.
It does a reset similar to PCI FLR by resetting all the MSIX
registers. We have to re-init msix interrupts because of this. This
workaround is only specific to this platform and is not expected to carry
over to the later generations.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The channel status register for v3.3 is now 64bit. Use readq if available
on v3.3 platforms.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

These should be good for the IOAT DMA devices on the Intel Atom S1269,
S1279, and S1289 platforms. We are also adding IOAT v3.3 definition for
the new DMA engine.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Adding Haswell PCI device IDs for ioatdma and simplify the detection of
certain Xeon CPUs that has alignment bugs so that modifications can be
changed at a single place going forward.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Dan Williams <djbw@fb.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

If the device-tree blob is present during boot, then register the SDMA
controller with the device-tree DMA driver so that we can use device-tree
to look-up DMA client information.

Signed-off-by: Jon Hunter <jon-hunter@ti.com>
Reviewed-by: Felipe Balbi <balbi@ti.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

If CONFIG_OF is not set the of_node of the device will always be NULL.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The patch to add the common DMA binding added a dummy dw_dma_slave
structure into the dw_dma_chan structure in order to configure the
masters correctly. It turns out that this can be simplified if we
pick the DMA masters in the dwc_alloc_chan_resources function instead
and save them in the dw_dma_chan structure directly.

This could be simplified further once all users that today use
dw_dma_slave for configuration get converted to device tree based
setup instead.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Cc: linux-arm-kernel@lists.infradead.org
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Since we will have not only DT cases in future let's rename DT related methods
to reflect their belonging.

The rename was done as follows:
	struct dw_dma_filter_args	-> struct dw_dma_of_filter_args
	dw_dma_generic_filter()		-> dw_dma_of_filter()
	dw_dma_xlate()			-> dw_dma_of_xlate()
	dw_dma_id_table			-> dw_dma_of_id_table

There is no functional change.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Let's use capital letter as a first one in the comments.
There is no functional changes.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The 'mask' parameter is not modified in __dma_request_channel and really
shouldn't be. Make this explicit by making the parameter const.

Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

there is hardcode which enabled the clock of dmaengine before,
this patch takes the clock by standard clock API and enable it
in probe.

Signed-off-by: Barry Song <Baohua.Song@csr.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

When we pause the channel after transfer is completed we might stuck in the
dwc_chan_pause() because the FIFO_EMPTY flag will never be asserted. To avoid
the endless loop we introduce a timeout here (*). The proper solution is to
somehow get the residue in FIFO and avoid busyloop when transfer is done, but
this task is not simple and fast.

Unfortunately we can't use cpu_relax() in conjunction with jiffies checker, due
to we have interrupts disabled by spin_lock_irqsave() and there is a big chance
that no interrupts will come to update the jiffies..

(*) The worst case is
	AHB write * FIFO size / hclk = 5.12 us,
    where
	AHB write = 2 cycles,
	hclk = 100 MHz,
	burst size = 1 byte,
	FIFO size = 256 bytes.
    The proposed 40us timeout might be considered as a big one, though we enter
    to that state only when we have the transfer already completed.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

kfree on NULL pointer is a no-op.

Signed-off-by: Syam Sidhardhan <s.syam@samsung.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Since commit 84c1e63c12 (dma: Remove erroneous __exit and __exit_p() references)
the following section mismatch happens:

WARNING: drivers/built-in.o(.text+0x20f94): Section mismatch in reference from the function ipu_remove() to the function .exit.text:ipu_idmac_exit()
The function ipu_remove() references a function in an exit section.
Often the function ipu_idmac_exit() has valid usage outside the exit section
and the fix is to remove the __exit annotation of ipu_idmac_exit.

Remove the '__exit' annotation from ipu_idmac_exit in order to fix it.

Signed-off-by: Fabio Estevam <fabio.estevam@freescale.com>
Acked-by: Maxin B. John <maxin.john@enea.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Tegra only supports, and always enables, device tree. Remove all ifdefs
and runtime checks for DT support from the driver.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Use the newly introduced devm_ioremap_resource() instead of
devm_request_and_ioremap() which provides more consistent error handling.

Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Reviewed-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

Comparison between buffers is stored to the dedicated structure.

Note that the verify result is now accessible only via file 'results' in the
debugfs.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>

The patch provides a storage for the test results in the linked list. The
gathered data could be used after test is done.

The new file 'results' represents gathered data of the in progress test. The
messages collected are printed to the kernel log as well.

Example of output:
	% cat /sys/kernel/debug/dmatest/results
	dma0chan0-copy0: #1: No errors with src_off=0x7bf dst_off=0x8ad len=0x3fea (0)

The message format is unified across the different types of errors. A number in
the parens represents additional information, e.g. error code, error counter,
or status.

Note that the buffer comparison is done in the old way, i.e. data is not
collected and just printed out.

Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>