The Linux DMA API: Coherent and Streaming Mappings Explained

Almost every driver that moves real data uses the Linux DMA API. It is also one of the areas where a driver can look correct, pass a quick test, and then corrupt data months later on a different board. This article walks through what the DMA API actually does, the difference between coherent and streaming mappings, where cache maintenance happens, how the kernel decides whether your device is coherent at all, and how to debug a driver that gets it wrong.

This is the rules-first companion to Inside the Linux DMA Core: From dma_map_phys() to the Cache Ops, which traces the same interfaces down through kernel/dma/mapping.c into the arm64 cache hooks. Read this one for the contract your driver has to honour; read that one when you want to see the implementation underneath it.

Why the Linux DMA API exists: three address spaces

The kernel documentation is explicit that there are three kinds of addresses in play:

• CPU virtual address — what kmalloc() or ioremap() returns. The MMU translates it.
• CPU physical address — phys_addr_t. What you see in /proc/iomem.
• DMA (bus) address — dma_addr_t. The address the device puts on the bus. An IOMMU or a host bridge may translate it to something completely different from the physical address.

On a simple SoC the DMA address often equals the physical address, which is exactly why a broken driver can work on one board and fail on the next one that has an IOMMU or a bus offset. The job of the DMA API is to give you a valid dma_addr_t for a buffer, set up any IOMMU entry required, and perform whatever cache maintenance the platform needs. You get all of that only if you use it correctly.

Before any mapping, tell the core what your device can address:

#include <linux/dma-mapping.h>

if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32))) {
        dev_warn(dev, "no suitable DMA available\n");
        return -ENODEV;
}

The default assumption is 32-bit DMA addressing. If your device drives more address bits, or fewer, say so. The documentation notes that dma_set_mask_and_coherent() does not fail for masks larger than 32 bits, so the common pattern of “try 64, fall back to 32” is not the recommended form; select the mask from what the hardware can actually do.

What memory you are allowed to DMA to

This rule is the source of a lot of hard bugs. Memory from the page allocator or from kmalloc() / kmem_cache_alloc() is DMA-able. Memory from vmalloc() is not. Kernel image addresses, module image addresses, stack addresses and the return of kmap() are not DMA-able either.

A DMA buffer on the stack is the classic case: it may appear to work on a coherent x86 test machine and then corrupt neighbouring stack data on an ARM board with non-coherent caches. Allocate the buffer with kmalloc(), or embed it in a device structure with the cache-line isolation the kernel provides.

Coherent mappings

A coherent mapping is memory that the CPU and the device can both access with no explicit software flushing. Allocate it once, usually at probe:

dma_addr_t dma_handle;
void *cpu_addr;

cpu_addr = dma_alloc_coherent(dev, size, &dma_handle, GFP_KERNEL);
if (!cpu_addr)
        return -ENOMEM;

/* ... program dma_handle into the device ... */

dma_free_coherent(dev, size, cpu_addr, dma_handle);

You get back two things: a virtual address for the CPU and a dma_addr_t for the device. The kernel documentation lists the right uses for coherent memory: network card DMA ring descriptors, adapter mailbox structures, device firmware executed out of main memory. In other words, small structures that both sides access repeatedly, where a cache flush on every access would be unreasonable.

Two points people get wrong here:

• Coherent does not mean ordered. The CPU can still reorder stores to coherent memory. If the device must see word0 before word1 is marked valid, you need a barrier: desc->word0 = address; wmb(); desc->word1 = DESC_VALID;
• Coherent memory can be expensive. On a non-coherent ARM platform the kernel typically maps it as uncached. Reading a large buffer through an uncached mapping is slow. Do not use dma_alloc_coherent() for bulk payload buffers just to avoid thinking about cache maintenance.

If you need many small coherent chunks with an alignment or boundary constraint, use the DMA pool API — dma_pool_create(), dma_pool_alloc(), dma_pool_free(), dma_pool_destroy() — instead of subdividing pages yourself. dma_pool_alloc() and dma_pool_free() may be called from interrupt context, and dma_alloc_coherent() may be called in interrupt context with GFP_ATOMIC; dma_free_coherent() and dma_pool_destroy() may not.

Streaming mappings and the ownership rule

A streaming mapping takes a buffer that already exists — a network packet, a filesystem block — and makes it visible to the device for one transfer:

dma_addr_t dma_handle;

dma_handle = dma_map_single(dev, addr, size, DMA_TO_DEVICE);
if (dma_mapping_error(dev, dma_handle))
        goto map_error;

/* ... start the transfer, wait for completion ... */

dma_unmap_single(dev, dma_handle, size, DMA_TO_DEVICE);

In a real driver the unmap usually happens in the completion interrupt rather than in a wait loop, which is one reason the streaming map and unmap calls are safe to call from interrupt context. If that path is unfamiliar, The Linux Generic IRQ Subsystem: From Device Tree to Handler covers how the handler gets there.

Three things are mandatory and are routinely skipped:

1. Check dma_mapping_error(). Mapping can fail — no IOMMU space, no bounce buffer. Using the returned address without checking can produce anything from a panic to silent data corruption.
2. Give the exact direction. DMA_TO_DEVICE, DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. The direction is what tells the platform whether to clean the cache, invalidate it, or both. DMA_BIDIRECTIONAL always works but can cost more.
3. Unmap with the same size and the same function. Something mapped with dma_map_single() must be released with dma_unmap_single(), not dma_unmap_page(). The DMA address space is a shared resource; leaking mappings can eventually make the machine unusable.

For scatter-gather, dma_map_sg() returns the number of entries it actually mapped, which can be fewer than nents because the implementation is allowed to merge consecutive entries. Iterate with for_each_sg() over the returned count and read sg_dma_address() and sg_dma_len(). But when you unmap or sync, pass the original nents, not the returned count. Getting this backwards is a common bug.

The ownership rule is the heart of streaming mappings. Between dma_map_*() and dma_unmap_*(), the buffer belongs to the device. The CPU must not touch it. If you do need to look at the data in between, borrow the buffer back explicitly:

dma_sync_single_for_cpu(dev, dma_handle, size, DMA_FROM_DEVICE);
/* CPU may now read the buffer */

dma_sync_single_for_device(dev, dma_handle, size, DMA_FROM_DEVICE);
/* device owns it again */

On a non-coherent platform, dma_sync_single_for_cpu() with DMA_FROM_DEVICE invalidates the cache lines covering the buffer, so the CPU sees what the device wrote rather than a stale cached copy. dma_sync_single_for_device() with DMA_TO_DEVICE cleans the cache, so the device sees what the CPU wrote rather than data still sitting dirty in cache. Skip the sync and you get exactly the intermittent, load-dependent corruption that costs a week to find.

Cache-line sharing: the corruption you cannot see in the code

On a CPU with non-coherent caches, cache maintenance operates on whole cache lines, not on individual bytes. If a DMA_FROM_DEVICE buffer shares a cache line with a field the CPU writes, one of the two writes can be lost: the CPU writes its field, the device writes the buffer, and the invalidate or the writeback destroys the other. This is why the kernel requires architectures to set ARCH_DMA_MINALIGN so that a kmalloc()‘d buffer does not share a cache line with other data, and why current kernels provide the __dma_from_device_group_begin() / __dma_from_device_group_end() markers to isolate a DMA buffer embedded inside a driver structure:

struct my_device {
        spinlock_t lock1;
        __dma_from_device_group_begin();
        char dma_buffer1[16];
        char dma_buffer2[16];
        __dma_from_device_group_end();
        spinlock_t lock2;
};

Both macros take an optional GROUP identifier, which you use when one structure holds more than one such group: __dma_from_device_group_begin(buffer1) … __dma_from_device_group_end(buffer1), with the same name at both ends. The unnamed form above is what the documentation shows for a single group.

On cache-coherent platforms these macros expand to zero-length array markers. On non-coherent platforms they enforce the minimum DMA alignment, which the documentation notes can be as large as 128 bytes. If you need a safe alignment value at runtime, dma_get_cache_alignment() takes no arguments and returns the processor cache alignment: the minimum alignment and width to observe when mapping memory or doing a partial flush. It may report more than the real cache line, but always a power of two that whole cache lines fit into.

Who decides that your device is coherent

Nothing in your driver decides this. On a device-tree platform, the dma-coherent property in the device node does. The DT core reads it in of_dma_is_coherent() (drivers/of/address.c), walking up the tree, and the result is passed into of_dma_configure(), which installs coherent or non-coherent DMA operations for that device. So:

• If the SoC integration really is I/O-coherent for that master and the node has dma-coherent, the sync calls become no-ops and cost nothing.
• If the property is missing on a master that is in fact coherent, you pay for cache maintenance you do not need. That is safe for correctness but slower than necessary.
• If the property is present on a master that is not coherent, the kernel skips cache maintenance and you get data corruption that no amount of reading the driver will explain. When a new board corrupts DMA data and the same driver is known good elsewhere, check the device tree before you touch the driver.

Note also where that device tree comes from. As Device Tree Ownership Is Moving Up Into Firmware describes, on a growing number of platforms the node your kernel sees is assembled or amended by firmware at boot, so the dma-coherent property may not be in the DTS file you have in front of you. Read it back from the running system with /proc/device-tree rather than trusting the source tree.

The important consequence for driver authors: always write the sync calls, even if your current board is coherent. On a coherent platform they cost nothing; on the next board they are the difference between working and corrupting. If you have a performance-critical fast path, dma_need_sync() returns true only when dma_sync_single_for_{device,cpu} are actually required for a given DMA address.

Debugging with CONFIG_DMA_API_DEBUG

The kernel can check your DMA API usage at runtime. Enable “Enable debugging of DMA-API usage” (CONFIG_DMA_API_DEBUG, defined in kernel/dma/Kconfig). It has a real performance cost, so it is a development-kernel option, not a production one.

With it enabled, the core keeps a record of every mapping and complains when the rules are broken. For example, releasing a mapping with the wrong function produces a warning like this:

forcedeth 0000:00:08.0: DMA-API: device driver frees DMA memory with wrong
        function [device address=0x00000000640444be] [size=66 bytes] [mapped as
single] [unmapped as page]

By default only the first error is printed, to avoid flooding the log. Everything else is controlled from debugfs:

raghu@techveda.org:~$ sudo mount -t debugfs none /sys/kernel/debug
raghu@techveda.org:~$ ls /sys/kernel/debug/dma-api/
all_errors  disabled  driver_filter  dump  error_count  min_free_entries
nr_total_entries  num_errors  num_free_entries

raghu@techveda.org:~$ cat /sys/kernel/debug/dma-api/error_count
0
raghu@techveda.org:~$ echo 1 | sudo tee /sys/kernel/debug/dma-api/all_errors
raghu@techveda.org:~$ echo my_driver | sudo tee /sys/kernel/debug/dma-api/driver_filter

all_errors set to a non-zero value prints a warning for every error found instead of only the first, and num_errors is the count of warnings still to be printed before the code goes quiet. driver_filter limits the output to one driver; write an empty string to it to see all errors again. dump lists the current DMA mappings, which is how you find a mapping leak: watch num_free_entries fall while your driver runs, and read min_free_entries for the lowest it has ever reached. The same filter can be set at boot with dma_debug_driver=<drivername>, and the whole facility can be turned off at boot with dma_debug=off — note that it cannot be switched back on at runtime, so that choice costs you a reboot. The debug code preallocates 65536 entries; if that is not enough it says so in the kernel log, and dma_debug_entries=<n> raises the number.

Two checks are worth knowing about specifically. The debug code records whether you called dma_mapping_error() on an address, and warns at unmap time, with a call trace, if you never did. It also tracks device-writable mappings by cache line, so two DMA_FROM_DEVICE or DMA_BIDIRECTIONAL buffers that fall in the same cache line report cacheline tracking EEXIST, overlapping mappings aren't supported — which is the cache-line sharing problem described above, caught at map time instead of as corrupted data.

When DMA corruption does reach the point of an oops rather than a quiet wrong result, the same discipline applies to reading the trace: Reading a Kernel Oops, Part 2 covers decoding it back to the source line.

A short checklist for review

• DMA mask set before the first mapping, and it matches the hardware.
• Buffers come from kmalloc() or the page allocator, never from vmalloc() and never from the stack.
• Every dma_map_*() has a matching dma_unmap_*() of the same kind and size, including on every error path.
• dma_mapping_error() checked on every dma_map_single() and dma_map_page() result.
• Direction is the true direction, not DMA_BIDIRECTIONAL out of convenience.
• No CPU access to a mapped streaming buffer without a dma_sync_*_for_cpu() and dma_sync_*_for_device() pair around it.
• dma_unmap_sg() and dma_sync_sg_*() get the original nents, not the count returned by dma_map_sg().
• Descriptors and control structures in coherent memory; bulk payload in streaming mappings.

Driver-side DMA is one of the topics we spend real bench time on in our Linux device drivers training, because reading the API is not the same as having debugged a cache-coherency problem on a live board.

Key takeaways

• The Linux DMA API translates between CPU addresses and device (bus) addresses, and performs the cache maintenance a non-coherent platform needs.
• Use coherent mappings for small, long-lived shared structures such as descriptor rings; use streaming mappings for payload buffers moved once.
• Between map and unmap, a streaming buffer belongs to the device. Use dma_sync_single_for_cpu() and dma_sync_single_for_device() if the CPU must touch it in between.
• Coherency is a platform property expressed by the dma-coherent device-tree property, not something the driver chooses. Write the sync calls regardless.
• CONFIG_DMA_API_DEBUG plus /sys/kernel/debug/dma-api/ catches wrong-function unmaps, unchecked mapping errors and mapping leaks before a customer does.

Frequently asked questions

When should I use dma_alloc_coherent() instead of dma_map_single()?
Use dma_alloc_coherent() for small structures that the CPU and device both access repeatedly over the life of the driver, such as descriptor rings and mailboxes. Use dma_map_single() for an existing buffer that is handed to the device for one transfer, such as a network packet or a block of file data.

Can I DMA to a buffer allocated with vmalloc() or declared on the stack?
No. The kernel documentation states that memory from vmalloc(), kernel or module image addresses, and stack addresses may not be used for DMA. Use kmalloc() or the page allocator, both of which return DMA-able memory.

Why does my driver work on one board and corrupt data on another?
The most common reasons are that the second board is not I/O-coherent while the first one was and the driver omits the dma_sync_* calls, or that the device node carries a dma-coherent property that does not match the hardware. Check the device tree and the sync calls before rewriting driver logic.

How do I find a DMA mapping leak?
Build a development kernel with CONFIG_DMA_API_DEBUG, then watch /sys/kernel/debug/dma-api/num_free_entries while the driver runs and read /sys/kernel/debug/dma-api/dump to list the mappings currently outstanding. Set driver_filter to your driver name to remove noise from the rest of the system.

Further reading

• Dynamic DMA mapping Guide — kernel.org, Documentation/core-api/dma-api-howto.rst
• Dynamic DMA mapping using the generic device — kernel.org, the API reference and the DMA API debugging section
• DMA attributes — kernel.org, DMA_ATTR_SKIP_CPU_SYNC, DMA_ATTR_FORCE_CONTIGUOUS and others
• DMA and swiotlb — kernel.org, how bounce buffering works when a device cannot reach the memory
• drivers/of/address.c — Bootlin Elixir (v6.16), where of_dma_is_coherent() reads the dma-coherent property