Swap on Embedded Linux: zram, Flash Swap or No Swap?

Swap on embedded Linux is a product decision that is usually made by default instead of on purpose. A development board image ships with swap enabled, a production image ships with it disabled, and nobody records why. This article compares three configurations (no swap, zram, and swap on flash storage) and recommends one for the common case. It also states when each of the other two is the better choice.

The context: why swap on embedded Linux needs a decision

The kernel can only reclaim file-backed pages, such as executables and mapped files, by dropping them and reading them again later. Anonymous memory, such as heap and stack, has no file to return to. Without a swap device, that memory stays resident until the process frees it. In the kernel, paging of anonymous memory is controlled by CONFIG_SWAP, which depends on MMU && BLOCK && !ARCH_NO_SWAP and defaults to y.

Three forces shape the choice on a device. Memory is fixed at design time and cannot be added after shipment. Flash storage (eMMC, SD card, NAND) has limited write endurance, and the product is expected to run for years. The product must also stay responsive, or restart cleanly, when memory runs short; a device that is slow but not dead is often the worst outcome in the field.

The options: three configurations compared

Option 1: no swap

The kernel is built or configured without a swap device. Memory use is bounded by RAM, and the out-of-memory (OOM) killer is the only remaining mechanism. The behaviour is simple to reason about and there are no swap writes to flash.

• For: no added flash writes, no compression CPU cost, predictable latency when memory is sufficient.
• Against: no buffer for short peaks. The systemd-oomd documentation notes that without swap a system can livelock quickly and may stop responding in a reasonable time, because the userspace killer has less time to react.

Option 2: zram, a compressed swap device in RAM

The zram driver (CONFIG_ZRAM) provides /dev/zramX block devices whose contents are compressed and kept in system memory. Used as swap, it lets the kernel move cold anonymous pages into a smaller compressed form instead of failing an allocation. The kernel documentation advises that a zram device larger than twice system memory is wasteful, because a 2:1 compression ratio is the expected figure, and that an unused device costs about 0.1% of its disk size.

• For: no flash wear, much faster than any storage device, and the compressed size is visible in /sys/block/zram0/mm_stat. The mem_limit attribute caps the memory that zram may consume.
• Against: compression uses CPU time, and data that does not compress (already compressed or encrypted data) gains nothing; mm_stat reports such pages in huge_pages. Compressed data still occupies RAM, so zram changes the ceiling but does not remove it.

The compression algorithm is selected through comp_algorithm before disksize is written, and the available choices depend on the CONFIG_ZRAM_BACKEND_* symbols that were enabled. The default compressor is lzo-rle. A related feature, zswap (CONFIG_ZSWAP, which depends on SWAP), compresses pages into a RAM pool in front of a real swap device and writes them to that device when the pool is full. It therefore still needs storage swap and does not remove flash writes.

Option 3: swap on flash storage

A swap partition or file is placed on eMMC, SD card or another storage device. This is the only option that extends memory beyond RAM by a large amount, and it is required for hibernation. It is also the option with the highest risk on small flash parts. The kernel documentation for vm.swappiness describes the value as a rough relative I/O cost of swapping against filesystem paging, so on slow flash the system should be tuned to avoid swapping, not to use it freely. A 2012 kernel mailing list discussion among Linaro storage developers also noted that swap allocates blocks in a pattern that does not match what eMMC devices need for fast operation; the discussion is dated, so measure on your own part.

• For: large capacity, supports memory-hungry but infrequent workloads, supports hibernation.
• Against: consumes write endurance, is slow when it is actually used, and a device that reaches swap under load can become unresponsive before the OOM killer acts.

The decision: swap on embedded Linux should not default to flash

For swap on embedded Linux in the common product (512 MB to a few GB of RAM, eMMC or SD storage, a fixed set of services), the recommended configuration is as follows. Do not place swap on the main flash device. Treat memory as a budget and enforce it per service with cgroup v2: memory.max is the hard limit that invokes the OOM killer inside that cgroup, memory.high throttles a cgroup before that point, and memory.oom.group makes the kernel kill a service as a whole instead of leaving it half-running. Combine this with a restart policy so that a killed service returns to a known state.

Add zram only after measurement. Check how much anonymous memory is cold and how well it compresses on the real workload, then size the device conservatively. A small device can absorb short peaks and give a userspace monitor time to act. If the workload is mostly file-backed or incompressible, zram adds CPU cost without benefit and should stay disabled.

raghu@techveda.org:~$ sudo modprobe zram
raghu@techveda.org:~$ echo lz4 | sudo tee /sys/block/zram0/comp_algorithm
raghu@techveda.org:~$ echo 256M | sudo tee /sys/block/zram0/disksize
raghu@techveda.org:~$ sudo mkswap /dev/zram0
raghu@techveda.org:~$ sudo swapon /dev/zram0
raghu@techveda.org:~$ cat /sys/block/zram0/mm_stat

The first three values in mm_stat are the original data size, the compressed data size and the memory actually used, in bytes. The lz4 choice in the commands above requires CONFIG_ZRAM_BACKEND_LZ4; reading comp_algorithm shows the supported algorithms with the active one in square brackets. Compare the mm_stat values after a representative run before deciding on a permanent size. On systemd-based images, zram-generator can create the device from /etc/systemd/zram-generator.conf; its documented default is zram-size = min(ram / 2, 4096), so set the size explicitly for a product. The OpenEmbedded meta-oe layer also carries a zram recipe under recipes-extended/zram; confirm its installed files and init method on the release branch you build.

Use the other options when these conditions hold:

• Swap on storage is the better choice when the device has high-endurance storage (industrial eMMC with a measured health margin, or an SSD), when occasional large but cold workloads are expected, or when hibernation is a product requirement. Keep vm.swappiness low and monitor wear.
• No swap and no zram is the better choice when the CPU is small and latency is strict, when the working set is mostly file-backed (for example memory-mapped model files), or when the memory budget is already enforced by cgroup limits and testing shows no peaks.

Consequences: what the choice costs later

Detection of memory pressure. With any option, rely on pressure stall information, not on free memory. /proc/pressure/memory reports a some line (some tasks stalled) and a full line (all non-idle tasks stalled), averaged over 10, 60 and 300 seconds. A trigger written to the file can wake a monitor through poll(), with a minimum window of 500 ms. The kernel documentation describes sustained full pressure as thrashing.

Userspace OOM handling. systemd-oomd requires cgroup v2, memory accounting and a kernel with PSI (Linux 4.20 or later). It strongly recommends swap, and its swap-based action is ignored on systems without swap, although pressure-based action still works with more threshold tuning. If the product uses no swap at all, plan to test those thresholds. The kernel itself offers vm.panic_on_oom (value 1 panics on OOM) for products where a reboot is cleaner than a degraded state, in combination with a hardware watchdog.

Flash health. If storage swap is used, read the eMMC health fields during test and in the field. The mmc extcsd read command from mmc-utils reports EXT_CSD_DEVICE_LIFE_TIME_EST_TYP_A and _TYP_B in 10% steps and EXT_CSD_PRE_EOL_INFO as an overall reserved-block status. Record these values in fleet telemetry so that wear becomes visible before failure.

Memory budgeting for edge AI. Model weights that are mapped from a file are reclaimable page cache, not anonymous memory, so they do not benefit from swap and are better handled by sizing; see the related article on file-backed weights on edge devices. Related training is listed on the Embedded Linux with Yocto page.

Key takeaways

• Swap on embedded Linux is a choice about where the memory ceiling is enforced and what happens when it is reached.
• Do not put swap on the main flash device of a product unless storage endurance has been measured and a reason exists.
• zram is optional, costs CPU, and helps only when the cold anonymous memory compresses.
• Enforce per-service limits with cgroup v2 and use PSI to detect pressure.
• Read eMMC life-time fields during testing and in the field if any swap writes reach flash.

Frequently asked questions

Is zram a replacement for more RAM?
No. Compressed pages still occupy RAM, and pages that do not compress gain nothing. The kernel documentation expects about a 2:1 ratio when sizing a device, and mm_stat shows the measured values so the figure can be checked on the real workload.

Can zram write cold pages to flash?
Yes, when CONFIG_ZRAM_WRITEBACK is enabled and a partition is set in backing_dev before disksize is written. When writeback_limit_enable is set, the writeback_limit attribute sets a budget in 4 KB units, so writes to flash can be capped.

Does systemd-oomd require swap?
Its documentation strongly recommends swap and says swap-based actions are ignored without it. Pressure-based actions still work, but pressure rises more abruptly, so thresholds may need more tuning.

Further reading

• zram: Compressed RAM-based block devices (kernel documentation)
• zswap (kernel documentation)
• Documentation for /proc/sys/vm (kernel documentation)
• PSI: Pressure Stall Information (kernel documentation)
• Control Group v2 (kernel documentation)
• systemd-oomd.service(8) manual page
• systemd zram-generator
• NetModule Linux: reading eMMC life time with mmc-utils
• LKML: swap on eMMC and other flash (2012)