Skip to main content

TECH VEDA

Corporate on-site training - Submit proposal Pick your modulesSharpen your kernel skills: deep dives, drivers, Yocto, CVEs, careers — updated daily. Read the blog →
Insights

Cached Memory on Embedded Linux: Can the Kernel Reclaim All of It?

Is cached memory free memory on Linux? Test the belief against the v7.3 source: what Cached includes, what MemAvailable leaves out, and how to check a board.

Cached Memory on Embedded Linux: Can the Kernel Reclaim All of It?

The belief that cached memory is free memory is true only under conditions. Clean page cache from a disk or flash filesystem can be reclaimed. The Cached line also includes tmpfs and shared memory, and without swap the kernel cannot free those pages. On an embedded board, read MemAvailable and Shmem, and treat files in tmpfs as RAM that is in use.

Many engineers learn one rule early. Linux uses spare RAM as a cache, so a low “free” value does not matter. Cached memory is returned to applications when they need it. This article tests that belief about cached memory against the kernel source at tag v7.3 and its documentation. It shows where the belief is correct and where it fails. The failures matter most on embedded boards, which may run without swap and may keep logs and runtime files in RAM.

The belief about cached memory and where it comes from

The belief appears in many forms. One common form is “free plus buff/cache is the real free memory”. Another is “cache can be dropped at any time, so ignore it”. A Linux distribution wiki article on this topic describes the buffer and cache memory as memory that can be reclaimed immediately for applications. Forum answers say the same thing in other words. They describe the cache as memory that can be emptied almost instantly.

The strongest form of the belief is this. The kernel puts unused RAM to work as page cache. When an application needs memory, the kernel reclaims page cache and gives the pages to the application. So the memory that is really available is roughly the free memory plus the cache. A low MemFree value on its own is therefore not a problem.

This form is a reasonable statement, and much of it is correct. It also has a history. BusyBox free prints a line called -/+ buffers/cache, which adds the cache to the free memory. It prints that line only when the kernel does not provide MemAvailable, which means kernels older than 3.14. The comment in the BusyBox source describes that line as counting buffer cache as free memory.

The kernel developers themselves said that the old calculation was once acceptable. The commit that added MemAvailable in Linux 3.14 was written by Rik van Riel. It says that adding “free” and “cached” was fine about ten years earlier. It also says that the same calculation was “pretty much guaranteed to be wrong” on the systems of 2014. The rest of this article explains why.

What the source says

Three pieces of the kernel source show what cached memory includes. All three were read at tag v7.3.

What Cached counts. The documentation in Documentation/filesystems/proc.rst describes Cached as the in-memory cache for files read from the disk, “as well as tmpfs & shmem”. The code in fs/proc/meminfo.c agrees. It computes Cached as all file pages, minus the swap cache, minus the buffers. Pages of tmpfs files and shared memory are file pages, so they are included.

Which pages can be reclaimed as cache. The function folio_is_file_lru() in include/linux/mm_inline.h decides which list a page goes on. Its comment states that a tmpfs page goes on the anonymous list, not on the file list. The test is whether the page is “swap backed”. The tmpfs code in mm/shmem.c marks its pages as swap backed. A page on the anonymous list cannot be dropped. The kernel can only free it by writing it to swap.

What MemAvailable counts. The function si_mem_available() in mm/show_mem.c calculates MemAvailable. It starts with the free pages and subtracts the reserved pages. It then adds the pages on the active and inactive file lists. It does not add all of them. It keeps back half of these pages, or the low watermark, whichever is smaller. It treats reclaimable slab memory in the same way. Pages on the anonymous list are not added at all, so tmpfs and shared memory are excluded.

Key insight: Cached and MemAvailable do not count the same pages. Cached includes tmpfs and shared memory, and MemAvailable leaves them out.

Where the belief about cached memory is correct

The belief is correct for clean page cache. These are pages that contain data read from a filesystem on storage, such as ext4 on eMMC or a read-only squashfs image. The data in these pages also exists on the storage device. The kernel can drop such a page and use it for something else. If the data is needed again, the kernel reads it again from storage.

On a board where most of the cached memory is this kind of page, the belief is correct. A low MemFree value after a long run is normal. It shows that the kernel has used the spare RAM to keep file data. It does not show a memory leak. An engineer who restarts a service because MemFree is low may be looking at this normal behaviour.

The belief is also correct for most reclaimable slab memory, such as the dentry and inode caches. This memory is not part of Cached. It appears as SReclaimable, and MemAvailable includes part of it.

Where cached memory cannot be reclaimed

The belief fails for several kinds of pages that appear in Cached or seem to be cache. Each of them is common on embedded systems.

Files in tmpfs and shared memory

The documentation in Documentation/filesystems/tmpfs.rst states that tmpfs keeps its files in the kernel’s internal caches. Its pages can be moved to swap when memory is short, if swap is enabled. Some embedded products are built without swap. On such a product, a tmpfs page stays in RAM until the file is deleted or the filesystem is unmounted. The kernel cannot reclaim it, and the out-of-memory (OOM) killer cannot free it either. Killing a process does not delete the files that the process wrote.

The default OpenEmbedded Core fstab mounts /run and /var/volatile as tmpfs. Anything written under these directories uses RAM that the kernel cannot reclaim. POSIX shared memory under /dev/shm and System V shared memory are also counted in Shmem and Cached.

The tmpfs documentation gives a further warning. The default size of a tmpfs mount is half of the physical RAM. It also states that if tmpfs instances are made too large, the machine will deadlock, because the OOM handler cannot free that memory. Two tmpfs mounts with the default size are each allowed half of the RAM, so together they are allowed all of it.

Key insight: On a board without swap, the Shmem value is RAM in use, even though it is also counted in Cached. Subtract it before you treat any part of Cached as available.

ramfs and an initramfs root filesystem

The document Documentation/filesystems/ramfs-rootfs-initramfs.rst explains that ramfs has no backing store. Its pages are never marked clean, so the kernel cannot free them. The proc.rst documentation lists ramfs pages under Unevictable. The same document states that rootfs is an instance of ramfs, or of tmpfs if tmpfs is enabled. A product that runs entirely from an initramfs keeps its whole root filesystem in RAM. If rootfs is ramfs, the kernel cannot reclaim those pages at all. If rootfs is tmpfs, the kernel can only move them to swap.

Locked pages

A real-time application may call mlock() or mlockall() to avoid page faults. Locked file pages are still file pages, so they are counted in Cached. They are on the unevictable list, not on the file lists, so MemAvailable does not count them. The Mlocked line shows how much memory is locked.

Dirty pages

A dirty page contains data that has not yet been written to storage. The kernel must write it before it can reuse the page. These pages are on the file lists, so MemAvailable counts them. On a board with slow flash storage, reclaiming them takes time, and an application that needs memory quickly may wait. The Dirty and Writeback lines show how much data is waiting.

The page cache that the system needs

Even clean page cache is not all available. The comment in si_mem_available() states that not all of the page cache can be freed, otherwise the system will start swapping or thrashing. The executable code of running programs is also file page cache. If the kernel drops it, the programs read it again from storage soon after. This is why MemAvailable keeps back part of the page cache in its estimate. The documentation adds that the effect of these factors varies from system to system. MemAvailable is an estimate, not an exact measurement.

What to do on your own board

Use MemAvailable as the main measure of spare memory, not free memory plus cached memory. Then check the parts of Cached that the kernel cannot reclaim. The following command prints the relevant lines.

root@raspberrypi5:~# grep -E "^(MemFree|MemAvailable|Cached|Shmem|Unevictable|Mlocked|Dirty|Writeback):" /proc/meminfo

Compare Shmem with Cached. If Shmem is a large part of Cached, much of the cache cannot be reclaimed. If Unevictable is large, check for ramfs, an initramfs root filesystem or locked memory.

Next, find which tmpfs mounts are using the memory.

root@raspberrypi5:~# df -h /run /var/volatile /dev/shm

The “Used” column of each tmpfs mount is RAM in use. On a board without swap, it stays in use until the files are deleted. Set an explicit size= option on every tmpfs mount in fstab, based on what the product needs. Do not rely on the default of half the RAM for each mount. Rotate or limit the logs that are written to tmpfs.

Check what your free command reports. The BusyBox free source computes its “used” column as the total memory minus the free memory and minus the buff/cache value. Its buff/cache value includes Cached, so tmpfs files appear under buff/cache and not under “used”. The procps-ng free computes “used” as the total memory minus MemAvailable, so the two programs can show different “used” values on the same board. In both programs, the “available” column is MemAvailable, and the “shared” column is the shared memory count. Read the “available” and “shared” columns, not “free” plus “buff/cache”.

For a measurement of whether the board is short of memory, use pressure stall information in /proc/pressure/memory. The adjacent TECH VEDA article on how much RAM an embedded Linux product needs explains PSI and per-service peak memory.

Key insight: Give every tmpfs mount an explicit size limit. The default allows each mount half of the RAM, and the kernel cannot reclaim tmpfs pages on a board without swap.

What was checked and what could not be verified

The verdict rests on the kernel source and documentation at tag v7.3. The files read were Documentation/filesystems/proc.rst, Documentation/filesystems/tmpfs.rst, Documentation/filesystems/ramfs-rootfs-initramfs.rst, fs/proc/meminfo.c, mm/show_mem.c, include/linux/mm_inline.h and mm/shmem.c. The release in which MemAvailable first appeared was confirmed by reading fs/proc/meminfo.c at tags v3.13 and v3.14. The behaviour of the free programs was read from the BusyBox and procps-ng sources. The OpenEmbedded Core fstab was read from its master branch.

This article does not give measured numbers from a real board. The share of cached memory that is tmpfs or locked memory depends on the product, so no general figure is given. The time needed to write dirty pages depends on the storage device and was not measured. Products built with other build systems may mount different directories as tmpfs. Check the fstab and the systemd mount units of your own image.

Key takeaways

  • The belief that cached memory is free memory is true only for clean page cache from a filesystem on storage.
  • Cached includes tmpfs and shared memory. Without swap, the kernel cannot reclaim those pages.
  • MemAvailable excludes tmpfs, shared memory and locked pages, and it keeps back part of the page cache. It is an estimate.
  • The default OpenEmbedded Core image mounts /run and /var/volatile as tmpfs, so files there use RAM.
  • Set an explicit size= on every tmpfs mount, and read the “available” and “shared” columns of free.
Was this worth your time?

Frequently asked questions

Is cached memory free memory on Linux?
Only in part. Clean page cache from a filesystem on storage can be reclaimed. The Cached line also includes tmpfs and shared memory, which the kernel cannot reclaim without swap.

Does MemAvailable include tmpfs files?
No. MemAvailable adds only the pages on the file lists. Tmpfs pages are swap backed, so they are on the anonymous list and are not added.

When did MemAvailable appear in /proc/meminfo?
In Linux 3.14. The line is not present in fs/proc/meminfo.c at tag v3.13 and is present at tag v3.14.

Why do BusyBox free and procps-ng free show different “used” values?
BusyBox computes “used” as the total memory minus the free memory and minus buff/cache. Procps-ng computes “used” as the total memory minus MemAvailable. Tmpfs files therefore appear under buff/cache in BusyBox, but they reduce the available memory in both programs.

What is the default size of a tmpfs mount?
Half of the physical RAM, according to the tmpfs documentation. Each mount has its own limit, so several mounts with the default size can together use all of the RAM.

Further reading

RB
Raghu Bharadwaj

Founder, TECH VEDA — 20+ years teaching the Linux kernel, device drivers and embedded systems.

Follow on LinkedIn

Get new posts by email

Kernel, embedded Linux and AI-era engineering — a few sharp reads a month. No spam.

We email occasionally and never share your address.