Edge AIDid It Run on the NPU? Tracing One Inference on an Embedded Linux Board
One inference on an embedded Linux board, traced through its file mapping, memory, threads, device node and driver on i.MX 8M Plus…
On-device and edge AI for embedded Linux engineers: accelerators and NPUs, model runtimes and quantization, federated learning, and the systems engineering behind deploying AI at the edge.
Edge AIOne inference on an embedded Linux board, traced through its file mapping, memory, threads, device node and driver on i.MX 8M Plus…
Edge AIAn on-device LLM slows down when the kernel reclaims its weights and KV cache. What eviction costs, how restoration works, and how…
Edge AIThe accel subsystem gives every NPU a /dev/accel node and all of DRM's plumbing, but its six in-tree drivers share no ioctls.…
Edge AIGPU isolation can partition SMs correctly and still miss deadlines on an edge board, because the power mode caps the GPU clock…
Edge AITensorRT, LiteRT, ONNX Runtime and OpenVINO on real embedded silicon: which ones reach the NPU on an i.MX, an AM62A, an RK3588…
Edge AIConcurrent inference on an edge board collapses once processes exceed half the CPU cores. The kernel-side reason, and how to measure it…
Edge AIFile-backed weights let an edge device run a model from the page cache instead of a second private copy. What it saves,…
Edge AIChoosing an Edge AI SoC on TOPS alone goes wrong. What the NPU, memory, power and software stack really decide, with figures…
Edge AITo the kernel, an edge model is two mappings: a reclaimable file-backed one and a tensor arena that is anonymous and cannot…
Edge AIThe mainline NPU driver for RK3588 is real, in drivers/accel since Linux 6.18. What it does, what it does not, and where…
Edge AIOperator fallback pushes unsupported ops back to the CPU and fragments the graph. What it costs on real devices, and how to…
Edge AISpeculative decoding speeds up on-device LLM inference by amortizing the memory-bandwidth cost of decoding. Here is how it works and where it…
Edge AISelf-learning edge AI comes in three technical forms. How each one works, and where each stands today.
Edge AIEdge AI in automotive and industrial systems runs beside a deterministic, safety-critical control loop. How real-time Linux, TSN and mixed criticality help.
Edge AITinyML vs Edge AI on Linux is not a TOPS comparison. It is a choice of machine. A decision guide, the measured…
One email every two weeks — the best of our kernel & embedded writing, plus a hand-picked link or two. No spam.
// 1,200+ embedded engineers already read it
We email occasionally and never share your address.