Edge AIEdge AI
6 TOPS on Paper. Which Inference Runtime Reaches It?
TensorRT, LiteRT, ONNX Runtime and OpenVINO on real embedded silicon: which ones reach the NPU on an i.MX, an AM62A, an RK3588…
RB Sep 30 · 13 min
Linux kernel & Device drivers starts on 24th Oct 2026 enrollingCorporate on-site training - Submit proposal Pick your modulesSharpen your kernel skills: deep dives, drivers, Yocto, CVEs, careers — updated daily. Read the blog →Embedded Linux fast track starts 30th sept 2026 enrollingEmbedded Linux Mastery track starts 30th sept 2026 enrollingLinux systems engineering starts 30th sept 2026 enrolling
~/ blog
Edge AI
On-device and edge AI for embedded Linux engineers: accelerators and NPUs, model runtimes and quantization, federated learning, and the systems engineering behind deploying AI at the edge.
Edge AI
Edge AIEdge AI
Concurrent Inference Stops Scaling at Half Your CPU Cores
Concurrent inference on an edge board collapses once processes exceed half the CPU cores. The kernel-side reason, and how to measure it…
RB Sep 21 · 21 min
Edge AIEdge AI
File-Backed Weights: Why Your Edge Device Holds the Model Twice
File-backed weights let an edge device run a model from the page cache instead of a second private copy. What it saves,…
RB Sep 17 · 19 min
Edge AIEdge AI
How to Choose an Edge AI SoC: NPU, GPU, DSP, CPU, Memory, Power and the Software Stack
Choosing an Edge AI SoC on TOPS alone goes wrong. What the NPU, memory, power and software stack really decide, with figures…
RB Sep 2 · 23 min
Edge AIEdge AI
The Half of an AI Model the Kernel Cannot Reclaim
To the kernel, an edge model is two mappings: a reclaimable file-backed one and a tensor arena that is anonymous and cannot…
RB Aug 26 · 24 min
Edge AIEdge AI
The Mainline NPU Driver on RK3588, and What It Cannot Do Yet
The mainline NPU driver for RK3588 is real, in drivers/accel since Linux 6.18. What it does, what it does not, and where…
RB Aug 19 · 7 min
Edge AIEdge AI
Operator Fallback: Why Your Accelerated Model Still Runs on the CPU
Operator fallback pushes unsupported ops back to the CPU and fragments the graph. What it costs on real devices, and how to…
RB Aug 18 · 10 min
Edge AIEdge AI
Speculative Decoding: Faster On-Device LLMs
Speculative decoding speeds up on-device LLM inference by amortizing the memory-bandwidth cost of decoding. Here is how it works and where it…
RB Aug 5 · 8 min
Edge AIEdge AI
Self-Learning Edge AI: Where the Technology Stands
Self-learning edge AI comes in three technical forms. How each one works, and where each stands today.
RB Jul 29 · 9 min
Edge AIEdge AI
Edge AI in Automotive and Industrial Systems
Edge AI in automotive and industrial systems runs beside a deterministic, safety-critical control loop. How real-time Linux, TSN and mixed criticality help.
RB Jul 22 · 9 min
Edge AIEdge AI
TinyML vs Edge AI on Linux: How to Choose the Right Machine
TinyML vs Edge AI on Linux is not a TOPS comparison. It is a choice of machine. A decision guide, the measured…
RB Jul 20 · 14 min
Edge AIEdge AI
4-Bit Weight Quantization: Why the Memory System Decides
Why 4-bit weight quantization is a memory-system problem: DRAM bandwidth, unified memory on Jetson, kernel fusion, and the real accuracy and speed…
RB Jul 15 · 11 min
Edge AIEdge AI
LLM Inference From Flash: Running Models Larger Than Your DRAM
LLM inference from flash: how to run a model larger than your DRAM by streaming weights from storage, explained for embedded engineers.
RB Jul 10 · 10 min
Edge AIEdge AI
Edge AI in the Next 10 Years: The Long Horizon
Where edge AI heads by 2035: AI-native 6G networks, neuromorphic silicon, the EU AI Act, and a ten-year roadmap for embedded Linux…
RB Jul 3 · 10 min
Edge AIEdge AI
Edge AI in the Next 10 Years: The Stack and the Fleet
The embedded Linux stack for edge AI — Yocto, runtimes, secure OTA — and federated learning with Flower to improve models across…
RB Jul 2 · 10 min
The fortnightly digest
One email every two weeks — the best of our kernel & embedded writing, plus a hand-picked link or two. No spam.
// 1,200+ embedded engineers already read it
We email occasionally and never share your address.