For a product that will stay in the field for five years or more, and whose SoC already has upstream support for the peripherals you use, start from a mainline long-term kernel and keep your own patch set small. Choose the vendor BSP kernel when a needed block, such as an NPU, ISP or video codec, has no upstream driver and time to market matters more than later maintenance. The main trade-off is early effort against later effort: mainline costs more before the first release and less in every upgrade after it.
Every embedded Linux product starts from a kernel tree. Either it is the one the silicon vendor ships in its board support package (BSP), or it is a kernel from kernel.org with board support taken from upstream. The vendor BSP vs mainline decision is made early, often by default, and it determines the cost of every security update and kernel upgrade for the life of the product. This article sets out both options, recommends one for the common case, and states when the other is the right choice.
The context
The decision arises when you select a SoC or module and set up the first build. The vendor provides an evaluation board, a BSP with a kernel, a bootloader and a Yocto layer, and the board boots on the first day. Upstream support for the same SoC may be complete, partial or absent, and this is rarely visible from the vendor’s marketing material.
Three forces act on the decision. The first is product life. The kernel project releases a new mainline version every nine to ten weeks, and kernel.org lists a small number of longterm series with a projected end-of-life date for each. A device that ships for ten years will pass several of these. The second is who supplies security fixes. The third is hardware that has no upstream driver at all, which is common for GPUs, NPUs, camera pipelines and video codecs.
The options
Option A: the vendor BSP kernel
The vendor kernel is a fork of a kernel.org release with the vendor’s drivers and board support added. It is validated by the vendor on the vendor’s boards, and it is the only source of support for blocks that have no upstream driver. NXP, for example, describes two Linux paths on its i.MX parts, a mainstream path and an LTS path, and its LTS updates include fixes from both upstream and downstream releases.
- Advantages: fastest route to a working board; full feature coverage, including accelerators; a support contact for problems on the vendor’s hardware.
- Disadvantages: the patch set can be large and was written for the vendor’s schedule. An LWN report on measuring vendor kernels notes that users must apply out-of-tree patches when they migrate to newer releases, and that some patches are little more than quick fixes. Collabora observes that vendors usually track an LTS kernel for security updates only from time to time, and may reduce support for older products as they move to newer ones.
Option B: mainline-first
Here the product runs a kernel.org release, normally a longterm series, with board support that is already upstream or that you submit upstream. In Yocto, the linux-yocto recipes track kernel.org releases, so this path fits the standard build flow. Your own changes are kept as a short, documented series.
- Advantages: security and bug fixes arrive through the stable process without waiting for a vendor. Collabora notes that once support is upstream, the effort to move to newer versions is dramatically reduced because there are few or no out-of-tree patches. Toradex also states that upstream review improves the quality of its code.
- Disadvantages: upstream review takes longer than internal review. Some peripherals will not work, or will work with reduced features. You become responsible for validating the kernel on your own hardware, because no vendor does it for you. Toradex, for instance, reports that all its supported 32-bit i.MX modules ran on the mainline kernel from BSP 6.0.0, while for the i.MX8M Mini and Plus modules it offered mainline only as an experimental alternative to the NXP-based BSP.
The decision: vendor BSP vs mainline
For the common case, choose mainline-first. The common case is a product with a field life of five years or more, a SoC family with in-tree support for the peripherals you need, and a team that can maintain a small patch series. The reasoning is about where cost falls. A vendor kernel has a low initial cost and a recurring cost at every upgrade. A mainline kernel has a higher initial cost and a lower recurring cost. Over several years the recurring cost dominates.
The stable process supports this choice. A patch is accepted into a stable kernel only if the same fix, or an equivalent one, already exists in mainline, and it must be obviously correct, tested and no larger than 100 lines with context. Code that is upstream therefore receives fixes by default. Code in a vendor fork receives them only when someone backports them.
Choose the vendor BSP instead when any of these applies:
- A block you must use, such as an NPU, ISP or hardware codec, has no upstream driver, and the product cannot ship without it.
- The product life is short, or the volume is too small to justify the validation effort.
- The schedule does not allow for the upstream work, and the vendor commits in writing to security updates for the whole product life.
A hybrid is often practical. Use the vendor BSP for the prototype and for early software development, and plan the move to mainline before volume production. Whichever kernel you ship, measure the real gap first. The commands below count the commits your tree carries beyond a mainline tag, and show whether the running kernel is tainted by externally built modules.
raghu@techveda.org:~$ git rev-list --count --no-merges $(git merge-base v6.12 <vendor-branch>)..<vendor-branch>
raghu@techveda.org:~$ cat /proc/sys/kernel/taintedA tainted value with bit 12 set (4096) indicates a module built outside the kernel tree. The post Inherited a Vendor BSP? Measure These Five Things First describes this audit in full. Replace v6.12 with the mainline version your vendor tree is based on.
Consequences
Maintenance. With the vendor kernel you carry the delta between vendor and mainline, and you rebase it yourself or wait for the vendor. With mainline you carry only your own changes. The Hidden Cost of Out-of-Tree Drivers and Private Kernel Patches explains how the second kind of delta grows.
Security. A vendor kernel is only as current as the vendor’s last release for your SoC. A mainline longterm kernel is current as long as kernel.org maintains the series, and you must plan the move to the next series before the end-of-life date shown on kernel.org. The Yocto Project follows a similar pattern: a new LTS release every two years, each supported for four years.
Staffing. Mainline-first requires engineers who can write a device tree binding, send a patch and respond to review. The team must hold this skill, which affects hiring. Vendor-first requires engineers who can debug a large foreign tree.
Boot and memory. Neither option decides the footprint. A trimmed vendor configuration can reach the same size as a mainline one, so treat footprint as a configuration task.
Exit cost. Moving from vendor to mainline later is possible, but it is easier when the device tree and your own drivers followed upstream conventions from the start. The post Architecting Mainline-Friendly Products covers the design rules, and In-Tree vs Out-of-Tree Driver covers where your own drivers should live.
Key takeaways
- The cost of a vendor kernel is paid at every upgrade; the cost of a mainline kernel is paid mostly before the first release.
- Check upstream support for each peripheral you use before choosing a SoC, not after.
- Choose the vendor BSP when a required block has no upstream driver, the product life is short, or the schedule cannot absorb the upstream work.
- Measure the commit delta and the taint flag, whichever kernel you ship.
- Plan the move to the next longterm series before the current one reaches its end-of-life date.
The Yocto and kernel topics in this article are taught in the Embedded Linux with Yocto course at TECH VEDA.
Frequently asked questions
Why does mainline-first lower the cost of kernel upgrades?
Once board support is upstream, there are few or no out-of-tree patches to port to each new kernel, so the work of moving to a newer version is much smaller.
When is the vendor BSP kernel the better choice?
When a block that the product requires, such as an NPU, ISP or hardware codec, has no upstream driver, or when the product life or volume is too small to justify the upstream work.
Can a product start on the vendor kernel and move to mainline later?
Yes. A common approach is to use the vendor BSP for the prototype and early development, then move to a mainline longterm kernel before volume production, provided the device tree and drivers follow upstream conventions.
Does a mainline kernel receive security fixes automatically?
Fixes reach a longterm series through the stable process, which accepts only patches already present in mainline. You must still update the product to the latest release of the series and move to a newer series before the current one reaches end-of-life.
Further reading
- kernel.org: Releases and longterm series
- Rules for Linux -stable kernel patches
- Submitting devicetree (DT) binding patches
- Collabora: What to do about differing product life cycles
- LWN: Analyzing the patchiness of vendor kernels
- Toradex: Upstream-first, mainline kernel support is reality
- NXP: Linux paths for the life of i.MX products
- Yocto Project: Release process and LTS releases
