Free resource
The order to work in before you send a fault upstream, and the check that proves each step. Written for an engineer who knows the code but has never sent a report to a kernel mailing list.
Get the printable PDFRead this first
If the fault has security consequences, stop. It does not go to a public mailing list, and it cannot be recalled once it is sent. Report it privately to the kernel security team at security@kernel.org, in plain text with no attachments, and state the affected kernel version range. Reports without a version range are not processed. Everything below assumes an ordinary, non-security fault.
There is no central bug tracker for the Linux kernel. A report is mail to a named maintainer and a public list, competing with everything else in that inbox. That is why preparation, rather than the difficulty of the bug, usually decides whether anyone replies.
Steps 01-03
The three questions that determine the whole route. Answering them last is the most common way a report reaches the wrong people.
This is first because it is the only step that cannot be corrected later. Once a fault with security consequences is on a public archive, the disclosure has happened.
Upstream developers cannot fix code they did not write. A vendor kernel carries patches that exist nowhere upstream, so a report against it usually cannot be acted on.
cat /proc/version, and check whether it came from a vendor, a distribution or kernel.org.A regression is something that worked on one kernel and works worse or not at all on a newer one built with a similar configuration. Regressions carry more weight and follow a different route, so decide now rather than later.
make olddefconfig from the working configuration, so a configuration difference is not mistaken for a code regression.Steps 04-06
The report has to be against code the maintainers are still working on, or there is nothing for them to change.
Stages 01 to 03 above are complete. Enter your email and we send the full 13-stage checklist, laid out to print.
No mailing list, no spam. By email we send the file once and stop.
The taint flag records that something has already happened which can produce unrelated follow-up errors. A maintainer cannot separate your fault from that noise, so tainted reports are frequently set aside.
cat /proc/sys/kernel/tainted. Zero means clean; any other value is a bitfield of reasons.tools/debugging/kernel-chktaint, shipped by many distributions in a linux-tools or kernel-tools package.CPU:, after the process id and command name.rmmod does not clear it. Only a reboot does. Checking the flag on the old kernel and assuming the new one inherits the result is the usual mistake.The shorter the path to the fault, the more likely someone can run it on their own hardware. This is the step that converts a report into something testable.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →Steps 07-09
A trace full of hexadecimal offsets is work you are asking the maintainer to do, using a build only you have.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →These are the details a maintainer needs before they can even classify the report, and asking for them costs a round trip of several days.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →This step exists because the board is often not yours. A kernel log is a permanent public document once it reaches an archive.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →Steps 10-13
A report has no default destination. This step decides whether anyone reads it.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →Developers skim. The subject, the first sentence and the first paragraph decide whether the rest is read, which is why they are written last.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →Regressions are tracked separately and treated with more urgency, but only if you register them.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →A report is the start of an exchange. Most reports that die after being sent die because the reporter stopped answering, not because the maintainer stopped caring.
4 checks, the proof and the failure modes — in the full checklistEnter your email for the full version →What it leads to
A report on an archived list is permanent and citable. When someone fixes the fault, the kernel process asks them to credit you with a Reported-by: tag and a Closes: tag pointing at the URL of your report, and private trackers are not allowed there. A report filed only in an internal system cannot be referenced by the fix. Over a few years, a set of well-described reports is a public record of problems you found, which is harder to build and harder to fake than a list of technologies.
The debugging that produces a report like this is the same work as debugging a driver that fails during probe. If you want to practise it deliberately, that is what our Linux device drivers training covers.
The complete version, laid out to print and keep. Sent once.
No mailing list, no spam. By email we send the file once and stop.
Questions
Related tools and resources
Read free + printable PDFBoard Bring-Up ChecklistA board that will not boot fails in a predictable order, and days go to debugging the wrong stage. 13 stages with the proof that each one worked.Open →
Read free + printable PDFEmbedded Linux GlossaryBSP, DTB, SBOM - the acronyms that fill docs, forum posts and reviews, decoded. 25 terms with the misunderstanding corrected: a BSP treated as a kernel, an SBOM produced by scanning.Open →
Free guideChoosing Your First SBCWhich board fits what you want to build - learning Linux internals, home automation, vision, robotics. Matched by use case, with the four checks before you buy.Open →
Live once a monthFree monthly masterclassOne free hour, live, every month, with a working kernel engineer. Bring your questions - including the one about how to get into this field. Recordings stay on YouTube.Open →
Free career guideCareer roadmapsWhich skills stay in demand as AI spreads, and the order to learn them - for the embedded Linux and kernel paths.Open →