Linux Clocksource and sched_clock: Three Clocks on One Board

Read a timestamp from two places on the same board and you can get two different answers. The number in front of a dmesg line and the value clock_gettime(CLOCK_MONOTONIC) returns do not come from the same code, are not corrected the same way, and do not behave the same across a suspend. Both come from hardware one timer driver registered, and the Linux clocksource layer is only one of the three things it registered.

Three clocks, and one driver that registers all of them

The i.MX general-purpose timer driver is a good place to start because it does everything in one function. This is its whole clocksource setup path at v7.3, with an ARM delay-timer block removed:

/* drivers/clocksource/timer-imx-gpt.c, v7.3 */
static int __init mxc_clocksource_init(struct imx_timer *imxtm)
{
	unsigned int c = clk_get_rate(imxtm->clk_per);
	void __iomem *reg = imxtm->base + imxtm->gpt->reg_tcn;

	sched_clock_reg = reg;

	sched_clock_register(mxc_read_sched_clock, 32, c);
	return clocksource_mmio_init(reg, "mxc_timer1", c, 200, 32,
			clocksource_mmio_readl_up);
}

One hardware register, two separate registrations. sched_clock_register() installs it as the fast clock for printk and tracing; clocksource_mmio_init() installs it as a Linux clocksource for the timekeeper, rated 200, over a 32-bit counter. Further down, the driver registers a third object, a clock_event_device also rated 200, which only delivers an interrupt at a requested time and never answers the question “what time is it”.

On an i.MX6DL taking its timer from osc_per, the driver divides the 24 MHz oscillator by eight: #define V2_TIMER_RATE_OSC_DIV8 3000000, with the comment /* 24 / 8 = 3 MHz */ beside the prescaler write. A 32-bit counter at 3 MHz returns to zero after about 1,431 seconds, and that one number drives most of the behaviour below.

What a Linux clocksource is made of

The structure is deliberately small. These are the fields that matter for reading time, in declaration order at v7.3:

/* include/linux/clocksource.h, v7.3, abridged */
struct clocksource {
	u64			(*read)(struct clocksource *cs);
	u64			mask;
	u32			mult;
	u32			shift;
	u64			max_idle_ns;
	u32			maxadj;
	u64			max_cycles;
	u64			max_raw_delta;
	const char		*name;
	...
	int			rating;
	enum clocksource_ids	id;
	enum vdso_clock_mode	vdso_clock_mode;
	unsigned long		flags;
	...
};

Every Linux clocksource supplies read, mask, name, rating and flags; the rest is computed at registration. Converting ticks to nanoseconds is one multiply and one shift, in clocksource_cyc2ns(): ((u64) cycles * mult) >> shift.

There is no division, and that is the point: this runs on every clock_gettime() call and every timer expiry calculation, and a 64-bit division on a 32-bit Arm core is a library call. The cost is that mult and shift are only a fixed-point approximation, derived in clocksource_freq2mult() as mult = (from << shift) / freq. A larger shift is more precise, but cycles * mult must stay inside 64 bits for every delta the kernel converts, so choosing shift fixes how long an interval must convert correctly. Every safety property below follows from it.

Two safety margins in the Linux clocksource math

__clocksource_update_freq_scale() does the work, and it starts by fixing that interval:

/* kernel/time/clocksource.c, v7.3 */
		sec = cs->mask;
		do_div(sec, freq);
		do_div(sec, scale);
		if (!sec)
			sec = 1;
		else if (sec > 600 && cs->mask > UINT_MAX)
			sec = 600;

		clocks_calc_mult_shift(&cs->mult, &cs->shift, freq,
				       NSEC_PER_SEC / scale, sec * scale);

sec starts as the counter’s wrap time, which for a counter wider than 32 bits can be decades. Holding the arithmetic exact over decades would force a small shift and a coarse mult, so the kernel caps it at ten minutes; the comment above the code describes the result as roughly 0.06 ppm granularity for NTP. The cap applies only when the mask is strictly greater than UINT_MAX, so a 32-bit clocksource never reaches it and keeps its full wrap interval as the conversion range.

Then two margins are applied. The first reserves room for NTP, which steers the clock by adjusting mult at runtime. clocksource_max_adjustment() returns cs->mult * 11 / 100 under the comment “We won’t try to correct for more than 11% adjustments (110,000 ppm)”. If mult + maxadj would not fit in 32 bits, registration halves mult and decrements shift until it does, so a Linux clocksource can end up less precise than its frequency allows purely to leave room for a correction that may never be applied.

The second margin is in clocks_calc_max_nsecs(), whose comment gives both method and reason:

/*
 * NOTE: This function includes a safety margin of 50%, in other words, we
 * return half the number of nanoseconds the hardware counter can technically
 * cover. This is done so that we can potentially detect problems caused by
 * delayed timers or bad hardware, which might result in time intervals that
 * are larger than what the math used can handle without overflows.
 */

The result is max_idle_ns, the longest the tick code may leave this clocksource unread. Running the v7.3 algorithms for two real cases gives the table below. Note that the wide counter’s hardware wraps after roughly 95 years, and the kernel still will not let the tick code sleep more than about seven minutes without reading it.

sched_clock is a different clock, and it has to be polled

sched_clock() is not built on the current Linux clocksource. It is a parallel implementation in kernel/time/sched_clock.c whose only goal is to be cheap, because printk, ftrace and the scheduler call it from contexts where taking a seqlock and walking the timekeeper is unacceptable. It keeps its own conversion data:

/* kernel/time/sched_clock.c, v7.3 */
	do {
		seq = raw_read_seqcount_latch(&cd.seq);
		rd = cd.read_data + (seq & 1);

		cyc = (rd->read_sched_clock() - rd->epoch_cyc) &
		      rd->sched_clock_mask;
		res = rd->epoch_ns + cyc_to_ns(cyc, rd->mult, rd->shift);
	} while (raw_read_seqcount_latch_retry(&cd.seq, seq));

That is the whole read path: a counter read, a masked subtraction from a stored epoch, one multiply-shift and an addition. The two-element read_data array with a latch counter is what makes it safe from an NMI with no lock at all.

The masked subtraction is also the weakness. cyc is correct only if the counter has advanced by less than one full wrap since epoch_cyc was stored, which for the 32-bit counter above is 1,431 seconds. So an hrtimer refreshes the epoch, re-arming with hrtimer_forward_now(hrt, cd.wrap_kt), and cd.wrap_kt comes from clocks_calc_max_nsecs(new_mult, new_shift, 0, new_mask, NULL), inheriting the same 50 percent margin. The maxadj argument is zero, and that is a real difference rather than an oversight: sched_clock is never steered by NTP. It is cheap and monotonic, and it is not the clock for a timestamp that must agree with wall time.

Registration reports what it worked out, with the format "sched_clock: %u bits at %lu%cHz, resolution %lluns, wraps every %lluns". For the two cases above the computed values are 32 bits at 3000kHz, resolution 333ns, wraps every 715827882841ns and 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns — under twelve minutes and about 73 minutes respectively. Rates of 4 MHz and above print in MHz and anything slower prints in kHz, which is why 3 MHz appears as 3000kHz. The function also opens with if (cd.rate > rate) return;, so the faster clock wins and probe order does not matter.

Rating decides which Linux clocksource the timekeeper uses

More than one Linux clocksource can be registered, and a single integer decides. The bands come from the kernel-doc comment on struct clocksource: 1 to 99 unfit for real use, 100 to 199 base level, 200 to 299 good, 300 to 399 desired, 400 to 499 the ideal clocksource. The list is kept sorted by rating, so selection walks it, skips anything without CLOCK_SOURCE_VALID_FOR_HRES in one-shot mode, and returns the first survivor.

This is where the two drivers diverge. The i.MX MMIO clocksource asks for 200. The arm64 architected timer asks for 400:

/* drivers/clocksource/arm_arch_timer.c, v7.3 */
static struct clocksource clocksource_counter = {
	.name	= "arch_sys_counter",
	.id	= CSID_ARM_ARCH_COUNTER,
	.rating	= 400,
	.read	= arch_counter_read,
	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
};

Its mask is not a constant. The driver sets it from the counter width, and that calculation explains why wide counters are narrower than their register:

static int arch_counter_get_width(void)
{
	u64 min_cycles = MIN_ROLLOVER_SECS * arch_timer_rate;

	/* guarantee the returned width is within the valid range */
	return clamp_val(ilog2(min_cycles - 1) + 1, 56, 64);
}

At 24 MHz, 56 bits already covers roughly 95 years, so the computed value falls below the floor and the clamp returns 56. That is the 0xffffffffffffff mask in the table. The same width goes to sched_clock_register(scr, width, arch_timer_rate), so on arm64 both clocks run off one counter at one width.

Two overrides exist, clocksource= on the kernel command line and current_clocksource in sysfs, but neither bypasses the high-resolution requirement: a request lacking CLOCK_SOURCE_VALID_FOR_HRES in one-shot mode is logged as not currently compatible and deferred.

The Linux clocksource watchdog usually does not run on an embedded board

There is a mechanism for catching a clock that drifts. A Linux clocksource marked CLOCK_SOURCE_MUST_VERIFY goes on a watchdog list and is compared twice a second against a reference chosen from those not marked that way, with a tolerance at v7.3 of 500 ppm if both sides claim CLOCK_SOURCE_CALIBRATED and 4000 ppm otherwise. One that fails loses CLOCK_SOURCE_VALID_FOR_HRES, is re-rated to zero and is replaced, after printing Marking clocksource <name> unstable due to frequency skew.

Neither driver read for this article sets CLOCK_SOURCE_MUST_VERIFY. The architected timer sets only CLOCK_SOURCE_IS_CONTINUOUS, plus CLOCK_SOURCE_SUSPEND_NONSTOP when the counter keeps running in suspend, and clocksource_mmio_init() sets only CLOCK_SOURCE_IS_CONTINUOUS for every MMIO clocksource in the tree. On a board whose clocksources are all of that kind the watchdog list is empty, and clocksource_start_watchdog() returns immediately when that list is empty. Nothing is checked, because there was never a second independent counter to check against. If your product depends on the timer holding a stated tolerance, that remains your measurement to make.

The field nobody sets: vdso_clock_mode

One field decides whether user space can read time without entering the kernel. enum vdso_clock_mode begins with VDSO_CLOCKMODE_NONE, so that value is zero, and on arm64 the architecture adds two members, VDSO_CLOCKMODE_ARCHTIMER and VDSO_CLOCKMODE_ARCHTIMER_NOCOMPAT.

The architected timer driver declares static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_ARCHTIMER; and assigns clocksource_counter.vdso_clock_mode = vdso_default; before registering. clocksource_mmio_init() does not. It allocates its wrapper with kzalloc_obj() and fills in five fields, none of them this one, so every MMIO clocksource is registered with vdso_clock_mode left at zero. That value is copied into the page user space reads, on every timekeeping update:

/* kernel/time/vsyscall.c, v7.3, abridged */
	clock_mode = tk->tkr_mono.clock->vdso_clock_mode;
	vc[CS_HRES_COARSE].clock_mode	= clock_mode;
	vc[CS_RAW].clock_mode		= clock_mode;
	...
	/*
	 * If the current clocksource is not VDSO capable, then spare the
	 * update of the high resolution parts.
	 */
	if (clock_mode != VDSO_CLOCKMODE_NONE)
		update_vdso_time_data(vdata, tk);

The user-space side acts on it. The generic vDSO code does if (vc->clock_mode == VDSO_CLOCKMODE_NONE) return false;, and its caller turns that into return clock_gettime_fallback(clock, ts);. The consequence is concrete: on a board whose selected Linux clocksource is an MMIO timer, every clock_gettime(CLOCK_MONOTONIC) in every process is a real system call, and the library call looks identical in your source either way.

To put a number on the difference I compared the vDSO path against the same call forced through syscall(SYS_clock_gettime, ...), two million iterations each at -O2. The vDSO path took about 25 ns per call and the forced system call about 145 ns, roughly six times more, repeatable across runs. That was measured on an x86_64 virtual machine using the tsc clocksource, not on an embedded board, it isolates only the cost of entering the kernel, and it is a reason to measure rather than your board’s number. An application that timestamps a 1 kHz stream three times per record is making three thousand system calls a second it did not intend to make.

What happens across a suspend

The three clocks part company again here. sched_clock is frozen deliberately: sched_clock_suspend() replaces the read function with suspended_sched_clock_read(), which returns the stored epoch_cyc, so printk and ftrace timestamps do not advance over the suspended interval. The clocksource layer does the opposite work, picking a suspend clocksource from those flagged CLOCK_SOURCE_SUSPEND_NONSTOP and passing the measured difference to __timekeeping_inject_sleeptime(), which adds it to the wall clock, subtracts it from wall_to_monotonic and adds it to the boot-time offset. A comment in kernel/time/timekeeping.c states the result: CLOCK_MONOTONIC does not include suspend time while CLOCK_BOOTTIME does.

How to check this on your own board

Three checks, in the order worth running them. First, see what the Linux clocksource layer chose and what else was offered:

raghu@techveda.org:~$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
raghu@techveda.org:~$ cat /sys/devices/system/clocksource/clocksource0/available_clocksource

The second file is filtered. When the tick is in one-shot mode it lists only clocksources carrying CLOCK_SOURCE_VALID_FOR_HRES, so a registered clocksource can be missing from it. Second, read what the kernel computed at registration:

raghu@techveda.org:~$ dmesg | grep -E 'clocksource|sched_clock'
sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns
arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns
clocksource: Switched to clocksource arch_sys_counter

Compare the bit count and rate in the sched_clock line against the datasheet for the timer you believe you are using: a narrower counter or lower rate means a different hardware block was probed. Third, find out whether user space is paying for a system call:

raghu@techveda.org:~$ strace -c -e trace=clock_gettime ./your_program

A vDSO-capable clocksource produces no clock_gettime entries at all, because the call never reaches the kernel. A non-zero count is the vdso_clock_mode field showing through, and on arm64 the next thing to check is whether the architected timer was probed at all.

Driver-level work of this kind, including the timer and interrupt paths a board bring-up depends on, is covered in our Linux Systems Engineering training.

Limits, and what was not verified

Every field, constant, flag and log format quoted here was read from mainline v7.3. The watchdog code has been reworked more than once and I did not diff it against the longterm series, so treat the ppm figures as v7.3 values rather than properties of the kernel you ship. The arithmetic in the table and the two registration lines came from reimplementing the kernel’s own functions and running them, cross-checked against the line real 24 MHz arm64 boards print, not from a board in front of me. The MMIO claim is a claim about clocksource_mmio_init(): a driver that defines its own struct clocksource may set vdso_clock_mode, and I did not survey the tree for ones that do, nor count how many set CLOCK_SOURCE_MUST_VERIFY.

Key takeaways

• A timer driver typically registers three objects: a Linux clocksource for the timekeeper, a sched_clock source for printk and tracing, and a clockevent device for interrupts.
• Conversion is (cycles * mult) >> shift with no division, so shift is chosen against an overflow budget. A mask wider than 32 bits caps the conversion range at 600 seconds; a 32-bit mask never reaches that cap.
• Two margins shrink it further: 11 percent of mult reserved for NTP, and a 50 percent cut in clocks_calc_max_nsecs(), leaving a 56-bit counter at 24 MHz with max_idle_ns of about 441 seconds.
• sched_clock keeps its own mult, shift, mask and epoch, takes no NTP adjustment, and needs an hrtimer to refresh the epoch before the counter wraps, which at 3 MHz over 32 bits is under twelve minutes.
• Rating alone decides selection: 400 for the arm64 architected timer against 200 for a typical MMIO timer, and an override must still satisfy CLOCK_SOURCE_VALID_FOR_HRES.
• clocksource_mmio_init() never sets vdso_clock_mode, so every clock_gettime() there becomes a system call; across a suspend, only CLOCK_BOOTTIME includes the suspended interval.

Frequently asked questions

Why does the kernel use multiply and shift instead of dividing by the counter frequency?
Because the conversion runs on every clock_gettime call and every timer calculation, and a 64-bit division is a library call on a 32-bit Arm core. The cost is that mult and shift are fixed-point approximations, so the kernel must pick a shift that will not overflow for the longest interval it may convert.

How can I tell whether clock_gettime on my board is a real system call?
Run your program under strace with clock_gettime traced. If the clocksource in use is vDSO-capable the call never reaches the kernel and no entries appear. A non-zero count means the selected clocksource has vdso_clock_mode set to VDSO_CLOCKMODE_NONE, which is the default for any clocksource registered through clocksource_mmio_init.

Why do my printk timestamps and CLOCK_MONOTONIC disagree after a resume?
Because sched_clock is frozen deliberately while the system is suspended: its read function is replaced with one that returns the stored epoch value. CLOCK_MONOTONIC also excludes the suspended interval, by a different route, and CLOCK_BOOTTIME is the only one of the three that includes it.

Does the clocksource watchdog protect me from a drifting timer?
Only if a clocksource is marked CLOCK_SOURCE_MUST_VERIFY and there is a second, independent continuous clocksource to compare it against. Neither the arm64 architected timer nor any clocksource registered through clocksource_mmio_init sets that flag, so on many embedded boards the watchdog list is empty and the watchdog timer never starts.

Further reading

• include/linux/clocksource.h at v7.3 — struct clocksource, the rating bands, the flags
• kernel/time/clocksource.c at v7.3 — mult and shift, both margins, selection, the watchdog
• kernel/time/sched_clock.c at v7.3 — the latch read path and the polling hrtimer
• kernel/time/vsyscall.c at v7.3 — how vdso_clock_mode reaches user space
• drivers/clocksource/mmio.c at v7.3 — the fields clocksource_mmio_init does not set
• drivers/clocksource/arm_arch_timer.c at v7.3 — arch_sys_counter and arch_counter_get_width
• drivers/clocksource/timer-imx-gpt.c at v7.3 — one register registered three ways
• Kernel documentation: timekeeping
• kernel.org releases