Commit b6e61eef authored Jul 31, 2009 by Linus Torvalds Committed by Ingo Molnar Aug 10, 2009

x86: Fix serialization in pit_expect_msb()



Wei Chong Tan reported a fast-PIT-calibration corner-case:

| pit_expect_msb() is vulnerable to SMI disturbance corner case
| in some platforms which causes /proc/cpuinfo to show wrong
| CPU MHz value when quick_pit_calibrate() jumps to success
| section.

I think that the real issue isn't even an SMI - but the fact
that in the very last iteration of the loop, there's no
serializing instruction _after_ the last 'rdtsc'. So even in
the absense of SMI's, we do have a situation where the cycle
counter was read without proper serialization.

The last check should be done outside the outer loop, since
_inside_ the outer loop, we'll be testing that the PIT has
the right MSB value has the right value in the next iteration.

So only the _last_ iteration is special, because that's the one
that will not check the PIT MSB value any more, and because the
final 'get_cycles()' isn't serialized.

In other words:

 - I'd like to move the PIT MSB check to after the last
   iteration, rather than in every iteration

 - I think we should comment on the fact that it's also a
   serializing instruction and so 'fences in' the TSC read.

Here's a suggested replacement.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Reported-by: "Tan, Wei Chong" <wei.chong.tan@intel.com>
Tested-by: "Tan, Wei Chong" <wei.chong.tan@intel.com>
LKML-Reference: <B28277FD4E0F9247A3D55704C440A140D5D683F3@pgsmsx504.gar.corp.intel.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

parent f4b9a988

arch/x86/kernel/tsc.c

+22 −7

Original line number	Diff line number	Diff line
		@@ -275,15 +275,20 @@ static unsigned long pit_calibrate_tsc(u32 latch, unsigned long ms, int loopmin)
		* use the TSC value at the transitions to calculate a pretty
		* good value for the TSC frequencty.
		*/
		static inline int pit_verify_msb(unsigned char val)
		{
		/* Ignore LSB */
		inb(0x42);
		return inb(0x42) == val;
		}

		static inline int pit_expect_msb(unsigned char val, u64 tscp, unsigned long deltap)
		{
		int count;
		u64 tsc = 0;

		for (count = 0; count < 50000; count++) {
		/* Ignore LSB */
		inb(0x42);
		if (inb(0x42) != val)
		if (!pit_verify_msb(val))
		break;
		tsc = get_cycles();
		}
		@@ -336,8 +341,7 @@ static unsigned long quick_pit_calibrate(void)
		* to do that is to just read back the 16-bit counter
		* once from the PIT.
		*/
		inb(0x42);
		inb(0x42);
		pit_verify_msb(0);

		if (pit_expect_msb(0xff, &tsc, &d1)) {
		for (i = 1; i <= MAX_QUICK_PIT_ITERATIONS; i++) {
		@@ -348,7 +352,18 @@ static unsigned long quick_pit_calibrate(void)
		* Iterate until the error is less than 500 ppm
		*/
		delta -= tsc;
		if (d1+d2 < delta >> 11)
		if (d1+d2 >= delta >> 11)
		continue;

		/*
		* Check the PIT one more time to verify that
		* all TSC reads were stable wrt the PIT.
		*
		* This also guarantees serialization of the
		* last cycle read ('d2') in pit_expect_msb.
		*/
		if (!pit_verify_msb(0xfe - i))
		break;
		goto success;
		}
		}

Admin message