- The hw.tsc_calibrate_test=1 and hw.lapic_calibrate_test=1 tunables can be specified to test results of the calibration for different delays (from 100 milliseconds to 2 seconds in 100 millisecond steps).
- With this change the TSC and LAPIC calibration each should take only 200 milliseconds, instead of the original 1 second and 2 second delays.
- This change tries to make the TSC calibration more exact, by averaging the TSC values from before and after reading the timer. By sampling the latency of reading the (HPET) timer, we can make sure that the start and end measurements of TSC and the (typically HPET or i8254) timer didn't get interrupted (e.g. by an SMI on hardware, or by the host when running
virtualized), and filter out those outliers.
- Additionally for the TSC calibration the new code does 2 measurements at the start and end of the delay time, separated by 20 milliseconds. This should make results even more consistent.
- The hw.calibrate_tsc_fast=0 tunable can be set, to revert to the old TSC calibration code.
- Use the TSC to calibrate the LAPIC timer, when the TSC is invariant. Although this indirect calibration might accumulate inaccuracies, this still seems better. Since the TSC runs very fast, we can get a very accurate value in 200ms or even less. To forcibly disable the TSC based LAPIC calibration, set the hw.lapic_calibrate_fast=0 loader tunable.
- The fallback (without using the TSC) LAPIC calibration is slightly improved, by measuring the sysclock timestamp at the start and end of the measurement explicitly with sys_cputimer->count(). Also the lapic timer is explicitly read after starting the countdown. It also proves to be useful in at least some virtualization environments (e.g. QEMU with TCG emulation), to do some LAPIC timer access before actually measuring anything.
- The HPET and LAPIC mmio read accesses are no barrier for Intel and AMD cpus. So we explicitly have to avoid out-of-order execution of the rdtsc() call that follows the sys_cputimer->count(), by using rdtsc_ordered() which uses lfence or mfence on Intel and AMD CPUs respectively.
4098a6e56c pc64 - Improve TSC and LAPIC timer calibration code.
sys/platform/pc64/apic/lapic.c | 174 +++++++++++++++++++++++++++++++++++++----
sys/platform/pc64/isa/clock.c | 161 ++++++++++++++++++++++++++++++++++----
2 files changed, 304 insertions(+), 31 deletions(-)