Linux RHEL6.9 constant time drift

[opethdisciple]

Any one have any thoughts on why our servers in our data center keep time drifting?

It's fairly usual for them to drift every month or so by a few mins, which then manifests it's self in us not being able to SSH into the machines...

But yesterday a lot of the servers where suspiciously exactly 10mins out of sync with the NTP server.

We did all the usual checks and all seems to be in order. A manual sync with the NTP server gets everything back in line....

Most of our servers are KVM hosts.

I'm doing a Google around now to see if I can find an answer but I'm no NTP expert.

Maybe someone has some experience?

Thanks

 

[mrkev]

Does ntpd or chronyd ? I'm not sure what service RHEL uses by default.

For ntp: use 'ntpq -p' to see what servers are synchronised
For chrony: you need to check the service status to see if it's working. If using systemctl that will be 'systemctl status chronyd'

Depending on the outcome of the above will give you an indication of where the problem may be.

 

[memyselfandi]

Not seen this with a good few thousand RHEL 6 servers using NTPD (which I would expect on RHEL 6.x rather than Chronyd which is an option on RHEL 7)

In addition to what mrkey asks above (although RHEL6 doesn't use systemctl), what are you sync'ing against (e.g. internal time server sync'ed to an online server, internal time source device, internet time source, and what is its stratum level?
Are you syncing to an odd or even number of time sources?
What options are you using on NTPD if you are using it?