Time is of The Essence
Author: Derek Smiley, Sr Network Engineer, Synergy Information Solutions
Time synchronization among network devices is important to consider when building out infrastructure, and not just within the datacenter; client endpoints, kiosk terminals, wireless controllers and associated access points, slot machine virtual bank controllers and their electronic gaming machines (EGMs, or slot machines), etc... All of these devices should be synced to the same time source, and done so in a secure method.
Network Time Protocol (NTP) is used to synchronize all participating devices to within a few milliseconds of Coordinated Universal Time (UTC); i.e. local time zones or daylight-saving time is not transmitted. Each participating device can be configured to apply its own TZ/DST settings for local log viewing if desired, but the goal is to have logging time stamps correlate on all devices.
Take the following example...
The Datacenter Server(s) and Domain Endpoint(s) both have their time synced to the domain controllers. The Switch and IP Phone both have their time synced to an internal NTP source other than the domain controllers. The Router never had NTP, or its clock settings, configured appropriately.
Imagine trying to analyze the root cause, or anything really, based on the disparate time stamps in the log files for the devices in this example. The router's time stamps are completely irrelevant and while the domain devices and other network devices with NTP configured may be in the same ballpark, that just isn't sufficient accuracy for analysis or reporting purposes.
Now imagine the same scenario with properly configured and synchronized clocks...
Since each device is syncing with the same time source, correlating information from their logs would be much easier in this scenario.
Synergy has worked closely with many of our customers to implement GPS time servers in their environments as stratum 0 time sources; i.e. reference clocks. We then configure redundant stratum 1 sources; i.e. primary time servers... which is generally the core switches as all network traffic typically can route to/through the cores. All other network infrastructure, or client endpoints, are configured to sync with the primary time servers. Time synchronization for domain-joined endpoints such as user workstations, jackpot kiosks, and other devices on the gaming floor is accomplished by syncing the domain controllers to the primary time servers.
Given the impact that improper time synchronization can have, NTP packets should be authenticated where possible. With NTP authentication, each device synchronizes to a time source only if that source carries one of the authentication keys configured on the client; i.e. devices will drop any packets that fail the authentication check, which prevents them from updating the local clock. This is paramount to preventing MITM or relay-delay attacks where a bad actor could compromise traffic.
Depending on the size of the environment and how many NTP clients exist, there is typically a small (almost negligible) amount of overhead for authenticated NTP packets. We've deployed NTP authentication without issue in environments with over one-thousand network infrastructure devices; L2/L3 switches, data center infrastructure, wireless, etc. This has all proven useful for security and reporting, sometimes combined, on several occasions where our clients have needed to collect information from devices living in different layers of the network for event reconnaissance.