In a distributed computing settings, one should make sure to synchronize to clocks to the same reference. NTP and PTP are complementary services to synchronize clocks in distributed computing environments. In situations, where the gold standard of White Rabbit receivers are not available, you can use any of these services to assure the best possible time stamps in your data. You have two options: NTP is very easy to configure, PTP more precise, with achievable errors in time offset determination of Δt ~ 20 µs vs. 200 ns, respectively. Their different performance is inherent to the protocols. The reference clock providing both services is itself derived from a GPS signal and is capable of hardware-timestamping.

NTP

The Network Time Protocolexternal, NTP, is very widespread and the timing precision is sufficient for many applications. The biggest drawback of NTP though is that there are too many clocks out there, with very poor precision and very large offsets to each other. I.e. from inside the GSI campus, we measure already between the Stratum-1 servers at the Cesium fountain clocks in Braunschweig, Paris and at NIST a time different by ~10ms. And moving away from these Stratum-1 servers to the plethora of NTP servers in the global pools, makes things only worse. If every client now even uses a different pooled NTP server to synchronize to, you may get lost, depending on the needs of the experiments.

NTP can be more precise, if you have all settings under your control. Fortunately, we have taken all efforts and operate our own NTP service, to make sure all clients at our network see the same reference with a very low latency and best possible precision and within the error margins all internal time stamps are consistent with our reference clocks.

DHCP-clients should automatically receive all neccessary data for their auto-configuration. If for any reason your system does not autoconfigure, please point your machine manually at the NTP server pool time.cryring.lan.

PTP

The Precision Time Protocolexternal, PTP, is defined by IEEE 1588 or 802.1AS and can achieve 100ns precision with proper hardware. Most notably, the network cards used in your computer must be capable of providing hardware timestamping. Cheap NICs usually can't and the system falls back to software timing, sacrificing about a factor of 20 in precision, depending on the load on your machine. So far, we observe that Software-PTP can still be better than NTP even with moderate system load. In cryring.lan, we operate a Grand Master. To synchronize a clock on your Debian/Linux client, please first enable the packages chrony and linuxptp.

Raspbian and Debian come with sensible defaults and only minor configuration tweaks are required to set it up. Please add the following lines to the file /etc/linuxptp/timemaster.conf:

[ptp_domain 0]
interfaces eth0

[timemaster]
ntp_program chronyd

(adjust eth0 to the network device connected to cryring.lan.) Then, just restart timemaster by using the command sudo systemctl restart timemaster. This should now restart all required (sub-)daemons (timemaster, chrony, ptp4l, phc2sys) with the correct parameters.

Verifying it works:

Shortly after restarting timemaster, you should be able to observe in your logfiles (e.g. /var/log/syslog) entries as the following:
Feb 13 12:27:33 yourpc ptp4l: [2754715.996] [0:eth0] port 1: new foreign master b49691.fffe.c09e85-1

You can verify chrony is using PTP as reference clock as shown with the following example:
username@yourpc$ chronyc sourcestats 
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
PTP0                       14   8    54     +0.001      0.111     +1ns  1293ns
ntp.somewhere.else         10   6   77m     +0.081      0.064  +4352us  3066us
...

Hardware support

Some manufacturers build capability for hardware timestamping into some of their network interface card models. Please check the data sheet of your network interface card for IEEE 1588 or 802.1AS features before buying. Hardwaretimestamping is essential for achieving the design precision possible for PTP. On existing hardware, you can check for hardware timestamping in the output of the command ethtool -T ...

Please let us, know should you experience problems.

-- MichaelLestinsky - 2022-02-13
Topic revision: r3 - 2022-04-11, MichaelLestinsky - This page was cached on 2022-09-26 - 11:38.

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSI Wiki? Send feedback | Legal notice | Privacy Policy (german)