In a distributed computing settings, one should make sure to synchronize to clocks to the same reference and the gold standard is White Rabbit . However, in certain situations, White Rabbit is not easily available to the experiments, e.g. due to receivers not being easily available. In such cases, we prepared NTP and PTP service to synchronize your experiment clocks. NTP is very easy to configure, PTP is a lot more precise. The achievable offset uncertainties are Δt ~ 20 µs and 200 ns, respectively. Their different performance is inherent to the protocols.

We are operating a GPS locked oven controlled crystal oscillator (OCXO) as reference clock providing time services for cryring.lan.

NTP

The Network Time Protocol (NTP) is very widespread and by default, the timing precision is sufficient for many office use cases. If you want to synchronize distributed clocks for experiments at storage rings, you need to prepare your setup with much more care. NTP can be rather precise, if you have all settings under your control and a very simple and deterministic network path between client and server. We operate a pool of NTP servers at cryring.lan and all machines participating in this pool are synchronized with each other to better than 1 µs relative to the reference clock.

DHCP-clients should auto-configure all neccessary settings. If for any reason your system does not autoconfigure, please point your machine manually to the NTP server pool time.cryring.lan. Please contact us for precision tweaks to boost the performance further.

PTP

The Precision Time Protocol (PTP) is defined by IEEE 1588 or 802.1AS and can achieve 100ns precision with proper hardware. Most notably, the network cards used in your computer must be capable of providing hardware timestamping. Cheap NICs usually can't and the system falls back to software timing, sacrificing about a factor of 20 in precision, depending on the load on your machine. So far, we observe that Software-PTP can still be better than NTP even with moderate system load. In cryring.lan, we operate a Grand Master. To synchronize a clock on your Debian/Linux client, please first enable the packages chrony and linuxptp.

Raspbian and Debian come with sensible defaults and only minor configuration tweaks are required to set it up. Please add the following lines to the file /etc/linuxptp/timemaster.conf:

[ptp_domain 0]
interfaces eth0

[timemaster]
ntp_program chronyd

(adjust eth0 to the network device connected to cryring.lan.) Then, just restart timemaster by using the command sudo systemctl restart timemaster. This should now restart all required (sub-)daemons (timemaster, chrony, ptp4l, phc2sys) with the correct parameters.

Verifying it works:

Shortly after restarting timemaster, you should be able to observe in your logfiles (e.g. /var/log/syslog) entries as the following:

Feb 13 12:27:33 yourpc ptp4l: [2754715.996] [0:eth0] port 1: new foreign master b49691.fffe.c09e85-1

You can verify chrony is using PTP as reference clock as shown with the following example:

username@yourpc$ chronyc sourcestats 
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
PTP0                       64   8   251     +0.001      0.011     +1ns   593ns
ntp.somewhere.else         10   6   77m     +0.081      0.064  +4352us  3066us
...

Hardware support and performance tweaks

Some manufacturers build capability for hardware timestamping into some of their network interface card models. Please check the data sheet of your network interface card for IEEE 1588 or 802.1AS features before buying. Hardware-timestamping is essential for achieving the design precision possible for PTP. On existing hardware, you can check for hardware timestamping in the output of the command ethtool -T ...

Please also turn off EEE (Energy Efficient Ethernet and/or Auto Powerdown) on your network devices for improved timing performance.

If your system-CPU is capable of frequency scaling, try to disable this functions (e.g. on Linux, see the documentation for the cpufreq subsystem)

Please let us, know should you experience problems.

-- MichaelLestinsky - 2022-02-13