Run the ordinary and boundary clocks using linuxptp
The
linuxptp package is a PTPv2 implementation according to the IEEE 1588 standard for Linux.
It includes the certain programs (
ptp4l,
phc2sys,
phc_ctl,
timemaster) and utility tool (
pmc).
ptp4l is an implementation of the PTP boundary clock (BC) and ordinary clock (OC).
If the hardware timestamping is supported it is used to synchronize the PTP hardware clock (PHC) on the network interface card (NIC) to a master clock.
Otherwise, with the software timestamping it synchronizes the system time to the master clock.
Moreover,
ptp4l is implemented for the user space and assisted by the Linux kernel with its features:
- socket option SO_TIMESTAMPING for packet timestamping
- PHC subsystem that allows to access the PHC devices via system calls
- some drivers support hardware and/or software timestamping
phc2sys is used to synchronize two or more clocks.
With the hardware timestamping it is typically used to synchronizes the system clock with PHC on NIC, which itself is synchronized by the
ptp4l program.
phc_ctl can be used directly control a PHC device. Typically, it is used for debugging purpose.
In general,
phc2sys should be used to control the PHC devices.
timemaster is used to run NTP with PTP as reference clocks.
It uses
ptp4l and
phc2sys in combination of
chronyd or
ntpd to synchronize the system clock to NTP and PTP time sources.
The PTP time is provided by
phc2sys and
ptp4l to
chronyd/ntpd, which can compare all time sources and use the best sources to synchronize the system clock.
The
pmc tool implements a PTP management client and is used to send PTP management messages (get/set/command) to
ptp4l in run-time.
This document describes how to build the simple OC and BC using
linuxptp.
The components that were used for demonstration:
- Linux host with linuxptp
- kernel v5.15.0
- linuxptp v3.1.1
- EB tools (eb-info, eb-console) and the wishbone driver
- (optional) saftlib tools v2.5.1
- network interface card (NIC) with the PTP hardware clock (PHC):
- BlueLAN converged network adapter with dual interface (recognized as enp2s0f0, enp2s0f1), link
- White Rabbit (WR) switch
- model WRS-3/18 from Creotech
- software version: v6.0.1
- configuration version: default and v1.5.2
- 802.1x in WRS does not care of the PTP frames => PTP will always function
- SFPs: AXCEN AXGE-3454-0531 (1490nm), AXGE-1254-0531 (1310nm)
- fiber optic cables (LC-LC, simplex)
- (optional) timing receivers (TR)
- Pexaria5 installed in the Linux host (dev/wbm0) and/or SCU3
- gateware version: fallout-v6.1.2
- (optional) digital oscilloscope
- (optional) LEMO cables
1. Prerequisites
1.1. Confirm no time synchronization on a local system
In order to avoid mess, the time synchronization of a local system must be disabled.
On Debian-based distros (Ubuntu, Linux Mint) the network time synchronization is handled by
timesyncd (instead of
ntpd).
The status of
timesyncd can be queried by
timedatectl:
$ timedatectl
Local time: Thu 2022-11-03 11:27:46 CET
Universal time: Thu 2022-11-03 10:27:46 UTC
RTC time: Thu 2022-11-03 10:27:46
Time zone: Europe/Berlin (CET, +0100)
System clock synchronized: yes # system time is successfully synchronized
NTP service: active # timesyncd is up and running
RTC in local TZ: no
With
timedatectl the NTP service can be turned off/on.
$ sudo timedatectl set-ntp false # turn off
$ sudo timedatectl set-ntp true # turn on
1.2. BlueLAN CNA I350-F2
It is a dual port, converged network adapter (CNA).
In general, the CNA adapters combines two functions of storage area network (SAN) and general-purpose computer networking into a single hardware:
- host-bus adapter (HBA)
- network interface controller (NIC)
These adapters support the Fiber-Channel over Ethernet (
FCoE) and Gigabit Ethernet (
GbE) technologies and enable an access to SAN and LAN via single interface.
The chosen CNA provides two network interfaces for Gigabit SFP trancievers:
$ ifconfig
...
enp2s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.38.124 netmask 255.255.0.0 broadcast 169.254.255.255
ether 80:61:5f:08:b3:2a txqueuelen 1000 (Ethernet)
RX packets 53382 bytes 3947163 (3.9 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 51124 bytes 8750019 (8.7 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x72300000-723fffff
enp2s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.131.24 netmask 255.255.0.0 broadcast 169.254.255.255
ether 80:61:5f:08:b3:2b txqueuelen 1000 (Ethernet)
RX packets 57199 bytes 13727442 (13.7 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 84617 bytes 10944351 (10.9 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x72200000-722fffff
...
Moreover, each network interface is capable of hardware timestamping:
$ ethtool -T enp2s0f0 # repeat it also for 'enp2s0f1'
Time stamping parameters for enp2s0f0:
Capabilities:
hardware-transmit
software-transmit
hardware-receive
software-receive
software-system-clock
hardware-raw-clock
PTP Hardware Clock: 0 # its PHC is /dev/ptp0
Hardware Transmit Timestamp Modes:
off
on
Hardware Receive Filter Modes:
none
all
1.3. Usage of ptp4l
The typical usage of
ptp4l with hardware timestamping is to synchronize a PHC device on NIC to a PTP master.
At least following options/arguments are required:
- -i: specify a network interface to send/receive PTP packets
- -s: slave clock mode (default is the master mode)
- -E: end-to-end delay measurement mechanism (default)
- -2: raw Ethernet (802.3) as transport protocol (default is UDP/IPv4)
- hardware timestamping (default)
A command given below synchronizes the PHC clock (/dev/ptp0) of the first network interface (ie., enp2s0f0) to a PTP master (ie., WR switch):
$ sudo ptp4l -i enp2s0f0 -E -2 -s -m --step_threshold=0.00002
ptp4l[8875.899]: selected /dev/ptp0 as PTP clock
ptp4l[8875.943]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[8875.943]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[8876.992]: port 1: new foreign master 70b3d5.fffe.91e37e-2
ptp4l[8881.348]: selected best master clock 70b3d5.fffe.91e37e
ptp4l[8881.348]: foreign master not using PTP timescale
ptp4l[8881.348]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[8884.494]: master offset -4 s0 freq +31255 path delay 454
ptp4l[8885.585]: master offset -39 s2 freq +31223 path delay 461
ptp4l[8885.585]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[8886.617]: master offset -36 s2 freq +31187 path delay 461
ptp4l[8887.723]: master offset 27 s2 freq +31239 path delay 457
ptp4l[8888.783]: master offset 18 s2 freq +31238 path delay 458
ptp4l[8889.627]: master offset 22 s2 freq +31248 path delay 456
ptp4l[8890.725]: master offset 33 s2 freq +31265 path delay 456
ptp4l[8891.531]: master offset 15 s2 freq +31257 path delay 458
ptp4l[8892.483]: master offset 1 s2 freq +31248 path delay 458
ptp4l[8893.649]: master offset -17 s2 freq +31230 path delay 459
ptp4l[8894.816]: master offset -4 s2 freq +31238 path delay 458
The '--step_threshold=n' option is used to adjust the time faster. If the time jump is greater than 'n' seconds, time is adjusted by stepping the clock instead of changing the frequency.
Use '--summary_interval
' option to get summary statistics, instead of log output every clock update.
The interval is a power of two in seconds (ie., 5 specifies 32-second interval)
Status of the PTP clock synchronization in the log output includes:
- offset between master and slave clocks, nanoseconds
- clock servo state, s0=unlocked, s1=clock step, s2=locked
- frequency adjustment of clock, parts per billion
- path delay, nanoseconds
The state change "UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED" indicates successful synchronization with the PTP master clock.
1.4. Usage of phc2sys
Its typical usage is to adjust the system clock based on PHC of a slave clock (network interface that synchronizes to a PTP master).
This way a local system can operate as an OC.
As it is used to synchronize two or more clocks, it is also used to synchronize the PHC devices of other master clocks in a BC.
Required options/arguments are:
- -s: specify a source clock by a network interface (or PHC device), it is a slave clock in a BC
- -c: specify other clock by a network interface (or PHC device), it is a master clock in a BC
- -w: wait until ptp4l instance is synchronized
A command given below synchronizes the PHC devices on each network interface of BlueLAN CNA.
Here, the PHC device (/dev/ptp1) on the second network interface (enp2s0f1) is synchronized to the PHC device on the first network interface (enp2s0f0).
$ sudo phc2sys -s enp2s0f0 -c enp2s0f1 -w -m -S 0.00002
phc2sys[8987.646]: enp2s0f1 phc offset 381755 s0 freq +29187 delay 233433
phc2sys[8988.648]: enp2s0f1 phc offset 382261 s1 freq +29691 delay 236393
phc2sys[8989.650]: enp2s0f1 phc offset 2710 s2 freq +32401 delay 235513
phc2sys[8990.652]: enp2s0f1 phc offset 1161 s2 freq +31665 delay 236201
phc2sys[8991.654]: enp2s0f1 phc offset -92 s2 freq +30761 delay 234713
phc2sys[8992.656]: enp2s0f1 phc offset -1591 s2 freq +29234 delay 235289
phc2sys[8993.658]: enp2s0f1 phc offset -2838 s2 freq +27510 delay 222794
The '-S n' option can be used to set the step threshold (in seconds) of the servo.
Use '-u ' option to print the summary statistics instead of individual update sample. Requires the number of clock updates.
1.5. Usage of phc_ctl
Get the (PTP) time of PHC devices.
Both commands given below read the time of PHC on the first network interface of BlueLAN CNA.
$ sudo phc_ctl enp2s0f0 -- get; sudo phc_ctl /dev/ptp0 -- get
phc_ctl[6414.864]: clock time is 1631095590.304911375 or Wed Sep 8 12:06:30 2021
phc_ctl[6414.877]: clock time is 1631095590.317937114 or Wed Sep 8 12:06:30 2021
1.6. Usage of pmc
This tool can be used to get the current status/configuration of a slave clock.
$ sudo pmc -u -b 0 'GET CURRENT_DATA_SET'
sending: GET CURRENT_DATA_SET
80615f.fffe.08b32a-0 seq 0 RESPONSE MANAGEMENT CURRENT_DATA_SET
stepsRemoved 1
offsetFromMaster -15.0
meanPathDelay 476.0
1.7. Other
A few useful commands (not relevant to linuxptp) are given below:
$ lspci | grep -i i350 # check if a BlueLAN CNA is detected by the host system
02:00.0 Ethernet controller: Intel Corporation I350 Gigabit Fiber Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation I350 Gigabit Fiber Network Connection (rev 01)
$ ifconfig # list available network interfaces
enp2s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.38.124 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::8261:5fff:fe08:b32a prefixlen 64 scopeid 0x20<link>
ether 80:61:5f:08:b3:2a txqueuelen 1000 (Ethernet)
RX packets 19031 bytes 1406692 (1.4 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 27500 bytes 5641200 (5.6 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x72300000-723fffff
enp2s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.131.24 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::8261:5fff:fe08:b32b prefixlen 64 scopeid 0x20<link>
ether 80:61:5f:08:b3:2b txqueuelen 1000 (Ethernet)
RX packets 14091 bytes 2684066 (2.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 39824 bytes 6479204 (6.4 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x72200000-722fffff
$ ls /sys/class/ptp/ # list available PHCs
ptp0 ptp1 ptp2
$ cat /sys/class/ptp/ptp1/clock_name # get the name of a PHC device
80615f08b32a # returns a value same as MAC address of the associated network interface
2. Run an OC and BC with linuxptp
Here, a host system will operate as an OC and BC by using linuxptp.
Figure 2.1 depicts the structure of the demo OC and BC clocks.
In each case, a WR switch will be used as a BC. It will be labeled as a parent BC to distinguish it from a BC running on the host system.
By default, its wri1 interface is in the slave mode, all other WR interfaces are in the master mode. Itself it can run in the free-running (FR) mode.
2.1. Running an ordinary clock (OC)
Any network interface of the BlueLAN CNA can be used: they have to be connected to the master mode ports of the WR switch.
In order to run an OC a single instance of ptp4l has to be started with one of the network interfaces in the slave mode.
Of course, another instance of phc2sys is required to synchronize the host system time to PHC on chosen network interface.
$ sudo ptp4l -i enp2s0f0 -E -2 -s --step_threshold=0.00002 -m
(optional) $ sudo phc2sys -s enp2s0f0 -w -S 0.00002 # synchronize the local system time
Look at hier for an example.
2.2. Run a boundary clock (BC)
BC provides multiple clocks: typically it is a switch with one slave clock at one network interface and master clocks at other network interfaces.
Therefore, both network interfaces of the BlueLAN CNA are used as follows:
- PTP slave clock will use the first network interface
- PTP master clock will use the second network interface
BC will be created in two ways:
- manual: each PTP clock is created by a separate instance of ptp4l
- automatic: PTP clocks are defined in an external configuration file for ptp4l
For all demo BCs following options are used:
- hardware timestamping
- L2 (802.3) transport
- end-to-end delay mechanism
- clock stepping threshold is 20 us (step_threshold=0.00002)
- poll timeout for TX timestamp is 30 ms (tx_timestamp_timeout=30)
Run a BC manually
In order to run a BC in the host system, at least two instances of ptp4l must run.
It is achieved by running ptp4l with each network interface of the BlueLAN CNA and synchronize both instances with phc2sys:
- launch ptp4l as a slave clock with the first network interface (enp2s0f0), which is connected to the master mode port of the WR switch (parent BC)
- start ptp4l as a master clock with the second network interface (enp2s0f1)
- start phc2sys to synchronize these master and slave clocks in our demo BC
All instances of ptp4l and phc2sys can be launched in given order:
- start phc2sys to synchronize the master and slave clocks (source reference is PHC on enp2s0f0)
- start ptp4l as a slave clock with the enp2s0f0 interface
- start ptp4l as a master clock with the enp2s0f1 interface
Figure 2.2 depicts the instances of ptp4l and phc2sys running in the demo BC.
As an example, all necessary commands for running a BC manually are given below (invoke them on separate terminal):
$ sudo phc2sys -s enp2s0f0 -c enp2s0f1 -w -m -S 0.00002
phc2sys[9220.219]: Waiting for ptp4l...
...
phc2sys[9281.288]: enp2s0f1 phc offset 231222 s0 freq +30400 delay 235417
phc2sys[9282.290]: enp2s0f1 phc offset 230411 s1 freq +29591 delay 234185
phc2sys[9283.292]: enp2s0f1 phc offset 3199 s2 freq +32790 delay 233993
phc2sys[9284.294]: enp2s0f1 phc offset 57 s2 freq +30608 delay 235545
phc2sys[9285.295]: enp2s0f1 phc offset 188 s2 freq +30756 delay 236120
phc2sys[9286.297]: enp2s0f1 phc offset 591 s2 freq +31215 delay 234921
phc2sys[9287.299]: enp2s0f1 phc offset 2155 s2 freq +32957 delay 234697
...
$ sudo ptp4l -i enp2s0f0 -E -2 -s -m --step_threshold=0.00002 --tx_timestamp_timeout=30 # start the slave clock
ptp4l[9236.618]: selected /dev/ptp0 as PTP clock
ptp4l[9236.667]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[9236.667]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[9238.144]: port 1: new foreign master 70b3d5.fffe.91e37e-2
ptp4l[9242.886]: selected best master clock 70b3d5.fffe.91e37e
ptp4l[9242.886]: foreign master not using PTP timescale
ptp4l[9242.886]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[9245.227]: master offset -12347 s0 freq +31288 path delay 478
ptp4l[9246.419]: master offset -12427 s2 freq +31221 path delay 478
ptp4l[9246.419]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[9247.586]: master offset -12437 s2 freq +18784 path delay 478
ptp4l[9248.446]: master offset -1130 s2 freq +26360 path delay -138
ptp4l[9249.398]: master offset 4106 s2 freq +31257 path delay -750
...
ptp4l[9257.812]: master offset -36 s2 freq +31212 path delay 479
ptp4l[9258.938]: master offset -58 s2 freq +31180 path delay 479
ptp4l[9260.004]: master offset -49 s2 freq +31171 path delay 479
ptp4l[9261.113]: master offset -32 s2 freq +31174 path delay 479
...
$ sudo ptp4l -i enp2s0f1 -E -2 -m --step_threshold=0.00002 --tx_timestamp_timeout=30 # start the master clock
[sudo] password for eochirs:
ptp4l[9272.405]: selected /dev/ptp1 as PTP clock
ptp4l[9272.447]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[9272.447]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[9279.644]: port 1: LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
ptp4l[9279.644]: selected local clock 80615f.fffe.08b32b as best master
ptp4l[9279.644]: port 1: assuming the grand master role
Run a BC automatically
In this setup ptp4l uses an external configuration file, which includes network interfaces, their clock roles and all other necessary options (like as 'boundary_clock_jbod').
Configuration files are available for both versions of ptp4l: v3.1.1 and v1.9.2
It also requires that phc2sys must be run with the '-a' option, so that clock roles can be determined automatically from the active ptp4l process.
Important ptp4l options that are considered for BC:
- boundary_clock_jbod=1 (must be enabled)
- clock_type=BC (only for v3.1.1)
- masterOnly=1 for enp2s0f1 (only for v3.1.1)
$ sudo phc2sys -a -m -S 0.00002
phc2sys[9631.954]: Waiting for ptp4l...
...
phc2sys[9660.985]: reconfiguring after port state change
phc2sys[9660.985]: selecting enp2s0f1 for synchronization
phc2sys[9660.985]: selecting enp2s0f0 as the master clock
phc2sys[9660.986]: enp2s0f1 phc offset -161497 s0 freq +33611 delay 234824
phc2sys[9661.988]: enp2s0f1 phc offset -162427 s1 freq +32682 delay 236008
phc2sys[9662.990]: enp2s0f1 phc offset -4930 s2 freq +27752 delay 237064
phc2sys[9663.992]: enp2s0f1 phc offset -1292 s2 freq +29911 delay 234666
phc2sys[9664.994]: enp2s0f1 phc offset 1121 s2 freq +31937 delay 235433
phc2sys[9665.996]: enp2s0f1 phc offset -1923 s2 freq +29229 delay 236937
...
$ sudo ptp4l -f bluelan_bc_v311.conf -m # start the BC (summary_interval=32 seconds)
ptp4l[9659.654]: selected /dev/ptp0 as PTP clock
ptp4l[9659.654]: port 2: just a bunch of devices
ptp4l[9659.731]: port 1: INITIALIZING to SLAVE on INIT_COMPLETE
ptp4l[9659.787]: port 2: INITIALIZING to MASTER on INIT_COMPLETE
ptp4l[9659.787]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[9660.360]: port 1: new foreign master 70b3d5.fffe.91e37e-2
ptp4l[9664.053]: selected best master clock 70b3d5.fffe.91e37e
ptp4l[9664.053]: foreign master not using PTP timescale
ptp4l[9664.053]: port 2: master state recommended in slave only mode
ptp4l[9664.053]: port 2: defaultDS.priority1 probably misconfigured
ptp4l[9680.933]: rms 454 max 1036 freq +31060 +/- 259 delay 462 +/- 5
ptp4l[9696.434]: rms 13 max 25 freq +31128 +/- 13 delay 454 +/- 1
ptp4l[9711.554]: rms 13 max 25 freq +31127 +/- 18 delay 454 +/- 1
ptp4l[9728.002]: rms 10 max 18 freq +31090 +/- 8 delay 454 +/- 1
...
2.3. Check the PTP synchronization
The checks are relevant to the demo boundary clock, which is run in Section 2.2.
2.3.1. Check the (PTP) time
PTP uses the International Atomic Time (TAI) as its timescale.
With the phc_ctl tool the time of the PHC devices can be directly verified.
An example below shows that the PHC clocks of the demo BC are not synchronized.
$ sudo phc_ctl /dev/ptp0 -- get # PTP time of the slave clock in the demo BC
phc_ctl[14405.321]: clock time is 1587506971.943044312 or Wed Apr 22 00:09:31 2020
$ sudo phc_ctl /dev/ptp1 -- get # PTP time of the master clock in the demo BC
phc_ctl[14408.640]: clock time is 1631103584.010256074 or Wed Sep 8 14:19:44 2021
Above failure can be easily fixed by running an instance of phc2sys that synchronizes both clocks.
All three commands below are equal:
$ sudo phc2sys -s /dev/ptp0 -c /dev/ptp1 -w -m # specify the PHC devices
$ sudo phc2sys -s enp2s0f0 -c enp2s0f1 -w -m # or specify the network interfaces
$ sudo phc2sys -s /dev/ptp0 -c enp2s0f1 -w -m # or mix them
If a TR is used as an OC (connected to the master clock of the demo BC), then PTP time on a parent BC (WR switch) and OC (TR) can be examined.
Both of them (WR switch and TR) provide CLI tools to monitor the WR PTP synchronization, which show the TAI time.
On a host system with the target TR (dev/wbm0), use eb-console:
$ eb-console dev/wbm0 # start console for the target TR
Connected to uart at address 60500 # press 'Enter' to get the prompt
wrc# gui # invoke the 'gui' command to display the WR PTP core monitor
TAI Time: Tue, Apr 21, 2020, 22:21:34
Link status:
wru1: Link up (RX: 41407, TX: 35765) IPv4: BOOTP running
Mode: WR Off
PTP status: slave
Synchronization status:
Clock offset: -66285 ns
One-way delay averaged: 0 ns
Observed drift: 255 ns
On the WR switch, launch wr_mon (demo BC is connected to wri6 port):
nwt0019m66#wr_mon # invoke the 'wr_mon' command to display the WR synchronization monitor
WR Switch Sync Monitor WP3a-wrpc_fixes-24-gf313343b [q = quit]
WR time (TAI) : 2020-04-21 22:21:32.005181 Leap seconds: 37
Switch time (UTC): 2020-04-21 22:20:55.005221 TAI-UTC : +36.999960
TimingMode: FR PLL locking state: LOCKED
Do not wonder if there is a small difference between TAI time.
Each tool uses the different interface and layout to show the synchronization status, therefore, the status information are displayed with different delays.
Adjust the WR (TAI) time on the WR switch and check that the WR time of the target TR is changed too.
nwt0019m66#wr_date get # get the actual WR (TAI) time
1587507794.898556304 TAI
2020-04-21 22:23:14.898556304 TAI
2020-04-21 22:22:37.898556304 UTC
nwt0019m66#date -s 14:00 # change the system time
Tue Apr 21 14:00:00 UTC 2020
nwt0019m66#wr_date set host # adjust the WR time to the system time
nwt0019m66#wr_date get
1587477663.526469712 TAI
2020-04-21 14:01:03.526469712 TAI
2020-04-21 14:00:26.526469712 UTC
If time of the target TR will not be adjusted directly, then use '--step_threshold=' option with the ptp4l instance for the slave clock.
Also consider the '-S ' option for the phc2sys instance.
This option will adjust the time by stepping the clock instead of changing the frequency.
Use clock stepping carefully, specially for the real-time systems.
All timers that are set to expire between new time and the current time will fire once the time is adjusted.
2.3.2. Synchronization accuracy
2.3.2.1. Log output analysis
Example log messages of ptp4l and phc2sys are shown in the Listing 2.1.
It shows the summary statistics in the 32-second interval.
The statistics include the root-mean-square (rms) and maximum absolute of the offset between the master and slave clocks (nanoseconds), the frequency offset mean and its standard deviation (parts/billion), and path delay mean and its standard deviation (nanoseconds).
Listing 2.1. Log output of ptp4l and phc2sys (summary statistics are taken in 32-second interval)
$ sudo ptp4l -i enp2s0f0 -E -2 -s -m --step_threshold=0.00002 --summary_interval 5
ptp4l[13102.436]: selected /dev/ptp0 as PTP clock
ptp4l[13102.491]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[13102.491]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[13103.004]: port 1: new foreign master 70b3d5.fffe.91e37e-2
ptp4l[13106.594]: selected best master clock 70b3d5.fffe.91e37e
ptp4l[13106.594]: foreign master not using PTP timescale
ptp4l[13106.594]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[13108.845]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[13139.667]: rms 1066 max 3368 freq +30921 +/- 631 delay 466 +/- 9
ptp4l[13170.775]: rms 9 max 20 freq +31028 +/- 11 delay 455 +/- 1
ptp4l[13202.494]: rms 12 max 31 freq +31021 +/- 17 delay 456 +/- 0
ptp4l[13234.972]: rms 8 max 26 freq +31007 +/- 10 delay 455 +/- 1
ptp4l[13267.485]: rms 12 max 28 freq +30984 +/- 13 delay 457 +/- 1
ptp4l[13299.381]: rms 9 max 24 freq +31004 +/- 10 delay 456 +/- 1
...
$ sudo phc2sys -s enp2s0f0 -c enp2s0f1 -w -m -S 0.00002 -u 32
phc2sys[13108.895]: Waiting for ptp4l
...
phc2sys[13140.957]: enp2s0f1 rms 14245 max 57241 freq +31022 +/- 2303 delay 235010 +/- 2644
phc2sys[13173.019]: enp2s0f1 rms 3887 max 10767 freq +31070 +/- 3459 delay 233107 +/- 4320
phc2sys[13205.081]: enp2s0f1 rms 2862 max 7610 freq +30968 +/- 2491 delay 234898 +/- 3350
phc2sys[13237.143]: enp2s0f1 rms 3135 max 11065 freq +31279 +/- 2832 delay 233998 +/- 3734
phc2sys[13269.205]: enp2s0f1 rms 2915 max 11156 freq +30714 +/- 2440 delay 234247 +/- 3370
phc2sys[13301.267]: enp2s0f1 rms 2095 max 7465 freq +30959 +/- 1819 delay 234849 +/- 1877
phc2sys[13333.328]: enp2s0f1 rms 1714 max 5573 freq +31076 +/- 1475 delay 235021 +/- 1159
phc2sys[13365.390]: enp2s0f1 rms 3178 max 8554 freq +30973 +/- 2757 delay 233854 +/- 2956
...
2.3.2.2. (Optional) Measure the delay between the pulse-per-second (PPS) signals
Since WR switch and the target TR can generate the PPS signal, measuring the delay between the generated PPS signals with an oscilloscope would be sufficient for the accuracy analysis.
Short version: the result of measurement of the delay between PPS signals generated by the parent BC (WR switch) and OC (TR) is shown in Table 2.1 and Table 2.2.
Long version:
Use saftlib tools to generate PPS signal on the target TR device.
First, check saftd and make sure that the target TR is attached to the saftd.
$ saft-ctl bla -j # display attached TR devices
$ sudo saftd tr0:dev/wbm0 # start saftd with the target TR device, if saftd is not started at all
$ sudo saft-ctl attach tr0:dev/wbm0 # or attach the local TR device (dev/wbm0), if it is not attached
Assume the target TR device is attached to saftd as tr0 and ready for usage.
Then, use saft-pps-gen to start the PPS generation on the target TR device:
$ saft-pps-gen tr0 -s # starts the PPS generation
ECA configuration done for 11 IO(s)!
Next pulse at: 0x16a2d5dc041be800 -> 2021-09-08 11:39:16.000000000
Next pulse at: 0x16a2d5dc3fb6b200 -> 2021-09-08 11:39:17.000000000
Next pulse at: 0x16a2d5dc7b517c00 -> 2021-09-08 11:39:18.000000000
...
The PPS signal is output at all available output pins.
If PPS signal generation fails and the saftlib tool returns "Timing Receiver is NOT locked!", then try it again after rebooting the target TR (fpgareset) with eb-reset.
For the WR switch, the PPS signal is output at the SMC connector labeled as 'PPS OUT'.
Make sure that the PPS generation is enabled in the switch configuration:
# wrs_menuconfig
PPS generation ---> Force PPS for tersting purpose # this option must be checked
Table 2.1. Delays between PPS signals generated by WRS and SCU3
| statistics |
manual BC |
auto BC |
| mean |
46,14 us |
-0,42 us |
20,31 us |
12,81 us |
-0,25 us |
6,55 us |
| min |
-71,17 us |
-18,14 us |
-4,48 us |
-3,54 us |
-8,68 us |
-5,79 us |
| max |
144,66 us |
9,72 us |
67,06 us |
70,85 us |
9,16 us |
26,08 us |
| sdev |
43,72 us |
5,25 us |
17,94 us |
14,33 us |
3,13 us |
6,66 us |
| counts |
~1K |
~1K |
~1,1K |
~1,12K |
~1K |
~1K |
Table 2.2. Delays between PPS signals generated by WRS and Pexaria5
| statistics |
manual BC |
auto BC |
| mean |
11,18 us |
0,68 us |
6,17 us |
-5 us |
-0,07 us |
-5,73 us |
| min |
-18,28 us |
-2,85 us |
-23,24 us |
-21,85 us |
-4,85 us |
-22,04 us |
| max |
28,83 us |
3,46 us |
0,9 us |
1,48 us |
2,25 us |
1,58 us |
| sdev |
8,78 us |
1,1 us |
5,3 us |
5,14 us |
1,01 us |
5,05 us |
| counts |
~1K |
~1K |
~1K |
~1K |
~1K |
~1K |
Table 2.3. Delays between PPS signals generated by WRS and SCU3 (directly synchronized to WRS)
| statistics |
optical cable |
UTP Cat6 cable |
| mean |
-86,05 ns |
- |
| min |
-88,97 ns |
- |
| max |
-84,29 ns |
- |
| sdev |
1,352 ns |
- |
| counts |
~1K |
- |
A screenshot of measurement is available in this file as an example: last measurement in Table 2.2 (signal on channel 1 (C1, olive) is PPS from WRS, signal on channel 2 (C2, pink) is PPS from TR)
3. External sources
- [1] Configuring PTP using ptp4l, link
- [2] Linux PTP using PHC adjust phase quick start guide, link
- [3] PTP on Linux ~ Introduction to linuxptp, link
- [4] Synchronizing time with Linuxptp, link
-- EnkhboldOchirsuren - 11 Nov 2022 
conf 
png 
pdf 
Aikido
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.