How-To: White Rabbit to MIL Gateway

Introduction

This is How-To about the White Rabbit to MIL Gateway, wr-mil. The how-to of the 50 Hz synchronization of UNILAC is included as well. This gateways translates timing messages on the White Rabbit network to telegrams on a MIL Eventbus.

Dedicated documentation can be found here.

A list with relevant Frontends/SCUs can be found here.

Monitoring

A simple monitoring app is available via Launcher -> Verschiedenes -> 'WR-MIL Monitor'

Figure: Screenshot of the wr-mil monitor. The status is displayed for various gateways. See text for details.

The figure above shows an example of the wr-mil monitor. The upper eight lines show information about nodes at UNILAC. The upper seven are wr-mil gateways, while the eighth entry is for the 50 Hz synchronization. Lines 9 and 10 give information about the wr-mil gateways at SIS18 and ESR.

Display

wr-mil (except 'pzu_f50')

• # MIL: gateway number
• domain: MIL timing domain
• version: firmware version
• state: state of firmware; expected value is 'OpReady'
• status: status of firmware; expected values is '1'
• #sent/fw: firmware, number of sent MIL telegrams
• #miss(e)d/fw: firmware, number of missing MIL telegrams; difference between number of sent and detected MIL telegrams; expected value is '0'
• #err/fw: firmware, number of MIL telegrams that have been reported as 'broken' by the gateware of the MIL piggy on the SCU; expected value is '0'
• #burst/fw: firmware, number 'burst' telegrams, where the spacing to the beginning of the previous telegram is less than 25 us; expected value is '0'
• #match/x86: host, number of received telegrams, where the payload matches the one of the sent/received MIL telegrams (see mode)
• r [%]: ratio of matched telegrams; due to a bug in the older saftlib, a ratio slightly below 100% is expected
• mode: select what is compared
  • 1: compare received MIL telegrams with WR timing messages
  • 2: compare received MIL telegrams with sent MIL telegrams
  • 3: as mode '2' but excludes UTC telegrams
  • 4: as mode '1' but discarding data from MIL piggy
  • 5: compare sent MIL telegrams with WR timing messages
• ave: average offset [us]
• sdev: standard deviation of offset [us]
• min: minimum offset
• max: maximum offset

pzu_f50
For the line, the following columns have a different meaning

• #sent/fw: firmware, number of phase words sent to the Data Master
• #miss(e)d/fw: firmware, number of missing CMD_UNI_F50DM messages from the Data Master
• #err/fw: firmware, number of cycles with Data Master not being locked
• #burst/fw: n/a
• #match/x86: host, number of matches (received CMD_UNI_F50DM and CMD_UNI_F50TUNE messages)
• r [%]: ratio of matching messages
• mode: select how comparison is done
  • 0: no comparison
  • 2: hard locking (simulation mode for testing)
  • 4: hard locking of Data Master
• ave: average, offset UNILAC cycle start vs 50 Hz mains
• sdev: standard deviation, offset UNILAC cycle start vs 50 Hz mains
• min: minimum, offset UNILAC cycle start vs 50 Hz mains
• max: maximum, offset UNILAC cycle start vs 50 Hz mains

User Control

• '0..9: The statistics information is aggregated by the server program on the frontend. Thus, all monitor applications display the same information everywhere. The statistics can be reset for frontend. By pressing the relevant key 0..9, the information on the server program is reset. With the next execution, the data is reset for all monitor applications.
• h: display help text; basically, the URL of this wiki page is displayed
• q: quits the monitor application

Command Line Interface

Command line interfaces provides additional information.

Diagnostics

wr-mil

[ruth@scuxl4710 ~]# wrmil-ctl dev/wbm0 diag
common: diags ...
firmware boot at                    : 2024-11-11 12:46:15 TAI
diagnostics reset at                : 2024-11-11 12:46:22 TAI
version                             : 000012
mac                                 : 0x00267b0003fe
ip                                  : 192.168.160.235
used shared mem [byte]              : 480
state (# of changes)                : OpReady    (0)
# of transfers                      : 000000000000
# of injections                     : 000000000000
status of act transfer              : 0x0
# late events                       : 000000000000
'offset done' (processing time) [us]:       11.056       // expected processing time
communication latency [us]          :        4.136       // expected value
sum status (# changes)              : 0x1 (0)
overall status                      : OK
wrmil: info  ...

GID                                 : 0x0000000000001c3  // GID for which messages are translatted
UTC trigger evt ID                  : 0x0000000000000ff  // EvtNo that triggers generation of UTC Events
UTC MIL delay [us]                  : 0d000000000000056  // offset between consecutive UTC events
UTC trigger delay [us]              : 0d000000000000650  // 1st UTC is sent after trigger evt and this delay
MIL latency [ns]                    : 0d000000000000500  // latency correction for all MIL events
UTC offset [ms]                     : 0d001199142000000  // milliseconds at 01/01/2008  (since 1970)
request fill event                  : 0d000000000000000  // default: off
MIL dev (0: piggy, 1.. :SIO)        : 0x000000000000001  // here: SIO in 1st slot
MIL data monitoring                 : 0d000000000000002  // here: number, timestamping and decoding of received MIL telegrams
# MIL events sent                   : 0d000000298684662  // number of MIL events sent so far
# MIL events received (TAI)         : 0d000000298684662  // number of received MIL events via ECA TLU (this is unusually perfect)
# MIL events received (data)        : 0d000000298684662  // number of decoded MIL events in MIL piggy
# MIL events received (error)       : 0d000000000000849  // number of received MIL events with bit errrors
# high frequency bursts             : 0d000000000000000  // number of MIL events with an offset smaller than 25us

wr-f50

ruth@scuxl4711:~# wrf50-ctl dev/wbm0 diag
common: diags ...
firmware boot at                    : 2024-11-11 11:21:52 TAI
diagnostics reset at                : 2024-11-11 11:22:51 TAI
version                             : 000012
mac                                 : 0x00267b000769
ip                                  : 192.168.161.187
used shared mem [byte]              : 488
state (# of changes)                : OpReady    (0)
# of transfers                      : 000000000000
# of injections                     : 000000000000
status of act transfer              : 0x0
# late events                       : 000000000000
'offset done' (processing time) [us]:       87.600    // expected value; linear regression takes some time
communication latency [us]          :        3.939    // expected vallue
sum status (# changes)              : 0x1 (0)
overall status                      : OK
wrf50: info  ...

offset to 50 Hz mains [us]          :               0
mode                                :               4
act period of mains [us]            :       20004.480 // measured values of 50 Hz mains
act period of DM [us]               :       20004.344
set period of DM [us]               :       20004.584
act offset DM: DM - mains [us]      :          -0.013 // offset between cycle start and 50 Hz rising edge
min offset DM: DM - mains [us]      :         -66.071
max offset DM: DM - mains [us]      :          90.148
act period change DM: act - prev[us]:          -0.014 // how much the period of the data master changes 
min period change DM: act - prev[us]:         -16.789 // between cycles
max period change DM: act - prev[us]:           9.478
act offset mains: act - predict [us]:          -0.028
min offset mains: act - predict [us]:         -88.845
max offset mains: act - predict [us]:          61.205
DM lock state                       :               3
DM lock date                        : 2024-11-15 13:32:01 TAI // date of last transition unlocked -> locked
DM # locks                          :            1485 // number of transitions unlocked -> locked
DM # cycles                         :        17798778
DM # sent phase words               :        17798757

Parameters, Internal Timing Groups and Event Numbers

Relevant information is given in the command line help.

Timing messages are written to the input of the ECA from

• the timing network
• the lm32 softcore

The lm32 softcore used GIDs and EvtNo that are outside the ranges used by LSA. These numbers are also documented in the command line help.

wr-mil

[ruth@scuxl4711 ~]# wrmil-ctl -h
Usage: wrmil-ctl [OPTION] <etherbone-device> [COMMAND]

  -h                  display this help and exit
  -e                  display version
  -i                  show gateway information

All following parameters are to be used with command 'configure' 
parameters -w and -s are mandatory
  -w <MIL device>     MIL device for sending MIL messages; 0: MIL Piggy; 1..: SIO
  -s <MIL domain>     MIL domain; this can be
                      0: PZU-QR; UNILAC, Source Right
                      1: PZU-QL; UNILAC, Source Left
                      2: PZU-QN; UNILAC, Source High Charge State Injector (HLI)
                      3: PZU-UN; UNILAC, High Charge State Injector (HLI)
                      4: PZU-UH; UNILAC, High Current Injector (HSI)
                      5: PZU-AT; UNILAC, Alvarez Cavities
                      6: PZU-TK; UNILAC, Transfer Line
                      7: PZ-SIS18
                      8: PZ-ESR
  -l <latency corr>   [ns] latency correction for all MIL telegrams, default 0
  -g                  'latency' shall be negative
  -t <trigger>        UTC: evtNo of trigger event [0..255], default 0xf6
                      (UNILAC requires 0xff
  -o <offset>         UTC: offset [s]                     , default yr 2008
  -d <delay>          UTC: delay after trigger event [us] , default 0
  -u <delay>          UTC: delay between UTC events  [us] , default 30
  -m <on>             monitoring of received MIL telegrams, default 0
                      0: no monitoring
                      1: number and timestamping of MIL telegrams    , GID 0xfe1
                      2: includes '1', adds decoding of MIL telegrams, GID 0xff1
                      (deadline of decoded MIL telegrams is t0 + 1ms)
                      monitoring is implemented via local messages to the ECA
                      Tip: MIL telegrams to be sent are available too, GID 0xff0
  -f                  request sending of fill events if no events > 10 seconds

  configure           command requests state change from IDLE or CONFIGURED -> CONFIGURED
                      'configure' requires parameters -w, -s
  startop             command requests state change from CONFIGURED -> OPREADY
  stopop              command requests state change from OPREADY -> STOPPING -> CONFIGURED
  recover             command tries to recover from state ERROR and transit to state IDLE
  idle                command requests state change to IDLE

  diag                shows statistics and detailed information
  cleardiag           command clears FW statistics

Tip: For using negative values with commands such as 'snoop', consider
using the special argument '--' to terminate option scanning.

Use this tool to control the wr-mil gateway from the command line
Example1: 'wrmil-ctl dev/wbm0-s 0 -w 1 -m 1 -l 0  configure'

Report software bugs to <d.beck@gsi.de>
Version 00.00.12. Licensed under the LGPL v3.

wr-f50

[ruth@scuxl4712 ~]# wrf50-ctl -h
Usage: wrf50-ctl [OPTION] <etherbone-device> [COMMAND]

  -h                  display this help and exit
  -e                  display version
  -i                  show 50 Hz sync information

All following parameters are to be used with command 'configure' 
  -o <offset>         offset [us] to zero transition of 50 Hz mains, default 0
                      'offset': t_cycle_mains - t_cycle_DM and must be positive
  -m <mode>           mode selection, default 1 
                      0: off
                      2: hard locking, Data Master simulation mode
                      4: hard locking of Data Master
  configure           command requests state change from IDLE or CONFIGURED -> CONFIGURED
  startop             command requests state change from CONFIGURED -> OPREADY
  stopop              command requests state change from OPREADY -> STOPPING -> CONFIGURED
  recover             command tries to recover from state ERROR and transit to state IDLE
  idle                command requests state change to IDLE

  diag                shows statistics and detailed information
  cleardiag           command clears FW statistics

Use this tool to control the UNILAC 50 Hz synchronization from the command line
Example1: 'wrf50-ctl dev/wbm0 -m 3 configure'

For debugging, some messages are available via the ECA with gid 0xfc0. EvtNo:
    0xa01: rising and falling edge of HW signal from 50 Hz module
    0xfc0: start of Data Master cycle
    0xfc1: message to Data Master; set-value of cycle length param field

Report software bugs to <d.beck@gsi.de>
Version 00.00.12. Licensed under the LGPL v3.

Usage

Just config via NFS-init and monitoring

NFS-Init

Configuration of the frontend is done via nfs-init.

WR-MIL

Example for PRO, UNILAC HSI('pzu_uh'), location LSB6, SL7 ramdisk:

[me@asl789 scuxl4711]$ ll
total 4
lrwxrwxrwx 1 me    bel 49 May 21 14:22 10_timing-rte -> ../global/timing-rte-tg-fallout-v6.2.0-no-graylog     // timing RTE
lrwxrwxrwx 1 me    bel 29 May 22 08:43 20_wrmil_wrmil-tools -> ../global/timing-rte-db-yocto                  // common stuff
lrwxrwxrwx 1 me    bel 29 May 22 08:43 30_wrmil_wrmil-pro-pzuuh-lsb6-config -> ../global/timing-rte-db-yocto  // load and init firmware
lrwxrwxrwx 1 me    bel 29 May 22 08:43 40_wrmil_wrmil-pro-pzuuh-lsb6-systemd -> ../global/timing-rte-db-yocto

Example for INT, note the differences in the names of the symbolic links (click):

[me@asl789 scuxl4712]$ ll
lrwxrwxrwx 1 me    bel 49 Feb 13  2024 10_timing-rte -> ../global/timing-rte-tg-fallout-v6.2.0-no-graylog
lrwxrwxrwx 1 me    bel 29 Apr 18  2024 20_wrmil-dev_wrmil-tools -> ../global/timing-rte-db-yocto
lrwxrwxrwx 1 me    bel 29 Apr 18  2024 30_wrmil-dev_wrmil-int-pzuuh-bg2-config -> ../global/timing-rte-db-yocto
lrwxrwxrwx 1 me    bel 29 Apr 18  2024 40_wrmil-dev_wrmil-int-pzuuh-bg2-systemd -> ../global/timing-rte-db-yocto
                                               ^^^^       ^^^ ^^^^^ ^^^
                                                  |         |     |   |- location
                                                  |         |     |- what
                                                  |         |- environment
                                                  |- path 

WR-F50

Example for PRO, location LSB6, SL7 ramdisk:

[me@asl789 scuxl4712]$ ll
total 4
lrwxrwxrwx 1 me    bel 49 May 21  2024 10_timing-rte -> ../global/timing-rte-tg-fallout-v6.2.0-no-graylog
lrwxrwxrwx 1 me    bel 29 May 27 11:07 20_wrmil_wrmil-tools -> ../global/timing-rte-db-yocto
lrwxrwxrwx 1 me    bel 29 May 27 11:07 30_wrmil_wrmil-pro-f50sync-lsb6-config -> ../global/timing-rte-db-yocto
lrwxrwxrwx 1 me    bel 29 May 27 11:07 40_wrmil_wrmil-pro-f50sync-lsb6-systemd -> ../global/timing-rte-db-yocto
                                                                |- what

Code, Build, Deployment

Code is here: bel_projects/tree/.../modules/wr-mil (includes wr-f50 firmware and software)

For branch and hash of the current version check the logbook

For building and deployment check the comments given at the top of the 'Makefile'

Example for building and deployment for PRO (assuming target system runs Yocto ramdisk)

make clean                                                                  // clean
make make MASP=NO ENV=pro SYSENV=ACC7 PREFIX= all                           // builds software, firmware and generate nfs-init scripts
make PREFIX= SYSENV=ACC7 STAGING=/common/export/timing-rte/wrmil deploy     // copies stuff to location where nfs-init will find it

-- DietrichBeck - 18 Nov 2024