WR-MIL Gateway

Introduction

The White Rabbit to MIL gateway can work in one GID only. Its main features are.

• listen to so-called legacy event numbers, EvtNo 0..255
• convert received timing messages to MIL telegrams following a defined mapping
• on-time, send the converted MIL telegrams to the MIL Eventbus
• generate so-called UTC telegrams; this is triggered via a special timing message
• deprecated: if required, generate so-called 'fill events'

Presently (2024) nine MIL Event bus domains exist at GSI. The corresponding timing group IDs are

• PZU_QR, UNILAC Timing - Source Right
• PZU_QL, UNILAC Timing - Source Left
• PZU_QN, UNILAC Timing - Source High Charge State Injector (HLI)
• PZU_UN, UNILAC Timing - High Charge State Injector (HLI)
• PZU_UH, UNILAC Timing - High Current Injector (HSI)
• PZU_AT, UNILAC Timing - Alvarez Cavities
• PZU_TK, UNILAC Timing - Transfer Line
• SIS18_RING, SIS18 Ring, includes some timing messages for HEST
• ESR_RING, ESR Ring

More background on MIL telegrams is available here.

A dedicated How-To can be found here.

The MILbus is a single master bus. Here, the master is an SCU. The cable from the SCUs connects to hardware driver modules. Each module drives one MIL strand. A MIL strand is mainly one big cable with a driver at the start and a terminator at the end. Converter boxes (aka 'Blechdose') are inserted into the MIL strand, the whole strand is basically a daisy chain. Each converter box has multiple outputs (differential LEMO) to which clients - TIF, SEs ... - are connected.

A test setup consists of an SCUs, a MILbus cable, a converter box and a MIL slave.

A very simple test setup consists of an SCU, to which a MIL slave is connected via a special cable.

Figure: Connections between SCU and converter box.

Setup

An SCU is used as target platform. To guarantee on-time generation of MIL event telegrams, shared resources (MIL task ram, SCU bus, Wishbone bus, ...) are avoided by using one SCU per MIL event bus (= MIL timing domain).

Figure: Installation of seven WR to MIL gateways at UNILAC / LSB6. Top right: SCU and hardware for UNILAC 50 Hz synchronization.

Figure: SCU with MIL piggy and SIO as White Rabbit to MIL gateway.

The figure above shows a White Rabbit to MIL gateway. The main component is an SCU. A SIO (FG 901.152) serves a the MIL bus master, the MIL cable has a D-Sub 9 pol plug and a red mark. For monitoring, the so-called MIL piggy on the left of the SCU front panel is connected to the MIL bus via a grey cable with 2-pole LEMO plug. The MIL piggy is configured such, that for each incoming MIL telegram its data is written to a FIFO and a TTL pulse is generated. A very short LEMO00 cable connects the output of the MIL piggy (left) to one of the inputs of the SCU (right).

ECA and Firmware Activity

The main work is done by the ECA.

As the input from the host and the status information to the host is transmitted as timing messages via the ECA, the firmware is basically a simple timing message <-> MIL Devicebus gateway.

To this simple gateways chopper specific functionality has been added:

• Write 'Strahlweg Maske', 'Strahlweg Register' and 'Anfordermaske' to the chopper control unit.
• Read status information from the chopper control unit

Figure: White Rabbit to MIL gateway on SCU. The central components are the ECA and the lm32 softcore (3d-boxes). Numbers indicate data flow for certain activities; 1: receive timing message from the White Rabbit network, 2: generation of a MIL event, 3: snoop MIL events (optional monitoring), 4: read and process data of MIL event (optional monitoring).

The figure above indicates the procedures at the gateway.

1: Receive Timing Message
If a timing message is received from the White Rabbit network. The ECA checks if it fulfills the rules (GID, EvtNo). The ECA rule is configured with a negative offset of 500us , which is plenty of time for processing and sending the telegram onto the MIL bus.

2: Send MIL Telegram
The lm32 firmware maps relevant information from the White Rabbit timing message into the 16 bit data of a MIL telegram. For precise timing, the lm32 does not send the telegram to the MIL bus directly. Instead, the wishbone channel of the ECA is used: The lm32 firmware builds a timing message and copies the 16 bit MIL telegram to the Parameter field. The deadline of this timing message equals the deadline of the original timing message from the data master (only a small offset is subtracted, which takes into account delays and transmission time on the MIL bus). This timing messages is written to the input of the ECA. On-time, the wishbone channel copies the 16bit MIL telegram to the destination address. In this case, the destination address is register in the SIO that finally send the MIL telegram to the MIL bus.

3: Snoop MIL Telegram
For optional monitoring, the gateway offers the possibility of snooping on the MIL bus. To avoid conflicts with sending MIL telegrams, monitoring only relies on the MIL piggy. For each received MIL telegram, the piggy is configured to generate a TTL pulse on an output and to store the data in a FIFO. The event of a MIL telegram is timestamped using the ECA; this requires to connect the TTL output of the piggy to a bi-directional IO of the SCU via a Lemo cable. The IO is configured as input and to trigger the TLU of the ECA. Long story short: A received MIL event is converted to a timing message at the ECA input. The lm32 is configured to receive this timing message. Obviously, this timing message only carries the timestamp of the MIL telegram received but not the data of the MIL telegram.

4: Obtain MIL Event Data
When the lm32 detects an incoming MIL event via a timing message, it reads the MIL telegram data from the FIFO of the piggy. The lm32 firmware then builds a timing message with the data of the MIL telegram and writes it to the input of the ECA.

Host
All interfaces connected to the FPGA/ECA can receive all information 1..4 via the ECA. This is extremely helpful for debugging on the host. This is also used for a dedicated saftlib application, that can be used to compare various combinations of messages with each other. As an example, one can compare the deadlines of received MIL telegrams with the original timing messages from the Data Master. The comparison may include deadlines, synchronization and successful mapping of data.

Some remarks

MIL telegrams have 20 bits in total, 16 data bits plus 4 bits for manchester encoding. Bus speed is 1 MHz. Thus sending a MIL telegram takes 20 us. However, MIL protocol has no built-in 'balancing' such as the 8b/10b encoding done with Ethernet. To be on the safe side, it is assumed that a 5us pause is required between MIL telegrams. Thus, it takes 25us for transmitting one MIL telegram.

Figure: Typical MIL telegrams have a jitter (max,min) of about 3us.

Bugs

During the beamtime 2021, several reports of missing MIL events appeared. For example, the electron cooler in ESR was sometimes not switched off because of a missing event. The reason for the missing events is not found yet. This is a collection of observations related to the missing events. It may help in finding the cause of the probelm.

• The WR-event that corresponds to a missing MIL-event is definitely there. The events are not missing in the schedule of the Data Master.
• The conversion of a WR-event into MIL-event happens in an LM32 program. The LM32 sends MIL-events with the correct deadline into the ECA, these events have the ID=0xffffffff0000MMMM, where M represents the bits of the MIL-event. The ECA-WB-Master output channel writes the MIL-event into the MIL-Piggy. By configuring the ECA-IO channel to pulse one of the SCU-IOs at the same event, it can be seen that the firmware indeed injects the event into the ECA. However sometimes the MIL-Piggy doesn't send anything. The Missing Events can be observed on the oscilloscope.
• 
• It is unclear why this happens. Speculation: the MIL-Piggy doesn't accept that WB-Write from the EAC-WB-Master output channel for some reason, or the WB-Master output channel has a bug and doesn't execute the WB-write, or the MIL-Piggy accepts the WB-write but has a bug and doesn't send the MIL-Event. => Make a SignalTap that targets the WB-interfaces of ECA-WB-Master output channel and the WB-interface of the MIL-Piggy
• The MIL-Events of the prduction systems (SIS18 and ESR) in BG2.009 have glitches after the last Manchester-bit. This doesn't seem to be a problem because it always happens after the last bit (the parity bit). These glitches do not appear on a similar setup in my office. The reason/source of the glitches is not located yet.
• 
• The rate of missing events depends on the schedule, but approximately every 30-60 minutes a MIL event is lost. Often the missing MIL-events are not essential for machine control (e.g. missing UTC timestamp bits). Missing MIL events that are important appear a few times per day.

-- DietrichBeck - 18 Nov 2024