How-To: Install Frontend

Introduction

This how-to describes steps necessary to install and commission a frontend

Preparation

• CCT, Machine Model and Services
  • have a meeting on what the frontend is supposed to do and how it is integrated into the control system
  • make sure relevant persons are invited and attend the meeting or are at least informed (LSA, machine-model, technical group)
  • agreement on proposal of design/solution
  • agree on nomen, have nomen ready to use
  • have a written protocol/minutes
• Technical Group
  • establish contact
  • understand technical environment of the equipment
  • visit location of technical environment
  • clarify how the frontend will be connected to the equipment (signals, interfaces, cables, plugs ...)
  • VERY VERY VERY IMPORTANT: rack space
  • clarify cooling
  • IMPORTANT: clarify power outlets; installing additional power outlets might take time
• Insfrastructure, INN, TOS
  • ACC/WR network
    • are sufficient network outlets available?
    • reserve the port(s) needed (even if there seem to be unused ports, they might already be reserved)
    • if not: INN/TOS will propose a solution
    • checkout required length of ACC and White Rabbit patch cables
    • IMPORTANT: providing additional outlets might take a lot of time (> 6 months), if new cables need to be pulled to the building/room/cave
    • to be on the safe side, planning should start ~ 1 year ahead of time
    • remark:
      • INN/TOS: will provide cables into the room/cave
      • are ACC/WR outlets directly accessible in the same rack?
      • if not, check out with the machine coordinator or STV how (long) patch cables will be pulled from the INN/TOS outlets to the FEC
      • maybe additional cables inside the cave need to be pulled
      • clarify, who will pull these cables and maybe you need to organize these cables and helping hands
• Hardware
  • getting the hardware needs to be planned. Sometimes, hardware needs to get ordered or built
  • request hardware sufficiently long time prior installation (at least one year)
    • SCU: HEL
    • SuperMicro: via INN
    • Timing Receiver: TOS
    • additional modules: ...

Detailed Design, Low Level (HEL, TOS), FESA, APP, LSA, Model

• clarify set and get values and their data format.
  • this is CRUCIAL
  • changing the set/get values at a later stage will be tedious and very time consuming (voice of experience)
• if similar frontends will be installed at various machines, involve ALL modellers in the design phase
• clarify the future representation of set/get values in ParamModi
• clarify how parameters are treated in the machine model
  • the modelers know everything about it ...
  • but you should have green light from the modelers
• clarify how the parameters are treated by LSA
  • the LSA people know everything about this ...
  • but you should have green light from the LSA people
• clarify how the set/get values will be handled/provied by the FESA class
  • the FESA people know everything about this ..
  • but you should have green light from the FESA people
• if dedicated FPGA gateware/firmware is involved
  • have this organized or do it yourself

IMPORT: Make sure the full path from the Settings (= ParamModi) to the last bit in the FPGA is known and there is green light

Hardware Installation

After some development things should get installed. Here: SuperMicro

• Preparation
  • get stuff from INN
    • SuperMicro
    • have your SuperMicro registered (IP, VLAN)
    • two ACC network cables are needed; you know the required length already (see top)
    • two power cables will be needed; if you ask kindly, you'll get these cables from INN too; if the frontend gets installed at the top of a very tall rack, you may want to ask for extra long power cables (if your forget this point you'll need to revisit INN another time)
  • get stuff from TOS
    • Timing Receiver
    • WR network cable; you'll need to specify the type of connectors (LC or SC)
    • have your timing receiver registered (IP, VLAN)
• Setting Up
  • find some space in your lab for testing
  • mount the timing receiver(s) in the SuperMicro, see here
  • connect two ACC network cables, see here
  • connect White Rabbit cable(s) to the timing receiver
  • connect the SuperMicro to power
  • configure management
    • logon the management interface with the password provided by INN
    • add a new user and set a password to the one that is appropriate for Rufbereitschaft and in your team
    • logoff and make sure you can logon with the new user
    • change the default password
    • IMPORTANT:
      • configure the behavior in case just one LAN port is active, see here
      • I (db) prefer mode 'dedicated'
  • configure PXE boot, see here
  • configure nfs-init, just basic stuff like timing RTE is required at this stage
• Basic Testing
  • power-cycle the SuperMicro
  • make sure ssh login works
  • check for correct operating system (ram disk)
    • 'uname -a'
    • 'cat /etc/os-release'
    • ...
  • check the timing reciver(s), see here
    • 'eb-info'
    • 'eb-ls'
    • 'saft-ctl'
    • 'eb-mon'
      • link up?
      • tracking?
      • got IP?
• Burn-In
  • have your Frontend up and running for at least 24 yours in the lab
  • burn-in is preferred at an ambient temperature similar to the one where it will be installed
  • after 24 hours, repeat basic testing
• Final Frontend Installation
  • preferably done together with the technical group of the equipment
  • either ask for help from the technical group or ACO colleagues
  • connect everything
  • power-up the Frontend and redo basic testing (see above)

Software Installation and Integration

• make sure all required low-level gateware/firmware/software is installed (nfs-init...)
• ask the FESA developer to deploy the FESA class and to export the device to the FESA database
• check you can talk to the device using 'FESA Explorer'
• ask the LSA people to import the the device into the LSA database
• check you can talk to the device using 'Device Control'
• check that MASP receives proper information about the device
• ask someone from the LSA team to 'repair'/'recalculate' the pattern 'setting' to be used for testing with 'ParamModi'
• arrange with your ACO colleagues (INT) or operation (PRO) that you get a time window for testing

Final Test

The great moment has arrived!

At this stage you should know the relevant people from model, LSA, APP, SER, FEC, technical group

Do a 'dry test' from ParamModi to Frontend

• using the Scheduling App, plan in the pattern for testing
• using ParamModi, set proper values for the device
  • you have agreed much earlier with the developers where and via which tab the corresponding fields will be available
  • it might be, that you have to remind the modelling colleagues that it is now time to deploy a new version of ParamModi that includes your new device
• do a 'Vollversorgung' of your pattern in ParamModi
• checkout 'Ergebnis der letzten Datenversorgung anzeigen'
• checkout the set-values via FESA Explorer
• There is a high risk things won't work at the first try. Debugging starts. You might need help from
  • model
  • APP
  • LSA
  • Services
  • FESA
  • ...
• start the pattern using the BSS App

Do a full test including equipment. You might need help from

• technical group; they might help with local diagnostics
• STV ('Zuschaltung' required?)
• machine coordinator (allowance for operation, interference with other equipment)

-- DietrichBeck - 13 Jun 2023