Integrating DIM into the Control System Studio on the Data Access Layer

This page documents the progress of the project. The project is not finished yet. First, I (Martin Feldmann) will give an introduction on what the project is about and the components it uses. Then I will explain the goals, as well as the plan to reach these goals. At last, I will describe how much progress was made and give all the information we discovered so far.

Introduction

The Control System Studio (CSS) ( http://css.desy.de/content/index_eng.html ) is build on the Eclipse Rich Client Platform. It has the look and feel of the eclipse IDE, but it is not an IDE for the development of code. It is an application meant to design control systems for experiments. Because of the nature of the rich client platform it is capable of loading an enormous ammount of plugins for all types of tasks. Because CSS is about control systems, it features displaying, monitoring and editing of process variables, using the plugins appropiate for the given task. So CSS already has support for EPICS, TINE and TANGO, though I cannot say if it really works for TINE and TANGO. This was achieved by implementing the data access layer of CSS, called DAL. DAL is a special layer between the application and the protocol used. DAL is the interface of CSS for all data which comes from external sources, to unify the handling of the data on the application side.

On the other hand is DIM ( http://dim.web.cern.ch/dim/ ), a communication system for distributed environments, which is used to propagate process variables across a network.

The goal of the project is, to integrate DIM into CSS by implementing the data access layer (DAL).

Concept

There was no real concept. We just started out by exploring how CSS works, and used the really bad documentation as a guide. You can download the documentation from the CSS website. Because EPICS was already working with CSS, we also tried to understand how it works and used this as a guide as well.

Preparations

To continue developments regarding the integration of DIM into DAL, several preparations have to be made:

First, you need a version of Eclipse for RCP/Plug-in Developers. You can choose from this list or download it directly from here. Furthermore you need an up to date version of the JRE or JDK, which you can get from the sun site, if it is not already installed on your machine.

Next, install DIM. Just follow the instructions from the website. Here is the link to the intstructions on the Windows installation. At this point, step 1 and 2 are sufficient. Starting and using DIM comes after you installed everything.

It is recommended that you have an OPC IOC Shell from EPICS with some demo variables, just to be able to test the system with EPICS process variables.

To be able to download the latest CSS sources, you first have to register an account at desy. This easy and fast procedure is described here. Although you should be careful with the password you choose, which is highly restricted, and store it somewhere for your personal use, because you will have to change it every few months and you may not use a password which you have had before, because they seem to store your password history.

If you have made it, you can try and download the sources of CSS now. Therefore you have to start your eclipse RCP version, and configure your CVS settings like this. Now, checkout/ import the project set files of module css-applications from CSSTeamProjectSets, which is described here.

At last, configure the run configuration following this description.

Now you should be able to push the run button and start or debug CSS from within your eclipse IDE.

First Steps

To get an impression on what we are talking about, I will give you a step by step explanation on a typical control systems use case:

In this use case, you are the operator or developer of a control system component. First you start your EPICS soft IOC shell and type dbl, which should show you the EPICS demo process variables available. This soft IOC simulates your control system environment in the form of process variables, which are actually on your machine only.

The next step is to start your eclipse and run the CSS application. Once CSS is active, click on the menubar and go to CSS/Preferences/CSS Core/Control System, where you push the radio button of EPICS and then click apply and ok. Another click on the menubar CSS/Display/Synoptic Display Studio will start the SDS plugin and open up a new view. Choose one of the available widgets on the sidebar, like Meter or Label, and place it somewhere on the board. If you click on the new element, you can see its properties on the right side. Choose "Double Value" as "Value Type", leave "$channel$" as "Primary PV" and click on "Alias" where you choose "channel" as the "Name" and "EpicsDemo1" as the "Value". Now you can push the "Display in Run Mode" button, and hopefully you will see the new widget which displays the value of your EpicsDemo1 process variable from your soft IOC.

This is the end of the demo, and it shows us exactly what we want to implement for the DIM protocol.

The next section covers information about the actual progress of the project, what we learned while working on it and some guidelines.

Overview of measures

We want to integrate DIM into the DAL. Therefore we must create one or more plugins for the RCP application, which encapsulate DIM functions (a client and a server) in the processes of DAL. Moreover you have to understand the procedures of DAL in order to know which methods to implement. Because the task of understanding DAL is quite difficult, you must have a basis on which you can test things, your own plugin. In the next sections I will describe how to create a plugin and activate DIM for your testing.

Creating a new plugin

Finally, if you succeeded to do the preparations and first steps sections, we will start the coding. The first step is to create a plugin for CSS. You have two choices:

1. Follow the steps described on the official homepage, or
2. you can just copy and paste an existing project.

Because copy and paste is easier and provides you with an already working plugin, we decided to go that way. Just copy the project org.csstudio.platform.libs.epics and rename it to org.csstudio.platform.libs.dim. Moreso, because now you can see how the original plugin works and adjust it to your needs (which will be covered in the next section so don't do anything else in this step for now).

You should use the same procedure for creating another plugin, org.csstudio.platform.libs.dal.epics, copy and rename it to org.csstudio.platform.libs.dal.dim

Configuring and testing the new plugins

To use and test your new plugins, several things have to be done:

1. Adjust the code of the plugin

• Go to your new project org.csstudio.platform.libs.dim and open the file EpicsPlugin.java. Rename the file and the class name to DimPlugin. Change the public static final String ID from "org.csstudio.platform.libs.epics" to "org.csstudio.platform.libs.dim". That's it for now. You can, and later on you should delete all things related to java channel access, e.g. the public enum MonitorMask, the stuff related to JNI and JCA in the public final void start method and also in the public void installPreferences method.

• Adjust the second plugin in a smiliar way. Go to org.csstudio.platform.libs.dal.dim and open the file Activator.java. Here you change the public static final String PLUGIN_ID to "org.csstudio.platform.libs.dal.dim".

• If you want to test your new plugins at the end of this section, just put an additional println command in the constructors or in the start methods. In this way you can check the console if your new plugins were started.

2. Adjusting the plugin.xml

• Any plugin in an RCP application is configured in a special file called plugin.xml. This file is mandatory, because it links between the application and its plugins. It is located in the base directory of the plugin. A double click will show the file in an organzied manner, with tabs at the bottom of the editor. Go through every tab and look closely at the offered fields. Most changes are obvoius, but I will list the important ones here:
  • org.csstudio.libs.dim: Go to the overview tab. In the general information section, change the ID, version, name and provider as you like. The important thing here is to change the activator correctly, like org.csstudio.platform.libs.dim.DimPlugin. Check in the MANIFEST.MF tab if the changes have been assigned correctly.
  • org.csstudio.libs.dal.dim: Change the ID, version, name and provider as you like, as well as the activator to org.csstudio.platform.libs.dal.dim.Activator. Go to the dependencys tab and change the entry org.csstudio.platform.libs.epics to org.csstudio.platform.libs.dim. The runtime tab should only display org.csstudio.platform.libs.dal.dim and org.dim.css.dal.dim as exported packages, and the extensions tab should list only the name of the plug (e.g. DIM plug) if you expand the entry org.csstudio.platform.libs.dal.plugs. Check in the MANIFEST.MF and in the plugin.xml tab if the changes have been assigned correctly. You also have a refernce here to the factory class used to instantiate properties as well as to the ID, which should be set to "DIM".

3. Configuring the run configuration

• Go to the eclipse run configurations dialog. Check that "Run a product" is selected, which should be "org.csstudio.desy.product.product". Next, go to the "Plug-ins" tab and make sure that your new plugins are listed and selected for startup.

4. Making DIM available for your plugins

• To use DIM instead of EPISC, you first have to download and install DIM as described in the preparations section of this article. You start DIM as described in step 3. If you started the test server, which is also avaiable in the DID utility, you can see the values it provides, shown in its own dialog.

• Now, to let CSS make use of your own DIM plugin, you have to add it as an option in the preferences, called the set of default control systems. This is located exactly where you chose EPICS as the default control system last time (CSS/Preferences/CSS Core/Control System). To get it as an option you have to add it to the enumeration public enum ControlSystemEnum implements IAdaptable, which is located in the plugin project org.csstudio.platform in the package model.pvs with the file name ControlSystemEnum.java . Just add a line like DIM("dim", "DIM", true, "icons/controlsystem-epics"). The important parts here are the private String _dalName and the private boolean _supportedByDAL, which we set to "DIM" and "true" respectively.

• So if you did this correctly, CSS will display the option "DIM" as a radio button in the prefernces section next time you start it. Moreover, it will search for a plugin referenced by the string identifier "DIM", which you should have set in the activator of org.csstudio.platform.libs.dal.dim .

5. Testing your new plugins

• To test your new plugins you start the application pushing the run button (your new run configuration). If CSS starts, go to CSS/Preferences/CSS Core/Control System and choose DIM as the default control system. You can use the widget you created during the first steps section, or create a new one. The procedure is the same as in the first steps section. At this point the name of the PV doesn't matter. You don't even have to start a DIM name server or DID. Just click on the run button of the widget and look at your console. (To look at the console you have to change the view of CSS back to CSS standard.) If the message of your println command is displayed, congratulations, you are in the game and everything worked as it should. Now it is time for the next step, starting the development.

Developments so far, what's next

In theory, you just have to set up a DIM server and client pair somewhere in the DAL API architecture. But where should that be? To answer this question, you have to understand how DAL works, which is not easy, because DAL seems quite complex and its documentation is almost non existent. Here are the documents about DAL which were available to me. I will summarize it below.

This is the summary of the documentation (every point here applies only to the plugin project org.csstudio.platform.libs.dal.dim). You need the following classes:

• DIMPlug
• PropertyFactoryImpl
• PropertyProxyImpl
• MonitorProxyImpl

I will explain the functionality of these classes, their methods and attributes in the following sections. The explanation only covers the components I could understand. This is intended for a better understanding. You can either try and make your own version, from scratch or from the EPICS files.

DIMPlug

The class DIMPlug extends the abstract class AbstractPlug. This relationship is mandatory because it is an integral part of the RCP application system.
Important attributes of DIMPlug:
• public static final String PLUG_TYPE = "DIM";
• private static DIMPlug sharedInstance;
Constructor:
• In the constructor you should register the properties you want to use. (You also have to implement these properties at some point.) You can do this by calling this.registerPropertyProxyImplementationClass(DoublePropertyImpl.class, DoublePropertyProxyImpl.class); for a property of the type double. Observing this call, you can detect a mapping between the DAL class DoublePropertyImpl, which is used on the DAL side as a representation for a double type, and your own class DoublePropertyProxyImpl, which represents a property of type double on your side. (This means, the DoublePropertyProxyImpl will later act as a link between DAL and DIM.)
You need the following methods because of the inheritance from AbstractPlug. You also need to check all of these methods for dependencies and adjust them accordingly, either by deleting unnecessary dependencies or by altering names, strings or functionality:
• protected <TT extends PropertyProxy<?>> TT createNewPropertyProxy(String uniqueName, Class<TT> type) throws ConnectionException
• public String getPlugType()
These methods operate on the instance of private static DIMPlug sharedInstance.
• public static AbstractPlug getInstance(Properties configuration) throws Exception
• public static AbstractPlug getInstance(AbstractApplicationContext ctx) throws RemoteException
• public synchronized void releaseInstance() throws Exception
Devices are not supported, so you can just throw an exception for these methods, like in EPICSPlug.
• protected Class<? extends AbstractDevice> getDeviceImplementationClass(String uniqueDeviceName)
• protected Class<? extends DeviceProxy> getDeviceProxyImplementationClass(String uniqueDeviceName)
You don't need to change these methods:
• protected DirectoryProxy createNewDirectoryProxy(String uniqueName) throws ConnectionException
• protected <T extends DeviceProxy> T createNewDeviceProxy(String uniqueName,Class<T> type) throws ConnectionException
The following methods need special attention. Basically they are needed because of the inheritance from AbstractPlug. But their implementation is dependent on jca, the general channel concept and the classes supporting the interface of the EPICS data. In my understanding, these methods return the specific classes implementing the data type of the DAL side, like DoublePropertyProxyImpl and so on. But to be able to return the DAL type, they access self provided intermediate format classes, called DBRType for EPICS. An intermediate type for DIM data has yet to be implemented. Here, your own classes should be returned, which implement these data types.
• protected Class<? extends SimpleProperty<?>> getPropertyImplementationClass(String uniquePropertyName)
• public Class<? extends SimpleProperty<?>> getPropertyImplementationClass(Class<? extends SimpleProperty<?>> type, String propertyName) throws RemoteException
• public Class<? extends PropertyProxy<?>> getPropertyProxyImplementationClass(String propertyName)
There are other methods and attributes which I think you need (but I am not really sure on this). I didn't change them, at the most I deleted some lines with dependencies of jca, and I will list them here:
• public static final String DEFAULT_AUTHORITY = "DEFAULT";
From the interface PlugContext, no change necessary, but I am not sure if you even need them:
• public RemoteInfo createRemoteInfo(String uniqueName) throws NamingException
• public DirContext getDefaultDirectory()
Timer stuff:
• private Timer timer;
• class ScheduledTask extends TimerTask
• private synchronized Timer getTimer()
• public TimerTask schedule(Runnable r, long delay, long rate)
Other thoughts:
• For the plugin project org.csstudio.platform.libs.dim you only need one class, the DimPlugin
• You need to import the DIM sourcecode, either as a library or as the sourcecode itself. Most probably, these sources have to be changed, or rather for the sake of compatibility, extended.
  • Problems with todays version of DIM: DIM is not generic nowadays, which makes nice problems, because at the DAL everything is generic, and you do not know which types DAL wants to have at runtime. So you have to find a way around it, one thought here being the extension of the DimInfo class by XyzTypeDimInfo, which inherits an abstract and generic method getValue, which is implemented in the actual subtype. That means, you have the method infoHandler, which you overwrite in each of these subclasses, so that it calls exactly the typed method referring to this subtype.
Here is an example of this (also including the monitorChanged call which is explained later):

public class MyDimInfo<T> extends DimInfo {

   protected T value;
   protected MonitorProxyImpl<T> monitor;

   public T getValue(){
      return value;
   }
   
   public void infoHandler()
   {
      //nothing new here
      System.out.println("Received: " + getDouble());
      System.out.println("Quality: " + getQuality());
      System.out.println("Timestamp: " + getTimestamp());
   }
}


public class MyDimInfoDouble extends MyDimInfo<Double> {

   public void infoHandler()
   {
      System.out.println("Received: " + getDouble());
      System.out.println("Quality: " + getQuality());
      System.out.println("Timestamp: " + getTimestamp());

      //update the value for access from another class
      value = getDouble();
      //call monitor changed to notify the DAL side about its change
      monitor.monitorChanged( this );
   }
}

PropertyFactoryImpl

You need this class, but you only have to change the names from EPICS to DIM, so that's quite easy.

PropertyProxyImpl

This class seems to represent the DAL side of a process variable, at least for EPICS. It extends AbstractProxyImpl and implements PropertyProxy<T>, SyncPropertyProxy<T> and DirectoryProxy. These relationships seem reasonable for our project. In EPICS, PropertyProxyImpl also implements ConnectionListener and GetListener, which are both from the gov.aps.jca package. However, there is also an interface called ConnectionListener in the package org.epics.css.dal.context package. Implementing this interface does not seem necessary on first glance, because with DIM, there is no connection in the sense of an EPICS-like channel.
Mechanism to typed values: This class represents something like a process variable. But it has no ultimate member variable where an actual value is stored. (The actual value seems to be stored in the MonitorProxyImpl class, so more on that later.) Instead, a complex mechanism is invoked resolving the actual value.
• First, the class PropertyProxyImpl is generic. A class resolving an actual typed value must therefore extend it, e.g. a double value from a process variable would finally be resolved from a class DoublePropertyProxyImpl, which extends PropertyProxyImpl. This means that you need an extra class for every actual data type you want to support, like double, long or string. (That's just for the DAL side of it.)
• The actual value is then received through a call of getValueAsync or getValueSync, respectively set through a call of setValueSynch or setValueAsynch. These methods are the integral parts to receiving or sending a value from EPICS, DIM or any other protocol.

Get and set methods - How the actual value is handled

So, here the magic begins. There are requests for the asnchronous get and set methods, as well as listeners for these requests and for the synchronous methods. For EPICS, it works like this:
• Set up the request or listener
• Activate the channel: plug.flushIO();
• Retrieve the value from the event
The problem here is, that each of these calls can be traced up to the sources of jca. So there will be an event originating from jca, which triggers all these calls and stores the actual value. This is rather useless for us, because DIM sources are not easily substitutable, though they are much simpler than the jca system. Moreover, there is the concept of a channel, which we also don't have in DIM.
The Personalpages.main task here is to figure out how this fits for DIM, or how DIM fits into this structure. My idea about retrieving the value is described in the next section, but if it works, it should in principle be possible to just take out all jca dependencies and stay with the DAL side about requests and responses, adjusting these calls accordingly.
Hint: insert setConnectionState(ConnectionState.CONNECTED); at the end of the PropertyProxyImpl constructor. You probably need this line if you take out the section where the channel is created.

MonitorProxyImpl

This class extends RequestImpl and implements MonitorProxy, Runnable and for EPICS it also implements MonitorListener from jca. I do not fully understand what this class really does, so you have to check this and be careful with my estimations. I believe this class monitors, or listens to the value of the channel. You can leave all of the Runnable implementation from EPICS as it is, because that's just thread stuff and should work in the same way.
Important things to see here: The class has an attribute protected T value; which should be the actual value this whole project is all about. It is set in the method public void monitorChanged(MonitorEvent ev) which is inherited from the jca interface MonitorListener, where it is received from a jca class called MonitorEvent and transformed to a java value from its type description class DBR. At the end it notifies the system (the DAL side) about the changed value. Thus, this method seems to be the all important one in this class, and you should implement a method equal to it, but changing the dependency from jca to DIM.
My idea for the implementation was to extend DimInfo in such a way that you have a subclass for each type (double, long...), and override the infoHandler method, storing the actual DIM value in an attribute of that subclass. This value would always be up to date, because infoHandler is always called on value change or periodically. Moreover, infoHandler should also call the equivalent to the monitorChanged method and pass the new value to it, where it will be assigned to the DAL side and it gets notified about the change. In this way it should be possible to reuse a minimum intersection of the EPICS implementation, without the jca dependencies.

PlugUtilities

Another interesting class is the PlugUtilities. This class converts values from Java to the DBR types and back and manages the implemented classes. It is just an auxiliary class, but you have to provide something equal one way or another. It has no priority and you will notice the need for it automatically when it occurs.

Hints to adjusting existing code

If you want to adjust code you copied from epics, or from anywhere else, here are some hints:
• If you delete or comment the import statements, you can check for dependencies, because the dependent code will be marked as error/ unknown by eclipse. Now that you have identified which code belongs to whom, you can delete or comment it and try if it still works. The first goal should be to get a basic version, which doesn't have to do anything, but which is also free of dependencies from anything but DAL itself.

The trailed class diagram visualizes the relationships on the boundary to jca and to the according java base classes.

-- MartinFeldmann - 29 Jun 2010

• DAL Property Proxy and Monitor Proxy Class Diagram:
  

• DAL.zip: The available DAL documentation