Distributed Systems

Three classes serve to build distributed systems based on CS. The communication is based on TCP/IP.

1. QueueListener
2. QueueClient
3. QueueServer

QueueListener

Each node in a distributed system must run an object of QueueListener. In the data base the object "Listener" has been configured. Typically, this object is the first objected to be instantiated by the SuperProc when the SuperProc starts. In the data base, check the Super process and have a look at "processes to load during start up".

QueueClient

Each remote node needs it's own QueueClient object on the local node. All events ("calls", "notifications") to a remote node are directed to the QueueClient object of the remote node. By convention, the QueueClient object to the remoted node "remote123.gsi.de" must have the name "QueueClient_remote123.gsi.de".

QueueServer

A remote node has a QueueServer object for the local node. All events received from the local node are distributed to the objects on the remote node via the QueueServer object. By convention, the QueueServer object on the remote node "remote123.gsi.de", that receives events from the local node "local123.gsi.de", must have the name "QueueServer_local123.gsi.de".

Principles of operation

The connection is a one way road and goes from a QueueClient to the QueueServer. For a two way connection, there must be a QueueClient and a QueueServer object on each node.

When connecting from a local node "local123.gsi.de" to a remote node "remote123.gsi.de" the following things happen.

1. On the local node, the "QueueClient_remote123.gsi.de" is created.
2. The "QueueClient_remote123.gsi.de" connects to the listener on the remote node
3. The listener on the remote node obtains a free TCP port and returns it to the "QueueClient_remote123.gsi.de" on the local machine.
4. The "QueueClient_remote123.gsi.de" on the local node tries to connect to the "QueueServer_local123.gsi.de" on the remote node.
5. The listener on the remote node creates the "QueueServer_local123.gsi.de" and passes the free TCP port to the QueueServer
6. The "QueueServer_local123.gsi.de" on the remote node listens on the free TCP port for a connection from the "QueueClient_remote123.gsi.de"
7. QueueClient and QueueServer connect
8. The one way connection from the local to the remote node is established, now we have to take care about the connection from the remote node to the local node.
9. The "QueueServer_local123.gsi.de" on the remote node creates the "QueueClient_local123.gsi.de" object on the remote node.
10. Steps 2) to 7) are performed to establish the connection from the remote to the local node

Establishing a connection

Two possibilities exist to establish a connection

1. Just send an event to an object on the remote node ("Ping"). If the connection is not yet established, a QueueClient object to the remote node is created. The rest works by itself. In this case however, the event to the remote node will become lost if the connection was not yet established. Also, creating QueueClient and QueueServer on the fly will make remote node and local node quite busy for about 10 seconds
2. If you know beforehand which nodes should connect to each other, it will be better to establish the connections already when the CS systems are starting up. As an example, one can include the "QueueClient_remote123.gsi.de" object in the list of "objects to load during startup" in the database entry of the Super process for the local machine.

-- DietrichBeck - 06 Jul 2005