MQ-based Online API

MQ-based Online API

Intro

New Online API proposal: Message-Queue-based data exchange for STAR Online domain;

Purpose

Primary idea is to replace current DB-centric STAR Online system with industrial-strength Message Queueing service. Online databases will, then, take a proper data storage role, leaving information exchange to MQ server. STAR, as an experiment in-progress, is still growing every year, so standard information exchange protocol is required for all involved parties to enable efficient cross-communications.

It is proposed to leave EPICS system as it is now for Slow Controls part of Online domain, and allow easy data export from EPICS to MQ via specialized epics2mq services. Further, data will be stored to MySQL (or some other storage engine) via mq2db service(s). Clients could retrieve archived detector conditions either via direct MySQL access as it is now, or through properly formatted request to db2mq service.

[introduction-talk] [implementation-talk]

Primary components

• qpid: AMQP 0.10 server [src rpm] ("qpid-cpp-mrg" package, not the older "qpidc" one);
• python-qpid : python bindings to AMQP [src rpm] (0.7 version or newer);
• Google Protobuf: efficient cross-language, cross-platform serialization/deserialization library [src rpm];
• EPICS: Experimental Physics and Industrial Control System [rpm];
• log4cxx: C++ implementation of highly successful log4j library standard [src rpm];

Implementation

• epics2mq : service, which queries EPICS via EasyCA API, and submits Protobuf-encoded results to AMQP server;
• mq2db : service, which listens to AMQP storage queue for incoming Protobuf-encoded data, decodes it and stores it to MySQL (or some other backend db);
• db2mq : service, which listens to AMQP requests queue, queries backend database for data, encodes it in Protobuf format and send it to requestor;
• db2mq-client : example of client program for db2mq service (requests some info from db);
• mq-publisher-example : minimalistic example on how to publish Protobuf-encoded messages to AMQP server
• mq-subscriber-example : minimalistic example on how to subscribe to AMQP server queue, receive and decode Protobuf messages

Use-Cases

• STAR Conditions database population: epics2mq -> MQ -> mq2db; db2mq -> MQ -> db2mq-client;
• Data exchange between Online users: mq-publisher -> MQ -> mq-subscriber;

How-To: simple usage example

1. login to [your_login_name]@onl11.starp.bnl.gov ;
2. checkout mq-publisher-example and mq-subscriber-example from STAR CVS (see links above);
3. compile both examples by typing "make" ;
4. start both services, in either order. obviously, there's no dependency on publisher/subscriber start order - they are independent;
5. watch log messages, read code, modify to your needs!

How-To: store EPICS channels to Online DB

1. login to onl11.starp.bnl.gov;
2. checkout epics2mq service from STAR CVS and compile it with "make";
3. modify epics2mq-converter.ini - set proper storage path string, add desired epics channel names;
4. run epics2mq-service as service: "nohup ./epics2mq-service epics2mq-converter.ini >& out.log &"
5. watch new database, table and records arriving to Online DB - mq2db will automatically accept messages and create appropriate database structures!

How-To: read archived EPICS channel values from Online DB

1. login to onl11.starp.bnl.gov;
2. checkout db2mq-client from STAR CVS, compile with "make";
3. modify db2mq-client code to your needs, or copy relevant parts into your project;
4. enjoy!

$> qpid-stat -c -S cproc -I localhost:5672
Connections
  client-addr           cproc            cpid   auth        connected       idle           msgIn  msgOut
  ========================================================================================================
  127.0.0.1:54484       db2mq-service    9729   anonymous   2d 1h 44m 1s    2d 1h 39m 52s    29      0
  127.0.0.1:56594       epics2mq-servic  31245  anonymous   5d 22h 39m 51s  4m 30s         5.15k     0
  127.0.0.1:58283       epics2mq-servic  30965  anonymous   5d 22h 45m 50s  30s            5.16k     0
  127.0.0.1:58281       epics2mq-servic  30813  anonymous   5d 22h 49m 18s  4m 0s          5.16k     0
  127.0.0.1:55579       epics2mq-servic  28919  anonymous   5d 23h 56m 25s  1m 10s         5.20k     0
  130.199.60.101:34822  epics2mq-servic  19668  anonymous   2d 1h 34m 36s   10s            17.9k     0
  127.0.0.1:43400       mq2db-service    28586  anonymous   6d 0h 2m 38s    10s            25.7k     0
  127.0.0.1:38496       qpid-stat        28995  guest@QPID  0s              0s              108      0