STAR DB API v2

This page holds STAR database API v2 progress. Here are major milestones :

1. Components Overview (UML)
2. Core Components technology choice:
   1. [done] New version of XML configuration file + XML Schema for verification (.xsd available, see attachment);
   2. [done] New XML parser library :
      • [done] rapidXML (header-only, fast, non-validating parser) - will use this one as a start;
      • [not done] xerces-c (huge, validating, slow parser);
   3. [done] Encryption support for XML configuration :
      • [done] XOR-SHIFT-BASE64;
      • [done] AES128, AES256;
      • [CYPHER-TYPE]-V1 means that [CYPHER-TYPE] was used with parameter set 1 (e.g.: predefined AES key 'ABC');
   4. [done] Threads support for Load Balancer, Data preload, Cache search
      • pthreads - popular, widely used;
        
   5. [done] Database Abstraction Layer reshape :
      • OpenDBX (MySQL, Postresql, Sqlite3, Oracle) - API level abstraction, will require "personalized" sql queries for our system;
   6. [done] In-memory cache support [requires C++ -> DB -> C++ codec update]:
      • memcached (local/global, persistent, with expiration time setting);
      • simple hashmap (local - in memory cache, per job, non-persistent);
   7. [in progress] Persistent cache support :
      • hypercache (local, persistent, timestamped, file-based cache);
3. Supported Features list
   • OFFLINE DB: read-heavy
   • ONLINE DB: write-heavy
4. AOB

--------------------------------------------------------------------------------------------------------------
New Load Balancer (abstract interface + db-specific modules) :

• dbServers.xml, v1, v2 xml config support;
• db load : number of running threads;
• db load : response time;
• should account for slave lag;
• should allow multiple instances with different tasks and databases (e.g. one for read, another for write);
• should randomize requests if multiple "free" nodes found;

OPEN QUESTIONS

Should we support <databases></databases> tag with new configuration schema?