Offline DB performance study

Offline Database is READ-heavy (99% reads / 1% writes due to replication), therefore it should benefit from various buffers optimization, elimination of key-less joins and disk (ram) IO improvements.

Typical Oflline slave R/W percentage could be seen here :
db08 performance profile

First results :

================================================================================
 Sample | 50 BFC events, pp500.ittf chain 
--------+----------------+------------------------------------------------------ 
 Host   | dbslave03.star.bnl.gov:3316 - Offline 
--------+----------------+------------------------------------------------------ 
 Date   | Fri, 08 May 2009 16:01:12 -0400 
================================================================================

 SERVER GLOBAL VARIABLES

================================================================================
 Variable Name	 |	Config Value	| Human-Readable Value
================================================================================
 preload_buffer_size 	 |	32768		| 	32 KB 
-------------------------+------------------------------------------------------
 read_buffer_size    	 |	1048576		| 	1 MB 
-------------------------+------------------------------------------------------
 read_rnd_buffer_size	 |	262144		| 	256 KB 
-------------------------+------------------------------------------------------
 sort_buffer_size    	 |	1048576		| 	1 MB  
-------------------------+------------------------------------------------------
 myisam_sort_buffer_size |	67108864	| 	64 MB  
-------------------------+------------------------------------------------------
 join_buffer_size	 |	131072		| 	128 KB  
-------------------------+------------------------------------------------------
 thread_stack		 |	196608		| 	192 KB  
================================================================================
 key_buffer_size	 |	268435456	| 	256 MB  
-------------------------+------------------------------------------------------
 query_cache_size	 |	33554432	| 	32 MB  		
-------------------------+------------------------------------------------------
 max_heap_table_size	 |	16777216	| 	16 MB  
-------------------------+------------------------------------------------------
 tmp_table_size	 |	33554432	| 	32 MB  
================================================================================
 bulk_insert_buffer_size |	8388608		| 	8 MB  
-------------------------+------------------------------------------------------
 concurrent_insert	    |	1		| 	1  
================================================================================

 PERFORMANCE PARAMETERS

================================================================================
 PARAMETER		 |	Measured Value	| 	Acceptable Range
================================================================================
 Total number of queries | 	6877 		| 	> 1000. 
-------------------------+------------------------------------------------------
 Total time, sec  	    |	1116		| 	any
-------------------------+------------------------------------------------------
 Queries/second (QPS)	    | 	6.16		| 	> 100.
-------------------------+------------------------------------------------------
 Queries/second - SELECT | 	5.89		|	> 100.0	 
-------------------------+------------------------------------------------------
 Queries/second - INSERT | 	0.17		|	any	 
-------------------------+------------------------------------------------------
 Queries/second - UPDATE | 	0		|	any	 
-------------------------+------------------------------------------------------
 Query cache efficiency  | 	45 %		|	> 20%	 
-------------------------+------------------------------------------------------
 Query cache overhead    | 	6 %		|	< 5%	 
-------------------------+------------------------------------------------------

Initial test revealed strange things: there seem to be too many queries (~7000 queries for only two real db sessions??!), and Query Cache efficiency is 45%! Logs revealed the mystery - we somehow allow many repetitive queries in our API. Further investigation found more issues - see below.

a) Repetitive ID requests (they are queried one after another, no changes or queries in between):
....
3 Query       select elementID from ladderIDs                                                                       
3 Query       select elementID from ladderIDs
....
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs                                                                   
3 Query       select elementID from CorrectionIDs
....
3 Query       select elementID from SectorIDs where Sector=1                                                        
3 Query       select elementID from SectorIDs where Sector=1                                                        
3 Query       select elementID from SectorIDs where Sector=1                                                        
3 Query       select elementID from SectorIDs where Sector=1
....
3 Query       select elementID from HybridIDs where Barrel=1 AND Ladder=1 AND Wafer=1 AND Hybrid=1                  
3 Query       select elementID from HybridIDs where Barrel=1 AND Ladder=1 AND Wafer=1 AND Hybrid=1                  
3 Query       select elementID from HybridIDs where Barrel=1 AND Ladder=1 AND Wafer=1 AND Hybrid=1                  
3 Query       select elementID from HybridIDs where Barrel=1 AND Ladder=1 AND Wafer=1 AND Hybrid=1                  
3 Query       select elementID from HybridIDs where Barrel=1 AND Ladder=1 AND Wafer=1 AND Hybrid=1

c) Repetitive date/time conversions via mysql function, like those below :
4 Query       select from_unixtime(1041811200) + 0 as requestTime                                                   
4 Query       select unix_timestamp('20090330040259') as requestTime

b) Repetitive DB Initializations for no real reason :
3 Init DB     Geometry_tpc                                                                                          
3 Init DB     Geometry_ftpc                                                                                         
3 Init DB     Geometry_tof                                                                                          
3 Init DB     Geometry_svt                                                                                          
3 Init DB     Geometry_ssd                                                                                          
3 Init DB     Geometry_tpc
(that's what was need to do a query agains Geometry_tpc database)

Full logs could be found in attachment to this page.