Library release structure and policy

Under:

Introduction

The below policy is a revised document from the initial policy instated in STAR at the beginning of its software development phase in February 2000.

Four canonical versions of the STAR software are provided at any given time. Those are dev, new, pro, and old. These canonical versions evolve in time i.e. it is understood that the content of (for example) new today is not fixed and likely to be different within some time frame (and following the above release policy and procedure). Two additional areas are described for completeness but are NOT canonical in the sens that their usage and meaning may evolve without prior warning nor discussion. Purpose and policy for each version is described below.

The official STAR software releases are otherwise named SLXXY where XX is a year number and Y is a letter from a to z. Those releases are fixed (and should not be changed or patched unless exceptional circumstances not described in this policy - patch should be in those cases rare and under a strict control and validation procedure).

There are no other official areas than the one mentioned herein. Please, consult the table below for additional information.

Notes, primary and secondary environment


The supported STAR software platforms, October 2017:

  • Primary Environment: Scientific Linux 7.3 using gcc 4.8.5 version supporting 32 bits (mandatory for code developers ; compilation in optimized and non-optimized). Goal is to enable 64 bits under this OS.
  • Secondary Environment: Scientific Linux 6.4 using gcc version 4.8.2/32 bits; we assume no necessary support for 64 bits and only the non-optimized version will be assembled.
  • As before, switching to the 64 bits environment is achieved by using the command '% setup 64bits' and similarly, '% setup 32bits' - however, the 64bits environment is still not expected to work properly (Geant3 based simulation will not work, ROOT based analysis will).


Previous support from summer 2015:

  • Primary Environment: Scientific Linux 6.4 using gcc version 4.8.2/32 bits (mandatory for code developers ; compilation in optimized and non-optimized
  • Secondary Environment: Linux RedHat 5.3 using gcc  4.8.2 - unification of compiler should make integration easier.

Past support before 2015:

  • Primary Environment: Scientific Linux 6.4 using gcc version 4.4.7/32 bits (mandatory for code developers ; compilation in optimized and non-optimized)
  • Secondary Environment: Linux RedHat 5.3 using gcc  4.3.2 (recommended for code developers ; compilation requirement reduced to non-optimized mode)


The supported STAR software platform as per 2009/2011 were:

  • Primary Environment: Scientific Linux 5.3 using gcc version 4.3.2/32 bits (mandatory for code developers ; compilation in optimized and non-optimized)
  • Secondary Environment: Linux RedHat 4.4 using gcc  3.4.6 (recommended for code developers ; compilation requirement reduced to non-optimized mode)
  • Other OS/Compiler used
    • Scientific Linux 5.3, gcc 4.3.2/64 bits mode - please, use the command '% setup 64bits' to switch to this environment and try compiling. As per April 20th 2011, the count-down for making this alternate compilation one of the primary environment had begun. Policy effective due date: July 20th.
    • Scientific Linux 5.3, gcc 4.5.1/32 bits may be used at a later time to leverage advanced C++ features (SIMD, Threads)


While multi-platform and/or multi-compiler impose a constraint on the developer, the procedure ensures portability and easier evolution to later version of the compiler. Time given in the library definitions section are in EST time, i.e. they correspond to the Tier0 (BNL) site local times.
 

Library definitions

Note: Infrastructure developers are mandated to work from private areas and not directly from a deployed canonical version (unless otherwise mentioned). However, full testing of a given engineered solution, ready to be deployed, may require a full library re-compilation. Whenever needed, this shall not occur without approval, advanced notice and Email announcement by the software librarian.


Version

.dev and adev

Usage and scope

Leading edge development versions, for core software developers only

Guarantees

While we appreciate feedback, there are absolutely no functional guarantees (compilation or runtime) for those areas. If you chose to work with this area, it is at your own risk.

Description

Changes to code by the core software group do not go directly into dev, but into one of .dev or adev within the following restrictions/scope:
1. For infrastructure changes (such as ROOT version changes and evaluation, new features, interface changes and developments etc ...) .dev is used which NO ONE outside the core infrastructure group should be using. .dev is a main area of work used by the core infrastructure team and manipulation of code in this area do not require approval from the Software librarian.
2. If a patch deployment and testing to dev (and as described in the next section) OR a minor adjustment not requiring compilation and test are needed, adev may be used for its testing and evaluation. In all case, the operation must be completed before 11 PM every day. The tight relationship between adev and dev is described above. Changes are restricted to either the Software Librarian, the Infrastructure leader or higher authority.

Patching Policy

N/A in .dev
Must be complete by 11 PM for adev

Release Policy

For adev, code must compile on the primary and secondary platforms.

At the end of the first and third week of each month (on Thursdays and whenever applicable), major changes to .dev may be checked in CVS and consequently propagate into dev by the next day. These commits should be made only after successful testing of .dev . The new dev environment is tested against code developers activities for the following week allowing for subtle adjustments not initially envisioned.


Version

dev

Usage and scope

Version containing the latest developments of code modules in the repository.

Guarantees

This environment was released upon successful compilation of approved code on the main supported production platform and compiler in both optimized and non-optimized. There are no compilation guarantees for other compiler/platforms.

Description

Although code is this area are likely to be buggy (or at least, not fully tested), they contain the latest algorithms and structures being worked on by the user and sub-system developers.

Submission of code to the repository is frequent (daily). An effort to exclude major technical problems must be made by developers, but it is understood that behavior of code can not be all guaranteed. However, all code checked in by developers should be tested for basic operation prior to check-in in CVS that is,
1. Compiling and running using the dev library and the mandatory supported platform and/or supported compilers
2. Basic test conducted by the developers for assuring the code actually runs
3. QA histograms are affected in the correct way or developer test suite shows satisfactory result and/or improvement.

Note:
Since 2002, dev is also used during data acquisition periods for regular and automated snapshot productions called FastOffline productions. While the same quality assurance procedure applies (must compile prior to check-in, must check if it runs), and emphasis is made to preserve sanity of this area at all cost. Especially, committing to CVS code/libraries not previously checked for compilation will lead to its removal from daily compilation as well as from any chain related to that code and used in the FastOffline production. Removed code/libraries will not be put back into operation (compilation and production) until code is demonstrated to be compilable and stable that is, passing the basic quality assurance requirement as stated above.

The following schedule is used for updates to dev: it is updated automatically by the AutoBuild script starting from the adev area and the procedure will start daily at 11 PM. This procedure do not send any Email notice automatically. It will however execute the following steps:
1. Every day at 11 PM, AutoBuild updates all code currently available in dev with its (newer) counterpart available in the CVS repository and as committed during the day. There is NO automatic check out of new code, code trees or libraries: they are added on a per request basis upon which they are susceptible to all quality and policies requirements previously stated.
2. The AutoBuild procedure attempts to compile in non-optimized and optimized mode respectively under the primary environment conditions. This operation is transparent to the users of dev.
3. Shall the compilation succeed, dev is “released” that is, the area become automatically visible to the users. Note that this does not warrant a working library as checks are the responsibility of the developers.
4. Other supported platform and/or compilers compilation are made after a successful primary environment based release which, in turn, implies a 24 hours behind update on the secondary environment (and other platforms). Secondary environment functional operation are required.

Functionality tests of dev are made every night by running a small number of events (~ 50) through the most common full production chains and detector geometries on the primary environment. Several data output format will be provided (supported data-summary-tape format, histograms, Micro-DST and/or other supported formats, ...) for both real and simulated data and span over a range of data source (centrality, species, etc ...). Such provision may be based on the result of the nightly tests, test suite made at Tier0 and Tier1 sites as achievable. Those tests are targeted toward eliminating gross errors, possible remaining crashes, obvious memory leaks etc ...

Patching Policy

Bugs will be reported by the QA or Software Librarian to the software code/ component/ sub-system responsible person. The given component should be fixed within a week as well as within development guidelines specified above.

Release Policy

On the second and forth week of each month (on Thursdays and whenever applicable), dev is tested with a larger (and more complete) test suite over the week-end. The test suite would remain the same apart from the statistics (at least 200/500 events). Basic test should be successfully run on the secondary platform. Shall all tests perform according to expectations, the content of dev is a candidate for propagation and re-build of new. Thus, under normal circumstances, dev is moved to new on a bi-weekly basis.

Note that there are no mention of tagging at this stage although it may be more practical to use this mechanism for code propagation.

The migration of dev to new should be discussed at the Software and Computing meeting on Wednesday and requires positive approval from the Software Librarian and the QA/QC team (validation tests must pass).

Shall it be approved, the migration may proceed the following day.


Version

new

Usage and scope

A relatively stable (weeks old) version for full integration testing, detailed QA, and public use by people needing a near leading edge version of the code.

Guarantees

Code compiles on at least two platforms and/or compiler versions and has passed basic nightly tests QA criteria.

Description

A new version is released to allow further tests in a relatively stable and QA-ed environment. More developed QA procedures may include a desired set of on-going user analysis and sample productions.

Patching Policy

Identified bugs and engineering of a solution addressing a given problem may be using the new environment for stability purposes. In such case:
(a) relevant test must be successful in new
(b) the solution MUST also be evaluated in the dev environment prior to commits into CVS

Release Policy

Within the next two weeks period (as driven by the migration dev to new), this area may (or may not) be migrated to pro as follow:
1. If all quality checks on all supported platforms and compilers are passed successfully and no major problems are reported by users (analysis, developers), and after discussion and prior announcement, it will be released as the pro version. Hence, following the migration of dev, this migration from new to pro may occur as frequently as once every two weeks.
2. If a data production has been announced, a base sample as described by the QA procedure will be made in new. Shall the time period for PWG approval be passed without major functional problems encountered, new may become pro without further modifications.
3. If a data production and migration to pro is envisioned, embedding runs must be tried in that version, necessary macro fully operational.

Shall the tests not be successful or satisfactory (unexpected results from QA), either a patch will be engineered or the new version will be replaced at the next scheduled migration of dev to new. Therefore, new changes on a more rapid cycle than the pro version, so any given new version may or may not make it to pro.

A version that is not migrated to pro will be kept for a short period of time (of the order of a week) to allow people to migrate away from it on their own schedule. During this period it can be used via % starver <em>version</em> command, e.g. % starver SL04a.

If the bug fixes are successful an official tag is generated for the release. If not, a tag does not have to be generated until the next dev to new release, as long as this will happen within the next two weeks. If production have been run from the release, it must be tagged for reproducibility.


Version

pro

Usage and scope

The current production version

Guarantees

All components of the software are tagged and frozen for a production release, so that the release uniquely defines a stable, reproducible configuration.

Description

This level is a tagged and QA-ed version of the software library. It is used by production. The pro version is created from the latest new version at the time of the release.

Patching Policy

Whenever production has started, the only changes to a pro release will be to fix important bugs that impair functionality supposed to work in the release. Such fixes will be announced, discussed, documented immediately and scheduled with advance notice. Such fixes must be exceptional (and typically related to a critical component such as IO) as Physics content must identified prior to this stage. Should a grave Physics issue be identified, a new pro version should be released from new (see next paragraph).

A new migration from new to pro is preferable to a retag of pro. However, shall the CVS retag be necessary (such case may arise if new contains code functionally incompatible with pro), the corrected versions will be identified by adding "_n" to the version number (e.g. SL04a_1 for the first corrected version). “n” will be an incremental number depending on the previous patch level. The tags and sub-sequent related tags will be clearly documented. All code (functional and patched) will be re-tagged with the XXX_n level tag, thus, leading to a consistent bulk CVS tagged library version. A patch made in pro must be applied to the dev and new versions (same compatible patch or modified for the more recent environment to correct the targeted problem). All corrections to pro version should also be applied to the current dev,new versions in an appropriate way.

Modifications will never be made to a production release to add new functionality, change base code (ROOT or base classes) or fix functionality not yet operational when the release was made. If users want to have access to new functionality that has arrived since the release, they either need to use a newer version (new, dev) or wait for a new pro release containing the functionality.

Shall the procedure for QA procedure mention for the release of new be used for the migration new to pro, production cycle should not be restarted but a new later release of pro scheduled. Production would then resume from the new release, created following the same new to pro release policy.

Release Policy

When a pro version is retired, becomes the old version.

Proposed change

“any change to the production code should be reflected in the library tag”

Version

old

Usage and scope

The previous successful production release.

Guarantees

All quality checks mandated by pro.

Description

Previous old versions can be retrieved by the CVS tag. Old versions are not supported any longer (there are no bug fixes applied) but are preserved and retrievable by the software librarian.

Production level sub-system code or core software infrastructure developments should NOT be done with any old version of the library

Patching Policy

Patches are NOT allowed at this level

Release Policy

There are no existing canonical releases beyond that level