The Offline Conditions DB framework
Last Updated: October 19th, 2009 (F. Carminati)
Content
- Important links
- Introduction
- CDB access framework
- AliCDBEntry, AliCDBId, AliCDBMetaData
- Access to the OCDB. AliCDBStorage's Put(), Get() and GetAll() functions
- Storage and retrieval with AliCDBManager
- NEW! - Local caching of remote OCDB files
- OCDB access in Simulation and Reconstruction
- CDB tags - Access to the file catalog with tags
- General guidelines for OCDB objects
- Backward compatibility - Policy for CDB objects
Important links
ALICE Detector Calibration Requirement Tables and related documentation
ALICE Calibration and Alignment status table (MS PowerPoint slide)
ALICE Calibration and Alignment status Reports (MS Word document)
Intoduction
The Offline Conditions Database (OCDB) is the place where the calibration and alignment data is stored. It is not a "database" in the literal sense of the word (like Oracle, MySQL, etc): it is a set of entries in the AliEn file catalog that point to the physical entities (ROOT files stored in the various storage elements of the grid) containing the calibration and alignement data.
The organization of the database is handled by the CDB access framework, an AliRoot-based package.The OCDB was designed under the following principles:
- The calibration and alignment database contains ROOT TObjects stored into ROOT files;
- Calibration and alignment objects are RUN DEPENDENT objects;
- The database is READ-ONLY (automatic versioning of the stored objects);
- The objects in the OCDB are univocally identified by:
- a ( logical ) path name (path of the file in the AliEn file catalog);
- a validity expressed as a run range ;
- a main (grid) version number;
- a local subversion number, only for locally stored objects.
The local subversion was introduced to avoid version clashing during tranferral of OCDB object from Grid to local storages and vice versa. In local storage only the local version is increased, while in Grid storage only the Grid version is increased. When the object is transferred from local to Grid the Grid version is increased by one; when the object is transferred from Grid to Local the Grid version is kept fixed and the subVersion is reset to zero. The adopted versioning schema is shown in the following picture.

CDB access framework
The schema of the AliROOT-based CDB access framework is shown in the next picture.

-
a GRID folder in the file catalog containing logical file names, each one pointing to a Root file. Each calibration or alignment object is contained in a single ROOT file;
-
a LOCAL folder containing Root files, each one containing one single Root object, with a structure similar to the Grid one;
-
a local Root file ( DUMP ) containing one or more objects. The objects are stored into Root TDirectories defined by the object's name and run range.
Storage activation: AliCDBManager
Grid storages
-
grid (the grid host)
-
user (the user name)
-
folder (the path of the base folder)
-
se (the storage element to be used for data storage)
-
cacheFolder (the local folder used to cache the remote files: for the description of this functionality and of its further parameters see later)
Local storages
Dump storages
Activation of Default, Specific, Drain storages
AliCDBManager's method SetDefaultStorage("const char* uri") is used to activate the particular storage used by the manager's Get and Put functions (see later). Example:
AliCDBManager::Instance()->SetDefaultStorage("alien://folder=<baseFolder>");
A pointer to the default storage is returned by the function GetDefaultStorage() , and the default storage is unset by the function UnsetDefaultStorage() . To check activation of the default storage use Bool_t IsDefaultStorageSet() .
AliCDBManager::SetSpecificStorage(const char* calibPath, const char* uri) allows to activate one or more "calibration-specific" storages. Specific storages are useful in case a detector expert wants to test a particular set of conditions data, stored (say) on a local disk, while leaving the reconstruction algorithms use the main OCDB locations for all the other objects.
The implementation of SetSpecificStorage goes through the mapping of the detector name (or calibration type) string with the parameters ( AliCDBParam object) describing the storage. When a conditions object is stored/retrieved using AliCDBManager 's Put/Get functions, the object path name is parsed and if a match in the list of the specific storages is found, then the corresponding storage is returned and used. If not, then the default storage (which should be activated first) is used. For more details, see later.
Examples of specific storage activation:
AliCDBManager::Instance()->SetSpecificStorage("TPC/*","alien://folder=<baseFolder>");
AliCDBManager::Instance()->SetSpecificStorage("ITS/Calib/DeadPixelMaps","alien://folder=<baseFolder>");
AliCDBManager::Instance()->SetSpecificStorage("*/Align/*","alien://folder=<baseFolder>");
AliCDBManager's SetDrain(const char* uri) activates the drain storage. When activated, the drain storage stores every object retrieved from the OCDB. The drain storage is unset by UnsetDrain() ; to check activation of the default storage use Bool_t IsDrainSet() .
AliCDBEntry, AliCDBId, AliCDBMetaData
-
The calibration/alignment object container class (anything inheriting from ROOT's TObject class);
-
The parameters that univoquely identify the conditions data, described by the class AliCDBId ;
-
The object's metadata , described by the class AliCDBMetaData .
-
SetObject(TObject*) , TObject* GetObject() ;
-
SetId(const AliCDBId&) , AliCDBId& GetId( ) ;
-
SetMetaData(AliCDBMetaData*) , AliCDBMetaData* GetMetaData() ;
-
PrintId() , PrintMetaData() ;
-
SetOwner(Bool_t) , Bool_t IsOwner() .
-
during storage it is used to specify the path and run range of the object;
-
during retrieval it is used as a "query": it contains the path of the object, the required run and (if needed) the version and subversion to be retrieved (if version and/or subversion are not specified the highest ones are looked for).
-
an object describing the name (path) of the object (AliCDBPath ). The path name must have a fixed, three-level directory structure: " level1/level2/level3" ;
-
an object describing the run validity range of the object (AliCDBRunRange );
-
the version and subversion numbers (automatically set during storage).
-
TString& GetPath(), SetPath(const char* path), Tstring& GetLevel(int lev): return the full object's path or one of its levels;
-
AliCDBRunRange& GetAliCDBRunRange() + run number setters/getters;
-
Note: "Infinity" in the OCDB is 999999999 and is returned by static function: AliCDBRunRange::Infinity() ;
-
-
version/subVersion getters/setters ( GetVersion( ) , etc.);
-
The object class name (automatically set during storage);
-
The name of the person who created the object (related methods: SetResponsible(const char*) , GetResponsible() );
-
The LHC beam period (related methods: SetBeamPeriod(Int_t*) , GetBeamPeriod() );
-
The AliROOT version used to create the object (related methods: SetAliRootVersion(const char*) , GetAliRootVersion() );
-
Any comment (related methods: SetComment(const char*) , GetComment() );
Access to the OCDB. AliCDBStorage's Put(), Get() and GetAll() functions
Creation and storage of a calibration object into the OCDB
|
AliZDCCalibData *calibda = new AliZDCCalibData(); // creation of the AliCDBId object (identifier of the object) // creation and filling of the AliCDBMetaData // Activation of local storage // put object into database using AliCDBStorage::Put method
sto->Put(calibda, id, md); |
/$HOME/CDB/ZDC/Calib/Pedestals/Run1_10_v0_s0.root
If an object with the same path name and overlapping run range is stored again, it will be automatically set a higher subversion, e.g.:
/$HOME/CDB/ZDC/Calib/Pedestals/Run1_10_v0_s1.root
/$HOME/CDB/ZDC/Calib/Pedestals/Run5_10_v0_s2.root
and so on. If the object is stored into a Grid OCDB then only the grid version is taken into account and incremented (starting from 1), e.g.:
/alice/data/CDB/ZDC/Calib/Pedestals/Run1_10_v1.root
/alice/data/CDB/ZDC/Calib/Pedestals/Run1_10_v2.root
To retrieve an object from the OCDB one can use AliCDBStorage 's method:
AliCDBEntry* Get(const char* path, Int_t runNumber, Int_t version=-1, Int_t subVersion=-1)
If the version/subVersion are specified then the object with the requested version/subVersion is looked for (and if it is not found Get() returns a null pointer), if they are not specified then the object with the highest version/subVersion is retrieved. The Get method looks for the valid ROOT file in the OCDB, opens it and returns the pointer to the AliCDBEntry stored inside.
|
// Activation of local storage AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage("local://$HOME/DBFolder"); // Get the AliCDBEntry which contains the object "ZDC/Calib/Pedestals", // valid for run 5, highest version AliCDBEntry* entry = sto->Get("ZDC/Calib/Pedestals",5); // specifying the version: I want grid version 2 AliCDBEntry* entry = sto->Get("ZDC/Calib/Pedestals",5,2); // specifying version and subversion: I want version 2 and subVersion 1 AliCDBEntry* entry = sto->Get("ZDC/Calib/Pedestals",5,2,1);
|
Retrieval of more objects with GetAll(const char* path, Int_t runNumber, Int_t version=-1, Int_t subVersion=-1)
GetAll() returns a TList containing all the retrieved AliCDBEntry 's. Example:
Tlist *list = sto->GetAll("ZDC/*",5);
AliCDBEntry *entry = dynamic_cast<AliCDBEntry*> (list->At(0));
Selection criteria
|
// storage activation
AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(...);
// I want version 2_1 for all "ZDC/Calib/*" objects for runs 1-100
sto->AddSelection("ZDC/Calib/*",1,100,2,1);
// and I want version 1_0 for "ZDC/Calib/Pedestals" objects for runs 5-10
sto->AddSelection("ZDC/Calib/Pedestals",5,10,1,0);
AliCDBEntry* entry = 0;
entry = sto->Get("ZDC/Calib/Pedestals", 5); // object with version 1_0 is retrieved
entry = sto->Get("ZDC/Calib/GainFactors", 5); // object with version 2_1 is retrieved
entry = sto->Get("ZDC/Calib/Pedestals", 25); // object with version 2_1 is retrieved
entry = sto->Get("ZDC/Calib/Pedestals", 200); // object with higest version is retrieved
|
-
RemoveSelection(...) : See AliCDBStorage's header for all possible overloads of this function;
-
RemoveAllSelections() ;
-
PrintSelectionList() .
Further AliCDBStorage functions: GetLatestVersion( ), GetLatestSubVersion( )
Two further AliCDBStorage functions:
-
Int_t GetLatestVersion(const char* path, Int_t run);
-
Int_t GetLatestSubVersion(const char* path, Int_t run, Int_t version=-1);
return the latest version/subVersion found in OCDB for object with path " path " and valid for run " run ". If no valid object is found the functions return -1. Example:
AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(...);
Int_t version = sto-> GetLatestVersion("ITS/Calib/Data",5);
Storage and retrieval with AliCDBManager
|
// initialization
AliCDBManager *man = AliCDBManager::Instance();
man->SetDefaultStorage("alien://folder=/alien/data/OCDB");
man->SetSpecificStorage("*/Align/*", "alien://folder=/alien/data/align_OCDB");
man->SetSpecificStorage("ITS/*", "local://ITS_OCDB");
// object retrieval
AliCDBEntry *entry;
entry = man->Get("TPC/Calib/Data", 5); // Default storage is queried
entry = man->Get("TPC/Align/Data", 5); // specific storage "align_OCDB" is queried
entry = man->Get("ITS/Calib/Data", 5); // specific local storage "ITS_OCDB" is queried
|
AliCDBManager's "centralized" run number
AliCDBManager 's function SetRun(Int_t) can be used to specify a unique run number in the manager. If the centralized number is set the run number may be not specified in AliCDBManager::Get() . If it is specified the centralized run number is not used for retrieval. Example:
|
// initialization, set centralized run number
AliCDBManager *man = AliCDBManager::Instance();
man->SetDefaultStorage("alien://folder=/alien/data/OCDB");
man->SetRun(10);
// object retrieval
AliCDBEntry *entry;
entry = man->Get("TPC/Calib/Data"); // Object valid for run 10 is retrieved
|
AliCDBManager's cache
-
Cache must be active.
-
Activate it with SetCacheFlag(Bool_t)
-
By default the cache is ON.
-
-
Set a centralized run number in the manager with SetRun(Int_t) .
|
// initialization, set centralized run number and activate cache
AliCDBManager *man = AliCDBManager::Instance();
man->SetDefaultStorage("alien://folder=/alien/data/OCDB");
man->SetRun(10);
man->SetCacheFlag(kTRUE);
// object retrieval
AliCDBEntry *entry;
entry = man->Get("TPC/Calib/Data"); // Object is cached
entry = man->Get("TPC/Calib/Data"); // Object returned from cache
entry = man->Get("TPC/Calib/Data", 20); // Object is queried from OCDB
|
Local caching of remote OCDB files
-
operateDisc=<kTRUE, kFALSE, 0, 1> - default kTRUE
-
cacheSiz=<size in bytes> - default 1GB
-
cleanupInt=<interval in seconds> - default 0
OCDB access in Simulation and Reconstruction
-
try out the CDB framework with "reasonable" data. That is: decide which are the quantities that compose the calibration information for each subdetector;
-
try to fill the framework with reasonable calibration data; store and retrieve them;
-
Simulate "uncalibrated" data, apply calibration and reconstruct them;
-
Simulate "decalibrated" data, retrieve and calibrate them and then reconstruct.
How to implement simulation/reconstruction code? A simple example
|
AliZDCCalibData* AliZDCReconstructor::GetCalibData() const
{
// Getting calibration object for ZDC set
AliCDBEntry *entry = AliCDBManager::Instance()->Get("ZDC/Calib/Data");
if(!entry) AliFatal("No calibration data from calibration database!")
AliZDCCalibData *calibdata = (AliZDCCalibData*) entry-> GetObject();
if (!calibdata) AliFatal("Wrong calibration data from calibration database !");
return calibdata;
}
|
|
Table name: CDB |
|||||
|---|---|---|---|---|---|
| first_run (int) |
last_run |
version |
path_level_0 |
path_level_1 |
path_level_2 |
|
Table name: CDB_MD |
||||
|---|---|---|---|---|
|
object_classname
(varchar(255))
|
responsible
(varchar(255))
|
beam_period
(int)
|
aliroot_version
(varchar(255))
|
comment
(varchar(255))
|
AliCDBStorage's QueryCDB()
|
AliCDBStorage *sto = AliCDBManager::Instance()->GetStorage(...);
sto-> QueryCDB(5); // queries the OCDB file catalog for all files valid for run 5
sto-> QueryCDB(5,"ITS/*"); // look for files valid for run e and for path "ITS/*"
sto-> QueryCDB(5,"ITS/*", 1) ; // same as above, but look for files valid for version1 only
AliCDBMetaData md;
md.SetResponsible("Duck, Donald");
sto-> QueryCDB(5,"ITS/*", 1, &md); // use CDB_MD tags. Look for files created by Duck, Donald.
|
|
AliCDBManager* man = AliCDBManager::Instance();
man->SetDefaultStorage("alien://folder=mainOCDB");
man->SetSpecificStorage("ITS/*","alien://folder=ITS_Folder");
man->SetRun(5); // Default and specific storages are queried for files valid for run 5 now!
|
General guidelines for OCDB objects
Every object derived from a TObject can be stored in an AliCDBEntry and become an OCDB object. A part from this necessary requirement, a few guidelines should be followed as good practice in designing OCDB objects.
- Avoid pointless growing in the size of the objects; that would result in increased size occupied in memory and bigger OCDB folders. To this purpose:
- chose pointers to objects instead of objects as data members;
- use types adequate to the precision of the CDB information saved in the given data member: e.g. no need for Double_t if the precision is 1%, etc.;
Backward compatibility - Policy for OCDB objects and related classes
Changes in classes whose instances are stored in OCDB objects can introduce backward incompatibilities. This means that a given AliRoot version is not able to open the object extracted from the AliCDBEntry in a previously populated OCDB folder. Backward incompatibility is extremely harmful, since it limits the scope and validity of OCDB base folders to one or few AliRoot releases. For this reason the writing of code leading to backword incompatibility has been strongly discouraged up to v4-12-Release; starting with v4-13-Release a strict policy has been introduced to completely ban backward incompatibilities. The policy is summarised here below:
-
Backward-incompatible changes of classes which are stored in CDB objects are not allowed. This means in particular:
-
-
Removal of members is not allowed
-
Changing the type of members is not allowed
-
Addition of members is instead allowed. If the added member is meant to substitute an old one, for the addition to be backward-compatible there are two options:
- Comply to the following instructions concerning old and new data members in the new class:
- keep the old member and add the new one as a different one (i.e. with a new name);
- provide in the constructor of the new class a default value for the new member so that the new member is set also when reading old objects;
-
in new objects let the old member point to a null pointer;
- let the getter return the member that is available (the old one if non-zero, the new one otherwise).
The instructions above allow to read the old objects without increasing the size for new objects. Alternatively the new class can provide a convert function, which sets the new member based on the old member, if it is non-zero.
-
Alternatively, write a custom streamer that is able to read the old and the new data. An example of how to write a similar streamer can be found in the root users guide (Input/Output chapter, section "Migrating to ROOT 3", page 183).
- Comply to the following instructions concerning old and new data members in the new class:
-
-
Removal of CDB objects from default OCDBs, in particular from the repository is not allowed
-
Addition of CDB objects to default OCDBs, in particular to the repository is allowed
In addition, it is important to motivate the need for modifying the OCDB objects both in comment lines inside the class itself and in the comment bound to the corresponding svn commit.
Let's furthermore emphasise that in case of a future replacement, the guidelines given above to keep small the size of the OCDB objects will become even more important.
