The model of development using CMT is rather similar to that when using SRT, although many of the details are different. The basic model is to create a test release, into which the package or packages under development can be checkout from CVS, modified, tested and later checked back into CVS.
This note describes first how to create a test release, how to develop within one, for the situations where a single or multiple packages have been checked out, and how to establish a run-time environment for performing testing.
The other important scenario which will be addressed later on describes how to work out a bug fix to be applied to an already tagged and published version of a package.
The contents of this note will be subject to modifications, at least in the short term, until the conventions described here become stable. We expect such tuning to occur especially in the early days of the migration towards CMT. Therefore please do not consider the conventions or suggestions described in this document as being yet the final and definitive Atlas policy.Moreover, the feedback and contributions from users are highly welcome so as to construct as efficiently as possible the work model and policy.
The recommended method of establishing a login environment that supports CMT is to create a requirements file in your home directory. A suitable file is located in the doc/requirements.login file of the TestRelease package described in more detail later. [We should add this package to the list of packages that form an Atlas release.]
Copy this template requirements file to your home directory
cp <dir>/TestRelease/TestRelease-*/doc/requirements.login ${HOME}/requirementsModify it to suit your local environment. Then, once only, configure your environment:
source /afs/cern.ch/sw/contrib/CMT/v1r9/mgr/setup.[c]sh [1][2] cmt configThen, either interactively, or in your login script, do one of:
source setup.[c]sh source setup.[c]sh -tag=egcs source setup.[c]sh -tag=gcc source setup.[c]sh -tag=sunDepending on which platform or compiler you wish to use. The default when -tag=<tag> isn't specified is egcs on a Linux machine, or CC on a Solaris machine.
If you modify the requirements file, run:
cmt configto rebuild the setup.[c]sh file.
Notes:
- This location is site-specific. Ask your local administrator for the location of CMT at your site.
- Use setup.sh or setup.csh according to your login shell.
Decide on the location of your test release and create a suitable directory if one doesn't already exist:
> mkdir <dir> > cd <dir>Checkout the desired packages from CVS. It is recommended that CMT is used to do this since that will ensure that the correct directory hierarchy is created:
> cmt co [-r] <pkg> e.g.
> cmt co Generators/GeneratorModules [1] > cmt co -r GeneratorModules-00-00-30 Generators/GeneratorModules [2] > cmt co -requirements Generators/GeneratorModules/GeneratorModules-00-00-30/cmt/requirements [3]Notes:
- This checks out the package from the head of CVS.
- This checks out the specified tag of the package.
- This checks out the packages and versions that are directly referenced in the specified requirements file.
- If the CMTCVSOFFSET environment variable hadn't been setup, the commands would have had to have been of the form:
> cmt co offline/Generators/GeneratorModulesand another tier in the directory hierarchy (offline/) would be created.
The CMTCONFIG environment variable is used to determine the platform, compiler version, and build options (e.g. optimized, debug, profiled). If you are using the login requirements file described in Section 2, it is configured to allow you access to the Linux egcs and gcc compilers, and the Solaris CC compiler.
However, explicit modification of the CMTCONFIG environment variable allows you to access other configurations. Currently, valid values of CMTCONFIG are:
i386_linux22 [1] Linux-gcc [2] sun4x_57 [3] [more...]Notes:
- This is the debug, unoptimized Linux egcs compiler.
- This is the debug, unoptimized Linux gcc 2.95.2 compiler.
- This is the debug, unoptimized Solaris CC compiler.
The CMTPATH search path is used by CMT to bind together partial releases. In this case it is used to bind together the test release and the underlying base releases. If you have used the login requirements file as recommended in Section 2, it sets up the CMTPATH variable automatically to what's described here.
In the following, the names of the environment variables (apart from CMTPATH itself) have no particular significance, they are just a convenience:
> export TestArea=<dir> > export AtlasArea=/afs/cern.ch/atlas/software/dist/<rel> > export GaudiArea=/afs/cern.ch/atlas/offline/external/Gaudi/0.8.1 > export CMTPATH=${TestArea}:${AtlasArea}:${GaudiArea}or
> setenv TestArea <dir> > setenv AtlasArea /afs/cern.ch/atlas/software/dist/<rel> > setenv GaudiArea /afs/cern.ch/atlas/offline/external/Gaudi/0.8.1 > setenv CMTPATH ${TestArea}:${AtlasArea}:${GaudiArea}Where <rel> is the base release number (e.g. 2.2.0).
Any package that appears in an upstream fragment of CMTPATH will hide the equivalent package in a downstream fragment. The first fragement is generally the location of the test release, the second in this case is the main ATLAS release, and the third is the Gaudi release that Athena is based on. Potentially other partial releases could be present in the search path, but this is the current minimal configuration.
Go to the cmt directory within the package to be developed:
> cd <dir>/<pkg>/<tag>/cmtwhere <pkg> is the package name (e.g. Generator/GeneratorModules) and <tag> is the tag (e.g. GeneratorModules-00-00-30). The relevant commands for compiling and linking this package are:
> cmt config [1] > gmake clean [2] > gmake [3]Notes:
- This is the equivalent of "../src/configure" in the SRT world and only needs to be done if there is a change in the dependencies.
- This is the equivalent of "gmake clean" in the SRT world.
- This is the equivalent of "gmake install" in the SRT world. There is no equivalent to the SRT installation phase.
If multiple packages are checked out within the test release, it is possible that there are no direct dependencies between them. One could either treat them as multiple independent packages and build them all separately as discussed above, or you can setup an environment to allow you to build them all together. The easiest way of doing this is to use the TestRelease package that is provided to help you build multiple packages in a test release. Checkout the TestRelease package from the head of CVS (this should always be functional):
> cd <dir> > cmt co TestRelease > cd TestRelease/TestRelease-*/cmtThen modifiy the skeleton requirements file that is provided in this package to add the packages that you have checkout in your test release. It should be fairly obvious where to do this in the file since it's been annotated and examples are commented out. For example, if you've checked out the GeneratorModules package as in the previous example, add the following line:
use GeneratorModules GeneratorModules-* GeneratorsThen use the "cmt broadcast" command to broadcast the specified commands to all packages in the first fragment of the $CMTPATH, which in this case is your test release.
> cmt broadcast cmt config > cmt broadcast gmake clean > cmt broadcast gmake
CMT provides mechanisms for creating a run-time environment. Within the cmt directory of each package are files setup.csh and setup.sh. A run-time environment can be established simply by:
> source setup.shor
> source setup.cshUse whichever is appropriate for your shell.
This will also setup your LD_LIBRARY_PATH and PATH appropriately. However, because of limitations of the allowed length of LD_LIBRARY_PATH in particular, CMT uses symbolic links to reference shared libraries. These links are created using the "gmake" command, and therefore in in general, one should run "gmake" in a package at least once, together with sourcing the appropriate setup.[c]sh script during each login session.
Note however that the run-time environment is not necessarily the same as that established by the normal package dependencies. For example, in order to establish the athena run-time environment, it is necessary that a dependency to the Control/AthenaCommon package exists. This might not be the case for any of the packages that are being developed. In this case, the easiest way of establishing a run-time environment is to use the TestRelease package described in the previous section. Add the relevant "use" statements within this package in order to setup a run-time environment. Examples are commented out in the requirements file. For example, in order to run the Atlfast fast simulation, you could add the following lines:
use Control Control-* use Generator Generator-* use Atlfast Atlfast-* SimulationThen the commands for establishing a run-time environment are:
> cmt config > gmake [1] > source setup.[c]sh [2]Notes:
- It is important that "gmake" is run on this package, in case library symbolic links need to be established.
- This command sets up the run-time environment.
Note that the same directory can be used to control building of the test release and the establishment of a run-time environment.
Dependencies between package are specified through "use" statements within the requirements files. Such statements also specify the versions of the packages, although wildcards can be used to weaken the relationship. ATLAS recommends the following relationships:
- A. General dependencies between one package and another should be expressed in the form of:
use OtherPackage OtherPackage-00-* OtherPackageParente.g.use GeneratorModules GeneratorModules-00-* Generators- B. Container packages should depend upon the exact versions of their children packages. Thus the Generators package should list its children using the format:
use GeneratorModules-00-00-31 Generators use GeneratorObjects-00-00-31 Generators [etc.]
Several standard patterns have been created to ease the job of the package writer. They cover the situation for most standard packages; those that create linker or component libraries, or applications.
Component Library
A package that creates a component library in the Gaudi sense (e.g. Algorithms, Services or Converters) should use one of the following standard patterns:
apply_pattern component_interface [1] apply_pattern component_implementation [2] apply_pattern component_library [3]Notes:
- This applies to a package that just specifies the interface (via header files) to a set of components.
- This applies to a package that just implements a set of components (and will therefore have a "use" relationship with another package that specifies the interface).
- This applies to a package that simultaneously specifies both the interface and implementation of a set of components.
Linker Library
A package that creates a library that needs to be linked against, typically dynamically at run-time), should use the following standard pattern:
apply_pattern installed_libraryApplication
A package that creates an application should use the following pattern:
application <name> <list of files>e.g.application athena AthenaMain.cxxThen it may also define an alias to this application using the following pattern:
apply_pattern application_alias application=xyz [1]
- This defines by default the alias xyz pointing at the xyz.exe application expected in the binary directory of the package.
First comments:
+ I think this idea of "systematically" build a temporary container
package is very good. In principle instead of :
> cd
> mkdir TestRelease
> mkdir TestRelease/cmt
> cd TestRelease/cmt
> cat > requirements
it might be possible to do :
> cd
> cmt create TestRelease v1
> cd TestRelease/v1/cmt
> vi requirements
I have to understand the kind of persistency is applied to this
TestRelease. It seems to be volatile but...
What you have done works although CMT understands the directory
hierarchy :
...//TestRelease/cmt
as 'The package at version "TestRelease" ' which might not
be
what you meant ?? However this understanding by CMT has NO impact on
what we do as long as we don't try to exploit the package name (ie. we
don't care CMT-generated macros such as _tag, or this test
release is not "used").
I played around with this and as long as I didn't use "package "
within the requireemnts file it seemed to work. Originally I had in fact
made it a pseudo-package with a fake tag, but it seems very volatile and
obviously shouldn't be checked into CVS. Presumably it could live in CVS
just to give a template.
+ I will check the list of env. variables
+ I'll probably also describe the alternate primary setup scenario
(while yours is fine, very simple and quite understandable by
newcomers) which probably offers more flexibility in long term, ie the
one based on the login requirements file. This scenario anyway
requires
at least :
export CMTBASE=/afs/cern.ch/sw/contrib/CMT
export CMTVERS=v1r9
source setup.sh [1]
[1] this needs to be prepared once by a "cmt config" although the
scenario of re-config'ing when one changes CMT version has to be
explicited. In you scenario, it's a matter of changing CMTVERS, in
mine
it's a matter of running again "cmt config". Well the complete
sequence
export CMTBASE=/afs/cern.ch/sw/contrib/CMT
export CMTVERS=v1r9
source $CMTBASE/$CMTVERS/mgr/setup.sh
cmt config
is a way to combine both scenarios (the "cmt config" command here
is
not really heavy and can be re-done continuously without any harm but
could also be commented out as long as CMTVERS is not changed...)
I was just worried if anyone ever did a "gmake" on this requirements
file.
**********************************************************
I guess the right style would be to :
=======================================================================
1) have a core part package for a particular domain which could be
linked against as a normal library by "internal" packages in the same
domain
2) have the interface package as foreseen
3) have the true implementation component package which would make
use of the core part.
=======================================================================
******************************************************
o other scenarios
o CVS scenarios
o binary tag management
platform/compiler conventions
special builds (insure, debug, etc...)
o external packages
o transporting the software
o ...