Train Tips

My experience as a wagon of the GSI train: after and before SVN


-- JuanCastillo - 16 Apr 2014


Creating and customizing an AliAnalysisTask

The AliAnalysisTask is the standard C++ class, mother of the analysis tasks that deal with ALICE data.
Below we describe how to create an analysis task, based on existing "working" examples. It is assumed you have a working AliRoot

  • Download the "train" example. For example, in Section "Working example and current status" here: Alice Experiment: Offline Project
  • Unzip it, make the necessary par files running makepars.sh
  • Copy for example from my working folder ( cd /lustre_alpha/alice/jcastill/bin_64/train ) the AliAnalysisTaskMD.* files (2), the MULTIPLICITY.par and runLocal.C
  • Choose a data list. It contains a list of paths to ROOT files with data.
    Data can be not available anymore due to deletion. In my case, /u/sma/data/mc/v4-10-Release/fulleta_pp_2.txt. In general:

    • Old productions: /u/sma/data/mc. Each directory contains version-related lists:
           electrons  pdc07  speed_june16  v4-09-Release  v4-11-Release  v4-13-Rev-01   v4-14-Rev-04  v4-16-Release
           lustre     PDC08  v4-05-Rev-03  v4-09-Rev-01   v4-12-Release  v4-13-Rev-05   v4-15-Rev-01
           pdc06      speed  v4-06-Rev-03  v4-10-Release  v4-13-Release  v4-14-Release  v4-15-Rev-02

    • Lustre productions: /u/sma/data/mc/lustre (subdirectory of "old productions").
           jpsi_woojin       old_v4-16-Release  speed09      v4-14-Rev-04   v4-16-Rev-03  v4-16-Rev-07           v4-16-Rev-08_old
           LHC09a4           old_v4-16-Rev-03   summary.ods  v4-15-Rev-02   v4-16-Rev-05  v4-16-Rev-08           v4-17
           old_v4-14-Rev-04  old_v4-16-Rev-05   summary.xls  v4-16-Release  v4-16-Rev-06  v4-16-Rev-08_20090622
  • Test that the example you copied is working with your AliRoot installation, and so on...
  • Customize your task: change the name, choose what you get, etc.

Running a train

ALERT! The next tips are for running an OLD version (v4) of the train. If you need to run a NEW one, please contact your local "train driver"

  1. Create the main folder for the train, and a subfolder for the tasks Ex:main, train_juan, tasks folder, template
  2. Copy the tasks in its folder.
  3. In the main folder, copy and customizethe next scripts
    • For running: /u/sma/train/v4/train.sh. You better change:
      • RESULTSDIR : there you store the output files. It is created after the date and the sample. Ex: /train_juan/2008-07-31/v4-10/pp1
      • MYDIR and WD: they control the working directory during the production, where the tasks are unzipped. Ex: /tmp/jcastill170901080731v4-10/pp1, where pp1 corresponds to the job (there will be one folder per job). Here, in out.log you will be able (for example with more) read what is being done, how the par files are processed, the tasks loaded, and so on.
      • For testing your changes in this script, be sure you have something like root -b -q AnalysisTrain.C\(\"/u/sma/data/mc/$2.txt\",2,$task\) > out.log 2>&1 (that is, you run only over 2 samples).
    • For submitting jobs, copy and customize (or create your own) submit.sh job submission script. Yo have an example in /lustre_alpha/alice/jcastill/bin_64/train_juan/submit.sh (no email will be sent as a report on your job status)
  4. Add your tasks to the train. For that, you need to modify template/AnalysisTrain.C. Of course all the task and par files should be "mentioned" there. fRun controls if the task is including or not in the run: for example, change the number if (fRun==4 || fRun==100) for if (fRun==4 || fRun==1000) to eliminate the task included in that if.
  5. Test the modified scripts. For example: train_juan$ ./train.sh v4-10/pp1 v4-10-Release/small_lists/fulleta_pp_1_1. Sample output:
   ROOT version: v5-19-04, AliROOT version: v4-13-Release
   Starting directory /lustre_alpha/alice/jcastill/bin_64/train_juan/template
   Local directory /tmp
   /tmp/jcastill170901080731v4-10/pp1
   ...
   Cleaning ...
   mv: cannot stat `*.root': No such file or directory
   mv: cannot stat `*.log': No such file or directory
   Done.

ALERT! IMPORTANT: if you want to monitor out.log, you need to be login in the same machine. It's not enough with being in lustre

Handling the batch farm

Source: Using the SLAC batch farm

Some useful commands:

jcastill@lxi020:/lustre_alpha/alice/jcastill/bin_64/train_juan$ bjobs
JOBID   USER    STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
643429  jcastil RUN   alice-t3   lxi020.gsi. lxb266.gsi. *-01/pdc_7 Aug  4 11:15
643430  jcastil RUN   alice-t3   lxi020.gsi. lxb267.gsi. *trd_pdc_0 Aug  4 11:15
643431  jcastil RUN   alice-t3   lxi020.gsi. lxb345.gsi. *trd_pdc_1 Aug  4 11:15
643432  jcastil RUN   alice-t3   lxi020.gsi. lxb339.gsi. *trd_pdc_2 Aug  4 11:15
643433  jcastil PEND  alice-t3   lxi020.gsi.             *trd_pdc_3 Aug  4 11:15
643434  jcastil PEND  alice-t3   lxi020.gsi.             *trd_pdc_4 Aug  4 11:15
643435  jcastil PEND  alice-t3   lxi020.gsi.             *trd_pdc_5 Aug  4 11:15
643436  jcastil PEND  alice-t3   lxi020.gsi.             *trd_pdc_6 Aug  4 11:15
643437  jcastil PEND  alice-t3   lxi020.gsi.             *trd_pdc_7 Aug  4 11:15
jcastill@lxi020:/lustre_alpha/alice/jcastill/bin_64/train_juan$ bkill 643437
Job <643437> is being terminated

Joining the SVN train

To join GSI SVN train, you need to ask the train driver to create a folder for your analysis.
The folder you will find later, once you install the copy of the train. It should be something like yourname_taskname, in this case jcastill_mdana.

Copying the train structure

Create a folder on your home directory.
To test that your task works, you will need access to lustre, therefore, don't dig it too much.
In our case, it will be jcastill_mdana (yes,like the wagon itself big grin )

Go to the folder (jcastill_mdana)and type:

svn co https://subversion.gsi.de:/alice/train/trunk trunk

ALERT!: NOTE: the command must stay like this. If you change the output directory trunk for your own (like mytrain) you will not suceed on adding your wagon to the GSI analysis train.

Now you should have a new folder trunk in jcastill_mdana. Go inside and check if yourname_taskname is there.

A simple running Analysis Task

Testing your task

/jcastill_mdana/trunk$ ./run user -l v4-16-Rev-08/pdc1_resdb.txt -n 2 -f
/jcastill_mdana/trunk$ ./run user -t

Explanation of the parameters:

  • user: run mode. Keep this if you want to run alone.
  • v4-16-Rev-08/pdc1_resdb.txt : STEER/lists/v4-16-Rev-08/pdc1_resdb.txt data list. By default it goes to STEER/lists/ to look for your list
  • 2: number of files to process (2).

Sample printout (commented) if everything is fine (with a user-created par file):
Processing STEER/macros/StartAnalysis.C("STEER/lists/v4-16-Rev-08/pdc1_resdb.txt",2,0,"local","jcastill")...
Assing file: /lustre/alice/sma/v4-16-Rev-08/10TeV/pdc1/00000/AliESDs.root
Assing file: /lustre/alice/sma/v4-16-Rev-08/10TeV/pdc1/00001/AliESDs.root
=====================================
processing dir: jcastill_mdana (/u/jcastill/jcastill_mdana/trunk/jcastill_mdana)  //task dir
-------------------------------------
[...] //here you should see compilation messages from your task...
Linking libMULTredux.so ...                                                                                                   
done                                                                                                                          
Adding task 'jcastill_MD'                                                                                                     
Here                                                                                                                          
(class AliAnalysisTask*)0x10e0af0                                                                                             
task: jcastill_mdana  ACTIVE=0 POST_LOOP=0                                                                                    
   INPUT #0: TChain <-  [cAUTO_INPUT]                                                                                         
   OUTPUT #0: TTree ->  [NO CONTAINER]                                                                                        
   OUTPUT #1: TList ->  [jcastill_mdana]                                                                                      
   Container: jcastill_mdana  type: TList POST_LOOP=0    = Data producer: task jcastill_mdana    = Consumer tasks: -none-     
Filename: jcastill_mdana.root                                                                                                 
===== RUNNING LOCAL ANALYSIS Analysis Train ON TREE esdTree                                                                   
Linking done                                                                                                                  

=============================
    Running Statistics         
-----------------------------  
Real time 0:00:43, CP time 16.300

ALERT! NOTE: The root and aliroot used are those defined in STEER/scripts/env

Commiting the task

~/jcastill_mdana/trunk$ svn add *
A         AddTask_jcastill_MD.C
A         AliAnalysisTaskMD.cxx
A         AliAnalysisTaskMD.h
A         config
A         libMULTredux.pkg
A         Makefile
A         MDclassQA.cxx
A         MDclassQA.h
A         MDcorrelations.cxx
A         MDcorrelations.h
A         MULTreduxLinkDef.h
A         PROOF-INF.MULTredux
A         PROOF-INF.MULTredux/BUILD.sh
A         PROOF-INF.MULTredux/SETUP.C
~/jcastill_mdana/trunk/jcastill_mdana$ svn ci
Authentication realm: <https://subversion.gsi.de:443> GSI Subversion repository alice
Password for 'jcastill':
Adding         jcastill_mdana/AddTask_jcastill_MD.C
Adding         jcastill_mdana/AliAnalysisTaskMD.cxx
Adding         jcastill_mdana/AliAnalysisTaskMD.h
Adding         jcastill_mdana/MDclassQA.cxx
Adding         jcastill_mdana/MDclassQA.h
Adding         jcastill_mdana/MDcorrelations.cxx
Adding         jcastill_mdana/MDcorrelations.h
Adding         jcastill_mdana/MULTreduxLinkDef.h
Adding         jcastill_mdana/Makefile
Adding         jcastill_mdana/PROOF-INF.MULTredux
Adding         jcastill_mdana/PROOF-INF.MULTredux/BUILD.sh
Adding         jcastill_mdana/PROOF-INF.MULTredux/SETUP.C
Adding         jcastill_mdana/config
Adding         jcastill_mdana/libMULTredux.pkg
Transmitting file data .............
Committed revision 305.
Topic revision: r4 - 2014-04-16, JuanCastillo
 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSI Wiki? Send feedback
Imprint (in German)
Privacy Policy (in German)