-- JuanCastillo - 16 Jul 2008


Parallel ROOT Facility.
The same code can be run locally and in PROOF.

References & Links:

Firs test: ROOT vs PROOF

Login in a 64 machine: ssh user@lxetch64.gsi.de -X
Follow the steps of the basic Marian's GSI tutorial (not the same that at his website) here

Step 1: TChain::Draw locally and on Proof

Some images:

Local Analysis Output: After chain->Draw("particles.fPx");

Local Analysis Output

Proof query process: After TProof *proof = TProof::Open("jcastill@gsiaf.gsi.de");

Proof query process

PROOF output: After:


PROOF output

TIP This is a canva with all the member of the "particle" object we're annalyzing here big grin just in case you want to try annother examples
example member of

Step 2: Making a selector "Analyzer"

Based on one of the trees of the chain (the tree in the first file). Staying in aliroot, write

   TFile *f = TFile::Open("root://gsiaf.gsi.de//tutorial/pythFile1.root"); 
   TTree *tree = (TTree*)f->Get("pythiaEvents");

This is generating Analyzer.C and Analyzer.h onto your "home" folder.Dumping the files in another folder:
  • Be sure you have write permissions for everybody. For example, for dumpin onto "aliroot_utorial":
  • Give to ROOT writting permissions: chmod 0777 aliroot_tutorial
  • Give added to the name of the selector the path where you have it. In aliroot, write: tree->MakeSelector("/u/jcastill/aliroot_tutorial/Analyzer");

ALERT! NOTE: If you generate the selector in another place (Ex: tree->MakeSelector("selector/Analyzer");), change onto the code all the "paths" (name given by default, in this case selector/Analyzer) for only the name of your selector (Analyzer). Otherwise you will obtain an error:
Error in <TSelector::GetSelector>: file selector/Analyzer.C does not have a valid class deriving from TSelector

Step 3: Modifying your selector

We modify it to create/fill/display the number of particles per event.

Add or modify in the header:

     #include "TH1F.h"  //add
     const Int_t kMaxparticles = 4000; //change  
     TH1F* fHisto; //add 

In the code:

  • In function Analyzer::SlaveBegin that is link with "the proof workers" add:

    fHisto = new TH1F("nParticles", "Number of particles per event", 1000,0,1000); 

  • In function Analyzer::Process that is the action the worker is doing, add:


  • Once all the processes are terminated, we need to find our histo. To function Analyzer::Terminate add:

Step 4: Processing or compiling your selector

The output it's the same if you do it under PROOF or under AliRoot. big grin
Just don't forget to "setproof" for your chain of files.
Also it's not trivial to have the chain "filled".

  • Process the chain with this selector: chain->Process("Analyzer.C") Selector processed output

Selector processed output: chain->Process

  • Compile the selector chain->Process("Analyzer.C+") Selector compiled output:

Selector compiled output: chain->Process

ALERT! NOTE: If you compile your selector, a shared library (Analyzer_C.so) will be created. Appart from that, the output it's the same

TIP TIP: It it's not working, try quitting aliroot and beginning again from step 1: load the libraries, create the chain, create the proof and open it, set the chain to proof and compile/process your selector. And check also the path of your selector.

How to use

Files in a TChain or in a TDSet.
Analysis in a selector (TSelector, AliSelector. AliSelectorRL)
I/O is sent using dedicated list
Alice Analysis Framework has its ows arranngement for selector and I/O


Classes derived from it can run locally and in PROOF


Before running a query you need to populate the input list:==gProof->AddInput(myObj)==
Mergin of other classes: you need to implement Merge(TCollection*)

Loading packages

Via PAR file. Like Java Jar.
  • GZipped tar file
  • PROOF-INF directory:
    • BUILD.sh, build the pacakge, execute it per slave.
    • SETUP.C, et enviroment, load libraries, execute it per slave.

API to manage and activate packages:


If things go wrong

Case: crash and you cant connect with your session anymore.
Try: TProof::Reset("username@gsiaf.gsi.de")

Case: crashes of the selector.
  • Compile it again running chain->Process("ana.c")
  • Increase the log level of proof.

Look at the log files (it's all at the web)

See: Proof wiki page, CAF


See presentations miranov course

Batch machines
10 nodes (lxb284-lxb295 without 292, 288)
architecture: 8 cores, 16 GB.
system: debian etch, 64 bits
root version for proof: v5-17-04 (64bits)
libraries used need to be compiled under 64 bits

Enable aliroot libs on proof

  • 1. Enable par files
  • 2. Use a macro (ask Marian):

void ProofEnableAliRoot(const char* location = "/usr/local/grid/AliRoot/v4-06-Release")
  • 3. Load any local macro in all the working machines: gProof->Load("mymacro.C")

Idea of using independent components

You write what you want to do inside your analysis task.
Problem: debugging. You're not controlling the code below.


Write component
check stability
obtimise cuts
integrate into Analysis Framework

Bacht analysis

Slide 17: submit job: root -b -q submitBatch.C\(\"../lists/ppSmall.txt\",20\) (20 = number of files per job)

PROOF analysis

Slide 18: Before what is written there:
* Change proofAna.C macro (write your name)
* Load "Jens enviroment".Once in root: .x /u/wiechula/.rootlogon.C
* Now proceed.


Source: (a little bit old) ALICE Offline CAF page

The grid certification

You must HAVE a valid GRID certificate in your .globus folder, in your CERN lxplus home directory.
You MUST register as a VO user User Registration for Alice

The alien shell

  • Go to bash shell.Example: In your prompt, write [lxplus217] ~ > bash
  • Load it with their corresponding enviroment variables. Once in bash, "Source" the script for that, writting:

bash-3.00$ source /afs/cern.ch/alice/caf/caf-lxplus-new.sh -alien v4-13-Release

Sample output:

ROOT has now been enabled. Next alien-token-init will be called and you will be prompted for your
certificate password. Note that you have to have your certificate in /afs/cern.ch/user/c/castill1/.globus for that.

Setting central config:
export alien_API_SERVER_LIST="pcapiserv01.cern.ch:10000|pcapiserv02.cern.ch:10000|pcapiserv04.cern.ch:10000"
export alien_API_PORT=10000
export TERMINFO=/usr/share/terminfo
You have installed as gcc /usr/bin/gcc !!!
Client was compiled with   gcc version 3.4
Your local compiler is     gcc version 3.4
The installation itself contains gcc version 3.2
Great! Your compiler setup is ok!
Attention: You don't have a valid grid proxy - doing grid-proxy-init for you ...
Your identity: /O=GermanGrid/OU=GSI/CN=Juan Castillo
Enter GRID pass phrase for this identity:
Creating proxy ............................................................... Done
Your proxy is valid until: Thu Jul 17 01:36:02 2008
=> Trying to connect to Server [0] http://pcapiserv01.cern.ch:10000 as User castill1
=> Trying to connect to Server [1] http://pcapiserv02.cern.ch:10000 as User castill1
=> Trying to connect to Server [2] http://pcapiserv04.cern.ch:10000 as User castill1
failed to connect: server:authentication:role
Error: Connection to castill1@:10000[pcapiserv01.cern.ch:10000|pcapiserv02.cern.ch:10000|pcapiserv04.cern.ch:10000] failed!
FAILED: Token creation failed

Setting environment for use of ALICE CAF on LXPLUS

Default CAF versions are:
ROOT: v5-21-01-alice
AliRoot: v4-13-Release
Platform: slc4_amd64_gcc34

ROOT has been enabled. You are ready to use CAF...
PAR files can be found in /afs/cern.ch/alice/caf/sw/ALICE/PARs/v4-13-Release

ALERT! The error is due to the VO box token. See Alien Home

Testing PROOF

  • Login in a lxplus machine
  • Type aliroot
  • Try to access to PROOF: write root[]TProof::Open("<user>@lxb6046.cern.ch")

Sample output:

   Starting master: opening connection ...
   Starting master: OK
   Opening connections to workers: OK (33 workers)
   Setting up worker servers: OK (33 workers)
   Welcome to ALICE's CERN Analysis Facility
   PROOF set to parallel mode (33 workers)
  (class TProof*)0xb74bc0


Topic attachments
I Attachment Action Size Date Who Comment
local_output.pngpng local_output.png manage 5.1 K 2008-02-13 - 12:16 JuanCastillo Local Analysis Output
particle_members.pngpng particle_members.png manage 12.2 K 2008-02-13 - 16:06 JuanCastillo example member of "particle" object
proof_output.pngpng proof_output.png manage 4.4 K 2008-02-13 - 12:16 JuanCastillo PROOF output
proof_query_progress.pngpng proof_query_progress.png manage 17.0 K 2008-02-13 - 13:22 JuanCastillo Proof query process
selector_output_compiled.pngpng selector_output_compiled.png manage 6.9 K 2008-02-13 - 14:19 JuanCastillo Selector compiled output: chain->Process
selector_output_processed.pngpng selector_output_processed.png manage 6.3 K 2008-02-13 - 14:16 JuanCastillo Selector processed output: chain->Process
Topic revision: r11 - 2008-07-18, JuanCastillo
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSI Wiki? Send feedback
Imprint (in German)
Privacy Policy (in German)