1 Submitting jobs into the gLite Grid

Job Description Language

The Job Description Language (JDL) was developed as part of the Condor Project to describe resources like jobs and hosts. The workload management components involved in the job submission process need to understand JDL. In general JDL consists of simple attribute-value-pairs divided by semicolon. Some attribute describe the job itself, like the executable, input-output files and so on, while others specify requirements on the execution environment. To learn more details about JDL take a look to the DataGrid Job Description Language Howto.

A very simple example looks like:
  Executable = "/bin/hostname";
  StdOutput = "std.out";
  StdError = "std.err";
  OutputSandbox = { "std.out","std.err" };

Remote Software

If you require special software packages, there is no need to send them together with every job. Installed software is published via GLUE. They can be put into the requirements of JDL, e.g.:
Requirements = Member("VO-alice-ROOT", other.GlueHostApplicationSoftwareRunTimeEnvironment)

You can check for the software tags in the information system:
lcg-infosites --vo dgtest ce tag

Examples

Creating Random Numbers with ROOT

This JDL specifies the application files MakeRandom.sh and MakeRandom.C which will be transfered to the worker node via the input sandbox and the ROOT environment which is required at the target host to run the executbale. The software tag and the initialization of the ROOT environment differs on various compute elements.
Executable = "MakeRandom.sh";
Arguments = "";
StdOutput = "stdout";
StdError = "stderr";
InputSandbox = {"MakeRandom.sh","MakeRandom.C"};
OutputSandbox= {"stdout","stderr","RandomHist.jpg"};
VirtualOrganisation = "dgtest";
Requirements = Member("VO-dgtest-ROOTv5.13.04", other.GlueHostApplicationSoftwareRunTimeEnvironment);
RetryCount   = 3;
The result histogram RandomHist.jpg will be transferred back to you as part of the output sandbox.

Send it into the Grid and retrieve the result:
glite-job-list-match --vo dgtest MakeRandom.jdl
glite-job-submit --vo dgtest -o MakeRandom.id MakeRandom.jdl
glite-job-status --verbosity 0 --input MakeRandom.id
glite-job-output --input MakeRandom.id --dir $HOME

Creating a Gauss fit with ROOT

This example (Gauss.jdl) will teach you how results (here a ROOT-file) produced in a job will be registered in the file catalogue and transfered to the storage to be saved. The following script will execute ROOT with the source file Gauss.C and store the result.
#!/bin/bash
source /opt/root/.rootlogin
root -b -q Gauss.C
export LFC_HOST=iwrlfc.fzk.de
lcg-cr --vo dgtest -v -d iwrgks-3-2.fzk.de -l lfn:/grid/dgtest/icfa10/Gauss.root file://$PWD/gauss.root

Useful commands you would need:
lfc-mkdir /grid/dgtest/$USER
lfc-ls -l /grid/dgtest/$USER
lcg-del --vo dgtest -a lfn:/grid/dgtest/$USER/Gauss.root

After you have produced and stored your data successfully the next job will retrieve them, process the fit and transfer the result back to you.
#!/bin/bash
export LFC_HOST=iwrlfc.fzk.de
lcg-cp --vo dgtest -v lfn:/grid/dgtest/icfa05/Gauss.root file://$PWD/gauss.root
source /opt/root/.rootlogin
root -b -q GaussFit.C

-- VictorPenso - Oct 2006

Topic attachments
I Attachment Action Size Date Who Comment
Gauss.CC Gauss.C manage 0.2 K 2006-10-15 - 08:24 VictorPenso Source file for the Gauss example
Gauss.jdljdl Gauss.jdl manage 0.3 K 2006-10-15 - 08:30 VictorPenso JDL for Gauss example
GaussFit.CC GaussFit.C manage 0.2 K 2006-10-16 - 11:50 VictorPenso Source for the Gauss example
JobDescriptonLanguage.pdfpdf JobDescriptonLanguage.pdf manage 544.1 K 2006-10-13 - 17:32 VictorPenso DataGrid JDL Howto
MakeRandom.CC MakeRandom.C manage 0.7 K 2006-10-15 - 07:34 VictorPenso Source file for Random Number example
MakeRandom.shsh MakeRandom.sh manage 0.2 K 2006-10-16 - 09:08 VictorPenso Shell Script for Random Numbers with ROOT
Topic revision: r6 - 2006-10-16, VictorPenso
 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSI Wiki? Send feedback
Imprint (in German)
Privacy Policy (in German)