1Submitting jobs into the gLite Grid

Job Description Language

The Job Description Language (JDL) was developed as part of the Condor Project to describe resources like jobs and hosts. The workload management components involved in the job submission process need to understand JDL. In general JDL consists of simple attribute-value-pairs divided by semicolon. Some attribute describe the job itself, like the executable, input-output files and so on, while others specify requirements on the execution environment. To learn more details about JDL take a look to the DataGrid Job Description Language Howto.

A very simple example looks like:

  Executable = "/bin/hostname";
  StdOutput = "std.out";
  StdError = "std.err";
  OutputSandbox = { "std.out","std.err" };

Remote Software

If you require special software packages, there is no need to send them together with every job. Installed software is published via GLUE. They can be put into the requirements of JDL, e.g.:

Requirements = Member("VO-alice-ROOT", other.GlueHostApplicationSoftwareRunTimeEnvironment)

You can check for the software tags in the information system:

lcg-infosites --vo dgtest ce tag

Examples

Creating Random Numbers with ROOT

This JDL specifies the application files MakeRandom.sh and MakeRandom.C which will be transfered to the worker node via the input sandbox and the ROOT environment which is required at the target host to run the executbale. The software tag and the initialization of the ROOT environment differs on various compute elements.

Executable = "MakeRandom.sh";
Arguments = "";
StdOutput = "stdout";
StdError = "stderr";
InputSandbox = {"MakeRandom.sh","MakeRandom.C"};
OutputSandbox= {"stdout","stderr","RandomHist.jpg"};
VirtualOrganisation = "dgtest";
Requirements = Member("VO-dgtest-ROOTv5.13.04", other.GlueHostApplicationSoftwareRunTimeEnvironment);
RetryCount   = 3;

The result histogram RandomHist.jpg will be transferred back to you as part of the output sandbox.

Send it into the Grid and retrieve the result:

glite-job-list-match --vo dgtest MakeRandom.jdl
glite-job-submit --vo dgtest -o MakeRandom.id MakeRandom.jdl
glite-job-status --verbosity 0 --input MakeRandom.id
glite-job-output --input MakeRandom.id --dir $HOME

Creating a Gauss fit with ROOT

This example (Gauss.jdl) will teach you how results (here a ROOT-file) produced in a job will be registered in the file catalogue and transfered to the storage to be saved. The following script will execute ROOT with the source file Gauss.C and store the result.

#!/bin/bash
source /opt/root/.rootlogin
root -b -q Gauss.C
export LFC_HOST=iwrlfc.fzk.de
lcg-cr --vo dgtest -v -d iwrgks-3-2.fzk.de -l lfn:/grid/dgtest/icfa10/Gauss.root file://$PWD/gauss.root

Useful commands you would need:

lfc-mkdir /grid/dgtest/$USER
lfc-ls -l /grid/dgtest/$USER
lcg-del --vo dgtest -a lfn:/grid/dgtest/$USER/Gauss.root

After you have produced and stored your data successfully the next job will retrieve them, process the fit and transfer the result back to you.

#!/bin/bash
export LFC_HOST=iwrlfc.fzk.de
lcg-cp --vo dgtest -v lfn:/grid/dgtest/icfa05/Gauss.root file://$PWD/gauss.root
source /opt/root/.rootlogin
root -b -q GaussFit.C

-- VictorPenso - Oct 2006