CernVM-FS

The distribution of custom software for user groups on the HPC infrastructure is provided using the Cern Virtual Machine File-System (CernVM-FS). CernVM-FS is a read-only network file system based on HTTP and optimized to deliver software in a fast, scalable, and reliable way. Files and file meta-data are cached and downloaded on demand.

Cluster nodes, desktops and interactive machines mount software repositories in the directory /cvmfs/. Every sub-directory is called like the maintainers department or experiment name followed by the GSI domain name. The IT Department for example provides /cvmfs/it.gsi.de with common software available to all users. Experiment groups have repositories like /cvmfs/hades.gsi.de/ for the HADES experiment or /cvmfs/csee.gsi.de/ for the Common Systems and Experiment Electronics department.

---

---

Available Software

For a list of software at GSI made available via CVMFS see SoftwareInCvmfs.

Usage

Sub-directories in /cvmfs/ will be automatically mounted on demand as soon as they are accessed. This means directories are visible when listing with the ls command only if they are mounted already.

It is up to the maintainers how to structure the software inside a repository, and how to enable users to load required environment variables to use it. The HPC department recommends to use the environment-modules package providing a convenient command called module (→ EnvironmentModules).

Repositories

The following table contains a list of all software repositories available:

Path	Support	Description
/cvmfs/alice.gsi.de	p.malzacher@gsi.de, k.schwarz@gsi.de	ALICE Experiment
/cvmfs/fairroot.gsi.de	Contact the developers	FairRoot software
/cvmfs/hades.gsi.de	j.kempter@gsi.de	HADES Experiment
/cvmfs/sist.gsi.de	t.radon@gsi.de	SiSt Department at GSI
/cvmfs/theory.gsi.de	t.neff@gsi.de	Theory Department at GSI
/cvmfs/it.gsi.de	v.penso@gsi.de	Common software provided by GSI IT
/cvmfs/gamma.gsi.de	s.saha@gsi.de	Software for PreSPEC -AGATA-analysis
/cvmfs/csee.gsi.de	s.linev@gsi.de	Common Systems/Experiment Electronic (ROOT, Go4, DABC, EPICS)
/cvmfs/labview.gsi.de	h.brand@gsi.de, p.zumbruch@gsi.de	LabVIEW
/cvmfs/epics.gsi.de	p.zumbruch@gsi.de	EPICS
/cvmfs/r3b.gsi.de	b.loeher@gsi.de, r.plag@gsi.de, d.kresan@gsi.de	R3B experiment group
/cvmfs/panda.gsi.de	paul.buehler@oeaw.ac.at	Panda Experiment

Note that not all repositories are supported on all Debian platforms.

Instructions for maintainers

Please contact hpc@gsi.de to commission a new software repository.

Each software repository is hosted on a dedicated server called lxcvmfs. Write access to these nodes is restricted to the repository maintainer(s). In order to install or updated software on a repository server, maintainers have to deploy the software package on a build-server lxbuild. From there a publishing script synchronizes the software repository with the CernVM-FS server.

Repository Access

We need a public SSH key from the responsible software maintainer. Create one with:

»  mkdir key; ssh-keygen -q -t rsa -b 4096 -f key/id_rsa

Please send us only the id_rsa.pub file containing the public part of your key.

Don't forget to specify this private key when logging in to our build hosts, e.g.:

»  ssh -i key/id_rsa lxbuild02.gsi.de

Along with your public key send us the name of the repository you want to become software maintainer for, and your user account name.

Build Servers

The HPC Department operates completely independent CernVM-FS infrastructures for each Debian Version. (This avoids potential incompatibilities between binaries compiled for different Debian releases). Maintainers access the infrastructure for a particular Debian version using a dedicated build server.

Build Server	Platform	CernVM-FS server	Repository
lxbuild04.gsi.de	Debian 7 (Wheezy)	lxcvmfs11.gsi.de	it.gsi.de
		lxcvmfs12.gsi.de	fairroot.gsi.de
		lxcvmfs13.gsi.de	alice.gsi.de
		lxcvmfs14.gsi.de	hades.gsi.de
		lxcvmfs15.gsi.de	csee.gsi.de
		lxcvmfs16.gsi.de	theory.gsi.de
		lxcvmfs17.gsi.de	epics.gsi.de
		lxcvmfs18.gsi.de	labview.gsi.de
		lxcvmfs19.gsi.de	gamma.gsi.de
		lxcvmfs20.gsi.de	r3b.gsi.de
lxbuild05.gsi.de	Debian 8 (Jessie)	lxcvmfs21.gsi.de	it.gsi.de
		lxcvmfs22.gsi.de	alice.gsi.de
		lxcvmfs23.gsi.de	fairroot.gsi.de
		lxcvmfs24.gsi.de	csee.gsi.de
		lxcvmfs25.gsi.de	theory.gsi.de
		lxcvmfs27.gsi.de	panda.gsi.de
		lxcvmfs28.gsi.de	hades.gsi.de
		lxcvmfs29.gsi.de	gamma.gsi.de
		lxcvmfs30.gsi.de	r3b.gsi.de
		lxcvmfs31.gsi.de	epics.gsi.de
		lxcvmfs32.gsi.de	labview.gsi.de
		lxcvmfs33.gsi.de	htit.gsi.de

Note that the name of the software repositories are similar for all Debian versions, while the repository servers are tied to a particular Debian release.

Software Installation

There is no backup of software on the CernVM-FS facility, neither on the build servers nor the repository servers!

It is assumed that you maintain your software with a version control system and that you can reproduce your software stack anytime. Login to the build server for your target platform and follow these three basic steps:

1. Install your software in the corresponding repository directory. (Don't forget to check permissions and executable bits.)
2. Adjust the list of files which will NOT be synced to the repository server (if needed).
3. Call the script to publish your software.

It is up to the software maintainers to structure the directories inside their software repository. We strongly encourage you to attempt the smallest possible installation in terms of storage size. This means you should use out-of-source compilation or the install functionality of your build tool. Furthermore we recommend to never change software already in use! If you really need to overwrite already installed software please contact us beforehand.

Maintainers have the option to exclude files and directories from the synchronization process by enlisting them in a file named like the repository but suffixed *.gsi.de.exclude. (Take a look to the Rsync manuals 'exclude pattern rules' section for more details.)

Common obstacles and their solutions:

• If you encounter missing library dependencies of standard software, please let us know. We will install them in the cluster for you.
• Make sure that the file and directory permissions are correct, for example with commands like: find . -type f -exec chmod o+r {} \; and/or find . -type d -exec chmod o+rx {} \;
• If you decide to build your software on another host, make sure the platform is compatible. (Be careful with static linking!)
• Test your software by executing it first on the build host, and after publishing on the cluster.

Publishing

Publish software with the cvmfs command, respectively cvmfs-remote on Debian Wheezy and Jessie.

» sudo cvmfs-remote publish REPO

The option REPO needs to be replaced by the name of your repository, e.g. alice.gsi.de. Depending on the amount of files newly added to the software repository synchronization and publishing can take several minutes.

Resigning

Repositories need to be resigned at least every 30 days!

» sudo cvmfs-remote resign REPO

Otherwise the software will expire and the repository will become unavailable. This mechanism is in place to prevent clients from using outdated software.

You may want to deploy a Cronjob to automatically resign your repository.

MAILTO=j.doe@gsi.de
0 0 1,20 * * sudo cvmfs-remote resign REPO

Add a Cronjob to your user account with the command crontab -e. Adjust the mail address and the repository name like it is illustrated in the example above.

Best Practice

You should provide a README file in your repository pointing users to information resources, documentation and support.

This file should contain:

• Name and purpose of the software.
• Small how-to for beginners.
• A description of the logical structure of the repository.
• Links to documentation and support pages.
• Your contact information.

You should provide a script to enable users to add your software into their shell environment.

The HPC department recommends the use of Environment Modules like described above in the section CvmFs.