[DESY Home] [DESY IT Home] [DESY IT Physics Computing] [Grid Computing at DESY] [DESY Computing Seminar] [Imprint]

Grid Computing at DESY DESY

[Home] [Mon/Admin] [Grid@DESY] [Certs & VOs] [VOMS] [CVMFS] [User Guide] [Install Guide] [Notes] [Talks & Posters] [Glossary] [Documentation] [Links]

In order to ensure response in case of problems, use the Global Grid User Support (GGUS) and/or your VO support rather than private e-mail contacts or internal mailing lists.


Grid User Guides at DESY

For the VO registration refer to the list of VOs supported at DESY.


There is no Grid UI for SL6 via AFS available. The SL6 NAF WGS are equipped with the most recent EMI-3 UI and are kept up-to-date, e.g. 'nafhh-atlas', 'nafhh-belle', 'nafhh-cms', 'nafhh-ilc', 'naf-hone', 'nafhh-zeus' as well as 'pal'.

At 31. October 2014 the AFS-based SL5 UI was REMOVED!


We collect here a sample of commands which could be used for first tests. For a complete description of commands refer to the gLite 3.2 User Guide.
The DECH Regional Support Team has compiled a Grid primer, which works at SCAI for the VO dech.

The primary entry point for Grid user support is the Global Grid User Support (GGUS). Please send questions or report problems to this URL.


FAQ

Contents


Documentation

gLite 3.2 User Guide

[top]


Virtual Organization

See Grid Certification and VOs at DESY for details.

[top]


Proxy Creation with VOMS

Create a proxy, using your standard Grid certificate in ~/.globus/usercert.pem / ~/.globus/userkey.pem against your VO.

  voms-proxy-init -verify -debug -voms myvo

  voms-proxy-info -all

If the user belongs to a group within the VO and/or holds a special role, use the following commands with the correct entries:

  voms-proxy-init -verify -debug -voms myvo:/myvo

  voms-proxy-init -verify -debug -voms myvo:/myvo/mygroup

  voms-proxy-init -verify -debug -voms myvo:/myvo/Role=myrole

  voms-proxy-init -verify -debug -voms myvo:/myvo/mygroup/Role=myotherrole

It is possible to explicitly give the path of the cert, extend the validity time, or change the the standard location of the proxy given in X509_USER_PROXY:

  voms-proxy-init -verify -debug -cert mycert.pem -key mykey.pem

  voms-proxy-init -verify -debug -valid 192:00

  voms-proxy-init -verify -debug -file myproxy.pem

Make sure you do NOT request a proxy which is longer than allowed by VOMS. A warning such as 'The validity of this VOMS AC in your proxy is shortened to ...' would appear. The limit for ILC and CALICE is currently 192 hours.


[top]


Information System

  > lcg-infosites --vo dteam
  > lcg-infosites --vo dteam all
  > lcg-infosites --vo dteam ce | se | lfc | wms

  > lcg-info -vo ilc --list-ce --attrs Memory,MaxWCTime,MaxCPUTime,FreeCPUs,RunningJobs,OS,OSVersion,OSRelease --query CE="grid-cr0.desy.de:8443/cream-pbs-desy"

  > lcg-info --list-se --attrs SE --query SE="dcache-se-desy.desy.de"

  > lcg-info --vo ilc --sed --list-service --query ServiceType=org.glite.wms

Tests:

  > ldapsearch -xLLL -p2170 -h grid-giis.desy.de -b o=grid '(&(GlueSiteUniqueID=*)(GlueSiteOtherInfo=GRID=*))' GlueSiteOtherInfo

[top]


Job Submission

The usual way to run jobs in the Grid is by way of GLITE middleware.

Refer to the list of VOs managed at DESY.

Login to your favorite UI (see above) and make sure you have an AFS token:

  > ssh _UI_
  > tokens
  > klog

Your user certificate should be present:

  > ls $HOME/.globus
usercert.pem  userkey.pem

Simple test job

  > cat job.sh
#! /bin/sh
 
#########################################################################
hostname -f
date
id
pwd
ls -al
df .
env
voms-proxy-info -all
#########################################################################

CREAM-CE submisson

  > cat cream.jdl
[
  Executable    = "job.sh";
  StdOutput     = "stdout";
  StdError      = "stderr";
  InputSandbox  = {"job.sh"};
  OutputSandbox = {"stdout","stderr"};
  outputsandboxbasedesturi = "gsiftp://localhost";
]

  > glite-ce-job-submit -a -r grid-cr0.desy.de:8443/cream-pbs-desy cream.jdl
_jobid_

  > glite-ce-job-status _jobid_

  > glite-ce-job-output _jobid_

ARC-CE submission

  > cat arc.xrsl
&
(jobname="ARC")
(executable="job.sh")
(stdout="out")
(stderr="err")

  > arcsub -c grid-arcce0.desy.de arc.xrsl
_jobid_

  > arcstat _jobid_

  > arcget _jobid_

[top]


Data Management using the new LFC catalog

For the DESY VOs, the LCG File Catalog (LFC) is available.

The LFC offers features such as directory structures and ACL. and shows better performance.

Finding the apprpriate LFC:

  lcg-infosites --vo _VO_ lfc

Enabling the usage of LFC: (chose your favorite VO or use the generic host name)

  export LFC_HOST=`lcg-infosites --vo  lfc`

List the LFC for a VO:

  lcg-infosites --vo _VO_ lfc

Available commands on the UI are:

lfc-chmod        lfc-getacl       lfc-mkdir        lfc-setacl
lfc-chown        lfc-ln           lfc-rename       lfc-setcomment
lfc-delcomment   lfc-ls           lfc-rm

List contents:

  lfc-ls -l  /grid
  lfc-ls -lR /grid
  lfc-ls -lR /grid/_VO_

Creating a hierachical file system structure. Note: It has to start with lfn:/grid/_VO_:

  lfc-mkdir -p /grid/_VO_/test

List access rights:

  lfc-getacl /grid/_VO_/test

Example: (we use here the VO 'ilc') (Note: you must hold a valid proxy AND a be a member of the particular VO!)

  > export LFC_HOST=`lcg-infosites --vo  lfc`
  > lcg-cr -v --vo ilc file:/bin/sh -l lfn:/grid/ilc/test/ag_test1 -d dcache-se-desy.desy.de
  > lfc-ls -l -R /grid/ilc/test

/grid/ilc/test:
-rwxrwxr-x   1 41775    flc                  585428 Jan 26 10:08 ag_test1

  > lfc-getacl /grid/ilc/test
# file: /grid/ilc/test
# owner: 41775
# group: flc
user::rwx
group::rwx              #effective:rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r-x

[top]


Data Management

GFAL

  > gfal-ls -l --xattr=user.status

  > gfal-copy  file:... srm://...

lcg-tools (deprecated)

List replicas:

 > lcg-lr -v --vo dteam lfn:/grid/dteam/desy/DESYTestFile

 > lcg-lg -v --vo dteam lfn:/grid/dteam/desy/DESYTestFile

Copy and register a file:

  > lcg-cr -v --vo dteam -l lfn:/grid/dteam/desy/test4711 file:$PWD/file4711
guid:94462221-4871-4a45-ab31-2f3ce46e0cdb

Copy a file back:

  > lcg-cp -v --vo dteam lfn:/grid/dteam/desy/test4711 file:/tmp/test4711

Replicate a file to another SE:

  > lcg-rep -v --vo dteam lfn:/grid/dteam/desy/test4711 -d grid-se2.desy.de

List replicas associated with a LFN.

  > lcg-lr  -v --vo dteam lfn:/grid/dteam/desy/test4711

Using 'lcg-fetch' requires the following variables to be set:

  export LFC_HOST=`lcg-infosites --vo  lfc`
  export STORAGE_DIR=$PWD

  voms-proxy-init -voms _vo_

  lcg-fetch -v --vo _vo_ --SE _se_ --tar _file-to-be-copied_

Access can be tested by means of local protocols used at DESY, such as dCache copy dccp:

  > dccp -d2 gsidcap://...

A global basic Grid protocol used by all high-level protocols is GridFTP:

  > globus-url-copy gsiftp://...

A high-level Grid management protocol is the Java-based SRM:

  > mkdir -p $HOME/.srmconfig
  > srmcp -debug=true srm://...
[top]
Last modified: Thu Dec 1 11:30:20 MET 2005
by the DESY Grid Team: http://grid.desy.de/