[DESY Home] [DESY IT Home] [DESY IT Physics Computing] [Grid Computing at DESY] [DESY Computing Seminar] [Imprint]

Grid Computing at DESY DESY

[Home] [Mon/Admin] [Grid@DESY] [Certs & VOs] [VOMS] [CVMFS] [User Guide] [Install Guide] [Talks & Posters] [Glossary] [Documentation] [Links]

In order to ensure response in case of problems, use the Global Grid User Support (GGUS) and/or your VO support rather than private e-mail contacts or internal mailing lists.


Grid for ILC / CALICE

DESY participates in the Grid activities of WLCG as a member of the EU Grid project EGEE and operates a complete production-grade Grid infrastructure for the HERA experiments and some DESY groups.

Within the activities on Grid Computing at DESY, DESY hosts the Virtual Organizations (VO) for ILC called 'calice' and 'ilc'.

[top]


VO Administrator Information

DESY operates a for all DESY VOs Virtual Organization Membership Service (VOMS). Find the appropriate contact for the VO and follow the appropriate link for registration.

The VO Admins might find the links to the VO Membership Registration Service (VOMS) here:

CALICE: https://grid-voms.desy.de:8443/voms/calice
ILC: https://grid-voms.desy.de:8443/voms/ilc

[top]


Site Administrator Information

Standard yaim variables for DESY:

  > cat ./siteinfo/siteinfo.def
SITE_NAME=DESY-HH

WMS_HOST=grid-wms.desy.de
PX_HOST=grid-px.desy.de
BDII_HOST=grid-bdii.desy.de

VO specific: (adjust to your site)

  > cat ./siteinfo/vo.d/calice
SW_DIR=/opt/vo/calice
DEFAULT_SE=dcache-se-desy.desy.de 
VOMS_SERVERS=vomss://grid-voms.desy.de:8443/voms/calice?/calice/
VOMSES="'calice grid-voms.desy.de 15102 /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de calice'"
VOMS_CA_DN="/C=DE/O=GermanGrid/CN=GridKa-CA"
WMS_HOSTS=ilc-wms.desy.de

  > cat ./siteinfo/vo.d/ilc
SW_DIR=/opt/vo/ilc
DEFAULT_SE=dcache-se-desy.desy.de
VOMS_SERVERS=vomss://grid-voms.desy.de:8443/voms/ilc?/ilc/
VOMSES="'ilc grid-voms.desy.de 15110 /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de ilc' 'ilc voms.fnal.gov 15023 /DC=org/DC=doegrids/OU=Services/CN=http/voms.fnal.gov ilc'"
VOMS_CA_DN="'/C=DE/O=GermanGrid/CN=GridKa-CA' '/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1'"
WMS_HOSTS=ilc-wms.desy.de

YAIM will produce:

  > cat /etc/grid-security/vomsdir/calice/grid-voms.desy.de.lsc
/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de
/C=DE/O=GermanGrid/CN=GridKa-CA

  > cat /etc/grid-security/vomsdir/ilc/grid-voms.desy.de.lsc
/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de
/C=DE/O=GermanGrid/CN=GridKa-CA

[top]


Getting Started as a User

[gLite 3 User Guide]

In the Grid authentication is done via user certificates which are issued by the national Certificationn Authorities (CA) to which the user's home institutes are connected. Find your CA at EUGridPMA.

In order to register as a user of one of the ILC VOs - as soon as you have a vailid Grid certificate - refer to CALICE (VOMS) and ILC (VOMS).

DESY hosts the ILC data repository such as Monte Carlo events and CALICE test beam data.

At DESY, all Scientifc Linux 5 hosts can run the Grid User Interface software which is located in AFS. The SL6 NAF WGS have a local UI installation. See the DESY Grid User Guide for details.

Create a VOMS proxy for authentication and authorization with the Grid (example VO 'ilc'). This requires the presence of the VOMS server's host certifcate, see VOMS at DESY:

  > cat $X509_VOMS_DIR/calice/grid-voms.desy.de.lsc
/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de
/C=DE/O=GermanGrid/CN=GridKa-CA

  > cat $X509_VOMS_DIR/ilc/grid-voms.desy.de.lsc
/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de
/C=DE/O=GermanGrid/CN=GridKa-CA

  > voms-proxy-init -verify -voms ilc
  > voms-proxy-info -all

  > voms-proxy-init -verify -voms ilc -valid 192:0
  > voms-proxy-info -all

Make sure you do NOT request a proxy which is longer than allowed by VOMS. A warning such as 'The validity of this VOMS AC in your proxy is shortened to ...' would appear. The limit for ILC and CALICE is currently 192 hours.


For support questions please use the GGUS portal.

You might want to share issues etc. with your colleagues:


There is a job submission framework GANGA (Ganga - a computational task management tool for easy access to Grid resources.) which might be of interest. It is in use by ATLAS and LHCb. At DESY one can do a first tests by simply calling:

  /afs/desy.de/project/ganga//install//bin/ganga

Share your experiences and opinions, e.g. via the user's mailing lists!

  • GANGA
  • Ganga Introduction
  • GangaTutorial43
  • Ganga mailing lists

    [top]


    Short Collection of Grid Commands

    Here comes a set of basic Grid commands. This works for 'calice' as well as for 'ilc'. See also DESY Grid User Guide.

    Get VO infos:

      > lcg-infosites --vo calice all
      > lcg-infosites --vo calice lfc
      > lcg-infosites --vo calice ce
      > lcg-infosites --vo calice se
    
      > lcg-info --list-attrs
      > lcg-info --vo calice --list-ce --attrs TotalCPUs,MaxCPUTime,MaxWCTime
      > lcg-info --vo calice --list-ce --attrs TotalCPUs,MaxCPUTime,MaxWCTime --query CE="grid-ce5.desy.de:2119/jobmanager-lcgpbs-desy"
    
      > lcg-info --vo ilc --sed --list-service --query ServiceType=org.glite.wms
    

    Infos on installed software (example for CE 'grid-ce5.desy.de'):

      > lcg-tags -vo ilc --ce grid-cr5.desy.de --list
    

    To enable the access to the VOMS server, some info is needed on your UI if not already there. Ask your system admin to include from the web pages VOMS at DESY so following:

      > ls -l $GLITE_LOCATION/etc/vomses/calice-grid-voms.desy.de
      > ls -l $GLITE_LOCATION/etc/vomses/ilc-grid-voms.desy.de
    

    or set up in your local space (home dir):

      > mkdir -p $HOME/.glite
      > vi $HOME/.glite/vomses
    ...
    "calice" "grid-voms.desy.de" "15102" "/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de" "calice"
    "ilc" "grid-voms.desy.de" "15110" "/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de" "ilc"
    ...
    

    Grid proxy for authentication and authorization. (You need to hold a valid Grid certificate and you need to be a member of the VO):

      > voms-proxy-init -verify -voms calice
      > voms-proxy-info -all
    

    List the catalogue and copy a file by means of its Logical File Name (LFN):

      > export LFC_HOST=`lcg-infosites --vo calice lfc`
      > lfc-ls -l /grid/calice
      > lfc-ls -l /grid/calice/tb-desy/raw/conv_v0402/Run202349.000.slcio
    
      > lcg-lr -v --vo calice lfn:/grid/calice/tb-desy/raw/conv_v0402/Run202349.000.slcio
      > lcg-cp -v --vo calice lfn:/grid/calice/tb-desy/raw/conv_v0402/Run202349.000.slcio file:/tmp/Run202349.000.slcio
    

    Files may be put into the DESY mass storage system (example VO 'ilc'):

      > voms-proxy-init -verify -voms ilc
      > voms-proxy-info -all
    
      > lcg-cr -v --vo ilc -l lfn:/grid/ilc/test/test4711 file:/bin/sh -d dcache-se-desy.desy.de
    
      > export LFC_HOST=`lcg-infosites --vo ilc lfc`
      > lfc-ls -l /grid/ilc/test/test4711
    
      > lcg-lr -v --vo ilc lfn:/grid/ilc/test/test4711
    
      > lcg-del -a -v --vo ilc lfn:/grid/ilc/test/test4711
    

    Registering files which are already in the DESY mass storage (example VO 'ilc'):

      > voms-proxy-init -verify -voms ilc:/ilc/eudet-jra1
      > voms-proxy-info -all
    
      > export LFC_HOST=`lcg-infosites --vo ilc lfc`
      > lfc-ls -l /grid/ilc/eudet-jra1/2007/tb-desy-june
    
      > lcg-rf -v --vo ilc -l lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/testag2 srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/testag2
    Using grid catalog type: lfc
    Using grid catalog : grid-lfc.desy.de
    set timeout to 0 seconds
    Site URL to be registered: srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/testag2
    File size: 585908
    Alias created in Catalog: lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/testag2
    guid:e2a6fbbc-c32e-4bf8-b9bc-709f45d0263f
    
      > lfc-ls -l /grid/ilc/eudet-jra1/2007/tb-desy-june/testag2
    -rw-rw-r--   1 44021    1417                 585908 Jun 19 10:12 /grid/ilc/eudet-jra1/2007/tb-desy-june/testag2
    
      > lcg-lr -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/testag2
    srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/testag2
    

    In case a file should be replicated to another Grid site's SE:

      > voms-proxy-init -verify -voms ilc:/ilc/eudet-jra1
      > voms-proxy-info -all
    
      > lcg-lr -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
    srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
    
      > lcg-rep -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw -d grid05.lal.in2p3.fr
    Using grid catalog type: lfc
    Using grid catalog : grid-lfc.desy.de
    Source URL: lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
    File size: 91251853
    VO name: ilc
    Destination specified: grid05.lal.in2p3.fr
    Source URL for copy: gsiftp://dcache38.desy.de:2811//pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
    Destination URL for copy: gsiftp://grid06.lal.in2p3.fr/grid06.lal.in2p3.fr:/dpmpart/part8/ilc/2007-06-21/file07a2209f-91f5-46f5-8a87-aa9d87b7ee2f.709781.0
    # streams: 1
    # set timeout to 0
         87101440 bytes   2126.50 KB/sec avg   2158.00 KB/sec inst
    Transfer took 123210 ms
    
    Destination URL registered in LRC: srm://grid05.lal.in2p3.fr/dpm/lal.in2p3.fr/home/ilc/generated/2007-06-21/file07a2209f-91f5-46f5-8a87-aa9d87b7ee2f
    
      > lcg-lr -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
    srm://grid05.lal.in2p3.fr/dpm/lal.in2p3.fr/home/ilc/generated/2007-06-21/file07a2209f-91f5-46f5-8a87-aa9d87b7ee2f
    srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
    

    Submitting jobs to the Grid (example VO 'ilc'):

      > cat cream.sh
    #! /bin/sh
    
    #########################################################################
    /bin/hostname -f
    /bin/date
    /usr/bin/id
    /bin/pwd
    /bin/ls -al
    /bin/df .
    /usr/bin/env
    $GLITE_LOCATION/bin/voms-proxy-info -all
    #########################################################################
    
      > cat cream.jdl
    [
      Executable    = "cream.sh";
      StdOutput     = "stdout";
      StdError      = "stderr";
      InputSandbox  = {"cream.sh"};
      OutputSandbox = {"stdout","stderr"};
      outputsandboxbasedesturi = "gsiftp://localhost";
    ]
    
      > glite-ce-job-submit -a -r grid-cr0.desy.de:8443/cream-pbs-desy cream.jdl
    
      > glite-ce-job-status 
    

    To keep track of jobIds, one can specify a file which is used as a repository of jobIds:

      > glite-ce-job-status -o 
      > glite-ce-job-output -o 
    

    [top]


    Links

    [top]


    Last modified: Fri Dec 2 15:25:21 MET 2005
    by the DESY Grid Team: http://grid.desy.de/