| [DESY Home] [DESY IT Home] [DESY IT Physics Computing] [Grid Computing at DESY] | [DESY Computing Seminar] | [Imprint] |
| Grid Computing at DESY | |
|---|
| [Home] |
[Mon/Admin  ]
| [Grid@DESY] [Certs & VOs] [VOMS] [User Guide] [Install Guide] | [Talks & Posters] | [Glossary] [Documentation] [Links] |
In order to ensure response in case of problems, use the
Global Grid User Support (GGUS)
and/or your VO support rather than private e-mail contacts or internal mailing lists.
DESY participates in the Grid activities of WLCG as a member of the EU Grid project EGEE and operates a complete production-grade Grid infrastructure for the HERA experiments and some DESY groups.
Within the activities on Grid Computing at DESY, DESY hosts the Virtual Organizations (VO) for ILC called 'calice' and 'ilc'.
[top]
DESY operates a for all DESY VOs Virtual Organization Membership Service (VOMS). Find the appropriate contact for the VO and follow the appropriate link for registration.
The VO Admins might find the links to the VO Membership Registration Service (VOMS) here:
| CALICE: | https://grid-voms.desy.de:8443/voms/calice |
| ILC: | https://grid-voms.desy.de:8443/voms/ilc |
[top]
Standard yaim variables for DESY:
> cat ./siteinfo/siteinfo.def SITE_NAME=DESY-HH RB_HOST=grid-rb.desy.de WMS_HOST=grid-wms.desy.de PX_HOST=grid-px1.desy.de BDII_HOST=grid-bdii.desy.de
VO specific: (adjust to your site)
> cat ./siteinfo/vo.d/calice SW_DIR=/opt/vo/calice DEFAULT_SE=dcache-se-desy.desy.de VOMS_SERVERS=vomss://grid-voms.desy.de:8443/voms/calice?/calice/ VOMSES="'calice grid-voms.desy.de 15102 /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de calice'" VOMS_CA_DN="/C=DE/O=GermanGrid/CN=GridKa-CA" WMS_HOSTS=ilc-wms.desy.de > cat ./siteinfo/vo.d/ilc SW_DIR=/opt/vo/ilc DEFAULT_SE=dcache-se-desy.desy.de VOMS_SERVERS=vomss://grid-voms.desy.de:8443/voms/ilc?/ilc/ VOMSES="'ilc grid-voms.desy.de 15110 /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de ilc' 'ilc voms.fnal.gov 15023 /DC=org/DC=doegrids/OU=Services/CN=http/voms.fnal.gov ilc'" VOMS_CA_DN="'/C=DE/O=GermanGrid/CN=GridKa-CA' '/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1'" WMS_HOSTS=ilc-wms.desy.de
YAIM will produce:
> cat /etc/grid-security/vomsdir/calice/grid-voms.desy.de.lsc /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de /C=DE/O=GermanGrid/CN=GridKa-CA > cat /etc/grid-security/vomsdir/ilc/grid-voms.desy.de.lsc /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de /C=DE/O=GermanGrid/CN=GridKa-CA
Some services still need the certificate of the VOMS server for authentication of users. Please make sure you get:
> cd /etc/grid-security/vomsdir > wget http://grid.desy.de/voms/grid-voms.desy.de.10039.pem > ls -l /etc/grid-security/vomsdir ... -rw-r--r-- 1 root root 5132 Mar 17 15:13 grid-voms.desy.de.10039.pem ...
[top]
In the Grid authentication is done via user certificates which are issued by the national Certificationn Authorities (CA) to which the user's home institutes are connected. Find your CA at EUGridPMA.
In order to register as a user of one of the ILC VOs - as soon as you have a vailid Grid certificate - refer to CALICE (VOMS) and ILC (VOMS).
DESY hosts the ILC data repository such as Monte Carlo events and CALICE test beam data.
At DESY, all Scientifc Linux 4/5 hosts can run the Grid User Interface software which is located in AFS. See the DESY Grid User Guide for details.
Create a VOMS proxy for authentication and authorization with the Grid (example VO 'ilc'). This requires the presence of the VOMS server's host certifcate, see VOMS at DESY:
> cat $X509_VOMS_DIR/calice/grid-voms.desy.de.lsc /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de /C=DE/O=GermanGrid/CN=GridKa-CA > cat $X509_VOMS_DIR/ilc/grid-voms.desy.de.lsc /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de /C=DE/O=GermanGrid/CN=GridKa-CA > voms-proxy-init -verify -debug -voms ilc > voms-proxy-info -debug -all
For support questions please use the GGUS portal.
You might want to share issues etc. with your colleagues:
There is a job submission framework GANGA (Ganga - a computational task management tool for easy access to Grid resources.) which might be of interest. It is in use by ATLAS and LHCb. At DESY one can do a first tests by simply calling:
/afs/desy.de/project/ganga//install/ /bin/ganga
Share your experiences and opinions, e.g. via the user's mailing lists!
[top]
Here comes a set of basic Grid commands. This works for 'calice' as well as for 'ilc'. See also DESY Grid User Guide.
Get VO infos:
> lcg-infosites --vo calice all > lcg-infosites --vo calice lfc > lcg-infosites --vo calice ce > lcg-infosites --vo calice se > lcg-info --list-attrs > lcg-info --vo calice --list-ce --attrs TotalCPUs,MaxCPUTime,MaxWCTime > lcg-info --vo calice --list-ce --attrs TotalCPUs,MaxCPUTime,MaxWCTime --query CE="grid-ce5.desy.de:2119/jobmanager-lcgpbs-desy" > lcg-info --vo ilc --sed --list-service --query ServiceType=org.glite.wms
Infos on installed software (example for CE 'grid-ce5.desy.de'):
> lcg-tags -vo ilc --ce grid-cr5.desy.de --list
To enable the access to the VOMS server, some info is needed on your UI if not already there. Ask your system admin to include from the web pages VOMS at DESY so following:
> ls -l $GLITE_LOCATION/etc/vomses/calice-grid-voms.desy.de > ls -l $GLITE_LOCATION/etc/vomses/ilc-grid-voms.desy.de
or set up in your local space (home dir):
> mkdir -p $HOME/.glite > vi $HOME/.glite/vomses ... "calice" "grid-voms.desy.de" "15102" "/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de" "calice" "ilc" "grid-voms.desy.de" "15110" "/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de" "ilc" ...
Grid proxy for authentication and authorization. (You need to hold a valid Grid certificate and you need to be a member of the VO):
> voms-proxy-init -verify -debug -voms calice > voms-proxy-info -debug -all
List the catalogue and copy a file by means of its Logical File Name (LFN):
> export LFC_HOST=`lcg-infosites --vo calice lfc` > lfc-ls -l /grid/calice > lfc-ls -l /grid/calice/tb-desy/raw/conv_v0402/Run202349.000.slcio > lcg-lr -v --vo calice lfn:/grid/calice/tb-desy/raw/conv_v0402/Run202349.000.slcio > lcg-cp -v --vo calice lfn:/grid/calice/tb-desy/raw/conv_v0402/Run202349.000.slcio file:/tmp/Run202349.000.slcio
Files may be put into the DESY mass storage system (example VO 'ilc'):
> voms-proxy-init -verify -debug -voms ilc > voms-proxy-info -debug -all > lcg-cr -v --vo ilc -l lfn:/grid/ilc/test/test4711 file:/bin/sh -d dcache-se-desy.desy.de > export LFC_HOST=`lcg-infosites --vo ilc lfc` > lfc-ls -l /grid/ilc/test/test4711 > lcg-lr -v --vo ilc lfn:/grid/ilc/test/test4711 > lcg-del -a -v --vo ilc lfn:/grid/ilc/test/test4711
Registering files which are already in the DESY mass storage (example VO 'ilc'):
> voms-proxy-init -verify -debug -voms ilc:/ilc/eudet-jra1 > voms-proxy-info -debug -all > export LFC_HOST=`lcg-infosites --vo ilc lfc` > lfc-ls -l /grid/ilc/eudet-jra1/2007/tb-desy-june > lcg-rf -v --vo ilc -l lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/testag2 srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/testag2 Using grid catalog type: lfc Using grid catalog : grid-lfc.desy.de set timeout to 0 seconds Site URL to be registered: srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/testag2 File size: 585908 Alias created in Catalog: lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/testag2 guid:e2a6fbbc-c32e-4bf8-b9bc-709f45d0263f > lfc-ls -l /grid/ilc/eudet-jra1/2007/tb-desy-june/testag2 -rw-rw-r-- 1 44021 1417 585908 Jun 19 10:12 /grid/ilc/eudet-jra1/2007/tb-desy-june/testag2 > lcg-lr -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/testag2 srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/testag2
In case a file should be replicated to another Grid site's SE:
> voms-proxy-init -verify -debug -voms ilc:/ilc/eudet-jra1
> voms-proxy-info -debug -all
> lcg-lr -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
> lcg-rep -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw -d grid05.lal.in2p3.fr
Using grid catalog type: lfc
Using grid catalog : grid-lfc.desy.de
Source URL: lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
File size: 91251853
VO name: ilc
Destination specified: grid05.lal.in2p3.fr
Source URL for copy: gsiftp://dcache38.desy.de:2811//pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
Destination URL for copy: gsiftp://grid06.lal.in2p3.fr/grid06.lal.in2p3.fr:/dpmpart/part8/ilc/2007-06-21/file07a2209f-91f5-46f5-8a87-aa9d87b7ee2f.709781.0
# streams: 1
# set timeout to 0
87101440 bytes 2126.50 KB/sec avg 2158.00 KB/sec inst
Transfer took 123210 ms
Destination URL registered in LRC: srm://grid05.lal.in2p3.fr/dpm/lal.in2p3.fr/home/ilc/generated/2007-06-21/file07a2209f-91f5-46f5-8a87-aa9d87b7ee2f
> lcg-lr -v --vo ilc lfn:/grid/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
srm://grid05.lal.in2p3.fr/dpm/lal.in2p3.fr/home/ilc/generated/2007-06-21/file07a2209f-91f5-46f5-8a87-aa9d87b7ee2f
srm://dcache-se-desy.desy.de/pnfs/desy.de/ilc/eudet-jra1/2007/tb-desy-june/native/run000615.raw
Submitting jobs to the Grid (example VO 'ilc'):
> cat test.jdl
Executable = "test.sh";
StdOutput = "out";
StdError = "err";
InputSandbox = {"test.sh"};
OutputSandbox = {"out","err"};
> cat test.sh
#! /bin/sh
/bin/date
/bin/hostname -f
/usr/bin/id
voms-proxy-info -debug -all
> glite-wms-job-list-match -a test.jdl
> glite-wms-job-submit -a [-o test.jids] test.jdl
> glite-wms-job-status | [-i test.jids]
> glite-wms-job-output | [-i test.jids]
Tune your JDL by selecting only CEs (queues) with sufficient max cpu and walltime (values are given in minutes!):
> cat test2.jdl
Executable = "test.sh";
StdOutput = "out";
StdError = "err";
InputSandbox = {"test.sh"};
OutputSandbox = {"out","err"};
Requirements = other.GlueCEPolicyMaxCPUTime > 2880 &&
other.GlueCEPolicyMaxWallClockTime > 4320;
Only for job submission special configuration files are needed. If you have a gLite User Interface, but the VOs 'ilc' and 'calice' are not configured copy a local config file to ypur local space and edit it accordingly:
> ls -l $GLITE_LOCATION/etc/cms/glite_wms.conf > cp $GLITE_LOCATION/etc/cms/glite_wms.conf .
> cat ./glite_wms.conf
[
WmsClient = [
WMProxyEndpoints = "https://ilc-wms.desy.de:7443/glite_wms_wmproxy_server";
ErrorStorage = "/tmp";
ShallowRetryCount = 10;
rank = - other.GlueCEStateEstimatedResponseTime;
PerusalFileEnable = false;
RetryCount = 3;
virtualorganisation = "ilc";
MyProxyServer = "grid-px.desy.de";
AllowZippedISB = true;
requirements = other.GlueCEStateStatus == "Production";
ListenerStorage = "/tmp";
OutputStorage = "/tmp";
];
]
> glite-wms-job-submit -c ./glite_wms.conf -a test.jdl
[top]
[top]
| Last modified: Fri Dec 2 15:25:21 MET 2005 | by the DESY Grid Team: http://grid.desy.de/ |