[DESY Home] [DESY IT Home] [DESY IT Physics Computing] [Grid Computing at DESY] [DESY Computing Seminar] [Imprint]

Grid Computing at DESY DESY

[Home] [Mon/Admin] [Grid@DESY] [Certs & VOs] [VOMS] [CVMFS] [User Guide] [Install Guide] [Notes] [Talks & Posters] [Glossary] [Documentation] [Links]

In order to ensure response in case of problems, use the Global Grid User Support (GGUS) and/or your VO support rather than private e-mail contacts or internal mailing lists.


gLite Installation Log

Contents

[top]


Introduction

This page contains the description of the gLite installation/upgrade at DESY.

gLite Schema

[top]


Documentation

  • EGEE JRA1: Middleware
  • gLite
  • Service Reference Cards
  • The LCG software development portal
  • gLite Middleware - Bugs
  • project-eu-egee-pre-production-service@cern.ch
  • project-eu-egee-pre-production-users@cern.ch
  • YAIM
  • The LCG Troubleshooting Guide
  • Twiki: Berkeley Database Information Index
  • WLCG GlueSchemaUsage
  • WMSMonitor
  • LCAS/LCMAPS Overview

  • site-info.def

    SGE

  • VOMS configuration update
  • Implementation Of SGE
  • LCG-on-SGE
  • How to run an SGE farm on LCG
    Firewalls / Ports

    [top]


    Sources

    DESY SL:

    SL47 i386 Repo (DESY) [http://swrep.desy.de/SL/47/i386/SL/RPMS/]

    SL53 x86_64 Repo (DESY)> [http://swrep.desy.de/SL/53/x86_64/SL]

    CERN SL:

    CERN Repo

    WLCG OPS:

  • Release news for EGEE Operations Meeting Agenda

    gLite Docu:

  • Release plan for gLite 3.2

  • gLite 3.1
  • gLite 3.2

  • Updates to 3.1 (sl4_i386)
  • Updates to 3.2 (sl5_x86_64)

    gLite Repository:

  • R3.0
  • R3.1
  • R3.2

    YUM Repos:

  • YUM repos 3.1
  • YUM repos 3.2

    ETICS repository:

  • ETICS repo

    jpackage:

  • JPP Mirrors

    CAs:

  • LCG-CAs

    [top]


    Installation

  • QI

    YUM: Repos 3.1

      cd /etc/yum.repos.d/
    
      wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-BDII.repo
    
      yum clean all
      yum install glite-BDII
    
    

    [top]


    YAIM

    Docu:

    YAIM configuration variables siteinfo-def variables site-info.def

      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -c`-info.def  -n 
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -c`-info.def  -n 
    
      /opt/glite/yaim/bin/yaim -r -s  -n glite-WN -f config_vomsdir -d 6
    
      vi /var/ncm-yaimexec/VersionDatabase
    

    [top]


    Admin:

    Banning users:

      vi /opt/misc/config/CE/ban_users.db
    
      ssh grid-batch5
      /root/CE-ban_users.sh
    
    

    LCMAPS:

      cat /opt/misc/edg-mkgridmap.conf
    ##############################################################################
    #
    # edg-mkgridmap.conf written by AG on 2010-01-22 to use VOMS only
    #
    ##############################################################################
    
    gmf_local /opt/edg/etc/grid-mapfile-local
    
      cp /opt/misc/edg-mkgridmap.conf /opt/edg/etc/edg-mkgridmap.conf
      rm /etc/grid-security/dn-grid-mapfile
    

    Moving gridmapdir:

      cd _old_/gridmapdir
      tar -cvf /root/gridmap.tar .
    
      cd _new_/gridmapdir
      tar -xvf /root/gridmap.tar
    

    Moving info:

      cd _old_/info
      tar -cvf /root/info.tar .
    
      cd _new_/info
      tar -xvf /root/info.tar
    

    [top]


    GIIS (BDII_site):

    SiteCertMan/GIIS BDII check

      yum update glite-BDII_site
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n BDII_site
    
      chkconfig bdii on
    
      vi /etc/sudoers
    ...
    #AG Defaults    requiretty
    ...
    
      ln -sv /opt/bdii/etc/bdii.conf
      ln -sv /opt/glite/etc/gip/site-urls.conf
     

    Debug

      man slapd.conf
    ...
    Log levels are additive, and available levels are:
                          1      (0x1 trace) trace function calls
                          2      (0x2 packets) debug packet handling
                          4      (0x4 args) heavy trace debugging (function args)
                          8      (0x8 conns) connection management
                          16     (0x10 BER) print out packets sent and received
                          32     (0x20 filter) search filter processing
                          64     (0x40 config) configuration file processing
                          128    (0x80 ACL) access control list processing
                          256    (0x100 stats) stats log connections/operations/results
                          512    (0x200 stats2) stats log entries sent
                          1024   (0x400 shell) print communication with shell backends
                          2048   (0x800 parse) entry parsing
                          4096   (0x1000 cache) caching (unused)
                          8192   (0x2000 index) data indexing (unused)
                          16384  (0x4000 sync) LDAPSync replication
                          32768  (0x8000 none) only messages that get logged whatever log level is set
    
      vi /etc/bdii/bdii-slapd.conf
    ...
    loglevel        256
    ...
    
      vi /etc/syslog.conf
    ...
    #AG slapd
    local4.*                                                /var/log/slapd.log
    
    service syslog restart
    service bdii restart
    
      less /var/log/slapd.log
    

    [top]


    BDII (BDII_top):

    [twiki: InformationSystem] [twiki: BDII] [twiki: GIP] [site-info.def]

    Configuration:

      yum update glite-BDII_top 
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n BDII_top
    
      chkconfig bdii --list
      chkconfig bdii on
    
      chkconfig ldap2.4 --list  
      chkconfig ldap2.4 off
      /etc/init.d/ldap2.4 stop
    
      ln -sv /opt/bdii/etc/bdii.conf
      ln -sv /opt/glite/etc/gip/top-urls.conf
      ln -sv /var/log/bdii/bdii-update.log
      ln -sv /var/log/glite/glite-info-update-endpoints.log
    
      cat /etc/cron.hourly/glite-info-update-endpoints
      save /etc/glite/glite-info-update-endpoints.conf
      vi /etc/glite/glite-info-update-endpoints.conf
    ...
    output_file = /opt/glite/etc/gip/top-urls0.conf
    ...
    
      /etc/init.d/bdii restart
    
      ls -l /opt/bdii /var/log/bdii /var/lib/bdii /opt/glite /var/run/bdii
    

    [top]


    PX (SL4):

    glite-PX

      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n PX
    
      ln -sv /opt/glite/etc/myproxy-server.conf
    
    

    [top]


    MON (SL4): (obsolete; see APEL)

    Installation, Configs, Logs:

      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n MON
    
      ln -sv /opt/glite/etc/glite-apel-publisher/publisher-config-yaim.xml
      vi /etc/cron.d/edg-apel-publisher
    
      ln -sv /var/log/apel.log
    
      vi /etc/tomcat5/tomcat5.conf
    ...
    CATALINA_OPTS="-Xmx2025M -server -DRGMA_HOME=/opt/glite -Dsun.net.client.defaultReadTimeout=240000 -Dsun.net.client.defaultConnectTimeout=189000 -Dsun.net.inetaddr.ttl=1800"
    JAVA_HOME="/usr/lib/jvm/java-1.6.0-sun"
    
      ln -sv /var/log/tomcat5/catalina.out
    
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-batch5.desy.de' IDENTIFIED BY ''; 
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-cr4.desy.de' IDENTIFIED BY ''; 
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-cr5.desy.de' IDENTIFIED BY ''; 
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-cr6.desy.de' IDENTIFIED BY ''; 
    
    

    [top]


    WMS

    gLite Workload Management Service

    Adding seperate partition:

      fdisk /dev/i2o/hdb
      -> p
      -> n
      -> v
      -> w
      -> q
    
      mkfs.ext3 /dev/i2o/hdb
    
      vi /etc/fstab
    /dev/sda6               /local                  ext3    defaults        1 2
    

    IPCS:

      ipcs -sl
      echo "kernel.sem = 250 32000 32 1024" >> /etc/sysctl.conf
      sysctl -p
      ipcs -sl
    

    SandboxDir:

      mkdir -pv             /local/SandboxDir/
      chown -Rv glite.glite /local/SandboxDir/
      ln -sv                /local/SandboxDir  /var/glite/.
    

    MySQL:

      mkdir -pv             /local/mysql/
      chown -Rv mysql.mysql /local/mysql/
      ln -sv                /local/mysql/  /var/lib/.
    
      #/etc/init.d/mysqld stop
      #cp -vrp /var/lib/mysql /local/.
    
    

    Update: glite-WMS-3.1.30-0.slc4.i386

      vi /etc/cron.d/glite-wms-check-daemons.cron
    
      /etc/init.d/gLite stop
    
      yum --disablerepo=* --enablerepo=glite-WMS update
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n WMS
    
      vi /opt/glite/etc/glite_wms.conf
     ...
     MaxRetryCount  =  1;
     ...
    
      vi /opt/condor-c/local.`hostname -s`/condor_config.local
    #AG NORDUGRID_GAHP= $(SBIN)/nordugrid_gahp
    NORDUGRID_GAHP = /opt/condor-7.4.1/sbin/nordugrid_gahp
    
      /etc/init.d/gLite restart
    

    Installation:

      #yum install glite-WMS glite-wms-client
      #yum update glite-WMS glite-wms-client
      #yum update egi-trust-core lcg-vomscerts
    
      yum --disablerepo=* --enablerepo=glite-WMS update
    
      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      vi /opt/condor-c/etc/condor_config
    ...
    ALLOW_WRITE = *
    ...
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n WMS
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n WMS
    
      ln -s /opt/glite/yaim/log/yaimlog .
    
      ls -l /etc/grid-security/ | grep gridmapdir
    drwxrwx---   2 root glite 200704 Mar  7 12:23 gridmapdir
    
      #\cp -v /opt/misc/config/etc/grid-security/groupmapfile      /etc/grid-security/groupmapfile
      #\cp -v /opt/misc/config/etc/grid-security/voms-grid-mapfile /etc/grid-security/voms-grid-mapfile
      #\cp -v /etc/grid-security/voms-grid-mapfile                 /etc/grid-security/grid-mapfile
    
      #
      # not needed anymore (2010-06-17)
      #
      #\cp -v /opt/misc/config/etc/grid-security/glite_wms_wmproxy.gacl     /opt/glite/etc/glite_wms_wmproxy.gacl
    
      
      ln -sv /opt/glite/etc/glite_wms_wmproxy.gacl .
      ln -sv /etc/grid-security/gridmapdir .
      ln -sv /opt/glite/etc/glite_wms.conf .
      ln -sv /etc/grid-security/groupmapfile .
      ln -sv /etc/grid-security/grid-mapfile .
      ln -sv /etc/grid-security/voms-grid-mapfile .
    
      ln -sv /var/log/globus-gridftp.log .
      ln -sv /var/log/gridftp-session.log .
      ln -sv /var/log/glite/wmproxy.log .
      ln -sv /var/glite/logmonitor/CondorG.log .
      ln -sv /var/log/glite/lcmaps.log .
     
    
      vi glite_wms.conf
    ...
    WorkloadManagerProxy =  [
        ...
        SDJRequirements  =  RegExp(".*sdj$", other.GlueCEUniqueID);
        #maxInputSandboxFiles = 16;
        #maxOutputSandboxFiles = 16;
    
    WorkloadManager =  [
        ...
        MaxRetryCount  =  1;
    ...
    
      /etc/init.d/gLite restart
      #/opt/glite/etc/init.d/glite-wms-wmproxy restart
      
      vi /etc/cron.d/glite-wms-wmproxy.restart.cron
    15 15-23/3 * * * root . /opt/glite/etc/profile.d/grid-env.sh ;  /opt/glite/etc/init.d/glite-wms-wmproxy restart > /var/log/glite/glite-wms-wmproxy.restart.cron.log
    
    
      #
      # not needed anymore (2010-06-17)
      #
      #vi glite_wms_wmproxy.gacl
      #/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-wms10.desy.de
    
      #
      # Add wmsmon (replaces rbwmsmon)
      #
      rpm -ivh /opt/misc/tools/WMS/wmsmon/gLite-WMS-monitoring-1.0-8.noarch.rpm
      [Should move to Quattor soon]
    
      #
      # Adjust config on grid-mon0 to collect WMS data:
      # vi /etc/rbwmsmond.conf
      # service rbwmsmond restart
      #
    
      #
      # Make sure mapping for CMS is done to  '.cmssgm' rather than 'sgmcms'
      #
      grep cms /etc/grid-security/voms-grid-mapfile
    
      #
      # Clean of ICE needed:
      #
      /opt/glite/etc/init.d/glite-wms-ice stop
      \rm -r /var/glite/ice/persist_dir
      /opt/glite/etc/init.d/glite-wms-ice start
    
      #
      # Check proper version of YAIM
      # glite-WMS-3.1.29 needs glite-yaim-wms-4.0.7-6 (although RPM page requires only  >glite-yaim-wms-4.0.7)
      # See GGUS ticket 60733
      # Ensure it by updating all gLite packages to the most recent ones
      #
      yum --disablerepo=* --enablerepo=glite-WMS update
     
      #
      # Once re-running YAIM check the setting of  NORDUGRID_GAHP in
      # /opt/condor-c/local.$LOCAL_HOSTNAME/condor_config.local:
      #
      grep NORDUGRID_GAHP /opt/condor-c/local.`hostname -s`/condor_config.local
    #AG NORDUGRID_GAHP= $(SBIN)/nordugrid_gahp
    NORDUGRID_GAHP = /opt/condor-7.4.1/sbin/nordugrid_gahp
    

    Monitor:

      /opt/glite/sbin/glite_wms_wmproxy_load_monitor
    

    Draining:

      vi /var/glite/.drain
    
    

    Finding a job:

    Finding a job:
    
      sshr grid-lb2
    mysql -p
    mysql> use lbserver20;
    
    mysql> select * from states where jobid='slYlCTn1tRzvDZFsjeycnA';
    
    mysql> select jobid,status,parent_job,STD_lastUpdateTime from states where jobid='slYlCTn1tRzvDZFsjeycnA';
    mysql> select jobid,prog,host                            from events where jobid='slYlCTn1tRzvDZFsjeycnA';
    
    
      sshr grid-wms10
    mysql -p
    mysql> use lbproxy; 
    mysql> select jobid,name,value from short_fields where jobid='2HZYF_mWWEAyQ7A1Z3xulQ';
    


    Operational tweaks

    Jobs sent to some CE stay in Running state forever

    Stuck log_monitor

    log_monitor keeps on crashing
     /opt/glite/etc/init.d/glite-wms-lm restart
    
    Try to remove possibly corrupted irepository.dat
    
     mv /var/glite/logmonitor/internal/irepository.dat /var/glite/logmonitor/internal/irepository.dat.bak
    
    and restart log_monitor
    
     /opt/glite/etc/init.d/glite-wms-lm start
    
    Conder Commands
    List all Condor jobs
     condor_q
    
    Remove a Condor job
     condor_rm -forex 
    
    Look at /root/wms-cleanup-condor.sh for some examples
    

    Transfer UI to WMS and WN from WMS:

      /var/log/gridftp-session.log
    

    Handle 'LCMAPS' error: (wrong hard link in gridmapdir):

      glite-wms-job-submit ...
    ...
    Warning - LCMAPS failed to map user credential
    
    Method: getFreeQuota
    ...
    
    
      li /etc/grid-security/gridmapdir | less
    
      \rm %2d....
    

    Monitoring:

      glite_cream_load_monitor --show
      mysql -p
    mysql> use creamdb;
    mysql> show tables;
    mysql> explain job_status;
    mysql> select * from job_status where jobId = "CREAM419742390";
    

    Find cancelled CREAM jobs:

    BATCH:
    ...
    -E-12599 09/28/2010 11:35:52 id:7644358 u:   cmsusr082 q:       cms h:grid-wn0504 e:  1 c:     2 w:   952 0.00 m:  13 v: 459 grid-cr5
    ...
    
    CREAM:
      less /opt/glite/var/log/glite-ce-cream.log
    ...
    28 Sep 2010 11:35:52,725 INFO org.glite.ce.creamapi.jobmanagement.cmdexecutor.AbstractJobExecutor (AbstractJobExecutor.java:2094) - (Worker Thread 33) JOB CREAM138669944 STATUS CHANGED: RUNNING => DONE-FAILED [failureReason=Cannot move ISB (retry_copy ${globus_transfer_cmd} gsiftp://wms202.cern.ch: 2811/var/glite/SandboxDir/zi/https_3a_2f_2fwms202.cern.ch_3a9000_2fziYv-2eReD2CbQtjgo5Ehw/input/.BrokerInfo file:///home/cmsusr082/home_cream_138669944/CREAM138669944/.BrokerInfo): error: globus_ftp_client: the server responded with an error500 500-Command failed. : globus_l_gfs_file_open failed.500-globus_xio: Unable to open file /var/glite/SandboxDir/zi/https_3a_2f_2fwms202.cern.ch_3a9000_2fziYv-2eReD2CbQtjgo5Ehw/input/.BrokerInfo500-globus_xio: System error in open: Permission denied500-globus_xio: A system call failed: Permission denied500 End.] [localUser=cmsusr082] [gridJobId=https://wms202.cern.ch:9000/ziYv-2eReD2CbQtjgo5Ehw] [lrmsJobId=7644358] [delegationId=12856076822E512159wms2022Ecern2Ech]
    ...
    
    

    Relate job from batch to cream:

    
    cream_039793384 -> /var/glite/var/cream_sandbox/../03/CREAM039793384
    
    root@grid-batch5: [~] grep 27152403 /var/torque/server_logs/20120816 | grep "job name"
    
    08/16/2012 05:50:56;0008;PBS_Server;Job;27152403.grid-batch5.desy.de;Job Queued at request of iceprd015@grid-cr4.desy.de, owner = iceprd015@grid-cr4.desy.de, job name = cream_039793384, queue = desy
    
    root@grid-cr4: [~] ll /var/glite/var/cream_sandbox/iceprd/_C_DE_O_GermanGrid_OU_DESY_CN_Dariusz_Gora_icecube_Role_production_Capability_NULL_iceprd015/03/CREAM039793384
    
    [top]

    LB

    Firewall / Port issues:

    Firewalls / Ports

      WMS, UI: 9000
      WMS, CE: 9001
      UI:      9003
    

    Installation:

      #yum install ca-policy-egi-core
      #yum install glite-LB
    
      yum update glite-LB
      yum update
    
      #export GLITE_USER=glite
      #grep edguser /etc/passwd
      #/usr/sbin/useradd -g edguser -s /bin/bash -c "EDG User" -d /home/edguser -u 995 edguser
      #chown -Rv edguser /var/bdii /var/log/bdii
    
      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n LB
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n LB
    
      ln -vs /var/log/glite/glite-lb-purger.log .
      ln -vs /opt/glite/etc/glite-lb/glite-lb-authz.conf .
    

    Cleanup:

      ls -hl /var/lib/mysql
    
      /etc/init.d/gLite stop
      /etc/init.d/mysqld stop
    
      mv -v /var/lib/mysql /var/lib/mysql.old
    
      /etc/init.d/mysqld start
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n LB
    

    RTM: (deprecated; now done automatically)

      #cat /opt/misc/tools/LB/rtm.sql | mysql -p
    
      #/etc/init.d/mysqld restart
    
    Note: (deprecated) There is a new RTM monitoring for LB server version 2.0 or higher. It allows pushing information to RTM.
    DN of the central monitoring host needs to in /opt/glite/etc/LB-super-users, as described above.
      vi /tmp/bkindex.in 
    [
            JobIndices = {
                    [ type = "system"; name = "lastUpdateTime" ]
            }
    ]
    
      cat /tmp/bkindex.in | glite-lb-bkindex -rv
    
    Note: (deprecated) This extention to the database can take several minutes, if the LB has been in production for some time already. At least for partially the database gets locked and no job submission is possible.

    On a fresh LB server RTM can be activated using YAIM by adding these lines to the siteinfo.def file: (still to be tested!)

      GLITE_LB_RTM_ENABLED=true
      GLITE_LB_RTM_DN="/C=UK/O=eScience/OU=Imperial/L=Physics/CN=rtmsrv00.hep.ph.ic.ac.uk/emailAddress=janusz.martyniak@imperial.ac.uk"
    

    [top]


    RB (deprecated)

    Sandbox cleaner:

       vi /etc/cron.d/cleanup-sandboxes
    PATH=/sbin:/bin:/usr/sbin:/usr/bin
    31 * * * * edguser /opt/lcg/sbin/cleanup-sandboxes.sh -v >> /var/log/cleanup-sandboxes.log 2>&1
    

    BDII:

    /  vi /opt/edg/etc/edg_wl.conf
    ...
    II_Contact = "grid-bdii1.desy.de";
    ...
    

    DESY specific scripts:

      /opt/misc/tools/RB/RB_cleanup_LB.install
    
      /opt/misc/tools/GANGLIA/gRBmetric.install
      /opt/misc/tools/GANGLIA/gmetric.install
    

    DESY currently still runs two RBs (grid-rb1.desy.de and grid-rb4.desy.de). It is recommended though to switch to the WMS', using the gLite commands glite-wms-*.

    The config files are

       $EDG_LOCATION/etc/edg_wl_ui_cmd_var.conf
    and for each VO
       $EDG_LOCATION/etc/THEVONAME/edg_wl_ui.conf
    
    Please comment out the LoggingDestination in edg_wl_ui_cmd_var.conf.

    [top]


    UI

    YAIM:

      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n glite-UI
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n glite-UI
    

    Correct vomses for SCAI:

      vi /opt/glite/etc/vomses/dech-glite-io.scai.fraunhofer.de
    

    A UI for CA certs and CRLs:

      cat /etc/yum.repos.d/lcg-CA.repo
    [CA]
    name=CAs
    baseurl=http://linuxsoft.cern.ch/LCG-CAs/current
    enabled=1
    
      cd /etc/yum.repos.d/
    wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/lcg-CA.repo
    wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/glite-UI.repo
    
      yum install ca-policy-egi-core
      yum install fetch-crl
    
      vi /opt/glite/libexec/fetch-crl.sh
    

    [top]


    lcg-CE (SL4 / 3.1.*)

    Twiki: LCG-CE

    Installation/Configuration: (see also /opt/misc/config/CE)

      ipcs -sl
      echo "kernel.sem = 250 32000 32 1024" >> /etc/sysctl.conf
      sysctl -p
      ipcs -sl
    
      #yum install ca-policy-egi-core
      #yum install lcg-CE
      ###yum install glite-TORQUE_server
      #yum install glite-TORQUE_utils 
    
      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n lcg-CE -n TORQUE_utils
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n lcg-CE -n TORQUE_utils
    
      /opt/glite/yaim/bin/yaim -r -s /opt/misc/yaim/`hostname -s`-info.def -n lcg-CE -n TORQUE_utils -f config_gip_ce
    

    LCMAPS: (we enforce the usage of VOMS-proxies; we explicitly map some (pilot) local users)

    
      diff /opt/misc/config/etc/grid-security/groupmapfile       /etc/grid-security/groupmapfile
      diff /opt/misc/config/etc/grid-security/voms-grid-mapfile  /etc/grid-security/voms-grid-mapfile
      diff /opt/misc/config/etc/grid-security/grid-mapfile-local /opt/edg/etc/grid-mapfile-local
    
      \cp -v /opt/misc/config/etc/grid-security/groupmapfile       /etc/grid-security/groupmapfile
      \cp -v /opt/misc/config/etc/grid-security/voms-grid-mapfile  /etc/grid-security/voms-grid-mapfile
      \cp -v /opt/misc/config/etc/grid-security/grid-mapfile-local /opt/edg/etc/grid-mapfile-local
    
      diff /opt/misc/config/edg/etc/edg-mkgridmap.conf /opt/edg/etc/edg-mkgridmap.conf
    
      \cp -v /opt/misc/config/edg/etc/edg-mkgridmap.conf           /opt/edg/etc/edg-mkgridmap.conf
    
      /opt/misc/config/etc/grid-security/edg-mkgridmap.sh
    
      diff /opt/misc/config/etc/lcmaps/lcmaps.db /opt/glite/etc/lcmaps/lcmaps.db
    
      \cp -v /opt/misc/config/etc/lcmaps/lcmaps.db /opt/glite/etc/lcmaps/lcmaps.db
    
      cat /opt/glite/etc/lcmaps/lcmaps.db
    ...
    # policies
    withvoms:
    poolaccount -> posix_enf | vomslocalgroup
    vomslocalgroup -> vomslocalaccount
    vomslocalaccount -> posix_enf | vomspoolaccount
    vomspoolaccount -> posix_enf
    ...
    
    
      \rm -vf /etc/logrotate.d/syslog.old
    
      logrotate -f /etc/logrotate.conf
      vi /etc/logrotate.d/globus-gass-cache-marshal
    /var/log/globus/globus-gass-cache-marshal.log {
    ...
    

    Check:

      ln -sv /etc/cron.d/edg-apel-pbs-parser
      ln -sv /opt/glite/etc/glite-apel-pbs/parser-config-yaim.xml
    
      ln -sv /opt/edg/etc/edg-mkgridmap.conf
      ln -sv /etc/grid-security/voms-grid-mapfile
    
      ln -sv /opt/edg/etc/lcmaps /root/.
      ln -sv /etc/grid-security  /root/.
    
      ln -sv /opt/glite/etc/gip/ldif /root/.
    

    maui:

      cd ~/quattor/scdb9/CREAM
      ./diagnose.install
    

    GIP:

      /opt/glite/libexec/glite-info-wrapper
      ls -l /opt/glite/var/cache/gip/
    
      /opt/glite/etc/gip/plugin/glite-info-dynamic-ce
      /opt/glite/etc/gip/plugin/glite-info-dynamic-scheduler-wrapper
      /opt/glite/etc/gip/plugin/glite-info-dynamic-software-wrapper
    
      less /opt/lcg/share/doc/lcg-info-dynamic-scheduler/lcg-info-dynamic-scheduler.txt
    
      less /opt/lcg/var/gip/ldif/static-file-CE.ldif
    
      ls -l /opt/glite/etc/gip/plugin
    
      cat /opt/glite/etc/gip/plugin/glite-info-dynamic-ce
    #!/bin/sh
    
    /opt/lcg/libexec/lcg-info-dynamic-pbs /opt/glite/etc/gip/ldif/static-file-CE.ldif grid-batch5.desy.de
     | sed -e 's/GlueCEStateStatus: Production/GlueCEStateStatus: Production/'
    
      cat /opt/glite/etc/gip/plugin/glite-info-dynamic-scheduler-wrapper
    #!/bin/sh
    /opt/lcg/libexec/lcg-info-dynamic-scheduler -c /opt/glite/etc/lcg-info-dynamic-scheduler.conf
    
      cat /opt/glite/etc/gip/plugin/glite-info-dynamic-software-wrapper
    #!/bin/sh
    /opt/lcg/libexec/lcg-info-dynamic-software /opt/glite/etc/gip/ldif/static-file-Cluster.ldif
    
      /opt/lcg/libexec/lrmsinfo-pbs    -> diagnose -g    (lists all jobs)
    

    APEL:

      vi /etc/cron.d/edg-apel-pbs-parser
    34 01,13 * * * root env ...
    

    Firewall:

      nmap -P0 grid-ce4.desy.de -p 2119,2811,20000
    
      ldapsearch -x -H ldap://grid-ce4.desy.de:2170 -b o=grid 
    

    Second CE for same batch server:

    grid-batch5:
    -----------
      qmgr -c "print server" 
    
      qmgr -c "print server" > pbs.conf
    
      ssh root@grid-ce5
      man pbs_server_attributes
      man pbs_queue_attributes
    
      vi /etc/hosts.equiv
    
      vi /etc/ssh/shosts.equiv
      vi /etc/ssh/ssh_known_host
    
      vi /etc/ssh/sshd_config
    ...
            HostbasedAuthentication yes
            IgnoreUserKnownHosts yes
            IgnoreRhosts yes
    
      #cp /opt/misc/config/CE/shosts.equiv     /etc/ssh/.
      #cp /opt/misc/config/CE/ssh_known_hosts  /etc/ssh/.
    
    
    WN:
    --
      /etc/ssh/ssh_known_hosts
    

    lcg-CE with batch server:

      cd /etc/yum.repos.d/
      wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-TORQUE_server.repo
    
      yum clean all
      yum install lcg-CE glite-TORQUE_server glite-TORQUE_utils
    
      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n lcg-CE -n TORQUE_server -n TORQUE_utils
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n lcg-CE -n TORQUE_server -n TORQUE_utils
    
      vi /etc/ssh/shost.equiv
    
      vi /var/spool/pbs/server_priv/nodes
    

    [top]


    Batch server

    The batch server, although it is not a Grid service on its own nor does it need any grid middleware, can also be installed using YAIM. Below are the instructions you might need:

    cd /etc/yum.repos.d/
    wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-TORQUE_server.repo
    wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-TORQUE_client.repo
    wget http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.1/glite-TORQUE_utils.repo
    yum update
    cd ~
    yum install  glite-TORQUE_server glite-TORQUE_utils
    vi /root/siteinfo.def
    vi /root/wn-list.conf
    vi /root/users.conf
    vi /root/groups.conf
    chmod -v 755 jdk-1_5_0_14-linux-i586-rpm.bin
    ./jdk-1_5_0_14-linux-i586-rpm.bin
    /opt/glite/yaim/bin/yaim -c -s siteinfo.def -n TORQUE_server -n TORQUE_utils
    qmgr -c "set queue atlas acl_groups = atlasusr"
    qmgr -c "set queue atlas acl_groups += atlasprd"
    qmgr -c "set queue atlas acl_groups += atlassgm"
    
    Use the qmgr to add any group of local users that also should submit to this queue. Depending on the setup, you might only want to have two queues, one default and one testing. Adapt adequately.

    Commands to drain or disable queues (and publishing this correctly to the Information System):

    qmgr
    
    #Disable queue via 
    set queue QNAME enabled=false  
    #Infosystem scripts will publish queue status as 'Draining'.
    
    #Stop queue via 
    set queue QNAME started=false
    #Infosystem script will publish queue status as 'Stopped'.
    
    [top]

    VOMS

    SL5:

    Docs:

  • VOMS Server Installation and Configuration Guide

    Installation/Configuration:

      #yum install ca-policy-egi-core
      #yum install glite-VOMS_mysql --enablerepo=sl-base
    
      /opt/misc/yaim/get-hostcert.sh grid-voms.desy.de
      #/opt/misc/yaim/get-hostcert.sh grid-vomrs.desy.de
    
      scp gellrich@pal:.globus/usercert.pem  /root/gellrich_usercert.pem
    
      cp -v /opt/misc/DESY-CA/* /etc/grid-security/certificates/. 
    
      less /opt/glite/yaim/defaults/glite-voms.pre
      less $GLITE_LOCATION//yaim/examples/siteinfo/services/glite-voms_mysql
    
      vi ../vo.d/desy
    VOMS_DB_NAME="voms_desy"
    VOMS_PORT="15104"
    VOMS_DB_USER="voms"
    VOMS_DB_PASS="..."
    VOMS_ADMIN_SMTP_HOST="smtp.desy.de"
    VOMS_ADMIN_MAIL="..."
    VOMS_CORE_TIMEOUT=691200
    VOMS_ADMIN_USER_REGISTRATION="false"
    
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n VOMS
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n VOMS
    
    
      vi /etc/tomcat5/tomcat5.conf
    ... 
    CATALINA_OPTS="-Xmx2048M -server -Dsun.net.client.defaultReadTimeout=240000 -XX:MaxPermSize=1024M"
    

    Host certificates:

      ls -l /etc/grid-security/hostcert.pem
    -rw-r--r-- 1 root   root  5133 Apr 27 10:39 hostcert.pem
      ls -l /etc/grid-security/hostkey.pem
    -r-------- 1 root   root   887 Apr 27 10:39 hostkey.pem
    
      ls -l /etc/grid-security/tomcat-cert.pem
    -rw-r--r-- 1 tomcat root  5133 Apr 27 10:43 tomcat-cert.pem
      ls -l /etc/grid-security/tomcat-key.pem
    -r-------- 1 tomcat root   887 Apr 27 10:43 tomcat-key.pem
    

    Further steps: "calice desy ghep hermes hone icecube ilc ildg olympus xfel.eu zeus"

      vi vos.sh
      #! /bin/sh -x
    
      vos="calice desy ghep hermes hone icecube ilc ildg olympus xfel.eu zeus"
    for vo in $vos; do
    
      echo $vo
    
        /opt/glite/sbin/voms-db-deploy.py add-admin --vo $vo --cert /etc/grid-security/hostcert.pem
        /opt/glite/sbin/voms-db-deploy.py add-admin --vo $vo --cert /opt/misc/tmp/grid-vomrs.desy.de-cert.pem 
    
        voms-admin --vo=$vo --nousercert add-ACL-entry /$vo ANYONE VOMS_CA 'CONTAINER_READ,MEMBERSHIP_READ' TRUE
        voms-admin --nousercert --vo=$vo add-ACL-entry /$vo /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-vomrs.desy.de /C=DE/O=GermanGrid/CN=GridKa-CA ALL TRUE
        voms-admin --vo=$vo get-ACL /$vo
    
      done
    

    SL4:

  • gLite VOMS server (MySQL backend version)
  • VOMS Core Services (pdf)
  • VOMS Admin and User Guide (pdf)
  • VOMS Installation and Configuration Guide (pdf)

    Installation (not up-to-date!):

      ls -l /etc/yum.repos.d
    total 32
    -rw-r--r--  1 root root 226 Jul 14 17:27 DAG.repo
    -rw-r--r--  1 root root 131 Jul 14 17:27 glite-VOMS_mysql.repo
    -rw-r--r--  1 root root 768 Jul 14 17:27 jpackage.repo
    -rw-r--r--  1 root root  72 Jul 14 17:27 lcg-CA.repo
    
      #yum install lcg-CA
      #yum install glite-VOMS_mysql
    
      yum install glite-VOMS_mysql classpathx-mail geronimo-jaf --exclude=sun-jaf
    
      #rpm -q jakarta-commons-collections-tomcat5
      #yum install jakarta-commons-collections-tomcat5
      #ln -sv /usr/share/java/commons-collections-tomcat5.jar /usr/share/tomcat5/common/lib/[commons-collections-tomcat5].jar
    
      #rpm -q javamail
      #yum install javamail
      #ln -sv /usr/share/java/glassfish-javamail.jar /usr/share/tomcat5/common/lib/[javamail].jar
    
      /opt/misc/yaim/get-hostcert.sh grid-voms.desy.de
      /opt/misc/yaim/get-hostcert.sh grid-vomrs.desy.de
    
      scp gellrich@pal:.globus/usercert.pem  /root/gellrich_usercert.pem
    
      cp -v /opt/misc/DESY-CA/* /etc/grid-security/certificates/.
    

    Configuration:

      /etc/init.d/mysqld start
      /usr/bin/mysqladmin -u root password '...'
      /usr/bin/mysqladmin -u root -h `hostname -f` password '...'
      /etc/init.d/mysqld restart
    
      #mysql -p
    #mysql> GRANT ALL PRIVILEGES ON *.* to 'voms'@'localhost' identified by '...';
    
      chkconfig mysqld on
      chkconfig --list mysqld
    
      ls -l /opt/glite/etc/config
      cp -v /opt/misc/install/VOMS//*.xml /opt/glite/etc/config/.
    
      #mkdir -p /var/glite/etc
      #ls -l  /var/glite/etc/voms-admin
      #cp -vr /opt/misc/install/VOMS/voms-admin /var/glite/etc/.
    
      vi /opt/glite/etc/config/glite-global.cfg.xml
    JAVA_HOME value="/usr/lib/jvm/java-1.5.0-sun/"
    #JAVA_HOME value="/usr/lib/jvm/java-1.6.0/"
    
      vi /opt/glite/etc/config/glite-security-utils.cfg.xml
    cron.mailto value="localhost"
    
      vi /opt/glite/etc/config/glite-voms-server.cfg.xml
    voms.db.type value="mysql"
    voms.db.host value="localhost"
    voms.admin.smtp.host value="smtp.desy.de"
    voms.mysql.admin.password value="..."
    
      export GLITE_LOCATION=/opt/glite
      export GLITE_LOCATION_VAR=/var/glite
      export GLITE_LOCATION_LOG=/var/log/glite
    
      cp /opt/misc/backups/grid-voms0.desy.de/voms-db_grid-voms0.desy.de_20091216-011101.sql.gz .
      gunzip voms-db_grid-voms0.desy.de_20091216-011101.sql.gz
      cat voms-db_grid-voms0.desy.de_20091216-011101.sql | mysql -p
    
    
      #mysqladmin -p drop voms_calice
      #cat /opt/misc/tmp/voms_calice.sql | mysql -p
    
      #cat /var/glite/etc/voms-admin/*/vomses
    
      /opt/glite/etc/config/scripts/glite-voms-server-config.py -c
    
      /opt/glite/etc/config/scripts/glite-voms-server-config.py --configure [--vo=]
      /opt/glite/etc/config/scripts/glite-voms-server-config.py --start     [--vo=]
      /opt/glite/etc/config/scripts/glite-voms-server-config.py --stop      [--vo=]
    
      ln -sv /var/log/tomcat5/catalina.out
    
    
      vi /var/glite/etc/voms-admin//vomses
    ...
    grid-voms.desy.de
    ...
    
    
      vi /opt/glite/etc/voms/>vo>/voms.conf
    ...
    --uri=grid-voms.desy.de:151NN
    ...
    
    
      /opt/glite/bin/voms-admin --vo=olympus --nousercert add-ACL-entry /olympus ANYONE VOMS_CA 'CONTAINER_READ,MEMBERSHIP_READ' TRUE
    
      /opt/glite/bin/voms-admin --nousercert --vo=olympus add-ACL-entry /olympus /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-vomrs.desy.de /C=DE/O=GermanGrid/CN=GridKa-CA ALL TRUE
      voms-admin --vo=olympus get-ACL /olympus
    
      voms-admin --verbose --vo desy --nousercert list-user-roles "/O=GermanGrid/OU=DESY/CN=Andreas Gellrich" "/C=DE/O=GermanGrid/CN=GridKa-CA"
    

    Old:

      #vi /var/glite/etc/voms-admin/calice/voms.database.properties
      #/opt/glite/sbin/voms-admin-configure --vo=calice upgrade
      
      #vos="calice desy ghep hermes hone icecube ilc ildg xfel.eu zeus"
      #for vo in $vos; do voms-admin --vo=$vo --nousercert add-ACL-entry /$vo ANYONE VOMS_CA 'CONTAINER_READ,MEMBERSHIP_READ' TRUE ; done
    

    SL3:

  • gLite VOMS installation guide

    Installation:

      apt-get update
      apt-get install ca-policy-egi-core
      apt-get install glite-VOMS_mysql
    

    Configuration:

      sshr grid-voms1
      scp /opt/glite/etc/config/*.xml grid-voms2:/opt/glite/etc/config/.
    
      #cd /opt/glite/etc/config
      #cp templates/* .
    
      #rm vo-list.cfg.xml
      #rm glite-rgma-servicetool-externalServices.cfg.xml
    
      #grep changeme *
    
      vi /opt/glite/etc/config/glite-global.cfg.xml
    JAVA_HOME value="/usr/java/j2sdk1.4.2_12"
    tomcat.CATALINA_OPTS value="-Xms256M -Xmx2048M -server -Dsun.net.client.defaultReadTimeout=240000"
    
      vi /opt/glite/etc/config/glite-rgma-common.cfg.xml
    rgma.server.hostname value="grid-rgma.desy.de"
    rgma.schema.hostname value="grid-rgma.desy.de"
    rgma.registry.hostname lcgic01.gridpp.rl.ac.uk
    
      vi /opt/glite/etc/config/glite-rgma-servicetool.cfg.xml
    rgma.servicetool.siteId value="grid-rgma.desy.de"
    
      vi /opt/glite/etc/config/glite-security-utils.cfg.xml
    cron.mailto value="localhost"
    
      vi /opt/glite/etc/config/glite-voms-server.cfg.xml
    voms.db.type value="mysql"
    voms.db.host value="localhost"
    voms.admin.smtp.host value="smtp.desy.de"
    voms.mysql.admin.password value="..."
    
      VO settings
    
      ls -l /etc/grid-security/
    
      scp gellrich@grid-ui:.globus/usercert.pem /root/gellrich_usercert.pem
    
      /opt/glite/etc/config/scripts/glite-voms-server-config.py -c
    
      /opt/glite/etc/config/scripts/glite-voms-server-config.py --configure
    
      /opt/glite/etc/config/scripts/glite-voms-server-config.py --start
    
      cat /opt/glite/etc/voms/*/voms.conf
    
      echo "--timeout=691200" >> /opt/glite/etc/voms/calice/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/desy/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/ghep/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/hermes/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/hone/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/icecube/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/ilc/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/ildg/voms.conf
      echo "--timeout=691200" >> /opt/glite/etc/voms/zeus/voms.conf
    

    Edit 'configuration' page:

      cat /var/glite/etc/voms-admin/*/vomses
    ... "grid-voms.desy.de" ... "/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de" ...
    ...
    

  • https://grid-voms1.desy.de:8443/vomses/

    Remove a VO (example VO 'baikal'):

      rm /var/glite/etc/voms-admin/baikal/*.save
      /opt/glite/sbin/voms-admin-configure --verbose remove --vo baikal --dbapwdfile ./x
    

    [top]


    VOMRS

  • VOMRS Project
  • VomrsInstallGuide
  • VOMRS FAQ
  • vomrs-grid-support@cern.ch

    Installation: (to be done on top of a VOMS server with one dummy VO. See above)

      cp -v /opt/misc/DESY-CA/* /etc/grid-security/certificates/.
      
      mysql> GRANT ALL PRIVILEGES ON *.* to 'voms'@'grid-voms1.desy.de' identified by '';
    
      cd /root
    
      Downloads
    
      wget http://www.fnal.gov/docs/products/vomrs/RPMS/vomrs-1.3-4a.noarch.rpm 
      wget http://www.fnal.gov/docs/products/vomrs/RPMS/vomrs-client-1.3-4a.noarch.rpm
    
      rpm -ivh vomrs-1.3-4a.noarch.rpm
      rpm -ivh vomrs-client-1.3-4a.noarch.rpm
    
      ls -l /opt/vomrs-1.3
      export VOMRS_LOCATION=/opt/vomrs-1.3
    

    Configuration of VOs:

      cd /var/lib/tomcat5/
      ln -sv /var/log/tomcat5 logs
      ln -sv /etc/tomcat5     conf
    
      cp -r /opt/misc/config/VOMRS/cfg/ .
    
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/calice.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/desy.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/ghep.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/hermes.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/hone.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/icecube.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/ilc.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/ildg.cfg 
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/olympus.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/xfel.eu.cfg
      /opt/vomrs-1.3/sbin/vomrs_configure --autorun -f ~/cfg/zeus.cfg
    
      cp /opt/misc/backups/grid-vomrs1.desy.de/voms-db_grid-vomrs1.desy.de_20091211-102505.sql.gz .
      gunzip voms-db_grid-vomrs1.desy.de_20091211-102505.sql.gz
      cat voms-db_grid-vomrs1.desy.de_20091211-102505.sql.gz | mysql -p
    
      #/opt/vomrs-1.3/sbin/release_scripts/vomrs_db_upgrade_1.3.4
    
      #cat /opt/vomrs-1.3/etc/profile.d/vomrs.sh
    #...
    #export JAVA_HOME=$JAVA_HOME
    #export PATH=$PATH:$JAVA_HOME/bin
    
    
      /opt/vomrs-1.3/sbin/add_admin --vo calice --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo desy --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo ghep --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo hermes --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo hone --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo icecube --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo ilc --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo ildg --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo olympus --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo xfel.eu --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
      /opt/vomrs-1.3/sbin/add_admin --vo zeus --file /root/gellrich_usercert.pem --org DESY --firstname Andreas --lastname Gellrich --phone "+49 40 8998 2732"
    

    Start:

      less /opt/vomrs-1.3/sbin/vomrs_hermes_load.cfg
      vi /opt/vomrs-1.3/var/etc/vomrs_hermes/vomrs.xml
    
      service vomrs start [hermes]
    
      https://grid-vomrs1.desy.de:8443/vo/hermes/vomrs
    
      ln -sv /var/log/tomcat5/catalina.out
      ln -sv /var/log/vomrs/vomrs_hermes.log
    

    Prepare VOMS server:

      openssl x509 -subject -noout -in /etc/grid-security/hostcert.pem
    
      vos="calice desy ghep hermes hone icecube ilc ildg xfel.eu zeus"
      for vo in $vos; do /opt/glite/sbin/voms-db-deploy.py add-admin --vo $vo --dn /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-vomrs.desy.de --ca /C=DE/O=GermanGrid/CN=GridKa-CA --email gridka-ca@iwr.fzk.de ; done
    

    The VOMRS client:

      /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs.desy.de 8443 vo/ilc getServices
    
      /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs.desy.de 8443 vo/ilc getServiceRoles RegisterMember
    
      /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs.desy.de 8443 vo/ilc getServiceArguments RegisterMember VOAdmin
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs1.desy.de 8443 vo/desy GetServiceArguments AddVOAdmin VOAdmin
    - Client CN=host/grid-vomrs1.desy.de, OU=DESY, O=GermanGrid, C=DE accepted
    DN CA
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs1.desy.de 8443 vo/desy GetServiceArguments AddSite VOAdmin
    - Client CN=host/grid-vomrs1.desy.de, OU=DESY, O=GermanGrid, C=DE accepted
    INSTITUTION
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs1.desy.de 8443 vo/desy GetServiceArguments AddGroup VOAdmin
    - Client CN=host/grid-vomrs1.desy.de, OU=DESY, O=GermanGrid, C=DE accepted
    GROUP ACCESS DESCRIPTION
    

    Examples:

    gn="Dum"
    sn="Dummy"
    voms="grid-vomrs.desy.de"
    vo="calice"
    dn="/C=DE/O=GermanGrid/OU=DESY/CN=$gn $sn"
    ca="/C=DE/O=GermanGrid/CN=GridKa-CA"
    email="Andreas.Gellrich@desy.de"
    inst="DESY"
    ph="4711"
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient \
    ${voms} 8443 \
    vo/${vo} \
    RegisterMember \
    "${dn}" "${ca}" \
    "$sn" \
    "${inst}" \
    "/O=GermanGrid/OU=DESY/CN=Andreas Gellrich" "/C=DE/O=GermanGrid/CN=GridKa-CA" \
    "full" \
    "${email}" \
    "N" \
    "First name,$gn,Last name,$sn,Phone,$ph"
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs1.desy.de 8443 vo/desy AddSite CERN
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs1.desy.de 8443 vo/desy AddGroup /desy/gkss Open GKSS
    
    /opt/vomrs-1.3/client/bin/vomrs_soapclient grid-vomrs1.desy.de 8443 vo/desy AddVOAdmin "/O=GermanGrid/OU=DESY/CN=Christoph Wissing" "/C=DE/O=GermanGrid/CN=GridKa-CA"
    

    [top]


    APEL

    SL5 stand-alone service:

    Docs:

  • GOC Accounting (Plots and docs)
  • APEL FAQ

    Accounting:

  • APEL Synchronisation Test

      yum install ca-policy-egi-core
      yum install glite-APEL --enablerepo=sl-base
    
      #rpm -ivH http://linuxsoft.cern.ch/EGEE/gLite/R3.2/glite-GENERIC/sl5/x86_64/RPMS.externals/fetch-crl-2.7.0-2.noarch.rpm
    
      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      /etc/init.d/mysqld start
      /usr/bin/mysqladmin -u root password ''
      /usr/bin/mysqladmin -u root -h grid-apel0.desy.de password ''
      /etc/init.d/mysqld restart
    
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n APEL
    
      chkconfig mysqld --list
      #chkconfig mysqld on
    

    Error handling:

      vi /opt/glite/bin/apel-publisher
    ...
    -Xmx2048m
    ...
    
    
      myisamchk --help
    
      myisamchk --check /var/lib/mysql/accounting/BlahdRecords.MYI
    
      myisamchk -o /var/lib/mysql/accounting/BlahdRecords.MYI
    

    MySQL:

      mysql -p
    
      mysql> GRANT ALL ON *.*  TO 'accounting'@'grid-apel0.desy.de' IDENTIFIED BY '';
      mysql> GRANT ALL ON *.*  TO 'accounting'@'localhost' IDENTIFIED BY '';
    
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-batch5.desy.de' IDENTIFIED BY '';
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-ce5.desy.de'    IDENTIFIED BY '';
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-cr4.desy.de'    IDENTIFIED BY '';
      mysql> GRANT ALL ON accounting.*  TO 'accounting'@'grid-cr5.desy.de'    IDENTIFIED BY '';
    
    
      mysql> SHOW GRANTS FOR 'accounting'@'localhost';
    
      mysql> SHOW GRANTS FOR 'accounting'@'grid-batch5.desy.de';
      mysql> SHOW GRANTS FOR 'accounting'@'grid-ce5.desy.de';
      mysql> SHOW GRANTS FOR 'accounting'@'grid-cr4.desy.de';
      mysql> SHOW GRANTS FOR 'accounting'@'grid-cr5.desy.de';
    
      mysql> use accounting;
      mysql> SHOW TABLES;
      mysql> OPTIMIZE TABLE BlahdRecords;
      mysql> OPTIMIZE TABLE EventRecords;
      mysql> OPTIMIZE TABLE GkRecords;
      mysql> OPTIMIZE TABLE LcgProcessedFiles;
      mysql> OPTIMIZE TABLE LcgRecords;
      mysql> OPTIMIZE TABLE MessageRecords;
      mysql> OPTIMIZE TABLE RepublishInfo;
      mysql> OPTIMIZE TABLE SpecRecords;
      mysql> OPTIMIZE TABLE SpecRecords_28593;
    

    [top]


    LFC

    LCG File Catalog The LCG Troubleshooting Guide

    LFC API / CLI

      #wget ftp://fr2.rpmfind.net/linux/dag/redhat/el5/en/x86_64/dag/RPMS/perl-XML-RegExp-0.03-1.2.el5.rf.noarch.rpm
      #wget ftp://fr2.rpmfind.net/linux/dag/redhat/el5/en/x86_64/dag/RPMS/perl-XML-DOM-1.44-2.el5.rf.noarch.rpm
      #yum --enablerepo=dag install glite-LFC_mysql
      #yum install glite-LFC_mysql
    
      /opt/misc/yaim/get-hostcert.sh `hostname -f`                         # must contain real hostname!
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n LFC_mysql
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n LFC_mysql
    
      ln -sv /opt/lcg/etc/NSCONFIG
      ln -sv /opt/lcg/etc/lcgdm-mapfile
      ln -sv /var/log/lfc/log /root/lfc.log
      ln -sv /var/log/dli/log /root/dli.log
    
      cat /opt/lcg/etc/NSCONFIG
    lfc/msql@grid-lfc1.desy.de/cns_db
    

    Migrate DB from other LFC:

      cat lfc-db_grid-lfc0.desy.de_20110106-140001.sql | mysql -p 
      /etc/init.d/mysqld restart
    
      mysqladmin -p -u root -h localhost       password '...'
      /etc/init.d/mysqld restart
    
    mysql> GRANT ALL PRIVILEGES ON *.* TO 'lfc'@'grid-lfc2.desy.de' IDENTIFIED BY "...";
    
    mysql> show grants for 'root'@'grid-lfc0.desy.de';
    mysql> show grants for 'root'@'grid-lfc1.desy.de';
    mysql> show grants for 'root'@'grid-lfc2.desy.de';
    
    mysql> show grants for 'lfc'@'grid-lfc0.desy.de';
    mysql> show grants for 'lfc'@'grid-lfc1.desy.de';
    mysql> show grants for 'lfc'@'grid-lfc2.desy.de';
    

    MySQL:

      mysql -p
    mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'grid-db0.desy.de' IDENTIFIED BY "...";
    
      /etc/init.d/mysql restart
    

    When renewing the host certificate, make sure it is copied to /etc/grid-security/lfcmgr as well:

    root@grid-lfc1: [~] ll /etc/grid-security/host*.pem
    -rw-r--r--    1 root     root         5132 Jan  7 14:49 /etc/grid-security/hostcert.pem
    -r--------    1 root     root          887 Jan  7 14:49 /etc/grid-security/hostkey.pem
    root@grid-lfc1: [~] ll /etc/grid-security/lfcmgr/lfc*.pem
    -rw-r--r--    1 lfcmgr   lfcmgr       5132 Jan  9 15:52 /etc/grid-security/lfcmgr/lfccert.pem
    -r--------    1 lfcmgr   lfcmgr        887 Jan  9 15:53 /etc/grid-security/lfcmgr/lfckey.pem
    

    [top]


    DPM

    Disk Pool Manager

    The installation was done on one host.

    default-info.def:

    DPM_HOST=grid-se0.desy.de
    DPMPOOL=datapool
    DPM_FILESYSTEMS="grid-se0.desy.de:/data"
    DPMMGR=dpmmgr
    DPMUSER_PWD=xxx
    DPMFSIZE=1G
    DPM_DB_HOST=grid-se0.desy.de
    DPM_DB_USER=dpmmgr
    DPM_DB_PASSWORD=_secret_
    

    Tests:

      dpm-qryconf
    POOL datapool DEFSIZE 1024.00M GC_START_THRESH 0 GC_STOP_THRESH 0 DEFPINTIME 0 PUT_RETENP 86400 FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GID 0 S_TYPE -
                                  CAPACITY 134.83G FREE 127.95G ( 94.9%)
      grid-se0.desy.de /data CAPACITY 134.83G FREE 127.95G ( 94.9%)
    
      pp dpnsdaemon
      pp dpm
      pp srmv1
      pp srmv2
    
      ls -la /data
    
      cat /etc/sysconfig/dpm
    
      cat /etc/sysconfig/dpnsdaemon
    
      cat /etc/sysconfig/dpm-gsiftp
    
      cat /etc/sysconfig/rfiod
    
      cat /opt/lcg/etc/DPMCONFIG
    dpmmgr/msql@grid-se0.desy.de
    
      cat /etc/shift.conf
    RFIOD TRUST grid-se0.desy.de
    RFIOD WTRUST grid-se0.desy.de
    RFIOD RTRUST grid-se0.desy.de
    RFIOD XTRUST grid-se0.desy.de
    RFIOD FTRUST grid-se0.desy.de
    DPM TRUST grid-se0.desy.de
    DPNS TRUST grid-se0.desy.de
    

    Logs:

      /var/log/dpm-gsiftp/dpm-gsiftp.log
      /var/log/dpm/log
    

    Usage tests:

      export DPNS_HOST=grid-se0.desy.de
    
      dpns-ls -lR /
      dpns-ls -l /dpm/desy.de/home
    
      dpns-getacl /dpm/desy.de/home/geant4
      dpns-entergrpmap --gid 122 --group geant
      dpns-chown root::geant /dpm/desy.de/home/geant4
    
      dpns-mkdir /dpm/desy.de/home/dteam/test
      dpns-ls -l /dpm/desy.de/home/dteam
    
      lcg-cp -v --vo dteam gsiftp://grid-se0.desy.de/etc/passwd file:$PWD/x
    
      lcg-cp -v --vo dteam file:/bin/sh gsiftp://grid-se0.desy.de/dpm/desy.de/home/dteam/SET_testfile
    

    [top]


    Ganglia

    http:

      rpm -qf /etc/httpd/conf/httpd.conf
    
      chkconfig httpd on
      service httpd start
      
      cd /var/www/html
    
      cd /var/www/cgi-bin
    

    #  vi /etc/php.ini
    #...
    #memory_limit = 128M ;
    #...
    
      vi /etc/gmetad.conf
      chkconfig gmetad on
      service gmetad start
    
      ln -sv /usr/share/ganglia /var/www/html/.
    
      vi /var/www/html/ganglia/get_context.php
    ...
    if (!$sort)
          $sort = "by name";
    ...
    
      vi /var/www/html/ganglia/conf.php
    ...
    $default_range = "day";
    ...
    
      vi /var/www/html/ganglia/graph.php
    ...
    "-W DESY --width $width --height $height $upper_limit $lower_limit ".
    ...
    

    More features:

      vi /var/www/html/ganglia/graph.php
    ...
          else if ($graph == "power_report")
             {
                $style = "POWER";
    
                $lower_limit = "--lower-limit 0 --rigid";
                $extras = "--base 1024";
                $vertical_label = "--vertical-label 'W'";
    
                $series = "DEF:'POWER'='${rrd_dir}/POWER.rrd':'sum':AVERAGE "
                   ."LINE2:'POWER'#$mem_cached_color:'In' ";
             }
    ...
    
      vi /var/www/html/ganglia/get_context.php
    ...
    $reports = array(
    ...
    
      vi /var/www/html/ganglia/conf.php
    ...
      $optional_graphs = array('power');
    ...
    
      vi /usr/share/ganglia/templates/default/cluster_view.tpl
    
      vi /usr/share/ganglia/templates/default/host_view.tpl
    
    

    [top]


    VOBOX

    Get host certs for the machine Install the repos and the software:
      #yum update
      #yum install ca-policy-egi-core
      #yum install glite-VOBOX
    
    Configure the VOBOX:
      /opt/misc/yaim/get-hostcert.sh $(/bin/hostname -f)
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n glite-VOBOX
    
    For all accounts, that are needed, make sure a login is possible:
      grep NP /etc/shadow
    atlasusr000:*NP*:13909:0:99999:7:::
    atlasusr101:*NP*:13909:0:99999:7:::
    
    Make sure CA certificates are installed, if they are not:
      yum update ca-policy-egi-core
      /etc/init.d/gsisshd restart
    
    Probably, you want to maintain /etc/grid-security/grid-mapfile by hand:
      rm /etc/cron.d/edg-mkgridmap
      vi /etc/grid-security/grid-mapfile
    "/O=GermanGrid/OU=DESY/CN=Yves Kemp2" atlasusr000
    
      vi /etc/cron.d/edg-mkgridmap
    # ...
    
    If you want to mount PNFS, first have an export allowed by the DOT group:
      vi fstab
    ...
    dcache-dir-atlas:/atlas   /pnfs/fs/atlas  nfs intr,hard,rw,noac,hard,vers=2 0 0
    
      mkdir -vp /pnfs/fs/atlas
      mount     /pnfs/fs/atlas/
      ln -sv    /pnfs/fs /pnfs/desy.de
    

    SCAS

    Documentation glite-SCAS

    SCAS:

  • SCAS
  • Installation recipe for SCAS daemon (in 20 steps)
  • SCAS (and gLExec) Preliminary Test Plan

    GLExec:

  • GLExec
  • GLexec installation

    SCAS server Installation:

      yum install glite-SCAS
    
    SCAS server Configuration:
      
      /opt/misc/yaim/get-hostcert.sh $(/bin/hostname -f)
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/$(/bin/hostname -s)-info.def -n SCAS
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/$(/bin/hostname -s)-info.def -n SCAS
    
      ln -sv /opt/glite/etc/scas.conf .
      ln -sv /opt/glite/etc/lcas/ban_users.db .
    
      \cp -v /opt/misc/config/etc/grid-security/groupmapfile      /etc/grid-security/groupmapfile
      \cp -v /opt/misc/config/etc/grid-security/voms-grid-mapfile /etc/grid-security/voms-grid-mapfile
      \cp -v /etc/grid-security/voms-grid-mapfile                 /etc/grid-security/grid-mapfile
    
      \cp -v /opt/misc/config/CE/edg-mkgridmap.conf /opt/edg/etc/edg-mkgridmap.conf
      \cp -v /opt/misc/config/CE/grid-mapfile-local /opt/edg/etc/grid-mapfile-local
      \rm /etc/grid-security/dn-grid-mapfile
      /opt/edg/sbin/edg-mkgridmap --output=/etc/grid-security/dn-grid-mapfile
      cp /etc/grid-security/dn-grid-mapfile /etc/grid-security/grid-mapfile.tmp; cat /etc/grid-security/voms-grid-mapfile >> /etc/grid-security/grid-mapfile.tmp; mv /etc/grid-security/grid-mapfile.tmp /etc/grid-security/grid-mapfile
    

    Logs:

      ln -sv /var/log/scas-mkgridmap.log .
      ln -sv /var/log/glite/scas.log .
    

    GLExec client installation:

      vi /opt/glite/etc/glexec.conf
    ...
    user_white_list              = *
    user_white_list              = ".atlasplt,.cmsplt"
    ...
    log_destination              = "syslog" | "file"
    

      ls -l /opt/glite/sbin/glexec
    
    -r-xr-xr-x 1 root glexec 79792 Jun 11  2009 /opt/glite/sbin/glexec   [chmod -v 5555] 'log-only'
    
    -r-sr-sr-x 1 glexec glexec 79792 Jun 11  2009 /opt/glite/sbin/glexec [chmod -v 6555] 'setuid'
    
    
      chown -v root   /opt/glite/sbin/glexec; chmod -v 5555 /opt/glite/sbin/glexec
      chown -v glexec /opt/glite/sbin/glexec; chmod -v 6555 /opt/glite/sbin/glexec
    

    [top]


    ARGUS

    Documentation:

  • Argus, The EGEE Authorization Service
  • Argus: Policy Administration Point (PAP): Configuration

    ARGUS server Installation:

      yum install glite-ARGUS
    
    ARGUS server Configuration:
      
      /opt/misc/yaim/get-hostcert.sh $(/bin/hostname -f)
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/$(/bin/hostname -s)-info.def -n ARGUS_server
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/$(/bin/hostname -s)-info.def -n ARGUS_server
    

    Configurations:

      /opt/argus/pap/conf/pap_authorization.ini
      /opt/argus/pap/conf/pap_configuration.ini
    
      /opt/argus/pdp/conf/pdp.ini
    
      /opt/argus/pepd/conf/pepd.ini
    

    Services:

      /etc/init.d/pap-standalone
      /etc/init.d/pdp
      /etc/init.d/pepd
    

    Logs:

    
    
    

    [top]


    CREAM

  • CREAM CE Home Page

    useful links:

  • Install cream32
  • How to Check and Test your CREAMCE
  • Error Messages Reported by CREAM to Client
  • CREAM and Blparser Configuration
  • How To Purge Jobs From The CREAMDB
  • Using CEMonitor CLI
  • How To Drain A CREAM-CE

    Installation:

    #
    # yum
    #
    
      # due to dependency problems of globus 
      yum install c-ares-1.3.0-4.sl5
    
      #yum install glite-CREAM glite-TORQUE_server glite-TORQUE_utils
      yum install glite-CREAM glite-TORQUE_utils
    
      yum install perl-Date-Manip  
      yum install xml-commons-apis
    
      vi /etc/security/limits.conf
    #AG
    tomcat          soft    nofile  63536
    tomcat          hard    nofile  63536
    
    tomcat          soft    nproc   16384
    tomcat          hard    nproc   16384
    

    #
    # as long as we haven't upgraded the torque server
    #
      ### server was updated on 2012-01-19 yum downgrade torque-2.3.13-1.el5.x86_64 torque-client-2.3.13-1.el5.x86_64 libtorque-2.3.13-1.el5.x86_64
    
    #
    # file system
    #
    
      CREAM_SANDBOX_PATH="/var/glite/var/cream_sandbox"
    
      -> /opt/misc/yaim/`/bin/hostname -s`-info.def
      #CREAM_SANDBOX_PATH="/home/glite/var/cream_sandbox"
      #chmod go+xr /home/glite
      #chmod go+rx /home/glite
      #mkdir -p /home/glite/var/cream_sandbox
      #chown glite.glite /home/glite/var
      #chown tomcat.tomcat /home/glite/var/cream_sandbox
      #chmod g+w /home/glite/var/cream_sandbox
      #ln -s /home/glite/var/cream_sandbox /opt/glite/var/.
    
    #
    # admins
    #
      vi /etc/grid-security/admin-list
    "/O=GermanGrid/OU=DESY/CN=Andreas Gellrich"
    "/O=GermanGrid/OU=DESY/CN=Christoph Wissing"
    "/C=DE/O=GermanGrid/OU=DESY/CN=Dmitry Ozerov"
    
    #
    # yaim
    #
      /opt/misc/yaim/get-hostcert.sh `hostname -f`
    
      #/opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`/bin/hostname -s`-info.def -n creamCE -n TORQUE_server -n TORQUE_utils
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`/bin/hostname -s`-info.def -n creamCE -n TORQUE_utils
    
      /opt/glite/yaim/bin/yaim -r -s /opt/misc/yaim/`/bin/hostname -s`-info.def -n creamCE -f config_cream_blparser
    
    #
    # tomcat
    #
      save /etc/tomcat5/tomcat5.conf
      cp   /etc/tomcat5/tomcat5.conf.save /etc/tomcat5/tomcat5.conf
    ...
    #AG
    JAVA_OPTS="-server -Xms512m -Xmx4096m -Dglite.log.path=/opt/glite/var/log"
    ...
      service tomcat5 restart
    
    #
    # configs
    #
      save /opt/lcg/etc/cleanup-grid-accounts.conf
      vi   /opt/lcg/etc/cleanup-grid-accounts.conf
    ...
    #AG
    IDLE=10
    
      vi /etc/cron.d/cleanup-grid-accounts 
     1 */3 * * * root /opt/lcg/sbin/cleanup-grid-accounts.sh -v -F >> /var/log/cleanup-grid-accounts.log 2>&1
    
    #
    #
    #
      vi /opt/glite/etc/glite-ce-cream/cream-config.xml
    
    #
    # apel
    #
      vi /etc/cron.d/edg-apel-pbs-parser
    
    #
    # mysql
    #
      vi /etc/my.cnf
    [mysqld]
    max_connections=450
    
    # AG (default is 8M)
    innodb_buffer_pool_size=64M
    
    ...
    
    #
    # https://ggus.eu/tech/ticket_show.php?ticket=71830
    # https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind1108&L=lcg-rollout&D=0&P=124288
    #
    # (qstat -f does NOT pass 'Resource_List.walltime')
    #
      > save /opt/lcg/libexec/lcg-info-dynamic-scheduler
      > cp /opt/misc/tmp/lcg-info-dynamic-scheduler /opt/lcg/libexec/lcg-info-dynamic-scheduler
    
    #
    # logs
    #
      ln -s /var/log/tomcat5/catalina.out .
    
      ln -sv /opt/glite/var/log/glite-ce-cream.log .
      ln -sv /opt/glite/var/log/glite-ce-monitor.log .
      ln -sv /opt/glite/var/log/glite-pbsparser.log .
      ln -s /var/log/tomcat5/glite-security-trustmanager.log .
    
      ln -s /var/log/apel.log .
    
      #ln -sv /opt/glite/var/log/glite-ce-bnotifier.log . 
    
    #
    # for batch server on other host
    #
      cd ~/quattor/scdb9/desy-tools/CREAM
      ./CREAM-check-tomcat.install
      ./blahp-logs-cron.install
      ./diagnose.install
    
     
      ssh-keygen -t dsa -f ~/.ssh/id_dsa -N '' ; cat ~/.ssh/id_dsa.pub
    
      su - edguser
      ssh-keygen -t dsa -f ~/.ssh/id_dsa -N '' ; cat ~/.ssh/id_dsa.pub
    
      #
      # Note: grid-batch5:.ssh/authorized_keys2 is handled by Quattor
      #
      #       cfg/sites/desy-it/service/grid-ce-ssh-keys.tpl
      #
    
    #
    # TORQUE server
    #
    vi /etc/hosts.equiv
    
    qmgr -c "
    set queue atlas      acl_hosts += grid-cr5.desy.de
    set queue cms        acl_hosts += grid-cr5.desy.de
    set queue des        acl_hosts += grid-cr5.desy.de
    set queue operations acl_hosts += grid-cr5.desy.de"
    

    Config:

      vi /opt/glite/etc/glite-apel-pbs/parser-config-yaim.xml
    
      /opt/glite/etc/blparser.conf
      /opt/glite/etc/glexec.conf
      #/opt/glite/etc/blah.config
    

    Tests:

      glite-ce-job-submit -a -r grid-cr5.desy.de:8443/cream-pbs-desy cream.jdl
      glite-ce-job-status <...>
    
      /opt/glite/bin/glite_cream_load_monitor --show
    

    TORQUE 2.5.7 / MUNGE

      root@grid-vm06: [~] /usr/sbin/create-munge-key
      root@grid-vm06: [~] ls -l /etc/munge/munge.key
    -r-------- 1 munge munge 1024 Nov  3 17:08 /etc/munge/munge.key
      root@grid-vm06: [~] chkconfig munge on
      root@grid-vm06: [~] /etc/init.d/munge start
      
      root@grid-vm10: [~] cp munge.key /etc/munge/munge.key
      root@grid-vm10: [~] ls -l /etc/munge/munge.key
    -r-------- 1 munge munge 1024 Nov  3 17:09 /etc/munge/munge.key
      root@grid-vm10: [~] chown munge.munge /etc/munge/munge.key 
      root@grid-vm10: [~] chkconfig munge on
      root@grid-vm10: [~] /etc/init.d/munge start
    

    CREAM administration:

      vi /etc/grid-security/admin-list
    "/O=GermanGrid/OU=DESY/CN=Andreas Gellrich"
    "/O=GermanGrid/OU=DESY/CN=Christoph Wissing"
    "/C=DE/O=GermanGrid/OU=DESY/CN=Dmitry Ozerov"
    
      UI:
      glite-ce-disable-submission --debug grid-cr5.desy.de
      glite-ce-enable-submission  --debug grid-cr5.desy.de
    
      glite-ce-service-info -L 2 grid-cr5.desy.de
    

      glite-ce-monitor-getinfo --cert $X509_USER_PROXY grid-cr5.desy.de
      glite-ce-monitor-gettopics --cert $X509_USER_PROXY grid-cr5.desy.de
    

    Cleanup:

      df -i
    
      find /opt/glite/var/cemonitor/CREAM_JOBS -type f -atime +60 -exec rm -rv {} \;
    

    [top]


    CLUSTER

    Documentation:

  • gLite CLUSTER

      yum install glite-CLUSTER
    

    Configurations:

    
    
    

    Services:

    
    
    

    Logs:

    
    
    

    [top]


    VO DIR NFS Server

    Documentation:

  •   # (by Quattor) yum install nfs-utils
    

    Configurations:

      vi /etc/exports
    #
    # allow access
    #
    /local grid-sm*.desy.de(rw,no_root_squash,sync) *.desy.de(ro,async,no_root_squash)
    
      vi /etc/sysconfig/nfs
    ...
    RPCNFSDCOUNT=32
    ...
      cat /proc/net/rpc/nfsd
    
      /etc/init.d/nfs restart
    
      chkconfig nfslock --list
      chkconfig nfs --list
    
      chkconfig nfslock on
      chkconfig nfs on
    
      vi /etc/sudoers.forcream
    ...
    Runas_Alias GLEXEC_BELLESGM = \
          sgmbelle
    ...
    Runas_Alias GLEXEC_CMSSGM = \
          sgmcms
    ...
    

    [top]


    CLUSTER

    Note: Does assume lcg-CEs only?

    Documentation:

  • EMI: gLite CLUSTER
  • Wiki: gLite CLUSTER

      yum install ca-policy-egi-core
      yum install glite-CLUSTER
    

    Configurations:

      /opt/misc/yaim/get-hostcert.sh `/bin/hostname -f`
    
      /opt/glite/yaim/bin/yaim -v -s /opt/misc/yaim/`hostname -s`-info.def -n CLUSTER
      /opt/glite/yaim/bin/yaim -c -s /opt/misc/yaim/`hostname -s`-info.def -n CLUSTER
    

    Services:

    
    
    

    Logs:

    
    
    

    [top]


    by the DESY Grid Team: http://grid.desy.de/