[DESY Home] [DESY IT Home] [DESY IT Physics Computing] [Grid Computing at DESY] [DESY Computing Seminar] [Imprint]

Grid Computing at DESY DESY

[Home] [Mon/Admin] [Grid@DESY] [Certs & VOs] [VOMS] [CVMFS] [User Guide] [Install Guide] [Notes] [Talks & Posters] [Glossary] [Documentation] [Links]

In order to ensure response in case of problems, use the Global Grid User Support (GGUS) and/or your VO support rather than private e-mail contacts or internal mailing lists.


ARC-CE

ARC-CE for HT Condor

Docu:

  • Computing with HTCondor
  • Example Build of an ARC/Condor Cluster
  • ARC HTCondor Basic Install
  • HTCondorTM Version 8.2.3 Manual

  • HTCondorTM Version 8.4.1 Manual
  • 3.5 Policy Configuration for Execute Hosts and for Submit Hosts
  • 3.10 Monitoring

  • Defragmentation_for_multicore_jobs
  • fallow-1.1-1.x86_64.rpm

  • The NorduGrid-ARC Information System
  • ARC batch system back-end interface guide with support for GLUE2
  • Condor useful commands
  • Useful Condor Commands

  • GridPP: Publishing tutorial

    Installation:

      #
      # puppet: grid_arc_ce   (repos, SL, and some basic stuff)
      #
    
      // ARC-CE
      #yum install fetch-crl
      #yum install nordugrid-arc-compute-element
      #yum install globus-gridftp-server globus-gsi* globus-gss* globus-xio globus-io
    
      // Condor
      #cd /etc/yum.repos.d/
      #wget http://research.cs.wisc.edu/htcondor/yum/repo.d/htcondor-stable-rhel6.repo
      #yum clean all
      #yum install condor --disablerepo=emi*
    
      // most recent norsugrid repo
      #rpm -ivh http://download.nordugrid.org/packages/nordugrid-release/releases/15.03/centos/el6/x86_64/nordugrid-release-15.03-1.el6.noarch.rpm --nodeps
      #yum clean all
      #yum update nordugrid-arc --disablerepo=emi*
    
      #
      # hostcert now handled by puppet
      #
      #/opt/misc/yaim/get-hostcert.sh `hostname -f`
    

    Configuration:

      #
      # rsyslog messages
      #
      cat /etc/rsyslog.d/SystemLogRateLimitInterval.conf
    #$SystemLogRateLimitInterval 10
    #$SystemLogRateLimitBurst 500
    $SystemLogRateLimitInterval 0 # turn off rate limiting
    
      #
      # puppet does the job!
      #
    
      # ls -l /etc/grid-security/vomsdir
      # ls -l /etc/vomses
    
      #less /etc/grid-security/grid-mapfile
      #less /etc/grid-security/groupmapfile
    
      #
      # puppet copies a template
      #
      cp /etc/arc.conf.template /etc/arc.conf
    
      #ls -l /etc/condor/condor_config.local
      #ls -l /etc/condor/pool_password
      #ls -l /etc/condor/config.d/
    
      #
      # capabilities
      #
      cp -pr /opt/misc/edg/var/info /opt/edg/var/.
    
      #vi /etc/ld.so.conf.d/condor.conf
    /usr/lib64/condor
    
      #
      # puppet creates a template only!
      #
      cp /etc/arc.conf.template /etc/arc.conf
    

      save /usr/share/arc/ARC1ClusterInfo.pm
      vi   /usr/share/arc/ARC1ClusterInfo.pm
    ...
    #AG $log->warning('EMIES Failure state attribute cannot be determined.');
    $log->debug('EMIES Failure state attribute cannot be determined.');
    ...
    

    Services:

      service condor start
      service a-rex start
      service gridftpd start
      service nordugrid-arc-ldap-infosys start
    

    Ports:

      condor_collector: 9618,9620 (for arc_ce and worker)
    

    Logs:

      ls -l /var/spool/arc/grid/*/log
    
      ln -s /var/log/arc/bdii/bdii-update.log
      ln -s /var/log/arc/gridftpd.log
      ln -s /var/log/arc/grid-manager.log
      ln -s /var/log/arc/gm-jobs.log
      ln -s /var/log/arc/infoprovider.log
    
      ln -s /var/spool/arc/jobstatus/
    
      ln -s /var/log/condor/EventLog
      ln -s /var/log/condor/MasterLog
      ln -s /var/log/condor/ProcLog
      ln -s /var/log/condor/SchedLog
      ln -s /var/log/condor/ShadowLog
    

    Tests:

      condor_config_val CONDOR_HOST
      condor_config_val LOG
    
      condor_status -any -pool condor01.desy.de
    
      condor_status -schedd 
    
      su - grid
      condor_run "/bin/hostname"
    
      condor_status -any
    
    [top]
    by the DESY Grid Team: http://grid.desy.de/