Here is the main CRAB documentation: CRAB.#

Quick Recipe for users#

We assume you have gone through getting a UW-HEP account, a grid certificate, etc….#

1) Typical setup:
mkdir /scratch/weinberg
cd /scratch/weinberg     # must be in /scratch!
scramv1 project CMSSW CMSSW_3_5_8_patch3
cd CMSSW_3_5_8_patch3/src/
kinit weinberg@CERN.CH
cvs co your/analyzer  # (or create your analyzer)
scramv1 b
cd your/analyzer

2) Create a crab.cfg here.  See example in section below.

4) Load CRAB at wisconsin:
source /cms/sw/CRAB_2_10_2p3/

5) Create the jobs
crab -create

6) Submit the jobs
crab -submit

7) wait, then check on your jobs,
crab -status

8) If they're working, the output root files should be in your hdfs directory:

9) Optional, if you want the log and err files do
crab -getoutput

10) Optional, if you want the integrated luminosity of the data you ran on
crab -report
# Crab returns something like:
# Luminosity section summary file: /scratch/mbanderson/.../res/lumiSummary.json
# Give that full path to that file to lumiCalc: -c frontier://LumiProd/CMS_LUMI_PROD -i res/lumiSummary.json recorded

For information on creating an analyzer see Creating Analyzer for more information.#

Figure out what dataset you want to run over using DAS#

# Note: to continue submitting CRAB jobs for a particular WF to Condor. crab -submit 1 -continue crab_0_$DATE_$TIME#

We recommend using the remoteGlidein scheduler. Other options are to use the glidein server and glite.#

Example CRAB cfg file#

This is a typical crab.cfg file.#

jobtype                = cmssw
scheduler              = remoteGlidein
use_server             = 0

datasetpath            = /MinBias/Summer09-STARTUP3X_V8K_900GeV-v1/GEN-SIM-RECO
#datasetpath            = /MinimumBias/BeamCommissioning09-Dec19thReReco_336p3_v2/RECO
pset                   = test/
total_number_of_events = -1
events_per_job         = 40000
output_file            = SinglePhotonAnalyzer.root

copy_data              = 1
return_data            = 0
storage_element        = T2_US_Wisconsin
user_remote_dir        = ./
check_user_remote_dir  = 0

rb                     = CERN
proxy_server           =
virtual_organization   = cms
retry_count            = 0

#se_white_list          = T2_US_Caltech, T2_US_UCSD, T2_US_Wisconsin

Advanced Set-up#

Put the following code anywhere into your .bash_profile:#

# CMSSW & CRAB Version
export CMSSW_VER=CMSSW_3_3_1
export CRAB_VER=CRAB_2_10_2p3

#Load CMSSW environment settings
cd ${CMSSW_VER}/src
cd ~

#Load CRAB environment
source /cms/sw/${CRAB_VER}/

Now, whenver you login, you can just#

cd /path/to/your/analysis/workdir

# Check Jobs status in local Condor queue.

# Check Job status in CRAB
crab -status

# Retrieve the output
crab -getoutput


# Login to CERN

# Load the CMS Environment
source /afs/

# Start your grid proxy
voms-proxy-init -voms cms

# Load up CRAB
source /afs/

If your grid-proxy isn’t installed at CERN but it is at Wisconsin, copy it this way:#

# Login to CERN

# Copy your grid proxy
scp -r .globus

# Set the permissions on these files
chmod 700 ~/.globus/private/
fs setacl -dir ~/.globus/private -acl system:anyuser none
chmod 0600 ~/.globus/private/userkey.pem
chmod 640 ~/.globus/usercert.pem

Data Location#

If you would like to see where your data is located, use Data Aggregation System.#


# To see a list of all your jobs
condor_q $USER

# To watch your jobs

# If some jobs go on hold
# See the hold reason

# Find out more info about a job
#  jobID = the number on the left when you type condor_q
grid-job-remote-info <jobID>

# Grab the stdout file from a job still running
#  run the above command to find the location of stdout
globus-job-run <site-you-submitted-to> /bin/cat /location/of/stdout

globus-job-run /bin/cat /osglocal/users/cms/uscms1370/.globus/job/