runWiscJobs: a tool for using HTCondor and HDFS

To use runWiscJobs, first obtain an OSG grid certificate. If you are a member of a large collaboration like CMS, LSST, or LZ, you should register with a Virtual Organization (VO), but if you are a member of a small experiment without a VO, you can specify the GLOW VO when you need to provide one. Follow the instructions under How to get a Grid User Certificate here:

Log into a login machine:


Obtain a proxy:

voms-proxy-init -valid 142:00

or if you have a VO:

voms-proxy-init -valid 142:00  -voms VO_NAME

where VO_NAME is the name of your VO.

Then you can use to run your jobs with HTCondor:

 python /usr/local/bin/ \
        --WorkFlow="test.mac_test1" \
        --Executable="/afs/" \
        --Experiment=lz \
        --nJobs=10 \
        --TransferInputFiles="/afs/" \
	--HDFSProdDir=outfiles \
        --MakeUniqueOutput \
        --UserEnv="UserName=`whoami` " \
        --OutputDir="test.mac" \

You can see all the arguments for by doing:

 python /usr/local/bin/ --help

Note that you need to pass an Experiment name to It might be lz, lux, miniclean, ricochet, etc.

Executable is your script or program that runs your simulation program.

HDFSProdDir is the directory where your Executable should place the output files you would like copied to HDFS. This directory will be under the working directory on the worker node. If your executable handles on its own the copying of output files to HDFS, set this parameter to “None”.

If your executable generates output files all with the same name, you can use the –MakeUniqueOutput option so that a unique tag gets added to each output file name when it is copied into HDFS.

If your jobs are successful, the output appears under to /hdfs//user//.

Diagnostic logs appear under /nfs_scratch//.