Running JugWorker

First, of course, you need to install JugMaster. This may include setting up storage and file servers.

Next you need to decide if the batch of jobs you are going to create fall into an existing execution class or if you need to define a new one. An execution class is an arbitrary grouping designed by you to indicate sets of jobs that all have the same execution requirements. Once you decide what execution class you are going to use, you may need to configure a worker_class to handle that class of jobs. This is done with jug_worker_setup. By default, worker classes are configured to handle jobs in the execution class "test" and "condor". The "test" jobs simply run wherever a test worker is running (probably just the machine where Jug was installed) and the "condor" jobs are submitted to the Condor batch system, assuming you have one.

Once you have chosen an execution class, you need to write a job description and submit a batch of jobs.

If you chose an execution class that is handled by a queue manager, then your jobs should start running automatically. If not, then you can run workers by hand.

To run workers by hand, you need to "compile" a version of JugWorker that knows where your master RPC server is running, and which has all the other configurations you may desire. The "compiled" version is actually a self-contained Python script containing configuration settings. See jug_make_worker.

Next you need to run the compiled version of JugWorker. Typically, you would run it on a batch system. For example, to run JugWorker on a Condor cluster, you would use something like the following Condor submit file:

Universe=Vanilla
Executable=$ENV(JUG_HOME)/workers/condor.JugWorker
error=JugWorker.$(Cluster).$(Process).log
log=JugWorker.$(Cluster).$(Process).condor.log
transfer_files=on_exit
notification = never
queue 25

This is assuming that you previously did jug_make_worker --worker_class=condor.

There is no need to have any Jug software installed on the execution nodes. The whole purpose of jug_make_worker is to produce an easily transportable, self-contained script that may be run anywhere. (The execution node does need to have python installed, however.)