JugMaster Release History

Version 1.2
Date: February 21, 2007 (mostly developed in early 2006)
  • Added --show_all option to jug_ls. This dumps file metadata, such as size, md5, storage date, job id, and run and storage IPs.
  • Fixed a bug in jug_abort when restarting parent-child batches. The bug could cause the restarted child to be marked finished without processing any of the parent jobs.
  • Added the ability to import arbitrary python modules in Jug submit files.
  • Made Jug submit file syntax support python operators instead of C operators. The C ? operator has been replaced by iff().
  • Added python-style indexing and hashing of lists and strings.
  • Added new_seed(), which may be used to allocate unique ranges of random seeds when submitting new batches of jobs.
  • Added cleanup of idle queued jobs when the queued job gets finished first (or stolen away) by some other worker. Previously, the queued job had to get run by the batch system before it could be removed from the queue, which could block other work from starting when the queue is full.
  • Improved handling of network problems in JugWorker's communication with the master.
  • Added batch() to submit language. Allows referencing properties of other batches when creating a new batch. Very handy for chained batches.
  • Added getenv() to submit language.
  • Added python-style if-elif-else to the submit file language.
  • Added reporting of worker state in jug_ps.
  • Added monitoring of rundir timestamps and size.
  • Added search operators to batch selection in CGI and command tools.
    Version 1.1
    Date: March 25, 2005
  • Fixed the case where idle jobs pre-assigned to existing workers would decrease the number of active jobs that could run under a workload constraint (since the pre-assigned jobs also count towards the limit placed by the constraint).
  • Fixed a bug causing workers to crash with a locking exception. This bug was only noticable on heavily loaded workers, such as a queue worker managing hundreds of jobs.
  • Many small enhancements to the web interface.
  • Added a pacman version 3 installation cache.
  • Made compatible with python2.3.
    Version 1.0
    Date: Jan 24, 2005
  • Added queue workers, which can automate the process of submitting job execution workers to a batch system.
  • Added interruptible job option to control whether jobs are signalled to shut down when JugWorker is being shut down.
  • Added store_info URL flag, allowing access to additional information about the file, such as size, md5 checksum, and url.
  • Added max_failure_frequency so user can control batch failure throttling.
  • Added eg() to parameter definitions so an example value can be shown to the user when a required parameter is not provided.
  • Added storage of arbitrary run "Output Vars", so custom attributes can be saved in the database. This uses the existing jug_monitor.vars mechanism.
  • Added support for expressions in "${}" constructs. The expressions are evaulated when the run is being created. This allows one to perform simple math when substituting things such as JUG_SEED into environment variables.
  • Added support for newlines within environment values. This is convenient if you assign an environment variable to a list of values.
  • Added execution constraints so that a maximum number of active jobs may be specified for an execution class.
  • Added worker authorization cookies. Users don't need to be aware of this, because jug_worker_setup configures the worker automatically. This is not to be deemed an assurance of strong authentication. Jug only weakly authenticates processes, but it does provide very specific authorization control. Authority to request work belonging to a specific execution class is now added to the list of things that are controlled.
    Version 0.9
    Date: Sep 30, 2004
  • Added batch input_files as a convenient alternative to workware = {"{unpack=no}..."}.
  • Made it possible to easily wrap JugCGI.py within another CGI script.
    Version 0.8
  • Added a pacman installer.
  • Replaced various startup scripts with jug_daemon.
  • Added 'include' to the submit file syntax.
  • Added submit file parameters, which may be passed on the command-line or when one script includes another.
    Version 0.7
  • Added JugCGI for viewing general status.
  • Added handle_urls to make input file access protocols extendable.
  • Adding option for timing statistics in jug_batch_status.
  • Added jug_get_file.
  • Added mirror_jobs_when_idle.
  • Added variables and simple expressions to submit files.
  • Added automatic throttling when a batch of jobs starts failing at a suspiciously high rate.
    Version 0.6
    Date: Feb 13, 2004
  • Added not_required and run_wrapper to the package options.
  • Added file service to the storage worker.
  • Fixed a bug that was causing occasional hangs during shutdown when encountering serious errors, such as running out of disk space.
    Version 0.5
    Date: Dec 22, 2003
  • Fixed a bug in firewall mode.
  • Added jug_storage_setup and jug_fileserver_setup.
    Version 0.4
    Date: Dec 18, 2003
  • Added support for stage-outs from behind firewalls, with the --ports option.
  • Improved reporting of queue state in jug_run_status.
    Version 0.3
    Date: Sep 22, 2003
  • Added reassignment of jobs at end of run when some workers are idle and some have jobs waiting in their run queue.
  • Added execution_class.
    Version 0.2
    Date: Aug 4, 2003
  • Added worker state restoration in case of reboots.
  • Added jug_run_status and jug_batch_status.
  • Added work_done, output_bytes.
  • Added run_output.storage_url.
  • Added pre_installed_path_search.
    Version 0.11
    Date: July 9, 2003