Links to Workspaces
For entry points to workspaces (Jupyterlabs), see cms01.hep.wisc.edu.
Workspace Choice
GPUs are accessible through Condor with all workspace choices – both with and without GPUs locally available. However, one can only use GPUs locally when using a workspace labeled as having GPUs locally available. For testing and development with GPU-accelerated code, it is recommended to use locally-hosted GPUs first. For running accelerated code at scale with Condor, a workspace does not need to have locally-hosted GPUs.
Notes are provided next to workspaces to help guide your choice of image. These broadly correspond to versions of coffea and accompanying packages that have been tested for compatibility.
Quickstart
Welcome to the Analysis Facility at the University of Wisconsin – Madison!
The Analysis Facility gives users access to the UW Tier-2 computing cluster, and is configured around the analysis tools and concepts of coffea. So, it is best suited for users looking for access to coffea and associated tools and/or distributed computing resources.
The Analysis Facility is reachable at cms01.hep.wisc.edu. You will be presented with a choice of `coffea` image to use – those containing “202X” are “CalVer” (or “new”) coffea, while those containing “0.7” are “SemVer” coffea. To run the notebooks in this repository on the Analysis Facility, one can open a terminal session in the AF, and then clone this repository.
Here in the Analysis Facility’s Jupyterhub, users can run code through Jupyter notebooks, or through the terminal. All code runs in an environment with coffea and common analysis tools available.
To open a new notebook or a terminal window, click the “plus” sign next to the words “Welcome.ipynb” at the top of this notebook.
Important: To leave the Analysis Facility, first make sure you save any notebooks you edited. Then, please go to the top left of your browser window (next to the orange Jupyter logo), and click File -> Hub Control Panel. There, please select “Shut Down Server”, and then logout in the top right corner of your window. You can then close your browser window.
If your work requires user-specific setup steps (ie: creating a VOMS proxy for data access), this should be done in a terminal window before running your workflow. To obtain a voms proxy, add ‘-vomses /etc/vomses’ to the voms-proxy-init command.
Examples
Examples are available at this Github repository. It includes interactive examples that can be run on the Analysis Facility to gain familiarity with columnar analysis tools in the context of the Wisconsin AF.
Virtual Environments at the AF
If users need Python packages not included by default in the AF image, it is suggested to manage additional packages with Python virtual environments. To create a virtual environment at the AF, please use the command-line utility new-af-venv. For example, running $ new-af-venv my-env-name creates a virtual environment named my-env-name in the current working directory. Using this utility guarantees that a notebook kernel is created (see next paragraph), and that the environment can be shipped to Condor workers. Activate the virtual environment with $ source my-env-name/bin/activate, and then install packages with $ python3 -m pip install name-of-package. To deactivate the virtual environment, $ deactivate.
To use a virtual environment in a notebook, go to the top right corner of an open notebook to select the kernel. Using the drop-down menu, select the kernel called my-env-name.
To send virtual environment packages to Condor workers, use the ship_env option of the cowtools GetCondorCllient function. Make sure that the desired virtual environment is activated if running a script (or selected as a kernel if running a notebook).
Contact
To report issues or ask questions, please contact help@hep.wisc.edu and rsimeon@wisc.edu.