Further thoughts on the DAQ failures:
The readout works fine when in PickOne mode. The lasers turn on and the profiles are present.
It does _not_ work fine in full DAQ mode. In certain regions lasers don't appear. It is not clear whether this is because they never turn on or because the DAQ doesn't wait long enough. That would have to be checked by running the laser control panel and watching for lasers to turn off and on.
One new feature of our readout system is that the DIM DNS server is the DAQ machine. At certain points in the DAQ process the system is running at > 98% of CPU capacity. At other times the network I/O use is heavy. (On average the I/O rate is tolerable, but it can be bursty.)
If each DIM command must first check the DNS server for routing (ie. if there is no caching) then it is imaginable that the heavily loaded server might not reply quickly enough. I do not use blocking DIM calls; merely a delay, so the DCOPS might be asked to digitize before the laser was actually turned on.
The problem with this model of the failures is that I would not expect the failures to cluster in a single section, but be more random. I would also expect them to be concentrated on the _second_ laser readout.
The reason for this is that at the beginning of a read cycle, DIM commands are sent to all the "laser 1" SLM lasers to begin simultaneous readout. At this point in the cycle nothing is being read out and no profiles are present to be fitted, so I would expect the CPU and I/O load to be very small. When the "laser 1" processing is done for an SLM the "laser 2" is told to commence. At this point other SLMs may still be fitting and the CPU load may be heavy. So for the "laser 2" part of the readout I would expect failure modes to be reading out when "laser 1" had not gotten the signal to turn off yet, and also reading out when "laser 2" had not gotten the signal to turn on, with perhaps a few when both lasers are on.
This isn't what we see.
Modified 03-March-2010 at 08:34
http://hep.physics.wisc.edu/~jnb/cms/03Mar2010
| Previous notes | Next notes | Main slide directory |