July 2006

Bogus(?) Superclusters

Some of the superclusters give very weird outpout. There is no eta or phi information and their energies are sometimes amazingly high. I even got a SC with a negative energy!

From my analysis:

The highest SuperCluster has energy: 352.13, Phi: nan and Eta: nan

Elsewhere:

BOGUS!!! Endcap SuperCluster has energy: -0.325042, Phi: nan and Eta: nan

Those "nan"s should be numerical.... I modified my code to catch these bogus SuperClusters. What was found, is that all of them appear in the EndCaps:

Their energies vary quite a bit. Here I'm plotting only up to 250GeV, but they reach up to 1200GeV!!!

Finally 0_8_0

Well, I didn't manage to make my farmout script work (superclusters never appeared). Thanks to the new updated version of Nikitenko's allreco though everything looks okay. I've submitted a set of signal jobs and some background. In the meantime I'm working on my analysis script to make it identify some elctrons...

0_8_0 Revisited

So the scramv1 problems were fixed - we needed to source one file to be configured correctly. I don't know if this would affect the farmout jobs as well. I'm testing it now.

Unfortunately the files I produced yesterday are no good. My fears of the small file size proved true. They seem to contain only gene events. Although the rest of the branches are there, I couldn't see any reco SClusters or jets :(

Back to square 1(?)...

Update: We're one step further! (hopefully the last) After looking at the problem with Ajit and Dan, the problem was pinpointed at these lines of the cfg file:

# include "SimGeneral/HepPDTESSource/data/pdt.cfi"
es_source = HepPDTESSource {
string pdtFileName = "/afs/cern.ch/cms/external/lcg/external/HepPDT/2.02.02/slc3_ia32_gcc323/data/PDG_mass_width_2004.mc"
}

It looks that in pre2 the mentioned cfi file was exactly like the code I have above! However, in 0_8_0 final, there is no path - only the filename. Obviously something has changed in the source code; that's why problems appeared.

In the 2 events I tested it though I didn't see any superclusters in the data... hmmm....maybe it's the Analyzer it needs to be changed now(?)

btw, I looked up that __PROD thing, but apparently it shouldn't worry us at all. From this note on branch names:

In some cases, a branch name may be shortened, according to the following rules, in order:

1. If the process name is PROD, the "_PROD" is replaced by an empty string.


2. If the resulting branch name ends in __. (two underscores and a period), the two underscores are removed.

Migration to 0_8_0

Looks like in 080 some files have changed names (from CMSSW/RecoLocalCalo/EcalRecProducers/data/) that's why Nikitenko's script wouldn't work. I changed them to point to the renamed ones. As Nikitenko points, in file

Geometry/TrackerRecoData/data/trackerRecoGeometryXML.cfi

we have to change

es_source = XMLIdealGeometryESSource

to

es_source trackerIdealGeom = XMLIdealGeometryESSource

To do that, when you're in the CMSSW_0_8_0/src directory you'll have to checkout from CVS:

cmscvsroot CMSSW; cvs login;
cvs co -r CMSSW_0_8_0 Geometry/TrackerRecoData

The module named ecaluncalibrechit changed to ecalWeightUncalibRecHit (this module is inside ecalWeightUncalibRecHit.cfi - one of the renamed files).

The module named ecalrechit was renamed to ecalRecHit (ecalRecHit.cfi) - apparently lowercase DOES matter :)

Still working on it... I've got a 2 event job which is still running after 3min:40sec... could it be??? ...6min:40secs... looking good... Nope :(

I've got an error message saying:

<FrameworkError ExitStatus="8001" Type="CMSException" >
cms::Exception caught in cmsRun
---- FileNotFound BEGIN
can't open pdt file cannot open /afs/cern.ch/cms/external/lcg/external/HepPDT/2.02.02/slc3_ia32_gcc323/data/PDG_mass_width_2004.mcA cms::Exception is going through EDProducer:
Module type=HepMCCandidateProducer, Module label=genCandidates, Process name=PROD
---- FileNotFound END
</FrameworkError>

The really annoying part is that the cfi file defines it as: string pdtFileName = "/afs/cern.ch/cms/external/lcg/external/HepPDT/2.02.02/slc3_ia32_gcc323/data/PDG_mass_width_2004.mc" and that file exists! Hmmm I'm kinda stuck...

edit: I just realized that the "A" at the end of the filename belongs to the next sentence. How embarassing. Still though it doesn't work. I tried copying the file locally to my own web space and rerunning but the same error occured.

Hmmm... the problem seems to be in the doGenJetMET sequence. I'll try disabling this and see what happens. This means that the path p1 has to be changed as well.

Last update: All the above managed to give me a final root file. The farmout script that worked is in my bin directory. BUT That script has the doGenJetMET sequence disabled. This obviously has the result of not having GenJets in the resulting root file. One other thing that bothers me is that the root file is only 1.4 MB (two events). This means that either somehow the event size has been magically reduced by a factor of ~5 or that I'm missing stuff (the doGenJetMET sequence is one of them for sure!).

The root file looks okay though; all other necessary branches are there at least:

recoSuperClusters_correctedIslandBarrelSuperClusterProducer__PROD.
recoCaloJets_icone5__PROD.
recoCaloJets_mcone5__PROD.
recoTracks_CTFWMaterial__PROD.

(one other issue is that extra __PROD part which I don't know if its due to the new version or other things) I'll now create around 500 events to get some statistics and look at the plots.

If you want to look at the first test events I produced and comment (please do!) you'll find them here:

/afs/hep/cms/lazaridis/FullCMS080Sim-HZZ2e2nu-2events

btw... I don't think I can do anything about the scramv1 error...

SCRAM Error: It appears that the module "Template.pm" is not installed.
Please check your installaion. If you are an administrator,
you can find the Perl Template Toolkit at www.cpan.org or at
the web site of the author (Andy Wardley):
www.template-toolkit.com
You should install version 2.xx (2.13 or better).

working...

Useful one-line script (Mike did all the hard work, I just added the quotes and the commas):

ls /afs/hep.wisc.edu/cms/lazaridis/FullCMS080Sim-PythiaSignal | grep root | awk '{print "\"file:/afs/hep.wisc.edu/cms/lazaridis/FullCMS080Sim-PythiaSignal/" $0 "\","}'

To create the list of files to be given in untracked vstring fileNames of PoolSource.

Hey, after the change of the double loops and the usage of structs and vectors where I store the data, everything sped up amazingly! I used to take about 2 and a half to 3 hours to process - now in 15 minutes it's finished! Talking about code optimization eh? :)

I also have a couple of hisotograms: Electron candidates (cuts on DeltaR between SCs and Jets and Tracks and SCs), and DeltaR's between SCs and Jets and Tracks and SCs.

Full framework progress

I started following the same technique as with FWLight: Built successfully iterators to go through all the tracks, SCs and jets of an event. Instead of using nested iterators (jets inside SCs for example) what I do now is I created vectors where Jet and Track info are being stored. Then this information is being used at the SC iterator. Still we need double loops, but the nested one doesn't access the root file anymore, but the memory. I hope this will speed things up significantly. Sample of struct used:

struct JetData
{
unsigned int number;
double eta;
double phi;
double energy;}

Tomorrow's todo: Add histograms; maybe create a FWLight script to combine the resulting root files if time permits.

Transition to full framework

After a lot of frustration with FWLight, we're switching to the full CMSSW framework. I tried running the tutorial at

https://uimon.cern.ch/twiki/bin/view/CMS/WorkBookMakeAnalysis#Using_the_full_CMSSW_framework

but without any luck. It may have been made for another version of CSMSSW, or it might have been me. I'll give it one more try tomorrow and if it doesn't work, I'll just start by building my own module and see how it goes.

first analysis steps (on tracks, SCs, jets)

So, today we've been collaborating with Marc and we were trying to start writing some analysis code and delve into the secrets of Tracks, SuperClusters (SCs) and Jets :)

What we accomplished is:

• We started building on a generic script I've prepared with pointers at the trees: recoTracks_CTFWMaterial, recoSuperClusters_islandSuperClusterProducer_islandBarrelSuperClusterCollection and CaloJets_mcone5. The respective pointers are of type: std::vector<reco::Track>, std::vector<reco::SuperCluster> and std::vector<CaloJet>.
• We've calculated eta, phi and energy for Tracks, SCs and Jets. For Tracks, because eta, phi were not directly available we used a ROOT TLorentzVector as Prof. Dasu proposed and used that to find eta and phi from the methods: vector.eta() and vector.phi().
• Used etas and phis to find Delta(r) between tracks&SCs and SCs&Jets.
• Also, we've calculated the ratio of {Ecal/(Ecal+Ehcal)}. Currently I'm running an analysis to create this plot to see how it looks - one hour and a half and counting so far...

After 2 hours and some minutes... we have this. Before you get too baffled, let me tell you this: only SCs&Jets with Delta(r)<=0.3 were adder to the plot. I plan to plot all of them pretty soon.

Tracks, Clusters, SuperClusters and Jets

So far, I've created three analysis scripts to process the reconstructed Tracks, SuperClusters and Jets. They're pretty crude scripts, no eta, phi or Delta(r) selection is being made. As soon as these are working as I need to, if there is time I'll try to instegrate them as modules to the full CMSSW framework.

I've got a couple of histograms (analysis still running): DiJetMass, LeadingJetsEta, LeadingJetsPhi, LeadingJetsPt.

Analysis finished: Plots, Scripts, Presentation.

Some interesting web pages...

Just to pick them up tomorrow where I left them:

ECalClustering

EgammaSimpleAnalyzer.cc

list of collections made by the egamma producers

Notes on Analyzer.cc

Analyzer.cc has a special interest because it is the first example of EDAnalyzer (analysis with the full framework and not FWLight).

First let's note that in the cfg file, the following strings are passed to the Analyzer:

string calotowers = "caloTowers"
string tracks = "CTFWMaterial"
string jets = "icone5"
string jetsgen = "icone5gen"

These apparently are the root file branches that contain the information he's looking for. These are later passed to the module at the constructor:

// get names of modules, producing object collections
m_tracksSrc = iConfig.getParameter<string>("tracks");
m_jetsSrc = iConfig.getParameter<string>("jets");
m_jetsgenSrc = iConfig.getParameter<string>("jetsgen");
m_calotowers = iConfig.getParameter<string>("calotowers");

Now, at the actual analysis code we have the definitions (after a using namespace edm statement):

Handle<HepMCProduct> evt;	MC info
Handle<CaloTowerCollection> calotowers;	calo towers collection
Handle<CaloJetCollection> jets;	calo jet collection
Handle<GenJetCollection> jetsgen;	calo jet collection
Handle<TrackCollection> tracks;	track collection

Then, there's an iteration over the number of jetsgen where the Jets Pt, eta and phi are printed out and hJetEt->Fill((*jetsgen)[j].pt()); fills the Et histogram. The eta and phi histograms are filled outside this loop with:

hJetEta->Fill((*jetsgen)[0].eta(),(*jets)[1].eta());
hJetPhi->Fill((*jetsgen)[0].phi(),(*jets)[1].phi());

I'll have to look a bit more at this last part; it confuses me a bit.

Some code cleanup and results from Nikitenko's recipe using condor and my signal cfi.

Cleaned up a bit my root scripts. Started some preliminary processing of the Signal job I submitted to condor somedays ago. 9973 events made it to the end. I'll have to look at the error messages to see what happened.

The jobs I submitted using Nikitenko's recipe to condor (my signal cfi) seems it worked fine - at least I have root files this time. I processed one of them with the Analyzer script and got plots for JetEt, JetEta and JetPhi. By the way, the modified version of the script to submit condor jobs using this recipe and 0_8_0_pre2 is in my /bin directory under the name farmoutFullCMS080SimulationJobs.

July 7

Tried to generate gene+simu+digi events with CMSSW_0_8_0_pre2. As expected, the result was lots of errors. I have kept an error output and a log file as reference. As a test, I tried to continue with clusterization to see what happens and after initialization cmsRun crashed with this output:

[SiStripNoiseService::configure]
%MSG-e BeginJob: 07-Jul-2006 10:44:29 CDT pre-events
A cms::Exception is going through EDProducer:

%MSG-e BeginJob: 07-Jul-2006 10:44:29 CDT pre-events
A cms::Exception is going through EDProducer:

%MSG-e FwkJob: 07-Jul-2006 10:44:29 CDT pre-events
<FrameworkError ExitStatus="8001" Type="CMSException" >
cms::Exception caught in cmsRun
---- NoProxyException BEGIN
No data of type "DDCompactView" with label "" in record "IdealGeometryRecord"
Please add an ESSource or ESProducer to your job which can deliver this data.
A cms::Exception is going through EDProducer:
Module type=SiStripClusterizer, Module label=ThreeThresholdClusterizer, Process name=Clusterization
---- NoProxyException END

</FrameworkError>

cms::Exception caught in cmsRun
---- NoProxyException BEGIN
No data of type "DDCompactView" with label "" in record "IdealGeometryRecord"
Please add an ESSource or ESProducer to your job which can deliver this data.
A cms::Exception is going through EDProducer:
Module type=SiStripClusterizer, Module label=ThreeThresholdClusterizer, Process name=Clusterization
---- NoProxyException END

%MSG-i SiStripClusterizer: 07-Jul-2006 10:44:29 CDT pre-events
[SiStripClusterizer::~SiStripClusterizer] Destructing object...

Now I'm thinking of what to do next...

July 6

Working on A. Nikitenko's Exercise with tau jets: full chain from generation to reconstruction and analysis with CMSSW_0_8_0_pre2. Note: In the analysis part, step 3 we have to do scramv1 b to build the Analyzer module. All output here.

Also, I found a working solution to yesterday's issue. I do not know if this is the best solution possible, but it works. The trick is to define the chain and all the problematic variables outside the function -- i.e. as global variables. So now my scripts look like this:

TChain *chain;
int nfiles;
Int_t nev;
void CPlotElectronsTest(const char* fName = "/afs/hep/cms/lazaridis/FullCMSSim-PythiaSignal")
{
gSystem->Load("libFWCoreFWLite.so");
AutoLibraryLoader::enable();

cout << "TChaining... ";
chain = new TChain("Events");
TString fNames = TString(fName) + TString("/PythiaSignal-0041.root");
nfiles = chain->Add(fNames);

// Number of entries in chain nev = chain->GetEntries();
cout << nfiles << " files with " << nev << " entries" << endl;

// Open first file and set addresses. This needed in addition to what is done in event loop. chain->GetEvent();

... and so on from here.

July 5

1. Found out something very interesting today... the Pythia seed has to be between 0 and 900000000. Setting it to a larger number resets it to -1. That's why all our root files had the same events...

2. I really don't know if this is a Root issue or Cint or just my fault. When trying to TChain some files for further processing with a script named ChainTest.C:

void ChainTest(const char* fName = "some_filename")
{
gSystem->Load("libFWCoreFWLite.so");
AutoLibraryLoader::enable();
TChain *chain = new TChain("Events");
chain->Add(fNames);
chain->GetEntries();
chain->GetEvent(0);
...
}

During execution at the GetEvent line I get the error message:

Error: Symbol chain is not defined in current scope
CPlotElectronsTest.C:21:
Error: Failed to evaluate chain->GetEvent(0)

I was using the exact same script to process my previous root files. The difference is that the first contained only generated particles whereas the second digitized events. In the first case works fine!

If the script is modified as:

{
const char* fName = "some_filename");
gSystem->Load("libFWCoreFWLite.so");
AutoLibraryLoader::enable();
TChain *chain = new TChain("Events");
chain->Add(fNames);
chain->GetEntries();
chain->GetEvent(0);
...
}

Everything works perfectly for both root files! If a solution is found I'll post it here as well.