Project

General

Profile

Bug #1782

Tracker kalman fit pz reconstruction broken

Added by Dobbs, Adam about 8 years ago. Updated almost 8 years ago.

Status:
Open
Priority:
High
Category:
Tracker
Target version:
Start date:
23 October 2015
Due date:
% Done:

0%

Estimated time:
Workflow:
New Issue

Description

The Kalman fit pz reconstruction is currently out by a very significant amount when compared with the TOF calculation from real data, and when doing MC studies.


Files

tracker_questions.zip (2.74 MB) tracker_questions.zip Dobbs, Adam, 23 October 2015 15:13
run_7469_with_muon_tof_cut.pdf (1.11 MB) run_7469_with_muon_tof_cut.pdf Blackmore, Victoria, 02 November 2015 12:14
run_7469_muons_219_to_221MeV.pdf (391 KB) run_7469_muons_219_to_221MeV.pdf Blackmore, Victoria, 02 November 2015 12:14
run_7469_muons_219_to_221MeV_wPvalCut.pdf (246 KB) run_7469_muons_219_to_221MeV_wPvalCut.pdf Blackmore, Victoria, 02 November 2015 14:52
run_7469_muons_219_to_221MeV_wPvalCut_0pt05.pdf (316 KB) run_7469_muons_219_to_221MeV_wPvalCut_0pt05.pdf Blackmore, Victoria, 02 November 2015 15:06
momentum_comparison.pdf (28 KB) momentum_comparison.pdf Blackmore, Victoria, 04 December 2015 16:18
pvalues.pdf (13.7 KB) pvalues.pdf Blackmore, Victoria, 04 December 2015 16:18
pvalues_a.pdf (163 KB) pvalues_a.pdf Blackmore, Victoria, 08 December 2015 16:57
pvalues_b.pdf (154 KB) pvalues_b.pdf Blackmore, Victoria, 08 December 2015 16:57
3-200_mu__incCuts_plots.pdf (278 KB) 3-200_mu__incCuts_plots.pdf Step I Blackmore, Victoria, 18 December 2015 11:30
momentum_comparison.pdf (24.4 KB) momentum_comparison.pdf with cut on Pz at tracker between 180 and 600 MeV Blackmore, Victoria, 18 December 2015 11:30
#1

Updated by Dobbs, Adam about 8 years ago

Email from Victoria (22/10/15):

It's about time I brought this up, as maybe there's a simple answer.

I've attached a zip of (many) plots, to help explain the question --
each pdf in there is formatted in the same way, so "page 2, top right"
is always the same plot in all of them.

So.. to the question:

I can look at particles in a specific time-of-flight window and,
assuming they're muons, get an estimate of the particles longitudinal
momentum. I assume the distance they travel is L = sqrt(dx^2 + dy^2 +
dz^2), and dx = x(TOF1) - x(TOF0) etc. This should give an estimate
within around 5MeV of the 'truth'... they don't travel an enormously
larger excess path length than the straight line assumption as they
'wiggle' between Q789.

If I accept only momenta between 199 and 201 MeV, I can then see what
momenta the tracker predicts for the particle. For this example, what I
see is a board distribution of Pz in the tracker, with a mean of ~
156MeV, and rms of 25 MeV. Do you know why?

(I get my TOF info from TOF space points, and my tracker info from
tracker track points. The only cut I'm applying is on the momentum
from time-of-flight)

To illustrate, take a look at the zip:
- run_7469_all_particles.pdf --> plots all particles, for all
time-of-flights, be they e, mu or pi.
- run_7469_with_muon_tof_cut.pdf --> plots all muons, by taking a
broad cut on time-of-flight
- run_7469_muons_199_to_201MeV.pdf --> takes the previously
mentioned momentum slice from 199-201 MeV (according to TOF)
- run_7469_muons_219_to_221MeV.pdf --> takes a momentum slice from
219 to 221 MeV (so you can see the same effect at higher momentum
according to the TOF)

In those pdfs:
- Page 1, left: the time of flight, acquired from TOF space point
hit times (TOF1 - TOF0)
- Page 1, right: the momentum, assuming its a muon, from that
time-of-flight
- Page 2, top right: The p-value of the tracker track point (I was
at a bit of a loss for a good place to stick this plot, so there it is!)
- Page 5, top right: Pz from the tracker at station 5 (also see
pages 6--9 for the other stations)

Statistics are low for a small momentum slice, but it should, surely,
still be centred on something closer to what I select at the TOFs.
There'll be some momentum loss from air etc (and TOF1), but it looks
like a 20MeV effect, which is a bit large...

There's nothing much there correlating Pz in the tracker with x or y
(see Page 5, middle & bottom right) -- if large radius particles had
gone through the outsides of the diffuser and come in, my gut feeling is
that we'd see some funny shape in those distributions.. but it looks
flat with Pz. Plus most of them shouldn't have gone through the
outside of the diffuser, most of them should have gone through the
middle where there was nothing (as it was open).

I don't get why the p-values look the way they do, but maybe you can
explain that...

Other thoughts: does air in the upstream tracker really screw us up
that badly? Does the non-uniform field screw us up? (I suspect it's
non-uniform.. M1 & M2 weren't powered and they didn't try and account
for it.. nor for the PRY enhancing the field, so it was probably more
than 4T)

Is there already a simulation out there with air in the trackers that
shows that we should expect a 20MeV momentum loss between TOF1 and the
tracker?

Chris R. suggested earlier I see if pattern recognition tells the same
story, so I will see if I can do that... but in the meantime, I'm just a
bit puzzled by how such a tight momentum spread becomes so broad and
messy by the time it gets to the tracker.. and do I trust the tracker?
(I do want to trust the tracker.. but to do that I also need to figure
out if 20MeV is expected or not!)

#2

Updated by Dobbs, Adam about 8 years ago

Reply from Chris Hunt (22/10/15)

Hi Victoria,

You've basically written my todo list I think.

The Pz reconstruction genuinely doesn't work. There is a systematic bug (even in MC) that I haven't managed to fix. It gets close - but not close enough, although I didn't think it was out by 20MeV... I have an idea as to the cause but I have had no time to have a good go at fixing it for a quite a while, due to the onslaught of data based tasks.

Same for the p-values there are a few issues floating around that just haven't been looked at yet. I've started using a 0.01 cut as most of the bad tracks end up in the first bin anyway, so that seems fine. Interestingly a lot of the poor track fits are tracks without all 15 trackpoints so I think there is something there that can be easily avoided with some intelligent cuts, at least for the time being.

Personally I don't think the air will really make much difference certainly not 20MeV. Can't remember numbers, but the tracker's not really very long. But the field will be noticeable, as you said not having M2 is a bit of a pain.

Once the CM is over and I can throw the alignment note at people to review I will back onto Kalman to fix those (and other) issues. Sorry its not much use right now, but there just hasn't been enough time/people.

Generally, Patrec seems to reconstruct more reliably, so it may well be a safer choice for the near future, but it is a little less sensitive to noise, etc, as you don't have the p-value type data available at first glance. It would be good to compare. The kalman transverse components have always seemed ok for the most part, but I really would not trust the pz reconstruction.

Cheers

Chris

#3

Updated by Dobbs, Adam about 8 years ago

Reply from Victoria (22/10/15):

Sorry to hear the to-do list is getting longer!

(Though I can't count.. 200 - 156 != a 20 MeV drop, but more like 40!)

Looking at the PDG, we should get ~ 10 MeV loss in the TOF and ~ 0.5MeV
in the air between TOF1 and the most upstream part of the tracker. 40
just doesn't fit..

Can you see what kind of difference the MC has when you get a chance?

#4

Updated by Blackmore, Victoria about 8 years ago

Ken caught a logic problem: Using time of flight between TOF0 and TOF1 gives me total momentum, not Pz.

I've added some plots to the end of the giant-string-of-plots to compare P from TOF0-->TOF1, with sqrt(Px^2 + Py^2 + Pz^2) from the upstream tracker. I've also taken the difference as TOF P - tracker P.

These plots start at page 15 on the attached pdfs (the other plots are the same as attached to the initial issue). If you take a "220" MeV beam according to the TOFs, you still end up with a mean of 187 MeV total P at the tracker. There's still a significant broadening in total momentum (19.6 MeV rms in this case)... so I don't think this problem is magically solved by realising that I was comparing total P with Pz.

#5

Updated by Blackmore, Victoria about 8 years ago

Insisting that the p-value returned by the track fit is > 0.5 gives the attached plots (which should directly compare to the previous ones). In this case, a 2MeV slice based on the TOFs alone lets us see 1087 particles. After insisting that the p-value from the tracker > 0.5, this reduces to 260 particles.

The difference in total momentum is still on the order of 30 MeV (and the mean could be easily swayed in these small samples).

So.. in addition to the question of why the momentum seems to be so different is the question of why the p-values are all pretty grim? (And am I right to say they need to be bigger than 0.5 or is another value a better threshold?)

#6

Updated by Blackmore, Victoria about 8 years ago

Durga remembers using a p-value of 0.05, so here's the same string of plots with that as the cut instead. Nothing really changes (though there's more particles in the plot... 619 down from 1087)

#7

Updated by Dobbs, Adam about 8 years ago

Thanks Victoria, this is very helpful to know (especially the extra info about the p values), your case is made. I believe Chris us now working on this as his top priority...

#8

Updated by Blackmore, Victoria about 8 years ago

So here's the lessons I've learned today:

1. I need time and space to write good analysis code
2. Under pressure, I write poor analysis code and make stupid mistakes
3. People assume that I have (1) and don't check I'm not stuck in (2)

So in the plots beforehand, when I was asking for advice and wondering what I'd done wrong... someone should have said "that's funny... everything's out by a factor 0.8" and I'd have realised that I'd missed a factor of relativistic beta (v/c ~ 0.8 for our muons), and everyone would've been much happier!

I attach a new comparison plot, total momentum for all particles (no cuts) from time-of-flight assuming they're muons (not normalising to the electron peak, as previous plots weren't), total momentum from Kalman, and total momentum from pattern recognition (both at plane 1 of the upstream tracker). This time I remembered to type the vital "P = gamma * m * beta" that I always meant to type in the tof calculation...

I've attached the P-value plot because it still looks odd to me and I'd like to understand why it looks the way it does...

(It's nice to be able to think away from the noise of the control room!)

(I hope this makes everyone's weekends better -- if so, I can live with the embarrassment!)

#9

Updated by Rogers, Chris about 8 years ago

Great news! Well done for figuring it out.

#10

Updated by Blackmore, Victoria about 8 years ago

So... looking a bit more into the oddness of the p-values in run 7469

There are two pdf's attached, with the same plots but slightly different cuts:

  • pvalues_a.pdf contains plots with p-value cuts (either side of) of 0.05
  • pvalues_b.pdf contains plots with p-values > 0.0001 and cuts (either side of) 0.05 -- this was to try and remove the huge peak at zero and get a better idea of the 'double hump' nature of the p-value distribution
  • Both of these pdf's have a time-of-flight cut between TOF0 and TOF1, accepting only particles with tof > 26.5ns (and they must all have gone through TOF1 and the upstream tracker)

The plots are as follows:

  • page 1: The p-value distribution, taken from MAUS::SciFiTrackPoint->P_value()
  • page 2:
    • Top: 2D histogram of (total) momentum at TOF1 and the P-value of the particle at the upstream tracker
    • Bottom: 2D histogram of (total) momentum at tracker plane 1, and the P-value of the particle
  • page 3:
    • Top left: Px at plane 1 of the upstream tracker and the P-value
    • Top right: Py at plane 1...
    • Bottom left: Pz at plane 1 ...
    • Bottom right: Total P at plane 1 (a repeat of page 2, bottom)
  • page 4: A lot of histograms for which the legend defeated me...
    • Thin black line: Total P at TOF1, for all particles regardless of their P-value at the tracker (or, if you're looking at pvalues_b.pdf, have P-values > 0.0001)
    • Thin green line: Total P at TOF1 for particles with P-value > 0.05
    • Thin red line: Total P at TOF1 for particles with P-value < 0.05
    • Thick black line: Total P at plane 1 of the upstream tracker for all particles regardless of their P-value
    • Thick green line: Total P at plane 1 of the tracker for particles with P-value > 0.05
    • Thick red line: Total P at plane 1 of the tracker for particle with P-value < 0.05
  • page 5: The histograms from page 4, split into (top) TOF, and (bottom) tracker only plots...
  • page 6: The histograms from page 4, split into individual plots so you can see the stats boxes more easily
  • page 7: Correlation between momentum at TOF1 and momentum at plane 1 of the tracker for...
    • (left) All particles, regardless of their P-value
    • (middle) Particles with P-value > 0.05
    • (right) Particles with P-value < 0.05

I had wondered if the plots on pages 2 & 3 would point to one 'hump' in the P-value distribution being at low transverse momentum and one at high... but it looks like its fairly evenly distributed across the full range.

Can someone tell me how a P-value is assigned to the tracks? (What are the two hypotheses being tested exactly...? Good track fit/bad track fit, or something else?) I'm just generally confused about the shape of the distribution, and its probably because I'm ignorant about how those numbers are assigned...

#11

Updated by Rogers, Chris about 8 years ago

Nb: do you have a correct field map including the effect of iron for the run? Is this something that we (I) should chase?

#12

Updated by Hunt, Christopher about 8 years ago

Hi Victoria,

I've spent a little time returning to the p-value calculation and there are a couple of bugs in it. I've fixed one, and I think I'm about to fix the second, but I'll find out later this week.

In short, the P-value corresponds to the probability of calculating a worse chi-squared than the one returned, given the expected chi-squared distribution for the number of degrees of freedom.

I'm still trying to decide if I have the correct NDF, but the algorithm itself is pretty much fixed in my devel branch.

In theory, the p-value distribution should be flat if we have:Chris Hun
- The correct NDF
- Correct description of the system
- Correct estimation of the errors

I need to push a couple of tracker updates to the merge branch this week. The P-value calculation will be one of them.

#13

Updated by Blackmore, Victoria almost 8 years ago

Just a comment on another (related) comparison that I noticed whilst comparing the (3, 200) mu+ beam measured during Step I and run 7469 (which is supposed to be a 3, 200 mu+ beam..

In Step I, the mean muon momentum at TOF1 was ~ 222MeV
In 7469, the mean muon momentum is more like ~ 200 MeV (both TOF and tracker see about this)

The quadrupole and dipole currents were the same, but the DS was off in 7469. (Edit: the DS was on for the first 10-20 minutes of the run, but broke during it.. that's why the DS is 'on' in the CDB)

Francois's MC shows that we scrape away high momentum muons along the beamline.. and it kind of makes sense that without the DS we scrape off more, but I was somewhat surprised by the difference (the widths of the momentum distributions are about the same, for what it's worth, and obviously there're vastly more particles in the Step I data)

Just.. something we should watch out for.

#14

Updated by Blackmore, Victoria almost 8 years ago

Actually, it's not as bad as I first thought.. but there is still a difference.

I've attached the string of plots made from the (3, 200) mu+ beam accumulated during Step I (3-200_mu+_incCuts_plots.pdf), which has a momentum selection between 180 and 600 MeV applied to it.

Making the same selection on run 7469 (using the Kalman estimated of Pz at the tracker, which is why there are some muons at <180 in the TOF and PattRec distributions), then the means go to ~ 210MeV (see momentum_comparison.pdf attached here).

There's still a difference, which we should try and understand..

#15

Updated by Blackmore, Victoria almost 8 years ago

Of course... ISIS were also operating at 700 MeV during that run. So perhaps the first thing to do is compare some of our recent reference runs (now they're back at 800MeV) with some from October and see if the proportions changed.

(I'll see what I find, assuming this works as an aide mémoire for me!)

Also available in: Atom PDF