Project

General

Profile

Bug #1453

MAUS skipping events

Added by Pidcott, Celeste over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Category:
Simulation
Target version:
-
Start date:
09 April 2014
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

I ran simulate_mice for a sample of muons, and when I tried to run the outputted json through my global recon import mapper it failed on event 23. I then went back and looked at the json file that simulate_mice had produced, and looking through the events, there is a spill_number 22 and a spill_number 24, but no spill 23. This is the second time this has happened today, but the first time it happened was at event 1158, so it isn't necessarily the most reproducible of problems. Nonetheless, I have attached my simulate_mice, datacard and the first 30 lines of the json output in case these would be helpful.


Files

simulate_mice_json.py (1.71 KB) simulate_mice_json.py Pidcott, Celeste, 09 April 2014 17:14
datacard_200MeV_mu_plus_copy.py (2.3 KB) datacard_200MeV_mu_plus_copy.py Pidcott, Celeste, 09 April 2014 17:14
example.json (1.62 MB) example.json first 30 lines of json output Pidcott, Celeste, 09 April 2014 17:14
spill23.json (158 KB) spill23.json json document for spill 23, contains a NaN Rajaram, Durga, 10 April 2014 20:15
spill23-nanfixed.json (158 KB) spill23-nanfixed.json Rajaram, Durga, 10 April 2014 20:16

Related issues

Related to MAUS - Bug #1454: NaN in SciFi PR tracksClosedDobbs, Adam11 April 2014

Actions
#1

Updated by Rajaram, Durga over 7 years ago

Is example.json supposed to have the missing spill? Because...I see a "spill_number":23 in example.json at the end of line 25

... 
tof2":[{"charge":8592,"charge_product":2143,"detector":"tof2","dt":-0.09331161040770297,"part_event_number":0,"phys_event_number":0,"pixel_key":"TOFPixelKey 2 3 5 tof2","slabX":3,"slabY":5,"station":2,"time":60.13439171926642}]}},"trigger_event":{}}],"run_number":0,"spill_number":23}

Also, just so I debug with the right version -- are you running against the trunk? Or against an earlier release?

#2

Updated by Rajaram, Durga over 7 years ago

Can you also provide the error output you get when it failed?

#3

Updated by Pidcott, Celeste over 7 years ago

That is curious, I typically use a bash script that lets me view my json files in a human readable format in less, and when I search for the string spill_number in there, not only is there not a 23, but spill_number appears at a different point in the structure:

"global_event": {
        "track_points": [], 
        "tracks": [], 
        "space_points": [], 
        "primary_chains": []
      }
    }
  ], 
  "spill_number": 22, 
  "errors": {}, 
  "daq_event_type": "physics_event", 
  "mc_events": [
    {
      "tracks": [
        {

but then looking at the file in the standard non-human readable format in less, I then can find spill_number 23 as you did, so I'm not sure what has happened there.

And this was in 0.8.0, but I will be trying it in 0.8.3 once it has finished installing on my machine.

Error output :

TRANSFORM/MERGE/OUTPUT:  Processed 23 events so far, 1 events in buffer.
  Missing an end_of_run spill...
  ...creating one to flush the mergers!
TRANSFORM: Shutting down transformer
MERGE: Shutting down merger
INPUT: Shutting down inputer
OUTPUT: Shutting down outputer
Clearing Globals
Traceback (most recent call last):
  File "bin/newGlobal/GlobalReconImport.py", line 31, in <module>
    run()
  File "bin/newGlobal/GlobalReconImport.py", line 28, in run
    my_output, datacards)
  File "/home/celeste/MICE/MAUS/PIDframework/src/common_py/Go.py", line 131, in __init__
    self.get_job_footer())
  File "/home/celeste/MICE/MAUS/PIDframework/src/common_py/framework/single_thread.py", line 97, in execute
    map_buffer = DataflowUtilities.buffer_input(emitter, 1)
  File "/home/celeste/MICE/MAUS/PIDframework/src/common_py/framework/utilities.py", line 67, in buffer_input
    value = next(input_emitter)
  File "/home/celeste/MICE/MAUS/PIDframework/build/InputPyJSON.py", line 114, in emitter
    json.loads(next_value)
  File "/home/celeste/MICE/MAUS/merge/third_party/install/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/home/celeste/MICE/MAUS/merge/third_party/install/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/celeste/MICE/MAUS/merge/third_party/install/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
#4

Updated by Rajaram, Durga over 7 years ago

So...what's happening is that the json document for spill 23 is failing to get decoded
  • because, it has
    "chisq_dof":-nan
  • and JSON thinks not-a-number, nan, is a string and since nan is not enclosed within quotes, json.loads fails to parse
  • As a quick and dirty test, I edited spill23.json and changed nan to
     "chisq_dof":-999
    and it goes through fine
  • See attached spill23.json and spill23-nanfixed.json
    bin/utilities/json_browser.py --input-file <filename> 
    will fail on spill23.json input, but pass with spill23-nanfixed

Raising this as a tracker bug

#6

Updated by Rajaram, Durga over 7 years ago

Raised as tracker issue #1454

#7

Updated by Rajaram, Durga over 7 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF