- 
                Notifications
    You must be signed in to change notification settings 
- Fork 16
Description
There are at least two circumstances when we expect an indicator to produce no output:
- DSEW CPR changes its upload schedule a couple times a year, so we run it daily just in case and to allow for holiday-driven variances. On days when no source report was posted to healthdata.gov, the DSEW CPR indicator produces no output.
- The Quidel COVID tests indicator produces no output if no new source data is available since the last time it ran. This happens when the indicator is run multiple times in the same day without resetting the input cache.
The validator currently throws an exception when it encounters no output. Here's one from the quidel-covidtest run on 2022-12-14 at ~19:15:
Traceback (most recent call last):
 File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
   return _run_code(code, main_globals, None,
 File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
   exec(code, run_globals)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/runner.py", line 83, in <module>
   run_indicator_pipeline(indicator_module.run.run_module,
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/runner.py", line 63, in run_indicator_pipeline
   validation_report = validator.validate()
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/validator/validate.py", line 61, in validate
   all_frames = aggregate_frames(frames_list)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/validator/utils.py", line 82, in aggregate_frames
   return pd.concat(all_frames).reset_index(drop=True)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/pandas/util/_decorators.py", line 331, in wrapper
   return func(*args, **kwargs)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 368, in concat
   op = _Concatenator(
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 425, in __init__
   raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate
For cases like dsew-cpr, when (currently) most days have no output, this exception just creates noise in the #system-monitoring channel, which risks exposure fatigue -- we won't notice when a more serious exception is thrown because this one gets thrown all the time.
For cases like quidel-covidtest, we'd like to know when no new source files were found, but it's not an exception-level notification. A warning would do.
We should modify the validator to handle no-output scenarios without crashing. We may wish to permit an indicator to configure whether no-output should generate a warning or be ignored.
Context:
- validator source (it's a doozy, be prepared)
- sample "suppressed errors" validator configuration format