Skip to content

Validator throws an exception when no output is produced by the indicator #1746

@krivard

Description

@krivard

There are at least two circumstances when we expect an indicator to produce no output:

  • DSEW CPR changes its upload schedule a couple times a year, so we run it daily just in case and to allow for holiday-driven variances. On days when no source report was posted to healthdata.gov, the DSEW CPR indicator produces no output.
  • The Quidel COVID tests indicator produces no output if no new source data is available since the last time it ran. This happens when the indicator is run multiple times in the same day without resetting the input cache.

The validator currently throws an exception when it encounters no output. Here's one from the quidel-covidtest run on 2022-12-14 at ~19:15:

Traceback (most recent call last):
 File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
   return _run_code(code, main_globals, None,
 File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
   exec(code, run_globals)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/runner.py", line 83, in <module>
   run_indicator_pipeline(indicator_module.run.run_module,
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/runner.py", line 63, in run_indicator_pipeline
   validation_report = validator.validate()
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/validator/validate.py", line 61, in validate
   all_frames = aggregate_frames(frames_list)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/delphi_utils/validator/utils.py", line 82, in aggregate_frames
   return pd.concat(all_frames).reset_index(drop=True)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/pandas/util/_decorators.py", line 331, in wrapper
   return func(*args, **kwargs)
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 368, in concat
   op = _Concatenator(
 File "/mnt/data7/indicators/runtime/quidel_covidtest/env/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 425, in __init__
   raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate

For cases like dsew-cpr, when (currently) most days have no output, this exception just creates noise in the #system-monitoring channel, which risks exposure fatigue -- we won't notice when a more serious exception is thrown because this one gets thrown all the time.

For cases like quidel-covidtest, we'd like to know when no new source files were found, but it's not an exception-level notification. A warning would do.

We should modify the validator to handle no-output scenarios without crashing. We may wish to permit an indicator to configure whether no-output should generate a warning or be ignored.

Context:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions