Consider new ways to calculate direction

[capnrefsmmat]

There are several things to consider:

• The deaths signal is sparse (many zeros) and high-noise (no smoothing), so direction can jump around a lot in individual counties. Should it smooth more? Is it meaningful to declare direction in these cases?
• We set a threshold for what counts as increasing or decreasing on the other signals. Are we happy with this threshold?

Note that currently, direction is calculated for all signals server-side, so the ideal solution would not require special-casing specific signals.

[krivard]

Current method of calculating direction just runs a linear regression on the last 7 points, so we do get some amount of smoothing for free without having to produce a smoothed signal per se. The problem with deaths is that, because it's unsmoothed, the window size of 7 points is insufficient to filter out the noise.

I'm not unilaterally opposed to special-casing signals, just to making changes to the epidata acquisition code on a friday afternoon...

If we want something more general-purpose, we could flag the filename in a way that gets stripped out before it reaches the API.

[krivard]

Tagging @huisaddison

[capnrefsmmat]

I just wanted to record a thought I had earlier today, after talking with Alberto Cairo:

Direction is hard to formalized, as we've seen. We want the user to be able to tell whether a given indicator is going up or down, whether that trend is meaningful compared to the typical noise in the trend, and how that trend compares to other locations across the US. The casual user presumably wants to know "How bad is my (city, state, county) compared to everywhere else?"

But direction doesn't directly answer that question. A discretized direction signal just tells us "Is my (city, state, country) getting worse?", with no sense of scale for "worse."

I think this is actually a problem we should solve with visualization instead of a more clever direction signal definition.

Specifically: Click on one region. The time series appears in the time series plot. But in the background appear many other time series, in faint gray, from other regions. These faint lines show us the typical variation in the time series. So I can see immediately, by eye, whether the trend in this one seems different from the typical noise.

We lose the ability to compare across a map; but we gain the ability to compare across time series. I think the latter is more often what we want.

Unfortunately this still leaves deaths as a problem, because the variance of the time series changes a lot by population. I'm not sure that can be worked around: the time series for a rural area where one person dies every three days is never going to be meaningful, no matter what we do to it.

[krivard]

@tibshirani, here is the 7-day-trend/direction issue we want to add as a candidate for 1.5. Alex brings up above that another way to communicate the idea behind the 7-day-trend may not actually involve calculating trends at all, but displaying the current signal in a context that lets people see how much better or worse the signal is doing relative to others in its class. I think we should prototype both approaches over the next two weeks to see which we like better.

[tibshirani]

sounds like a good idea

…

On Sun, Jun 21, 2020 at 5:48 AM krivard ***@***.***> wrote: @tibshirani <https://github.com/tibshirani>, here is the 7-day-trend/direction issue we want to add as a candidate for 1.5. Alex brings up above that another way to communicate the idea *behind* the 7-day-trend may not actually involve calculating trends at all, but displaying the current signal in a context that lets people see how much better or worse the signal is doing relative to others in its class. I think we should prototype both approaches over the next two weeks to see which we like better. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVXOYIX25KOJYRVNUHKKLLRXX6TBANCNFSM4M4LIDFQ> .

-- I get so much email that I might not reply to an incoming email, just because it got lost. So don't hesitate to email me again. The probability of a reply should increase. Prof. Robert Tibshirani Depts of Biomedical Data Sciences, and Statistics Stanford Univ Stanford CA 94305 tibs@stanford.edu http://www-stat.stanford.edu/~tibs

[krivard]

Consider computing historical variability per-location, not per geo_type (if we can compute it fast enough)

[krivard]

With no further feedback, we're going with a 14-day linear regression.