MDC handle legacy counts

[qqmyers]

What this PR does / why we need it:

This PR adds 'legacy' download counts prior to Make Data Count being enabled to the metrics display and adds some messages helping to explain why, for example, the file download counts may not match the MDC downloads.

Which issue(s) this PR closes:

Closes #6542

Special notes for your reviewer:

Set the :MDCStartDate setting to the day MDC logging was enabled , e.g. '2020-12-31'
MDC logging and display also need to be enabled.

Suggestions on how to test this:

Datasets with downloads prior to MDC should show the legacy counts, others should not. See issue for images.

Does this PR introduce a user interface change?:

Yes - see issue #6542 for images/description

Is there a release notes update needed for this change?:

Probably should note that MDCStartDate should be set (default is not set which leaves legacy counting off).

Additional documentation:

If this looks like something that will be merged, I can add docs/release notes. For now, just wanted to get the changes out.

[coveralls]

Coverage: 20.335% (+0.2%) from 20.184% when pulling 791319e on QualitativeDataRepository:IQSS/6542 into e4961fe on IQSS:develop.

[djbrooke]

Thanks @qqmyers for this PR. I'm holding this from review/QA a few days so that I can discuss with Merce and others about prioritizing a migration of old counts to new counts. Hopefully we can figure out this piece and just display one number.

[djbrooke]

@qqmyers and others who are interested, a bit of an update... We're still discussing how to best handle the old counts and have reached out to MDC and received feedback. Thanks @pdurbin @TaniaSchlatter and @kcondon for discussing this after standup today. I think our first pass of this will be to get a PoC for an API that migrates (maybe not the best term for this) the old counts to the tables/logs for the new counts. Thanks @pdurbin for offering to create an issue for this. The goal remains to show one number on the page, so if we're able to do that satisfactorily, we can revisit the need for this PR.

[qqmyers]

@djbrooke - is the ~migration going to be done via log file mining? Or is this something that can be done solely with info already in the DB. I ask because I don't think QDR has all logs, but if there's a way to generate one number that has a clear meaning given what we do have, that's probably preferable for QDR as well.

In the meantime, after discussion at QDR, I'm going to modify the code/strings slightly to be a little more clear. Depending on the requirements for ~migrating, we may then go ahead and deploy this or wait for more info. No need to review/merge if others won't need it. FWIW, we did discuss making details visible in a popup, but unless the main 'total' number includes the pre-MDC counts somehow, this won't be enough.

[djbrooke]

@qqmyers I believe what we're proposing will only use information already in the DB, but definitely keep an eye out for the new issue from @pdurbin for more details. The goal remains that we can include pre-MDC counts and MDC counts in a single number and provide some explanation in a tooltip/popover.

[pdurbin]

Here's the new issue but for now it's just an image. We're in sprint planning now but after I'll beef up the description: IQSS/dataverse.harvard.edu#75

[qqmyers]

FWIW: The last couple commits make some changes that QDR is implementing after some internal review. There are some wording changes and an addition to show the MDC turn-on date, which helps explain the pre-MDC number appearing.

I think this is what QDR is going to move forward with for now. We'll watch the progress for a single number solution, and if this PR ever becomes useful, it's here.

[djbrooke]

Thanks @qqmyers for the update, I'll close it for now. We're also hopeful for the single number solution in IQSS/dataverse.harvard.edu#75 :)

[qqmyers]

FWIW: Here's what we've ended up with on QDR. Newer datasets (past Oct 1, 2019) show only MDC counts while older datasets show two numbers with a pop-up explanation for the pre-MDC counts.

[qqmyers]

I've updated the branch (only 8854 commits!) and am reopening so it can be looked at for IQSS/dataverse-pm#8. I did not run the updated version - the main conflict point was in some css in the MDC metrics block so that could be off.

[pdurbin]

At standup we decided we'd like to give this PR a try: IQSS/dataverse.harvard.edu#75 (comment)

[qqmyers]

This has been merged with develop. @pdurbin - not sure where it should go on the board(s).

[pdurbin]

I took a quick look and left a comment.

We gave it a 10 today at standup. It's a small code change but testing will take some time. We put it on the main board under Sprint Ready.

I'd like to remind the collective consciousness that @donsizemore implemented support for Counter Processor in https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible which might help make testing easier.

[pdurbin]

Can :MDCStartDate be changed from a database setting to MPCONFIG?

And can we please have a release note about the new setting?

[landreev]

I added a quick release note (via a PR, since I don't have a write on the branch directly).
Other than that it's ready to go.