Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As admin, I want to create an export of all submissions to compare them with registration systems #4930

Open
Tracked by #4920
joeribekker opened this issue Dec 16, 2024 · 2 comments · May be fixed by #4962
Open
Tracked by #4920

Comments

@joeribekker
Copy link
Contributor

The goal is to see if Open Forms submitted more or less forms then the number of cases created in backend systems. So, the admin wants to compare the submitted forms to the created cases in another system.

To this end, we'll expand on the existing statistics page, and add an export function.

Tasks

  1. Add date-range filter to statistics: https://pypi.org/project/django-daterange-filterspec/
  2. Add export bulk action: Export statistics
  3. Upon export, generate the statistics details based on timeline logger events from succesfull submissions
  4. Export the columns: public reference, form name, submission datetime

Note: I know the date in the existing view doesn't match the expectations. It now filters on "last submission" date. In a followup issue, this will be addressed.

@robinmolen
Copy link
Contributor

What do we do about the submissions removal limit options? These can be set to "Submissions will be deleted", and completely remove the submission data. This would then result into less exportdata then expected.

(I think the successful submission log is preserved, so we will be able to count the amount of submissions that were successful)

I doubt we want to remove the "Submissions will be deleted" option. So do we just export all the submissions that are remaining? (This would than be different then the expected amount, and would also be different the amount we show in the statistics)

This could of course also happen with manual removal of submissions..

@sergei-maertens
Copy link
Member

The suggested library is not maintained and doesn't appear to test/support Python > 3.10 and Django > 4.0, so we will not be adding this dependency.

@sergei-maertens sergei-maertens linked a pull request Dec 20, 2024 that will close this issue
10 tasks
sergei-maertens added a commit that referenced this issue Dec 20, 2024
Export submission statistics based on the timeline logs.

Note: if the timeline logs are pruned, this affects the exports. It's
up to the users to periodically create these exports and save them
somewhere if they periodically prune log records.

Note 2: filtering on forms only works on new log records, as existing
log records don't have the form ID stored in the structured data.

Submissions that were deleted for which existing log records are
present will display 'unknown' for some columns because the relevant
information has been deleted. Only from 3.0 onwards are we snapshotting
the data required for the exports.
sergei-maertens added a commit that referenced this issue Dec 20, 2024
Ensures that the filters are correctly implemented.
@sergei-maertens sergei-maertens moved this from Todo to Implemented in Development Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Implemented
Development

Successfully merging a pull request may close this issue.

3 participants