Add metadata to data being sent to API endpoint in json dump registration backend #5053

viktorvanwijk · 2025-01-28T16:13:39Z

Closes #5012

Changes

Add metadata to data being sent to API endpoint in JSON dump registration backend

Checklist

Check off the items that are completed or not relevant.

Impact on features
- Checked copying a form
- Checked import/export of a form
- Config checks in the configuration overview admin page
- Problem detection in the admin email digest is handled
Release management
- I have labelled the PR as "needs-backport" accordingly
I have updated the translations assets (you do NOT need to provide translations)
- Ran ./bin/makemessages_js.sh
- Ran ./bin/compilemessages_js.sh
Dockerfile/scripts
- Updated the Dockerfile with the necessary scripts from the ./bin folder
Commit hygiene
- Commit messages refer to the relevant Github issue
- Commit messages explain the "why" of change, not the how

codecov · 2025-01-30T16:04:24Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.73%. Comparing base (53752fa) to head (58b6492).
Report is 9 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5053   +/-   ##
=======================================
  Coverage   96.72%   96.73%           
=======================================
  Files         770      771    +1     
  Lines       26534    26588   +54     
  Branches     3452     3458    +6     
=======================================
+ Hits        25666    25720   +54     
  Misses        606      606           
  Partials      262      262

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

viktorvanwijk · 2025-01-31T10:11:03Z

Few design considerations:

Tried making the JSONDumpOptionsSerializer.required_metadata_variables field read_only, but then it won't be available in validated_data of the serializer, which is passed to the JSONDumpRegistration.register_submission. Thought about overwriting validated_data and adding the required_metadata_variables manually, but didn't seem like the right thing to do. Or adding a non-field constant for the default values of required_metadata_variables, so it can be accessed easily from the plugin without instantiating the serializer, but not sure if there are any best practises for that.

To show all metadata variables (additional and required) in the metadata selection box, and grey out all variables which are required by default, thought about adding the registration variables to the VariablesSelection component. But, because they are just static variables, they would all be added to this group in the drop-down menu. And afaik we cannot guarantee unique keys between regular static variables and registration variables (or registration variables of different plugins as well). So I would have to redo the grouping of variables in this component, and wasn't sure it would be worth it, or even desired behaviour.

sergei-maertens

Looks good, I think it's mostly polishing from here on :)

src/openforms/forms/json_schema.py

src/openforms/js/components/admin/form_design/registrations/json_dump/JSONDumpOptionsForm.js

src/openforms/registrations/contrib/json_dump/config.py

src/openforms/registrations/contrib/json_dump/plugin.py

src/openforms/registrations/contrib/json_dump/registration_variables.py

sergei-maertens

Neat!

One more thing that caught my attention in storybook is that your help texts for the form fields don't end with a <kbd>.</kbd>, which looks odd and is inconsistent with (most of) the other form fields.

So, form label can be without, help text/tooltip should be with period as they are typically actual sentences (or should be!).

...penforms/js/components/admin/form_design/registrations/json_dump/fields/MetadataVariables.js

src/openforms/registrations/contrib/json_dump/config.py

...penforms/js/components/admin/form_design/registrations/json_dump/fields/MetadataVariables.js

Also add support for an extra static variables registry in generate_json_schema

If there is no authentication, the value can be an empty string

If there is no content (in case of an HTTP 204 response for example), converting the response to json results in an error, so we just return an empty string

* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables. * Fix imports * Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this. * Fix help texts of JSON dump option fields. They are sentences that should end with a period

viktorvanwijk added enhancement topic: registration owner: venlo labels Jan 28, 2025

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch 6 times, most recently from 047a0c9 to 54efef5 Compare January 30, 2025 15:55

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch from 54efef5 to 8f487bf Compare January 30, 2025 16:14

viktorvanwijk marked this pull request as ready for review January 31, 2025 08:20

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch 2 times, most recently from 205d6aa to 5a5e6a8 Compare January 31, 2025 13:45

viktorvanwijk requested a review from sergei-maertens February 3, 2025 08:52

sergei-maertens reviewed Feb 3, 2025

View reviewed changes

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch from 5a5e6a8 to a191924 Compare February 4, 2025 13:07

sergei-maertens reviewed Feb 4, 2025

View reviewed changes

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch 3 times, most recently from 53eacd6 to d81405f Compare February 5, 2025 09:49

sergei-maertens requested changes Feb 5, 2025

View reviewed changes

...penforms/js/components/admin/form_design/registrations/json_dump/fields/MetadataVariables.js Outdated Show resolved Hide resolved

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch from d81405f to 857c799 Compare February 5, 2025 10:26

viktorvanwijk added 6 commits February 5, 2025 11:28

✨ [#5012] Add JSON dump registration variables

98b1c5c

✨ [#5012] Add metadata to JSON dump configuration options

12949ac

Also add support for an extra static variables registry in generate_json_schema

🐛 [#5012] Fix json schema of AuthType

ee17cf5

If there is no authentication, the value can be an empty string

✅ [#5012] Update tests

0e9bbe7

🐛 [#5012] Fix handling of response after post request in JSON dump

728a876

If there is no content (in case of an HTTP 204 response for example), converting the response to json results in an error, so we just return an empty string

viktorvanwijk force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch from 857c799 to 377bd53 Compare February 5, 2025 10:30

🌐 [#5012] Update translations

58b6492

sergei-maertens force-pushed the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch from 377bd53 to 58b6492 Compare February 5, 2025 12:03

sergei-maertens approved these changes Feb 5, 2025

View reviewed changes

sergei-maertens merged commit eea9c23 into master Feb 5, 2025
17 of 18 checks passed

sergei-maertens deleted the feature/5012-add-metadata-to-data-being-sent-to-api-endpoint-in-json-dump-registration-backend branch February 5, 2025 12:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add metadata to data being sent to API endpoint in json dump registration backend #5053

Add metadata to data being sent to API endpoint in json dump registration backend #5053

viktorvanwijk commented Jan 28, 2025 •

edited

Loading

codecov bot commented Jan 30, 2025 •

edited

Loading

viktorvanwijk commented Jan 31, 2025 •

edited

Loading

sergei-maertens left a comment

sergei-maertens left a comment

Add metadata to data being sent to API endpoint in json dump registration backend #5053

Add metadata to data being sent to API endpoint in json dump registration backend #5053

Conversation

viktorvanwijk commented Jan 28, 2025 • edited Loading

codecov bot commented Jan 30, 2025 • edited Loading

Codecov Report

viktorvanwijk commented Jan 31, 2025 • edited Loading

sergei-maertens left a comment

Choose a reason for hiding this comment

sergei-maertens left a comment

Choose a reason for hiding this comment

viktorvanwijk commented Jan 28, 2025 •

edited

Loading

codecov bot commented Jan 30, 2025 •

edited

Loading

viktorvanwijk commented Jan 31, 2025 •

edited

Loading