Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating DSR data pre-processor to support manual data #3693

Merged
merged 2 commits into from
Jun 28, 2023

Conversation

galvana
Copy link
Contributor

@galvana galvana commented Jun 27, 2023

Closes #3686

Description Of Changes

The DSR report builder pre-processes the raw DSR data to separate the collection address into its separate parts, the dataset name and the collection name. The collection address for manual integrations is just the collection name without a dataset. The missing separator : from manual collection addresses causes an error when generating the DSR report. The fix uses a default dataset name of manual for manual collection addresses. I understand that using this approach might cause a collision with a non-manual datasets named manual but the product team (@rsilvery) agreed this was acceptable for now.

Code Changes

  • Updated the way the collection addresses are parsed for the DSR report builder

Steps to Confirm

  • Configure a manual integration
  • Submit an access request
  • Verify the generated HTML report

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Issue Requirements are Met
  • Update CHANGELOG.md

@galvana galvana linked an issue Jun 27, 2023 that may be closed by this pull request
@galvana galvana marked this pull request as ready for review June 27, 2023 23:18
@galvana galvana requested a review from RobertKeyser June 27, 2023 23:18
@cypress
Copy link

cypress bot commented Jun 27, 2023

Passing run #2972 ↗︎

0 4 0 0 Flakiness 0
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.

Details:

Merge 89619f7 into c7effb1...
Project: fides Commit: 4d916bca43 ℹ️
Status: Passed Duration: 01:06 💡
Started: Jun 28, 2023 10:41 PM Ended: Jun 28, 2023 10:42 PM

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@codecov
Copy link

codecov bot commented Jun 28, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (f47bc5d) 87.10% compared to head (89619f7) 87.10%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3693   +/-   ##
=======================================
  Coverage   87.10%   87.10%           
=======================================
  Files         310      310           
  Lines       19031    19053   +22     
  Branches     2437     2439    +2     
=======================================
+ Hits        16577    16597   +20     
- Misses       2026     2028    +2     
  Partials      428      428           
Impacted Files Coverage Δ
.../privacy_request/dsr_package/dsr_report_builder.py 94.93% <100.00%> (+0.06%) ⬆️

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@galvana galvana merged commit 028ee83 into main Jun 28, 2023
@galvana galvana deleted the 3686-fix-manual-integrations-for-dsr-package branch June 28, 2023 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Manual DSR - split throws not enough values to unpack error
2 participants