Skip to content
This repository has been archived by the owner on Jun 20, 2023. It is now read-only.

Privacy Preserving Analytics (main-feature-branch) (EXPOSUREAPP-4321) #2344

Merged
merged 22 commits into from
Feb 15, 2021

Conversation

d4rken
Copy link
Member

@d4rken d4rken commented Feb 12, 2021

WIP

This PR will be added via merge commit when done.
All PRs into this branch need to be reviewed and squashed.

d4rken and others added 11 commits January 29, 2021 10:30
…2225)

* Skeleton classes for Analytics.

* Rebased branch.

* Adjust interface names.

* Adjust TODOs, to calm the LINTER. WIP
Update ppa main to include new protos and stuff from 1.13.x
* PPA User Input Strings

* Data Donation: PPA User Input.
First draft.

TODO:
Districts
Tests

* Fix interaction with radio button..
Fix state matching for NRW
Add district mapping and display.
Fix "unspecified" element position.

* Test skeletons.

* Finish unit tests.

* Wish the LINTer a nice evening.

* Move extension functions into their own file and make them public to allow label-related code to be reused on the settings screen.
Additional unit tests that check the federal state short code mapping.

* XML formatting.

* Address additional PR comments.

* Adjust radiobutton accent color.
* added implementation for ExposureRiskMetadataDonor,

the de.rki.coronawarnapp.server.protocols.internal.ppdd.PpaData namespaces is populated by protos that are not yet present but will be added soon

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* updated imports to match new proto root classes, (still missing from repo)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* switched to kotlin only solution for proto <-> settings string conversion

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* added unit test for exposure risk metadata serialisation

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added error handling to previousExposureRiskMetadata
Added test that checks for changes in ExposureRiskMetadata serialization

Signed-off-by: Kolya Opahle <k.opahle@sap.com>
* Switched protobuf container to builder to allow for instantiation
added UserMetadataDonor

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Update UserMetadataDonor.kt
* remove redundant district prefixes

proposal from UA after reviewing the rather technical names of the initial version

* Adjust unit test to new districts JSON.

* add missing spaces to district names

* remove trailing whitespace from distict names

Co-authored-by: Matthias Urhahn <matthias.urhahn@sap.com>
* Strings for onboarding

* view and image

* View attached to onboarding process

* Complete onboarding screen

* Tests and screen capture

* Accessibility

* Reformatting

* New screen with additional info

* New screen with routing

* Small change in legal text

* Missing sentence in description of contact journal (EXPOSUREAPP-4941) #2309

Co-authored-by: Ralf Gehrer <ralfgehrer@users.noreply.github.com>

* Change statistics overview header to 'bundesweite' Statistiken (EXPOSUREAPP-4975) #2305

Co-authored-by: Ralf Gehrer <ralfgehrer@users.noreply.github.com>

* Show normal behavior box when tracing disabled (EXPOSUREAPP-4768) #2306

use toolbar

Co-authored-by: Ralf Gehrer <ralfgehrer@users.noreply.github.com>

* Delta onboarding analytics screen

* Delta onboarding navigation

* Strings fix

* Comment for Kolya

* Layout, nav grav and fragments refactoring

* Merge conflicts and formatting

* Add missing tools:layout and format the file

Co-authored-by: janetback <68653277+janetback@users.noreply.github.com>
Co-authored-by: Ralf Gehrer <ralfgehrer@users.noreply.github.com>
Co-authored-by: Chilja Gossow <49635654+chiljamgossow@users.noreply.github.com>
Co-authored-by: Mohamed Metwalli <Mohamed.Metwalli@sap.com>
* added settings entry

* added basic layout for the settings screen

* updated fragment layout and navigation

* linting

* adapted settings entry icon

* linting

* fixed strings and nav_graph

* added analytics disabled icon for settings entry

* fixed settings entry icon and text

* added layout to new navgraph items

* updated the layout and addressed the comments

* removed blank lines from layout
* Updated settings icons

* added navigation to analytics more information fragment

* linting

* fixed max length

* removed unneeded marging at botton of scrollview

* added dimens

* aligned fonts

* added row click

* updated doc

* aligned fonts
@d4rken d4rken added maintainers Tag pull requests created by maintainers feature request A new feature proposal for the app author merge PR to be merged by author labels Feb 12, 2021
@d4rken d4rken added this to the 1.13.0 milestone Feb 12, 2021
@d4rken d4rken added enhancement Improvement of an existing feature and removed feature request A new feature proposal for the app labels Feb 12, 2021
d4rken and others added 10 commits February 12, 2021 17:15
# Conflicts:
#	Corona-Warn-App/src/main/res/values-de/legal_strings.xml
#	Corona-Warn-App/src/main/res/values-de/strings.xml
#	Corona-Warn-App/src/main/res/values/legal_strings.xml
* initial commit of Analytics upload code (lots of missing pieces)
kind of working: Definition of backends and endpoints, donor data collection (no donors implemented here)
probably broken: SafetyNet attestation, actual server communication

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* added checks for analytics submission abortion

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* added analytics submission periodic worker definition

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* lint fixes

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed kotlin support for IntoSet injection
Added AnalyticsModule to DataDonationModule
Added WorkerBinder definition for DataDonationAnalyticsPeriodicWorker
Added DataDonationAnalyticsPeriodicWorker scheduling to CoronaWarnApplication
Added missing mocks to WorkerBinderTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed injectData call in ExposureRiskMetadataDonor
Added more logging to Analytics
Implemented first AnalyticsTests
Fixed auto formatted removal of io.kotest.matchers.collections.shouldContainAll in WorkerBinderTest (curse you Optimize imports)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implemented Test for ExposureRiskMetadataDonor

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added check for isEnabled during Analytics submission
Added DataDonationAnalyticsOneTimeWorker to allow for random distribution of Analytics submission events

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Moved contribution collection to own function to help with test menu features

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added analytics testing features

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* small fixes to the WorkerBinderTest
added missing worker binding to WorkerBinder

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Put Analytics test options into an extra card.

* in my defense, it was quite late

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Moved AnalyticsSettings to own package
Updated the @IntoSet providers in AnalyticsModule to match the actual DonorModules

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added analyticsEnabled preference to AnalyticsSettings

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added a deleteData function to the DonorModule interface which can be used to remove persisted data if a user revokes analytics consent

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added skeleton to persist last analytics submission on tester devices
Currently the tester version also does not store the data

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed borked unit tests

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implemented File based storage of Last submitted analytics file on tester devices

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added calls to enable and disable analytics to the OnboardingAnalyticsViewModel

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed OnboardingAnalyticsFragmentTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Turns out AnalyticsConfigMapping was not implemented yet, fixed that

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added a timestamp to the stored Analytics Submission

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed unit tests in AnalyticsTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Updated AnalyticsConfigMapper to throw an exception if app config is invalid

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implemented most of the comments from @d4rken

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* if -> when magic

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Moved analytics enable/disable to the Analytics class to allow for deletion on disable

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Made the AnalyticsTest a bit more concise and improved the abort checks

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Changed Api provider to reusable

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Aligned DataDonationAnalyticsApiV1 response with Survery format

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed OnboardingAnalyticsFragmentTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Merge with ppa-main exploded this for some reason, fixed now

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added analytics deactivation if no config is present on remote

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fix error message if there is no last analytics data.
We only need to check the values if the object itself is non-null.

* Refactor upload response code check. Use a single `return when` to let the compiler enforce correct case handling.

* Use a specific time object, as Long and just `getDelay` could be confusing.
Android mostly uses long=miliseconds and not hours.

* Only get the config once for the whole analytics submission process.
Prevent unnecessary config checks, and if the config cache expires, don't let the config change in the middle of submisison.

* LINTER ¯\_(ツ)_/¯

Co-authored-by: Matthias Urhahn <matthias.urhahn@sap.com>
# Conflicts:
#	Corona-Warn-App/src/deviceForTesters/java/de/rki/coronawarnapp/test/datadonation/ui/DataDonationTestFragment.kt
#	Corona-Warn-App/src/deviceForTesters/java/de/rki/coronawarnapp/test/datadonation/ui/DataDonationTestFragmentViewModel.kt
#	Corona-Warn-App/src/deviceForTesters/res/layout/fragment_test_datadonation.xml
# Conflicts:
#	Corona-Warn-App/src/main/res/values/strings.xml
…2330

* initial commit of Analytics upload code (lots of missing pieces)
kind of working: Definition of backends and endpoints, donor data collection (no donors implemented here)
probably broken: SafetyNet attestation, actual server communication

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* added checks for analytics submission abortion

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* added analytics submission periodic worker definition

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* lint fixes

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed kotlin support for IntoSet injection
Added AnalyticsModule to DataDonationModule
Added WorkerBinder definition for DataDonationAnalyticsPeriodicWorker
Added DataDonationAnalyticsPeriodicWorker scheduling to CoronaWarnApplication
Added missing mocks to WorkerBinderTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed injectData call in ExposureRiskMetadataDonor
Added more logging to Analytics
Implemented first AnalyticsTests
Fixed auto formatted removal of io.kotest.matchers.collections.shouldContainAll in WorkerBinderTest (curse you Optimize imports)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implemented Test for ExposureRiskMetadataDonor

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added check for isEnabled during Analytics submission
Added DataDonationAnalyticsOneTimeWorker to allow for random distribution of Analytics submission events

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Moved contribution collection to own function to help with test menu features

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added analytics testing features

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* small fixes to the WorkerBinderTest
added missing worker binding to WorkerBinder

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Put Analytics test options into an extra card.

* in my defense, it was quite late

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Moved AnalyticsSettings to own package
Updated the @IntoSet providers in AnalyticsModule to match the actual DonorModules

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added analyticsEnabled preference to AnalyticsSettings

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added a deleteData function to the DonorModule interface which can be used to remove persisted data if a user revokes analytics consent

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added skeleton to persist last analytics submission on tester devices
Currently the tester version also does not store the data

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed borked unit tests

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implemented File based storage of Last submitted analytics file on tester devices

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added calls to enable and disable analytics to the OnboardingAnalyticsViewModel

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed OnboardingAnalyticsFragmentTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Turns out AnalyticsConfigMapping was not implemented yet, fixed that

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added a timestamp to the stored Analytics Submission

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed unit tests in AnalyticsTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Updated AnalyticsConfigMapper to throw an exception if app config is invalid

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implemented most of the comments from @d4rken

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* if -> when magic

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Moved analytics enable/disable to the Analytics class to allow for deletion on disable

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Made the AnalyticsTest a bit more concise and improved the abort checks

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Changed Api provider to reusable

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Aligned DataDonationAnalyticsApiV1 response with Survery format

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Integration of the Analytics settings UI with the Analytics backend code

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added test for the analytics status

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed OnboardingAnalyticsFragmentTest

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Refactored the SettingsPrivacyPreservingAnalyticsFragment ui state handling

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Merge with ppa-main exploded this for some reason, fixed now

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added analytics deactivation if no config is present on remote

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed AnalyticsTest borked by merge

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Fixed AnalyticsTest borked by merge

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Addressed some linting comments

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

Co-authored-by: Matthias Urhahn <matthias.urhahn@sap.com>
Copy link
Contributor

@kolyaopahle kolyaopahle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@harambasicluka harambasicluka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on device (Pixel 3) by using the test menu.

@harambasicluka harambasicluka marked this pull request as ready for review February 15, 2021 15:30
@harambasicluka harambasicluka requested review from a team February 15, 2021 15:30
@sonarcloud
Copy link

sonarcloud bot commented Feb 15, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 11 Code Smells

38.7% 38.7% Coverage
2.3% 2.3% Duplication

@d4rken d4rken merged commit 053d9ca into release/1.13.x Feb 15, 2021
@d4rken d4rken deleted the feature/4321-ppa-main branch February 15, 2021 15:41
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
author merge PR to be merged by author enhancement Improvement of an existing feature maintainers Tag pull requests created by maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants