Skip to content
This repository has been archived by the owner on Jun 20, 2023. It is now read-only.

Migration to google ENF client v2 (window mode) (EXPOSUREAPP-3456) #1677

Merged
merged 68 commits into from
Nov 20, 2020

Conversation

chris-cwa
Copy link
Contributor

No description provided.

BMItr and others added 30 commits November 2, 2020 20:44
…3537, EXPOSUREAPP-3518) (#1546)

* Split and hide the protobuf config behind interfaces with individual mappers responsible for creating the desired formats.

* Merge branch 'release/1.7.x' into feature/3455-more-frequent-riskscore-updates-configs

# Conflicts:
#	Corona-Warn-App/src/deviceForTesters/java/de/rki/coronawarnapp/test/risklevel/ui/TestRiskLevelCalculationFragmentCWAViewModel.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/transaction/RiskLevelTransaction.kt
#	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/transaction/RiskLevelTransactionTest.kt

* Make the AppConfig observable.
Provide the server time offset.
Offer a lastUpdatedAt timestamp.
Add an app config specific test screen.
Clean up test screens a bit and move debug options out of API test options.

* Fix test regression due to refactoring (moved code around).

* Store the server timestamp and offset at retrieval.
Switch to config storage via json to be able to store additional meta data fields (i.e. time).

* KLint and Me have a hate relationship based on both mutual admiration.

* Fix time offset parsing being locale dependent.

* Fix broken unit tests.

* Improve offset accuracy, move before unzipping.

* Fix overly long livedata subscription to results (viewmodel scope vs observer scope)

* Add mapping for the new protobuf configs + tests.

* For cached (retrofit) response, we need to check the cacheResponse and its timestamps
to determine an accurate time offset.

* Exposure a boolean property to tell us when a fallback config is being used.

* Hide the observable flow<ConfigData> behind a method that can automatically triggers refreshes.

* Use a common mapper interface.

* set old risklevelcalculation deprecated

* Created skeleton for new risk calculation and aggregation

* Implementing steps to aggregate results form exposure windows - wip

* Address PR comments and KLints.

* Fix refactoring regression.

* ktlint

* Added ExposureWindowRiskLevelConfig and ExposureWindowRiskLevelConfigMapper for new config api (not yet introduced)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added first Implementation of exposure window based calculateRisk function

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added generics to Range.inRange

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added Ugly Hack to RiskLevelTransaction to allow for compilation during testing

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Linting and injecting RiskLevelCalculation into TestRiskLevelCalculationFragmentCWAViewModel, currently wont build because ExposureWindowRiskLevelConfig has no Provider

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Linting extravaganza

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Lint Wars Episode VI: Return of the trailing Comma

* Improve config unzipping code.

* Add flag to forward exception thrown during HotDataFlow.kt initialization.

* Don't specify a default context via singleton.

* Move download and fallback logic into it's own class just responsible for sourcing the config: "AppConfigSource".
"AppConfigProvider" is now only responsible for making it available.

* Simplify current concepts for making the app config observable until we have a default configuration.

* Improve app config test screen, delete options, better feedback.
Show toast instead of crash on errors.

* Fixed GSON serialization not encoding/decoding the byte array correctly.
Added specific type adapters for instant and duration to get cleaner json.

* Remove type adapters from base gson due to conflict with CalculationTrackerStorage.

* We want to default to forced serialization of instant by our converters, instead of using the default serialization which will differ
between Java8.Instant and JodaTime.Instant, to prevent future headaches there, register explicit converters by default,
and overwrite them if necessary (currently only needed for CalculationTrackerStorage.kt).

* Improve AppConfigServer code readability by moving code into extensions.

* Fix merge conflicts

* Added missing import to WorkerBinderTest

* fixed unit tests

* Removed auto formatting on unrelated files (revert + cherry pick in other commit)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implementing steps to aggregate results form exposure windows

* Renamed ExposureWindowRiskLevelConfig to ExposureWindowRiskCalculationConfig

* adjusted & refactored Windows aggregation

* removed example Values

* satisfy lint

* make Aggregation work with Instant now

* Use long while calculation

* Added normalizedTimePerDayToRiskLevelMappingList to AppConfig

* normalizedTimePerDayToRiskLevelMappingList from AppConfig

* satisfy lint

* Get AppConfig on init and listen for updates

* exposureData to aggregatedRiskPerDateResult

* Corrected name in ConfigParserTest

* use instant for specific aggregation logs

* satisfy CI

* satisfy detekt

* exposure history exception & log adjustment

* Fixed unittests for new config parser and risk levels

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added some logging to the calculateRisk function and removed the suspend qualifiers as AppConfig is fetched during init

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

Co-authored-by: Matthias Urhahn <matthias.urhahn@sap.com>
Co-authored-by: BMItter <Berndus@gmx.de>
Co-authored-by: harambasicluka <64483219+harambasicluka@users.noreply.github.com>
…-enfv2-main-branch

# Conflicts:
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/AppConfigModule.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/risk/RiskLevelTask.kt
…(EXPOSUREAPP-3456) (#1575)

* Split and hide the protobuf config behind interfaces with individual mappers responsible for creating the desired formats.

* Merge branch 'release/1.7.x' into feature/3455-more-frequent-riskscore-updates-configs

# Conflicts:
#	Corona-Warn-App/src/deviceForTesters/java/de/rki/coronawarnapp/test/risklevel/ui/TestRiskLevelCalculationFragmentCWAViewModel.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/transaction/RiskLevelTransaction.kt
#	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/transaction/RiskLevelTransactionTest.kt

* Make the AppConfig observable.
Provide the server time offset.
Offer a lastUpdatedAt timestamp.
Add an app config specific test screen.
Clean up test screens a bit and move debug options out of API test options.

* Fix test regression due to refactoring (moved code around).

* Store the server timestamp and offset at retrieval.
Switch to config storage via json to be able to store additional meta data fields (i.e. time).

* KLint and Me have a hate relationship based on both mutual admiration.

* Fix time offset parsing being locale dependent.

* Fix broken unit tests.

* Improve offset accuracy, move before unzipping.

* Fix overly long livedata subscription to results (viewmodel scope vs observer scope)

* Add mapping for the new protobuf configs + tests.

* For cached (retrofit) response, we need to check the cacheResponse and its timestamps
to determine an accurate time offset.

* Exposure a boolean property to tell us when a fallback config is being used.

* Hide the observable flow<ConfigData> behind a method that can automatically triggers refreshes.

* Use a common mapper interface.

* set old risklevelcalculation deprecated

* Created skeleton for new risk calculation and aggregation

* Initial

* Implementing steps to aggregate results form exposure windows - wip

* Address PR comments and KLints.

* Fix refactoring regression.

* ktlint

* Json parsing

* Added ExposureWindowRiskLevelConfig and ExposureWindowRiskLevelConfigMapper for new config api (not yet introduced)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added first Implementation of exposure window based calculateRisk function

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added generics to Range.inRange

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Added Ugly Hack to RiskLevelTransaction to allow for compilation during testing

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Linting and injecting RiskLevelCalculation into TestRiskLevelCalculationFragmentCWAViewModel, currently wont build because ExposureWindowRiskLevelConfig has no Provider

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Linting extravaganza

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Lint Wars Episode VI: Return of the trailing Comma

* Improve config unzipping code.

* Add flag to forward exception thrown during HotDataFlow.kt initialization.

* Don't specify a default context via singleton.

* Move download and fallback logic into it's own class just responsible for sourcing the config: "AppConfigSource".
"AppConfigProvider" is now only responsible for making it available.

* Check test cases

* Simplify current concepts for making the app config observable until we have a default configuration.

* Implementing steps to aggregate results form exposure windows

* cleaned todo

* Adjusted default values

* Improve app config test screen, delete options, better feedback.
Show toast instead of crash on errors.

* Fixed GSON serialization not encoding/decoding the byte array correctly.
Added specific type adapters for instant and duration to get cleaner json.

* Remove type adapters from base gson due to conflict with CalculationTrackerStorage.

* refactored Windows aggregation

* We want to default to forced serialization of instant by our converters, instead of using the default serialization which will differ
between Java8.Instant and JodaTime.Instant, to prevent future headaches there, register explicit converters by default,
and overwrite them if necessary (currently only needed for CalculationTrackerStorage.kt).

* Improve AppConfigServer code readability by moving code into extensions.

* Fix merge conflicts

* removed example value

* Added missing import to WorkerBinderTest

* fixed unit tests

* Removed auto formatting on unrelated files (revert + cherry pick in other commit)

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* Implementing steps to aggregate results form exposure windows

* Renamed ExposureWindowRiskLevelConfig to ExposureWindowRiskCalculationConfig

* adjusted & refactored Windows aggregation

* removed example Values

* satisfy lint

* make Aggregation work with Instant now

* Use long while calculation

* Added normalizedTimePerDayToRiskLevelMappingList to AppConfig

* normalizedTimePerDayToRiskLevelMappingList from AppConfig

* satisfy lint

* Get AppConfig on init and listen for updates

* exposureData to aggregatedRiskPerDateResult

* Corrected name in ConfigParserTest

* use instant for specific aggregation logs

* satisfy CI

* satisfy detekt

* Mock exposure windows

* Full test process

* Fix gitignore

* Improved logging

* Correct test cases dates handling

* Config fix and logs

* Small clean up

* fixed some naming and conversion issues with json test case parsing

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

* TRL Encodings in config did not match TRL Encodings used in js example

* Removing a return that broke the calculation

* Actual tests and formatting

* Formatting

Co-authored-by: Matthias Urhahn <matthias.urhahn@sap.com>
Co-authored-by: BMItter <Berndus@gmx.de>
Co-authored-by: Kolya Opahle <k.opahle@sap.com>
Co-authored-by: harambasicluka <64483219+harambasicluka@users.noreply.github.com>
* Implemented exposure window provider, adjusted ENF Client

* Implemented new provideDiagnosisKeys to activate ExposureWindow mode, deprecated old diagnosis related content

* Adjusted ENFClientTest

* conflict fix - wip

* test fix
…PP-3540) (#1605)

* Implemented the opportunity to use ExposureDetectionTracker without identifier

* Old provideDiagnosisKeys = NO-OP

* clean

* maxLength clean

* refactoring and extension function

* adjusted wording

* Throw UnsupportedOperationException in ENFClient in case

* ktlint clean

* fix tests for now

* commented out Exception temporarily

* adjusted enfTest -wip, throw unsupportedOperationException
* Initial switch to new app config version (no tests/lint)

* Fixed some unit tests, not all are working yet

* Fixed ConfigParserTest

* ktlint fixes

* ktlint

* ExposureWindows to map is now beautiful

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

Co-authored-by: BMItter <Berndus@gmx.de>
#1634)

* Adjusted tests accordingly - removed deprecated provideDiagnosisKeys

* satisfy detekt
…-enfv2-main-branch

# Conflicts:
#	Corona-Warn-App/src/deviceForTesters/java/de/rki/coronawarnapp/test/risklevel/ui/TestRiskLevelCalculationFragmentCWAViewModel.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/risk/DefaultRiskLevels.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/risk/RiskLevelTask.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/risk/RiskLevels.kt
#	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/risk/RiskLevelsTest.kt
* Added numberOfDaysWithHighRisk and numberOfDaysWithLowRisk

according to new tech spec

* Transformed minimumDistinctEncountersForRisk and mostRecentDateForRisk into extension fun

* typealias for ProtobufRiskLevels

Signed-off-by: Kolya Opahle <k.opahle@sap.com>

Co-authored-by: Kolya Opahle <k.opahle@sap.com>
…appconfig-name

Fixed asset appconfig resolution for v2 config (EXPOSUREAPP-3456)
Signed-off-by: Kolya Opahle <k.opahle@sap.com>
* activate WindowExposure mode

* extirpated tokens!

* detekt, ktlint

* fixed tests

* - v1 methods

* - get v1 exposure summary

* - fixme

* updated exposure summary repo

* Adding mock dependencies that are required to instantiate the worker factories in our binding test.

* Added numberOfDaysWithHighRisk and numberOfDaysWithLowRisk

according to new tech spec

* Transformed minimumDistinctEncountersForRisk and mostRecentDateForRisk into extension fun

* make sure list of windows and aggregated result belong together

* sending values for current UI

* fixed ktlint

* renamed ExposureResultStore

* fixed tests

Co-authored-by: Matthias Urhahn <matthias.urhahn@sap.com>
Co-authored-by: BMItter <berndus@gmx.de>
Signed-off-by: Kolya Opahle <k.opahle@sap.com>

Conflicts:
	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/risk/DefaultRiskLevels.kt
	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/nearby/windows/ExposureWindowsCalculationTest.kt
	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/risk/RiskLevelsTest.kt
…-enfv2-main-branch

# Conflicts:
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/AppConfigModule.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/ExposureDetectionConfig.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/download/DefaultAppConfigSource.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/mapping/ExposureDetectionConfigMapper.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/mapping/KeyDownloadParametersMapper.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/sources/remote/AppConfigApiV1.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/appconfig/sources/remote/AppConfigServer.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/diagnosiskeys/download/DownloadDiagnosisKeysTask.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/storage/TracingRepository.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/ui/main/home/HomeFragmentViewModel.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/ui/tracing/details/RiskDetailsFragmentViewModel.kt
#	Corona-Warn-App/src/main/java/de/rki/coronawarnapp/update/UpdateChecker.kt
#	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/appconfig/download/DefaultAppConfigSourceTest.kt
#	Corona-Warn-App/src/test/java/de/rki/coronawarnapp/appconfig/mapping/ExposureDetectionConfigMapperTest.kt
chris-cwa and others added 3 commits November 20, 2020 11:22
- Added requireAtLeast to ENFVersion
- Added UnsupportedENFVersionException
- Throw UnsupportedENFVersionException in provideDiagnosisKeys
- Adjusted Tests
@sonarcloud
Copy link

sonarcloud bot commented Nov 20, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

66.1% 66.1% Coverage
0.0% 0.0% Duplication

kolyaopahle and others added 4 commits November 20, 2020 13:30
…DayToRiskLevelMappingMissingException

Signed-off-by: Kolya Opahle <k.opahle@sap.com>
* Unarm the quota check for now.

* Make key provider unit tests more specific.

* Add time unit to parameter name.
Copy link
Member

@d4rken d4rken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@BMItr
Copy link
Contributor

BMItr commented Nov 20, 2020

have to check testmenu changes according to last commit changes

@harambasicluka
Copy link
Contributor

harambasicluka commented Nov 20, 2020

Not sure how to handle this edge case: I traveled back in time, imho the active tracing time displayed should be zero. But what happen if a user travels 500 days in future does a risk calculation and goes back to the current date then he would need 500 days to reach the correct state.
EDIT: Ok, user should just reset the app (not uninstall) to get in to a valid state again.
image

@harambasicluka
Copy link
Contributor

I went 4 days back on WRU-XD reset the data from OS level. I went 23h in future -> unknown risk, plus 2 -> low risk (visible in ENF menu in OS), traveled 3h in future -> no update, still on green & so on over multiple days. I'll try to reproduce it and see when it occurred, but currently it says Begegnungen wurden noch nicht überprüft even if there are exposure checks in the ENF settings on OS level.

Will now check the logs.

On both screenshots the current date is 19.11.20.
image
image

@harambasicluka
Copy link
Contributor

I went 4 days back on WRU-XD reset the data from OS level. I went 23h in future -> unknown risk, plus 2 -> low risk (visible in ENF menu in OS), traveled 3h in future -> no update, still on green & so on over multiple days. I'll try to reproduce it and see when it occurred, but currently it says Begegnungen wurden noch nicht überprüft even if there are exposure checks in the ENF settings on OS level.

Should be fixed with the latest update from release/1.7.x. Has to be retested after bot is merged in release/1.8.x

@sonarcloud
Copy link

sonarcloud bot commented Nov 20, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 1 Code Smell

56.4% 56.4% Coverage
0.0% 0.0% Duplication

@d4rken d4rken merged commit 5da95c3 into release/1.8.x Nov 20, 2020
@d4rken d4rken deleted the feature/3456-enfv2-main-branch branch November 20, 2020 14:51
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
maintainers Tag pull requests created by maintainers prio PRs to review first.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants