Calculate and display confidence for risk scoring

[slobentanzer]

Feature description

The risk assessment displayed in the app should be accompanied by a confidence scoring based on the amount of corona-warn-app users in the vicinity of the user. If that is not possible, remove "Low Risk" altogether, because it cannot be confidently stated.

If possible, include a mechanism to discern the likelihood of few or many corona-warn-app-contacts to inform this risk score. Compare the corona-warn-app contacts to the absolute number of contacts had by the device. This could be time- or distance-based, as is the main function.

Problem and motivation

The main screen of the app shows a large green "Low Risk" icon. However, display of "Low Risk" is possible in spite of massive contact with Covid-positive individuals, simply if these individuals do not use the app themselves. Showing a low risk to a user who has been in a crowd, of which only few others have used the app, results in overconfidence regarding the low risk, particularly in statistically naive users, and thus to potentially harmful underestimations of one's own infection risk.

If the logging cannot be adjusted to represent a credibility scoring of the low risk, the display of "Low Risk" should be removed altogether, and the app should unambiguously state that it is only valuable in tracking positive contacts and cannot inform about low risk.

Is this something you're interested in working on

I don't have the required mobile technology skills.

---

Internal Tracking ID: EXPOSUREAPP-2077

[daimpi]

For privacy reasons the app is designed in such a way that it cannot know how many (non-positive) contacts you had with other ppl even if they use CWA: #124 (comment)

Regarding the "low risk/green" status: There is some improvement planned at least for those cases where a a "green" risk-contact happened:

• https://github.com/corona-warn-app/cwa-backlog/issues/23
• Show more details about "risk encounters" #100

One idea which is currently being discussed is to introduce a "yellow" risk status for such cases: https://github.com/corona-warn-app/cwa-app-android/issues/899#issuecomment-663475550

[slobentanzer]

thank you, daniel

i gather from your answer that the functionality required to infer absence of risk contact or a confidence measure cannot be implemented due to the underlying technology. is that correct?

if so, as referred to in the discussions you mentioned, i think it is not clear to many naïve users that the app is only designed to track positive contacts, and not to infer the inverse, absence of infection risk. i therefore think that the display of "low risk", and particularly on a large green panel, is not warranted by the app's actual functionality.

it would be strongly advised to let this be reflected in the design of the user interface, such that the average user understands that the app can only inform about risk contacts, not about the absence of risk contacts. please consider passing this on to the epidemiologic/public health personnel in your team.

[daimpi]

i gather from your answer that the functionality required to infer absence of risk contact or a confidence measure cannot be implemented due to the underlying technology. is that correct?

That is correct. But strictly speaking it's not just a technological limitation: in order to reliably infer absence of risk you would need 100% CWA uptake in the population, 100% compliance with uploading positive keys in case of infection as well as 100% test coverage for all infections.

This is o/c not possible, even if there were no technological barriers at all (which exist for a privacy reasons).

Btw: if you want to get the maximal information possible out of the Exposure Notification Framework (ENF), even bypassing the restriction that you cannot count non-positive RPIs, and you have a rooted device, you might want to check out @mh-'s corona-warn-companion-android 🙂.

[slobentanzer]

i don't really agree on that logic. for computation of a confidence metric you do not need 100% coverage in any of these categories. in fact, if you knew the coverage (as explained in my initial post), you could already infer a lot on the confidence of the risk score. similarly, if you knew the percentage of contacts met that actually do have the app installed, you could say a lot about the confidence of the "low risk" score displayed.

confidence in a statistical sense is always an estimation, a way of saying how much you can trust the result. based on what you have said, the average corona warn app user cannot put any trust in a display of "low risk", and that is why i think the displayed message needs to be changed to something that enables the average user to understand this (eg, "no risk contacts so far", "no risk estimation yet", etc).

the app does only inform about positive contacts in cases where this happens (and the positively tested user also enters his data into the app), and cannot confidently state a measure of "safety" based on the absence of such contacts.

[daimpi]

Yes, in principle you could make a stochastic model which tries to estimate risk by incorporating more data. I don't think this is easy though even if you had information on the number of contacts with the app: what if you have high regional heterogeneity in app uptake? Does the fact that you only encounter few other CWAs now indicate that you're more save? What if you have cross-correlation of app uptake with other behavioral markers like compliance with hygiene rules etc…?

I'm not saying it's impossible, I would just need to see a proper reasoned quantitative analysis on how much more confidence could actually be gained that way to be convinced.

But that's anyway more of a academic debate at this point, as the number of contacts is not available in the first place ^^.

[GisoSchroederSAP]

Let me quickly jump in to explain a few but important decisions:
(a) To simplify the usage of the app, from the very beginning the decision was made: There are two colors - green (low risk), red (significant risk).
(b) Every now and then the proposals for introduction of a third color (yellow or light green) come up, but the initial decision stays, at least for now.
(c) As much as I understand the intention of the abstract "confidence" scoring idea, neither incorporation of the number of persons (with or without app), nor incorporation of the "absence" of risk contacts would end this discussion, sorry. "Confidence" is somewhat psychological and based on thresholds, believe, trust, and experience. You could introduce a bunch of other factors that may have influence to this risk level calculation (think about the aerosol discussion, the epidemiological impact of age or medical preconditions...) How do you know, the "confidence score" itself would not be questioned by the users?
(d) You may have reviewed the documentation already, including the sample paper with the math model risk calculation (here), where the risk calculation is based on - mainly on two factors: distance and time, with some adjustable parameters. Even the current calculation is way too technical and rational for the "average end user" - that's why the decision (a) was made: To translate the calculated risk into colors.
(e) And yes, I am aware that still with theses two colors there are questions: 1 risk contact, still green? We constantly try to discuss the UI and the "wording" - the phrases regarding "low risk" are currently under investigation and will be tuned.

[mf179]

I would agree with the assumptions that it is not possible to determine any reasonable confidence (-score) for the "green overall low risk" statement for the shared reasons.

It is even worse. There isn´t any proof that the recorded close+long encounters were not with any infected person. (person doesn´t know about infection, not yet tested, decided not to share positive result with app, ...) This is all shown as "green low risk".

From that the alternative proposed by slobentanzer should be adopted: remove the green/low risk statement. Make it simply a "green status" and explain that status accordingly.

[slobentanzer]

let me put it bluntly: it is clear that the app cannot inform about low risk, only about high risk. thus, the phrase "low risk" and all related content should be removed, otherwise you are displaying nonfactual information.

the decision to make it "green and red" was wrong in the first place, and probably made by people who don't know about the technical implications of the actual tracking. if any, it should be "grey and red," if you need a colour coding, to visualise "not enough information vs actual risk." maybe you can consider this stance in your constant discussion of the UI.

[Ein-Tim]

I just wanted to add my opinion to this:

It is (imho) ok that the App shows "Low-Risk" because it does say "LOW-Risk" and not "No-Risk", so the App says there is a Risk but it is Low.
That the App only knows what it knows should be made clearer but in these new Texts, shared by @abro1i the App says:

[...]Ihr Infektionsrisiko wird unter Berücksichtigung aller in der Corona-Warn-App vorliegenden Daten dennoch als niedrig eingestuft.[...]

[GisoSchroederSAP]

Okay, I Think I understand all your ideas so far. Just let my please clarify:
From the development perspective we are open to your ideas, proposals and the expected concerns and impact. Those thought we definitely consolidate and discuss with the client. Still, the clear decision so far is:

• There will be a color code green (calculated low risk)/red (calculated potential risk).
• There will be a kind of wording explaining what the user shall understand with the status text.
  This wording is constantly under review.

[mf179]

Thanks for taking it up, GisoSchroederSAP.

Slobentanzer is right, outside the "red status" the app has NO CLUE about any risk. And so that shall be removed.

I found it quite annoying that it required me to exchange a few mails with the RKI-hotline to get it confirmed that "Risiko-Begegnung mit niedrigem Risiko" is no risk at all. RKI even referred me to the Tagesschau-article, which exists as the app fails to provide a decent description. So Ein-Tim, the FAQ is correct. That encounter is no-risk (harmless/unbedenklich), and this is not not compliant with a "low risk statement". For just traced close/long encounters without any related diagnosis key (yet) it is not known whether a it´s a risk or not until a diagnosis key is shared for it. So nothing is known. And so there is also nothing known about any overall risk.

I understand slobentanzers argument. A gray screen is however a bit tricky as it is also used for telling the user that he didn´t behaved well and has no/not enough tracing yet. Formally slobentanzer is right, in both current cases (green&gray) the app has no clue about the user´s real overall risk. Only the red status/risk/screen can be determined as an increased/higher risk (with confidence) by having one or more real-risk-encounters with persons that shared a positive test result.

This issue I brought up alraedy under "backlog #23" where somebody described "App says: "1 Risiko-Begegnung" but the screen says "Niedriges Risiko". I didnt find information, what this exactly means".
As there is more under backlog # 23 it will in best case only result in correcting "Risiko-Begegnung" to "Begegnung".
And I hope the issue of the "overall low risk statement" can be solved here under # 110.

I repeat here what I proposed under corona-warn-app/cwa-documentation#416 (this 416 is a spin-off from backlog# 23). The proposal is:
On the green main screen "n Risikobegegnungen mit niedrigem Risiko" should be changed to "n unbedenkliche Begegnungen" in line with the FAQ.

Further on the green main screen there shouldn´t be any risk statement. So "Niedriges Risiko" should be changed to "Status grün". (or whatever color is preferred)

The klick on the related main screen link should explain the "Status grün" as:
"Ihr Status ist grün da bisher alle aufgezeichneten Begegnungen als unbedenklich erachtet werden.
Etwaig angezeigte "unbedenkliche Begegnungen" sind Begegnungen mit einer positiv getesteten Person. Diese Begegnungen waren jedoch nur kurz oder mit größerem Abstand und sind daher unbedenklich.
Es kann jedoch nicht ausgeschlossen werden, daß ein Infektionsrisiko bestand, da z.B. nicht jeder die CWA benutzt.
Daher sollten sie in jedem Falle die Hygieneregeln befolgen. Auflistung derer."

From the discussion/exchanges so far I got the impression that the RKI sets the criteria for what is considered a "risk" and not neccessarily the developers. From that it might be most fruitful you discuss it with them.

[Ein-Tim]

@mf179

I found it quite annoying that it required me to exchange a few mails with the RKI-hotline to get it confirmed that "Risiko-Begegnung mit niedrigem Risiko" is no risk at all.

I don't really think thats true. The Risk is low, how it says, but you can't say that there is no Risk of Infections, right?
I'm very confused now, @tkowark, @mtb77, @svengabr could you pleas make this clearer?

[mf179]

Ok Ein-Tim, for the very specific "Risikobegegnung mit niedrigem Risiko" the risk is known as harmless. It is a bit confusing as the "overall risk" has no term. I meant no clue about the overall risk.

One solution is to show no "harmless encounters". As stated somewhere else, I think it is good for demonstrating that the app works (finds [harmless] contacts with positiv tested) and perhaps raises awareness. But it doesn´t bring anything for the apps main or even sole function: "breaking the infection chain".

I don´t get what you mean with the reference to the "verworfen issue". To me the wording is just ambigous. It is meant as "the low/no-risk encounters are shown to the user (just for info) and are considered as harmless (not discarded) during the overall risk assessment.

[daimpi]

I personally think the green background with the "low risk" statement is fine in case of no encounters.

@mf179

One solution is to show no "harmless encounters"

I disagree with this as I've explained here.

On the green main screen "n Risikobegegnungen mit niedrigem Risiko" should be changed to "n unbedenkliche Begegnungen" in line with the FAQ.

Just changing the term "exposure" to "encounter" like shown in this preview is sufficient imo.

@Ein-Tim regarding your questions whether green encounters actually carry "no risk": in the real world they o/c carry some risk as we have e.g. aerosolized spread and problems with the distance-attenuation mapping of BLE signals (cf. here and here).

But we could construct an ordering of risk levels vis-a-vis the different possible displays in CWA:

1. Green (low risk) with no encounters
2. Green (low risk) with some encounters
3. Red (increased risk) with some encounters

In this ordering 1. represents the lowest, 2. a higher, and 3. the highest risk level ceteris paribus.

[mf179]

What is an encounter, daimpi? With a positive tested person?
Your 1. Green would then rather mean: no (yet) known encounter with a positive person. Or ? If you don´t get a diagnosis key for a close+long encounter you cannot know whether the encounter is a risk or not. So for 1. you cannot describe any overall risk.
2 includes the same yet unclassified encounters besides those identified as "low/no risk encounter".

In my understanding the target is to avoid false alarms, i.e. a clear distinction between red (be concerned, take actions) and green (be not concerned, don´t overreact). And the differentiation is determined by some parameter setting. Stating about a "low risk" is counterintuitive and counterproductive. It confuses/scares the user. And the "green/low risk" can still not exclude that there was a high risk in reality, which is the main deficiency here.

Seems we are moving in circles.

[daimpi]

What is an encounter, daimpi? With a positive tested person?

Yes. The German term is "Begegnung" as you can see in the preview linked above.

Your 1. Green would then rather mean: no (yet) known encounter with a positive person. Or ? If you don´t get a diagnosis key for a close+long encounter you cannot know whether the encounter is a risk or not. So for 1. you cannot describe any overall risk.

Nope. I was intentionally speaking about an ordering of risk levels which is about relative risk and stating that those comparisons are ceteris paribus. So I think my original statement is correct.

In my understanding the target is to avoid false alarms, i.e. a clear distinction between red (be concerned, take actions) and green (be not concerned, don´t overreact). And the differentiation is determined by some parameter setting.

As I've stated above: there are good reasons (e.g. ~50% recall in Fraunhofer IIS testing) to doubt that making such a confident and sharp binary distinction is reasonable.
Also: such a determination profits immensely from additional information that only the user possesses (e.g. it makes a massive difference whether I was mainly outside, or stuck in a crammed room the day of the encounter), which is why I'm pushing to give more info to the user also in case of green encounters.

Stating about a "low risk" is counterintuitive and counterproductive. It confuses/scares the user. And the "green/low risk" can still not exclude that there was a high risk in reality, which is the main deficiency here.

Why do you think this is the case? For me the green "low risk" status is not per se confusing. Do you think some better communication on how the app works would alleviate your concerns?

[mf179]

Let´s take an example.

Persons A and B watch a movie every day, sitting next/close to each other. Both use the CWA. A gets Covid19, tested postive, but doesn´t share it with the app. What entitels the CWA of B to claim: "you have [just] a low [overall] risk"?

This is a warn app, as the name says. It knows just two states, warning and no-warning. And as such it should be communicated.
color/status gray - not fully operational yet
color/status red - based on the available information it is decided to issue a warning
color/status white - the available information doesn´t cause any warning, but an infection risk cannot be excluded
(white seems to represent the situation best - not enough info [for a warning])

If all or some "low/no risk encounters" are wanted to result in a warning/red, the decision threshold would be modified accordingly.

[Ein-Tim]

@mf179

Now I start to really understand your point. 👍
I think it is not necessary to introduce new colors/etc. (And tbh I don't think the RKI will do that)
Just an disclaimer what makes 100% clear that the App just knows what it knows would be good, but I think most of the User already known this...

[mf179]

Green should be changed to white as the green/low-risk lulls the user into a false sense of security. The user needs to be made aware that he should not ignore any symptoms just as the app declares everything is fine "green / low-risk".

And the "green low-risk-encounters" should also not be denoted as risky. Those fall under "white".

The app is made for automating the tracing of infections / breaking the chain and thereby offloading the health authorities, who do that manually. I called it counter-productive as current ambiguous risk statements create load for the health authorities.

I don´t think that an average user can know what the various "green risk statements" mean. I would accept that the people here know it, but those are not average users. Thinking to have some clue about what the app does, I also contacted authorities to know what I shall do with the "low-risk-encounter".

[mtb77]

@mf179 ok - so as @Ein-Tim mentioned, I guess now we understood your point, so its mostly about the green color code, you'll never know if you might have a risk, so the app should not weighting you in false security - right?

Sascha

Corona-Warn-App Open Source Team

[daimpi]

@mf179 As I stated above: I don't think that the "low risk status" and the green color code "lulls the user into a false sense of security" if there have been no encounters. And the new description of green encounters also explains them nicely, as it clearly states no further actions need to be taken by the user.

I very much disagree with the idea of hiding encounters from the user, unless there is an "expert mode" where this data can be accessed if required.

Something I can get behind is general improved communication on how the app works, so that there is no confusion for average users what the app can and cannot do 🙂.

[mf179]

Yes, Sascha, this about the color green.
But it is also the text that comes for explanation, when clicking on the link for the "green low risk".
It explains: "sie haben ein niedriges Infektionsrisiko, da ..."

So green color and "low [overall] risk" as well as the explanation of what that "green risk" is need to be reconsidered.
I suggest change green to white and explain the "white status" as the app hasn´t determined any warning based on its available data. An infection risk can however not be excluded and so the user should in any case follow the usual (AHA) guidance and not ignore any symptoms.

Daimpi, I have nothing againts showing "no risk encounters", but for info only. It is well understood that there is no zero risk with such encounters. But I think the app wants to make a black-and-white-decision. At the moment there seems no major problem with the capacity of the health system. However the app (environment) seems dimensioned for coping with a few thousand infections per day (https://www.golem.de/news/corona-warn-app-telekom-rechnete-mit-10-000-neuinfizierten-pro-tag-2008-150391.html). We will hopefully/probably not get this. But the app may still need to alert only the users that get the "red status". And all others should not get any hint that might encourage them to bother the health system. Besides of course, that they shall take symptoms serious.

[svengabr]

There are currently many discussions regarding green risk status.

See also:
corona-warn-app/cwa-website#307
https://github.com/corona-warn-app/cwa-backlog/issues/23

It was discussed with the RKI to add a 3rd color code beside green and red but it was declined by the RKI. The upcoming version of the app will have a slightly overhaul of the risk status screen.

You can see a preview of the screens how they will most likely look like afterwards:
https://github.com/corona-warn-app/cwa-backlog/issues/23#issuecomment-682485410

The latest information is that there is no plan to add an additional color code like yellow or gray.

Best regards,
SG

Corona-Warn-App Open Source Team

[daimpi]

@mf179

Daimpi, I have nothing againts showing "no risk encounters", but for info only. It is well understood that there is no zero risk with such encounters.

I don't fully understand: are you saying that you're fine with hiding encounters from the user? I've mentioned the reasons why this would be a bad idea multiple times above and fortunately CWA is also currently not hiding any encounters from the user.

But I think the app wants to make a black-and-white-decision.

I also don't think that's a great idea for the reasons mentioned above. But in principle I don't think this is as problematic as completely hiding encounters would be. And if the app would show some more real info (e.g. day of encounter) on green encounters I'd be quite happy even if there is no 3rd color.

But the app may still need to alert only the users that get the "red status". And all others should not get any hint that might encourage them to bother the health system.

I see your point wrt not overloading the healthcare system. But I very much disagree with the idea that "hiding encounters" would be a solution. Exactly the opposite: when it comes to triaging resources, more information is important as e.g. the doctors which have to decide whether a test should be performed or not, profit from more info on the potential encounters. When I know my patient had a risk encounter 13 days ago but still doesn't show any symptoms, that might be quite different to a situation where I know the encounter just happened 2 days ago. Not hiding encounters has the added advantage, that we're not hiding the ~50% false negative encounters the app produces (and that's a conservative number for the reasons mentioned above).

Besides of course, that they shall take symptoms serious.

Sure, but if you hide encounters, you prevent the user/ care-provider from making a better informed determination on how to evaluate potential symptoms, especially b/c Covid-19 has symptoms that overlap with other diseases which will become more prevalent as we move into autumn.

[slobentanzer]

sorry to butt in here, but at this point the discussion does not really represent the original issue. in fact, i was advocating for clearly using it only to trace positive contacts (because it can do that, potentially), and make no assumptions about low risk (because it CAN'T).

however, as @svengabr and @GisoSchroederSAP have said above, the decision to show the low risk status, and to do it in green, was made by the RKI independent of whether it makes sense or not. it is also pretty clear to me at this point that there is no interest in amending this, because apparently "the users understand this." which i don't think, but i don't call the shots, obviously.

recent evaluations seem to indicate i may be right, because the agreement with the app in the population sinks quite rapidly, and now more than half of users seem to think we will not benefit from its use. i'm inclined to agree to that, and seeing all the resistance here, i don't really have any hope that this will be resolved. so thanks for your efforts, and i'll retreat from this thread. at least SAP made some money.

[mf179]

I agree with slobentanzer. Further, I assume this is a commercial product and it seems to lack any professional quality management. At least obviously for what is not just software.

The issue discussed here is reported for almost two months. The "Risikobegegnung mit niedrigem Risiko" for more than two months meanwhile. The proposed wording presented as "solution" here and under # 23 is still wrong. It contradicts the correct FAQ.

Last repetition to explain again, also for svengabr: I was rather forced to move to this issue here from # 23 and_ to another issue that was closed with bending the rules as backlog#23 is not seriously caring about the issues discussed here.
This here with its thumbs-up represents the attitude of the discussion on # 23: "
Coronawahn commented 10 days ago: Mir ist diese Wortklauberei ehrlich gesagt ziemlich egal, denn wenn ich eine Info bekomme wie z.B. eine Risiko-Begegnung am XX.XX.XXXX (um XX:XX Uhr), mit X Meter Abstand und einer Länge von X Minuten
brauche ich keine weitere Erklärung."

Discussion under issue # 23 focuses on enhancements that none of the majority of the users needs. If there is a serious interest by the "CWA Team" in fixing the issues, you should make someone responsible for the "risk statments" outside the red status.

It took a week until someone even understood what the issue with the green risk statements is. This is elementary logic, which makes me wonder whether there is a lack of competency regarding this. Now it is restarted. Nobody wants a third color. Current green is wrong and should become white. Current (green) "low risk" statment is a false statement under various conditions.

It turns out to be just a waste of time spending effort here on fixing the CWA.
Now that Google and Apple announced that they integrate also the rest of the warning functionality. it remains to hope that they better understand what they do. And you guys here perhaps start to consider the end-of-life of the product.

[Ein-Tim]

@slobentanzer & @mf179

First of all, everything I write here is not against you personally, I only want to describe my view on this topic. ❤️

I understand that it is frustating for you that your point, which is a good one, is not accepted by the RKI, but please don't just give up but consider how this problem could also be solved!

@slobentanzer

[...]and make no assumptions about low risk (because it CAN'T).

You are right, the App can't really make the statement "Low Risk" because it could be that a person don't upload his Key if he/she is positive or even he/she doesn't use the CWA.
But I still think it is not necessary to indtroduce new colours, etc. An understandable text should make clear that you could have been in (close) conatct with somebody having COVID-19. I, again, beg you to open an Issue dedicated to only this and make a proposal for this Text.

recent evaluations seem to indicate i may be right, because the agreement with the app in the population sinks quite rapidly, and now more than half of users seem to think we will not benefit from its use[...]

This is not completely right, here you can find a German article from the "SZ" with detailed information and a link to a study about Public opinion with regard to the CWA.

and seeing all the resistance here, i don't really have any hope that this will be resolved. so thanks for your efforts, and i'll retreat from this thread. at least SAP made some money.

I think I can speak for the most of us that we do not reject your proposals from the very beginning, we just all wanted to understand your point. And if everbody understood it and still don't support your idea, than they just think it is not necessary to do that... Thats the good side from open source, you always have min. 2 opinions.

---

@mf179

The issue discussed here is reported for almost two months. The "Risikobegegnung mit niedrigem Risiko" for more than two months meanwhile. The proposed wording presented as "solution" here and under # 23 is still wrong. It contradicts the correct FAQ.

Could you please explain what is contradictory (if you've done so before just link your comment) thank you!

Last repetition to explain again, also for svengabr: I was rather forced to move to this issue here from # 23 and_ to another issue that was closed with bending the rules as backlog#23 is not seriously caring about the issues discussed here.

That's why I've pleased you from the very beginning to just open an Issue dedicated to 1 point and not do all in one (in other Issues?!?).

Now it is restarted. Nobody wants a third color. Current green is wrong and should become white. Current (green) "low risk" statment is a false statement under various conditions.

Right, the RKI said it doesn't want to change the colour, so I think you are blaming the wrong people here...

Now that Google and Apple announced that they integrate also the rest of the warning functionality. it remains to hope that they better understand what they do. And you guys here perhaps start to consider the end-of-life of the product.

Please see this statement from the Telekom, there you can see that the App will still live on, even with Exposure Notification Express...

[GisoSchroederSAP]

Thanks, @Ein-Tim , for the last comment here.
I see, beside some questionable statements, a fruitful discussion here about different opinions in various dimensions. You are discussing and/or questioning

• color coding
• verbal explanations in the CWA UI
• risk calculation (or evaluation)
• logical contradictions
• the difference between "exposure" and "encounter" and the role of ENF versus the role of the App
• and even the professional expertise of those that made the decisions (and/or those that implemented the app?!)

Even when I already stepped in twice to provide a statement, it seems I did not fully reach you.
So, let me try to step back and ask a couple of questions:

1. Did you already reviewed the mentioned paper "Epidemiological Motivation of the Transmission Risk Level" to accept the difference between transmission risk level & value and total risk score?
2. Do you believe there is a global acceptance for any color coding, i.e. do you think, this long discussion would immediately be finished by introducing a third white/grey/yellow color? (Same question would come up for the verbal explanation in the app)
3. What do you think - do we all share the same idea of the purpose of the CWA app?
4. Do you at least accept, there might be a good reason to design and implement the app the way it is?

I am just asking, because it looks to me, there are different intentions and different interpretations all over.
From the design perspective, this follows also the RKI intention, @mf179 made a nice note:

red (be concerned, take actions) and green (be not concerned, don´t overreact)

Taking "Guidance for the user based on the knowledge of the local app" as the purpose of the addressed UI element, we constantly review and adjust those verbal explanations with our professional quality management team, with the RKI experts, with community users and even with the BMG (ministry of health).

Looking back to the keywords of this ticket's contributions I totally agree with the summary 4days ago: Yes, we are moving in circles. Therefore, I'd like to thank you for your thoughts and concerns so far. We always will share your valuable input in the communication between the parties, including RKI. Based on this dialogue, please accept:

• Decision is made for the binary color code.
• Phrases, wording and explanations will be further improved to transport the "intended message".

Thanx.

[slobentanzer]

listen, guys,
to me this is also not at all personal. and i am kind of sorry that you have to deal with this, as you are clearly not the ones responsible for the epidemiological side of things. i am having a hard time explaining what this thread should have been about.

it has always been about confidence, in the particular case of absence of contacts. nothing else. i have no particular quarrel with the binary coding, please don't confuse my statements with those from @mf179. however, it seems you do not realise, or are strongly opposed to acknowledging, the fact that a user of the app will invariably base their decisions on the "risk" you present to them in the app. That's what "Guidance for the user based on the knowledge of the local app" means.

if a user has lots of positive contacts, it is fair to assume that the display of "high risk" has merit in this case. but the opposite is not the case. how is that so difficult to acknowledge? if someone lives in a high-risk environment, such as working in a hospital or nursing home (non-medical staff or anyone not familiar with statistics and epidemiology), or in a small village with a large outbreak, it is never warranted to show them "low risk". in these situations, they are always at high risk, if they do not isolate. i think even the RKI will agree with me on that.

and here is the problem: a potentially asymptomatic or pre-symptomatic infected person may use the judgement in the app (again, "Guidance for the user based on the knowledge of the local app") to decide whether to go to a social gathering, concert, or the likes. where they have the potential to become another super-spreader. to return to my initial argument: there is no confidence (because, as you said, it cannot be implemented) for a display of "low risk."

@GisoSchroederSAP, your advice of reading sebastian wolf's paper also tells me that you do not understand my point. "Epidemiological Motivation of the Transmission Risk Level" only concerns itself with the "positive case," and develops a stochastic model only for estimation of risk based on a positive contact. there isn't even mention of the opposite case. if the app is designed solely for contact tracing, it has no right of telling anyone they are at "low risk." i have been trying to make this very point from the start, and i am simply getting frustrated by now. not that it has any bearing on you, because what can i do?

[GisoSchroederSAP]

@slobentanzer, I see you point quite well and I acknowledge your conclusions. However, they are based on a different model - I hope you accept this point as well: While you state:

If a user has lots of positive contacts, it is fair to assume that the display of "high risk" has merit in this case. but the opposite is not the case. how is that so difficult to acknowledge?

...I simply disagree with the pure logic. If the app traces multiple contacts for 1min or less with a distance of 5 or more meters (which may happen for cleaning personell in hospitals), the underlying model during risk evaluation will probably rate those contacts as low risk from the transmission risk level perspective.

That's why I referred to the paper: You might be right about a general statement for a "high risk environment" and may warn users according to "potentially high risk clusters". But cluster recognition and evaluation is not the intended design of the app (yet) and cluster information is not part of the transmission risk calculation. In fact, the purpose of the app was clearly mentioned as motivation for the used approach based on transmission risk:

The better we can assess the probability of a transmission from A to B, the more accurate is the combined risk score that is used to warn the user to take further action.

From my perspective these different viewpoints (btw, already commented by @Ein-Tim ) make the statement weak:

if the app is designed solely for contact tracing, it has no right of telling anyone they are at "low risk."

Here we have the point, also mentioned multiple times: The message of the app regarding risk neither told the user whether we recommend to take action or not, nor did it explain why the risk status is green or red. Therefore, the wording was improved and verified by all parties with the clear message: Based on the math model and based on the facts known about the traced contacts, it is not likely[=green] or likely [=red] the user got infected via transmission from A to B.
I am quite convinced, this is a valid message to the user. It's all about " probability of transmission between the two persons being in close contact ". If the probability exceeds an (app-defined) level, the green color changes to red, and the description tells the user a high risk of being infected via transmission from a prior traced contact - regardless of the "high or low risk environment" setting. The attached screenshot hopefully supports the intention and the scope for the risk calculation, as described above. No cluster or high risk environment information is incorporated yet (you may want to address this topic to the wishlist).

[svengabr]

We got an official statement from the Robert Koch Institute:

German

Als Herausgeber der Corona-Warn-App bedankt sich das Robert Koch-Institut für die vielen Rückmeldungen zur Risikostatusanzeige auf dem HomeScreen. Diesem Dank schließen wir uns als Entwicklerteam der App an. Nach Abwägung aller Argumente und in Abstimmung mit dem Bundesministerium für Gesundheit hat das RKI entschieden, die zweistufige Risikodarstellung (grün und rot) beizubehalten.

Die App muss möglichst einfach und allgemeinverständlich in der Anwendung sein. Durch die Unterscheidung von niedrigem und erhöhtem Risiko können den Nutzerinnen und Nutzern auf ihren Risikostatus abgestimmte Handlungsempfehlungen gegeben werden. Die zusätzliche Anzeige von Begegnungen, die bestimmte Kriterien erfüllen, erhöht die Transparenz. Für alle diejenigen, die sich mit der Risikobewertung und ihren Parametern eingehender beschäftigen möchten, haben RKI und Entwicklerteam detaillierte Beschreibungen auf GitHub zur Verfügung gestellt.

Die Texte in der Corona-Warn-App werden wir unter Berücksichtigung Ihres Feedbacks weiter anpassen. Mit einem anstehenden Update werden wir weitere Erläuterungen zum besseren Verständnis der angezeigten Risikobegegnungen geben. Zudem wird geprüft, ob mit einem späteren Update eine Risikobegegnungshistorie in der App dargestellt werden kann.

Auch zukünftig werden wir Anregungen und Vorschläge zur Verbesserung der Corona-Warn-App aufgreifen, diskutieren und, soweit sinnvoll und möglich, umsetzen. Wir freuen uns, wenn Sie sich weiter mit konstruktivem Feedback beteiligen.

---

English

As the publisher of the Corona-Warn-App, the Robert Koch-Institute as well as the development team would like to thank you for the many feedbacks on the risk status display on the HomeScreen. After weighing up all arguments and in coordination with the Federal Ministry of Health, the RKI has decided to keep the two-level risk display (green and red).

The app must be as simple and generally understandable as possible in its application. By differentiating between low and increased risk, users can be given recommendations for action based on their risk status. The additional display of encounters that meet certain criteria increases transparency. For those who want to take a closer look at risk assessment and its parameters, RKI and the development team have provided detailed descriptions on GitHub.

We will further improve the texts in the Corona-Warn-App and we will continue to listen for feedback of the community. With an upcoming update, we will provide further explanations to help you better understand the risk encounters displayed. In addition, we will check whether a later update will allow a risk encounter history to be displayed in the app.

In the future, we will continue to take up, discuss and, where reasonable and possible, implement suggestions and proposals for improving the Corona Warning App. We would be pleased if you continue to contribute with constructive feedback.

Further text improvements are being introduced in the upcoming hotfix release 1.3.1.

RC1 of 1.3.1 is already available:
https://github.com/corona-warn-app/cwa-app-android/releases/tag/1.3.1-SNAPSHOT-RC1
https://github.com/corona-warn-app/cwa-app-ios/releases/tag/v1.3.1-RC1

Since the decision from the RKI is final, I will close this issue now.

Best regards,
SG

Corona-Warn-App Open Source Team