Divisiveness/consensus metric in automatic report

[phoqui]

Could anyone explain the metric/calculation behind the 'How divisive was the conversation?' section of the automatic report?
I can't seem to find an explanation anywhere.

[metasoarous]

Hi @phoqui. Thanks for asking this question.

The "divisiveness" metric is defined in our methods paper using the term "extremity" (see pages 9-10). Basically, it's a measure of how strongly predictive a comment is of where participants fall in the 2D "opinion space" represented in the visualization. It's more or less the norm (distance from [0,0]) of the PCA vector entries corresponding to the comment in question. Our rationale for looking at this metric specifically is to be able to quickly get a sense of which comments best explain the opinion space and groupings.

Admittedly, this has sometimes been confused with a more raw measure of the total vote variance or overall level of agreement/disagreement associated with the comment. You may sometimes notice a comment will show up on the low-end of the "divisiveness" scale that nevertheless has a higher vote variance (significant number of agree and disagree votes). This happens when those voting patterns don't correlate well with the primary axes of the conversation (as defined by the PCA, as represented in the visualization). While this is to be expected based on the way the metric is defined, it is unfortunate that the name we chose for it in the report doesn't quite convey the nuance we would like. But it's also not clear what name would convey this nuance, so we're considering adding a note of explanation in the report. We're also considering just switching the report to use raw vote variance, since this will be easier to explain (especially for less technical consumers of the report).

Please let us know if this explanation makes sense, or if you have any further questions or feedback.

Thanks again