Bias en non-discriminatie t.o.v. de levenscyclus

[OliwiaDetmers]

Vragen voor deze discussie:

• Welke vormen van bias kunnen er ontstaan in iedere fase van de levenscyclus ?
• Wat kan je doen om dat zo veel mogelijk te voorkomen ?
• Welke andere verantwoordelijkheden hebben overheidsorganisaties per fase van de levenscyclus ?

De volgende 3 referenties bevatten al best een breed overzicht van verschillende soorten biases met een indicatie in welke fases van de levenscyclus ze van belang zijn. Ik heb nog geen vertaling naar Nederlands gedaan.

Ref. 1: NIST, Towards a Standard for identifying and managing bias in artificial intelligence (https://www.nist.gov/publications/towards-standard-identifying-and-managing-bias-artificial-intelligence)
Ref. 2: An Introduction to Algorithmic Fairness, H. Weerts 2021 (https://arxiv.org/pdf/2105.05595v1.pdf)
Ref. 3: The Fairness Handbook (https://openresearch.amsterdam/nl/page/87589/the-fairness-handbook)

bias definitie: a systematic and disproportionate tendency towards something (Ref. 2)
Soorten Bias (Ref. 1)
Human biases reflect systematic errors in human thought based on a limited number of heuristic principles and predicting values to simpler judgmental operations. […]
Systemic biases result from procedures and practices of particular institutions that operate in ways which result in certain social groups being advantaged or favored and others be- ing disadvantaged or devalued. […]
Statistical and computational biases stem from errors that result when the sample is not representative of the population. These biases arise from systematic as opposed to random error and can occur in the absence of prejudice, partiality, or discriminatory intent. […]

Hierbij ook nog belangrijke nuances:

• Ref 1.: Likely due to expectations based on techno-solutionism and a lack of mature AI process governance, organizations often default to overly technical solutions for AI bias issues. Yet, these mathematical and computational approaches do not adequately capture the societal impact of AI systems.[…] Using a socio-technical approach to AI bias makes it possible to evaluate dynamic systems of bias and understand how they impact each other and under what conditions these biases are attenuated or amplified. Adopting a socio- technical perspective can enable a broader understanding of AI impacts and the key decisions that happen throughout, and beyond, the AI lifecycle–such as whether technology is even a solution to a given task or problem.
• Ref 2.[…] most issues arise at the intersection of social bias and statistical bias. […] biases are hard to precisely dissect and often overlap. In practice, it is usually difficult if not impossible to know exactly which biases are at play. Luckily, a thoughtful development process that leaves room for self-reflection goes a long way in mitigating harms.

Hieronder heb ik een aantal voorbeelden van bias benoemd per fase van de levenscyclus. Deze lijst is zeker niet uitputtend, gelet ook op de benoemde nuances. Hierbij de uitnodiging om het verder aan te vullen en aan te scherpen.

[OliwiaDetmers]

Probleemanalyse

• Ref 1: These early decisions and who makes them can reflect systemic biases within organizational settings, individual and group heuristics, and limited points of view. Systemic biases are also reflected in the datasets selected within pre-design. All of these biases can affect later stages and decisions in complex ways, and lead to biased outcomes.
• Ref 2: Translating a real-world problem into a machine learning task can be difficult. By definition, a model is a simplification of reality. […] To this end, some amount of abstraction is required. This involves removing details to focus attention on general patterns. […] By abstracting away the context surrounding a model and its inputs and outputs, you may accidentally abstract away some of the consequences as well.

Voorbeelden van biases en traps:

societal bias A prejudice for or against a person or a group (Ref. 2) Societal or social biases are often stereotypes. Common examples of societal or social biases are based on concepts like race, ethnicity, gender, sexual orientation, socioeconomic status, education, and more. (Ref. 1)
institutional bias In contrast to biases exhibited at the level of individual persons, institutional bias refers to a tendency exhibited at the level of entire institutions, where practices or norms result in the favoring or disadvantaging of certain social groups. Common examples include institutional racism and institutional sexism (Ref. 1)
optimism bias A cognitive bias that causes people to overestimate the likelihood of positive events and underestimate the likelihood of negative events. Closely related to solutionism trap (Ref. 2)

Traps (Ref. 2, Selbst et al., 2019):
abstraction trap A failure to consider how social context is interlaced with technology due to abstracting away context surrounding a machine learning model and its inputs and outputs.

• formalism trap A failure to take into account sociotechnical context when formalizing the problem. Closely related to construct validity bias.
  portability trap Taking an existing solution and applying it in a different situation without taking into account the differences between the two contexts.
• ripple effect trap A failure to consider the ways in which the introduction of a machine learning model may affect behavior of other actors in the system and change the social context.
• solutionism trap A failure to consider that machine learning might not be the best solution to a problems.

[OliwiaDetmers]

Ontwerp

In many cases, fairness-related harms can be traced back to the way a real-world problem is translated into a machine learning task. Which target variable do we intend to predict? What features will be included? What (fairness) constraints do we consider? Many of these decisions boil down to what social scientists refer to as measurement: the way we measure (abstract) phenomena. (Fairlearn, https://fairlearn.org/v0.7.0/user_guide/fairness_in_machine_learning.html#fairness-of-ai-systems:)

Voorbeelden van biases:

construct validity bias A statistical bias that occurs when a feature or target variable does not accurately measure the construct it was designed to measure. (Ref. 2)

Zie ook biases bij probleemanalyse

[OliwiaDetmers]

Data verkenning en data preparatie

Ref. 3: This phase tends to produce most bias due to the prejudices and
harmful historical patterns embedded in the data.

Voorbeelden van biases:

historical bias A social bias that is encoded in the data through biased human decision-making or structural biases embedded in society. (Ref. 2)
representation bias A bias that occurs when some groups, defined by sensitive characteristics, are underrepresented in the data (Ref. 2)
selection bias A statistical bias that occurs when data collection or selection procedures results in a non-random sample of the population. Closely related to representation bias. (Ref. 2)
measurement bias A statistical bias that occurs when data collection practices result in systematic errors in data. Can cause construct validity bias. (Ref. 2)
confirmation bias A tendency to put attention towards information that confirms one’s own beliefs. (Ref. 2)

[OliwiaDetmers]

Ontwikkelen

Ref 3: In the modelling phase, the model is built and trained. Here, fairness issues can arise when an unfit model is selected or when the modelling choices result in the prioritization of an objective that leads to more errors for underrepresented groups.

Voorbeelden van biases:

aggregation bias A bias that occurs when a single machine learning model is used for groups that have distinct data distributions, resulting in inaccurate predictions for (some) groups. (Ref. 2)
learning bias occurs when the model prioritizes some objective, e.g., accuracy, that damages a fairness-related outcome. (Ref. 3)
omitted variable bias A statistical bias that occurs when a model leaves out one or more relevant variables. (Ref. 2)

[OliwiaDetmers]

Validatie

Ref 3: During the evaluation stage of the model development cycle, the performance of the model on the test set is evaluated.
Assumptions behind Evaluations:
• Decisions can be evaluated as an aggregation of separately evaluated individual decisions. This includes assuming that
outcomes are not affected by the decisions for others, an assumption known as no interference.
• All individuals can be considered symmetrically, i.e., identically. This assumes, for example, that the harm of denying a loan to someone who could repay is equal across all people.
• Decisions are evaluated simultaneously. This means that they are evaluated in a batch as opposed to serially, and therefore they do not consider potentially important temporal dynamics.

Voorbeelden van biases:

evaluation bias A bias that occurs through the use of performance metrics and procedures that are not appropriate for the context in which the model will be used. (Ref. 2)

[OliwiaDetmers]

Implementatie

Ref 3: the model is being deployed in a real-world setting, where its predictions are part of a system that affects individuals and groups of people. Ideally, the population that the model sees in the real-world resembles that of the development sample, but this is not always the case.

Ref 2: Once the system is deployed, it may be used, interpreted, or interacted with inappropriately, resulting in unfair outcomes. The underlying cause of these outcomes is a mismatch between the system’s design and the context in which it will be applied. Indeed, biases in deployment can often be attributed to abstraction traps.

Voorbeelden van biases:

automation bias caused when people prefer the results generated by algorithms over
those of humans (Ref. 3)

deployment bias occurs when decision-makers and other end users behave unexpectedly with the AI system, hereby resulting in unfair outcomes and interventions (Ref. 3)

Zie traps bij de probleemanalyse

[OliwiaDetmers]

Monitoren

Voorbeelden van biases:

reinforcing feedback loop Feedback mechanisms that amplify an effect. In the context of algorithmic fairness, it refers to the amplification of existing biases when new data is collected based on the output of a biased model. See also selection bias. (Ref. 2)

[OliwiaDetmers]

Archiveren

Nog geen specifieke voorbeelden van bias