A recent post on the ICO's AI Auditing
Framework blog explores human bias and discrimination in AI
systems, together with some of the technical and organisational
measures which can be implemented to mitigate the legal risks
associated with these issues.
What causes bias and discrimination in AI?
Assuming that the underlying algorithms are not inherently
unbalanced, the reason that some AI systems exhibit bias and/or
discriminatory behaviour may be that the data used to train the AI
is in some way already biased and/or discriminatory.
The blog post uses a hypothetical scenario to illustrate this,
of an AI system used to calculate credit risks for a bank, which is
giving women lower credit scores than men. The blog explains that
the likely reasons for this are either 1) that male borrowers are
over-represented in the training data, which will skew the
performance of the AI in favour of male borrowers, or 2) that the
data set used for training reflects past
