Fairness Score and Process Standardization: Framework for Fairness
  Certification in Artificial Intelligence Systems
  Decisions made by various Artificial Intelligence (AI) systems greatly
influence our day-to-day lives. With the increasing use of AI systems, it
becomes crucial to know that they are fair, identify the underlying biases in
their decision-making, and create a standardized framework to ascertain their
fairness. In this paper, we propose a novel Fairness Score to measure the
fairness of a data-driven AI system and a Standard Operating Procedure (SOP)
for issuing Fairness Certification for such systems. Fairness Score and audit
process standardization will ensure quality, reduce ambiguity, enable
comparison and improve the trustworthiness of the AI systems. It will also
provide a framework to operationalise the concept of fairness and facilitate
the commercial deployment of such systems. Furthermore, a Fairness Certificate
issued by a designated third-party auditing agency following the standardized
process would boost the conviction of the organizations in the AI systems that
they intend to deploy. The Bias Index proposed in this paper also reveals
comparative bias amongst the various protected attributes within the dataset.
To substantiate the proposed framework, we iteratively train a model on biased
and unbiased data using multiple datasets and check that the Fairness Score and
the proposed process correctly identify the biases and judge the fairness.
