How to build a fair AI system?

Artificial intelligence is rapidly being deployed in enterprises in all walks of life, and it is expected that enterprises will double their spending on artificial intelligence systems in the next three years. However, AI is not the easiest technology to deploy, and even a fully functional AI system may bring business and customer risks.

One of the main risks highlighted in recent news reports about AI in credit, recruitment, and healthcare applications is the potential for bias. Therefore, some of these companies are regulated by government agencies to ensure that their AI models are fair.

Machine learning models are trained on real examples to mimic the historical results of unseen data. Training data may be biased due to a variety of reasons, including the limited number of data items representing protected groups and the potential for human bias in the process of collating data. Unfortunately, models trained on biased data often perpetuate the biases in the decisions they make.

Ensuring the fairness of business processes is not a new paradigm. For example, in the 1970s, the US government prohibited the adoption of fair loan laws such as the Equal Credit Opportunity Act (ECOA) and the Fair Housing Act (FHAct) to discriminate against credit and real estate transactions. In addition, the Equal Pay Law, the Civil Rights Law, the Rehabilitation Law, the Employment Age Discrimination Law and the Immigration Reform Law all provide extensive protection measures to prevent discrimination against certain protected groups.

Building a fair AI requires a two-step process: understanding bias and addressing potential biases. In this article, we will focus on the first topic.

Understand bias

Before solving the problem, you need to confirm its existence. No company will bias its AI system towards users with malicious intent. Instead, the lack of awareness and transparency in the model development life cycle led to the inadvertent introduction of bias.

The following is a list of best practices to better understand and reduce bias in the ML development life cycle.

(1) Obtain support from major stakeholders

An unfair system is similar to a security risk that can have a significant business impact. Material resources are needed to implement fair governance processes. Without the support of the leadership team, the tasks necessary to implement the process may not be able to obtain sufficient development capabilities that exceed other business priorities. Therefore, a strong fairness-centric AI process starts with the identification system of all AI stakeholders, including the management team.

(2) Appointment of “internal defenders”

After making sure to buy, appoint a champion responsible for establishing fair procedures. Advocates communicate with various teams, including legal and compliance representatives, to draft specific use cases with the company’s field (e.g., healthcare, recruitment, etc.) and team (e.g., recommend rehospitalization, determine insurance premiums, evaluate Credit rating) related criteria, etc. There are several measures of bias, such as equal opportunity and demographic equal. The choice of fairness metric depends on the use case and is in the hands of the practitioner.

After finalizing the guidelines, the “defenders” will train the relevant teams. To make it feasible, AI fair workflow can ensure data and model deviation. In addition, it requires access to protected attributes that are assessed for fairness, such as gender and race. In most cases, it is difficult to collect protected attributes, and in most cases, it is illegal to use them directly in the model.

However, even if the protected attribute is not used as a model function, there may be proxy bias, and another data field (such as postal code) may be associated with the protected attribute (such as race). Without protection attributes and measuring them, it is difficult to identify deviations. One way for the team to address this gap is to infer protected attributes, such as using census data in the case of loan underwriting models to infer gender and ethnicity.

Measurement deviation

Next, we need to measure the deviation. Unfortunately, the inherent opacity of many types of machine learning models makes measuring their deviations difficult. The interpretability of artificial intelligence is a recent research development that unlocks the black box of artificial intelligence and enables people to understand what is happening inside the artificial intelligence model. This leads to a transparent assessment of deviations to ensure that decisions driven by AI are responsible and trustworthy.

This specific report is a model used to assess risks to make loan decisions. It has additional metadata on the protected “race” attribute. Using this report, users can use various fairness indicators to view group fairness and different effects. It is recommended that you focus on specific indicators (for example, “false positive rate”) and specific privilege categories (for example, white people) to measure bias based on the domain requirements of the use case.

In addition to tabular models such as the loan model above, there will also be deviations in the text and image models. For example, the image below shows a text model that is measuring the toxicity of user-generated annotations.

Note how the heat map shows that the model is much less biased relative to the “female” and “atheist” identity groups. In this case, ML developers may wish to add more representative examples of biased identity groups to the training set.

Fair considerations in production models

Regardless of whether there is a deviation before deployment, once the model provides services for real-time traffic, deviations may occur. The change in bias is usually caused by the provision of input data to the deployed model, which is statistically different from the data used to train the model. Therefore, the best practice is to monitor the relevant deviation indicators in the model after deployment. The screenshot below depicts the appearance of the monitoring model accuracy indicator (one of the related indicators used to track potential deviations).

In short, artificial intelligence provides a unique opportunity to quantify and resolve the deviations in the human-dominated and opaque decision-making system so far.