By Anthony Kaylin, courtesy of SBAM Approved Partner ASE
The Biden administration is taking a hard look at the discriminatory aspects of artificial intelligence or AI. In the past few years, AI bias has been in the news. For example, Amazon dropped an AI recruitment program because it found that it was biased towards white males. Other cases reported bias against African Americans in healthcare algorithms. Facial recognition software has an underwhelming library of faces and has been identified as a leading cause of racial injustice.
From the HR perspective, the use of AI is growing, especially the use of chat boxes and prediction modeling. A question HR has to answer is whether AI-based tools in recruiting and hiring really outperform human decision-making. With HR stretched and resources limited, AI has taken over the low-level tasks thereby freeing up time for higher level work. Also, if the algorithm is written correctly, AI can provide unbiased analysis of those who applied and sort through large amounts of data to provide the best ranking of candidates.
But the Amazon experience is something to be aware of. Amazon for many years tried developing a system for automating the recruitment process. Amazon received thousands of resumes and needed to review and rank the best candidates. The system was being developed to do this. As a result, Amazon fed the system a decade’s worth of resumes from people applying for jobs at Amazon.
More specifically, Amazon tested the system on software developer jobs and other technical positions. It reviewed resumes for the past ten years for the positions. As it turned out, primarily men applied, and men were selected by the AI. More specifically,
“Amazon’s system taught itself that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools.”
Amazon recognized even with tooling and redoing the algorithm there was no guarantee the results would differ. So, it scrapped that program.
During the Trump administration, 10 U.S. senators sent a joint letter to Janet Dhillon, the then chair of the Equal Employment Opportunity Commission, on Dec. 8, 2020, urging the EEOC to use its powers under Title VII of the Civil Rights Act of 1964 to “investigate and/or enforce against discrimination related to the use of” AI hiring technologies.
The Senators specifically asked the EEOC to review three questions:
- Can the EEOC request access to “hiring assessment tools, algorithms, and applicant data from employers or hiring assessment vendors and conduct tests to determine whether the assessment tools may produce disparate impacts?”
- If the EEOC were to conduct such a study, could it publish its findings in a public report?
- What additional authority and resources would the EEOC need to proactively study and investigate these AI hiring assessment technologies?
With the Biden administration in power, it is likely that the EEOC will review hiring cases and ask for more information on any AI tools used, especially in gender discrimination cases. HR needs to be asking vendors for the information on the algorithms used and any studies they conducted to show lack of bias in the programs.
HR also should be aware that legislation down the road may be passed that would require algorithm impact studies. In other words, employers would be required to evaluate automated systems that make decisions about people by studying the system’s design, development, and training data, in search of “impacts on accuracy, fairness, bias, discrimination, privacy, and security.” This language was specifically written in the Algorithmic Accountability Act in 2019, which will likely be reintroduced again into the current congress. Whether it can pass into law or not is a matter that will be dependent on the nonpartisan nature of the bill, but eventually down the road it will likely pass in one form or another.
Although AI will help HR do its function, HR needs to be wary of any promises of AI vendors, and conduct studies on AI impacts, especially when dealing with people processes, such as recruitment, succession planning, and the like.