AI + ML
Stanford researchers argue need for transparency and independent testing
AI algorithms exhibit racial bias in job candidate screening, and they discriminate more frequently against those applying for multiple jobs at different companies, according to Stanford-led researchers.The boffins evaluated algorithmic hiring decisions across multiple employers that use the same hiring vendor. The resulting algorithmic monoculture, they say, is problematic.The vendor in this instance was talent platform pymetrics, acquired by Harver in 2022. Harver did not immediately respond to a request for comment.
The researchers – Rishi Bommasani, Sarah H. Bana, Kathleen A. Creel, Dan Jurafsky, and Percy Liang – obtained a pymetrics dataset spanning the period from December 2018 through December 2022. It contained 4,197,168 job applications submitted by 3,372,132 applicants to 1,746 positions.
The dataset details hiring recommendations provided to 156 employers with a total annual revenue of $225 billion. It spans 11 industries, including finance, manufacturing, and warehousing.When people applied for jobs at these companies, they were directed to pymetrics' machine learning platform to play assessment games. The platform's algorithm measures gameplay performance and recommends on average 58.2 percent of applicants per position. Employers decide who to interview, typically rejecting candidates who were not recommended by the hiring platform.The researchers contend that the pymetrics algorithm is unfair."We find substantial evidence of racial disparities in AI-based candidate screening," the researchers said.They made that determination by applying the US Equal Employment Opportunity Commission’s "four-fifths rule," which at least on paper elicits agency attention when a given group's hiring selection rate is less than 80 percent of the most recommended group of job applicants."We discovered that 26 percent of Black applicants and 15 percent of Asian applicants applied to positions where the AI system discriminated against their racial group," the researchers said.If those Black and Asian candidates had their job applications advanced at the same rate as the most favored group (commonly White applicants), about 40,000 more job candidates would move on to the next screening stage.What's more, the report authors say that when people submit multiple applications at different companies that use the same hiring algorithm, they're more likely to be rejected everywhere than if the companies used different hiring technologies. They found 10 percent of job seekers who submit four applications were rejected from all the places where they applied for jobs.
