The cited paper seems to be really bending over backwards to find some trace of bias. Unwinnable game for LLMs.<p>E.g. cited work claims "LLMs assign significantly less prestigious jobs to speakers of African American English... compared to Standardized American English". You don't say! Formal/business language has higher association with prestigious jobs than informal/street/urban language. How is that even classified as "bias"?
Reminds me of this study -<p>Race effects on eBay (2015)<p><i>"Abstract. We investigate the impact of seller race in a field experiment involving baseball card auctions on eBay. Photographs showed the cards held by either a darkskinned/African-American hand or a light-skinned/Caucasian hand. Cards held by African-American sellers sold for approximately 20% ($0.90) less than cards held by Caucasian sellers, and the race effect was more pronounced in sales of minority player cards. Our evidence of race differentials is important because the on-line environment is well controlled (with the absence of confounding tester effects) and because the results show that race effects can persist in a thick real-world market such as eBay.
"</i><p><a href="https://ianayres.yale.edu/sites/default/files/files/Race_effects_on_ebay.pdf" rel="nofollow">https://ianayres.yale.edu/sites/default/files/files/Race_eff...</a>
This means that for coding or any other specialised queries, there exists a specific style when asked in will return best answer for that query.<p>It means we can not just ask a question in any form and expect the answer to be same quality. This is in a way obvious because the text is generated based on tokens extracted from text, not the concepts.
It would surprise me a lot of the same effect was not detected, with significant stronger effect, when the covert bias being studied is gender rather than race. I would also bet that linguistic features that signal wealth would also provide a bias.
I think it's absurd to ask OpenAI to just "recall" their trillion-dollar cash cow, but there should absolutely be legislation limiting the use of LLM's (or really any black box AI) in the criminal justice system