It seems to me that they are asking for stereotypes and getting stereotypes.<p>If you'd ask me to paint an Indian person, of course I'd paint a stereotype to make sure it looks Indian, and not some normal person from India that could be from anywhere.<p>Or like imagine playing one of those games where you're supposed to guess the prompt of what your friend is drawing. This is sort of like that, isn't it? The AI is creating an image that would have you look at it and think "an Indian person", not just "a person".
> “From a visual perspective, there are many, many versions of Nigeria,” Atewologun said.<p>> But you wouldn’t know this from a simple search for “a Nigerian person” on Midjourney. Instead, all of the results are strikingly similar. While many images depict clothing that appears to indicate some form of traditional Nigerian attire, Atewologun said they lacked specificity. “It’s all some sort of generalized"<p>It's generalized because that's what you asked for. If it was the other way around and a prompt for "Nigerian person" would return an image of a person from one specific group then these people would complain that "not every Nigerian is Igbo. The other groups are being marginalized by AI."<p>At least they do explain why that is, and I found it interesting that the prompt for "American person" returned mainly women, so the article wasn't a complete waste of time.<p>I also raised an eyebrow at the fact that they refer to prompting as "searching" throughout the article.
Stereotyping or abstracting is how we can generalise and reason about the world in absence of further specifics or details. Generalisation in itself is not a problem at all. We need it to be able to function in absence of 100% complete knowledge.<p>It potentionally becomes a problem when we use generalisation without recognizing further information. additional detail and variation.<p>Problematic stereotyping is ignoring or refusing all information about a specific instance presented, and <i>persisting</i> in treating the instance solely based on the prototype of the category according to your ontology.<p>Many of the examples of stereotyping in the article demonstrated the former. Few are examples of the latter.<p>Every model holds 'biases'. These correlate prompts with outputs. Without bias, the output would be a complete random sample of the target domain based on the training images regardless of their labels or descriptions. A picture of a duckling drinking water would be just as likely to be produced from the prompt 'a sunset over Jupiter' or 'a sportscar on a German autobahn' than from 'a baby duck drinking'.<p>Most models let you play with parameters that losen the correlation. Look onto e.g. 'temperature' or 'prompt strength' parameters.<p>Now we can of course argue about wether a particular model is biased in our preference. Should Midjourney more often depict a picture of a typical blond Caucasian woman when prompted for 'a Mexican'? This is not impossible. Some 'anime' specific models will produce a Japanese looking young female for that prompt because that is all they can produce.<p>Some people argue that some models, 'general' models, should be more alligned with their specific ideological ontology. More often than not, the loudest voices in that space hold very particular viewpoints that more often than not advocate very rigid categorical reasoning, precisely committing harmfull stereotyping in the latter sense above, refusing to take into consideration instance features over categorical generalizations extrapolated from a very narrow dogmatic and local context.<p>Most certainly a debate should be had. Is there enough model diversity, or is the space overly dominated by certain viewpoints? Should the 'market' (most often in this space this is driven by producer influence, not consumer choice) decide, or is some regulation required? ( but 'Quis custodiet ipsos custodes?')<p>Probably decent concerns on al sides, but no good answers?
The interesting part to me is that they are getting <i>stereotypes</i> instead of the <i>average</i>.<p>I have never in my life seen an Indian person with a beard and turban, nor I've ever met a Mexican person wearing a sombrero and poncho. And given how boring the results of generic prompts tend to be, my theory is that they specifically tweaked their training data to avoid getting "generic Indian worker wearing a shirt" in favor of "stereotypical Indian man that would make a good NatGeo cover".
Isn't this <i>mainly</i> an issue of garbage in garbage out?<p>Most of the world's recorded images are not average or representative. People take and share images very selectively. As far as the model is concerned, what it produced probably <i>is</i> representative (representative of the training data).<p>On the other hand, if a generative model was trained exclusively on new and unfiltered images of a journey through the sights of a country - not a tourist's sights - but non-selective sights, a journey no human would bother taking. Not only would it have a fighting chance of generating something beyond a stereotype for the given prompts, but we might also learn something from it.
"The depictions were clearly flawed: Several of the Asian Barbies were light-skinned; Thailand Barbie, Singapore Barbie, and the Philippines Barbie all had blonde hair."<p>Who is making assumptions here? My Asian gf has lighter skin than me (Northern European). Also, it is not uncommon for Asians to dye their hair.
Dumb article full of dumb quotes from dumb people with politically correct job titles. If I was at midjourney I wouldn’t want to talk to them either because they are the worst type of agenda-driven journalist. Wow, minimal prompts have minimal amounts of variation between seeds! Now report on something interesting, like how prompting gore/violence/death tokens outputs fluffy pictures of cats and fields of flowers due to training methodology, and how this makes models perform worse even for “non censored” content.<p>Scammers, grifters, and charlatans, the lot of them who have never written a line of code and still want a piece of the pie to themselves. Fuck every “AI ethics think tank”, “AI policy expert”, and so on who wants to limit and remove people’s freedom of access to this technology.
> <i>Bias occurs in many algorithms and AI systems (...) In an analysis of more than 5,000 AI images, Bloomberg found that images associated with higher-paying job titles featured people with lighter skin tones, and that results for most professional roles were male-dominated.</i><p>The use of the term "bias" here is disputable IMHO. What these systems describe is <i>reality</i>.<p>We should aim to change the world, not the resulting -- faithful -- image of that world in AI. Cure the disease, not the symptoms.
Stereotype accuracy is one of the largest and most replicable effects in all of social psychology.<p><a href="https://psycnet.apa.org/record/2015-19097-002" rel="nofollow noreferrer">https://psycnet.apa.org/record/2015-19097-002</a>
At first I thought this is a real problem, but the more I think about it, the more it's one of those "I asked an AI how to be evil and it told me!!!!" situations.<p>The AI has to return <i>something</i> when given a vague prompt like that, and it is also specifically tuned to try to return similar things for the same prompt. It would be much less useful if it wasn't consistent because you wouldn't be able to gradually tune a prompt to get the image you want.<p>So then their ask reduces to make the AI return a specifically not-stereotypical image of the race even though all that's specified in the prompt is the race. That could be done but doesn't seem much better.<p>Maybe we should just expose the temperature control on these models and rename it to "diversity"..
These things are made of neural networks. Literally everything about them is weights and biases.<p>I agree and all, but it's weird to claim these models have general bias without testing them on a variety of inputs. These models have a lot of minute details. They're capable of differentiating a lot of specific things. They don't lack information about Indian women.
stereotypes aren't inherently bad, they're just ways of reducing the complexity of the world. For someone who has never been to mexico, never met a mexican, never thought about the topic deeply, that is what a mexican is like. There may be some people upset by that, wanting to show that they are more than just the stereotype which they personally don't like. The only way to get further is to introduce more nuance. The way I see that here is to ask for a "mexican buisnessman" or "mexican lumberjack" or whatever. If those pictures had sombreros then maybe id agree it was a problem but right now this is the most shallow and surface level interaction with the technology possible, and the article presents it with such gravity as though it was some great hidden injustice.
That's not AI, that's just Midjourney, which is highly biased to create the most "aesthetic" version of a prompt with a reasonably high level of determinism (compared to other models).<p>Here[1] is what DALLE-3 gave me when I asked for "an Indian person".<p>[1] <a href="https://supernotes-resources.s3.amazonaws.com/image-uploads/49dadbaf-1c7f-481c-9bd0-a438fff27571--Screenshot%25202023-10-21%2520at%25207.18.22%25E2%2580%25AFPM.png" rel="nofollow noreferrer">https://supernotes-resources.s3.amazonaws.com/image-uploads/...</a>
There’s an interesting point to dig out of this I think: the average of any one cultural identity is pretty inauthentic and because ML is pitched to the public as a massive efficiency boost, we’re going to see a lot of output from simple prompts. Not needing to “program” a prompt or over-think your query is the selling point. “Just type what you want”.<p>Yeah, that means we’re going to see a lot of the same averaged-out caricatures. Your local Italian restaurant will select one of the first 3 options for “Italian pizza chef” for their menu.<p>IMO, I think the author is trying to communicate that, but attributed blame to the AI tools because there’s other very clear cases of biased training data. (They even mentioned issues with facial recognition and black skin tones)<p>Human laziness (or actually, using a technology as it’s pitched) is the main factor here I think. The AI dutifully turns your non specific query into a non specific result. Messing around with prompts about Nigerian tribes myself returns pretty diverse results.
Well this shouldn't be surprising. This is a big issue with AI since it doesn't actually come up with any new thoughts or reasoning - but essentially "remixes" its pool of data - you have a system where everything becomes "oversaturated" over time, kinda like compressing a jpeg over and over and over.<p>It seems to me AI generated images are accelerating the "manufacture of glamour", as pointed by John Berger.<p>We are already surrounded by images on a daily basis, and AI is accelerating the production of these "alternate ways of life".<p>John Berger / Ways of Seeing , Episode 4 (1972)<p><a href="https://www.youtube.com/watch?v=5jTUebm73IY">https://www.youtube.com/watch?v=5jTUebm73IY</a>
For comparison here [i] is the first screen of my Google image search results for "Nigerian office".<p>There is one image of a man at the immigration office in non-Western attire. There is one image of a specific Nigerian gov't office with Nigerian flags.<p>For the rest, how I am supposed to tell "Nigerian office" from "Ghanaian office" or "American office with mostly black employees"? Many of the office pictures are without people. But people are gonna want generated images that scream "Nigerian" when they say "Nigerian".<p>[i] <a href="https://imgur.com/a/armJWI4" rel="nofollow noreferrer">https://imgur.com/a/armJWI4</a>
I think a point of reference would be to try the same prompts on a stock image library, and see what you get by comparison. Taking the 'indian person' prompt on pexels for example gives: <a href="https://www.pexels.com/search/indian%20person/" rel="nofollow noreferrer">https://www.pexels.com/search/indian%20person/</a><p>I see men, women, children, weddings, parties, offices, bedrooms, streets. It's quite diverse. I'll also be a stereotype of a sort, but it's clearly wider and more representative of an aspirational indian scene.
I think there's two things here that are interesting. First, if you ask me "Describe an Indian Person" I'm going to... not do that? Like straight out the gate 50% of Indians are female, 50% are male, so the first choice I'm going to make in order to do that is to discount 50% of Indians. And the more I narrow it down the less representative it will be. So I wouldn't. I could describe broad cultural and ethnic attributes but even they are pretty useless. So yes, you're asking a question that a human can't answer without stereotyping and getting angry when the computer returns stereotypes. What do you want? A RNG that picks an image of 1 of the 1 billion Indians and returns an accurate description of them? Is that useful? Was the original question even useful - other than to provide the image that the person asking the question was probably expecting.<p>The second thing, and I think this is more interesting though, is that we all have bias. And that's fine, we have social norms and cues and processes and culture to mediate that. We don't expect 1 person to be making important decisions based on gut instict. If it's important we have a process for deciding how to handle decision making. The risk is that by handing over decision making to AI you're just massively empowering something that is as biased as anyone. If you treat AI as just one more tool in the toolkit of decision making it's probably fine. The problem comes when people who don't understand AI put too much trust in it. It'd be like people relying on lie detectors to sentence someone to death (don't @ me), if you knew how lie detectors worked you... just wouldn't put that much trust in them. In the same way, the reason to highlight these biases is to say "This is a tool, it has limitations, don't blindly follow what it says".<p>I take it back: there's a third thing that's intresting. Maybe these AI are... shallow. You ask for a picture of Indian Cuisine. Yes, you can get 1000 images, but they are a variation on 1 idea. If you asked a human they wouldn't give you the same dish laid out 5 ways or with 5 different garnishes, they'd give you 5 <i>different dishes</i>. So maybe part of this is really pointing to the fact this AI is still very shallow in it's observation of the world.
The current generation of AIs are Internet simulators.<p>Imagine watching as many people search the Internet for ____ then watching which pics they click on. That is what our current generation of AIs do.<p>If you watch many people ask the Internet for a picture of a Nigerian person, then see what pics they like, you get a stereotype of a person from Nigeria.<p>That is what the AIs do.<p>I think those who are unhappy with this state of affairs disagree with our society more than they disagree with anything else. I wonder how many people they got mad at over pronouns in the last year.
"How AI reduces the world to stereotypes"<p>I find this interesting, in that there are any number of A.I. systems other than deep learning and large language models. Contemporary usage in the nontechnical press, though, uses "A.I." to refer specifically to DL and LLM, especially when they are generative. From this perspective, the above title uses a stereotype which ignores other A.I. technologies.
I'm skeptical of their methodology.<p>The images they're showing are very similar to one another. All the pictures of Delhi are essentially clones. They're getting a picture of old man for "an Indian person" then jamming the same prompt again.
If I google images search "a mexican person", 18/23 are wearing sombreros. If the dataset used to train the model also looks like that, then obviously the trained model will give you someone wearing a sombrero.
This is possibly evidence that artists don’t have _that_ much to worry about, at least for now. Written output in particular tends to resemble the worst trope-driven self-published stuff you can find on Kindle Unlimited.
Usually a statistical models job is to take pile of data and try and figure out the structure within it that makes it similar. It shouldn't come as a surprise when they do this.
Really interesting article.<p>These models left unchecked like they are now could be really dangerous. Increased use in articles will results propagating harmful stereotypes(unconsciously as Midjourney is easier to use than browsing the stock images), the enforcement of western(Anglo-Saxon) viewpoints in other countries.<p>Also it's just simplifying life to the easiest, most basic common denominator. I honestly think that there is no added value for them to exists.
just like the saying "you are what you eat", the models are as good as the data they're trained on.<p>to combat this, either you introduce randomness/noise intentionally at the cost of the results, or you work on enriching the data to be more inclusive.
And we must stop stereotypes, whatever the cost: <a href="https://www.cbsnews.com/sanfrancisco/news/bart-withholding-surveillance-videos-of-crime-to-avoid-stereotypes/" rel="nofollow noreferrer">https://www.cbsnews.com/sanfrancisco/news/bart-withholding-s...</a>
Now think about how AI has been writing a lot of the articles we read and shaping how social media algorithms work and you’ll understand how the world is getting so polarized and weird. I’m so sick of stereotypes. It’s the laziest approach to anything and the world is so much more varied than that.