The ongoing Elon Twitter saga: "insert demographic" melts down

Status
Not open for further replies.
Fubsy said:
@unwashed potato!

Good point. However, then that would mean that Google's Gemini would be training on data that has an anti-white bias, something we're told does not exist. Which is it?

It's either been programmed that way, or it's learnt from what's been fed into it. Neither is a good look.
Click to expand...
But it doesn't exist that is has a bias does it? The issue is it doesn't have a bias, in that, it should learn that certain events were entirely white only, or black only, and it doesn't know this.

It's poor yep. But it's early. Is anyone educated in the field expecting other wise?

Seems more just people like c Kent trying to act like they are being oppressed and the big bad Google is trying to come after them. Paranoia is all I see when people as so scared like this.

And it's just so thooght less, as like I pointed out, why if Google programmed it that way, are they trying to program it to no longer be that way after like 1 week of release. Google were thinking some how they'd get away with it, once the ai has killed off all the whites?

Paranoia I tell you!
 
Last edited:
Wasn't the point that traditionally AI etc has had a white bias due to the data and models.

Google tried to fix that but it went too far so they pulled it.

Sounds pretty Musk like TBF, innovate and break things quickly, fix them quickly.
 
An AI that's globally trained will very likely be biased towards the far larger cohorts of other ethnicities considering the inherently utilitarian nature of the code.
 
Last edited:
unwashed potato! said:
But it doesn't exist that is has a bias does it? The issue is it doesn't have a bias, in that, it should learn that certain events were entirely white only, or black only, and it doesn't know this.

It's poor yep. But it's early. Is anyone educated in the field expecting other wise?

Seems more just people like c Kent trying to act like they are being oppressed and the big bad Google is trying to come after them. Paranoia is all I see when people as so scared like this.

And it's just so thooght less, as like I pointed out, why if Google programmed it that way, are they trying to program it to no longer be that way after like 1 week of release. Google were thinking some how they'd get away with it, once the ai has killed off all the whites?

Paranoia I tell you!
Click to expand...

All well and good giving them the benefit of the doubt. I appreciate that. They no doubt will fix it, if they don't it will just get worse and they'll lose credibility, which may be a deciding factor in the AI space for market share.

Consistent in giving benefit of doubt though, not so sure. Like when Media Matters highlighted Holocaust denier ads next to leading brands on X lead to them pulling and funding. Very forgiving then, even though there is some doubt about how frequently that happened.
 
Fubsy said:
even though there is some doubt about how frequently that happened
Click to expand...
This is the bit you decided on yourself. It is in no way in doubt the frequency or lack of frequency

Speaking of giving the benefit of the doubt. I guess Elon was just high on drugs at the point, and isn't a moron with computers?


 
Last edited:
unwashed potato! said:
You make a good point though. People like c Kent are so paranoid that the great system is against them that they think Google purposefully programmed their ai to be anti white. Absolutely nuts. It's literally jsut learning from data is scrapes from.

And the crazy thing is with c Kents silly paranoid theory, is that Google just released this ai, and he thinks they purposefully programmed it to behave like this, and then as soon as they are discovered, they are changing it. Like wtf was the point.
Click to expand...

They did to some extent without accounting for some of the consequences. It's not that they want it to be "anti-white", though in some cases there is a bit of the old "woke" ideology creeping into the text model that I guess could be perceived that way but really the main thing is that they want to have diverse responses; don't just show a white man every time someone asks for a picture of a "scientist" etc.

Fubsy said:
Good point. However, then that would mean that Google's Gemini would be training on data that has an anti-white bias, something we're told does not exist. Which is it?
Click to expand...

Nope, it's not that either. You're both a bit wrong here, the underlying model isn't the issue, or rather not directly w.r.t the image stuff, the issue with images is a layer above that and an attempt to correct the underlying model.

This is more like the real answer:
Mercenary Keyboard Warrior said:
Its neither, its using current demographics to weight a response.
Which clearly will give odd results if they are applied to a historical context where you knew that those results did not apply.
Click to expand...

It's not really trying too hard to map to demographics even in modern contexts either, it's just diversifying in general but MKW is basically right.

Google wants to see diverse responses to fit with a diverse user base as MKW points out - sounds reasonable right? So the way they've done this with images is with prompt injections - you request something but your prompt isn't the prompt given to the model, instead, it's modified with "diversity" prompts, this has perhaps overcorrected a bit (there are only like 4.2% black people in England) but that's OK and is better to risk things that way for them than say showing a white man every time someone asks for a British doctor and then getting blasted by the mainstream press for being racist again. (Google has had issues in the past re: bias like image classification mixing up black people and gorillas).

These diversity prompts are not that well thought out, so not only are they perhaps overcorrecting but as MKW points out in a historical context they're absolutely absurd.

The "woke" stuff also becomes a minor issue (but perhaps overhyped here), perfectly reasonable/good intentions at play there too - they're trying to stop it from being used to generate racist images too so things like specifically requesting a "Chinese woman" generated an objection re: stereotypes etc.. but then asking for a British woman generates one white face and one black and one Asian etc.. (That's not matching demographics but just ensuring diversity/variety) So obvs it could draw a Chinese woman, it more or less just drew one among the four it generated for a British woman. But you can throw in activities - a Chinese woman washing clothes, a Chinese woman standing outside the massage parlour... and.... Chinese woman in LA being robbed by black man. And they're quite wary of people specifying minority groups in some contexts so there is some censorship mechanism to try and avoid the obvious from the obvious types who post on the obvious image boards etc.

As a result of that you can obviously contrive situations where you ask it for a prompt with one type of person and then with another type of person and highlight the inconsistency and call it out as being racist... that probably does include some contexts where the white guy can be the bad guy etc.. so they've not really got it quite right so far and need to have a rethink of how to ensure some representation, how to avoid abuse etc..
 
@dowie

Good post.

From what I've seen though (albeit, this is all 3rd party, no actual usage myself as it was US only and I cba setting my VPN up the other day when this all kicked off), this AI image generation was only providing diversity in one way*. How is that explained?

*e.g. I saw example results of someone asking for white people and getting only black back, where another prompt for black people returned black people. Could have been crap, and a forged response, but based on other results, wouldn't have surprised me.

Edit: also, prompt injection sounds dodgy as ****, I mean, that's full on manipulation? I can't trust a model that uses that, and that needs to be explicitly highlighted at the point of entering a prompt or with the result.
 
Last edited:
Fubsy said:
@dowie

Good post.

From what I've seen though (albeit, this is all 3rd party, no actual usage myself as it was US only and I cba setting my VPN up the other day when this all kicked off), this AI image generation was only providing diversity in one way*. How is that explained?

*e.g. I saw example results of someone asking for white people and getting only black back, where another prompt for black people returned black people. Could have been crap, and a forged response, but based on other results, wouldn't have surprised me.

Edit: also, prompt injection sounds dodgy as ****, I mean, that's full on manipulation? I can't trust a model that uses that, and that needs to be explicitly highlighted at the point of entering a prompt or with the result.
Click to expand...
If you don't want prompt injection then you will have to host your own open source model., or you can realize that it is a meaningless implemention detail
 
Fubsy said:
Edit: also, prompt injection sounds dodgy as ****, I mean, that's full on manipulation? I can't trust a model that uses that, and that needs to be explicitly highlighted at the point of entering a prompt or with the result.
Click to expand...

Perhaps prompt manipulation is a better description as prompt injection also refers to users trying to bypass some of the controls with the way they craft their prompts.

But yeah what you've highlighted is where some of the criticism of it being "woke" comes in, it's going to be more sensitive about some races, demographics than others so you can set up scenarios where it will generate an image if requested for one group but not another and that inconsistency can then be highlighted, posted on twitter etc..

If you want an example that indicates some of what the actual prompts the underlying model ends up with I pointed this out in the ChatGPT thread last week:
dowie said:
I think some of the above is the insertion of "diversity" prompts, like you're not interacting with the model directly rather you type a prompt but then Google adds some diversity words to the prompt then the model gets the modified prompt.

The model has a representation of say people in the 1800s in Scotland but it's been fed modified prompt to draw people of different ethnicities etc..

You can see an example here where it's refused to draw the image (presumably a restriction on drawing specific people after the Taylor Swift incident) and has replied back with a description, of course that description includes the (unnecessary for this image) diversity stuff:


They will probably adjust this and make it a bit more sensible in the future as it's more likely they want that stuff to apply to general current-day images but I guess they wanted to avoid negative PR upon launch.
Click to expand...

See the text in the reply; the prompt was a one-line request but the reply/explanation from the model including a description of what it would try to generate - that's indicative of the sort of modified prompt the model ends up with when generating an image "The group includes people of various ethnicities,..."
 
Last edited:
I think I'll start following that thread. Interesting.

Re-writing of history with AI is akin to Orwell's Ministry of Truth. It's horrifying.

At least when humans make up utter BS on Twitter there's a community notes feature that calls it out, or on here we can thrash it out for days/pages.
 
Fubsy said:
I think I'll start following that thread. Interesting.

Re-writing of history with AI is akin to Orwell's Ministry of Truth. It's horrifying.

At least when humans make up utter BS on Twitter there's a community notes feature that calls it out, or on here we can thrash it out for days/pages.
Click to expand...


Except no one is re-writing history.
LLMs hallucinate and are just plain wrong all the time. People don't understand what they do, they are simply next word predictors like the smart keyboards on phone. For images it is not different and just creates statistical patterns of data. They do not natively possess knowledge in an explicit form, only in probabilities.
 
D.P. said:
Except no one is re-writing history.
LLMs hallucinate and are just plain wrong all the time. People don't understand what they do, they are simply next word predictors like the smart keyboards on phone. For images it is not different and just creates statistical patterns of data. They do not natively possess knowledge in an explicit form, only in probabilities.
Click to expand...

From what dowie posted earlier, that AI "knows" that those guys are all white - it pretty much states that. However, a human, has programmed a policy that does not allow it to generate the image and then provides an inaccurate description of what said image would look like.

That doesn't quite fit with your explanation, which I admit is an accurate description of what can and does happen. This case looks to be slightly more questionable though.
 
Fubsy said:
From what dowie posted earlier, that AI "knows" that those guys are all white - it pretty much states that.
Click to expand...

The policy limitation there I think was drawing faces of real identifiable people (see issues with Taylor Swift etc..) they'll generally crack down on drawing specific people now, but the response from the LLM indicates that the prompt it is reading and the prompt that was given to the diffusion model to draw the image was modified to request diverse people... so there are a few things at play there, the modification of the prompt and the rejection of a request to draw specific people.

Note the response says "I can offer you a text description that incorporates the diverse ethnicities and genders you mentioned..." but obvs the user didn't mention any diverse ethnicities or genders, the prompt had been modified before either the LLM or diffusion model got to see it.

But yeah I don't think that has much to do with hallucinations.
 
Last edited:
Fubsy said:
From what dowie posted earlier, that AI "knows" that those guys are all white - it pretty much states that. However, a human, has programmed a policy that does not allow it to generate the image and then provides an inaccurate description of what said image would look like.

That doesn't quite fit with your explanation, which I admit is an accurate description of what can and does happen. This case looks to be slightly more questionable though.
Click to expand...


No one except google knows exactly the details, and dowie is not an authority on genAI. The reality is likely a mixture of different factors from training set biases, fine tuning issues, prompt management to people simply trying to find limitations in the model and claiming that is a bias.

The fact is none of these models should ever be trusted, they do not produce facts from a database but probabilistic distributions of data. Those probabilities can always be skewed, intentionally or otherwise.

Look how often the generative image tools create humans with 3 legs or 7 fingers etc. The models have no explicit knowledge about human anatomy. There is no way to know of the models have statistical knowledge of the skin color of various people. If there was not substantial training data then it is extremely unlikely. And then for things like drawing random people, without explicit input on skin color then what do you expect the model to do? White is a minority so defaulting to black people is perfectly fine.


There is then the fact responsible AI has legal requirements and given half the planet wants to use genAI for neferous means then of course prompt injection and filtering is pervasive. Otherwise you have endless amounts of fake images of politicians doing something illegal or immoral. Companies have to protect themselves from that.
 
Status
Not open for further replies.
Back
Top Bottom