this post was submitted on 04 Nov 2023
294 points (93.0% liked)

Technology

58061 readers
31 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’::By contrast, prompts for ‘Israeli’ do not generate images of people wielding guns, even in response to a prompt for ‘Israel army’

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 10 months ago* (last edited 10 months ago)

The thing is, it's almost impossible to perfectly prevent something like this before it happens. The data comes from humans, it will include all the biases and racism humans have. You can try to clean it up if you know what you want to avoid, but you can't make it sterile for every single thing that exists. Once the AI is trained, you can pre-censor it so that it doesn't generate certain types of images you know are "true" from the data but not acceptable to depict - e.g "jews have huge noses in drawings" is a thing it would learn because that's a caricature we have used for ages - but again, only if you know what you are looking for and you won't make it perfect.

If the word "palestine" makes it generate children with guns, it's simply because the data it trained on made it think those two things are correlated somehow, and that wasn't known until now. It will get added to the list of things to censor next time.