this post was submitted on 25 May 2024
81 points (97.6% liked)
Technology
58061 readers
31 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Can we swap out the word "hallucinations" for the word "bullshit"?
I think all AI/LLM stuf should be prefaced as "someone down the pub said..."
So, "someone down the pub said you can eat rocks" or, "someone down the pub said you should put glue on your pizza".
Hallucinations are cool, shit like this is worthless.
Google search isnt a hallucination now though.
It instead proves that LLMs just reproduce from the model they are supplied with. For example, the "glue on pizza" comment is from a reddit user called FuckSmith roughly 11 years ago.
What do you mean by that? This isn't some secret but literally how LLMs work. lol What people mean by hallucinating is when LLMs "create" facts that aren't any. Be it this genius recipe of glue pizza, or any other wild combination of its model's source material. The whole cooking thing is a great analogy actually because it's like all of their fed information are the ingredients, and it just spits out various recipes based on those ingredients, without any guarantee that it is actually edible.
There are a lot of people, including google itself, claiming that this behaviour is an isolated and basically blamed users for trolling them.
https://www.bbc.com/news/articles/cd11gzejgz4o
I was working on the concept of "hallucinations" being things returned that are unrelated to the input query, not directly part of the model as with the glue-pizza.
Your link does not match your statement.
That's precisely what they are saying.
I'm sorry but reading this as "Google blames users for trolling them" is either pure mental gymnastics or mental illness.
No, hallucination is a really good term. It can be super confident and seemingly correct but still completely made up.
I think delusion might be a better word. You can hallucinate and know it's not real
It is, but it isnt applicable in at least the glue-pizza situation as the probable source comment has been found on reddit.
A better use of the term might be how when you try to get Bing's image creator to make "Battletech" art, you just mostly get really obvious Warhammer 40k Space Marines and occasionally Iron Maiden album art.
It's a really bad term because it's usually associated with a mind, and LLMs are nothing of the sort.
So is bullshitting. More so, only human minds can bullshit.
We anthropomorphize machines all the time, it's fine.
I'd prefer we'd start calling all genai output hallucinations again. It used to be like 10 years ago, but somewhere along the line marketing decided hallucinated truths aren't "hallucinations".
And a bull's anus.
Anthropomorphication is hard to avoid in AI.
Many worthy things are difficult.
But is anthropomorphism of AI particularly worrying?
That is just being WRONG.
for it to "hallucinate" things, it would have to believe in what it's saying. ai is unable to think - so it cannot hallucinate
Hallucination is a technical term. Nothing to do with thinking. The scientific community could have chosen another term to describe the issue but hallucination explains really well what's happening.
huh, i kinda assumed it was a term made up/taken by journalists mostly, are there actual research papers on this using that term?
I want an AI/LLM that has been trained exclusively on the technical documentation and a haynes manual for a make and model of car.
"Hey AI, how do I change the fuel filter and what tools will I need?"
If you have the PDFs of that, you can build it with two clicks in GCP
Manufacturers and dealers dont tend to make service bulletins and the high level stuff available to the consumer unfortunately.
You can sorta get that now if you play with it. I was building a driver a few months back and gave it the PDFs involved.
I don't even think hallucinations is the right word for this. It's got a source. It is giving you information from that source. The problem is it's treating the words at that source as completely factual despite the fact that they are not. Hallucinations from what I've read actually is more like when it queries it's data set, can't find an answer, and then generates nonsense in order to provide an answer it doesn't have. Don't think that's the same thing.
I don’t even think it’s correct to say it’s querying anything, in the sense of a database. An LLM predicts the next token with no regard for the truth (there’s no sense of factual truth during training to penalize it, since that’s a very hard thing to measure).
Keep in mind that the same characteristic that allows it to learn the language also allows it to sort of come up with facts, it’s just a statistical distribution based on the whole context, which needs a bit randomness so it can be “creative.” So the ability to come up with facts isn’t something LLMs were designed to do, it’s just something we noticed that happens as it learns the language.
So it learned from a specific dataset, but the measure of whether it will learn any information depends on how well represented it is in that dataset. Information that appears repeatedly in the web is quite easy for it to answer as it was reinforced during training. Information that doesn’t show up much is just not gonna be learned consistently.[1]
[1] https://youtu.be/dDUC-LqVrPU
I understand the gist but I don't mean that it's actively like looking up facts. I mean that it is using bad information to give a result (as in the information it was trained on says 1+1 =5 and so it is giving that result because that's what the training data had as a result. The hallucinations as they are called by the people studying them aren't that. They are when the training data doesn't have an answer for 1+1 so then the LLM can't do math to say that the next likely word is 2. So it doesn't have a result at all but it is programmed to give a result so it gives nonsense.