this post was submitted on 14 Jul 2023
2 points (75.0% liked)
Technology
58061 readers
31 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I think you're reading something there other than what I said. Look, today's LLM's ingest a ton of text - more accurately tokens - and builds up statistics of which tokens it sees in that context. So statistically if you see the sentence "A nice cup of " statistically the next word is maybe 48% coffee, 28% tea, 17% water and so on. If earlier in the text it says something about heating a cup of oil, that will have a muuch higher chance. It then picks one of the top tokens at (weighted) random, and then the text (array of tokens) is fed in again into the LLM and a new prediction is made. And so on it continues until you stop the loop (usually from a end token or a keyword you're looking for). Larger LLM's are better at spotting more subtle patterns - or more accurate it got more layers of statistics that's applied - but it still has the fundamental issue of going one token at a time and just going by what's most likely to be the next token.
Have you tried that when it's correct too? And in that case you mention it has a clean break and then start anew with token generation, allowing it to go a different path. You can see it more clearly experimenting with local LLM's that have fewer layers to maintain the illusion.
We're trying to make a flying machine by improving pogo sticks. No matter how well you design the pogo stick and the spring, it will not be a flying machine.