We have identified how millions of concepts are represented inside Claude Sonnet, one of our deployed large language models. This is the first ever detailed look inside a modern, production-grade large language model.
I often see a lot of people with outdated understanding of modern LLMs.
This is probably the best interpretability research to date, by the leading interpretability research team.
It’s worth a read if you want a peek behind the curtain on modern models.
I would not expect almost human-like conversation on being told that is just statistics. I’d expect something like the old Markov chain jobs. What kind of knowledge leads you to have higher expectations?
Also, how does Bayesian statistics enter into this?
ELIZA from 1966 was enough to convince people that computer program they were talking to was human. People are now being sold on getting answers to their questions via natural language prompts and those answers are pretty much plausibly sounding sentences that happen to be right sometimes due to probability calculations.
Bayesian statistics is very different from what’s being taught up until high school (at least here) and is foundational to earlier machine learning applications like spam filters. It’s hard to imagine understanding what LLMs do without basics.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !technology@lemmy.world
This is a most excellent place for technology news and articles.
I would not expect almost human-like conversation on being told that is just statistics. I’d expect something like the old Markov chain jobs. What kind of knowledge leads you to have higher expectations?
Also, how does Bayesian statistics enter into this?
ELIZA from 1966 was enough to convince people that computer program they were talking to was human. People are now being sold on getting answers to their questions via natural language prompts and those answers are pretty much plausibly sounding sentences that happen to be right sometimes due to probability calculations.
Bayesian statistics is very different from what’s being taught up until high school (at least here) and is foundational to earlier machine learning applications like spam filters. It’s hard to imagine understanding what LLMs do without basics.
Those aren’t the basics, though. That’s how saying it’s statistics is misleading. A Bayesian network is not a neural network.