Dario Amodei:
“100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it.
And AI Models actually do learn during the context window, without changing the weights”
x.com/rohanpaul_ai...
Dario Amodei:
View original threadSo I guess Anthropic’s continual learning take is “moar context”
my left shoulder (😈): “i’m so blackpilled on Anthropic”
my right shoulder (😇): “actually all big LLM advances have been super simple strategies that seemed too dumb to work”
my left shoulder (😈): “i’m so blackpilled on Anthropic”
my right shoulder (😇): “actually all big LLM advances have been super simple strategies that seemed too dumb to work”
14
they’re willingly missing the continual learning bandwagon. that’s one helluva be
but again, for GPT-3 OpenAI essentially bet the entire company in one big yolo run
en.wikipedia.org/wiki/GPT-3
Anthropic tends to be more focused. It’s their style to ignore complicated continual learning strategies
but again, for GPT-3 OpenAI essentially bet the entire company in one big yolo run
en.wikipedia.org/wiki/GPT-3
Anthropic tends to be more focused. It’s their style to ignore complicated continual learning strategies
11
remember when o1-preview was announced?
i spent the first many days shitting on OpenAI because “they’re just doing Chain of Thought”
that was super dumb, lol it really was. Yet here we are
i spent the first many days shitting on OpenAI because “they’re just doing Chain of Thought”
that was super dumb, lol it really was. Yet here we are
8