A new paper dropped from DeepMind: Deliberation in Latent Space via Different...

A new paper dropped from DeepMind: Deliberation in Latent Space via Differentiable Cache Augmentation

The trouble is, it's not very readable. I tried making a thread here, but it got far too long, so it's a blog now:

timkellogg.me/blog/2024/12...

The big notes here are:

* Use an "expert" LLM to augment a generalist
* Yes, it's like RAG, but it doesn't replace RAG