entropix: qwen vs llama

entropix: qwen vs llama

when using entropix, qwen 2.5 7B coder seems to produce much clearer entropy paths for entropix to follow, vs llama 3.1 8B

this makes qwen+entropix a great combination for writing code

for the uninitiated — entropix is a dynamic sampler that uses logit entropy/varentropy (variance) to decide between sampler strategies like highest prob, high temp, CoT, etc.

timkellogg.me/blog/2024/10...

the graphs above are produced with llmri github.com/xjdr-alt/llm...