Bluesky Thread

an audible “no fucking way..” escapes my mouth

View original thread
an audible “no fucking way..” escapes my mouth

they claim that you can get LLMs to give you well calibrated confidence scores

like, i knew you could do something like this with logits/attention values, but the idea that the LLM could speak it back to you.. 🤯
Maria Antoniak @mariaa.bsky.social
Now I have some evidence to back up me telling all my collaborators to add confidence predictions to their prompts. Anecdotally, I've found this helpful for classification tasks.

arxiv.org/abs/2412.14737
39 4
afaict they’re being honest about the scope & limitations of their study. probably a credible paper chatgpt.com/share/67a696...
chatgpt.com
ChatGPT - Study Caveats and Limitations
Shared via ChatGPT
6
39 likes 4 reposts

More like this

×