Bluesky Thread

The year is 2026. President Musk has outlawed using proprietary model outputs...

January 07, 2025 View original thread

The year is 2026. President Musk has outlawed using proprietary model outputs to train smaller models. Attorney General Sam Altman has been tasked with quashing illegal distilleries from creating illicit open source moonshine LLMs

21 3

but fr this is a great paper arxiv.org/abs/2408.16737

arxiv.org

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling

Training on high-quality synthetic data from strong language models (LMs) is a common strategy to improve the reasoning performance of LMs. In this work, we revisit whether this strategy is compute-op...

6 1

More like this