Bluesky Thread

Mistral Small 3

January 30, 2025 View original thread

Mistral Small 3

A 24B LLM that's VERY fast with great function calling

More important, MISTRAL IS OPEN SOURCE AGAIN!!!!!!

mistral.ai/news/mistral...

A scatter plot comparing AI model performance on MMLU-Pro against latency in milliseconds per token. The x-axis represents latency (milliseconds per token), and the y-axis represents performance (MMLU-Pro score).

- **Mistral Small 3** (highlighted in orange with a castle emoji) is positioned in the upper-left region, indicating high performance and low latency.
- **GPT-4o Mini** is slightly lower in performance but has higher latency.
- **Qwen-2.5 32B** is positioned higher in performance but with greater latency.
- **Gemma-2 27B** has lower performance and the highest latency among the models.

The benchmark is based on Apache 2.0 models using vLLM with a batch size of 16 on 4xH100 GPUs, with GPT-4o Mini data sourced from OpenAI's API.

23 3

i had mistral-small produce an economic model for a 25% tariff on Mexico and fed the answer into R1 and R1 had A LOT to say, mistral did not hold up to R1’s high standards

More like this