Bluesky Thread

Whoah.. sonnet was not distilled

January 29, 2025 View original thread

Whoah.. sonnet was *not* distilled

"3.5 Sonnet was not trained in any way that involved a larger or more expensive model (contrary to some rumors)."

—Dario Amodei

darioamodei.com/on-deepseek-...

DeepSeek does not "do for $6M5 what cost US AI companies billions". I can only speak for Anthropic, but Claude 3.5 Sonnet is a mid-sized model that cost a few $10M's to train (I won't give an exact number). Also, 3.5 Sonnet was not trained in any way that involved a larger or more expensive model (contrary to some rumors). Sonnet's training was conducted 9-12 months ago, and DeepSeek's model was trained in November/December, while Sonnet remains notably ahead in many internal and external evals. Thus, I think a fair statement is "DeepSeek produced a model close to the performance of US models 7-10 months older, for a good deal less cost (but not anywhere near the ratios people have suggested)".

30 3

Dario doesn't agree with me

Highlighted text: 'the export controls were not the main factor causing them to "innovate"'

Full quote:

The performance of DeepSeek does not mean the export controls failed. As I stated above, DeepSeek had a moderate-to-large number of chips, so it's not surprising that they were able to develop and then train a powerful model. They were not substantially more resource-constrained than US AI companies, and the export controls were not the main factor causing them to "innovate". They are simply very talented engineers and show why China is a serious competitor to the US.

4 1

Finally, let's stop this "DeepSeek distilled o1" nonsense. Dario doesn't even believe it. DeepSeek is not the enemy of anyone.

Highlighted text: "I don't see DeepSeek themselves as adversaries and the point isn't to target them in particular"

Full quote:
Given my focus on export controls and US national security, I want to be clear on one thing. I don't see DeepSeek themselves as adversaries and the point isn't to target them in particular. In interviews they've done, they seem like smart, curious researchers who just want to make useful technology.

But they're beholden to an authoritarian government that has committed human rights violations, has behaved aggressively on the world stage, and will be far more unfettered in these actions if they're able to match the US in AI. Export controls are one of our most powerful tools for preventing this, and the idea that the technology getting more powerful, having more bang for the buck, is a reason to lift our export controls makes no sense at all.

9 1

yesterday i wrote that the US should completely lift the bans, and i’m not sure dario is really invalidating my arguments

bsky.app/profile/timk...

Tim Kellogg @timkellogg.me

The USA Should Lift AI Export Controls

This is an overview of what we've learned today: DeepSeek is skirting around AI export controls intended to slow Chinese AI progress with some low-level hacking

I suggest that the US could maintain it's lead by lifting the ban

timkellogg.me/blog/2025/01...

This image features a red brick wall painted with the design of the **flag of China**. The flag consists of a large yellow star in the upper left corner, surrounded by four smaller yellow stars in an arc formation. The brick texture adds a rough, artistic effect to the national emblem. Let me know if you need any additional analysis!

i still remember that 2023 leaked google memo declaring that open source had won

since then, it’s felt like a constant battle back and forth, when o3 hit i was convinced closed had won. a month later, open is still in it

bsky.app/profile/sung...

Sung Kim @sungkim.bsky.social

Google engineer stated in 2023, “We Have No Moat, And Neither Does OpenAI” ( semianalysis.com/2023/05/04/g... ).

However, we do have a moat in AI chips, which employs a lot more people than these AI labs. To maintain that moat, we need to lift the short-sighted AI chips export controls.

obligatory bsky.app/profile/timk...

Tim Kellogg @timkellogg.me

all y’all see a moat, i see a leaky basement

More like this