Bluesky Thread

dear GPT-5, i’m pretty sure “storagely” is not a word, but that oddly makes a...

View original thread
dear GPT-5, i’m pretty sure “storagely” is not a word, but that oddly makes a ton of sense so maybe i’ll start using it
• FlashAttention still does softmax; it just computes it numerically/storagely better.
40 2
honestly this model really does feel “big”, but in a totally different way than R1, K2 & Opus. like a whole ass new direction for depth

it’s like that nerdy kid whose words can’t keep up with their brain
7
1 hour later
my reading on softmax & entropy in LLMs has led me into economics

you can indeed calculate the “temperature” (same thing as the LLM parameter) of wealth redistribution across the years for an economy

and it’s been done:

apps3.aps.org/aps/units/ma...
Chart titled “Time evolution of income temperature” showing income-related metrics from ~1983 to 2001.
	•	(a) Income separating exponential and power-law (r*) – black line with star markers, rising from ~95k in 1983 to ~140k in 2001.
	•	(b) r* / T – green line with square markers, decreasing from ~6.5 in 1983 to ~3 in the late 1990s, stabilizing around 3–3.5.
	•	(c) Average income (T) – red line with circle markers, rising from ~20k in 1983 to ~40k in 2001.
	•	(d) Inflation (CPI) – blue line with triangle markers, steadily increasing.
	•	(e) GDP per capita – purple line with diamond markers, increasing gradually in line with average income.

Right-hand annotation:
	•	“The nominal average income T doubled: 20 k$ 1983 → 40 k$ 2001, but it is mostly inflation.”

X-axis: Years (1983–2001).
Y-axis left: k$ (0–150).
Y-axis right: ratio r*/T (0–10).

Source: Victor Yakovenko, Statistical mechanics of money, income and wealth. Slide 26.
5
expanding slightly — a low temperature drives “wealth” (or logit values, for LLMs) into fewer hands

whereas higher temperatures will redistribute equally without regard to the model’s (economy’s) biases

so temperature is a parameter that can exaggerate the internal bias or structure
3
40 likes 2 reposts

More like this

×