i tore this apart this morning, the gist:

i tore this apart this morning, the gist:

- yes, it separates knowledge from reasoning 🎉
- it substitutes MHA computational complexity for knowledge graph schema design 🤔

i’m partly ecstatic, this is huge, but also disappointed bc KG design is largely unsolved

a big problem with the KG tie in is that KGs lose information, a ton

you have to boil everything down to entity-relation-entity triples, and a lot of knowledge (e.g. procedural knowledge) doesn’t fit into that

this probably made a lot more sense in the early days of LLMs

the achilles heel is that it goes against the bitter lesson

instead of throwing more compute at a problem, we solve it through brain power

but it’s worse. the bitter lesson was in regards to researchers, this would require ALL USERS to have superb KG design skills

that’s a bad scaling dynamic

ChatGPT sesh

my process:

1. read the paper, kinda, until i have unfounded confidence on the topic

2. spout my unfounded confidence into an o3-mini-high chat

3. let o3 bring me back to earth, touch grass

chatgpt.com/share/67db6a...