New concepts will save us from AI
I was introduced to Zeno’s paradoxes during a Math Olympiad class in Singapore at age around 11. They were presented like problems that needed to be solved. It was annoying to keep trying to explain them away and constantly being met with some objection from my friend. I found the paradoxes annoying for years to come.
Until I finally realised while studying philosophy that their whole point was to be reductios ad absurdum against the concepts of space, time, plurality and infinities. Zeno meant to show that if you accept those concepts you get all these contradictions, therefore there must be something wrong with the concepts.
The situation with AI safety feels similarly annoying. Every way we try to square it, either the AI is superintelligent and runs away from us, or someone gets very powerful and threatens democracy, or there’s an arms race - all futures seem really bad right now. The people who work on AI futures seem to be great at holding some amount of denial and just trying anyway, or having a corner of their thinking that is quite wishful.
I wonder if that is because extrapolating intelligence so far (and especially, beyond our understanding) is breaking our concepts of economy, intelligence, what it means to be human, et cetera. Our inability to think of a solution might not be because no solutions exist, but that we lack the concepts we need to even craft a solution.
What could these new concepts be?
- Concepts we form as we work with AI more and integrate it into our lives (think: no one was using words like prompt, ChatGPT, agent, context engineering, token, chain-of-thought, RAG, etc. just 3 years ago)
- Some empirical patterns in the behaviour of intelligence beyond humans (or the specific behaviour of the kind of models that come to be dominant) that we just haven’t had the opportunity to observe
- Some new clarity about human consciousness
- Protocols for control, alignment, collaboration, et.c. that we are unable to design right now
- Many things across various fields of intelligence that the AI thinks up with intelligence only slightly greater than ours (think of your favourite concepts ever and imagine 100s more of those)
- Concepts more accessible from different value frameworks which we will have as we live in a society where intelligence is an API call
- A new notion of technological progress