There is a way of seeing the world where you look at a blade of grass and see “a solar-powered self-replicating factory”. I’ve never figured out how to explain how hard a Super-Intelligence can hit us, to someone who does not see from that angle. It’s not just the one fact.
Biological systems at the molecular level are impossibly advanced nanotech that we are hopelessly far from engineering ourselves from scratch. pic.twitter.com/mGKZXvm78E
A self-replicating solar-powered thing that did not rely on humans would be a miracle. Everything is possible. Imagining it does not imply the probability is > 1e-100.
Imagine putting on a cap & reading silently to yourself…except every word appears on a screen!
Yes, the AI literally reads your brainwaves
You silently think: “High quality film with twists”
BrainGPT says out loud: “Good flim, twists interesting”
The model is only 40% accurate right now, but that number will likely rise rapidly. And soon AI may not need the cap to read your brainwaves, because you leak tons of data that future AIs will be able to pick up.
Where might this go?
There are already over a billion surveillance cameras on earth, and the main reason there aren’t more is because because humans can’t go through all of the footage. But AI can.
So, if you thought there were a lot of cameras now, you aint seen NOTHING yet. And they’ll now actually be used to surveil.
In other words, the AIs will have “billions of eyes”. And the AIs won’t just see your face, they’ll see your thoughts.
If we aren’t careful, we’re hurtling towards a surveillance dystopia with no private thoughts. Orwell on steroids.
Some will read this and think “thus we must open source/decentralize” – but as Vitalik says, that doesn’t necessarily solve the problem!
If AGI is winner take all, open source may just accelerate us to the cliff faster. And if we open source everything, we’ll have no kill switch. And no safety guardrails. And since there will be more people in the race, it’ll be harder to coordinate.
AI just reconstructed a Pink Floyd song from brain activity. And it sounds shockingly clear. Think about the potential of this tech for people struggling with communication. We’re living in the future.
Here’s us from the perspective of a system just 50x faster than us
Reminder:
Without internationally enforced speed limits on AI, I think humanity is very unlikely to survive. From AI’s perspective in 2-3 years from now, we look more like plants than animals: big slow chunks of biofuel showing weak signs of intelligence when undisturbed for ages (seconds) on end.
Over the next decade, expect AI with more like a 100x – 1,000,000x speed advantage over us. Why?
Neurons fire at ~1000 times/second at most, while computer chips “fire” a million times faster than that. Current AI has not been distilled to run maximally efficiently, but will almost certainly run 100x faster than humans, and 1,000,000x is conceivable given the hardware speed difference.
“But plants are still around!”, you say. “Maybe AI will keep humans around as nature reserves.” Possible, but unlikely if it’s not speed-limited. Remember, ~99.9% of all species on Earth have gone extinct:
When people demand “extraordinary” evidence for the “extraordinary” claim that humanity will perish when faced with intelligent systems 100 to 1,000,000 times faster than us, remember that the “ordinary” thing to happen to a species is extinction, not survival. As many now argue, “I can’t predict how a world-class chess AI will checkmate you, but I can predict who will win the game.”
And for all the conversations we’re having about “alignment” and how AI will serve humans as peers or assistants, please try to remember the video above. To future AI, we’re not chimps; we’re plants.
