The original AI alignment person, his work on the prospect of a runaway intelligence explosion and existential risk from AI has been extremely influential and practically gave breath to the whole AI Safety Motion.
He has been thinking about this long and hard and has been key to building the Alignment theory framework since at least as early as 2004, when AI was hardly on anyone’s mind.
Eliezer S. Yudkowsky is an American artificial intelligence researcher and writer on decision theory and ethics, best known for popularising ideas related to artificial intelligence alignment.
He is the founder of and a research fellow at the Machine Intelligence Research Institute (MIRI), a private research nonprofit based in Berkeley, California. He also founded the popular LessWrong community originally as a sister site and offshoot of Robin Hanson’s Overcoming Bias . LessWrong is dedicated to the study of human rationality, the general field of decision making and pulls together a variety of material from mathematics, economics, cognitive science, and other disciplines relevant when considering how individuals or groups act in complex environments rationally.
People often draw him as a pessimist, but in fact he is an ultra-optimist, who was originally a major pro-AI enthusiast and wrote beautiful pieces about the Singularity, until at some point, his optimism about Technological Progress momentum drove him to the conclusion that the Artificial Intelligence will soon blow past the humanity fast, to the point where the difference will be bigger than that of humans and cockroaches. He is actually much more of an optimist than most accelerationists out there. What makes him a doomer is the fact that he has thought long and hard about the values-alignment problem and seeing the trend, he estimates we will reach God-Level AI before we know how to configure it to care about us.
Like most “AI Doomers”, he is a techno-optimist who only advocates for a pause, for just enough time, so that we can change the order of events.
If only we could get alignment before capabilities, we would unlock the closest thing we can have to paradise.
