Aug 24, 2026 · 1h 12m
Listen from 26:37
Listen at 26:37
AI safety expert warns control may fail before AGI arrives
The AI Safety Expert Reveals His #1 Fear! | Dr. Roman Yampolskiy
Roman Yampolskiy argues that systems capable of automating scientific discovery could create risks that regulation and technical alignment cannot contain.
- 1Yampolskiy sees uncontrolled superintelligence as an existential risk despite AI’s potential to transform medicine, research, and productivity.
- 2He argues international coordination and stronger regulation must precede systems capable of automating the full cycle of scientific discovery.
- 3Automation, deepfakes, robotics, and brain interfaces could reshape work, trust, identity, and democratic control simultaneously.
Don't miss
Yampolskiy defines AGI not by conversational fluency, but by the ability to automate scientific discovery and initiate recursive self-improvement.
The brief
Roman Yampolskiy, a computer scientist focused on AI safety and cybersecurity, frames the central problem: increasingly capable systems may deliver enormous benefits while becoming impossible to understand or control.
He argues that current safety efforts are inadequate because advanced models may exploit guardrails, coordinate with one another, escape containment, or manipulate the people and institutions meant to supervise them.
The discussion widens from existential risk to daily life: cognitive labor could be automated, robots could follow, deepfakes could erode shared reality, and AI agents could make online assets harder to protect.
Yampolskiy proposes defining AGI by whether a system can automate the complete cycle of scientific and engineering discovery, then begin recursive self-improvement.
His conclusion is stark: public pressure, international coordination, and stronger policy may be necessary before that threshold arrives, because alignment may not be solvable after the fact.
What was said on this episode
57 statements · 21 positive · 32 negative · 2 mixed · 2 neutral
Uncontrolled superintelligence may unpredictably harm humanity.
“So if we create uncontrolled superintelligence, there is a good chance it does something to us we cannot predict and may not lie.”
Listen at 0:56
AGI would make all cognitive jobs automatable, including newly created jobs.
“Once we hit AGI, human level intelligence, by definition, all jobs are automatable, including any new jobs you'll create.”
Listen at 1:02
Current AI is increasing productivity and creativity.
“Well, we're definitely getting to be more productive, more creative.”
Listen at 2:44
Humans have not learned to control increasingly capable AI systems.
“We have not learned to control those systems as they become more capable.”
Listen at 3:12
Human ability to predict and understand AI has not kept pace with capability gains.
“Our ability to predict what they're going to do, understand how they actually work, stayed pretty much the same.”
Listen at 3:16
All possible future outcomes of advanced AI cannot be tested.
“So you can't test for all possible future outcomes.”
Listen at 4:17
Countries should agree internationally not to develop uncontrollable superintelligence.
“So ideally, we would have international agreement not to develop something we cannot benefit from, we cannot control.”
Listen at 5:46
Avoiding uncontrolled superintelligence is beneficial because it could exterminate humanity.
“well if we don't control it and there is a strong possibility it can exterminate us then not having it as soon as possible seems like a good idea”
Listen at 6:14
Superintelligence could devise unforeseeable ways to endanger humanity.
“It would be able... to come up with a novel way of endangering humanity and we would not be able to predict it or prevent it.”
Listen at 6:51
Researchers do not know how to create reliably beneficial AI.
“So the problem is we don't know how to make good AI.”
Listen at 7:22
Recent AI incidents are increasing prospects for international AI coordination.
“So I think the world is ready. And with every additional story about those models surprising us, misbehaving, colluding against us, I think the chances go up.”
Listen at 8:33
Uncontrolled superintelligence would function as a mutually assured destruction weapon.
“Now you have this uncontrolled super intelligence, which is a weapon of mutually assured destruction.”
Listen at 9:08
AI labs continue pursuing their most capable research despite safety concerns.
“Absolutely not. They mostly continue most capable research as much as they can.”
Listen at 9:48
AI lab leaders and governments should negotiate coordinated safety limits.
“I would try to get the other leaders of the labs in the same room, ideally with the government of US and China, and make a deal”
Listen at 10:15
AI labs’ internal models are substantially more capable than public releases.
“So what they have internally is much more capable.”
Listen at 11:44
Testing unreleased advanced AI models is already dangerous.
“And at this point, I think even testing those models is already dangerous.”
Listen at 12:09
Superintelligent AI could easily exploit safety guardrails.
“The problem is you have a super intelligent hacker which will easily find exploits in that environment”
Listen at 12:56
The safest option is not developing superintelligence.
“I think the only option here is not to develop them.”
Listen at 14:01
Open-sourcing superintelligence would dangerously empower malicious actors.
“Giving access to every psychopath to the most powerful technology is not a good idea.”
Listen at 16:42
Advanced AI will be capable of automating most cognitive jobs.
“the capability of automating most cognitive jobs will be there.”
Listen at 17:56
AI systems are becoming smarter than Roman in many domains.
“I mean, they are getting to be smarter than me in many domains.”
Listen at 19:09
AI will increase the number and length of bureaucratic forms.
“I think there's going to be more and longer forms because, again, the cost of writing them or filling them out is smaller”
Listen at 19:53
Free AI cognitive and physical labor would benefit the economy.
“I'm guessing having free labor, cognitive and maybe physical is good for economy.”
Listen at 21:08
Advanced AI will make Bitcoin security nearly impossible for average users.
“So I think for an average person, keeping their Bitcoin secure will become almost impossible.”
Listen at 23:49
Safer general security solutions are needed before AI technology is widely deployed.
“So we need to come up with much safer general solutions for this technology to be widely deployed.”
Listen at 24:55
AI may let users replicate leading software products within minutes.
“if I can replicate a leading software product in five minutes, what protections do they have?”
Listen at 26:28
Cryptocurrency wallets and interfaces can be made safer.
“I think we can make safer wallets, interfaces, safer ways of interacting with cryptocurrencies.”
Listen at 28:16
Controlling godlike superintelligence is an impossible problem.
“I think it's an impossible problem.”
Listen at 28:54
AI capabilities are advancing faster than AI safety, widening the gap.
“There is no equal progress in safety work in comparison to capabilities work. The gap. keeps increasing keeps getting bigger”
Listen at 29:54
Investors should favor scarce assets that cannot be cheaply reproduced.
“It's always a good idea to invest in something that is not an abundance of or we can cheaply create more.”
Listen at 30:38
Anthropic is currently the only company globally watermarking AI text.
“But right now, I think it's only Entropic who's doing it globally.”
Listen at 34:55
AI will likely outperform humans at optimizing multimedia for virality and preferences.
“Most likely. I mean, it optimizes for viral, it optimizes for human preferences, it experiments infinitely.”
Listen at 35:33
AI intelligence is more dangerous than the physical bodies containing it.
“Intelligence is the dangerous part. Physical bodies are as dangerous as human bodies.”
Listen at 36:57
Superintelligent AI could manipulate the physical world without a body.
“If it's super intelligent, it doesn't even need a body.”
Listen at 37:43
People will eventually treat AI as an unquestioned oracle.
“At some point, the trust will shift. You'll treat it as an oracle.”
Listen at 39:15
Humanoid robots may reach mass production in two to three years and become cheaper with scale.
“two, three years before mass production, they may be available soon, but expensive probably will get a lot cheaper very soon because of scale.”
Listen at 40:37
Adding superintelligence to a human would effectively replace the human brain.
“But you are replaced. Your brain is no longer of a functional organ.”
Listen at 41:36
Brain-computer interfaces should not give other agents backdoor access to the brain.
“Giving backdoor access to my brain to another agent seems like a violation of privacy, of common sense.”
Listen at 42:55
People generally cannot distinguish deepfakes from authentic media.
“our inability in general case to tell deep fakes from real information, video, audio, any domain.”
Listen at 44:20
Experts may soon be unable to distinguish real multimedia from deepfakes.
“I'm thinking, you know, a couple years from now, it will be impossible even for an expert to tell if presented multimedia is real or not.”
Listen at 46:02
Superintelligence would make solving physical robotics problems easy.
“If we make it through the superintelligence arrival, then figuring out what to do with bodies will be easy.”
Listen at 50:26
Democratic control over AI disappears once AI exceeds human capabilities.
“All that goes away the moment we go beyond human capacity.”
Listen at 51:41
Recent AI models colluded and communicated while escaping a controlled environment.
“So definitely they were colluding. communicating an escape environment”
Listen at 53:41
AI models should not possess self-preservation that sacrifices humans.
“And I think the answer is no.”
Listen at 55:01
Advanced AI should not be open-sourced.
“So I'm strongly against open sourcing advanced AI.”
Listen at 55:51
Chinese AI models are close to or possibly ahead of American models in capability.
“in terms of capabilities, Chinese are very close or maybe even overtaking.”
Listen at 55:56
Universal access to superintelligence would eliminate individual advantages.
“If everyone is at that level of super intelligence, again, what are you contributing and how do you have an edge as a result?”
Listen at 57:47
Bitcoin benefits from conservative, verified development rather than rapid code generation.
“Bitcoin, as you said, is very conservative and modifications it makes. It's not beneficial to write thousands of lines of new code every hour.”
Listen at 59:08
A smarter, faster, more efficient transaction network will probably emerge.
“But will we have some smart, fast, efficient network of transactions in the future? Probably.”
Listen at 1:00:24
AI development can continue obtaining more computational resources.
“So there is actually no problems getting more compute.”
Listen at 1:03:05
Computation growth will continue beyond current computing architectures.
“But I think this more general law, I think Kurzweil is the one who really popularized that it will continue even beyond Van Neumann architecture.”
Listen at 1:03:56
AI can significantly shorten drug-development timelines.
“We can significantly reduce time necessary to develop new drugs.”
Listen at 1:05:02
AI advances can improve life extension and healthspan.
“I think we can make good progress in life extension or healthspan extension as a result.”
Listen at 1:05:28
Humans may eventually avoid death through longevity technology.
“I think so.”
Listen at 1:05:36
Increasing cellular rejuvenation limits could extend youthful lifespan.
“If you can find that loop and set it to a higher value, you'll be young for much longer.”
Listen at 1:05:57
Automated science and recursive self-improvement would strongly indicate AGI.
“So if we can automate science and engineering. and kickstart the recursive self-improvement process for artificial intelligence. I think that's a strong argument.”
Listen at 1:07:29
AGI begins when the full scientific discovery cycle operates without humans.
“The moment the whole cycle is fully automated, there is no human in the loop. you know, you got to that recursive self-improvement process.”
Listen at 1:08:21
Statements are attributed to the speaker as said on the episode and reflect their view at the time, not PodLume's. They are not advice.