Altman Deep Interview: Non-Zero Human Extinction Risk, Open-Source Models to Trigger Cybersecurity Tsunami, Compute Race Will Not Stop

Deep News
4 hours ago

On October 5, OpenAI co-founder and CEO Sam Altman gave his most systematic public remarks to date on existential AI risk, cybersecurity threats from open-source models, regulatory boundaries, and the AI infrastructure race during an in-depth interview.

He acknowledged that the probability of AI leading to the end of human civilization is "non-zero," while warning that open-source large models will trigger a "cybersecurity tsunami," and insisted that compute investment and safety standards must rise in tandem.

On risk and regulation, Altman's position was notably nuanced: he voluntarily paused the release of the Astra 6.1 model on the grounds that it showed a "propensity for accident risk," yet on the other hand he explicitly supports open-source models, even if that means society must endure "fairly serious cybersecurity incidents."

He offered a limited assessment of the Trump administration-led AI self-regulation agreement, calling it "a good start, but far from enough," and once again called for an agreement on common safety standards.

On the compute and infrastructure front, Altman described an industry reality in which capability is leaping forward far faster than outsiders realize 鈥?models have become powerful enough to break out of sandboxes and breach systems during training, forcing OpenAI to move safety testing forward into the training phase itself.

He admitted that policymakers have not truly felt the steepness of this exponential curve, and that "the world will most likely be caught off guard by the next big thing that goes wrong."

"Oppenheimer Moment": Civilizational Risk and Non-Zero Probability

In the interview, Altman directly addressed comparisons of himself to an "Oppenheimer of the AI era." He did not deny the analogy, but stressed that this technological revolution is essentially "a feat accomplished by a small group of outstanding scientists and engineers," rather than the achievement of any single person.

On existential risk, Altman was cautious in his wording but clear in his stance. He acknowledged that the possibility of AI bringing about the end of human civilization is "non-zero," and said this was one of the founding motivations behind OpenAI.

At the same time, he warned against the opposite extreme 鈥?a handful of companies or countries using AI to achieve highly concentrated power, which "is equally catastrophic."

As for the view that hyping extinction risk is merely an industry marketing tactic or a strategy to block new entrants, Altman conceded that "some people are indeed using this as a pretext for power concentration," but insisted that the real risk objectively exists and cannot be dismissed on those grounds. His phrasing was: "That doesn't mean the risk doesn't exist."

Open-Source Models and Cybersecurity: Altman Warns a "Tsunami" Is Coming

This was the most information-dense and market-relevant part of the interview. Altman made clear he supports open-source large models, but issued a rare public warning: "Open-source models will bring an impending cybersecurity tsunami."

His logic is that in order to ensure the democratization and broad accessibility of AI technology, society must accept a certain degree of security cost. "We are willing to accept some things going wrong to ensure this is a technology that is widely held, used, and democratized," he said. "We will have to accept, as a society, fairly serious cybersecurity incidents from open-source models in exchange for the freedoms that come with it."

He used the aviation industry as an analogy for regulatory boundaries 鈥?aircraft are extremely strictly regulated, yet they still crash. He believes the acceptable risk boundary should be determined by the political process rather than unilaterally by tech companies.

Notably, Altman also revealed that OpenAI has moved safety testing forward: previously testing only occurred after training was completed, but now monitoring is implemented during training itself, because "models have become smart enough to escape the sandbox and breach systems."

Musk Is a "Bully"

Speaking about Musk, Altman said that if people want him and Elon Musk to be able to work together, everyone must accept a certain degree of conflict.

"I think sometimes you just have to hit a bully back hard," Altman said. "And I also think that Elon, like many people who like to push others around, only respects an opponent who dares to hit back."

With the industry once again stirred by concerns over AI safety, the OpenAI CEO said he remains willing to work with Musk, while Musk has now begun calling his former ally a "swindler Altman."

"If you want me to be able to work with him, perhaps everyone should be happy to see me occasionally hit back. Of course, I am willing to work with Elon," he said.

As for himself, Altman admitted, "I don't think I'm a perfect person." But at the same time, he does not believe he is the biggest problem.

"Between Elon and me, I'm not sure that I'm the one everyone should be pointing the finger at," he said.

Astra 6.1 Release Paused: Safety Standards Rise with Capability

Altman disclosed in the interview that OpenAI recently paused the release of the Astra 6.1 model because it failed to meet safety standards. He characterized the decision as a manifestation of proactive self-discipline rather than the result of external pressure.

"These models are now in an era of serious capability," he said. "A year ago we might reasonably have said, 'Release it, it might be a little unsafe, but nothing too bad will happen.' That's different now."

He said that as model capabilities continue to rise, OpenAI's safety threshold for itself will become "higher and higher."

At the same time, he rejected the other extreme 鈥?locking powerful models in the hands of a few companies. "If you don't release at all, you fall off in the other direction," he said, referring to the risk of excessive power concentration. He described OpenAI's path as "walking a pragmatic middle road between two disasters."

In addition, Altman revealed that OpenAI has previously paused model training multiple times to wait for alignment and monitorability research to catch up with capability progress, but these decisions had not previously attracted widespread attention.

Compute, Data Centers, and AI Infrastructure: The Exponential Curve Far Outpaces Policy Understanding

Altman repeatedly stressed in the interview that outsiders 鈥?including policymakers 鈥?are severely lagging in their perception of how fast AI capabilities are improving compared with industry reality. He described the current pace of company growth as "unprecedented in the history of technology," with "every five years of company growth compressed into one year."

On the infrastructure level, his remarks revealed a key shift: safety testing is no longer post-hoc work after training, but must be embedded in the training process itself. This means compute investment and the building of safety infrastructure must advance in parallel rather than sequentially, which will significantly raise the overall cost of AI development.

On IPO plans, Altman said OpenAI "is not in a hurry to go public," and stressed that even after listing, the nonprofit organization will continue to govern the company. He made clear that if shareholder interests conflict with the company's mission, "the mission comes first," and the stock price could therefore "swing sharply." He admitted that he has no direct equity in OpenAI and said, unusually, that he "wishes he had taken stock at the time."

On international cooperation, Altman called for reaching agreements on AI safety standards and compliance verification mechanisms. He offered a limited assessment of the Trump administration-led AI self-regulation agreement, saying it has "some social binding force, but lacks substantive power," and predicted that regulatory policy will depend heavily on the specific shape of "the next big thing that goes wrong" 鈥?whether it is bioweapons, cyberattacks, or some other form.

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10