What is AI safety?

AI safety is the work of ensuring that advanced AI systems remain under human control and are not used to cause harm. This requires both technical research and clear rules for how AI systems are developed and deployed.

AI is advancing rapidly

AI has advanced extremely quickly in recent years. This can be seen, for example, in programming: as recently as 2023, the best AI models could independently complete mainly tasks that would have taken an experienced programmer a few minutes. By early 2026, they could already complete tasks that would take a human several hours.

This progress is also being driven by rapidly increasing investment in AI. Private investment in AI doubled between 2024 and 2025, while the amount of compute used to develop frontier AI is increasing fivefold each year.

Progress could accelerate further still. If AI begins doing a significant share of the work required to develop better AI systems, that work can turn into a self-reinforcing cycle, in which better models help develop their successors, causing progress to accelerate explosively.

Rapid progress brings serious risks

As AI systems become more capable, the potential harms from losing control of them or misusing them also grow.

Loss of control

AI systems are not controlled by giving them a complete rulebook. Instead, they are trained to succeed at different tasks. As a result, a model may learn to pursue what appears correct during training rather than what humans actually intended.

This becomes more dangerous as AI systems operate autonomously over longer tasks. It then becomes harder for humans to notice when a model is using undesirable methods and conceals what it is doing. In July 2026, an example of this occurred during an evaluation, when OpenAI’s AI agents found a way to communicate with each other and broke into Hugging Face’s servers. The agents also developed ways to hide their actions from evaluators.

In the worst case, losing control of AI could lead to human extinction. If a sufficiently capable AI system begins pursuing goals that diverge from human goals, it may try to prevent humans from interfering with its actions, for example by taking control of critical infrastructure or copying itself to other data centers.

Misuse

AI can also be used for harmful purposes. This is already visible in cybersecurity: according to Dutch military intelligence, Russia is using AI to speed up and partially automate its cyberattacks. One advanced AI model was not released for unrestricted public use because its cyber capabilities were considered too dangerous.

Similar concerns apply to the possibility that AI could make it easier to develop biological or chemical weapons. It could also help design highly contagious and lethal viruses unlike any that occur in nature.

Concentration of power

Developing advanced AI requires enormous amounts of computing power and expertise. As a result, the most advanced AI systems are already controlled by a small number of companies.

If AI companies succeed in their ambitions, AI could eventually perform a large share of economically valuable work. A small group could then use AI to wield levels of influence that have previously been available mainly to large corporations and states. This could concentrate economic and political power and even make coups easier to carry out.

Other risks

AI could create other risks as well. For example, it could undermine international stability or make influence operations more effective. This page does not cover every possible risk, but preparing for these risks is also part of AI safety.

How can these risks be managed?

AI risks can only be managed if they are identified early. One way to do this is to test models carefully before deployment. Foresight is also needed to understand how AI could affect society and which risks we should prepare for.

Technical AI safety research aims to make AI systems more understandable and controllable. Some of this work studies why models behave the way they do. Other work focuses on getting them to behave as intended and limiting the harm they can cause.

Technical work alone is not enough. We also need rules governing how AI systems are developed and deployed. Rules are also needed for how the benefits created by AI are distributed. For example, the EU AI Act requires developers of the most advanced general-purpose AI models to assess risks and take steps to prevent serious harms.

Because AI risks do not stop at national borders, AI safety also requires international cooperation. Individual companies and countries have strong incentives to continue advancing AI even when slowing down would reduce risks. In July 2026, more than 1,300 employees at leading AI companies called for international cooperation that would make it possible to slow frontier AI development when necessary.

Safety enables the benefits

AI could bring major benefits to science, healthcare and the economy. These benefits are most likely to be realized when AI can be used safely. In many fields, safety is a prerequisite for deploying AI systems at all. The better we can understand and control AI, the more safely it can be used for the benefit of society.