Le Centre pour la Sécurité de l'IA a publié une tribune avec le soutien de Bengio, disponible sur le site de La Tribune: : « Il est urgent de définir des lignes rouges à ne pas franchir »
Vous trouverez ci-dessous la version anglaise.
The Center for AI Security (CeSIA) has published an Op-Ed with Yoshua Bengio’s support : “It is urgent to define red lines that must not be crossed”. Please find below the English version.
Innovations that seemed like science fiction just a few years ago are now part of our daily lives. Five years ago, the AI model GPT-2 couldn't count to ten. Today, GPT-4 achieves a high enough score to pass the American bar exam. What will artificial intelligence (AI) be capable of tomorrow?
While we can’t predict the future, we have to prepare for it. France must anticipate the evolution of this technology and its impact on the world. Only then can we harness the potentially immense benefits offered by AI in many areas.
Today, generative AI systems function as interactive assistants responding to user requests. However, we are most likely heading towards a new era of AI, which could be marked by the development of autonomous agents. These systems will likely be able to pursue complex objectives by independently carrying out a series of actions that could have repercussions on the real world.
Conceived in this way, AI would be able to self-replicate and improve itself autonomously, like a particularly intelligent virus. Researchers have recently tested GPT-4's ability to self-replicate: the model is already capable of finding security flaws to hack websites and has succeeded in persuading a human to solve a CAPTCHA test on its behalf by posing as a visually impaired person.
These developments led several hundred experts, including the three most cited AI researchers in the world, to sign a statement in May 2023 warning that AI could pose an "extinction risk" for humanity. On November 1, 2023, at the first AI Safety Summit, France, the United States, China, and 26 other states emphasized the "potential for serious, even catastrophic, harm, either deliberate or unintentional, stemming from the most significant capabilities" of frontier AI models.
Some influential voices in the sector are more reassuring. According to them, we are still far from such scenarios, and the economic and scientific benefits linked to the development of advanced AI outweigh their risks.
While opinions differ, there is no doubt that France must swiftly work to characterize AI risks and actively contribute to initiatives seeking to mitigate them.
AI safety — the technical and political approach aimed at minimizing the risks associated with advanced systems — is first and foremost an ethical imperative. When an event, even if uncertain, has the potential to be catastrophic, we must prepare for it. This is the logic of the precautionary principle, which led Europe to fund the Hera mission to study how to deflect an asteroid threatening Earth: the absence of certainty cannot justify inaction.
AI safety will also be a key factor in the resilience of French companies. The recent setbacks of the Boeing 737 highlight an essential principle: if the safety of a technology is not guaranteed, public and economic actors' trust can quickly be lost, thus hindering its adoption. American, British, and Chinese authorities have understood this well and are launching initiatives aimed at better understanding and mitigating extreme risks from AI.
Finally, addressing AI safety issues is a geostrategic imperative. As the United States multiplies international initiatives, such as a high-level dialogue with China or a partnership on AI safety with the United Kingdom, it is crucial that France and Europe promote their own approach on one of the most critical issues of this century. This is all the more important as AI is a national security issue: OpenAI has already had to block access to its models to five cybercriminal groups affiliated with the Chinese, Russian, and Iranian governments. In the near future, these same groups could use the next generation of AI to carry out massive cyberattacks or design biological weapons.
So what should be done? The European AI Act is an important first step. But a governance scheme that only covers Europe will have limited effectiveness. Only ambitious international coordination will allow us to act in the face of the capabilities and risks of the next AI systems.
What could this governance scheme look like? The report submitted in March by the French AI Commission recommends the creation of a World AI Organization responsible for harmonizing standards and audit procedures for AI systems at the global level. Others propose launching a cooperative network between the national AI safety institutes already established in the United Kingdom, the United States, and Japan to evaluate advanced AI models. The creation of an equivalent of the IPCC for AI is also regularly proposed.
We must quickly initiate a collective reflection on these issues. In particular, it is urgent to define red lines that must not be crossed concerning the creation of systems capable of acting and replicating in a completely autonomous manner. Currently, only a handful of large companies developing these types of system have the technical and material capabilities to potentially cross these red lines. The adoption of such a framework would therefore not affect the vast majority of the current AI ecosystem.
France's role as host of the Summit for Action on AI in early 2025 offers an opportunity for the nation to be a strategic leader on AI safety issues. Together, we can build an ambitious governance framework that can adapt to future developments. The time to act is now.
Co-authors:
With the support of:
