OpenAI Charts Path to Safeguarding Against AI Catastrophes

NEW YORK: ChatGPT developer OpenAI on Monday updated its guidelines for assessing the “AI catastrophe risk” of artificial intelligence in model development.

The announcement comes a month after CEO Sam Altman was fired by the company’s board of directors, only to be rehired days later after an uproar from employees and investors. According to US media reports, board members criticized Altman for his support of accelerated development of open AI, even if that meant distancing himself from some questions about the technology’s potential risks.

In a “preparedness framework” released Monday, the group said it “believes that scientific research on the catastrophic risks of AI is far behind where it should be.” The framework “will help bridge this gap,” the group said.

The monitoring and evaluation team, announced in October, will focus on “state-of-the-art models” that have better capabilities than the most advanced AI software currently being developed. The team will evaluate each new model and assign a risk level ranging from “low” to “severe” in four main categories.

According to this framework, only models with a risk rating of “moderate” or lower can be developed. The first category is concerned with cybersecurity and the model’s capacity to conduct large-scale cyberattacks.

The second category measures the software’s tendency to contribute to the creation of something harmful to humans, such as chemical mixtures, organisms (like viruses), or nuclear weapons.

The third category concerns the model’s potential, such as its ability to influence human behavior.

The last category of risk is related to the model’s potential autonomy, i.e. whether it can escape the control of the programmer who created it.

Once a risk has been identified, it is presented to the OpenAI Security Advisory Group, and the OpenAI leadership determines the changes to be made to the model to mitigate the associated risk. The Board of Directors is notified and can overrule the leadership’s decision.

We’re on Telegram, Facebook, and Twitter too! If you want to stay connected with us, be sure to follow us. We’d love to stay in touch!