Daniel Kokotajlo, a researcher at OpenAI, resigns citing concerns over the company’s handling of AI security issues and disagreement over a non-disparagement agreement, sparking a conversation about regulations for AGI companies.
In April, Daniel Kokotajlo, a researcher at OpenAI, resigned, citing concerns over the company’s handling of security issues related to artificial intelligence. Kokotajlo, who worked in policy and governance, expressed his worries on the online forum “LessWrong,” stating he lost confidence in OpenAI’s ability to responsibly manage the potential risks of AI technology.
Kokotajlo criticized OpenAI’s culture for adopting a “move fast and break things” approach, which he deemed inappropriate for such powerful and misunderstood technology. Furthermore, he revealed that OpenAI pressured him to sign a non-disparagement agreement, threatening the loss of his vested equity, worth $1.7 million, if he refused. Kokotajlo declined to sign, choosing to retain his freedom to voice concerns publicly.
Following Kokotajlo’s departure, OpenAI’s CEO Sam Altman issued an apology, stating he was unaware of the situation and should have known. However, the company’s practice of requiring such agreements from employees, who might lose their equity if they did not comply, has reportedly been ongoing for years.
This event has sparked a larger conversation about the risks and regulations surrounding artificial general intelligence (AGI) companies. Current and former OpenAI employees have advocated for a “Right to Warn” pledge, urging companies to revoke non-disparagement agreements, create anonymous reporting mechanisms for safety concerns, and support a culture of open criticism.
The pledge highlights the lack of effective regulation for AGI companies and calls for systematic protections to ensure employee warnings about safety risks are heeded.
