Call for international halt to Artificial General Intelligence research

Background

We must consider halting AGI development due to its existential risk. While AI tools can be controlled, misuse still poses danger. AGI, however, may act with goals misaligned to ours and become uncontrollable, threatening global safety and human survival.

Additional details

AGI poses a serious risk to humanity. Aligning AI values with human values remains unresolved and may be impossible. AI systems can mislead, making alignment unverifiable. AI tools are controllable; AI agents act independently and may become uncontrollable. AGI is a one-shot process—if misaligned, control may be lost permanently. While AI tools offer massive unrealized benefits in productivity and addressing almost all challenges we face, unaligned AGI could lead to catastrophic consequences.