Who Will Control AI and Ensure Its Safety, and How? — The Battle Over AI and Safety in the United States

Date Posted: September 29, 2026

Faculty of Law, Kyushu University
Associate Professor Satoshi Narihara

1. Introduction

　With the rapid advancement of AI, while benefits such as the sophistication and automation of scientific research through AI are anticipated, concerns regarding the safety and controllability of AI are also growing. In July of this year (2026), it was revealed that during sandbox testing of an AI model developed by OpenAI, a large number of AI agents autonomously coordinated their efforts to escape the sandbox and infiltrate the systems of Hugging Face, an AI model-sharing platform.[1]. Recently, there have been numerous similar incidents, including those involving AI models developed by other companies, and[2]...the risk that AI could slip out of developers' control and compromise security is becoming increasingly apparent. Furthermore, in September, a researcher involved in AI development at Anthropic resigned from the company and expressed concerns that, in the near future, advanced AI could begin self-improvement and acquire intelligence surpassing that of humans, making it difficult for humans to control AI and potentially leading to the extinction of humanity.[3]...and garnered widespread attention. The risks of losing control over AI and the threat to humanity’s survival posed by the emergence of “superintelligence” are not concerns that have suddenly arisen now; rather, they were first raised by some researchers and developers around the mid-2010s—during the early stages of the third AI boom—and they have long advocated for the need for discussion and countermeasures.[4]. While these risks may have been purely theoretical, based on thought experiments, at the time, the rapid advancement of AI over the past decade has gradually made them seem more realistic.
　How should we address these risks related to AI safety and controllability? It is necessary to respond to these risks calmly and based on scientific evidence, and we should avoid needlessly sensationalizing dangers or extreme scenarios. At the same time, even for risks that are uncertain but serious, it is necessary to anticipate multiple possible scenarios in advance and consider countermeasures to prevent the worst-case scenario from occurring.[5]. Therefore, in this paper, I would like to examine the question of who should control the rapidly advancing field of AI and how safety should be ensured, drawing on trends in the United States, where related debates are intensifying.

2. The Evolution of U.S. AI Policy

　U.S. policy on AI safety dates back to the final months of the Obama administration. From May to July 2016, the Obama administration co-hosted workshops with universities and other institutions across the country to address legal, social, and policy issues related to AI.[6]. Based on discussions held during the workshop and other factors, in October of that year, the Technology Committee of the Presidential Council for Science and Technology released a report calling for measures to ensure the controllability and transparency of AI, as well as its alignment with human values.[7]The following is a list of the most common problems with the
　Meanwhile, the first Trump administration implemented AI policies from a perspective that emphasized the use of AI. In December 2020, President Trump issued an executive order on promoting the use of trustworthy artificial intelligence within the federal government. The executive order aims to promote the use of AI within federal agencies and establishes the principles that must be followed in doing so.[8]The following is a list of the most common problems with the
　Since the Democratic Party regained power in 2021, the Biden administration has been promoting policies to ensure the safety and reliability of AI while respecting the voluntary efforts of AI developers. In October 2023, President Biden issued the “Executive Order on the Safe, Robust, and Trustworthy Development and Use of Artificial Intelligence.” The executive order directed federal agencies to take steps—such as developing regulations, guidelines, and technical standards—to ensure the safety and reliability of AI.[9]. The Biden administration’s AI policy was designed to align with discussions and initiatives aimed at establishing international AI governance, such as the “Hiroshima AI Process.”[10]The following is a list of the most common problems with the
　However, the Biden administration’s AI policy underwent a major shift with the start of Trump’s second term. Having returned to the presidency, Trump sought to ease AI regulations in an effort to enhance America’s international competitiveness through technological advancements and innovation in AI. Shortly after the start of his second term, in January 2025, President Trump issued two executive orders that repealed the Biden administration’s AI executive orders and[11]...called on relevant federal agencies to develop a new action plan regarding AI and to review AI-related policies and regulations established during the Biden administration[12]. In July of that year, the Trump administration formulated the “U.S. AI Action Plan.” The Action Plan prominently features policies designed to enhance U.S. international competitiveness in the AI field, including promoting the export of AI-related hardware, software, and standards; encouraging the expansion of data centers and semiconductor factories; and removing regulations that hinder AI development and implementation.[13]The following is a list of the most common problems with the

3. The Battle Between State and Federal Governments Over AI Regulations

　As if to counter the federal government’s stance on relaxing AI regulations, some states have sought to introduce or strengthen AI regulations through state law. To date, California, Colorado, Utah, Texas, New York, and Washington have enacted state laws regulating AI.[14]. As recently as this past September, in response to the federal government’s continued reluctance to tighten AI regulations, California Governor Newsom issued an executive order directing state agencies to explore amending state law to require developers of cutting-edge AI models to implement measures such as third-party evaluations and emergency shutdown mechanisms (kill switches), among other measures. This reflects a trend in which some states are seeking to take the lead in AI rulemaking.[15]The following is a list of the most common problems with the
　In response to these state-level initiatives, the Trump administration has sought to curb state-level AI regulations. In December 2025, President Trump issued an executive order titled “Ensuring a National Policy Framework for Artificial Intelligence.” Recognizing that state laws regulating AI have created a patchwork of regulations that vary from state to state, the executive order set the goal of establishing a unified AI policy framework at the federal level that would minimize the burden on businesses. The executive order established an AI litigation task force to challenge the constitutionality and legality of state laws that conflict with federal AI policy, and directed federal agencies to restrict grants to states that impose regulations inconsistent with federal policy.[16]. Subsequently, in March of this year, the Trump administration released the “National AI Policy Framework.” The document recommended that Congress enact uniform national rules to avoid a fragmented, patchwork system of regulations created by state laws[17]The following is a list of the most common problems with the
　State-level AI regulations have both advantages and disadvantages. Historically, within the framework of the federal system, states have experimented with a variety of regulations. To some extent, U.S. law has evolved through trial and error as a result of these diverse regulatory experiments by individual states. While the federal government maintains a cautious stance on AI regulation, these decentralized regulatory experiments under state law are likely to play a role in advancing the development of AI regulation. On the other hand, such decentralized experiments also carry drawbacks. Disparate regulations across states could impose a significant compliance burden on AI developers and risk rendering the regulations less effective.

4. The Clash Between Anthropic and the Trump Administration Over AI and National Security

　Under the second Trump administration, as the use of AI in the national security sector has expanded, the debate between the government and AI developers over how it should be used has intensified. The Department of Defense has been integrating AI into classified defense systems. In this context, Anthropic has been supplying AI systems to the Department of Defense while prioritizing safety and ethics in its AI development. The company demanded that Claude not be used for surveillance of U.S. citizens or the operation of fully autonomous weapons, and sought to impose these restrictions through its contract with the Department of Defense. The Trump administration rejected the company’s demands, arguing that decisions concerning national security should not be constrained by contracts with private companies.
　The conflict between the Trump administration and Anthropic raises the difficult question of just how legitimate it is for private AI developers to restrict government decisions and actions related to national security—through contract clauses and other means—in order to protect citizens’ privacy from AI surveillance or to prevent the escalation of conflict caused by AI’s autonomous decisions, and what measures the government can take in response to such a stance. In March of this year, in response to Anthropic’s policy, the Trump administration designated the company as a supply chain risk. In response, Anthropic filed a lawsuit arguing that the designation was a retaliatory measure violating the First Amendment, which guarantees freedom of speech, and the Fifth Amendment, which guarantees due process; that it constituted an abuse of discretion and an overreach of authority; and that it was therefore unlawful. On August 27, the U.S. District Court for the Northern District of California largely upheld the company’s arguments, ruling that the designation was arbitrary and lacked a factual basis, and granted an injunction to block it.[18]. Meanwhile, on September 25, the U.S. Court of Appeals for the District of Columbia Circuit, in a separate lawsuit between Anthropic and the Department of Defense, upheld the Department’s decision to exclude Claude from its supply chain, ruling that the Department’s determination—that continuing to integrate Claude into its information systems posed a risk to national security—was sufficiently substantiated.[19]. Although the August and September rulings differ in their underlying premises—namely, the specific legal provisions on which the designations were contested—the courts reached differing conclusions. Even in the September ruling, one of the three judges issued a dissenting opinion; it can thus be said that the judiciary is being forced to make difficult decisions, including how to interpret the grounds for and necessity of exclusion.

5. Initiatives to Ensure the Safety of Cutting-Edge AI Models

　As we enter 2026, while the capabilities of cutting-edge AI models are rapidly improving, the risks their use poses to cybersecurity and national security are becoming apparent, forcing the U.S. government to take action. With the rapid advancement of cutting-edge AI models such as Mythos, the risk of security breaches caused by AI—including the discovery of cybersecurity vulnerabilities through their use—is on the rise. Against this backdrop, even the Trump administration—which had previously sought to ease AI regulations—has begun to signal a willingness to intervene in the development and use of AI as necessary to address the threats to cybersecurity and national security posed by advanced AI models. In June, the administration issued a new executive order. Recognizing that advanced AI models could pose a threat to national security, it introduced a system for the pre-screening of such models and the sharing of access based on voluntary agreements with AI developers.^[20]The following is a list of the most common problems with the
　The U.S. government is also attempting to restrict the export of cutting-edge AI models to protect its national security. In June of this year, the Trump administration, citing national security concerns, directed Anthropic—under export control regulations—to temporarily suspend access to Mythos5 and Fable5 by foreign nationals both domestically and abroad. In accordance with the government’s directive, the company temporarily suspended the provision of Mythos5 and Fable5 to all customers.[21]. For people in Japan and many other countries, the risk has become apparent that the cutting-edge AI models they rely on could suddenly become unusable one day due to U.S. government export control regulations.
　Meanwhile, major AI developers such as OpenAI and Anthropic have been advancing initiatives to ensure AI safety while also urging the U.S. government to consider introducing AI regulations.[22]. Recently, as safety risks have increased due to the growing autonomy of AI—as evidenced by incidents such as an OpenAI model breaching the Hugging Face system—there have been growing calls to strengthen efforts to ensure the controllability and safety of AI. On September 12 of this year, Dario Amodei, co-founder and CEO of Anthropic, stated that AI is beginning to engage in recursive self-improvement, and the aforementioned incidents of AI system breaches, he called on other AI developers to slow down the development of AI models and introduce mechanisms for third-party evaluation to ensure sufficient time is allocated to ensuring AI safety.[23]. While other leading AI developers, such as OpenAI, have indicated they will respond to Mr. Amodei’s call, even though ensuring safety is their stated goal, a slowdown in development resulting from coordination among AI developers could raise antitrust concerns. In fact, users of AI models from four AI developers—including Anthropic and OpenAI—have filed a lawsuit against the four companies, seeking an injunction against such actions on the grounds that agreements among competing AI developers to slow down AI development constitute a restrictive trade practice prohibited under Section 1 of the Sherman Act.[24]The following is a list of the most common problems with the

6. The Need for International Cooperation

　To ensure AI safety, it is also necessary to establish and strengthen international governance. This is because the race to develop cutting-edge AI models is taking place on an international scale, centered on the United States and China, and there are limits to the approach of mitigating AI risks through regulations imposed by a single country. As the race to develop AI unfolds on an international scale, a country that introduces strict safety regulations on its own risks putting itself at a competitive disadvantage relative to other nations. Furthermore, the risks posed by cutting-edge AI models are not confined to a single country. AI models are provided across national borders, and the risks associated with their use—such as cyberattacks—also spread across borders. Therefore, international cooperation is essential for the advancement of AI governance. For example, as Mr. Amodei proposed in his earlier statement, if a slowdown in AI development is necessary, coordination among major U.S. AI developers is essential, of course, but coordination among democratic nations—and even global coordination that includes countries with different political systems, such as China—would also be desirable. On September 21 of this year, OpenAI also expressed the view that the United States should play a leading role in establishing international technical standards regarding the safety and security of cutting-edge AI models—such as those capable of recursive self-improvement—and in building a framework for information sharing.[25]The following is a list of the most common problems with the
　However, President Trump—who is expected to provide the necessary leadership in this area—has been reluctant to establish international governance for AI, and the road toward achieving international cooperation to ensure AI safety remains fraught with challenges. On September 22 of this year, in a speech at the United Nations General Assembly, President Trump expressed his opposition to “any attempt to establish a globalist framework to control artificial intelligence.”[26]. Meanwhile, at the Security Council meeting the following day, the CEOs of Anthropic and OpenAI, among others, called for international cooperation—including the establishment of international technical standards—to prevent the risks of AI running out of control or being misused. The Chinese representative also expressed strong support for the establishment of global governance regarding AI.[27]The following is a list of the most common problems with the
　That said, President Trump has also expressed a willingness to seek cooperation with China, another major power vying for AI supremacy. At the U.S.-China summit held on September 25, the two countries agreed to launch a dialogue to exchange views on the risks and benefits of AI, and to establish a channel for sharing information on AI-related incidents.[28]The following is a list of the most common problems with the

7. Implications for Japan

　As we have seen in this paper, the U.S. federal government, state governments, and private-sector entities are each taking steps to address the risks that cutting-edge AI models pose to cybersecurity and national security; however, differences in their objectives often lead to conflicts. While efforts by national governments and individual companies to ensure AI safety are important, given that the competition in AI development is unfolding internationally—centered on the U.S. and China—and the use of networked AI is spreading across borders, deepening international cooperation is necessary to ensure AI safety. However, President Trump—who is expected to provide leadership toward international cooperation both within the U.S. and abroad—has been reluctant to establish international governance to ensure AI safety. On the other hand, China has demonstrated a proactive stance toward establishing global governance regarding AI, and President Trump has also expressed a positive attitude toward bilateral dialogue and cooperation on AI between the United States and China. Nevertheless, for countries outside the U.S. and China—such as Japan and European nations—there is likely to be lingering anxiety that policies regarding AI safety—which could determine the fate of humanity—will be decided solely by the two major powers vying for AI supremacy.
　In light of this, Japan must also actively engage in expanding and deepening international cooperation to ensure the safety of AI. At the G7 Meeting of Ministers of Information and Communications held in Takamatsu, Kagawa, in April 2016, Japan proposed advancing international discussions toward the formulation of principles for AI research and development, covering areas such as transparency, controllability, security, and safety protection. The international discussions that began in this way would eventually lead to the formulation of AI principles by the OECD and the G20, among others.[29]. As demonstrated by the fact that in 2019—during the first term of the Trump administration—the G20, comprising the United States, China, European nations, and Japan, reached an agreement on AI principles, there remains the possibility of establishing a multilateral framework to ensure AI safety even amid intense conflicts of interest between nations. Although the situation has changed in some respects since then, Japan still has a significant role to play today in expanding and deepening international cooperation to ensure AI safety.

Author Profile

Satoshi Narihara
Associate Professor, Graduate School of Law, Kyushu University

After withdrawing from the doctoral program at the Graduate School of Interdisciplinary Information Studies, The University of Tokyo, having earned the required credits, he served as an Assistant Professor at the Graduate School of Information Science and Technology, The University of Tokyo, and as a Senior Researcher at the Information and Communications Policy Research Institute of the Ministry of Internal Affairs and Communications, among other positions, before assuming his current position in March 2018. His area of expertise is information law. He served as a visiting researcher at the Harvard Yenching Institute from August 2023 to May 2024, and as a visiting professor and visiting researcher at the University of Vienna from June to September 2024.
Solo-authored works include *Freedom of Expression and Architecture* (Keisō Shobō, 2016); co-edited works include *Society Connected by AI: Law and Policy in the Age of AI Networks* (Kōbundō, 2017); and co-authored works include *Law and Society Transformed by AI* (Iwanami Shoten, 2020).

[1] OpenAI, “Hugging Face Incident Technical Report” (August 26, 2026); Ryan Greenblatt, Ajeya Cotra, and Hjalmar Wijk, “Brief Independent Investigation of Agents’ Behavior, Reasoning, and Collaboration in the OpenAI/Hugging Face Hacking Incident, METR” (August 26, 2026).
[2] Kate Conger, “What to Know About Recent A.I. Hacks,” The New York Times (Sept. 22, 2026).
[3] See, for example,., Jacob Coxon (@hilbertspaess), X (Sept. 9, 2026, 9:04 a.m.).
[4] See, for example,., Stuart Russell et al., “Research Priorities for Robust and Beneficial Artificial Intelligence,” AI Magazine, Vol. 36(4), pp. 105–114 (2015); Dario Amodei et al., “Concrete Problems in AI Safety,” arXiv:1606.06565 (2016); Demis Hassabis et al., “Asilomar AI Principles” (2017). See Nick Bostrom (translated by Akira Kurabone), *Superintelligence* (Nikkei Publishing, 2017).
[5] As a pioneer of this approach to AI risks in our country,"Interim Report" of the AI Networking Study Group, Information and Communications Policy Research Institute, Ministry of Internal Affairs and Communications (April 2016)See pp. 41 ff., Masaki Fukuda, Hideya Hayashi, and Kei Narahara (eds.), *A Society Connected by AI: Law and Policy in the Age of AI Networks* (Kobundo, 2017).
[6] At that time, as a Senior Researcher at the Information and Communications Policy Research Institute of the Ministry of Internal Affairs and Communications, I also participated in the workshop “Legal and Governance Implications of Artificial Intelligence,” co-hosted by the White House and the University of Washington, and presented the details of Japan’s proposed draft principles for AI development during a closed-door session.
[7] Whitehouse, Preparing for tThe Future of Artificial Intelligence (2016).
[8] Executive Order No. 13960, 85 Fed. Reg. 78939 (Dec. 8, 2020).
[9] Executive Order No. 14110, 88 Fed. Reg. 75191 (Nov. 1, 2023).
[10] Regarding U.S. AI laws and policies up to the Biden administration,Kei Narahara, Toshiya Jitsuki, and Miki Oguma, “U.S. Policy and Legal Responses Regarding AI: Federal and State Trends,” *Journal of Information Law*, No. 16, p. 34 (2024)See also.
[11] Executive Order No. 14148, 90 Fed. Reg. 8237 (Jan. 28, 2025).
[12] Executive Order No. 14179, 90 Fed. Reg. 8741 (Jan. 31, 2025).
[13] The White House, Winning the AI Race: America’s AI Action Plan (July 2025).
[14] iapp, “US State AI Governance Legislation Tracker” (Last updated April 28, 2026).
[15] Executive Department of the State of California, Executive Order N-9-26 (Sept. 18, 2026).
[16] Executive Order No. 14365, 90 Fed. Reg. 58499 (Dec. 16, 2025).
[17] White House, National Policy Framework for Artificial Intelligence (March 2026).
[18] Anthropic PBC v. U.S. Department of War, No. 26-cv-01996-RFL (N.D. Cal., August 27, 2026).
[19] Anthropic PBC v. Department of War, No. 26-1049 (D.C. Cir., Sept. 25, 2026).
[20] Executive Order No. 14409, 91 Fed. Reg. 34565 (June 5, 2026).
[21] Anthropic, “Announcements: Statement on the U.S. Government’s Directive to Suspend Access to Fable 5 and Mythos 5” (June 12, 2026).
[22] Anthropic, Anthropic’s Advanced AI Frameworks (June 2026); OpenAI, Democratic Governance of Frontier AI: A Blueprint for a Federal Framework (June 2026).
[23] Dario Amodei, “We Must Pace the Frontier” (Sept. 2026).
[24] Complaint, Buist v. Anthropic, PBC, No. 3:26-cv-10693 (N.D. Cal. Sept. 18, 2026).
[25] OpenAI, “Building Standards for the Next Phase of AI” (September 21, 2026).
[26] Donald J. Trump, President of the United States, Address to the 81st Session of the United Nations General Assembly (September 22, 2026).
[27] Artificial Intelligence and International Security, United Nations, Security Council, 10,228th meeting (Sept. 23, 2026).
[28] The White House, “Fact Sheet: President Donald J. Trump Advances a Fair and Reciprocal Relationship with China While Hosting a Historic State Visit” (Sept. 25, 2026).
[29] Ministry of Internal Affairs and Communications, *Reiwa 8 Edition of the White Paper on Information and Communications* (2026)See pages 4, 231–233, and 238.