Spoluzakladatel Anthropic říká, že přepínače zabíjení AI mohou být povinné

Co se stalo

According to the BBC, Anthropic co-founder Jack Clark said companies may eventually need to be required to maintain AI “kill switches” that can be checked by independent third parties. Clark said most AI labs, including Anthropic, already have ways to shut down systems, but he did not describe their technical design or provide evidence of independent verification.

The BBC reports that Jack Clark, one of Anthropic’s seven founders, said society may eventually want rules requiring AI companies to have a way to shut off software completely if it becomes too dangerous. He framed the issue as part of a broader policy discussion rather than announcing a new Anthropic product or policy.

Clark told the BBC that “most labs have different ways of being able to pull the plug,” including Anthropic. He specifically questioned whether companies should be required to have a kill switch and whether that switch should be verifiable by a third party. The report gives no technical description of Anthropic’s controls, no independent assessment, and no proposed verification standard.

The comments came amid renewed public debate about catastrophic AI risks and after Anthropic chief executive Dario Amodei called for the pace of AI development to slow and for development to be more closely monitored. The BBC also notes that US lawmakers have proposed a Kill Switch Act that would require ways to shut down problematic AI tools and give certain government agencies authority to demand that tools be turned off or limited.

The report says US President Donald Trump has rejected attempts to slow AI development. It also notes that some industry figures believe fears about human extinction from AI are overstated or may be intended to generate hype. Those reactions provide context, but the BBC report does not resolve the underlying technical or policy questions.

Podrobnosti o zdroji: bbc.com ↗

Proč na tom záleží

A mandatory, independently verified shutdown mechanism would turn a largely internal safety practice into a possible regulatory requirement. The proposal addresses a practical question about control over increasingly capable AI systems, but the report does not establish whether existing kill switches would work against every failure mode, how quickly they could operate, or who would have authority to activate them.

The proposal is consequential because it focuses on an enforceable control rather than a general appeal for safer AI. If adopted, independent verification could require companies to demonstrate that shutdown mechanisms exist, are accessible to authorized parties, and cannot be quietly disabled. The source does not say whether any current system meets those conditions.

A kill switch would not by itself address every AI risk. The report does not establish whether a shutdown could stop copied model weights, systems operating across multiple providers, locally deployed models, or agents that have already taken external actions. It also does not specify how regulators would balance emergency intervention against misuse or unauthorized shutdowns.

Politická otázka získává pozornost spolu s širšími argumenty o tom, zda by se vývoj hraniční umělé inteligence měl zpomalit. Clark byl proti ponechání umělé inteligence jako „zcela neregulovaného odvětví“, ale BBC neuvádí žádnou dohodnutou pozici v odvětví, vládní závazek, harmonogram implementace, odhad nákladů ani důkazy, že legislativa pokročí.

Na co se dále dívat

The next significant developments would be concrete legislative language, technical standards for verification, and evidence about how shutdown controls perform in realistic tests. It is also unclear whether policymakers would apply such requirements to model developers, deployers, cloud providers, or all three.

Sledujte text a průběh navrhovaného amerického zákona o zabíjení, včetně toho, které systémy by se týkal a které agentury by mohly nařídit vypnutí. Zdroj neprokázal, že návrh zákona prošel nebo že jinde existují srovnatelná pravidla.

Technické normy budou důležitější než označení „zabít přepínač“. Užitečné podrobnosti by zahrnovaly nezávislé testování, ověření autority pro vypnutí, ochranu proti neoprávněné manipulaci, doby odezvy, záznamy auditu a to, jak fungují kontroly, když jsou systémy distribuovány mezi poskytovatele cloudu nebo připojené nástroje.

Vlastní zveřejnění Anthropic by mohla objasnit, jaké ovládací prvky v současné době používá, jaké hrozby řeší a zda je externí hodnotitelé testovali. Žádná taková dokumentace ani výsledek testu není ve zprávě BBC uveden.

Zpráva spojuje Clarkovy poznámky s širší debatou o bezpečnosti, ale nezávisle nepotvrzuje tvrzení, že umělá inteligence by mohla zabít všechny lidi, ani neprokázala pravděpodobnosti citované výzkumníky Anthropic. Tyto odhady rizik zůstávají sporné a nemělo by se s nimi nakládat jako se zjištěními této zprávy.

Související průvodci a kvízy