사회 가이드

Frontier AI Safety Frameworks and Responsible Scaling Policies

A frontier AI safety framework, such as a responsible scaling policy, is a published commitment by an AI lab to test its models for dangerous capabilities.

4분 읽기
마지막 업데이트 2026년 9월 15일

읽기 시작 퀴즈 풀기

이 페이지에서4분 읽기

개요
심층 분석
전략적 영향
The Future of Frontier AI Safety Frameworks and Responsible Scaling Policies
실제 구현
위험 및 가드레일
구현 로드맵
계속 탐색하세요
자주 묻는 질문

개요

If a model crosses a set threshold, the lab commits to adding stronger safeguards, or to not deploying or not training further until those safeguards are in place. These frameworks matter because they are currently the main way the most advanced AI models are governed for catastrophic risk, though most of them are voluntary and written by the labs themselves.

심층 분석

Anthropic published the first Responsible Scaling Policy in September 2023. It introduced AI Safety Levels (ASLs), loosely modeled on biosafety levels, where higher levels require stronger security and deployment safeguards. A later version, in October 2024, restated the policy in terms of capability thresholds and required safeguards and described the role of a Responsible Scaling Officer. OpenAI released its Preparedness Framework as a beta in December 2023. It scored risk in tracked categories, originally including CBRN threats, cybersecurity, persuasion and model autonomy. A 2025 revision reorganized it around High and Critical capability thresholds and dropped persuasion as a tracked category. Google DeepMind published its Frontier Safety Framework in May 2024 and has revised it since. It defines Critical Capability Levels (CCLs) that trigger response plans. At the AI Seoul Summit in May 2024, 16 companies signed the Frontier AI Safety Commitments. They agreed to publish safety frameworks, set thresholds for intolerable risk, and not develop or deploy a model if its risks could not be kept below those thresholds. Many frameworks were published before the Paris AI Action Summit in February 2025. Critics point to several weaknesses. Labs write their own thresholds and can revise them on their own. Wording such as "meaningful uplift" leaves room for interpretation. External verification is limited, and commercial competition gives labs an incentive to read results generously. A common misconception is that these frameworks are laws. Most are voluntary, although this is changing. California's SB 53 requires large frontier developers to publish such a framework, and the EU's General-Purpose AI Code of Practice, a voluntary route to showing compliance with the AI Act, asks signatories to adopt one.

기술적 통찰력

Most frameworks follow an if-then structure. If evaluations show a capability threshold has been reached, then specific safeguards must be in place before training continues or before deployment. Evaluations include automated benchmarks, expert red-teaming, uplift studies that compare novices with and without model access, and agentic task suites. A central difficulty is elicitation. A model may look weaker than it is if testers do not use strong prompting, scaffolding, tools or fine-tuning. So frameworks often require a safety margin or testing at intervals of effective compute. Safeguards come in two kinds. Deployment safeguards include classifiers, refusal training, account monitoring and staged access. Security safeguards protect model weights from theft, for example through access controls, insider-threat programs and hardened infrastructure. Governance pieces include capability reports, internal review boards, whistleblower channels and board notification when thresholds are crossed.

전략적 영향

위험과 안전

치명적인 AI 피해와 일상적인 AI 피해는 누가 위험을 이해하고 누가 조치를 취할 수 있는지에 따라 달라집니다.

더 명확한 결정들

공공 및 전문 지식은 강력한 안전 정책이 정치적으로 가능한지 여부를 결정합니다.

과장된 과장을 뚫고 나가기

명확한 설명은 과대광고, 연구실 홍보, 모호한 윤리 연극에 의한 포착을 줄입니다.

The Future of Frontier AI Safety Frameworks and Responsible Scaling Policies

Frameworks are moving from voluntary pledges toward regulatory expectations. California now requires large frontier developers to publish them, the EU's Code of Practice asks signatories to adopt them, and more jurisdictions may follow. Open questions include who audits compliance, how thresholds get standardized across labs, and whether evaluations can keep up with rapidly improving agentic systems. Independent testing bodies, including government AI institutes, may take on a bigger role in checking lab claims. The central tension will likely remain: labs design the rules they are judged by, which makes transparency and outside scrutiny important.

실제 구현

Before release, a lab tests whether a new model gives meaningful uplift to someone trying to acquire biological weapons. If it crosses the threshold, the lab adds stricter misuse filters and security before deployment.

In May 2025, Anthropic said it had activated ASL-3 deployment and security protections for Claude Opus 4 as a precaution, because it could not rule out that the model had reached the relevant capability threshold.

OpenAI's Preparedness Framework directs a Safety Advisory Group to review capability reports and advise leadership on whether safeguards are adequate before a release.

A lab tracking autonomous replication or AI research acceleration runs agentic evaluations. These test whether the model can complete long, multi-step tasks without human help.

위험 및 가드레일

실존적 위험을 공상과학처럼 다루면서 능력을 합성합니다.
높은 자율성 하에서 정렬과 표면 제품 안전성을 혼동합니다.
영어가 아니거나 전문가가 아닌 청중에게는 품질이 낮은 소스만 남겨 둡니다.

구현 로드맵

제품 손상, 오용, 통제력 상실/잘못 정렬 위험을 분리합니다.
일정과 심각도에 대한 귀하의 견해를 바꿀 수 있는 증거가 무엇인지 물어보십시오.
마케팅 주장보다 기본 소스와 구체적인 평가를 선호하세요.
인식뿐만 아니라 경력, 정책, 자금 조달 또는 기술 등 하나의 행동 경로를 식별하십시오.

계속 탐색하세요

Free newsletter

Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.

One email each weekday. Unsubscribe in one click. We never sell or share your address.

Test yourself

Take the Frontier AI Safety Frameworks and Responsible Scaling Policies quiz

Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.

퀴즈 시작

Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation

자주 묻는 질문

What is Frontier AI Safety Frameworks and Responsible Scaling Policies?

A frontier AI safety framework, such as a responsible scaling policy, is a published commitment by an AI lab to test its models for dangerous capabilities. If a model crosses a set threshold, the lab commits to adding stronger safeguards, or to not deploying or not training further until those safeguards are in place. These frameworks matter because they are currently the main way the most advanced AI models are governed for catastrophic risk, though most of them are voluntary and written by the labs themselves.

What is the basic logic of a responsible scaling policy?

These frameworks are built on if-then commitments that tie capability thresholds to required safeguards.

Which company introduced AI Safety Levels (ASLs)?

Anthropic's September 2023 Responsible Scaling Policy introduced ASLs, loosely modeled on biosafety levels.

What does Google DeepMind call the thresholds in its Frontier Safety Framework?

DeepMind uses Critical Capability Levels (CCLs), which trigger response plans when they are reached.

What did companies agree to at the AI Seoul Summit in May 2024?

The Frontier AI Safety Commitments were voluntary pledges to publish frameworks and to not proceed if risks could not be kept below the thresholds.

Which category did OpenAI's 2025 Preparedness Framework revision drop as a tracked category?

The revision reorganized the framework around High and Critical thresholds and removed persuasion as a tracked category.

계속 학습하세요