PRŮVODCE společností

Model Risk Management (SR 11-7) a AI

SR 11-7 is the Federal Reserve's 2011 supervisory guidance on model risk management, issued jointly with OCC Bulletin 2011-12.

Poslechněte si tento návodAI-vyprávěný · ~3 min

Na této stránce4 min čtení
  1. Přehled
  2. Hluboký ponor
  3. Real-World Implementace
  4. The Future of Model Risk Management (SR 11-7) and AI
  5. Uvedení do praxe
  6. Pokračujte v objevování
  7. Často kladené otázky

Přehled

It expects banks to develop, validate, govern and monitor the models they rely on. Banks now apply it to machine learning and large language models, which strains traditional validation because these models are often opaque, supplied by vendors and non-deterministic. The guidance matters because a bank using AI for credit, fraud, compliance or customer service has to show supervisors it understands and controls how those models can fail.

Hluboký ponor

SR 11-7 was issued in April 2011 by the Federal Reserve together with the OCC, and the FDIC adopted it in 2017. It defines a model broadly: a quantitative method, system or approach that applies statistical, economic, financial or mathematical theories to turn input data into quantitative estimates. A model has an input part, a processing part and a reporting part. Model risk comes from two sources: fundamental errors in the model, and using a sound model incorrectly or outside its intended purpose. The guidance rests on three pillars. The first is sound development, implementation and use. The second is validation, which has three core elements: evaluating conceptual soundness, ongoing monitoring (including process verification and benchmarking), and outcomes analysis such as back-testing. The third is governance: board and senior management oversight, written policies, a complete model inventory, and documentation detailed enough that someone unfamiliar with the model could understand how it works. Throughout, the guidance calls for "effective challenge," meaning critical review by people who are objective, informed, competent and influential enough to force changes. Vendor models get no exemption. Banks are expected to get appropriate documentation from vendors. Where proprietary details are withheld, banks should rely more on sensitivity analysis, benchmarking and outcomes testing. A common misconception is that SR 11-7 doesn't reach AI because it predates modern machine learning. Its definition is technology-neutral, and supervisors have treated AI as within scope. Banks usually either classify generative AI tools as models or govern them under a broader AI risk framework that uses the same validation principles. Related references include the OCC's 2021 Comptroller's Handbook booklet on model risk management and the NIST AI Risk Management Framework, released in January 2023. For credit decisions, adverse action notice requirements under the Equal Credit Opportunity Act still apply when the model is complex.

Real-World Implementace

A bank adds a vendor LLM that summarizes customer complaints to its model inventory, assigns it a risk tier, and gives it to an independent validation team before production use.

Validators build a labeled set of several hundred complaints to measure how often the LLM's summaries leave out an issue that must be escalated for regulatory reasons. They set an acceptable error threshold before approving the tool.

A machine learning credit model goes through outcomes analysis against actual defaults, plus fair lending testing. Explanation methods help produce the specific adverse action reasons lenders must give applicants.

A monitoring dashboard tracks shifts in input data and samples LLM output quality every week. When the vendor releases a new model version, the dashboard triggers a targeted revalidation.

The Future of Model Risk Management (SR 11-7) and AI

Banks are expanding model inventories and building evaluation methods for generative AI. Supervisors have discussed AI governance in speeches and requests for information, but whether formal updates to model risk guidance will come, and what they would say, remains uncertain. The core principles in SR 11-7 (know the model's purpose, test it independently, document its limits, monitor it over time) apply well to LLMs even where specific methods are still being worked out. Expect the most attention on vendor transparency and continuous monitoring.

Uvedení do praxe

Proč na tom záleží

  • Riziko a bezpečnost. Katastrofické a každodenní škody AI závisí na tom, kdo rozumí rizikům a kdo může jednat.
  • Jasnější rozhodnutí. Veřejná a odborná gramotnost určuje, zda je silná bezpečnostní politika politicky možná.
  • Prorážením humbuku. Jasná vysvětlení snižují zachytávání humbukem, PR v laboratoři a vágní etické divadlo.

Pozor na

  • Zacházení s existenčním rizikem jako sci-fi, zatímco schopnosti kombinují.
  • Matoucí bezpečnost povrchových produktů se zarovnáním pod vysokou autonomií.
  • Neanglické a neodborné publikum ponechává pouze nekvalitní zdroje.

Další kroky

  1. Oddělte rizika poškození produktu, nesprávného použití a ztráty kontroly/nesouladu.
  2. Zeptejte se, jaké důkazy by změnily váš pohled na časové osy a závažnost.
  3. Upřednostňujte primární zdroje a konkrétní hodnocení před marketingovými tvrzeními.
  4. Identifikujte jednu akční cestu: kariéru, politiku, financování nebo dovednosti – nejen povědomí.

Pokračujte v objevování

Newsletter zdarma

Získejte denní přehled o AI

Tři ověřené příběhy ze světa AI každý všední den ráno, psané srozumitelnou angličtinou. Navždy zdarma, bez reklam.

Jeden e-mail každý všední den. Odhlásit se můžete jedním kliknutím. Vaši adresu nikdy neprodáváme ani nesdílíme.

Otestujte se

Take the Model Risk Management (SR 11-7) and AI quiz

Okamžitá zpětná vazba ke každé odpovědi a sdílitelný certifikát s ověřitelným ID po absolvování kurzu.

Spustit kvíz

Podpořte bezplatnou AI vzdělávání. AI Understanding je nezisková organizace 501(c)(3) — žádné reklamy, žádná platba za přístup, nikdy. Darujte

Často kladené otázky

What is Model Risk Management (SR 11-7) and AI?

SR 11-7 is the Federal Reserve's 2011 supervisory guidance on model risk management, issued jointly with OCC Bulletin 2011-12. It expects banks to develop, validate, govern and monitor the models they rely on. Banks now apply it to machine learning and large language models, which strains traditional validation because these models are often opaque, supplied by vendors and non-deterministic. The guidance matters because a bank using AI for credit, fraud, compliance or customer service has to show supervisors it understands and controls how those models can fail.

Který dokument OCC byl vydán vedle SR 11-7 Federálního rezervního systému v roce 2011?

OCC vydal stejné pokyny jako Bulletin 2011-12, takže tyto dva dokumenty jsou často citovány společně.

Jaké jsou tři základní prvky validace podle SR 11-7?

Ověření zahrnuje, zda je návrh správný, zda model funguje tak, jak bylo zamýšleno, a jak jsou jeho výstupy v porovnání se skutečnými výsledky.

Co vyžaduje SR 11-7 ke kontrole, aby se považovala za „účinnou výzvu“?

Efektivní výzva znamená kritickou analýzu schopnými, nezávislými lidmi, jejichž zjištění skutečně vedou ke změnám.

How should a bank handle a vendor model whose proprietary details are withheld?

Vendor models are still validated. When internal details are unavailable, testing of behavior and outputs carries more weight.

According to the guide, which of these is part of an LLM application's model boundary for change management?

Prompts, retrieval data, parameters, tools and guardrails all shape the output, so changing them counts as a model change.