應用指南

影片內容審核中的人工智慧

AI 影片內容審核使用指紋比對與機器學習分類器,尋找違反平台規則的上傳或直播影片,通常在使用者看到之前就已被偵測到。

聆聽此指南AI 語音敘述 · ~8 min

本頁閱讀時間 9 分鐘
  1. 概述
  2. 重點摘要
  3. 深入探討
  4. 批判性閱讀平台透明度報告。
  5. 現實世界的實施
  6. 人工智慧在影片內容審核中的未來
  7. 付諸實踐
  8. 資料來源與延伸閱讀
  9. 不斷探索
  10. 常見問題

概述

規模很重要:YouTube 表示每分鐘有超過 500 小時的影片進入,且在 2023 年底,自動化系統先行標記了 96.1% 被移除的影片。

重點摘要

  1. 影片審核結合已知素材的雜湊比對、新素材的分類器、人為審查以獲取上下文,以及使用者舉報。
  2. YouTube 於 2023 年 10 月至 12 月的報告顯示,9,012,232 份被移除影片中有 96.1% 首先由自動化系統標記。
  3. 感知雜湊,例如 Meta 的開源 PDQ 與 TMK+PDQF,能比對已知影片的編輯副本,而不會分享有害檔案。
  4. Facebook 在上傳後 24 小時內封鎖了超過 1.2 百萬份新西蘭克里斯特城攻擊影片的複製品,但其分類器錯過了原始直播。
  5. Meta 的監督委員會發現一個分類器錯誤移除了一則乳癌意識貼文,並建議在自動化行動時告知使用者。

深入探討

Video content moderation is the work of finding and acting on videos that break a platform's rules, such as graphic violence, child sexual abuse material, terrorist propaganda, nudity, harassment or dangerous acts. AI in this context means automated systems that inspect videos and their audio, text and metadata, then either take action directly or send the video to human reviewers. Platforms rely on automation because the volume is far beyond human capacity. In its October to December 2023 report filed under California law AB 587, YouTube said more than 500 hours of content are uploaded every minute and that the vast majority of content it reviews and acts on is first detected by automated systems.

There are two main families of technique. The first is matching against known content. A system computes a hash, a short digital fingerprint, of a video or its frames and compares it with a database of fingerprints from material already judged violating. Cryptographic hashes such as MD5 only match exact copies. Perceptual hashes such as Meta's PDQ for images and TMK+PDQF for video are designed so that visually similar files produce similar fingerprints, which catches copies that have been resized, re-encoded or lightly edited. Because only fingerprints are shared, organizations can pool them without exchanging the harmful files themselves.

The second family is classification of new content. Machine learning classifiers are trained on labeled examples to estimate how likely a video is to contain a given kind of violation. A pipeline can sample frames, runs image models on them, analyzes the audio and any speech, reads on-screen text and the title or description, and combines these signals into scores per policy area. Each score is compared with thresholds chosen by the platform: very high-confidence cases may be removed or restricted automatically, uncertain cases go to human moderators, and low scores pass. YouTube's report describes this as a collaborative loop in which human moderators confirm decisions and their feedback is used to retrain the models.

The final stage is the action and the appeal. Actions are not just removal. YouTube's report lists restricting a video, for example age-restricting content unsuitable for all audiences, as well as removal and channel strikes, with three strikes in 90 days able to end a channel. Users are notified and can appeal, and YouTube says appeals go to human review. The results show how early automation acts: in that quarter, YouTube removed 9,012,232 videos, 96.1% of them first flagged by automated systems, 3.4% by ordinary users and 0.6% by organizations in its Priority Flagger program. Just over half of removed videos, 51.51%, had zero views, and another 26.43% had between 1 and 10 views.

Shared databases grew out of specific crises. After a gunman livestreamed the March 2019 attack on two mosques in Christchurch, New Zealand, Facebook reported that the live broadcast was viewed fewer than 200 times and about 4,000 times in total before removal, that the first user report came 29 minutes after the stream began, and that in the following 24 hours it blocked more than 1.2 million copies at upload and removed about 300,000 more. It found more than 800 visually distinct variants and used audio matching to catch edits whose visuals had changed but whose soundtrack had not. That year GIFCT, the Global Internet Forum to Counter Terrorism, expanded its Hash-Sharing Database from content produced by groups on the United Nations Security Council sanctions list to include content from perpetrators who livestream or record attacks, and later added attacker manifestos and URLs. In August 2019 Facebook released PDQ and TMK+PDQF as open source.

Compared with alternatives, each method has a clear role. Human-only review cannot keep pace with hundreds of hours of uploads per minute. User reporting is valuable for context but slow: in the Christchurch case no user reported the live video until after the broadcast ended. Hash matching is precise and cheap but only finds content someone has already identified, and determined uploaders try to evade it with edits. Classifiers can find new material but make more mistakes. In practice, platforms layer all four, using hashes for known material, classifiers for triage, humans for context and appeals, and user reports as an additional signal.

The limitations are significant and documented. Classifiers need many training examples, and Facebook explained that its systems missed the Christchurch livestream partly because such events are rare and partly because similar-looking footage, such as livestreamed video games, would cause floods of false alarms. Classifiers also miss context. In a January 2021 decision, Meta's Oversight Board found that a machine learning classifier had removed a Brazilian user's Instagram post about breast cancer symptoms under the nudity policy, failing to recognize the Portuguese words for breast cancer on the image, even though awareness posts were explicitly allowed. The Board recommended telling users when automation acted, allowing human review of such appeals, and auditing samples of automated removals. Automated systems can also perform unevenly across languages and cultures, and they struggle with satire, news reporting and documentary uses that YouTube's policies explicitly exempt.

In practice, building video moderation requires clear written policies, labeled training data for each policy area, reviewer teams and tooling, appeal processes, and metrics such as the share of removals reversed on appeal. Smaller platforms rarely build all of this themselves. They often join hash-sharing programs such as GIFCT's, which does not dictate how members enforce their own rules, use open-source fingerprinting tools like PDQ and TMK+PDQF, and buy classification from vendors. Livestreams add a time constraint, since harm can occur before review, so platforms combine real-time classifiers with rapid escalation paths. The central design choice is where to set thresholds: lower thresholds catch more violations but remove more legitimate speech, and the right balance differs by harm type, which is why many platforms publish transparency reports so outsiders can examine the trade-offs.

04工作範例

批判性閱讀平台透明度報告。

  1. 開啟 YouTube 的 AB 587 報告,查詢 2023 年 10 月至 12 月的影片總移除數,為 9,012,232 件。

  2. 找出首次偵測來源的移除圖表,並註明自動標記佔 96.1%。

  3. 找出依觀看次數分類的移除圖表,並加入 0 次觀看與 1‑10 次觀看的比例,分別為 51.51% 與 26.43%。

  4. 註明報告未顯示的項目,例如有多少移除是錯誤的或在上訴後被逆轉。

  5. 與監督委員會 2021 年的建議比較:平台應報告在人工審查後被推翻的自動移除數量。

  6. 寫下報告未回答的關於準確性的問題。

它顯示了什麼

約有 77.9% 的被移除影片觀看次數不超過 10 次,顯示自動化早期介入,但僅靠報告無法得知其正確率。請對照報告中的圖表標籤,逐一核對每個數字。

現實世界的實施

YouTube 的 2023 年 10 月至 12 月報告顯示,共移除 9,012,232 部違反社群守則的影片,其中 96.1% 為自動標記首次偵測,且 51.51% 在未獲得任何觀看前即被移除。

2019 年 3 月基督城襲擊事件後,Facebook 在 24 小時內阻擋超過 120 萬份攻擊影片的上傳,透過影片與音訊比對,針對超過 800 種不同編輯變體進行匹配。

GIFCT 的雜湊共享資料庫允許會員公司分享像 PDQ 和 TMK 這類的感知雜湊,用於恐怖分子和暴力極端主義者的圖像與影片,這樣一個平台的發現可以協助其他平台阻止重複上傳。

Facebook 在 2019 年 8 月將其 PDQ 照片雜湊和 TMK+PDQF 影片雜湊演算法開源,讓其他公司與非營利組織能夠為兒童剝削、恐怖宣傳與血腥暴力進行指紋辨識。

申訴與稽核:在 2020 年 Instagram 分類器錯誤移除了一篇乳癌意識提升貼文後,Meta 的監督委員會建議,當自動化系統採取行動時應告知使用者,並在申訴時將自動移除的裸露內容轉交人工審查。

人工智慧在影片內容審核中的未來

Several directions are visible in public sources. Hash-sharing has broadened in scope, with GIFCT's database expanding in 2019 to cover recorded attacks and later to manifestos and URLs, and open-source fingerprinting tools lowering the barrier for smaller platforms. Calls for transparency about automation, such as the Oversight Board's 2021 recommendations to tell users when automation acted and to publish how often automated removals are reversed, push platforms to measure errors as well as removals. Open problems remain hard. Rare, novel events like the 2019 Christchurch livestream still provide few training examples, context-dependent material such as news, satire and medical information remains error-prone, and performance across languages and regions is uneven. Synthetic video made with generative AI raises further questions about how detection and labeling should work alongside moderation. How regulators, platforms and independent researchers will measure moderation quality, rather than just volume, is an open question, and published transparency reports are the best current evidence for judging progress.

付諸實踐

為什麼這很重要

  • 配裝選擇. 應用級設計決定了人工智慧是否能改善實際結果。
  • 團隊與工作流程. 良好的工作流程整合可以創造使用者值得信賴的生產力效益。
  • 風險與安全. 範圍明確的用例可以減少變更疲勞和實施風險。

留意

  • 將損壞的流程自動化可能會加劇現有問題。
  • 團隊可能會過度自動化並消除所需的人工判斷。
  • 如果不持續評估輸出,品質可能會出現偏差。

後續步驟

  1. 繪製目前工作流程並確定摩擦最大的步驟。
  2. 在完全自動化之前定義人工檢查點。
  3. 對使用者進行提示、升級路徑和品質標準的訓練。
  4. 追蹤任務級結果以確認持續價值。

資料來源與延伸閱讀

  1. California Department of Justice (filed by YouTube)YouTube 2023 年第四季報告,加州 AB 587 法案
  2. Meta Newsroom關於紐西蘭恐怖攻擊事件的進一步更新
  3. Meta Newsroom開源照片與影片比對技術,讓網路更安全
  4. Global Internet Forum to Counter Terrorism雜湊共享資料庫
  5. Oversight Board乳癌症狀與裸露內容 (2020-004-IG-UA)
  6. YouTubeYouTube 社群準則
  7. Meta (GitHub)PDQ:感知影像雜湊 (ThreatExchange)
  8. Meta (GitHub)TMK+PDQF:影片雜湊 (ThreatExchange)

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

參加 AI 影片內容審查測驗

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

什麼是影片內容審核中的人工智慧?

AI 影片內容審核利用指紋比對和機器學習分類器,來找出上傳或直播中違反平台規則的影片,通常是在人們看到之前。這之所以重要是因為規模:YouTube 表示,每分鐘有超過 500 小時的影片上傳,而在 2023 年底,自動化系統首次標記了其移除影片的 96.1%。

為什麼視訊審核被描述為“多模式”?

視訊結合了多種訊號類型,因此有效的審核需要視覺、運動分析、語音轉文字和 OCR 協同工作。

感知雜湊如何協助平台辨識已重新上傳的已知有害影片?

感知影片指紋可與已知素材的指紋比對,以識別近似重複的副本;它本身不會辨識全新有害內容。

為什麼平台仍然將許多案例轉交給人工審核員?

人工智慧會對內容進行分類和評分,但需要判斷意圖和上下文的模棱兩可的案例會升級給人們。

語音轉文本如何有助於審核?

在影片審查流程中,語音辨識可從音訊產生文字稿,文字分類器再檢查其中的政策訊號,例如仇恨言論或威脅。

堅韌的影片指紋在經過編碼或影像變更後能協助辨識什麼?

感知影片雜湊旨在支援在某些保留內容的轉換下匹配視覺相似的副本;其堅韌度取決於特定方法與轉換。