技術指南

使用 nvidia-smi 和 DCGM 監控 GPU 使用率

nvidia-smi and NVIDIA DCGM expose GPU activity, memory, power, temperature, clock, and health information for diagnosing a running workload.

聆聽此指南AI 語音敘述 · ~3 min

本頁閱讀時間3分鐘
  1. 概述
  2. 深入探討
  3. 現實世界的實施
  4. The Future of GPU Utilization Monitoring with nvidia-smi and DCGM
  5. 付諸實踐
  6. 不斷探索
  7. 常見問題

概述

A single utilization percentage does not explain performance: compare multiple metrics with request or training throughput and profile the actual bottleneck.

深入探討

NVIDIA's nvidia-smi reports device status and utilization summaries, while DCGM provides monitoring, health, and profiling functions suited to managed systems and clusters. The GPU utilization percentage in nvidia-smi is the share of the recent sample period during which one or more kernels were executing. It is not the percentage of theoretical FLOPs achieved. Memory utilization is a separate activity measure and is not the same as the amount of VRAM allocated. A device can be active nearly all the time yet fail to deliver expected throughput. Kernels may be memory-bound, use few compute units, wait on dependencies, or perform operations that do not use specialized tensor hardware. Conversely, low utilization between steps can point to slow input decoding, small batches, CPU preprocessing, synchronization, or an undersized workload. Compare GPU metrics with tokens or samples per second, step time, latency, and host CPU activity. Memory capacity and memory traffic also answer different questions. nvidia-smi reports used memory in units such as MiB; DCGM profiling can report whether device-memory traffic is active. A high memory-activity ratio can signal bandwidth pressure, while high allocated capacity may simply reflect caching. Watch memory growth over time for leaks and include reserved memory from the framework. Power, clock frequency, and temperature help explain throttling or power limits. ECC and health diagnostics can reveal hardware or communication issues. DCGM's profiling metrics are interval averages; they do not identify an individual slow kernel or source line. Use a profiler when aggregate telemetry suggests a bottleneck but cannot locate it. Sample cadence matters. A short burst can disappear in a long reporting interval, and container or virtualization boundaries may affect which processes are visible. Monitor representative steady-state and burst workloads, and correlate metrics with logs and job phases. Use the tools to form a hypothesis, then verify it with performance profiling.

現實世界的實施

An operator checks GPU process memory, temperature, clocks, power, and utilization during a long training run.

A cluster dashboard samples DCGM metrics to compare SM activity, tensor-pipe activity, DRAM traffic, and interconnect use across nodes.

A data scientist sees low GPU activity between batches and profiles data loading before increasing model compute.

An engineer separates memory allocated in MiB from memory-activity percentage when assessing GPU capacity and bandwidth.

The Future of GPU Utilization Monitoring with nvidia-smi and DCGM

GPU monitoring tools will continue adding hardware counters and fleet-level health signals. Better dashboards may connect telemetry with workloads, model stages, and power use. Aggregate utilization will still need interpretation because kernels, memory traffic, and communication create different bottlenecks. Teams should combine DCGM or nvidia-smi snapshots with framework profilers and application throughput for actionable diagnosis. Correlate sampled signals with application traces and job phases. Recheck metric definitions when tools, drivers, or device generations change. Keep sampling windows documented per device.

付諸實踐

為什麼這很重要

  • 成本與預算. 多年來,架構決策決定著效能和營運成本。
  • 更明確的決策. 技術教育幫助團隊選擇正確的堆疊,而不僅僅是最新的堆疊。
  • 品質管控. 更好的工程選擇可以減少生產中的可靠性事故。

留意

  • 優化一項基準測試可以隱藏更廣泛的系統弱點。
  • 基礎設施和維護成本常常被低估。
  • 隨著系統變得更加複雜,安全性和可觀察性差距可能會擴大。

後續步驟

  1. 在實施之前定義延遲、品質和成本目標。
  2. 在實際負載和資料條件下進行基準測試。
  3. 儀器監控錯誤、漂移和使用者影響。
  4. 在擴展之前準備回滾和事件回應路徑。

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

Take the GPU Utilization Monitoring with nvidia-smi and DCGM quiz

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

What is GPU Utilization Monitoring with nvidia-smi and DCGM?

nvidia-smi and NVIDIA DCGM expose GPU activity, memory, power, temperature, clock, and health information for diagnosing a running workload. A single utilization percentage does not explain performance: compare multiple metrics with request or training throughput and profile the actual bottleneck.

nvidia-smi GPU 使用率大約會報告什麼?

The reported utilization is a time-based activity measure, not achieved compute efficiency.

記憶體利用率與記憶體使用量有何不同?

Memory activity and allocated capacity measure different behavior.

GPU 報告高利用率但吞吐量低,可能的解釋是什麼?

Continuous kernel activity does not guarantee high arithmetic efficiency.

在訓練步驟之間 GPU 活動低可能表示什麼?

The GPU may be waiting for CPU preprocessing, data transfer, or synchronization.

哪個 DCGM 指標可以協助評估記憶體流量活動?

DCGM profiling includes a measure of device memory traffic activity.