Multi-Instance GPU Partitioning
Multi-Instance GPU (MIG) is an NVIDIA technology that slices a single physical GPU into multiple isolated hardware partitions.
Overview
It matters because it lets one expensive accelerator serve many small workloads at once without them interfering with each other.
Deep Dive
Introduced with the NVIDIA A100 (Ampere) and continued on H100 and newer data-center GPUs, MIG carves a GPU into up to seven independent instances. Unlike software time-slicing, MIG provides true hardware isolation: each instance gets its own dedicated streaming multiprocessors (SMs), L2 cache slices, memory controllers, and a fixed slice of high-bandwidth memory. An A100 with 40GB can be split into seven 5GB instances, or fewer larger ones. Each partition behaves like a smaller standalone GPU, so a noisy or crashing job in one instance cannot starve or corrupt another. This guaranteed quality-of-service makes MIG ideal for inference serving, multi-tenant clusters, and development environments where many users share one card.
Technical Insight
MIG works by physically gating the GPU's internal crossbar so each instance has a fixed path to its own memory slice and SMs. NVIDIA defines profiles as fractions like 1g.5gb (one compute slice, 5GB) up to 7g.40gb. A GPU Instance reserves memory and SMs; within it a Compute Instance subdivides the SMs further. Because the partitions are hardware-enforced, faults, ECC errors, and memory bandwidth stay confined to a single instance.
Strategic Impact
Cost and budget
Architecture decisions drive performance and operating cost for years.
Clearer decisions
Technical education helps teams choose the right stack, not just the newest one.
Quality control
Better engineering choices reduce reliability incidents in production.
The Future of Multi-Instance GPU Partitioning
As GPUs grow to 80GB, 141GB, and beyond, partitioning becomes more attractive because individual models rarely need a whole card for inference. Expect tighter Kubernetes and cloud integration, dynamic repartitioning without draining the node, and finer-grained profiles. Competing vendors are pursuing similar SR-IOV-style GPU virtualization, and serverless inference platforms increasingly rely on partitioning to pack many models densely and cut idle waste.
Real-World Implementation
A cloud provider splits one A100 into seven instances so seven customers each get a guaranteed, isolated GPU slice for inference.
A university research cluster gives each PhD student a 10GB MIG instance for prototyping instead of monopolizing whole cards.
An inference service packs several small language and vision models onto one H100, each in its own partition with predictable latency.
A Kubernetes cluster advertises MIG instances as schedulable resources so pods request 'nvidia.com/mig-1g.5gb' like any other resource.
Risks & Guardrails
Optimizing one benchmark can hide broader system weaknesses.
Infrastructure and maintenance costs are often underestimated.
Security and observability gaps can grow as systems become more complex.
Implementation Roadmap
Define latency, quality, and cost targets before implementation.
Benchmark under realistic load and data conditions.
Instrument monitoring for errors, drift, and user impact.
Prepare rollback and incident response paths before scaling.
Keep Exploring
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Instance GPU Partitioning quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Multi-Task Learning
Frequently asked questions
What is Multi-Instance GPU Partitioning?
Multi-Instance GPU (MIG) is an NVIDIA technology that slices a single physical GPU into multiple isolated hardware partitions. It matters because it lets one expensive accelerator serve many small workloads at once without them interfering with each other.
What kind of isolation does MIG provide between instances?
MIG enforces isolation in hardware, dedicating SMs, L2 cache slices, and memory to each instance so workloads cannot interfere.
On which NVIDIA architecture was MIG first introduced?
MIG debuted with the Ampere-based A100 and continued on H100 and later data-center GPUs.
What is the maximum number of instances a MIG-capable GPU can be split into?
A MIG-capable GPU such as the A100 can be partitioned into up to seven independent instances.
In a MIG profile like '1g.5gb', what does the '5gb' represent?
The profile encodes compute slices (1g) and the dedicated memory (5GB) given to the instance.
Which workload is MIG especially well suited for?
MIG shines when many small, isolated workloads (like inference) share one card with guaranteed quality of service.