AI Glossary Term
What does “Sparse Model” mean?
Definition
A model where many parameters are zero or inactive to reduce computation.
How Sparse Model works
Example
A pruned network sets most of its weights to zero. On hardware that can skip those zeros, the model does less arithmetic per prediction.
Why it matters
Sparsity is a way to get large capacity with less compute. Real savings depend on software and hardware support.
Common questions
Sparse vs mixture of experts?
Mixture of experts is one kind of sparse model, where only a few expert subnetworks run for each input.
Does sparsity lower quality?
It can if too aggressive; well-designed sparsity retains most accuracy.
How is sparsity created?
By pruning after training, training with sparsity built in, or using designs like mixture of experts.