Model Serialization Formats
Model serialization is how a trained machine learning model gets saved to disk so it can be loaded and run later, on a different machine or in a different language.
Overview
The format you choose affects portability, speed, file size, and even security.
Deep Dive
After training, a model is just numbers (weights) plus a description of its architecture. Serialization writes that state into a file. Different ecosystems use different formats. Python's pickle and PyTorch's default .pt files are convenient but tie you to Python and can execute arbitrary code on load, making them a security risk with untrusted files. ONNX (Open Neural Network Exchange) is a framework-neutral format that lets a model trained in PyTorch run in another runtime or language. SavedModel and the older HDF5 serve TensorFlow and Keras. For large language models, safetensors has become popular because it stores only tensor data in a simple, fast, memory-mappable layout with no code execution, making it both safer and quicker to load. GGUF is widely used for running quantized LLMs efficiently on local hardware.
Technical Insight
The key trade-off is between framework-native and interchange formats. Native formats (pickle, .pt) capture full Python objects but require the same code to deserialize and may run hidden code. Interchange formats like ONNX export the computational graph and weights into a standardized schema (using protocol buffers) so any compatible runtime can execute it. Safetensors goes minimal: a small JSON header describing each tensor's name, shape, and dtype, followed by the raw bytes, enabling zero-copy memory mapping.
Strategic Impact
Cost and budget
Architecture decisions drive performance and operating cost for years.
Clearer decisions
Technical education helps teams choose the right stack, not just the newest one.
Quality control
Better engineering choices reduce reliability incidents in production.
The Future of Model Serialization Formats
Expect continued consolidation around safe, portable formats. Safetensors is becoming the default for sharing model weights publicly because it removes the code-execution risk of pickle, and GGUF is the de facto standard for local LLM inference with quantization. ONNX keeps expanding as the bridge between training frameworks and optimized deployment runtimes on edge devices, browsers, and accelerators. Overall the trend favors formats that are language-neutral, memory-efficient, and secure by design.
Real-World Implementation
A team trains a model in PyTorch, exports it to ONNX, and runs it inside a C# application with no Python dependency.
Hugging Face distributes model weights as safetensors so users can download them without risk of malicious code execution.
A developer downloads a GGUF file of a quantized LLM to run it locally on a laptop CPU.
A TensorFlow service loads a SavedModel directory containing the graph and variables for serving predictions via an API.
Risks & Guardrails
Optimizing one benchmark can hide broader system weaknesses.
Infrastructure and maintenance costs are often underestimated.
Security and observability gaps can grow as systems become more complex.
Implementation Roadmap
Define latency, quality, and cost targets before implementation.
Benchmark under realistic load and data conditions.
Instrument monitoring for errors, drift, and user impact.
Prepare rollback and incident response paths before scaling.
Keep Exploring
Free newsletter
Keep up with AI in 3 minutes a day
One short email each weekday with the three AI stories that actually matter. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Model Serialization Formats quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Model Pruning
Frequently asked questions
What is Model Serialization Formats?
Model serialization is how a trained machine learning model gets saved to disk so it can be loaded and run later, on a different machine or in a different language. The format you choose affects portability, speed, file size, and even security.
Why is Python's pickle format considered a security risk for models?
Unpickling can run arbitrary code, so loading an untrusted pickle file can compromise your system.
What is the main advantage of the ONNX format?
ONNX is a framework-neutral interchange format, enabling portability across runtimes, languages, and hardware.
Why has safetensors become popular for sharing model weights?
Safetensors avoids the code-execution risk of pickle and loads quickly via memory mapping, making it safe and efficient.
GGUF is most associated with which use case?
GGUF is the de facto format for distributing and running quantized LLMs locally and efficiently.
What does an interchange format like ONNX primarily store?
ONNX captures the model's computation graph and parameters in a standard schema so any compatible runtime can execute it.