torch.compile and PyTorch 2 Graph Compilation

PyTorch normally executes tensor operations eagerly, which is convenient for debugging. torch.compile can capture compatible computation and compile it for a backend. In common PyTorch 2 workflows, TorchDynamo captures Python frames and TorchInductor generates optimized code for supported devices. The goal is to reduce overhead and improve kernels or operation fusion, but actual gains depend on the model, shapes, hardware, backend, and workload. A compiled function may take longer on the first call because tracing and compilation occur. Subsequent calls can reuse compiled variants when inputs and execution patterns match. Changes in shapes, dtypes, control flow, or guards may trigger additional compilation. Dynamic-shape options can reduce some recompiles but may affect optimization. A graph break occurs when execution cannot be captured as part of a compiled graph; PyTorch then runs a portion eagerly. Graph breaks may be correct but reduce the opportunity for optimization. Compilation is a performance tool, not a semantic fix. Compare eager and compiled outputs within appropriate tolerances, test gradients if training, and include representative edge cases. Measure cold-start and steady-state performance separately. Include data loading, transfer, synchronization, and postprocessing if those contribute to production latency. Avoid reporting speedups from tiny synthetic inputs if deployment uses larger or variable workloads. Start with default settings and inspect logs or diagnostics if speed does not improve. Some models benefit substantially, while others see little gain or become slower due to compilation overhead. Custom operators, Python-heavy control flow, unsupported operations, and frequent shape changes can limit capture. Static export or other deployment paths may be more suitable for a different goal. Keep an eager baseline and pin the PyTorch version and backend configuration. Compilation support and behavior evolve, and not every operation is supported equally across CPU, CUDA, and other backends. Adopt compilation only after end-to-end measurement demonstrates value without correctness regressions.