Hidden Technical Debt in ML Systems

Technical debt in ML systems includes the future cost of maintaining shortcuts, hidden assumptions and tangled dependencies. The classic paper by Sculley and colleagues argues that ML systems can incur debt through data dependencies, feedback loops, undeclared consumers and boundary erosion, in addition to ordinary code quality issues. A model is only one part of a system that also includes data collection, features, training, evaluation, deployment and monitoring. Data dependencies can be unstable or poorly documented. A feature may change meaning upstream without a code change in the model repository. A pipeline can silently depend on a data source that is slow, sparse or governed by another team. Monitoring and schema contracts make these dependencies visible. Undeclared consumers occur when a feature or prediction is used by multiple downstream systems that are not tracked, making updates risky. Feedback loops arise when model outputs influence the data later used for training. A ranking model chooses what users see; clicks from those exposures then become labels. The next model may reinforce existing preferences or exposure patterns. Entanglement means components' behavior depends on one another, so changing a shared feature or model can affect multiple outputs in nonlocal ways. Boundary erosion occurs when responsibilities between components blur and changes propagate unexpectedly. Debt reduction is ongoing work. Map data and model dependencies, define owners and interfaces, version features, test integration contracts, monitor input quality and model outcomes, and retire unused components. Reproducible training and clear artifact lineage make rollback possible. Avoid adding complex abstractions that do not reduce actual risk. A model may have strong offline metrics while the surrounding system remains fragile. Review maintenance cost as part of model lifecycle decisions, and include feedback effects and hidden consumers when planning migrations.