The Track Record of AI Predictions

AI forecasts are hard to grade because the same phrase can describe a benchmark result, a narrow job or a broad human capability. A testable forecast names its target, time window and success condition. Vague claims can seem prescient later when their meaning shifts to fit events. The history includes ambitious forecasts. In The Shape of Automation for Men and Management (1965), Herbert Simon said machines would be technologically capable within twenty years of doing any work a person could do. This was a broad technical-capability claim, not a prediction that workers would be replaced or systems deployed throughout the economy by the deadline. Simon distinguished capability from economic adoption: people could retain comparative advantage in work they performed better. By 1985, no system had demonstrated the stated universal capability; progress on narrow tasks alone did not establish the full claim. Earlier researchers also gave optimistic timelines for machine translation and chess. Historical claims need their original wording and conditions. Underestimation also happens. People may overlook how quickly computing, data or a method can improve, or assume a capability will remain difficult because earlier attempts struggled. Some forecasts are conditional on hardware, funding, data or policy. A fair assessment records the original claim, date, assumptions, target and evidence at the deadline. It separates a demonstration from reliable performance and from broad social adoption. A useful prediction ledger includes successful, failed and unresolved claims. Preserve the original source, avoid selecting only famous misses, and state how the outcome is defined. For current forecasts, ask for a measurable endpoint and probability, then revisit it on schedule. Historical examples teach caution about confidence and scope; they do not prove that every current prediction will fail or that rapid progress cannot happen.