Companies GUIDE
Covariant RFM-1 and Robotic Foundation Models
Covariant's RFM-1 is a robotics foundation model trained on warehouse manipulation data so robot arms can reason about and pick unfamiliar objects with language and vision.
Listen to this guideAI-narrated · ~2 min
On this page2 min read
Deep Dive
Covariant announced RFM-1 in March 2024 as an 8-billion-parameter multimodal model for warehouse robotics. Its company announcement says it trains on text, images, video, robot actions, and physical measurements, using autoregressive next-token prediction across modalities. The announcement describes capabilities including language-guided programming, answering questions, and generating videos that predict object interactions. In August 2024, Amazon said it received a non-exclusive license to Covariant’s robotic foundation models and hired the founders plus about a quarter of Covariant’s employees; Covariant said it would continue serving existing customers.
Real-World Implementation
Giving a warehouse robot a language instruction and asking it about an object in view
Using video prediction to consider how objects might respond to candidate robot actions
Evaluating picking systems on unfamiliar items in cluttered bins
Distinguishing Covariant’s RFM-1 product from its earlier Covariant Brain warehouse platform
The Future of Covariant RFM-1 and Robotic Foundation Models
RFM-1’s 2024 announcement illustrates how robotics firms explored foundation models trained across physical interactions and other modalities. Amazon’s license arrangement was non-exclusive, and Covariant continued to serve customers. Claims about robot generalization should be assessed on the specific hardware, warehouse tasks, and evaluation conditions.
Putting it into practice
Why it matters
- Vendor strategy. Vendor roadmaps influence what features your team can build next.
- Cost and budget. Commercial terms and deployment options affect long-term cost and risk.
- Risk and safety. Company incentives shape product defaults, safety posture, and openness.
Watch out for
- Launch announcements may outpace stability in real production workflows.
- API pricing or policy shifts can break assumptions overnight.
- Single-vendor dependency increases lock-in and migration costs.
Next steps
- Evaluate providers using your own tasks and datasets.
- Review privacy, security, and legal terms before integration.
- Maintain a fallback plan across models or vendors.
- Monitor release notes so roadmap changes do not surprise teams.
Keep Exploring
Free newsletter
Keep up with AI in 3 minutes a day
One short email each weekday with the three AI stories that actually matter. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Covariant RFM-1 and Robotic Foundation Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Frequently asked questions
What is Covariant RFM-1 and Robotic Foundation Models?
Covariant's RFM-1 is a robotics foundation model trained on warehouse manipulation data so robot arms can reason about and pick unfamiliar objects with language and vision.
What was the name of Covariant's foundation model introduced in 2024?
Covariant introduced RFM-1, the Robotics Foundation Model 1, trained on robot data plus text and images.
What did Amazon’s 2024 commercial agreement with Covariant include?
Amazon described a non-exclusive model license and said Covariant founders and some employees would join its robotics team; Covariant would continue serving customers.
What core idea does RFM-1 borrow from large language models?
RFM-1 treats text, images, sensor readings, and actions as tokens and predicts the next one, learning to generalize like an LLM.
What was RFM-1 designed to help warehouse robots do?
Covariant positioned RFM-1 for warehouse picking and described generalization to varied items; performance depends on the robot and task.
Who were among Covariant's notable founders?
Covariant was co-founded by AI researchers including Pieter Abbeel, Peter Chen, and Rocky Duan, with roots at UC Berkeley and OpenAI.
Keep learning
Related guides
More guides picked for this topic