Chuyện gì đã xảy ra
PrismML released Bonsai 2 27B, a compressed large language model designed to run on personal computers and high-end smartphones without cloud infrastructure. The model reduces Alibaba’s Qwen3.8 27B to 5.9 gigabytes using ternary weight compression, retaining 98% of the original’s performance. The weights are available under the Apache 2.0 License and support NVIDIA GPUs via CUDA and Apple devices via MLX.
PrismML, a startup founded by Caltech researchers and led by Babak Hassibi, released Bonsai 2 27B on Thursday. The model is a compressed version of Alibaba’s Qwen3.8 27B, reduced to 5.9 gigabytes through ternary weight compression, which converts 16-bit parameters to three values: positive one, negative one, or zero. This technique eliminates redundant parameters, allowing the model to retain 98% of the original’s performance while fitting on local hardware.
The release marks a significant improvement over the original Bonsai model from March, which retained 95% of performance. According to PrismML, the original model has been downloaded over 11 million times, with additional smaller models accumulating 2.6 million downloads, indicating strong developer adoption for device-based AI. The new model runs on NVIDIA GPUs via CUDA and on Apple devices (Mac, iPhone, iPad) via MLX, utilizing custom low-bit kernels to ensure efficient local execution.
The model weights are available immediately under the Apache 2.0 License, allowing for broad integration and modification. PrismML raised a $22.25 million seed round backed by Khosla Ventures, Cerberus Capital, and Caltech, with Ion Stoica, Databricks co-founder, serving as an adviser. The company’s approach focuses on leveraging the redundancy in larger models to achieve high performance parity with minimal file size, a strategy Hassibi suggests will become more effective as model sizes increase.
Chi tiết nguồn: techjuice.pk ↗
Tại sao nó quan trọng
This release demonstrates that reasoning-capable AI can operate locally on consumer hardware, addressing privacy concerns and eliminating cloud latency and subscription costs. By achieving high performance parity with a significantly smaller file size, PrismML advances the shift from cloud-dependent AI to distributed, device-based reasoning, making advanced AI capabilities accessible without massive data center infrastructure.
The ability to run reasoning-capable AI locally on consumer devices addresses critical privacy concerns by ensuring user data never leaves the device. This eliminates the need for cloud transmission, reducing latency and removing ongoing subscription costs associated with cloud-based AI services. For developers and end-users, this represents a fundamental shift toward distributed, device-based reasoning that is both cost-effective and private.
The high retention rate of 98% performance in a 5.9GB file demonstrates that compression techniques can effectively preserve the intelligence of large models. This has practical implications for industries requiring , such as healthcare, finance, and personal assistants, where data sovereignty and real-time response are paramount. The open-source nature of the release further accelerates adoption by allowing the community to build upon and optimize the model for specific hardware constraints.
Cơ chế tương tác: Nó thực sự hoạt động như thế nào
Khám phá công nghệ cơ bản đằng sau sự phát triển này một cách tương tác.
Which component of an AI application is the machine-learning model itself?
Xem gì tiếp theo
Developers should monitor the availability of custom low-bit kernels for different hardware architectures and the company's stated goal of compressing hundreds-of-billions-parameter models in the coming months. Additionally, the potential integration with Apple devices, hinted at by the timing of the release, warrants attention for future device-side AI capabilities.
PrismML has stated its next goal is to compress models with hundreds of billions of parameters within the next couple of months. Success in this area would further validate the scalability of ternary compression for frontier-class models, potentially making even more advanced AI capabilities available on local hardware.
The timing of the release has led to speculation about potential integration with Apple devices. While CEO Babak Hassibi declined to comment on reports of talks with Apple, the support for Apple’s MLX framework suggests a strategic alignment with Apple’s ecosystem. Future announcements regarding official partnerships or device-specific optimizations could significantly impact the adoption of local AI on iOS and macOS platforms.