引導式測驗 · 硬 等級
混合模型和稀疏模型測驗
查看 Mixtral 路由、專家啟動、總參數與活動參數,以及稀疏模型的實際記憶體和部署權衡。
問題 1 的 8
在 Mixtral 8x7B 中,有多少專家處理每個單獨的代幣?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- 在 Mixtral 8x7B 中,有多少專家處理每個單獨的代幣?
- 兩位專家
- 所有八位專家
- 恰好一位專家
- 每次隨機數
- 哪個元件決定將令牌發送給哪些專家?
- 分詞器
- 路由器(門控網路)
- 注意力層
- 嵌入表
- 儘管 Mixtral 8x7B 總共有大約 47B 個參數,但每個代幣大約有多少個活躍參數?
- 約470億
- 約70億
- 約130億
- 約700億
- What must a Mixtral serving setup account for even when only two experts run per token?
- Only the active parameters ever need to be stored anywhere
- All experts must always fit in GPU memory with no offloading option
- Unused experts can be deleted without changing the checkpoint
- The full expert weights must be available to the system, with residency or offloading depending on deployment
- Mistral AI 在哪個許可下發布了 Mixtral,推動了開放採用?
- 阿帕契2.0
- 僅專有、閉源
- 通用公共授權 v3
- 知識共享非商業性
- What training steps does Mistral report for the Mixtral 8x7B Instruct variant?
- 它消除了對任何訓練資料的需要
- Supervised fine-tuning and direct preference optimization (DPO)
- 它完全消除了對 GPU 的需求
- 它保證模型永遠不會出錯
- How many expert networks are available in each Mixtral 8x7B sparse mixture-of-experts layer?
- 兩位專家
- Seven experts
- Eight experts
- Twelve experts
- Which languages does Mistral list for Mixtral 8x7B?
- Yes; all unused expert weights are deleted
- Yes; it means the model is dense with 12.9B total parameters
- No; it means every parameter is active for every token
- English, French, Italian, German, and Spanish