Deepseek training 2T and plans 8T model

DeepSeek is currently training a 2T-parameter model and has future plans to develop an 8T-parameter model. Their existing models include Flash, with a parameter count of 552 billion, and Pro, which features 1.6T total parameters and 49B activated weights per token. These developments are notable in the context of models like Mythos / Fable, which is estimated to have a 10T parameter count.

This report details DeepSeek's current 2T model training and future 8T model plans, unlike previous reports that focused on their 552B Flash or 1.6T Pro models.