A base model is valuable both for the capabilities that pre-training builds and for the starting point it offers for further training. Different domains, reasoning tasks and agentic applications call for different data, training objectives and behaviors. Releasing the weights at the end of pre-training lets developers build on the general capabilities already in place and choose for themselves where to take the model next.
Today we are releasing ERNIE-5.5-Flash-Base, a sparse Mixture-of-Experts (MoE) language model with 280B total parameters, 14B of which are activated per token. It was pre-trained from scratch on 33.5T tokens with a training context of up to 128K, and supports a 1M-token context for fine-tuning and inference. The weights are the checkpoint at the end of pre-training, before any mid- or post-training, and can serve as a common starting point for domain adaptation, instruction tuning and reinforcement-learning research.
