Upstage has released Solar Open 2, an open-weight large language model with 250 billion parameters designed for agentic workflows such as office productivity and coding. The model utilizes a Hybrid-Attention Mixture-of-Experts architecture that activates only 15 billion parameters per token to deliver efficient inference.

  • It features a 1M-token context window achieved by removing positional encoding in favor of intrinsic order encoding within linear-attention layers.
  • Training efficiency is improved by initializing the model with selective weight transfer from Solar Open 1, carrying over only 2.3% of weights.
  • The architecture interleaves three linear-attention layers with one softmax-attention layer, reducing KV cache memory usage to roughly a quarter of an all-softmax model.
  • The model supports English, Korean, and Japanese and is competitive with other open-weight models on agent benchmarks.

The release provides a high-capacity model optimized for long-context tasks and tool-calling capabilities while maintaining low inference costs.