Seed-OSS is a series of open-source language models developed by ByteDance's Seed Team, designed for powerful long-context, reasoning, agent and general capabilities, and versatile developer-friendly features.

We have released Seed-OSS-36B to the open-source community under the Apache-2.0 license. Trained with 12T tokens, Seed-OSS-36B has achieved impressive results on mainstream benchmarks while maintaining good practical performance at a low cost.

Key Features

Native Long Context: Trained with up-to-512K long context natively.

Flexible Control of Thinking Budget: Allowing users to flexibly adjust the reasoning length as needed. This capability of dynamically controlling the reasoning length enhances inference efficiency in practical application scenarios.

Enhanced Reasoning Capability: Specifically optimized for reasoning tasks while maintaining balanced and excellent general capabilities.

Agentic Intelligence: Performs well in agentic tasks such as tool-using and issue resolving.

Research-Friendly: Given that the inclusion of synthetic instruction data in pre-training may affect the post-training research, we released pre-trained models both with and without instruction data, providing the research community with more diverse options.

Evaluation Results

Seed-OSS-36B-Base

Incorporating synthetic instruction data into pretraining leads to improved performance on most benchmarks. We adopt the version augmented with synthetic instruction data (i.e., w/ syn.) as Seed-OSS-36B-Base. We also release Seed-OSS-36B-Base-woSyn trained without such data (i.e., w/o syn.), offering the community a high-performance foundation model unaffected by synthetic instruction data.

Seed-OSS-36B-Instruct

Seed-OSS-36B-Instruct is a post-trained model version developed by the Seed team based on Seed-OSS-36B-Base. It approaches or even reaches the best level among open-source models of the same size in evaluations of mathematics, code, reasoning, agent capabilities, and long-text processing capabilities.

We hope that Seed-OSS-36B can bring new possibilities to the open-source community, and we welcome developers and researchers to use these models for experiments and applications. The Seed-OSS series will continue to iterate, and we look forward to sparking more innovative ideas together with developers.