Over the past few years, pretraining scaling laws have led to a broad consensus: model capabilities improve in a relatively predictable way as data and compute scale. But once large models enter real-world settings, a more practical question comes into focus: can they continue learning through sustained interaction with their environments and keep improving over time?

We recently released EdgeBench, an ultra-long-horizon benchmark built to measure learning from real-world environments. The benchmark comprises 134 realistic and diverse tasks spanning six major capability domains, each allowing agents to operate continuously for at least 12 hours.

Based on long-horizon interaction runs across these tasks, we find that the overall agent performance in "environment learning" closely follows a log-sigmoid curve, with an average goodness of fit of R² = 0.998. Across frontier models from different generations, agent learning speed also appears to double roughly every three months.

EdgeBench has now open-sourced 51 tasks and the full evaluation framework, enabling the broader community to further study how agents learn from real-world environments.

Project link: https://seed.bytedance.com/edgebench
Paper link: https://edge-bench.org/paper.pdf
Code link: https://github.com/ByteDance-Seed/EdgeBench
Data link: https://huggingface.co/datasets/ByteDance-Seed/EdgeBench

An ultra-long-horizon benchmark for measuring real-world environment learning

In real-world settings, model performance depends not only on what is learned during training. Much of the knowledge required for solving practical tasks is not explicitly present in training corpora. Real-world work also relies not merely on readily available information, but on repeated trial and error, feedback interpretation, and continual revision.

At the same time, real-world environments are constantly evolving. New tools, new problems, and new knowledge continue to emerge, meaning that no static training dataset can ever cover everything in advance. As a result, an agent's ability to learn from its environment and continuously improve task performance is becoming increasingly important.

Most existing benchmarks primarily measure knowledge and capabilities that models already possess. EdgeBench, by contrast, focuses on how agents learn from real-world environments when given sufficient time, feedback, and room for improvement. EdgeBench has the following characteristics in task design:

• Designed for environment learning: Each workspace, feedback signal, and evaluator is closely grounded in real-world practice, and the score reflects how the agent learns and improves within the environment.

• Ultra-long horizon: Every task supports more than 12 hours of continuous operation, and some extended experiments run for over 72 hours, allowing experience to accumulate over time. Among tasks with recorded human effort, human experts spent an average of 57.2 hours completing a single task, with the maximum reaching 320 hours.

• Coverage across six domains: The 134 tasks span scientific problems & ML, systems & software engineering, professional knowledge work, combinatorial optimization, formal math & theorem proving, and interactive games & simulators. They were iteratively developed by domain experts based on real-world problems, and more than 90% were built from scratch.

Task taxonomy of EdgeBench

Unveiling scaling laws in environment learning

In EdgeBench, an agent does not finish after a single submission. Instead, it continuously interacts with the task environment and receives feedback such as scores, error messages, and hints for improvement. As a result, these curves do not reflect an Agent's ability to solve a task in one shot. Rather, they capture an agent's ability to absorb feedback, adjust its strategy, and progressively improve its results through sustained interaction.

Based on roughly 38,000 hours of environment interaction across 134 real-world tasks, we find that agent performance is not static. Agents can continue to learn and improve through sustained interaction with their environments.

A closer look shows that growth trajectories in environment learning are far from uniform. Agents differ not only in final performance, but also in how they learn, absorb feedback, and improve over time. Some improve steadily, some make rapid early gains before plateauing, and others remain stuck for a long time before eventually breaking through.

Agent learning curves over a 12-hour period for three tasks, illustrating, from left to right, steady improvement, rapid early progress, and a late-stage breakthrough.

Take a 12-hour run on the gravitational-wave task as an example: GPT-5.5 improved its score from 42.8 to 67.0 over 247 scored attempts. This case shows that environment learning is more than simple iterative tuning: major score gains often arise when the agent redefines, decomposes, and reorganizes the problem in response to feedback.

The agent first makes the task measurable, then breaks down the failure, identifies the bottlenecks, and finally preserves what works while correcting the remaining errors.

If we shift from a single task to aggregate performance across tasks, a broader pattern emerges: this learning process is not random fluctuation, but follows a highly stable pattern.

For each model, the study covers all 134 tasks, with 3 independent runs per task, yielding 402 learning curves per model. Viewed individually, any single-task trajectory may appear noisy and volatile. But once these trajectories are aggregated and averaged by interaction time, they converge to a simple yet highly precise log-sigmoid curve:

This functional form not only fits the average learning trajectories of different models consistently, but does so with very high precision, achieving a mean R² of 0.998. This is not merely an empirical fitting result; it can also be understood through the lens of graph exploration theory.

Averaged across 134 tasks, overall Agent performance improves as environment interaction time increases. For more analysis, please refer to the original paper.

We also ask a further question: does the speed at which models learn from the environment change across generations?

To minimize the interference from differences in prior knowledge and baseline capability, we selected 18 tasks on which models showed similar initial performance, ran 2-hour evaluations on successive generations of models released between September 2025 and May 2026, and used the performance gains over this period to characterize their "learning speed". The results show that as model generations evolve, the speed of environment learning increases significantly. For the most advanced models available at the time, this speed approaches a doubling every three months.

Trends in learning speed across different generations of large language models.

Closing thoughts

As the speed at which models learn from the environment continues to increase, future differences between models may lie not only in their initial capabilities, but increasingly in how quickly they can learn after entering an environment. The ability to interpret feedback, accumulate experience, and adjust strategies will become ever more central to building durable advantages in long-horizon, open-ended tasks.

We also hope EdgeBench can provide a useful reference for related research and practical applications, and we look forward to more future work focusing on how models learn, adapt, and improve in open-ended environments. At the same time, we will continue to understand and study the capabilities that sustain value creation in real-world settings, as well as the key capabilities that will define the upper bound of future AI, and keep exploring the uncharted frontiers of intelligence.