Selecting smaller language models reduces step expenses but risks triggering repetitive failure loops that multiply corrective workload and rapidly drain operational funds. In our illustrative Play, altering the model size factor reveals this balance: Compute Budget rises from 254 to 404 at step 15. However, heightened error rates quickly reverse these savings, as compounding retries overwhelm the per-step cost benefits.
Key takeaways
- Halving Model Size Factor from 10 to 5 preserves Compute Budget at 404 units instead of 254 units at step 15.
- Increasing Base Failure Rate from 4 to 6 causes Compute Budget to deplete completely to 0 units by step 15.
- Compute Budget depletion accelerates over time when task failures multiply, dropping from 838 units at step 3 to 408 units at step 8 under higher error conditions.
- Prioritizing low error rates protects organizational capacity more effectively than simply reducing baseline model size.
Does cheaper model execution protect your compute budget?
Anyone funding an agent deployment must choose between burning more compute on heavy models or risking repetitive mistakes with lightweight ones. In this model, the baseline setting finishes with Compute Budget at 254 units at step 15, while reducing unit model scale finishes with 404 units. Conversely, elevating task mistakes drains the available pool down to 0 units at step 15. All these figures are illustrative model values that demonstrate how corrective feedback loops shape total system expenditure.
When an agent fails, it must try again. That extra work burns resources regardless of how small the base model is. The central question is whether the savings from a lean architecture hold up when errors begin to snowball into extended recovery cycles.
What happens when you adjust the operational levers?
The first lever is Model Size Factor, which sets the baseline resource drain per operation. Its starting value is 10, and dropping it down to 5 leaves Compute Budget at 404 units instead of 254 units at step 15.
Early in the schedule at step 1, the lighter setup preserves 964 units compared to 954 units in the baseline. By step 8, that spread widens, holding 684 units against 604 units, showing steady linear preservation of processing capacity across successive cycles.
The second lever is Base Failure Rate, which fixes the volume of execution failures that impede net resolution. Its starting value is 4, and raising this lever to 6 drops Compute Budget to 0 units by step 15.
Looking at step 3, the higher error setting preserves 838 units compared to 854 units in the standard case. By step 8, Compute Budget drops down to 408 units, and it hits 0 units at step 15, showing that failure growth accelerates the drain over time.
How do the interconnected model parts push on each other?
The playable structure relies on two central reservoirs that interact continuously. Unresolved Tasks holds work items waiting for successful completion, starting at an initial stock of 4 tasks. Fresh assignments enter through Task Arrival, while finished work exits via Task Resolution.
Compute Budget begins at 1000 units and steadily decreases as resources are spent. Compute Burn pulls from this pool during every active phase. This drain is driven by direct task attempts and corrective retries. When Base Failure Rate pushes work back into Unresolved Tasks, those pending jobs demand further processing, burning capacity in proportion to Model Size Factor until the reserve runs dry.
What do these results mean for agent design choices?
In this model, cutting model scale yields consistent arithmetic savings, but elevated failure rates create compounding depletion. Halving model weight stretches the reserve safely through step 15, whereas a moderate increase in errors empties the entire supply prematurely. This pattern shows that failure mitigation protects organizational capacity far more effectively than purely shaving execution overhead.
Anyone overseeing an automated workflow should watch the accumulation of pending retries before celebrating lower per-step execution costs. If an architecture makes frequent mistakes, those corrective attempts wipe out the benefits of running smaller components. In this model, keeping error loops in check is the primary requirement for operational survival.
What does this playable model leave out?
The scenario makes several simplifying assumptions to clarify the core feedback. It uses a fixed time step where each round represents one hour of operational activity. Task Arrival feeds work into the system at a constant pace rather than fluctuating unevenly across different cycles.
The model also treats failure dynamics and unit costs as fixed numbers without intermediate adjustments. It contains only two reservoirs and two primary controls, excluding secondary factors that might modify retry policies. Real systems are more complicated than any model, but omitting external complications makes the underlying relationship between failure loops and resource exhaustion easy to follow.
Where does the topic come from?
This playable model was inspired by the publication titled When Is Smaller Greener? Right-Sizing Language Models for Agentic AI, published by arXiv on 30 October 2026. The specific numbers, steps and outcomes discussed throughout this essay represent the model's own illustrative scenario.
Can you test these trade-offs yourself?
You can explore these trade-offs directly by adjusting the values in the playable model. Try lowering Base Failure Rate or raising Model Size Factor to see how quickly the resource curve responds to different operational balances across each phase of play.
The parts of the model
The question it asks: How does choosing smaller language models for agentic tasks trade lower per-step compute costs against compounding retries caused by task errors?
- Task Arrival · Source
- Generates baseline incoming agent work items in tasks per hour.
- Unresolved Tasks · Pool
- Stores pending and failed agent work items awaiting successful resolution in tasks.
- Task Resolution · Sink
- Disposes of successfully completed agent work items in tasks per hour.
- Compute Budget · Pool
- Tracks available organizational processing capacity in compute units.
- Compute Burn · Sink
- Consumes processing capacity through baseline execution and corrective retries in compute units per hour.
- Model Size Factor · Lever
- Sets the relative parameter scale and baseline compute burn per execution step.
- Base Failure Rate · Lever
- Determines the volume of task execution failures that reduce net task resolution.
Assumptions and limits
- Each step of Play represents one hour of agent operations.
- Unresolved tasks generate immediate corrective retry steps that burn compute proportional to the Model Size Factor.
- All numeric values such as baseline task arrival and unit costs are illustrative.
A teaching model with illustrative values; each Play is a scenario, not a forecast.
Sources
- arXiv (arxiv.org)
Thank you to arXiv for use of its open access interoperability.
About this model
Machinations makes these models with AI from a public source, and each one is a teaching scenario: its numbers are illustrative values chosen for the model, not measurements or forecasts. You can open this diagram in the Community to remix it and change it.