FastContext Architecture: Measuring Local Exploration Delegation

Abstract: Microsoft recently proposed FastContext, an architecture where a frontier coding agent delegates repository exploration to a small, local language model. While the specific FastContext-4B model exhibited poor reliability, we isolated and tested the underlying delegation architecture using an off-the-shelf Qwen 3.5 4B model via Grove's exploration tool. Across 50 exploratory tasks, the results show a 61% aggregate reduction in billed exploration costs with minimal quality degradation (0.05 mean score gap), traded against a 2–7× increase in wall-clock latency.

The Delegation Hypothesis

Frontier Agent (e.g. Claude 3.5 Sonnet) mcp__grove__explore file:line context Local Delegate (e.g. Qwen 3.5 4B) Source Tree 10M+ lines
The FastContext Local Delegation Architecture

Last month, Microsoft published FastContext: a paper, a GitHub repository, and a family of fine-tuned local models (4B–30B). The core premise was architectural: frontier coding agents spend significant tokens on iterative repository exploration (grep, glob, read). By wrapping a local LLM behind an --explore tool, the main agent can delegate these search loops. The local model performs the heavy lifting of reading files and isolating lines of code, returning only curated context. Since local inference costs nothing per token, this architecture aims to drastically reduce the main agent's token consumption.

Microsoft claimed up to a 60% reduction in token consumption alongside minor end-to-end resolution gains. However, the repository and models were subsequently withdrawn. To evaluate the architectural premise independently, we tested the delegation mechanism using an isolated testbench of 50 exploratory tasks from the is-grep-enough dataset.

Model Implementation vs. Architecture

The FastContext architecture is fundamentally sound, but the initial model implementation was flawed. The FastContext-4B model exhibited low stability in tool calling and struggled to drive tool results toward reliable answers compared to off-the-shelf models. The original paper benchmarked its trained explorers against only two baselines: no delegation, and delegation to the same frontier model. It omitted the critical baseline for a local, cost-free pitch: a cheap, off-the-shelf small model.

Testing an un-tuned qwen3.5:4b model reveals that it significantly outperforms the purpose-built FastContext-4B at repository exploration:

model usable call grounded NOCALL (empty)
qwen3.5:4b (reference) 100% 99%* 0
gemma4:e4b 97% 88% 0
gemma3:4b (prompted-JSON) 95% 86% 4
fc-q8-fixed (FastContext-4B-RL) 88% 85% 9

The table above measures the first-move capability: turning a question into a usable tool call. A weak first move compounds. The FastContext model's empty completions burn turns in the explore loop, typically clustering on hard, long queries (e.g., L2, L5-redis) where model persistence is most required.

Cost Analysis: Billed USD vs. Token Counts

To test the architecture itself, we ran the same exploratory tasks two ways: the paid cloud model (Claude 3.5 Sonnet) exploring the repository itself, versus the same model delegating every exploration to a local qwen3.5:4b behind a single explore tool.

Measure it in dollars, not tokens. Quoting raw token reductions is misleading because token costs vary by tier. For example, Claude Code delegates read tasks to a subagent running on Haiku, which is ~5× cheaper per token than Sonnet. On the hardest cell (tokio-L5), the baseline spends 4.1M subagent tokens. While this dominates the raw token count, the actual cost is only ~$0.65. Consequently, we measure the billed total_cost_usd to capture the true economic impact.

Baseline metered costs range from $0.17 to $1.02 per cell. Delegation shifts the entire exploration tier off the cloud provider's bill onto a local qwen3.5:4b that costs $0. The table below compares the baseline versus the grove-explore delegation across 18 runs (one per repo × rung, under various grove steering modes).

Q is a 0–1 quality score from a blind judge, calculated as: completeness × grounding.

cell mode base $ deleg $ Δ cost wall (base→deleg) base Q deleg Q
django-L2strict0.2040.080−61%60s→128s (2.1×)0.970.72
django-L2standard0.2040.077−62%60s→149s (2.5×)0.970.95
django-L3strict0.1710.099−42%70s→157s (2.3×)1.000.85
django-L4strict0.2210.160−28%71s→214s (3.0×)0.970.95
django-L5strict0.2560.090−65%109s→146s (1.3×)1.000.95
django-L5balanced0.2560.106−59%109s→260s (2.4×)1.000.97
redis-L2strict0.2130.245+15%68s→481s (7.1×)0.980.83
redis-L2standard0.2130.094−56%68s→628s*0.980.63
redis-L3strict0.2300.116−49%65s→267s (4.1×)0.951.00
redis-L4strict0.3320.125−62%111s→304s (2.7×)0.950.95
redis-L4balanced0.3320.159−52%111s→385s (3.5×)0.951.00
redis-L5strict0.4150.185−55%190s→890s (4.7×)0.850.95
tokio-L2strict0.3940.155−61%167s→330s (2.0×)0.900.98
tokio-L2standard0.3940.077−80%167s→245s (1.5×)0.900.79
tokio-L3strict0.4670.119−74%100s→236s (2.4×)0.950.90
tokio-L4strict0.2650.137−48%143s→339s (2.4×)0.950.92
tokio-L5strict1.0250.143−86%243s→489s (2.0×)0.880.85
tokio-L5balanced1.0250.147−86%243s→566s (2.3×)0.880.85
Cost Reduction: Baseline vs Delegation (USD)
Dumbbell plot showing billed USD cost per cell. Blue dot (●) is baseline Sonnet, orange dot (●) is Qwen delegation.

* redis-L2 standard was timed while another run shared the GPU; its wall-clock time is inflated. Strict-arm aggregate (the 12 default-steering cells): $4.19 → $1.65, −61%, cheaper on 11/12, mean Q 0.95 baseline vs 0.90 delegation.

  • Cost Reduction: Delegation yields a −61% aggregate cost reduction (−51% mean per-cell), scaling up to −86% on the most exploration-heavy cell (tokio-L5).
  • Quality Maintenance: Blind-grading against frozen reference keys shows the self-exploring baseline scoring Q = 0.95 mean (across 12 Full cells), while delegation scores 0.90. The 0.05 gap occurs alongside a 61% cost cut.
  • The Exception: The redis-L2 cell is the only instance where delegation costs more (+15%). This is a pure text-search task where strict steering forced the 4B model toward structural tools, resulting in thrashing (19 explore round-trips). Under standard steering, it used pure grep and achieved a −56% cost reduction vs baseline. Steering must be task-shaped, not one-size-fits-all.

Trace Analysis: Tool Delegation Patterns

The aggregate numbers obscure how the outer model utilizes the delegation tool. Given a complex query, the frontier model decomposes the mechanism into an ordered sequence of narrow locator questions and fires them at the local model.

For example, in django-L5, the tool-neutral prompt was:

I'm planning a change to how persisting a model instance interacts with the database, so I need to understand the full journey of saving a single object. Starting from the public save entry point, through the once-per-save bookkeeping where lifecycle notifications are emitted, then how it decides between updating an existing row and inserting a new one, and finally how that choice is turned into an actual SQL statement run against the database connection — walk me through that whole flow and how the stages connect.

Claude generated five focused explore questions mapping to the code path:

  1. Where is Model.save() defined and what does it call in django/db/models/base.py?
  2. Where is Model.save_base() defined and where does it emit pre_save/post_save signals?
  3. Where is Model._save_table() defined and how does it decide between calling _do_update vs _do_insert?
  4. Where are Model._do_update() and _do_insert() defined, and what QuerySet/query-compiler methods do they call to run SQL?
  5. In django/db/models/sql/compiler.py, what do SQLUpdateCompiler.execute_sql and SQLInsertCompiler.execute_sql do to run SQL against the database cursor?

The frontier model already predicts the structural shapes (save_base, _save_table, SQLInsertCompiler). It delegates the grounding: verifying the exact file:line citations. Reading files to confirm these anchors is the expensive portion of exploration, and it shifts entirely to the local model.

Latency Trade-offs

The economic benefits of local delegation are traded directly against latency. A cold local model consumes ~45s on the initial call. Across the measured cells, inner exploration accounts for ~88% of end-to-end wall time. While dollar costs drop 51–86%, wall-clock latency increases by 2–7×. Local delegation trades money and privacy for time, making it viable for background processing or privacy-bound egress boundaries, rather than interactive human-in-the-loop debugging. (Note: Latency measurements for local model execution were observed on an AMD Ryzen 5 7600X with 32GB RAM and an NVIDIA RTX 3060 12GB GPU. Execution times will vary based on available compute).

Implementation in Grove

This architecture is currently implemented in Rust within grove. The grove serve --explore command exposes a single mcp__grove__explore locator tool to the outer coding agent. An inner pure-Rust explorer (core::explore) drives the bounded, ≤6-turn loop validated by this study, utilizing four inner tools (Grove, Read, Glob, Grep) and returning filesystem-validated file:line citations via an OpenAI-compatible local backend.

The study's three steering modes are available natively: standard (model tool selection), balanced (reconnaissance followed by plan commitment), and strict (grove-first). A fallback mechanism ensures that if the local model is unhealthy at startup, grove reverts to its seven structural tools.

Measurement Methodology: Real-world repositories (redis/C, django/Python, tokio/Rust) were pinned at fixed SHAs across five complexity rungs (L1-L5) with tool-neutral prompts. Each arm ran in an isolated container. Answers were blind-judged against a reference key that remained hidden during the run.

Provenance & References

A controlled, reproducible benchmark of agent code navigation. Every figure recomputes from committed evidence — nothing here is asserted that you cannot check.