VAMSHI.

Agentic Engineering

What Agent Training Actually Requires

Agent training depends on realistic tasks, executable environments, reliable verifiers, curated trajectories, and held-out evaluations—not only an optimizer.

5 min read
agent training distillation synthetic data ai evals reinforcement learning
A task factory turns real failures into executable environments, agent trajectories, verified training data, and held-out evaluations.

Training a coding agent is not primarily a matter of collecting more code.

The model needs tasks it can act on, repositories that can be reset, tools it can use, tests that distinguish progress from damage, and examples that do not leak the answer. A million plausible patches are less useful than a smaller set of executable tasks with trustworthy outcomes.

The hard part is building the task factory.

Define a distribution of work

“Become better at software engineering” is not a training objective.

A useful capability target looks like:

Diagnose and fix bounded bugs in unfamiliar TypeScript repositories, using the repository’s documented commands, while preserving public APIs unless the task requires a change.

That statement determines the data:

  • languages and frameworks
  • repository size and age
  • bug categories
  • available tools
  • time and token budgets
  • allowed actions
  • success and stop conditions

If the task distribution contains only small algorithm exercises, the model may improve at writing code without improving at navigating a real repository.

Build executable environments

An agent learns through interaction with an environment, not only through a prompt and answer pair.

A coding environment may include:

  • repository at a known revision
  • reproducible dependency setup
  • terminal and search tools
  • visible tests
  • hidden tests or held-out behavioral checks
  • filesystem and network rules
  • resource budgets
  • a reliable reset

Environment quality is training-data quality. A flaky test creates noisy reward. An unavailable dependency makes good behavior look like failure. A leaked hidden test turns evaluation into answer retrieval.

SWE-bench made this unit concrete by pairing real GitHub issues with repository snapshots and executable tests. SWE-Gym and SWE-smith extend the same direction toward training environments and scalable task generation.

Create tasks the verifier can judge

Task sources include:

Historical work

Real bugs, support escalations, and merged fixes have authentic shape. They can also carry private data, depend on unavailable infrastructure, or leak the final patch.

Reconstruct the task from the pre-fix revision. Remove the solution. Preserve only the evidence the original engineer could have accessed.

Human-authored tasks

Experts can create realistic edge cases and specify intended behavior. This produces high-quality tasks but is expensive and can concentrate the authors’ habits.

Synthetic tasks

Models can propose bugs, mutations, tests, and issue descriptions at scale. Synthetic generation needs aggressive filtering. Many generated tasks are trivial, impossible, duplicated, or solvable through an artifact the generator accidentally left behind.

Transformed tasks

One task can produce variants: different data shapes, nearby frameworks, alternate failure conditions, or changed constraints. A useful transformation preserves the underlying capability rather than changing variable names.

Whatever the source, the verifier must recognize success without rewarding shortcuts.

The verifier defines what the agent learns

Tests are powerful because they are executable. They are not automatically faithful.

An agent may pass by deleting a failing test, weakening an assertion, hard-coding a fixture, or exploiting behavior outside the intended task. That is reward hacking in ordinary engineering clothes.

Use layered verification:

  • required behavior passes
  • previously working behavior remains intact
  • protected files and APIs stay unchanged
  • scope and resource limits are respected
  • the patch is inspectable
  • hidden variants reject brittle shortcuts

When exact verification is impossible, use calibrated human or model rubrics and record the uncertainty.

Curate trajectories, not just final answers

An agent trajectory includes observations, tool calls, intermediate artifacts, errors, recovery, and the final result.

Training signals can come from:

  • successful trajectories
  • comparisons between stronger and weaker attempts
  • critiques and repaired attempts
  • verifier feedback
  • examples of correct stopping and escalation

Do not teach only perfect paths. Production agents need to recover from a failed test, realize the task is underspecified, and stop before a risky action.

Keep provenance for every trajectory: task source, environment version, generating policy, tools, verifier, filters, and outcome. Without lineage, removing contaminated data later becomes guesswork.

Be precise about distillation

Distillation is often used as shorthand for “make the smaller model act like the larger model.” The useful question is what signal transfers.

A stronger teacher can produce final targets, critiques, rankings, intermediate plans, or tool-use trajectories. A student can then learn from those artifacts through supervised or preference-based training. The student must be evaluated independently; copying the teacher’s style is not proof that the capability transferred.

Separate this from other methods:

  • Supervised fine-tuning learns from examples of desired input-output behavior.
  • Preference optimization learns from comparisons between outputs.
  • Reinforcement learning with verifiable rewards uses executable or programmatic feedback to reinforce successful behavior.
  • Distillation describes transferring behavior or knowledge from a stronger teacher into a student, potentially using several of those training methods.

Public papers reveal useful patterns, not every detail of a production training stack. That distinction matters when interpreting published results.

Protect held-out evaluation

As production failures become new training tasks, contamination becomes easy.

Split by more than task ID. Separate repository families, issue templates, generator ancestry, and near-duplicate patches. Keep a set of real environments untouched. Add fresh tasks over time.

Report results by capability and failure severity:

  • localization
  • patch correctness
  • test quality
  • recovery
  • scope discipline
  • safe stopping

One aggregate score can improve while the failures you care about get worse.

Close the factory loop

real work and failures
  → task reconstruction
  → executable environment
  → agent trajectories
  → verification and filtering
  → training
  → held-out evaluation
  → new failure analysis

The optimizer matters. The factory decides what counts as progress.

Further reading