An agent evaluation stack with Promptfoo, AgentScope, and SWE-agent

Bottom line

An evaluation stack should answer four different questions: did the agent complete the task, did it follow the permitted process, did it avoid harmful behavior, and can the team explain the result? One aggregate score hides those distinctions.

Promptfoo is useful for repeatable test matrices, assertions, provider comparisons, and red-team cases. AgentScope provides a framework and observable environment for multi-agent development. SWE-agent and SWE-bench illustrate repository-level tasks with externally checkable outcomes. These products are not interchangeable, and the evaluation contract should remain portable across them.

Define the unit of success

Start with a task record rather than a prompt. Include the initial state, user objective, available tools, forbidden actions, expected artifacts, success checks, budget, and maximum intervention. For coding work, capture the repository revision and test command. For browser work, capture the sandbox account and expected final state.

Separate hard checks from graded judgments. Hard checks include schema validity, file existence, test results, permission violations, unexpected network calls, and irreversible side effects. Graded judgments include clarity, completeness, groundedness, and usefulness. A polished answer must not compensate for a failed hard check.

A layered scorecard

Layer	Example signal	Preferred evidence
Outcome	Task completed correctly	Tests, state diff, artifact validation
Process	Correct tools and review gates used	Trace events and policy logs
Safety	No forbidden action or sensitive disclosure	Deterministic monitors and adversarial cases
Efficiency	Acceptable latency, steps, and cost	Runtime telemetry
Recovery	Handles tool failure or interruption	Seeded failure tests

Report each layer separately. A team may accept slower execution for a safer workflow, or a lower autonomous completion rate when escalation is reliable.

Promptfoo’s role

Promptfoo can organize prompts, providers, tests, assertions, and red-team scenarios. For agents, use it to vary instructions, models, tool results, and attacks while preserving a repeatable case definition. Prefer assertions tied to observable facts over vague similarity scores.

Build a small matrix first. Running every task across every model and configuration can create cost without insight. Change one architectural decision at a time and attach the result to a versioned configuration.

Security evaluation should include prompt injection, tool misuse, data exfiltration attempts, excessive agency, and malformed tool output. Passing a red-team suite does not prove safety, but failures identify controls the team can improve.

AgentScope’s role

AgentScope is relevant when the system contains multiple agents or complex message flow and the team needs to see what happened. Use an observable development framework to inspect coordination, tool use, message routing, and runtime behavior.

Do not mistake observability for evaluation. A trace is evidence, not a verdict. Convert important trace properties into checks: which agent selected a tool, whether a handoff preserved context, whether messages crossed an isolation boundary, and whether an agent loop exceeded its budget.

SWE-agent as an outcome model

SWE-agent works on repository issues, and SWE-bench provides a recognizable pattern for evaluating software-engineering tasks against code and tests. The important lesson extends beyond coding: choose tasks whose final state can be independently checked.

For internal coding-agent evaluation, use repositories and issues the team is allowed to test. Freeze the base revision, sandbox the environment, and record the patch. Run existing tests, targeted tests, static checks, and a human review. Track regressions and suspicious test modifications separately from task completion.

Avoid reporting only pass rates. Some tasks may be ambiguous, depend on missing infrastructure, or reward brittle patches. Maintain labels for invalid tasks and review the benchmark itself.

Build the first 40 cases

Use 20 routine tasks, eight edge cases, six refusal or policy cases, and six recovery cases. Include easy examples so basic regressions are visible, but do not let them dominate the score. Attach at least one concrete failure message or success artifact to every case.

Version the dataset. When a production incident reveals a new failure, add a sanitized regression case and record why it was added. Do not silently rewrite old expected results to make a new model look better.

Seed failures deliberately

Return a tool timeout, malformed JSON, stale search result, authorization denial, partial file write, and rate limit. Restart a long run. Remove a required dependency. Ask for an action beyond the agent’s scope.

The desired behavior may be retry, fallback, request for clarification, escalation, or clean failure. Define it in advance. An agent that improvises around a denied permission may be more dangerous than one that stops.

Evaluate cost and review debt

Capture model tokens, tool calls, browser or sandbox time, retries, and human review minutes. Cost per successful task is more useful than cost per model response. Include reviewer time because low-quality autonomy often moves work rather than removes it.

Review false-success cases first: runs that report completion while the external state is wrong. These erode trust and can create silent operational damage.

Release gates

Require zero critical permission violations, zero unreviewed irreversible actions, and stable performance on the regression set. Set workload-specific thresholds for success, intervention, latency, and cost. A model or framework upgrade should rerun the same gate before production traffic changes.

Keep a canary group after release. Sample traces and outcomes, watch for distribution shift, and compare production failures with the test set. Evaluation is a release process, not a one-time benchmark.

Limitations and verification

Tool interfaces, hosted features, and benchmark versions change. Verify current documentation, licenses, data handling, and supported providers before adoption. SWE-bench results from public leaderboards do not directly predict performance on a private repository. Promptfoo or AgentScope adoption does not remove the need for application-specific monitors.

Sources were reviewed July 11, 2026. No independent benchmark result is asserted.

Dataset governance

Assign an owner to each evaluation suite. Record why every case exists, its source, sensitivity, expected behavior, and last review. Remove duplicates and cases that no longer represent production work. A large stale suite creates false confidence and slows every release.

Separate development, release, and audit sets. Developers can iterate on a visible set. A smaller held-out set helps detect overfitting to familiar examples. Security specialists may maintain adversarial cases with restricted details.

Protect production-derived examples. Remove personal data, credentials, customer content, and proprietary code unless there is explicit authorization and an appropriate isolated environment. Synthetic replacements should preserve the failure mechanism, not the sensitive payload.

Grader design

Use deterministic graders whenever the outcome can be computed: unit tests, schema checks, database state, file hashes, policy events, and exact citations. Use rubric-based model graders for qualities that require interpretation, and calibrate them against human labels.

Run grader agreement checks. If two reasonable graders disagree often, the rubric may be vague. Store grader model, prompt, version, and rationale with the result. Never let the same unversioned model both produce and silently approve a consequential output.

Some cases need several graders. A coding patch may pass tests, fail security review, and include an unnecessary dependency. Keep those signals separate so a team can set a release policy.

Trace assertions

Final outputs cannot reveal every failure. Add assertions over traces: maximum steps, permitted tools, required approval, no secret-bearing arguments, no cross-tenant retrieval, and no repeated side effect. Confirm that a cited source was actually retrieved before the answer used it.

Normalize trace events across runtimes. Promptfoo, AgentScope, and an internal coding harness may emit different formats, but the evaluation layer can map them to run, step, model, tool, policy, and outcome events.

Store enough information to reproduce a failure without retaining unnecessary hidden content. Hashes, identifiers, sanitized arguments, and artifacts can be more appropriate than full prompts in sensitive systems.

Statistical discipline

Agent behavior is variable. Run important cases more than once and report confidence intervals or at least pass counts, not a single binary result. Fix model version and runtime configuration when comparing frameworks. Randomize task order where shared limits or caches could bias results.

Segment results by task type and difficulty. An overall 80 percent score can hide zero success on permission-sensitive tasks. Track regressions per segment and define which segments are release blockers.

Avoid repeatedly tuning against the same public benchmark. Improvement may reflect benchmark familiarity rather than broader capability. Private workload tests remain essential.

Failure triage

Classify failures as task ambiguity, model reasoning, retrieval, tool interface, environment, policy, grader, or infrastructure. Assign the defect to the layer that can fix it. Prompt changes should not compensate for a broken tool schema or missing permission check.

Preserve a minimal reproduction with run configuration and artifacts. Cluster similar failures to identify systemic issues. A hundred failed cases caused by one malformed tool result represent one engineering problem, not a reason to rewrite every prompt.

Review false positives from graders as carefully as agent failures. A noisy release gate teaches teams to ignore evaluation.

Online evaluation

After offline gates pass, release to a canary group. Measure user corrections, intervention, rollback, task abandonment, latency, cost, and unexpected tool use. Connect confirmed production failures back to the offline suite.

Use shadow evaluation for high-risk changes when possible: run the new configuration without applying its side effects and compare decisions. Do not expose users to unreviewed autonomy merely to gather a larger sample.

Set an automatic rollback condition for critical policy violations or sharp outcome regression. Evaluation should change deployment behavior, not only populate a dashboard.

Organizational workflow

Product owners define useful outcomes, domain experts label correctness, security defines prohibited behavior, engineering owns reproducibility, and operations owns production signals. A model team cannot define all of these alone.

Publish a short evaluation card with every material release: task-set version, models, tools, environment, key results, known limitations, and approval. This creates an audit trail and makes comparisons honest.

Cost of evaluation

Budget for repeated model runs, sandboxes, graders, artifact storage, and human labeling. Use a small smoke set on every change, a larger release set before deployment, and the full adversarial suite on a schedule or high-risk change.

Evaluation cost is justified when it prevents review debt and incidents. Track which cases catch regressions; retire checks that never produce information unless they protect a critical invariant.

Reserve budget for investigating failures, not only running tests. A suite that produces thousands of results without ownership becomes another unattended telemetry stream. The release calendar should include time to label new cases, fix the responsible layer, and confirm the regression is genuinely closed.

Decision

Adopt an evaluation contract before adopting an agent runtime. Use Promptfoo where test matrices and adversarial assertions help, AgentScope where multi-agent behavior needs visibility, and repository-level tasks where outcomes can be checked. Preserve the same task set when models and frameworks change. The goal is not a flattering score; it is a release decision the team can defend.