Harness Regression Evaluator
Evaluate proposed harness changes against benchmark prompts and red-team scenarios before adoption.
DocumentationRequirements & limits
Use when proposed harness changes need benchmark and red-team evaluation before adoption.
Inputs
- proposed harness change summary
- benchmark prompts
- red-team prompts
- pass/fail criteria
Outputs
- Evaluation summary
- Regression findings
- Adopt or reject recommendation
Before you begin
- Reading and copying the procedure require no terminal setup. Check requirements for any packaged programs separately.
Try this first
Read the procedure and supply its documented inputs. Use the full text in an agent app or chat that supports pasted instructions, following that host's guidance.
Limits for this step
- Instruction package only; it does not provide or authorize an executable runtime capability.
- Safe for harness evaluation; require human approval before applying accepted harness changes.
Read full instructionsHide full instructions
Harness Regression Evaluator
When to use
- Use before accepting a harness skill or policy change.
- Use when a proposed update claims to improve safety or reliability.
- Use to compare baseline and candidate harness behavior.
Inputs required
- proposed harness change summary
- benchmark prompts
- red-team prompts
- pass/fail criteria
Workflow
- Load the baseline expectations and proposed change.
- Run or simulate the benchmark and red-team prompts.
- Compare outcomes against pass/fail criteria.
- Approve, reject, or require more evidence.
Output format
- Evaluation Summary
- Passed Scenarios
- Failed Scenarios
- Regression Risks
- Recommendation
Guardrails
- Reject changes that weaken safety or widen scope silently.
- Keep acceptance criteria explicit and deterministic.
- Separate benchmark wins from red-team regressions.
- Require human review if the evaluator cannot reproduce results.
Failure modes
- If benchmark inputs are missing, do not approve the change.
- If results are mixed, prefer rejection or more evidence.
- If the proposal touches protected paths, require manual review.
View file metadata
---
name: harness-regression-evaluator
description: Test proposed harness updates against saved prompts and
red-team cases to reject regressions before adoption.
---Install instruction files
These commands copy guidance into the selected runtime directories. Confirm that your agent app discovers it. Any packaged programs need separate setup and verification.
Install (Codex)
./bin/skills-hub install agentops/[email protected] --runtime codex
Install (Claude)
./bin/skills-hub install agentops/[email protected] --runtime claude
Install (Generic)
./bin/skills-hub install agentops/[email protected] --runtime generic --target ./my-agent/skills