The fastest way to make AI agents unsafe is to let every engineer invent their own private workflow. One person runs agents on dirty branches. Another allows shell access without review. Another trusts final summaries. Another shares raw traces with secrets. None of this looks broken during a demo. It breaks when the agent touches production-shaped work.
A reliability checklist is not bureaucracy. It is the minimum operating contract for non-deterministic software that can act.
Before the agent runs
- Start from a clean branch, clean worktree, or disposable sandbox.
- Write the success criteria before the prompt: tests, files, records, artifacts, or review steps.
- Declare forbidden actions: publish, push, purchase, destructive change, external message.
- Decide which tools are read-only, write-capable, or approval-gated.
- Capture the current environment: commit, dependencies, configuration, and test command.
This is standard automation thinking. You would not debug a flaky browser test without knowing browser version, fixture state, and test data. Agent runs deserve the same discipline.
During the run
The trace should show what happened, not what the model later claims happened. Capture tool names, arguments, outputs, exit codes, file diffs, timings, and approval decisions. OpenTelemetry’s GenAI work is pushing the industry toward standard telemetry for model calls, tools, and agent steps. The principle matters even if your first version is just local JSONL.
Do not overbuild the dashboard first. Build the evidence trail first.
After the run
- Compare the final claim against actual state.
- Rerun the agreed verification command yourself or through CI.
- Review the diff, not only the summary.
- Redact traces before sharing them.
- Classify failures: model reasoning, prompt ambiguity, tool error, environment issue, missing permission, bad grader, or skipped verification.
- Convert repeated failures into regression evals.
LangChain’s checklist recommends reading 20-50 real traces before building heavy eval infrastructure. That is excellent advice. You will learn more from actual agent mistakes than from abstract benchmark worship.
Capability evals vs regression evals
Do not mix these. Capability evals answer “can this agent handle harder tasks?” They should include tasks the agent currently struggles with. Regression evals answer “did we break behavior that used to work?” They should be boring and close to 100% pass.
A healthy reliability program has both. Capability evals push the frontier. Regression evals protect the floor.
The checklist that matters
- Trace every meaningful step.
- Verify state, not prose.
- Use code-based graders whenever the environment can answer objectively.
- Keep approval gates around irreversible or external effects.
- Separate raw local traces from sanitized reports.
- Version prompts, tool definitions, and harness rules with code.
- Feed production failures back into datasets.
This is where Agentic AI Reliability becomes more than a phrase. The teams that win will not be the teams that trust agents blindly. They will be the teams that make agent work inspectable enough to scale.
Sources and further reading
- LangChain, Agent Evaluation Readiness Checklist
- OpenTelemetry, AI Agent Observability
- Anthropic, Demystifying evals for AI agents
