Measuring Agents in Production or Characterizing Agents in Production

Sources: https://openreview.net/attachment?id=mWxEAgz3xu&name=originally_submitted_PDF

Sources for data: Two sources.

1. interview case studies with deployment teams (big tech to startups)
2. Surveyed 306 practitioners across 26 domains, filtered to 86 deployed agents. (Time is April to Sept 2025).

Other key points:

1. In user facing systems, orgs prefer controllability and traceability over autonomy. While in academic research, autonomy, complexity and long-horizon agents are focused on.
2. Prompting > Fine-Tuning: 70% production systems rely on prompting without weight or fine-tuning. Reason is better models are released quickly. Keeping up with fine-tuned models becomes infeasible. User controlled system design makes it easier to keep up with this change.
3. Truly autonomous agents almost non-existent in enterprise environments.
4. Human evaluation / Human in the loop as primary method of evaluation. LLM-as-a-judge used as secondary, complementary checks.
5. 75% of teams abstain from benchmarks as standard ones don't map to real-world business logic. They rely more on A/B testing and expert user feedback.
6. Agents aren't usually deployed as independent systems, rather as a productivity tools for humans.
7. Such agentic systems are usually latency tolerant.
8. Use of frameworks:

a. in surveys 61% use third party frameworks with langgraph, langchain leading.

b. in case studies, 85% use custom in-house solution. While two teams migrated off crewAI when moving to productions. (THis was for flexibility, simplicity and security).

c. The trend: More mature the framework, more likely it is to drop framework.

9. Reliability is the primary bottleneck. Interestingly, despite reliability being unsolved, teams ship agents by:

a. Constraining environment and autonomy.

b. Read-only modes where agent generates reports, human review, sandboxed verification before prod integration on real use-cases, role based access controls etc.

Some eval related takeaways:

1. Combine HITL with LLM as judge: Judge gives a confidence score, all low confidence ones go to humans for review. From high confidence ones, randomly sampled ones go to humans.
2. A convergent recipe that teams seem to use are: establish gt sets, collect user interactions, iteratively expand the set with expert review. Task success, user outcomes, qualitative improvements rather than a clean performance delta (x% accuracy to y%)
3. Coding agents are rare case where fast automated verification exists. For others it's difficult to setup standardized benchmark, due to cost of expert-review, and changing client-specific requirements.

TITILE: Measuring Agents in Production or Characterizing Agents in Production