Agent Review Studio
Local, browser-based workspace for reviewing AI agent execution traces and catching regressions between versions
🔗 Visit Agent Review Studio
Description
When you tweak a prompt or swap a model for your AI agent, knowing whether it actually got better — not just "felt" better on the one example you tried — needs structured evaluation, not vibes. Agent Review Studio is a local-first tool for exactly that: you define evaluation cases, review the agent's full execution trace for each one, label where a claim or action failed, and compare versions side by side.
It runs entirely in the browser (built with Vite, data stored in IndexedDB), so there's no server to set up and nothing leaves your machine unless you export it. It supports multiple workspaces with archive/restore, pairs each claim with its supporting evidence inside a navigable execution trace tree, combines automated evaluators with human labeling for judgment calls, and produces baseline-versus-candidate metrics with regression gates — so you can set a threshold and get a clear pass/fail on whether a change is safe to ship. Results export as JSON evaluation packs. It's agent-agnostic (works with traces from any agent framework), free, open source, and at v1.5.0 with structured, ongoing releases.
💬 Our review
The short version: Agent Review Studio brings real evaluation discipline — baselines, regression gates, claim/evidence pairing — to a process most teams currently do by eyeballing a few transcripts, and doing it fully local/browser-based removes the usual friction of standing up an eval server.
Against LangSmith or Braintrust, both mature hosted eval platforms with deeper integrations and team collaboration features, Agent Review Studio's edge is being local-first and agent-agnostic — no account, no data leaving your machine, and no lock-in to a specific agent framework. The trade-off is exactly what you'd expect from a young, single-maintainer project (1 GitHub star): no team collaboration layer, no hosted dashboards for sharing results with stakeholders, and you're trusting a newer project with something as important as your regression-testing process. Worth using for solo or small-team agent development where local, private evaluation is enough; look at LangSmith or Braintrust once you need shared team dashboards.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Gratuit, open source.
Pros
100% local/navigateur — aucune donnée envoyée à un serveur
Portes de régression baseline-vs-candidate avec seuils configurables
Appariement claim/evidence dans un arbre de trace navigable
Agnostique au framework d'agent, gratuit et open source
Cons
Pas de collaboration d'équipe ni de dashboard partagé
Projet jeune, mono-mainteneur (1 star GitHub)
Moins d'intégrations que les plateformes hébergées établies