AgentSnap
A free, open-source Python testing tool that snapshot-tests AI agents — recording every LLM and tool call so you can catch when a prompt tweak or model swap silently breaks your agent's behavior.
🔗 Visit AgentSnap
Description
If you've ever changed one line in a prompt and had no idea whether it broke something else three steps downstream, you know the core problem with testing AI agents: their output can shift for reasons that have nothing to do with a code bug. AgentSnap borrows an old idea from web development — snapshot testing, where you save a 'known good' output and get alerted the moment a new run doesn't match it — and applies it to AI agents instead of web pages.
AgentSnap is an open-source Python testing framework (MIT license, Python 3.10+) that records every LLM and tool call an agent makes, then replays those recordings in CI for deterministic, zero-cost regression tests, or re-runs them live against real APIs to confirm behavior against the current model. It detects three kinds of regressions — structural (a different tool got called), argument (same tool, different inputs), and semantic (same tool and inputs, but meaningfully different output, checked via an LLM judge or offline embeddings) — and integrates with LangGraph, LangChain, Pydantic AI, and CrewAI across Anthropic, OpenAI, Gemini, Cohere, Mistral, Groq, and OpenRouter.
💬 Our review
The short version: if your team ships an AI agent and keeps finding out about broken tool-calling or drifted outputs from users instead of tests, AgentSnap gives you a free, lightweight way to catch that in CI before it ships — worth trying even at its current early stage.
AgentSnap is free and open-source, so the only real cost is integration time, and its pytest-fixture approach slots into an existing test suite rather than demanding a new testing philosophy. The three-tier regression detection (structural/argument/semantic) is a genuinely useful design choice: most ad-hoc agent testing only catches the first tier. The catch is maturity — with 1 GitHub star at the time of writing, this is a brand-new project with no production track record, so treat it as promising rather than proven and expect to file your own issues. Compared to hand-rolling a snapshot harness, AgentSnap saves real setup time for multi-provider agents; compared to full observability platforms, it's narrower in scope (testing, not live monitoring) but far simpler to adopt for a single repo.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Projet open-source gratuit sous licence MIT, aucune offre commerciale.
Pros
Gratuit et open-source, licence MIT
Trois niveaux de détection de régression (structurel, arguments, sémantique)
Mode replay déterministe pour CI sans coût API
Intègre LangGraph, LangChain, Pydantic AI, CrewAI
Cons
Projet très récent, seulement 1 star GitHub à ce jour
Pas de suivi en production ni de communauté établie
Nécessite du temps d'intégration pour instrumenter un agent existant