config-drift-checker
GitHub Actions tool that catches regressions in your Claude Code setup whenever models or configs change
🔗 Visit config-drift-checker
Description
Teams that lean on CLAUDE.md files, custom skills, and hooks to shape how their AI coding assistant behaves have a quiet problem: when Anthropic ships a new model version, or someone tweaks a hook, there's no automatic way to know if the assistant got worse at the tasks it used to handle well. config-drift-checker is a GitHub Action that runs your Claude Code configuration against a pinned performance baseline on every pull request, so drift gets caught before it ships.
It works by running real agent executions in throwaway copies of your repository, grading the results, and comparing them against the baseline — not a static lint check, an actual behavioral test. When a new model or Claude Code platform version comes out, it runs canary tests, and if performance holds up it opens a PR to update the pinned baseline automatically, so the baseline stays current without manual babysitting. It's distributed as an npm package for CI integration, licensed FSL-1.1-Apache-2.0 (reverts to plain Apache 2.0 after two years), and free aside from the Anthropic API calls it makes while testing.
💬 Our review
The short version: config-drift-checker addresses a real, underserved problem — nobody else is regression-testing prompt/config behavior the way you'd regression-test code — but it's a young, narrow tool built for one specific platform (Claude Code).
There's no direct competitor doing exactly this for Claude Code configs specifically; the closest comparisons are general LLM-eval frameworks (which test model outputs, not your CLAUDE.md/skills/hooks setup end-to-end) or just manually re-testing after every update, which most teams simply don't do. Its value is proportional to how much you depend on a carefully tuned Claude Code configuration — a team with a thin CLAUDE.md and no custom skills won't get much out of it, while a team with elaborate hooks and skills that took months to tune has a real safety net here. Worth adopting if your Claude Code setup is load-bearing for your workflow; skip it if you're using Claude Code mostly out of the box.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Licence FSL-1.1-Apache-2.0 (repasse en Apache 2.0 pur après 2 ans), gratuit hors coût des appels API Anthropic pendant les tests.
Pros
Exécution réelle d'agents dans des copies jetables du repo, pas juste un lint statique
Détecte automatiquement les canaris sur nouveaux modèles/versions Claude Code
Ouvre lui-même une PR de mise à jour de baseline quand c'est sûr
Gratuit, intégration GitHub Actions native
Cons
Spécifique à Claude Code, aucune valeur si vous utilisez un autre assistant
Utile surtout si votre configuration (CLAUDE.md/skills/hooks) est déjà élaborée
Coûts d'API Anthropic pendant les tests à votre charge