Simurg
Open-source library that detects LLM output corruption mid-stream and aborts generation before bad tokens reach users, triggering automatic retries.
🔗 Visit Simurg
Description
When an LLM's output starts glitching mid-response — repeating itself, drifting into gibberish, or breaking structurally — most systems only catch it after the fact, once the bad text is already in front of the user. Simurg watches the token stream as it's generated and cuts it off the moment things go wrong, before anyone sees it.
Simurg is an open-source Python library (NumPy-only dependency) that monitors LLM token streams in real time for signs of decoding corruption: repetition loops, language drift, garbage output, and structural breakdown. It holds the first 350 characters of a response in a buffer, releasing them only after verification, then continuously re-checks every 400 characters. If corruption is detected, it halts generation and triggers a retry with a fallback model — a "zero-leak" protocol designed so corrupted text never reaches the user when the failure starts early. Under the hood, five detector ensembles (n-gram surprise, repetition sketches, SimHash drift, self-calibration, interpretable rules) feed a conformal fusion layer that combines their scores with statistical false-alarm guarantees. It's model-agnostic, working with any OpenAI-compatible inference API (vLLM, llama.cpp, TGI, Ollama, and others), reports a median detection latency of about 590 characters past corruption onset, and processes roughly 197,000 characters per second on a single CPU core. A rule-based tier works out of the box with no training required; an optional learned tier and a "SIMURG Monolith" variant add hallucination detection via logprob entropy and self-consistency checks. It's free and Apache-2.0 licensed.
💬 Our review
The short version: Simurg is a technically serious, genuinely useful safety net for teams running their own LLM inference, but it's a young open-source project (38 stars) that solves a narrower problem — decoding corruption, not general hallucination — than its README might suggest at first glance.
Unlike heavier guardrails frameworks such as Guardrails AI or NVIDIA NeMo Guardrails, which focus on validating structured outputs and enforcing content policies, Simurg is laser-focused on catching decoding-level failures (repetition, garbage tokens, structural breakdown) in real time and triggering a fallback retry — a genuinely different and complementary problem. The conformal-calibration approach and the five-detector ensemble suggest real engineering thought went into avoiding false positives, and the fact that it needs no training to get started (rule-based tier) lowers the adoption barrier considerably. The catch: at a median ~590-character detection latency, some corrupted text can still slip into the buffer before an abort fires, and being model-agnostic means you're responsible for wiring it into your own inference pipeline — there's no managed service. If you're self-hosting inference via vLLM, llama.cpp, or Ollama and want a lightweight corruption tripwire, it's worth trying; if you need broader content/structure validation, pair it with something like Guardrails AI instead.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Bibliothèque Python gratuite, licence Apache-2.0, aucune offre hébergée
Pros
fonctionne avec n'importe quelle API compatible OpenAI
aucun entraînement requis pour le palier de base
débit élevé (197k+ caractères/seconde sur un seul cœur CPU)
gratuit, licence Apache-2.0, dépendances minimales (Python + NumPy)
Cons
projet encore jeune (38 étoiles, 15 forks)
latence de détection d'environ 590 caractères, du texte corrompu peut atteindre le buffer avant l'arrêt
ajoute une couche de surveillance à intégrer soi-même dans le pipeline d'inférence