Best alternatives to Kullback in 2026

Testing an AI agent is harder than testing regular software, because an agent's "correct" behavior often depends on a whole simulated world — the tools it had access to, the data it saw, the rules it was supposed to follow. Kullback's approach is to build that world automatically from a record of what actually happened, rather than asking a developer to hand-write it. Kullback is an open-source harness that processes AI agent execution logs to automatically reconstruct the tools, data, and operational rules used during past runs. It ingests traces, rebuilds the operational environment, replays the logs for verification, and produces a verdict based on the final state of the data rather than just reading the conversation transcript. It extracts data records, tool functions, policy rules, simulated users, and task verifiers from the traces, with published results based on real historical retail-transaction traces.