kateryna.ai

What It Is

Kateryna is an LLM cognitive uncertainty detection tool designed for RAG pipelines. Its core goal is to identify high-risk answers where the model “sounds confident but has no retrieved evidence.” It uses a three-state classification system: +1 means Grounded with supporting evidence, 0 means uncertain, and -1 means Ungrounded—confident but unsupported. It is not positioned as a replacement for generative models, but as an additional validation layer between RAG retrieval results and LLM outputs.

Core Capabilities

Kateryna focuses on cross-checking an LLM’s linguistic confidence against evidence retrieved by RAG. When the knowledge base returns 0 relevant chunks but the model still gives a definitive answer, Kateryna marks it as -1. Publicly listed features include three-state detection, RAG confidence scoring, linguistic analysis, and adapters for OpenAI, Anthropic, and Ollama. In official testing, across 7 hallucination-prone questions, the model fabricated 5 answers, and Kateryna marked those 5 as Ungrounded, with a reported detection accuracy of 78%. However, the sample size is small, so the conclusion should be treated cautiously.

Pricing and Integration

The core version is open source under the MIT License. It can be installed via pip install kateryna and is available on GitHub and PyPI. The Pro version has not yet been released. Planned features include audit logs, analytics dashboards, domain packs for legal/medical/financial use cases, threshold calibration, contradiction detection, document auditing, SOC2 logs, priority support, and SLA, but pricing has not been disclosed. Enterprise features are available by contacting the team.

Pros and Cons

Its strengths are a clearly defined problem scope, especially for blocking hallucinations before and after RAG systems go live; three-state logic is more useful for engineering decisions than simple confidence scores; and open source availability plus local Ollama support lowers the barrier to testing. Its limitations are that it depends on the recall quality of the RAG system itself, has no clear baseline without RAG, has limited public evaluation scale, and its Pro, compliance, and enterprise support features are still marked as Coming Soon.

Who It’s For and China Access

Kateryna is suitable for AI engineering teams building knowledge-base Q&A, customer service bots, compliance Q&A, and internal enterprise search assistants—especially teams already using RAG and needing to monitor hallucination rates. China access and payment methods have not been disclosed. GitHub and PyPI can usually serve as technical trial entry points, but actual access stability is unknown. For alternatives or complements, consider RAGAS, TruLens, LangSmith, Arize Phoenix, DeepEval, or a self-built evaluation workflow using domestic vector databases and local models.