Catch agent failures
before your users do.

Kalytera evaluates every interaction at every step, in real time — not a sample, not after the fact — and tells you exactly what broke, where, and why.

Enterprise AI agents run complex multi-step workflows. Unlike traditional software, agents think and act differently in every interaction — standard quality checks aren't built for that. Eval and observability platforms exist, but they use sampled data and after-the-fact analysis. They miss failures mid-workflow, including one-off failures that could be catastrophic.

• Evaluates every interaction at every step, the moment it happens
• Gives every interaction a quality score calibrated to your industry
• Surfaces one-off catastrophic failures immediately in the Failure Feed
• Groups repeating failures into named patterns with plain English root cause

From pip install to your first scored interaction in under two minutes.

$ pip install kalytera

import kalytera

kalytera.configure(
  api_key="kly_live_...",
  api_endpoint="https://api.kalytera.dev",
  agent_id="billing-agent",  # optional
)

@kalytera.watch
def support_agent(user_input):
    ...  # nothing else changes

kalytera.trace(
  session_id="session-001",
  step_number=1,
  step_name="classify_intent",
  input=user_message,
  output=agent_reply,
)

Every interaction gets a quality score from 0 to 100, computed from four dimensions evaluated by an LLM judge on every step.

An interaction passes when overall_score ≥ 0.7. Configurable per agent from the dashboard.

A healthcare agent and a marketing agent shouldn't be graded the same way. Kalytera ships with quality weightings tuned per industry.

Every failed interaction is classified into one of seven failure types.

Three views. Each answers one question: is my agent healthy, what's failing right now, and exactly what happened in this one interaction.

Quality score trend over 7 days, today's pass rate, active failure count, and top 3 failure types ranked by frequency.

Every failure the moment it happens. One-off failures shown individually. Repeating patterns grouped automatically once 5+ failures share the same type and step, with a one-sentence root cause.

Full step-by-step trace. Per-step quality score. The failing step highlighted. Failure reason in one plain English sentence — not a JSON dump.

Authenticates Kalytera and sets defaults. Call once at startup before any trace() calls.

kalytera.configure(
  api_key: str,
  api_endpoint: str = "http://localhost:8000",
  agent_id: str | None = None,
) -> None

A decorator that instruments any agent function. Captures input, output, and latency automatically.

@kalytera.watch
def your_agent(user_input: str) -> str:
    ...

Works with any Python callable — LangChain chains, CrewAI agents, AutoGen agents, or a plain function.

Manually log one step of an interaction. Use when your agent can't be wrapped with @watch.

kalytera.trace(
  session_id: str,
  step_number: int,
  step_name: str,
  input: str,
  output: str,
  tool_calls: list | None = None,
  metadata: dict | None = None,
  session_ended: bool = False,
) -> None

Four tables. Everything Kalytera stores, nothing more.

id · agent_id · session_id · step_number · step_name
input · output · tool_calls · latency_ms · timestamp

log_id · accuracy · goal_alignment · decision_quality
completeness · overall_score · passed · failure_type · failure_reason

agent_id · pattern_type · pattern_value · failure_count
failure_rate · pct_of_all_failures · root_cause · is_worsening