Built for the Agents of SigNoz hackathon

Don't just trust an AI agent. Make it prove every step.

Agent K investigates problems in your AI app, backs up every finding with proof, and only acts when the safety rules say it is safe.

View on GitHub

Evidence-backed findings. One safe action. Every decision on the record.

agent-k · investigation
alert received: error rate breached the objective
query SigNoz: failed traces after deploy v2
evidence: 214 failed spans linked to deploy v2
query SigNoz: deployment marker for support-api
evidence: prompt template changed at 10:02
confidence recalibrated by code: 0.94
policy gate: ALLOW rollback (all checks pass)
action: roll back to previous version
verified: error rate back to normal
Powered by

Standing on open, honest tools

SigNoz is the operating system for the whole investigation. Everything the agent sees, does and records lives there.

What it does

An investigator you can actually check

Agent K is not a chatbot for your logs. It is a workflow that produces evidence, takes one careful action, and watches its own work.

trace214 failed spans linked to deploy v2

Evidence feed

Every finding comes with proof

Each claim links straight to the traces, logs, metrics or deployment that back it up. No proof, no claim.

Agent K
MCP
SigNoz

SigNoz over MCP

Investigates through SigNoz

Agent K asks SigNoz for evidence through the Model Context Protocol (MCP). Every query is recorded as its own step.

$0cost per run
0ktokens used
0sseconds to answer

Sample values, shown for illustration.

Self-telemetry

Watches its own work

It records its own cost, tokens, time and queries, so you can audit how it reached an answer.

Loop breaker

Stops repeated queries

Cost watchdog

Caps spend per run

Guardrails

Knows when to stop

A loop breaker halts repeated queries and a cost watchdog caps spend. When either trips, it escalates to a human.

The Three Laws

Rules enforced by code, not by prompts

Three simple rules decide what Agent K may claim, what it may do, and what it must record. Software enforces each one.

Law 1

No claim without evidence

Agent K never guesses. Every finding links to the exact data that backs it up.

  • Every claim links to a real SigNoz trace, log, metric or deployment.
  • A link checker confirms every published link still resolves.
claimconfidence 0.94

The v2 prompt template caused the rise in failed answers.

tracefailed requests after deploy v2
deploymentdeployment marker for support-api v2

Law 2

No action without budget

Agent K can do only one thing: undo the last release. And only when every safety check passes.

  • The allowlist holds exactly one action: roll back to the last version.
  • If any check fails, it acts on nothing and escalates to a human.

All checks pass

  • Objective breached
  • Cause tied to a deployment
  • Confidence above threshold

Any check fails

  • No deployment cause found
  • Action not on the allowlist
  • Escalate to a human

Law 3

No self without telemetry

Every step is recorded, so you can see what Agent K did, how long it took, and what it cost.

  • Token counts and estimated cost are recorded for every model call.
  • Loop-breaker and cost-watchdog events are visible in the dashboard.
$0.004
cost / investigation
18.2k
tokens
42s
duration
11
MCP queries

Sample values shown for illustration, not fixed targets.

How it works

From alert to auditable report

Ten fixed steps run in order. The model proposes ideas inside some steps, but code decides what happens next.

01

Alert received

A SigNoz alert fires and reaches Agent K over a webhook.

Safety by construction

The sandbox is real, not a promise

Two design choices make the safety story hold up under scrutiny. Neither depends on trusting the model.

An isolated rollback service

Agent K never holds the keys to your infrastructure. A separate service does the one thing it is allowed to do.

Agent KPOST /rollbackdeployerDocker
  • A small deployer service is the only holder of the Docker access.
  • Agent K makes one authenticated call that carries no image reference.
  • The deployer picks the last known-good version and records a marker.

A code-only policy gate

The decision to act lives in plain code that you can read and test. The model is not in the loop.

  • Checks the allowlist, the alert, the cooldown and the confidence.
  • Checks the cause and the sandbox before anything runs.
  • Denies the action when any single check fails.
Four controlled incidents

Two safe to fix, two safe to refuse

Each incident is seeded on purpose as a controlled test. Agent K approves a rollback twice and refuses it twice, and it explains why each time.

Prompt-regression deployment

Rollback allowed

Symptom

A broken prompt template ships in a new version and failed answers rise right after the deploy.

Investigation

Agent K connects the error spike to the deployment marker and shows the failed traces.

Verdict

The cause is a recent deployment and the error budget is breached, so a rollback is allowed.

signoz signalError rate spikes after deploy v2
signalobjectivedeploy v2peak 66 err/min

Reported honestly as four out of four on four controlled scenarios, not as general production accuracy.

Questions

Straight answers for a careful reviewer

See it for yourself

Prove what it knows. Earn the right to act.

Read the code, watch the demo, and follow the build. Agent K shows its work at every step.

View on GitHub
Read the build blog