Reviewing Knowledge

Your part of the learning loop. RunLore investigates incidents and proposes what it learned as a pull request to your knowledge-base repo. You are the editor-in-chief: you review the finding, refine it, and decide whether it becomes permanent, recall-able knowledge. This page is everything you do.

RunLore never writes to the catalog directly — it only opens PRs. Nothing enters your knowledge base until a human merges it. That’s the whole point: the knowledge is yours, reviewed, and in your Git.

Guided workflow: the kb-steward Claude Code skill walks you through PR triage, post-incident capture, and seeding the KB with your platform’s context — interview-style, PR-gated like everything else.

Four ways knowledge gets in

Knowledge reaches your catalog by four routes. They differ in who starts them and how much ceremony they carry — but all four end the same way: a pull request on your KB repo that a human merges. None of them writes to your catalog directly.

Investigation finding

The default path, and the subject of the rest of this page. RunLore investigates, verifies, and opens a PR labelled runlore + triggered. See Learning Loop for how a finding becomes an entry.

Thread capture — reply to the bot where the incident is being discussed

The knowledge an incident generates usually appears in the thread, not in the investigation: someone says “this always happens after a spot reclaim” and that sentence is the durable fact. Thread capture is how it stops evaporating there.

Reply in the investigation’s own thread:

@runlore note: this recurs after every spot-node reclaim, not a CNI fault

RunLore files it against that investigation’s KB pull request, or opens a standalone note PR when there is none, and replies with a link to what it wrote. Your words are recorded verbatim, attributed to you.

Notes accumulate. On the investigation’s own KB pull request each note is a comment — review feedback on a draft somebody else wrote, which a human folds in at merge time. On a standalone note PR, where the entry is nothing but your notes, every note after the first is appended to the entry itself, so a conversation that refines an answer — the cause, then the proof, then the fix — merges as one entry holding all of it rather than only the opening statement. The reply tells you which of the two happened.

Where a captured note pays off. The four routes above all end in a merged pull request, but they do not all produce the same kind of entry. A captured note carries no symptom → cause → resolution chain, so once merged it widens what kb_search finds during an investigation — measurably useful — while the adversarial verify pass has so far rejected every recalled operator note rather than letting it answer an incident outright. Learning Loop §10 has the measurement and why that is currently the safe behaviour.

Optionally, with a model.chat block configured, plain questions in the thread are answered too — from the investigation’s own evidence — and RunLore proposes a note itself when your message contained something durable. A model-drafted note is marked as model-drafted, never filed as your words.

Announcing a write to everyone else. By default a captured note is reported only into the thread it came from, so the knowledge lands but nobody outside that thread learns that it did. Setting notify.thread.announce_kb_updates (default false) announces every landed write, naming the pull request, the entry, who wrote it and which chat system they typed it in, and quoting the note. The value says where it lands — true and channel both mean each notifier’s own channel or room, thread means the thread the note was typed in, both means both. Two consequences to know before turning it on:

• channel gives you two messages for one write: the thread reply, which answers the person who typed, and the channel post, which reaches everyone who was not reading that thread. With several sinks that is the feature rather than duplication. With one transport it is an echo — the thread already lives in that channel — and thread is the answer: the announcement goes into the thread instead, and any sink that cannot reach it still gets its own channel.
• The announcement carries note content. A note written in a thread nobody else was watching is broadcast to every sink you have configured — the other chat system, any webhook endpoint. That is an operator’s call to make knowingly, which is why the key is off unless you set it. Routing it into a thread narrows who sees it on the originating transport; it does not stop the other sinks receiving it.

Setup and the full cost picture: Slack · Matrix · what conversational replies cost

kb-steward skill — the guided, interview-style route

A Claude Code skill that interviews you and writes well-formed OKF entries from your answers. Use it when you are capturing something larger than a one-line note: a post-incident review, your platform’s baseline context, or a pass over a backlog of open KB PRs. It is the writing half; RunLore is the diagnosing half.

Editing the repo directly

Your catalog is an ordinary Git repository of Markdown files. Clone it and open a PR by hand — for bulk corrections, or to import runbooks you already have. The OKF format is what the indexer expects — §2 below shows an entry field by field, and Design explains why that shape — and lore validate-kb <catalog-dir> checks a catalog against the same merge gate before you push.

1. What you see when RunLore finds something

Two things happen when an investigation produces a confident, verified finding:

1. A chat message (Slack/Matrix) — the ranked root cause with confidence, the evidence trail, suggested next steps, and a link to the KB pull request. It’s a notification, not where you act.
2. A pull request on your KB repo (forge.kb_repo), labelled runlore + triggered (“raw finding, not yet worked”). This PR is where you review and approve.

If a finding isn’t confident/verified (e.g. the investigation was inconclusive), RunLore opens no PR — by design, it won’t fabricate knowledge.

Expected triage volume

Be honest with yourself about the queue this creates. In a 6-day pilot RunLore drafted 29 PRs (~3.5/day), and ~72% were closed without merging — not because they were wrong, but because they weren’t worth permanent knowledge: benign infrastructure churn, synthetic test canaries, and near-duplicates of entries the catalog already held. That’s a real reviewer-fatigue risk: a queue that’s mostly noise trains people to stop looking.

Three forge knobs cut that volume before a PR is ever opened, so the queue you see is closer to the ~28% worth keeping:

• skip_verdicts: ["no_action"] — the biggest lever. A no_action verdict is RunLore’s own “benign / self-healed / synthetic; nothing to do” classification. Skipping it keeps those findings out of the review queue entirely while still notifying chat, so nobody’s blind to them — they just don’t become PRs.
• min_confidence (default 0.75) — the quality bar. Findings the model isn’t confident about stay chat-only, never a PR.
• dup_score (default 5.0) — the dedup threshold. A finding that closely matches an existing catalog entry is coalesced (a comment on the open PR) or dropped, instead of drafting a near-duplicate.

None of these fabricate or hide knowledge — every skipped finding is still delivered to chat. They only decide what’s worth a permanent, reviewed entry. Recommended starting point for a production install: skip_verdicts: ["no_action"], then tune min_confidence up if benign churn still leaks through. See configuration.md.

2. Anatomy of a proposed entry

Every PR adds one Markdown file (an OKF entry) under a type directory — incidents/kubecontaineroomkilled-for-oom-app-pod-a3938440.md (title slug + a short fingerprint, so two incidents sharing a title never collide). Here is a real one RunLore opened for an OOMKilled pod:

---
type: Incident
title: KubeContainerOOMKilled for oom-app pod
description: The container 'hog' is OOMKilled because its memory limit (100Mi) is too low.
resource: runlore-test/oom-app
tags: [runlore, incident, pod, runlore-test]  # + workload kind & namespace — recall signal
timestamp: "2026-07-03T09:14:00Z"             # OKF-recommended last-change stamp
fingerprint: a393844afec8e3d1…                # deterministic identity (resource + cause)
confidence: 0.9                               # queryable extension fields (OKF: frontmatter
provenance: [flux/oom-app-values]             #   is for what you filter/index on)
---

## Decision
- **why keep:** … the memory limit is too low …
- **confidence:** 90%

## Symptom
KubeContainerOOMKilled for oom-app pod

Affected resource: Pod runlore-test/oom-app

## Investigate                              # the evidence trail (what it observed)
- pod_status: hog OOMKilled (exit 137); memory limit 100Mi …

## Cause                                    # ranked root causes, each with a %
1. **The memory limit of 100Mi is too low for the workload.** (90%)

## Resolution                               # suggested, reversible-first
- Increase the memory limit for the 'hog' container. (reversible=false)

## Unresolved                               # what it honestly couldn't determine
- (none)

## Citations                                # OKF §8: the causing-change provenance
[1] flux/oom-app-values

The Decision card makes the merge call quick; the sections make the entry reusable knowledge for the next person (and for RunLore’s instant recall).

The same PR also keeps the OKF bundle self-describing: the entry’s link line is added to your index.md (when the bundle has one) and a **Creation** record is appended to log.md — all in the one diff you review.

Writing entries by hand? The field-by-field contract — which frontmatter keys the loader parses, which the merge gate requires, and what each one does for recall — is documented in okf-format.md, shipped as part of the kb-steward skill but useful on its own.

Related knowledge in the PR

When the catalog holds nearby entries — or the trigger has fired before — the drafted PR ends with a Related knowledge section: the closest existing entries at draft time (linked, with their BM25 score and affected resource) and — when the trigger has fired before — a Trigger seen ×N line pointing at the previous entry. Use it to answer the two review questions cheaply: is this a duplicate of something merged? and does this incident keep coming back? Scores are corpus-relative hints, not a ranking guarantee. A genuinely novel first sighting (no catalog hits, never seen before) gets no section at all — there’s nothing to show.

3. Reviewing it — three checks

Formatting is already enforced for you — a CI check on the KB repo validates each entry’s structure (lore validate-kb). Your job is the judgment the machine can’t make:

1. Is the cause real? Does the Investigate evidence actually support the Cause? (The evidence is quoted verbatim from live cluster/cloud/Git state.)
2. Does the cause explain this symptom? A confident, well-formed entry can still describe a related problem rather than the one that fired. Check the top cause answers the Symptom, not just “something nearby is wrong”.
3. Is it durable, generalizable knowledge — or transient noise? A self-resolving blip (a bootstrap race, a one-off capacity spike) is not worth a permanent entry, even at high confidence. Keep entries that will help the next incident.

Searching the KB from the CLI

The same BM25 index the agent uses for recall is available to humans:

# against a local checkout of your KB repo
lore kb search "crashloop web configmap" --dir ./my-kb

# or via config (catalog.dir), after `lore catalog sync`
lore kb search "oom worker"

# one entry in full — path, filename, or a query with a unique hit
lore kb show crashloop-web

--json emits the hits for scripting. --ledger /var/lib/runlore/outcomes.jsonl adds a RESOLVE column (how often each entry’s recalls preceded the incident actually resolving) when you have a copy of the outcome ledger.

This is also the 30-second evaluation path: clone any OKF knowledge repo and point --dir at it — no cluster, no model, no config.

4. Add context / refine it

The PR is just Markdown — edit it like any other PR:

• Sharpen the Cause, add a Resolution step, link a runbook or dashboard, correct the title, add tags.
• Add operational context RunLore didn’t have (“this recurs after every node rotation”, “owned by the data team”).
• Your edits are the knowledge — they’re what future recalls return.

(Prefer a quick comment over an edit? Comment for discussion, but only the merged file content is indexed.)

5. Approve or reject

• Approve = merge the PR. The entry is committed to your catalog, re-indexed within minutes, and becomes instant-recall-able for every future matching incident (RunLore answers in milliseconds instead of re-investigating). Add the accepted label if you want an explicit audit trail.
• Reject = close the PR. Use this for transient noise, a wrong/duplicate finding, or knowledge you don’t want to keep. Nothing is lost — RunLore will re-propose if the incident genuinely recurs.

That single merge/close decision is “validating a KB entry”. There is no separate approval UI — your Git review is the gate.

6. What happens after you merge

• Instant recall. On the next matching incident, RunLore recalls the entry (after confirming it against current state and re-verifying it) instead of running a full investigation — saving minutes and tokens. On-call sees an explicit ⚡ Instant recall block in chat quoting this entry’s cause, its human-reviewed resolution, and its resolve-rate — the visible payoff of your review.
• Outcomes feed back. RunLore records whether incidents that recalled this entry actually resolved. An entry with a poor real-world resolve-rate is automatically decayed out of recall — so bad knowledge stops being used without anyone policing it. (See the learning loop.)

7. Label reference

8. Optional: the backlog groomer

The backlog groomer keeps the PR queue tidy without you: it suppresses re-drafts of entries you already rejected, dedups near-identical PRs, closes stale unreviewed ones after a configurable age, promotes solved→ready-to-merge when the incident resolved, and opens a knowledge-gap issue when an unsolved pattern recurs. It only ever comments/labels/closes — it never merges.

It runs automatically inside the serve pod (leader-only, every 6 h by default) — in dry-run first: check the logs for curate dry-run: skipped forge write lines (and the audit log for "actor":"curate" records) to see what it would do, then set curate.sweeps.mode: apply to let it act, or mode: "off" to silence it. Every automated action, applied or skipped, lands in the same tamper-evident audit chain as cluster actions when actions.audit_log_path is set. A lore curate CronJob remains available for out-of-server runs. See getting-started.

The groomer also carries on-call feedback into your review: when responders hold standing 👎 votes on the investigation behind a pending entry (the 👍/👎 buttons or Matrix reactions on the chat notification), it posts a comment on the open PR — “On-call feedback: N standing 👎 votes on the investigation behind this entry” — naming the trigger. Treat it as a red flag on check 1/2 above: re-read the Cause against the evidence before merging. The same 👎 also re-arms re-investigation (it bypasses the recurrence cooldown), so a fresher, possibly corrected conclusion may arrive while the PR is still open — worth waiting for before you merge a contested diagnosis. The comment appears once per contested trigger, even across repeated groomer runs.

In one sentence: RunLore drafts; CI checks the format; you check that the cause is real, explains the symptom, and is worth keeping — then merge to teach it, or close to skip it.