Security Model

What RunLore is allowed to do, how that’s enforced, and the honest limitations. This is the runtime security model — how the agent behaves in your cluster. For the LLM-specific trust story — prompt injection, redaction boundaries, untrusted-output handling, network guards — see the LLM security architecture. For reporting a vulnerability, see SECURITY.md. For the deeper design rationale, see Design §9.

The guiding principle: safety is enforced in code, not promised in prose. The agent’s own claims (and the LLM’s output) are never trusted for an authorization decision.

Read-only by default

RunLore reads your cluster, metrics, logs, and network — its only writes are markdown to Git via reviewed PRs. Two independent layers keep it that way:

1. RBAC grants no write verbs by default. The ServiceAccount gets get/list/watch cluster-wide and nothing else (see RBAC below).
2. The action policy defaults to off (actions.mode). No cluster-mutating tool is wired to the LLM; the model can only propose an action, never dispatch one.

The Curator is cluster-read-only — its “writes” are PRs and issues against your knowledge-base repo, never the cluster.

The action gate (climbing the autonomy ladder)

When you enable actions (the suggest → approve → auto ladder), every executable action passes a server-authoritative gate (internal/action) that re-derives its safety from a canonical op registry and discards the model’s own metadata:

• Reversibility and blast radius come from the registry (only suspend / resume / reconcile are executable, all reversible, blast-radius 1) — an unknown op is treated as irreversible and refused.
• The policy envelope enforces reversible_only, max_blast_radius, an allowed-kinds list, and a namespace allowlist. flux-system and kube-system are always denied as targets, regardless of config; an empty allow.namespaces permits nothing.
• The gate is re-validated at every execution boundary (approval handling and auto), so a stale or tampered decision can’t slip through (defense in depth).
• The target name is corroborated against the resources the investigation actually observed server-side (the triggering workload + everything the GitOps read tools returned). An unobserved target never auto-executes (the action is downgraded to a suggestion); under approve it stays in the queue but carries an explicit possible-injection warning for the human approver.
• auto mode starts paused (kill-switch engaged, fail-closed) and is gated behind confidence/rate/blast limits. It exists but is not recommended on real clusters.

The action config is fail-closed: approve/auto won’t start without an approval token and an audit-log path (both modes execute cluster mutations, so both must be audited), and auto additionally requires an authenticated webhook, a positive confidence threshold and rate cap, and a non-empty namespace allowlist (see Configuration → actions).

External MCP tools

Remote MCP tools run outside RunLore’s action gate: the gate stops RunLore from executing cluster operations, but a remote tool that mutates state server-side would do so the moment it is called. Treat every configured MCP server as part of your TCB. Two controls bound this: per-server mcp.servers[].tools allowlists (a tool not listed is never registered, so the model can never call it), and mcp.require_allowlist: true to refuse startup unless every server is allowlisted. Tool output remains untrusted data (redaction + no-instruction-following), and per-server discovery failures are isolated.

Secret redaction at the LLM and egress boundaries

Tool output and incident text flow to a model provider and, for findings, into your KB PR and chat. internal/redact masks secret-shaped values at three boundaries:

1. Ingress — incident text before it enters the prompt.
2. Tool output — every tool result (pod/controller logs, git diffs, status/event messages) before it reaches the model provider.
3. Delivery — the finished investigation (root-cause summaries, evidence, suggested actions) before it’s copied into the KB PR body and chat.

Coverage (high-precision, masks the value while keeping structure): PEM private keys, JWTs, GitHub / Slack / AWS / Google / Stripe keys, user:pass@host URLs, Authorization headers, generic *(password|secret|api_key|token|…): <value> pairs, and the values under a kind: Secret manifest’s data:/stringData: block — including one surfaced inside a git diff. A masked Secret value is also learned: its base64 blob is decoded and both forms are scrubbed from the whole payload, so the same secret quoted decoded in a log line or encoded in an event does not outlive the manifest that names it. One shape is masked structurally rather than textually: a container’s env puts the sensitive word in the value of name: and the credential under the literal key value:, which no key-name rule can see, so resource_spec walks the decoded object and masks it before anything is rendered.

Least-privilege RBAC

The chart’s RBAC is scoped tightly (deploy/helm/runlore/templates/rbac.yaml):

• ClusterRole (read-only, cluster-wide): get/list/watch on Flux/ArgoCD resources and events, get/list on pods (status only — not pods/log). No write verb. patch is intentionally never granted cluster-wide.
• Namespaced Role for pod/controller logs: pods/log (raw log bodies, which can carry secrets/PII) is granted only over rbac.controllerLogNamespaces (default flux-system) — never cluster-wide.
• Defense-in-depth app-layer guard: because pod logs are streamed to the external LLM, the pod_logs tool is also constrained in the agent config to {the incident’s own namespace} ∪ config.investigation.pod_log_namespaces — a request for any other namespace is rejected before the cluster is queried, not just denied by RBAC. The chart auto-defaults pod_log_namespaces to rbac.controllerLogNamespaces, so the app-layer allowlist tracks the RBAC scope by default (no silent drift); leaving both at the defaults limits raw-log reads to the incident namespace plus flux-system. The app guard must stay a superset of the RBAC namespaces, or pod_logs is blocked at the app layer for namespaces RBAC would otherwise permit.
• resource_spec and list_resources’ read-only kind allowlist: rbac.resourceSpecRules grants get and list on the spec-bearing kinds these tools read (Services, workloads, NetworkPolicies, HPAs, PVCs, PVs, Nodes, scrape CRs …). Both verbs are needed because two tools share the rules: resource_spec reads one object by name (get), while list_resources enumerates a kind (list) and is registered automatically alongside it. Granting get alone leaves list_resources permanently denied — and the agent reports that as a data gap rather than an error, so it reasons around the missing evidence and nothing says why. Adding a rule REPLACES this list rather than merging into it — a values overlay or Kustomize patch that names one CRD leaves you with that one rule and silently drops the shipped ten, with no install-time error. To extend, copy the whole list and append. It ships populated — this is a default grant, not an opt-in menu: a stock install adds 10 rules — 28 resource types, 23 of them granted nowhere else — of cluster-wide get and list, on top of the ClusterRole’s 9 base rules. Set rbac.resourceSpecRules: [] to decline it in full (that costs you resource_spec and nothing else — workload_ownership’s owner-chain kinds are granted separately and unconditionally, so narrowing this list can never silently truncate an owner chain). Kinds with neither .spec nor .status — ServiceAccount, EndpointSlice, StorageClass, ConfigMap — are deliberately absent: the read would return nothing while the grant stayed entirely real, and a cluster-wide ServiceAccount read in particular exposes the IRSA / Workload-Identity role ARNs in its annotations. It is an allowlist, never a wildcard, on purpose: resources: ["*"] includes secrets. The tool refuses the Secret kind before and after resolution — a spelling that case-folds to “secret”, and any resource literally named secrets in any API group, is refused — but that is an application-layer policy. RBAC is the boundary. A kind missing from the list is reported to the model as a denial, never as a missing object, so a gap degrades the tool instead of fabricating evidence.
• Namespaced Role for actions: only when rbac.allowActions is set, get/patch on kustomizations/helmreleases over rbac.actionNamespaces — a bounded, opt-in blast radius that must mirror config.actions.allow.namespaces.

Credentials & the GitHub App

• Short-lived tokens, no PAT. The GitHub App mints an RS256 JWT (~9 min), exchanges it for a ~1-hour installation token, and refreshes ~1 minute before expiry. There is no long-lived personal access token; revocation is central (uninstall the App).
• Scope the App to the KB repo (Contents/PRs/Issues read-write), plus optional read-only on your GitOps source repos for the what-changed diff. Disable the App’s webhook. See Getting started → GitHub App.
• The clone credential is confined to one host — on the paths that read a repo URL from the cluster. RunLore attaches the forge token only to clones of forge.git_host (derived from github_api_url / gitlab.base_url unless you set it); any other host clones anonymously. This matters because a GitOps spec.source.repoURL is cluster state — a namespace admin who can create an Argo CD Application or a Flux GitRepository would otherwise choose where the token is sent. A GitHub Enterprise install with subdomain isolation must name forge.git_host, and fails config load until it does, so the credential is never quietly withheld from your own GitOps repo either. The confinement is implemented as the TokenHost field on the differ that what_changed and source_diff share.
• The catalog git-sync is outside that confinement, and rests on a different argument. The catalog syncer has no host field, so it sends its credential — catalog.git.token_env, or the shared forge GitHub App identity when that is unset — to whatever catalog.git.url names, on any host. forge.git_host does not gate this path and setting it changes nothing here. The reason this is not the same exposure as the GitOps case is that catalog.git.url is operator config, not cluster state: changing it means changing runlore.yaml, which no namespace admin can do. Treat it accordingly — point it at a host you would hand the forge credential to, and give it a dedicated read-scoped catalog.git.token_env when the catalog lives somewhere the forge App identity should not reach.
• Secrets by indirection. Every credential is referenced by the name of an env var / Secret key, never inlined in config — so config can’t leak a secret (see Configuration).
• Webhook auth. The incident webhook accepts a bearer token (server.webhook_token_env). It is mandatory once any model is configured (the serve path fails closed — an unauthenticated webhook must not reach the LLM and bill the model) and also enforced by config.Validate under actions.mode=auto. It is warning-only for the model-less log-only investigator. Pair it with a restrictive NetworkPolicy.
• Failed-auth backoff. Failed authentications on the control endpoints and the alert webhook are rate-limited per remote host: after 10 consecutive failures the host is blocked for 1s, doubling up to a 60s cap, and the block is checked before the token compare. A correct token always clears the counter. Behind a shared NAT this can delay a legitimate caller for at most one block window during a live attack. Tokens should be ≥128-bit random values (e.g. openssl rand -hex 16); the backoff is a brake on weak tokens, not a substitute for a strong one.

Tamper-evident audit log

Every action attempt — inputs, gate result, op, target, actor, outcome — is appended to a hash-chained JSON log (internal/audit): each record carries the previous record’s hash, the file is 0600 and fsync’d after every write, and a Verify pass detects the first broken link. Outcomes recorded: executed / dry-run / skipped / denied / failed.

The chain is load-bearing, not just an artifact tests check:

• Verified on startup, fail-closed under approve/auto. Both executing modes are required to set actions.audit_log_path (enforced by config validation), so the guarantee always has a chain to verify — neither can silently downgrade to an unaudited run. When the agent opens the log it re-walks the existing chain in a single read pass and reuses that same handle for appends (no verify→append re-read window). If a link is broken and actions.mode is approve or auto, startup fails — RunLore refuses to execute and audit cluster mutations against a history it can no longer vouch for. Under off/suggest (nothing executes) it logs a loud warning and keeps appending, so a read-only deployment isn’t blocked by a damaged file. An empty or absent log is a valid (zero-record) chain.
• Verifiable on demand. lore audit verify --path <audit.jsonl> (or --config <runlore.yaml> to read actions.audit_log_path) re-walks the chain out-of-band: it prints OK: chain intact (<N> records) and exits 0, or prints the first broken link and exits non-zero. Run it from CI, a cron, or an incident review.

Verification catches insertion, edit (any byte of a recorded field), and mid-chain deletion — each breaks a prev_hash/hash link.

Honest residual limit — tail-truncation. Dropping the most-recent records leaves a shorter but internally consistent prefix, which still verifies. Chain verification alone therefore cannot detect that the tail was lopped off. Fully closing this needs an external anchor (e.g. periodically publishing the head hash + record count to an append-only store the writer can’t rewrite), which is out of scope: a sidecar high-water mark doesn’t help — a privileged writer that can truncate the log can truncate the sidecar too, and making it crash-consistent is fiddly. Until an external anchor exists, mitigate operationally: keep the log on durable storage with restricted write access (ideally a medium where the agent’s own identity cannot rewrite history), and back it up.

The feedback channels (👍/👎) — exposure & trust model

Human feedback ratings weigh recalled-knowledge trust and re-arm the recurrence cooldown, so the channels that carry them are part of the security surface. The two channels have opposite exposure profiles and one shared trust model.

Shared trust model — votes are workspace/room-scoped opinions. Feedback is deliberately unprivileged: any authenticated member of your Slack workspace / Matrix room can rate (it is an opinion feeding the learning loop, not a cluster mutation — approve/reject keep their allowlist). The blast radius of a hostile voter is bounded by construction: one live vote per (incident trigger, user), latest wins (no stacking), a vote is one Bernoulli observation in the same Beta posterior as resolve signals (several independent voters are needed to move an established entry), recalled answers still pass the adversarial verify pass, recall confidence is hard-capped at 0.90, and the worst a 👎 campaign achieves is extra fresh investigations (cost, not wrong answers — decay fails toward re-investigation, never toward trusting). Every vote is an append-only ledger line carrying the voter’s stable id, so a campaign is auditable after the fact.

Slack (notify.slack.feedback_buttons) — an exposed endpoint, hardened. Clicks arrive on POST /slack/interactions, which must be reachable from Slack’s servers. Every request is verified against the app signing secret (HMAC-SHA256 over the raw body, ±5-minute timestamp window against replay, constant-time compare) before any parsing-derived action; unsigned or stale requests are rejected and the body read is capped at 1 MiB. Replay within the window is idempotent by the vote dedup. The message-update callback (response_url) is restricted to https://*.slack.com with a bounded client (no SSRF). Expose only the paths you use, not the pod: route /slack/interactions — and, if thread capture is also enabled, /slack/events too — through your ingress/gateway, and nothing else; the same listener also serves the alert webhook (open when server.webhook_token_env is unset!), /metrics, and the token-gated control endpoints, none of which belong on the internet. If any part of the server is reachable from outside, set server.webhook_token_env regardless of action mode.

Matrix (notify.matrix.feedback_reactions) — nothing exposed, one explicit check. Reactions arrive over the client-server /sync long-poll — an outbound HTTPS request authenticated by the notifier’s existing access token. No inbound endpoint, no signing secret, no NetworkPolicy change; responses are size-capped before decoding. The one attack Matrix enables that Slack cannot is attribution forgery: any room member could post their own message carrying the io.runlore.trigger_key content field and vote on it, misdirecting ratings to an arbitrary incident. The listener closes this by resolving its own identity (/whoami) at startup and counting a vote only when the reacted-to event was sent by the bot itself — and it refuses to listen at all until that identity is known. Operational requirement: because vote identity is room membership, use an invite-only room (and prefer disabling federation for it); in a federated room, remote homeservers assert their own users’ identities.

Slack thread capture (notify.slack.thread_capture) — a second exposed endpoint

Opt-in and separate from the feedback channels above: replying @runlore note: … inside an investigation thread lets a human write what they know straight into a knowledge-base PR, never applied automatically — but it is not reviewed and merged like any other curated entry. It has its own, deliberately different lifecycle: a note PR is markerless by design (no DupFingerprint), so curate’s dedup pass never auto-closes it as a “duplicate” of another note on the same recurring incident — closing one outright would discard a human’s contribution. It is not exempt from the stale sweep, though: an untouched note past the configured staleness window is closed like any other stale draft, with a comment that says so and invites reopening — routine housekeeping, not a rejection, and nothing is discarded (reopening the PR restores it for review). It shares the Slack interactivity callback’s exposure story above, but none of its vote trust model, which is why it gets its own section here instead of folding into that one.

Mentions arrive on POST /slack/events, a Slack Events API Request URL subscribed to app_mention only — never message.channels — so RunLore reads nothing in a channel it was not addressed in. The route exists on every deployment, but is a no-op unless opted in: with notify.slack.thread_capture off (the default) it answers 404, exactly like /slack/interactions above when no feedback path is configured. Once enabled, every request passes the identical check as the interactivity callback, before any parsing-derived action: HMAC-SHA256 over the raw body against the app signing secret, the same ±5-minute timestamp window against replay, constant-time compare, and a 1 MiB body cap. The Events API subscription handshake (Slack’s url_verification challenge) is verified the same way, so an attacker cannot repoint your endpoint by guessing the Request URL. See Expose only the paths you use above — /slack/events needs the same ingress treatment as /slack/interactions, nothing more.

Matrix thread capture (notify.matrix.thread_capture) — a widened /sync filter

Opt-in and separate from the feedback channels above, same as Slack’s: replying @runlore note: … inside an investigation thread lets a human write what they know straight into a knowledge-base PR — reviewed and merged like any other curated entry, never applied automatically. It shares Matrix feedback’s exposure story above (nothing inbound-reachable), but not its vote trust model, which is why it gets its own section here too.

The /sync filter widens while this is on — stated plainly, not softened. With notify.matrix.thread_capture off (the default), RunLore’s /sync filter requests only ["m.reaction"]: the process receives reactions in the configured room and nothing else. With notify.matrix.thread_capture: true, the filter widens to ["m.reaction","m.room.message"] — the process now receives message events from the configured room, where before it received only reactions. That is what the option does the moment it is on; it is not a hypothetical worst case.

RunLore does not act on every message it receives. Two checks run before anything else happens: is the message addressed to it (m.mentions, or the bot’s own MXID/localpart appearing in the body — the same detection addressed() implements in the listener), and is it rooted, via the m.thread relation or the m.in_reply_to fallback, in one of RunLore’s own investigation messages (the same sender-is-the-bot trust anchor the feedback-reactions attribution check above uses). A message that fails either check is dropped immediately, and its body is never logged. But — the honest part — every message in the room does transit the process first: the homeserver delivers it over /sync, and RunLore parses it far enough to run those two checks, before deciding to discard it. A member’s ordinary chatter, and anything typed in a thread RunLore never started, reaches the process and is read, even though nothing from it is ever stored or written anywhere.

The trade Matrix makes here is the transport’s genuine advantage over Slack. Enabling this needs no exposed HTTP endpoint, no ingress change, and no new Kubernetes permission: the widened filter rides the same outbound long-poll RunLore already runs for feedback_reactions, so there is nothing new to route through your ingress or open in a NetworkPolicy. Weigh the two transports’ opposite trade-offs before choosing between them: Slack’s thread capture keeps its inbound event stream limited to app_mention but adds an exposed, internet-reachable endpoint; Matrix’s stays unexposed but widens what it reads from a room it already polls.

Operational requirement, same reasoning as matrix.feedback_reactions above: use an invite-only room. With thread capture on, every room member’s messages reach the process (even though only an addressed, correctly-rooted one is ever acted on), so an open room widens who can even attempt to be heard by it.

Honest limitations

• The model sees cluster data. Even with redaction, tool output reaches your model provider. The strongest mitigation is self-hosting the model in-cluster. The redaction recall gaps above are real.
• RCA can be wrong. Frontier RCA is sub-50% on real incidents; unresolved is a first-class output and an adversarial verify pass can only lower confidence. Treat findings as hypotheses, and the human PR review as the load-bearing quality gate.
• Prompt injection is bounded, not impossible. A poisoned alert or KB entry can bias an RCA, but it cannot trigger a write — the action gate ignores model-authored authorization fields, and recall is disabled under auto-execution so a poisoned catalog entry can’t short-circuit into an action.