Core · Investigation
Root cause with receipts — not another dashboard safari.
RemOps investigates with the tools you already trust: metrics, deploys, cluster events, and code signals. You get a causal chain, confidence, and a remediation plan ready for approval.
01 · Correlate
Forty dashboards become one evidence pack
Investigation pulls read-only context from connected integrations and the alert’s own labels. Deploys, pod restarts, latency cliffs, and config diffs get lined up on a single timeline so “what changed?” is answered with receipts.
- Alert labels seed the search (namespace, service, pod)
- Deploy and PR signals ranked by time proximity
- Cluster events and restarts folded into the same story
- Fallback heuristics when deep RCA is unavailable — still produces an approvable path
47 signals correlated
Datadog metrics · deploy markers · kube events · config history
02 · Hypotheses
Multiple theories. Ranked. Verified against prod evidence.
RemOps does not cling to the first plausible story. Competing hypotheses are scored, weak ones discarded, and the survivor is the one that survives the telemetry — the way a good war room works when seniors are awake.
- Parallel theories with supporting / contradicting evidence
- Confidence scores you can challenge on the dashboard
- Explicit “not root cause” notes so juniors learn the pattern
03 · Report
What happened. Why. What to do next.
The deliverable is a report a VP can skim and an SRE can verify: timeline, impact, root cause, and the exact change that started the fire. No “it might be the network.”
What happened
Alert fired for payments.api p99 > 400ms at 3:14 AM. Latency peaked at 1.84s. Checkout errors followed within two minutes.
Root cause
PR #4821 reduced payments-db-pool max_connections from 100 to 10. Under nighttime batch + checkout overlap, new connections queued and p99 collapsed.
Impact
Blast radius: payments-api only. Downstream wallet service degraded but did not page. No data loss.
04 · Remediation plan
Investigation ends with something you can approve
Plans name the action type, target, parameters, and expected verification. Status moves to AWAITING_APPROVAL. That is the handoff into the human gate — not a vague “consider rolling back.”
- Concrete actions (patch, restart, scale, revert)
- Dry-run and policy checks before you see the Approve button
- Next step: Human approval
Rollback payments-db-pool
Patch ConfigMap max_connections: 100, rollout restart deploy/payments-api, verify p99 < 300ms for 5 minutes.
dry-run passed · policies green · blast radius: 1 service
Shipping every week.
- Evidence packsTimeline + correlated signals on every incident.
- Confidence scoringHypotheses ranked with supporting evidence counts.
- Remediation parsingInvestigation output becomes approvable actions.
- Fallback pathHeuristic report when deep RCA cannot complete — still reaches approval.
Frequently asked questions
Typically tens of seconds to a couple of minutes depending on integrations and complexity. Soft-launch environments may use a faster heuristic path that still produces an approvable plan.
You review the full report on the dashboard — evidence, root cause, and plan. Reject or approve the remediation; re-run paths are available when new context appears.
Reject the plan. Nothing executes. The audit trail keeps the proposed cause so your team can correct course without a silent write to prod.
More context improves quality, but intake labels alone can start a useful investigation. Connect Datadog/Grafana/K8s/GitHub as you harden the loop.
Human approval. RemOps will not execute until someone with access signs off.
Trade dashboards for a verdict.
Connect your stack once. The next alert arrives with a causal chain and a plan.