Observability & AIOps

When we help

• Alerts nobody acts onMany alerts fire, and the ones that matter drown among the rest.
• Failures users find firstProblems show up when someone reports them, not in monitoring.
• Three tools for one questionMetrics, logs and traces live in different places, and troubleshooting means comparing timestamps.
• Network and systems apartA network fault shows up as a service fault, and finding the link takes time.

Metrics, logs and traces

Three kinds of data that answer different questions. Connected, for example through OpenTelemetry, they let you follow a failure from the alert to its cause.

• MetricsHow much and how often Metrics Numbers over time: response times, error rate, load and capacity. They are cheap to store and good for alerts and trends.
• LogsWhat happened Logs Event by event. Logs with common fields can be searched and tied to a specific request.
• TracesWhere the time goes Traces A request's path through the services, with the time spent in each step. A trace shows where a slow request actually gets stuck.

Alerts that point to something to do

An alert should mean someone needs to act. What is merely good to know belongs on a dashboard.

• Alerts on what users noticeErrors, response times and availability for the service, not every single port or CPU spike.
• One action per alertEvery alert has an owner and a runbook that says what to do.
• The right recipientThe alert goes to the team that owns the service, and severity decides whether someone is woken up.
• Alerts that lead nowhereAn alert nobody acts on is changed or removed.

AIOps: anomalies and probable causes

For us, AIOps means analyzing operational data to find anomalies and probable causes sooner, across networks and systems.

• AnomaliesPatterns that differ from normal for that particular time, before they cross a fixed threshold.
• Probable causesEvents in the network and systems are lined up against each other, so the alert shows what changed or stopped working first.
• Fewer alerts for the same failureRelated alerts are grouped into one case instead of many.
• Analysis of operations, not your own modelsBuilding, deploying and optimizing your own models is a different job.

One service at a time

We start where visibility is weakest and expand with the same pattern, so every step is usable right away.

• AssessmentWhich alerts exist and which lead to action What was missing the last time something went wrong Tools, data volumes and cost Result What to remove, change and add
• First serviceMetrics, logs and traces Dashboard and alerts on what users notice One runbook per alert Result One service with full visibility
• More servicesThe same pattern for the next service Common fields and names Network and systems in the same view Result Visibility that works the same everywhere
• AIOpsAnomalies in operational data Events lined up against each other Follow-up on what the analysis catches Result Anomalies and probable causes that show up sooner

The visibility is yours

Collection, dashboards and alerts live as code with you, and every alert has an owner and a runbook.

• Follow a slow request from the alert to its cause
• Add an alert through the code
• Walk through an alert with its runbook
• Collection and alerts as codeThe configuration in your repository, with a history of every change.
• Dashboards per serviceWhat shows whether the service is healthy, and where it is not.
• Runbooks per alertWhat the alert means and what to do.
• Access and ownersAccounts and keys stay with you, with a named owner per alert and dashboard.

Related services