Cloud server monitoring is the practice of collecting metrics, logs, and traces from cloud-hosted servers and alerting on the conditions that matter. The team that monitors well has alerts that fire before users notice, dashboards that show the right context, and a clear distinction between ‘interesting’ and ‘urgent’. The team that monitors poorly has alert fatigue, pages at 3am that mean nothing, and a culture that ignores alerts entirely - which is worse than not having them at all.
Table of contents
- The four golden signals
- What to alert on (and what not to)
- Logs, metrics, and traces
- Tooling: Prometheus, Grafana, and the cloud-native stack
- Setting up monitoring that works
- FAQ
The four golden signals
Latency, traffic, errors, and saturation - the four golden signals from Google’s SRE book. Every service has these four signals, and every service can be monitored by tracking them. Latency is how long requests take. Traffic is how many requests per second. Errors are the rate of failed requests. Saturation is how close to capacity the service is.
Latency matters more than the team expects. A service at 100ms p99 is very different from a service at 1s p99, even if both are ‘fast’. The team that tracks p50, p95, p99 latency separately sees the long tail that affects real users. A service with low p50 and high p99 has a small fraction of users with terrible experience, and the team that only tracks averages misses it.
Errors should be tracked by HTTP status code AND by application-level error semantics. A 500 response is an error. A 200 response that says {"status": "failed"} is also an error. The team that monitors application-level errors (business-logic failures) catches bugs that status codes alone miss. The team that only watches 5xx rates misses silent failures where the API returns 200 but the work did not happen.
What to alert on (and what not to)
Alert on user-impacting conditions, not on internal metrics. CPU at 90% is interesting; response time at 5s is urgent. Disk at 95% full is interesting; disk write failures are urgent. The team that pages on user-impacting conditions has pages that mean something; the team that pages on every metric spike has pages that mean nothing.
Use the ‘two alerts per week’ rule for paging alerts. If the team pages on more than two conditions per week, the on-call engineer is overloaded and starts ignoring pages. The team that has fewer paging alerts and more dashboard-only metrics has a sustainable on-call rotation.
Severity levels matter. Critical pages (3am wake-up) should be reserved for conditions that require immediate human intervention - service down, data loss, security incident. Warning pages (next business day) cover degradation that needs investigation. Info pages (dashboard only) cover trends that the team should track. The team that distinguishes severity has the right urgency for each condition.
Logs, metrics, and traces
Metrics are aggregated numbers over time. CPU usage, request count, error rate, latency. Metrics are cheap to store (Prometheus retains 15 days of in-memory data, then archives to disk) and cheap to query (sub-second on millions of data points). The team that uses metrics for dashboards and alerts gets fast queries and clear trends.
Logs are discrete events with context. A request log, an error log, an audit log. Logs are expensive to store (a busy service generates gigabytes per day) and expensive to query (full-text search across large datasets). The team that uses logs for debugging specific incidents and metrics for ongoing visibility gets the right tool for each job.
Traces are the path of a single request through the system. A trace shows the call from the load balancer to the API server to the database to the cache, with timing at each hop. The team that uses traces for latency debugging sees which hop is slow; the team that only uses metrics sees the total latency but not where it comes from.
Tooling: Prometheus, Grafana, and the cloud-native stack
Prometheus is the de facto metrics standard for cloud-native monitoring. It runs as a single binary, scrapes metrics from instrumented services via HTTP, and stores them in a time-series database. The team that runs Prometheus has open-format metrics (PromQL) and a vendor-neutral export path. The downside: Prometheus is single-host and needs federation or remote storage for multi-region deployments.
Grafana is the de facto dashboard standard. It queries Prometheus, CloudWatch, Datadog, and dozens of other backends. The team that uses Grafana gets one dashboard tool across all data sources. The team that uses the cloud provider’s native dashboard (CloudWatch, Azure Monitor) is locked in but has simpler setup.
Loki is the de facto log aggregation standard for cloud-native stacks. It indexes logs by labels (like Prometheus metrics) rather than full-text, which makes it cheap to store and fast to query. The team that uses Loki gets the same operational model for logs and metrics. The team that uses Elasticsearch for logs has more query power but higher cost.
Setting up monitoring that works
Start with the managed platform’s built-in monitoring. RunxBuild, AWS CloudWatch, GCP Cloud Monitoring, and Azure Monitor all ship default dashboards and metrics. The team that uses built-in monitoring has zero setup time and gets the platform’s defaults; the team that rolls its own from day one has higher setup cost.
Add application-level instrumentation once the platform metrics are not enough. Instrument key paths with Prometheus client libraries (or OpenTelemetry exporters). Track business metrics that the platform cannot see: signups per minute, payment successes per minute, queue depth. The team that instruments business metrics catches failures that infrastructure metrics miss.
Build dashboards for the team, alerts for on-call. Dashboards should be explorable (lots of context, lots of metrics). Alerts should be specific (one condition, one action). The team that confuses the two has dashboards that page and alerts that don’t show enough context to debug.
FAQ
What is the difference between monitoring and observability?
Monitoring is collecting predefined metrics and alerting when they cross thresholds. Observability is the ability to ask arbitrary questions about system behavior using metrics, logs, and traces. The team that monitors well has alerts; the team that has observability can debug novel issues that no predefined alert would catch.
How often should I check dashboards?
Dashboards should be checked during incident response and during post-incident review, not continuously. The team that watches dashboards all day has alert fatigue; the team that uses dashboards as a debugging tool during incidents uses them effectively.
Should I use a managed monitoring service or self-host?
Managed (Datadog, New Relic, Honeycomb) has lower setup cost and includes features the team would otherwise build (alerting, dashboards, log search). Self-host (Prometheus, Grafana, Loki) has lower per-host cost at scale and no vendor lock-in. The team with 5-20 hosts uses managed; the team with 100+ hosts often self-hosts.
What is the cost of monitoring at scale?
Datadog bills per host (~$15-30/host/month for the full product). New Relic bills per GB ingested. Self-hosted Prometheus+Grafana+Loki costs the disk and CPU for the monitoring stack but is otherwise free. The team that monitors 100 hosts spends $1500-3000/month on Datadog or ~1-2 hosts worth of resources on self-hosted.
How long should I retain metrics?
15-30 days for high-resolution metrics (the recent past, useful for incident response). 1 year for downsampled metrics (5-minute averages, useful for trend analysis). The team that retains forever runs out of disk; the team that retains only a week cannot investigate slow-burn issues.
What is SLO-based alerting?
SLO (Service Level Objective) alerting fires when the service is at risk of missing its error budget. Example: 99.9% availability SLO with a 30-day window means 43 minutes of allowed downtime per month. The team that uses SLO alerting pages on conditions that actually threaten user experience, not on every metric blip.
How this fits the rest of the stack
For a sense of what the full project costs before it commits, the RunxBuild hosting calculator shows the line items together. The API, the database, the storage, the worker, the bandwidth - each one is a separate number, and the team’s mental model for the platform is the sum of those numbers.
Useful related references: