O b
All posts in the "Observability" category.
-
Loki in Production: Labels, Per-Stream Retention, and the LogQL Alerts We Run
The production side of Loki — the label set we run, the 14-rule per-stream retention table, the LogQL alerts we actually rely on (audit, syslog, firewall), and the ingestion-rate gotcha that bit us early.
-
Deploying Loki on Kubernetes: SimpleScalable on Nutanix Objects
Deploying Grafana Loki in SimpleScalable mode on Kubernetes with Nutanix Objects as the S3 backend — the real values we run, schema configuration, and why we picked SimpleScalable over Distributed.
-
Alloy in Production: The DaemonSet Config Running The Conveyor's Observability
The real Alloy DaemonSet configuration running across fleet — pod logs, kube-audit, node metrics, kubelet, cAdvisor, etcd, apiserver, CoreDNS, CNPG, and synthetic probes — plus the production lessons that shaped it.
-
Grafana Alloy on Kubernetes: Three Deployments, One Collector
How we deploy Grafana Alloy on Kubernetes using three separate topologies — a DaemonSet for pod logs and node metrics, a Deployment with a MetalLB VIP for syslog and SNMP, and a Deployment for OTLP traces. Plus why Telegraf earned a permanent seat at the table.
-
Nutanix Objects as the Storage Backend for Loki and Mimir
How we configured Nutanix Objects as the S3-compatible storage backend for Grafana Loki, Mimir, and Tempo — bucket architecture across two data centers, F5 GSLB for cross-DC HA, credential management with External Secrets Operator, and a dual-retention strategy that survives compactor mistakes.
-
Building an LGTM Observability Stack on Nutanix: Why We Did It and What It Looks Like
Architecture overview of a self-hosted LGTM observability stack (Loki, Grafana, Tempo, Mimir) running on Nutanix — why we built it with zero software budget on recycled hardware, what each component does, and the honest trade-offs five months in.