Setting Up a Monitoring Stack in Docker Compose
Using Grafana, Prometheus, Node Exporter, cAdvisor, and Loki, Exposed via NGINX
This guide provides instructions on setting up a comprehensive monitoring stack using Grafana, Prometheus, Node Exporter, cAdvisor and Loki. These components are orchestrated with Docker Compose and exposed via an NGINX reverse proxy, making them accessible through a single domain.
Components
Grafana: The analytics and monitoring solution with support for multiple data sources, including Prometheus.
Prometheus: The monitoring and alerting toolkit, collecting metrics from configured targets at specified intervals.
Node Exporter: A Prometheus exporter for hardware and OS metrics exposed by *NIX kernels.
cAdvisor: Analyzes resource usage and performance characteristics of running containers.
Loki: A horizontally-scalable, highly-available, multi-tenant log aggregation system.
NGINX: Used as a reverse proxy to expose Grafana on the internet securely.
Prerequisites
Docker and Docker Compose installed on your host machine.
Domain name configured to point to the host machine (for NGINX configuration).
Configuration Files Overview
docker-compose.yml
: Defines the services, networks, and volumes for the Docker containers.prometheus.yml
: Configuration file for Prometheus to define scrape targets and intervals.grafana.ini
: Configuration file for Grafana's settings, including SMTP for email notifications.loki-config.yml
: Configuration file for Loki to define the storage backend and other settings.domain.conf
: NGINX configuration for proxying requests to Grafana and handling WebSocket connections.
Installation Steps
Configure Docker Daemon: Enable Docker metrics by adding the following to
/etc/docker/daemon.json
and restarting Docker:{ "metrics-addr": "127.0.0.1:9323" } # Then reload Docker configurations: sudo systemctl reload docker
Install Loki plugin for Docker: Install the Loki plugin for Docker to enable log collection:
docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
Clone the Repository: Clone the repository to your host machine:
git clone https://gist.github.com/398e94fb0aebb994b25223d99e7ae769.git monitoring-stack cd monitoring-stack
Start the Services: Start the monitoring stack using Docker Compose:
docker-compose up -d
Access Grafana: Access Grafana at
http://<your-domain>
and log in with the default credentials (username:admin
, password:admin
).
Important Notes:
Replace placeholder values in grafana.ini with your SMTP credentials to enable email notifications.
Adjust the domain.conf to match your domain and specific requirements.
Review Docker and NGINX logs in case of any issues during setup.
How to send container logs to loki:
docker run --name nginx-loki -d \ -p 8080:80 \ --log-driver=loki \ --log-opt loki-url=http://localhost:3100/loki/api/v1/push \ nginx:latest