Grafana Loki setup in Minikube

Photo by Chris Ried on Unsplash

Grafana Loki setup in Minikube

Basic concepts

  1. Loki - is a log aggregation tool, similar to ElasticSearch. But it is a slightly different in the way it stores the logs, and the way it indexes the logs.
  2. Loki allows you to collect and centralize all the logs from your containers, pods, nodes, clusters, and all Kubernetes-based infrastructure.
  3. Loki does not index the logs, but indexes the metadata of the logs.
  4. Promtail - is the agent that gets installed on all your worker nodes. It is responsible for collecting logs from all the pods running on the respective nodes and then it sends to the Loki.
  5. We also have Grafana as part of the Loki stack, which allows you to visualize, explore and search your logs.

Start minikube with 3 nodes (master, grafana and loki)

minikube start --nodes 3

Add helm chart repository

  1. Add repo:
    helm repo add grafana https://grafana.github.io/helm-charts
    
  2. Check repository list:
    helm repo list
    
  3. Update installed repo:
    helm repo update
    

Create custom values file for Helm

  1. Show default values for Grafana loki-stack chart:
    helm show values grafana/loki-stack
    
  2. Create your own custom values file 'loki-stack-values.yaml':
    # Enable Loki with persistence volume
    loki:
    enabled: true
    size: 1Gi
    promtail:
    enabled: true
    grafana:
    enabled: true
    sidecar:
     datasources:
       enabled: true
    image:
     tag: 8.3.5
    

Install Loki stack

  1. Install Loki stack:
    helm install loki-stack grafana/loki-stack --values loki-stack-values.yaml -n loki --create-namespace
    
  2. Check installed Helm release:
    helm list -n loki
    
  3. Check deployment in loki namespace:
    kubectl -n loki get all
    

Port forward Loki stack service into localhost port 3000

In another command terminal:

kubectl -n loki port-forward svc/loki-stac-grafana 3000:80

Get admin credentials for Grafana

  1. List secrets in loki namespace:
    kubectl -n loki get secret loki-stack-grafana -o yaml
    
  2. Copy admin password and decode it:
    echo "[admin_password]" | base64 -d
    
  3. Log into Grafana on localhost:3000
  4. Check added Loki datasource: loki

Test logs

  1. Press 'Explore'
  2. In top bar select 'Loki'
  3. Press 'Log browser'
  4. Select 'namespace'. For example 'kube-system'
  5. Select 'kube-apiserver': logs
  6. Press 'Show logs': logs2

Reference:

  1. Grafana Loki