Grafana (MCP)

The Grafana MCP server provides comprehensive access to your Grafana instance and its ecosystem. It enables Holmes to search dashboards, run PromQL and LogQL queries, investigate incidents, manage alerts, and explore metrics — all through a single MCP connection.

Prerequisites

• A running Grafana instance (Grafana Cloud or self-hosted)
• A Grafana MCP server deployed and accessible from Holmes
• A Grafana service account token (see below)

Creating a service account token:

1. In Grafana, go to Administration → Users and Access → Service Accounts
2. Click Add service account and set the role to Viewer
3. Click Create, then go into the created service account
4. Click Add service account token → Generate token (no expiration or longest duration available)
5. Copy the token (starts with glsa_...)

Configuration

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

mcp_servers:
  grafana:
    description: "Grafana observability and dashboards"
    config:
      url: "https://your-grafana-instance/mcp"
      mode: streamable-http
      extra_headers:
        X-Grafana-API-Key: "<YOUR_TOKEN>"
    icon_url: "https://cdn.simpleicons.org/grafana/F46800"
    # These instructions were tested and produce improved results
    llm_instructions: |
      Always use Grafana tools (e.g. query_prometheus) for metrics/PromQL. Do not use kubectl top or prometheus/metrics toolset.
      NEVER answer based on truncated data. Retry with topk/bottomk or more filters until the query succeeds.
      For high-cardinality metrics (>10 series), ALWAYS use topk(5, <query>). Check cardinality first with count() if unsure.
      Standard metrics: CPU=container_cpu_usage_seconds_total, Memory=container_memory_working_set_bytes, Throttling=container_cpu_cfs_throttled_periods_total.
      NEVER use type "promql" embeds. ALWAYS embed charts: << {"type": "chart", "tool_call_ids": ["<tool_call_id>"], "generateConfig": "function generateConfig(toolOutputs) { /* parse toolOutputs[0].data array, return Chart.js config */ }", "title": "Title"} >>
      Embed at most 2 charts with line spacing between them.

Replace <YOUR_TOKEN> with your Grafana service account token.

After making changes to your configuration, run:

holmes toolset refresh

Create Kubernetes Secret:

kubectl create secret generic holmes-secrets \
  --from-literal=grafana-api-key="glsa_..." \
  -n <namespace>

Configure Helm Values:

# values.yaml
additionalEnvVars:
  - name: GRAFANA_API_KEY
    valueFrom:
      secretKeyRef:
        name: holmes-secrets
        key: grafana-api-key

mcp_servers:
  grafana:
    description: "Grafana observability and dashboards"
    config:
      url: "https://your-grafana-instance/mcp"
      mode: streamable-http
      extra_headers:
        X-Grafana-API-Key: "{{ env.GRAFANA_API_KEY }}"
    icon_url: "https://cdn.simpleicons.org/grafana/F46800"
    # These instructions were tested and produce improved results
    llm_instructions: |
      Always use Grafana tools (e.g. query_prometheus) for metrics/PromQL. Do not use kubectl top or prometheus/metrics toolset.
      NEVER answer based on truncated data. Retry with topk/bottomk or more filters until the query succeeds.
      For high-cardinality metrics (>10 series), ALWAYS use topk(5, <query>). Check cardinality first with count() if unsure.
      Standard metrics: CPU=container_cpu_usage_seconds_total, Memory=container_memory_working_set_bytes, Throttling=container_cpu_cfs_throttled_periods_total.
      NEVER use type "promql" embeds. ALWAYS embed charts: << {"type": "chart", "tool_call_ids": ["<tool_call_id>"], "generateConfig": "function generateConfig(toolOutputs) { /* parse toolOutputs[0].data array, return Chart.js config */ }", "title": "Title"} >>
      Embed at most 2 charts with line spacing between them.

Then deploy or upgrade your Holmes installation:

helm upgrade --install holmes robusta/holmes -f values.yaml

Create Kubernetes Secret:

kubectl create secret generic holmes-secrets \
  --from-literal=grafana-api-key="glsa_..." \
  -n <namespace>

Configure Helm Values:

# generated_values.yaml
holmes:
  additionalEnvVars:
    - name: GRAFANA_API_KEY
      valueFrom:
        secretKeyRef:
          name: holmes-secrets
          key: grafana-api-key

  mcp_servers:
    grafana:
      description: "Grafana observability and dashboards"
      config:
        url: "https://your-grafana-instance/mcp"
        mode: streamable-http
        extra_headers:
          X-Grafana-API-Key: "{{ env.GRAFANA_API_KEY }}"
      icon_url: "https://cdn.simpleicons.org/grafana/F46800"
      # These instructions were tested and produce improved results
      llm_instructions: |
          Always use Grafana tools (e.g. query_prometheus) for metrics/PromQL. Do not use kubectl top or prometheus/metrics toolset.
          NEVER answer based on truncated data. Retry with topk/bottomk or more filters until the query succeeds.
          For high-cardinality metrics (>10 series), ALWAYS use topk(5, <query>). Check cardinality first with count() if unsure.
          Standard metrics: CPU=container_cpu_usage_seconds_total, Memory=container_memory_working_set_bytes, Throttling=container_cpu_cfs_throttled_periods_total.
          NEVER use type "promql" embeds. ALWAYS embed charts: << {"type": "chart", "tool_call_ids": ["<tool_call_id>"], "generateConfig": "function generateConfig(toolOutputs) { /* parse toolOutputs[0].data array, return Chart.js config */ }", "title": "Title"} >>
          Embed at most 2 charts with line spacing between them.

Then deploy or upgrade your Robusta installation:

helm upgrade --install robusta robusta/robusta -f generated_values.yaml --set clusterName=YOUR_CLUSTER_NAME

MCP endpoint path

The Grafana MCP server serves on /mcp, not /sse or /mcp/messages. Make sure your Holmes config URL ends with /mcp.

Available Tools

The Grafana MCP server exposes ~57 tools organized by category:

Category	Key Tools	Description
Dashboards	search_dashboards, get_dashboard_by_uid, get_dashboard_panel_queries	Search, retrieve, and analyze dashboard configurations and panel queries
Datasources	list_datasources, get_datasource_by_name	Discover and inspect configured datasources
Prometheus	query_prometheus, list_prometheus_metric_names, list_prometheus_label_values	Run PromQL queries, discover metrics, and explore label dimensions
Loki	query_loki_logs, query_loki_stats, list_loki_label_names	Execute LogQL queries, retrieve log patterns and statistics
Alerting	list_alert_rules, get_alert_rule_by_uid, list_contact_points	Inspect alert rule configurations and notification channels
Incidents	list_incidents, create_incident, get_incident	Search, create, and manage Grafana Incidents
OnCall	get_current_oncall_users, list_oncall_schedules	View on-call schedules, shifts, and team assignments
Sift	get_sift_investigation, find_error_pattern_logs, find_slow_requests	Run Sift investigations for automated log and trace analysis
Pyroscope	fetch_pyroscope_profile, list_pyroscope_profile_types	Fetch continuous profiling data
Navigation	generate_deeplink	Generate deeplink URLs for Grafana resources

For the full list of tools, see the Grafana MCP Server documentation.

Testing the Connection

holmes ask "List all Grafana dashboards"

Common Use Cases

holmes ask "show me memory and cpu usage by namespace for the past day?"

holmes ask "Run a PromQL query to show CPU usage for the checkout-api pods over the last hour"

holmes ask "Search Loki logs for errors in the user-service namespace in the last 30 minutes"

holmes ask "What alert rules are currently configured and which ones are firing?"

holmes ask "Who is currently on-call for the platform team?"

Additional Resources

• Grafana MCP setup guide
• Grafana MCP Server (upstream)
• Grafana Service Account Tokens