跪拜 Guibai
← All articles
AIOps

Natural-Language Monitoring: Prometheus MCP + WorkBuddy Build AIOps Inspection Reports

By JasonPan ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Natural-language monitoring is only trustworthy when the AI is constrained to real data and read-only actions; this setup puts those constraints in the prompt itself. Teams already running Prometheus can try MCP-based AIOps with three containers and no changes to their existing stack.

Summary

Prometheus and Node Exporter collect Linux CPU, memory, disk, network, and load metrics. A Prometheus MCP Server wraps the Prometheus API as MCP tools, and WorkBuddy calls those tools to turn natural-language requests into PromQL queries. Docker Compose runs three containers: Prometheus on host port 19090, Node Exporter on the internal Docker network, and the MCP server on 8080. A curl query against the Prometheus API confirms both scrape jobs are up before WorkBuddy connects.

WorkBuddy connects through a custom MCP connector pointed at http://<server-ip>:8080/mcp. The test loaded 28 MCP tools, including query, range_query, and list_targets. A user then creates an 'expert' in WorkBuddy with a detailed system prompt that enforces read-only behavior, one PromQL query per call, real metrics only, and explicit disclosure of missing data.

The prompt also requires a Grafana-style HTML report with KPI cards, trend charts, resource details, AI diagnosis, and a conclusion. That HTML is a static snapshot, not a live dashboard. In the sample report, CPU was about 2.4%, memory about 45%, and root disk about 66.5%.

Takeaways
— Three containers make up the stack: Prometheus, Node Exporter, and prometheus-mcp-server.
— Prometheus listens on host port 19090, the MCP server on 8080, and Node Exporter is exposed only inside the Docker network.
— The MCP server starts with --prometheus.url=http://prometheus:9090 and --mcp.transport=http.
— A successful startup is verified by querying `up` and seeing value 1 for both the `prometheus` and `node` scrape jobs.
— WorkBuddy adds the MCP server as a custom connector using type http and URL http://<server-ip>:8080/mcp.
— The test loaded 28 MCP tools, including query, range_query, and list_targets.
— The inspection prompt mandates one valid PromQL query per tool call and forbids fabricating or filling missing metrics.
— The HTML report is a one-time static snapshot; sample readings were CPU ~2.4%, memory ~45%, and root partition ~66.5%.
Conclusions

LLM-generated monitoring reports stay reliable only when the prompt forbids root-cause conclusions from a single metric and requires missing data to be labeled. That prompt does both, keeping the AI in the role of report generator.

Prompt engineering carries most of the operational weight here. The system prompt reads like an SRE checklist: read-only by default, one PromQL query per call, real metrics only, units and timezone specified, and a static snapshot declared up front.

Declaring the HTML a one-time snapshot is a deliberate honesty measure. A single range_query covers a fixed window, so labeling the report as static keeps expectations aligned with what the data can actually show.

Concepts & terms
MCP (Model Context Protocol)
An open protocol that gives AI applications a standard way to call external tools. Here it lets WorkBuddy invoke Prometheus query tools through an HTTP endpoint.
Node Exporter
A Prometheus exporter that exposes Linux host metrics such as CPU, memory, disk, network, and load for scraping.
PromQL
Prometheus's query language for selecting and aggregating time series metrics, used by the MCP server's query and range_query tools.
WorkBuddy
The AI client in this setup; it connects to MCP servers, interprets natural-language instructions, calls tools, and generates HTML reports.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗