跪拜 Guibai
← All articles
Agent · AI Programming · OpenAI

A Natural-Language Data Analyst That Queries SQL, Plots Charts, and Sanitizes Its Own Output

By 怕浪猫 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Natural-language SQL agents are easy to prototype and dangerous to deploy. This architecture addresses the two failure modes that block production use: destructive queries and accidental PII exposure. The three-layer security model and mandatory audit trail are directly portable to any internal analytics tool that lets non-technical users query a database.

Summary

A LangChain agent backed by GPT-4o reads a condensed database schema, generates SELECT-only SQL, and executes it against SQLite. A second tool produces matplotlib charts from query results. The pipeline runs inside a three-layer security fence: prompt-level rules, tool-level keyword blocking, and output-level sanitization that masks phone numbers, emails, and ID numbers.

An audit logger records every query with a risk score, flagging unlimited queries that lack a LIMIT clause as medium risk. The reporting layer converts raw results into structured markdown with statistical summaries, trend analysis, and actionable recommendations, enforcing a rule that every claim must cite a specific number rather than vague directional language.

The full agent wraps execution, sanitization, and auditing into a single `analyze()` call, making it straightforward to embed in a Streamlit frontend or an internal analytics bot.

Takeaways
— Schema descriptions are condensed to `table(col1 TYPE, col2 TYPE)` format to save tokens while preserving enough structure for correct SQL generation.
— Only SELECT queries are permitted; any statement containing DROP, DELETE, UPDATE, INSERT, ALTER, or CREATE is blocked at the tool level before execution.
— Output sanitization uses regex patterns to mask phone numbers, emails, and ID numbers in query results before they reach the user.
— A query auditor logs every interaction with a risk level: high for destructive keywords or sensitive column names, medium for queries missing a LIMIT clause, and low otherwise.
— Reports must cite specific numbers — "increased by 23.5%" rather than "has increased" — enforced through the prompt that drives the reporting LLM call.
— Results exceeding 50 rows are truncated to the first 50 rows in the response to avoid overwhelming the context window or the user.
Conclusions

Schema condensation is a cheap, high-leverage optimization. Dropping sample rows and keeping only column names and types cuts token usage dramatically without measurably degrading SQL accuracy for most analytical queries.

The risk-scoring heuristic that flags any query without LIMIT or COUNT as medium risk is a practical, low-effort guardrail against accidental full-table scans that could spike database load or leak large datasets.

Wrapping the entire agent lifecycle — execution, sanitization, audit — into a single method call reduces the surface area for mistakes when integrating into a frontend. The caller doesn't need to remember to sanitize or log separately.

Concepts & terms
Schema Condensation
A technique for reducing the token cost of database schema descriptions sent to an LLM by stripping sample data and formatting tables as compact `table_name(col1 TYPE, col2 TYPE)` strings rather than full CREATE TABLE statements or row samples.
Three-Layer Security Fencing
A defense-in-depth pattern for LLM agents that applies security rules at three stages: the system prompt (declaring what the agent must not do), the tool implementation (blocking forbidden operations before execution), and the output layer (redacting sensitive data from results before they reach the user).
Query Auditing with Risk Scoring
Logging every database query an agent executes along with a computed risk level — high for destructive keywords or sensitive column access, medium for queries lacking row limits, low otherwise — to create an audit trail for compliance and incident investigation.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗