Full Stack · Testing · Artificial Intelligence

One AI Gateway, Six Layers: The Blueprint for Enterprise AI Testing as Infrastructure

By 狂师 · Oct 23, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Testing teams are being pushed to quantify AI efficiency, while most AI capability still lives on individual laptops, cannot enter intranets, and produces no numbers management trusts. This platform design is one concrete attempt to make AI testing an organization-level infrastructure: measurable, retainable, and privately deployable.

Summary

A from-scratch enterprise AI testing platform is the centerpiece of AI Evolution Society's 2026 practical curriculum. The proposed architecture separates six concerns into six layers: user access, scenario services, AI capability, scheduling execution, data, and infrastructure. Scenario services are plugins, so a new test type mounts without touching the core; the AI capability layer is isolated behind a self-developed gateway, so model swaps and cloud-to-local switches do not ripple into business code.

The same design answers a survey the group ran before starting: nearly 95% of member companies have begun AI transformation, but 80% of real AI projects remain in knowledge management and development efficiency, and fewer than 40% reach RPA, Agent, or intelligent quality inspection. The platform is positioned as the missing unified carrier for more than 30 testing Agent Skills, and as the place where management can actually pull numbers like '80% of test cases run by AI' or 'weekly iteration changed to daily iteration'.

The specified stack runs from Python/FastAPI, LangChain/LangGraph, LiteLLM, Vue 3, PostgreSQL with pgvector, Redis, and RQ to Playwright, Locust, OWASP ZAP, DeepEval, Midscene.js, Docker Compose, and Prometheus/Grafana/Loki. The gateway is deliberately self-built rather than wrapped LiteLLM directly because its routing, degradation, and metering logic is course material.

Takeaways

— Nearly 95% of surveyed member companies have started AI transformation; about half report large-scale multi-department use, but 80% of live AI projects sit in knowledge management and development efficiency.

— Deep business AI projects such as RPA, Agent, and intelligent quality inspection have less than 40% penetration, and 60% of enterprises cite the talent gap as the top blocker.

— The platform design answers four failures of scattered AI tools: capabilities leave with departing staff, quality standards stay inconsistent, management cannot collect metrics, and public-cloud tools cannot run in intranet environments.

— The architecture has six layers: L1 user access, L2 scenario services, L3 AI capability, L4 scheduling execution, L5 data, L6 infrastructure; each layer answers one complete question.

— Dependencies flow only downward, allowing any layer to be refactored or replaced independently; scenarios are plugins, so adding a new test type does not require changing the core.

— The AI gateway is the platform's single control point: it exposes OpenAI-compatible APIs, routes by scenario, supports private/local inference switching, and unifies retries, rate limiting, and usage metering.

— The tech stack includes Python/FastAPI, LangChain/LangGraph, LiteLLM, Vue 3, PostgreSQL with pgvector, Redis, RQ, HTTPX/pytest, Playwright, Locust, OWASP ZAP, DeepEval/Midscene.js, Docker Compose, Nginx, and Prometheus/Grafana/Loki.

— For enterprises, the platform's stated value is efficiency plus certainty: dashboards expose AI test generation ratio, automated execution ratio, failure self-healing rate, and regression time.

Conclusions

The survey's own numbers frame the pitch: with 80% of AI projects stuck in low-threshold scenarios and less than 40% reaching deep business use, the hard part is not starting AI but moving it from personal productivity to shared business process.

Choosing a self-built AI gateway over LiteLLM swaps production convenience for teaching value; the routing, degradation, and metering logic may be worth studying, but a delivery team prioritizing reliability would likely prefer maintained code.

The claim that AI becomes the worker while people review is the sharpest break from ZenTao/TestLink-era platforms, yet the design does not specify how the human review step is enforced or measured; without that detail, 'quality standards solidified in the process' remains an intention.

Concepts & terms

AI gateway

A centralized layer through which all platform components call large language models. It exposes one OpenAI-compatible API, handles model routing, retries, degradation, rate limiting, and usage metering, and can switch entirely to local inference for private deployment.

Agent Skill

A self-contained, callable unit of AI-agent capability, such as document parsing, test data construction, script generation, failure diagnosis, or report generation. The group had built more than 30 of these for AI testing workflows.

RAG (Retrieval-Augmented Generation)

A technique that retrieves relevant information from a vector store and feeds it to a language model before generating output, used in the platform's AI capability layer to ground test knowledge.

MCP (Model Context Protocol)

An open protocol for connecting AI assistants to external tools and data sources; the platform's AI dialogue assistant uses it, along with Agent Skills, to let natural-language commands trigger platform operations.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗