AI Writes the Demo, You Write the Production System: A Java Engineer's Guide to the Last 10%

Yesterday I posted an article on Juejin titled 'Three Hurdles for Java Teams Adopting AI,' and a commenter said:

Let the boss do it.

I replied:

The boss uses AI to go all-in, developing up to 90%, but can't handle the remaining 10% himself, then hands it to the employees saying: 'Look, I got it to 90% in three days. The rest is yours — wrap it up quickly, we're going live tomorrow!' This is definitely not a joke; it's a real-life joke, hahaha.

After I sent it, I laughed for a long time, but then I couldn't quite laugh anymore.

Because I experienced this exact scenario just last month. On Monday, the boss threw over a demo video, saying 'Someone else built an Agent with AI in three days; you get one live this week too.' By Wednesday, he came to me with a piece of cursor-generated code: 'It's already 90% done. You just need to add security, auditing, and deployment.'

I opened the code and saw: it connects to the public GPT-4 API, customer phone numbers are lying in the logs, the tool invocation chain is hidden inside a prompt, and there's no callback for errors.

The so-called 90% is the 90% that can run a demo. What it lacks to go live is the soul of production-grade software.

This article starts from that real-life joke in the comments and discusses how a Java team can patch that 10% mess into production-ready engineering capability.

1. First, See Clearly: Where That 10% Is Hiding

AI writes code fast. A CRUD, an Agent skeleton, an MCP call — Cursor, Copilot, or Claude Code can get it running in minutes.

But a runnable demo ≠ production-ready. There are at least five dimensions of difference:

Dimension	90% Demo	100% Production-Ready
Model	Directly calls OpenAI / cloud API	Private deployment / gateway / rate limiting & circuit breaking
Data	Customer info sent directly just to make it work	Data stays within the domain, masked, auditable
Tools	Just needs to be callable	Calls must be traceable, rollback-able, with timeouts
Errors	Retry on error, refresh if that fails	Exception handling, degradation, manual fallback
Compliance	Go live first, talk later	Security review, audit logs, data governance

See it? What AI is best at is writing that 'runnable 90%'.

What it's not good at is precisely the 10% that determines whether it can go live: edge cases, data security, observability, auditability, rollback capability.

And these happen to be the bread and butter of backend engineers.

2. How a Java Team Can Patch the Three Hurdles of AI Adoption

The last article broke down three hurdles. Today, applying them to this '90% vs 10%' scenario, you'll see at a glance where you need to patch.

Hurdle 1: The Python Gap

Many bosses' demos are run using Cursor + Python scripts, but your company's ERP, permissions, MyBatis, and Dubbo are all in Java.

That 10% is: how to integrate the demo into the existing Java system, rather than starting over.

Route A: Spring AI (fits the Spring ecosystem)

// Minimal skeleton: Register existing Services as tools callable by the Agent
@RestController
public class AgentController {
    private final ChatClient chatClient;

    public AgentController(ChatClient.Builder builder, OrderService orderService) {
        this.chatClient = builder
                .defaultTools(new OrderTools(orderService))
                .build();
    }

    @PostMapping("/agent/chat")
    public String chat(@RequestBody String question) {
        return chatClient.prompt().user(question).call().content();
    }

    public record OrderTools(OrderService svc) {
        @Tool("Query order status by order number")
        public String queryOrder(@ToolParam("Order Number") String orderNo) {
            return svc.status(orderNo);
        }
    }
}

Route B: LangChain4j (lighter, runs without Spring)

public interface OrderAgent {
    @UserMessage("Customer asks: {{it}}")
    String answer(String question);

    @Tool("Query order status")
    String queryOrder(String orderNo);
}

OrderAgent agent = AiServices.builder(OrderAgent.class)
        .chatLanguageModel(OpenAiChatModel.builder()
                .apiKey(System.getenv("OPENAI_KEY")).modelName("gpt-4o").build())
        .build();

Core idea: Don't switch languages for AI; turn your Services into AI's toolbox.

Hurdle 2: Data Leaving the Domain

To be fast, the boss's demo most likely connects to a cloud-based large model. The first thing you need to change in that 10% you inherit is the model gateway.

@Configuration
public class LocalModelConfig {
    @Bean
    public ChatClient localChatClient(ChatClient.Builder builder) {
        OpenAiApi localApi = OpenAiApi.builder()
                .baseUrl("http://192.168.10.20:11434/v1") // Internal network Ollama
                .apiKey("ollama")
                .build();
        return builder
                .model(OpenAiChatModel.builder().openAiApi(localApi).build())
                .build();
    }

    @Bean
    public ChatClient rateLimitedClient(ChatClient client) {
        return client.mutate()
                .defaultAdvisors(new RateLimitAdvisor(100))
                .build();
    }
}

Data staying within the domain is the baseline; rate limiting and circuit breaking are the decency.

Hurdle 3: The Black Box That Cannot Be Audited

This is the most valuable part of that 10%. AI tool calls, parameter passing, and error correction are all black boxes, making post-mortems impossible when things go wrong.

Spring AI 2.0 places tool invocation logic into the Advisor chain, which is perfect for instrumentation:

public class AuditAdvisor implements Advisor {
    private static final Logger audit = LoggerFactory.getLogger("agent-audit");

    @Override
    public AdvisedResponse adviseCall(AdvisedRequest request, CallAroundAdvisorChain chain) {
        long start = System.currentTimeMillis();
        audit.info("AGENT_REQ ts={} user={} tools={}",
                start, request.userText(), request.toolNames());

        AdvisedResponse response = chain.nextAroundCall(request);

        audit.info("AGENT_RES ts={} costMs={} contentLen={}",
                start, System.currentTimeMillis() - start,
                response.response().getResult().getOutput().getText().length());
        return response;
    }
}

Determinism, auditability, rollback capability — these are the stock-in-trade of enterprise backend engineering. Welding them onto the Agent now is your moat.

3. Wrap-up Checklist: Next Time You Face This 10%, Just Tick These Off

Patch the language bridge: Use Spring AI / LangChain4j to integrate the demo into the existing Java stack.
Patch the model gateway: Private Ollama / vLLM, unified baseUrl, add rate limiting and circuit breaking.
Patch the audit trail: Advisor records input parameters, output parameters, and latency for every tool call.
Patch exception fallback: Have degradation strategies for tool call failures, model timeouts, and malformed results.
Patch data governance: Mask sensitive fields, ensure logs don't contain raw data, isolate permissions by business role.
Patch the release process: Complete security review, stress testing, and rollback drills before even mentioning go-live.

Remember: Ready to go live ≠ Runnable.

4. Final Words

AI won't make backend engineers unemployed, but it will eliminate those who can only write demos.

The boss using AI to reach 90% in three days isn't a bad thing. What's bad is someone genuinely thinking the remaining 10% is just 'wrapping up.'

That 10% is the soul of production-grade software: security, auditing, stability, maintainability. And this is precisely the decade-old bread and butter of backend engineering.

So next time the boss asks you to 'wrap it up,' you can reply:

Bro, I can take over the 90% AI wrote in a day. But this last 10% — that's where the real valuable work is.

Have you ever been handed a mess from an AI demo? Share in the comments, and I'll pick real ones to break down in the next article.

Want to systematically follow a 'backend-to-AI Agent practical roadmap'? Complete materials at: wangzhongyang.com

#AI Agent #Java #Backend Development #Spring AI #LangChain4j #Large Model Implementation #Programmer Transition to AI #Agent Engineering