跪拜 Guibai
← All articles
Artificial Intelligence · Database · Product Manager

The Five Backend Requirements That Sound Simple but Aren't

By 吴琼琼 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Every backend engineer has been handed a one-sentence requirement that unravels into weeks of work. Recognizing the hidden scope — data volume, failure modes, downstream systems — before committing to an estimate is what separates a senior engineer from someone who just ships the first version and gets paged at 3 AM.

Summary

Product managers can describe a requirement in 10 seconds that takes weeks to implement correctly. A request to add one database column for a recommendation score actually demands a new history table, a weighted algorithm across five metrics, a daily incremental-update pipeline with sharded workers, and a caching strategy — all to avoid a 6-hour full recalculation on 2 million products. An export-to-Excel button on a 2-million-row table requires async job queues, streaming writes, and progress tracking just to keep the server from running out of memory.

Security shortcuts are equally expensive. Removing login from an internal admin panel to "move faster" led to scraped user data within two weeks. A CTO's demand to migrate from MySQL to MongoDB because "NoSQL is faster" ignored the fact that the user system was deeply relational; the real fix was three days of index optimization, which dropped query times from 200ms to 5ms.

The common thread is that stakeholders measure simplicity by how short the sentence is, while developers measure it by how many systems the change touches. Scheduled tasks are the purest example: a cron job that fails silently, runs twice in a distributed environment, or lacks idempotency becomes a data-integrity time bomb.

Takeaways
A "just add a field" request can require new tables, APIs, scheduled jobs, caching layers, and a migration plan — ask what the field actually does before estimating.
Exporting 2 million rows synchronously will OOM the server; use async job queues, streaming writes, and user notifications instead.
Removing authentication to ship faster is a false economy — scraped data or a security incident costs far more than the login form you skipped.
Switching databases because "it's faster" is a red flag; profile the existing system first — missing indexes are usually the real bottleneck.
Scheduled tasks need alerting on failure, idempotency for retries, distributed locking to prevent duplicate runs, and timeout controls so a hung job doesn't block the queue.
Product managers often add scope with "since we're at it" — clarify the minimum viable requirement before building the expanded version.
Conclusions

The gap between a stakeholder's "simple" and an engineer's "simple" is a scope problem, not a communication problem: stakeholders describe the outcome, engineers enumerate every system that must change to produce it.

Security treated as a feature to add later is security treated as optional — and in practice, "later" never arrives until after an incident.

Technology-switching decisions driven by conference talks rather than profiling data are a recurring pattern in engineering organizations; the fix is to require a POC with comparative benchmarks before any migration is approved.

The most dangerous phrase in product development is "we might as well" — it turns a minimal change into an unbounded project without anyone explicitly deciding to expand scope.

Concepts & terms
Incremental update (in data pipelines)
Recalculating only the records that changed since the last run, rather than reprocessing the entire dataset. Requires a mechanism to mark "dirty" records — often a Redis set or a database flag — and a periodic full recalculation to correct accumulated drift.
Idempotency (in scheduled tasks)
A task is idempotent if running it multiple times produces the same result as running it once. Achieved through batch IDs, deduplication keys, or conditional writes, so that retries after a failure don't create duplicate data.
Distributed lock
A mechanism — often backed by Redis, ZooKeeper, or a database — that ensures only one instance of a scheduled task runs across multiple servers. Prevents duplicate execution when the same cron job is deployed on several nodes.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗