Frontend · JavaScript · Programmer

The Standard Chunked Upload Recipe Fails on Real Networks — Here's the Fix

By ErpanOmer · Jul 7, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

File upload is a solved problem in demos but a persistent source of production incidents. The gap between the interview answer and a working implementation is three concrete engineering decisions — concurrency control, memory discipline, and state persistence — that directly determine whether an upload survives a train tunnel or a coffee-shop network.

Summary

Interview-ready chunked uploads — slice with `Blob.slice`, fire with `Promise.all`, merge on the backend — break down the moment they hit a real network. Browsers cap per-domain connections to six, so 100 simultaneous requests queue up and time out. A single failed chunk rejects the entire batch, discarding already-uploaded work. The fix is a native async queue that caps concurrency at four and pushes failed chunks to the back of the line for silent retry.

Real devices expose deeper traps. Converting every `Blob` to an `ArrayBuffer` before upload can OOM-crash a tab on a 5 GB file; `File.slice()` returns a lightweight pointer, and data should stay as a reference until it enters `FormData`. Hashing multi-gigabyte files on the main thread freezes the UI for tens of seconds — sampling the head, tail, and a few middle segments inside a Web Worker drops that to hundreds of milliseconds. Mobile browsers kill background tabs, so each completed chunk must write its state into IndexedDB or localStorage. On reload, the uploader reads the persisted map, skips finished chunks, and resumes without losing progress.

Takeaways

— Browsers limit concurrent connections to the same origin to roughly six; firing 100 requests at once queues most of them until they time out.

— A single failed chunk inside `Promise.all` rejects the whole batch, losing progress on every other chunk.

— A concurrency queue with a configurable cap (e.g., four) and silent retry keeps the pipeline moving under packet loss without surfacing errors to the user.

— Failed chunks should be pushed back onto the end of the task queue rather than retried immediately, giving the network time to recover.

— `File.slice()` returns a lightweight reference, not a data copy; reading chunks into `ArrayBuffer` or Base64 before the actual network call can exhaust heap memory on multi-gigabyte files.

— Full-file hashing on the main thread freezes the UI; sampling the first 2 MB, last 2 MB, and a few interior segments inside a Web Worker cuts hash time from tens of seconds to hundreds of milliseconds.

— Mobile browsers routinely kill background tabs, so every successfully uploaded chunk must persist its state to IndexedDB or localStorage for seamless resume on reload.

Conclusions

The standard interview answer for large-file upload is so widely taught that it has become a liability — it describes a demo that works only on localhost with a perfect connection.

Production upload logic is less about clever algorithms and more about respecting three hard physical limits: the browser's connection pool, the device's memory ceiling, and the network's unreliability.

Sampling-based file hashing is an under-discussed technique that trades a tiny collision risk for a 100x speedup, making it practical for browser-side integrity checks on large files.

Concepts & terms

Browser per-origin connection limit

Modern browsers restrict the number of simultaneous TCP connections to a single domain (typically six). Exceeding this limit causes requests to queue, which in weak-network scenarios leads to cascading timeouts.

Concurrency queue with fallback retry

A pattern that caps the number of in-flight async tasks and, on failure, re-inserts the failed task at the end of the queue rather than aborting the entire batch. This prevents connection saturation and allows transient network errors to self-heal.

Sampling hash

A technique for computing a file's content hash by reading only selected byte ranges (e.g., head, tail, and a few middle segments) instead of the entire file. It reduces hash computation from tens of seconds to sub-second times with an acceptably low collision probability.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗