跪拜 Guibai
← All articles
Backend · Go · Interview

Go Concurrency: The Right Way to Parallelize API Calls (Context, Errgroup, and Boundary Protection)

By Sailing ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Go backend teams that fan out to multiple downstreams risk exactly the failures this article describes: partial reads, runaway goroutines, and dependency overload. The layered approach gives a concrete, testable pattern for keeping aggregation interfaces fast and stable under load.

Summary

A typical aggregation interface needs to call logs, metrics, and alerts downstream in parallel. A naive implementation with `go func()` and `time.Sleep` works at first, but production soon reveals missing data, goroutine accumulation, ignored cancellations, and overwhelmed dependencies. The core problem isn't the number of goroutines; it's the absence of a defined lifecycle, error propagation, panic recovery, and resource ceilings.

Correct concurrency control starts with `context.WithTimeout` for a unified deadline, then `errgroup` for task management and error aggregation. But `errgroup` does not recover panics, so a `safeGo` wrapper is needed to convert panics into errors. For partial success, `errgroup.WithContext` must be avoided because it cancels all tasks on the first error; `sync.WaitGroup` lets each downstream run to completion. Callers must also check a `Partial` flag, since a nil error doesn't mean complete data.

Beyond a single request, service-level semaphores limit total downstream concurrency, while entry-level in-flight limiters reject excess requests before goroutines multiply. Connection pool settings and gateway-level limits complete the picture. The article lays out four layers: entry, request, service, and cluster. Each handles a different bottleneck: fast rejection, per-request task limits, per-instance downstream concurrency, and cross-instance traffic control.

Takeaways
— Using time.Sleep to wait for goroutines never guarantees task completion and introduces data races.
— Locks and channels handle synchronization but not lifecycle management: failures, cancellation, and resource limits remain unaddressed.
— Request-level concurrency limits (like errgroup.SetLimit) do not cap service-level concurrency; 1000 requests each with 3 concurrent tasks can still send 3000 concurrent downstream calls.
— Service-level semaphores don't prevent goroutine pile-up when many requests block on acquisition; an entry-level in-flight limiter with fast rejection is required.
— errgroup.WithContext cancels the shared context on the first error, so it's unsuitable for partial success scenarios; use sync.WaitGroup to let all downstreams finish independently.
— A panic wrapper around goroutines (e.g., safeGo) is mandatory because errgroup.Wait does not recover panics.
— Aggregate all downstream errors with errors.Join instead of keeping only the first, preserving the full failure picture.
— Always check ctx.Err() after waiting in partial success logic to distinguish client cancellation from downstream partial failure.
— The four layers of concurrency control are: entry in-flight limiter, request-level errgroup.SetLimit, service-level semaphore, and cluster-level gateway rate limiting.
— Horizontal scaling amplifies downstream concurrency: more Pods mean more per-request fan-out, so downstream protection remains critical.
Conclusions

The article exposes a common blind spot: developers treat concurrency as a per-request concern and ignore cross-request and entry-level accumulation, which is why goroutine pile-ups and downstream outages keep recurring.

The distinction between all-or-nothing and partial success is a design decision that must be explicit; reusing errgroup.WithContext for partial success silently converts it into fail-fast, defeating the purpose.

The emphasis on client-side context support and connection pool limits shows that a Loader can pass context correctly, but if the HTTP client or transport doesn't honor it, cancellation remains ineffective.

The advice to refactor concurrency first and adjust retries separately is pragmatic: combining concurrency, retries, and timeouts at once increases the blast radius of mistakes.

Concepts & terms
errgroup
A Go package that provides synchronization and error propagation for a group of goroutines. WithContext cancels the shared context when any task returns an error, and Wait blocks until all tasks finish. It does not recover panics.
context.WithTimeout
Creates a derived context that is automatically cancelled after a specified duration. It propagates cancellation signals down the call chain, so any function receiving the context can stop work promptly.
Semaphore
A concurrency primitive that limits the number of simultaneous accesses to a resource. Here it's implemented with a buffered channel to cap downstream concurrency per service instance.
In-flight limiter
A mechanism that limits the number of requests currently being processed by a service. It uses a non-blocking acquire to reject excess requests with a 429 status instead of letting goroutines pile up.
Partial success
A strategy where an aggregation endpoint returns data even if some downstream calls fail, marking the response as incomplete. It requires explicit tracking (e.g., a Partial flag) and careful separation from overall cancellation.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗