Why Your RUM Data Is Wrong: The 6 Traps of Performance Monitoring
Optimizing against wrong metrics wastes months, and most self-built SDKs get at least one metric wrong. This article quantifies every common mistake and provides a testable architecture that makes your performance data defensible—a practical baseline for any team building its own RUM.
This article tackles the final—and most deceptive—front of performance: knowing whether you really got faster. Using only the Node.js standard library, the author builds a complete RUM (Real User Monitoring) stack and benchmarks why common misconceptions fail. For example, summing layout shifts inflates p75 CLS by 2.68x, mixing mobile 1% and desktop 10% samples cuts true p75 by −30.8%, and averaging p90s across shards is systematically biased while summing histograms is not.
The solution hinges on a design principle: write metric definitions as pure functions so they can be tested, replayed from real entry logs, and shared between lab and field. The article then walks through implementing CLS session windows, LCP freeze on first input, INP trimming, weighted histograms, byte-aware beacon batching, and buffer eviction policies. Every claim is backed by a benchmark script that produces the cited numbers, plus a 56-assertion test suite that locks each definition to its correct behavior.
Most self-built SDKs implement CLS as a sum, a systematic error that inflates p75 by 2.68x on realistic pages.
The non-invertibility of quantiles means any aggregate that loses distribution shape is inherently lossy; histograms are the only additive approximation.
The observer effect is more about serialization and buffer management than the act of measuring; monitoring can become a self-induced long task.
The deliberate trimming of INP spike outliers reveals a design tension: metrics optimize for reproducibility over pathology, so rare extreme events need separate tracking.