Ferroelectric Sensor Arrays Compute While They See: In-Sensor Vision Against the Memory Wall

Edge vision workloads are power- and latency-bound, and this architecture promises an order-of-magnitude efficiency gain by moving compute into the sensor. The real hurdle is that analog non-idealities and missing design tooling, not raw physics, will decide whether ferroelectric in-sensor chips reach production.

The von Neumann memory wall hits vision workloads hardest: DRAM reads cost roughly 640 pJ versus 3.1 pJ for a 32-bit multiply, so processors spend most of their time waiting on data. In-sensor computing attacks this by fusing the photodiode, weight storage, and multiply-accumulate into a single ferroelectric crossbar array. Each pixel stores a weight in its polarization state, converts incoming light into a current, and the column line sums currents via Kirchhoff's law — the neural network's core weighted addition happens physically, with no off-array data movement.

The article argues this architecture can deliver one to two orders of magnitude energy efficiency gains over traditional CMOS image sensor plus digital processor pipelines. It also shows the path to brain-inspired vision: the retina itself is a low-power analog processor, and ferroelectric arrays align naturally with event-driven spiking neural networks. But the engineering chain is long — device-to-device variation, cycle-to-cycle fluctuation, IR drop, ADC overhead, and missing EDA tooling all need solving before a lab demo becomes a mass-produced edge chip.