Optical Flow Sensor: A Direction-Selective Bionic Retina Design
Juchen Zhou, Bonan Yan, Yuchao Yang
cs.AR, cs.CV, cs.ET, eess.IV
2026-07-30
A retina-like optical-flow sensor computes motion vectors on-chip from DVS ON/OFF event timing, cutting power to 1/303 of an FPGA-DVS pipeline at microsecond latency with 3.3× less output data.
Optical flow, the per-pixel motion vector field between moments, is a foundational signal for robots, drones, and autonomous systems. Frame cameras compute it by dense sampling and global computation, which is slow and power-hungry. Event cameras (DVS), modeled on the retina, emit a signal only when a pixel's brightness changes, so they produce sparse data at microsecond precision. But that event stream only says "something changed here"; turning it back into optical-flow vectors still needs an external FPGA or processor, which eats the power the sensor saved. This paper removes that last step by computing optical flow inside the sensor chip itself.
The core idea discretizes the optical-flow equation into event pairs between neighboring pixels. When a pixel's brightness drops it fires an OFF event; a neighbor whose brightness rises fires an ON event. If the same motion edge sweeps across two adjacent pixels, the OFF-then-ON sequence (or its mirror) is temporally correlated, and the OFS uses that correlation as the test for an optical-flow event.
The circuit runs as follows:
Several design choices are explicit tradeoffs: a 2-bit TDC resolution is picked as the accuracy/area/power sweet spot; a 3×3 neighborhood beats 4-connectivity and 5×5 because detection rate rises monotonically with neighborhood size but so does cost; all in-pixel circuits are digital to resist transistor mismatch.
The readout interface, OF-AER (Optical-flow Address-Event Representation), is also simplified: instead of reading out pixel by pixel, it aggregates at the row level. Any pixel detecting flow triggers its row's readout request; an arbiter serves rows at 1 µs each, for a peak throughput of 128 Meps. Row-level rather than pixel-level aggregation relaxes the clock requirement, lowering power without sacrificing throughput.
Two implementations are offered: a CMOS version (TSMC 65 nm) using photodiodes with 1 µs latency, and an optical-memristor version whose sensing devices sit in the upper metal layers and spare the silicon substrate, giving smaller area and lower power (3.3 mW vs 7.4 mW), but the memristor's slower response pushes latency to about 10 µs and its photosensitivity is weaker.
Compared with a DVS + FPGA baseline:
| Metric | DVS + FPGA | CMOS OFS | Memristor OFS |
| Flow sense + compute power | 1 W | 7.4 mW | 3.3 mW |
| vs baseline | n/a | 135× less | 303× less |
| Per-event latency | 1.1 µs | 1 µs | 10 µs |
| Output bandwidth | 2.33 MB/s | 700 KB/s (3.3× less) | n/a |
| Sensor footprint | 2mm×2mm | 2.9mm×2.7mm | 2.56mm×2.4mm |
Two structural bonuses. First, noise rejection: isolated noise events lack spatial correlation and fail the "neighbor ON within window" gate, so they are filtered out for free, giving 188× to 259× suppression. Second, because the output is optical-flow vectors rather than raw events, the event rate drops from 388 keps to 100 keps, lightening the downstream load.
On accuracy, the Average Endpoint Error (AEE) on the MVSEC dataset ranges from 159.73 to 274.28 px/s; on the indoorflying3 sequence it is 179.51 px/s, which the authors concede trails the comparison method [11], blaming the quantization in the equation and the 2-bit TDC truncation.
For teams building robot vision, drones, or high-speed motion capture, this collapses the optical-flow chain from "camera plus FPGA" onto a single chip: power drops from watts to milliwatts, latency holds at microseconds. That is a real win for edge devices with tight power budgets and real-time constraints, such as micro-drones, AR/VR headsets, and industrial high-speed inspection. The memristor variant sketches a path that trades latency for even smaller area and lower power. Directionally, it is a step for event cameras from "emit events" toward "emit usable motion semantics on-chip."
One caveat up front, though: this is a pre-silicon design.
The most important point: the power figures come from SPICE circuit simulation and the area from a pre-silicon layout, not from a measured chip. The "experimental results" are circuit simulations plus a Python functional reproduction; the physical silicon does not exist yet. Whether mismatch, parasitics, and real readout bandwidth erode these numbers on actual silicon is unknown.
Accuracy is also a weakness: the 2-bit TDC truncates magnitude coarsely, and on MVSEC the method trails prior work, which the authors do not hide. The memristor version trades photosensitivity for area and power and "requires more complex resistance readout schemes," so its engineering maturity is lower. Finally, the 128 Meps peak throughput is a row-scan theoretical ceiling; past 1 Meps, rows contend for the arbiter and queue, so the timestamp error (ATE) rises, reaching about 74.3 µs at the extreme point.