Neuromorphic system enhances machine vision in extreme lighting environments

A research team has recently developed a neuromorphic exposure control (NEC) system that enhances machine vision under extreme lighting variations.

Published in Nature Communications, this biologically inspired system mimics human peripheral vision to achieve unprecedented speed and robustness in dynamic perception environments. The research team was led by Professor Jia Pan and Professor Yifan Evan Peng from the Department of Computer Science and Department of Electrical & Electronic Engineering under the Faculty of Engineering at the University of Hong Kong (HKU), in collaboration with a researcher at Australian National University.

Traditional automatic exposure (AE) systems rely on iterative image feedback, creating a chicken-and-egg dilemma that fails in sudden brightness shifts (e.g., tunnels, glare). The NEC system solves this by integrating event cameras—sensors that capture per-pixel brightness changes as asynchronous “events”—with a novel Trilinear Event Double Integral (TEDI) algorithm. This approach: Operates at 130 million events/sec on a single CPU, enabling edge deployment.

“Like how our pupils instantly adapt to light, NEC mimics biological synergy between retinal pathways,” explained Shijie Lin, the first author of the article. “By fusing event streams with physical light metrics, we bypass traditional bottlenecks to deliver lighting-agnostic vision.”

In tests, the team has validated NEC across mission-critical scenarios:

1. Autonomous Driving: Improved detection accuracy (mAP +47.3%) when vehicles exit tunnels into blinding sunlight.
2. Augmented Reality (AR): Achieved 11% higher pose estimation (PCK) for hand tracking under surgical lights.
3. 3D Reconstruction: Enabled continuous SLAM in overexposed environments where conventional methods fail.
4. Medical AR Assistance: Maintained clear intraoperative visualization despite dynamic spotlight adjustments.

Professor Pan said, “This breakthrough represents a significant leap in machine vision by bridging the gap between biological principles and computational efficiency. The NEC system not only addresses the limitations of traditional exposure control but also paves the way for more adaptive and resilient vision systems in real-world applications, from autonomous vehicles to medical robotics.”

Professor Peng commented, “Our collaborative work has been instrumental in pushing the boundaries of neuromorphic engineering. By leveraging event-based sensing and bio-inspired algorithms, we’ve created a system that is not only faster but also more robust under extreme conditions. This is a testament to the power of interdisciplinary research in solving diverse complex engineering challenges.”

In the long term, the NEC paradigm offers a novel event-frame processing scheme that reduces the processing burden of high-resolution events/images and incorporates bio-plausible principles into the low-level control of the machine eyes. This opens new avenues for camera design, system control, and downstream algorithms. The team’s success in embodying neuromorphic synergy in various systems is a milestone that can inspire many optical/image/neuromorphic processing pipelines and implies direct economic and practical implications for the industry.