Abstract In-sensor neuromorphic vision platforms with near-infrared (NIR) sensitivity are essential for intelligent imaging in low-light and multispectral environments. Here, a retinocortical dual-mode platform based on evolved-synaptic transistor (Evo-SynT) devices using upconversion nanoparticles (UCNP) and their integration into evolved-retina optical synapse (EROS) arrays is introduced. Evo-SynT devices exhibit key synaptic features, including paired-pulse facilitation indices exceeding 183.93% and 136.36% at a 0.5 s interval under 808 nm and 940 nm illumination, respectively, and analog weight modulation across 512 conductance states. A 12×12 EROS array enables dual-mode operation: retinal-like in-sensor preprocessing and cortical-like in-memory classification. The EROS array improves pedestrian detection accuracy under low-light conditions from 0.7806 to 0.8481 (808 nm) and 0.9071 (940 nm), and achieves classification accuracies of 77.19% and 79.40%, respectively. These results highlight the EROS platform as a scalable solution for integrated NIR-sensitive neuromorphic vision systems. Data availability The data supporting the findings of this study are available within the paper and its Supplementary Information files. The source data underlying the graphs and plots presented in the figures are provided with this paper. Additional data related to this study are available from the corresponding authors upon request. Requests for data will typically be responded to within two weeks, and the data will remain available for at least three years following publication. Source data are provided in this paper. Code availability All custom codes used for data processing, analysis, and simulation in this study are provided with this paper as Source Code files. References Huang, P.-Y. et al. Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction. Nat. Commun. 14, 6736 (2023). Zhou, G. et al. Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing. Nat. Commun. 14, 8489 (2023). Huang, H. et al. In-sensor
Retinocortical in-sensor neuromorphic vision platform for NIR-augmented artificial vision
Read the original article
nature.com →