Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness
Summary AI-generated
- TL;DR
- By adding a computer model of early, subcortical brain vision to standard artificial intelligence networks, researchers have made computer vision systems significantly more robust to visual distortions and noise.
- Problem
- While modern computer vision models are highly accurate, they are easily confused by image distortions, noise, or lighting changes that humans easily ignore. This vulnerability limits their reliability in real-world applications where visual conditions are unpredictable.
- Method
- The researchers developed "Early Vision Networks" (EVNets), which add a neuro-inspired front-end to standard convolutional neural networks. This front-end includes a new "SubcorticalBlock" modeled after the early stages of biological vision, paired with an existing model of the primary visual cortex.
- Results
- EVNets outperformed standard networks by 9.3% on a benchmark testing resistance to corruptions, adversarial attacks, and domain shifts, while also showing a stronger human-like bias toward recognizing shapes rather than textures. Furthermore, combining this brain-inspired architecture with advanced data training techniques yielded an additional 6.2% improvement over using training techniques alone.
- Takeaways
- Mimicking early biological vision processes directly in network architectures naturally improves their alignment with actual brain responses. Additionally, combining biological design principles with modern machine learning training methods provides complementary benefits for building more reliable AI.
- For industry
- For companies deploying computer vision in unpredictable environments—such as autonomous driving or quality inspection—this research offers a way to make AI systems much more dependable. By integrating biological vision principles, these models become significantly more resilient to common real-world challenges like glare, blur, and unexpected image corruptions.
- Why it matters
- This work bridges the gap between neuroscience and artificial intelligence, demonstrating that biological constraints can guide the development of safer, more trustworthy AI. The resulting models can improve the safety of critical technologies like medical imaging and autonomous vehicles, while providing researchers with better computational tools to study human vision.
Abstract
Convolutional neural networks (CNNs) trained on object recognition achieve high task performance but continue to exhibit vulnerability under a range of visual perturbations and out-of-domain images, when compared with biological vision. Prior work has demonstrated that coupling a standard CNN with a front-end (VOneBlock) that mimics the primate primary visual cortex (V1) can improve overall model robustness. Expanding on this, we introduce Early Vision Networks (EVNets), a new class of hybrid CNNs that combine the VOneBlock with a novel SubcorticalBlock, whose architecture draws from computational models in neuroscience and is parameterized to maximize alignment with subcortical responses reported across multiple experimental studies. Without being optimized to do so, the assembly of the SubcorticalBlock with the VOneBlock improved V1 alignment across most standard V1 benchmarks, and better modeled extra-classical receptive field phenomena. In addition, EVNets exhibit stronger emergent shape bias and outperform the base CNN architecture by 9.3% on an aggregate benchmark of robustness evaluations, including adversarial perturbations, common corruptions, and domain shifts. Finally, we show that EVNets can be further improved when paired with a state-of-the-art data augmentation technique, surpassing the performance of the isolated data augmentation approach by 6.2% on our robustness benchmark. This result reveals complementary benefits between changes in architecture to better mimic biology and training-based machine learning approaches.