Multivariate correlation analysis on multimodal neurophysiological data

Abstract

This thesis investigates the coupling between visual, neural and motor signals during immersive driving in a virtual-reality (VR) environment. The analysis is based on a multimodal dataset recorded at Columbia University’s Laboratory for Intelligent Imaging and Neural Computing (LIINC), comprising optical flow video features, electroencephalography (EEG), and steering-wheel motor data. The aim is to examine how visual–neural and neural–motor correlations vary with driving difficulty and individual performance. After extensive preprocessing mainly on the EEG recordings—filtering, bad channel and epoch rejection, independent component analysis, and artifact correction—Canonical Correlation Analysis was applied to identify spatiotemporal components maximizing the correlation between EEG and each peripheral modality. Significance was assessed using phase-randomized surrogate data with false-discovery-rate correction. Components showing consistent significance across conditions or the highest overall canonical correlations were retained for group analysis. Fisher transformed correlations were then entered into Linear Mixed-Effects models to test effects of fog difficulty, driving session, and performance group. Results indicate stronger optical flow–EEG coupling under high fog density, particularly in high-performance drivers, while EEG–motor coupling remained relatively stable across conditions. These findings suggest that visual difficulty enhances visual–neural synchronization, possibly reflecting increased attentional or perceptual engagement, whereas motor control loops are less sensitive to environmental complexity. The study provides a reproducible multivariate framework for analyzing multimodal neurophysiological data and lays the groundwork for future investigations of cognitive load, visuomotor adaptation, and predictive neural coding in naturalistic tasks.