Date of Award
Fall 2023
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Interdepartmental Neuroscience Program
First Advisor
Hirsch, Joy
Abstract
The neural processing of real human faces during direct interaction remains unknown. Prior research has established that face simulations do not capture the full breadth of processing that occurs during real face viewing, demonstrating the need for new approaches. Here we present multimodal methodological developments which combine fNIRS with EEG and eye-tracking, as well as a novel paradigm for disrupted viewing of another real face as well as a robot control, which enables probing of the impact of time and continuity on real face specific neural processing networks. We hypothesized that the spatial and temporal complexity of information extraction from live faces recruits different processing regions over time, thus making the temporal duration and continuity of both real face presentation and visual sensing for acquisition contributing factors to real face neural processing. We investigate these hypotheses by comparing the relationship of two visual sensing metrics (fixation length and dwell time) to the neural processing of a uniform amount of face viewing time (3200-ms per event), subdivided into two different viewing-epoch durations (1600- and 800-ms) with short, 200-ms disruptions between each epoch. Paradigms were employed for simultaneous acquisition of neural hemodynamics and eye-behavior, alongside a “SmartGlass” divider which enabled implementation of different face viewing patterns. We compare the 1600-ms viewing epochs with the human to that of the robot to establish real face specificity and we compare the 1600-ms human face viewing epoch to the 800-ms viewing epochs to identify time dependent changes in neural processing. Main effect contrasts of task > rest confirmed prior findings of significant lateral cortex activity within the right supramarginal gyrus during 1600ms human face viewing but not robot face viewing. Application of regressors incorporating mean fixation length and dwell time identified activity in the lateral and dorsal parietal cortices, respectively, during 1600-ms viewing epochs of the real face but not the robot face. This suggests that visual sensing modulates neural processing in a real face specific manner. sLORETA source estimation models identified significant differences between real and simulated face viewing in the alpha frequency band in both the lateral and dorsal parietal cortices, pointing to a role of temporal integration as the driver of activity. These results were then compared to neural activity during the 800-ms viewing epochs in order to understand the importance of continuity of face view and length of undisrupted face viewing time for these relationships. During the 800-ms viewing-epochs there was no significant main effect of task in the supramarginal gyrus. Linear regressors with mean fixation length during the 800-ms epochs found no relationship with the lateral cortex, consistent with the hypothesis that fixation length is an essential gating factor during live-face processing. Similar regression analyses were conducted using face dwell time and also found no significant dwell time relationship with the dorsal parietal cortex during 800-ms epochs. Average fixation lengths were longer during 1600-ms as compared to 800-ms epochs (p=0.005), with no change in cumulative face viewing time (dwell time), suggesting that fixation lengths adapt to real face presentation patterns and may act as a gating mechanism for engagement of dynamic visual processing regions. These findings introduce effects of live-face-viewing duration and visual sensing as model components for neural processing of live faces. They underscore the significance of naturalistic face-viewing paradigms and point to an important role of the temporal dimension for understanding real live face processing.
Recommended Citation
Kelley, Megan, "A Multi-Modal Investigation of the Basic Neural Processing Mechanisms of Real Human Face Viewing" (2023). Yale Graduate School of Arts and Sciences Dissertations. 1156.
https://elischolar.library.yale.edu/gsas_dissertations/1156
