Autonomic nervous system (ANS) control may be disrupted by cerebrovascular disease. We investigated the relationship between alterations in white matter integrity and regulation of the ANS in 23 participants who sustained a stroke within 5 years. These participants underwent diffusion tensor imaging, and fractional anisotropy values were calculated (DTI-FA) for each hemisphere and lobe. Cognitive and physical exertion tasks were performed while recording an electrocardiogram. Respiratory sinus arrhythmia (RSA) decreased more during a verbal fluency task with lower left hemisphere DTI-FA. Further, the physical stressor yielded decreases in RSA with lower frontal DTI-FA and higher temporal lobe DTI-FA, p<.05 (perhaps a release effect on the central autonomic network). Decrements in ANS regulation may have functional consequences that alter behavior, as well as potentially increasing the risk for further vascular disease. © 2012 Society for Psychophysiological Research.