© 2016 Wiley Periodicals, Inc. The importance of relationships between handedness, language lateralization and localization, and white matter tracts for language performance is unclear. The goal of the study was to investigate these relationships by examining arcuate fasciculus (AF) structural asymmetry (DTI) and functional asymmetry (fMRI) in language circuits, handedness, and linguistic performance. A large sample of right-handed (n = 158) and atypical-handed (n = 82) healthy adults underwent DTI at 3 T to assess number of streamlines and fractional anisotropy (FA) of the AF, and language fMRI. Language functions were assessed using standard tests of vocabulary, naming, verbal fluency, and complex ideation. Laterality indices (LIs) illustrated degree of asymmetry and lateralization patterns for the AF (streamlines and FA) and verb generation fMRI. Both handedness groups showed leftward lateralization bias for streamline and fMRI LIs and symmetry for FA LI. The proportion of subjects with left, right, or symmetric lateralization were similar between groups if based on AF LIs, but differed if based on fMRI LIs (p = 0.0016). Degree of right-handedness was not associated with AF lateralization, but was associated with fMRI language lateralization (p = 0.0014). FA LI was not associated with performance on language assessments, but streamline LI was associated with better vocabulary and complex ideation performance in atypical-handed subjects (p = 0.022 and p = 0.0098, respectively), and better semantic fluency in right-handed subjects (p = 0.047); however, these did not survive multiple comparisons correction. We provide evidence that AF asymmetry is independent of hand preference, and while degree of right-handedness is associated with hemispheric language lateralization, the majority of atypical-handed individuals are left-lateralized for language. Hum Brain Mapp 37:3297–3309, 2016. © 2016 Wiley Periodicals, Inc.