People with schizophrenia typically show visual processing deficits on masking tasks and other performance-based measures, while people with bipolar disorder may have related deficits. The etiology of these deficits is not well understood. Most neuroscientific studies of perception in schizophrenia and bipolar disorder have focused on visual processing areas in the cerebral cortex, but perception also depends on earlier components of the visual system that few studies have examined in these disorders. Using diffusion weighted imaging (DWI), we investigated the structure of the primary sensory input pathway to the cortical visual system: the optic radiations. We used probabilistic tractography to identify the optic radiations in 32 patients with schizophrenia, 31 patients with bipolar disorder, and 30 healthy controls. The same participants also performed a visual masking task outside the scanner. We characterized the optic radiations with three structural measures: fractional anisotropy, mean diffusivity, and tract volume. We did not find significant differences in those structural measures across groups. However, we did find a significant correlation between the volume of the optic radiations and visual masking thresholds that was unique to the schizophrenia group and explained variance in masking performance above and beyond that previously accounted for by differences in visual cortex. Thus, individual differences in the volume of the optic radiations explained more variance in visual masking performance in the schizophrenia group than the bipolar or control groups. This suggests that individual differences in the structure of the subcortical visual system have an important influence on visual processing in schizophrenia.