Racial/ethnic and socioeconomic disparities in COVID-19 burden have been widely reported. Using data from the state health departments of Alabama and Louisiana aggregated to residential Census tracts, we assessed the relationship between social vulnerability and COVID-19 testing rates, test positivity, and incidence. Data were cumulative for the period of February 27, 2020 to October 7, 2020. We estimated the association of the 2018 Social Vulnerability Index (SVI) overall score and theme scores with COVID-19 tests, test positivity, and cases using multivariable negative binomial regressions. We adjusted for rurality with 2010 Rural–Urban Commuting Area codes. Regional effects were modeled as fixed effects of counties/parishes and state health department regions. The analytical sample included 1160 Alabama and 1105 Louisiana Census tracts. In both states, overall social vulnerability and vulnerability themes were significantly associated with increased COVID-19 case rates (RR 1.57, 95% CI 1.45–1.70 for Alabama; RR 1.36, 95% CI 1.26–1.46 for Louisiana). There was increased COVID-19 testing with higher overall vulnerability in Louisiana (RR 1.26, 95% CI 1.14–1.38), but not in Alabama (RR 0.95, 95% CI 0.89–1.02). Consequently, test positivity in Alabama was significantly associated with social vulnerability (RR 1.66, 95% CI 1.57–1.75), whereas no such relationship was observed in Louisiana (RR 1.05, 95% CI 0.98–1.12). Social vulnerability is a risk factor for COVID-19 infection, particularly among racial/ethnic minorities and those in disadvantaged housing conditions without transportation. Increased testing targeted to vulnerable communities may contribute to reduction in test positivity and overall COVID-19 disparities.