Background: Prescription opioids (PO) have been widely used for chronic non-cancer pain, with commensurate concerns for overdose. The long-term effect of these medications on non-overdose mortality in the general population remains poorly understood. This study’s objective was to examine the association of prescription opioid use and mortality in a large cohort, accounting for gender differences and concurrent benzodiazepine use, and using propensity score matching. Methods: 29,025 US community-dwellers were enrolled in the REasons for Geographic And Racial Differences in Stroke (REGARDS) cohort between 2003 and 2007, and followed through December 31, 2012. At baseline there were 1907 participants with PO; 1864 of them were matched to participants without PO, based on the model-derived propensity to receive opioid prescriptions. Causes of death were expert-adjudicated. Results: Over median follow-up of 6 years there were 4428 deaths (413 among persons with PO). The risk for all-cause mortality was 12% higher, in absolute terms, for persons with PO compared to those without PO in the overall sample, with gender differences (interaction p =.0008). The risk of death was increased for women with PO (hazard ratio [HR] 1.21 [95% Confidence Interval (CI) 1.04–1.40]), but not men (HR 0.92 [95% CI 0.77–1.10]). Women with PO were at higher risk of cardiovascular disease (CVD) death (HR 1.43 [95% CI 1.12–1.84]), sudden death (HR 2.02 [95% CI 1.29–3.15]) (a subset of CVD death), and accidents (HR 2.18 [95% CI 1.03–4.60]). These risks were not observed for men with PO. Conclusion: Over 6 years of follow-up, women but not men who had opioid prescriptions were at higher risk of all-cause mortality, CVD death, sudden death, and accidents. Special caution in prescribing opioids for women may be warranted until these findings are confirmed.