Background: A total of 20% of the U.S. population resides in rural areas, yet is served by 3% of oncologists, and 7% of nononcology specialists. Access to care issues can be compounded by lower socioeconomic status (SES) in rural areas, yet this issue is unexplored among older patients with breast cancer. Methods: Using Surveillance Epidemiology and End Results-Medicare, 109,608 patients diagnosed at ≥65 years with breast cancer between 2000 and 2011 were identified. Residence status was combined with Federal Poverty levels: urban (high, medium, and low poverty) and rural (high, medium, and low poverty). Five-year overall survival (OS) and healthcare utilization [HCU: visits to primary care provider (PCP), oncologist, nononcology specialist, and emergency department (ED)] were examined using urban/low poverty as reference. The residence, HCU, and mortality association was examined using mediation and moderation analyses. Results: Median age was 76 years; 12.5% were rural, 15.6% high poverty. Five-year OS was 69.8% for rural and 70.9% for urban. Both urban- and rural/high-poverty patients had a 1.2-fold increased mortality hazard. Rural/high-poverty patients had a higher rate of PCP [year 1 (Y1): incidence rate ratio (IRR) ¼ 1.23; year 2 (Y2)–year 5 (Y5): IRR ¼ 1.19] and ED visits (Y1: IRR ¼ 1.82; Y2–Y5: IRR ¼ 1.43), but lower nononcology specialist visit rates (Y1: IRR ¼ 0.74; Y2–Y5: IRR ¼ 0.71). Paucity of nononcology specialist visits mediated 23%–57% of excess mortality risk. The interaction between residence/SES and paucity of nononcology specialist visits accounted for 49%–92% of excess mortality risk experienced by rural/high-poverty patients versus urban/low poverty. Conclusions: Urban–rural residence mortality differences among older patients with breast cancer are highly predicated by poverty level. Impact: Rural/high-poverty patients demonstrate less use of nononcology specialists compared with urban/low poverty, with disparities moderated by specialist use.