Intractable epilepsy can be challenging for patients and for their families. Disability rates in patients are high, causing tremendous physical and emotional burden on family caregivers. Additionally, caregivers may experience affiliate stigma, where they perceive and internalize the negative societal views of a condition and exhibit a psychological response. Affiliate stigma has been rarely studied in caregivers of those with intractable epilepsy. This study examined the relationship between affiliate stigma and the levels of burden experienced by caregivers, as well as how these levels may vary between those caring for children and adults. This cross-sectional approach used a self-administered survey offered to caregivers of family members with confirmed diagnoses of intractable epilepsy. We measured burden with the 30-item Carer's Assessment of Difficulties Index (CADI) and affiliate stigma with a six-item scale examining caregivers' perceptions of stigma directed toward themselves and their family members with epilepsy. Four nested ordinary-least-squares regression models were estimated using stigma scale scores to predict levels of perceived burden adjusting for demographic variables. Age of the patient with epilepsy was dichotomized (pediatric/adult) to assess a possible moderating effect of patient's age on the relationship between stigma and caregiver burden. Respondents (N = 136) were predominantly White (83%), female (75%), and married (69%), with an average age of 43 years. Patients with epilepsy were 52% male with ages ranging from 2 to 82 years. Each of the regression models yielded positive associations (p < 0.001) between perceived levels of caregiver burden and affiliate stigma. Additionally, the age of the family member with epilepsy moderated (p < 0.05) the effect, with the relationship stronger for caregivers of adults. In a highly select group of patients with refractory epilepsy recruited mostly from a cannabidiol (CBD) clinic, this study demonstrated that caregivers experience affiliate stigma, which is significantly associated with higher burden levels. Additionally, this study identified specific needs, which when met, may improve caregivers' physical and mental health.