The complexities of navigating pregnancy while living with HIV predispose women to additional stress. Finding ways to minimize psychosocial challenges during the perinatal period may maximize the well-being of mothers living with HIV and their children. The goal of this study was to explore psychosocial challenges experienced by women living with HIV (WLWH) during pregnancy and the postpartum. We conducted individual in-depth interviews with 20 WLWH recruited from an HIV treatment cohort study in Mbarara, Uganda as part of a larger study exploring perinatal depression. We conducted content analyses to identify themes related to challenges of WLWH during pregnancy and the postpartum. Participants had a median age of 33 years [IQR: 28-35], a median of 3 living children [IQR: 2-5], and 95% had achieved HIV-RNA suppression. Challenges were organized around the following themes: HIV-related stigma from health professionals, HIV status disclosure dilemma, unintended pregnancy and intimate partner violence, HIV and environmental structural barriers and distress and fear related to maternal and child health. Stigma centered on discrimination by health care professionals and personal shame associated with being pregnant as a WLWH. This led to difficulty engaging in HIV care, particularly when coupled with structural barriers, such as lack of transportation to clinic. Participants experienced intimate partner violence and lacked support from their partners and family members. Distress and fear about the health and uncertainty about the future of the unborn baby due to maternal deteriorating physical health was common. The perinatal period is a time of stress for WLWH. Challenges experienced by WLWH may compromise successful engagement in HIV care and may reduce quality of life for women and their children. Strategies aimed at alleviating the challenges of WLWH should involve the larger structural environment including partners, family and community member as well as policy makers, funders and program implementers to work together for the common cause. These consolidated efforts may not only lower the risk of psychological distress but has potential to create long lasting solutions to benefit the wider community.