We conducted a rapid, mixed-methods assessment to understand how COVID-19 affected Latinx sexual minority men (LSMM) and transgender women (LTGW). Using a computer-assisted telephone interviewing software, one interviewer called 52 participants (randomly sampled from a larger HIV prevention pilot study aiming to increase HIV knowledge and testing frequency; n = 36 LSMM and n = 16 LTGW) between 04/27/20-05/18/20. We quantified core domains using the Epidemic-Pandemic Impacts Inventory scale and provided important context through open-ended qualitative questions assessing: 1) COVID-19 infection history and experiences with quarantine; 2) Health and healthcare access; 3) Employment and economic impact of COVID-19. Participants reported increases in physical conflict or verbal arguments with a partner (13.5%) or other adult(s) (19.2%) due to stressors associated with the safer-at-home order. Participants also reported increased alcohol consumption (23.1%), problems with sleep (67.3%) and mental health (78.4%). Further, disruptions in access to Pre-Exposure Prophylaxis or PrEP–a daily pill to prevent HIV–occurred (33.3% of 18 participants who reported being on PrEP). Many said they received less medical attention than usual (34.6%), and LTGW reported delays in critical gender-affirming hormones/procedures. Half of the participants lost their jobs (50.0%); many undocumented participants relayed additional financial concerns because they did not qualify for financial assistance. Though no COVID-19 infections were noted, COVID-19 dramatically impacted other aspects of health and overall wellbeing of LSMM and LTGW. Public health responses should address the stressors faced by LSMM and LTGW during the COVID-19 pandemic and the impact on wellbeing.