The majority of ER-targeted tail-anchored (TA) proteins are inserted into membranes by the Guided Entry of Tail-anchored protein (GET) system. Disruption of this system causes a subset of TA proteins to mislocalize to mitochondria. We show that the AAA+ ATPase Msp1 limits the accumulation of mislocalized TA proteins on mitochondria. Deletion of MSP1 causes the Pex15 and Gos1 TA proteins to accumulate on mitochondria when the GET system is impaired. Likely as a result of failing to extract mislocalized TA proteins, yeast with combined mutation of the MSP1 gene and the GET system exhibit strong synergistic growth defects and severe mitochondrial damage, including loss of mitochondrial DNA and protein and aberrant mitochondrial morphology. Like yeast Msp1, human ATAD1 limits the mitochondrial mislocalization of PEX26 and GOS28, orthologs of Pex15 and Gos1, respectively. GOS28 protein level is also increased in ATAD1 -/- mouse tissues. Therefore, we propose that yeast Msp1 and mammalian ATAD1 are conserved members of the mitochondrial protein quality control system that might promote the extraction and degradation of mislocalized TA proteins to maintain mitochondrial integrity.