Mitochondria are dynamic organelles that play essential roles in cell growth and survival. Processes of fission and fusion are critical for the distribution, segregation, and maintenance of mitochondria and their genomes (mtDNA). While recent work has revealed the significance of mitochondrial organization for mtDNA maintenance, the impact of mtDNA perturbations on mitochondrial dynamics remains less understood. Here, we develop a tool to induce mitochondria-specific DNA damage using a mitochondrial-targeted base modifying bacterial toxin, DarT. Following damage, we observe dynamic reorganization of mitochondrial networks, likely driven by mitochondrial dysfunction. Changes in the organization are associated with the loss of mtDNA, independent of mitophagy. Unexpectedly, perturbation to exonuclease function of mtDNA replicative polymerase, Mip1, results in rapid loss of mtDNA. Our data suggest that, under damage, partitioning of defective mtDNA and organelle are de-coupled, with emphasis on mitochondrial segregation independent of its DNA. Together, our work underscores the importance of genome maintenance on mitochondrial function, which can act as a modulator of organelle organization and segregation.