Although breast cancer treatments reduce mortality, their adverse effects can increase depression which impacts one’s quality of life (QoL). Physical activity (PA) seems to improve the QoL of breast cancer survivors (BCS). However, an unanswered question is the influence of PA on the QoL in BCS with depressive symptoms. Thus, we analyzed the influence of PA on the QoL in BCS with persistent depressive symptoms during 12 months of follow-up. The sample included 70 female BCS. Depression and QoL domains (i.e., functional capacity, physical limitations, body pain, general health status, vitality, social and emotional aspects, and mental health) were assessed at baseline and follow-up periods by the Hospital Anxiety and Depression Scale and SF-36, respectively. Habitual PA was assessed by Baecke’s questionnaire. Our results indicate a prevalence of 17.1% of depressive symptoms. Non-depressives BCS improved their physical limitations and general health status domains over time, but there were no observed differences in depressive BCS. BCS with persistent depressive symptoms (baseline and follow-up) showed worse QoL scores than non-depressives in all domains, regardless of confounding factors. When adjusted for PA, the difference between BCS depressives and non-depressives lost its significance in the functional capacity domain. In conclusion, habitual PA practice positively influenced the functional capacity domain of the QoL in BCS.