Background. Age-related changes in body composition affect physical fitness in older adults. However, whether the autonomic response is associated with body fat percentage and its implication for physical fitness is not fully understood. Aim. To understand the association between physical fitness, body composition, and heart rate variability in older people and its mediating factors. Methods. A cross-sectional study with 81 older adults was conducted, assessing Short Physical Performance Battery (SPPB), Two-minute Step Test (TMST), body composition, and cardiac autonomic response. Correlation and mediation analyses were performed. Results. Body fat percentage negatively correlated with physical fitness (SPPB: r = −0.273, p = 0.015; TMST: r = −0.279, p = 0.013) and sympathetic activity (sympathetic nervous system (SNS) index: r = −0.252, p = 0.030), yet positively correlated with parasympathetic tone (root mean square of successive differences (RMSSD): r = 0.253, p = 0.029; standard deviation of NN intervals (SDNN): r = 0.269, p = 0.020). Physical fitness associated with sympathetic nervous system index (SPPB: r = 0.313, p = 0.006; TMST: r = 0.265, p = 0.022) and parasympathetic nervous system index (TMST: r = −0.344, p = 0.003). Muscle mass mediated body fat’s impact on physical fitness, while physical fitness mediated body fat’s impact on autonomic response. Conclusion. Body composition and cardiac autonomic response to exercise are associated with physical fitness in older people, highlighting a possible protective effect of muscle mass against the decline in physical fitness associated with increased body fat.