The optimal intra-set rest for cluster sets (CLS) during plyometric-jump training (PJT) to improve physical fitness remains unclear. The objective of this quasi-experimental study was to compare the effects of PJT with traditional (TRS) vs. CLS structures, using different intra-set rests, on the physical fitness of healthy participants. Forty-seven recreationally active young men performed 3–5 sets of 10–12 repetitions of upper- and lower-body PJT exercises twice a week for six weeks using different set configurations: TRS group (no intra-set rest), and the CLS10, CLS20 and CLS30 groups with 10, 20 and 30 s of intra-set rest, respectively, while the total rest period was equated. Pretest-posttest measurements were carried out 48 h before and after the intervention and the rating of fatigue (ROF) was also assessed using a numerical scale (0–10 points) 20 min after the first and last (i.e., 12th) session. There was no significant difference in the mean energy intake between groups (p > 0.05). The repeated measures ANOVA revealed that all groups showed similar improvements (p < 0.05) in body mass, body mass index, fat-free mass, one repetition maximum (dynamic strength) and repetitions to failure (muscular endurance) in back squat and chest press, handgrip strength, standing long jump, 20 m sprint, 9-m shuttle run (change of direction speed), and ROF. Of note, the ROF was lower for the CLS20 and CLS30 groups, independent from the training effect. The physical fitness of recreationally active young men improved after 6 weeks of PJT involving intra-set rest intervals of 0 s, 10 s, 20 s, or 30 s. However, an intra-set rest of 20 s and 30 s seems to induce lower exercise-induced fatigue perception.