We present evolutionary models for young low-mass stars and brown dwarfs taking into account episodic phases of accretion at early stages of the evolution, a scenario supported by recent large surveys of embedded protostars. An evolution including short episodes of vigorous accretion ((M) over dot >= 10(-4)M(circle dot) yr(-1)) followed by longer quiescent phases ((M) over dot < 10(-6)M(circle dot) yr(-1)) can explain the observed luminosity spread in H-R diagrams of star-forming regions at ages of a few Myr, for objects ranging from a few Jupiter masses to a few tenths of a solar mass. The gravitational contraction of these accreting objects strongly departs from the standard Hayashi track at constant T(eff). The best agreement with the observed luminosity scatter is obtained if most of the accretion shock energy is radiated away. The obtained luminosity spread at 1 Myr in the H-R diagram is equivalent to what can be misinterpreted as an similar to 10 Myr age spread for non-accreting objects. We also predict a significant spread in radius at a given T(eff), as suggested by recent observations. These calculations bear important consequences for our understanding of star formation and early stages of evolution and on the determination of the initial mass function for young (<= a few Myr) clusters. Our results also show that the concept of a stellar birthline for low-mass objects has no valid support.