We show that the key ingredients for creating recurrent traveling spatial phase defects in drifting patterns are a noise-sustained structure regime together with the vicinity of a phase transition, that is, a spatial region where the control parameter lies close to the threshold for pattern formation. They both generate specific favorable initial conditions for local spatial gradients, phase, and/or amplitude. Predictions from the stochastic convective Ginzburg-Landau equation with real coefficients agree quite well with experiments carried out on a Kerr medium submitted to shifted optical feedback that evidence noise-induced traveling phase slips and vortex phase-singularities.