The morphology of galaxies is shaped by stellar activity, feedback, gas and dust properties, and interactions with surroundings, and can therefore provide insight into these processes. In this paper, we study the spatial offsets between stellar and interstellar medium emission in a sample of 54 main-sequence star-forming galaxies at z similar to 4-6 observed with the Atacama Large Millimeter/submillimeter Array (ALMA), and drawn from the ALMA Large Program to INvestigate C+ at Early times (ALPINE). We find no significant spatial offset for the majority (similar to 70 per cent) of galaxies in the sample among any combination of [C ii], far-infrared continuum, optical, and ultraviolet emission. However, a fraction of the sample (similar to 30 per cent) shows offsets larger than the median by more than 3 sigma significance (compared to the uncertainty on the offsets), especially between [C ii] and ultraviolet emission. We find that these significant offsets are of the order of similar to 0.5-0.7 arcsec, corresponding to similar to 3.5-4.5 kiloparsecs. The offsets could be caused by a complex dust geometry, strong feedback from stars and active galactic nuclei, large-scale gas inflow and outflow, or a combination of these phenomena. However, our current analysis does not definitively constrain the origin. Future, higher resolution ALMA and JWST observations may help resolve the ambiguity. Regardless, since there exist at least some galaxies that display such large offsets, galaxy models and spectral energy distribution fitting codes cannot assume co-spatial emission in all main-sequence galaxies, and must take into account that the observed emission across wavelengths may be spatially segregated.