Context. Outflows from low-mass star-forming galaxies are a fundamental ingredient for models of galaxy evolution and cosmology. Despite seemingly favourable conditions for outflow formation in compact starbursting galaxies, convincing observational evidence for kiloparsec-scale outflows in such systems is scarce. Aims. The onset of kiloparsec-scale ionised filaments in the halo of the metal-poor compact dwarf SBS 0335-052E was previously not linked to an outflow. In this paper we investigate whether these filaments provide evidence for an outflow. Methods. We obtained new VLT/MUSE WFM and deep NRAO/VLA B-configuration 21 cm data of the galaxy. The MUSE data provide morphology, kinematics, and emission line ratios of Hβ/Hα and [O ¯III]λ5007/Hα of the low surface-brightness filaments, while the VLA data deliver morphology and kinematics of the neutral gas in and around the system. Both datasets are used in concert for comparisons between the ionised and the neutral phase. Results. We report the prolongation of a lacy filamentary ionised structure up to a projected distance of 16 kpc at SBHα = 1.5 - 10-18 erg s cm-2 arcsec-2. The filaments exhibit unusual low Hα/Hβ 2.4 and low [Oa ¯III]/Hα ∼ 0.4 - 0.6 typical of diffuse ionised gas. They are spectrally narrow (∼20 km s-1) and exhibit no velocity sub-structure. The filaments extend outwards from the elongated Ha ¯I halo. On small scales, the NHI peak is offset from the main star-forming sites. The morphology and kinematics of Ha ¯I and Ha ¯II reveal how star-formation-driven feedback interacts differently with the ionised and the neutral phase. Conclusions. We reason that the filaments are a large-scale manifestation of star-formation- driven feedback, namely limb-brightened edges of a giant outflow cone that protrudes through the halo of this gas-rich system. A simple toy model of such a conical structure is found to be commensurable with the observations.