Using Wigner-deformed Heisenberg oscillators, we construct 3D Chern-Simons models consisting of fractional-spin fields coupled to higher-spin gravity and internal non-abelian gauge fields. The gauge algebras consist of Lorentz-tensorial Blencowe-Vasiliev higher-spin algebras and compact internal algebras intertwined by infinite-dimensional generators in lowest-weight representations of the Lorentz algebra with fractional spin. In integer or half-integer non-unitary cases, there exist truncations to gl(l, l +/- 1) or gl(l vertical bar l +/- 1) models. In all non-unitary cases, the internal gauge fields can be set to zero. At the semi-classical level, the fractional-spin fields are either Grassmann even or odd. The action requires the enveloping-algebra representation of the deformed oscillators, while their Fock-space representation suffices on-shell.