The interplay of spin and lattice fluctuations in two-dimensional magnets without inversion symmetry is investigated. We find a general form for the magnetoelastic coupling between magnons and existing chiral phonons based on the symmetries of the crystalline lattice. We show that in hexagonal lattices, the coupling of magnons and chiral phonons derives from an anisotropic exchange spin model containing topological phases of magnons. Using the Heisenberg-Kitaev-Γ model, we show how magnon-polaron edge states with circular polarization arise from this interaction. Our findings exploit the polarization degrees of freedom in spin-lattice systems, thus setting the ground for the transfer of angular momentum between chiral phonons and magnons.