We investigate inclusive prompt photon and semi-inclusive prompt photon-hadron production in high energy proton-nucleus collisions using the color glass condensate formalism, which incorporates non-linear dynamics of gluon saturation at small x via the Balitsky-Kovchegov equation with a running coupling. For inclusive prompt photon production, we rewrite the cross section in terms of direct and fragmentation contributions and show that the direct photon (and isolated prompt photon) production is more sensitive to gluon saturation effects. We then analyze azimuthal correlations in photon-hadron production in high energy proton-nucleus collisions and obtain a strong suppression of the away-side peak in photon-hadron correlations at forward rapidities, similar to the observed mono-jet production in deuteron-gold collisions at forward rapidity at the Relativistic Heavy Ion Collider. We make predictions for the nuclear modification factor R-p(d)A and photon-hadron azimuthal correlations in proton(deuteron)-nucleus collisions at the Relativistic Heavy Ion Collider and the Large Hadron Collider at various rapidities.