We present the first results from the polarimetry mode of the Gemini Planet Imager (GPI), which uses a new integral field polarimetry architecture to provide high contrast linear polarimetry with minimal systematic biases between the orthogonal polarizations. We describe the design, data reduction methods, and performance of polarimetry with GPI. Point-spread function (PSF) subtraction via differential polarimetry suppresses unpolarized starlight by a factor of over 100, and provides sensitivity to circumstellar dust reaching the photon noise limit for these observations. In the case of the circumstellar disk around HR 4796A, GPI's advanced adaptive optics system reveals the disk clearly even prior to PSF subtraction. In polarized light, the disk is seen all the way in to its semi-minor axis for the first time. The disk exhibits surprisingly strong asymmetry in polarized intensity, with the west side greater than or similar to 9 times brighter than the east side despite the fact that the east side is slightly brighter in total intensity. Based on a synthesis of the total and polarized intensities, we now believe that the west side is closer to us, contrary to most prior interpretations. Forward scattering by relatively large silicate dust particles leads to the strong polarized intensity on the west side, and the ring must be slightly optically thick in order to explain the lower brightness in total intensity there. These findings suggest that the ring is geometrically narrow and dynamically cold, perhaps shepherded by larger bodies in the same manner as Saturn's F ring.