A primary aim of the Nuclear Spectroscopic Telescope Array (NuSTAR) mission is to find and characterize heavily obscured Active Galactic Nuclei (AGNs). Based on mid-infrared photometry from the Wide-Field Infrared Survey Explorer (WISE) and optical photometry from the Sloan Digital Sky Surveys, we have selected a large population of luminous obscured AGNs (i.e., "obscured quasars"). Here we report NuSTAR observations of four WISE-selected heavily obscured quasars for which we have optical spectroscopy from the Southern African Large Telescope and W. M. Keck Observatory. Optical diagnostics confirm that all four targets are AGNs. With NuSTAR hard X-ray observations, three of the four objects are undetected, while the fourth has a marginal detection. We confirm that these objects have observed hard X-ray (10-40 keV) luminosities at or below similar to 10(43) erg s(-1). We compare X-ray and IR luminosities to obtain estimates of the hydrogen column densities (N-H) based on the suppression of the hard X-ray emission. We estimate N-H of these quasars to be at or larger than 10(25) cm(-2), confirming that WISE and optical selection can identify very heavily obscured quasars that may be missed in X-ray surveys, and they do not contribute significantly to the cosmic X-ray background. From the optical Balmer decrements, we found that our three extreme obscured targets lie in highly reddened host environments. This galactic extinction cannot adequately explain the more obscured AGNs, but it may imply a different scale of obscuration in the galaxy.