The scarcity of liquid water in the hyperarid core of the Atacama Desert makes this region one of the most challenging environments for life on Earth. The low numbers of microbial cells in the soils suggest that within the Atacama Desert lies the dry limit for life on our planet. Here, we show that the Ca-sulfate crusts of this hyperarid core are the habitats of lithobiontic micro-organisms. This microporous, translucent substrate is colonized by epilithic lichens, as well as endolithic free-living algae, fungal hyphae, cyanobacteria and non photosynthetic bacteria. We also report a novel type of endolithic community, "hypoendoliths", colonizing the undermost layer of the crusts. The colonization of gypsum crusts within the hyperarid core appears to be controlled by the moisture regime. Our data shows that the threshold for colonization is crossed within the dry core, with abundant colonization in gypsum crusts at one study site, while crusts at a drier site are virtually devoid of life. We show that the cumulative time in 1 year of relative humidity (RH) above 60% is the best parameter to explain the difference in colonization between both sites. This is supported by controlled humidity experiments, where we show that colonies of endolithic cyanobacteria in the Ca-sulfate crust undergo imbibition process at RH > 60%. Assuming that life once arose on Mars, it is conceivable that Martian micro-organisms sought refuge in similar isolated evaporite microenvironments during their last struggle for life as their planet turned arid.