Due to its harmful effects on human health and agriculture, ground-level ozone concentrations are continually monitored nowadays in most places in the world. However, predicting ground-level ozone concentrations is difficult and thus poses a major concern in urban areas worldwide. In this paper, we investigate the use of deep recurrent neural nets to forecast ground-level ozone concentrations at Santiago (Chile), one of the most polluted cities in South America. It is found that the accuracy of current prediction models for peaks of the ozone concentration, 1-day ahead, is often lower than expected, which limits their practical utility as tools to anticipate critical pollution events. To address this issue, we propose to adopt a multitask learning criterion in which the model is not only trained to predict the expected value at the next time step but multiple quantiles of the response distribution. Experiments on real data illustrate that this approach improves the prediction accuracy for high values of the time series.