For ground-based optical imaging with current CCD technology, the Poisson fluctuations in source and sky background photon arrivals dominate the noise budget and are readily estimated. Another component of noise, however, is the signal from the undetected population of stars and galaxies. Using injection of artifical galaxies into images, we demonstrate that the measured variance of galaxy moments (used for weak gravitational lensing measurements) in Dark Energy Survey (DES) images is significantly in excess of the Poisson predictions, by up to 30 per cent, and that the background sky levels are overestimated by current software. By cross-correlating distinct images of 'empty' sky regions, we establish that there is a significant image noise contribution from undetected static sources (US), which, on average, are mildly resolved at DES resolution. Treating these US as a stationary noise source, we compute a correction to the moment covariance matrix expected from Poisson noise. The corrected covariance matrix matches the moment variances measured on the injected DES images to within 5 per cent. Thus, we have an empirical method to statistically account for US in weak lensing measurements, rather than requiring extremely deep sky simulations. We also find that local sky determinations can remove most of the bias in flux measurements, at a small penalty in additional, but quantifiable, noise.