Altitudinal gradients provide valuable information about the effects of environmental variables on changes in species richness and composition as well as the distribution of below ground fungal communities. Since most knowledge in this respect has been gathered on aboveground communities, we focused our study towards the characterization of belowground fungal communities associated with two different ages of Norway spruce (Picea abies) trees along an altitudinal gradient. By sequencing the internal transcribed spacer (ITS) region on the Illumina platform, we investigated the fungal communities in a floristically and geologically relatively well explored forest on the slope of Mt. Iseler of the Bavarian Alps. From fine roots and rhizosphere of a total of 90 of Norway spruce trees from 18 plots we detected 1285 taxa, with a range of 167 to 506 (average 377) taxa per plot. Fungal taxa are distributed over 96 different orders belonging to the phyla Ascomycota, Basidiomycota, Chrytridiomycota, Glomeromycota, and Mucoromycota. Overall the Agaricales (438 taxa) and Tremellales (81 taxa) belonging to the Basidiomycota and the Hypocreales (65 spp.) and Helotiales (61 taxa) belonging to the Ascomycota represented the taxon richest orders. The evaluation of our multivariate generalized mixed models indicate that the altitude has a significant influence on the composition of the fungal communities (p < 0.003) and that tree age determines community diversity (p < 0.05). A total of 47 ecological guilds were detected, of which the ectomycorrhizal and saprophytic guilds were the most taxon-rich. Our ITS amplicon Illumina sequencing approach allowed us to characterize a high fungal community diversity that would not be possible to capture with fruiting body surveys alone. We conclude that it is an invaluable tool for diverse monitoring tasks and inventorying biodiversity, especially in the detection of microorganisms developing very ephemeral and/or inconspicuous fruiting bodies or lacking them all together. Results suggest that the altitude mainly influences the community composition, whereas fungal diversity becomes higher in mature/older trees. Finally, we demonstrate that novel techniques from bacterial microbiome analyses are also useful for studying fungal diversity and community structure in a DNA metabarcoding approach, but that incomplete reference sequence databases so far limit effective identification.