Accurate seismic risk modeling requires knowledge of key structural characteristics of buildings. However, to date, the collection of such data is highly expensive in terms of labor, time and money and thus prohibitive for a spatially continuous large-area monitoring. This study quantitatively evaluates the potential of an automated and thus more efficient collection of vulnerability-related structural building characteristics based on Deep Convolutional Neural Networks (DCNNs) and street-level imagery such as provided by Google Street View. The proposed approach involves a tailored hierarchical categorization workflow to structure the highly heterogeneous street-level imagery in an application-oriented fashion. Thereupon, we use state-of-the-art DCNNs to explore the automated inference of Seismic Building Structural Types. These reflect the main-load bearing structure of a building, and thus its resistance to seismic forces. Additionally, we assess the independent retrieval of two key building structural parameters, i.e., the material of the lateral-load-resisting system and building height to investigate the applicability for a more generic structural characterization of buildings. Experimental results obtained for the earthquake-prone Chilean capital Santiago show accuracies beyond κ = 0.81 for all addressed classification tasks. This underlines the potential of the proposed methodology for an efficient in-situ data collection on large spatial scales with the purpose of risk assessments related to earthquakes, but also other natural hazards (e.g., tsunamis, or floods).