The following paper examines the reliability and structure validity of a quantitative observational teacher pedagogical competencies scale implemented at one of Chile’s largest higher education institutions. In a context in which new students accessing post-secondary education are challenging traditional teaching methods, the evaluation of this instrument is presented as a relevant case study for those interested in promoting teachers’ pedagogical competencies. Reliability analyses considered the KR-20 coefficient and corrected item-total correlations. Structure validity was assessed through an exploratory factor analysis in which the concept’s theoretical and latent structures were compared. The results suggest that the scale has high levels of internal consistency. Additionally, although the scale’s theoretical and latent structures do not match exactly, relevant common elements are found. The considerations for applying these types of educational measurement instruments are discussed.