For more than twenty years, we have witnessed a continuous and significant growth in the scope and quality of research in Prognostics and Health Management (PHM). Prognostic algorithms and risk assessment metrics naturally play a critical role in this regard, since they provide the necessary information to take preventive measures and avoid catastrophic system failures. Unfortunately, the problem of failure prognostics has been treated many times from a heuristic, and mostly intuitive, standpoint. Indeed, the PHM community has often validated contributions to the state-of-the-art solely based on the performance experienced under specific run-to-failure experiments, and accepted lack of mathematical rigor in the formulation of the prediction problem itself. In this paper, we revisit the fundamentals of the prognostic problem, providing constructive criticism to inconsistencies found in approaches that have been adopted by many researchers within the PHM community. In addition, we propose a rigorous mathematical framework for failure prognostics, introducing failure probability measures for both discrete- and continuous-time dynamical systems that truly formalize the prognostic problem. We further discuss the philosophical implications of these novel notions in the context of a paradigm change, using as an illustrative example the problem of Lithium-Ion battery condition monitoring.