In this work, we address two main objectives. The first one is to provide a rigorous foundation to the maximum entropy principle in statistical physics, by making use of the Fenchel-Rockafellar duality. The second objective is to discuss the well-foundedness of the so-called escort distributions in the context of non-extensive entropy maximization. The duality treatment of maximum entropy confirms the non-rigorous results obtained via the usual variational calculus, however, the use of escort distributions yields undefined behavior when used consistently, and only leads to the desired results when used in an ad-hoc manner.