Lumped elements models of vocal folds are relevant research tools that can enhance the understanding of the pathophysiology of many voice disorders. In this paper, we use the port-Hamiltonian framework to obtain an energy-based model for the fluid-structure interactions between the vocal folds and the airflow in the glottis. The vocal fold behavior is represented by a three-mass model and the airflow is described as a fluid with irrotational flow. The proposed approach allows to go beyond the usual quasi-steady onedimensional flow assumption in lumped mass models. The simulation results show that the proposed energy-based model successfully reproduces the oscillations of the vocal folds, including the collision phenomena, and it is useful to analyze the energy exchange between the airflow and the vocal folds.