This paper proposes the use of Artificial Neural Networks (ANN) for the efficient bidding of a Photovoltaic power plant with Energy Storage System (PV-ESS) participating in Day-Ahead (DA) and Real-Time (RT) energy and reserve markets under uncertainty. The Energy Management System (EMS) is based on Multi-Agent Deep Reinforcement Learning (MADRL). The MADRL scheme aims to maximize the profit of the hybrid PV-ESS plant through an efficient bidding in both markets. Results show that the MADRL framework can fulfill both the financial and physical constraints faced by the PV-ESS plant while providing energy and ancillary services. Daily market incomes have comparable mean values regarding traditional optimization approaches (average value of 1839 USD), but with a 45.3% smaller variance. Furthermore, it maintains a reference-tracking performance of 86.63% for one-year-round participation, against a 73.05% and 79.13% performance obtained with scenario-based robust and stochastic programming implementations, respectively.