Chile is one of the main exporters of sweet cherries in the world and one of the few in the southern hemisphere, being their harvesting between October and January. Hence, Chilean cherries have gained market in the last few years and positioned Chile in a strategic situation which motivates to undergo through a deep innovation process in the field. Currently, cherry crop estimates have an error of approximately 45%, which propagates to all stages of the production process. In order to mitigate such error, we develop, test and evaluate a deep neural-based approach, using a portable artificial vision system to enhance the cherries harvesting estimates. Our system was tested in a cherry grove, under real field conditions. It was able to detect cherries with up to 85% of accuracy and to estimate production with up to 25% of error. In addition, it was able to classify cherries into four sizes, for a better characterization of the production for exportation.