This article presents an initial comparative performance study between a Deep Reinforcement Learning (DRL) agent and baseline heuristic within the context of different intercore spacings in multicore fiber (MCF) networks. The analysis focuses on a case study involving a three-core fiber configuration within the Eurocore topology. The study's findings indicate that the DRL agents outperformed the reference heuristic in terms of blocking probability, particularly under specific inter-core spacing conditions. This superior performance can be attributed to the adaptability of DRL agents in adjusting their learned policies during training. These findings suggest that DRL algorithms have significant potential in efficiently addressing resource allocation challenges within MCF-EON networks, even in scenarios characterized by stringent constraints.