Recognizing variable stars is a task of interest in the astronomy community. Currently, this task has taken advantage of deep learning algorithms. However, these algorithms require a large amount of data to achieve high levels of precision. In this work, self-supervised learning is proposed to improve the classification of variable stars considering a reduced amount of data using recurrent networks. The experiments in Gaia dataset show that the proposed approach allows to improve performance, when compared with traditional initialization schemes, up to 7% and 13% in real databases in semi-supervised learning scenarios. In future work, we propose considering experiments with other variable star databases.