Keeping students engaged can improve their learning. Gamification is a technique that has been shown to improve engagement, transferring the positive effects of videogames. However, the mobile games industry has identified that player retention decreases quickly over time, but it is unclear if this decreasing retention pattern transfers to gamification activities. To check if the behavioral engagement was stable over a 14-week engineering course, we gamified an existing virtual learning environment (VLE), conducting an observational study for two semesters. Actions performed by students within the VLE were used to measure behavioral engagement, which allows us to track student behavior weekly. To analyze the gamification effect, we divided user actions into two groups: academic and gamified. The results showed that the gamified actions performed by students in the VLE decreased every week, which was significantly higher than the decrease in academic actions. This shows that the decreasing retention pattern seen in mobile games was also present in this study. These results suggest that, just like in mobile games, to sustain the behavioral engagement over a long period of time in a gamified class, additional modern game-design techniques should be considered.