Background: Several epidemiological studies report a negative association between Cancer and Alzheimer's disease (AD). Objective: To characterize the trajectories of memory loss in individuals with early amnestic cognitive impairment with and without history of previous cancer. Methods: Cognitive deterioration was assessed using the Montreal Cognitive Assessment (MoCA) or MoCA-Memory Index Score (MoCA-MIS) biannually in subjects with early amnestic cognitive impairment followed-up retrospectively from 2007 to 2021. History of Cancer was obtained from clinical records. Simple linear regressions of MoCA-MIS scores were calculated for each subject and analyzed with K-means cluster analysis to identify subgroups with different cognitive decline trajectories. χ2 and t tests were used for descriptive categorical and continuous variables and mixed multiple linear regressions to determine cognitive decline covariates. Results: Analysis of the trajectory of cognitive decline in 141 subjects with early amnestic cognitive impairment identified two subgroups: Fast (n = 60) and Slow (n = 81) progressors. At baseline Fast progressors had better MoCA-MIS (p < 0.001) and functionality (CDR p = 0.02, AD8 p = 0.05), took less anti-dementia medications (p = 0.005), and had higher depression rates (p = 0.02). Interestingly, Fast progressors slowed their speed of memory decline (from 1.6 to 1.1 MoCA-MIS points/year) and global cognitive decline (from 2.0 to 1.4 total MoCA points/year) when Cancer history was present. Conclusion: Two trajectories of amnestic cognitive decline were identified, possibly derived from different neurophysiopathologies or clinical stages. This study suggests that a history of previous Cancer slows down amnestic cognitive decline, specifically in a subgroup of subjects with depression at baseline and accelerated deterioration at follow-up.