Objective To determine whether the extent of overlap of the genetic architecture among the sporadic late-onset Alzheimer's Disease (sLOAD), familial late-onset AD (fLOAD), sporadic early-onset AD (sEOAD), and autosomal dominant early-onset AD (eADAD). Methods Polygenic risk scores (PRSs) were constructed using previously identified 21 genome-wide significant loci for LOAD risk. Results We found that there is an overlap in the genetic architecture among sEOAD, fLOAD, and sLOAD. The highest association of the PRS and risk (odds ratio [OR] = 2.27; P = 1.29 × 10−7) was observed in sEOAD, followed by fLOAD (OR = 1.75; P = 1.12 × 10−7) and sLOAD (OR = 1.40; P = 1.21 × 10−3). The PRS was associated with cerebrospinal fluid ptau181-Aβ42 on eADAD (P = 4.36 × 10−2). Conclusion Our analysis confirms that the genetic factors identified for LOAD modulate risk in sLOAD and fLOAD and also sEOAD cohorts. Specifically, our results suggest that the burden of these risk variants is associated with familial clustering and earlier onset of AD. Although these variants are not associated with risk in the eADAD, they may be modulating age at onset.