We present a study of the HD 165054 astrometric calibration field that has been periodically observed with the Gemini Planet Imager (GPI). HD 165054 is a bright star within Baade's Window, a region of the galactic plane with relatively low extinction from interstellar dust. HD 165054 was selected as a calibrator target due to the high number density of stars within this region (similar to 3 stars per square arcsecond with H < 22), necessary because of the small field of view of the GPI. Using nine epochs spanning over five years, we have fit a standard five-parameter astrometric model to the astrometry of seven background stars within close proximity to HD 165054 (rho < 2 ''). We achieved a proper motion precision of similar to 0.3 mas yr(-1) and constrained the parallax of each star to be less than or similar to 1 mas. Our measured proper motions and parallax limits are consistent with the background stars being a part of the galactic bulge. Using these measurements, we find no evidence of any systematic trend of either the plate scale or the north angle offset of GPI between 2014 and 2019. We compared our model describing the motions of the seven background stars to observations of the same field in 2014 and 2018 obtained with Keck/NIRC2, an instrument with excellent astrometric calibration. We find that the predicted position of the background sources is consistent with that measured by NIRC2, within the uncertainties of the calibration of the two instruments. In the future, we will use this field as a standard astrometric calibrator for the upgrade of GPI and potentially for other high-contrast imagers.