It is postulated that quantum gravity is a sum over causal structures coupled to matter via scale evolution. Quantized causal structures can be described by studying simple matrix models where matrices are replaced by an algebra of quantum mechanical observables. In particular, previous studies constructed quantum gravity models by quantizing the moduli of Laplace, weight, and defining- function operators on Fefferman-Graham ambient spaces. The algebra of these operators underlies conformal geometries. We extend those results to include fermions by taking an osp(1/2) "Dirac square root" of these algebras. The theory is a simple, Grassmann, two-matrix model. Its quantum action is a Chern-Simons theory whose differential is a first-quantized, quantum mechanical Becchi-Rouet-Stora-Tyutin operator. The theory is a basic ingredient for building fundamental theories of physical observables.