The linearized Einstein equations in D spacetime dimensions can be written as twisted self-duality equations expressing that the linearized curvature tensor of the graviton described by a rank-2 symmetric tensor is dual to the linearized curvature tensor of the "dual graviton" described by a tensor of (D - 3, 1) Young symmetry type. In the case of four dimensions, both the graviton and its dual are rank-2 symmetric tensors [Young symmetry type (1, 1)], while in the case of 11 spacetime dimensions relevant to M theory, the dual graviton is described by a tensor of (8, 1) Young symmetry type. We provide in this paper an action principle that yields the twisted self-duality conditions as equations of motion, keeping the graviton and its dual on equal footing. In order to construct a local, quadratic, variational principle for the twisted linear self-duality equations, it is necessary to introduce two "prepotentials." These are also tensors of mixed Young symmetry types and are obtained by solving the Hamiltonian constraints of the Hamiltonian formulation either of the Pauli-Fierz action for the graviton or of the Curtright action for its dual, the resulting actions being identical. The prepotentials enjoy interesting gauge-invariance symmetries, which are exhibited and generalize the gauge symmetries found in D = 4. A variational principle where the basic variables are the original Pauli-Fierz field and its dual can also be given, but contrary to the prepotential action, the corresponding action is nonlocal in space-while remaining local in time. We also analyze in detail the Hamiltonian structure of the theory and show that the graviton and its dual are canonically conjugate in a sense made precise in the text.