Traditionally, non-parametric impulse and frequency response functions are estimated by taking the ratio of power spectral density estimates. However, this approach may often lead to non-causal estimates. In this letter, we derive a closed form expression for the impulse response estimator by smoothed empirical transfer function estimate, which allows optimal enforcement of causality on non-parametric estimators based on spectral analysis. The new method is shown to be asymptotically unbiased and of minimum covariance in a positive semidefinite sense among a broad class of linear estimators. Numerical simulations illustrate the performance of the new estimator.