Standard quantum tomography of a single qudit achieves an infidelity that scales in the worst case as O(1/root N) for a sample of size N. Here, we propose a suitable generalization of the two-stage adaptive quantum tomography for a qubit to the case of a single qudit. This achieves an infidelity of the order of O[1/root N-0(N - N-0)] for all quantum states, where N-0 and N - N-0 are the ensemble sizes employed in the two stages of the method. This result is based on a second-order Taylor series expansion of the infidelity that is obtained by means of the Frechet derivative and measurement outcomes modeled by a multinomial distribution. Numerical simulations indicate that the choice N-0 = N/2 leads to an infidelity that scales approximately as O(1/N) for all quantum states in a wide range of dimensions, that is, a quadratic improvement of the infidelity when compared to standard quantum tomography in the case of low-rank states.