We present an efficient multi-GPU approach for the Exchange Monte Carlo method designed for the simulation of disordered spin systems. Parallel computation is organized using a two-level scheme, allowing the algorithm to scale its performance in the presence of faster GPUs as well as multiple GPUs. Performance results show that spin-level performance is between one and two orders of magnitude faster than a sequential CPU implementation and approximate to 7 times faster than a parallel multi-core CPU implementation running on 16 cores. Multi-GPU performance scales with almost 99 % of efficiency when using two GPUs at size L = 256.