The study has been carried out on the prospects of probing the sterile neutrino mixing with the magnetized iron calorimeter (ICAL) at the India-based Neutrino Observatory (INO), using atmospheric neutrinos as a source. The so-called 3 + 1 scenario is considered for active-sterile neutrino mixing and lead to projected exclusion curves in the sterile neutrino mass and mixing angle plane. The analysis is performed using the neutrino event generator NUANCE, modified for ICAL, and folded with the detector resolutions obtained by the INO collaboration from a full GEANT4-based detector simulation. A comparison has been made between the results obtained from the analysis considering only the energy and zenith angle of the muon and combined with the hadron energy due to the neutrino induced event. A small improvement has been observed with the addition of the hadron information to the muon. In the analysis we consider neutrinos coming from all zenith angles and the Earth matter effects are also included. The inclusion of events from all zenith angles improves the sensitivity to sterile neutrino mixing by about 35% over the result obtained using only down-going events. The improvement mainly stems from the impact of Earth matter effects on active-sterile mixing. The expected precision of ICAL on the active-sterile mixing is explored and the allowed confidence level (C.L.) contours presented. At the assumed true value of 10. for the sterile mixing angles and marginalization over Delta m(41)(2) and the sterile mixing angles, the upper bound at 90% C.L. (from two-parameter plots) is around 20 degrees for theta(14) and theta(34), and about 12 degrees for theta(24).