Performance in autonomous driven vehicles is susceptible of degradation when traversing different terrains, thus needing motion controllers to be tuned for different terrain profiles. Such tuning stage is a time consuming process for the programmer or operator, and it is often based on intuition or heuristic approaches, and once tuned, the performance of the vehicle varies according to the terrain nature. In this context, we provide a visual based approach to identify terrain variability and its transitions, while observing and learning the performance of the vehicle using machine learning techniques. Based on the identified terrain and the knowledge regarding the performance of the vehicle, our system self-tunes the motion controller, in real time, to enhance its performance. In particular, the trajectory tracking errors are reduced, the control input effort is decreased, and the effects of the wheel-terrain interaction are mitigated preserving the system robustness. The tests were carried out by simulation and experimentation using a robotized commercial platform. Finally, implementation details and results are included in this paper, showing an enhancement in the motion performance up to 92.4% when the highest accuracy of the terrain classifier was 84.3%.