In recent years, Convolutional Neural Networks (CNN) have been widely used for real-world applications in the field of computer vision. Their class-leading performance, however, depends heavily on the architecture used for a given problem. In most cases, the architectures are manually optimized by the researchers, a time-consuming process hard to achieve without prior knowledge of CNN. In this paper, we propose a new genetic algorithm for the optimization of the CNN architecture for a given image classification problem. This algorithm extends and refines existing research in the field, by allowing depth exploration, introducing a novel sequential crossover operator, using an incremental selective pressure schedule over evolution (favoring higher diversity in early generations) and by evaluating individual performances over the validation set with early stopping. The technique is validated in three image classification dataset, namely, CIFAR10, MNIST and Caltech256 datasets, which are widely used benchmarks for image classification algorithms. We evaluate the performance and total execution time over these datasets, and compare our results with those achieved by the best genetic methods published so far. In all cases, we achieve better results in terms of test accuracy, consistently over different datasets, while remaining in the same orders of magnitude of execution time of the fastest approaches.