This paper presents a digital architecture for face detection on infrared (IR) images. We use Local Binary Patterns (LBP) to build a feature vector for each pixel, which represents the texture of the image in a vicinity of that pixel. We use a Support Vector Machine (SVM), trained with 306 images of 51 different subjects, to recognize human face textures. Finally, we group the classified pixels into rectangular boxes enclosing the faces using an algorithm for connected components. These boxes can then be used to track, count, or identify faces in a scene, for example. We implemented our architecture on a Xilinx XC6SLX45 FPGA and tested it on 306 IR images of 51 subjects, different from the data used to train the SVM. The circuit correctly identifies 100% of the faces in the images, and reports 4.5% of false positives. We also tested the system on a set of IR video streams featuring multiple faces per image, with varied poses and backgrounds, and obtained a hit rate of 94.5%, with 7.2% false positives. The circuit uses less than 25% of the logic resources available on the FPGA, and can process 313 640x480-pixel images per second with a 100MHz clock, while consuming 266mW of power.