Gaze direction determination can be a powerful anticipatory perceptual mechanism for determining the next action of other individuals, humans or robots. It can allow cooperation, synchronization or competition between robots. This is of particular importance in the case of anthropomorphic robots, which in addition of having a human-like body, should behave as humans and have similar attention mechanisms for tracking and gazing other individuals and objects. We address this problem by proposing a gaze direction determination system for robots. This system is based primarily on a robot head pose detection system that consists of two processing stages: computation of scale-invariant local descriptors of the scene and matching of these descriptors against descriptors of robot head prototypes already stored in a database. These prototypes correspond to images of robot heads taken under different view angles. After the robot head pose is detected, the robot gaze direction is determined by a composed coordinate transformation that considers the three-dimensional pose of the observing robot's camera, the detected robot head pose with respect to the observing camera, and the head model of the observed robot. Results of the successful application of the proposed system in real robots are presented. (C) Koninklijke Brill NV, Leiden and The Robotics Society of Japan, 2009