Apparatus, method and program for detecting act of looking aside

More accurate detection of an act of looking aside is to be achieved. A direction-of-face detection unit detects the orientation of face of a driver, and a direction-of-sight line detection unit detects the orientation of sight line of the driver, based on an image photographed by the photographing unit. A field-of-view detection unit uses the sight line orientation of the driver as a reference to detect a range depending on the difference between the face orientation and sight line orientation of the driver as the field of view of the driver. A judging unit judges that the act of looking aside is going on when the face orientation of the driver is out of the face orientation-judging range. Also, the judging unit judges based on the driver's field of view whether or not the driver is looking aside when the face orientation of the driver is within the face orientation-judging range. The invention is applicable to an apparatus for detecting an act of looking aside by a driver.