Method, apparatus and computer program product for providing face pose estimation

TECHNOLOGICAL FIELD

p-0002Embodiments of the present invention relate generally to image processing technology and, more particularly, relate to a method, apparatus and computer program product for providing face pose estimation.

BACKGROUND

p-0003Face detection and recognition is becoming an increasingly more important technology. In this regard, for example, face detection may be useful in biometrics, user interface, gaming and other areas such as creating context for accessing communities in the mobile domain. Face detection may also be important going forward in relation to initiatives such as metadata standardization.

p-0004Although face detection techniques continue to improve, many current methods require either a high computation capability (e.g., statistical methods of detecting faces by scanning images in a traversing way on multiple scales) or suffer from limited face detection performance (e.g., structure-based methods with relatively high false alarms of face detection). Furthermore, some statistical face detection mechanisms have degraded performance for multi-view face detection in relation to front face detection. As another complicating issue, faces encountered in a particular image may not always be oriented in the same direction relative to the camera, which could negatively impact face detection. For example, in-plane rotation (e.g., faces that are rotated along the axis from the face to the observer) may complicate face detection in some cases.

p-0005Accordingly, the tendency for developing devices with continued increases in their capacity to create content, store content and/or receive content relatively quickly upon request, the trend toward electronic devices (e.g., mobile electronic devices such as mobile phones) becoming increasingly ubiquitous in the modern world, and the drive for continued improvements in interface and access mechanisms to unlock the capabilities of such devices, may make it desirable to provide further improvements in the area of face detection.

BRIEF SUMMARY OF SOME EXAMPLES

p-0006A method, apparatus and computer program product are therefore provided to enable an improved face detection mechanism. In this regard, in some exemplary embodiments, a mechanism is provided that is capable of incorporating a fusion of certain aspects of statistical and structural face detection methods in order to realize advantages associated with both, while reducing disadvantages associated therewith. As such, embodiments of the present invention may provide a relatively robust ability for detecting faces even under different conditions (e.g., having different head poses or facial views).

p-0007In an exemplary embodiment, a method of providing face pose estimation for use in face detection is provided. The method may include utilizing a selected portion of classifiers in detectors to determine coarse pose information for a candidate face in an image, determining fine pose information for the candidate face based at least in part on the determined coarse pose information, and employing another portion of the classifiers in the detectors to perform face detection based at least in part on the fine pose information to determine whether the candidate face corresponds to a face.

p-0008In another exemplary embodiment, a computer program product for providing face pose estimation for use in face detection is provided. The computer program product includes at least one computer-readable storage medium having computer-executable program code instructions stored therein. The computer-executable program code instructions may include program code instructions for utilizing a selected portion of classifiers in detectors to determine coarse pose information for a candidate face in an image, determining fine pose information for the candidate face based at least in part on the determined coarse pose information, and employing another portion of the classifiers in the detectors to perform face detection based at least in part on the fine pose information to determine whether the candidate face corresponds to a face.

p-0009In another exemplary embodiment, an apparatus for providing face pose estimation for use in face detection is provided. The apparatus may include a processor. The processor may be configured to utilize a selected portion of classifiers in detectors to determine coarse pose information for a candidate face in an image, determine fine pose information for the candidate face based at least in part on the determined coarse pose information, and employ another portion of the classifiers in the detectors assoc to perform face detection based at least in part on the fine pose information to determine whether the candidate face corresponds to a face.

p-0010In yet another exemplary embodiment, an apparatus for providing face pose estimation for use in face detection is provided. The apparatus may include means for utilizing a selected portion of classifiers in detectors to determine coarse pose information for a candidate face in an image, means for determining fine pose information for the candidate face based at least in part on the determined coarse pose information, and means for employing another portion of the classifiers in the detectors to perform face detection based at least in part on the fine pose information to determine whether the candidate face corresponds to a face.

p-0011Embodiments of the invention may provide a method, apparatus and computer program product for employment, for example, in mobile or fixed environments. As a result, for example, computing device users may enjoy an improved capability for face detection.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

p-0012Having thus described some embodiments of the invention in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:

p-0013<figref idrefs="DRAWINGS">FIG. 1</figref> illustrates an example of a division of a rotation plane into a plurality of sectors according to an exemplary embodiment of the present invention;

p-0014<figref idrefs="DRAWINGS">FIG. 2</figref> shows an example of a statistical mechanism for pose evaluation according to an exemplary embodiment of the present invention;

p-0015<figref idrefs="DRAWINGS">FIG. 3</figref> illustrates one example of a fusion of structural and statistical methods for face detection according to an exemplary embodiment of the present invention;

p-0016<figref idrefs="DRAWINGS">FIG. 4</figref> shows a block diagram illustrating one example of a fusion of structural and statistical methods for face detection according to an exemplary embodiment of the present invention;

p-0017<figref idrefs="DRAWINGS">FIG. 5</figref> illustrates an example of a push-switch based pose estimator in accordance with an exemplary embodiment of the present invention;

p-0018<figref idrefs="DRAWINGS">FIG. 6</figref> illustrates a block diagram of an apparatus for providing face pose estimation according to an exemplary embodiment of the present invention;

p-0019<figref idrefs="DRAWINGS">FIG. 7</figref> shows an example of coarse pose evaluation that may be accomplished using a coarse pose evaluator in connection with an exemplary embodiment of the present invention; and

p-0020<figref idrefs="DRAWINGS">FIG. 8</figref> is a flowchart according to an exemplary method for providing face pose estimation for use in face detection according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF SOME EMBODIMENTS

p-0021Some embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms “data,” “content,” “information” and similar terms may be used interchangeably to refer to data capable of being transmitted, received and/or stored in accordance with embodiments of the present invention. Moreover, the term “exemplary”, as used herein, is not provided to convey any qualitative assessment, but instead merely to convey an illustration of an example. Additionally, the terms near and far are used herein in the relative sense so as to refer to objects being closer and further from some point with respect to one another, but not to otherwise represent any particular or quantifiable position. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments of the present invention.

p-0022Some embodiments of the present invention may provide a mechanism by which improvements may be experienced in relation to face detection and/or recognition. In this regard, for example, some embodiments may provide for a real-time or near real-time face detection scheme, which may be suitable for metadata assignment, biometrics, gaming, and numerous other activities on hand-held or other computing devices.

p-0023For many images taken, the individual or individuals in the image may have various different in-plane and out of plane rotations defining their respective pose or facial view. These rotations defining the poses of faces in the image may complicate face detection and recognition. Two basic categories of responses have been developed for various techniques that deal with these situations. In this regard, a first category includes pose evaluation using multiple trained detectors. For this approach, multiple face detectors are trained given corresponding training sets with various poses. The training sets include a plurality of faces oriented with various different amounts of in-plane and/or out of plane rotations (e.g., various poses). Image patches may then be attempted to be passed through the trained detectors, and the one through which the image patch passes and is recognized as a face, may be indicative of the corresponding pose of the face therein. <figref idrefs="DRAWINGS">FIG. 1</figref> shows an example of a division of the rotation plane into 20 sectors. For each sector, a face detector may be trained based on a face with a corresponding pose. As can be seen from <figref idrefs="DRAWINGS">FIG. 1</figref>, each sector defines a particular in-plane rotation and a particular out of plane rotation. By proceeding around all of the sectors and considering tolerances within each sector, the 20 trained detectors may be able to detect faces having various different poses within an image to provide an estimate of face pose information in a multi-view face detection application or system.

p-0024However, one disadvantage of this approach is that the approach only provides a pose scope, and not necessarily precise pose information. In addition, there is a trade-off between the number of sectors and the difficulty of convergence in detector training. For example, sector A of <figref idrefs="DRAWINGS">FIG. 1</figref> covers pose variations from −45° to 45° of in plane rotation. Thus, a corresponding training database for providing training examples for training the detector has to include many examples to cover incremental changes within the range, which may lead to difficult convergence in training because of relatively large pattern variations. This disadvantage could be partially solved by increasing the number of sectors in order to make convergence easier. However, increasing the number of sectors (e.g., by doubling) may provide a corresponding increase (e.g., double) in the time consumed for pose evaluation. The increase in time consumption may not be tolerable for a real-time system.

p-0025Another category of approaches to multi-view face detection may include pose evaluation by training a pose classifier. In this regard, for an image patch, the pose may initially be estimated by a pose classifier. Then, the image patch is sent to a corresponding face detector of the classified pose to verify whether the image patch includes a face or not. In the example embodiment of <figref idrefs="DRAWINGS">FIG. 1</figref>, which includes twenty sectors (e.g., sector A), the pose classifier is trained to distinguish the corresponding 20 poses. This approach also only provides a pose scope instead of precise pose. Meanwhile, training a unique pose classifier to distinguish as many as 20 poses can be a difficult task to realize as it relates to a multiple category classification problem, which is a classical difficulty in pattern analysis. In some cases, it may be possible to train 20 pose classifiers and use each one to classify one pose and remaining poses. However, this solution is also potentially time consuming and therefore may not be effectively useful in real-time scenarios.

p-0026Accordingly, embodiments of the present invention may be aimed at addressing problems related to difficulty in training convergence, coarse precision or lack of accuracy in pose evaluation, and low time efficiency in pose evaluation. In particular, embodiments of the present invention may provide a unique fusion of structure based and statistical based face detection methods. In this regard, embodiments of the present invention may initially employ coarse pose determination using a statistical method, followed by fine pose determination using a structural method to analyze an image patch for a face. Normalized data (e.g., rotated to verticality) may then be used to analyze for the existence of a face in the image patch (e.g., using another statistical method). Accordingly, some embodiments provide for a fine pose determination prior to performing a pose-based face detection application. Thus, embodiments of the present invention may provide fine pose information with relatively low increase in workload for a face-based application. In general terms, embodiments of the present invention may be referred to as relating to a push-switch based face pose evaluator because information is pushed between two different methods and then a method switch is made to push information back to the original method.

p-0027Face detection may be accomplished in some cases using cascaded classifiers forming a detector. In this regard, as shown in the example of <figref idrefs="DRAWINGS">FIG. 1</figref>, twenty different detectors could be formed by cascading weak classifiers. A weak classifier may be defined as a classifier which is only slightly correlated with the true classification. In contrast, a strong classifier may be a classifier that is arbitrarily well-correlated with the true classification. Weak classifiers may be easier to create than strong classifiers. However, a plurality of weak classifiers may, in some cases, provide results that may be on par with results provided by a strong classifier with less cost.

p-0028<figref idrefs="DRAWINGS">FIG. 2</figref> illustrates a potential mechanism for pose evaluation that could be used in conjunction with the example of <figref idrefs="DRAWINGS">FIG. 1</figref>. In this regard, n detectors (e.g., 20 detectors in the example of <figref idrefs="DRAWINGS">FIG. 1</figref>) may be comprised of cascaded sets of weak classifiers trained on faces with a particular pose or range of poses to detect faces having characteristics similar to the corresponding training poses. As such, each detector may be trained for a corresponding particular pose. Thus, for example, if an image patch is passed through detector <b>1</b> and results in a face detection, the detected face can be said to have a corresponding pose similar to that associated with the training faces used to train the detector (e.g., pose <b>1</b>). Meanwhile, if the image patch does not generate a face detection in relation to detector <b>1</b>, the image patch may still generate a face detection with a corresponding pose for any one or more of the other detectors (e.g., detector <b>2</b> to detector n). Therefore, in operation, the image patch may be input into all of the detectors in order to realize a multi-view face detector configured to detect possible faces with various corresponding poses. The pose information for each possible face may be simultaneously determinable based on the detector that detected the face.

p-0029Since face detection is central to a face-based application, it may be beneficial to initially determine pose information prior to performing face detection. The performance of pose determination may initially be done in a relatively coarse fashion using a statistical method such as the mechanism shown in <figref idrefs="DRAWINGS">FIG. 2</figref>. Based on the coarse pose information determined, fine pose information may then be accomplished using a structure based method thereby fusing statistical and structural methods. <figref idrefs="DRAWINGS">FIG. 3</figref> illustrates one example of a fusion of structural and statistical methods for face detection according to an exemplary embodiment. In this regard, as shown in <figref idrefs="DRAWINGS">FIG. 3</figref>, a statistical method <b>20</b> of pose evaluation (e.g., similar to the initial stages shown in <figref idrefs="DRAWINGS">FIG. 2</figref>) may be performed by several detectors that each comprise a cascaded series of weak classifiers in which each detector corresponds to a particular range of poses (e.g., covering a predetermined range of out of plane rotation angles and/or in-plane rotation angles). As a result of passing an image patch through the statistical method <b>20</b>, coarse pose information <b>22</b> may be passed through a structural method <b>24</b> to get fine pose information <b>26</b>. After normalizing the patches, a statistical method <b>28</b> (e.g., similar to the remaining stages or portions from the embodiment of <figref idrefs="DRAWINGS">FIG. 2</figref>) may be employed on the normalized patches <b>30</b> to provide a face detection output <b>32</b> that may be relatively accurate and robust without greatly increasing overhead costs.

p-0030A characteristic aspect of the embodiment shown in <figref idrefs="DRAWINGS">FIG. 3</figref> relates to the combination of detectors comprising several weak classifiers with a structure-based method to form a pose evaluator. Thus, an embodiment of the present invention could be realized by employing the initial stage or stages of the statistical method of <figref idrefs="DRAWINGS">FIG. 2</figref> for coarse pose determination and then employing a structural method to perform fine pose determination. Some or all of the remaining weak classifiers of the statistical method for face detection may then be used for corresponding face detection in each respective detector. However, separate generic statistical methods could be used for the respective operations associated with the statistical method <b>20</b> and the statistical method <b>28</b> of <figref idrefs="DRAWINGS">FIG. 3</figref> instead of using portions of the statistical method shown in <figref idrefs="DRAWINGS">FIG. 2</figref> as shown in <figref idrefs="DRAWINGS">FIG. 4</figref>.

p-0031<figref idrefs="DRAWINGS">FIG. 4</figref> shows a block diagram of the example described in reference to <figref idrefs="DRAWINGS">FIG. 3</figref> with an example image patch. In this regard, the image patch <b>40</b> may be analyzed for coarse pose determination using statistical method <b>42</b>. For example, a coarse pose determination may be achieved indicating a pose with between about 90 and 135 degrees of in-plane rotation and approximately no out of plane rotation. A structural method <b>44</b> may then be employed to obtain a fine pose determination of 135 degrees of in-plane rotation and no out of plane rotation. The image patch <b>46</b> may then be rotated to verticality for normalization and another statistical method <b>48</b> (or a portion of the same statistical method) may be employed for face detection. As can be seen from <figref idrefs="DRAWINGS">FIGS. 2-4</figref>, embodiments of the present invention may perform face detection for face-based applications without requiring a separate pose classifier. Accordingly, time efficiency of exemplary embodiments of the present invention may be increased relative to existing methods.

p-0032<figref idrefs="DRAWINGS">FIG. 5</figref> illustrates an example of a push-switch based pose estimator in accordance with an exemplary embodiment of the present invention. In this regard, as shown in <figref idrefs="DRAWINGS">FIG. 5</figref>, for a given image patch, a statistical-based method (e.g., Adaboost (adaptive boosting) based statistical method <b>50</b> as described in reference to <figref idrefs="DRAWINGS">FIG. 3</figref> above) may initially be employed for coarse pose estimation <b>52</b>. The coarse pose estimation <b>52</b> may be provided to a structure based detection method <b>54</b> to result in a fine pose evaluation <b>56</b>. The patch may be rotated to verticality by the structure based scheme and pushed back to the Adaboost based statistical method <b>50</b> for face detection, recognition, etc.

p-0033As indicated above, Adaboost may be an example of a statistical method that may be employed in connection with embodiments of the present invention. Other examples of statistical methods may include Gentle-Boost, RealBoost, FloatBoost, and/or the like. Meanwhile, exemplary structural methods may include employment of detectors in various different orientations that provide for coverage of respective different in-plane rotation angles, and employment of a series of differently oriented facial templates against which features of an image may be compared. In some cases, a result of employment of structure based scheme may be that the image patch analyzed is rotated to verticality.

p-0034<figref idrefs="DRAWINGS">FIG. 6</figref> illustrates one example of a schematic block diagram of an apparatus for enabling face pose estimation for face detection according to an exemplary embodiment of the present invention. An exemplary embodiment of the invention will now be described with reference to <figref idrefs="DRAWINGS">FIG. 6</figref>, in which certain elements of an apparatus for enabling face pose estimation for face detection are displayed. The apparatus of <figref idrefs="DRAWINGS">FIG. 6</figref> may be employed, for example, on a mobile terminal (e.g., a mobile phone, gaming device, personal digital assistant (PDA) and/or the like) or a variety of other devices, both mobile and fixed (such as, for example, a network device, personal computer, laptop computer, digital camera, or the like). Alternatively, embodiments may be employed on a combination of devices. Accordingly, some embodiments of the present invention may be embodied wholly at a single device or by devices in a client/server relationship. Furthermore, it should be noted that the devices or elements described below may not be mandatory and thus some may be omitted in certain embodiments.

p-0035Referring now to <figref idrefs="DRAWINGS">FIG. 6</figref>, an apparatus for enabling face pose estimation for face detection is provided. The apparatus may include or otherwise be in communication with a processor <b>70</b>, a user interface <b>72</b>, a communication interface <b>74</b> and a memory device <b>76</b>. The memory device <b>76</b> may include, for example, volatile and/or non-volatile memory. The memory device <b>76</b> may be configured to store information, data, applications, instructions or the like for enabling the apparatus to carry out various functions in accordance with exemplary embodiments of the present invention. For example, the memory device <b>76</b> could be configured to buffer input data for processing by the processor <b>70</b>. Additionally or alternatively, the memory device <b>76</b> could be configured to store instructions for execution by the processor <b>70</b>. As yet another alternative, the memory device <b>76</b> may be one of a plurality of databases that store information and/or media content.

p-0036The processor <b>70</b> may be embodied in a number of different ways. For example, the processor <b>70</b> may be embodied as various processing means such as a processing element, a coprocessor, a controller or various other processing devices including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a hardware accelerator, or the like. In an exemplary embodiment, the processor <b>70</b> may be configured to execute instructions stored in the memory device <b>76</b> or otherwise accessible to the processor <b>70</b>. As such, whether configured by hardware or software methods, or by a combination thereof, the processor <b>70</b> may represent an entity capable of performing operations according to embodiments of the present invention while configured accordingly. Accordingly, when the processor <b>70</b> is embodied as an ASIC, FPGA or the like, the processor <b>70</b> may be specifically configured hardware for conducting the operations described herein. Alternatively, when the processor <b>70</b> is embodied as an executor of software instructions, the instructions may specifically configure the processor <b>70</b>, which may otherwise be a general purpose processing element, to perform the algorithms and operations described herein.

p-0037Meanwhile, the communication interface <b>74</b> may be any means such as a device or circuitry embodied in either hardware, software, or a combination of hardware and software that is configured to receive and/or transmit data from/to a network and/or any other device or module in communication with the apparatus. In this regard, the communication interface <b>74</b> may include, for example, an antenna (or multiple antennas) and supporting hardware and/or software for enabling communications with a wireless communication network (e.g., network <b>78</b>). In fixed environments, the communication interface <b>74</b> may alternatively or also support wired communication. As such, the communication interface <b>74</b> may include a communication modem and/or other hardware/software for supporting communication via cable, digital subscriber line (DSL), universal serial bus (USB), Ethernet or other mechanisms.

p-0038The user interface <b>72</b> may be in communication with the processor <b>70</b> to receive an indication of a user input at the user interface <b>72</b> and/or to provide an audible, visual, mechanical or other output to the user. As such, the user interface <b>72</b> may include, for example, a keyboard, a mouse, a joystick, a display, a touch screen, a microphone, a speaker, or other input/output mechanisms. In an exemplary embodiment in which the apparatus is embodied as a server or some other network devices, the user interface <b>72</b> may be limited, or eliminated. However, in an embodiment in which the apparatus is embodied as a mobile terminal or personal computer, the user interface <b>72</b> may include, among other devices or elements, any or all of a speaker, a microphone, a display, and a keyboard or the like.

p-0039In an exemplary embodiment, the processor <b>70</b> may be embodied as, include or otherwise control a coarse pose evaluator <b>80</b>, a fine pose evaluator <b>82</b>, a face detector <b>84</b>, and in some cases also a candidate rotator <b>86</b>. However, the candidate rotator <b>86</b> may be a portion of the fine pose evaluator <b>82</b> in some instances. The coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, and the candidate rotator <b>86</b> may each be any means such as a device or circuitry embodied in hardware, software or a combination of hardware and software (e.g., processor <b>70</b> operating under software control, the processor <b>70</b> embodied as an ASIC or FPGA specifically configured to perform the operations described herein, or a combination thereof) that is configured to perform the corresponding functions of the coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, and the candidate rotator <b>86</b>, respectively, as described below.

p-0040In an exemplary embodiment, any or all of the coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, and the candidate rotator <b>86</b> may include instructions, code, modules, applications and/or circuitry for providing respective portions of face detection services. However, it should be noted that code, circuitry and/or instructions associated with the coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, and the candidate rotator <b>86</b> need not necessarily be modular. Moreover, as discussed below, in some cases, the coarse pose evaluator <b>80</b> and the face detector <b>84</b> may each implement portions of the same statistical based method or scheme (e.g., including a series of cascaded detectors) and in some cases the implemented portions may overlap. In some embodiments, communication between the coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, and the candidate rotator <b>86</b> is conducted via the processor <b>70</b>. However, the coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, and the candidate rotator <b>86</b> are alternatively in direct communication with each other or may have no communication with each other in other embodiments.

p-0041In an exemplary embodiment, the coarse pose evaluator <b>80</b> employs a series of trained detectors of a confidence based or statistical based method of face detection for determining an estimate of the pose of any identified face within a particular image patch. The coarse pose evaluator <b>80</b>, however, does not implement all of the classifiers of the detectors of the statistical based method. Instead, only a selected number of the classifiers may be employed to get a rough estimate of pose in a relatively short time due to the avoidance of using all of the classifiers of the detectors.

p-0042Given an image patch for a face related application, a typical trained detector could be used to determine whether the image patch included a face. However, given the number of trained detectors in a typical statistical based method scheme and the corresponding number of trained classifiers therein, time efficiency may be low if the image patch is analyzed using all of the classifiers. Accordingly, embodiments of the present invention may utilize less than the full complement of trained classifiers for pose determination in combination with a structural method to refine the pose information determined by the trained classifiers of the statistical method that are employed. Thus, the coarse pose evaluator <b>80</b> may be configured to quickly perform coarse pose evaluation to enable improved real-time performance by improving detection speed.

p-0043The coarse pose evaluator <b>80</b> may be configured to employ data used for training that includes relatively small pose variations. Even though relatively small pose variations are shown in the training data, given a face beyond the training scope, the probability of correctly classifying the given face with respect to its corresponding pose is relatively high since similarities between the training data and evaluated face are highest in both appearance and structure.

p-0044An exemplary statistical based method may include a plurality of detectors in which each detector is a trained detector D_n={C₁,C₂, . . . ,C_m} with m cascades, where C_i={W₁,W₂, . . . ,W_t} is the ith cascade with t weak classifiers. In some embodiments, the number of weak classifiers in each detector is more than 1,000. Each weak classifier would typically contribute a confidence score of indicating the degree to which the testing image patch is likely to be a face with the pose associated with the corresponding classifier. As an example, for a Gentle-Boost statistical method, the definition of W_imay be:

p-0045<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>W</mi><mi>i</mi></msub><mo>=</mo><mfrac><mrow><msub><mi>P</mi><mo>+</mo></msub><mo>-</mo><msub><mi>P</mi><mo>-</mo></msub></mrow><mrow><msub><mi>P</mi><mo>+</mo></msub><mo>+</mo><msub><mi>P</mi><mo>-</mo></msub></mrow></mfrac></mrow></mtd><mtd><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

p-0046where P₊ and P₋ are positive probability and negative probability in a partition area. It can be further written as:

p-0047<maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>W</mi><mi>i</mi></msub><mo>=</mo><mrow><mfrac><msub><mi>P</mi><mo>+</mo></msub><mrow><msub><mi>P</mi><mo>+</mo></msub><mo>+</mo><msub><mi>P</mi><mo>-</mo></msub></mrow></mfrac><mo>-</mo><mfrac><msub><mi>P</mi><mo>-</mo></msub><mrow><msub><mi>P</mi><mo>+</mo></msub><mo>+</mo><msub><mi>P</mi><mo>-</mo></msub></mrow></mfrac></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>2</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

p-0048So W_iis actually a normalized score to measure, in a particular partition, which probability is more dominant. If the confidence score is large, it is more likely that the image patch being analyzed includes a face with the pose associated with the corresponding classifier. Each weak classifier W_jiin detector D_jcontributes a score, and the scores are fused with a rule (e.g., the summation rule below) to produce a reliable confidence score:

p-0049<maths id="MATH-US-00003" num="00003"><math overflow="scroll"><mtable><mtr><mtd><mrow><mover><msub><mi>w</mi><mi>j</mi></msub><mi>_</mi></mover><mo>=</mo><mfrac><mrow><munderover><mo>∑</mo><mi>i</mi><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle></munderover><mo></mo><msub><mi>W</mi><mi>ji</mi></msub></mrow><mi>N</mi></mfrac></mrow></mtd><mtd><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

p-0050where N is the number of weak classifier used for pose evaluation. Thus, a pose of image patch I may be determined by:

p-0051<br />Pos(<i>I</i>)=arg max( <o><i>W</i>_j</o>).   (4)

p-0052By implementing only N weak classifiers (e.g., 6 in one example) a pose may be determined by utilizing a fraction of the weak classifiers of each trained detector (e.g., only the first 6 weak classifiers in one example) for pose estimation. Thus, it is not necessary to train an extra pose classifier and still relatively fast performance can be achieved. Accordingly, the coarse pose evaluator <b>80</b> is configured to implement a selected portion of the classifiers for each detector of a statistical based analysis scheme in order to obtain confidence scores used to achieve relatively good (albeit coarse) pose evaluation performance.

p-0053<figref idrefs="DRAWINGS">FIG. 7</figref> shows an example of coarse pose evaluation that may be accomplished using the coarse pose evaluator <b>80</b>. Notably, as shown in <figref idrefs="DRAWINGS">FIG. 7</figref>, in some embodiments convergence performance may be increased at least in part by enabling a reduction in the coverage range of the detectors. For example, sector A in <figref idrefs="DRAWINGS">FIG. 1</figref> covered from −45 to 45 degrees of in-plane rotation, which may require a relatively large amount of training data to enable reliable pose detection and face detection over such a range. Meanwhile, in an exemplary embodiment of the present invention, the coverage range may be reduced to −15 to 15 degrees of in-plane rotation because rotated faces can be calibrated by a structural method by the fine pose evaluator <b>82</b> (thus the rotation of the face indicated outside the −15 to 15 degrees coverage area to within the −15 to 15 degrees coverage area may be accomplished by the structural method). Thus, it is not necessary for trained detectors in accordance with embodiments of the present invention to have the capability to detect large rotations of faces. In practice, the coarse pose evaluator <b>80</b> may therefore implement less classifiers in each detector (e.g., N may be smaller).

p-0054The fine pose evaluator <b>82</b> may be configured to implement a structure based analysis scheme to provide fine pose information including a relatively accurate indication of the pose of any face detected in a particular image patch. The structure based analysis scheme may be one of the examples described above or any other structure based analysis scheme. Though some embodiments may estimate coarse pose information using the statistical scheme, utilizing the structure based scheme may enable refinement of the coarse pose information. In this regard, for example, if the face has a relatively large rotation, the face may not have a high likelihood of being correctly classified by using only the statistical based method employed by the coarse pose evaluator <b>80</b> (e.g., by classifiers of the detector associated with a corresponding pose), because for ensuring good convergence, only small pose variations are generally permitted in training data. Accordingly, embodiments of the present invention utilize the structure based analysis scheme to get fine pose information. In some embodiments, the face may be rotated to verticality to enable face detection. In some cases, the rotation to verticality may be accomplished by the candidate rotator <b>86</b> either before or after operation of the fine pose evaluator <b>82</b> in determining a relatively accurate pose to define pose information for the corresponding face candidate. By vertically aligning the face candidate, the structural method may be better enabled to analyze structural features of the image patch to determine accurate pose information.

p-0055The face detector <b>84</b> may be configured to detect face candidates in a vertically aligned image patch with a fine pose determination already made using the statistical based analysis scheme used by the coarse pose evaluator <b>80</b>. In some cases, the face detector <b>84</b> may use all or a portion of the remaining cascaded weak classifiers not initially used by the coarse pose evaluator <b>80</b>. However, in other cases, an unrelated set of detectors employing a statistical method may be employed. In operation, after candidate faces have been detected, the candidate rotator <b>86</b> may rotate the detected candidates to vertical after which the face detector <b>84</b> may more easily and quickly determine whether the candidates correspond to faces. In an exemplary embodiment, the apparatus of <figref idrefs="DRAWINGS">FIG. 6</figref> may be in communication with a media capturing module (e.g., a camera) to receive image data for use in analysis as described herein. However, in alternative embodiments, the apparatus of <figref idrefs="DRAWINGS">FIG. 6</figref> may receive images from a storage location (e.g., the memory device <b>76</b>) or another device.

p-0056As indicated above, in some embodiments, the coarse pose evaluator <b>80</b> and the face detector <b>84</b> may utilize Adaboost or another statistical based analysis scheme. However, Adaboost is merely one example of a specific implementation of the coarse pose evaluator <b>80</b> and the face detector <b>84</b> and other mechanisms may alternatively be employed. Adaboost (adaptive boost) is a machine learning algorithm that may be utilized in conjunction with other learning mechanisms to improve performance. In an exemplary embodiment, Adaboost may be used in conjunction with a structure based analysis scheme for pose evaluation. Then, after determining coarse pose information using selected classifiers from the detectors associated with Adaboost that have been trained using face datasets with relatively small pose variations, the structure based analysis scheme may refine the pose information to provide fine pose information and rotate the corresponding image patch to verticality. Other Adaboost classifiers may then be employed to complete face detection on the vertically rotated image.

p-0057The vertical rotation of the image by the structure based analysis scheme may make it unnecessary for training data with large rotation angles to be utilized by the statistical based analysis scheme. Thus, less weak classifiers may need to be trained in each detector and detectors may achieve convergence more rapidly. According to some embodiments in which coarse pose information is initially acquired using a statistical based analysis scheme, search scope of the structure based analysis scheme is refined for finding a precise rotation angle. Thus, based on the size of an input image patch, appropriate operators may be constructed to better estimate structure information to improve efficiency of the structure based analysis scheme. Moreover, some embodiments of the present invention fuse statistical and structure based analysis schemes for the determination of pose information to augment face detection.

p-0058Embodiments of the present invention may, in some cases, decrease time needed to accomplish pose estimation relatively accurately without appending extra computational load in face based applications. In this regard, a push-switch based face pose evaluator as described herein fuses both statistical and structure based analysis schemes for reliable pose estimation. Thus, for example, convergence performance in detector training may be improved, pose estimates may be quickly and accurately determined, and does so without requiring an extra pose classifier to train.

p-0059In some cases, the structure based analysis scheme employed by the fine pose evaluator <b>82</b> may utilize eye positioning for fine pose information determination. As such, fine pose information may not be determinable for certain face rotations where the eyes are not visible. In such situations, only coarse pose information may be provided.

p-0060Embodiments of the present invention may be utilized in connection with multi-view face detection and recognition and also in connection with human-computer interface (HCI) applications in order to provide a user interface for mobile phone operation, gaming, and carrying out tasks using computers.

p-0061<figref idrefs="DRAWINGS">FIG. 8</figref> is a flowchart of a system, method and program product according to some exemplary embodiments of the invention. It will be understood that each block or step of the flowchart, and combinations of blocks in the flowchart, can be implemented by various means, such as hardware, firmware, and/or software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory device of a mobile terminal or other apparatus employing embodiments of the present invention and executed by a processor in the mobile terminal or other apparatus. As will be appreciated, any such computer program instructions may be loaded onto a computer or other programmable apparatus (i.e., hardware) to produce a machine, such that the instructions which execute on the computer (e.g., via a processor) or other programmable apparatus create means for implementing the functions specified in the flowchart block(s) or step(s). These computer program instructions may also be stored in a computer-readable memory that can direct a computer (e.g., the processor or another computing device) or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block(s) or step(s). The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block(s) or step(s).

p-0062Accordingly, blocks or steps of the flowchart support combinations of means for performing the specified functions, combinations of steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that one or more blocks or steps of the flowchart, and combinations of blocks or steps in the flowchart, can be implemented by special purpose hardware-based computer systems which perform the specified functions or steps, or combinations of special purpose hardware and computer instructions.

p-0063In this regard, one embodiment of a method for providing pose evaluation for face detection as illustrated, for example, in <figref idrefs="DRAWINGS">FIG. 8</figref> may include utilizing a selected portion of classifiers in detectors (e.g., associated with a statistical based analysis scheme) to determine coarse pose information for a candidate face in an image at operation <b>100</b>. The method may further include determining fine pose information for the candidate face (e.g., by employing a structure based analysis scheme) based at least in part on the determined coarse pose information at operation <b>110</b> and employing another portion of the classifiers in the detectors (e.g., associated with the statistical based analysis scheme) to perform face detection based at least in part on the fine pose information to determine whether the candidate face corresponds to a face at operation <b>120</b>.

p-0064In some embodiments, the method may include further optional operations, an example of which is shown in dashed lines in <figref idrefs="DRAWINGS">FIG. 8</figref>. Optional operations may be performed in any order and/or in combination with each other in various alternative embodiments. As such, the method may further include an operation of rotating the image to verticality prior to performing the face detection at operation <b>115</b>.

p-0065In some embodiments, certain ones of the operations above may be modified or further amplified as described below. It should be appreciated that each of the modifications or amplifications below may be included with the operations above either alone or in combination with any others among the features described herein. In this regard, for example, determining fine pose information for the candidate face by employing the structure based analysis scheme comprises utilizing a face template. In some cases, the statistical based analysis scheme may be Adaboost. In an exemplary embodiment, utilizing the selected portion of classifiers may include utilizing N number of initial classifiers in the detectors of the statistical based analysis scheme. Alternatively or additionally, employing another portion of the statistical based analysis scheme may include utilizing at least a portion of remaining classifiers not used for determining the coarse pose information.

p-0066In an exemplary embodiment, an apparatus for performing the method of <figref idrefs="DRAWINGS">FIG. 8</figref> above may comprise a processor (e.g., the processor <b>70</b>) configured to perform some or each of the operations (<b>100</b>-<b>120</b>) described above. The processor may, for example, be configured to perform the operations (<b>100</b>-<b>120</b>) by performing hardware implemented logical functions, executing stored instructions, or executing algorithms for performing each of the operations. Alternatively, the apparatus may comprise means for performing each of the operations described above. In this regard, according to an example embodiment, examples of means for performing operations <b>100</b>-<b>120</b> may comprise, for example, the processor <b>70</b>, coarse pose evaluator <b>80</b>, the fine pose evaluator <b>82</b>, the face detector <b>84</b>, the candidate rotator <b>86</b>, and/or an algorithm executed by the processor <b>70</b> for processing information as described above.

p-0067Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe exemplary embodiments in the context of certain exemplary combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and/or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.