Information processing apparatus and locus data recognition method

BACKGROUND OF THE INVENTION

p-00021. Field of the Invention

p-0003The present invention relates to an information processing apparatus and a locus data recognition method.

p-00042. Description of the Related Art

p-0005A conventional device having an input unit, such as a touch panel, extracts locus data of one character input in an input frame (field) or by various character separation methods. Furthermore, such a conventional device executes character recognition in a unit of one character by using the extracted locus data as input data in the character recognition processing.

p-0006In this case, it is assumed that the locus data does not include locus data other than the one included in the recognition target character. In other words, in character recognition processing, only recognition of a character is executed. Furthermore, in gesture recognition, only recognition of a gesture is executed during gesture recognition processing.

p-0007However, a pen-input graphic editing system discussed in Japanese Patent Application Laid-Open No. 06-208654 does not consider that an editing request and a character are input at the same time. Accordingly, the conventional method separately determines whether specific input data is an editing request or a character.

p-0008In the pen-input graphic editing system discussed in Japanese Patent Application Laid-Open No. 06-208654, if the size and the position of input data is not clear, it is not certain whether the input data is determined to be an editing request or a character. In addition, in the pen-input graphic editing system discussed in the Laid-Open No. 06-208654, even if an editing request is correctly recognized, a subsequent character is likely to be misrecognized.

p-0009In addition, a data processing apparatus discussed in Japanese Patent Application Laid-Open No. 08-305821 can extract only a predetermined locus. Accordingly, the data processing apparatus discussed in the Laid-Open No. 08-305821 can correctly recognize input data only when a gesture and a character are input in a predetermined input order. But if the order of inputting a gesture or a character is not determined or used, the irregularly input gesture or character cannot be effectively recognized.

p-0010In addition, in an image recognition apparatus disclosed in U.S. Pat. No. 5,781,663, a gesture recognition unit, a character recognition unit, and a graphic recognition unit recognize input loci and merely selects a best result from a plurality of recognition results.

p-0011Therefore, the image recognition apparatus disclosed in U.S. Pat. No. 5,781,663 does not consider a method for separating a gesture and a character which are mixed in loci.

p-0012An optimal result can be output by executing recognition processing on all locus combinations by using a combination of conventional recognition units, such as a gesture recognition unit and a character recognition unit. However, in this case, it is necessary to execute recognition processing on a large number of combinations. Accordingly, the recognition cannot be completed in a practical calculation time.

p-0013In addition, in the above-described conventional method, an incorrect combination may match a dictionary pattern. In this case, misrecognition may occur. Accordingly, the above-described conventional method is not practical. Furthermore, in the above-described conventional method, a separated recognition result may be obtained by a combination in which a specific locus is included in both separated data.

p-0014If data is input as a sentence, the above-described conventional method can separate each character. However, if a gesture and a character are mixed and included in input data, a conventional method that uses a conventional separation algorithm is not appropriate and cannot separate mixed gestures and characters.

p-0015For example, if a control command parameter is written within a gesture locus having a circular shape (“∘”), the above-described conventional method cannot appropriately recognize the parameter by recognizing that the gesture locus and the parameter constitute one character.

p-0016In addition, when a control command parameter is written within a frame (field) of a circumscribed rectangle of the input gesture locus, if the input data is separated into a locus of the gesture and a part of the input character, and a locus of the other characters, the above-described conventional method cannot recognize both locus data.

SUMMARY OF THE INVENTION

p-0017The present invention is directed to a method for correctly separating a gesture locus and a character locus from mixed loci including a gesture locus and a character locus within a practical processing time.

p-0018According to an aspect of the present invention, an information processing apparatus includes a gesture locus data recognition unit configured to execute processing for recognizing gesture locus data included in locus data according to characteristic data of the locus data and gesture characteristic shape data included in gesture dictionary data and output a result of the processing, a separation unit configured to separate gesture locus data and locus data other than the gesture locus data from the locus data according to the result of the recognition by the gesture locus data recognition unit, and a character locus data recognition unit configured to execute processing for recognizing locus data of a character included in the locus data other than the gesture locus data according to the characteristic data of the locus data other than the gesture locus data which is separated by the separation unit and the locus characteristic data of a character included in a character dictionary data, and output a result of the processing.

p-0019An exemplary embodiment having the above-described configuration can correctly separate a gesture locus and a character locus from loci in which a gesture locus and a character locus are mixed, within a practical processing time.

p-0020In addition, the present invention can be implemented by a locus data recognition method, a program for the locus data recognition method, and a storage medium storing the program.

p-0021According to yet another aspect of the present invention, an information processing apparatus is capable of increasing a character locus separation ratio and a character locus recognition ratio by restricting an input character locus according to a relationship between a gesture locus and a character locus accompanying the gesture locus. In addition, according to yet another aspect of the present invention, an information processing apparatus can separate a gesture locus and a character locus from loci having the same shape according to a setting of a condition for a relative positional relationship.

p-0022Further features and aspects of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

p-0023The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the invention and, together with the description, serve to explain the principles of the present invention.

p-0024<figref idrefs="DRAWINGS">FIG. 1</figref> illustrates an exemplary system configuration according to an exemplary embodiment of the present invention.

p-0025<figref idrefs="DRAWINGS">FIG. 2</figref> illustrates an exemplary hardware configuration of an information terminal <b>1</b>.

p-0026<figref idrefs="DRAWINGS">FIG. 3</figref> illustrates an exemplary functional configuration of the information terminal <b>1</b> (case <b>1</b>).

p-0027<figref idrefs="DRAWINGS">FIG. 4</figref> illustrates an example structure of data of a gesture dictionary (case <b>1</b>).

p-0028<figref idrefs="DRAWINGS">FIG. 5</figref> illustrates an example of a locus of input data (case <b>1</b>).

p-0029<figref idrefs="DRAWINGS">FIG. 6</figref> illustrates an example of locus data (case <b>1</b>).

p-0030<figref idrefs="DRAWINGS">FIG. 7</figref> is a flow chart illustrating an example of gesture recognition processing.

p-0031<figref idrefs="DRAWINGS">FIG. 8</figref> is a flow chart illustrating an example of locus separation processing (case <b>1</b>).

p-0032<figref idrefs="DRAWINGS">FIG. 9</figref> is a flow chart illustrating an example of separation processing.

p-0033<figref idrefs="DRAWINGS">FIG. 10</figref> is a flow chart illustrating an example of character recognition processing.

p-0034<figref idrefs="DRAWINGS">FIG. 11</figref> illustrates an example of a screen that displays a result of gesture recognition (case <b>1</b>).

p-0035<figref idrefs="DRAWINGS">FIG. 12</figref> illustrates an exemplary functional configuration of an information terminal <b>1</b> (case <b>2</b>).

p-0036<figref idrefs="DRAWINGS">FIG. 13</figref> illustrates an example of a structure of data of a gesture dictionary (case <b>2</b>).

p-0037<figref idrefs="DRAWINGS">FIG. 14</figref> illustrates an example of a locus of input data (case <b>2</b>).

p-0038<figref idrefs="DRAWINGS">FIG. 15</figref> illustrates an example of locus data (case <b>2</b>).

p-0039<figref idrefs="DRAWINGS">FIG. 16</figref> is a flow chart illustrating an example of positional condition determination processing.

p-0040<figref idrefs="DRAWINGS">FIG. 17</figref> is a flow chart illustrating an example of locus separation processing (case <b>2</b>).

p-0041<figref idrefs="DRAWINGS">FIG. 18</figref> illustrates an example of a screen displayed as a result of gesture recognition (case <b>2</b>).

DESCRIPTION OF THE EMBODIMENTS

p-0042Various exemplary embodiments, features, and aspects of the invention will be described in detail below with reference to the drawings.

p-0043Now, a first exemplary embodiment of the present invention will be described below with reference to drawings.

p-0044<figref idrefs="DRAWINGS">FIG. 1</figref> illustrates an exemplary system configuration of a system according to the present exemplary embodiment. Referring to <figref idrefs="DRAWINGS">FIG. 1</figref>, software for displaying a still image recorded on a recording medium is installed on an information terminal <b>1</b>, which is an example of an information processing apparatus.

p-0045When a user inputs a hand-drawn locus on a liquid crystal display (LCD) device <b>7</b> of the information terminal <b>1</b> by using an instruction input device <b>6</b>, a locus is displayed on an image displayed on the LCD <b>7</b> of the information terminal <b>1</b>. The instruction input device <b>6</b> will be described below. A pen <b>2</b> is a pen used for a general transparent resistance film digitizer.

p-0046<figref idrefs="DRAWINGS">FIG. 2</figref> illustrates an exemplary hardware configuration of the information terminal <b>1</b>. Referring to <figref idrefs="DRAWINGS">FIG. 2</figref>, a universal serial bus (USB) interface <b>3</b> is an interface connected with other apparatuses. Data communication can be executed between the information terminal <b>1</b> and other apparatuses via the USB interface <b>3</b>.

p-0047When a user presses a switch provided on the device, a key switch <b>4</b> detects the pressing of the switch and notifies the result of the detection to a central processing unit (CPU) <b>8</b>. Data communication with other apparatus on a network can be executed via a local area network (LAN) interface (I/F) <b>5</b>.

p-0048The instruction input device <b>6</b> includes a transparent resistance film digitizer and is a device for inputting positional coordinates. When the user presses the surface of the instruction input device <b>6</b> with the pen <b>2</b> (<figref idrefs="DRAWINGS">FIG. 1</figref>), the instruction input device <b>6</b> transmits X-axis and Y-axis positional coordinate data of the pressed position to the CPU <b>8</b> via a system bus <b>12</b>.

p-0049The LCD device <b>7</b> includes an LCD device, an LCD control circuit, and a display memory. The LCD device <b>7</b> is connected to the CPU <b>8</b> via the system bus <b>12</b>. The LCD device <b>7</b> displays a locus and a character according to an instruction from the CPU <b>8</b>.

p-0050The CPU <b>8</b> is connected to a random access memory (RAM) <b>9</b>, a read-only memory (ROM) <b>10</b>, and the LCD device <b>7</b> via the system bus <b>12</b>. The CPU <b>8</b> executes recognition processing according to a program stored on the ROM <b>10</b>. The program will be described below.

p-0051The RAM <b>9</b> functions as a work area for the CPU <b>8</b>. the ROM <b>10</b> stores a program of the recognition processing according to the present exemplary embodiment and a dictionary, which will be described below.

p-0052The memory card <b>11</b> stores image information, for example. The system bus <b>12</b> is a data bus used for data communication among the CPU <b>8</b>, the RAM <b>9</b>, the ROM <b>10</b>, and other components of the information terminal <b>1</b>.

p-0053<figref idrefs="DRAWINGS">FIG. 3</figref> illustrates an exemplary functional configuration of the information terminal <b>1</b> (case <b>1</b>).

p-0054Each of components of the information terminal <b>1</b>, such as a gesture recognition unit <b>14</b>, a separation unit <b>16</b>, a recognition condition setting unit <b>17</b>, a character separation unit <b>18</b>, a character recognition unit <b>19</b>, and a command processing unit <b>21</b>, can be implemented by the CPU <b>8</b> by executing a program stored on the ROM <b>10</b>.

p-0055As described above, the instruction input device <b>6</b> detects positional coordinate data, such as a locus written with the pen <b>2</b>, and inputs the detected positional coordinate data to the CPU <b>8</b> as a locus. The input locus is stored on a specific area (a locus storage unit <b>13</b>, which will be described in detail below) of the RAM <b>9</b> under control of the CPU <b>8</b>.

p-0056The locus storage unit <b>13</b> records positional coordinate data rows of the input locus in a unit of a locus. The gesture recognition unit <b>14</b> acquires locus data (positional coordinate data rows of the locus) from the locus storage unit <b>13</b>.

p-0057The gesture recognition unit <b>14</b> extracts characteristic data of the acquired locus from the positional coordinate data row of the acquired locus. In addition, the gesture recognition unit <b>14</b> executes matching processing for comparing the extracted characteristic data and data of the gesture dictionary <b>15</b> (for example, a gesture characteristic shape).

p-0058The gesture recognition unit <b>14</b> outputs a recognition result, such as a gesture code (a code indicating the shape of the gesture) of a gesture characteristic shape most similar to the extracted characteristic data (gesture characteristic shape data) or a command type.

p-0059The gesture dictionary <b>15</b> is dictionary data including a gesture code, a gesture characteristic shape (locus characteristic shape), and a gesture parameter condition of a registered gesture. The separation unit <b>16</b> separates a gesture locus (gesture locus data) from the other loci (the other locus data). More specifically, if a gesture locus and a character locus are input to the separation unit <b>16</b> at the same time, the separation unit <b>16</b> separates the loci from each other according to a result of recognition by the gesture recognition unit <b>14</b>.

p-0060The recognition condition setting unit <b>17</b> sets the number of characters to the character separation unit <b>18</b> according to the gesture parameter condition described in the gesture dictionary <b>15</b>. The character separation unit <b>18</b> separates character loci, which are transmitted from the separation unit <b>16</b>, into locus data of one character one by one up to the number of characters transmitted from the recognition condition setting unit <b>17</b>.

p-0061The character recognition unit <b>19</b> executes matching between the characteristic data of the locus data of one character, which is transmitted from the character separation unit <b>18</b>, and locus characteristic data of a character stored in the character recognition dictionary <b>20</b>, which is an example of a character dictionary data.

p-0062The character recognition unit <b>19</b> outputs the character code of the character of the locus data whose matching degree is the highest, as a recognition result.

p-0063The command processing unit <b>21</b> executes processing related to the command (the display and setting of the printing size “A4”, for example) according to the command type transmitted from the gesture recognition unit <b>14</b> and the gesture command parameter indicated by the character code transmitted from the character recognition unit <b>19</b>.

p-0064In addition, triggered by an instruction to display the locus from the gesture recognition unit <b>14</b>, the command processing unit <b>21</b> displays a locus or a recognition result on the LCD device <b>7</b>.

p-0065<figref idrefs="DRAWINGS">FIG. 4</figref> illustrates an example of a structure of data of a gesture dictionary (case <b>1</b>). Referring to <figref idrefs="DRAWINGS">FIG. 4</figref>, one piece of gesture dictionary data includes a gesture code, a gesture characteristic shape (characteristic data of the gesture shape), a gesture parameter condition, a gesture separation condition, and a command type.

p-0066As a registered gesture characteristic shape, data drawn in a unique way having a shape different from that of one stroke constituting a normal character, such as an alphanumeric, a kana character (a Japanese phonetic syllabary), or a kanji character (a Chinese character), is registered.

p-0067For example, a gesture code “0002”, which is a gesture code for executing printing, includes a circular locus and a numeral for designating the number of prints. A gesture code “0002” is written starting from a point different from the starting point of writing a normal numeral “0”.

p-0068If the user writes the gesture “0002” and a numeral “3” with the pen <b>2</b>, then the information terminal <b>1</b> recognizes that an instruction to “print on three sheets” has been given. Furthermore, if the user writes the gesture “0002” and a numeral “11” with the pen <b>2</b>, then the information terminal <b>1</b> recognizes that an instruction to “print on eleven sheets” has been given.

p-0069In the present exemplary embodiment, only gesture loci are registered in the gesture dictionary. However, it is also useful if a locus constituting a gesture locus and a character, e.g., “/−3”, is registered in the gesture dictionary.

p-0070With the above-described configuration, the information terminal <b>1</b> sets a character locus code, which indicates a character locus, as a recognition result.

p-0071<figref idrefs="DRAWINGS">FIG. 5</figref> illustrates an example of a locus of input data (case <b>1</b>). Referring to <figref idrefs="DRAWINGS">FIG. 5</figref>, still images <b>23</b> and <b>24</b>, which are displayed as a reduced rectangle in <figref idrefs="DRAWINGS">FIG. 5</figref>, are displayed on a display input screen <b>22</b>.

p-0072<figref idrefs="DRAWINGS">FIG. 6</figref> illustrates an example of locus data (case <b>1</b>). Referring to <figref idrefs="DRAWINGS">FIG. 6</figref>, the user inputs a gesture and a character starting from a first stroke starting point <b>25</b> and via a first stroke second point <b>26</b> and a first stroke end point <b>30</b>. In addition, the user inputs the gesture and the character by performing a second stroke, which starts from a starting point <b>31</b>, a third stroke, which starts from a starting point <b>32</b>, a fourth stroke, which starts from a starting point <b>33</b>, a fifth stroke, which starts from a starting point <b>34</b>, and a sixth stroke, which starts from a starting point <b>35</b>.

p-0073<figref idrefs="DRAWINGS">FIG. 7</figref> is a flow chart illustrating an example of gesture recognition processing (processing for recognizing gesture locus data). When a gesture locus and a character locus, which is a gesture command parameter, have been completely input, the processing in the flowchart of <figref idrefs="DRAWINGS">FIG. 7</figref> starts. A processing method for executing recognition every time a locus is input is also feasible. However, for sake of simplicity, in the present exemplary embodiment, the processing in the flowchart of <figref idrefs="DRAWINGS">FIG. 7</figref> starts after all loci are completely input.

p-0074If a predetermined length of time has passed or if the user presses an area outside the input area, after the user has input loci, the processing described below starts. More specifically, when the loci <b>25</b> through <b>35</b> illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref> are input, the processing illustrated in <figref idrefs="DRAWINGS">FIG. 7</figref> starts.

p-0075After the gesture recognition processing has started, the gesture recognition unit <b>14</b> secures a work area. Furthermore, the CPU <b>8</b> initializes the work area and loads positional coordinate data rows of all the input loci thereon.

p-0076The input loci are normalized to a coordinate range having the same value as that at the time of generating the dictionary. For example, the input loci are normalized to the range of 0 to 100.

p-0077In step S<b>72</b>, the gesture recognition unit <b>14</b> reads data of one locus (one stroke). In the example illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref>, the gesture recognition unit <b>14</b> reads the locus data <b>25</b> through <b>30</b> of the first stroke into a storage buffer.

p-0078In step S<b>73</b>, the gesture recognition unit <b>14</b> executes locus characteristic extraction processing. For example, the gesture recognition unit <b>14</b> divides a locus into ten equal parts and outputs positional coordinate points thereof as characteristic points.

p-0079In step S<b>74</b>, the gesture recognition unit <b>14</b> reads the gesture characteristic shape from the gesture dictionary <b>15</b>. More specifically, at first, the gesture recognition unit <b>14</b> reads positional coordinates of the gesture characteristic shape of the gesture “0001”. Then, the gesture recognition unit <b>14</b> reads the gesture characteristic shapes of the gestures “0002”, “0003”, and “0004”.

p-0080In step S<b>75</b>, the gesture recognition unit <b>14</b> executes processing for calculating a matching degree of characteristic data of the locus and the data of the gesture characteristic shape stored in the gesture dictionary <b>15</b>.

p-0081More specifically, If all of the positional coordinate points, which have been obtained by equally dividing the normalized locus into ten parts, and positional coordinate points of the gesture characteristic shape registered in the gesture dictionary <b>15</b> are the same, then the gesture recognition unit <b>14</b> determines that the matching degree is 100%.

p-0082For example, the gesture recognition unit <b>14</b> determines that the points <b>25</b> through <b>30</b> of the first stroke of the locus illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref> has a matching degree of 90% with the gesture code “0003” registered in the gesture dictionary <b>15</b>.

p-0083The gesture recognition unit <b>14</b> determines that no gesture characteristic shape corresponding to the subsequent locus <b>31</b> has been registered in the gesture dictionary <b>15</b>.

p-0084In step S<b>76</b>, the gesture recognition unit <b>14</b> stores the recognition result of the locus. More specifically, the gesture recognition unit <b>14</b> stores recognition results of the first through the last locus in a recognition result storage buffer.

p-0085In step S<b>77</b>, the gesture recognition unit <b>14</b> determines whether all loci have been completely evaluated. If it is determined that all loci have been completely evaluated (YES in step S<b>77</b>), then the processing advances to step S<b>78</b>. On the other hand, if it is determined that any unprocessed locus remains (NO in step S<b>77</b>), then the processing returns to step S<b>72</b> to process a subsequent locus.

p-0086In step S<b>78</b>, the gesture recognition unit <b>14</b> stores the gesture code whose degree of matching has been determined to be high during the evaluation of the locus. In the example illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref>, the gesture recognition unit <b>14</b> stores the gesture code “0003”, whose degree of matching with the locus of the first stroke has been determined to be 90% Then, the gesture recognition unit <b>14</b> ends the gesture recognition processing.

p-0087By executing processing illustrated in <figref idrefs="DRAWINGS">FIG. 7</figref>, a first stroke (locus), of the input loci, is usually recognized as an input gesture and a result of the gesture recognition is stored. For the other strokes, a gesture recognition result, such as “no corresponding gesture extracted” or “matching degree low” is output.

p-0088If the user does not input a correct gesture, a gesture recognition result “no corresponding gesture extracted” may be output as the gesture recognition result of all the loci. In this case, it is not useful to execute the subsequent processing. Accordingly, the information terminal <b>1</b> suspends the processing.

p-0089In the present exemplary embodiment, it is supposed that each gesture (gesture characteristic shape) registered in the gesture dictionary <b>15</b> includes only one stroke. However, the present exemplary embodiment is not limited to this. More specifically, it is also useful if a gesture characteristic shape including two strokes and a gesture characteristic shape including one stroke are mixed and registered in the gesture dictionary <b>15</b>.

p-0090In this case, the following is executed as the processing in the flow chart of <figref idrefs="DRAWINGS">FIG. 7</figref>.

p-0091At first, the gesture recognition unit <b>14</b> executes the recognition processing on all the loci by using the data of one stroke registered in the gesture dictionary and stores the locus whose degree of matching is the highest and the degree of matching thereof.

p-0092Then, the gesture recognition unit <b>14</b> executes the recognition processing on a combination of two input loci by using the data of two strokes registered in the gesture dictionary. Furthermore, the gesture recognition unit <b>14</b> stores the combination of two loci whose degree of matching is the highest and the matching degree thereof.

p-0093More specifically, the gesture recognition unit <b>14</b> compares the matching degree of the two loci and the matching degree of the one locus and uses the gesture code whose matching degree is higher as the gesture recognition result.

p-0094In the above-described manner, the information terminal <b>1</b> can separate two strokes or one stroke from the locus as a gesture locus (gesture locus data) according to the recognition result and the other loci (data other than the gesture locus data) as a character locus by executing locus separation processing illustrated in <figref idrefs="DRAWINGS">FIG. 8</figref>. If a gesture including three strokes, the above-described method can be applied to the gesture including three strokes.

p-0095As for a combination of loci matching with the two-stroke gesture dictionary data, if data of a locus including first through fifth strokes has been input, loci input in the vicinity of the first and the second strokes (or the second and the third strokes, the third and the fourth strokes, or the fourth and the fifth strokes) or loci at close positions in the order of input are used as a candidate of a combination.

p-0096Now, locus separation processing will be described in detail below. <figref idrefs="DRAWINGS">FIG. 8</figref> is a flow chart illustrating an example of the locus separation processing (case <b>1</b>). When the locus separation processing starts, the separation unit <b>16</b> secures a work area and initializes the work area.

p-0097Referring to <figref idrefs="DRAWINGS">FIG. 8</figref>, in step S<b>82</b>, the separation unit <b>16</b> reads a recognition result of one locus. More specifically, the separation unit <b>16</b> reads the recognition result illustrated in <figref idrefs="DRAWINGS">FIG. 7</figref>.

p-0098For example, if the recognition result of the first locus illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref> indicates the matching degree of 90% with the gesture code “0003”, then the separation unit <b>16</b> reads the same as the recognition result.

p-0099In step S<b>83</b>, the separation unit <b>16</b> determines whether the evaluation value (the degree of matching) of the recognition result is equal to or less than a gesture separation condition. If it is determined that the evaluation value of the recognition result is equal to or less than the gesture separation condition (YES in step S<b>83</b>), then the processing advances to step S<b>84</b>. On the other hand, if it is determined that the evaluation value of the recognition result is greater than the gesture separation condition (NO in step S<b>83</b>), then the processing returns to step S<b>82</b>.

p-0100More specifically, if the recognition result of the locus indicates the degree of matching of 90% with gesture code “0003”, since the gesture separation condition of the gesture code “0003” is the matching degree of 80% or more, the separation unit <b>16</b> returns to step S<b>82</b>. In other words, in this case, the separation unit <b>16</b> does not store the locus as one of the character loci.

p-0101If the recognition result indicates “no corresponding gesture code extracted”, then the separation unit <b>16</b> separates the loci as character loci.

p-0102In step S<b>84</b>, the separation unit <b>16</b> stores the locus read in step S<b>82</b> in the storage buffer as the character locus. If it is determined that data of all the character loci has been stored, then the separation unit <b>16</b> transmits the character loci to the character separation unit <b>18</b>.

p-0103In step S<b>85</b>, the separation unit <b>16</b> determines whether all the loci have been completely processed. If it is determined that all the loci have been completely processed (YES in step S<b>85</b>), then the processing illustrated in <figref idrefs="DRAWINGS">FIG. 8</figref> ends. On the other hand, if it is determined that all the loci have not been completely processed yet, then the processing returns to step S<b>82</b> to read data of a subsequent locus.

p-0104Now, character locus separation processing will be described in detail below.

p-0105<figref idrefs="DRAWINGS">FIG. 9</figref> is a flow chart illustrating an example of separation processing. The separation processing described below is, to be brief, processing for separating positional coordinate data of a character locus including a plurality of characters into positional coordinate data of each character. When the character locus separation processing starts, the character separation unit <b>18</b> secures a work area and initializes the work area.

p-0106Referring to <figref idrefs="DRAWINGS">FIG. 9</figref>, in step S<b>92</b>, the recognition condition setting unit <b>17</b> reads a gesture parameter condition, which is an example of conditional information, from the gesture dictionary. Then, the recognition condition setting unit <b>17</b> sets the read gesture parameter condition on the character separation unit <b>18</b>.

p-0107The recognition condition setting unit <b>17</b> reads the gesture parameter condition of the corresponding gesture code from the gesture dictionary (<figref idrefs="DRAWINGS">FIG. 4</figref>) and sets the same on the character separation unit <b>18</b>.

p-0108For example, if the gesture input by the user is a printing gesture, the user can designate the number of prints at the same time in inputting the printing gesture. In the present exemplary embodiment, the number of prints up to ninety-nine sheets can be designated. Accordingly, numerals of up to two digits (i.e., a numeral from 1 to 99) are set as the gesture parameter condition.

p-0109In the input example illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref>, the printing size is designated by the gesture code “0003”. Accordingly, the gesture parameter condition is “two-digit alphanumeric”, such as “A4”, “A3”, or “B5”. However, the present exemplary embodiment is not limited to this. More specifically, since printing on a card or a letter is conceivable, it is also useful if the gesture parameter condition includes a two-digit alphanumeric or a kanji character.

p-0110In step S<b>93</b>, the character separation unit <b>18</b> reads the character loci stored by the separation unit <b>16</b> in the storage buffer in the processing illustrated in <figref idrefs="DRAWINGS">FIG. 8</figref>.

p-0111In step S<b>94</b>, the character separation unit <b>18</b> generates a virtual character frame for one character according to the number of characters in the read gesture parameter condition and the circumscribed frame of the loci. More specifically, if the gesture code is “0003”, if the gesture parameter condition is “two-digit alphanumeric”, and if the character locus is “A4”, then the character separation unit <b>18</b> generates two virtual character frames “□ □”.

p-0112In step S<b>95</b>, the character separation unit <b>18</b> reads one locus. More specifically, the character separation unit <b>18</b> loads a positional coordinate data row of one locus on the work area.

p-0113In step S<b>96</b>, the character separation unit <b>18</b> determines whether all the loci have been completely processed. If it is determined that all the loci have been completely processed (YES in step S<b>96</b>), then the processing advances to step S<b>910</b>. On the other hand, if it is determined that all the loci have not been completely processed yet (NO in step S<b>96</b>), then the processing advances to step S<b>97</b>.

p-0114In step S<b>97</b>, the character separation unit <b>18</b> determines whether the locus read instep S<b>95</b> exists within the virtual character frame currently used. More specifically, the character separation unit <b>18</b> executes the determination serially on the virtual character frame for the first character, the virtual character frame for the second character, and so on.

p-0115If it is determined that the locus read in step S<b>95</b> exists within the virtual character frame currently used (YES in step S<b>97</b>), then the processing advances to step S<b>99</b>. On the other hand, if it is determined that the locus read in step S<b>95</b> does not exist within the virtual character frame currently used (NO in step S<b>97</b>), then the processing advances to step S<b>98</b>.

p-0116In the determination in step S<b>97</b>, at first, the character separation unit <b>18</b> separates (extracts) the character locus input in the left frame of the virtual character frames “□ □” as the first character. Then, the character separation unit <b>18</b> separates (extracts) the character locus input in the right frame of the virtual character frames “□ □” as the second character.

p-0117In step S<b>98</b>, the character separation unit <b>18</b> prepares for separating data input in a subsequent virtual character frame. More specifically, the character separation unit <b>18</b> moves the positional data of the virtual character frame so that the data is put into the right frame from the left frame and decrements the counter for the locus.

p-0118In step S<b>99</b>, the character separation unit <b>18</b> stores the locus data of N characters (N is an integer) obtained as a result of the character separation.

p-0119In step S<b>910</b>, the character separation unit <b>18</b> transmits the data of the locus of the character separated and stored character by character in step S<b>99</b>.

p-0120Now, an example of the character recognition processing will be described in detail below.

p-0121<figref idrefs="DRAWINGS">FIG. 10</figref> is a flow chart illustrating an example of character recognition processing (processing for recognizing character locus data). When the character recognition processing starts, the character recognition unit <b>19</b> secures a work area and initializes the work area.

p-0122Referring to <figref idrefs="DRAWINGS">FIG. 10</figref>, in step S<b>102</b>, the recognition condition setting unit <b>17</b> reads the character type of the parameter from the gesture dictionary and sets the read parameter character type on the character recognition unit <b>19</b>. More specifically, in the input example illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref>, because the gesture code is “0003”, the recognition condition setting unit <b>17</b> reads “alphanumeric” as the character type and sets the read character type on the character recognition unit <b>19</b>.

p-0123In step S<b>103</b>, the character recognition unit <b>19</b> executes a setting of the character type for a character recognition engine. In the example illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref>, the character recognition unit <b>19</b> sets “alphanumeric” on the character recognition engine as the character type. By executing the above-described processing, the character recognition dictionary includes only alphanumeric.

p-0124In step S<b>104</b>, the character recognition unit <b>19</b> loads the locus for one character on the work area. In step S<b>105</b>, the character recognition unit <b>19</b> extracts the characteristic data of the locus data. More specifically, the character recognition unit <b>19</b> equally divides one locus into ten parts and converts the equally divided loci into vector data in eight directions.

p-0125In step S<b>106</b>, the character recognition unit <b>19</b> reads the dictionary data of the set character type from the character recognition dictionary. For example, if “alphanumeric” is set as the character type, the character recognition unit <b>19</b> reads only the alphanumeric dictionary data from the character recognition dictionary.

p-0126In step S<b>107</b>, the character recognition unit <b>19</b> executes matching processing between the characteristic data of the locus data and the locus characteristic data of the character included in the dictionary data of the character recognition dictionary. Then, the character recognition unit <b>19</b> sets the character code of the dictionary locus characteristic data whose matching degree is the highest as the recognition result.

p-0127In step S<b>108</b>, the character recognition unit <b>19</b> determines whether all the character loci have been completely processed. If it is determined that all the character loci have been completely processed (YES in step S<b>108</b>), then the processing advances to step S<b>109</b>. On the other hand, if it is determined that all the character loci have not been completely processed yet (NO in step S<b>108</b>), then the processing returns to step S<b>104</b>.

p-0128In step S<b>109</b>, the character recognition unit <b>19</b> executes processing for outputting the character code of the recognized character to the command processing unit <b>21</b> as the recognition result. <figref idrefs="DRAWINGS">FIG. 11</figref> illustrates an example of a screen that displays a result of gesture recognition (case <b>1</b>).

p-0129When the locus illustrated in <figref idrefs="DRAWINGS">FIG. 5</figref> is input by the user, if the gesture of the command to set the printing size is recognized by the gesture recognition unit <b>14</b>, and if the printing size “A4” is recognized by the character recognition unit <b>19</b>, then the command processing unit <b>21</b> displays the screen illustrated in <figref idrefs="DRAWINGS">FIG. 11</figref> on the LCD device <b>7</b>.

p-0130The screen displayed in the example illustrated in <figref idrefs="DRAWINGS">FIG. 11</figref> is a mere example. More specifically, it is also useful if font characters display the printing size “A4” and an icon indicating the locus or the command displays the gesture. In addition, it is also useful if the command processing unit <b>21</b> displays another candidate (changes the printing size “A4” to “B4” for display) when the user presses each character or an icon with the pen <b>2</b>.

p-0131The information terminal <b>1</b> according to the present exemplary embodiment and having the above-described configuration executes the above-described processing. As a result, the present exemplary embodiment can appropriately separate, recognize, and process a hand-drawn gesture locus and a character locus input on a free flat plane area including no frame, in a free order.

p-0132Furthermore, by executing the above-described processing, the information terminal <b>1</b> can appropriately separate and recognize the gesture locus and the character locus regardless of the order of inputting the same (whether the gesture locus and the character locus are input in this order, input in the order of the character locus and the gesture locus, or input in the order of character locus, gesture locus, and character locus).

p-0133Now, a second exemplary embodiment will be described in detail below. In the above-described first exemplary embodiment, because a gesture has a specific shape and is input by a particular way of inputting, the information terminal <b>1</b> separates the gesture locus and the character locus, which is the gesture command parameter, according to a result of the gesture recognition. However, according to the type of units and devices mounted on the information terminal <b>1</b>, an exemplary embodiment is conceivable that uses a locus having the same shape as a normally written character, as a gesture.

p-0134The present exemplary embodiment includes a positional condition determination unit, which is configured to determine a condition of the position between the gesture locus and other loci, in addition to the first exemplary embodiment.

p-0135<figref idrefs="DRAWINGS">FIG. 12</figref> illustrates an exemplary functional configuration of an information terminal <b>1</b> (case <b>2</b>). In the following description, points different from the first exemplary embodiment will be primarily described in detail below.

p-0136Referring to <figref idrefs="DRAWINGS">FIG. 12</figref>, a positional condition determination unit <b>37</b> determines whether a corresponding gesture locus satisfies a positional condition recorded in the gesture dictionary.

p-0137<figref idrefs="DRAWINGS">FIG. 13</figref> illustrates an example of a structure of data of a gesture dictionary (case <b>2</b>). One piece of gesture dictionary data includes a gesture code, a gesture characteristic shape (characteristic data of the gesture shape), a gesture parameter condition, a gesture separation condition, a command type, and data of a positional condition.

p-0138In the present exemplary embodiment, a “positional condition” refers to a positional condition between a position of a gesture locus and the position of a character locus, which is a gesture command parameter input at the same time in inputting the gesture locus.

p-0139If a gesture code is “0002”, the positional condition includes a positional condition for inputting a parameter character locus within the gesture locus.

p-0140If a character such as “∘∘”, is registered in the dictionary, it is useful if a detailed condition is stored such that a locus of two strokes or more is to be included or if a detailed condition as to the shape of the locus is stored.

p-0141<figref idrefs="DRAWINGS">FIG. 14</figref> illustrates an example of a locus of input data (case <b>2</b>). Referring to <figref idrefs="DRAWINGS">FIG. 14</figref>, the display input screen <b>22</b> displays a first stroke locus <b>38</b>, a second stroke locus <b>39</b>, and a third stroke locus <b>40</b>.

p-0142<figref idrefs="DRAWINGS">FIG. 15</figref> illustrates an example of locus data (case <b>2</b>). <figref idrefs="DRAWINGS">FIG. 16</figref> is a flow chart illustrating an example of positional condition determination processing. In the present exemplary embodiment, it is supposed that before the processing illustrated in <figref idrefs="DRAWINGS">FIG. 16</figref> starts (before the processing illustrated in <figref idrefs="DRAWINGS">FIG. 16</figref> is called), the gesture recognition processing by the gesture recognition unit <b>14</b> has been completed.

p-0143When the positional condition determination processing starts, the positional condition determination unit <b>37</b> secures a work area and initializes the work area.

p-0144Referring to <figref idrefs="DRAWINGS">FIG. 16</figref>, in step S<b>162</b>, the positional condition determination unit <b>37</b> loads the result of one locus on the work area. More specifically, in the example illustrated in <figref idrefs="DRAWINGS">FIG. 14</figref>, the positional condition determination unit <b>37</b> reads the gesture code “0002”, which indicates the locus <b>40</b>.

p-0145In step S<b>163</b>, the positional condition determination unit <b>37</b> reads the positional condition from the gesture dictionary according to the read gesture code. If it is recognized that the locus <b>40</b> is the gesture code “0002”, then the positional condition determination unit <b>37</b> reads the positional condition such that the character locus, which is the positional condition of the gesture code “0002”, exists within the gesture locus. In step S<b>164</b>, the positional condition determination unit <b>37</b> reads the other loci on an inspection work area.

p-0146In step S<b>165</b>, the positional condition determination unit <b>37</b> determines whether all the loci read in step S<b>164</b> satisfy the positional condition read in step S<b>163</b>. If it is determined that all the loci read in step S<b>164</b> satisfy the positional condition read in step S<b>163</b> (YES in step S<b>165</b>), then the processing advances to step S<b>166</b>. On the other hand, if it is determined that all the loci read in step S<b>164</b> do not satisfy the positional condition read in step S<b>163</b> (NO in step S<b>165</b>), then the processing advances to step S<b>167</b>.

p-0147In the example illustrated in <figref idrefs="DRAWINGS">FIG. 14</figref>, the positional condition determination unit <b>37</b> reads a condition such that if the recognition result of the locus <b>40</b> having a circular locus “∘” as an outer locus is the gesture code “0002”, then the character locus of the gesture command parameter exists within the gesture locus.

p-0148In this case, the first stroke locus “<b>1</b>” <b>38</b> and the second stroke locus “<b>0</b>” <b>39</b> is input within the outer circular locus <b>40</b>. Accordingly, the positional condition determination unit <b>37</b> determines that all the loci read in step S<b>164</b> satisfy the positional condition read in step S<b>163</b>.

p-0149On the other hand, the recognition result of the second stroke locus “<b>0</b>” <b>39</b> is the gesture code “0002” but both the first and the third strokes are input outside the second stroke locus “<b>0</b>” <b>39</b>. Accordingly, the positional condition determination unit <b>37</b> determines that the positional condition is not satisfied.

p-0150In step S<b>166</b>, the positional condition determination unit <b>37</b> stores the locus number of the locus that satisfies the positional condition and the gesture code thereof. In step S<b>167</b>, the positional condition determination unit <b>37</b> determines whether all the loci have been processed.

p-0151If it is determined that all the loci have been processed (YES in step S<b>167</b>), then the positional condition determination unit <b>37</b> outputs the locus number and the gesture code of the locus that has been determined to satisfy the positional condition stored in step S<b>166</b>. Then, the processing illustrated in <figref idrefs="DRAWINGS">FIG. 16</figref> ends.

p-0152On the other hand, if it is determined that all the loci have not been processed yet (NO in step S<b>167</b>), then the processing returns to step S<b>162</b>.

p-0153<figref idrefs="DRAWINGS">FIG. 17</figref> is a flow chart illustrating an example of locus separation processing (case <b>2</b>). When the locus separation processing starts, the separation unit <b>16</b> secures a work area and initializes the work area.

p-0154Referring to <figref idrefs="DRAWINGS">FIG. 17</figref>, in step S<b>172</b>, the separation unit <b>16</b> reads a recognition result of one locus. The recognition result of one locus is a recognition result after the positional condition determination processing is executed.

p-0155For the first stroke locus “<b>1</b>” <b>38</b>, no corresponding gesture exists. Accordingly, no corresponding gesture code exists. For the second stroke locus “<b>0</b>” <b>39</b>, no corresponding gesture code exists because the positional condition is not satisfied. For the third stroke locus “<b>0</b>” <b>40</b>, the corresponding gesture code “0002” exists because the positional condition is satisfied.

p-0156In step S<b>173</b>, the locus that the separation unit <b>16</b> determines whether the locus read in step S<b>172</b> is a gesture locus. If it is determined that the locus read instep S<b>172</b> is a gesture locus (YES in step S<b>173</b>), then the processing advances to step S<b>172</b>. On the other hand, if it is determined that the locus read in step S<b>172</b> is not a gesture locus (NO in step S<b>173</b>), then the processing advances to step S<b>174</b>.

p-0157In step S<b>174</b>, the separation unit <b>16</b> stores the locus read in step S<b>172</b> in the storage buffer as the character locus. More specifically, in the example illustrated in <figref idrefs="DRAWINGS">FIG. 14</figref>, the positional condition determination unit <b>37</b> stores the first and second loci “<b>10</b>” as the character locus, as the result of the gesture recognition and the result of the determination as to whether the locus satisfies the positional condition.

p-0158In step S<b>175</b>, the separation unit <b>16</b> determines whether all the loci have been completely processed. If it is determined that all the loci have been completely processed (YES instep S<b>175</b>), then the processing illustrated in <figref idrefs="DRAWINGS">FIG. 17</figref> ends. On the other hand, if it is determined that all the loci have not been completely processed yet (NO in step S<b>175</b>), then the processing returns to step S<b>172</b> to process a subsequent locus. After that, the processing similar to that in the first exemplary embodiment is executed.

p-0159If the combination of the loci “<b>1</b>”, “<b>0</b>”, and “∘” <b>38</b> through <b>40</b> illustrated in <figref idrefs="DRAWINGS">FIG. 14</figref> are recognized as an instruction for printing ten sheets, then the command processing unit <b>21</b> displays a screen (verification screen) <b>50</b> illustrated in <figref idrefs="DRAWINGS">FIG. 18</figref> on the liquid crystal display (LCD) device <b>7</b>.

p-0160<figref idrefs="DRAWINGS">FIG. 18</figref> illustrates an example of a screen displayed as a result of the gesture recognition (case <b>2</b>).

p-0161The information terminal <b>1</b> according to the present exemplary embodiment and having the above-described configuration executes the above-described processing. Accordingly, the present exemplary embodiment can appropriately determine whether an input locus is a gesture locus or a character locus according to the positional condition even if the gesture and the character have the same locus.

p-0162More specifically, the present exemplary embodiment can appropriately separate a gesture locus and a character locus even if the loci are input at the same time. Therefore, the present exemplary embodiment can appropriately execute recognition processing.

p-0163With the above-described configuration, the present exemplary embodiment can allow the user to freely set the shape of a gesture to be registered. Accordingly, the present exemplary embodiment can realize the information terminal <b>1</b>, on which a shape that the user can easily input or a locus by whose shape the user can easily think of the type of the corresponding command can be registered.

p-0164The present invention can also be achieved by providing a system or an apparatus with a storage medium storing program code of software implementing the functions of the embodiments and by reading and executing the program code stored in the storage medium with a computer of the system or the apparatus (a CPU or a micro processing unit (MPU)).

p-0165In this case, the program code itself, which is read from the storage medium, implements the functions of the embodiments described above, and accordingly, the storage medium storing the program code constitutes the present invention.

p-0166In addition, the functions according to the embodiments described above can be implemented not only by executing the program code read by the CPU of the system or the computer, but also implemented by the processing in which an OS or the like carries out a part of or the whole of the actual processing based on an instruction given by the program code.

p-0167Further, in another aspect of the embodiment of the present invention, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted in the system or the computer or a function expansion unit connected to the system or the computer, a CPU and the like provided in the function expansion board or the function expansion unit carries out a part of or the whole of the processing to implement the functions of the embodiments described above.

p-0168If the present invention is applied on the storage medium, the program code corresponding to the above-described flowcharts is stored on the storage medium (computer-readable storage medium).

p-0169With the above-described configuration, each of the above-described exemplary embodiments of the present invention can recognize each of loci input by the user in an arbitrary order and execute processing according to an instruction indicated by the loci.

p-0170Each exemplary embodiment of the present invention having the configuration described above can increase the separation ratio and the recognition ratio of the character locus according to the relationship between the gesture locus and the character locus input at the same time in inputting the gesture locus.

p-0171In addition, the second exemplary embodiment can separate the loci having the same shape owing to the setting of the condition for the positional relationship between the gesture locus and the gesture command parameter character locus. Therefore, the present invention can increase the user convenience.

p-0172Furthermore, if only one-stroke locus is registered in the gesture dictionary as a gesture locus, the size of the dictionary can be reduced and the processing can be executed at a high processing speed.

p-0173With the above-described configuration, each exemplary embodiment of the present invention described above can appropriately separate a gesture locus and a character locus from loci wherein a gesture locus and a character locus are mixed, within a practical processing time.

p-0174While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all modifications, equivalent structures, and functions.

p-0175This application claims priority from Japanese Patent Application No. 2008-334758 filed Dec. 26, 2008, which is hereby incorporated by reference herein in its entirety.