Image-based protruded part recognizing and tracking method

Image-based identification and tracking of prominent parts

<p>S10, S11,..., S17. . . Steps</p><p>a, b, ..., e. . . Feature points (protruding parts)</p><p>30. . . Feature area range</p><p>31. . . Feature area</p><p>32. . . Scanning area.</p>

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the following examples are specially cited, in conjunction with the accompanying drawings, and are described in detail as follows:

Figure 1 is a flowchart showing the operation flow of the image-based method for identifying and tracking protrusions according to an embodiment of the present invention.

Figure 2 is a schematic diagram to illustrate the detection of protruding parts.

Figure 3a shows the range of characteristic regions in the first image.

Figure 3b shows the image area range of the detection feature area in the second image.

Figure 4 is a schematic diagram to illustrate the identification of the trigger mechanism.

The present invention relates to an image recognition and tracking method, and in particular to an image-based protruding part recognition and tracking that is based on the image tone and is based on the characteristics of the protruding part to effectively identify and track the protruding parts in the two images Tracking method.

For application developers, in addition to the functional considerations of the application, the interactive interface between the application and the user is a key consideration. The interactive interface between the general program and the user is a traditional keyboard and mouse. The user can issue commands to the application program through the keyboard and mouse to perform related operations and calculations.

In addition, with continuous innovation and research and development, a new generation of human-machine interfaces based on human "hand" has also been tried to develop. However, because the technology is not yet mature, this new generation based on "hand" The human-machine interface must be limited to a special use environment. For example, it must be operated on a reflective plane to reduce environmental interference. However, this limitation will reduce the freedom of use. When using a "hand"-based interface, a background model must be established in advance for subsequent segmentation and use, thereby increasing its inconvenience.

In addition, if the skeleton (Skeleton) is used as the feature of gesture recognition, it is more susceptible to the influence of noise (NOise); and if the contour is used as the recognition basis, it is more difficult to completely segment the contours of the hands and body parts.

In view of this, the main purpose of the present invention is to provide an image-based protrusion recognition based on the image hue (Hue) and based on the characteristics of the protrusion in the feature area to effectively identify and track the protrusions in the two images And tracking methods.

In order to achieve the above-mentioned object of the present invention, it can be achieved by the image-based protruding part identification and tracking method provided by the present invention.

According to the image-based protruding part identification and tracking method of the embodiment of the present invention, first, an image is converted into a hue, saturation, brightness (HSV) image format for representation, and the characteristic regions in the image are identified according to a hue range , And extract the area contour of the corresponding characteristic area. After that, for each pixel on the area contour, it is determined whether the distance from the pixel to the center of gravity of the characteristic area is greater than the distance from the pixel before and after the pixel to the center of gravity. If the distance from the pixel point to the center of gravity is greater than the distance from the previous pixel point and the next pixel point to the center of gravity, then the pixel point is determined to be a prominent part in the characteristic area.

In addition, the present invention further receives another image to perform the above-mentioned protruding part recognition, so as to identify the protruding part in the image. In addition, if the difference between the angle between the protrusion and the center of gravity and the horizontal line in the two images is less than a predetermined angle threshold and the distance between the protrusion and the center of gravity in the two images is greater than a predetermined distance threshold, A trigger message is sent to trigger a specific application or function command.

Example

Figure 1 is a flowchart showing the operation flow of the image-based method for identifying and tracking protrusions according to an embodiment of the present invention. It should be noted that this embodiment takes the fingertips (protruding parts) of the hand area as an example to identify and track the protruding parts, but it is not limited to this.

First, in step S10, an image is received, and in step S11, the image is converted into a hue, saturation, and brightness (HSV) image format for representation. It should be noted that if the image is expressed in the hue, saturation, and brightness image format when the image is received, step S11 can be omitted.

Then, in step S12, the characteristic region in the image is identified according to a tonal range, and in step S13, the region contour of the corresponding characteristic region is captured. Since this embodiment is used to identify the fingertips of the hand region, the characteristic region refers to the hand region, and the color tone range is the skin tone range. Assuming that Pixel(i,j) represents the hue information of the pixel at the coordinate (i,j) in the image, and then use the predefined maximum hue value Hue_Max and minimum Hue_Min (hue range) to cut the characteristic area. as follows:

<img file="TW594591B_D0001.tif" />

When the hue information of a pixel falls within the hue range, the pixel belongs to the pixel in the characteristic area; otherwise, it does not belong.

Then, in step S14, for the pixels on the area contour, it is determined whether the distance from the pixel to the center of gravity of the characteristic area is greater than the distance from the previous pixel to the next pixel to the center of gravity. If the distance from the pixel point to the center of gravity is not greater than the distance from the pixel point before or after the pixel point to the center of gravity (No in step S14), then the determination of step S16 is performed (described later). And if the distance from the pixel to the center of gravity is greater than the distance from the previous pixel to the next pixel to the center of gravity (Yes in step S14), then in step S15, it is determined that the pixel is a prominent part in the characteristic area, which means tip.

Figure 2 is a schematic diagram to illustrate the detection of protruding parts. In this example, Pixel_CenterC(P_CC) represents the center of gravity of the feature area; PiXel_Contour(i)(P_C(i)) represents the i-th contour point (pixel point) on the contour of the area; Distance(i)=(Pixel_Contour(i) )_Pixel_Centerc) ² , D(i) represents the distance from the i-th contour point to the center of gravity. The formula for judging whether a contour point is a relative extreme value (protruding part) is as follows: Pixel_Contour(i)=ExtremaPoint;if Dis tan ce(i-l)&lt;Dis tan ce (i)&amp;Dis tan ce(i1)&lt;Dis tan ce( i) Among them, ExtremaPoint represents a feature point (protruding part). When the distance from the contour point to the center of gravity is greater than the distance from the adjacent contour point to the center of gravity, the contour point is judged as a feature point.

After that, in step S16, it is judged whether all the pixels on the contour of the area have been judged for the protruding part. If not all pixels on the contour of the area have been judged for the protruding part (No in step S16), then in step S17, skip to the next pixel on the contour of the area, and return to step S14 to continue judging pixels Whether the distance to the center of gravity of the feature area is greater than the distance from a pixel before and after the pixel to the center of gravity. And if all the pixels on the contour of the area have been judged for the protruding part (Yes in step S16), then the identification of the protruding part of the entire feature area is ended.

When tracking the protruding parts of the characteristic region, another image is received, and the above-mentioned protruding part recognition (such as step S10 to step S17) is performed to identify the characteristic regions and protruding parts in the image. When identifying the characteristic areas in this image, one of the predetermined areas in the image can be identified by the tonal range (skin tone range) in the image, where the predetermined area ranges correspondingly include the characteristic areas in the previous image. The area of all protruding parts and one of the areas around this area have been positioned and moved. Note that the purpose of identifying feature regions in an image with a predetermined area is to avoid searching the entire image and consuming most of the computing resources. Except for the initial image, the entire image must be scanned. After the feature points are found, other images will be used. The characteristic area can be identified according to the established area range.

Figure 3a shows the detected feature area 30 and feature points a, b, c, d, and e in a first image. In Figure 3a, the feature area range 30 includes all feature points a, b, c, d, e, and f of the feature area 31, and has a height H and a width W. Since the human hand area has a certain range size and the displacement between two consecutive images is limited, a neighboring block range can be defined for searching, as shown in Figure 3b. In Fig. 3b, the scanning area 32 includes the position of the characteristic area 30 and the surrounding area corresponding to the location shift d area, that is, the scanning area 32 has a height (H+2d) and a width (W+2d).

It is worth noting that before the recognition of the second image, the entire image can be converted to the hue, saturation, brightness (HSV) image format, or only the predetermined area defined in the image can be converted to Hue, saturation, brightness (HSV) image format representation.

In addition, for the trigger mechanism of the man-machine operation interface, after the feature points are identified, the change in the location of the feature points can be used as the movement of the feature to determine whether a trigger behavior occurs.

Figure 4 is a schematic diagram to illustrate the identification of the trigger mechanism. If Mag(i) represents the distance between the feature point (P_FP) and the center of gravity (P_CC) in the i-th image, and Theta(i) represents the angle between the line between the feature point and the center of gravity and the horizontal line. If the position change of the feature point satisfies the following conditions, the trigger mechanism is considered to be activated (similar to the action of pressing the left mouse button).

Trigger=true; if(abs(Theta(i)OneTheta(i-1))<Angle Threshold)&(αbs(Mag(i)-Mag(i-1))>MagThreshold) Among them, AngleThreaShold is the preset angle threshold Limit, and MagThreaShold is the predetermined distance threshold. In other words, if the angle between the characteristic point (protruding part) in the second image and the line connecting the center of gravity and the horizontal line is less than a predetermined angle Threshold value and the difference between the distance between the protruding part and the center of gravity in the second image and the distance between the protruding part and the center of gravity in the first image is greater than a predetermined distance threshold, which means that the trigger mechanism is activated, and a trigger message is sent for specific Triggering of application programs or function commands.

It should be noted that the conditions for determining the activation of the trigger mechanism can be designed or changed according to different applications, and are not limited to the determination rules described in this embodiment. In addition, since there may be multiple feature points in the feature area, the feature points can be filtered again and the trigger content and trigger conditions corresponding to different feature points can be defined.

Therefore, with the image-based protruding part identification and tracking method provided by the present invention, the protruding parts in the two images can be effectively identified and tracked based on the image hue (Hue) and according to the features of the protruding parts in the feature area .

In addition, the present invention has the following advantages: First, the use of hue information for segmentation can reduce the degree of influence of light. In addition, because the cutting is directly set in the hue division of the feature area to be searched (in this embodiment, the hand) Within the region, even in non-specific environments, the feature region can be segmented directly without the step of establishing a background model; second, in this embodiment, using the fingertip as the feature point has the advantage of being clear and easy to identify , The processing in the positioning and tracking stage is simpler and saves system resources than the outline or backbone-based approach.

Although the present invention has been disclosed in preferred embodiments as above, it is not intended to limit the present invention. Anyone familiar with the art can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the present invention The scope of protection shall be subject to the scope of the attached patent application.

Schematic description

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the following examples are specially cited, in conjunction with the accompanying drawings, and are described in detail as follows:

Figure 1 is a flowchart showing the operation flow of the image-based method for identifying and tracking protrusions according to an embodiment of the present invention.

Figure 2 is a schematic diagram to illustrate the detection of protruding parts.

Figure 3a shows the range of characteristic regions in the first image.

Figure 3b shows the image area range of the detection feature area in the second image.

Figure 4 is a schematic diagram to illustrate the identification of the trigger mechanism.

Symbol description of main components

S10, S11,..., S17. . . Steps

a, b, ..., e. . . Feature points (protruding parts)

30. . . Feature area range

31. . . Feature area

32. . . Scanning area.