Object recognition device, object recognition method, and program

0001The present invention relates to an object recognition device, an object recognition method, and a program.

0002Conventionally, there is a technique of performing image recognition on a captured image to recognize an object included in the image. In this object recognition technology, in recent years, an object has been recognized with high accuracy by using a deep learning technology in which a neural network is multi-layered. This deep learning technique generally recognizes an object with high accuracy by stacking layers in a plurality of stages in an intermediate layer between an input layer and an output layer of a neural network. In this deep learning technique, in particular, a convolutional neural network (CNN) is attracting attention because it has higher performance than recognizing an object based on a conventional image feature amount.

0003According to the convolutional neural network, the labeled recognition target image is learned and the main target included in the recognition target image is recognized. Therefore, when there are a plurality of main objects in the training image, the convolutional neural network manually specifies the area rectangle including the main objects and assigns a label to the image corresponding to the specified area. I was learning.

0004In this learning process, it was necessary to give a label to the designated area. In the subsequent learning process, the image corresponding to the designated area is input to the convolutional neural network, and the feature vector output from the convolutional neural network is learned. Further, in the conventional target recognition process, a candidate area is extracted based on a local feature by performing edge extraction processing or the like on the recognition target image, and the candidate area is input to a convolutional neural network to be characterized. After extracting the vector, classification was performed, and the classified candidate region with the highest certainty was obtained as the recognition result. Conviction is a quantity that indicates how much the subject similarity of an image learned with a certain image region and label is relatively higher than the similarity of another class.

<p num="0005"><nplcit num="1"><text>Rich feature hierarchies for accurate object detection and semantic segmentation, Tech report, Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, UC Berkeley (2014)</text></nplcit></p>

<p num="0006"> However, the object recognition technology using the conventional convolutional neural network cannot automatically narrow down to one desired object when there are a plurality of objects in the training image, and thus is included in the image as described above. It was necessary to manually specify multiple areas containing the target one by one.</p><p num="0007"> That is, in the learning process in the object recognition technology using the conventional convolutional neural network, it is necessary to add one label to one recognition target image and create the learning data, so that the burden for specifying the area is burdensome. large. In addition, it is necessary to create and store an image (hereinafter referred to as a teacher image) to be processed in the learning process for improving the recognition accuracy of the object exclusively for the processing of the convolutional neural network.</p><p num="0008"> Further, in the recognition process in the object recognition technology using the conventional convolutional neural network, the recognition target is often the one with a clear outline of an object or the like, and the teacher image is often the one with a clear outline of the object or the like. For this reason, good recognition accuracy could not be exhibited for an image in which it is difficult to determine a specific object. For example, an image containing a large number of objects such as the scenery of the entire city such as Akihabara and Asakusa, an image in which it is difficult to distinguish a specific object, and a mixture of short-range objects and long-range objects in the image. It was not possible to process images with high recognition accuracy, such as images with large differences in size and images in which objects of very different sizes are mixed.</p><p num="0009"> Further, there is a similar problem not only in the object recognition technique using the conventional convolutional neural network but also in other existing object recognition techniques.</p><p num="0010"> Therefore, the present invention has been made in view of the above problems, and is an object recognition device, an object recognition method, and a program capable of determining a main object in an image and the position of the object with higher recognition accuracy. Is intended to provide.</p>

<p num="0011"> (1) One aspect of the present invention is an image conversion unit that sets a plurality of candidate areas of an arbitrary area in a recognition target image, and a recognition process for image information included in each candidate area set by the image conversion unit. The target candidate is calculated based on the recognition calculation unit that calculates the certainty of each target candidate in each candidate area, the certainty of each target candidate calculated by the recognition calculation unit, and the area position of each candidate area. It is an object recognition device having a map generation unit that creates a map for each, and a position calculation unit that specifies a target position based on a target candidate map created by the map generation unit.</p><p num="0012"> (2) One aspect of the present invention is the above-mentioned object recognition device, and the map generation unit statistically processes the certainty of the target candidate for each position of the recognition target image, and obtains the result of the statistical processing. A map of the target candidate is created based on the map, and the position calculation unit extracts the target candidate from the map and specifies the extracted area as the target position.</p><p num="0013"> (3) One aspect of the present invention is the object recognition device described above, wherein the map generation unit has a plurality of overlapping candidate regions among a plurality of candidate regions set by the image conversion unit. Statistical processing of certainty.</p><p num="0014"> (4) One aspect of the present invention is the above-mentioned object recognition device, in which the image conversion unit extracts a feature amount of the recognition target image and based on the extracted feature amount, the inside of the recognition target image. A candidate area is set according to the size of the feature included in the image, and a candidate area having an area smaller than a predetermined area area is deleted from the plurality of candidate areas set in the recognition target image.</p><p num="0015"> (5) One aspect of the present invention is the above-mentioned object recognition device, which has a learning processing unit that learns an object based on a learning image, and the learning processing unit is an entire learning image supplied from the outside. The learning result is generated using the above, and the recognition calculation unit calculates the certainty of the target candidate using the image of the candidate region set by the image conversion unit and the learning result.</p><p num="0016"> (6) In one aspect of the present invention, a step of setting a plurality of candidate regions of an arbitrary area in the recognition target image and recognition processing are performed on the image information included in each of the candidate regions, and the target candidate is in each of the candidate regions. A step of calculating the certainty of each target candidate, a step of creating a map for each target candidate based on the certainty of each target candidate, and a region position of each candidate area, and a target based on the target candidate map. It is an object recognition method having a step of specifying the position of.</p><p num="0017"> (7) In one aspect of the present invention, a computer is made to set a plurality of candidate areas of an arbitrary area in the recognition target image, recognition processing is performed on the image information included in each candidate area, and the target is targeted to each of the candidate areas. The confidence level for each candidate is calculated, a map for each target candidate is created based on the certainty level for each target candidate and the area position of each candidate area, and the target position is determined based on the target candidate map. It is a program to identify.</p>

<p num="0018"> According to one aspect of the present invention, the main object in the image and the position of the object can be determined with higher recognition accuracy.</p>

0019<figref num="1">It is a block diagram which shows the functional structure of the object recognition apparatus 1 of embodiment which applied this invention.</figref><figref num="2">It is a block diagram which shows one configuration example of the object recognition apparatus 1 of the Embodiment to which this invention is applied.</figref><figref num="3">It is a figure which shows the learning image P10 which imaged the tower-like structure from a short distance and from below.</figref><figref num="4">It is a figure which shows the learning image P11 which imaged a building from a long distance and from above.</figref><figref num="5">It is a figure which shows the learning image P12 which imaged the building at night.</figref><figref num="6">It is a flowchart which shows the procedure of the recognition process in the object recognition apparatus 1 shown as the embodiment of this invention.</figref><figref num="7">It is a figure which shows an example of the candidate area set in the recognition target image input to the object recognition device 1 to which this invention is applied, and the certainty degree for each candidate area.</figref><figref num="8">It is a figure which shows the relationship between the certainty degree and area, and a candidate area in the object recognition apparatus 1 to which this invention is applied.</figref><figref num="9">It is a figure which illustrated the map created by the map generation part 16 in the object recognition apparatus 1 to which this invention was applied.</figref><figref num="10">It is a figure which shows an example of the candidate area set in the recognition target image P1 as a comparative example.</figref><figref num="11">It is a figure which shows an example of another candidate area set in the recognition target image P1 as a comparative example.</figref>

0020Hereinafter, an object recognition device, an object recognition method, and a program to which the present invention is applied will be described with reference to the drawings. First, the functional description of the object recognition device 1 of the embodiment to which the present invention is applied will be described. FIG. 1 is a block diagram showing a functional configuration of the object recognition device 1 according to the embodiment to which the present invention is applied. The target recognition device 1 recognizes the main target among the targets included in the recognition target image, and outputs the attributes (name, type) and position of the recognized target. The target recognition device 1 includes an image conversion unit 12, a recognition calculation unit 14, a map generation unit 16, and a position calculation unit 18.

0021The image conversion unit 12 is supplied with a recognition target image from the outside. The image conversion unit 12 may set a plurality of candidate areas for the recognition target image by an arbitrary method, and the plurality of candidate areas may overlap each other. The image conversion unit 12 will be described as setting a candidate area based on the characteristics of the image to be recognized. However, the method for setting the candidate area is not limited to this, and a method such as shifting a frame of several sizes may be adopted.

0022The recognition calculation unit 14 performs recognition processing on the image information included in each candidate area set by the image conversion unit 12. The recognition calculation unit 14 refers to the learning result stored in the learning result database 140, and increases the certainty as the image information included in each candidate area resembles the target candidate. As a result, the recognition calculation unit 14 calculates the certainty of each target candidate in each candidate area. The target candidate is a target that can be recognized by a person who sees an image such as an object or a landscape included in the recognition target image, and is a candidate that may be included in the recognition target image. In addition, the conviction is the certainty of the target candidate, and how much the similarity of the subject of the image learned together with the arbitrary image and the label is relatively higher than the similarity of other classes (target candidates). It is an amount indicating. When a plurality of target candidates are calculated for the image included in the candidate area, the recognition calculation unit 14 obtains certainty for each target candidate. As a result, the recognition calculation unit 14 obtains one or a plurality of target candidates and the certainty of each target candidate for each candidate area.

0023The map generation unit 16 calculates an evaluation value for each position of the recognition target image based on the class ID and the certainty. The map generation unit 16 calculates an evaluation value for each pixel of the image to be recognized, or for each representative position in which a set of pixels is regarded as one position. The map generation unit 16 performs a predetermined statistical process on the certainty and calculates an evaluation value. This evaluation value becomes higher as the evaluation that the main target included in the recognition target image is the target corresponding to the class ID is higher. In the statistical processing, for example, the evaluation value is calculated by summing the certainty for each class ID for each position of the recognition target image. According to this statistical processing, the higher the evaluation value is calculated, the more the candidate areas for which the same class ID is calculated overlap. Further, in the statistical processing, the coefficient and the certainty according to the area of the candidate region may be used, and the evaluation value may be higher as the area of the candidate region is larger. Further, in the statistical processing, the evaluation value may be calculated by summing the class ID and the certainty and using the coefficient according to the area of the candidate area. According to this conversion process, the higher the evaluation value is calculated as the candidate regions for which the same class ID is calculated overlap and the area of the candidate regions is larger. In the statistical conversion process, other processing may be performed as long as it is possible to calculate an evaluation value that gives a high evaluation that the main target included in the recognition target image is the target corresponding to the class ID. As a result, the map generation unit 16 can create a map including the evaluation value for each class ID for each position of the recognition target image. That is, the higher the possibility that the target corresponding to the class ID exists, the higher the evaluation value is distributed to generate the map.

0024The position calculation unit 18 calculates the position of the target (class ID) mainly included in the recognition target image with reference to the map created by the map generation unit 16. The position calculation unit 18 determines that the target corresponding to the class ID exists in the region where the evaluation value of the class ID is high in the recognition target image. The position calculation unit 18 may, for example, extract a region of the recognition target image whose evaluation value exceeds a predetermined threshold value, and determine that an target corresponding to the class ID exists in the extracted region. In addition, the position calculation unit 18 performs edge extraction processing on the map, extracts an area surrounded by edge lines whose evaluation values are gradually increased, and targets the extracted area corresponding to the class ID. It may be determined that it exists. As a result, the position calculation unit 18 can obtain the class ID and the position information of the target corresponding to the class ID. This position information may be the contour position of the target or the center position of the target.

0025Hereinafter, the specific configuration and operation of the target recognition device 1 shown in FIG. 1 will be described. FIG. 2 is a block diagram showing a configuration example of the object recognition device 1 according to the embodiment to which the present invention is applied. The target recognition device 1 is mounted on, for example, a portable communication terminal. The target recognition device 1 is connected to the image processing device 200 and the image display unit 300.

0026The image processing device 200 includes an image generation unit 202 and an AR (Augmented Reality) processing unit 204. The image generation unit 202 generates a recognition target image supplied to the target recognition device 1. The image generation unit 202 is composed of, for example, a camera device, a communication unit that receives image data from the outside, or a storage unit that stores image data. The image processing device 200 supplies the recognition target image generated by the image generation unit 202 to the target recognition device 1 when the target included in the recognition target image is recognized.

0027The AR processing unit 204 performs AR processing by performing image processing on the recognition target image generated by the image generation unit 202. This AR process includes a process of adding, deleting, emphasizing, or attenuating information to the environment represented by the image to be recognized. The AR process includes, for example, a process of adding an explanatory image of the target included in the recognition target image to the recognition target image captured by the camera device. The AR processing unit 204 causes the target recognition device 1 to output the recognition target image from the image generation unit 202 when the target recognition device 1 identifies the target included in the recognition target image. In the present embodiment, the target recognition device 1 will be described as recognizing the landmark image included in the recognition target image and performing AR processing for adding a description about the landmark image to the recognition target image.

0028In the present embodiment, the result (landmark) of the target recognized by the target recognition device 1 will be used for AR processing, but the result of the target recognition by the target recognition device 1 will be used for other purposes. Of course, it may be used for. For example, the image processing device 200 may cause the target recognition device 1 to identify an object included in an image displayed by the SNS application in response to a request output from the SNS (social networking service) application. The SNS application deletes or attenuates the target from the recognition target image when the recognition target image includes a preset target based on the result (class ID and position) of the target being recognized by the target recognition device 1. Image processing such as adding other information is performed.

0029The target recognition device 1 includes an image conversion unit 12, a recognition calculation unit 14, a map generation unit 16, a position calculation unit 18, an interface unit 20, a learning image storage unit 22, and a learning processing unit 24. The target recognition device 1 is a computer having a control device for executing software and a storage unit. The target recognition device 1 is a software functional unit that functions when a processor such as a CPU (Central Processing Unit) executes a program stored in a storage unit (not shown). Further, the target recognition device 1 may be a hardware functional unit such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit). The storage unit in the target recognition device 1 includes, for example, an HDD (Hard Disc Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), and a ROM (Read Only). It is realized by Memory) or RAM (Random Access Memory). In addition, various programs such as firmware and application programs, processing results by various function units, information acquired from the outside by the interface unit 20, and the like are stored in the storage unit.

0030The interface unit 20 exchanges information with the image processing device 200 mounted on the portable communication terminal, and also exchanges information with each unit of the target recognition device 1. The interface unit 20 outputs the recognition target image to the image conversion unit 12 as a processing target in response to the supply of the recognition target image from the image processing device 200. The interface unit 20 supplies the class ID and the position as the processing result of the target for the image to be recognized from the position calculation unit 18, and outputs the class ID and the position to the image processing device 200. This class ID and position are information representing the classification of the landmark image included in the recognition target image. The class ID is information for identifying the main target included in the recognition target image, and examples thereof include a building name, an urban area name, and a landscape name.

0031The learning image storage unit 22 has a function of accumulating a learning image to which a class ID is added. The learning image storage unit 22 has an image conversion unit 220 and an image data database 222. The learning image storage unit 22 downloads a learning image from a server device or the like (not shown) that stores an image as a learning material. The learning image storage unit 22 downloads the learning image according to the instruction of the AR application executed in the target recognition device 1, for example. A label is added to this learning image. The attribute information of the learning image is written in this label. The attribute information includes the classification of landmarks included in the learning image. Attribute information is added as a class ID. In the present embodiment, the learning image storage unit 22 downloads the image material including the landmark as a learning image. The landmark image included in this learning image is preferably an image that can be recognized as a landmark by a human. Further, the landmark image in the learning image may have an arbitrary area with respect to the image area, and the position in the image may be an arbitrary position.

0032The image conversion unit 220 is supplied with a learning image used for performing learning processing from the outside. The image conversion unit 220 preprocesses the learning image according to the processing method of the learning process in the subsequent stage. The image conversion unit 220 performs, for example, area change processing of the learning image, adjustment of the aspect ratio, and the like, and converts the learning image into a format that can be performed. It is desirable that the image conversion unit 220 is trained by the learning processing unit 24 using the entire learning image, but the learning processing unit 24 may perform learning by dividing or cutting out an area of the learning image. The image data database 222 stores the learning image before conversion supplied from the outside and the learning image converted by the image conversion unit 220. The image data database 222 stores the learning image converted by the image conversion unit 220 as a batch file in which the learning process can be executed in the learning processing unit 24. In addition, the image data database 222 accumulates the training image before conversion and the training image after conversion for each landmark classified by the label.

0033The learning processing unit 24 performs learning based on an image with a label (class ID) stored in the image data database 222. The learning process executed by the learning processing unit 24 converts the image data into numerical data such as a vector. The learning result obtained by this learning process is stored in the learning result database 140 in the recognition calculation unit 14. The learning result obtained by the learning processing unit 24 is referred to by the recognition calculation unit 14.

0034The image conversion unit 12, the recognition calculation unit 14, the map generation unit 16, and the position calculation unit 18 operate as described with reference to FIG. As a result, the target recognition device 1 outputs the class ID and the position to the interface unit 20 in response to the supply of the recognition target image.

0035The learning method by the learning processing unit 24 may be a convolutional neural network or another machine learning method (SVM, etc.), and the learning result database 140 stores the learning results according to the learning method. To. The recognition calculation unit 14 obtains the class ID and the certainty corresponding to the main target included in the recognition target image based on the learning result stored in the learning result database 140.

0036When a convolutional neural network is used to calculate the class ID and certainty, the convolutional neural network is learned to classify the images to be recognized into an arbitrary number of classes. The learning processing unit 24 learns the filter coefficient and the bias value in the convolution layer so that the learning result of the class ID: 1 can be obtained with respect to the supply of the learning image of the class ID: 1. Specifically, the convolutional neural networks learned by classifying the classes into "coast", "Akihabara", "Shibuya", "Tokyo Tower", and "Tokyo Metropolitan Government Building" are "coast", "Akihabara", "Shibuya", Classify into "Coast", "Akihabara", "Shibuya", "Tokyo Tower", and "Tokyo Metropolitan Government Building" according to the supply of recognition target images including "Tokyo Tower" and "Tokyo Metropolitan Government Building". Can be done. Further, in this learning process, the convolutional neural network includes, for example, those shown in FIGS. 3 to 5 as learning images. The learning image P10 shown in FIG. 3 is an image of a tower-shaped building (landmark) taken from a short distance and from below, and the learning image P11 shown in FIG. 4 is an image of a tower-shaped building taken from a long distance and from above. It is an image, and the learning image P12 shown in FIG. 5 is an image of a tower-shaped building captured at night. In this learning process, the target recognition device 1 performs the learning process on the entire learning image without extracting a specific object and cutting out a candidate region.

0037Next, the target recognition process in the target recognition device 1 configured as described above will be described. FIG. 6 is a flowchart showing a procedure of recognition processing in the target recognition device 1 shown as an embodiment of the present invention.

0038First, the image conversion unit 12 waits until the recognition target image is input (step S200), and in response to the recognition target image being input (step S200: YES), sets an area of an arbitrary area in the recognition target image. Multiple settings (step S202). At this time, the image conversion unit 12 extracts the feature amount of the recognition target image, and based on the extracted feature amount, an area (candidate area) having an arbitrary area according to the size of the feature included in the recognition target image. Set multiple. The image conversion unit 12 supplies an image for each candidate area to the recognition calculation unit 14.

0039At this time, the image conversion unit 12 may delete the candidate area having an area smaller than the predetermined area area among the plurality of candidate areas set for the recognition target image. For this predetermined area area, it is desirable that the ratio including the main target in the recognition target image is set. For example, the size of the recognition target image may be such that the viewer of the image pays attention to it. The predetermined area may be set in advance, or may be set based on the area of a plurality of candidate areas set for the recognition target image.

0040FIG. 7 is a diagram showing an example of a candidate region set in the recognition target image input to the target recognition device 1 to which the present invention is applied and the certainty of each candidate region. As shown in FIG. 7, when the tower-shaped building is supplied as an image of a candidate region including a large image area at a substantially central position, the image conversion unit 12 includes a plurality of candidate regions including the tower-shaped building A. , A plurality of candidate areas set in the building B other than the tower-shaped building and a plurality of candidate areas set in the building C are set. As shown in FIG. 7, the image conversion unit 12 can set a plurality of candidate regions with little leakage for the object included in the recognition target image. What kind of existing method is used to set the candidate regions? Any method may be used.

0041The recognition calculation unit 14 receives the area candidate from the image conversion unit 12, and uses the learning result data to calculate the class ID and the certainty of the area image passed from the image conversion unit 12 (step S204). The recognition calculation unit 14 selects the next candidate area (step S208) and calculates the class ID and the certainty (step S204) until the calculation of the certainty of all the candidate areas set by the image conversion unit 12 is completed. ) And repeat. The recognition calculation unit 14 calculates an evaluation value for each position of the recognition target image by the map generation unit 16 according to the completion of the calculation of the certainty for all the candidate areas (step S206: YES), and the evaluation value. Generate a map of (step S210). Then, the position calculation unit 18 outputs the class ID and the position representing the main target included in the recognition target image to the interface unit 20 based on the map generated by the map generation unit 16 (step S212).

0042In the example shown in FIG. 7, it shows how the certainty for each class ID is calculated for each of a plurality of candidate regions. FIG. 8 is a diagram illustrating the relationship between the certainty of each candidate region and the area used by the map generation unit in the object recognition device 1 to which the present invention is applied when calculating the evaluation value. In FIG. 8, the area counts candidate areas exceeding one million pixels. As shown in Fig. 8 (a), there are many candidate areas with high certainty and large area for tower-shaped building A. As for building B, as shown in Fig. 8 (b), there are many candidate areas with high certainty but small area. As for building C, as shown in Fig. 8 (c), there are many candidate areas with low certainty and small area. Utilizing the characteristics shown in FIG. 8, the target recognition device 1 statistically processes the certainty of the position of the recognition target image for each class ID, and the main target is the area where the evaluation value obtained as a result of the statistical processing is high. It may be an existing area.

0043FIG. 9 is for explaining the relationship between the position and the evaluation value represented by the recognition target image P recognized by the recognition calculation unit 14 and the map created by the map generation unit 16 in the target recognition device 1 to which the present invention is applied. This map, which is a diagram showing a visualized example, is created for Ta (tower type building), Tb (mountain), Tc, Te, and Tf for each class ID. In the example shown in FIG. 9, the map for each class ID is created so that the higher the evaluation value position, the higher the brightness becomes. The map differs as shown in (a) to (d) due to the difference in statistical processing for calculating the evaluation value, but any of the maps (a) to (d) shown in FIG. 9 is used as the target. The position may be calculated. The position calculation unit 18 performs edge extraction processing on the map of each class ID, extracts the area surrounded by the edge line whose evaluation value is gradually increased, and corresponds to the class ID in the extracted area. It may be determined that the target is present. According to FIG. 9, Ta (tower type building) existing at a substantially central position of the recognition target image is extracted by performing edge extraction processing on the map whose class ID is Ta (tower type building). be able to. As a result, the target recognition device 1 can output the processing result including the class ID (Ta) representing the tower type building and the extracted position of the tower type building from the interface unit 20.

0044In the explanation shown in FIG. 8, the map generation unit 16 created a map of the evaluation values for each class ID, but the evaluation value map was not created for the class IDs that are unlikely to be included in the recognition target image. You may. If the evaluation value calculated by statistically processing the certainty is low, there is a high possibility that the main target will not be extracted from the map of the evaluation value, and the map generator 16 has the class ID in which such an evaluation value is calculated. Is determined not to create a map. Before creating the map, the map generation unit 16 calculates an evaluation index that can be calculated with a smaller amount of calculation than the map creation process. For example, the map generation unit 16 obtains the total value of the evaluation function with the area and certainty of the candidate area as parameters for each class ID, compares the total value between the class IDs, and creates a map of the evaluation value. Determine the ID and the class ID that does not map the evaluation value. As a result, the object recognition device 1 can suppress the amount of processing for creating the map while suppressing the decrease in the recognition accuracy of the main object.

0045As described above, according to the target recognition device 1 to which the present invention is applied, a plurality of areas of an arbitrary area are set in the recognition target image, the certainty of the target candidate included in each area is calculated, and the target candidate is calculated. Since a map for each target candidate is generated based on the certainty level and the position of the target candidate and the target position is specified from the map, the target included in the area with high certainty is simply included in the recognition target image. It is possible to suppress erroneous recognition that it is an object, and it is possible to determine the main object in the image to be recognized with higher recognition accuracy.

0046For example, even if a landscape of an urban area containing a large number of buildings or a landscape containing a large number of buildings but containing a single landmark is input as a recognition target image, it is clearly defined in the recognition target image. It is possible to suppress the output of the classification result that the single target included is the main target in the recognition target image. More specifically, according to the object recognition device 1 to which the present invention is applied, the name of the city area of the symbolic townscape can be recognized as the main object, and the landmark occupying a large area can be recognized as the main object. ..

0047FIG. 10 shows an example of a candidate region set in the recognition target image P1 as a comparative example. FIG. 11 shows an example of another candidate region set in the recognition target image P1 as a comparative example. As a comparative example, if the landmarks output from the convolutional neural network are obtained as the classification result based only on the certainty obtained by inputting the images of the candidate regions A1, B1, and C1 shown in FIG. 10 into the convolutional neural network. Of the candidate areas A1, B1, and C1 set for the recognition target image P1, the certainty of the candidate area B1 is the highest. However, the main target included in the recognition target image P1 is the building T1, which is included in the image of the candidate area A1.

0048Similarly, as a comparative example, the landmarks output from the convolutional neural network are classified based only on the certainty obtained by inputting the images of the candidate regions A2, B2, and C2 shown in FIG. 11 into the convolutional neural network. Among the candidate areas A2, B2, and C2 set for the recognition target image P2, the certainty of the candidate area B2 is the highest. However, the main target included in the recognition target image P2 is the building T2, which is included in the image of the candidate area A2.

0049In this way, when a plurality of objects are included in the recognition target image as in the comparative example, erroneous recognition may occur if the main target included in the recognition target image is recognized based only on the certainty. .. On the other hand, according to the object recognition device 1 to which the present invention is applied, the class ID and the certainty are calculated for each candidate area, and the certainty is statistically processed for each class ID. Therefore, for example, it is shown in FIG. 10 or FIG. As described above, it is possible to increase the probability that the target included in the candidate area having a large area among the candidate areas can be recognized as the main target.

0050Further, according to the object recognition device 1 to which the present invention is applied, the learning processing unit 24 learns using the entire learning image supplied from the outside, so that the object included in the entire recognition target image is recognized. It is possible to determine the main target in the recognition target image with higher recognition accuracy.

0051That is, according to the object recognition device 1 to which the present invention is applied, it is not necessary to cut out the object that is the correct answer in the learning process, so that the learning is performed regardless of the position of the object that is the correct answer in the learning image. Learning processing can be performed on the entire image, and the main target in the recognition target image can be determined with high recognition accuracy regardless of the form in which the main target is included in the entire image in the recognition processing. it can.

0052Specifically, if teacher data is created so that a specific object can be recognized as in existing technology and features are matched, it is difficult to distinguish the area of the object such as a landscape (for example, an urban area, a boulevard, etc.). Recognition performance cannot be exhibited for images and large buildings whose appearance differs greatly from near to distant views, but according to the target recognition device 1 to which the present invention is applied, learning processing is performed for the entire image such as a landscape. Therefore, even an image such as a landscape can recognize the main object with high recognition accuracy.

0053Further, according to the object recognition device 1 to which the present invention is applied, it is not necessary to cut out an image including an object that is a correct answer for the learning process, so that the amount of preprocessing of the learning process can be suppressed, for example, on the Internet or the like. The general image used can be used as a learning image, and the efficiency of the learning process can be improved. Further, according to the object recognition device 1 to which the present invention is applied, since it is not necessary to edit the learning image so that one learning image becomes one label, a dedicated teacher image is used for preprocessing of the learning process. There is no need to prepare, and the processing amount can be further suppressed.

0054Further, according to the object recognition device 1 to which the present invention is applied, among the plurality of regions set by the image conversion unit 12, a plurality of certainty degrees for a plurality of overlapping regions are statistically processed, so that the certainty degree is high. As the number of overlapping areas increases, it is possible to suppress erroneous recognition that the main target is included in the recognition target image, and it is possible to determine the main target in the recognition target image with higher recognition accuracy.

0055Further, according to the object recognition device 1 to which the present invention is applied, among the plurality of regions set by the image conversion unit 12, it is based on the area of the candidate region in addition to the plurality of certainty degrees for the plurality of overlapping regions. Since statistical processing is performed, it is possible to suppress erroneous recognition that the candidate area with a high degree of certainty and a large area is the main target included in the recognition target image, and the main target in the recognition target image can be suppressed with higher recognition accuracy. It can be determined.

0056Further, according to the object recognition device 1 to which the present invention is applied, the area of the area smaller than the predetermined area area is deleted from the plurality of areas set in the recognition target image, so that the area is based on the area area of each area. By utilizing the identification of the target included in the recognition target image, the main target in the recognition target image can be determined with high recognition accuracy even if the processing load is reduced.

0057Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

0058Further, a computer program for realizing the function of the target recognition device 1 described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. Good. The "computer system" referred to here may include hardware such as an OS and peripheral devices. The "computer-readable recording medium" is a writable non-volatile memory such as a flexible disk, a magneto-optical disk, a ROM, or a flash memory, a portable medium such as a DVD (Digital Versatile Disk), or a built-in computer system. A storage device such as a hard disk.

0059Furthermore, a "computer-readable recording medium" is a volatile memory inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line (for example, DRAM (Dynamic)). It also includes those that hold the program for a certain period of time, such as Random Access Memory)). Further, the program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned functions in combination with a program already recorded in the computer system.

00601 ... Target recognition device, 12 ... Image conversion unit, 14 ... Recognition calculation unit, 16 ... Map generation unit, 18 ... Position calculation unit, 20 ... Interface unit, 22 .. .Learning image storage unit, 200 ... image processing device, 202 ... image generation unit, 204 ... AR processing unit