Photographing apparatus

The present invention relates to a photographing device for photographing an object, and more particularly to a photographing device having a function of guiding an object to be photographed to an appropriate position or state.

[0002] A technique is known in which an object is photographed by using a photographing device, and the object is identified by using the image obtained by the photographing. (See, for example, Patent Document 1.) The system described in Patent Document 1 includes a camera and a display device for displaying an image taken by the camera. Then, the display device is provided with an image of the object to be photographed taken by the camera and a contour (reference contour) of the image that would be obtained if the object to be photographed is photographed at an appropriate position. , Overlay and display. As a result, the user can correct the position of the object to be photographed, and can obtain an image of the object to be photographed in a state of being arranged at an appropriate position.

[0003] Further, a technique of guiding an object to a predetermined position based on an image of the object taken by a camera is also known. (For example, refer to Patent Document 2.) However, the system described in Patent Document 2 is provided in a parking lot, etc., and uses a message display, a blinking lamp, etc. when a car is detected from a captured image. Then, the car is guided to the destination along a predetermined route. That is, this system does not guide the object to be photographed to a position where the optimum image can be taken.

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-273498 (see FIGS. 12 to 14, paragraphs 0070 to 0073) [Patent Document 2] Japanese Patent Application Laid-Open No. 64-55674 (Page 2) [0005] [ Problem to be Solved by the Invention In the method described in Patent Document 1, the user corrects the position of the object to be imaged while looking at the reference contour displayed on the display device. That is, in this method, a display device for displaying a reference contour and a captured image is an essential requirement.

[0006] Further, in this method, the user can move the object to be photographed to an appropriate position for photographing, but whether or not the object is in a suitable posture for photographing. It cannot be confirmed. Here, the "posture" is a concept including an angle or inclination of the object with respect to the camera.

[0007] An object of the present invention is to improve the identification accuracy when identifying an object by using an image obtained by photographing the object with a photographing device. It is also to guide the object to be photographed to an appropriate position or posture.

[Means for Solving the Problems] The photographing apparatus of the present invention includes a photographing means for photographing an object, a target photographing state storage means for storing target photographing state information representing the target shooting state of the object, and the target photographing. Guidance determination means for determining how to guide the object based on the state information and the image captured by the imaging means, and how to guide the object based on the determination result by the guidance determination means. It has a guidance instruction output means for instructing, and an image output means for outputting an image captured by the photographing means.

[0009] In this photographing device, how to guide the object is determined by comparing the state of the actually photographed image with the target photographing state. Then, based on the determination result, an instruction on how to guide the object is given. Therefore, if the object is moved or rotated according to the guidance instruction, the state of the object can be brought closer to the target shooting state. As a result, a good image according to the purpose can be obtained.

[0010] In the imaging device, the target position of the object may be defined as the target imaging state information, and the guidance determination means may determine the direction in which the object should be guided. In this case, since the object is guided to the target position and photographed, a good image can be obtained.

[0011] Further, in the imaging device, the target posture of the object may be defined as the target imaging state information, and the guidance determination means may determine the rotation direction in which the object should be guided. In this case, since the object is guided to the target posture and photographed, a good image can be obtained.

[0012] The photographing apparatus according to another aspect of the present invention includes a photographing means, a target photographing state storage means for storing target photographing state information representing a target photographing state when a background image is photographed by using the photographing means, and the target. Guidance determination means that determines the direction or angle in which the imaging means should be guided based on the imaging state information and the image captured by the imaging means, and the direction in which the imaging means should be guided based on the determination result by the guidance determination means. Alternatively, it has a guidance instruction output means for instructing an angle.

[0013] In this photographing device, since the position or angle of the photographing means is optimized, a good background image can be obtained, and an image of the object to be photographed can be accurately cut out from the photographed image.

[Embodiment of the Invention] FIG. 1 is a configuration diagram of an embodiment of a system in which a photographing apparatus according to the present invention is used. This system compares the image taken by the photographing device 1, the database 2 in which the image data of one or a plurality of objects is registered, and the image photographed by the photographing device 1 with each image registered in the database 2. The identification device 3 for identifying the object photographed by the photographing device 1 (or determining whether or not the object photographed by the photographing device 1 is registered in the database 2) is provided.

[0015] In order to improve the identification accuracy in this system, when an object is photographed, the object needs to be arranged at an appropriate position and in an appropriate posture. Therefore, the photographing apparatus according to the present invention has a function of guiding the object to be imaged to an appropriate position and posture. Then, the user of this system can move the object or rotate the object according to the above guidance. The "posture of an object" is a concept including an angle or inclination of the object with respect to the camera.

[0016] FIG. 2 is a configuration diagram of a photographing apparatus according to the first embodiment. The photographing device 10 of the first embodiment includes a photographing unit 11, a photographed image storage unit 12, a target shooting state storage unit 13, a guidance direction determination unit 14, a guidance direction instruction output unit 15, and a photographed image output control unit 16. ..

The photographing unit 11 is a camera and outputs image data of the captured image. The image data is not particularly limited, but is, for example, digital data in a bitmap format. The captured image storage unit 12 stores the image data output from the captured image storage unit 11.

[0018] The target shooting state storage unit 13 stores target shooting state information representing the target shooting state of the object to be photographed. Here, the target shooting state information includes information indicating a target position where the object to be photographed should be arranged at the time of shooting. The target position is a position where the characteristics of the object can be sufficiently detected if the object to be photographed is photographed at that position. In this case, the target shooting state information may be image data obtained in advance by shooting with the object to be photographed or an object equivalent thereto arranged at the target position.

[0019] The guidance direction determination unit 14 refers to the target shooting state information, determines whether or not the object to be photographed by the photographing unit 11 is arranged at an appropriate position, and the object to be photographed is appropriate. When it is not arranged at the position, it is determined in which direction the object to be photographed should be guided. The guidance direction determination unit 14 includes a feature extraction unit 17 that analyzes the image data stored in the captured image storage unit 12 and extracts the features thereof, and determines the object to be photographed based on the output of the feature extraction unit 17. It shall be recognizable.

[0020] The guidance direction instruction output unit 15 notifies the user of the determination result by the guidance direction determination unit 14. That is, when the object to be photographed is arranged at an appropriate position, the user is notified to that effect. On the other hand, when the object to be photographed is not arranged at an appropriate position, the user is presented with the direction in which the object to be photographed should be moved. This allows the user to place the object to be photographed in an appropriate position for photographing.

[0021] The captured image output control unit 16 outputs the image data stored in the captured image storage unit 12. Here, this image data is sent to, for example, the identification device 3 shown in FIG. The captured image output control unit 16 outputs the image data stored in the captured image storage unit 12 only when the guidance direction determination unit 14 determines that the object to be photographed is arranged at an appropriate position. You may try to do it. That is, when it is determined that the object to be photographed is arranged at an appropriate position, the image taken in that state is expected to be a sufficiently good image for identifying the object to be photographed. Therefore, if the image data is sent to the identification device 3 only in that case, the identification accuracy can be improved.

Next, the operation of the photographing apparatus 10 of the first embodiment will be described with reference to FIGS. 3 to 5. Here, in the target shooting state storage unit 13, as target shooting state information, a target image obtained by shooting with an object to be photographed or an object equivalent thereto arranged at a target position is stored in advance. It is assumed that

In the example shown in FIG. 3, the object to be photographed is arranged at a position shifted upward from the target position. In this case, first, the captured image obtained by the photographing unit 11 is stored in the captured image storage unit 12. Subsequently, the guidance direction determination unit 14 compares the captured image stored in the captured image storage unit 12 with the target image registered in the target shooting state storage unit 13 as the target shooting state information. As a result, it is detected that the object to be photographed is arranged at a position shifted upward with respect to the target position. Then, the guidance direction instruction output unit 15 outputs the detection result by the guidance direction determination unit 14. Specifically, for example, the message "Please move downward" is output.

[0024] In the example shown in FIG. 4, the object to be photographed is too close to the photographing unit 11. In this case, when the captured image stored in the captured image storage unit 12 and the target image registered in the target shooting state storage unit 13 are compared, the object to be photographed in the captured image is the target image. It is detected that it is larger than the object to be photographed inside. As a result, the guidance direction determination unit 14 detects that the object to be photographed is too close to the photographing unit 11. Then, the guidance direction instruction output unit 15 outputs the detection result by the guidance direction determination unit 14. Specifically, for example, the message "Keep away from the camera" is output.

In the example shown in FIG. 5, the object to be photographed is too far from the photographing unit 11. In this case, when the captured image stored in the captured image storage unit 12 and the target image registered in the target shooting state storage unit 13 are compared, the object to be photographed in the captured image is the target image. It is detected that it is smaller than the object to be photographed inside. As a result, the guiding direction determination unit 14 detects that the object to be photographed is too far from the photographing unit 11. Then, the guidance direction instruction output unit 15 outputs the detection result by the guidance direction determination unit 14. Specifically, for example, the message "Please bring it closer to the camera" is output.

[0026] As described above, when the photographing object 10 of the first embodiment is arranged at a position deviated from the target position, the photographing device 10 outputs a message for guiding the imaged object to the target position. To do. As a result, the user can easily move the object to be photographed to the target position. As a result, an image of the object taken in a good state can be obtained, so that the identification accuracy of the object is improved.

[0027] FIG. 6 is a flowchart showing the operation of the photographing apparatus 10 according to the first embodiment. It is assumed that the target shooting state information is registered in advance in the target shooting state storage unit 13.

[0028] In step S1, first, the photographing unit 11 takes a picture. Then, the image captured by the photographing unit 11 is stored in the captured image storage unit 12. In step S2, the "current shooting state" is calculated by analyzing the image stored in the shot image storage unit 12. The "shooting state" means the position of the object to be photographed in the examples shown in FIGS. 3 to 5.

[0029] In steps S3 and S4, the "current shooting state" obtained in step S2 and the "target shooting state" stored in the target shooting state storage unit 13 are compared. In the examples shown in FIGS. 3 to 5, the positions and sizes of the objects to be photographed in the image are compared. Then, when the "current shooting state" is within the range of the "target shooting state", it is considered that the object to be photographed is arranged at the target position, and the guidance process is terminated. On the other hand, when the "current shooting state" is not within the range of the "target shooting state", it is considered that the object to be photographed is not arranged at the target position, and the guidance processing of steps S5 to S6 is executed.

[0030] In step S5, the direction in which the object to be photographed should be guided is determined. Here, the guidance direction is calculated as "a direction that brings the" current shooting state "closer to the" target shooting state "". Specific examples are as described with reference to FIGS. 3 to 5. Then, in step S6, a message for notifying the user of the guidance direction is output. As a result, the user can bring the object to be photographed closer to the target position. After this, the process returns to step S1 and the next image is taken.

[0031] The process of the above flowchart is repeatedly executed until the object to be photographed is placed at the target position. Then, the guidance process ends when the object to be photographed is placed at the target position.

FIG. 7 is a configuration diagram of a photographing apparatus according to a second embodiment. The photographing device 20 of the first embodiment includes a photographing unit 11, a photographed image storage unit 12, a target shooting state storage unit 21, a guided attitude determination unit 22, a guided attitude instruction output unit 23, and a photographed image output control unit 16. .. The photographing unit 11, the photographed image storage unit 12, and the photographed image output control unit 16 are as described in the first embodiment.

[0033] The target shooting state storage unit 21 stores target shooting state information representing the target shooting state of the object to be photographed. Here, the target shooting state information includes information representing the target posture of the object to be photographed at the time of shooting. The target posture refers to a posture in which the characteristics of the object can be sufficiently detected if the object to be photographed is photographed in that posture. In this case, the target shooting state information may be image data obtained in advance by shooting the object to be photographed or an object equivalent thereto in the target posture.

[0034] The guided posture determination unit 22 refers to the target shooting state information, determines whether or not the object to be photographed by the photographing unit 11 is in an appropriate posture, and at the same time, the object to be photographed is in an appropriate posture. If not, it is determined how to rotate the object to be photographed. The guided posture determination unit 22 includes a feature extraction unit 17 that analyzes the image data stored in the captured image storage unit 12 and extracts the features thereof, and the imaged object is based on the output of the feature extraction unit 17. The posture should be recognizable.

[0035] The guidance posture instruction output unit 23 notifies the user of the determination result by the guidance posture determination unit 22. That is, when the object to be photographed is in an appropriate posture, the user is notified to that effect. On the other hand, when the object to be photographed is not in an appropriate posture, the direction and amount of rotation of the object to be imaged are presented to the user. As a result, the user can correct the object to be photographed into an appropriate posture for photographing.

Next, the operation of the photographing apparatus 20 of the second embodiment will be described with reference to FIGS. 8 to 9. In the example shown in FIG. 8, it is assumed that the shape of the object to be photographed is a cube. Further, it is assumed that the target shooting state storage unit 21 stores in advance the target image obtained by shooting the object to be photographed or an object equivalent thereto in the target posture as the target shooting state information. .. In this example, the target posture is a state in which one side of the cube faces the camera. Therefore, the shape of the object shown in the target image is a square.

[0037] In this example, the posture of the object to be photographed is tilted to the left. In this case, when the captured image stored in the captured image storage unit 12 and the target image registered in the target shooting state storage unit 21 are compared, the shape of the object shown in the target image is square. On the other hand, the shape of the object to be photographed in the photographed image is not square. At this time, the guidance direction determination unit 22 can recognize the tilt angle of the object to be photographed by analyzing the shape of the object to be photographed in the photographed image. That is, in this example, the guidance direction determination unit 22 detects that the object to be photographed is tilted to the left and the tilt angle thereof. Then, the guidance direction instruction output unit 23 outputs the detection result by the guidance direction determination unit 22. Specifically, for example, the message "Please tilt a little to the right" is output.

[0038] In the example shown in FIG. 9, it is assumed that the object to be photographed has a plate shape. Further, as shown in FIG. 9A, the photographing unit 11 shall output uniform irradiation light in the direction to be photographed. Further, it is assumed that the target shooting state storage unit 21 stores the target brightness distribution information as the target shooting state information. Here, the target brightness distribution refers to the brightness distribution of an image obtained by taking a picture with an object to be photographed or an object equivalent thereto in a target posture. In this example, the target posture of the object to be photographed is such that the irradiation light vertically hits a predetermined surface of the plate-shaped object to be photographed, as shown by the broken line in FIG. 9 (a). To do. Then, the target brightness distribution becomes information indicating that the brightness is constant over the entire object to be photographed, as shown by the broken line in FIG. 9 (b).

[0039] In this example, as shown in FIG. 9A, the object to be photographed is tilted with respect to the target posture. Specifically, the distance from the photographing unit 11 to the C region is shorter than the distance from the photographing unit 11 to the A region. Then, when the object to be photographed is photographed in this state, the brightness of the C region becomes higher than the brightness of the A region as shown in FIG. 9 (b). Therefore, the posture (here, the inclination) of the object to be photographed can be detected by comparing the brightness distribution obtained by this photographing with the target brightness distribution. After that, the guidance direction instruction output unit 23 outputs the detection result by the guidance direction determination unit 22 as in the example shown in FIG. Specifically, a guidance instruction to rotate the object to be photographed in the direction indicated by the arrow D in FIG. 9A is output.

[0040] In FIG. 9, for the sake of simplicity, the inclination of the object to be photographed is detected based on the one-dimensional brightness distribution, but the inclination of the object to be photographed is based on the two-dimensional brightness distribution. May be detected.

FIG. 10 is a flowchart showing the operation of the photographing apparatus 20 according to the second embodiment. Here, the processing of steps S11 to S14 is basically the same as that of steps S1 to S4 shown in FIG. However, the target shooting state storage unit 21 stores target posture information representing the target posture of the object to be photographed. Then, in steps S13 to S14, it is determined whether or not the posture of the object to be photographed is within the range of the target shooting state.

[0042] When the posture of the object to be photographed is outside the range of the target shooting state, in step S15, the direction in which the object to be photographed is rotated and the amount of rotation thereof are calculated. Here, this rotation direction / rotation amount is calculated so as to bring the "current shooting state" closer to the "target shooting state". Specific examples are as described with reference to FIGS. 8 to 9. Then, in step S6, a message for notifying the user of the rotation direction / rotation amount is output. As a result, the user can bring the posture of the object to be photographed closer to the target posture. After this, the process returns to step S1 and the next image is taken.

[0043] The process of the above flowchart is repeatedly executed until the posture of the object to be photographed reaches the target posture. Then, the guidance process ends when the posture of the object to be photographed reaches the target posture.

[0044] In the first and second embodiments described above, the position or orientation of the object to be photographed is detected, and it is determined whether or not to guide the object to be photographed based on the detection result. , The induction process may be performed in consideration of other factors.

[0045] For example, consider a system in which an object is photographed using a camera and the object is identified using a pattern detected by analyzing the photographed image. That is, it is assumed that the object to be identified has a pattern that can be detected by analyzing the captured image. In this case, the feature extraction unit 17 shown in FIG. 2 or FIG. 7 detects the pattern of the object to be photographed by analyzing the image obtained by the photographing unit 11. On the other hand, the target shooting state storage units 13 and 21 define a target value of the amount of patterns to be shot by the shooting unit 11 as the target shooting state information. Here, as the "amount of pattern", for example, when the pattern is a line pattern, "the number of lines" or "the sum of the lengths of each line" is defined.

[0046] The guidance direction determination unit 14 or the guidance posture determination unit 22 compares the amount of patterns detected by the feature extraction unit 17 with the amount of patterns defined as the target shooting state information. Then, if the amount of the detected pattern is smaller, the induction process may be continued, and when the amount of the detected pattern is larger, the induction process may be terminated.

FIG. 11 is a configuration diagram of a photographing apparatus according to a third embodiment. The photographing device 30 of the third embodiment includes a photographing unit 11, a photographed image storage unit 12, an object discrimination unit 31, an object image storage unit 32, a background image storage unit 33, a guidance instruction output unit 34, and a photographed image output control unit 35. To be equipped. The photographing unit 11 and the photographed image storage unit 12 are as described in the first embodiment.

[0048] The object discrimination unit 31 determines whether or not the image of the object to be photographed is included in the image captured by the photographing unit 11. Then, when the image of the object to be photographed is included in the image photographed by the photographing unit 11, the image is stored in the object image storage unit 32 as an object image. On the other hand, when the image of the object to be photographed is not included, the image is stored in the background image storage unit 33 as a background image. However, even if an image of the object to be photographed is included, if the image is taken in an inappropriate state, the image is discarded. The object discrimination unit 31 includes a feature extraction unit 17 that analyzes the image data stored in the captured image storage unit 12 and extracts the features thereof. Then, based on the output of the feature extraction unit 17, it is possible to recognize whether or not an image of the object to be photographed is included.

The guidance instruction output unit 35 creates and outputs an instruction for guiding the object to be photographed based on the discrimination result by the object discrimination unit 31. The captured image output control unit 35 cuts out an image of the object to be imaged by generating a difference image between the object image stored in the object image storage unit 32 and the background image stored in the background image storage unit 32. (Background difference processing). Then, the cut-out image of the object to be photographed is sent to the identification device 3 shown in FIG. When background subtraction processing is performed in the identification device 3, the captured image output control unit 35 sends both the object image and the background image to the identification device 3.

[0050] FIGS. 12 and 13 are diagrams illustrating the operation of the object discrimination unit 31. In the example shown in FIG. 12, the object to be photographed is arranged in the field of view of the photographing unit 11. Therefore, in this case, the image taken by the photographing unit 11 includes the image of the object to be photographed. Then, the object discrimination unit 31 stores this image as an object image in the object image storage unit 32. After that, the object determination unit 31 checks whether or not the background image is stored in the background image storage unit 33. Then, if the background image is not stored, the guidance instruction output unit 34 generates an instruction for guiding the object to be photographed out of the photographing field of view of the photographing unit 11.

[0051] According to this guidance instruction, the user moves the object to be photographed out of the field of view of photography as shown in FIG. Then, after that, the photographing unit 11 takes a picture. In the example shown in FIG. 13, the object to be photographed is arranged outside the photographing field of view of the photographing unit 11. Therefore, in this case, the image of the object to be photographed is not included in the image photographed by the photographing unit 11. Then, the object discrimination unit 31 stores this image as a background image in the background image storage unit 32. After that, the object discrimination unit 31 checks whether or not the object image is stored in the object image storage unit 32. Then, if the object image is not stored, the guidance instruction output unit 34 generates an instruction for guiding the object to be photographed within the photographing field of view of the photographing unit 11.

[0052] According to this guidance instruction, the user moves the object to be photographed within the field of view of photography as shown in FIG. Then, after that, the photographing unit 11 takes a picture. In this way, the photographing device 30 outputs an instruction to guide the object to be photographed to the user, if necessary, in order to acquire both the object image and the background image.

FIG. 14 is a diagram illustrating an example of a determination method by the object discrimination unit 31. It is assumed that the object to be photographed has a specific color (here, red) on the entire surface thereof. Further, it is assumed that the object discrimination unit 31 has a function of detecting the ratio (area ratio) of the red image to the entire image captured by the photographing unit 11.

[0054] In the example shown in FIG. 14 (a), the red image is not detected. In this case, the object discrimination unit 31 considers that the image of the object to be photographed does not exist in the image photographed by the photographing unit 11, and stores the image in the background image storage unit 33.

In the example shown in FIG. 14 (b), the ratio of the red image to the entire image is 8%. In this case, the object discrimination unit 31 determines that the object to be photographed is too far from the image capturing unit 11, and discards the image. When the ratio of the red image to the entire image is smaller than a predetermined value (for example, 20%), the object discriminating unit 31 determines that the object to be imaged is too far from the photographing unit 11.

In the example shown in FIG. 14 (c), the ratio of the red image to the entire image is 30%. In this case, the object determination unit 31 determines that the object to be photographed is arranged at the target position, and stores the image in the object image storage unit 32. The object discrimination unit 31 determines that the object to be photographed is arranged at the target position when the ratio of the red image to the entire image is within a predetermined range (for example, 20 to 40%).

In the example shown in FIG. 14 (d), the ratio of the red image to the entire image is 80%. In this case, the object discrimination unit 31 determines that the object to be photographed is too close to the image capturing unit 11, and discards the image. When the ratio of the red image to the entire image is larger than a predetermined value (for example, 40%), the object discrimination unit 31 determines that the object to be photographed is too close to the image capturing unit 11.

[0058] FIG. 15 is a flowchart showing the operation of the photographing apparatus 30 according to the third embodiment . In step S21, the image captured by the photographing unit 11 is stored in the captured image storage unit 12. In step S22, a predetermined image process (for example, color component analysis process) is extracted from the image stored in the captured image storage unit 12.

[0059] In step S23, the ratio (ratio A) of the image of the specific color component to the entire image is calculated. In step S24, the ratio A is compared with the preset threshold.

[0060] If the ratio A is "smaller than the threshold value T3", it is considered that the captured image does not include the image of the object to be photographed, and in step S25, the image is stored in the background image storage unit 33. Store in. The threshold value T3 is, for example, "5%" in the example shown in FIG. Subsequently, in step S26, it is checked whether or not the object image is stored in the object image storage unit 32. Then, when the object image is not stored, in step S27, a guidance instruction for guiding the object to be photographed into the photographing field of view of the photographing unit 11 is output. On the other hand, when the object image is stored, both the object image and the background image are acquired, so the process ends.

[0061] If the ratio A is "greater than the threshold value T1 and smaller than the threshold value T2", it is considered that the captured image includes the image of the object to be photographed, and in step S28, the ratio A is considered to be included. The image is stored in the object image storage unit 32. The threshold value T1 and the threshold value T2 are "20%" and "40%", respectively, in the example shown in FIG. Subsequently, in step S29, it is checked whether or not the background image is stored in the background image storage unit 33. Then, when the background image is not stored, in step S30, an instruction for guiding the object to be photographed out of the field of view of the photographing unit 11 is output. On the other hand, when the background image is stored, both the object image and the background image are acquired, so the process ends.

[0062] The user moves the object to be measured according to the guidance instruction in step S27 or S30. Then, the process returns to step S21, and the photographing unit 11 takes a picture. As a result, both the object image and the background image are acquired.

[0063] If the ratio A is "greater than the threshold value T3 and smaller than the threshold value T1" or "greater than the threshold value T2", the captured image does not include the image of the object to be photographed. However, it is judged that the object to be photographed is placed in an inappropriate position. In this case, the image is discarded in step S31. Then, in step S32, a process for guiding the object to be photographed to the target position is performed. That is, when the ratio A is larger than the threshold value T3 and smaller than the threshold value T1, a guidance instruction for bringing the object to be photographed closer to the photographing unit 11 is output. On the other hand, when the ratio A is larger than the threshold value T2, a guidance instruction for moving the object to be photographed away from the photographing unit 11 is output. Then, the process returns to step S21.

[0064] FIG. 16 is a configuration diagram of a photographing device according to a fourth embodiment. The photographing device 40 of the fourth embodiment includes a photographing unit 11, a photographed image storage unit 12, a distance measuring unit 41, an object discrimination unit 42, an object image storage unit 32, a background image storage unit 33, a guidance instruction output unit 34, and photographing. The image output control unit 35 is provided. Here, the photographing unit 11, the photographed image storage unit 12, the object image storage unit 32, the background image storage unit 33, the guidance instruction output unit 34, and the photographed image output control unit 35 will be described in the first or third embodiment. That's right.

[0065] The distance measuring unit 41 is, for example, a known distance sensor, and measures the distance between the photographing unit 11 and the object to be photographed. Similar to the object discrimination unit 31 of the third embodiment, the object discrimination unit 42 determines whether or not the image of the object to be photographed is included in the image captured by the photographing unit 11. Then, when the image of the object to be photographed is included in the image photographed by the photographing unit 11, the image is stored in the object image storage unit 32 as an object image. On the other hand, when the image of the object to be photographed is not included, the image is stored in the background image storage unit 33 as a background image. However, the object discrimination unit 42 determines whether or not the image of the object to be photographed is included in the photographed image based on the output of the distance measurement unit 41.

FIG. 17 is a flowchart showing the operation of the photographing apparatus 40 according to the fourth embodiment. The processes of steps S21 and S25 to S32 are as described with reference to FIG.

[0067] In step S41, the distance (distance A) between the photographing unit 11 and the object to be photographed is calculated based on the output of the distance measuring unit 41. In step S42, the distance A is compared with the preset threshold.

If the distance A is greater than the threshold value T3 (for example, 16 cm), it is considered that the captured image does not include the image of the object to be photographed, and the process proceeds to step S25. If the distance A is "greater than the threshold value T1 (for example, 4 cm) and smaller than the threshold value T2 (for example, 8 cm)", the captured image includes the image of the object to be photographed. It is considered that there is, and the process proceeds to step S28. Further, if the distance A is "smaller than the threshold value T1" or "larger than the threshold value T2 and smaller than the threshold value T3", the captured image includes the image of the object to be photographed. However, it is determined that the object to be photographed is placed at an inappropriate position. In this case, the process proceeds to step S31.

[0069] As described above, the operation of the photographing device 40 of the fourth embodiment is basically the same as that of the photographing device 30 of the third embodiment. However, in the fourth embodiment, the distance from the photographing unit 11 to the object to be photographed is directly measured by using a distance sensor or the like. Therefore, even if the distance cannot be estimated only by image analysis, it is possible to reliably determine whether or not the object to be photographed is captured.

[0070] Further, according to the photographing apparatus of the third and fourth embodiments, the object area and the background area can be separated, so that an image in which only the background image is changed while holding the image of the object to be photographed can be obtained. Can be generated.

[0071] In the first to fourth embodiments described above, a function of guiding the object to be photographed to an appropriate position or posture is provided, and the identification accuracy of the object is improved by utilizing this function. However, if the background image is not appropriate, even if the image showing the object to be photographed is analyzed, the object may not be accurately cut out. That is, if the background image is not appropriate, the identification accuracy of the object may decrease. Therefore, in order to improve the identification accuracy of the object, it is desirable to prepare a suitable background image in advance. Therefore, the photographing apparatus of the fifth embodiment described below has a function for acquiring a suitable background image in advance.

FIG. 18 is a configuration diagram of a photographing apparatus according to a fifth embodiment. The photographing device 50 of the fifth embodiment includes a photographing unit 11, a photographed image storage unit 12, a target background photographing state storage unit 51, a background photographing state determination unit 52, and a photographing device guidance instruction output unit 53. The photographing unit 11 and the photographed image storage unit 12 are as described in the first embodiment.

[0073] The target background shooting state storage unit 51 stores information representing a desirable state as a background image as the target background shooting state information. Here, in general, if the background is too bright, there is a risk that the image of the object to be photographed cannot be accurately cut out even if the image of the object to be photographed is analyzed. Further, even if the distribution of the brightness of the background is not uniform, there is a possibility that the image of the object cannot be accurately cut out. Therefore, in this case, as the target background shooting state information, for example, information that defines the brightness of the background image or information that defines the brightness distribution of the background image may be stored. Alternatively, it may be image data obtained by photographing an appropriate background (target background) prepared in advance.

[0074] The background shooting state determination unit 52 refers to the target background shooting state information, and determines whether or not the background image (an image in which the object to be photographed is not captured) obtained by the photographing unit 11 is appropriate. Then, when it is determined that the background image is not appropriate, it is determined how to guide the position or posture of the photographing unit 11. The background shooting state determination unit 52 includes a feature extraction unit 17 that analyzes the image data stored in the captured image storage unit 12 and extracts the features thereof, and the background image is based on the output of the feature extraction unit 17. It shall be possible to recognize the state (brightness distribution, etc.).

[0075] The photographing device guidance instruction output unit 53 outputs a guidance instruction to the user based on the determination result by the background photographing state determination unit 52. As a result, the user can move the photographing unit 11 to an appropriate position. Alternatively, the photographing device guidance instruction output unit 53 may move / rotate the position of the photographing unit 11 based on the determination result by the background photographing state determination unit 52.

Next, the operation of the photographing apparatus 50 according to the fifth embodiment will be described with reference to FIGS. 19 and 20. Here, it is assumed that shooting is performed in a state where the object to be photographed does not exist in the field of view of the photographing unit 11, and the image is stored in the photographed image storage unit 12. Hereinafter, this image will be referred to as a background image.

In the example shown in FIG. 19, the brightness of the upper region of the background image obtained by the photographing unit 11 is high, and the brightness of the lower region is low. In this case, when the background shooting state determination unit 52 analyzes the brightness distribution of this background image and detects that the brightness of the upper region thereof is higher than the brightness defined as the target background shooting state information, the shooting unit 11 is set. It is determined that the image should be guided downward (or the photographing unit 11 should be directed downward). Then, this determination result is notified to the user by the photographing device guidance instruction output unit 53 as a guidance instruction.

[0078] In the example shown in FIG. 20, the image of the fluorescent lamp is included in the background image obtained by the photographing unit 11. In this case, in this background image, the brightness of the region corresponding to the fluorescent lamp (in this example, the rightmost region) is increased. Then, the background shooting state determination unit 52 analyzes the brightness distribution of this background image, and when it detects that the brightness of the right end region is higher than the brightness defined as the target background shooting state information, the shooting unit 11 is left. It is determined that the image should be guided toward the direction (or the photographing unit 11 should be directed to the left). Then, this determination result is notified to the user by the photographing device guidance instruction output unit 53 as a guidance instruction.

[0079] The user moves or rotates the photographing unit 11 according to the guidance instruction from the photographing device guidance instruction output unit 53. Thereby, the photographing apparatus of the fifth embodiment can obtain an appropriate background image. That is, a background image having low lightness or a background image having a uniform lightness distribution can be obtained. Therefore, if the object to be identified is photographed in an environment where this background image can be obtained, the identification accuracy is improved.

[0080] FIG. 21 is a flowchart showing the operation of the photographing apparatus 50 according to the fifth embodiment. In step S51, the background image is captured by the photographing unit 11 and stored in the captured image storage unit 12. The "background image" refers to an image taken in a state where the object to be photographed does not exist in the field of view of the photographing unit 11.

[0081] In step S52, the brightness and the brightness distribution of the entire image are calculated by analyzing the background image stored in the captured image storage unit 12. This process is executed by the feature extraction unit 17. In steps S53 and S54, the brightness information stored in the target background shooting state storage unit 51 is referred to, and it is determined whether or not the brightness or brightness distribution of the background image calculated in step S52 is within the permissible range.

[0082] If the lightness or lightness distribution of the background image is not within the permissible range, in step S55, a background image having a lower lightness is obtained, or a background image having a uniform lightness distribution is obtained. Determine how to guide part 11. Then, in step S56, the determination result is output as a guidance instruction.

[0083] The user moves or rotates the photographing unit 11 according to the guidance instruction. Then, the process returns to step S51, and the next background image is taken. The processes of steps S51 to S56 are repeatedly executed until the brightness or brightness distribution of the background image obtained by the photographing unit 11 falls within the permissible range.

[0084] The functions provided by the photographing devices of the first to fifth embodiments can be arbitrarily combined. That is, for example, the position or angle of the photographing unit 11 is guided by the function provided by the fifth embodiment, and then the position or posture of the object to be photographed is guided by the function provided by the first to fourth embodiments. You may try to shoot while shooting. Further, for example, after guiding the object to be photographed to an appropriate position by the function provided by the first embodiment, the image is taken while correcting the posture of the object to be imaged by the function provided by the second embodiment. You may.

Next, a method of outputting a guidance instruction will be described. In the following, the description will be made based on the photographing apparatus of the first embodiment, but the present invention is not limited to this, and the present invention can be applied to the photographing apparatus of the second to fifth embodiments.

[0086] The photographing device shown in FIG. 22 includes a character string generation unit 61 and a display device 62. Then, the character string generation unit 61 generates a character string corresponding to the guidance instruction determined by the guidance direction determination unit 14. Further, the display device 62 displays the character string generated by the character string generation unit 61. This character string is a guide message instructing the moving direction, moving amount, rotation direction, and rotation angle of the object to be photographed or the photographing unit 11. Here, it is easy to display the generated character string immediately. Therefore, according to this method, the user can quickly correct the position and orientation of the object to be photographed or the photographing unit 11, and the processing time for identifying the object is shortened.

[0087] The photographing device shown in FIG. 23 includes a graphic / symbol generation unit 63 and a display device 62. Then, the figure / symbol generation unit 63 generates a figure / symbol corresponding to the guidance instruction determined by the guidance direction determination unit 14. Further, the display device 62 displays the graphic / symbol generated by the graphic / symbol generation unit 61. This figure / symbol is, for example, an arrow mark indicating a moving direction, a moving amount, a rotation direction, and a rotation angle of the object to be photographed or the photographing unit 11. Therefore, in this case, even a user (for example, a foreigner) who cannot read the message displayed by the method shown in FIG. 22 can quickly correct the position and posture of the object to be photographed or the photographing unit 11. ..

[0088] The photographing apparatus shown in FIG. 24 includes a voice guidance generator 64 and a speaker 65. Then, the voice guidance generation unit 64 generates voice guidance corresponding to the guidance instruction determined by the guidance direction determination unit 14, and outputs the voice guidance via the speaker 65. Here, this voice guidance is a message instructing the moving direction, the moving amount, the rotation direction, and the rotation angle of the object to be photographed or the photographing unit 11. Therefore, according to this method, even a visually impaired user can correct the position and orientation of the object to be photographed or the photographing unit 11.

The photographing apparatus shown in FIG. 25 includes a stereophonic sound generation unit 66 and a plurality of speakers 67a and 67b. Then, the stereophonic sound generation unit 66 generates a guide sound corresponding to the guidance instruction determined by the guidance direction determination unit 14, and outputs the guide sound via the speakers 67a and 67b. Here, this guide sound indicates the moving direction, the moving amount, the rotation direction, and the rotation angle of the object to be photographed or the photographing unit 11. Specifically, the guidance instruction is expressed by the direction in which the guide sound is heard, the volume of the guide sound, the pitch of the guide sound, the cycle of the guide sound when it is output periodically, and the like. For example, when you want to bring the object to be photographed closer to the photographing unit 11, a sound higher than the reference sound is output as a guide sound, and when you want to move the object to be photographed away from the photographing unit 11, the guide sound is higher than the reference sound. Also outputs a low sound. At this time, the pitch of the guide sound is assumed to continuously change according to the distance from the photographing unit 11 to the object to be photographed.

[0090] FIG. 26 is a diagram illustrating a method of guiding an object to be photographed by using a guide sound. In this example, the object to be photographed is currently located at point X. Then, it is assumed that the guidance direction determination unit 14 determines that the object to be photographed should be guided to the Y point. Further, it is assumed that the user can hear the synthetic sound of the sound A output from the speaker 67a and the sound B output from the speaker 67b. In this case, the volume of the speakers 67a and 67b is adjusted so that the localization direction of the synthesized sound with respect to the user is the direction from the user to the Y point. As a result, the user can hear the guide sound from the direction of the Y point, so that the object to be photographed at the X point can be moved to the Y point.

FIG. 27 is a block diagram of a computer that provides guidance instruction functions (guidance direction determination unit 14, guidance posture determination unit 22, object determination unit 31, 42, etc.) according to the present invention. The guidance instruction function according to the present invention is realized by executing a program that describes the processing of the above-mentioned flowchart using a computer.

[0092] The CPU 101 loads the program describing the process shown in the above flowchart from the storage device 102 into the memory 103 and executes the program. The storage device 102 is, for example, a hard disk and stores the above program. The storage device 102 may be an external storage device connected to the computer 100. The memory 103 is, for example, a semiconductor memory and is used as a work area of the CPU 101. Here, the target shooting state information and the target background shooting state information are stored in, for example, the storage device 102. Further, the captured image storage unit 12, the object image storage unit 32, and the background image storage unit 33 are realized by, for example, the memory 103.

[0093] The recording medium driver 104 accesses the portable recording medium 105 according to the instruction of the CPU 101. The portable recording medium 105 includes, for example, a semiconductor device (PC card, etc.), a medium (flexible disk, magnetic tape, etc.) to which information is input / output by magnetic action, and a medium (optical disk, etc.) to which information is input / output by optical action. Etc.) shall be included. The communication control device 106 transmits / receives data via the network according to the instruction of the CPU 101. The output device 107 is, for example, a display device or a speaker, and outputs the generated guidance instruction.

[0094] FIG. 28 is a diagram illustrating a method of providing a software program according to the present invention. The program according to the present invention is provided, for example, by any of the following three methods.

(1) Provided by being installed on a computer. In this case, the program is, for example, pre-installed on the computer 100 before it ships.

(2) Portability Provided by being stored in a recording medium. In this case, the program stored in the portable recording medium 105 is basically installed in the storage device 102 via the recording medium driver 104.

(3) Provided by a program server provided on the network. In this case, the computer 100 acquires the corresponding program by downloading it from the program server. However, the computer 100 may request the server device to execute the above program and receive the execution result.

(Appendix 1) Photographed by a shooting means for photographing an object, a target shooting state storage means for storing target shooting state information representing the target shooting state of the object, the target shooting state information, and the shooting means. Guidance determination means for determining how to guide the object based on an image, guidance instruction output means for instructing how to guide the object based on the determination result by the guidance determination means, and An imaging device having an image output means for outputting an image captured by the imaging means.

(Appendix 2) In the imaging apparatus according to Appendix 1, the target imaging state information includes information indicating a target position of the object, and the guidance determination means indicates a direction in which the object should be guided. judge.

(Appendix 3) In the imaging apparatus according to Appendix 1, the target imaging state information includes information indicating a target posture of the object, and the guidance determination means is a rotation direction in which the object should be guided. To judge.

(Appendix 4) An object discriminating means for determining whether or not an image of an object to be photographed is included in an image photographed by the photographing means, which is the photographing apparatus according to Appendix 1. Further, there is an object image storage means for storing an image when the image of the object is included as an object image, and a background image storage means for storing the image when the image of the object is not included as a background image. Then, the image output means cuts out an image of the object based on the object image and the background image, and outputs the cut out image.

(Appendix 5) An object discriminating means for determining whether or not an image of an object to be imaged is included in an image photographed by the photographing means, which is the photographing apparatus according to Appendix 1. Further, there is an object image storage means for storing an image when the image of the object is included as an object image, and a background image storage means for storing the image when the image of the object is not included as a background image. Then, the image output means outputs both the object image and the background image.

(Appendix 6) In the imaging apparatus according to Appendix 4 or 5, when the object image is not stored in the object image storage means, the guiding determination means uses the imaging means to capture the object. The object is guided so that an image including the object is taken, and when the background image is not stored in the background image storage means, the object is taken so that the image containing the object is taken by the taking means. Guide the object.

(Appendix 7) The imaging apparatus according to Appendix 1, wherein the imaging means measures the distance between the imaging means and the object, and the imaging means is based on the measurement results of the measuring means. An object discrimination means for determining whether or not an image of an object to be photographed is included in the captured image, and an object image storage means for storing the image when the image of the object is included as an object image. And a background image storage means for storing an image when the image of the object is not included as a background image, and the image output means stores the image of the object based on the object image and the background image. Cut out and output the cut out image.

(Appendix 8) The imaging device according to Appendix 1, wherein the imaging means measures the distance between the imaging means and the object, and the imaging means is based on the measurement results of the measuring means. An object discrimination means for determining whether or not an image of an object to be photographed is included in the captured image, and an object image storage means for storing the image when the image of the object is included as an object image. Further, the background image storage means for storing the image when the image of the object is not included as the background image is further provided, and the image output means outputs both the object image and the background image.

(Appendix 9) In the photographing apparatus according to Appendix 7 or 8, when the object image storage means does not store the object image, the guiding determination means means the object by the photographing means. The object is guided so that an image including the object is taken, and when the background image is not stored in the background image storage means, the object is taken so that the image containing the object is taken by the taking means. Guide the object.

(Appendix 10) In the photographing apparatus according to Appendix 1, the image output means was photographed by the photographing means when it was determined that the guidance determining means did not need to guide the object. Output the image.

(Appendix 11) In the photographing apparatus according to Appendix 1, the object has a predetermined pattern, and the amount of the pattern to be photographed by the photographing means is defined as the target photographing state information. The guidance determination means is used when the amount of the pattern detected from the image of the object photographed by the photographing means is larger than the amount of the pattern defined as the target photographing state information. , It is determined that it is not necessary to guide the object.

(Appendix 12) Based on a photographing means for photographing an object of a specific color, a detecting means for detecting the ratio of an image of the specific color to the entire image photographed by the photographing means, and a detection result by the detecting means. An imaging device having a guidance instruction output means for instructing a direction in which the object should be guided and an image output means for outputting an image captured by the imaging means.

(Appendix 13) A direction in which the object should be guided based on the photographing means for photographing the object, the measuring means for measuring the distance between the photographing means and the object, and the measurement result by the measuring means. An imaging device having a guidance instruction output means for instructing the above and an image output means for outputting an image captured by the imaging means.

(Appendix 14) The shooting means, the target shooting state storage means for storing the target shooting state information representing the target shooting state when the background image is shot by the shooting means, the target shooting state information, and the above. The guidance determination means for determining the direction or angle in which the imaging means should be guided based on the image captured by the imaging means, and the direction or angle in which the imaging means should be guided based on the determination result by the guidance determination means. An imaging device having a guidance instruction output means for instructing.

(Appendix 15) The photographing apparatus according to Appendix 14, further comprising a moving means for moving the photographing means according to an instruction output from the guidance instruction output means.

(Appendix 16) The photographing apparatus according to any one of Supplementary note 1 to 11, 14, and 15, wherein the guidance instruction output means includes a display means and corresponds to a determination result by the guidance determination means. Display the character string to be displayed on the display means.

(Appendix 17) The photographing apparatus according to any one of Supplementary note 1 to 11, 14, and 15, wherein the guidance instruction output means includes a display means and corresponds to a determination result by the guidance determination means. Display the figure or symbol to be displayed on the display means.

(Appendix 18) The imaging device according to any one of Supplementary notes 1 to 11, 14, and 15, wherein the guidance instruction output means outputs voice guidance corresponding to a determination result by the guidance determination means. To do.

(Appendix 19) The imaging device according to any one of Supplementary notes 1 to 11, 14, and 15, wherein the guidance instruction output means generates a stereophonic sound corresponding to a determination result by the guidance determination means. To do.

(Appendix 20) A method of guiding an object to be photographed by using a photographing device, which is a target photographing state in which an object is photographed by using the photographing device and represents a target photographing state of the object prepared in advance. An object that determines how to guide the object based on the information and the image taken by the photographing device, and outputs a guidance instruction on how to guide the object based on the determination result. Guidance method.

(Appendix 21) A method of photographing an object using a photographing device, which is a first step of photographing an object using the photographing device, and a target representing a target photographing state of the object prepared in advance. The second step of determining how to guide the object based on the shooting state information and the image taken by the shooting device, and how to guide the object based on the determination result. A photographing method including a third step of outputting a guidance instruction and a fourth step of repeating the first to third steps until it is determined that it is not necessary to guide the object.

[Effect of the Invention] According to the present invention, since the object to be photographed can be guided to an appropriate position and posture, a good image for identifying the object can be obtained. Therefore, the accuracy of identifying the object is improved.

[0120] Further, since the position or angle of the photographing device is guided so that an appropriate background image can be obtained, the image of the object to be photographed can be accurately extracted, which also improves the accuracy of identifying the object. To do.

BRIEF DESCRIPTION OF THE DRAWINGS [Fig. 1] Fig. 1 is a configuration diagram of an example of a system in which a photographing apparatus according to the present invention is used.

FIG. 2 is a configuration diagram of a photographing apparatus according to the first embodiment.

FIG. 3 is a diagram (No. 1) for explaining the operation of the photographing apparatus according to the first embodiment.

FIG. 4 is a diagram (No. 2) for explaining the operation of the photographing apparatus according to the first embodiment.

FIG. 5 is a diagram (No. 3) for explaining the operation of the photographing apparatus according to the first embodiment.

FIG. 6 is a flowchart showing the operation of the photographing apparatus according to the first embodiment.

FIG. 7 is a configuration diagram of a photographing apparatus according to a second embodiment.

FIG. 8 is a diagram (No. 1) for explaining the operation of the photographing apparatus according to the second embodiment.

FIG. 9 is a diagram (No. 2) for explaining the operation of the photographing apparatus according to the second embodiment.

FIG. 10 is a flowchart showing the operation of the photographing apparatus of the second embodiment.

FIG. 11 is a configuration diagram of a photographing apparatus according to a third embodiment.

FIG. 12 is a diagram (No. 1) for explaining the operation of the object discrimination unit.

FIG. 13 is a diagram (No. 2) for explaining the operation of the object discrimination unit.

FIG. 14 is a diagram illustrating an example of a determination method by an object discrimination unit.

FIG. 15 is a flowchart showing the operation of the photographing apparatus according to the third embodiment.

FIG. 16 is a configuration diagram of a photographing apparatus according to a fourth embodiment.

FIG. 17 is a flowchart showing the operation of the photographing apparatus according to the fourth embodiment.

FIG. 18 is a configuration diagram of a photographing apparatus according to a fifth embodiment.

FIG. 19 is a diagram (No. 1) for explaining the operation of the photographing apparatus according to the fifth embodiment.

FIG. 20 is a diagram (No. 2) for explaining the operation of the photographing apparatus according to the fifth embodiment.

FIG. 21 is a flowchart showing the operation of the photographing apparatus according to the fifth embodiment.

FIG. 22 is a diagram (No. 1) for explaining a method of outputting a guidance instruction.

FIG. 23 is a diagram (No. 2) for explaining a method of outputting a guidance instruction.

FIG. 24 is a diagram (No. 3) for explaining a method of outputting a guidance instruction.

FIG. 25 is a diagram (No. 4) for explaining a method of outputting a guidance instruction.

FIG. 26 is a diagram showing a specific example of the method shown in FIG. 25.

FIG. 27 is a block diagram of a computer that provides a guidance instruction function according to the present invention.

FIG. 28 is a diagram illustrating a method of providing a software program according to the present invention.

[Explanation of symbols] 10, 20, 30, 40, 50 Imaging device 11 Imaging unit 12 Imaging image storage unit 13, 21 Target imaging state storage unit 14 Guidance direction determination unit 15 Guidance direction instruction output unit 16, 35 Capturing image output control Part 22 Guidance posture determination unit 23 Guidance posture instruction output unit 31, 42 Object discrimination unit 32 Object image storage unit 33 Background image storage unit 34 Guidance instruction output unit 41 Distance measurement unit 51 Target background shooting state storage unit 52 Background shooting state judgment unit 53 Imaging device guidance output unit