Workpiece removing device and workpiece removing method

The present invention relates to a work taking-out device and a work taking-out method for taking out works bulked in a container by a robot.

A device that captures the entire area of multiple workpieces randomly arranged (bulk) in a container with a camera, detects the workpieces based on the camera image, and automatically takes out the workpieces with a robot manipulator. It is known (see, for example, Patent Document 1). In the apparatus described in Patent Document 1, it is determined whether or not the loading state of the work in the container has changed, and when it is determined that the loading state has not changed, the work is imaged at the time of the next work taking out operation. Instead, by detecting the work based on the camera image taken in the past, the image taken by the camera is omitted, and the time for taking out the work is shortened.

<p><patcit num="1"><text>Japanese Patent No. 4199264</text></patcit></p>

<p> However, in the apparatus described in Patent Document 1, since there are few cases where the loaded state of the work has not changed in the work taking out work, there are not many opportunities to omit imaging, and when the loaded state of the work in the container has changed. Since it is necessary to re-image the entire area of a plurality of workpieces with a camera and detect the workpieces from the camera images, it is difficult to efficiently take out the workpieces.</p>

<p> The work taking-out device according to the present invention includes a camera that images a work loading area including a plurality of works stacked separately, a work detection unit that detects a work based on a camera image captured by the camera, and a detection result by the work detection unit. A work selection unit that selects the work to be taken out based on the above, a robot that takes out the work selected by the work selection unit, and a loading state determination unit that determines whether or not the loading state of the work has changed due to the operation of the robot. The work detection unit includes an area setting unit for setting a work detection area when detecting a work, and the area setting unit determines that the load state of the work has changed by the load state determination unit. It is a peripheral area of the change position of, and is characterized in that a work detection area is set as a part of the work loading area. Further, the work taking-out method according to the present invention includes an imaging procedure in which a work loading area including a plurality of loosely stacked workpieces is imaged by a camera, a workpiece detection procedure in which a workpiece is detected based on a camera image captured by the camera, and a workpiece. The work selection procedure that selects the work to be taken out based on the detection result by the detection procedure, the work take-out procedure that takes out the work selected by the work selection procedure by the robot, and whether or not the loading state of the work has changed due to the operation of the robot. In the area setting procedure, the load state of the work is changed by the load state determination procedure, including the load state determination procedure for determining the load state and the area setting procedure for setting the work detection area when the work is detected by the work detection procedure. When it is determined, it is a peripheral area of the loading state change position, and the work detection area is set as a part of the work loading area.</p>

<p> According to the present invention, since the work detection area is set in the peripheral area of the position where the load state of the work is changed, the work detection area is narrower than the case where the work detection area is set in the entire work loading area, and the work is Since the processing time required for detection can be greatly reduced, the robot has almost no time to wait for the image and detection of the work, and the work can be taken out efficiently.</p>

<figref num="1">It is a figure which shows the whole structure of the work taking-out apparatus which concerns on embodiment of this invention.</figref><figref num="2">It is a flowchart which shows an example of the process executed by the CPU in the robot controller of FIG.</figref><figref num="3">It is a figure which shows the detail of the work detection area setting process of FIG.</figref><figref num="4">It is a figure for demonstrating the method of the automatic change of the size of the work detection area.</figref><figref num="5">It is a figure for demonstrating operation of the work taking-out apparatus which concerns on embodiment of this invention.</figref>

Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 5. FIG. 1 is a diagram showing an overall configuration of a work taking-out device 1 according to an embodiment of the present invention. A plurality of works 5 of the same type are randomly arranged in the container 6, that is, they are piled up in bulk. The work taking-out device 1 includes a robot 2 that takes out a work 5 selected from the plurality of works 5 that are piled up in bulk, a camera 7 that is fixedly arranged above the container 6, and a camera that is imaged by the camera 7. It has a robot controller 8 that controls the robot 2 based on an image.

The robot 2 is an articulated robot having a rotatable joint axis, and the work 5 is gripped by a robot hand 2b provided at the tip of the robot arm 2a. Further, the robot hand 2b is provided with a visual sensor 3, and each work 5 is measured by the visual sensor 3. The visual sensor 3 is a laser projection type three-dimensional visual sensor, and is controlled by the visual sensor control unit 8h in the robot controller 8. The measurement data obtained by the visual sensor 3 is stored in the memory 8g in the robot controller 8, and the detailed three-dimensional position and orientation of the work 5 are obtained by the processing in the controller 8. The robot arm 2a is provided with an impact sensor 4, and the impact sensor 4 detects the impact acting on the robot arm 2a when the work is taken out.

The camera 7 is an electronic camera having an image pickup element such as a CCD, and is a well-known light receiving device having a function of detecting a two-dimensional image on a light receiving surface (on the CCD array surface) by imaging. The imaging operation of the camera 7 is controlled by the camera control unit 8f in the robot controller 8, and the imaging range is set so that the workpiece loading area (for example, the entire container) including the plurality of workpieces 5 is within the field of view. The camera image is taken into the camera control unit 8f and stored in the memory 8g.

The robot controller 8 has a work detection unit 8a that detects the work 5 based on the image stored in the memory 8g, and a work selection unit 8b that selects the work 5 to be taken out from the work 5 detected by the work detection unit 8a. When the workpiece loading state determining unit 8c determines whether the loading state of the workpiece 5 in the container 6 is changed, the workpiece loading state change position storing position where the loading state of the workpiece 5 is changed and the location storage portion 8d , The work detection area setting unit 8e that sets the work detection area when the work 5 is detected by the work detection unit 8a, and these together with the camera control unit 8f and the memory 8g, provide an image processing device that processes the camera image. Configure.

The work detection unit 8a detects the work 5 by pattern matching, for example. That is, the work 5 is detected by generating a work model corresponding to the work shape in advance and searching and extracting the target image corresponding to this work model from the camera images in the work detection area. In such processing in the work detection unit 8a, if the work detection area is wide, it takes time to search for the target image, and the robot waits for the work take-out work until the search for the target image is completed, so that the work take-out work is efficient. It becomes difficult to do it in a targeted manner. Therefore, in the present embodiment, the work taking-out work time is shortened by setting the work detection area as follows.

FIG. 2 is a flowchart showing an example of processing executed by the CPU in the robot controller 8. The process shown in this flowchart is started, for example, when a start command for work removal work is input. In step S1, the camera control unit 8f outputs an imaging command to the camera 7 so as to image the work 5 in the work loading area in the container, and stores the camera image obtained by the imaging in the memory 8g.

In step S2, the work 5 is detected by using the camera image in the work detection area among the camera images stored in the memory 8g by the processing (pattern matching) in the work detection unit 8a, and the detected work 5 is detected. Store in memory 8g. In the initial state immediately after the start of the process of FIG. 2, the work detection area is set to surround the entire container according to the work loading area. When a plurality of work detection areas are set in a part of the work loading area as described later (FIG. 5 (b)), the work 5 is detected for each work detection area.

In step S3, it is determined whether or not one or more work 5s are detected by the processing in step S2. If step S3 is denied, the process proceeds to step S4, and it is determined whether or not the predetermined end condition is satisfied. For example, when the number of fetched works 5 reaches a predetermined number, the end condition is determined to be satisfactory. If step S4 is affirmed, the process ends. If step S4 is denied, the process proceeds to step S5, a work detection area that surrounds the entire container is set, and the process returns to step S1. On the other hand, if step S3 is affirmed, the process proceeds to step S6.

In step S6, the work 5 to be taken out by the robot 2 is selected from all the work 5 in the container stored in the memory 8g in step S2 by the processing in the work selection unit 8b. In this case, for example, the work 5 which is higher than the surrounding work 5 and is not concealed is selected as the work 5 to be taken out.

In step S7, it is determined whether or not the work 5 can be selected by the process in step S6. If step S7 is denied, the process proceeds to step S8, and the detection condition and selection condition of the work 5 are changed so that the work 5 can be selected. For example, the amount of light at the time of imaging, the work pattern in pattern matching, etc. are changed, and the process returns to step S1. If step S7 is affirmed, the process proceeds to step S9.

In step S9, a control signal is output to the servomotor for driving the robot, the operation of the robot 2 (robot arm 2a or robot hand) is controlled, and the selected work 5 is taken out from the container. In this case, the moving position of the visual sensor 3 provided at the tip of the robot arm is calculated, the three-dimensional position and posture of the selected work 5 are measured by the visual sensor 3, and the robot hand is moved to the target position for taking out the work. After moving, the work 5 is grasped by the robot hand and taken out.

In step S10, whether or not an impact is detected by the impact sensor 4 while the robot 2 is operating by the process of the work loading state determination unit 8c, that is, before the selected work 5 is gripped, the robot 2 Determines whether or not an impact is generated due to contact with a work or the like. Instead of the impact sensor 4, it is possible to detect a sudden change in the load acting on the servomotor for driving the robot by a change in the motor current or the like, and thereby determine the presence or absence of an impact.

If step S10 is affirmed, the process proceeds to step S11. In this case, it is highly possible that the load state of the work 5 in the container, especially in the vicinity of the work 5 to be taken out, has changed due to the occurrence of the impact. Therefore, in step S11, the position of the work 5 to be taken out is stored as the load state change position by the process in the work loading state change position storage unit 8d, and the process returns to step S6 to redo the selection process of the work 5. In this redoing work selection process, the work 5 at a position distant from the loading state change position, that is, the work 5 to be taken out from the work 5 whose loading state is considered to have not changed may be selected. The loading state change position is a three-dimensional position of the work 5 detected by the visual sensor 3, and is stored as position data of the robot coordinate system.

In step S10, when an impact is detected when moving the robot hand to the position of the work 5 to be taken out, the loading state changes at the position where the robot 2 is impacted, not near the work 5 to be taken out. Probability is high. In this case, in step S11, the position of the tip of the robot arm when the robot 2 receives an impact may be stored in the memory 8 g as the load state change position. The position of the tip of the robot arm can be detected by various position sensors provided on the robot 2.

If step S10 is denied, the process proceeds to step S12. In step S12, it is determined whether or not the work 5 to be taken out by the robot hand can be grasped by the process of the work loading state determination unit 8c. Specifically, the success or failure of the gripping operation is determined based on the detection value of the open / close confirmation sensor of the hand chuck and, in the case of the robot hand using a suction cup, the detection value of the suction confirmation sensor. It is also possible to determine the success or failure of the gripping operation based on the detection value of the proximity sensor or the like that detects whether the work 5 is in the correct position with respect to the robot hand.

If step S12 is affirmed, the process proceeds to step S13. In this case, it is highly possible that the loading state of the work 5 in the container, particularly the loading state in the vicinity of the removal position of the work 5, has changed by grasping and removing the work 5. Therefore, in step S13, the position of the selected work 5 to be taken out is stored as the load state change position by the processing in the work loading state change position storage unit 8d, and the process proceeds to step S14. If the actual position of the work 5 deviates from the position of the selected work 5, the robot hand may be moved downward, for example, from the position of the selected work 5 to grip the work 5. In consideration, the position where the work 5 is actually gripped may be stored in the memory 8 g as the load state change position. If step S12 is denied, the process proceeds to step S14.

In step S14, the work detection area setting process shown in FIG. 3 is executed by the process in the work detection area setting unit 8e. First, in step S14a, it is determined whether or not the loading state change position exists, that is, whether or not the process of either step S11 or step S13 is executed. If step S14a is affirmed, the process proceeds to step S14b to acquire the load state change position in the robot coordinate system stored in the work load state change position storage unit 8d. In step S14c, the load state change position of the robot coordinate system is converted to the position on the image. Specifically, using the calibration data of the camera 7, the loading state change position is converted to the position on the camera image by a well-known method.

In step S14d, a work detection area having a predetermined shape and size is set at the loading state change position on the image. For example, when the work detection area is circular, the diameter or radius of the circle centered on the load state change position may be set, and when the work detection area is rectangular, the vertical length of the rectangular window centered on the load state change position. The directional length and the lateral length may be set. In any case, the work detection area is set to at least a part of the work loading area imaged by the camera 7, that is, a range narrower than the work loading area. If there are multiple loading state change positions, set the work detection area for each position.

When the process of step S14d is completed, the process returns to step S1 of FIG. 2 and a series of processes are repeated. In the repetitive processing, the work 5 is detected from the work detection area set as a part of the work loading area in step S2, and the work detection data in the work detection area is rewritten by this new work detection data. Store in memory 8g. That is, the work 5 is detected by using the area around the position where the load state of the work 5 has changed as the work detection area, and the work detection data in that area is updated. Therefore, since it is not necessary to detect the work 5 over the entire area of the work loading area, it is possible to shorten the processing time in the work detection unit 8a.

On the other hand, if it is determined in step S14a that the loading state change position does not exist, the process proceeds to step S14e. In step S14e, the work detection area is set in the entire work loading area that surrounds the entire container, and the process returns to step S2 in FIG. In this case, since the loading state of the work 5 has not changed, it is not necessary to take an image of the camera 7 again, and in step S2 by the repeated process, the work 5 is detected from the entire container.

By the way, regarding the setting process of the work detection area in step S14d, the size on the image of the work 5 captured by the camera 7 is the distance from the camera 7 to the work 5, that is, the height of the work 5 arranged in the container. It changes according to. In consideration of this point, the size of the work detection area may be automatically changed according to the height of the work 5. That is, when the work 5 is close to the camera 7, the work detection area is large, and when the work 5 is far from the camera 7, the work detection area is small, and the size of the work 5 included in the work detection area on the image is large. May be constant. This point will be described below.

FIG. 4 is a diagram for explaining a method of automatically changing the size of the work detection area. Here, the height direction of the work 5 is defined in the Z-axis direction in the figure. First, the image sizes SZ1 and SZ2 to be captured in the image captured by the camera 7 when the work 5 is at the heights of Z1 and Z2 from the reference position are set, respectively. At this time, the size reflected in the image is inversely proportional to the distance of the work 5 from the camera 7, so the distance from the reference position to the camera 7 (camera height) is Z0, and the distance from the camera 7 to the heights Z1 and Z2 is up to each work 5. Assuming that the distances of are A1 and A2, respectively, the following equation (I) holds. SZ2 / SZ1 = A1 / A2 = (Z0-Z1) / (Z0-Z2) (I)

The camera height Z0 can be calculated by the following equation (II), and assuming that the height of the loading state change position is Z3, the image size SZ3 at that time can be calculated by the following equation (III). Z0 = (SZ2 * Z2-SZ1 * Z1) / (SZ2-SZ1) (II) SZ3 = ((Z0-Z1) / (Z0-Z3)) * SZ1 (III)

When the work detection area is, for example, circular, the diameter D1 of the work detection area corresponding to the image size SZ1 is determined in advance. In step S14d, using this D1, the diameter D3 of the work detection region corresponding to the image size SZ3 is calculated by the following equation (IV). D3 = (SZ3 / SZ1) * D1 = ((Z0-Z1) / (Z0-Z3)) * D1 (IV)

The operation of the work taking-out device 1 according to the embodiment of the present invention will be described more specifically. For example, suppose that a plurality of works 5 are arranged in bulk in the container 6 as shown in FIG. 5 (a). In this case, first, the work detection area that surrounds the entire container is set, and after work 5 is detected in the entire container (step S2), the work 5a that is higher than the surrounding work 5 and is not hidden is , Selected as the work to be taken out by Robot 2 (step S6).

When the selected work 5a is taken out, for example, if the robot hand comes into contact with the work 5a before the robot hand grips the work 5a, the position of the work 5a shifts as shown in FIG. 5 (b). In this case, the impact is detected by the impact sensor 4, the position of the selected work 5a is stored as the load state change position (step S11), and the work 5 is selected again (step S6). At this time, as shown in FIG. 5A, when a work 5b different from the previous one is selected, the work 5b is taken out by the robot 2, and the position of the work 5b is stored as the load state change position ( Step S13).

In this case, as shown in FIG. 5 (b), the work detection areas 12a and 12b are set in the areas including the work 5a set at the time of impact detection and the work 5b selected at the time of taking out by the robot 2 (step). S14d). Then, after the entire inside of the container is imaged by the camera 7, the work 5 is detected in the work detection areas 12a and 12b using the camera image (step S2). As a result, the work detection area of the work 5 is limited, so that the time required for the work detection can be shortened, and the work removal work can be performed efficiently.

If the impact sensor 4 does not detect the impact when the work is taken out and the robot 2 does not detect the gripping motion of the work 5, the work detection area surrounding the entire container is set because there is no load state change position (step). S14e). Even if work 5 is not detected in the work detection area set in step S14d (for example, when only the search window 12b in Fig. 5 (b) is set), the work detection area that surrounds the entire container is set (step). S5). In these cases, work 5 is detected again from the entire container.

According to this embodiment, the following effects can be obtained. (1) The work loading status determination unit 8c determines whether or not the loading status of the work 5 has changed, and when it is determined that the loading status of the work 5 has changed, the work detection area setting unit 8e determines. The work detection area was set in the peripheral area of the change position of the loaded state by the process of, and the work 5 was detected in the work detection area by the process of the work detection unit 8a. Therefore, the work 5 is detected in a part of the work loading area, and the work detection area can be narrowed. Therefore, the time required for the work detection can be shortened, and the work removal work can be performed efficiently. ..

(2) When it is determined that the gripping operation of the work 5 by the robot 2 is successful, it is determined that the loading state of the work 5 has changed, so that the change in the loading state after the work 5 is taken out can be correctly determined. .. In this case, since the work detection area is set in the peripheral area of the position of the work 5 selected by the work selection unit 8b, the work detection process can be performed without waste in the place where the loading state is likely to change. Instead of this, if the work detection area is set in the peripheral area of the position where the work 5 is gripped by the robot 2, the actual position of the work 5 is deviated from the selected work 5 position. , Work detection processing can be performed without waste.

(3) If it is determined that an impact has acted on the robot 2 before the robot 2 grips the work 5, it is determined that the loading state of the work 5 has changed. Can be correctly judged for changes in. In this case, since the work detection area is set in the peripheral area of the position of the work 5 selected by the work selection unit 8b, the work detection process can be performed without waste in the place where the loading state is likely to change. Instead of this, if the work detection area is set in the area around the position where the impact is applied to the robot 2, the work detection area is detected even if the position where the impact is applied is far from the selected work 5. Processing can be performed without waste.

(4) When the load state change position exists in multiple places, such as when the work 5 existing at a position different from the impact detection position is taken out after detecting the impact acting on the robot 2 (for example, in the case of FIG. 5). ), Since the work detection areas 12a and 12b corresponding to the respective loading state change positions are set, the work 5 in the area where the loading state has changed can be detected accurately while suppressing the time required for the work detection. (5) If the size of the work detection area is automatically changed according to the distance between the selected work 5 and the camera 7, the work is displayed on the image regardless of the height of the work 5 in the container. The relative size of the work detection area with respect to 5 can be made constant, and the work 5 can be detected efficiently.

In the above embodiment, the work detection unit 8a, the work selection unit 8b, the work loading state determination unit 8c (loading state determination unit), the work loading state change position storage unit 8d, and the work detection area setting unit 8e (area setting unit). ), The camera control unit 8f, the memory 8g, and the visual sensor control unit 8h are included in the robot controller 8, but the configuration of the robot controller 8 is not limited to this. For example, the work detection unit 8a, the work detection area setting unit 8e, the camera control unit 8f, and the visual sensor control unit 8h are provided outside the robot controller 8 so that the result of image processing is transmitted to the robot controller 8 using communication means. You may. The work loading state change position storage unit 8d may be omitted, and the loading state change position may be stored in the memory 8g.

In the above embodiment, when it is determined that the loading state of the work 5 has changed, the work detection area is set as a part of the work loading area, but the shape of the work detection area may be any shape. By processing in the robot controller 8, the success or failure of the gripping operation of the work 5 is determined (step S12), and the presence or absence of an impact acting on the robot 2 is determined (step S10). The configuration of the work loading status determination unit and the configuration of the work loading status determination unit as the impact determination unit are not limited to those described above. That is, the present invention is not limited to the work taking-out device 1 of the embodiment as long as the features and functions of the present invention can be realized.

Summarizing the above, the work taking-out method according to the present embodiment includes an imaging procedure (step S1) in which a workpiece loading area including a plurality of workpieces 5 stacked separately is imaged by a camera 7 and a camera image captured by the camera 7. The work detection procedure (step S2) that detects the work 5 based on the work, the work selection procedure (step S6) that selects the work 5 to be taken out based on the detection result by the work detection procedure, and the work selection procedure. The work taking-out procedure (step S9) for taking out the work 5 by the robot 2, the loading state determination procedure (step S10, step S12) for determining whether or not the loading state of the work 5 has changed due to the operation of the robot 2, and the work detection. Including the area setting procedure (step S14d) for setting the work detection area when detecting the work 5 by the procedure, in the area setting procedure, when it is determined that the loading state of the work 5 has changed by the loading state determination procedure, It is a peripheral area of the loading state change position, and is characterized in that a work detection area is set in a part of the work loading area, and various modifications may be added to the above configuration as long as this feature can be realized. ..

1 Work take-out device 2 robot 2a robot arm 2b robot hand 3 Visual sensor 4 Impact sensor 5 work 5a Work that was selected first and failed to be taken out 5b The second selected work that was successfully taken out 6 container 7 camera 8 robot controller 8a Work detector 8b Work selection department 8c Work load status judgment unit 8d Work load status change position storage unit 8e Work detection area setting unit 8f Camera control unit 8g memory 8h Visual sensor control unit 12a Work detection area set by impact detection 12b Work detection area set by successful work removal