Software and methods for automated pallet inspection and repair

IS 2 365 239 T3

DESCRIPTION

Apparatus and procedure for automatic inspection and repair of pallets

Field of the invention

The invention relates generally to the repair of wooden pallets, and specifically to an automated process for scanning pallets and identifying individual items on the pallet for removal, replacement or repair. It also applies to pallets constructed of other materials such as plastic, metal, or composite materials.

Description of Related Art

The commercial movement of materials normally uses a wooden pallet on which the material is placed or secured. This pallet is typically constructed with a flat top deck consisting of wooden boards or planks nailed, bolted, or glued to parallel beams known as carriers or stringers. Bottom boards are likewise attached to the carriers. The frame allows the insertion of the forks of a forklift or other machine to lift and move the pallet and its load of materials. There are various pallet designs in use and they are generally characterized by the place of manufacture and use. For example, pallets made and used in Australia, New Zealand, the United States, Canada, and Europe are all of different designs. In some designs, for example, blocks with or in place of the carriers are used to separate the upper and lower plates. While wooden pallets are the most common, pallets of other materials such as plastic, metal or composite material are also used.

During normal use, pallets can be dropped, overloaded, crushed, or damaged. Damaged pallets are returned to the pallet supplier or another supplier for inspection, repair or replacement. The inspection process and decision is currently made by expert human inspectors, or by automated means who apply specific criteria, and decide whether a pallet is damaged, and if so, decide to repair or discard the pallet. The use of human inspectors is desirable as they can immediately inspect and repair each pallet in a single station. This can be done by presenting each pallet in turn, for example, on a conveyor belt, so that the inspector can see the pallet, decide if it is damaged, and repair or discard it. Human operators, on the other hand, are undesirable because the inspection and repair decision is not uniform, as each inspector will naturally apply the repair based on their judgment. It is also not convenient from a safety point of view as accidents or injuries can occur in such an environment.

Human operators may be replaced by an automated pallet inspection and repair apparatus. Some current automated systems use stereoscopic pairs of cameras to collect pallet geometry and topography information, then use computer programs to make the repair or scrap decision by consistently applying pallet-specific criteria. This is done first by determining the pallet design - Australian or European for example - then by comparing the geometry and topography of the individual pallet against the criteria for the pallet design. Current systems, however, only decide to repair or discard each inspected pallet, that is, each pallet meets or no longer meets the inspection criteria. If the pallet passes, it is placed back in service. If it does not pass, it is sent for repair. The repair process in today's automated systems, however, is similar to the previous manual inspection process. Pallets in need of repair are shipped by the conveyor, passing one or more human repair stations where the repairer inspects the pallet, determines repair needs, and then repairs them. In some cases, a pallet may be damaged to such an extent that it is determined to be beyond repair and may be discarded. A process in the repairman makes this determination have an additional disadvantage that the repairman may be inclined to declare that a pallet is beyond repair, since that minimizes the work to be done. Even in the best of cases, a system with automatic control and a human repairman has some of the same disadvantages (in consistency and safety) of the fully human inspection and repair process.

What is needed is a process to automatically inspect a pallet to determine if it needs to be repaired. If no repair is required, the pallet is placed back in service. If repair is necessary, the pallet is sent to an automated repair station with a list of repairs to be made. The repair station receives the pallet and makes the repairs on the list. In addition, a determination can be made that the pallet is beyond repair, in which case the pallet is sent to the repair station and disassembled so that undamaged components can be reused.

EP0943394A2 discloses a system for the repair of wooden pallets.

DE19645553A1 reveals that pallets are analyzed for certain characteristics and functions, and defective pallets are scrapped.

IS 2 365 239 T3

WO 2004/053739 A1, which was published between the priority date and the filing date of the present application describes a set-up optics and method for the inspection and repair of pallets. A laser, camera and image processing algorithms are used to measure the 3D coordinates of the pallets and to identify the defects of the elements on the pallets. 3D coordinates are filtered (for example, relative to a height threshold) and compared to specific criteria for each element. In addition, it is proposed to generate a formula of the repair tasks and carry out these tasks using an automatic repair station.

The invention provides an improved apparatus and method for the inspection and repair of a pallet as defined in independent claims 1 and 7, respectively. Other specific characteristics of the apparatus and the method are defined in the dependent claims.

Brief description of the drawings

For a more complete understanding of the invention and other advantages thereof, reference is now made to the following description of the preferred embodiments taken in conjunction with the accompanying drawings in which:

Figure 1 is a schematic top plan view of a pallet;

Figure 2 is a top schematic plan view diagram of an inspection station;

Figure 3 shows a schematic side elevation of a pallet, and

Figure 4 shows a logical flow diagram of a pallet repair process

Description of preferred embodiments

An automated pallet inspection and repair system and the apparatus comprises an inspection station connected to a computer. A pallet to be inspected moves relative to the inspection head. The pallet can be on a conveyor belt or moved with a robotic manipulator or other device. On the other hand, the pallet can be in a fixed location and the inspection head can move through it. The sensing head is comprised of an array of at least one laser and one camera, with the camera recording the profile of the reflected laser across a width of a pallet. Additional cameras can be used to scan larger areas, but stereoscopic camera pairs are not necessary. The resulting information from the sensing head is collected and processed by the computer to represent the geometry and topography of the pallet as a two-dimensional representation. The representation is analyzed so that the individual elements, (ie tables, planks, beams, blocks, etc.) are identified and located by coordinates. The design of the pallet is determined by the number, size and location of the elements. The elements are analyzed with the specific criteria for the determined pallet design. This includes the criteria for the item only (size, location, integrity, damage, missing or raised nails, etc.), the criteria between items (spacing, overlap, etc.), and the pallet design criteria (items missing or superfluous, etc.). If it is determined that the pallet has not passed the criteria, a list of specific repairs is generated. This list includes which item must be repaired and the nature of the repair (remove, replace, reconnect, repair, etc.) The data that comprises the list of repairs to be carried out accompany the pallet to a repair station, either physically or logically through the use of a monitoring system. The repair station is an automatic repairer, for example, a robotic arm using a nail gun, band saw or other saw, safe levers, etc., to implement the exact repairs determined necessary. After repair, the pallet is returned to service. If the pallet is determined to have passed the criteria, it is returned to service without stopping at the repair station. The analysis may also indicate that the pallet should not be repaired, but rather disassembled. In this case, the list of repairs only includes the steps to disassemble the elements for reuse, and the parts that can be reused and those that are to be discarded.

In the preferred embodiment, the present invention provides sufficient coordinated outputs to automate component repair, for example, by robotic arm movements, band saw placement and activation, nailing, etc.

Figure 1 shows a layout of a pallet 100. The pallet consists of top plates 102, labeled TB 0 through 7, and corresponding bottom boards labeled BBO through BB4. The upper boards are supported by three horizontal carriers or stringers 104, 106, and 108, labeled Bo, B1, and B2. Other designs will have different numbers of boards, boards of different sizes, and different spacing between boards, and may have different numbers and styles of carriers. Connector blocks and boards can be substituted for brackets in some designs. In the illustrated design, the top table 0 and the top table 7 are wider than the other six tables. For the purposes of the present invention, the pallet can be considered to be set in an xyz orthogonal configuration where the X axis is horizontal (with reference to Figure 1) through the bottom of the pallet parallel to the beam or carrier , 108. The y-axis runs vertically (with reference to Figure 1) along the left edge of the top board 1. The z-axis is

ES 2 365 239 T3 orthogonal to both x and y.

There are four processes that constitute the preferred automated pallet inspection and repair procedure. The first process is data capture. This means capturing all the data about the physical or structural composition of a pallet that is required to make determinations about the nature, scope and purpose of the repair process. The second process is the analysis of the captured data. The analysis simplifies the data and refers the data to known facts, so that a repair process, using certain fixed processes, can be specified. The third process is the generation of a list of detailed instructions based on the repair processes that are available and what repairs are needed, as determined by the data and analysis thereof. The detailed steps required to make repairs on a pallet is called a formula. The fourth process is to apply the formula using automated equipment. In preferred embodiments, an industrial robot reads a specific step of the formula and implements it according to a flexible schedule. Each step is processed in turn.

An embodiment of the present invention consists of two computer systems and the mechanical means to move the pallets. These systems can reside on the processors of one or more physical equipment or hardware, and can be distributed or collected. The first computer system is called the capture system, which collects information about the geometry and topography of the pallet. The second computer system is called the analysis system, which analyzes the geometry and topography of the pallet and determines the design of the pallet and then using the design specifications of the pallet analyzes the pallet and decides whether it should be repaired or returned to service. The two computer systems need not be separate or distinct. The mechanical means that move the pallet can be a chain or conveyor belt or a robotic arm or other system for transporting the pallet through the inspection head.

Figure 2 shows an application 200 of the mechanical means to move the pallets. This comprises a chain conveyor supporting a pallet 204 and moving it in the direction of the arrow, from left to right. Pallet 204 moves under two lasers 206 and 208. These lasers illuminate and sensors capture the entire width of pallet 204. In some embodiments, there is an overlap in the laser beams in the center as shown at 210. As the pallet 204 moves under the laser beams 206 and 208, information about the geometry and topography of a pallet is captured by the cameras and sent to the computer capture system 212. Such laser systems and the sensor are well known, as are the procedures for using such lasers to collect this information. The system can be replicated to collect data on the other faces of the pallet at the same time or asynchronously from the top deck. Obviously this can also be achieved by a number of different combinations of laser and camera than the double set described above.

The 212 capture computer system performs the following steps. First, it collects the information from the pallet profile, that is, it collects the geometry and topography information from the lasers and sensors. The sensors return a sequence of three-dimensional coordinates. The cameras / sensors are synchronized so that the overlapping points illuminated by multiple laser beams give the same coordinate values ​​when viewed from each sensor or camera. The analysis of the two lasers combine to give a series of coordinates for the width of the pallet. This process is repeated for each scan profile that the pallet moves in relation to the lasers and cameras.

When analyzing the top deck, the scanned data is filtered to give only the top surface geometry and topography. This is accomplished by discarding points that have a z coordinate that is below a certain threshold or filter line. This eliminates any point from the analysis that corresponds to a carrier or the bottom of the pallet or the transport conveyor or robotic manipulator, for example. It will be obvious that the same process could be applied in the analysis of any specific face or cover of the pallet. In some embodiments, individual planes are established for each carrier. Drawings can be combined, on average or referenced by separate data.

The laser scanners are held according to the speed of the pallet transport mechanism so that the scans occur at regular distances along the length of the pallet in the direction of movement. Typically this is set to scan at a linear distance of 1mm, although it could be at any chosen resolution distance.

Next, the corners of the pallet meet with a 45 degree filter. This places the four points at the ends of the pallet. That is, the point of (a minimum of x, and minimum), (minimum and maximum x), (maximum and minimum of x) and (maximum x and maximum) is found. These four points determine the corners of the pallet. These are normally called PP0, PP1, PP2, and PP3, respectively, where PP0 and PP2 lie on the X-axis, and PP0 and PP1 lie on the axis. PP0 and PP3 are diagonal, as are PP1 and PP2.

The software then finds the offsets between the image origin and the pallet origin to give the distances and the xy offset of the pallet. That is, the pallet size is calculated by subtracting the combinations of

IS 2 365 239 T3

PP0, PP1, PP2 and PP3. The data is also normalized by relocating the coordinates so that PP0 is at the origin of the coordinate system pallet, and PP1 and PP2 lie on the xe and axis, respectively. A second set of coordinates, based on the image data, is used in the calculation of automatic repair parameters.

To convert the three-dimensional topographic information to a two-dimensional geometric representation, first the location and height of the headlines (labeled Bo, B1, B2) are found in the image by inspecting the profiles most likely to be represent bearer places. When the carrier heights and locations have been determined, a series of filter planes are drawn displacement of the carriers (shown as item 302 in Figure 3). This can also be achieved by finding best fit planes for the surface of the drawing boards and a plane of the offset filter below. Each point in the three-dimensional representation is checked against the plane of the corresponding filter. Points above the filter plane are identified as belonging to joints, points below belonging to holders or other structural elements. The board points are then filtered and assembled into point arrays on the board edges that belong together. This can be done using any of the applied technical standard edges to find the set of points above the filter line and a next edge string algorithm. These edge matrices represent the boards or parts of the joints and are used in the subsequent analysis. If a two-dimensional (geometry only) scan and inspection head is used, electronic means (for example, sensor scope restrictions) are used to filter the data tables, with the same identification matrix and process. assembly taking place. The arrays do not contain height-related data, only 2D geometric position of the points above the filter line.

The 214 analysis computer system performs the steps indicated in Table 1.

TABLE 1

Step - Description

<td>He passed</td><td>Description</td>

<td> 1</td><td>The data stream from the inspection head is captured for the construction of a pallet model in the memory of the computer. This process uses the known inspection head solution and the known sensing head speed and distance pallets to construct a topographic three-dimensional model and subsequently a geometric two-dimensional model of the pallets being analyzed. Corner pallet points are calculated from topographic data and stored.</td>

<td> 2</td><td>Geometric model is decomposed to give orders of the points that represent the edges of each table (and the edges of the partial joints)</td>

<td> 3</td><td>Board assemblies are checked for integrity (that is, is the edge closed?) And consistency (that is, the edges do not intersect). Accumulated boards are divided by applying a virtual edge along the most probable line of intersection between the boards. When there are multiple arrays along a line parallel to the Y axis, and these arrays are less than a full board length in the Y direction and do not overlap or intersect, they are classified as a single (not working) board. Board matrices are sorted by their minimum value of X, and their corners are identified and stored.</td>

<td> 4</td><td>By comparing the number, type and location of the identified joints and the spacing distance between the pallet corner points with the range of possible known pallet geometries, a pallet type can be assigned to the pallets under analysis, determined by the closest match against the specification database. This adaptation process can be accelerated if the system only needs to wait for a single pallet style. A pallets that cannot be matched to a specific style of pallets is marked as undefined scan and later stopped.</td>

<td> 5</td><td>Board types can be assigned (eg intermediate, lead, etc.) for each board based on its location relative to the pallet corner points and its approximate width (from the board corner points). Boards within a given region of the corner pallet points are assumed to be the front joints.</td>

<td> 6</td><td>Pallet quality criteria are loaded from the database into the analysis system for the particular type of pallets explained above. Each board matrix will be checked against the appropriate board criteria for that type of card. Board controls can include board width, notches or missing material, ragged edges, excessive curvature, or any other criteria. Any table arrangement that fails these tests is marked as a board to be eliminated. The topographic data of the region</td>

IS 2 365 239 T3

<td></td><td>Corresponding to the board is also checked for hall thickness, split end, cracks, holes and other three dimensional characteristics, with the failures again being recorded for removal.</td>

<td> 7</td><td>The set of matrices are examined against one another's quality criteria to determine if other moving board repairs are necessary. An example would be the position of the front boards in relation to the corner pallet points, with a front board that is too far from the corner marked points that need to be adjusted, unless it has already been marked for removal. A hierarchy of board repair decisions is imposed with board removal of the highest priority, next board realignment, and any other operations lower.</td>

<td> 8</td><td>Gaps are compared against distance criteria, such as width spacing. Gaps that are larger than a board width are checked to see if a board allows the gap to be entered with the appropriate gaps on either side (Fig. 4). If a plaque is formed, a large phantom is constructed to represent the missing board, and is marked by a slate placement operation.</td>

<td> 9</td><td>The results are stored in the database. At this point, all matrices are marked, either valid boards, or as tables to be removed, adjusted, or tampered with.</td>

<td> 10</td><td>When this system is implemented in an automated pallet repair situation, further calculations and data manipulation are required. These calculations are specific to each machine in the repair cell, and could include the location (X, Y, and Z position and angles) needed to insert a band saw blade into a particular space to perform the ejection operation of a board that has been marked for deletion. When this blade does not fit the gap, the gasket is marked for removal by a different device.</td>

<td> 11</td><td>A formula is generated for cell repair, with a list of the operations (jobs) necessary to carry out to repair the pallets, and the data associated with each of these jobs. These are rated to speed up the repair cycle time.</td>

<td> 12</td><td>In a control system designed for grading or quality control purposes only, steps 10 and 11 are removed, and replaced by topographic analysis of protruding nails and other features.</td>

Figure 4 illustrates the logic flow for step 8 in Table 1, the examination process for all 400 separations. The software defines pattern storage to store gap values ​​to be used for each of the 402 profiles. Each gap is initialized to zero. Starting with the leftmost right edge of the joint, the gap values ​​are calculated for each table as shown in step 404. In step 406 the difference in values ​​for each table has been stored, so the mean difference can be calculated. At step 408 the mean difference is compared to the distance criteria based on the pallet design. If the gap is larger than the design criteria, the difference is marked as a bad gap, shown in step 418. In step 420, the gap is examined to see if it is large enough to fit a new board. If it is large enough to fit in a joint, step 424 calculates how many joints will fit in the space. That number of tables below is indicated for repair orders. The software then moves to step 422 to examine the next space. However, if the decision made in step 420 is that the gap is not wide enough to fit the new joint and still exceeds the criteria for the maximum difference, then step 428 is carried out. Step 428 determines that the gaskets should be removed and replaced to correct the gap.

In step 430 a check is made to see if one of the bounding tables is skewed. If the boundary boards are crooked, the offending board is indicated for removal and replacement or replacement, and the resulting difference will be re-evaluated 426. If none of the boundary tips are incorrect 430, then a check is made for see if one of the boards is missing all the wood (or other material for non-timber pallets) 432. If one of the missing material boards, 432, a command is indicated to remove the plate and re-evaluate the resulting differences in step 426. In step 432, if no boards are missing any material, then step 434 is performed. A review of the separation of neighbors to be made. If one of the neighbor spacings is smaller than the other then an order to remove the table and re-evaluate the stage of social differentiation that 426 produces. If in step 434 the gaps are equal in size then step 436 is performed, that is, the pallet is marked for manual inspection, or else a decision can be made to arbitrarily remove one of the tables.

In connection with step 408, the mean difference is compared to the design criteria, and if the mean difference is acceptable, a test is run to determine if there is a notch in the plate. This notch would give a false indication

IS 2 365 239 T3 of a bad gap. The notch test is shown in steps 410 to 416. At 410, the difference of values ​​over the entire length of the notch is added and the mean is calculated. In step 416, the calculated average is compared to the design criteria. If the average is greater than the design criteria and the gap is too large, processing continues with step 418 described above. If at step 416 the average gap passes the test against the design criteria then a check is performed at step 414 to determine if there are more gap values ​​to check. If there is more to check, step 412 is performed to add the next value and then subtract the first value and recalculate the mean. The process then continues and step 416. If at step 414 there are no more gap values ​​to check, then all gap values ​​have been determined to be acceptable and the process continues at step 422 to proceed to the next space. If there are no more spaces to test, then the process ends at step 438.

Returning now to step 426, a repair order indicates to remove an edge or change its position. The process then continues at 404 to recalculate the mean distance.

In this way, the pallet board is examined for repair boards and orders are stored for later use or the pallet is back in service. If the pallet needs repair, specific instructions are determined to remove, replace, replace boards, or add one or more boards, or to remove a protruding nail.

This offers a technical advantage over current pallet automated inspection and / or repair systems, which only determine a pass-through for the adequacy of the pallet, and no specific repair instructions are generated. Furthermore, the technique of the invention is sufficient to automate the repair process by connecting the process output to an automatic repair station. This station could include a robot arm that grabs the pallet to be repaired, then, using a band saw and the nail gun and other artifacts, removes and replaces the specified gaskets. The instructions for the robot arm would be, for example, remove the plate located between 22.5 cm and 40 cm from the leading edge, then the nail from a new board 22.0 cm from the leading edge.

This same logic can be applied to the inspection and repositioning of the carrier or replacement, or as an alternative to the lower pallet cover.

Controlling the robot in an automated repair cell based on the information generated by the pallet analysis system described above requires specific robot, PLC and computer system links or interfaces and software.

Traditional robot control design is relatively simple, based on the premise that the robot performs repetitive work (or a series of repetitive jobs) that can be predefined at the time the system is designed. Additionally, traditional robot systems require human intervention when there is a robot problem or accident. To use a robot for pallet repair, you must dynamically change your program for each pallet to be repaired, based on the particular operations that are required and the location of the joints or gaps for which these operations should be applied. In addition, it should automatically recover from minor accidents and problems, and accidents to an operator's main hall. To achieve this, the cell control software system is divided into three components - these are the robot control, the programmable logic controller (PLC) and the sub-formula generation-system repair described in steps 10 and 11 from Table 1. The formula for the generation of sub-system loads of the necessary robot operations and associated position data in the PLC. This can be done in a single batch of a complete pallet, or sequentially as required. In preferred embodiments the data is sent as a single batch. The data is checked for consistency and integrity, and the missing data is marked with the formula generation system.

The robot control contains a master job and a series of sub-jobs that the master job can call up when needed. The master job communicates with the PLC. The PLC tells the main job in the robot control, which sub-job to call, and sends the data that this sub-job will have to execute, for example, the location and angle of a special joint to be operated. This data is confirmed by the master job to the PLC, with the master job waiting for a handshake from the PLC before continuing. Upon receiving the handshake, the sub-job is called. The first task in each sub-job is to check the current location of the robot. The position and angle of the pallet gripper must be within a certain envelope (which can be a sphere or a cylinder around a predefined point or a line known as the sub-work start position) for the operation to continue. If the robot is within the allowed envelope, the sub-job continues with the operation. At each step of the operation, a handshake is exchanged with the PLC. This handshake allows the PLC to control the point in the sub-work that the robot is doing, which is necessary for any automatic recovery operation. When the sub-job is finished (or if an error is detected), the robot control goes back to the main job, which communicates with the PLC to obtain the next sub-job, and the process continues until the formula is complete. .

Each device in the repair cell is numbered, starting with device 02 and working up to 99

ES 2 365 239 T3 devices (this system could be extended to use any number of digits depending on the number of devices in the cell, however two digits are used in preferred embodiments). The robot sub-jobs that occur on each of these devices are the name of the device on which they occur, for example a two-digit code. The sub-jobs that travel between the control machines are named by combining the names of the two devices between which the robot must move (for example, the sub-job to move from machine 12 to machine 34 it would be 1234, while to move 34 to 12 it would be 3412). These numbers are generated by the subsystem repair formula and passed as part of the formula. The PLC passes these names in turn to the master worker robot, which calls the sub-job.

Some numbers are reserved for emergency or other retrievals, ie devices 00 and 01. This allows sub-job retrievals to be defined for each device as XX00 and XX01 where XX represents the name of the device. recovery jobs in general, reverse through the steps previously performed on that machine, back to the initial robot position for that device (for example, if sub-job 03 has problems, the recovery job would be 0300 or 0301 , depending on the alarm generated). Depending on the alarm condition, the current step formula is tried again, or alternatively, the formula is dynamically changed to overcome the problem. For example, if removing a board that is removed with device # 03, 04 and the device is designed to remove boards, additional measurements can be added to the formula dynamically to take the pallet in device 04 and perform the expulsion operation. Depending on the particular formula, this action can be carried out immediately, or after the completion of the other pending actions on device 03.

The system is based on the PLC being in maximum control at all times, and of handshakes between the robot and the PLC between any operation, no matter how small. For safety reasons, all robot operations should be reviewed in its home position and confirmed with the PLC.

An alternative embodiment of the formula system would be in a repair cell where the pallet is held in one location and the device inspection and repair head is presented to it. The analysis of the pallet will proceed according to the previous description, as well as the generation of the formula. In this style of embodiment, rather than the PLC instructing the robot to take the pallet to a particular repair device, the PLC instructs the robot (or other type of manipulator) to repair the pallet equipment. The position data transfer is identical to the description above, as well as the robot and compliance location testing procedures.