Method for detecting damage defect of curved tag based on template matching and similarity calculation

A Method for Detecting Damage Defects of Surface Label Based on Template Matching and Similarity Calculation

Technical field

[0001] The present invention relates to the field of machine vision and video image processing, and more particularly to a surface tag breakage defect detection method based on template matching and similarity calculation.

Background technique

[0002] In the food, beverage and pharmaceutical packaging areas, PET plastic, glass and other materials often need to have a label outside the container. Labels not only have a very important aesthetic effect on the product, but also indicate a large number of product information. There are defects in the label for the product is more serious quality problems, can not be listed on the sale.

[0003] The traditional bottle label damage defect detection in the filling product line back to the manual detection method to complete the detection work, usually there is low efficiency, high cost, poor stability and poor reliability and other issues, can not meet the large-scale industrialized Production needs. With the increasing speed of production, the gradual improvement of product quality requirements and the rapid increase in labor costs, manual detection of label defects more and more difficult to sustain. Image detection algorithm based on machine vision can automatically detect defective defects, whether it is to improve efficiency, reduce costs, or to improve the stability and reliability have greatly improved, with good prospects for development.

[0004] At present, the current research situation at home and abroad, commonly used in label defect detection algorithm based on image filtering method, based on edge extraction method, based on the depth of learning methods. However, these methods are usually more critical conditions for image capture, the operating environment is also more complex, in the input to the actual industrial applications are often limited.

There are also shortcomings or limitations in the application of the relevant scheme, such as China Patent No. ZL 201310703385.3, issued on December 30, 2015, and the name of the invention is: a flat wine The method comprises the following steps: image contrast ratio stretching transformation, image denoising processing, image threshold processing, image filling processing, and the like, wherein the image is obtained by using the illumination image acquisition system, The pre-processing of the bottle image; repositioning the bottle bottle and setting the position of the bottle in the bottle image, and correcting the label when the special point and the label of the pose relationship, and need to detect the image of the special point and label Pose relationship to compare, in order to determine whether the label has paste defects. This application is based on the positional relationship between the midpoint of the image and overcomes the problem that the conventional detection method is not easy to detect the label sticking defect of the flat bottle, but the application is used for the detection of the flat surface of the bottle, Body is not applicable, and the program itself is largely dependent on the three preset points, and the placement of the bottle has a certain requirement, must be in the side of the standard image acquisition, in the practical application of a certain degree of limitations The

[0006] Chinese Patent No. ZL201410610077.0, invented the name: label defect detection method; the application of the printed label for image acquisition; one by one the standard characters and labels on one of the characters to be superimposed and If the similarity value of the summing result is greater than or equal to the first threshold value, the standard character corresponding to the similarity value with the largest sum result is used as the character to be compared, and the current value of the character to be compared is recorded And further determining whether the similarity value of the character to be compared is smaller than the preset second threshold, and if so, stacking the corresponding standard character with the character to be compared and calculating the number of times the product is equal And if the number of times exceeds the preset third threshold value and the area of ​​the defective area exceeds a preset fourth threshold value, the defective area is color-coded. The application can identify the characters in the label defects, compared to the high efficiency. But the application of the application object is mainly a flat character label, does not apply to the deformation of the surface label, and the application is for the characters, bar code simple tag, does not apply to include graphics, text and other Complex content label.

[0007] Chinese Patent No. ZL 201310160165.0, issued on August 19, 2015, entitled "Method for detecting a near-cylindrical bottle label based on a 3D construction, which includes a bottle label Accurate positioning and label mask extraction, 3D modeling of the label of the bottle label, anti-column surface expansion of the bottle label image, and detection of the label image of the bottle body, which can realize real-time uninterrupted detection, But the defect detection of the application is to use the classifier to classify the feature vector of each block of the image to judge whether the current block is defective, the complicated process and the large amount of computation. Therefore, the efficiency is higher Low, and the eigenvector robustness is not strong enough, may be affected by light and other factors interfere with the impact of detection results.

The contents of the invention

[0008] 1. Technical problems to be solved by the invention

It is an object of the present invention to overcome the deficiencies of the prior art and to provide a method for detecting surface defect damage based on template matching and similarity calculation. The present invention includes image acquisition, image preprocessing, template region extraction, Detection and result display, and the image acquisition includes different images taken from three angles of the label to be tested and the template label. The preprocessing process realizes the segmentation of the label area and the image processing operation of the surface transformation into the plane; the template area extraction Step to achieve the test label three different angle images and template a panorama of a region of the corresponding one, so that the image collection in the bottle rotation angle is not limited, reducing the difficulty of the system; defect detection based on template matching and similarity calculation phase Combined with the method, through the short time to match the template to lock the possible defect area, and then use the exact similarity calculation to determine whether the existence of defects in ensuring the detection effect on the basis of effectively improve the detection efficiency.

[0010] 2. Technical program

[0011] In order to achieve the above object, the technical solution provided by the present invention is:

[0012] A method for detecting a breakage defect of a surface label based on template matching and similarity calculation of the present invention comprises the steps of:

[0013] Step 1: Image acquisition: Collecting a surface label image of a cylindrical bottle using a set of light sources and three cameras in a low-light environment; the initial image group of the collected lossless template is recorded as (Μμ, Μμ, Μμ) The initial image group to be recorded is recorded as (Fcll, F0, F3);

Step 2: Image Segmentation and Stretching: The image obtained in step 1 is divided into operations, and the divided image area of ​​the label region is obtained and stretched by the camera calibration method to obtain a plane of the surface label region Image, template label plane image group recorded as (151,52,53), the test label plane image group is recorded as (decay ^ 2, inside);

[0015] Step 3: Image splicing: The template label plane image group (Ms1, Ms2, Ms3) obtained in step 2 is stitched by the feature point matching method to obtain the template panoramic image M;

Step 4: Image Positioning: The positioning image plane (FllF2lF3) obtained in the second step is positioned in the template panorama image M, and the positioning area is divided into the map template group (M1, M2 , M3);

Step 5: Damage Defect Detection: The template matching method and the feature similarity FSnH are combined to calculate the similarity of each test-template group (Fi, Mi), and the similarity is lower than the set threshold Identified as a damaged defect area;

[0018] Step 6, the results show: show the results obtained in step 5 of the damage defect detection.

[0019] Furthermore, step 1 in a low-brightness environment using a CCD camera to collect the label image of the bottle, the three cameras are in the bottle as the center of the equilateral triangle vertex position, and placed in each camera just below Light source, to ensure that the direction of each camera camera in the central axis to maintain the level and through the bottle placed area center.

[0020] Further, the two pairs of the lossless template initial image group (Mcil 2) 3 and the initial image group to be measured (Fcll, FaFci3) are subjected to a preprocessing operation by presetting an ROI region slightly larger than the label area On the basis of this, the canny operator is used to detect the edge, so as to segment the label area accurately. Then, Zhang Zhengyou plane calibration method is used to obtain the inside and outside parameters of the camera, and then the surface image is transformed into a planar image.

Further, in the third step, the image contents of the template label plane image group (MS1, MS2, MS3) are overlapped, and the three images are concatenated by the SURF feature extraction and the feature point matching to obtain the template panoramic image M.

[0022] Further, the specific process of image positioning in step 4 is to use the template matching method to find out the corresponding regions of each image in the template panorama image M of the test label plane image group (FiF2J3), respectively, and (MhM ^ M3) corresponding to (F1J2J3) is obtained by dividing the corresponding regions.

Further, in step 5, the image plane group (Fl, F2, F3) to be measured corresponds to the image in the map template group (M1, M2, M3), divided into three sets of test- The template image group (Fi, Mi) performs the damage defect detection processing separately; by calculating the similarity between the image to be measured Fi and the template image Mi, if the similarity is lower than the set threshold, it is considered that the image to be measured and the template image exist The similarity calculation method uses the template matching method and the feature similarity degree FSIM to calculate the organic combination way to judge whether the existence of the defective defect exists and realize the location of the defective defect area.

[0024] Furthermore, step 5 calculates the similarity between the measured image Fi and the template image Mi as follows:

(1) Set the width w, the height h, the column direction step col_step, the row direction step length row_step the sliding window, the synchronized traversing the image to be measured Fi and the template image Mi, memorize an image to obtain the block The number of images is m;

[0026] (2) set the template matching similarity value threshold tl and t2, tl <t2; FS top similarity value threshold si;

[0027] (3) The image of the measured image Fi and the template image Mi in the current window is denoted as fk, mk, Kk m, and the template matching similarity is calculated; the template matching adopts the normalized squared matching method The similarity degree is t, if t <tl, the graph to be measured fk and the template map Hlk similarity is small, can determine the existence of broken defects in the graph; if t> t2, the graph to be measured fk and template map Hlk similarity It is possible to determine that there is no breakage defect in the graph to be measured; otherwise, the similarity of the graph to be tested fk and the template map mk is moderate and can not be judged and proceed to the next step;

[0028] (4) the similarity of the measured map fk and template map mk, using FSIM calculation method to calculate the similarity of the two maps, the similarity value is s; if more than si, the map fk and template map mk similar A high degree of judgment can be found in the map without damage defects exist; if s <sl, to determine the existence of broken defects in the map;

[0029] (5) Repeat step (3) until the completion of all the block image similarity calculation.

[0030] Furthermore, the calculation method of the feature similarity FSIM is as follows:

[0031] The two images to be calculated for similarity are denoted as fl (X) and f2 (X), respectively

[0032]

[0033] where

[0034]

(X) is the phase similarity between (X) and f2 (X) at pixel X, and is expressed as follows:

[0036]

[0037] SG (X) represents the gradient similarity of (X) to f2 (X) at pixel X, as follows:

[0038]

[0039] The above two kinds of ^ ^ and ^ are constant, used to prevent the denominator is 0, take 1'1 = 0.85,1 ^ = 1604: 6) and G2 (X) respectively represent the image fi (X) and f2 (X) at the pixel X; the gradient of the image f (X) is calculated as follows:

[0040]

[0041] where Gx (X) and Gy (X) are the partial derivatives of the image in X and y at pixel X, which can be obtained using the sobel operator:

[0042]

PCi (X) and PC2 (X) represent the phase coincidence of the image fi (X) and f2 (X) at the pixel X, respectively; the phase coincidence PC (X) of a pair of images f (X) Methods as below:

[0044]

[0045] where An (X) is the amplitude of the η cosine component,

For the phase offset function, expressed as:

[0046]

[0047] φ "(χ) is the local phase of the Fourier transform at pixel X,

Is the weighted average of the local phase of all Fourier transform components at X, T is the estimated noise, ε is a small normal quantity, 0.001 is obtained, the denominator is 0, W (X) is the filter band weighting function,

[0048]

[0049] where

[0050]

[0051] Amax (X) is the maximum corresponding amplitude of the filter bank at X, the constant c is 0.4, g is 10, and ε is zero.

[0052] Further, step 5 of the damage defect detection involves the following parameter settings:

(1) Sliding window size and step size: including width w, height h, column direction step c0l_Step, row direction step row_step, specific width w take 1/4 of image width, height h image The height of the 1/4, column to the step size col_step take the image width of 1/8, row to the step size row_step take the image height of 1/8;

[0054] (2) Similarity threshold: including template matching similarity lower limit threshold tl and upper limit threshold t2, FSM similarity si, specifically tl = 0.9, t2 = 0.95, sl = 0.8.

[0055] Further, the specific process of step 6 is that the sliding window synchronizes the template image and the image to be measured in step 5, and calculates the similarity between the corresponding template and the image to be measured to determine the current block Whether there are broken defects, if damaged defects, in the corresponding position in the image to be marked, the final display after the marked image.

[0056] 3. Benefit effect

[0057] The technical solution provided by the present invention has the following significant effect as compared with the known known art:

(1) The method according to the present invention provides a method for detecting a breakage defect of a surface label based on template matching and similarity calculation, and a template region is extracted by a template matching method in a template panorama, In order to carry out the follow-up defect detection steps; the existing surface tag defect detection technology for non-all-inclusive label, the need to collect images in the side of the label, followed by further stretching and integration, so the shooting angle has certain requirements The program proposed by the present invention is directed to an all-inclusive label (a non-all-inclusive label can also be treated as a full packet, and the blank portion is treated as a label and does not affect the detection result) And in the actual detection process, no matter from what angle to take the bottle to be detected, the current angle label can be found in the template panorama map corresponding to the template map, the invention of the proposed The processing method makes the rotation angle of the bottle in the image collection unrestricted, which reduces the difficulty of the system.

(2) The method for detecting the defect defect of the surface label based on the template matching and the similarity calculation of the present invention is based on the method of combining the template matching and the similarity calculation to detect the defect by a time- Possible defect area, and then use the exact similarity calculation to determine whether the existence of defects, to ensure the detection effect on the basis of effectively improve the detection efficiency.

Description of the drawings

[0060] FIG. 1 is a flowchart of a method for detecting a defective defect of the present invention;

[0061] FIG. 2 is a specific flowchart of a defect detection module in the method of the present invention;

[0062] FIG. 3 is a view showing a defect detection result of the present invention.

detailed description

[0063] The present invention will be described in detail with reference to the accompanying drawings and examples for a further understanding of the present invention.

[0064] According to the drawings, the present invention provides a new method for detecting surface defect damage of a surface label, which can quickly and effectively realize the automatic detection of the defective defects of the bottle label. The present invention mainly includes image preprocessing, template area extraction, defect detection, and the results show four parts. The present embodiment is capable of quickly and effectively detecting damage to a ring-shaped label on the surface of a cylindrical bottle.

[0065] The specific implementation method of the present invention will be described in detail with reference to the following examples.

[0066] Example 1

[0067] Referring to FIG. 1 and FIG. 2, a surface tag breakage defect detection method based on template matching and similarity calculation of the present embodiment includes the following steps:

[0068] Image preprocessing

The image preprocessing includes a series of steps such as image acquisition, image segmentation, image stretching, and template image splicing, for obtaining the plane label area of ​​the image label area and the template image of the template image, the specific steps are as follows:

[0070] (1) Image acquisition:

[0071] Specifically, the CCD camera is used to collect the label image of the bottle in a low-brightness environment. The three cameras are in the same position as the center of the equilateral triangle, and a light source is placed directly below each camera to ensure that The direction of the center of the camera's camera is kept horizontal and through the center of the center of the bottle. The collected lossless template group is written as (Mcil, Mci2, Mci3) and the group to be measured is recorded as (Fcil, Fci2, Fci3). The collected images are gray scale, subscript 0 indicates that the image is the original image, and subscripts 1-3 indicate the sequence number in the image group.

[0072] (2) Image segmentation and stretching:

[0073] The lossless template initial image group (Mc ^ maMci3) and the initial image group (Fc ^ FaFci3) not only contain the tag information, but also the body information and shooting background information. In order to detect the defect condition of the label, it is necessary to set the label area Split out. At the same time, the surface of the bottle in the image is curved surface. In order to facilitate the subsequent detection, the surface image is stretched by the camera calibration to obtain the corresponding plane image. (MS1, MS2, MS3) and the label image plane group (F1, F2, F3), subscript (F1, F2, F3) are obtained by dividing and stretching the six images together. S indicates that the image is a plane label area image that has been divided and stretched. details as follows:

[0074] Image segmentation: Image acquisition is performed in a low-light environment with no complicated background and segmentation operations are relatively simple. Because the position of the bottle is relatively fixed when the image is collected, that is, the position of the label area in the collected image is relatively fixed, the ROI area larger than the label area can be set in advance, which contains the label area and a small amount of background area, Canny operator for edge detection, you can accurately segment the label area.

[0075] Image Stretching: The segmented image is stretched to convert it from a curved image to a corresponding planar image. In this embodiment, the Zhang Zhengyou plane calibration method is used to obtain the camera's internal and external parameters, and the curved image is transformed into a planar image. OpenCV as an open source visual library provides a variety of calibration methods, which contains based on Zhang Zhengyou plane calibration algorithm to achieve the calibration method can be directly called.

[0076] (3) Template image splicing:

[0077] In order to facilitate the subsequent detection of the image to be detected in the template map to find the corresponding mapping image, you need to stitch the template image group to obtain its panoramic image. In the template label plane image group (Ms1, Ms2, Ms3), the image contents are overlapped in two ways. By splitting the SURF feature and feature point matching, the three images are concatenated to obtain the template panorama image M.

[0078] Template area extraction

[0079] There are three images in the image group to be measured, corresponding to a certain area in the template panorama M, respectively. (F ^ F2J3) in the template panorama image M, and the corresponding regions are respectively separated to obtain the mapping corresponding to (F1J2J3) by using the template matching method to find out the corresponding regions of the image in the template panorama image M Template Image Group (MhMhM3).

[0080] Defect detection

(F1, M1), (F2, M2), (F3, M3) are divided into three groups, which are different from the images in the mapped template image group (MhMhM3). [0081] The test label plane image group (FhF2J3) Defect detection processing. Taking (F1, M1) as an example, by calculating the similarity between the image to be measured ^ and the template image M1, if the similarity is lower than the set threshold, it can be considered that there is a large difference between the image to be measured and the template image. Damage defects exist. Similarity calculation method using template matching method and feature similarity FSIM calculation of organic combination of the way, not only can determine the presence of broken defects, but also to achieve the location of damaged defect area, the specific process is as follows:

[0082] (1) Set the width w, the height h, the column direction step col_step, the row direction step ro w_step the sliding window, the synchronization traversing the image decrement and the application, memorize an image to obtain the number of blocks The

[0083] (2) Set the template matching similarity value threshold tl and t2 (tl <t2), FS top similarity value threshold si. The greater the similarity value indicates that the higher the similarity between the graph to be plotted and the template, the less likely it is to break the defect.

[0084] ⑶ Piecewise image F1 and template image M1 The current window of the block image is recorded as fk, mk (l k m). Calculate the similarity of its template matching. The template matching uses the normalized squared difference matching method, and the similarity value is t (OStSl, the larger the value is, the more similar the t = 1). If t <tl, the plotted map fk and the template map mk similarity is small, can determine the existence of broken defects in the graph; if t> t2, to be measured map fk and template map mk similarity is high, If there is no broken defect in the graph, if tl <t <t2, the graph fk and the template map mk have similar similarity, can not judge, and proceed to the next step.

[0085] (4) the similarity is moderate (ie, tl <t <t2) to be measured map fk and template map mk, using FSIM calculation method to calculate the similarity of the two maps, the similarity value of 8. If there is no damage defect exists in the graph, if s <sl, it is judged that there is a breakage defect in the graph to be measured. The similarity of FSIM is calculated as follows:

[0086] FSIM calculation is versatile, in order to avoid the k subscript on the subsequent expression of the interference, will need to calculate the similarity of the two images were recorded as A (X) and f2 (x), then

[0087]

[0088] where

[0089]

[0090] Let a = ^ = USpc (X) denote the phase similarity of h (X) and f2 (X) at pixel X, as follows:

[0091]

Sc (X) represents the gradient similarity of (X) to f2 (X) at pixel X, as follows:

[0093]

[0094] The above two formulas

[0095] I) TgT2 is a constant, used to prevent the denominator is 0, take! ^ = 0.85, T2 = 160.

[0096] 2) G1 (X) and G2 (X) represent the gradient amplitudes of the images h (X) and f2 (X) at the pixel X, respectively. The gradient amplitude G (X) of a pair of images f (X) is calculated as follows

[0097]

[0098] where Gx (X) and Gy (X) are the partial derivatives of the image in X and y at pixel X, which can be obtained using the sobel operator

[0099]

[0Ί00] 3) PCi (X) and PC2 (X) represent the phase coincidence of the image fi (X) and f2 (X) at pixel X, respectively. The phase coincidence PC (X) of a pair of images f (X) is calculated as follows

[0101]

[0102] where An (X) is the amplitude of the η cosine component, Δφη (χ) is the phase offset function, expressed as

[0103]

[0104] qUx) is the local phase of the Fourier transform at pixel X,

Is the weighted average of the local phase of all Fourier transform components at X, T is the estimated noise, ε is a small normal quantity, 0.001 is obtained, the denominator is 0, W (X) is the filter band weighting function,

[0105]

[0106] where

[0107]

[0108] Amax (X) is the maximum corresponding amplitude of the filter bank at X, the constant c is 0.4, g is 10, and ε is zero.

[0109] (5) Repeat step (3) until the completion of all the block image similarity calculation.

[0110] where the parameter settings are as follows:

[0111] 1) Sliding window size and step size: including width w, height h, column direction step length COl_step, row direction step row_step, specific width w take 1/4 of image width, height h 1 of image height / 4, column to the step size col_step take the image width of 1/8, row to step size row_step take the image height of 1/8.

[0112] 2) Similarity threshold: including template matching similarity lower limit threshold tl and upper limit threshold t2, FS top similarity si, specifically tl = 0.9, t2 = 0.95, sl = 0.8.

[0113] The results show

[0114] The sliding window synchronizes the template image and the image to be measured synchronously and calculates the similarity between the corresponding template and the image to be measured to determine whether the current block has a defective defect. If there is a defective defect, In the corresponding position to mark the final display after the marked image to be tested.

(A3) is the initial image of the group to be measured, and Fig. (Bl) is the segmentation of the image to be measured in which one of the labels is damaged. (B3) is the result of the test, and the box marked area indicates that there is a defect in the block. As can be seen from Fig. 3, the proposed scheme of the present invention can realize the panoramic splicing of the image, effectively detect the defect defect of the label and locate the defect.

The present invention and its embodiments have been described with reference to the present invention, and the description thereof is not limited thereto, and only one of the embodiments of the present invention is shown in the drawings, and the actual structure is not limited thereto. Therefore, it will be understood by those skilled in the art that the structural aspects and embodiments similar to those described in the present invention are not to be inventive without departing from the inventive object of the present invention, and should fall within the scope of the present invention The