Image processor, fingerprint matching device, image processing program

【0001】

[Technical field to which the invention belongs]

The present invention relates to an image processing device, a fingerprint collation device, and an image processing program that process image data in which each pixel is represented by a multi-valued pixel value, which is captured by a one-dimensional image pickup element in which a plurality of image pickup elements are arranged. ..

【0002】

[Conventional technology]

In recent years, as a device for identifying a person, a fingerprint collation device (image processing device) for identifying a person by reading a person's fingerprint pattern as image data and executing a collation process on the fingerprint image is used. It is becoming like. In the fingerprint collation device (image processing device), for example, a fingerprint image is read by using a one-dimensional image sensor in which a plurality of image sensors are one-dimensionally arranged.

【0003】

Generally, in an image processing device that reads an image by a one-dimensional image sensor, data for adjustment is read by reading a reference white and black image in order to correct the influence of differences in characteristics of each image sensor and lens optical system. Need to be generated.

【0004】

For example, an image read by scanning m pixels in the y-axis direction with a one-dimensional image sensor in which n image sensors are arranged in the x-axis direction is image data of n × m pixels. Here, the data obtained by reading the reference white image is W = {w_ij| i = 1 ... n, j = 1 ... m}, B = {b = {b_ijIf | i = 1 ... n, j = 1 ... m}, the white level adjustment data aw_i, Black level adjustment data ab_iCan be calculated, for example, by the following equations (1) and (2).

【0005】

Also, the pixel value of each pixel of the read image data D = {d_ij| i = 1 ... n, j = 1 ... m} can be adjusted according to the following equation (3), for example. Where d'_ijIs the adjusted pixel data, and depth is the maximum pixel value of one pixel (for example, 255 when represented by 8 bits per pixel).

【0006】

[Number 1]<img file="JP2004362248A_D0001.tif" /> 【0007】

Further, conventionally, in order to correct the gradation characteristic of a scanned image, a device for adjusting the characteristic of shading correction has been considered (for example, Patent Document 1). In this apparatus, pre-reading (pre-scan) is performed, an adjustment value is calculated based on the obtained image data, and the reading condition is changed (corrected) using the adjustment value to execute the main reading. The adjustment value calculates the maximum value and the minimum value in the acquired image data, and is based on the calculated maximum value and the minimum value and the correction value instructed for the calculated maximum value and the minimum value. calculate.

【0008】

[Patent Document 1]

Japanese Unexamined Patent Publication No. 2000-270216 [0009]

[Problems to be Solved by the Invention]

As described above, in the conventional method of generating adjustment data in image correction, adjustment data is generated for each device in order to correct the influence of differences in characteristics of each image sensor, lens optical system, etc. for each device. It had to be generated. Further, in order to adjust the image by this method, it is necessary that the environment when using the scanner is fixed (an environment with a constant brightness or an environment in which outside light does not enter). If the environment is not constant, more adjustment data for each environment is required.

【0010】

Further, also in the apparatus described in Patent Document 1, it is necessary to perform pre-reading (pre-scan) and calculate the adjustment value based on the obtained image data.

【0011】

The present invention has been made in view of the above problems, and it is possible to make corrections from the image data captured as a processing target to eliminate the influence of variations in each image sensor of the one-dimensional image sensor, the lens optical system, and the like. It is an object of the present invention to provide a possible image processing device, fingerprint matching device, and image processing program.

【0012】

[Means for solving problems]

The present invention is an image processing device that processes image data in which each pixel is represented by a multi-valued pixel value, which is captured by a one-dimensional image pickup element in which a plurality of image pickup elements are arranged. For example, the first pixel value as the maximum value and the second pixel value as the minimum value are detected, and the pixel read by the image sensor from the detected first pixel value and the second pixel value is detected. The pixel value width is detected. Then, for each pixel of the image data read by the image sensor, normalized data representing the ratio of the pixel value to the pixel value width is generated, and further, the average value of the normalized data is generated. Based on the calculated average value and the pixel value that can be taken by the pixel, the pixel value of each pixel of the image data read by the image sensor is corrected, for example, based on the ratio of the two.

【0013】

In this way, the image data read by the one-dimensional image sensor as the processing target is corrected based on the pixel value of each pixel obtained from the image data, so that the image correction can be performed in advance. It is possible to eliminate the need for pre-scanning and generation of adjustment data.

【0014】

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an electronic circuit of a mobile phone equipped with a function of an image processing device according to an embodiment of the present invention. A mobile phone is realized by reading a program recorded on a recording medium and utilizing a computer function whose operation is controlled by the read program. The image processing apparatus in the present embodiment reads an image of a fingerprint pattern (hereinafter referred to as a fingerprint image), which is a characteristic pattern of a skin ridge, from the fingertip (palm side of the fingertip) of a person to be a subject, and this fingerprint. It is assumed that it is configured as a fingerprint collation device that executes collation processing on an image.

【0015】

The mobile phone shown in FIG. 1 is configured by connecting the CPU 10 to various devices such as a storage device 12, a RAM 14, a call unit 16, a display unit 18, a key unit 19, and a fingerprint reader 20 via a bus. ..

【0016】

CPU10 realizes various functions by executing a program stored in the program area of RAM14. In addition to controlling the function as a mobile phone, the CPU 10 controls a fingerprint collation process (image processing) that reads a fingerprint image by the fingerprint reading unit 20 and collates the fingerprint image. The fingerprint collation processing (image processing) includes an image data correction processing for correcting variations in the image pickup element, the lens optical system, and the like with respect to the image data read from the fingerprint reading unit 20.

【0017】

The storage device 12 stores programs, data, and the like, and is read out as needed and stored in the RAM 14. In the present embodiment, in addition to the image collation program (image processing program), registered fingerprint image data or the like used as a collation source for fingerprint collation is recorded.

【0018】

The RAM 14 stores programs and various data and is accessed by the CPU 10. In addition to various programs for controlling the mobile phone, the fingerprint reading unit 20 reads the fingerprint image and executes fingerprint verification on the fingerprint image. An image matching program (image processing program) and the like are stored. When the fingerprint collation process is executed by the image collation program, the fingerprint image data read by the fingerprint reader 20 is recorded, and various data used for the process are recorded.

【0019】

The call unit 16 is a unit for performing wireless communication as a mobile phone. The display unit 18 displays various data and the like when executing various functions realized by the CPU 10. The key unit 19 is composed of a plurality of keys including numeric keys for inputting telephone numbers and various function keys. The fingerprint reading unit 20 reads fingerprint image data, and is provided at a position where the fingerprint reading operation is easy, such as the upper part of the front surface as shown in the appearance example of the mobile phone of FIG. The fingerprint reader 20 in the present embodiment includes a light source 21, a lens optical system (selfock lens 22), a one-dimensional image sensor 24, an image control circuit 26, an A / D conversion circuit 28, and a transparent rotating roller 29. A part of the outer peripheral surface of the transparent rotating roller 29 is exposed to the outside through a slit provided in the housing of the mobile phone. The exposed portion of the transparent rotating roller 29 serves as a fingerprint image reading unit. When reading a fingerprint image, the fingertip of the subject is pressed against the reading unit, and in that state, the transparent rotating roller 29 is rotated in a predetermined direction (direction perpendicular to the rotation axis of the transparent rotating roller 29). It is done by moving while (see Fig. 6). The slit provided on the surface of the housing may have a width sufficient to allow the user to press the fingertip against the transparent rotating roller 29 to rotate the transparent rotating roller 29. Therefore, in order to read the fingerprint image, it is not necessary to secure a reading surface in which the entire fingerprint is accommodated, and the mounting area of the fingerprint reading unit 20 (transparent rotating roller 29) on the housing surface can be reduced.

【0020】

In the fingerprint reading unit 20, the light reflected from the fingertip, which is the subject irradiated from the light source 21 and pressed (contacted) with the reading unit, passes through the transparent rotating roller 29 and is transmitted to the one-dimensional image sensor 24 by the self-fock lens 22. It is focused. Under the control of the image pickup control circuit 26, the light collected via the self-fock lens 22 is photoelectrically converted by the one-dimensional image pickup device 24, and further converted as image data representing a fingerprint pattern by the A / D conversion circuit 28. The image data from the one-dimensional image sensor 24 is buffered in the RAM 14 at a predetermined timing.

【0021】

FIG. 3 shows a schematic configuration (side sectional view) of the mechanical portion of the fingerprint reading unit 20. As shown in FIG. 3, the housing of the mobile phone is provided with a slit along the rotation axis of the transparent rotating roller 29 so that a part of the outer peripheral surface of the transparent rotating roller 29 is exposed as a fingerprint image reading portion. ing. The transparent rotating roller 29 is made of a transparent material such as acrylic or glass so as to transmit light, and is mounted so as to expose a part of the outer peripheral surface from a slit provided in the housing and rotate. .. Further, the transparent rotating roller 29 has a hollow inside, and the inside of the roller is hollow, and the light source 21 (for example, LED (Light Emitting Diode)), the lens optical system (Selfoc lens 22), and the one-dimensional image sensor 24 are hollow. The image sensor including the above is mounted. These imaging function units are mounted so as not to be interlocked with the rotation of the transparent rotating roller 29.

【0022】

The Selfock lens 22 forms an image of a portion (reading portion) of the transparent rotating roller 29 on which the subject is in pressure contact on the one-dimensional image sensor 24. The imaging optical system is not limited to the selfock lens, and an imaging optical system composed of a rod lens group can be used.

【0023】

The one-dimensional image sensor 24 is composed of a CCD line sensor, a CMOS line sensor, or the like, and is mounted so that the arrangement of each image sensor 24a is parallel to the rotation axis of the transparent rotating roller 29, as shown in FIG. Has been done. By making the transparent rotating roller 29 hollow and mounting the imaging function unit inside, the mounting area of the fingerprint reading unit 20 on the housing surface and the mounting volume inside the housing are reduced.

【0024】

Next, the operation of the fingerprint verification device mounted on the mobile phone in this embodiment will be described. FIG. 5 is a flowchart for explaining the fingerprint collation process by the fingerprint collation device. First, when the execution of the fingerprint collation process is instructed through, for example, the key unit 19, the CPU 10 activates the image collation program to start the fingerprint collation process. The CPU 10 executes a fingerprint image capture process for capturing the fingerprint image to be collated through the fingerprint reader 20 (step S1).

【0025】

FIG. 6 shows an example of how to move a finger when the fingerprint reading unit 20 captures a fingerprint image. First, as shown in FIG. 6A, the fingertip of the subject is pressed against the reading unit on the transparent rotating roller 29, and in that state, a predetermined direction (direction perpendicular to the rotation axis of the transparent rotating roller 29). It is moved while rotating the transparent rotating roller 29. FIG. 6 (b) shows how the finger is moved toward the front.

【0026】

As shown in FIGS. 6 (a) and 6 (b), the one-dimensional image pickup element 24 executed in parallel with the rotation axis of the transparent rotating roller 29 by moving the transparent rotating roller 29 while rotating it at the fingerprint portion. , The entire fingerprint part can be scanned.

【0027】

The image signal output from the one-dimensional image sensor 24 is converted into digital data (fingerprint image data) by the A / D conversion circuit 28. The fingerprint image data converted by the A / D conversion circuit 28 is output from the fingerprint reading unit 20 and recorded in the RAM 14. In the fingerprint image data, for example, each pixel is represented by a pixel value of multiple values (for example, 0 to 255). The fingerprint image data optically scanned by the fingerprint reader 20 is biased toward pixel values due to the influence of variations in each image sensor of the one-dimensional image sensor 24 and the self-fock lens 22 (lens optical system). Variations may be included.

【0028】

Next, in order to correct the bias and variation of the pixel values, the CPU 10 performs image data correction processing on the fingerprint image data read by the fingerprint reading unit 20 (step S2). In the image data correction processing in the present embodiment, each pixel value is corrected based on the pixel value of each pixel of the fingerprint image data read as the processing target.

【0029】

FIG. 9 is a flowchart for explaining the details of the image data correction process. As shown in FIG. 7, the fingerprint image data read by the one-dimensional image sensor 24 has the data read by each image sensor arranged in the y-axis direction. In the fingerprint image data, the image data in one row cut out in the y-axis direction has similar properties to the image data in the other rows corresponding to each image sensor. That is, since the fingerprint pattern consists of parts corresponding to either the ridges or valleys of the fingerprint and is an almost uniform pattern, the image data in each column has properties (such as the ratio of pixels of each pixel value). And the average value do not differ significantly. Each pixel is corrected as described below by utilizing the characteristics of the fingerprint image data.

【0030】

First, for each image sensor of the one-dimensional image sensor 24, the pixel values that are the maximum value (first pixel value) and the minimum value (second pixel value) are detected from the image read by each image sensor (2). Step A1). Here, each pixel (pixel value) of the image data read by each image sensor has D = {d, as shown in FIG._ijIt is represented by | i = 1 ... n, j = 1 ... m}. Maximum pixel value max for each image sensor_iAnd minimum min_iIs calculated by the following equations (4) and (5).

【0031】

max_i= MAX (d_ij| j = 1 ... m); 1 i n ... (4) min_i= MIN (d_ij| j = 1 ... m); 1 i n ... (5) Here, MAX () and MIN () are functions for finding the maximum and minimum values, respectively.

【0032】

Next, the width of the data for each image sensor, that is, the maximum and minimum pixel value widths of the pixel values obtained in step A1 are calculated by the following equation (6) (step A2).

【0033】

range_i= max_i-min_i 1 i n ... (6) Next, for each image sensor of the one-dimensional image sensor 24, for each pixel captured by the image sensor, the pixel value d_ijPixel price range range_iNormalized data d'representing the ratio to_ijIs calculated by the following equation (7) (step A3).

【0034】

[Number 2]<img file="JP2004362248A_D0002.tif" /> 【0035】

Further, for each image sensor, the normalized data d'obtained for each pixel captured by each image sensor._ijThe average value of is calculated by the following formula (8) (step A4).

【0036】

[Number 3]<img file="JP2004362248A_D0003.tif" /> 【0037】

And the normalized data d'_ijCalculate the adjusted pixel data by combining the average values of (step A5). That is, the CPU 10 has the normalized data d'calculated in step A4._ijBased on the average value of and the pixel value (depth) that the pixel can take, the pixel value of each pixel of the image data read by the image sensor is corrected by the following equation (9).

【0038】

[Number 4]<img file="JP2004362248A_D0004.tif" /> 【0039】

Here, depth is the maximum pixel value of one pixel (for example, 255 when represented by one pixel with 8 bits). When generating pixel data adjusted by proportional calculation as expressed by the above-mentioned equation (9), the calculated pixel value may exceed the maximum value or the minimum value that one pixel can take. In that case, clipping processing is performed to adjust the pixel value so that one pixel is within the range that can be taken.

【0040】

Next, the CPU 10 executes a normalization process on the fingerprint image data corrected by the image data correction process in order to match the inclination and size with the registered fingerprint image data as the collation source (step S3).

【0041】

Then, the CPU 10 executes the collation of the normalized fingerprint image data with the registered fingerprint image data by the fingerprint collation process (step S4). Since the image data correction process is applied to the fingerprint image to be collated in the stage before the fingerprint collation process, a good collation result can be expected.

【0042】

The CPU 10 displays the result (OK, NG) of the fingerprint verification process on, for example, the display device 20 (step S5).

【0043】

By executing the image data correction process in this way, the fingerprint image data read by the one-dimensional image sensor 24 is varied in each image sensor of the one-dimensional image sensor 24 and the self-fock lens 22 (lens optical system). It can be corrected so as to eliminate such influences. In the image data correction process in the present embodiment, it is not necessary to create adjustment data or the like in advance, so that it is not necessary to perform pre-scanning for image correction in advance. Further, since the image correction corresponding to the change of the environment such as the external light at the time of reading the image is realized, it is possible to always obtain a stable fingerprint collation result regardless of the state of the environment.

【0044】

Next, modified examples (1) and (2) of the above-described embodiment will be described. Modification example (1) When dust or the like adheres to the surface of the transparent rotating roller 29 during the fingerprint image reading process, the dust portion has a pixel value different from the pixel value that the fingerprint pattern can normally take. Therefore, the pixel value may correspond to the maximum value or the minimum value. In such a case, it becomes impossible to correct the image correctly due to the influence of the pixel value of the pixel corresponding to the dust portion. In such a case, in the corrected fingerprint image data, the column corresponding to the dust portion has a pixel value different from that of the other columns as a whole, and a pattern such as a streak (or line) is generated.

【0045】

In the above description, it is assumed that the maximum and minimum pixel values are detected from the pixels captured by each image sensor of the one-dimensional image sensor 24, but in the modified example (1), it is specified from each of the maximum and minimum values. By detecting the pixel values of the pixels in the same order, the image data correction process is executed so as not to be affected by the pixels having the invalid pixel values read due to the influence of dust or the like.

【0046】

FIG. 10 is a flowchart showing detailed processing of the modification (1) corresponding to step A1 in FIG. It should be noted that the processing of other parts in the image data correction processing (FIG. 9) is performed in the same manner as in the above-described embodiment, and detailed description thereof will be omitted.

【0047】

When the method of the modification (1) is used, a designated value indicating the order of pixels with respect to the image data captured by each image sensor is recorded in the storage device 12. This specified value may be recorded in the storage device 12 in advance, or may be input and recorded by the user through the key unit 19. For example, it is assumed that the order from the maximum value (for example, x) and the order from the minimum value (for example, y) to be detected from the pixels read by each image sensor are recorded.

【0048】

First, the pixel group read by the first image sensor of the one-dimensional image sensor 24 is specified (step B1), and the pixel group is sorted (ascending or descending) based on each pixel value (step B2).

【0049】

Next, the pixel value of the xth pixel from the largest pixel value is set as the maximum value of the pixel width for generating normalized data, and the pixel value of the yth pixel from the smallest pixel is set as the maximum value. Detect as the minimum pixel width for generating normalized data (step B3).

【0050】

Next, the next image sensor to be detected for the maximum value and the minimum value is specified (step B4), and similarly, the pixel value of the xth pixel from the maximum value from the pixel group read by this image sensor. And detect the pixel value of the yth pixel from the minimum value (steps B5, B2, B3).

【0051】

For example, the pixel value of each pixel of the read image data D = {d_ijFor | i = 1 ... n, j = 1 ... m}, the data is sorted for each image sensor, and the data {d captured by the i-th image sensor._ij| j = 1 ... m}, the data with the largest pixel value is sd_i1, Sd the second largest data_i2Data sorted in descending order like {sd_ikWhen expressed as | k = 1 ... m}, max for normalizing the data for each image sensor._iAnd min_iIs calculated as follows.

【0052】

max_i= sd_ix ... (10) min_i= sd_{i (m-y + 1)}... (11) In this way, the maximum value max of the pixel value width for generating the normalized data according to the modification (1)._i, Minimum value min_iBy calculating, even if there are pixels with pixel values that do not normally exist in the fingerprint image due to dust or the like adhering to the transparent rotating roller 29, the maximum value is obtained after removing these pixels. Since the minimum value is detected, the influence of dust and the like can be removed and the image data can be corrected.

【0053】

Modification example (2) In the above-mentioned modification example (1), it is assumed that the xth and yth pixel values of the maximum value and the minimum value are detected from the pixel group read by each imaging element, but the maximum value is specified. Even if the maximum and minimum values of the pixel width for generating normalized data are calculated from the set of pixels up to the third (pixel group) and the set of pixels from the minimum value to the specified number (pixel group). good. Here, for example, the average value of each set of pixels (pixel group) is set as the maximum value and the minimum value of the pixel width.

【0054】

FIG. 11 is a flowchart showing the detailed processing of the modification (2) corresponding to step A1 in FIG. It should be noted that the processing of other parts in the image data correction processing (FIG. 9) is performed in the same manner as in the above-described embodiment, and detailed description thereof will be omitted. When the method of the modification (2) is used, a designated value indicating the order of pixels with respect to the image data captured by each image sensor is recorded in the storage device 12 in the same manner as the modification (1). It shall be.

【0055】

First, the pixel group read by the first image sensor of the one-dimensional image sensor 24 is specified (step C1), and the pixel group is sorted (ascending or descending) based on each pixel value (step C2).

【0056】

Next, the average of the pixel values of the pixels having the largest pixel value to the xth pixel is calculated, and this average value is set as the maximum value of the pixel width for generating the normalized data, and y from the smallest pixel. The average of the pixel values of the second pixel is calculated, and this average value is set as the minimum value of the pixel width for generating the normalized data (step C3).

【0057】

Next, the next image sensor to be detected for the maximum value and the minimum value is specified (step C4), and similarly, the pixels from the maximum value to the xth pixel from the pixel group read by this image sensor. The average of the values and the average of the pixel values from the minimum value to the y-th pixel are calculated and used as the maximum and minimum values, respectively (steps C5, C2, C3).

【0058】

For example, the average of the pixel values of the largest pixel to the xth pixel and the average of the pixel values of the smallest pixel to the yth pixel are calculated by the following equations (12) and (13).

【0059】

[Number 5]<img file="JP2004362248A_D0005.tif" /> 【0060】

In this way, stable results can be obtained by calculating the maximum value and the minimum value according to the modified example (2). That is, in the fingerprint image data, the data obtained from one image sensor includes many image portions corresponding to the ridges (peaks) and valleys (valleys) of the fingerprint. Therefore, in the set of pixels (pixel group) from each of the maximum value and the minimum value to the specified th, there are many pixels with similar pixel values, and by finding the average value of the pixel group, dust etc. Is attached to the transparent rotating roller 29, the influence of the pixels corresponding to the dust portion can be reduced.

【0061】

In the above description, the case where the image data reading device according to the present embodiment is mounted on a mobile phone is described as an example, but it may be mounted on another information device, or the image data reading device may be mounted. It may be configured independently as.

【0062】

Further, in the above description, the case of reading the fingerprint pattern is described, but it can also be applied to the case of reading image data such as a palm print pattern. That is, the images captured by the fingerprint reader 20 (one-dimensional image sensor 24) have similar average values and maximum / minimum values in the y-axis direction between the image sensors in the same manner as the fingerprint image described above. It can be applied in some cases.

【0063】

Further, the present invention is not limited to the above-described embodiment, and can be variously modified at the implementation stage without departing from the gist thereof. In addition, the contents of the above-described embodiments may be combined as appropriate as possible. The above-described embodiments include inventions at various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constitutional requirements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if an effect can be obtained, a configuration in which the constituent requirements are deleted can be extracted as an invention.

【0064】

[Effect of the invention]

As described above, according to the present invention, the influence of variations in each image sensor of the one-dimensional image sensor, the lens optical system, etc. is eliminated from the image data captured as the processing target without performing prescan or generation of adjustment data. It is possible to make corrections.

[Simple explanation of drawings]

FIG. 1 is a block diagram showing an electronic circuit configuration of a mobile phone equipped with an image processing device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an external example of a mobile phone on which the fingerprint reading unit 20 according to the present embodiment is mounted.

FIG. 3 is a diagram showing a schematic configuration (side sectional view) of a mechanical portion of the fingerprint reading unit 20 in the present embodiment.

FIG. 4 is a diagram showing an arrangement of each image pickup device 24a in the one-dimensional image pickup device 24.

FIG. 5 is a flowchart for explaining a fingerprint collation process by the fingerprint collation device according to the present embodiment.

FIG. 6 is a diagram showing an example of how to move a finger when capturing a fingerprint image by the fingerprint reading unit 20.

FIG. 7 is a diagram showing an example of fingerprint image data read by the one-dimensional image sensor 24.

FIG. 8 is a diagram showing an array of each pixel (pixel value) of image data read by each image sensor.

FIG. 9 is a flowchart for explaining the details of the image data correction process in the present embodiment.

FIG. 10 is a flowchart showing detailed processing of a modification (1) corresponding to step A1 in FIG.

FIG. 11 is a flowchart showing detailed processing of a modification (2) corresponding to step A1 in FIG.

[Explanation of symbols]

10 ... CPU12 ... Storage device 14 ... RAM 16 ... Call unit 18 ... Display unit 19 ... Key unit 20 ... Fingerprint reader 21 ... Light source 22 ... Selfock Lens (lens optical system) 24 ... 1D image sensor 24a ... Image sensor 26 ... Image control circuit 28 ... A / D conversion circuit 29 ... Transparent rotating roller