Method to conduct fingerprint verification and a fingerprint verification system

The invention relates to a method to conduct fingerprint verification and a fingerprint verification system.

The prior art discloses some different approaches to improve fingerprint verification systems with the goal to decrease the false acceptance rate in cases of fraud. One approach to fool known fingerprint verification systems consists to present them artificial fingers as described in <nplcit id="ncit0001" npl-type="b"><text>Matsumoto et al. "Impact of artificial 'gummy' fingers on fingerprint systems" in Optical Security and Counterfeit Deterrence Techniques IV, Bellingham, Washington, 2002-01-23/25, Vol. 4677, The International Society of Optical Engineering, pages 275/288</text></nplcit>.

Fingerprint recognition systems have been widely used in forensic applications. Originally, conventional technology was used with inked fingerprint dab and paper card based databases, wherein human experts conducted retrieval and matching. Since some years electronic and computer software based databases and screening methods had been developed and put into practice.

More recently, with the increased availability of fingerprint scanning devices, fingerprint technology has become one of the most important biometric identification methods. However, as every chain link in a security system, electronic fingerprint recognition systems are a possible target of unlawful attack and fraud.

In <patcit id="pcit0001" dnum="US2003039381A"><text>US 2003/039381</text></patcit>, a method is a method is described in which minutia points are classified by the distances of the nearest neighbour minutia points in angular quantized neighbourhood around the original minutia point. This method provides some local topological information about the minutia point and its immediate neighbours. However it does not provide global information about the minutia embedding in the minutia pattern geometry to identify in an unique way two corresponding minutia points. The method allows to select candidates for further processing out of a database.

<nplcit id="ncit0002" npl-type="s"><text>F. Pernus et al. discloses in the article "Minutiae based fingerprint recognition" in Proc. of the 5th Int. Conf. on pattern recognition, Miami, 1980, pages 1380-1382</text></nplcit>, a method to recognize corresponding fingerprints through minutiae matching using specific local features of the minutiae. The presented feature array of the minutia however does not characterize the topological embedding of the specific minutia in the geometrical pattern of all minutiae nor does it provide a standardized representation of this topological embedding in the global minutiae geometry pattern.

Based on this prior art, it is an object of the invention to provide an improved fingerprint verification system, which is capable of returning detailed matching information including precise registration of the template and the query fingerprint scan image.

This object is achieved with a method having the features of claim 1.

The matcher architecture uses a two stage matching, i.e. first a topological matching followed by a geometrical matching. This concept proved to be very efficient and transparent. One major advantage of the method according to the invention is that the results returned by the matching process allows for a precise registration template and query images.

Further advantageous embodiments are characterized through the features mentioned in the dependent claims.

Throughout this disclosure the following definitions shall be used. Fingerprint recognition is a general term for fingerprint identification as well as verification.

While fingerprint identification refers to the process of matching a query fingerprint against a template fingerprint database to establish the identity of an individual, fingerprint verification refers to determination of whether two fingerprints are from the same finger or not. Since verification is a comparison of a query fingerprint against an enrolled template fingerprint, it is also termed as one-to-one matching. Identification, on the other hand, is termed as one-to-many matching.

A fingerprint scan is a digital representation of a fingerprint as a result of a data acquisition process with a fingerprint-scanning device. In this work the typical data format of a raw fingerprint scan is the BMP-image file format, but there are multiple other valid formats.

A live finger is a genuine finger, forming part of the living body of a person whose identity should be verified. A gummy finger or an artificial finger is not genuine. It is artificially produced and used to deceive fingerprint systems.

From a fingerprint scan, an extraction process locates features in the ridges and furrows of the friction skin, called minutiae. Points are detected where ridges end or split, and their location, type, orientation and quality are stored and used for search.

There are 30 to 50 minutiae on a typical sensor scan, and matching takes place on these points rather than on all the pixels in the fingerprint image. With the above-mentioned sensor from Infineon the fingerprint image comprises 288 times 224 = 64'512 pixels which results in a pixel size of approx. 0.05 mm typical for such sensors.

Image registration is the process of aligning two or more images of the same scene. One image, called the base image, is considered the reference to which the other images, called input images, are compared. The object of image registration is to bring the input image into alignment with the base image by applying a spatial transformation to the input image.

The invention is now described by way of example on the basis of the accompanying drawings: <dl id="dl0001" compact="compact"><dt>Fig. 1</dt><dd>shows a target constructed around a minutiae point according to a first embodiment of the invention, and</dd><dt>Fig. 2</dt><dd>shows a target constructed around a minutiae point according to a second embodiment of the invention, and</dd><dt>Fig. 3</dt><dd>shows a diagram of the overall gray level image produced by a gummy finger detected according to another embodiment of the invention.</dd></dl>

A method for an automatic fingerprint identification and/or verification system comprises at least three main processing stages namely, <ol id="ol0001" compact="compact" ol-style=""><li>a.) fingerprint scanning</li><li>b.) feature extraction, e.g. fingerprint minutiae extraction, and</li><li>c.) matching.</li></ol>

The following description briefly explains the necessary steps relating to steps a.) and b.) before describing in depth step c.) with reference to the enclosed drawings.

The invention may be used with any conventional fingerprint scanning method, i.e. the aim of the scanning method is the creation of fingerprint images. Typical fingerprint scanning systems are disclosed in the above-mentioned paper of Matsumoto et al. A capacitive fingerprint sensor as used within the embodiments of the apparatus according to the invention is disclosed in "Microsystems for Biometrics FingerTIP-FTF 1100 MF1 V2.0 CMOS Chip and System Data Book 3.3", Munich, Germany, 2000, Status 05.00, Infineon Technologies AG, CC Applications Group, 38 pp.. Such a system can be used to provide images, e.g. so-called BMP images with a given gray level palette.

There are several different approaches to conduct feature extraction, e.g. fingerprint minutiae extraction. One excellent description of feature extraction technology can be found in <nplcit id="ncit0003" npl-type="s"><text>Garris et al. "Users's Guide to NIST Fingerprint Image Software (NFIS)", Gaithersburg, MD, USA, 2001-04, National Institute of Standards and Technology, 186 pp</text></nplcit>.. Said authors have published a software package for fingerprint feature extraction, the so-called NIST/NFIS software, which can be directly used on the above mentioned stored fingerprint images to extract minutiae.

In contrast to the standardized minutiae feature extraction, minutiae feature matching technology has been developed in different directions. Examples of prior art are the US patents of Ferris-Powers <patcit id="pcit0002" dnum="US5631972A"><text>US 5,631,972</text></patcit>, Kovacs-Vajna <patcit id="pcit0003" dnum="US20010040989A"><text>US 2001/0040989</text></patcit>, <patcit id="pcit0004" dnum="US5960101A"><text>Lo-Bavarian US 5,960101</text></patcit>, <patcit id="pcit0005" dnum="US4135147A"><text>Riganati et al. US 4,135,147</text></patcit> and Sparrow <patcit id="pcit0006" dnum="US5631971A"><text>US 5,631,971</text></patcit>. The following description provides a different solution for the problems related to minutiae feature matching.

Fingerprint scans are subjected to translation, rotation and distortion. It is up to the apparatus and method according to this invention to provide a better solution than correlation or convolution pattern matching methods known from prior art. Usually fingerprint recognition uses the singularity points of a fingerprint, the so-called core and delta features. However, the use of these features is not always easy or possible. Solid-state sensor scans may only capture a fraction of the fingerprint area. The core/delta-based recognition does not cope with fingerprint distortion.

As mentioned the available fingerprint minutiae point pattern may be subjected to translation, rotation and distortion, just depending on how exactly the fingertip is actually positioned on and pressed against the scanning device. Also, because of missing or additional minutiae in either point sets (called query and template), a complete matching of all minutiae is generally not possible.

Fingerprint minutiae have additional point attribute information as minutiae type (ridge end or bifurcation and minutiae direction facilitating the matching. The relative directional orientation and the relative position of neighboring minutiae are almost invariant to translation, rotation and - in some extent - distortion.

The method used for verification is based on the insight that only corresponding pairs of template-minutiae and query-minutiae have to be matched. This mating will be achieved by matching of the local environment of the minutiae.

The result set is consistency checked in order to minimize the probability of falsely matched minutiae pairs.

The results will then be used to register the query and the template image, which in turn will then be used for an iterative geometrical matching of minutiae pairs. Finally, the matcher step returns a set of mated minutiae pairs.

In order to detect the use of a gummy finger a subsequent filter stage evaluates the low pass filtered gray level intensity of the query compared to the template.

The embodiment according to the invention uses live fingerprint data form a fingertip sensor providing an image of 224 times 288 pixels with 8bit-grayscale and a resolution of 513 dpi.

The scan of the template had been saved as BMP image beforehand but also a transformed format could be used.

The scan of the query is a second raw fingerprint scan and provided as BMP image.

Both images are submitted to the NIST/NFIS-mindtct-package from the above-mentioned authors Garris, Watson, McCabe and Wilson providing a binarized image, minutiae feature data and minutiae quality information. All minutiae are represented with the three values pixel coordinates, direction and quality. The pixel coordinates are given in x and y coordinates, although different representation systems may be used. One pixel of a binarized image according to this embodiment comprises only the values black or white, gray values are not used.

The method according to this invention comprises two stages: <ul id="ul0001" list-style="dash" compact="compact"><li>a topological matcher</li><li>a geometrical matcher.</li></ul>

The topological matcher compares the local environment of template and query minutiae. This allows finding the relative position of template and query images replacing the less efficient core/delta-method used within prior art. The result is a set of mated template-query minutiae pairs. The topological matcher function returns the necessary parameters for the subsequent image registration by the geometrical matcher.

The topological matcher starts with the data of the query minutiae represented in Fig. 1 with an overlaid target structure 101. Every vector of the query minutiae comprises at least a position indication and an angle indication. The position indication 102 may be the center pixel position of the minutiae in the array of the sensor (here: an array of 224 x 288). The angle indication 110 can be defined as the angle of an intrinsic feature of the minutiae or of its surrounding line structure in relation to a predefined 0 degree angle of the array of the sensor.

Then a first minutiae vector 102 of the query template (within the example numbered 6) is selected.

A sector distribution based on the position 102 and angle 110 of the first minutiae vector is defined. This may be a simple square area distribution (not shown in the drawings), e.g. with a side length of 20 pixels with the position of the first minutiae vector at relative position (0,0), wherein the squares are oriented along said angle indication 110.

Another simple sector distribution is shown in Fig. 1. This is a target 101 constructed around a minutiae point 102, which is per definition located in the "bull's eye" at the relative coordinates (0,0). The distribution is defined by concentric circles 103 around said point 102 and radial lines 104 starting from the relative position of the first minutiae vector 102, i.e. (0,0). Within the embodiment shown in Fig. 1 there are 16 sectors zones 104 having an angular opening of 22,5 degree and the concentric circles have a radius of 20 pixels. Within other embodiments (not shown) the radius may start with e.g. 10, 20 or 30 pixels and increase or decrease in a monotone way. In Fig. 1 every sector 105 has the same relative radius of 20 pixels. Therefore the distribution of Fig. 1 provides 160 different sectors 105. The X-Y coordinates in relation to the sensor image shown as rectangle 119 are shown with the lines 120 and 121. The numbering of the pixels starts in corner 122. Ring sectors 109 outside the rectangle 119 relate to sectors which are marked "non valid", i.e. the value of the validity vector of said sectors is 0 and therefore only such sectors are used in the scalar product being part of the query image 119 as well as part of the template image.

Now the number of further minutiae vectors in any sector 105 of the sector distribution is determined. Some sectors 106 are empty (null minutiae) and sectors 107 have received one further minutiae vector (numbered 0 and 1 and 3). Sectors 108 have couples of two further minutiae vectors (numbered 11 and 13; 15 and 18; 27 and 29). In other words, each ring sector 105, 106, 107, 108 is considered a bin, and all neighbor minutiae are binned into their ring sectors. The minutiae shown in Fig. 1 are only those having a minimum reliability, e.g. of 0.2 on a scale between 0 and 1.

The target having r=10 rings and s=16 sectors corresponds to an r times s dimensional feature vector containing the hits. In case of a square distribution this corresponds to r rows and s lines. This vector called P is the vector of the chosen minutiae (here numbered 6). It is assumed that this vector is robust against translation, rotation, distortion or incompleteness of the fingerprint image.

Ring sectors 109, being outside of the image borders 119, are marked invalid.

Instead of matching the minutiae pattern directly, the local environments, characterized by the sector distribution are matched.

The query set with m minutiae (m=29 in Fig. 1) produces a feature vector Pqk (k=1 to m). In the same way the stored template set with n minutiae produces a feature vector Pti (i=1 to n).

Preferably each feature vector is accompanied by a validity vector Pvti and Pvqk, respectively, with the components being equal to 1, if the sector element (e.g. 106, 107, 108) is inside the image borders 119, or being equal to 0, if the sector element (e.g. 109) is outside the image borders 119. Such validity vectors are used to mask invalid elements of the feature vectors in the matching process.

Furthermore the reliability of the minutiae is used. Usually all minutiae with a reliability of less than 20 % are discarded, i.e. the respective feature vector P is set to 0. The reliability can be read out from the mindtct-Routine of the NFIS/NIST software

The matching process consists in calculating the normalized (and validity masked) scalar product for every combination of Pti and Pqk. This results in a match matrix M.

For any given minutia v in the query minutia set, the corresponding minutia u in the template set is found by searching the maximum element in column v of M.

In order to reject unwanted false matches at an early stage the maximum element has to be larger than a threshold value t1 and the second runner has to be smaller than a second threshold value t2 times the maximum element in the row. Typical values are t1=0.5 and t2=0.8. If this condition is met a matching pair is found: minutia v of the query set can be matched with minutia u of the template set where u is the index of the maximum element in column v.

All other elements in column v of M are subsequently set to zero and the resulting matrix can then be used as a look up table for the mated minutiae pairs using array sort and index functions.

Then, a consistency check is performed according to all of the following criteria: <ul id="ul0002" list-style="dash" compact="compact"><li>the matching pair must be of the same minutia type (either bifurcation-bifurcation or ridge-ridge),</li><li>the relative minutia orientation in the result set must be the same in the template as in the query, and</li><li>the computed scalar product of the vectors leading from mated minutiae to its mated list neighbors must be within some tolerance limits in the template and in the query.</li></ul>

Pairs that do not comply with these criteria are kicked out of the set.

The topological matcher therefore returns a set of mated minutiae pairs (template/query). Their respective pixel coordinates are easily looked up with the output generated by the feature extraction process.

The geometrical matcher uses the image coordinates of the minutiae of the result set of the topological matcher to register the query image, the input image, to the template image, the base image. In other words, the query image is transformed into the same frame of reference as the template image.

The topological matcher has provided sets of mated minutiae in order to compute the image transformation parameters of the image registration process. The transformation will compensate for distortion within the convex hull of already mated minutiae. The following transformation types are used: "linear conformal" with at least two pairs of mated minutiae, wherein straight lines remain straight and parallel lines are still parallel; "affine" with at least three pairs of mated minutiae, wherein straight lines remain straight and parallel lines are still parallel, but rectangles become parallelograms; and "projective" with at least four pairs of mated minutiae, wherein straight lines remain straight but parallel lines converge towards vanishing points which may or may not be within the image. Such transformation functions can e.g. be realized with the Software packet Matlab and the function cp2tform. Said transformation function cp2tform finds the transformation coefficients exactly, if the number of control points is the minimum number. If there are additional points, than a least squares solution is found.

The transformed image is compared with the template image. For every query minutiae it is determined, if the closest template minutiae is within a certain pixel distance tolerance. With the use of a 288x244 pixel sensor with the embodiment of the invention a distance tolerance between 1 to 7 pixels, preferably 3 to 5 pixels is used. This represents a tolerated deviation (threshold) between 0,3 % and 3 %, preferably between 1 % and 2 %.

After finding a candidate match and the above mentioned distance tolerance check, a number of consistency checks is performed: <ul id="ul0003" list-style="dash" compact="compact"><li>comparison of the minutiae type (e.g. ridge ending or bifurcation),</li><li>comparison of the minutiae direction,</li><li>evaluation of sufficient reliability of the matching template (e.g. &gt; 20%), and</li><li>evaluation of sufficient reliability of the query template, e.g. evaluation of the quality of the minutiae used for the registration process (e.g. &gt; 20%).</li></ul>

If a candidate match passes all these checks, it will be added to the result set. The geometrical matching is an iterative process, repeated until no additional mated minutiae pairs can be found.

Further sector distributions may comprise a hexagonal grid or rectangular sectors, etc..

Fig. 2 shows a target constructed around a minutiae point according to a second embodiment of the invention. Same features receive the same reference numerals throughout all figures. Within Fig. 2 31 minutiae are shown wherein minutia with the number 4 is the chosen minutia with its intrinsic direction 110. The same principles apply to retrieve the minutiae pairs. It has to be noted that of course identical targets are to be used for template and query analysis. Furthermore it can be seen from Fig. 2 that there may be minutiae 113 which are not inside one of the sectors 105. They are simply discarded.

The general principle of operation for the embodiment shown in Fig. 1 and 2 is based on identical topological structures defined around each minutia which is mapped on an algebraic structure. The algebraic structures can be compared in such a way that identical or very similar topologic structures around minutiae can be detected.

The efficiency of the method and device can even be improved, if a method and device for the detection of gummy fingers is used.

The detection of gummy fingers is a method step, which can be used in a stand-alone realization for any fingerprint matching method and apparatus; or, the method can readily be used with the fingerprint matching method mentioned above as additional step.

Fig. 3 shows a diagram of the overall gray level image produced by a gummy finger detected according to another embodiment of the invention. The X- and Y-axis 201 and 202 relates to the absolute coordinates of the pixels of the image provided by the sensor. The Z-axis 203 relates to the relative gray value of the pixels.

A second order surface fit 204 was used to compute the principal curvatures of the surface fitted to the BMP gray levels.It is clear that it is possible to apply this method for any graphic image file comprising the fingerprint grayscale values, e.g. TIF images, GIF images or compressed formats as JPEG or PNG. Beside the template and query image coordinates of the mated minutiae (incl. their total number) the principal curvatures of the fitted surface to the image gray levels are provided.

The gray level distribution was low pass filtered, e.g. with a MATLAB sliding neighbourhood filter with a block size of 20x20 pixel and subsequently a second order surface was fitted to the filtered image data. Said second level surface is almost flat, i.e. the principal curvatures are almost zero for fingerprint scans of genuine fingers. Typical values for the curvatures of genuine fingers are 0.0001 to 0.0002.

The second order surface 204 shows the fitted surface. The darker areas 207 show the low pass filtered gray level distribution of the scan. Only gray level of those areas are taken into account, which can be brought into congruence and having a sufficient quality in the template as well as in the query, a value which can be generated with the mindtct routine of said NFIS/NIST package.

However, as it can be seen from Fig. 3 by comparison of the curvature of query and template, that, in case a fingerprint provided by a gummy finger was used, a negative curvature of the gray level distribution as compared to their live, genuine counterparts was experienced. Typical values for the curvatures of false fingers are -0.0001 to -0.0030.

In direction of the X-axis 201 the gray value is in the middle of the sensed surface by a difference 205 greater than at the borders. This can also be experienced in direction of the Y-axis 202, since the gray value is in the middle of the sensed surface by a difference 206 greater than at the borders.

This effect can be used to discard queries presented through the use of gummy fingers. It is possible to rely on one of the two difference values or to require the existence of both negative curvatures to make the decision relating to the genuineness of the presented finger.

The method makes use of the following function: <maths id="math0001" num=""><math display="block"><mi mathvariant="normal">z</mi><mo mathvariant="normal">=</mo><msub><mi mathvariant="normal">b</mi><mn mathvariant="normal">1</mn></msub><mo mathvariant="normal">+</mo><msub><mi mathvariant="normal">b</mi><mn mathvariant="normal">2</mn></msub><mo>⁢</mo><mi mathvariant="normal">x</mi><mo mathvariant="normal">+</mo><msub><mi mathvariant="normal">b</mi><mn>4</mn></msub><mo>⁢</mo><msup><mi mathvariant="normal">x</mi><mn mathvariant="normal">2</mn></msup><mo mathvariant="normal">+</mo><msub><mi mathvariant="normal">b</mi><mn mathvariant="normal">5</mn></msub><mo>⁢</mo><msup><mi mathvariant="normal">y</mi><mn mathvariant="normal">2</mn></msup><mo mathvariant="normal">+</mo><msub><mi mathvariant="normal">b</mi><mn mathvariant="normal">6</mn></msub><mo>⁢</mo><mi>xy</mi></math><img file="EP1524620B1_D0001.tif" /></maths> wherein x and y are the free coordinates and z represents the gray level. The principal curvatures k₁ and k₂ can be calculated as the roots k of <maths id="math0002" num=""><math display="block"><mfenced open="|" close="|"><mtable><mtr><mtd><msub><mi>b</mi><mn>4</mn></msub><mo>-</mo><mi>k</mi></mtd><mtd><msub><mi>b</mi><mn>6</mn></msub><mo>/</mo><mn>2</mn></mtd></mtr><mtr><mtd><msub><mi>b</mi><mn>6</mn></msub><mo>/</mo><mn>2</mn></mtd><mtd><msub><mi>b</mi><mn>5</mn></msub><mo>-</mo><mi>k</mi></mtd></mtr></mtable></mfenced><mo>=</mo><mn>0</mn></math><img file="EP1524620B1_D0002.tif" /></maths>

The decision to issue a gummy finger warning is given, if the principal curvatures k₁ and k₂ of the query image are substantially negative, especially after comparison with the related principal curvatures k₁ and k₂ of the genuine counterpart. This method is based on the insight that the raw image of a gummy finder is darker at the edges and brighter in the center. This is due to the fact that a gummy finger has to be pressed against the sensor with uneven forces to compensate for curvature of the elastic material of the artificial finger.

Preferably the regression is only calculated on those parts of the template and query images that are contained in both images and whose region exceeds a certain quality threshold. Nevertheless, it is possible to compute raw image data without use of the template data. In such a case the threshold of the curvatures can be chosen k₁=k₂=0.