Efficient method and system for the acquisition of scene imagery and iris imagery using a single sensor

EFFICIENT METHOD AND SYSTEM FOR THE ACQUISITION OF SCENE IMAGERY AND IRIS IMAGERY USING A SINGLE SENSOR

This application relates to U.S. Provisional Patent Application No. 61/443,757, entitled "Method and System for Iris Recognition and Face Acquisition," filed on February 17, 2011, and "Efficient Method and System for Claims priority to U.S. Provisional Patent Application No. 61/472,279, entitled "the Acquisition of Scene Imagery and Iris Imagery using a Single Sensor," both of which are incorporated herein by reference in their entirety for all purposes. do.

The present invention relates to image processing techniques, and more particularly, to an efficient system and method for acquiring scene images and iris images using a single sensor.

Typically, biometric systems are designed to obtain optimal images by taking into account the specific constraints of that biometric type. If other data (eg, a face or background image) is to be acquired, different sensors are typically used, since the requirements for different image types are very different. However, such an approach adds cost to the overall solution and may also increase the size or footprint of the system.

US Patent Publication 20060050933 by Adam et al. attempts to solve the problem of acquiring data for use in face and iris recognition using a single sensor, but images such that the data acquired for each of the face and iris recognition components are individually optimal. It does not solve the problem of optimizing acquisition.

US Patent Publication 20080075334 by Determan et al. and US Patent Publication 20050270386 by Saitoh et al. disclose acquiring face and iris images for recognition using individual sensors for faces and individual sensors for iris. Saitoh describes a method for performing iris recognition that involves identifying the location of the iris using images of the face and iris, but uses two separate sensors that are individually focused on each of the face and iris and collects the data. Since they are acquired at the same time, user motion is irrelevant.

US Patent Publication 20080075334 to Determan et al. also discusses the use of one sensor for both face and iris, but the problem of optimizing image acquisition so that the data acquired for each of the face and iris recognition components are individually optimal. can't solve

US Patent Publication 20070206840 to Jacobson et al. also describes a system comprising acquiring images of a face and iris, but the problem of optimizing image acquisition so that the data acquired for each of the face and iris recognition components are individually optimal. It does not solve the problem, and it does not deal with how to obtain a system with a small size.

In certain aspects, methods and systems are described herein for obtaining, using a single sensor, a high quality image of an iris for biometric identification, and a high quality picture of any other scene, such as a human face. Embodiments of these systems and methods employ a single sensor for the purpose of determining or verifying an individual's identity using biometric recognition using the iris, as well as for the purpose of obtaining general images of scenes such as faces and places. can be used to allow an image to be acquired. The latter type of image can typically be acquired by, for example, a mobile phone user. Accordingly, the disclosed methods and systems may be incorporated into mobile and/or compact devices. The sensor may be a complementary metal oxide semiconductor (CMOS) sensor or another suitable type of image capture device. Methods and systems may set or adjust conditions to be nearly optimal in two acquisition modes, eg, an iris image acquisition mode and a picture (eg, non-iris) acquisition mode. In some embodiments, systems for acquiring such images, eg, implemented within a device, may have a significantly reduced physical size or footprint, eg, compared to devices using multiple sensors.

In one aspect, this disclosure describes a method of acquiring images of an iris and a scene using a single image sensor. The method may include capturing by an image sensor a view of the scene and a view of the iris within the at least one image. The image processing module may generate an image of the scene by applying a level of noise reduction to the first portion of the at least one image. The image processing module may generate an image of the iris to be used for biometric identification by applying the reduced level of noise reduction to the second portion of the at least one image.

In some embodiments, the image sensor may capture the view of the scene and the view of the iris as separable components within a single image. The image sensor may capture at least one image of the iris while illuminating the iris using infrared illumination. In certain embodiments, the image sensor may activate a plurality of sensor nodes of the image sensor. The first subset of sensor nodes may be adapted to capture an image of the iris primarily suitable for biometric identification. The second subset of sensor nodes may be adapted to primarily capture non-iris images.

In certain embodiments, the image processing module may apply noise reduction including an averaging or median function. The image processing module may apply noise reduction including reducing both time-varying and time-invariant noise from the captured image. The image processing module may subtract noise from one image of the iris and noise from another image of the iris. In certain embodiments, the image processing module may reduce ambient noise in one image using ambient noise from another image. The image processing module may reduce ambient noise from one image captured in the presence of infrared illumination using ambient noise from another image captured without infrared illumination. The image processing module may generate an image of the iris for use in biometric identification by performing gain or luminance control on the second portion of the at least one image.

In another aspect, this disclosure describes an apparatus for capturing images of an iris and a scene using a single image sensor. The device may include an image sensor and an image processing module. The image sensor may capture a view of the scene and a view of the iris within the at least one image. The image processing module may generate an image of the scene by applying a level of noise reduction to the first portion of the at least one image. The image processing module may generate an image of the iris to be used for biometric identification by applying the reduced level of noise reduction to the second portion of the at least one image.

In some embodiments, the image sensor captures the view of the scene and the view of the iris as separable components within a single image. The image sensor may include, for example, a complementary metal oxide semiconductor (CMOS) sensor. The image sensor may include a plurality of sensor nodes, a first subset of sensor nodes adapted to primarily capture an image of an iris suitable for biometric identification, and a second subset of sensor nodes primarily adapted to capture a non-iris image. can be adapted to In certain embodiments, the device includes an illuminator for illuminating the iris using infrared illumination, wherein the image sensor captures at least one image of the illuminated iris.

In some embodiments, the noise reduction performed comprises application of a mean or median function to the captured image. Noise reduction may include reducing both time-varying and time-invariant noise from the captured image. In certain embodiments, the image processing module subtracts noise from one image of the iris and noise from another image of the iris. The image processing module may reduce ambient noise from one image captured in the presence of infrared illumination using ambient noise from another image captured without infrared illumination. In some embodiments, the image processing module may generate an image of the iris to be used for biometric identification by performing gain or luminance control on the second portion of the at least one image.

Certain embodiments of the methods and systems disclosed herein may address various challenges in obtaining high quality images of the iris as well as high quality images of a scene using a single sensor. For example, one challenge is perhaps surprisingly related to the management of the noise properties of a sensor. The inventors have found that the requirements for image quality of iris recognition and standard scenes are sometimes contradictory for noise. As the pixel size of the imager gets smaller and smaller and thus the fundamental noise level at each pixel increases or becomes more pronounced, noise can be very important. We determine that a particular type of noise is actually desirable or tolerable for iris recognition, compared to the quality of iris images obtained in standard picture acquisition modes including, for example, noise reduction. Thus, we may prefer to keep noise in the processed image during acquisition of the iris image, perhaps counter-intuitively, to improve the performance of iris identification compared to images that have been subjected to conventional noise reduction.

Another challenge relates to the wavelength of illumination required for standard images and for iris images. Acquisition of iris images typically uses infrared illumination, whereas standard images typically rely on visible illumination. These can be seen as opposing constraints when integrated into a single system for the acquisition of both types of images. This disclosure describes several approaches to address this. For example and in one embodiment, different filters may be interleaved in front of the sensor. Filters may have different responses to infrared and visible responses. RGB (red, green, blue) filters and filter patterns may be adapted for use in different embodiments. For example, and in certain embodiments, the systems and methods may interleave filters that pass infrared light with other filters that pass primarily color images. Examples of this approach are in US Patent Publication 2007/0145273 and US Patent Publication 2007/0024931. Improvements to these approaches include the use of R, G, (G+I), B interleaved arrays (where I stands for infrared). This approach may have the advantage of maintaining or recovering the full resolution of the G (green) signal, which the human visual system is most sensitive to. Another embodiment of methods and systems addresses this challenge by using a removable or retractable IR-cut filter that can be placed automatically or manually in front of the sensor during standard image acquisition mode. In another embodiment, systems and methods may overlay an IR-blocking filter over only a portion of an image sensor dedicated to iris recognition.

Embodiments of the systems and methods described herein may address a third challenge related to deformation of an image from ambient lighting. In some embodiments where infrared filtering or illumination is not optimal, images of the surrounding scene reflected from the cornea or eye surface may be observed during acquisition of the iris image. This can sometimes seriously adversely affect the performance of iris recognition. Embodiments of the systems and methods described herein are capable of acquiring at least two images. One of the images may be captured with the controlled infrared illumination turned on, and at least a second image may be captured with the controlled infrared illumination turned off. An image processing module may process these at least two images to reduce or eliminate artifacts. For example, after aligning the images, the image processing module subtract the images from each other to remove artifacts images. Artifactual lighting or components do not change essentially between the two images, whereas the iris texture is illuminated by infrared illumination and exposed within one image, so the difference in images will remove artifacts while preserving the iris texture. can Methods and systems can overcome non-linearities in a sensor by identifying pixels that are in or close to a non-linear operating range (eg, saturated or dark) of the sensor, and can remove them from subsequent iris recognition processing, which This is because the image subtraction process in such areas may be non-linear, and artifacts may still remain. In yet another embodiment of the methods, deformation of images may be managed by exploiting certain geometric constraints of the location of the user, device, and deformation source. In certain embodiments, the fact that the user can hold the device in front of his face during iris acquisition mode can be used to reduce or block the transforming ambient light source in a sector of the acquired iris image. Methods and systems may, for example, limit iris recognition to this sector, avoiding issues related to image deformation.

The drawings below show specific exemplary embodiments of the methods and systems described herein, in which like reference numbers designate like elements. Each of the illustrated embodiments is illustrative, not limiting, of such methods and systems. 1A is a block diagram illustrating one embodiment of a networked environment with a client machine in communication with a server. 1B and 1C are block diagrams illustrating embodiments of computing machines for practicing the methods and systems described herein. 2 shows an embodiment of an image intensity profile corresponding to a portion of an image. 3A shows an image intensity profile of one embodiment for non-systematic noise. 3B shows an image intensity profile of one embodiment for systematic noise. 4 shows an image intensity profile of one embodiment for systematic noise. 5 shows an image intensity profile of one embodiment for sporadic noise. 6 shows an embodiment of an image intensity profile corresponding to a portion of an image that has been subjected to noise reduction. 7 is a diagram of one embodiment of an image of a view of a face including an iris texture. 8 depicts one embodiment for an image intensity profile representing iris texture. 9 shows one embodiment of an image intensity profile representative of iris texture after noise reduction. 10 depicts one embodiment for an image intensity profile indicative of iris texture and noise. 11 shows one embodiment of a system for acquiring scene images and iris images using a single sensor. 12 shows a chart showing the effect of noise on acquired images. 13 shows another embodiment of a system for acquiring scene images and iris images using a single sensor. 14 shows an embodiment of a system for acquiring a face image and an iris image using a single sensor. 15 shows a response profile based on a dual band pass filter. 16 shows an embodiment of the configuration of interleaved filters. 17 shows one embodiment for an image with artifacts reflected from the eye surface. 18 shows one embodiment for an image with artifacts reflected from the iris texture and the eye surface. 19 shows another embodiment of a system for acquiring a face image and an iris image using a single sensor. 20 shows one embodiment of an image representing an iris texture with artifacts removed. 21 shows one scenario for acquisition of face and iris images. 22 shows another embodiment for an image with artifacts reflected from the iris texture and the eye surface. 23 shows another embodiment of a system for acquiring a face image and an iris image using a single sensor. 24 shows another embodiment of a system for acquiring a face image and an iris image using a single sensor. 25 shows one embodiment of a system for acquiring facial images and iris images using a single sensor and mirror. 26 shows one embodiment of a method for acquiring a face image and an iris image using a single sensor and mirror. 27 depicts the effect of ocular dominance on acquisition of face images and iris images. 28 shows another embodiment of a system for acquiring facial images and iris images using a single sensor and mirror. 29 and 30 show the effect of visual dominance on acquisition of face images and iris images. 31 shows another embodiment of a system for acquiring facial images and iris images using a single sensor and mirror. 32 shows embodiments of a sensor and mirror configuration. 33 shows another embodiment of a system for acquiring facial images and iris images using a single sensor and mirror. 34 shows another embodiment of a system for acquiring facial images and iris images using a single sensor and mirror. 35 shows another embodiment of a system for acquiring a face image and an iris image using a single sensor. 36 shows another embodiment of a system for acquiring a face image and an iris image using a single sensor. 37 shows another embodiment of a system for acquiring a face image and an iris image using a single sensor. 38 is a flow diagram illustrating one embodiment of a method for acquiring a scene image and an iris image using a single sensor.

Before addressing other aspects of systems and methods for obtaining a scene image and an iris image using a single sensor, a description of system components and features suitable for use in the present systems and methods is helpful. can be 1A illustrates one or more client machines 102A-102N (generally referred to herein as "clients) 106 in communication with one or more servers 106A- 106N (referred to herein generally as "server(s) 106 "). One embodiment of a computing environment 101 including "machine(s) 102" is shown. A network is installed between the client machine(s) 102 and the server(s) 106 .

In one embodiment, computing environment 101 may include an appliance installed between server(s) 106 and client machine(s) 102 . The appliance may manage client/server connections and, in some cases, load balance client connections among multiple backend servers. The client machine(s) 102 may in some embodiments be referred to as a single client machine 102 or a single group of client machines 102 , the server(s) 106 being a single server 106 or may be referred to as a single group of servers 106 . In one embodiment, a single client machine 102 communicates with two or more servers 106 , and in another embodiment, a single server 106 communicates with two or more client machines 102 . In another embodiment, a single client machine 102 communicates with a single server 106 .

The client machine 102 may in some embodiments include client machine(s) 102 ; client(s); client computer(s); client device(s); client computing device(s); local machine; remote machine; client node(s); endpoint(s); endpoint node(s); or a second machine. Server 106, in some embodiments, includes server(s); local machine; remote machine; server farm(s); host computing device(s); or the first machine(s).

The client machine 102 may in some embodiments include software; program; executable instructions; virtual machine; hypervisor; web browser; web-based client; client-server applications; thin-client computing client; ActiveX control; Java applet; software related to voice over internet protocol (VoIP) communications, such as soft IP phones; applications for streaming video and/or audio; applications for facilitating real-time data communication; HTTP client; FTP client; oscar client; telnet client; or execute, operate, or provide an application, which may be any one of any other set of executable instructions. Still other embodiments include a client device 102 that displays application output generated by an application running remotely on a server 106 or other remotely located machine. In such embodiments, the client device 102 may display the application output within an application window, browser, or other output window. In one embodiment, the application is a desktop, while in other embodiments, the application is an application that creates a desktop.

Computing environment 101 includes two or more servers 106A-106N, such that servers 106A-106N can be logically grouped together within server farm 106 . Server farm 106 includes servers 106 that are geographically dispersed and logically grouped together within server farm 106, or servers 106 that are located close to each other and logically grouped together within server farm 106. can do. Geographically dispersed servers 106A-106N within server farm 106 may communicate using a WAN, MAN, or LAN in some embodiments, wherein different geographic areas may be located on different continents; different regions of the continent; different countries; different states; different cities; different campuses; different rooms; or any combination of the aforementioned geographic locations. In some embodiments, the server farm 106 may be administered as a single entity, while in other embodiments, the server farm 106 may include multiple server farms 106 .

In some embodiments, server farm 106 includes servers running a substantially similar type of operating system platform (eg, WINDOWS NT, UNIX, LINUX, or SNOW LEOPARD manufactured by Microsoft Corporation of Redmond, Washington). 106) may be included. In other embodiments, the server farm 106 includes a first group of servers 106 running a first type of operating system platform, and a second group of servers 106 running a second type of operating system platform. It can contain 2 groups. The server farm 106 may include servers 106 running different types of operating system platforms in other embodiments.

Server 106 may be any server type in some embodiments. In other embodiments, server 106 may include the following server types: a file server; application server; web server; proxy server; appliance; network appliances, gateways; application gateway; gateway server; virtualization server; deployment server; SSL VPN server; firewall; web server; application server or master application server; a server 106 running Active Directory; or any of the servers 106 running application acceleration programs that provide firewall functions, application functions, or load balancing functions. In some embodiments, server 106 may be a RADIUS server that includes a remote authentication dial-in user service. Some embodiments receive requests from the client machine 102 , send the request to the second server 106B, and respond in response from the second server 106B to the request generated by the client machine 102 . and a first server 106A. The first server 106A may obtain a list of applications available to the client machine 102 as well as address information associated with the application server 106 hosting the application identified within the list of applications. In addition, the first server 106A may provide a response to the client's request using the web interface, and communicate directly with the client 102 to provide the client 102 access to the identified application. have.

Client machines 102 may, in some embodiments, be client nodes seeking access to resources provided by server 106 . In other embodiments, server 106 may provide clients 102 or client nodes with access to hosted resources. Server 106 functions as a master node in some embodiments, communicating with one or more clients 102 or servers 106 . In some embodiments, the master node may identify and provide address information associated with the server 106 hosting the requested application to one or more clients 102 or servers 106 . In still other embodiments, the master node may be a server farm 106 , a client 102 , a cluster of client nodes 102 , or an appliance.

One or more clients 102 and/or one or more servers 106 may transmit data over a network 104 installed between machines and appliances within the computing environment 101 . Network 104 may include one or more subnetworks, and may be installed between any combination of clients 102 , servers 106 , computing machines and appliances included within computing environment 101 . . In some embodiments, network 104 may include a local area network (LAN); Urban Area Network (MAN); wide area network (WAN); a main network 104 comprising a number of subnetworks 104 located between client machines 102 and servers 106; a main public network 104 having a private subnetwork 104; a primary private network 104 having a public subnetwork 104; or a primary private network 104 with a private subnetwork 104 . Still other embodiments provide for the following network types: point-to-point networks; broadcast network; telecommunication networks; data communication networks; computer network; Asynchronous Transfer Mode (ATM) networks; Synchronous Optical Network (SONET) networks; Synchronous Digital Hierarchy (SDH) networks; wireless network; wired network; or a network 104 , which may be any one type of networks 104 including a wireless link, wherein the wireless link may be an infrared channel or a satellite band. The network topology of network 104 may be different in different embodiments, and possible network topologies include bus network topologies; star network topology; ring network topology; repeater-based network topology; or a tiered-star network topology. Additional embodiments may include a network 104 of mobile phone networks using a protocol to communicate between mobile devices, the protocol being AMPS; TDMA; CDMA; GSM; GPRS; UMTS; 3G; 4G; or any other protocol capable of transferring data between mobile devices.

One embodiment of a computing device 100 is shown in FIG. 1B , wherein the client machine 102 and server 106 shown in FIG. 1A are any of the computing devices 100 shown and described herein. It may be deployed and/or implemented as an embodiment. The computing device 100 includes the following components: a central processing unit 121 ; main memory 122; storage memory 128; input/output (I/O) controller 123; display devices 124A-124N; installation device 116; and a system bus 150 in communication with a network interface 118 . In one embodiment, storage memory 128 includes an operating system, software routines, and a client agent 120 . I/O controller 123 is further connected to keyboard 126 and pointing device 127 in some embodiments. Other embodiments may include an I/O controller 123 connected to two or more input/output devices 130A-130N.

1C illustrates one embodiment of a computing device 100 , wherein the client machine 102 and server 106 illustrated in FIG. 1A are any of the implementations of the computing device 100 shown and described herein. may be deployed and/or executed by way of example. Included within the computing device 100 are the following components: a bridge 170 and a system bus 150 in communication with a first I/O device 130A. In another embodiment, the bridge 170 further communicates with the main central processing unit 121 , the central processing unit 121 including the second I/O device 130B, the main memory 122, and the cache memory ( 140) can be further communicated. The central processing unit 121 includes I/O ports, a memory port 103, and a main processor.

Embodiments of computing machine 100 include the following component configurations: logic circuits that respond to and process instructions fetched from main memory unit 122 ; those manufactured by Intel Corporation; those made by Motorola; microprocessor units such as those manufactured by Transmeta Corporation of Santa Clara, CA; RS/6000 processors such as those manufactured by IBM Corporation; processors such as those manufactured by Advanced Micro Devices; or a central processing unit 121 characterized by any one of any other combination of logic circuits. Still other embodiments of central processing unit 122 include a microprocessor, microcontroller, central processing unit having a single processing core, a central processing unit having two processing cores, or a central processing unit having two or more processing cores. may include any combination of

Although FIG. 1C illustrates computing device 100 including a single central processing unit 121 , in some embodiments, computing device 100 may include one or more processing units 121 . In such embodiments, computing device 100 may store and execute firmware or other executable instructions, which when executed instruct one or more processing units 121 to execute instructions concurrently or on a single piece of data. instruct them to run concurrently. In other embodiments, computing device 100 may store and execute firmware or other executable instructions, which when executed direct one or more processing units to each execute a section of a group of instructions. For example, each processing unit 121 may be instructed to execute a portion of a program or a particular module within a program.

In some embodiments, processing unit 121 may include one or more processing cores. For example, the processing unit 121 may have two cores, four cores, eight cores, and the like. In one embodiment, processing unit 121 may include one or more parallel processing cores. The processing cores of the processing unit 121 may access the available memory as a global address space in some embodiments, or in other embodiments the memory within the computing device 100 is segmented and within the processing unit 121 . It can be assigned to a specific core. In one embodiment, one or more processing cores or processors within computing device 100 may each access local memory. In another embodiment, memory within computing device 100 may be shared among one or more processors or processing cores, while other memory may be accessed by specific processors or subsets of processors. In embodiments in which computing device 100 includes two or more processing units, multiple processing units may be included in a single integrated circuit (IC). These multiple processors may be linked together by an internal high-speed bus, which may be referred to as an element interconnect bus in some embodiments.

In embodiments in which computing device 100 includes one or more processing units 121 , or processing units 121 including one or more processing cores, the processors may simultaneously execute a single instruction on multiple pieces of data. or (SIMD), or in other embodiments, execute multiple instructions simultaneously on multiple pieces of data (MIMD). In some embodiments, computing device 100 may include any number of SIMD and MIMD processors.

Computing device 100 may include an image processor, a graphics processor, or a graphics processing unit in some embodiments. The graphic processing unit may include any combination of software and hardware, and may further input graphic data and graphic instructions, render a graphic from the input data and instructions, and output the rendered graphic. In some embodiments, the graphics processing unit may be included in the processing unit 121 . In other embodiments, computing device 100 may include one or more processing units 121 , wherein at least one processing unit 121 is dedicated to processing and rendering graphics.

One embodiment of the computing machine 100 includes a central processing unit 121 that communicates with the cache memory 140 via an auxiliary bus, also known as a backside bus, while another implementation of the computing machine 100 . An example includes a central processing unit 121 that communicates with cache memory via a system bus 150 . Local system bus 150 may be used by a central processing unit in some embodiments to communicate with more than one type of I/O device 130A- 130N. In some embodiments, local system bus 150 is a bus of the following types: a VESA VL bus; ISA bus; EISA bus; microchannel architecture (MCA) bus; PCI bus; PCI-X bus; PCI-Express bus; or NuBus. Other embodiments of computing machine 100 include I/O devices 130A- 130N, which are video displays 124 in communication with central processing unit 121 . Still other versions of computing machine 100 include processor 121 connected to I/O devices 130A- 130N via any of the following connections: HyperTransport, high-speed I/O, or InfiniBand. Further embodiments of the computing machine 100 include a processor 121 that communicates with one I/O bus 130A using a local interconnect bus and with a second I/O bus 130B using a direct connection. includes

Computing device 100 includes a main memory unit 122 and cache memory 140 in some embodiments. Cache memory 140 may be any type of memory, and in some embodiments, the following memory types: SRAM; BSRAM; or EDRAM. Other embodiments include the following memory types: Static Random Access Memory (SRAM), Burst SRAM or SynchBurst SRAM (BSRAM); dynamic random access memory (DRAM); Fast Page Mode DRAM (FPM DRAM); Advanced DRAM (EDRAM), Extended Data Output RAM (EDO RAM); Extended Data Output DRAM (EDO DRAM); Burst Extended Data Output DRAM (BEDO DRAM); Advanced DRAM (EDRAM); Synchronous DRAM (SDRAM); JEDEC SRAM; PC100 SDRAM; double data rate SDRAM (DDR SDRAM); Advanced SDRAM (ESDRAM); SyncLink DRAM (SLDRAM); Direct Rambus DRAM (DRDRAM); ferroelectric RAM (FRAM); or cache memory 140 and main memory unit 122, which may be any of any other memory type. Additional embodiments include a system bus 150; memory port 103; or a central processing unit 121 that can access the main memory 122 via any other connection, bus or port that allows the processor 121 to access the memory 122 .

One embodiment for computing device 100 includes the following installation devices 116: CD-ROM drive, CD-R/RW drive, DVD-ROM drive, tape drives of various formats, USB device, bootable It provides support for either media, a bootable CD, a bootable CD for GNU/Linux distribution such as KNOPPIX®, a hard drive or any other device suitable for installing applications or software. Applications may include client agent 120 or any portion of client agent 120 in some embodiments. Computing device 100 may further include storage device 128 , which may be either one or more hard disk drives or one or more redundant arrays of independent disks, the storage device comprising an operating system, software, programs, applications , or at least a portion of the client agent 120 . A further embodiment of the computing device 100 includes an installation device 116 used as the storage device 128 .

Computing device 100 may be connected to a standard telephone line, LAN or WAN link (eg, 802.11, T1, T3, 56kb, X.25, SNA, DECNET), broadband connection (eg, ISDN, Frame Relay, ATM, Gigabit). Network interface 118 for interfacing to a LAN, WAN, or Internet via various connections including, but not limited to, Ethernet, Ethernet-over-SONET), wireless connections, or some combination of any or all of the foregoing. may further include. Connections may also be made using various communication protocols (eg, TCP/IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, RS485, IEEE 802.11, IEEE 802.11a, IEEE 802.11b). , IEEE 802.11g, CDMA, GSM, WiMax and direct asynchronous access). One version of the computing device 100 is a gateway of any type and/or form, such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS), or the Citrix Gateway protocol manufactured by Citrix Systems, Inc. or a network interface 118 capable of communicating with additional computing devices 100' via a tunneling protocol. Versions of network interface 118 include: a built-in network adapter; network interface card; PCMCIA network card; cardbus network adapter; wireless network adapter; USB network adapter; modem; or any other suitable for interfacing computing device 100 to a network capable of communicating and performing in the methods and systems described herein.

Embodiments of computing device 100 include the following I/O devices 130A- 130N: keyboard 126; pointing device 127; mouse; trackpad; light pen; trackball; MIC; drawing tablet; video display; speaker; inkjet printer; laser printer; and dye-sublimation printers; or any other input/output device capable of performing the methods and systems described herein. I/O controller 123 may connect to multiple I/O devices 130A- 130N to control one or more I/O devices in some embodiments. Some embodiments of I/O devices 130A- 130N may be configured to provide storage or installation media 116, while other embodiments include a USB flash drive line of devices manufactured by TwinTech Industries, Inc. It can provide a universal serial bus (USB) interface to accommodate such USB storage devices. Still other embodiments include a system bus 150 and an external communication bus: such as a USB bus; Apple Desktop Bus; RS-232 serial connection; SCSI bus; FireWire bus; FireWire 800 bus; Ethernet bus; AppleTalk bus; Gigabit Ethernet Bus; asynchronous transfer modbus; HIPPI bus; Super HIPPI bus; SerialPlus bus; SCI/LAMP bus; FiberChannel bus; or an I/O device 130 , which may be a bridge between serial attached miniature computer system interface buses.

In some embodiments, computing machine 100 is capable of running any operating system, while in other embodiments, computing machine 100 is capable of running the following operating systems: versions of Microsoft Windows operating systems; different releases of Unix and Linux operating systems; Any version of the MAC OS produced by Apple Computer Corporation; OS/2 produced by IBM Corporation; Google's Android; any built-in operating system; any real-time operating system; any open source operating system; any proprietary operating system; any operating systems for mobile computing devices; or any other operating system. In another embodiment, the computing machine 100 may run multiple operating systems. For example, the computing machine 100 may run PARALLELS or another virtualization platform capable of running or managing a virtual machine running a first operating system, and the computing machine 100 may run a different system than the first operating system. 2 Run the operating system.

Computing machine 100 includes the following computing devices: a computing workstation; desktop computer; laptop or notebook computer; server; handheld computer; mobile phone; portable telecommunication devices; media playback device; game system; mobile computing devices; netbooks, tablets; devices of the IPOD or IPAD family of devices manufactured by Apple Computer Corporation; any of the PLAYSTATION family of devices manufactured by Sony Corporation; any of the Nintendo family of devices manufactured by Nintendo Corporation; any of the XBOX family of devices manufactured by Microsoft; or any other type and/or form of computing, telecommunications, or media device capable of communicating and having sufficient processor power and memory capacity to perform the methods and systems described herein. have. In other embodiments, the computing machine 100 may include the following mobile devices: a JAVA-enabled cellular phone or personal digital assistant (PDA); any computing device having different processors, operating systems and input devices compatible with the device; or a mobile device, such as any one of any other mobile computing device capable of performing the methods and systems described herein. In still other embodiments, computing device 100 includes the following mobile computing devices: at least one series of BlackBerry or other handheld device manufactured by Research in Motion Limited; iPhone manufactured by Apple Computers; Palm Pre; Pocket PC; Pocket PC phone; Android phone; or any other handheld mobile device. While specific system components and features that may be suitable for use in the present systems and methods have been described, additional aspects are addressed below.

2 shows an exemplary image of a typical scene or object (eg, a house) obtained by a conventional image sensor. The image sensor may include, but is not limited to, for example, a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) active pixel sensor. The graph or intensity profile corresponding to the image represents the intensity value (I) and the corresponding spatial position (X) of the pixels on the vertical axis, for the cross-sectional area indicated by the line P2. The bright and dark points in the intensity profile correspond to the bright and dark points in the image as shown. Typically, there may be substantial noise in the signal, represented by variations in intensity, even within uniformly illuminated areas (eg, areas corresponding to the door of a house). Noise can be derived from several sources, for example amplifier noise and shot-noise, anisotropic (systematic) noise, and sporadic noise. Shot noise is related to the quantum effect in which a finite number of photons are collected within a particular pixel-well within a finite period of time. The smaller the pixel size, the more shot noise can be generated. This is because there may be fewer photons from which to infer the measurement of incident illumination. As pixel dimensions become smaller, the focal length of the associated optics for a given image resolution may also decrease linearly. This may reduce the thickness of the lens/sensor component combination. However, as the requirements for sensor resolution increase, and as space constraints on sensors and their associated optics become more stringent, sensor and image pixel sizes are appropriately reduced to accommodate the requirements and constraints. should be The result of the reduction in pixel size is a substantial increase in the noise of the sensor. This type of noise, as well as amplifier noise, can be characterized as time-varying and unsystematic as shown in FIG. 3A .

Another type of noise is anisotropic or systematic/periodic noise. Periodic noise can be caused, for example, by differences in amplifier gains in the read path of the image sensor. For example, different rows and columns may pass through different amplifiers with slightly different gains. This type of systematic noise is shown in Figure 3b, in which the intensity profile, which should be uniformly flat, varies substantially periodically in one dimension (eg, across the image). 4 shows an example of sporadic noise introduced into an image, which may be apparent across multiple images. For example, occasional pixels in the array of sensor nodes may have degraded sensitivity, or may not function, have limited or excessive gain, such that the pixels may be brighter or darker as shown. .

Problems arising from noise are typically solved by performing noise reduction in the image processing module 220 . The image processing module 220 may use any type of spatially central filtering or region-selective averaging as shown in FIG. 5 . Many methods exist for performing noise reduction, and central filtering and region-selective averaging are identified for illustrative purposes only. 6 shows an intensity profile that can result from noise reduction. While the noise reduction may have essentially eliminated the noise, the image processing module 220 retained features (eg, bright and dark points) corresponding to real objects and edges in the scene. From the user's point of view, the image quality is typically considered unacceptable (eg, noisy) in FIG. 1 , while the quality is considered better in FIG. 6 .

7 shows an image of the iris I1 and the face F1. The image may be acquired using, for example, an optimal iris image acquisition system according to the specifications described in the National Institute of Standards and Technology (NIST) standards. These specifications may include those described in ANSI/INCITS 379-2004, Iris Image Interchange Format. Referring to FIG. 7 , the texture of the iris is represented by lines in the circular region indicated by I1 . 8 shows one representation of the intensity profile of the texture of the iris. In some embodiments, the similarity between FIG. 8 (intensity profile of the iris texture pattern) and FIG. 2 (intensity profile of the noise signal) may be quite evident. The reason for such similarity is that the source of each signal/pattern is characterized by a random process. In the case of the iris, a paper tear is signaled by the tearing of the iris tissue prior to birth, very similar to a different process each time it occurs. In the case of sensor noise, shot noise and other noises are generated by random time-varying physical processes.

The frequency characteristics of the iris signal "texture" have been characterized to some extent in NIST standards [ANSI/INCITS 379-2004, Iris Image Interchange Format], and for different iris diameter ranges, for example in millimeters (mm ), minimum resolution values corresponding to lines/pairs may be specified. The iris diameter may depend on the particular optical configuration. For example, for an iris diameter between 100-149 pixels, the defined pixel resolution may be at least 8.3 pixels per millimeter, with an optical resolution at 60% modulation of at least 2.0 line-pairs per millimeter. For an iris diameter between 150-199 pixels, the defined pixel resolution may be at least 12.5 pixels per millimeter, with an optical resolution at 60% modulation of at least 3.0 line-pairs per millimeter. For an iris diameter with more than 200 pixels, the defined pixel resolution may be at least 16.7 pixels per millimeter, with an optical resolution at 60% modulation of at least 4.0 line-pairs per millimeter. In certain embodiments, for other diameters, defined pixel resolution and/or optical resolution combinations may be suitable.

Fig. 9 shows the intensity profile of the iris texture after some of the noise reduction processing described above has been performed. In this exemplary case, the iris texture is essentially removed by noise reduction. This is because noise reduction algorithms such as region-specific averaging may not be able to distinguish between iris texture and noise. Thus, noise reduction, which is standard or common in most image capturing devices, may be a limitation when adapted to perform iris recognition.

The present systems and methods can address this problem by recognizing certain characteristics associated with iris recognition. FIG. 10 shows (eg, in NIST standards [ANSI/INCITS 379-2004, Iris Image Interchange Format) and ) shows the intensity profile of the optimally obtained iris texture. Certain iris recognition processes involve identifying a lack of statistical independence between the matched signal and the probe signal. One implication may be that matches are usually declared by comparisons that yield results that are not likely to be achieved by a random process. Thus, adding significant random and time-varying noise to the original iris signal may not 1) significantly increase the false match rate as false matches arise from non-random matching, and 2) the iris signal's may have a limited impact on the false rejection rate for an individual if the texture generally or essentially exceeds the texture of the sensor noise (e.g., if the images themselves appear noisy to the observer), 3 ) may increase the false rejection rate for the user (along with limited other results) if the texture of the iris signal has a similar or smaller magnitude compared to the magnitude of the sensor noise.

However, adding systematic noise to the original iris signal, for example as shown in Figure 3, can trigger a false match, since the comparison between the two data sets was not achieved by a random process. because it can be calculated. Accordingly, certain embodiments of methods and systems provide for improved performance of iris identification compared to images with reduced noise levels (eg, via noise reduction) in a captured iris image (eg, noise reduction). , even a significant level of noise) may be preferred (eg, counterintuitively). In some embodiments, the present systems may reduce or eliminate the level of unsystematic noise reduction applied to an image when the image is intended for iris recognition. The resulting images may appear extremely noisy to the viewer, perhaps compared to the processed image (eg, to which noise reduction has been applied). However, the performance of iris recognition can be greatly improved if a noisy image is used instead for iris recognition. In some specific hardware implementations, noise reduction algorithms may be enabled and hard-coded and not turned off. Some embodiments of the present methods and systems allow control over noise reduction algorithms to avoid noise reduction in frequency bands expected for iris texture as described elsewhere herein.

11 shows an example implementation of an approach in which the main processor may control an image signal processor, eg, a low level image signal processor. In the mode in which iris recognition is performed, a signal may be sent to the image signal processor to alter the noise reduction process as described above. Depending on the magnitude of the systematic noise, such noise can be removed (eg, using a dynamic low calibration, where pixels at the edge of the sensor are covered and can be used for sensor calibration), or the magnitude of the noise can be reduced to the iris texture. It can be left as is if it is significantly smaller than the signal magnitude of . For example, FIG. 12 presents a table summarizing multiple scenarios and explains how different types of noise can affect the performance of iris recognition and/or the quality of the visible image in different image acquisition modes.

Another challenge associated with obtaining an optimal standard scene image and iris image on the same sensor relates to the wavelength of illumination required for the standard image and iris image. Iris images typically require infrared illumination, while standard images typically require visible illumination. Sometimes there are conflicting constraints. Some embodiments of the present systems may be configured to address this by interleaving filters with different responses to infrared and visible light. Such systems may utilize one of a plurality of different configurations of such filters for an image sensor when capturing an image. One example of a filter that can be integrated or modified to create an interleaved filter is a filter with a Bayer RGB (red, green, blue) filter pattern (see, eg, US Pat. No. 3,971,065). Filters that pass (predominantly, significantly, or only) infrared light may be interleaved with other filters that pass (predominantly, significantly or only) color or visible light. Some embodiments of filters that provide selected filtering are described in US Patent Publication 20070145273 and US Patent Publication 20070024931. Some embodiments of the present systems and methods use R, G, (G+I), B interleaved arrays instead. Some of these systems have the ability to maintain full (or substantially full) resolution of the G (green) signal to which the human visual system is typically most sensitive.

In iris recognition mode, the magnitude of the G (green) response is typically much smaller than the magnitude of the infrared response due to incident infrared illumination. In some embodiments, an estimate of the infrared signal response (I) in the iris recognition mode may be recovered by subtracting the (G) signal from the adjacent (G+I) signal. In the standard image acquisition mode, the R, G, (G+I), and B signals can be processed to recover an estimate (G') of G in the pixel from which G+I has been recovered. For example, when an R, G, T, B pixel array is used, where T is fully transparent, various methods can be used to generate such estimates. A T pixel in such an implementation may include signals of R, G, B and I signals that are stacked or superimposed together. This can be a problem. If the T pixel filter is truly transparent, for effective performance, the sum of the R, G, B, I responses must still be within the dynamic range of the pixel. For a given integration time and pixel area over the entire image, this means that the dynamic range of R, G, B pixels cannot be fully utilized as saturation of the T pixel (R+G+B+I) may occur. do. It may be possible to set different pixel areas and gain for a T pixel compared to other R, G, B pixels, but it can be expensive to implement. One improvement that can be incorporated into the present systems is the use of a neutral density filter instead of a transparent filter. A neutral density filter can reduce the magnitude of illumination of all wavelengths (R, G, B, I) in that pixel, allowing sufficient or wide range of pixel capacities to be used in R, G, B pixels. noise can be reduced. As an example, a neutral density filter having a value of 0.5 to 0.6 may be selected. Typically the green signal can contribute about 60% of the luminance signal comprising R, G and B combined together.

If the T filter is truly transparent, at the expense of the signal-to-noise ratio of the R, G, and B pixels, to accommodate the range of the T pixel and keep it within a linear range, the overall dynamic range of the sensor will typically need to be reduced. will be. By incorporating the R, G, G+I, B filter array in some embodiments of the present systems and since no red and blue signals are present in the G+I pixel, the full dynamic range of the sensor is R, G, T, B can be increased relative to the dynamic range of the array, thus increasing the signal-to-noise ratio.

Another approach incorporated in some embodiments of the present methods and systems for obtaining an optimal standard scene image and iris image on the same sensor with respect to the wavelength of illumination is to multiplex an infrared cut filter over a standard image sensor or lens. or deploying. In one embodiment, for example as shown in FIG. 14 , some of the sensors (eg, 20% of the sensor or sensor nodes) may be designated primarily for iris recognition, while others (eg, , 80%) portion can be used for standard image acquisition. As in this example, a lower portion (eg, 80%) of the sensor may be covered by a standard IR cut filter. The remaining 20% of the sensor may remain uncovered. In the iris recognition mode, the covered area may be ignored. For example, an iris recognition application running on the image capturing device may guide the user to position his or her eyes within the sensing area of the uncovered 20% area. Feedback mechanisms may guide the user to move the image capturing device to position the user's iris within the appropriate capture area. For example, a face will be visible in the remaining 80% of the imager, so this can optionally be used for user guided feedback, with icons appearing instead of the eye area. In some embodiments, the image sensor may use the uncovered area to adjust its orientation to capture an image of the user's iris.

Another approach incorporated within some embodiments of the present systems and methods uses a dual band pass filter over all or a substantial portion of the color imager or sensor. Such a filter may pass both R, G, B signals and infrared signals in selected bands, such as those near 850 nm or 940 nm, and yield a frequency response as shown in FIG. 15 . . In another embodiment, the image acquisition system may use an IR cut filter that may be automatically or manually placed or slid in place over at least a portion of the image sensor when the device is in a standard image capture mode. For example, an IR cut filter may cover a portion of the image sensor to align with the user's eye to capture an iris image. Other portions of the image sensor may capture portions of the user's face, for example. An IR blocking filter may be disposed at one end of the sensor so that the sensor and thus the captured image are in three or more areas (eg, non-IR-blocking, IR-blocking and non-IR-blocking). rather than having two distinct regions (IR-blocking and non-IR-blocking). This allows a larger and more continuous non-iris portion of the scene (eg, face) to be obtained, which in turn can be used for eg face identification. In some embodiments, (eg, as an option) a visible light filter or an IR pass filter may be disposed over the image sensor when the device is in iris image capture mode.

In some embodiments, the image acquisition system may interleave infrared cut and infrared pass filters across the sensor, for example as shown in FIG. 16 . The interleaved filter may be configured in a variety of other ways, such as an arrangement of checker boxes, the use of stripes of varying widths or other alternating and/or repeatable patterns. In iris recognition mode, the response from sensor pixels/nodes below the IR pass filter bands is used for iris recognition, whereas the response from sensor pixels/nodes below the IR cut filter bands is the standard image acquisition mode is used in In some embodiments, both standard and iris images may be obtained using a single image capture, such as by separating the IR and non-IR image components corresponding to the interleaving pattern.

In some embodiments, the image obtained by the image sensor may be affected or deformed by ambient lighting. For example, in some embodiments, if infrared filtering and/or illumination is not optimal, images of the scene may be reflected from the surface of the eye (eg, the cornea) during acquisition of the iris image. An example of this is shown in FIG. 17 . The reflection of the image (eg, on the cornea of the eye) may be, as an example, a reflection of a scene including houses around the user. Such reflections may be referred to as artifacts. Above, we have described how systematic noise can adversely affect the performance of iris recognition. Artifacts can be obtained in similar methods: at least two images, one image with controlled infrared illumination turned on, as shown in FIG. 18 and one with controlled infrared illumination turned off, as shown in FIG. 17 . can be overcome using a method of obtaining at least a second image in The image processing module may process these at least two images to reduce or eliminate artifacts. For example, in some embodiments, the image processing module may subtract images from each other after aligning the images as shown in the processing diagram of FIG. 19 . Artifact-generating illumination is essentially unchanged between the two images, whereas the iris texture is illuminated by infrared illumination, so artifacts can be removed by taking the difference, while the iris texture is maintained. The remaining iris texture is shown in FIG. 20 by lines within the iris. The system may further overcome the sensor's nonlinearity, for example, by identifying pixels that are in or close to (eg, saturated or dark) the sensor's nonlinear operating range. The image processing module may exclude the identified pixels from subsequent iris recognition processing. Since the image subtraction process in such regions can be non-linear, artifacts can still be maintained using the subtraction approach.

Another embodiment of the present methods manages deformation of images by using certain geometric constraints on the location of the user, the image capturing device, and the source of the deformation or artifacts. The image processing module causes the image capturing device to deform within a sector of the acquired iris image, for example, as shown in FIG. 21 when the user holds the image capturing device in front of the user's face during the iris recognition mode It can be configured to recognize that it can reduce, or even block, sources of ambient light. 22 , the image processing module may limit iris recognition primarily or only to this sector, avoiding issues related to image deformation. In some embodiments, iris recognition based on this sector of the image may be weighted higher than other sectors in determining a biometric match.

In some embodiments, infrared illumination is not readily available or warranted during image capture. The image acquisition system 200 may be configured to control and/or provide infrared illumination. The image acquisition system may reduce power usage by illuminating with an infrared source (eg, LEDs) when the device is in iris recognition mode as shown in FIG. 23 .

24 depicts one embodiment of an image acquisition system 200 that utilizes some features of the systems and methods described herein. The image acquisition system 200 may be implemented within a device, such as a mobile and/or handheld device. The device may include a screen having a sensor. Infrared LEDs can provide illumination. A user may use a touch screen or other input device (eg, keyboard, button or voice command recognition) to switch between the iris recognition mode and the standard photo taking mode. The device may include an application through which the user may activate an image capturing mode. The application may automatically locate the user's iris, or further provide a feedback or guidance mechanism to guide the user to move the user's iris into the appropriate capture area. In some embodiments, the optional IR cut filter can be activated manually or automatically or moved over the image sensor when in iris image capture mode. Other filters (eg, an IR pass filter) may be integrated and/or activated in the appropriate mode(s). In certain embodiments, certain features of image acquisition system 200 may be included in an add-on accessory or sleeve for a mobile or existing device. As an example, such features may include an infrared illuminator, one or more filters, and/or an interface (eg, wireless or physical) to a mobile or legacy device.

In some embodiments, image acquisition system 200 may include infrared illuminators embedded within a screen of image acquisition system 200 for illumination of a user's eyes using infrared illumination. Screens and displays typically use white LED lighting below the LCD matrix. By replacing or adding some of the visible LEDs with near-infrared illuminators, a source of IR illumination can be provided by the display itself. In such an embodiment, the image acquisition system 200 may not require an additional fixture or area on the image acquisition system 200 to provide infrared illumination, thus saving space.

In certain embodiments, image acquisition system 200 may include a visible illuminator having, for example, two illumination intensities. The visible illuminator may be turned on at low power during the iris image acquisition mode. Low power lighting may be selected so as not to distract or annoy the user. In some embodiments, the luminance level in the low power mode may be at least twice as dark as the maximum luminance of the visible illuminator. The latter luminance level can be used to illuminate a wider scene, for example. A low-power visible illuminator can be used to retract the iris and increase the iris area whether or not the user is in the dark. However, since the visible illuminator can be close to the eye, some of the filters described above can still transmit significant visible light into the sensor. Thus, in some embodiments, visible light is turned off before images of the iris are acquired while the near infrared illuminator is turned on. In an alternative embodiment, the screen itself may be used as a source of visible light.

In some embodiments, one advantage of using a single sensor in the image acquisition system 200 is that the space occupied by the system can be minimized compared to the use of dual sensors. In either case, however, an important consideration is the ability of the user and/or operator to effectively use a single sensor or dual sensor device.

In some embodiments, a mirrored surface may be used to assist the user in guiding the user to align the user's iris with the appropriate capture area of the image sensor. The mirrored surface may feed back the user's location to the user as shown in FIG. 25 , in which case the user holds the device in front of him and a virtual image of some of the user's face is the distance from the user to the device. observed at twice the However, due to the nature of the human visual system, the ocular dominance and the requirements of the iris recognition system, the optimal size of the mirror may not scale linearly with the user's distance to the mirror as might be expected. . Indeed, under some conditions, increasing the size of the mirror to try and improve iris recognition performance may degrade performance or cause difficulties in alignment.

Visual dominance is the tendency to favor visual input from one eye or the other. It occurs in most people, 2/3 of people have a right eye dominance, and 1/3 of people have a left eye dominance. To maximize the size of the restored iris image while minimizing the size of the mirror used to guide the user, the present systems and methods address visual dominance and combine the properties of visual dominance with the constraints of iris recognition.

26 shows the reflected field of view of a mirror sized to allow both eyes to comfortably occupy the field of view. In some embodiments, the width of the mirror allows the reflection field of view at the viewing distance of the image acquisition device 200 to be at least about 50% wider than the reflection in the eye gap. For illustrative purposes, the user is shown in the center of the mirror. However, Fig. 27 actually shows that due to the visual dominance the user is usually located on one side of the mirror so that his dominant view is closer to the center of the mirror. If the width of the mirror's field of view is greater than 50% of the field of view of users' normal eye spacing (6.5-7 cm), the eyes may remain in the field of view. Thus, both eyes may be acquired by the image acquisition system 200 for people with visual dominance, since both eyes may still be within the field of view of the image sensor in such cases. However, the iris diameter in the captured image can be relatively small because the lens of the sensor is typically selected to cover a wide field of view.

28 shows a configuration for acquiring images of both eyes using a smaller mirror without considering visual dominance. The field of view of the mirror is smaller, thus minimizing its area on any image acquisition system 200 . Both eyes can be captured if the user is placed in the center of the mirror. However, as noted above, due to visual dominance, the user is typically placed to the right or left of this optimal position as shown in FIGS. 29 and 30 . In such a scenario, one of the eyes may be located outside the field of view of the camera. Thus, although this configuration has a moderately large mirror, even if the lens can be configured to capture both eyes (when in the central position), due to visual dominance, the image acquisition system 200 can actually only reliably capture one eye. can be caught

FIG. 31 shows a design that acquires a higher resolution iris image compared to FIG. 30 (ie, improves iris recognition performance), but uses a smaller mirror so that only the dominant vision is observed by the user. By limiting the size of the mirrors so that only the dominant vision is in the field of view, the tendency of the user's visual system to select either the left or right eye is a variable or unpredictable response within the field of view (e.g., shifted to the left or right). eyes), it is forced to be a binary response (eg, left or right eye). In some embodiments, image acquisition system 200 may operate or include a mirror having a diameter of about 14 mm at a working distance of about 9", such that the reflected field of view of the mirror is about two typical iris diameters (2). x 10.5 mm) Fig. 32 summarizes and shows the size of the effective field of view of the mirror and its relation to one or two-eye capture and also the size of the iris image obtained.

33 shows one embodiment of an image acquisition system 200 with an IR cut filter disposed over a portion of the sensor. A face or other image may be acquired by a part of the sensor, and an image for iris recognition is acquired by a part covered by an IR cut filter. Visual dominance tends to provide uncertainty in the horizontal direction due to the horizontal configuration of human eyes, so image acquisition system 200 can be suitably configured to have a horizontally shaped filter area above the sensor. FIG. 34 shows another embodiment where the user is viewing the sensor/lens assembly at an angle and the mirrors are tilted so that the eyes are closer to the top of the sensor rather than in the center of the sensor. This configuration makes it possible to place an IR blocking filter at one end of the sensor, so that the three regions (non-IR-blocking, IR-blocking and non-IR-blocking) are the case where the sensor is shown in FIG. 33 . rather than having two distinct regions (IR-blocking and non-IR-blocking). This allows a larger and more continuous non-iris portion of the scene to be obtained.

35 illustrates another embodiment of an image acquisition system 200 in which an operator may be holding an image acquisition device 200 to acquire an iris image of a user. In this embodiment, there is a see-through guidance channel that the operator can see to line up with the user's eyes. Additionally or alternatively, spaced guide markers may be placed on top of the image acquisition device 200 such that the operator aligns the user's eyes with, for example, two markers. 36 shows an enlarged view of one embodiment for a guide channel. In this embodiment, as shown, on the inner part of the guide channel, circular rings can be printed on the front and back of the guide channel. When the user is aligned, these rings can be seen as concentric circles to the operator. Otherwise, they will not be concentric (user's eyes misaligned). 36 also shows a visible illuminator (LED) on the device as well as infrared illuminators that may be used for iris recognition purposes. 37 shows another embodiment of an image acquisition system. In this embodiment, the LEDs are controlled by controllers, which are connected to a processor, which is also connected to a sensor used for iris recognition.

38 shows one embodiment of a method for capturing images of an iris and a scene using a single image sensor. The image sensor captures ( 382 ) a view of the scene and a view of the iris within the at least one image. The image processing module applies a level of noise reduction to the first portion of the at least one image to generate an image of the scene (384). The image processing module applies the reduced level of noise reduction to the second portion of the at least one image to generate an image of the iris for use in biometric identification (step 386).

Referring further to FIG. 38 , and more particularly, the image sensor 202 of the image acquisition system 200 captures ( 382 ) a view of the scene and a view of the iris within at least one image. The image sensor may capture a view of a scene within one image and a view of the iris within another image. In some embodiments, the image sensor may capture a view of the scene and a view of the iris within a single image. For example, the view of the scene may include at least a portion of the iris. The image sensor may capture a view of the scene and a view of the iris within the plurality of images. The image sensor may capture a view of the scene in some images and a view of the iris in other images. The image sensor may capture a view of the scene and a view of the iris within some images. The image sensor may capture more than one image over a period of time. The image sensor may, for example, capture two or more images within a short time frame of each other for later comparison or processing. The image sensor may capture two or more images under different conditions, for example, with or without infrared illumination, or with or without any type of filter discussed herein.

In some embodiments, image acquisition system 200 may include an iris capturing mode and a picture (eg, non-iris) capturing mode. The image sensor may capture an image of the view of the scene in picture capturing mode. The image sensor may capture an image of the view of the iris in the iris capturing mode. In certain embodiments, image acquisition system 200 may perform simultaneous capture of iris and non-iris images in another mode. The user may select a mode for image acquisition through, for example, an application running on the image acquisition device 200 . In some embodiments, the image acquisition system may capture the view of the scene and the view of the iris as separable components within a single image. The image acquisition system may use any embodiment and/or combination of interleaved filters, IR cut filters, IR pass filters, and other types of filters described herein to capture a view of the scene and/or a view of the iris. have.

In some embodiments, the image sensor includes a plurality of sensor nodes of the image sensor. The image sensor may activate a first subset of sensor nodes that are primarily adapted to capture an image of the iris suitable for biometric identification. The image sensor may activate a second subset of sensor nodes that are primarily adapted to capture a non-iris image. An IR pass (G+I) filter (eg passing G+I) or other filters may be applied over the sensor node that is adapted to primarily capture images of the iris. IR blocking, visible light pass, specific band pass or color filters may be applied over sensor nodes that are primarily adapted to capture non-iris images.

In some embodiments, the image sensor captures at least one image of the iris while illuminating the iris using infrared illumination. The image sensor may capture at least one image of the iris without infrared illumination. The image sensor may capture at least one image of the iris upon turning off the visible light illuminator. The image sensor may capture at least one image of the iris using illumination from a screen of the image acquisition system 200 . The image sensor may use a mirror of the image acquisition system 200 for guidance to capture at least one image of the iris when the iris is aligned with a portion of the sensor. The image sensor may capture at least one image of the iris when the iris is aligned with a portion of the sensor using a fluoroscopic guidance channel and/or markers by an operator.

With further reference to 384 , the image processing module may apply a level of noise reduction to the first portion of the at least one image to generate an image of the scene. The image acquisition system 200 may apply noise reduction to the image captured by the image sensor. The image acquisition system 200 may apply noise reduction to an image stored in the image acquisition system 200 , for example, a storage device or a buffer. The image acquisition system 200 may apply noise reduction including applying an average or median function or filter over some pixels of the images, eg, a 3x3 pixel window. The image acquisition system 200 may apply noise reduction including reduction of one or both of time-varying and time-invariant noise from the captured image. The image acquisition system 200 may process or exclude known defective pixels while performing image processing and/or noise reduction. The image acquisition system 200 may apply noise reduction using an image processing module, which may include one or more image signal processors 206 and/or other processors 208 . The image acquisition system 200 may apply noise reduction by identifying, processing, and/or compensating for the presence of systematic noise.

In some embodiments, the image processing module may apply noise reduction to the image captured in the non-iris capture mode. The image processing module may apply a level of noise reduction to a portion of the image that is not for iris biometric recognition, for example, a portion corresponding to an IR cut filter. The image processing module may apply noise reduction or filtering to normal or non-iris images. The image processing module may generate an image of the general scene that may be better recognized (eg, to a person) than the image before noise reduction.

With further reference to 386 , the image processing module may apply the reduced level of noise reduction to the second portion of the at least one image to generate an image of the iris for use in biometric identification. In some embodiments, the image processing module may disable noise reduction for an image to be used for iris biometric identification. The image processing module may determine that the noise level does not overwhelm the captured iris texture. The image processing module may perform iris biometric identification based on the raw or raw image captured by the image sensor. The image processing module may perform iris biometric identification based on the image captured by the image sensor after some processing, eg, removal of artifacts, sporadic noise and/or systematic noise.

In some embodiments, the image processing module may apply a reduced level of noise reduction to the image for use in iris biometric identification. The image processing module may apply a reduced level of noise reduction to the captured image while in the iris capturing mode. The image processing module may perform noise reduction for systematic and/or sporadic noise. The image processing module may disable noise reduction for unsystematic noise. The image processing module may apply the reduced level of noise reduction to a portion of the image extracted for iris biometric identification, for example, a portion corresponding to an IR pass filter. The image processing module may apply systematic noise reduction to a portion of the image extracted for iris biometric identification, for example, a portion corresponding to an IR pass filter.

In some embodiments, the image processing module 220 subtracts noise from one image of the iris and noise from another image of the iris. Such subtraction may reduce systematic noise and/or sporadic noise. The image processing module 220 may perform subtraction by aligning two images side by side. The image processing module 220 may align the two images using common reference points (eg, edges of shapes). The image processing module 220 may align the two images using pattern recognition/matching, correlation and/or other algorithms. The image processing module 220 may subtract noise corresponding to the overlapping portion of the two images. The image processing module 220 may reduce ambient noise in one image by using ambient noise from another image. Ambient noise may include signals from ambient light or illumination. Ambient noise may include artifacts from ambient lighting sources or reflections of ambient objects from the surface of the eye. In some embodiments, image processing module 220 may reduce ambient noise from one image captured in the presence of infrared illumination using ambient noise from another image captured in the absence of infrared illumination. .

In certain embodiments, image processing module 220 may recover infrared components from one or more (G+I) pixels imaged on the sensor node array. The image processing module 220 may subtract the G component from (G+I) using the G intensity value in the neighboring pixel. In some embodiments, the image processing module 220 may subtract the G component using the estimated G intensity value. The image processing module 220 may use the estimated G intensity value in processing the non-iris (eg, general scene) portion of the image. In some embodiments, the image processing module 220 may generate an image of the iris to be used for biometric identification by performing gain or luminance control or adjustment on a portion of at least one image. In some embodiments, the amount of infrared illumination may be insufficient or sub-optimal, such that gain or luminance control or adjustment may improve iris image quality. In certain embodiments, gain or luminance control or adjustment may be more desirable for addition of infrared illuminators, drawing power to provide infrared illumination, and/or control of infrared illumination (eg, under different conditions). have. Since the infrared signals are captured by some of the sensor nodes/pixels (eg, in an RGB(G+I) array), compensation through gain or luminance control or adjustment may be appropriate.

Having described specific embodiments of methods and systems, it will now be apparent to those skilled in the art that other embodiments incorporating the inventive concepts may be utilized. It should be understood that the systems described above may provide many of any or each of such components, and such components may be provided on a standalone machine or in some embodiments on multiple machines within a distributed system. . The systems and methods described above may be implemented as a method, apparatus, or article of manufacture using programming and/or engineering techniques to create software, firmware, hardware, or any combination thereof. Moreover, the systems and methods described above may be provided as one or more computer readable programs embodied on or in one or more articles of manufacture. As used herein, the term "article of manufacture" refers to code or logic, firmware, programmable logic, memory devices (eg, EEPROM, ROM, PROM, RAM, SRAM, etc.), hardware (e.g., integrated circuit chips, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.), electronic devices, computer-readable non-volatile storage units (e.g., , CD-ROM, floppy disk, hard disk drive, etc.). The article of manufacture may be accessed from a file server that provides access to computer readable programs via network transmission lines, wireless transmission media, signals propagating through space, radio waves, infrared signals, and the like. The article may be a flash memory card or magnetic tape. Articles of manufacture include hardware logic as well as software or programmable code embodied in a computer-readable medium executed by a processor. In general, computer readable programs may be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language, such as JAVA. Software programs may be stored as object code on or in one or more articles of manufacture.