Automatic identification of fingerprint inpainting target areas

Description of equivalent WO 2011022212 A1

AUTOMATIC IDENTIFICATION OF FINGERPRINT INP AINTING

TARGET AREAS

The invention is directed to biometric systems. In particular, the invention is directed to fingerprint inpainting including automatic identification of fingerprint inpainting target areas.

Biometric systems are used to identify individuals based on their unique traits. Biometrics are useful in many applications, including security and forensics. Some physical biometric markers include facial features, fingerprints, hand geometry, and iris and retinal scans. A biometric system can authenticate a user or determine the identity of sampled data by querying a database.

There are many advantages to using biometric systems. Most biometric markers are easily collectable, present in most individuals, unique between individuals, and permanent throughout the lifespan of an individual. However, these factors are not guaranteed. For example, surgical alterations may be used to change a biometric feature such that it does not match one previously collected from the same individual. Furthermore, different biometric features can change over time.

A common type of biometric identification is fingerprinting. A fingerprint is an impression of the raised friction ridges on the epidermis. In general, fingerprints have lasting permanence and are unique to an individual, making them a robust means for identification. Additionally, fingerprints are easily collectable, as they may be collected from many types of surfaces. Fingerprints are more intrusive than some less accurate biometric identification methods, such as facial recognition or voice print identification methods. Still, they are less intrusive than other accurate biometric identification methods, such as iris scans and DNA. As a result, fingerprints are currently the most common type of biometric identification and are likely to remain so for the foreseeable future.

The use of fingerprints as a form of biometric identification began with manual methods for collecting fingerprints and evaluating matches. Identification was performed at one time by manually comparing a collected fingerprint to fingerprints on a card collected using an "ink technique" (i.e., pressing and rolling an individual subject's inked finger). Such methods have now been automated by the use of automated identification systems to compare fingerprint images. The term "fingerprint image" as used herein refers to a digital image of a fingerprint. The "ink technique" is still in use today; however these cards are now scanned to create fingerprint images for use in automated identification systems. In addition to the "ink technique", fingerprint images can also be generated via the use of solid-state fingerprint readers. Solid-state fingerprint sensors generally work based on capacitance, thermal, electric field, laser, radio frequency, and/or other principles. Such fingerprint sensors typically generate 2-dimensional fingerprint images, although some fingerprint sensors generate 3 -dimensional fingerprint images.

Even though fingerprints are unique across individuals, they generally include several types or levels of common or "key" features. Automated

identification systems utilize such key features during fingerprint recognition processes. That is, these systems compare the locations, number, and types of key features in an acquired fingerprint image to determine the identity of the individual associated with the acquired fingerprint. Level 1 features of fingerprints include loops, whorls and arches formed by the ridges. These features describe the overall shape followed by the ridges. Level 2 features of fingerprints, or minutiae, are irregularities or discontinuities in the ridges. These include ridge terminations, bifurcations, and dots. Level 3 features of fingerprints include ridge pores, ridge shape, as well as scarring, warts, creases and other deformations.

Embodiments of the invention concern systems and methods for automatic identification of fingerprint inpainting areas. In a first embodiment of the invention, a method for inpainting areas in a fingerprint image is provided. The method includes dividing a fingerprint image into a plurality of image blocks and computing a plurality of block scores for the plurality of image blocks. The method also includes generating a blur matrix for the fingerprint image based on the plurality of block scores. The method further includes deriving an inpaint region (IR) matrix for the fingerprint image based on a weighting function and the blur matrix, the IR matrix identifying a portion of the plurality of image blocks for inpainting. In a second embodiment of the invention, a fingerprint processing system is provided. The system includes a storage element for storing a fingerprint image and a processing element communicatively coupled to the storage element. The processing element is configured for dividing the fingerprint image into a plurality of image blocks and computing a plurality of block scores for the plurality of image blocks. The processing element is also configured for generating a blur matrix for the fingerprint image based on the plurality of block scores. The processing element is further configured for deriving an inpaint region (IR) matrix for the fingerprint image based on a weighting function and the blur matrix, the IR matrix identifying a portion of the plurality of image blocks suitable for inpainting.

In a third embodiment of the invention, a computer-readable storage medium, having stored thereon a computer program for inpainting areas in a fingerprint image, is provided. The computer program has a plurality of code sections executable by a computer. The code sections cause the computer to perform the steps of dividing a fingerprint image into a plurality of image blocks and computing a plurality of block scores for the plurality of image blocks. The code sections also cause the computer to perform the step of generating a blur matrix for the fingerprint image based on the plurality of block scores. The code sections further cause the computer to perform the step of deriving an inpaint region (IR) matrix for the fingerprint image based on a weighting function and the blur matrix, the IR matrix identifying a portion of the plurality of image blocks suitable for inpainting.

FIG. 1 is a processed fingerprint image that is useful for understanding an inpainting process in accordance to an embodiment of the invention.

FIG. 2 is an image of an inpainting mask for the fingerprint image in FIG. 1 that is useful for understanding an inpainting process in accordance to an embodiment of the invention.

FIG. 3 is an image showing the inpainting provided for the blocks in FIG. 2 that is useful for understanding an inpainting process in accordance to an embodiment of the invention. FIG. 4 is an image of the processed fingerprint image in FIG. 1 after inpainting in FIG. 3 is applied that is useful for understanding an inpainting process in accordance to an embodiment of the invention.

FIG. 5 is a flowchart showing steps in an exemplary method for impainting areas of an acquired fingerprint image according to an embodiment of the invention.

FIG. 6 shows a first exemplary arrangements of blocks in a fingerprint images that is useful for understanding the various embodiments of the invention.

FIG. 7 shows a second exemplary arrangements of blocks in a fingerprint images that is useful for understanding the various embodiments of the invention.

FIG. 8 is a block diagram of a computer system that may be used in embodiments of the invention.

The present invention is described with reference to the attached figures, wherein like reference numerals are used throughout the figures to designate similar or equivalent elements. The figures are not drawn to scale and they are provided merely to illustrate the instant invention. Several aspects of the invention are described below with reference to example applications for illustration. It should be understood that numerous specific details, relationships, and methods are set forth to provide a full understanding of the invention. One having ordinary skill in the relevant art, however, will readily recognize that the invention can be practiced without one or more of the specific details or with other methods. In other instances, well-known structures or operations are not shown in detail to avoid obscuring the invention. The present invention is not limited by the illustrated ordering of acts or events, as some acts may occur in different orders and/or concurrently with other acts or events. Furthermore, not all illustrated acts or events are required to implement a methodology in accordance with the present invention.

As described above, fingerprint recognition processes typically rely on a minimum amount of matching between fingerprint data in a fingerprint template and a fingerprint of interest. The term "fingerprint template", as used herein, refers to a collection of information specifying the type, size, and/or locations of key features in one or more fingerprints associated with an individual. In particular, fingerprint recognition requires that at least some number of key features in the fingerprint of interest match with a high degree of certainty, the key features stored in the template. However, fingerprints are not always acquired under ideal conditions. For example, law enforcement investigators often need to rely upon partial or poor quality images of fingerprints acquired at a crime scene. Consequently, these fingerprint images may not contain a sufficient number of key features to allow a good match to a stored fingerprint image, frustrating further investigation of the crime. Although manual and automatic image processing techniques exist for removing some amounts of noise and blurring from acquired fingerprint images, such enhancement techniques may remove key features from the fingerprint image and exacerbate the fingerprint recognition process. A greater concern arises in the case of inpainting (i.e., the extrapolation of ridges to reconstruct fingerprint images). Although existing ridges can be used to reconstruct portions of a fingerprint image, the uniqueness of fingerprint often results in a failure to generate a key feature properly. Even worse, artificial key features may be generated during the reconstruction. As a result, the use of finger reconstruction techniques can actually reduce the likelihood of matching a fingerprint image to a stored fingerprint image or fingerprint template in a database.

To overcome these and other problems, embodiments of the invention provide systems and methods for inpainting fingerprint images, including automatic identification of fingerprint inpainting target areas. In particular, embodiments of the invention leverage fingerprint information gathered and/or generated during image processing (e.g., ridge flow direction and image quality) to selectively identify areas of partial fingerprint images to which inpainting is to be applied. By using the fingerprint information acquired during image processing, extrapolation of fingerprint features is performed only in those areas for which extrapolation is expected to accurately depict the missing features of the fingerprint. Thus, the amount of extrapolation is limited, reducing or eliminating the likelihood that key features will be altered or that artificial key features will be generated. An inpainting process in accordance with the various embodiments of the invention is conceptually illustrated with respect to FIGs. 1-4.

FIG. 1 is a processed fingerprint image 100 that is useful for understanding an inpainting process in accordance to an embodiment of the invention. In particular, FIG. 1 shows the result of image processing. The term "fingerprint image processing", as used herein, refers to any type of image processing and/or image characterization applied to an acquired fingerprint image. For example, fingerprint images often include unnecessary information such as scars, moisture- induced features, or areas without valuable ridges and furrows. Therefore, in order to eliminate the redundant information, filter the useful information, and enhance existing features, processes such as normalization (e.g., filtering and contrast enhancement), binarization (i.e., conversion to 1-bit image), quality markup (useless or poor quality data removal), and/or thinning processes (i.e., ridge enhancement) are used to generate the fingerprint to be used for identification. In the various embodiments of the invention, an acquired fingerprint image is divided into blocks of n x n pixels and each of the blocks is binarized. Subsequently, the blocks including poor information, are removed, resulting in the block-type edges or block-type voids shown in processed image 100. Additionally, information regarding the direction and type of features (i.e., ridge or valley) in each of the remaining blocks is stored.

Based on the processed fingerprint image 100, the areas for inpainting can then be selected in accordance with an embodiment of the invention to generate an inpainting mask. In general, an area associated with a block is selected based on the amount of fingerprint information available, namely the amount of surrounding blocks. FIG. 2 is an image 200 of an inpainting mask for image 100 that is useful for understanding an inpainting process in accordance to an embodiment of the invention.

As shown in FIG. 2, the blocks selected for inpainting mask in image 200 appear to lie along the borders of the processed fingerprint image 100 in FIG. 1. However, a closer comparison of images 100 and 200 shows that not all of the blocks lying along the of the processed fingerprint image 100 are selected. In the various embodiments of the invention, a methodology is provided to select only the blocks from image 100 lying in areas where the surrounding blocks are likely to have sufficient information to allow extrapolation of fingerprint features with a high degree of confidence are selected. This methodology is described below in greater detail with respect to FIGs. 6-7.

After the blocks are selected, the fingerprint features for these blocks can be extrapolated. This is conceptually illustrated in FIG. 3. FIG. 3 is an image 300 showing the inpainting for the blocks in FIG. 2. The fingerprint information for the blocks in image 200 is generated by using the information regarding the direction and types of features collected during image processing. Accordingly, fingerprint features for the inpainted areas in image 300 can be generated, as shown in FIG. 3, based on this fingerprint information. Afterwards, images 100 and 300 can be combined to produce the final fingerprint image, as shown in FIG. FIG. 4 is an image 400 of the processed fingerprint image 100 after the inpainting in FIG. 3 is applied. The combined image 400 can then be used for fingerprint recognition purposes.

One aspect of the invention, as described above, is the identification of blocks for inpainting a fingerprint image. Although such areas can be identified manually, such a process can be extremely time-consuming. Therefore, the various embodiments of the invention provide systems and methods for automating this identification process. In particular, the various embodiments of the invention combine a scoring of blocks in the fingerprint image with a weighting function to determine whether or not to apply inpainting to a particular block. This process is described below in greater detail with respect to FIG. 5

FIG. 5 is a flowchart showing steps in an exemplary method 500 for inpainting areas of an acquired fingerprint image according to an embodiment of the invention. Method 200 begins at step 502 and continues on to step 504. At step 504, a fingerprint image is received. In the various embodiments of the invention, a received fingerprint image may be generated using a variety of techniques, including ink techniques and solid-state scanning of an individual's fingerprints. However, the invention is not limited in this regard and fingerprint images can also be generated from latent, patent, or plastic fingerprints found at a location and imaged and/or retrieved using one or more forensic methods (e.g., dusting or chemically reactive techniques).

Once the fingerprint image is received at step 504, the image is divided into blocks for image processing at step 506. In one embodiment, the blocks can comprise n x n pixel blocks. Although n can be any integer value, such as 16, 24, or 32, the block size can be selected to limit the number of features present in the n x n pixel blocks. For example, block size can be selected to limit the blocks to show only a portion of one ridge feature or one valley feature. By limiting the block to show only the localized fingerprint features, all features in the block will be expected to have the same or similar directionality information. Such a configuration limits the amount of information that needs to be stored for each block.

Once the fingerprint image is divided into blocks at step 506, image processing of the fingerprint, on a block-by-block basis can be performed. As described above, the blocks can be pre-processed using normalization, binarization, and or thinning to enhance the features in the fingerprint. Additionally, the

directional information for the ridges and/or valleys in the block can be ascertained and stored. Furthermore, quality marking is performed to compute a quality score for each of the blocks. That is, for each block, the amount of fingerprint information is analyzed to determine a quality score indicative of the usefulness of the data in the data block. Therefore, blocks with poor image quality or incomplete fingerprint data can be given a low score. For example, if the data in a block does not allow the directional information for the block to be determined with a high degree of confidence or for valley and ridge portions to be clearly identified, the block can receive a low score. In another example, if the block contains incomplete information, the block can also receive a low score. In some embodiments, the block quality scores can be a function of the ridge flow coherency (i.e., any measure of the continuity of ridges are in a block) and strength (i.e., any measure how well defined the ridges are in a block, such as the gradient from a valley to the peak of a ridge) versus threshold value(s) or measure(s). However, the various embodiments of the invention are not limited to the methods described above and any other method for determining a quality score of a block can be used in the various embodiments of the invention.

Based on the quality scores generated at step 508, a blur matrix can be generated at step 510. In the various embodiments of the invention, the blur matrix comprises a reduced matrix as compared to the fingerprint image. That is, if a fingerprint image has pixel dimensions m x / and is divided into n x n blocks, the blur matrix comp ¹ rises an ^m //n x / yn matrix. In the various embodiments of the invention, the blur matrix comprises a binary matrix (i.e., a matrix in which each entry is zero or one). The entries for the binary blur matrix are selected based on the quality scores computed at step 508 and a quality score threshold value. Therefore, if a block quality score is greater than or equal to a threshold value, it is given a value of one. Conversely, if a block quality score is less than the threshold value, it is given a value of zero. In general the threshold value can be any amount; however, the threshold value can be adjusted by a user based on various factors. For example, if the amount of the fingerprint image missing is relatively high, a lower threshold value can be used to allow a larger number of blocks to be considered. If the amount of the fingerprint image missing is relatively low, a larger threshold value can be used to limit the introduction of artificial features. In some embodiments, the threshold value can be used as part of an iterative process to provide several inpainting scenarios.

The resulting binary matrix therefore identifies blocks (using a value of one) in the fingerprint image that not only contain useful fingerprint data for identification purposes, but also the blocks that contain useful information for extrapolating fingerprint information. The binary matrix also identifies blocks that can be potentially inpainted (using a value of zero).

Once the blur matrix is generated at step 510, the blur matrix can be used at step 512 to identify the blocks to be inpainted. In particular, a weighting function is evaluated for each location of the blur matrix. The weighting function determines for each block in the blur matrix whether the block has a minimum number of neighboring blocks to permit extrapolation of features into the block with a high degree of confidence. The development of an exemplary weighting function described in greater detail below with respect to in FIGs. 6 and 7. Once evaluated at step 512, the weighting function results in an ^m/ x y binary inpaint region (IR) matrix that defines that the blocks of the image received at step 504 that are to be inpainted.

Once the IR matrix is generated at step 512, the IR matrix can be used to perform inpainting of the received image at steps 514 and 516. First, the IR matrix is upscaled at step 514 to the resolution of the received fingerprint image. The term "upscaling", as used herein, refers to the process of mathematically converting a first image or matrix at a first resolution (i.e., matrix dimensions) to a second image or matrix at a second resolution higher than the first resolution. The upscaling at step 514 can be performed according to one or more interpolation techniques, such as piecewise constant interpolation, linear interpolation, polynomial interpolation, spline interpolation, and Gaussian processing techniques. However, the various

embodiments of the invention are not limited in this regard and any type of interpolation technique can be used.

The upscaled IR matrix generated at step 514 can then be used to apply inpainting at step 516. That is, the upscaled IR matrix is used as an inpaint mask to identify the areas of the fingerprint image which should be inpainted with fingerprint information based on fingerprint information from surrounding pixels. Inpainting is then performed according this inpaint mask using inpainting techniques based on extrapolation or other techniques. In the various embodiments of the invention, any type of inpainting method can be used. For example, linear, polynomial, conic, or French curve extrapolation methods, to name a few, can be used in the various embodiments of the invention. Additionally, other inpainting methods, such as those described in U.S. Patent Publication No. 2008/0080752 to Rahmes et al. and U.S.

Patent No. 6,987,520 to Criminisi et al., can also be used in the various embodiments of the invention. However, the various embodiments of the invention are not limited to any particular technique, and any extrapolation or inpainting method can be used with the various embodiments of the invention. Once inpainting is completed at step 516, method 500 continues on to step 518 to resume previous processing. For example, method 500 can be repeated for other fingerprint images or a fingerprint recognition process can be invoked.

As described above, one aspect of the invention is the development of the weighting function used for determining which of image blocks should be included in the IR matrix. The weighting function provides a relative measure of the number of blocks, surrounding a block of interest in the blur matrix, that are likely to include information for extrapolating information for the block of interest. In the various embodiments of the invention, the weighting function can be designed to provide such measure based on the position of surrounding blocks with respect to a block of interest.

For example, FIG. 6 shows a portion of a fingerprint image 600, where a block marked with a zero in the blur matrix (clear block 602) is surrounded by several blocks (shaded blocks 604, 606, 608, and 610) marked with a one in the blur matrix (i.e., block including good fingerprint data). In general, if a block 602 has one or more directly adjacent blocks (i.e., neighboring blocks including good fingerprint data, on a same row or column as block of interest, and contacting an edge of the block of interest), there is a high likelihood that the features in these directly adjacent blocks will carry over into block 602 and that extrapolation will be accurate. As a result, the weighting function can be designed to mark a block for inpainting if a sufficient number if directly adjacent blocks are identified in the blur matrix.

However, in some circumstances, a single adjacent block may not provide sufficient information for inpainting, as it would only define one boundary condition for inpainting in the block of interest. Therefore, in the various embodiments of the invention, the weighting function can be further designed to require two or more directly adjacent blocks in order to identify a block for inpainting. This results in the defining of at least two boundary conditions for the fingerprint information in the block of interest, increasing the accuracy of any subsequent extrapolation and reducing the likelihood of introducing artificial key features into the final fingerprint image. However, some blocks can still be marked for inpainting even though they do not have a sufficient number of directly adjacent blocks. This is conceptually illustrated in FIG. 7. FIG. 7 shows a portion of a fingerprint image 700, where a block marked with a zero in the blur matrix (clear block 702) is surrounded by several blocks (shaded blocks 704, 706, 708, and 710) marked with a one in the blur matrix (i.e., block including good fingerprint data). As described above, if a block 702 has a directly adjacent block, such as block 706, there is a high likelihood that the features in this directly adjacent block will carry over into block 702. However, as also described above, there can still be an issue as to whether the single directly adjacent block can provide sufficient information for accurately extrapolating fingerprint data for the block of interest. Therefore, identification of the block of interest for inpainting can be determined by also looking at other surrounding blocks. In particular, embodiments of the invention provide for the weighting function to consider the directly diagonal blocks (i.e., neighboring blocks including good fingerprint data, on a same diagonal as the block of interest, and contacting a corner of the block of interest). For example, if block 702 is surrounded by both a directly adjacent block 706 and one or more diagonally adjacent blocks, such as one or more of blocks 704, 708, and 710, block 702 can be identified for inpainting. As a result, the weighting function can be designed to mark a block for inpainting if at least one directly adjacent block and directly diagonal blocks are identified in the blur matrix. However, in some circumstances, a single directly diagonal block may not provide sufficient information for inpainting, as it cannot define how the fingerprint may vary on an opposing side or corner of the block of interest. For example, if image 700 included only blocks 704 and 706, it would be difficult to estimate the behavior for the features in block 702 with respect to other edges or corners. Therefore, in the various embodiments of the invention, the weighting function can be further designed to also require two or more diagonal adjacent blocks in order to mark a block of interest for inpainting. This increases the likelihood that subsequently inpainted features are accurate and reduces the likelihood of introducing artificial key features into the final fingerprint image. By taking into consideration the requirement of two or more directly adjacent blocks and two or more directly diagonal blocks, a weighting function can be developed. In particular, a weight of 2 can be provided for each directly adjacent block and a weight of 1 can be provided for each directly diagonal block {condition 1). Accordingly, if there are at least two directly adjacent blocks for a block of interest, there is at least a weight of (2 blocks x 2) = 4 associated with the block of interest. If there are at least two directly diagonal blocks for a block of interest and at least one directly adjacent block (condition 2), there is a weight of (2 blocks x 1) + 2 = 4 associated with the block of interest. Accordingly, a weighting function can be provided for a block of interest as shown below in Equation (1):

(# of directly diagonal blocks in Blur) +

2 x (# of directly adjacent blocks in Blur) &gt; 4 (1) where Blur is the blur matrix. Therefore, if the Equation (1) evaluates to true, the block can be inpainted. More generally, for an ^m/ x y I binary blur matrix, a convolution (Q kernel can be provided as shown below in Equation (2): <img file="WO2011022212A1_D0001.tif" />

The convolution kernel thus provides weighting of directly adjacent blocks equal to 2 and directly diagonal blocks equal to using. Using C, Equation (1) can be rewritten for the blur matrix as shown below in Equation (3):

IR = [(C &lt;8&gt; Blur) f| BIUI]≥ 4 , (3) where IR is the IR matrix and the intersection of the inverse of the blur matrix is provided so as not apply the convolution kernel to blocks of the fingerprint image with good fingerprint information. Using Equation (3), any blocks marked as a zero in the blur matrix and meeting either of condition 1 or condition 2, as described above, can then be automatically identified and the inpaint mask can be directly derived. The present invention can be realized in one computer system.

Alternatively, the present invention can be realized in several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software can be a general-purpose computer system. The general- purpose computer system can have a computer program that can control the computer system such that it carries out the methods described herein.

The present invention can take the form of a computer program product on a computer-usable storage medium (for example, a hard disk or a CD- ROM). The computer-usable storage medium can have computer-usable program code embodied in the medium. The term computer program product, as used herein, refers to a device comprised of all the features enabling the implementation of the methods described herein. Computer program, software application, computer software routine, and/or other variants of these terms, in the present context, mean any expression, in any language, code, or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code, or notation; or b) reproduction in a different material form.

The computer system 800 of FIG. 8 can comprise various types of computing systems and devices, including a server computer, a client user computer, a personal computer (PC), a tablet PC, a laptop computer, a desktop computer, a control system, a network router, switch or bridge, or any other device capable of executing a set of instructions (sequential or otherwise) that specifies actions to be taken by that device. It is to be understood that a device of the present disclosure also includes any electronic device that provides voice, video or data communication.

Further, while a single computer is illustrated, the phrase "computer system" shall be understood to include any collection of computing devices that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. The computer system 800 includes a processor 802 (such as a central processing unit (CPU), a graphics processing unit (GPU, or both), a main memory 804 and a static memory 806, which communicate with each other via a bus 808. The computer system 800 can further include a display unit 810, such as a video display (e.g., a liquid crystal display or LCD), a flat panel, a solid state display, or a cathode ray tube (CRT)). The computer system 800 can include an input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), a disk drive unit 816, a signal generation device 818 (e.g., a speaker or remote control) and a network interface device 820.

The disk drive unit 816 includes a computer-readable storage medium

822 on which is stored one or more sets of instructions 824 (e.g., software code) configured to implement one or more of the methodologies, procedures, or functions described herein. The instructions 824 can also reside, completely or at least partially, within the main memory 804, the static memory 806, and/or within the processor 802 during execution thereof by the computer system 800. The main memory 804 and the processor 802 also can constitute machine-readable media.

Dedicated hardware implementations including, but not limited to, application-specific integrated circuits, programmable logic arrays, and other hardware devices can likewise be constructed to implement the methods described herein. Applications that can include the apparatus and systems of various

embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit. Thus, the exemplary system is applicable to software, firmware, and hardware

imp lementations .

In accordance with various embodiments of the present invention, the methods described below are stored as software programs in a computer-readable storage medium and are configured for running on a computer processor.

Furthermore, software implementations can include, but are not limited to, distributed processing, component/object distributed processing, parallel processing, virtual machine processing, which can also be constructed to implement the methods described herein.

In the various embodiments of the present invention a network interface device 820 connected to a network environment 826 communicates over the network 826 using the instructions 824. The instructions 824 can further be transmitted or received over a network 826 via the network interface device 820.

While the computer-readable storage medium 822 is shown in an exemplary embodiment to be a single storage medium, the term "computer-readable storage medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term "computer-readable storage medium" shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure.

The term "computer-readable medium" shall accordingly be taken to include, but not be limited to, solid-state memories such as a memory card or other package that houses one or more read-only (non-volatile) memories, random access memories, or other re-writable (volatile) memories; magneto-optical or optical medium such as a disk or tape; as well as carrier wave signals such as a signal embodying computer instructions in a transmission medium; and/or a digital file attachment to e-mail or other self-contained information archive or set of archives considered to be a distribution medium equivalent to a tangible storage medium. Accordingly, the disclosure is considered to include any one or more of a computer- readable medium or a distribution medium, as listed herein and to include recognized equivalents and successor media, in which the software implementations herein are stored.

Those skilled in the art will appreciate that the computer system architecture illustrated in FIG. 8 is one possible example of a computer system. However, the invention is not limited in this regard and any other suitable computer system architecture can also be used without limitation.