Method and system of capturing an image of a card

TECHNICAL FIELD

0001The disclosed embodiments generally relate to image processing, and more particularly, to method and systems of capturing an image of a card.

BACKGROUND

0002Electronic devices, such as smartphones, are part of daily lives and have quickly become go-to devices, allowing users to accomplish many tasks with a simple tap and swipe such as, for example, making payments by “mobile wallets.” “Mobile wallets” refer to digital versions of real wallets that may contain multiple relationship cards. The term “relationship card,” or simply “card”, as used herein may refer to any physical card product that is configured to provide information, such as financial information (e.g., card numbers, account numbers, etc.), quasi-financial information (e.g., rewards balance, discount information, etc.) and/or individual-identifying information (e.g., name, address, etc.), when the card is read by a card reader. Examples of such cards include credit cards, debit cards, gift cards, rewards cards, frequent flyer cards, merchant-specific cards, discount cards, identification cards, membership cards, and driver's licenses, but are not limited thereto.

0003To add a card to a mobile wallet, the card must be imaged and digitalized. Conventional approaches for imaging and digitizing cards generally involve edge detection techniques by measuring brightness changes of card images, for example to detect whether the image brightness changes sharply (i.e., brightness discontinuities). However, many cards, such as gift cards, loyalty cards, and credit card, may contain magnetic stripes and/or colorful art work. For example, a magnetic stripe may be located at the top or bottom of a card, and may contain valuable information both below and above the magnetic stripe. For such cards, the conventional approaches may not be able to detect true edges of the cards. Therefore some valuable information of the cards may not be correctly captured.

0004The present disclosure is directed to addressing one or more of the problems set forth above and/or other problems associated with conventional imaging of cards.

SUMMARY

0005The disclosed embodiments relate to a method and a system of capturing an image of a card, wherein the card includes a magnetic stripe.

0006Consistent with a disclosed embodiment, a method of capturing an image of a card having a magnetic stripe may include obtaining a first image of the card by an imaging device, obtaining a plurality of images of the card via color delta analysis, and obtaining a third image of the card by comparing the first and the plurality of images.

0007Consistent with another disclosed embodiment, a system of capturing an image of a card having a magnetic stripe may include an imaging device and an image processing device. The system may be configured to obtain a first image of the card by the imaging device, obtain a plurality of images of the card via color delta analysis by the image processing device, and obtain a third image of the card by comparing the first and the plurality of images.

0008Consistent with yet another disclosed embodiment, a non-transitory computer-readable medium storing instructions that, when executed, cause a computer to perform a method of capturing an image of a card having a magnetic stripe. The method may include obtaining a first image of the card by an imaging device, obtaining a plurality of images of the card via color delta analysis, and obtaining a third image of the card by comparing the first and the plurality of images.

0009It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

0010The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate disclosed embodiments and, together with the description, serve to explain the disclosed embodiments. In the drawings:

0011<figref idref="DRAWINGS">FIG. 1</figref> is a block diagram of an exemplary system of capturing an image of a card, consistent with disclosed embodiments;

0012<figref idref="DRAWINGS">FIG. 2</figref> is a flow chart of an exemplary method of capturing an image of a card having a magnetic stripe, consistent with disclosed embodiments;

0013<figref idref="DRAWINGS">FIG. 3</figref> is a flow chart of another exemplary method of capturing an image of a card having a magnetic stripe, consistent with disclosed embodiments;

0014<figref idref="DRAWINGS">FIG. 4A</figref> is a schematic diagram of a card image without a magnetic stripe captured, consistent with disclosed embodiments;

0015<figref idref="DRAWINGS">FIG. 4B</figref> is a schematic diagram of the card in <figref idref="DRAWINGS">FIG. 4A</figref> with the magnetic stripe identified, consistent with disclosed embodiments;

0016<figref idref="DRAWINGS">FIG. 4C</figref> is a schematic diagram of the card image in <figref idref="DRAWINGS">FIG. 4A</figref> with the magnetic stripe captured, consistent with disclosed embodiments;

0017<figref idref="DRAWINGS">FIG. 5</figref> is a flow chart of yet another exemplary method of capturing an image of a card having a magnetic stripe, consistent with disclosed embodiments;

0018<figref idref="DRAWINGS">FIG. 6</figref> is a schematic diagram of a color analysis result, consistent with disclosed embodiments; and

0019<figref idref="DRAWINGS">FIG. 7</figref> is a flow chart of still yet another exemplary method of capturing an image of a card having a magnetic stripe, consistent with disclosed embodiments.

DETAILED DESCRIPTION

0020Reference will now be made in detail to the disclosed embodiments, examples of which are illustrated in the accompanying drawings. Wherever convenient, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

0021<figref idref="DRAWINGS">FIG. 1</figref> illustrates an exemplary system <b>10</b> for capturing and processing an image of a card <b>11</b>. The physical properties of the card (e.g., size, flexibility, location of various components included in the card) may meet the various international standards, including, e.g., ISO/IEC 7810, ISO/IEC 7811, ISO/IEC 7812, ISO/IEC 7813, ISO/IEC 7816, ISO 8583, ISO/IEC 4909, and ISO/IEC 14443. For example, a card may have a dimension of 85.60 mm (width) by 53.98 mm (height) by 0.76 mm (thickness), as specified in ISO/IEC 7810.

0022System <b>10</b> may include a computing system configured to receive and send information between the components of system <b>10</b> and components outside of system <b>10</b>. System <b>10</b> may include an imaging system <b>12</b> and an image processing system <b>14</b> communicating with each other through a network <b>16</b>. System <b>10</b> may include additional and/or alternative components.

0023In some embodiments, imaging system <b>12</b> may be a portable electronic device associated with a user, such as a smartphone or a tablet equipped with a camera for taking a live video or still image of a card. In other embodiments, imaging system <b>12</b> may include one or more computer systems associated with an entity. For example, imaging system <b>12</b> may be associated with an entity that provides imaging service (e.g., a photo studio). Imaging system <b>12</b> is configured to perform some or all the steps of the methods of capturing an image of a card, which will be described in detail below.

0024Image processing system <b>14</b> may include one or more computer systems associated with an entity that provides image processing services. For example, the entity may be a user, a cloud computing provider, an image service provider, a credit card issuer, a government agency, or other type of service entity that is capable of processing images of cards. Image processing system <b>14</b> is configured to perform some or all the steps of the methods of processing an image of a card, which will be described in detail below. For example, a credit card issuer may allow its customers to digitalize credit cards for mobile wallets. In this case, the credit card issuer may provide imaging processing system <b>14</b> for further processing images of credit cards provided by the customers. Alternatively, the credit card issuer may refer to a third party for providing imaging processing system <b>14</b>. In some embodiments, imaging system <b>12</b> may also provide image processing services.

0025Network <b>16</b> may include any type of network configured to facilitate communications and data exchange between components of system <b>10</b>, such as, for example, image processing system <b>14</b> and imaging system <b>12</b>. Network <b>16</b> may include a Local Area Network (LAN), or a Wide Area Network (WAN), such as the Internet. Network <b>16</b> may be a single network or a combination of networks. Network <b>16</b> is not limited to the above examples, and system <b>10</b> may be implemented with any type of network that allows entities (shown and not shown) of system <b>10</b> to exchange data and information. In some embodiments, a portion or an entire capability of image processing system <b>14</b> and imaging system <b>12</b> may be implemented in an application that may be loaded to a user device or distributed over a network.

0026System <b>10</b> may be configured to capture and process an image of card <b>11</b> to digitalize card <b>11</b>. In some embodiments, an entity such as a credit card issuer may provide card <b>11</b> to a customer for use in conducting transactions, including online transactions through a mobile wallet associated with a financial service account held by the customer. In some embodiments, the entity who provides card <b>11</b> may also provide image processing system <b>14</b>. In some embodiments, card <b>11</b> and image processing system <b>14</b> may be provided by different entities. To digitalize card <b>11</b>, imaging system <b>12</b> may capture a live video or still images of card <b>11</b> to obtain information contained in card <b>11</b>, such as credit card number, card holder name, card expiration date, etc. Imaging system <b>12</b> may communicate with image processing system <b>14</b> via network <b>16</b> to transfer the live video or still images for further processing in image processing system <b>14</b>. In some embodiments, imaging system <b>12</b> may subsequently receive a processed image of card <b>11</b> from image processing system <b>14</b> and present the processed image to the user for verification and/or selection.

0027In some embodiments card <b>11</b> may further include an information component <b>22</b> disposed on or n a card body <b>24</b>. As used herein, an “information component” may be one or more devices and/or elements configured to receive, store, process, provide, transfer, send, delete, and/or generate information. For example, information component <b>22</b> may be a microchip (e.g., an Europay, MasterCard, and Visa (EMV) chip), a communication device (e.g., Near Field Communication (NFC) antenna, Bluetooth® device, WiFi device), a magnetic strip, a barcode, Quick Response (OR) code, etc. Information component <b>22</b> may be secured (or affixed, attached) to card body <b>24</b> in such a way that allows card body <b>24</b> to carry information component <b>22</b> while maintaining a utility of information component <b>22</b> (i.e., allowing information component <b>22</b> to interact with card reader). Herein, the utility of information component <b>22</b> indicates that information component <b>22</b> functions properly, for example, a card containing RFID (radio frequency identification) shielded by a layer can be properly read by an RFID reader through the layer.

0028<figref idref="DRAWINGS">FIG. 2</figref> is a flow chart of an exemplary method <b>200</b> of capturing and processing an image of a card having a magnetic stripe. Method <b>200</b> may be implemented in system <b>10</b> of <figref idref="DRAWINGS">FIG. 1</figref> and may include the following steps.

0029In step <b>202</b>, an image (i.e., a first image) of card <b>11</b> having magnetic stripe <b>22</b> is obtained by imaging system <b>12</b>. Magnetic stripe <b>22</b> may be located on a top or bottom portion of card <b>11</b>, for example, at a location about 80% height of card <b>11</b>. In some embodiments, card <b>11</b> may also include card art work.

0030In this exemplary embodiment, imaging system <b>12</b> may include a smartphone (e.g., iPhone 4 and above) having an imaging device, an operating system, and software libraries (e.g., Rect-Detect, Core Image) to assist in capturing the image of card <b>11</b>. In some embodiments, imaging system <b>12</b> may include other electronic devices, (e.g., a tablet) and associated operating system and libraries for obtaining and initially processing the image of card <b>11</b>.

0031First, an image of card <b>11</b> is captured. Data representative of the image is then produced from the captured image. In conventional card image capture systems, the presence of magnetic stripe <b>22</b> may inhibit the accurate recognition of the boundaries of the card in the captured image. Accordingly, the disclosed system first analyzes the image data to determine if a magnetic stripe is included, and provides appropriate processing to generate and store a first version of the card image data.

0032In step <b>204</b>, color delta analysis is employed to generate a first plurality of versions of the captured card image data (e.g., representative of a plurality of images of card <b>11</b>). These versions may be obtained by applying several filters to the original image data. For example, CoreImage filters included as part of the Apple IOS operating system may be used. Specifically, the card image data may be processed by a grayscale filter with contrast increased, a filter with contrast highly enhanced, a combination filter that applies edge detection and overlays the edge enhancement on the original image, and a filter that simply returns a clean image of the original image. As used herein, a “clean” image refers to an image obtained by filtering the original image to remove undesireable image noise

0033Next, true edges of card <b>11</b> are detected by performing color delta analysis on the filtered versions of the card image data. As used herein, “color delta” refers to a difference between two color values, for example, a difference between a RGB value of a first pixel and a RGB value of a second pixel.

0034In step <b>206</b>, a third version of the card image data is generated by comparing the first version of the card image data and the multiple filtered versions of the card image data. In this exemplary embodiment, the first version of the card image data is compared with each of the multiple filtered versions of the card image data, discarding one image after each comparison, such that a second plurality of versions of the card image data are generated as a result of the comparisons. The second plurality of versions of images are images having a higher contrast. The third version is then determined by using mean values of the second plurality of the card image data. A confirmation image (i.e., a third image of card <b>11</b>) represented by the third version of card image data may then be presented to the user for verification and/or selection to confirm that the presented image includes complete information of the card.

0035Step <b>202</b> in <figref idref="DRAWINGS">FIG. 2</figref> may further include substeps. <figref idref="DRAWINGS">FIG. 3</figref> is a flow chart of an exemplary method <b>300</b> of capturing and processing an image of a card, which may be in combination with step <b>202</b> in <figref idref="DRAWINGS">FIG. 2</figref>, and may include the following steps

0036In step <b>2021</b>, a first rectangle is drawn and identified by imaging system <b>12</b> from the captured image data. In this exemplary embodiment, an iPhone video camera and Rect-Detect library is used to search the captured image data for a card shape. The first rectangle may cover the entire card or just a portion of the card. As shown in <figref idref="DRAWINGS">FIG. 4A</figref>, a first rectangle <b>404</b> covers only a portion of card <b>400</b> having a magnetic stripe <b>402</b>.

0037In step <b>2022</b>, a ratio of the first rectangle is calculated, for example, the ratio being a ratio of width to height of the rectangle.

0038In step <b>2023</b>, the calculated ratio is compared to a pre-specified card ratio value. For example, the pre-specified card ratio value may be 1.6179, i.e., the ratio of a standard credit card. If the calculated ratio is about the same as the pre-specified ratio value, the method determines that a complete image of the card has been obtained. Method <b>300</b> will proceed to step <b>2026</b> in which the first image of card <b>11</b> is generated by cropping along the first rectangle. The first image may be considered an image covering the entire card, and may contain some background that is not part of the card itself.

0039If the calculated ratio is substantially different than the pre-specified ratio value, for example, in a range of 1.35-1.6178 as shown in <figref idref="DRAWINGS">FIG. 4A</figref>, method <b>300</b> determines that additional processing is required. Method <b>300</b> will then proceed to step <b>2024</b>, in which a color analysis is performed on the first rectangle to search for a specific color of the magnetic stripe. For example, a specific consistent color of the magnetic stripe may be searched for in the captured card image data immediate above and below of the first rectangle, along a width of the first rectangle. As shown in <figref idref="DRAWINGS">FIG. 4B</figref>, a line <b>406</b> may be drawn along the middle of the first rectangle and the color sampled along the width of the rectangle as indicated by a double arrow line <b>408</b>. The locations for color sampling may be selected as shown by circles <b>410</b> such that the dimension of the magnetic stripe can be identified. For example, if the sampled color matches the specific color of the magnetic stripe (e.g, dark brown), color sampling will be continued in different positions. If the sampled color is consistent along the top or bottom of the first rectangle, the magnetic stripe is identified.

0040In step <b>2025</b>, a second rectangle is drawn in accordance to the pre-specified ratio value. As shown in <figref idref="DRAWINGS">FIG. 4C</figref>, a second rectangle <b>412</b> may be drawn based on the pre-specified ratio value to cover the entire card including the identified magnetic stripe <b>402</b>.

0041In step <b>2026</b>, the first image is obtained by cropping along the second rectangle. The first image may be considered an image covering the entire card, and may contain some background that is not part of the card itself.

0042In some embodiments, step <b>204</b> in <figref idref="DRAWINGS">FIG. 2</figref> may further include substeps. <figref idref="DRAWINGS">FIG. 5</figref> shows a flow chart of an exemplary method <b>500</b> of capturing and processing an image of a card, which may be in combination with step <b>204</b> in <figref idref="DRAWINGS">FIG. 2</figref>, and may include the following steps.

0043In step <b>2041</b>, image data of a card is received. The received image data (e.g., representative of a still image of the card) may be provided by the user, or may be received from imaging system <b>12</b>.

0044In step <b>2042</b>, an average RGB value of pixels in each pixel row of the received image data is calculated. A map correlating average RGB values with corresponding pixel rows of the received image data may be generated. In this exemplary embodiment, the average RGB value of pixels in each pixel row of the received image data may be calculated using built-in CoreImage filters provided by Apple in the IOS operating system.

0045In step <b>2043</b>, an average RGB value of pixels in each pixel column of the received image data is calculated. A map correlating average RGB values with corresponding pixel columns of the received image data may be generated. Similarly, the average RGB value of pixels in each pixel column of the received image data may be calculated using built-in CoreImage filters of IOS.

0046In step <b>2044</b>, a delta RGB value for each pair of average RGB values of adjacent pixel rows is calculated. A map correlating delta RGB values with corresponding pixel rows of the received image data may be generated.

0047In step <b>2045</b>, a delta RGB value for each pair of average RGB values of adjacent pixel columns is calculated. A map correlating delta RGB values with corresponding pixel columns of the received image data may be generated.

0048In step <b>2046</b>, delta RGB peak values near edges of the received image data are identified and determined as edges of the card. The delta RGB peak values may be high points close to the edges of the received image data where colors change most quickly. <figref idref="DRAWINGS">FIG. 6</figref> shows an exemplary map correlating delta RGB values with corresponding pixel rows of the still image. As shown in <figref idref="DRAWINGS">FIG. 6</figref>, delta RGB peak values <b>602</b>, <b>604</b>, <b>606</b>, and <b>608</b> respectively correspond to a left edge of card <b>601</b>, a left edge of magnetic stripe of card <b>601</b>, a right edge of magnetic stripe of card <b>601</b>, a right edge of card <b>601</b>, respectively.

0049In step <b>2047</b>, the received image data is cropped along the detected edges of the card. The resultant image data may be one of the plurality of versions of image data in step <b>204</b> of <figref idref="DRAWINGS">FIG. 2</figref>

0050In some embodiments, method <b>300</b> may further include running a high pass filter to remove meaningless noise in the delta RGB values.

0051In some embodiments, the delta RGB value may be a Euclidean distance between colors. As used herein, a Euclidean distance refers to a distance defined as √{square root over ((R₂−R₁)²+(G₂−G₁)²+(B₂−B₁)²,)} where R₁, R₂, G₁, G₂, B₁, B₂are red, green, and blue color values of a first pixel and a second pixel, respectively. In some embodiment, other distances to measure a delta RGB value may also be applied.

0052In some embodiments, method <b>300</b> may include running an additional pass at rectangle detection on the received image data to potentially compensate for any movement that may have occurred between the last frame of rectangle detection and the capture of the full size image. The result of this additional pass will be further processed.

0053In some embodiments, method <b>300</b> may include de-skewing a slightly expanded version of detected rectangle <b>412</b> (<figref idref="DRAWINGS">FIG. 4C</figref>). The expansion ensures that at least some amount of background around the actual card is included in the rectangle. Additional rectangle detection may be performed on the de-skewed image to capture a still image of the card. This may get a precise location of the card in the rectangle. If not, card bounds may be determined by inverting the scaling (i.e., a card ratio) mentioned previously.

0054In some embodiments, method <b>300</b> may further include running the de-skewed or cropped image with background through several filters, for example, grayscale filter with contrast increased, a filter with contrast highly enhanced, a combination filter that applies edge detection and overlays the edge enhancement over the original image, or a filter that simply returns a clean image of the original image.

0055In some embodiments, for each of the images obtained after applying filters, steps <b>2041</b>-<b>2047</b> of method <b>300</b> are applied to obtain the plurality of images of step <b>204</b> in <figref idref="DRAWINGS">FIG. 2</figref>.

0056In some embodiments, step <b>206</b> in <figref idref="DRAWINGS">FIG. 2</figref> may further include substeps. <figref idref="DRAWINGS">FIG. 7</figref> shows a flow chart of an exemplary method <b>700</b> of capturing and processing an image of a card, which may be in combination with step <b>204</b> in <figref idref="DRAWINGS">FIG. 2</figref>, and may include the following steps

0057In step <b>2061</b>, delta RGB values of the first image in step <b>202</b> are compared with corresponding delta RGB values of each of the plurality of images in step <b>204</b> to obtain a second plurality of images. One image will be discarded from each comparison if the value is off by more than <b>1</b>%. As a result, one image results from each comparison to produce the second plurality of images (i.e., representative of a second plurality of image data).

0058In step <b>2062</b>, mean values of delta RGB values of the second plurality of images are determined. As used here, a mean value of delta RGB refers an average delta RGB value averaged over corresponding delta RGB values of the second plurality of images.

0059In step <b>2063</b>, peak mean values of delta RGB values are identified and determined as edges of the card.

0060In step <b>2064</b>, one of the second pluralities of images is cropped along the identified edges of the card to produce the third image of the card in step <b>206</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The third image may be a final image of card <b>11</b>, and may also be a desirable image by the user. This third image will be added to mobile wallet.

0061Accordingly, some embodiments of the present invention may recognize a card in its entirety without cropping any of the edges. The card can be automatically cropped, thereby reducing the need for a user to capture the image and remove the background. Further, the disclosed embodiments may provide a higher success rates when capturing a card with a magnetic stripe. Additionally, the disclosed embodiments may be used to identify the legitimacy of documents and identification card. The color delta analysis may be used to look for patterns in images without having to scan them manually.

0062While illustrative embodiments have been described herein, the scope includes any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., of aspects across various embodiments), adaptations or alterations based on the present disclosure. For example, the order of the steps of the above exemplary method may be rearranged in any preferred or suitable order, or any step may be removed or added.

0063The disclosed embodiments may also provide a non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor of a computer, cause the computer to perform the above-described methods. The non-transitory computer-readable medium may be or include any type of volatile or non-volatile memory device, for example including floppy disks, optical discs, DVD, CD-ROMs, microdrive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.

0064The elements in the claims are to be interpreted broadly based on the language employed in the claims and not limited to examples described in the present specification or during the prosecution of the application, which examples are to be construed as non-exclusive. It is intended, therefore, that the specification and examples be considered as example only, with a true scope and spirit being indicated by the following claims and their full scope of equivalents.