Method for establishing pixel colour probabilities for use in OCR logic

The present invention relates to designing decision trees for optical character recognition (OCR) devices, and specifically to establishing probabilities of colour for particular pixels that are useful in generating OCR decision trees.

A decision tree is that part of the control system of an OCR machine that determines how the machine interprets picture element (pixel) information that is read in from scanning a document.

One technique for character recognition is to establish a decision tree in which different pixels are successively examined, and branches are taken at each pixel examination based on whether the pixel is black or white. Such decision trees are known and described in the art.

An optical character recognition device is generally only capable of recognising characters printed in a font that the OCR device has been trained to recognise. Typically, an OCR device is programmed by its manufacturer to recognise a few common type fonts. But, with the proliferation of laser printers and the widespread ability to generate numerous type fonts, many users desire the ability to control their OCR devices to recognise characters in a new font, different from any included by the manufacturer.

To design a decision tree for an OCR device to recognise characters in a particular font, the probability, for a given character, that a given pixel in a block of pixels representing that character is black is needed to identify pixels useful in differentiating that character from others. An approach to decision tree design is described in Casey et al., "Decision Tree Design Using a Probabilistic Model," <u>IEEE Transactions on Information Theory</u>, Vol. IT-30, No. 1, pp. 93-99 (1984).

("Character" is used herein to identify the printed or written symbol to be recognised by the OCR device. A character is typically a letter, a numeral, or some other symbol. It is to be recognised that "character" may also refer to a class of symbols, as when two symbols are very similar and the OCR decision logic does not necessarily distinguish between them, e.g. the numeral "1" and lower case "1" in same type fonts. Also, the following description refers to black and white pixels, although other combinations of distinguishable colours may also be used. Further, in the context of black and white pixels, methods and apparatus are described based on black pixels and probabilities of black. Decision tree logic can also be based on white pixels and probabilities of white.)

To obtain the probabilities of black for individual pixels in each character, hundreds of samples of each character in the font to be recognised must be examined. Collecting and identifying the samples is time consuming and expensive. Often, such large number of samples may not be available.

Thus, a need exists for a technique for designing OCR decision trees with very few (ideally only one) samples of each character.

A conventional decision tree generation process is shown in Fig. 1. A large plurality of samples in the font to be recognised is printed. The samples are scanned to generate an array of pixels. For each character or class of characters to be identified, all the samples of that character or class of characters are superimposed, and the probability that a particular pixel is black is counted as the number of times that pixel is black in the samples divided by the number of samples. From such probabilities, certain pixels can be selected for use in generating a decision tree to use in recognising characters in that font. This procedure usually requires an extremely large number of samples for each character or class of characters, typically on the order of 100 to 200 samples, for accurate character recognition.

EP-A-0343786 discloses a method of operating character recognition apparatus in which a plurality of sets of character recognition decision trees are stored in a memory. From the plurality, a set of recognition trees is chosen which has the smallest ratio of unidentified (reject) characters to identified characters, and these trees are modified in response to input signals from an operator on recognition of the characters not automatically recognised (or incorrectly recognised) by the apparatus.

The present invention provides an apparatus and method to reliably estimate the probability of the colour of a particular pixel of a character without examining large numbers of character samples. The method of the invention uses the unexpected experimental observation that the probabilities that pixels have certain colours can be reliably estimated as a function of their neighbourhood states, with the neighbourhood state of a particular pixel being a weighted sum of the values of the pixels in the immediate neighbourhood of the pixel under consideration.

According to one aspect of the invention there is provided a method of operating an optical character recognition device (and the corresponding device) which uses a decision tree for the recognition of characters in a first font, the device having a memory storing a probability table providing the probability, in a second font different from said first font, that a pixel is a first distinguishable colour as a function of its neighbourhood state, the method comprising the steps of: (i) scanning a document having characters in said first font to generate an array of pixels, each having a neighbourhood state; (ii) identifying in said array a cluster of pixels representing a character; (iii) selecting a pixel in said cluster; (iv) determining the neighbourhood state of said pixel in the exact same way as the neighbourhood state was determined for each pixel in the second font; (v) addressing said memory with the neighbourhood state of said selected pixel; (vi) reading from said memory the probability of said selected pixel being said distinguishable colour, associated with the neighbourhood state of said selected pixel, and assigning said probability to said selected pixel; (vii) repeating steps (iii) to (vi) for further selected pixels in said cluster; (viii) generating a new decision tree for said first font using said assigned probabilities of said distinguishable colour, without reliance on pre-existing decision trees ; and (ix) using the new decision tree for interpreting pixels to recognise characters.

Thus, the method of the present invention for automatically controlling a character recognition device to recognise characters in an unknown font includes scanning a document printed in the unknown font to generate an array of pixels. The immediate vicinity of each pixel in the array is then analyzed to compute a so-called neighbourhood state for the pixel. A block of pixels representing a character is identified. A pixel is selected in the block, and a memory is addressed with the neighbourhood state of the selected pixel. The memory has stored in it a probability table providing the probability in a second font that is different from the first unknown font, that a pixel is black as a function of its neighbourhood state. The probability of black associated with the neighbourhood state of the selected pixel is read from the memory, and that probability is assigned to the selected pixel. The decision tree is then generate for the unknown font using that assigned probability of black.

The neighbourhood state of a pixel is preferably determined by identifying and examining a preselected matrix of neighbour pixels as the pixel's neighbourhood.

According to a second aspect of the invention there is provided a method for the recognition of characters in an unknown font by means of an optical character recognition device which uses a decision tree to interpret pixels, the decision tree being created according to the following method steps: scanning a document in a first font to generated an array of pixels; for each pixel in said array, identifying a preselected matrix of neighbour pixels as the pixel's neighbourhood; determining from said pixel array the probability that a selected pixel is a first distinguishable colour, given the neighbourhood state of the pixel; storing a probability table providing, for each pixel neighbourhood state, the probability the selected pixel is said distinguishable colour; scanning a document in an unknown font to generate a second array of pixels; identifying in said array a plurality of clusters of pixels, each cluster being representative of a character; for each of a plurality of pixels in each of said clusters, identifying said preselected matrix of neighbour pixels as the pixel's neighbourhood, establishing the neighbourhood state of that pixel based on said pixel neighbourhood, addressing said table by the neighbourhood state of that pixel, and reading from said table the probability said pixel is said distinguishable colour; and generating a new decision tree for said unknown fo nt using the probabilities from said table for selected pixels, without reliance on pre-existing decision trees.

According to a third aspect of the invention there is provided a method of controlling an OCR device to recognise characters in an unknown font using a decision tree to interpret pixels for said recognition, the decision tree being generated using probabilities that pixels have a first distinguishable colour, said probabilities being assigned to pixels by the following sequence of steps: <ul id="ul0001" list-style="none"><li>(a) scanning a reference document in a first font and converting said scanned reference document into an array of pixels, wherein:</li><li>(i) each converted pixel has either a first colour or a second colour;</li><li>(ii) each converted pixel has a neighbourhood comprising a predetermined number of neighbour pixels of said converted pixel; and (iii) said neighbourhood has a neighbourhood state determined by the colours of the neighbour pixels;</li><li>(b) determining, for each pixel of said scanned reference document, the colour of that pixel;</li><li>(c) determining, for each pixel of said scanned reference document, its neighbourhood state;</li><li>(d) dividing the number of times a pixel in a neighbourhood having a first neighbourhood state has said first colour by the number of times in said scanned document said first neighbourhood state appears in said reference document to obtain a probability that a pixel has said first colour given that its neighbourhood has said first neighbourhood state;</li><li>(e) repeating step (d) for each possible neighbourhood state to generate a probability table comprising a list of all the possible neighbourhood states and the associated probability of each selected pixel having said first colour;</li><li>(f) storing said probability table in said OCR device; and</li><li>(g) scanning a second document in an unknown second font and converting said second document into an array of pixels, wherein:</li><li>(i) each converted pixel has either a first colour or a second colour;</li><li>(ii) each converted pixel has a neighbourhood comprising a predetermined number of neighbour pixels of said converted pixel, wherein said neighbourhood comprises neighbour pixels corresponding to the neighbourhood of the pixels of said reference document; and</li><li>(iii) said neighbourhood has a neighbourhood state determined by the colours of said neighbourhood pixels;</li><li>(h) determining, for each of a selected plurality of converted pixels in said cluster, the state of its neighbourhood; and</li><li>(i) for each selected converted pixel, looking up the corresponding neighbourhood state in said stored probability table and reading out from said probability table the probability said pixel has said first colour.</li></ul>

In order that the invention may be more fully understood it will now be described by way of example and with reference to the accompanying drawings in which:

Fig. 1 is a flow diagram of a conventional decision tree development process.

Figs. 2A, 2B, and 2C illustrate different pixel neighbourhoods.

Fig. 3 illustrates one scheme for pixel neighbourhood numbering for determining the pixel neighbourhood state.

Fig. 4 is a block diagram of apparatus suitable for generating pixel colour probability tables.

Fig. 5 is a block diagram of apparatus suitable for generating new OCR decision logic according to the invention.

Fig. 6 illustrates a block or cluster of pixels representing a character.

Fig. 7 is a flow diagram of OCR decision tree generation using the invention.

The colour of a pixel in a block of pixels depends on the character class to which the pixel belongs. Applicants have found that, unexpectedly, the colour of the pixel can be predicted solely as a function of the neighbourhood state of the pixel, without any knowledge of the character class to which it belongs.

In a given font, the colour of a pixel in a binary pixel pattern may be predicted, given the colours of the neighbouring pixels. Refer, for example, to Fig. 2. In Fig. 2A, a pixel X of unknown colour is shown surrounded by its eight adjacent neighbouring pixels, each of which is shown to have a value 0 (indicating the colour white). The central pixel X has a very high probability of being white (having a value 0). The probability is not exactly 1.00, as there may be a stray dot of black at that point. Similarly, referring to Fig. 2B, the central pixel X of the illustrated neighbourhood has a very high probability of being black (having a value 1), as its eight surrounding neighbour pixels are black.

An example of a pixel X having a neighbourhood that is half black and half white is shown in Fig. 2C. The likelihood that the pixel at the centre of this neighbourhood has value 1 can be statistically determined by examining a large number of samples. An examination of the ability to predict the colour of a pixel given the colour of neighbouring pixels is provided in Mohiuddin, et al., "Lossless Binary Image Compression Based on Pattern Matching," International Conference on Computers, Systems &amp; Signal Processing, Bangalore, India (Dec. 9 - 12, 1984).

The pixel neighbourhood comprises a preselected number of surrounding neighbour pixels in the horizontal and vertical directions. Thus, the pixel and its neighbourhood form a grid or matrix of pixels. In the illustrated embodiment, the neighbourhood comprises the eight adjacent surrounding pixels to form a square 3 X 3 grid. Other arrangements are possible, and the neighbourhoods need not be square.

The state of a pixel may be defined as a weighted sum of the elements of its neighbourhood. For example, arbitrarily, the neighbour pixels may be numbered clockwise starting from the west neighbour as shown in Fig. 3. The west neighbour is assigned the value 2⁰(=1). The northwest neighbour is assigned the value 2¹(=2), and so forth, with the southwest neighbour assigned the value 2⁷(=128). The actual numbering itself is not critical, as the objective of the numbering is only to provide a unique identification for each possible neighbourhood state. Since there are eight members in the neighbourhood, there could be a total of 256 different states.

The neighbourhood state is obtained by summing the values of the neighbours that are black. Thus, the pixel neighbourhood shown in Fig. 2A, using the numbering system shown in Fig. 3, has a neighbourhood state or value of 0, while the neighbourhood shown in Fig. 2B has a neighbourhood state value of 255. The neighbourhood shown in Fig. 2C has a neighbourhood state value of 30, the sum of 2¹+2+2³+2⁴.

From a scan of a document of text, the number of times each neighbourhood state occurs can be determined, as can the number of times out of those occurrences that the centre pixel is black. Thus, the probability that the centre pixel is black given that its neighbourhood state has a certain value can be determined. This is done by dividing the number of times pixels with a certain neighbourhood are black by the number of times that neighbourhood state occurs. From this, a table may be generated with an entry of the probability of black for each of the 256 possible neighbourhood states.

A block diagram of equipment suitable for generating and storing such a probability table is shown in Fig. 4. The scanner 11 scans a reference document from which the statistics are to be taken and supplies the binary pixel array to the neighbourhood state and probability computation logic 13. For each pixel in the array, the logic 13 adds together the assigned values for all the neighbour pixels that are black to obtain the neighbourhood state value for that pixel. The colour of that pixel under consideration is also determined. Once those statistics are gathered, for each neighbourhood state value, the percentage of time that the centre pixel was black can be readily determined. That percentage provides the probability of black given the neighbourhood state value. Thus, the logic 13 has some memory for temporary storage of various counts as the pixel arrays are examined. The computed probability of black as a function of the neighbourhoods state (probability table) is read into a memory 15 for later use.

According to the invention, applicants have discovered that, unexpectedly, the probability table providing the probability of black for a pixel given its neighbourhood state, generated as described above, is substantially invariant, regardless of the character class the pixel is located in. Thus, the probability table is also independent of the font being examined. Between vastly different fonts or even between fonts representing different languages, the probability table remains substantially constant. Therefore, the probabilities previously generated and stored for one font may be used in controlling the OCR device to recognise the new font.

Thus, to develop a decision tree for a new font, different from the font used in generating the probability table, the probability table generated for one font may be used to significantly reduce the amount of time and the number of symbol or character samples necessary to generate the statistics sufficient for the OCR machine to learn the new font and develop a decision tree for it. Additionally, multiple examples of certain characters in the new font may not be available, meaning that the probabilities of colour for pixels cannot be estimated using conventional methods. Consequently, optical character recognition cannot be carried out on the new font using conventional means.

Fig. 5 is a block diagram of a machine suitable for generating new OCR decision logic. The apparatus of Fig. 5 used to generate new OCR decision logic may be the same as that of Fig. 4 used in the development of the probability table. The scanner 21 may be the same as the scanner 11 of the system of Fig. 4. The neighbourhood state computation portion of the logic 25 may be the same as the portion of the neighbourhood state and probability computation logic 13 shown in Fig. 4. The memory 15, which stores the probability table, may be the same as the memory in which the probability table is originally stored in Fig. 4. To generate the OCR decision tree logic, a two way communication between the memory 15 and the logic 25 that performs neighbourhood state computation and decision tree generation, while the system of Fig. 4 only requires that the neighbourhood state and probability computation logic be able to read data into the memory 15, and need not read data from the memory.

The scanner 21 scans a teaching document printed in the unknown font to be identified by the machine. The scanner 21 converts the document into an array of pixels in a conventional manner. Each pixel is either black or white. Each pixel also has a neighbourhood of the type used in developing the probability table stored in the memory. The same neighbourhood is used around each pixel as was used in developing the probability table, with corresponding neighbour pixels assigned the same value. In this way, a pixel in the teaching document and a pixel in the reference document used in generating the probability table, both surrounded by pixels of the same colours in the same positions, will have the same neighbourhood state value. Using the numbering scheme shown in Fig. 3, the eight pixels surrounding that pixel under examination become the pixel's neighbourhood. The neighbour pixels are assigned values as shown in Fig. 3. The west neighbour pixel is assigned the value 2⁰=1, the northwest neighbour 2¹=2, and so forth, with the southwest neighbour assigned the value 2⁷=128.

Blocks or clusters of pixels, each block representing a character, may be identified in the array of pixels representing the scanned page. An exemplary character pattern of pixels representing the capital letter "A" (and greatly enlarged) is shown in Fig. 6. Such a pattern may be a square having 20 pixels on a side for a total cluster of 400 pixels representing the character.

A pixel to be examined is selected from within the character pattern. The neighbourhood state computation logic portion of the logic unit 25 determines the pixel's neighbourhood value using the same method for assigning values used in generating the probability table stored in the memory 15. Once the neighbourhood state or value is established for the pixel, that neighbourhood value is used to address the probability table previously stored in the memory 15. The associated probability stored for that neighbourhood state is read out from the memory. The probability read from the stored probability table then is assigned to the selected pixel in the character block for that character of the new font. That assigned probability is then used as the probability of black for that pixel in generating a decision tree for the new font in a known manner.

The decision tree logic design uses the different probabilities of black for various pixels in the different character pixel blocks to determine which pixels should be examined when presented with a new document in the new font. The objective is to find pixels that are useful in distinguishing different characters to be recognised. An appropriate method of designing a decision tree using such probability statistics is described in Casey and Nagy, "Decision Tree Design Using a Probabilistic Model," IEEE Transactions on Information Theory, Vol. IT-30, No. 1, pp. 93-99 (January, 1984).

If more than one sample of a character appears in the teaching document, and the samples are not identical, an average or ideal representative of the character or pixel block representing that character is used for establishing the colour of the neighbour pixels involved in the neighbourhood state determination.

The averaging of the character involves comparing the different occurrences of that character, and determining for each pixel in the character block whether the pixel is black more often than white or white more often than black. For each pixel that is more often black (at least 50%), that pixel is said to be black for the purposes of being a neighbour to determine the neighbourhood states of the pixels within that pixel block. Similarly, a pixel that is white more than 50% of the time is said to be white for the neighbourhood value determinations. In this way a more "typical" representative for each character class is generated.

Fig. 7 illustrates a flow diagram of the process of teaching an OCR device to recognise a new font, incorporating the present invention. A description of that flow diagram will summarise the preceding description of the preferred embodiment.

A reference document of text in any font is scanned (70). The frequency that a pixel is black given that it has a certain neighbourhood state is counted to generate a probability table providing the pixel probability of black as a function of neighbourhood state (71). This probability table generation function (72) needs to be done only once, and need not be repeated every time a document in a new font is to be recognised.

When decision tree logic is to be generated for a new font, a document printed in that new font is scanned (73). The clusters or blocks of pixels or character patterns representing different characters or classes of characters are identified (74). If more than one representative of a character is identified, those representatives are averaged to obtain a character representative.

Using the character representative, the pixel probabilities for pixels in each character block are generated (75). From those probabilities the tree design logic can create appropriate OCR decision tree logic (76).

With the invention, the generation of the probability table (the portion of the diagram in Fig. 7 left of the dashed line) may take place in a different machine than the generation of the character recognition tree (the portion of the diagram in Fig. 7 right of the dashed line). Thus, using the invention, the manufacturer of the optical character recognition device may store the probability table in the device memory, and may in many cases store a few character recognition decision trees in the device. The manufacturer also stores in the OCR device the programming necessary to enable the device to generate a character recognition decision tree from the pixel probability statistics. Then, if the user wishes to have the OCR device recognise a new font, different from any of the ones for which the manufacturer has stored decision logic trees, the user may have the OCR device generate a new decision tree for the new font.

A more detailed preferred embodiment is a method for controlling an OCR device to recognise characters that in a font which it has previously not been controlled to recognise by the steps of assigning a probability that a pixel has a first colour to said pixel and comprising: (a) scanning a reference document printed in a first font and converting said scanned reference document into an array of pixels, wherein: (i) each converted pixel has either a first colour or a second colour; and (ii) each converted pixel has a neighbourhood comprising a predetermined number of neighbour pixels adjacent said converted pixel; and (iii) said neighbourhood has a neighbourhood state determined by the colours of the neighbour pixels; (b) determining, for each pixel of said scanned reference document, the colour of that pixel; (c) determining, for each pixel of said scanned reference document, its neighbourhood state; (d) dividing the number of times a pixel in a neighbourhood having a first neighbourhood state has said first colour by the number of times in said scanned document said first neighbourhood state appears in said reference document to obtain a probability that a pixel has said first colour given that its neighbourhood has said first neighbourhood state; (e) repeating step (d) for each possible neighbourhood state to generate a probability table comprising a list of all the possible neighbourhood states and the associated probability of the centre pixel having said first colour; (f) storing said probability table in said OCR device; and (g) scanning a second document printed in an unknown second font and converting said second document into an array of pixels, wherein: <ul id="ul0002" list-style="none"><li>(i) each converted pixel has either a first colour or a second colour;</li><li>(ii) each converted pixel has a neighbourhood comprising a predetermined number of neighbour pixels adjacent said converted pixel, wherein said neighbourhood comprises the corresponding neighbour pixels as the neighbourhood of the pixels of said reference document; and</li><li>(iii) said neighbourhood has a neighbourhood state determined by the colours of said neighbour pixels; (h) determining, for each pixel, the state of its neighbourhood; and (i) for each pixel, looking up the corresponding neighbourhood state in said stored probability table and reading out from said probability table the probability said pixel has said first colour.</li></ul>

The method also includes in the step of scanning the second document identifying the first character printed on said second document and defined by the pixels in a pixel cluster; determining if said first character appears more than once in said second document; and if said first character appears more than once in said second document, averaging a plurality of the occurrences of said first character to obtain an average first character. Preferably the step of averaging the first character comprises the steps of comparing the different occurrences of said first character; determining for each pixel in the character window for said first character whether the pixel has said first colour more often than said second colour; for each pixel that has said first colour more often than said second colour, that pixel is determined to have said first colour in said averaged first character; and for each pixel that has said second colour more often than said first colour, that pixel is determined to have said second colour in said averaged first character.

With the present invention, it will be seen that only a single representative is required for each character in a new font. From that single representative, the OCR device can generate the pixel colour probabilities necessary for the tree design logic to create a decision tree for the new font. This greatly simplifies and speeds the generation of new decision trees for recognising new fonts.