Segmentation method and system for multiple raster content (mrc) representation of document

The MRC (multi or mixed raster content) representation of a document is versatile. This provides the ability to represent both color images and color or black and white text. The MRC representation allows a large number of "planes" to be used to represent the content of a document. MRC representation is becoming more and more important in the market. It has already been established as the leading color fax standard.

In MRC representation, an image is represented by one or more planes. The main advantage of the MRC representation of a document is that it provides an effective way to store, send and manipulate large digital color documents. This method takes advantage of the characteristic of the human visual system that the ability to discern small color changes is significantly reduced in the presence of high contrast edges. Edge information is usually separated from smoothly changing color information and encoded in one of the planes, called the selector plane (preferably at a high resolution of 2 bits or more per pixel). Following careful separation, the various planes can be compressed independently using standard compression schemes (such as JPEG and G4) with good compression and high quality.

<p> A method and system is needed to efficiently separate images into a set of planes, taking full advantage of the MRC representation.</p>

<p> Disclosed are methods and systems for separating image signals into a set of image planes. The system includes a minimum and maximum module, a dynamic threshold module, and a separation module. The min-max module receives an image signal and searches for the min and max within at least one window centered on the current pixel in the image signal. The dynamic threshold module is an index that represents the distance and orientation of the current pixel for each threshold plane, based on each of the minimum and maximum received from the minimum and maximum modules and the current pixel for the window. Each of the above is calculated, and a control signal is output based on the index. The separation module separates the image signal into a set of image planes according to the control signal by including a representation of the current pixel in at least one of the image planes.</p><p> The present invention provides methods and systems for separating image signals into a set of image planes. The image signal represents a digitally scanned document. Image planes are suitable for mixed raster content (MRC) representation of digitally scanned documents.</p>

Figure 1 shows a typical MRC representation. This representation includes four independent planes: foreground (FG), background (BG), selector (SEL), and rendering hints (HINTS). In the most common case, at a higher level, there can be a large number of foreground and selector pairs. However, in many applications, this representation is limited to three or four planes. Background planes are typically used to store continuous gradation information such as pictures and / or smoothly changing background colors. The selector plane usually holds a text image (binary) along with other edge information (eg, line art). The foreground plane usually retains the color of the corresponding text and / or line art. However, the MRC representation specifies only plains and their associated compression methods. It does not otherwise limit or execute the content of each of the planes. Each content of the plane can be properly defined by executing the MRC representation.

The MRC structure also considers a fourth plane, the Rendering Hint Plane, used to convey additional information about the content of the document. For example, a rendering hint plane can perform ICC (International Color Consortium) color hints that identify the best color matching method for different objects on the page.

The foreground and background planes are defined as two full color (L, a, b) or YCC planes. The selector plane is defined as the binary (1 bit depth) plane. One exemplary MRC representation specifies that the foreground and background are JPEG-compressed, and that the selector plane is ITU-G4 compressed (standard Group 4 facsimile compression). The rendering hint plane is considered to be arbitrary, but if it is used, a compression method similar to the Lempel-Ziv-Welch method can be used for the compression. In general, the foreground, background, selector, and rendering hint plane may all have different resolutions, and they do not have to maintain the original source input resolution.

The method of assembling a "segmented" MRC image back from its components (eg, plane) is to "pour" the foreground color over the background plane through the "mask" of the selector plane, thereby these. This is due to overwriting the content in front of the background plane at the position of. In other words, this assembly is accomplished by multiplexing pixel by pixel between foreground and background information based on the binary control signal of the selector plane. For example, if the selector value is 1, the foreground content is used, otherwise (ie, if the selector value = 0) the background content is used. The multiplexing operation is repeated pixel by pixel until all output pixels are defined.

The main advantage of the MRC representation of a document is that it provides an effective way to store, send, and manipulate large digital color documents. This method takes advantage of the characteristic of the human visual system that the ability to discern small color changes is significantly reduced in the presence of high contrast edges. The edge information is usually separated from the smoothly changing color information and coded in the selector plane (perhaps at a higher resolution than one selector sample per source pixel). Following careful separation, the various planes can be compressed independently using standard compression schemes (such as JPEG and G4) with good compression and high quality.

The segment module captures a full-color input image to be segmented, with three separate outputs for the three MRC planes: foreground plane FG, background plane BG, and selector plane Sel, as well as some additional signals. To generate. A block diagram of the segment module is shown in Figure 2. The segment module consists of two stages, the segmentation stage 24, followed by the MRC scale and tile tag generation stage 44. The segmentation stage 24 can be operated by one of two mutually exclusive modes (scan segmentation 24 or PDL segmentation 26).

The primary input for the scan segmentation module 24 is Src20. It further utilizes the 8-bit screen magnitude estimation signal SCM22 detailed in Applicant's pending US patent application D / A3011. The scan segmentation module 24 provides full-color (raw or unprocessed) foreground plane output Fgr30 and background plane output Bgr28, and Sel output 32 exported by module 24 (which may have been supersampled by module 24). Output.

In PDL mode 40, the PDL segmentation module 26 does not use SCM22, but instead can use the signal Rht34, which captures hint information from the PDL interpreter, and is encoded in the CEF hint plane Hnt38. The output from the PDL segmentation module 26 includes a full-color (raw or unprocessed) foreground plane Fgr30 and background plane Bgr28, a binary selector plane Sel32, and a hint plane Hnt38 if there is a hint. As shown above, the hint plane can be 1 bit or 8 bits deep.

The Src images for scan mode 42 and PDL mode 40 are typically processed differently. The scan segment module 24 relies on descreened inputs. This is not required for clean, noise-free images generated directly from the PDL source.

In the scanning process, it is assumed that the chroma component of the source input image Src20 is subsampled with a factor of 2 in the fast scanning direction (x: XCSS). No color image generated by the segmenter uses XCSS. No "adjustment" filtering action is required when the chroma sample of the source image is accessed. That is, four sample XCSS quadrants, L₀A₀L₁B₁And pixel 0 is L₀A₀B₁And pixel 1 is L₁A₀B₁,.

The output of the selector plane Sel is binary (1 bit depth), and the packed selector plane Spk packs 2x2 binary adjacent selector pixels (4 bits) into each other.

In PDL processing, the source input Src20 is assumed to be a full-color image, where the chroma channel is typically not subsampled and therefore has the same resolution as the luminance channel.

In general, the exported foreground, background, and selector plane may all have different resolutions for the input image. For example, the foreground and background planes may typically be downsampled and the selector plane may be upsampled from the original input resolution. The amount of upsampling or downsampling is programmable under software control.

The MRC scale and tile tag generation module 44 reads the first (raw: unprocessed) background Bgr28, foreground Fgr30, selector Sel32, and optionally the hint Hnt38 if present (PDL mode only). This produces a final color MRC layer, namely background Bgd46 and foreground Fgd, by subsampling and painting unspecified pixels in a "hole" or raw (unprocessed) image. In addition, the MRC scale and tile tag generation module 44 generates four relevant tile tag signals for background Ttb50, foreground Ttf52, selector Tts53, and optionally (PDL mode only) for rendering hint Tth54. The tile tag is one binary bit per tile (or strip) and indicates whether the entire current tile can be omitted. This also reduces the overall file size. Missing tiles are automatically filled with the default default color for each plane.

The scan segmentation module 24 is responsible for MRC segmenting the scanned document into three planes. The inputs to the scan segmentation module 24 include an input color signal Src20 and an 8-bit screen magnitude estimation signal SCM22. The scan segmentation module 24 outputs a full-color (raw: unprocessed) foreground plane Fgr28 and background plane Bgr30, and a selector Sel32 plane.

A block diagram of the scan segmentation module 24 is shown in FIG. The following is a brief description of the various modules that make up the scan segmentation module 24. The color input signal Src20 finds the minimum Min and maximum Max color values for dynamic thresholding in a 7x7 window centered on the current pixel of interest. Transferred to maximum module 60.

The minimum Min61 and maximum Max63 values are transferred to the dynamic threshold module 62 and the scanning MRC module 64. The dynamic threshold module 62 also uses the input color image Src20 and the 8-bit screen magnitude estimation signal SCM22. The dynamic threshold module 62 outputs a monochrome 8-bit signal Grr55 whose biased zero intersection represents the position of the edge in the selector plane. Also, the dynamic threshold module 62 is further used to communicate with the scanning MRC separation module 64 on a pixel-by-pixel basis when segmentation is applied, in which case how much additional enhancement is added. Generates an 8-bit segment augmentation control Enh59 that communicates whether it applies.

The purpose of block smoothing unit 56 is to move weak (also known as "false") edges away from strong edges to create contrast within the foreground and background JPEG Minimal Coding Unit (MCU) blocks. Is to prevent the high transition of. If there are no strong edges nearby, the weak edges are extruded from the JPEG block to the boundary between nearby blocks. This method eliminates unnecessary abrupt transitions within the JPEG block, thus increasing compression and overall quality. The output from the block smoothing unit 56 is an 8-bit smoothed Grs57 signal that represents the smoothed (filtered) form of the incoming signal Grr55.

The foreground erosion unit 200 is used to address the requirements of thin (but unbroken) text using linear YCC segmentation. A constant value is subtracted from the gray selector, which thins / erodes the foreground. This is only done if nearby tests demonstrate that thinning does not result in a dashed line, as described in more detail below.

Binary scale unit 66 provides the ability to supersample the smoothed gray selector signal Grs57 from the output of block smoothing 56. In normal 1: 1 mode, the Grs57 signal is thresholded to produce the binary selector plane output Sel32. However, for high quality text and line art reproduction, the selector plane may be supersampled at twice the input resolution (eg at 1200dpi for a 600dpi input). Supersample processing of the selector signal is performed by doubling the sample processing frequency prior to thresholding. The resulting higher resolution selector pixel is packed into the packed selector signal Spk122, with four adjacent ones packed at once.

The Mark Edge Processing Module 58 captures the packed high resolution selector output Spk122 to determine the number of on and off pixels in a 5x5 (high resolution) window centered on the current (low resolution) pixels of interest. Count. The output from the mark edge processing module 58 is a 2-bit signal See142. See signal 142 is set to 0 if all input pixels inside the 5x5 window are off (corresponding to a constant 3x3 background area). Similarly, See signal 142 is set to 3 if all input pixels inside the window are on (corresponding to a constant 3x3 foreground area). In addition, the See output is set to 1 or 2 if the 5x5 window is mostly in the background or mostly in the foreground, respectively.

Finally, the scanning MRC separation module 64 captures the full color source signal Src20 to be segmented, as well as the minimum and maximum colors (Min, Max) from the dependent minimum and maximum modules 60. Further, the MRC separation module 24 uses the See signal 142 from the mark edge processing module 58 and the segmentation and augmentation signal Enh 59 from the dynamic threshold module 62. The MRC separation module 64 actually produces two full-color outputs Fgr24 and Bgr30 as rough estimates for the foreground and background planes, respectively. Here, various modules of the scan segmentation module are described below.

A block diagram of the minimum and maximum dependent modules is shown in Figure 4. Dependent minimum and maximum module 60 inputs the Src signal 20 and examines a 7x7 window centered on the pixel 80 of interest to find the maximum L and minimum L pixels, where L is the luminance channel. .. The maximum output 68 is a pixel with a maximum L72. The minimum output 70 is a pixel with a minimum L74. The minimum output 70 is a pixel with a minimum L74. The resulting chroma value therefore depends on where the extremum was found.

The operation of the dependent minimum and maximum module 60 is shown in Figure 5. This operation proceeds in two stages. In the first stage, the dependent minimum and maximum module 60 searches the window for the largest 68 and the smallest 70 samples of the luminance component L. When the positions of the minimum brightness value 74 and the maximum brightness value 72 are found, they are output together with the chroma components (A, B) at these positions. This is where the X chroma subsample processing is interrupted even if the Src signal 20 reaches this module with the X subsampled chroma component. That is, the maximum and minimum color signals do not have X subsampled chrominance.

This filtering step is separable. For example, the final minimum 74 can be calculated by first calculating the minimum / maximum of each column and then finding the column minimum pixel with the minimum L. This means that the step-by-step work required for a window to be performed across Src images is to calculate one height 7 column and one width 7 row for both minimum and maximum output. become.

Referring to FIG. 6, the dynamic threshold module 62 applies adaptive thresholding to the incoming color source signal Src20 and a raw signed 8-bit gray selector where the zero intersection represents the transition of the selector plane. Generate signal Grr output 114. A gray selector value 0 marks a pixel with a selector value of 1 and is placed in the foreground. A gray selector value &lt;0 marks a pixel that is placed in the background. As shown in FIG. 6, the dynamic threshold module 60 is a pair of minimum / maximum dependencies (Min, Max) 90 and 92 from the minimum / maximum dependency module 60, respectively, and 8 from the screen estimation module (SEM). Utilizes the screen magnitude estimation signal Scm22 of the bit. The dynamic threshold module 62 also produces an 8-bit signal Enh118. The Enh signal 118 is communicated to the scanning MRC separation module 64 to determine how much enhancement is applied when the pixel is placed in the background and / or foreground plane.

The dynamic threshold module 62 operates with three segmentation modules: dynamic threshold, static threshold, and force on the foreground. Static thresholding is applied when the image is smooth (unchanged). In this mode, pixels with a brightness value greater than or equal to DefautThr76 are assigned to the background (Grr == 127 = -1), and pixels with a brightness value less than DefaultThr are assigned to the foreground (Gr =). = 129 = + 1). 127 and 129 (+ -1) represent small magnitude values for Grr114. These represent weak decisions that can be modified by subsequent block smoothing modules 56 by taking into account the position and polarity of nearby strong decisions. The strong determination is represented by a signed Grr magnitude&gt; 1 (encoded value &lt;127, or&gt; 129). Strong decisions are only generated in dynamic threshold mode, and only strong decisions can have a nonzero Enh code. Both static thresholding and the mode of force against the foreground produce only weak decisions.

In some configurations, the force mode for the foreground is enabled for halftone images, which mode is enabled by setting the HTFGScmThr84 to a value less than 256. Whenever Scm22 is equal to or greater than HTFGScmThr84, Grr114 is forced to have a minimum foreground value of 129 (= ± 1) and Enh118 is set to zero.

If force on the foreground is not prioritized, segmentation actively switches between generating weak static threshold decisions and generating strong dynamic threshold decisions. The signal EnhEn shows a strong decision and gates the output of the EhFVsScm function to Enh. The EhFVsScm function uses the screen magnitude estimated Scm as the domain value. If true, EnhEn also selects the signed 8-bit signal GSel as the source for the coding of Grr. GSel, described in more detail below, is the primary output of the dot product module. As shown in Figure 6, if the force on the foreground (HTFGEn) is not prioritized, the other two outputs of the dot product unit, Ccc and Cc0 (described below), are tested and the results are ORed to each other. Then EnhEn will be calculated.

EnhEn is enabled if the output is from the Ccc EhClrConThrVsMin function. Alternatively, EnhEn is enabled when Cc0 EhLumConThrVsMax and EhLumConThrVsScm functions are maximal. When the input signals for the EhClrConThrVsMin function and the EhLumConThrVsMax function are the luminance components of the Min and Max signals, respectively, the input signal for the EhLumConThrVsScm function is Scm.

The dot product unit 82 (second block from the upper left in FIG. 6) uses the full-color input signal Src20 and the full-color minimum 92 and maximum 90 (Min, Max) from the dependent minimum and maximum units. These values represent the extrema of brightness and the corresponding chroma values found in a centered (7x7) window on the current pixel of interest. The operation of this block is mainly to perform dot product multiplication of two vectors.

[Number 1]

GSel = minimum (127, maximum (-127 (&lt;X, Y &gt;&gt; / 256))) where &lt;X, Y&gt; is a dot product operation between two vectors X and Y [Equation 2]

&lt;X, Y&gt; = (X_L, X_A, X_B) (Y_L, Y_A, Y_B)¹= X_LY_L+ X_AY_A+ X_BY_BHere, X is as shown in Equation 3 below.<maths num="3"><img file="JP2005294877A_D0001.tif" /></maths>And Y are as shown in Equation 4 below.<maths num="4"><img file="JP2005294877A_D0002.tif" /></maths>

As an embodiment for further improvement, L_MNWhen == 0, the luminance component value in Equation 4 is L._MX/ 2 to L_MXIt changes to / 4. This is a primary attempt to adjust the brightness undershoot, which is typically produced by the previous sharpening stage. This helps prevent thin text features from being expanded by the segmentation process. That is, it is expressed in the following formula 5.<maths num="5"><img file="JP2005294877A_D0003.tif" /></maths>

The (L, A, B) values in Equation 4 or 5 are the corresponding color components of the incoming signal Src20. The X vector in Equation 3 is the vector difference between the maximum and minimum of (Min, Max). The Y vector in Equation 4 is the incoming signal Src20 minus the minimum and maximum averages. By taking the dot product of these two vectors, the output is perpendicular to the X vector and is proportional to its relative distance from the plane that intersects it halfway. {X_A, X_B, Y_L, Y_A, Y_B} And the final output G Cel can be negative. The absolute magnitude of the dot product output in Equation 1 is not as important as identifying zero intersections, so this result is simply divided by 256 (shifting to the right by 8) so that it fits in the 8-bit range. Converted back to. (Dot product normalization requires dividing by the magnitude of the vector). However, the output can often still overwhelm the 8-bit range (a factor of approximately 3 or up to 1.5 bits only), so if this becomes too large, the logic that limits the output magnitude to 127. Will need to be added. The dot product 82 output is shown in FIG. 6 as a signed 8-bit signal GSel or gray selector output. In order to limit the size of the dot product multiplier to 8 bits, both the X and Y components can be pre-converted by 1/2 and the final divisor can be changed to 64.

The dot product unit 82 also outputs two 8-bit signals that measure luminance and chroma contrast magnitude. The luminance portion Cc0106 is represented by the first component of the vector X.

[Number 6]

Cc0 = X_L= L_maximum-L_minimum

A scalar scale for the overall chroma contrast magnitude Ccc104 is further generated by adding the absolute values of the two chroma components of the vector X to each other.<maths num="7"><img file="JP2005294877A_D0004.tif" /></maths>The absolute value in the luminance component can be ignored because L is limited to the positive range [0 ... 255], with the maximum always greater than the minimum.

The decision logic function 91 in the left part of FIG. 6 manages the switch between the default and the active segmentation mode. Each function is represented by a small set of pairs of (x, y) points that represent piecewise linear functions. For x-values smaller than the first x-value, the output is the first y-value. For x value&gt; last x value, the output is the last y value.

The importance of the above logic is that either the dot product luminance contrast or the dot product chroma contrast must be large enough to operate under the active segmentation mode. The chroma contrast must be greater than the function of minimum brightness found in the (7x7) window. Similarly, the luminance contrast must be greater than the function of maximum luminance found in the same window, and even more than the function of screen magnitude Scm. Prior to export, the output of the signed gray selector gated by HTF GEn and ENhEn is encoded as an unsigned 8-bit signal Grr by adding 128.

The purpose of the block smoothing unit 56 is to move the weak (also known as "false") edge away from the strong edge as much as possible. This will increase the pulse width (decrease the frequency) in the JPEG plane, which will reduce the amount of compression and compression noise. If there is no strong edge to block it, the weak edge is completely swept out of the block. If a weak edge is trapped between two strong edges, it will be rearranged in the middle of the two. This method also coalesces / removes a large number of weak edge transitions that improve compression in all three planes.

The input to the block smoothing unit 56 is an 8-bit raw gray selector output Gr. From the thresholding process (under either active or default segmentation). The output from the block smoothing unit 56 is an 8-bit smoothed glass signal that represents the smoothed (filtered) form of the input signal Gr.

The very first step in the block smoothing unit 56 is to bias 128 (toggle Ms.) to make Gr.'A signed number. Next, the range of (Gr.-128) is investigated. If this is equal to -1 or 1, it is considered to be a weak edge BG or FG, respectively. Anything greater than -1 or 1 is considered to be either a strong BG or FG edge.

The block smoothing process involves four paths in a square temporary storage area (Tm [Sm] [sz]) that represents the size of the JPEG minimum coding unit (MCU) for the Bgd and Fgd planes, eg, Bgd and. If the Fgd is subsampled by a factor of 4 for JPEC compression, the MCU will be 16x16 if the chrominance and components are additionally subsampled by a factor of 2. For this block smoothing algorithm The required Tmp block is sized to Tmp [64] [64]. Implementation constraints require a smaller block size, in which case 32x32 would be suitable. Currently The pipeline configuration of is only required 32x32 blocks because no chroma subsampling is used when applying 1/4 reduction.

In the first pass (Grr-128) pixels are processed from left to right in each row in a separate Tmp. Whenever there is a strong edge, the corresponding Tmp position is initialized to +/- K (predefined constant == 2 * Tmp-1 dimension). The sign is the same as Grr-128. Alternatively, for weak edges, Tmp is written with the Tmp value before the magnitude was decremented by 1. The previous Tmp value is just the Tmp value on the left, as the first path is moving from left to right. The value before the first value in the row is defined to be 0. Decreasing the magnitude by 1 converts +4 to +3, -2 to -1, and 0 to 0. In addition, in this path, all weak values, +1 and -1 accumulations (WeakCnt) are calculated (only where Grr is weak).

In the second pass, each row of Tmp turns from right to left, this time comparing the value before the magnitude reduction with the current value. The value with the largest magnitude is placed in its current position. Again, the previous value for the first value on the right is assumed to be 0. Since the lines of paths 1 and 2 are independent, path 2 can be alternated with path 1. Paths 3 and 4 are the same as path 2 except that the directions are top-to-bottom and bottom-to-top.

After the fourth pass, the final smoothed result is produced by examining the Tmp. If the Tmp value has a maximum magnitude (+ K or -K), the value used is the value of the original strong edge from Grr. Alternatively, a code for weak foreground or background (128 + 1 or 128-1) is used depending on whether the Tmp value is positive or negative.

Referring to FIG. 7, the binary scale module 66 converts the input Grs120 of the 8-bit grayscale selector to the output Sel32 of the binary selector plane. For high quality text and line art, the binary selector output Sel32 can be maintained at a higher resolution than Src20. For example, the current design allows you to use twice the resolution in each direction (SEG_Sel_ScaleUp), so for a standard 600dpi scanner, the output resolution of the binary Sel signal can be 1200dpi. .. The selector logic module is responsible for interpolating the gray selector Grs input to a higher resolution and then using the signal as a threshold to generate the binary output Sel32. A packed copy of redundant Sel (Spk) is also generated at Src20 resolution.

A block diagram of binary scale module 66 is shown in Figure 8. The 8-bit gray selector input signal Grs120 is first interpolated in the gray domain with a factor of 2 in each direction using 2D bilinear interpolation 124. The output from the bilinear interpolation is passed through the multiplier 126 to choose whether to use the interpolated (supersampled) selector or the regular binary one with the same resolution as Src. Finally, the gray output is converted to binary using the threshold unit 128 to generate the selector signal Sel32.

Note that for the 2x interpolation factor, the binary Sel output data transfer rate is twice as fast as the source signal Src in each direction. This means that for every 8-bit input Grs, the binary scale module produces four binary output Sel samples. The second packed form (Spk) 122 of the Sel output is generated so that the selector bits of the four binaries are packed together, as shown in FIG. It is important to note that the selector output 140 uses vector notation to indicate higher output resolution. The output is still considered binary (ie, assuming the value is only either 0 or 1), but each incoming signal Grs input produces 4 selector bits at the output (2x). Interpolation coefficient is assumed). The four binary pixels are packed into an 8-bit packed signal Spk122, as shown above. If the interpolation factor is only 1, all 4 bits are the same.

Referring to FIG. 10, the mark edge module 58 takes the form of a packed high resolution selector Spk122 and is on and on in a 5x5 High Resolution window 155 centered on the current pixel 80 of interest. Count the number of off pixels. The output from the mark edge module 58 is the quadrature signal See142. If all input pixels inside the window are 0 (corresponding to a certain background area), the See signal 142 is set to 0. Similarly, the See signal 142 is set to 3 if all input pixels inside the window are on (corresponding to a certain foreground area). Further, if the contents of the window are mostly back ground or mostly foreground, the See output 142 is set to either 1 or 2. See has only 4 values and can be coded in 2 bits.

The operation of the mark edge module 58 is shown in FIG. The operation of the unit is as follows. The input to the edge processing module 58 is a packed binary selector signal Spk with the same resolution as Src. Edge processing module 58 maintains a 3x3 pixel context window centered on the current pixel of interest (at the original input resolution). Logically, the packed selector (Spk) selector contains four binary selector pixels for each Src resolution pixel in a 3x3 pixel window, as shown in Figure 11. The thicker lines represent the original Src resolution corresponding to the 6x6 pixel context window in the high resolution domain. However, only the content inside the 5x5 high resolution pixel area is used, and the shaded area in Figure 11 is excluded in the count.

The 5x5 high resolution context is designed to "detect" potential edges close to the current pixel of interest. The window pattern uses the full context of two [high resolution] pixels that extend from the top of the current pixel or just one down to the left and to the right. The unique window pattern prevents any edges associated with a large number of (low resolution) pixels from overlapping nearby pixels, that is, the position of any potential edge cannot be detected (ie shared) more than once. Will be done. Figure 11 further shows the possible edge positions of 4x4 = 16 in the current window of interest.

The Mark Edge Module 58 counts the number of high resolution pixels currently turned on in the 5x5 high resolution area. This number can be from 0 to 25. This is mapped from the mark edge module 58 to the output signal See as follows. See = 0 5 × 5 If the count is 0 (no foreground pixels were found) See = 1 If the count is in the range [1 ... 12] (mostly background pixels) See = 2 If the count is in the range [13 ... 24] (mostly foreground pixels) See = 3 If the count is 25 (only foreground pixels are found)

Seeing FIG. 3 again, the output signal See is transferred to the scanning MRC separation module 64. The See signal is at the original input resolution (typically 600dpi). The scanning MRC separation module 64 is responsible for dividing the incoming source signal Src into foreground and background planes. This module uses the full color minimum and maximum (Min, Max) outputs from the dependent minimum and maximum modules and the marked selector edge count signal See from the marked edge module. In addition, the scanning MRC separation module has the ability to enhance the edge rise via the segment augmentation control signal Enh from the dynamic threshold module.

The scanning MRC separation module 64 outputs two full-color raw: unprocessed estimates for each of the foreground output Fgr30 and the background output Bgr28. The tracking module, MRC scale, and tile tag generation module then further processes Fgr and Bfr to produce the final foreground Fgd and background Bgd output, respectively.

The scanning MRC separation module 64 takes in the full-color source signal Src to be segmented and produces values for one or sometimes both of the Fgr and Bgr outputs. The scanning MRC separation module has a special code of zero brightness and chroma (L = a = b = 0) to indicate empty (unsolicited) pixels in either the foreground Fgr or background Bgr output. .. As this process continues across pages, some foreground and background pixels remain unsolicited. The MRC scale and tile tag generation modules are then carefully painted with values for these unsolicited pixels to keep compression low and prevent additional JPEG ringing artifacts.

The scanning MRC separation module 64 uses the value of the selector edge count signal See from the mark edge module to determine whether to copy the enhanced Src pixels to the background, foreground, or both. The determination range is shown in FIG. Basically, the enhanced Src pixels are<maths num="8"><img file="JP2005294877A_D0005.tif" /></maths>If it is, it will be copied to the foreground and<maths num="9"><img file="JP2005294877A_D0006.tif" /></maths>If is, it is copied to the background. Therefore, if See == 0, the foreground is marked as undefined, and if See == 3, the background is marked as undefined.

First, the augmentation factor Enhf1 is the signal Enh incremented by exactly 1, so the maximum is 256 instead of 255.

[Number 12]

Enhf = Enh [Number 13]

Enhf1 = Enhf + 1

Next, we define two full-color enhanced forms of foreground and background (the purpose of which is described below).

[Number 14]

enhFG = LIM [Src + (Min-Src) (Enhf1 / 256)]

[Number 15]

enhBG = LIM [Src + (Max-Src) (Enhf1 / 256)]

Implementation memo. If the final Bgd and Fgd outputs are XCss (subsampled by X chroma) or scaled down, enhFG and enhBG may be XCSS.

In equations 14 and 15, Src is the full-color input signal, and Min and Max are the minimum and maximum dependent color outputs from the minimum and maximum dependent modules. The restriction function LIM allows the result for each of the components to be limited to an 8-bit range [1 ... 255], which is specially provided to mark undefined pixels. Zero code will be eliminated. Since Src and Min and Max are full color (L, A, B) vectors, the operation is in 3D space.

If the foreground is defined, that is, if See = {1, 2, or 3}, then the output Fgr value is [Equation 16].

If SEE = {1, 2, 3}, then Fgr = enhFG [number 17]

If SEE = 0, then Fgr = 0 is required.

If no foreground is used (ie See = 0), the foreground value is undefined by setting its value to the special code Fgr = 0 according to Equation 16 (for all three components). Marked as. Note: This practice extends Enhf to a 9-bit representation and its value is incremented by 1 to allow normalization by 256 instead of 255 (Enhf1).

A rigorous check of Equation 14 is that the amount of segment augmentation represented by Enhf1 interpolates the output foreground Fgr value between the current input signal value Src and the minimum Min of the dependent minimum maximum module (in 3D space). Make it clear. When Enhf = 0, no enhancement is performed and the output is set to the input signal Fgr = Src. This is a normal example unless there is sufficient contrast activity in the (7x7) window. If Enhf1 = 256 (maximum enhancement), the output is set to the minimum signal Fgr = Min. This represents an example of a pixel in the immediate vicinity of an edge, where it is advantageous to enhance the edge by painting the foreground as dark as possible (0 = black) as given by the nearest minimum. .. However, in general, segment augmentation Enhf can vary between the two extrema above, and the output foreground value is correspondingly weighted between the Src and Min values.

Similarly, when using background in segmentation, ie when See = {0, 1, 2}, the output Bgr value is [Equation 18].

If See = {0, 1, 2}, then Bgr = enhBG [number 19]

When See = 3, it is calculated by Bgr = 0.

As before, the output Bgr value varies between the input Src and Max values in proportion to the segment augmentation amount Enhf1, as given by Equation 15. Equation 18 is similar to Equation 16 except that the maximum Max is used instead of the minimum Min and the See range is different. Using Max for Bgr output makes it brighter instead of darker, as it is with the foreground.

Further, as shown by Equation 19, in correspondence with Equation 17, if neither the background nor the foreground is used (ie, See = 3), then each or the background value of (see = 0) is its. Marked as undefined by setting the value to the special code Bgr = 0 (for all three components).

The output from the MRC separation module is two partially painted full color plain Fgr and Bgr. Far from the edge of the selector plane, depending on whether it is bright or dark, typically only one of the foreground or background outputs contains the [enhanced] color of the current pixel. However, nearby edge information may be supported simultaneously by both foreground and background channels.

Referring to FIG. 2, the PDL segmentation module 26 is responsible for performing MRC segmentation on the plane in the three PDL documents. The input to the MRC segmentation module 26 of the PDL can include the input color signal Src20 and any rendering hint Rht34 that can be supplied by the PDL decomposer.

PDL's MRC segmentation module 26 outputs full-color foreground and background planes Fgr28 and Bgr30, binary selector plane Sel32, and stores some PDL hints, perhaps in the 8-bit hint plane Hnt.

A block diagram of the PDL MRC segmentation module 25 is shown in Figure 13. Starting from the left, the PDL segmenter reads the 8-bit rendering hint Rht from the input color signal Src20 and the PDL interpreter. The PDL segmenter 26 produces an 8-bit gray selector signal Grr similar in function to that used by the scanning process. In addition, the PDL segmenter outputs some PDL hints as MRC hints Hnt.

The gray selector signal Grr from the PDL's MRC segmentation module 25 is processed through the block smoothing unit 56 to generate a smoothed gray selector signal Grs that is transferred to the binary scale unit 66. The binary scale unit 66 thresholds the Grs signal to generate the binary selector signal Sel. The quality of the PDL data is not improved by supersample processing, so the selectors generated by the binary scale unit are always in Src resolution. The operation of the block smoothing unit and the binary scale unit will be described above.

Finally, the PDL's MRC separation module 25 is responsible for splitting the incoming source signal Src20 into the foreground and background planes Fgr30 and Bgr28, respectively. This separation is based on the binary selector plane Sel32.

The PDL segmentation module 26 is responsible for taking in the input color signal Src20 and generating the 8-bit gray selector signal Grr66. In addition, PDL segmentation module 26 stores some 8-bit PDL interpreter hint Rht as 8-bit PRC hints on the hint plane.

The operation of the PDL segmentation module is different from the scanning process described above. The scan segmentation process is based on dependent minimum / maximum analysis, followed by dynamic thresholding. However, for clean PDL data, segmentation is based on the classification of pixel content in a 3x3 window centered on the current pixel of interest. This classification takes precedence as a set of rules that determine whether the current pixel is associated with a foreground Fgr or a background Bgr plane.

For each incoming Src pixel, the content of the 3x3 window around this pixel is analyzed and classified into one or more subsequent class 158 shown in the table in Figure 14. The 3x3 window tests are prioritized as shown in the leftmost column in Figure 14. Thus, for example, a center pixel tagged to be an image pixel by the PDL interpreter takes precedence over any other combination, such as a center pixel tagged as black, white, or text.

The second column in the table in Figure 14 lists the class names in the C code simulation. The third column gives a brief description of the meaning of the class and how it is tested. Finally, the last column shows how the class is associated (ie, segmented) with the foreground or background plane. One exception to the 3x3 window exam is for classes 6 and 7. The methods for these classes are as follows.

1. First, any outer pixel that is neither NEAR to the center pixel nor FAR from the center pixel is classified as a BAD pixel. Here, the meaning of NEAR and FAR is Manhattan distance D._MIt is based on. That is, (D_MNEAR if &lt;PDLEqualDistLim)<maths num="20"><img file="JP2005294877A_D0007.tif" /></maths>Here, PDLEqualDistLim and PDLOtlDistThr are the thresholds of the two configurations. 2. The outer pixel that is not NEAR to the center pixel encountered and is not the first Bad that is FAR from the center pixel is the basis for the OTHER class. 3. The next encountered center pixel is not NEAR, and the pixel (which is not FAR from the center pixel or is not NEAR to the OTHER reference pixel above) is classified as a Bad pixel. 4. Finally, OTHER classes 6 and 7 depend on non-Bad pixels in the windows they encounter. Class 6 (Oth Dark) or 7 (Oth Lite) is differentiated based on the color value of the reference OTHER pixel.

The PDL separation module is responsible for dividing the incoming source signal Src into foreground and background planes Fgr and Bgr, respectively. This separation is based on the binary selector plane Sel. The separation process is initiated by initializing the foreground and background planes with a specially provided zero code (L = a = b = 0) to indicate unused pixels.

Then, the incoming color Src values, by confusion with "not used" specially a provided code is prevented sea urchin, is moved away from zero. The Val = Max (1, Src) max function guarantees that Val will never be zero in any of its planes. The separation process continues directly. If (Sel = 1), Fgd = Val or Bgd = Val

That is, each incoming color pixel is placed either in the foreground or in the background. Unlike in the case of scanned documents, this information is not placed on both planes, even near the edges. Therefore, the separation method is greatly simplified compared to the case of scanning.

Referring to FIG. 15, the MRC scale and tile tag generation module will apply additional processing to the coarse foreground and background estimates Fgr and Bgr to generate the final foreground and background outputs Fgd and Bgd. To do. The processing performed by the MRC scale and tile tag generation module is to first subsample the foreground and background values, ignoring undefined pixels. We then subsample this result with a factor of 8 to calculate the block mean (again ignoring undefined pixels). The third step is to insert the calculated block average into undefined pixels. The purpose is to reduce ringing artifacts in JPEG compression by painting undefined pixels with block averaging.

Additional logic inside the MRC scale and tile tag generation module also monitors foie gras and background output values to detect and flag certain black or white tiles. Similar logic is detected when the selector and hint are zero. A block diagram of the MRC scale and tile tag generation module is shown in Figure 17.

All four examples of subsample processing modules work in a similar manner. The sum of all pixels in the N × N area is calculated and a separate count of the number of pixels with valid = notZero is maintained. This sum is then normalized by the count of valid pixels to produce the output. The first stage of subsampling typically spans a 4x4 area that represents the overall extent of subsampling of Fgd and Bgd. The amount of subsample processing is specified by the parameters: Seg_Fdg_ScaleDn: SEG_Bgd_ScaleDn: SEG_Fgd_Dst_Css: and SEG_Bgd_Dst_Css.

The Css parameter controls whether the chroma sample was additionally subsampled by a factor of 2. The second stage always subsamples over an 8x8 area that represents the size of the JPEG block at the subsampled resolution. The final normalization of the subsampled output depends on the total weight value. However, it is still possible to avoid division in the formula by using a given multiplication table with a large number of choices for possible total weight values.

The fill-in-the-blank block inserts block mean Fga and Bga into Fgx and Bgx to replace all undefined pixels and generate the final foreground Fgd and background Bgd signals. The fill-in-the-blank block also produces a very low bandwidth output Tgb / Tgb of 1 bit per tile or strip that can be used to optimize compression when the CEF file is exported. Each fill-in-the-blank block monitors each pixel in the tile to test whether all pixels are within the limits set for the brightness and chrominance samples. If all tile pixels pass the test, the tile tag bits are set. Each pixel is subjected to three tests. LumRef-L &lt;= TileLumErr {where LumRef is 255 for Bgd and 0 for Fgd} abs (128-A) &lt;= TileChrmErrabs (128-B) &lt;= TileChrmErrSel and Hnt tiletag modules are just the equivalent of large NOR gates operating in one tile block. These produce 1 if all the binary pixels in the tile are 0. The tile size is programmable, but typical values vary from 64x64 to 512x512 pixels.

Seeing Figure 3 again, the foreground erosion unit 200 is used to address the requirements of thin (but unbroken) Kanji using linear YCC segmentation. A constant value is subtracted from the gray selector, which thins / erodes the foreground. If the pixels are converted from foreground to background, this is only done if nearby tests have demonstrated that thinning does not result in a dashed line. With reference to FIG. 16, the figure shows the operation of this module 200. Foreground Elotion Unit 200 attempts to fit some templates. If a fit is found, adjustments will be made. FIG. 16 shows two patterns. The shaded block 210 represents the background and the shaded block 214 represents a stronger foreground that is greater than the adjustment. The shaded block 216 represents a weak foreground that changes to the background when adjustments are drawn. Block 212 has not been identified. The weak foreground 216 can switch to background 210 only if one of the two patterns shown in FIG. 16 matches (each has four possible orientations).

<figref num="1">It is a figure which shows the MRC structure about a document.</figref><figref num="2">It is a block diagram of a segment module.</figref><figref num="3">It is a block diagram of a scanning segment module.</figref><figref num="4">It is a block diagram of the dependency minimum maximum module.</figref><figref num="5">It is a figure which shows the operation of the dependency minimum maximum module.</figref><figref num="6">It is a block diagram of a dynamic threshold module.</figref><figref num="7">It is a block diagram of a binary scale module.</figref><figref num="8">It is a figure which shows the operation of a binary scale module.</figref><figref num="9">It is a figure which shows the format of a packed selector.</figref><figref num="10">It is a block diagram of a mark edge module.</figref><figref num="11">It is a figure which shows the operation of the mark edge module.</figref><figref num="12">It is a figure which shows the decision range which defines the background and the foreground.</figref><figref num="13">It is a block diagram of the MRC segmentation module of PDL.</figref><figref num="14">It is a table which shows the class of a PDL segmentation module.</figref><figref num="15">It is a block diagram of the FG / BG cleanup module.</figref><figref num="16">It is a block diagram which shows the erosion of the foreground.</figref>