Electronic pan tilt zoom video camera with adaptive edge sharpening filter

FIELD OF THE INVENTION

[0001] This invention relates generally to digital image processing and more particularly to a video camera with electronic pan, tilt and zoom and an adaptive edge sharpening filter.

BACKGROUND OF THE INVENTION

[0002] Video cameras that utilize exclusively solid state components rather than a mechanical movement of lenses and other optical components to pan, tilt, and zoom have numerous economic and functional advantages. An electronic pan, tilt, zoom (“EPTZ”) camera is small, fast, inexpensive, quiet, reliable, durable, easy to manufacture and easy to upgrade. Cameras with electronic EPTZ features can be used in, for example, cameras incorporated in set-top boxes for video conferencing, in surveillance cameras, camcorders and digital still cameras, among other applications.

[0003] A digital EPTZ camera typically has three main independent components: an imager section, a sharpening filter, and a scaling circuit or “scaler”. The imager section typically includes a lens and a solid-state imager chip, such as a charge-coupled device (CCD) or a metal-oxide-on-silicon (MOS) type chip. The imager chip includes a plurality of photosensitive areas (that is, picture elements or “pixels”) arranged as a two-dimensional array which is scanned either interlaced or progressively, as is known in the art. Medium resolution imager chips which perform satisfactorily in video set-top applications have, for example, 640 pixels horizontally and 480 pixels vertically. The imager section preferably includes an analog-to digital section to provide digital serial video information. Imager sections and imager chips of this type are well known in the art.

[0004] Conventional EPTZ cameras sharpen the image before the image is scaled by the scaler. Virtually all modern imaging systems utilize a standard Laplacian sharpening filter to reduce blurring that results from hardware limitations. The standard Laplacian filter is known in the art and applies a high-pass function to both horizontal and vertical image axes. High-pass filters, however, tend to amplify noise and introduce high-frequency artifacts into the image, such as undershoots and overshoots, which are perceived by the viewer as “halos”.

[0005] The scaling circuit electronically adapts or “scales” the number of input pixels to match a predetermined output format. For example, video conferencing signals are typically transmitted in the Common Interface Format (CIF) of 352 pixels horizontally and 288 pixels vertically. The imager section described above outputs 640 pixels horizontally and 480 pixels vertically. The scaler has to convert the image to CIF format by “down-sampling”, in this example by a factor of 1.83 horizontally and a factor of 1.67 vertically. The scaler generates the respective new horizontal and vertical CIF addresses and computes new video information, e.g., new luminance (Y) and chrominance (C) values associated with the new addresses. Consequently, the scaler can also be used to perform the pan/tilt and zoom functions in an EPTZ camera.

[0006] Conventional EPTZ cameras typically do not sharpen the images after scaling. In particular, zoomed-in images tend to appear blurred. The visual appearance of blurry images is most effectively enhanced by sharpening “edges” in the image. An edge can be defined as a linear arrangement of pixels where the luminance (Y) signal changes significantly in a direction perpendicular to the direction of the linear pixel arrangement.

[0007] It is therefore an object of the present invention to provide an adaptive edge sharpening apparatus and method which sharpens an image without significantly amplifying noise.

[0008] It is a further object of the present invention to adapt the parameters used in the sharpening algorithm to the scaling ratio of the image. It is still another object of the invention to prevent overshoot and undershoot of the sharpened pixel values.

SUMMARY OF THE INVENTION

[0009] The adaptive edge-sharpening apparatus of the invention includes a conventional electronic imager section with an image sensor having pixels; a scaler which converts the pixel addresses of the electronic image or a portion thereof, i.e., a zoomed image, and the luminance (Y) and chrominance (C) values associated with these pixel addresses to output values conforming to a predetermined output format, such as the CIF format; preferably an edge detector for determining the location and orientation of an edge in the output pixels; a sharpening filter or “sharpener” for sharpening the output pixels depending on characteristic properties of the edge; and a “clipping” device which clips the luminance (Y) values of the sharpened pixel so as to fall between the smallest unsharpened numerical value and the greatest unsharpened numerical value, respectively, of the pixels located within a neighborhood of the sharpened pixel. A neighborhood of a pixel includes one or more pixels adjacent to the pixel and may also include the pixel itself. The clipping device which is a central feature of the present invention, therefore eliminates objectionable undershoots and overshoots which are commonly associated with high-pass filters, as discussed above.

[0010] In an advantageous embodiment, the edge detector detects for each edge pixel an edge orientation. Depending on the specific edge filter used, the edge detector, for example, determines if an edge is oriented horizontally, vertically, or at an angle with respect to the horizontal. Preferably only those pixels located on a line perpendicular to the edge orientation are subsequently used to sharpen the edge pixels.

[0011] In another advantageous embodiment, for each image a numerical threshold value is provided for locating and selecting edge pixels. The threshold value preferably depends on the scaling ratio used in the scaler (in the present example the scaling ratio is between approximately 0.5 and 2.0), but can also depend on other camera settings, such the pan and tilt position (reflecting the diminished performance of most optical systems away from the optical axis) and possibly also on the lighting conditions (low lighting introduces more noise). For example, the threshold value is preferably relatively small for a large magnification and relatively large for a small magnification. Alternatively, the threshold value can also be provided by the user.

[0012] In still another advantageous embodiment, a sharpening parameter is provided for controlling the sharpening. Like the threshold value, the sharpening parameter can also depend on the scaling ratio and the camera settings.

[0013] In yet another advantageous embodiment, pixels which form a local maximum or minimum and which therefore are not located on an edge, are not sharpened. Local maxima or minima can either be determined separately before the pixels are sharpened and subsequently excluded from being sharpened which, however, requires an additional computational step. Alternatively, pixels representing local maxima or minima will be left unsharpened if they are counted as being part of the neighborhood of pixels, because no pixel is allowed to exceed or fall below the value of any unsharpened pixel in the neighborhood of that pixel.

[0014] In still another advantageous embodiment, pixels can also be sharpened without edge detection, using the same Laplacian sharpening algorithm, and subsequently clipped.

[0015] The invention furthermore includes a method for adaptively detecting and sharpening the edges in images and for clipping sharpened pixel values so that they fall between the smallest unsharpened numeric values and the greatest unsharpened numeric values, respectively, of the pixels located within the neighborhood of the sharpened pixels.

BRIEF DESCRIPTION OF THE DRAWINGS

P-0016[0016] Other objects, features, and advantages of this invention will be described in the following detailed description with reference to preferred embodiments and the appended drawings, in which:

P-0017[0017]FIG. 1 is an electrical block diagram of a video camera, a control computer, a scaler and a sharpening filter;

P-0018[0018]FIG. 2 is an electrical block diagram of the scaler of FIG. 1;

P-0019[0019]FIGS. 3<i>a </i>and <i>b </i>are electrical block diagrams of the control logic of the scaler of FIG. 1;

P-0020[0020]FIG. 4 is an electrical block diagram of the sharpening filter of FIG. 1;

P-0021[0021]FIG. 5 is a graph of a sequence of individual pixels, illustrating a typical change in the pixel values across a typical edge within an image;

P-0022[0022]FIG. 6 is a graph of the pixel sequence of FIG. 5 after sharpening with a traditional Laplacian filter;

P-0023[0023]FIG. 7 is a graph of the pixel sequence of FIG. 5 in which the sharpened pixels are also clipped;

P-0024[0024]FIG. 8 is a flow chart of a digital image sharpening method; and

P-0025[0025]FIG. 9 is a perspective view of a set-top box incorporating the video camera of FIG. 1.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

P-0026[0026] Referring first to FIGS. 1 and 9, there is illustrated a set-top box <b>10</b> which includes a camera <b>20</b> and incorporates the edge-adaptive sharpening filter (not shown in FIG. 9) of the invention. The camera <b>20</b> is preferably an EPTZ camera which provides electronic pan/tilt and zoom movement. The set-top box <b>10</b> can also incorporate an optical receiver <b>22</b> for receiving an external optical input, for example an IR signal from a remote control unit (not shown). The set-top box <b>10</b> can also be enabled to receive acoustical input through a built-in microphone <b>30</b>, which is preferably directional. Remote microphone(s) (not shown) can also be used. The camera <b>20</b> can then automatically electronically pan, tilt and zoom to capture an image of, for example, a speaker in response to signals received through the directional microphone <b>30</b> or through the respective remote microphone(s). The set-top box <b>10</b> is preferably placed on top of a video monitor (not shown), with a video output signal from the set-top box <b>10</b> connected to the monitor, for example, via a cable (not shown).

P-0027[0027] The video camera <b>20</b> can also be adapted for movement relative to the set-top box <b>10</b> using conventional mechanical actuators and/or motors (not shown). For example, the image could be coarsely panned and tilted by mechanically adjusting the orientation of the camera <b>20</b> relative to the set-top box <b>10</b>, while a fine adjustment is performed electronically.

P-0028[0028] In FIG. 1 there is illustrated in the form of a block diagram an embodiment of an EPTZ video camera <b>90</b> which includes a conventional imager section <b>100</b>, a scaler <b>200</b>, and a sharpening filter <b>300</b> which includes a sharpener <b>305</b> and an edge detector <b>320</b>, all of which are in communication with a controller <b>400</b>.

P-0029[0029] An imaging device <b>110</b>, e.g., a CCD or CMOS imager chip, generates analog imaging signals in response to optical radiation impinging on the imaging device <b>110</b>. The analog imaging signals are subsequently processed conventionally with a correlated sampler (CDS) <b>120</b> and an analog-to digital converter <b>124</b> and transmitted to a digital signal processor (DSP). The DSP <b>130</b> converts the received signals to standard luminance (Y) video signals and chrominance (C) video signals with, in the preferred embodiment, a resolution of 640×480 pixels, and also adjusts, if necessary, picture contrast, brightness (that is, exposure), color balance, and imager setup based on instructions received from controller <b>400</b>. A timing and drive pulse generator <b>140</b> sends cam clock pulses <b>142</b> and horizontal and vertical sync pulses <b>144</b> to an address generator <b>210</b> which forms a part of scaler <b>200</b>.

P-0030[0030] As illustrated in FIG. 1, the scaler <b>200</b> includes a Y (luminance) scaler <b>230</b> and a C (chrominance) scaler <b>220</b>. The address generator <b>210</b> has a horizontal address generator <b>210</b>h for generating H (horizontal) pixel addresses and vertical address generator <b>210</b>v for generating V (vertical) pixel addresses, as illustrated in detail in FIGS. 2, 3<i>a </i>and <b>3</b><i>b</i>. In the present example, the scaler <b>200</b> is adapted to provide at its output pixel addresses and corresponding Y and C values in the CIF format of 352 pixels horizontally by 288 pixels vertically. As mentioned above, the scaler <b>200</b> down-samples an input image which has more than 352×288 pixels, by outputting a smaller number of pixels than in the input image and conversely up-samples an input image which has less than 352×288 pixels. For zooming/panning/tilting, the scaler selects a rectangular subset of the array of input pixels, with the size of the subset and its position within the full image determined by user preference. The scaled Y video signal outputted from the Y scaler is controlled to have the desired size (zoom ratio) as well as the desired horizontal position (pan) and vertical tilt position (tilt). It will be appreciated that no panning or tilting is possible when the image is completely zoomed out.

P-0031[0031] Referring now to FIG. 2, there is illustrated the Y scaler <b>230</b> together with a horizontal address generator <b>210</b>h and a vertical address generator <b>210</b>v. The Y scaler <b>230</b> includes a horizontal filter <b>240</b> and a vertical filter <b>270</b>. In the illustrated example, the luminance Y video signal, which is preferably 8 bits wide, is successively delayed by four one-pixel delays <b>242</b>, <b>244</b>, <b>246</b>, and <b>248</b>. Each delay <b>242</b>, <b>244</b>, <b>246</b>, and <b>248</b> temporarily stores a portion of the Y signal representative of a single pixel so that at any given instant, delays <b>242</b>, <b>244</b>, <b>246</b>, and <b>248</b> store four adjacent pixels from a single horizontal row in the image. Each of the four delayed portions of the Y signal is then multiplied by respective coefficients H1, H2, H3 and H4 in respective multipliers <b>252</b>, <b>254</b>, <b>256</b>, and <b>258</b>. It should be noted that each delay is combined with a multiplier to form a delay/multiplier circuit commonly referred to as a “tap” and that the number of taps determines the quality of the filter. The design of the present invention utilizes a four-tap filter which, with properly chosen tap weights, closely approximates the desired sin(x)/x lowpass filter response.

P-0032[0032] The coefficients H1, H2, H3, and H4 are determined by the desired zoom ratio and by the phase of the output pixels relative to the input pixels. These coefficients are chosen to produce a lowpass finite impulse response (FIR) filter is known in the digital signal processing field. At a high zoom ratio (that is, telephoto), the coefficients will be approximately shaped like a sin(x)/x curve, or equivalently generated by cubic convolution, and will include side lobes with negative values. At a low zoom ratio (that is, wide angle), the coefficients will be approximately sin(x)/x but at a magnification so that their values are all taken from the center lobe of the sin(x)/x curve and will all be positive. Coefficients scaled to the current zoom ratio at all anticipated phases are precalculated and loaded in the horizontal look-up table (HLUT) at the beginning of the frame to minimize on-the-fly computation.

P-0033[0033] These coefficients H1, H2, H3 and H4 will also be adjusted according to the exact phase of the output sample relative to the input sample, hence they must be updated for each new output sample. The coefficient generation occurs in H address generator/HLUT <b>210</b>h shown in FIG. 2.

P-0034[0034] Additional details of the H address generator/HLUT <b>210</b>h and the V address generator/VLUT <b>210</b>v are illustrated in FIGS. 3<i>a </i>and <b>3</b><i>b</i>. The H-in-counter <b>212</b>h is preset for each horizontal line of pixels to a value representative of the first desired pixel for inclusion in the output image (that is, the left edge of the output image) responsive to the user's desired horizontal pan angle. As a way to count the inactive input pixels until the first output pixel is needed, the H-in-counter <b>212</b>h is loaded with a negative number with a magnitude equal to the number of input pixels which will be ignored before reaching the first pixel in the horizontal line to be included in the output image. Then the H-in-counter <b>212</b>h is advanced by each edge of the camera serial output clock Cam clk. When the H-in-counter reaches zero, the first output pixel is produced.

P-0035[0035] In FIG. 3<i>a</i>, H-out-counter <b>214</b>h representing the number of the desired output pixel is clocked at a faster rate, e.g., by 2×clk (from clock bus <b>142</b> of FIG. 1) to accommodate the zoom function. The output of the H-out-counter <b>214</b>h is multiplied by the desired zoom ratio, in this case a number between 0.5 and 2, in multiplier <b>215</b>h, and the multiplier output, representing the location of the next output pixel as referred to the input, is compared to the output of H-in-counter <b>212</b>h by subtracting both numbers in adder <b>216</b>h and using comparator <b>217</b>h to determine when the output of the H-in-counter <b>212</b>h is within an integer of the desired location. If the output of the H-in-counter <b>212</b>h is within an integer of the desired location, the feedback signal to the H-out-counter <b>214</b>h, which has kept the counter stalled, is released, an output pixel is generated and output to become part of the camera's output image and the H-out-counter <b>214</b>h is incremented.

P-0036[0036] When the zoom ratio is near maximum telephoto, two output pixels are needed for each input pixel. Here, the H-out-counter <b>214</b>h is incremented, but the output of multiplier <b>215</b>h only changes by a fractional amount, not enough to change its integer value, and as only the integer portion of the multiplier feeds the adder <b>216</b>h, the equality in comparator <b>217</b>h is still met and another output sample is taken. As the internal circuitry is clocked at twice the image sensor clock, one extra pixel can be generated. For higher zoom ratios, a higher speed internal clock would be needed.

P-0037[0037] The output device must be able to handle the asynchronous nature of the output pixel stream, as the output pixels are available at a discontinuous rate depending on zoom ratio. This asynchronism can be handled, for example, by conventional video compression chips which read the incoming pixel stream into a frame of memory. If the output device is not capable of handling asynchronism, a first-in, first-out memory or a full frame memory must be interposed.

P-0038[0038] The vertical address generator/vertical look-up table (VLUT) <b>210</b>v, illustrated in FIG. 3<i>b</i>, is identical to the H address generator/HLUT <b>210</b>h, except it is clocked by horizontal sync pulses instead of camera clock pulses, and is reset during vertical sync instead of horizontal sync. When the zoom ratio is near maximum telephoto, two output lines are needed for each input line and the comparator <b>217</b>v triggers the output of a second line of data.

P-0039[0039] The scaled chrominance C signal from the C scaler <b>220</b> is positioned responsive to the same pan, tilt, and zoom signals ; and is controlled by the same address generator <b>210</b>, as the luminance Y signal so that the association of chrominance C and luminance Y signal components is unchanged.

P-0040[0040] After a pixel forming part of the luminance Y signal has been processed by the scaler <b>200</b>, it is passed on to sharpening filter <b>300</b>. In the present example, the pixel passed on is a CIF-sized pixel.

P-0041[0041] As illustrated in FIGS. 1 and 4, the sharpening filter <b>300</b> includes an edge detector <b>320</b> and a sharpener <b>305</b>. Details of the edge detector <b>320</b> and the sharpener <b>305</b> are depicted in FIG. 4. In the preferred embodiment of the present invention, edges and their orientation are determined first and the pixels located on an edge are subsequently sharpened in the direction perpendicular to the edge. However, as already mentioned above, pixels can also be sharpened without the requirement that the pixels to be sharpened are located on an edge. This aspect will be discussed in detail below.

P-0042[0042] The edge sharpening filter of the present example operates on three adjacent pixels on each of three successive lines. The unsharpened CIF Y video signal is received from the output of the Y scaler <b>230</b> by, in the present example, three H line delays <b>310</b>, <b>312</b>, and <b>314</b>, as illustrated in FIG. 4. The signals from each delay line are then passed on to single-clock-delay registers <b>315</b> so that horizontally adjacent pixels in each of the three lines are available simultaneously at the output of registers <b>315</b>.

P-0043[0043] The edge detector <b>320</b> can be a conventional edge detection filter, such as a Canny, Sobel or Kirsch filter which are known in the art. The edge detector of the present embodiment is a 3×3 Kirsch edge detector and has the following matrix form: <maths id="MATH-US-00001" num="1"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><mo>|</mo><mtable><mtr><mtd><msub><mi>k</mi><mn>11</mn></msub></mtd><mtd><msub><mi>k</mi><mn>12</mn></msub></mtd><mtd><msub><mi>k</mi><mn>13</mn></msub></mtd></mtr><mtr><mtd><msub><mi>k</mi><mn>21</mn></msub></mtd><mtd><msub><mi>k</mi><mn>22</mn></msub></mtd><mtd><msub><mi>k</mi><mn>23</mn></msub></mtd></mtr><mtr><mtd><msub><mi>k</mi><mn>31</mn></msub></mtd><mtd><msub><mi>k</mi><mn>32</mn></msub></mtd><mtd><msub><mi>k</mi><mn>33</mn></msub></mtd></mtr></mtable><mo>|</mo><mrow><mo>,</mo><mstyle><mtext> </mtext></mstyle><mo></mo><mrow><mi>wherein</mi><mo>[</mo><msub><mi>k</mi><mi>ij</mi></msub><mo>]</mo></mrow></mrow></mrow><mo>=</mo><mrow><mo>|</mo><mstyle><mtext> </mtext></mstyle><mo></mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr><mtr><mtd><mn>1</mn></mtd><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr><mtr><mtd><mn>1</mn></mtd><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr></mtable><mo>|</mo><mo>,</mo><mo>|</mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mn>1</mn></mtd><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr></mtable><mo>|</mo><mo>,</mo><mstyle><mtext> </mtext></mstyle><mo>|</mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr></mtable><mo>|</mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>and</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo>|</mo><mtable><mtr><mtd><mn>0</mn></mtd><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd></mtr></mtable><mo>|</mo></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mrow><mi>Equation</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mn>1</mn></mrow><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20030193584A1-20031016-M00001.TIF" id="EMI-M00001" he="31.9221" wi="390.0393" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00001" attachment-type="nb" file="US20030193584A1-20031016-M00001.NB" /></attachments></maths>

P-0044[0044] for four different edge orientations (vertical, +45°, horizontal and −45°). Other angles can be detected as well by selecting different matrices in Equation 1, as is known in the art.

P-0045[0045] The edge detector <b>320</b> illustrated in FIG. 4 operates on a 3×3 array of pixels comprising 3 adjacent pixels in each of 3 successive horizontal lines and forming a pixel matrix [a_ij]. In the following, the pixels of the first line are also designated a1, a2, and a3; the pixels of the second line are also designated b1, b2, and b3; and the pixels of the third line are also designated c1, c2, and c3. Pixel b2 is taken as the center pixel on which an edge test is being performed. To determine the orientation of the edge, the pixel matrix is convoluted with each of the Kirsch matrices of Equation 1 via the operation <maths id="MATH-US-00002" num="2"><math overflow="scroll"><mrow><mi>Res</mi><mo>=</mo><mrow><munderover><mo>∑</mo><mi>i</mi><mstyle><mtext> </mtext></mstyle></munderover><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mrow><mo>(</mo><mrow><munderover><mo>∑</mo><mrow><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi></mrow><mstyle><mtext> </mtext></mstyle></munderover><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mrow><msub><mi>a</mi><mi>ij</mi></msub><mo>×</mo><msub><mi>k</mi><mi>ij</mi></msub></mrow></mrow><mo>)</mo></mrow></mrow></mrow></math><img file="US20030193584A1-20031016-M00002.TIF" id="EMI-M00002" he="27.13095" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00002" attachment-type="nb" file="US20030193584A1-20031016-M00002.NB" /></attachments></maths>

P-0046[0046] For example, the horizontal Kirsch edge detector is applied as follows: <maths id="MATH-US-00003" num="3"><math overflow="scroll"><mtable><mtr><mtd><mtable><mtr><mtd><mrow><mi>Res</mi><mo>=</mo><mi /><mo></mo><mrow><mo>|</mo><mtable><mtr><mtd><msub><mi>a</mi><mn>1</mn></msub></mtd><mtd><msub><mi>a</mi><mn>2</mn></msub></mtd><mtd><msub><mi>a</mi><mn>3</mn></msub></mtd></mtr><mtr><mtd><msub><mi>b</mi><mn>1</mn></msub></mtd><mtd><msub><mi>b</mi><mn>2</mn></msub></mtd><mtd><msub><mi>b</mi><mn>3</mn></msub></mtd></mtr><mtr><mtd><msub><mi>c</mi><mn>1</mn></msub></mtd><mtd><msub><mi>c</mi><mn>2</mn></msub></mtd><mtd><msub><mi>c</mi><mn>3</mn></msub></mtd></mtr></mtable><mo>|</mo><mrow><mo>⊗</mo><mrow><mo>|</mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr></mtable><mo>|</mo></mrow></mrow></mrow></mrow></mtd></mtr><mtr><mtd><mrow><mi>Res</mi><mo>=</mo><mi /><mo></mo><mrow><mi>a1</mi><mo>+</mo><mi>a2</mi><mo>+</mo><mi>a3</mi><mo>+</mo><mn>0</mn><mo>+</mo><mn>0</mn><mo>+</mo><mn>0</mn><mo>-</mo><mi>c1</mi><mo>-</mo><mi>c2</mi><mo>-</mo><mi>c3</mi></mrow></mrow></mtd></mtr></mtable></mtd><mtd><mrow><mo>(</mo><mrow><mi>Equation</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mn>2</mn></mrow><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20030193584A1-20031016-M00003.TIF" id="EMI-M00003" he="42.9786" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00003" attachment-type="nb" file="US20030193584A1-20031016-M00003.NB" /></attachments></maths>

P-0047[0047] The value Res for the other orientations is calculated accordingly.

P-0048[0048] For a particular center pixel b2, the matrix [k_ij] which after convolution with [a_ij] provides the largest value of |Res|, determines the orientation of the edge on which the pixel b2 is located. Since the scaler <b>200</b> addresses every pixel of interest in an image, every pixel will at some point in time be in the position b2. However, as mentioned above, pixels on the marginal edge of the image, that is pixels which are not surrounded by neighboring pixels, are not considered by the edge detector <b>320</b> and are therefore also not sharpened.

P-0049[0049] Because discrete convolution can quickly become computationally extensive, the edge detector <b>320</b> is advantageously divided into a pre-calculation block <b>321</b> followed by an edge angle detection block <b>325</b>. The mathematical operation in both the pre-calculation block <b>321</b> and the edge angle detection block <b>325</b> are simple additions, unlike the multiplication of Equation 1, which can therefore be executed fast and with simpler circuitry. As noted above, the largest absolute value |Res|_maxfrom the four output values supplied by the edge angle detection block <b>325</b> is selected by the angle selector block <b>330</b> as the edge orientation.

P-0050[0050] In order to assess if the so determined edge is to be sharpened by sharpener <b>305</b>, the output |Res|_maxof edge angle selector block <b>330</b> is compared in comparator <b>335</b> to a preset sharpening threshold value T. If |Res|_maxis larger than the threshold value T, then the center pixel b2 is determined to be located on an edge and subject to sharpening.

P-0051[0051] The response of the edge detection filter does not depend on the value of the center pixel b2, since the matrix element k₂₂is always equal to zero, so that the product k₂₂×b<b>2</b> is also zero. b2 is deemed to be located on an edge, if the value of b2 lies between the values Px and Py of the two pixels adjacent to it in a direction perpendicular to the edge. Once the direction of the edge is determined, the points Px and Py in the direction perpendicular to the edge are fetched from switch <b>340</b> and the conditions Px<b2<Py or Px>b2>Py, respectively, are evaluated in decision block <b>343</b>. If neither of these two conditions is true, then pixel b2 is not sharpened, as indicated by the position of switch <b>370</b>. In an alternate embodiment, this test could be omitted if b2 is counted among the pixels in the neighborhood of pixels; the sharpened value of b2 (hereinafter referred to as b2′) is always between the largest value and the smallest unsharpened pixel value in the neighborhood to which b2′ is clipped.

P-0052[0052] Once it is determined that center pixel b2 is located on an edge, i.e. |Res|_max>T, then the neighborhood of the edge pixel is considered for sharpening. The embodiment of the invention uses a Laplacian filter for illustrative purposes; however, other filters, such as differences of Gaussians and unsharp masking, which are known in the art, can also be used. Like the edge filter discussed previously, the sharpening filter can be expressed as a matrix. The size of the matrix depends on the sharpening neighborhood, that is for a 3×3 array of pixels the sharpening matrix is also a 3×3 matrix. The sharpening neighborhood of pixels can be different from the neighborhood of pixels used for edge detection. If edge detection is employed, then the Laplacian filter for the four edge orientations discussed above (horizontal, vertical and ±45°) has the following form: <maths id="MATH-US-00004" num="4"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>S</mi><mo>=</mo><mrow><mo>|</mo><mtable><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mrow><mrow><mn>2</mn><mo></mo><mi>S</mi></mrow><mo>+</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr></mtable><mo>|</mo><mstyle><mtext> </mtext></mstyle><mo>|</mo><mtable><mtr><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mrow><mrow><mn>2</mn><mo></mo><mi>S</mi></mrow><mo>+</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd></mtr></mtable><mo>|</mo><mo>,</mo><mstyle><mtext> </mtext></mstyle><mo>|</mo><mtable><mtr><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mrow><mrow><mn>2</mn><mo></mo><mi>S</mi></mrow><mo>+</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mn>0</mn></mtd></mtr></mtable><mo>|</mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>and</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo>|</mo><mtable><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mrow><mrow><mn>2</mn><mo></mo><mi>S</mi></mrow><mo>+</mo><mn>1</mn></mrow></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr></mtable><mo>|</mo></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mrow><mi>Equation</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mn>3</mn></mrow><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20030193584A1-20031016-M00004.TIF" id="EMI-M00004" he="39.123" wi="332.2053" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00004" attachment-type="nb" file="US20030193584A1-20031016-M00004.NB" /></attachments></maths>

P-0053[0053] where the sharpening parameter S is a positive scalar value.

P-0054[0054] Without edge detection, the Laplacian filter may have other suitable forms, such as: <maths id="MATH-US-00005" num="5"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>S</mi><mo>=</mo><mrow><mo>|</mo><mtable><mtr><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mrow><mrow><mn>8</mn><mo></mo><mi>S</mi></mrow><mo>+</mo><mn>1</mn></mrow></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd><mtd><mrow><mo>-</mo><mi>S</mi></mrow></mtd></mtr></mtable><mo>|</mo></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mrow><mi>Equation</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mn>4</mn></mrow><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20030193584A1-20031016-M00005.TIF" id="EMI-M00005" he="31.9221" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00005" attachment-type="nb" file="US20030193584A1-20031016-M00005.NB" /></attachments></maths>

P-0055[0055] In the following discussion, it is assumed that edge detection is used and that b2 is located on a vertical edge. With a value of S=1, the matrix of equation (3) becomes: <maths id="MATH-US-00006" num="6"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>S</mi><mo>=</mo><mrow><mo>|</mo><mtable><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mn>3</mn></mtd><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd></mtr><mtr><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr></mtable><mo>|</mo></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mrow><mi>Equation</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mn>5</mn></mrow><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20030193584A1-20031016-M00006.TIF" id="EMI-M00006" he="31.9221" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00006" attachment-type="nb" file="US20030193584A1-20031016-M00006.NB" /></attachments></maths>

P-0056[0056] The edge pixel b2 is sharpened by convoluting the pixel matrix [a_ij] wherein b2 is the center matrix element a₂₂, with the sharpening matrix S of Equation 3 according to the formula: <maths id="MATH-US-00007" num="7"><math overflow="scroll"><mrow><mrow><msup><mi>b2</mi><mi>′</mi></msup><mo>≡</mo><msub><mi>b2</mi><mi>Sharpened</mi></msub></mrow><mo>=</mo><mrow><munderover><mo>∑</mo><mi>i</mi><mstyle><mtext> </mtext></mstyle></munderover><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mrow><mo>(</mo><mrow><munderover><mo>∑</mo><mrow><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi></mrow><mstyle><mtext> </mtext></mstyle></munderover><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mrow><msub><mi>a</mi><mi>ij</mi></msub><mo>×</mo><msub><mi>s</mi><mi>ij</mi></msub></mrow></mrow><mo>)</mo></mrow></mrow></mrow></math><img file="US20030193584A1-20031016-M00007.TIF" id="EMI-M00007" he="27.13095" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00007" attachment-type="nb" file="US20030193584A1-20031016-M00007.NB" /></attachments></maths>

P-0057[0057] or for a vertical edge:

<i>b</i><b>2</b><i>′=−S</i>*(<i>b</i><b>1</b><i>+b</i><b>3</b>)+(2<i>S</i>+1)*<i>b</i><b>2</b>

P-0058[0058] The appropriate sharpening filter of Equation (3) is implemented in the apparatus shown in FIG. 4 by first selecting with switch <b>340</b> the appropriate pair of neighboring pixels Px and Py which are perpendicular to the previously determined direction of the edge. If pixel b2 is on a vertical edge, as in the present example, then Px is the pixel value of one of pixels b1 and b3. and Py is the pixel value of the other one of pixels b3 and b1. The values of Px and Py are then added in a first adder <b>345</b> and multiplied by the negative value of the desired sharpening parameter S which is supplied to the input of the first multiplier <b>350</b>.

P-0059[0059] The value 2S+1 is supplied to the input of a second multiplier <b>355</b> and multiplied by the value b2, whereafter the outputs of first multiplier <b>350</b> and second multiplier <b>355</b> are added in a second adder <b>360</b>. As mentioned above, the edge pixel is sharpened only, as indicated by the position of switch <b>370</b>, if the absolute value of |Res| obtained by selector <b>330</b> is larger than the predetermined threshold value T and conditions in switch <b>343</b> hold true.

P-0060[0060] The value of sharpened center pixel b2′ at the output of adder <b>360</b> is subsequently “clipped” in clipping block <b>365</b> so as to lie between the unsharpened values of the pair of neighboring pixels Px and Py perpendicular to the edge. An unsharpened pixel in the context of the present invention is a pixel as originally supplied at the output of scaler <b>200</b> (see FIG. 1). This pixel may already have been processed by a sharpening algorithm internal to the camera. For example, if a center pixel b2 lies on vertical edge and has been chosen for sharpening, its value after sharpening is then clipped so that it lies between the values of the unsharpened pixels b1 and b3 to its left and right, respectively. This clipping allows greater sharpening of edges without causing objectionable overshoots or undershoots.

P-0061[0061] As mentioned above, the sharpening filter <b>300</b> includes a switch <b>370</b> which adaptively switches the video Y signal between the sharpened pixel values and the unsharpened pixel values depending on the characteristic features of the edge detection. For example, pixels which are located on edges and exceed the threshold T and which satisfy decision block <b>343</b>, are sharpened. Otherwise unsharpened pixels are passed on. It will be understood that the logic elements of the edge detector and the sharpening filter can be arranged in a different order as long as they perform the desired function.

P-0062[0062] The application of a Laplacian sharpening filter to the pixels of an exemplary edge and the beneficial effect of “clipping” the overshoot and undershoot, respectively, will now be explained with reference to FIGS. <b>5</b> to <b>7</b>. The edge depicted by pixels P<b>1</b> through P<b>7</b> of FIG. 5 extends over 7 pixels, with pixel values ranging from 0 (light object value) to 48 (dark object value) and would likely appear blurred. The original pixel values are listed in the second column of Table 1. When the Laplacian sharpening filter with S=1 is applied to pixels P<b>1</b> through P<b>7</b> (the pixels before pixel P<b>1</b> and after pixel P<b>7</b> are not located on an edge and are consequently unchanged), the values listed in the third column of Table 1 are obtained. These sharpened values which are obtained at the output of adder <b>360</b>, as discussed above, are depicted as pixels P<b>1</b>′ through P<b>7</b>′ in FIG. 6. It is evident from FIG. 6 that the slope of the curve through pixels P<b>3</b>′, P<b>4</b>′ and P<b>5</b>′ increases in comparison to FIG. 5, but that at the same time an undershoot at pixels P<b>1</b>′ and P<b>2</b>′ and an overshoot at pixels P<b>6</b>′ and P<b>7</b>′ are introduced. This undershoot and overshoot, respectively, could create an undesirable halo around the edge pixels. <tables id="TABLE-US-00001" num="1"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="5"><colspec colname="OFFSET" colwidth="14PT" align="left" /><colspec colname="1" colwidth="28PT" align="center" /><colspec colname="2" colwidth="63PT" align="center" /><colspec colname="3" colwidth="49PT" align="center" /><colspec colname="4" colwidth="63PT" align="center" /><thead><row><entry /><entry namest="OFFSET" nameend="4" align="center">TABLE 1</entry></row><row><entry /><entry /></row><row><entry /><entry namest="OFFSET" nameend="4" align="center" rowsep="1" /></row><row><entry /><entry /><entry>ORIGINAL</entry><entry>SHARPENED</entry><entry>CLIPPED</entry></row><row><entry /><entry>PIXEL</entry><entry>VALUE</entry><entry>VALUE</entry><entry>VALUE</entry></row><row><entry /><entry namest="OFFSET" nameend="4" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="5"><colspec colname="OFFSET" colwidth="14PT" align="left" /><colspec colname="1" colwidth="28PT" align="center" /><colspec colname="2" colwidth="63PT" align="char" char="." /><colspec colname="3" colwidth="49PT" align="char" char="." /><colspec colname="4" colwidth="63PT" align="char" char="." /><tbody valign="top"><row><entry /><entry>P1</entry><entry>0</entry><entry>−3</entry><entry>0</entry></row><row><entry /><entry>P2</entry><entry>3</entry><entry>−3</entry><entry>0</entry></row><row><entry /><entry>P3</entry><entry>12</entry><entry>9</entry><entry>9</entry></row><row><entry /><entry>P4</entry><entry>24</entry><entry>24</entry><entry>24</entry></row><row><entry /><entry>P5</entry><entry>36</entry><entry>39</entry><entry>39</entry></row><row><entry /><entry>P6</entry><entry>45</entry><entry>51</entry><entry>48</entry></row><row><entry /><entry>P7</entry><entry>48</entry><entry>51</entry><entry>48</entry></row><row><entry /><entry namest="OFFSET" nameend="4" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

P-0063[0063] To eliminate such halo effect, the pixel values are “clipped”, that is, the new value of each sharpened pixel Pi′ is limited to lie between the original, unsharpened values of its neighboring pixels P(i−1) and P(i+1) listed in the second column of Table 1. In those instances in which sharpening is applied in a direction orthogonal to the edge, as is the case in the preferred embodiment of this invention, and in which a 3×3 sharpening filter is applied to a 3×3 array of pixels, as in this illustration, P(i−1) and P(i+1) will be the unsharpened values of the immediately adjacent pixels on either side of the edge on which the pixel Pi is located.

P-0064[0064] The sharpening operation, using a 3×3 Laplacian filter of the form shown in equations 2 and 3, is carried out specifically as follows. The pixel to be sharpened is assumed to be pixel P<b>2</b> in FIG. 5. Pixel P<b>2</b> has an original, unsharpened numerical value of 3. We further assume S=1 in the Laplacian filter, so that the specific numerical values in the Laplacian filter are −1, 3, and −1. To sharpen pixel P<b>2</b>, the numerical value of pixel P<b>1</b>, which is 0, is multiplied by −1, the numerical value of pixel P<b>2</b> is multiplied by 3, and the numerical value of pixel P<b>3</b> is multiplied by −1. The three results (that is, 0, 9, and −12) are then added to yield the sharpened value of pixel P<b>2</b> which, in this case, is −3. This negative value represents an undershoot because the sharpened numerical pixel value of P<b>2</b> (that is, P<b>2</b>′) is less than the unsharpened numerical pixel value of pixel P<b>1</b> (=0) as shown in FIG. 6. With the traditional sharpening methods and sharpening filters, this undesirable undershoot would remain. The sharpening filter of the present invention, however, clips the sharpened value of P<b>2</b> (that is, P<b>2</b>′) so as to lie between the original, unsharpened numerical values of P<b>1</b> and P<b>3</b>.

P-0065[0065] Each edge pixel in FIG. 5 is subjected to the same sharpening and clipping operation resulting in the graph shown in FIG. 7. The clipped pixel values are listed in the fourth column of Table 1. As discussed previously, clipping with the apparatus of the invention is carried out in clipping block <b>365</b>, so as to limit b2′ to lie between the unsharpened pixel values of the neighborhood pixels.

P-0066[0066] As discussed before, pixels can be sharpened without prior edge detection by applying a different Laplacian filter, such as the filter of equation (4), to the neighborhood of pixels. With no edge detection, the values of all neighboring pixels of b2 are added in a first adder <b>345</b> and multiplied by the negative value of the desired sharpening parameter S which is supplied to the input of the first multiplier <b>350</b>. The value 8S+1 is supplied to the input of the second multiplier <b>355</b> and multiplied by the value b2, whereafter the outputs of first multiplier <b>350</b> and second multiplier <b>355</b> are added in a second adder <b>360</b>. Thereafter, the process is identical to the process used with edge sharpening.

P-0067[0067] If there is no edge detection, the values Px and Py to which b2 is clipped in switch <b>365</b>, are the minimum and maximum values, respectively, among all neighboring pixels. As discussed before, the center pixel b2 is excluded if switch <b>343</b> is used. Otherwise, b2 is included in the neighborhood of pixels.

P-0068[0068]FIG. 8 illustrates in the form of a flow diagram a method for carrying out the present invention of sharpening a digitally processed video image comprised of pixels in which each pixel has a numerical value and resides in a neighborhood of pixels.

P-0069[0069] In the present example, the CIF signal is received from the Y video output of scaler <b>200</b>. In a first step <b>510</b>, the pixel to be sharpened is evaluated so as to determine if the pixel is a local minimum or maximum.

P-0070[0070] As noted above, step <b>510</b> can be omitted if the pixel to be sharpened and clipped is counted among the pixels in the selected neighborhood of pixels. In this case, if the pixel were a local minimum or maximum, the process branches directly to step <b>580</b>, where the pixel is sharpened, but is subsequently clipped to itself in step <b>590</b>. Consequently, a local minimum or maximum is passed to output <b>600</b> without sharpening, independent of the method used.

P-0071[0071] If the pixel is not a local minimum or maximum, then the process flow branches in decision step <b>520</b> depending if an edge is to be detected or not. If the pixel is to be sharpened regardless of the presence of an edge, then the process flow branches directly to step <b>580</b>.

P-0072[0072] If an edge is to be detected first, then an edge filter, e.g., the Kirsch filter described above, is applied in step <b>530</b> to determine if the pixel is indeed located on an edge. In the next step <b>540</b>, the edge orientation is determined from the maximum value of |Res| and the edge is then sharpened only in the direction perpendicular to the edge direction. As explained before, an edge is sharpened only if the largest absolute value of Res (Equation 1) is larger than a predetermined threshold value T which can be provided at any step before step <b>560</b>, for example in step <b>550</b>. T can be provided either by the user or, preferably, automatically adaptively in response to, for example, camera settings (zoom ratio) and illumination conditions (noise). If in step <b>560</b> |Res| is smaller than T, then the pixel is not deemed to be located on an edge, and the process flow branches directly to output <b>600</b>, that is, the pixel is not sharpened. However, if |Res| is greater than T, then the pixel is presumed to lie on an edge and is sharpened in step <b>580</b> with a sharpening parameter S which is supplied in step <b>570</b>. Like the threshold parameter T, the parameter S can be provided either by the user or, preferably, automatically adaptively in response to, for example, the zoom ratio. S can be supplied at any step before step <b>570</b>.

P-0073[0073] After the pixel is sharpened in step <b>580</b>, the value of the sharpened pixel is “clipped” so as to lie between the unsharpened numerical values of any of the pixels within its sharpening neighborhood. If, as mentioned above, thresholding and sharpening of the pixel is restricted to the direction perpendicular to the edge, which is determined in step <b>540</b>, then in step <b>590</b>, the pixel value is also clipped only in the direction perpendicular to the edge. In step <b>600</b>, the sharpened and clipped pixel value is then provided at the CIF Y video output of sharpener <b>305</b> (FIGS. 1 and 4).

P-0074[0074] The foregoing is considered to be illustrative only of the principles of the invention. The video information is not limited to input from a video camera, but can included, e.g., scanned input, input received via network connection and/or input received from a storage device. Other video formats, such as RGB, MPEG, etc., can also be used. Other hardware and/or software which performs similar functions, can be substituted. Modifications of the preferred embodiments disclosed herein will be apparent to those skilled in the art that are within the scope of the following claims.