Motion detection of an input image signal during a current period with respect to a restored image signal in a previous period

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing method and apparatus and, more particularly, to motion detection processing of an input image signal.

2. Related Background Art

Along with recent trends toward smaller, multimedia-oriented computers, display devices in computers have remarkably advanced. As display devices, liquid crystal display devices have been popularly used. As one of such display devices, a display device using a ferroelectric liquid crystal display device (to be abbreviated as an FLCD hereinafter) is known.

As a feature of the FLCD, a high-definition display device with a large screen is achieved. However, since a liquid crystal element itself is a binary device which cannot change the transmittance of light in an analog manner, pseudo gradation levels are expressed by combinations of ON and OFF dots upon displaying a full-color image or the like. That is, digital halftone processing must be performed to reproduce a halftone image.

Since it is difficult to display data on a large screen at a rate of 30 frames per second or more, only a portion in which motion is detected in a frame must be rewritten by utilizing the memory characteristics as the feature of the FLCD. For this purpose, a portion including motion in an image must be detected. Conventionally, the following method of detecting motion is used.

FIG. 1 is a block diagram of an image processing apparatus for an FLCD, which is adapted to display an analog image signal supplied from a computer on a FLCD and includes a motion detection circuit.

Referring to FIG. 1, an analog image signal output from a computer (not shown) is converted by an A/D converter <b>1</b> into a digital signal having a predetermined number of gradation levels, e.g., a 24-bit digital signal (8 bits for each of R, G, and B colors).

A gradation conversion circuit <b>2</b> converts the R, G, and B digital signals output from the A/D converter <b>1</b> into a digital signal having a number of resolution levels that can be displayed by a display device <b>7</b>, e.g., a 3-bit digital signal (1 bit for each of R, G, and B colors) by known digital halftone processing such as the dither method, the error diffusion method, or the like. The digital signal output from the gradation conversion circuit <b>2</b> is stored in a binary frame memory <b>5</b>.

In additon, the digital signal output from the A/D converter <b>1</b> is input to a motion detector <b>9</b>. The motion detection <b>9</b> is described in detail below.

FIG. 2 is a block diagram showing the motion detector <b>9</b>.

Referring to FIG. 2, the image signal input from the A/D converter <b>1</b> is stored in a line buffer <b>901</b> having a predetermined capacity for K pixels. As shown in FIG. 2, the line buffer is designed to store K pixels, and pixel data stored in the line buffer are subjected to weighted mean processing given by equation (1) below via a multiplier <b>902</b>, an adder <b>903</b>, and an accumulator <b>904</b> so as to be converted into a feature value S_tcorresponding to a set of K pixels: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>S</mi><mi>t</mi></msub><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>0</mn></mrow><mrow><mi>K</mi><mo>-</mo><mn>1</mn></mrow></munderover><mo></mo><mrow><msub><mi>w</mi><mi>i</mi></msub><mo></mo><msub><mi>p</mi><mi>i</mi></msub></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00001" file="US06246780-20010612-M00001.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00001" attachment-type="nb" file="US06246780-20010612-M00001.NB" /></attachments></maths>

where w_iis a weighting coefficient determined in advance by a predetermined method, and an image which is being processed is assumed to belong to the t-th frame. Subsequently, the calculated feature amount S_tis compared with a feature amount S_t−1which was calculated based on image data of the previous frame and was stored in a feature amount memory <b>10</b> in FIG. 1, by a comparison controller <b>905</b>. In this case, a difference ΔS as the comparison result is compared with a predetermined threshold value T, and if the following inequality holds, it is determined that motion is detected from the corresponding line portion, and the line address of the pixels at that time is output to a frame memory control circuit <b>4</b> (FIG. <b>1</b>):

<maths><formula-text>ΔS=S_t−S_t−1&gt;T  (2)</formula-text></maths>

Note that the line address is the start address of K successive pixels.

The above-mentioned motion detection processing is performed in units of R, G, and B colors, and the calculated feature amount is newly stored in the feature amount memory <b>10</b>. The frame memory control circuit <b>4</b> outputs data of the motion-detected line from the digital signal (1 bit for each of R, G, and B colors) stored in the binary frame memory <b>5</b> on the basis of the address information output from the motion detector <b>9</b>. With the above-mentioned processing, continuous analog image signals are converted into signals having a predetermined number of gradation levels, and motion between adjacent frames is detected to partially rewrite an image to be displayed, thus realizing an image display operation in real time.

However, in the above-mentioned prior art, in order to attain motion detection, the feature amount obtained from image data of the previous frame must be stored in the feature amount memory <b>10</b>.

In particular, since the screen size has been increasing in recent years, the capacity required for the feature amount memory is also increasing, resulting in an increase in cost of the apparatus.

Upon execution of motion detection, binary pixel data may be directly compared with each other without calculating any feature amount. However, in this case, if an image signal includes noise from the transmission route, a binary image bit is easily inverted, resulting in a detection error.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the above situation, and has as its object to provide an image processing method and apparatus, which can reduce the capacity of a feature amount memory required for motion detection as much as possible so as to attain a cost reduction of the system, and rarely causes a detection error due to noise.

In order to achieve the above object, according to one preferred embodiment of the present invention, an image processing apparatus (method) comprises conversion means (step) for converting an input image signal having a number L₁of gradation levels into an image signal having a number L₂(&lt;L₁) of gradation levels, storage means (step) for storing the image signal converted by the conversion means (step), restoration means (step) for restoring the image signal stored in the storage means (step), and detection means (step) for detecting a motion of an image on the basis of the image signal input during a current period and the image signal in a previous period, which is restored by the restoration means (step).

Other objects, features and advantages of the invention will become apparent form the following detailed description taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a conventional image processing apparatus;

FIG. 2 is a block diagram showing a motion detector <b>9</b> shown in FIG. 1;

FIG. 3 is a block diagram showing an image processing apparatus according to the first embodiment of the present invention;

FIG. 4 is a detailed block diagram for explaining the processing of a motion detector <b>13</b> shown in FIG. 3;

FIG. 5 is a block diagram showing an image processing apparatus according to the second embodiment of the present invention;

FIG. 6 is a graph showing an example of a nonlinear function in a gradation conversion circuit shown in FIG. 5; and

FIG. 7 is a block diagram showing the arrangement of the comparison circuit <b>18</b> shown in FIG. <b>5</b>.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The preferred embodiments of the present invention are described hereinafter.

FIG. 3 is a block diagram showing an image processing apparatus according to the first embodiment of the present invention.

Referring to FIG. 3, the apparatus comprises an A/D converter <b>11</b>, a gradation conversion circuit <b>12</b>, a motion detector <b>13</b>, a frame memory control circuit <b>14</b>, a binary frame memory <b>15</b>, a display device <b>17</b>, and a filter (to be referred to as an LPF hereinafter) <b>16</b> having low-pass characteristics with coefficients shown in FIG. <b>4</b> and equation (5) shown later.

The operation of the image processing apparatus with the arrangement shown in FIG. 3 is explained below.

An analog image signal output from a computer (not shown) is converted by the A/D converter <b>11</b> into a digital signal having a predetermined number of gradation levels, e.g., a 24-bit digital signal (8 bits for each of R, G, and B colors). The gradation conversion circuit <b>12</b> converts the digital signal output from the A/D converter <b>11</b> into a digital signal having a number of gradation levels that are displayed by the display device <b>17</b>, e.g., a 3-bit digital signal (1 bit for each of R, G, and B colors) by the dither method. The digital signal output from the gradation conversion circuit <b>12</b> is stored in the binary frame memory <b>15</b>.

In addition, the digital signal output from the A/D converter <b>11</b> is input to the motion detector <b>13</b>. The motion detector <b>13</b> is described in detail below with reference to FIG. <b>4</b>.

Referring to FIG. 4, the image signal output from the A/D converter <b>11</b> is stored in a line buffer <b>301</b> having a predetermined capacity for K pixels. As shown in FIG. 4, the line buffer <b>301</b> is designed to store K pixels, and in this case, K pixels starting from the pixel at a coordinate position (i, j) in the scanning direction in the t-th frame are input to the line buffer <b>301</b>. The pixel data stored in the line buffer <b>301</b> are subjected to weighted mean processing given by equation (3) below via a multiplier <b>302</b>, an adder <b>303</b>, and an accumulator <b>304</b> so as to be converted into a feature value S_tcorresponding to a set of K pixels: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>S</mi><mi>t</mi></msub><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>0</mn></mrow><mrow><mi>K</mi><mo>-</mo><mn>1</mn></mrow></munderover><mo></mo><mrow><msub><mi>w</mi><mi>k</mi></msub><mo></mo><mrow><msub><mi>P</mi><mi>t</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mrow><mi>i</mi><mo>+</mo><mi>k</mi></mrow><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00002" file="US06246780-20010612-M00002.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00002" attachment-type="nb" file="US06246780-20010612-M00002.NB" /></attachments></maths>

where W_kis a coefficient determined in advance by a predetermined method, and P_t(i, j) indicates the pixel at a coordinate position (i, j) in an image of the t-th frame. Also binary image data of the (t−1)-th frame stored in the binary frame memory <b>5</b>, and pixel data for a total of nine neighboring pixels including the pixel at the position (i, j) as the center are fetched by the LPF <b>16</b>, and are subjected to a calculation given by equation (4) below so as to be converted into a mean value by weighted mean processing among surrounding pixels: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><msub><mover><mi>P</mi><mo>^</mo></mover><mrow><mi>t</mi><mo>-</mo><mn>1</mn></mrow></msub><mo></mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow><mo>=</mo><mfrac><mrow><munderover><mo>∑</mo><mrow><mi>m</mi><mo>,</mo><mrow><mi>n</mi><mo>=</mo><mrow><mo>-</mo><mn>1</mn></mrow></mrow></mrow><mn>1</mn></munderover><mo></mo><mrow><msub><mi>A</mi><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub><mo></mo><mrow><msub><mi>P</mi><mrow><mi>t</mi><mo>-</mo><mn>1</mn></mrow></msub><mo></mo><mrow><mo>(</mo><mrow><mrow><mi>i</mi><mo>+</mo><mi>m</mi></mrow><mo>,</mo><mrow><mi>j</mi><mo>+</mo><mi>n</mi></mrow></mrow><mo>)</mo></mrow></mrow></mrow></mrow><mrow><munderover><mo>∑</mo><mrow><mi>m</mi><mo>,</mo><mrow><mi>n</mi><mo>=</mo><mrow><mo>-</mo><mn>1</mn></mrow></mrow></mrow><mn>1</mn></munderover><mo></mo><msub><mi>A</mi><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub></mrow></mfrac></mrow></mtd><mtd><mrow><mo>(</mo><mn>4</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00003" file="US06246780-20010612-M00003.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00003" attachment-type="nb" file="US06246780-20010612-M00003.NB" /></attachments></maths>

where A_mnis a coefficient to be multiplied with each pixel, and consists of values given by equation (5) below: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>A</mi><mo>=</mo><mrow><mrow><mo>(</mo><mtable><mtr><mtd><msub><mi>A</mi><mrow><mrow><mo>-</mo><mn>1</mn></mrow><mo>-</mo><mn>1</mn></mrow></msub></mtd><mtd><msub><mi>A</mi><mrow><mn>0</mn><mo>-</mo><mn>1</mn></mrow></msub></mtd><mtd><msub><mi>A</mi><mrow><mn>1</mn><mo>-</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><msub><mi>A</mi><mrow><mo>-</mo><mn>10</mn></mrow></msub></mtd><mtd><msub><mi>A</mi><mn>00</mn></msub></mtd><mtd><msub><mi>A</mi><mn>10</mn></msub></mtd></mtr><mtr><mtd><msub><mi>A</mi><mrow><mo>-</mo><mn>11</mn></mrow></msub></mtd><mtd><msub><mi>A</mi><mn>01</mn></msub></mtd><mtd><msub><mi>A</mi><mn>11</mn></msub></mtd></mtr></mtable><mo>)</mo></mrow><mo>=</mo><mrow><mo>(</mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mn>2</mn></mtd><mtd><mn>1</mn></mtd></mtr><mtr><mtd><mn>2</mn></mtd><mtd><mn>4</mn></mtd><mtd><mn>2</mn></mtd></mtr><mtr><mtd><mn>1</mn></mtd><mtd><mn>2</mn></mtd><mtd><mn>1</mn></mtd></mtr></mtable><mo>)</mo></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>5</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00004" file="US06246780-20010612-M00004.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00004" attachment-type="nb" file="US06246780-20010612-M00004.NB" /></attachments></maths>

The mean value {circumflex over (P)}_t−1(i, j) is a restored value P_t−1(i, j) obtained by the weighted mean processing of surrounding pixels from the gradation-converted value.

The obtained restored value {circumflex over (P)}_t−1(i, j) is set in another line buffer <b>306</b> arranged in the motion detector <b>13</b>. Subsequently, the same processing as described above is performed for K binary pixel data and a mean value of the K pixels is set in the line buffer <b>306</b>. The pixel data stored in the line buffer <b>306</b> are subjected to weighted mean processing given by equation (6) below via a multiplier <b>307</b>, an adder <b>308</b>, and an accumulator <b>309</b>, and are converted into a feature value S_t−1corresponding to a set of K pixels: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>S</mi><mrow><mi>t</mi><mo>-</mo><mn>1</mn></mrow></msub><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>0</mn></mrow><mrow><mi>K</mi><mo>-</mo><mn>1</mn></mrow></munderover><mo></mo><mrow><msub><mi>w</mi><mi>k</mi></msub><mo></mo><mrow><msub><mover><mi>P</mi><mo>^</mo></mover><mrow><mi>t</mi><mo>-</mo><mn>1</mn></mrow></msub><mo></mo><mrow><mo>(</mo><mrow><mrow><mi>i</mi><mo>+</mo><mi>k</mi></mrow><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>6</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00005" file="US06246780-20010612-M00005.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00005" attachment-type="nb" file="US06246780-20010612-M00005.NB" /></attachments></maths>

where W_kis the same weighting coefficient as that used upon calculating S_tabove. The obtained two feature amounts S_tand S_t−1are input to a comparison controller <b>305</b>. The comparison controller <b>305</b> compares the two values, and then compares a difference ΔS as the comparison result with a predetermined threshold value T. If the following inequality holds, the controller <b>305</b> determines that motion is detected in the corresponding line portion, and outputs the line address of the pixels at that time to the frame memory control circuit <b>14</b> shown in FIG. <b>3</b>:

<maths><formula-text>ΔS=S_t−S_t−1&gt;T  (7)</formula-text></maths>

Note that the line address is the start address of K successive pixels. The above-mentioned motion detection processing is performed in units of R, G, and B colors.

The frame memory control circuit <b>14</b> controls the binary frame memory <b>15</b> to output data of the motion-detected line to the display device <b>17</b> on the basis of the address information output from the motion detector <b>13</b>. With the above-mentioned processing, continuous analog image signals are converted into signals having a predetermined number of gradation levels, and motion between adjacent frames is detected to partially rewrite an image to be displayed, thus realizing an image display operation in real time.

FIG. 5 is a block diagram showing an image processing apparatus according to the second embodiment of the present invention. Note that the same reference numerals in FIG. 5 denote the same parts as in FIG. 3, and a detailed description thereof is omitted.

The apparatus shown in FIG. 5 includes a comparison circuit <b>18</b>.

The operation of the image processing apparatus with the arrangement shown in FIG. 5 is described below.

An analog image signal output from a computer (not shown) is converted by the A/D converter <b>11</b> into a digital signal u having a predetermined number of gradation levels, e.g., 256 gradation levels per color. The signal u is converted into a signal having the number of gradation levels that can be displayed by the display device <b>17</b>, e.g., two gradation levels per color, by the gradation conversion circuit <b>12</b>, and the converted signal is output to the binary frame memory <b>15</b>.

Note that the display device <b>17</b> comprises a FLCD, and only a portion including a motion in an image for one frame as compared to the previous frame is output from the binary frame memory <b>15</b> under the control of the frame memory control circuit <b>14</b>. With this arrangement, motion between adjacent frames is detected by the comparison circuit <b>18</b>.

The gradation conversion circuit <b>12</b> comprises a cellular neural network in which neurons are locally coupled.

Upon converting an image having a size of M×N into that having two gradation levels, the operation of the cellular neural network is defined as: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><msub><mi>x</mi><mi>ij</mi></msub><mo>=</mo><mrow><mrow><munder><mrow><mstyle><mtext> </mtext></mstyle><mo>∑</mo></mrow><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></munder><mo></mo><msub><mi>A</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub><mo></mo><mrow><msub><mi>y</mi><mi>mn</mi></msub><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></mrow><mo>+</mo><mrow><munder><mrow><mstyle><mtext> </mtext></mstyle><mo>∑</mo></mrow><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></munder><mo></mo><msub><mi>B</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub><mo></mo><mrow><msub><mi>u</mi><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></mrow><mo>+</mo><mi>S</mi></mrow></mrow><mo></mo><mstyle>&lt;mtext /&gt;</mstyle><mo></mo><mrow><mrow><msub><mi>y</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi></mrow></msub><mo></mo><mrow><mo>(</mo><mrow><mi>t</mi><mo>+</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mi>f</mi><mo></mo><mrow><mo>(</mo><mrow><msub><mi>x</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi></mrow></msub><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo><mstyle>&lt;mtext /&gt;</mstyle><mo></mo><mrow><mrow><mi>f</mi><mo></mo><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mo>{</mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mrow><mrow><mi>for</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>x</mi></mrow><mo>≥</mo><mn>0</mn></mrow></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mrow><mi>for</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>x</mi></mrow><mo>&lt;</mo><mn>0</mn></mrow></mtd></mtr></mtable></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>8</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00006" file="US06246780-20010612-M00006.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00006" attachment-type="nb" file="US06246780-20010612-M00006.NB" /></attachments></maths>

where A_ijmnand B_ijmnare weights to be multiplied with a connection from the neuron at a position (m, n) to the neuron at a position (i, j). Also, u_mnand Y_mnare respectively input and output image data at the position (m, n). The cellular neural network generates an output image that minimizes a distortion function set as follows by the operation given by equations (8) above.

A distortion function dist(yi, ui) is defined as equation (9) below from a square error of the filtering result of the i-th gradation-converted output image y_iusing a predetermined filter H and the input image u_ibefore conversion: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><mi>dist</mi><mo></mo><mrow><mo>(</mo><mrow><msub><mi>y</mi><mi>i</mi></msub><mo>,</mo><msub><mi>u</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mfrac><mn>1</mn><mi>MN</mi></mfrac><mo></mo><msup><mrow><mo>(</mo><mrow><msub><mi>Hy</mi><mi>i</mi></msub><mo>-</mo><msub><mi>u</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow><mi>T</mi></msup><mo></mo><mrow><mo>(</mo><mrow><msub><mi>Hy</mi><mi>i</mi></msub><mo>-</mo><msub><mi>u</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>9</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00007" file="US06246780-20010612-M00007.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00007" attachment-type="nb" file="US06246780-20010612-M00007.NB" /></attachments></maths>

By comparing this distortion function and the energy function of the neural network, weights as the parameters of the network are set as follows:

<maths><formula-text>A=−H^TH+diag{H^TH}</formula-text></maths>

<maths><formula-text>B=X</formula-text></maths>

<maths><formula-text>S=0  (10)</formula-text></maths>

When the network operates based on these parameters and an output image is obtained, the value of the distortion function is minimized.

The operation of this embodiment is described below with reference to FIG. <b>5</b>.

A digital image signal uⁿof the n-th frame input from the A/D converter <b>11</b> is converted into an image yⁿby the operation of the neural network given by equations (11) below, and the converted image is output: <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><mrow><msubsup><mi>x</mi><mi>ij</mi><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mrow><munder><mrow><mstyle><mtext> </mtext></mstyle><mo>∑</mo></mrow><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></munder><mo></mo><msub><mi>A</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub><mo></mo><mrow><msubsup><mi>y</mi><mi>mn</mi><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></mrow><mo>+</mo><mrow><munder><mrow><mstyle><mtext> </mtext></mstyle><mo>∑</mo></mrow><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></munder><mo></mo><msub><mi>B</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow></msub><mo></mo><mrow><msubsup><mi>u</mi><mrow><mi>m</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>n</mi></mrow><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow></mrow><mo>+</mo><mi>S</mi></mrow></mrow><mo></mo><mstyle>&lt;mtext /&gt;</mstyle><mo></mo><mrow><mrow><msubsup><mi>y</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi></mrow><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mrow><mi>t</mi><mo>+</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mi>f</mi><mo></mo><mrow><mo>(</mo><mrow><msubsup><mi>x</mi><mrow><mi>i</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>j</mi></mrow><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo><mstyle>&lt;mtext /&gt;</mstyle><mo></mo><mrow><mrow><mi>f</mi><mo></mo><mrow><mo>(</mo><mi>x</mi><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mo>{</mo><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mrow><mrow><mi>for</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>x</mi></mrow><mo>≥</mo><mn>0</mn></mrow></mtd></mtr><mtr><mtd><mrow><mo>-</mo><mn>1</mn></mrow></mtd><mtd><mrow><mrow><mi>for</mi><mo></mo><mstyle><mtext> </mtext></mstyle><mo></mo><mi>x</mi></mrow><mo>&lt;</mo><mn>0</mn></mrow></mtd></mtr></mtable></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>11</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00008" file="US06246780-20010612-M00008.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00008" attachment-type="nb" file="US06246780-20010612-M00008.NB" /></attachments></maths>

where uⁿ_ij(t) and yⁿ_ij(t) are the values of the input image signal uⁿand the output image signal yⁿcorresponding to the position (i, j) on the screen in the t-th operation of the neural network. Also, f is a nonlinear function determined by the number of gradation levels that can be displayed by the display device <b>17</b>, and A_ijmnand B_ijmnare filter coefficients for a spatial filter operation applied to the yⁿ_ijand uⁿ_ij, which coefficients are determined in advance by equations (10) above. Note that the filter H is defined by a two-dimension Gaussian distribution which resembles human visual characteristics.

FIG. 6 shows an example of the nonlinear function f. The number of levels of an output with respect to an input is the same as the number of gradation levels that can be displayed by the display device <b>17</b>, and the output corresponds to image data having the number of gradation levels that can be displayed by the display device. In FIG. 6, the number of output gradation levels is <b>2</b>, and the display device <b>17</b> is a binary device which assumes either an ON or OFF state.

The gradation conversion circuit <b>12</b> calculates output data yⁿ(t) using equations (11), and calculates a sum total of differences between repetitive portions of the calculated image signals in one frame. When the sum total becomes equal to or smaller than a predetermined value T_s, as indicated by inequality (12) below, the circuit <b>12</b> outputs the output image signal yⁿ(t) at that time as a gradation-converted image yⁿto the binary frame memory <b>15</b>. <maths><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><munder><mo>∑</mo><mi>ij</mi></munder><mo></mo><mrow><mo></mo><mrow><mrow><msubsup><mi>y</mi><mi>ij</mi><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mi>t</mi><mo>)</mo></mrow></mrow><mo>-</mo><mrow><msubsup><mi>y</mi><mi>ij</mi><mi>n</mi></msubsup><mo></mo><mrow><mo>(</mo><mrow><mi>t</mi><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo></mrow></mrow><mo>&lt;</mo><msub><mi>T</mi><mi>s</mi></msub></mrow></mtd><mtd><mrow><mo>(</mo><mn>12</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img id="EMI-M00009" file="US06246780-20010612-M00009.TIF" img-content="math" img-format="tif" /><attachments><attachment idref="MATHEMATICA-00009" attachment-type="nb" file="US06246780-20010612-M00009.NB" /></attachments></maths>

Between the image signal yⁿoutput at that time and the input image signal uⁿ, the distortion function given by equation (9) above is minimized by the operation of the cellular neural network.

Subsequently, an image signal of the next (n+1)-th frame is input, and is converted into a digital image signal uⁿ⁺¹by the A/D converter <b>11</b> in the same manner as in the previous frame. Thereafter, the converted signal is input to the comparison circuit <b>18</b>. On the other hand, the image yⁿof the previous frame stored in the frame memory is input to the LPF <b>16</b>, and is converted as follows in accordance with equation (13) below:

<maths><formula-text>ûⁿ=Huⁿ  (13)</formula-text></maths>

uⁿ⁺¹and an are input to the comparison circuit <b>18</b> in units of lines, and motion between two frames in the corresponding line is detected on the basis of these data.

FIG. 7 is a block diagram showing the arrangement of the comparison circuit <b>18</b>.

uⁿ⁺¹and ûⁿinput to the comparison circuit <b>18</b> in units of lines are input to an adder <b>801</b> in units of

d=û_ijⁿ−u_ijⁿ⁺¹  (14)

where û_ijⁿand u_ijⁿ⁺¹are the i-th image data in the j-th line.

The difference d is input to a comparator <b>803</b>, and is compared with a predetermined threshold value Th1. When d is larger than Th1, the comparator <b>803</b> outputs “1”; otherwise, the comparator <b>803</b> outputs “0”. The comparator <b>803</b> supplies the output value to a counter <b>804</b>.

A pixel counter <b>802</b> counts the number of input pixels, and outputs a reset signal to the counter <b>804</b> when the count value has exceeded a pre-set value L. Upon reception of the reset signal, the counter <b>804</b> outputs the number K of “1” data input so far, and clears its internal counter to zero. A comparator <b>805</b> receives the number K of input data, and compares the input value with a predetermined threshold value Th2. When K has exceeded Th2, the comparator <b>805</b> outputs “1”; otherwise, the comparator <b>805</b> outputs “0”.

On the other hand, the difference d output from the adder <b>801</b> is also input to an accumulator <b>806</b>. The accumulator <b>806</b> accumulates the input values d in units of pixels, and outputs an accumulated value V to a comparator <b>807</b> upon reception of a reset signal from the pixel counter <b>802</b>. The comparator <b>807</b> compares the input value V with a predetermined threshold value Th3. When V has exceeded Th3, the comparator <b>807</b> outputs “1”; otherwise, the comparator <b>807</b> outputs “0”. Finally, in the comparison circuit <b>18</b>, an OR gate <b>809</b> ORs the results outputs from the comparators <b>805</b> and <b>807</b>, and outputs the OR as a rewriting flag for the currently effective line. A line counter <b>808</b> counts a line number on the basis of the output from the pixel counter <b>802</b>, and outputs the count value as the currently effective line No. data.

Referring to FIG. 5, the frame memory control circuit <b>14</b> controls the binary frame memory <b>15</b> to output data of a line to be output to the display device <b>17</b> on the basis of the rewriting flag and the line No. data received from the comparison circuit <b>18</b>. In this manner, since the LPF processing is performed upon comparison, and data are compared using a combination of a plurality of threshold values, stable detection is guaranteed even when noise is superimposed on an input image.

Note that various changes and modifications of the present invention may be made within the spirit and principal feature of the invention.

In other words, the foregoing description of embodiments has been given for illustrative purpose only and is not to be construed as imposing any limitation in any respect.

The scope of the invention is, therefore, to be determined solely by the following claims and not limited by the text of the specifications and alterations made within a scope equivalent to the scope of the claims fall within the true spirit and scope of the invention.