Apparatus and method for generating coefficient data, apparatus and method for processing information signal, program, and medium for recording the same

CROSS REFERENCES TO RELATED APPLICATIONS

The present application contains subject matters related to Japanese patent application no. 2004-124777 filed in the Japanese Patent Office on Apr. 20, 2004, the entire contents of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an apparatus and a method for generating coefficient data of an estimating equation used for converting a first information signal to a second information signal, to an apparatus and a method for converting the first information signal to the second information signal using the coefficient data, to a program for causing a computer to execute these methods, and to a medium for recording the program.

In particular, the present invention relates to an apparatus for generating coefficient data of an estimating equation for obtaining information data of the second information signal at a position of interest, in which coefficient seed data, which are coefficient data of a generation equation for generating the coefficient data of the estimating equation, are determined by using a plurality of training data items acquired from a teacher signal corresponding to the second information signal and a student signal corresponding to the first information signal and similarity of a student part of the plurality of training data items with respect to a part of the first information signal corresponding to the position of interest in the second information signal. The generation equation includes the similarity as a parameter. Thus, the present invention relates to an apparatus for generating coefficient data to obtain information data at the position of interest in the second information signal and for always providing the optimal coefficient data of the estimating equation, and therefore, providing improved quality of the output according to the second information signal.

2. Description of the Related Art

In recent years, a large number of technologies for improving the resolution or sampling frequency of image signals and audio signals have been proposed. For example, in order to upconvert a standard television signal having a standard resolution or low resolution to a high-resolution signal, known as an HDTV signal, or in order to carry out a sub-sampling interpolation operation on the standard television signal, it is known that an adaptive classification method provides a better result in performance compared to a known interpolation method.

In the adaptive classification method, in order to convert a standard television signal (SD signal) having a standard resolution or low resolution to a high-resolution signal (HD signal), a class to which pixel data at a pixel position of interest in the HD signal belongs is detected. Then, the pixel data at the pixel position in the HD signal is generated from a plurality of pixel data items in the SD signal using an estimating equation and coefficient data of the equation corresponding to the class. The coefficient data of the equation used in the conversion process including the classification is determined by learning for each class, for example, using a least-square method.

For example, as is disclosed in Japanese Unexamined Patent Application Publication No. 2002-218414, coefficient seed data, which is coefficient data of a generation equation including a parameter for adjusting resolution, is predetermined by learning for each class, for example, using a least-square method. Using the coefficient seed data and the parameter value, coefficient data of an estimating equation used for the conversion process including classification is obtained based on the generation equation.

As described above, coefficient data of an estimating equation for obtaining pixel data at a pixel position of interest in an HD signal corresponds to a class to which the pixel data at the pixel position of interest in the HD signal belongs. The coefficient data can provide average pixel data that belongs to the class in high accuracy, but is not very appropriate for an estimating equation to obtain pixel data at a pixel position of interest in an HD signal.

SUMMARY OF THE INVENTION

Accordingly, it is an object of the present invention to obtain, when converting a first information signal to a second information signal, the most appropriate coefficient data of an estimating equation for acquiring information data at a position of interest in the second information signal and to improve the quality of an output according to the second information signal.

According to the present invention, an apparatus generates coefficient data of an estimating equation used for converting a first information signal including a plurality of information data items to a second information signal including a plurality of information data items. The apparatus includes first data selection means for selecting, based on the first information signal, a plurality of information data items located around a position of interest in the second information signal; second data selection means for selecting, based on a first learning signal corresponding to the first information signal, a plurality of information data items located around a position of interest in a second learning signal corresponding to the second information signal; similarity determination means for acquiring the similarity of the plurality of information data items selected by the second data selection means with respect to the plurality of information data items selected by the first data selection means; third data selection means for selecting, based on the first learning signal, a plurality of information data items located around a position of interest in a second learning signal; first computing means for computing coefficient seed data using information data at each position of interest in the second learning signal, the plurality of information data items selected by the third data selection means corresponding to each position of interest, and a value of the similarity acquired by the similarity determination means corresponding to each position of interest, where the coefficient seed data is coefficient data of a generation equation for generating the coefficient data of the estimating equation, and the generation equation includes the similarity as a parameter; and second computing means for computing, based on the generation equation, the coefficient data of the estimating equation for determining the information data at the position of interest in the second information signal using the coefficient seed data computed by the first computing means and a value of the similarity that indicates the highest similarity.

According to the present invention, a method generates coefficient data of an estimating equation used for converting a first information signal including a plurality of information data items to a second information signal including a plurality of information data items. The method includes a first data selection step for selecting, based on the first information signal, a plurality of information data items located around a position of interest in the second information signal; a second data selection step for selecting, based on a first learning signal corresponding to the first information signal, a plurality of information data items located around a position of interest in a second learning signal corresponding to the second information signal; similarity determination step for acquiring the similarity of the plurality of information data items selected by the second data selection step with respect to the plurality of information data items selected by the first data selection step; a third data selection step for selecting, based on the first learning signal, a plurality of information data items located around a position of interest in a second learning signal; a first computing step for computing coefficient seed data using information data at each position of interest in the second learning signal, the plurality of information data items selected by the third data selection step corresponding to each position of interest, and a value of the similarity acquired by the similarity determination step corresponding to each position of interest, where the coefficient seed data is coefficient data of a generation equation for generating the coefficient data of the estimating equation, and the generation equation includes the similarity as a parameter; and a second computing step for computing, based on the generation equation, the coefficient data of the estimating equation for determining the information data at the position of interest in the second information signal using the coefficient seed data computed in the first computing step and a value of the similarity that indicates the highest similarity.

According to the present invention, a program includes program code for causing a computer to execute the above-described method for generating coefficient data. Additionally, according to the present invention, a computer-readable medium stores the program.

According to the present invention, when converting a first information signal to a second information signal, an estimating equation is used to determine information data at a position of interest in the second information signal, and coefficient data of the estimating equation are generated. Here, the information signal is, for example, an image signal and an audio signal. In the case of an image signal, information data are pixel data corresponding to each individual pixel. In the case of an audio signal, information data are sample data.

Based on the first information signal, a plurality of information data items around a position of interest in the second information signal (i.e., first comparison tap data) is selected.

Based on a first learning signal corresponding to the first information signal, a plurality of information data items around a position of interest in a second learning signal corresponding to the second information signal (i.e., second comparison tap data) is selected. For example, the second learning signal is stored in storage means, and the first learning signal is generated based on the second learning signal stored in the storage means. Alternatively, for example, the first information signal is used as the second learning signal and the first learning signal is generated based on the first information signal.

The similarity of the second comparison tap data selected from the first learning signal with respect to the first comparison tap data selected from the first information signal is acquired. For example, a sum of squared differences is obtained based on the first and second comparison tap data and is defined as the similarity. Alternatively, for example, a cross-correlation coefficient is obtained based on the first and second comparison tap data and is defined as the similarity.

Based on the first learning signal, a plurality of information data items around a position of interest in a second learning signal (i.e., prediction tap) is selected. Then, using information data at each position of interest, prediction tap data corresponding to each position of interest, and the similarity corresponding to each position of interest, coefficient seed data, which are coefficient data of a generation equation for generating coefficient data of an estimating equation, are determined. The generation equation includes the similarity as a parameter.

For example, using information data at each position of interest in a second learning signal, prediction tap data corresponding to each position of interest, and the similarity corresponding to each position of interest, a normal equation for determining coefficient seed data is generated, and then the normal equation is solved to determine the coefficient seed data.

Additionally, positions of interest in the second learning signal ST used for determining the coefficient seed data, for example, are selected from among positions of interest whose similarity values are in a predetermined range with respect to the similarity value that indicates the highest similarity. As will be described below, this can increase the accuracy of the coefficient seed data when acquiring the coefficient data by using the similarity value that indicates the highest similarity.

Subsequently, by using the coefficient seed data determined as described above, and the similarity value that indicates the highest similarity, coefficient data of an estimating equation for determining information data at the position of interest in the second information signal are determined based on the generation equation.

As described above, by using the first information signal as a second learning signal, the coefficient seed data are determined from many training data items whose first learning signal part resembles the part of the first information signal corresponding to the position of interest, that is, from many training data items that have high similarity. Accordingly, more accurate coefficient data for obtaining information data at the position of interest can be determined.

Thus, the determined coefficient data are always optimal as coefficient data of an estimating equation for obtaining the information data at a position of interest in the second information signal, and therefore, the quality of an output according to the second information signal can be improved.

According to the present invention, an information signal processing apparatus converts a first information signal including a plurality of information data items to a second information signal including a plurality of information data items. The information signal processing apparatus includes data selection means for selecting, based on the first information signal, a plurality of information data items located around a position of interest in the second information signal, coefficient data generation means for generating coefficient data of the estimating equation for determining information data of the second information signal at a position of interest, and computing means for computing, based on the estimating equation, the information data of the second information signal at the position of interest using the plurality of information data items selected by the data selection means and the coefficient data generated by the coefficient data generation means. The coefficient data generation means has an identical configuration to the above-described apparatus for generating coefficient data.

According to the present invention, an information signal processing method converts a first information signal including a plurality of information data items to a second information signal including a plurality of information data items. The information signal processing method includes a data selection step for selecting, based on the first information signal, a plurality of information data items located around a position of interest in the second information signal, a coefficient data generation step for generating coefficient data of the estimating equation for determining information data of the second information signal at a position of interest, and a computing step for computing, based on the estimating equation, the information data of the second information signal at the position of interest using the plurality of information data items selected by the data selection step and the coefficient data generated by the coefficient data generation step. The coefficient data generation step is identical to the step in the above-described method for generating coefficient data.

According to the present invention, a program includes program code for causing a computer to execute the above-described information signal processing method. Additionally, according to the present invention, a computer-readable medium stores the program.

According to the present invention, a first information signal is converted to a second information signal. That is, based on the first information signal, a plurality of information data items around a position of interest in the second information signal (i.e., prediction tap data) is selected. Coefficient data of an estimating equation for determining information data at a position of interest in the second information signal are then generated in the same manner as in the above-described apparatus or method for generating coefficient data. Subsequently, using the data of the prediction tap and the coefficient data, the information data at the position of interest in the second information signal are obtained based on the estimating equation. In this case, since optimal coefficient data of the estimating equation are always used, optimal information data at the position of interest in the second information signal are obtained. As a result, the quality of an output according to the second information signal can be improved.

According to the present invention, an apparatus generates coefficient data of an estimating equation used for converting a first information signal including a plurality of information data items to a second information signal including a plurality of information data items. The apparatus includes a first data selection unit configured to select, based on the first information signal, a plurality of information data items located around a position of interest in the second information signal; a second data selection unit configured to select, based on a first learning signal corresponding to the first information signal, a plurality of information data items located around a position of interest in a second learning signal corresponding to the second information signal; a similarity determination unit configured to acquire the similarity of the plurality of information data items selected by the second data selection unit with respect to the plurality of information data items selected by the first data selection unit; a third data selection unit configured to select, based on the first learning signal, a plurality of information data items located around a position of interest in the second learning signal; a first computing unit configured to compute coefficient seed data using information data at each position of interest in the second learning signal, the plurality of information data items selected by the third data selection unit corresponding to each position of interest, and a value of the similarity acquired by the similarity determination unit corresponding to each position of interest, where the coefficient seed data is used for generating the coefficient data of the estimating equation; and a second computing unit configured to compute, based on the generation equation, the coefficient data of the estimating equation for determining the information data at the position of interest in the second information signal using the coefficient seed data computed by the first computing unit and a value of the similarity that indicates the highest similarity.

According to the present invention, an information signal processing apparatus converts a first information signal including a plurality of information data items to a second information signal including a plurality of information data items. The information signal processing apparatus includes a data selection unit configured to select, based on the first information signal, a plurality of information data items located around a position of interest in the second information signal, a coefficient data generation unit configured to generate coefficient data of an estimating equation for determining information data of the second information signal at a position of interest, and a computing unit configured to compute, based on the estimating equation, the information data of the second information signal at the position of interest using the plurality of information data items selected by the data selection unit and the coefficient data generated by the coefficient data generation unit. The coefficient data generation unit has an identical configuration to the above-described apparatus for generating coefficient data.

That is, according to the present invention, when converting a first information signal to a second information signal, an estimating equation is used to obtain information data of the second information signal at a position of interest. To determine coefficient data of the estimating equation, coefficient seed data, which are coefficient data of a generation equation for generating the coefficient data of the estimating equation, are first determined by using a plurality of training data items acquired from a second learning signal corresponding to the second information signal and a first learning signal corresponding to the first information signal and similarity of the student part of the plurality of training data items with respect to a part of the first information signal corresponding to the position of interest in the second information signal. The generation equation includes the similarity as a parameter. Then, by using the coefficient seed data and a similarity value that indicates the highest similarity, the coefficient data of the estimating equation are determined based on the generation equation. Thus, optimal coefficient data of the estimating equation for obtaining information data at the position of interest in the second information signal are always provided. As a result, the quality of an output according to the second information signal can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram of an image signal processing apparatus according to a first embodiment of the present invention;

<figref idrefs="DRAWINGS">FIG. 2</figref> shows a relationship between positions of pixels of an SD signal (<b>525</b><i>i </i>signal) and an HD signal (<b>1050</b><i>i </i>signal);

<figref idrefs="DRAWINGS">FIG. 3</figref> shows an example of a pattern of a prediction tap;

<figref idrefs="DRAWINGS">FIG. 4</figref> shows an example of a pattern of a comparison tap;

<figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref> show phase differences of four pixels in a unit pixel block of an HD signal from the center prediction tap;

<figref idrefs="DRAWINGS">FIG. 6</figref> is a block diagram of a coefficient data generation unit;

<figref idrefs="DRAWINGS">FIG. 7</figref> is a block diagram of an image signal processing apparatus according to a second embodiment of the present invention;

<figref idrefs="DRAWINGS">FIG. 8</figref> is a block diagram of an image signal processing apparatus for implementing the present invention in software;

<figref idrefs="DRAWINGS">FIG. 9</figref> is a flow chart of the operation procedure of image signal processing; and

<figref idrefs="DRAWINGS">FIG. 10</figref> is a flow chart of the operation procedure of a coefficient data generation process.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A first embodiment of the present invention will be described next. <figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram of an image signal processing apparatus <b>100</b> according to the first embodiment. The image signal processing apparatus <b>100</b> converts an image signal Va, which is a standard definition (SD) signal called a <b>525</b><i>i </i>signal, to an image signal Vb, which is a high definition (HD) signal called a <b>1050</b><i>i </i>signal. Here, the image signal Va functions as a first information signal and the image signal Vb functions as a second information signal. The <b>525</b><i>i </i>signal is an interlaced image signal and has 525 lines per frame. The <b>1050</b><i>i </i>signal is an interlaced image signal and has 1050 lines per frame.

<figref idrefs="DRAWINGS">FIG. 2</figref> shows a relationship between pixels in a frame F having an SD (<b>525</b><i>i</i>) signal and an HD (<b>1050</b><i>i</i>) signal. Pixel positions in an odd field o are represented by solid lines and pixel positions of an even frame e are represented by dotted lines. The large dot is a pixel (SD pixel) of an SD signal and the small dot is a pixel (HD pixel) of an HD signal. As can be seen from <figref idrefs="DRAWINGS">FIG. 2</figref>, pixel data of an HD signal includes line data L<b>1</b> and L<b>1</b> at a position near a line of an SD signal and line data L<b>2</b> and L<b>2</b>′ at a position distant from a line of an SD signal. Here, L<b>1</b> and L<b>2</b> are line data in an odd field, and L<b>1</b>′ and L<b>2</b>′ are line data in an even field. The number of pixels in each line of an HD signal is twice that in each line of an SD signal.

Referring back to <figref idrefs="DRAWINGS">FIG. 1</figref>, the image signal processing apparatus <b>100</b> includes an input terminal <b>101</b>, a prediction tap selection unit <b>102</b>, and a comparison tap selection unit <b>103</b>. The input terminal <b>101</b> is used for inputting the image signal Va. The tap selection units <b>102</b> and <b>103</b> selectively extract, based on the image signal Va input to the input terminal <b>101</b>, a plurality of pixel data items around a pixel position of interest in the image signal Vb as data of a prediction tap and a comparison tap, respectively.

<figref idrefs="DRAWINGS">FIG. 3</figref> shows an example of a pattern of a plurality of pixel data items extracted as prediction tap data. In this example, eleven pixel data items X₁to X₁₁are extracted as prediction tap data. Here, <figref idrefs="DRAWINGS">FIG. 3</figref> shows an example in an odd field. y₁to y₄are pixel data items at a pixel position of interest in the image signal Vb. The position of pixel data item x₆is the position of a center prediction tap. In this case, a prediction tap is linked to an even field in addition to an odd field.

<figref idrefs="DRAWINGS">FIG. 4</figref> shows an example of a pattern of a plurality of pixel data items extracted as comparison tap data. In this example, five pixel data items xc₁to xc₅are extracted as comparison tap data. Here, <figref idrefs="DRAWINGS">FIG. 4</figref> shows an example in an odd field. y₁to y₄are pixel data items at a pixel position of interest in the image signal Vb. The position of pixel data item xc₃is the position of a center comparison tap. In this case, a comparison tap is only linked to an odd field.

Additionally, the image signal processing apparatus <b>100</b> includes a teacher image memory <b>104</b> functioning as storage means, and a coefficient data generation unit <b>105</b>. The teacher image memory <b>104</b> stores a teacher signal ST, which is an HD signal (<b>1050</b><i>i </i>signal) corresponding to the above-described image signal Vb. The coefficient data generation unit <b>105</b> generates coefficient data items Wi (i=1, . . . , n) of the following estimating equation (1) for determining pixel data at a pixel position of interest in the image signal Vb, which are used in an estimated prediction arithmetic unit <b>106</b>, using a plurality of pixel data items xci, which are data of a comparison tap extracted by the comparison tap selection unit <b>103</b>, and the teacher signal ST stored in the teacher image memory <b>104</b>. The estimated prediction arithmetic unit <b>106</b> will be described below. The coefficient data Wi is information for converting the image signal Va (<b>525</b><i>i </i>signal) to the image signal Vb (<b>1050</b><i>i </i>signal).

<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>y</mi><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><mrow><msub><mi>W</mi><mi>i</mi></msub><mo>·</mo><msub><mi>x</mi><mi>i</mi></msub></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

As shown in <figref idrefs="DRAWINGS">FIG. 2</figref>, when an SD signal (<b>525</b><i>i </i>signal) is converted to an HD signal (<b>1050</b><i>i </i>signal), four pixels of the HD signal must be obtained for one pixel of the SD signal for each odd field and even field. In this case, each of the four pixels in a 2-by-2 unit pixel block UB, which forms an HD signal in an odd field and an even frame, has a different phase difference from a center prediction tap thereof.

<figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref> show phase differences of four pixels in a 2-by-2 unit pixel block UB, which form an HD signal in an odd field and an even field respectively, from the center prediction tap. In the case of an odd field, the positions of four pixels HD<b>1</b> to HD<b>4</b> in the unit pixel block UB are shifted from a position of a center prediction tap SD<b>0</b> by k<b>1</b> to k<b>4</b> in the horizontal direction and by m<b>1</b> to m<b>4</b> in the vertical direction, respectively. In the case of an even field, the positions of four pixels HD<b>1</b>′ to HD<b>4</b>′ in the unit pixel block UB are shifted from a position of a center prediction tap SD<b>0</b>′ by k<b>1</b>′ to k<b>4</b>′ in the horizontal direction and by m<b>1</b>′ to m<b>4</b>′ in the vertical direction, respectively.

Thus, for each of an odd field and an even field, four pixels (HD<b>1</b> to HD<b>4</b>, or HD<b>1</b>′ to HD<b>4</b>′) exist at the pixel position of interest in the image signal Vb. Therefore, the above-described coefficient data items Wi generated by the coefficient data generation unit <b>105</b> consist of the coefficients data items Wi for the four pixels to determine the pixel data item y₁to y₄of the four pixels.

The coefficient data generation unit <b>105</b> determines the coefficient data items Wi of the estimating equation based on the following generation equation (2) including similarity R as a parameter. The similarity R will be described below in detail. <br /><i>Wi=w</i>_i0<i>+w</i>_i1<i>R+w</i>_i2<i>R</i>²<i>+w</i>_i3<i>R</i>³  (2)

A method for generating the coefficient data Wi will be described next.

First, coefficient seed data w_ij, which are coefficient data of the generation equation (2), are determined. Here, t_j(j=0 to 3) are defined as the following equations (3). <br />t₀=1, t₁=R, t₂=R², t₃=R³  (3)<br /> Using equations (3), equation (2) is rewritten as the following equation (4).

<maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>W</mi><mi>i</mi></msub><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>j</mi><mo>=</mo><mn>0</mn></mrow><mn>3</mn></munderover><mo>⁢</mo><mrow><msub><mi>w</mi><mi>ij</mi></msub><mo>·</mo><msub><mi>t</mi><mi>j</mi></msub></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>4</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

Finally, undetermined coefficients w_ijare determined by learning. That is, coefficients that minimize the squared error are found using a student signal SS corresponding to the image signal Va and a teacher signal ST corresponding to the image signal Vb. This solution is widely known as the least-square method.

Let the number of trainings be m, the residual error of the kth (1≦k≦m) training data item be e_k, and the total sum of the squared error be E. E is expressed as the following equation (5) using equations (1) and (2), where x_ikis a kth pixel data item at the ith prediction tap position and y_kis a kth pixel data item of the teacher signal corresponding to x_ik.

<maths id="MATH-US-00003" num="00003"><math overflow="scroll"><mtable><mtr><mtd><mtable><mtr><mtd><mrow><mi>E</mi><mo>=</mo><mi /><mo>⁢</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><msubsup><mi>e</mi><mi>k</mi><mn>2</mn></msubsup></mrow></mrow></mtd></mtr><mtr><mtd><mrow><mo>=</mo><mi /><mo>⁢</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><msup><mrow><mo>{</mo><mrow><msub><mi>y</mi><mi>k</mi></msub><mo>-</mo><mrow><mo>(</mo><mrow><mrow><msub><mi>W</mi><mn>1</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>x</mi><mrow><mn>1</mn><mo>⁢</mo><mi>K</mi></mrow></msub></mrow><mo>+</mo><mrow><msub><mi>W</mi><mn>2</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>x</mi><mrow><mn>2</mn><mo>⁢</mo><mi>K</mi></mrow></msub></mrow><mo>+</mo><mi>⋯</mi><mo>+</mo><mrow><msub><mi>W</mi><mi>n</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>x</mi><mi>nK</mi></msub></mrow></mrow><mo>)</mo></mrow></mrow><mo>}</mo></mrow><mn>2</mn></msup></mrow></mrow></mtd></mtr><mtr><mtd><mrow><mo>=</mo><mi /><mo>⁢</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><mrow><mo>{</mo><mrow><msub><mi>y</mi><mi>k</mi></msub><mo>-</mo><mrow><mo>[</mo><mrow><mrow><mrow><mo>(</mo><mrow><mrow><msub><mi>t</mi><mn>0</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>w</mi><mn>10</mn></msub></mrow><mo>+</mo><mrow><msub><mi>t</mi><mn>1</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>w</mi><mn>11</mn></msub></mrow><mo>+</mo><mi>⋯</mi><mo>+</mo><mrow><msub><mi>t</mi><mn>3</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>w</mi><mn>13</mn></msub></mrow></mrow><mo>)</mo></mrow><mo>⁢</mo><mstyle><mspace width="0.6em" height="0.6ex" /></mstyle><mo>⁢</mo><msub><mi>x</mi><mrow><mn>1</mn><mo>⁢</mo><mi>k</mi></mrow></msub></mrow><mo>+</mo><mi>⋯</mi><mo>+</mo></mrow></mrow></mrow></mrow></mrow></mrow></mtd></mtr><mtr><mtd><msup><mrow><mrow><mi /><mo>⁢</mo><mrow><mrow><mo>(</mo><mrow><mrow><msub><mi>t</mi><mn>0</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>w</mi><mi>n0</mi></msub></mrow><mo>+</mo><mrow><msub><mi>t</mi><mn>1</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>w</mi><mi>n1</mi></msub></mrow><mo>+</mo><mi>⋯</mi><mo>+</mo><mrow><msub><mi>t</mi><mn>3</mn></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>w</mi><mi>n3</mi></msub></mrow></mrow><mo>)</mo></mrow><mo>⁢</mo><mstyle><mspace width="0.6em" height="0.6ex" /></mstyle><mo>⁢</mo><msub><mi>x</mi><mi>nk</mi></msub></mrow><mo>]</mo></mrow><mo>}</mo></mrow><mn>2</mn></msup></mtd></mtr></mtable></mtd><mtd><mrow><mo>(</mo><mn>5</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

According to the solution of the least-square method, W_ijis determined such that the result of partial-differentiation of equation (5) with respect to w_ijis 0. This is expressed by the equation (6).

<maths id="MATH-US-00004" num="00004"><math overflow="scroll"><mtable><mtr><mtd><mrow><mfrac><mrow><mo>∂</mo><mi>E</mi></mrow><mrow><mo>∂</mo><msub><mi>w</mi><mi>ij</mi></msub></mrow></mfrac><mo>=</mo><mrow><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><mrow><mn>2</mn><mo>⁢</mo><mrow><mo>(</mo><mfrac><mrow><mo>∂</mo><msub><mi>e</mi><mi>k</mi></msub></mrow><mrow><mo>∂</mo><msub><mi>w</mi><mi>ij</mi></msub></mrow></mfrac><mo>)</mo></mrow><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>e</mi><mi>k</mi></msub></mrow></mrow><mo>=</mo><mrow><mrow><mo>-</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><mrow><mn>2</mn><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>t</mi><mi>j</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>x</mi><mi>ik</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>e</mi><mi>k</mi></msub></mrow></mrow></mrow><mo>=</mo><mn>0</mn></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>6</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

When X_ipjq, and Y_ipare defined by the following equations (7) and (8), equation (6) is rewritten as equation (9) using a matrix.

<maths id="MATH-US-00005" num="00005"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>X</mi><mi>ipjq</mi></msub><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><mrow><msub><mi>x</mi><mi>ik</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>t</mi><mi>p</mi></msub><mo>⁢</mo><msub><mi>x</mi><mi>jk</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>t</mi><mi>q</mi></msub></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>7</mn><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mrow><msub><mi>Y</mi><mi>ip</mi></msub><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover><mo>⁢</mo><mrow><msub><mi>x</mi><mi>ik</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>t</mi><mi>p</mi></msub><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><msub><mi>y</mi><mi>k</mi></msub></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>8</mn><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mrow><mrow><mrow><mo>[</mo><mtable><mtr><mtd><msub><mi>x</mi><mn>1010</mn></msub></mtd><mtd><msub><mi>x</mi><mn>1011</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mn>1013</mn></msub></mtd><mtd><msub><mi>x</mi><mn>1020</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mrow><mn>10</mn><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr><mtr><mtd><msub><mi>x</mi><mn>1110</mn></msub></mtd><mtd><msub><mi>x</mi><mn>1111</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mn>1113</mn></msub></mtd><mtd><msub><mi>x</mi><mn>1120</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mrow><mn>11</mn><mo>⁢</mo><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr><mtr><mtd><mi>⋮</mi></mtd><mtd><mi>⋮</mi></mtd><mtd><mi>⋰</mi></mtd><mtd><mi>⋮</mi></mtd><mtd><mi>⋮</mi></mtd><mtd><mi>⋰</mi></mtd><mtd><mi>⋮</mi></mtd></mtr><mtr><mtd><msub><mi>x</mi><mn>1310</mn></msub></mtd><mtd><msub><mi>x</mi><mn>1311</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mn>1313</mn></msub></mtd><mtd><msub><mi>x</mi><mn>1320</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mrow><mn>13</mn><mo>⁢</mo><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr><mtr><mtd><msub><mi>x</mi><mn>2010</mn></msub></mtd><mtd><msub><mi>x</mi><mn>2011</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mn>2013</mn></msub></mtd><mtd><msub><mi>x</mi><mn>2020</mn></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mrow><mn>20</mn><mo>⁢</mo><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr><mtr><mtd><mi>⋮</mi></mtd><mtd><mi>⋮</mi></mtd><mtd><mi>⋰</mi></mtd><mtd><mi>⋮</mi></mtd><mtd><mi>⋮</mi></mtd><mtd><mi>⋰</mi></mtd><mtd><mi>⋮</mi></mtd></mtr><mtr><mtd><msub><mi>x</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>310</mn></mrow></msub></mtd><mtd><msub><mi>x</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>311</mn></mrow></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>313</mn></mrow></msub></mtd><mtd><msub><mi>x</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>320</mn></mrow></msub></mtd><mtd><mi>⋯</mi></mtd><mtd><msub><mi>x</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn><mo>⁢</mo><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr></mtable><mo>]</mo></mrow><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>[</mo><mtable><mtr><mtd><msub><mi>w</mi><mn>10</mn></msub></mtd></mtr><mtr><mtd><msub><mi>w</mi><mn>11</mn></msub></mtd></mtr><mtr><mtd><mi>⋮</mi></mtd></mtr><mtr><mtd><msub><mi>w</mi><mn>13</mn></msub></mtd></mtr><mtr><mtd><msub><mi>w</mi><mn>20</mn></msub></mtd></mtr><mtr><mtd><mi>⋮</mi></mtd></mtr><mtr><mtd><msub><mi>w</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr></mtable><mo>]</mo></mrow><mo>=</mo><mrow><mo>[</mo><mtable><mtr><mtd><msub><mi>Y</mi><mn>10</mn></msub></mtd></mtr><mtr><mtd><msub><mi>Y</mi><mn>11</mn></msub></mtd></mtr><mtr><mtd><mi>⋮</mi></mtd></mtr><mtr><mtd><msub><mi>Y</mi><mn>13</mn></msub></mtd></mtr><mtr><mtd><msub><mi>Y</mi><mn>20</mn></msub></mtd></mtr><mtr><mtd><mi>⋮</mi></mtd></mtr><mtr><mtd><msub><mi>Y</mi><mrow><mi>n</mi><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo>⁢</mo><mn>3</mn></mrow></msub></mtd></mtr></mtable><mo>]</mo></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>9</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

Equation (9) is a normal equation for calculating the coefficient seed data w_ij. By solving this normal equation using a process of elimination, such as Gauss-Jordan elimination, the coefficient seed data w_ijare obtained.

Subsequently, the coefficient data Wi are determined based on the generation equation (2) by using the coefficient seed data w_ijand the value of the similarity R that indicates the highest similarity.

The coefficient data generation unit <b>105</b> will be further described next. <figref idrefs="DRAWINGS">FIG. 6</figref> is a block diagram of the coefficient data generation unit <b>105</b>. The coefficient data generation unit <b>105</b> generates coefficient data Wi based on the above-described method for generating coefficient data Wi.

The coefficient data generation unit <b>105</b> includes a student image generation unit <b>111</b>. The student image generation unit <b>111</b> carries out vertical and horizontal decimation on the teacher signal ST read out from the teacher image memory <b>104</b> to generate a student signal SS, which is an SD signal (<b>525</b><i>i </i>signal), corresponding to the image signal Va.

The coefficient data generation unit <b>105</b> also includes a prediction tap selection unit <b>112</b> and a comparison tap selection unit <b>113</b>. These tap selection units <b>112</b> and <b>113</b> selectively extract, based on the student signal SS generated by the student image generation unit <b>111</b>, a plurality of pixel data items around a pixel position of interest in the teacher signal ST as data of a prediction tap and an comparison tap, respectively. The tap selection units <b>112</b> and <b>113</b> correspond to the above-described tap selection units <b>102</b> and <b>103</b>, respectively.

The coefficient data generation unit <b>105</b> also includes a similarity determination unit <b>114</b>. The similarity determination unit <b>114</b> acquires the similarity R of a plurality of pixel data items sci, which is data of the comparison tap extracted by the comparison tap selection unit <b>113</b>, with respect to a plurality of pixel data items xci, which is data of the comparison tap extracted by the comparison tap selection unit <b>103</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref>.

For example, the similarity determination unit <b>114</b> obtains the sum of squared differences between a plurality of pixel data items sci and a plurality of pixel data items xci, as shown in the following equation (10). The similarity determination unit <b>114</b> regards the sum of squared differences as the similarity R. In equation (10), n=5 if the data of the comparison tap includes five pixel data items, as shown in <figref idrefs="DRAWINGS">FIG. 4</figref>.

<maths id="MATH-US-00006" num="00006"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>R</mi><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><msup><mrow><mo>(</mo><mrow><mi>xci</mi><mo>-</mo><mi>sci</mi></mrow><mo>)</mo></mrow><mn>2</mn></msup></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>10</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

Additionally, the similarity determination unit <b>114</b> obtains, for example, a cross-correlation coefficient and regards it as the similarity R, as shown in the following-equation (11). In equation (11), n=5 if the data of the comparison tap includes five pixel data items, as shown in <figref idrefs="DRAWINGS">FIG. 4</figref>.

<maths id="MATH-US-00007" num="00007"><math overflow="scroll"><mtable><mtr><mtd><mtable><mtr><mtd><mrow><mi>R</mi><mo>=</mo><mfrac><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><mrow><mrow><mo>(</mo><mrow><mi>xci</mi><mo>-</mo><mover><mi>xc</mi><mi>_</mi></mover></mrow><mo>)</mo></mrow><mo>⁢</mo><mstyle><mspace width="0.6em" height="0.6ex" /></mstyle><mo>⁢</mo><mrow><mo>(</mo><mrow><mi>sci</mi><mo>-</mo><mover><mi>sc</mi><mi>_</mi></mover></mrow><mo>)</mo></mrow></mrow></mrow><mrow><msqrt><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><msup><mrow><mo>(</mo><mrow><mi>xci</mi><mo>-</mo><mover><mi>xc</mi><mi>_</mi></mover></mrow><mo>)</mo></mrow><mn>2</mn></msup></mrow></msqrt><mo>⁢</mo><mstyle><mspace width="0.6em" height="0.6ex" /></mstyle><mo>⁢</mo><msqrt><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><msup><mrow><mo>(</mo><mrow><mi>sci</mi><mo>-</mo><mover><mi>sc</mi><mi>_</mi></mover></mrow><mo>)</mo></mrow><mn>2</mn></msup></mrow></msqrt></mrow></mfrac></mrow></mtd></mtr><mtr><mtd><mrow><mo>(</mo><mrow><mrow><mrow><mi>where</mi><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><mover><mi>xc</mi><mi>_</mi></mover></mrow><mo>=</mo><mfrac><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><mi>xci</mi></mrow><mi>n</mi></mfrac></mrow><mo>,</mo><mrow><mover><mi>sc</mi><mi>_</mi></mover><mo>=</mo><mfrac><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>⁢</mo><mi>sci</mi></mrow><mi>n</mi></mfrac></mrow></mrow><mo>)</mo></mrow></mtd></mtr></mtable></mtd><mtd><mrow><mo>(</mo><mn>11</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>

The coefficient data generation unit <b>105</b> also includes a teacher tap selection unit <b>115</b>. The teacher tap selection unit <b>115</b> selectively extracts, based on the teacher signal ST, a pixel data item y at a pixel position of interest in the teacher signal ST.

The coefficient data generation unit <b>105</b> also includes a normal equation generation unit <b>116</b>. The normal equation generation unit <b>116</b> generates a normal equation for determining the coefficient seed data w_ij(refer to equation (9)), which are the coefficient data of the generation equation (2), from pixel data y at each pixel position of interest in the teacher signal ST extracted by the teacher tap selection unit <b>115</b>, a plurality of pixel data items xi, which are data of the prediction tap and which are selectively extracted by the prediction tap selection unit <b>112</b> corresponding to the pixel data y at each pixel position of interest, and values of the similarity R acquired by the similarity determination unit <b>114</b> corresponding to the pixel data y.

In this case, a combination of one pixel data item y and a plurality of pixel data items xi corresponding to the pixel data item y forms one training data item. Many training data items having different similarity values between the teacher signal ST and the student signal SS corresponding to the teacher signal ST are generated. Thus, the normal equation generation unit <b>116</b> generates a normal equation including many training data items.

Also, in this case, the normal equation generation unit <b>116</b> generates normal equations for four pixels (i.e., HD<b>1</b> to HD<b>4</b>, or HD<b>1</b>′ to HD<b>4</b>′ in <figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref>). The normal equations corresponding to HD<b>1</b> to HD<b>4</b> or HD<b>1</b>′ to HD<b>4</b>′ are generated using only training data items generated by a pixel data item y having the same shift values from the center prediction taps SD<b>0</b> and SD<b>0</b>′ as HD<b>1</b> to HD<b>4</b> or HD<b>1</b>′ to HD<b>4</b>′, respectively.

The normal equation generation unit <b>116</b> may generate a normal equation using training data items corresponding to all pixel positions of interest in the teacher signal ST. Alternatively, positions of interest in the teacher signal ST for generating a normal equation may be selected from among pixel positions of interest whose values of the similarity R acquired by the similarity determination unit <b>114</b> are in a predetermined range with respect to the value of the similarity R that indicates the highest similarity. As will be described below, this can increase the accuracy of the coefficient seed data w_ijwhen acquiring the coefficient data Wi by using the value of the similarity R that indicates the highest similarity.

For example, when the similarity R is defined as a sum of squared differences shown in Equation (10), the value of the similarity R that indicates the highest similarity is 0. Accordingly, only pixel positions of interest having a value of the similarity R smaller than or equal to a predetermined value may be used to generate a normal equation. Additionally, for example, when the similarity R is defined as a cross-correlation coefficient shown in Equation (11), the similarity R is in the range from −1 to 1 and the value of the similarity R that indicates the highest similarity is 1. Accordingly, only pixel positions of interest having a value of the similarity R greater than or equal to 0.5 may be used to generate a normal equation.

The coefficient data generation unit <b>105</b> also includes a coefficient seed data computing unit <b>117</b> and a coefficient data computing unit <b>118</b>. The coefficient seed data computing unit <b>117</b> receives data of a normal equation from the normal equation generation unit <b>116</b>, solves the normal equation by a method such as elimination, and determines coefficient seed data w_ijfor four pixels. The coefficient data computing unit <b>118</b> determines coefficient data Wi for the four pixels based on the generation equation (2) using the coefficient seed data w_ijfor the four pixels determined by the coefficient seed data computing unit <b>117</b> and the value of the similarity R that indicates the highest similarity.

The operation of the coefficient data generation unit <b>105</b> shown in <figref idrefs="DRAWINGS">FIG. 6</figref> will be described next.

The student image generation unit <b>111</b> carries out vertical and horizontal decimation on a teacher signal ST to generate a student signal SS. The comparison tap selection unit <b>113</b> selectively extracts a plurality of pixel data items sci around a pixel position of interest in the teacher signal ST as comparison tap data. The plurality of pixel data items sci is supplied to the similarity determination unit <b>114</b>. Also, a plurality of pixel data items xci selectively extracted by the comparison tap selection unit <b>103</b> (refer to <figref idrefs="DRAWINGS">FIG. 1</figref>) is supplied to the similarity determination unit <b>114</b> as comparison tap data.

The similarity determination unit <b>114</b> acquires the similarity R of the plurality of pixel data items sci with respect to the plurality of pixel data items xci. In this case, for example, a sum of squared differences or cross-correlation coefficient is determined based on the plurality of pixel data items sci and the plurality of pixel data items xci, and is used as the similarity R (refer to equations (10) and (11)). The determined value of the similarity R is supplied to the normal equation generation unit <b>116</b>.

The prediction tap selection unit <b>112</b> selectively extracts a plurality of pixel data items xi around a pixel position of interest in the teacher signal ST as prediction tap data. The plurality of pixel data items xi is supplied to the normal equation generation unit <b>116</b>. On the other hand, the teacher tap selection unit <b>115</b> selectively extracts pixel data y at the pixel position of interest in the teacher signal ST based on the teacher signal ST. The pixel data y is supplied to the normal equation generation unit <b>116</b>.

The normal equation generation unit <b>116</b> generates a normal equation (refer to equation (9)) for determining coefficient seed data W_ijfor the four pixels (i.e., HD<b>1</b> to HD<b>4</b>, or HD<b>1</b>′ to HD<b>4</b>′ in <figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref>) (refer to the equation (2)) from pixel data y at each pixel position of interest in the teacher signal ST extracted by the teacher tap selection unit <b>115</b>, a plurality of pixel data items xi, which are prediction tap data selectively extracted by the prediction tap selection unit <b>112</b> corresponding to the pixel data y at each pixel position of interest, and the values of the similarity R acquired by the similarity determination unit <b>114</b> corresponding to the pixel data y at each pixel position of interest.

The coefficient data generation unit <b>105</b> receives data of the normal equation from the normal equation generation unit <b>116</b>, solves the normal equation by using a method such as elimination, and determines coefficient seed data W_ijfor the four pixels. The coefficient seed data W_ijfor the four pixels are supplied to the coefficient data computing unit <b>118</b>, which determines coefficient data Wi for the 4 pixels based on the generation equation (2) using the coefficient seed data W_ijfor the four pixels and the value of the similarity R that indicates the highest similarity.

Referring back to <figref idrefs="DRAWINGS">FIG. 1</figref>, the image signal processing apparatus <b>100</b> includes the estimated prediction arithmetic unit <b>106</b>, a post-processing unit <b>107</b>, and an output terminal <b>108</b>. The estimated prediction arithmetic unit <b>106</b> determines pixel data y at the pixel position of interest in the image signal Vb based on the estimating equation (1) using a plurality of pixel data items xi, which are data of the prediction tap selectively extracted by the prediction tap selection unit <b>102</b>, and the coefficient data Wi generated by the coefficient data generation unit <b>105</b>.

As described above, to convert an SD signal (<b>525</b><i>i </i>signal) to an HD signal (<b>1050</b><i>i </i>signal), four pixels of the HD signal (i.e., HD<b>1</b> to HD<b>4</b>, or HD<b>1</b>′ to HD<b>4</b>′ in <figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref> ) must be obtained for one pixel of the SD signal (i.e., SD<b>0</b> or SD<b>0</b>′ in <figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref> ) for each field. The estimated prediction arithmetic unit <b>106</b> acquires pixel data for each unit pixel block UB located at the pixel position of interest in the image signal Vb.

That is, the estimated prediction arithmetic unit <b>106</b> receives data xi of the prediction tap which corresponds to four pixels (pixels of interest) in the unit pixel block UB from the prediction tap selection unit <b>102</b>. Also, the estimated prediction arithmetic unit <b>106</b> receives coefficient data Wi for four pixels corresponding to the four pixels in the unit pixel block UB from the coefficient data generation unit <b>105</b>. Then, the estimated prediction arithmetic unit <b>106</b> individually computes pixel data items y₁to y₄of the four pixels that define the unit pixel block UB using the estimating equation (1).

The post-processing unit <b>107</b> arranges pixel data items y₁to y₄in the unit pixel block UB sequentially output from the estimated prediction arithmetic unit <b>106</b> in a line-at-a-time form and outputs them in a <b>1050</b><i>i </i>signal format. The output terminal <b>108</b> is used to output the image signal Vb (<b>1050</b><i>i </i>signal) received from the post-processing unit <b>107</b>.

The operation of the image signal processing apparatus <b>100</b> will be described next.

The image signal Va, which is an SD signal, is input to the input terminal <b>101</b>. The comparison tap selection unit <b>103</b> selectively extracts, based on the image signal Va input to the input terminal <b>101</b>, a plurality of pixel data items xci around the pixel position of interest in the image signal Vb as comparison tap data. The plurality of pixel data items xci is supplied to the coefficient data generation unit <b>105</b>.

By using the plurality of pixel data items xci and a teacher signal ST stored in the teacher image memory <b>104</b>, the coefficient data generation unit <b>105</b> generates coefficient data Wi (i=1, . . . , n) of the estimating equation (1) to acquire pixel data at a pixel position of interest in the image signal Vb. The pixel data is used by the estimated prediction arithmetic unit <b>106</b>. In this case, the coefficient data generation unit <b>105</b> generates coefficient data Wi for four pixels corresponding to the four pixels in the unit pixel block UB located at the pixel position of interest in the image signal Vb. The coefficient data Wi are supplied to the estimated prediction arithmetic unit <b>106</b>.

The prediction tap selection unit <b>102</b> selectively extracts, based on the image signal Va, a plurality of pixel data items xi around the pixel position of interest in the image signal Vb as prediction tap data. The plurality of pixel data items xi is supplied to the estimated prediction arithmetic unit <b>106</b>. Then, the estimated prediction arithmetic unit <b>106</b> individually computes pixel data items y₁to y₄of the four pixels (pixels of interest) in the unit pixel block UB based on the estimating equation (1) using the plurality of pixel data items xi and the coefficient data Wi for the four pixels.

The pixel position of interest in the image signal Vb sequentially moves. As it moves, the estimated prediction arithmetic unit <b>106</b> sequentially outputs pixel data items y₁to y₄of the four pixels (pixels of interest) in the unit pixel block UB, which are supplied to the post-processing unit <b>107</b>. The post-processing unit <b>107</b> arranges the pixel data items y₁to y₄in the unit pixel block UB sequentially output from the estimated prediction arithmetic unit <b>106</b> in a line-at-a-time form and outputs them in a <b>1050</b><i>i </i>signal format. That is, the post-processing unit <b>107</b> outputs the image signal Vb (<b>1050</b><i>i </i>signal), which is output to the output terminal <b>108</b>.

As described above, the coefficient data Wi of the estimating equation (1) for obtaining pixel data items y₁to y₄at the pixel position of interest is determined as follows. That is, first, by using a plurality of training data items acquired from a teacher signal, which corresponds to the image signal Vb, and a student signal SS, which corresponds to the image signal Va, and similarity R of the student signal SS in the plurality of training data items with respect to the image signal Va corresponding to the pixel position of interest in the image signal Vb, coefficient seed data w_ijof a generation equation including the similarity R as a parameter are determined. The coefficient seed data w_ijare used to generate the coefficient data Wi of the estimating equation. Then, using the coefficient seed data w_ijand the value of the similarity R that indicates the highest similarity, the coefficient data Wi are obtained based on the generation equation (2).

Consequently, according to the image signal processing apparatus <b>100</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref>, optimal coefficient data Wi of an estimating equation for obtaining pixel data items y₁to y₄at the pixel position of interest in the image signal Vb can always be determined, and therefore, the estimated prediction arithmetic unit <b>106</b> can provide optimal pixel data items y₁to y₄at the pixel position of interest. As a result, the quality of an image according to the image signal Vb can be improved.

A second embodiment of the present invention will be described next. <figref idrefs="DRAWINGS">FIG. 7</figref> is a block diagram of an image signal processing apparatus <b>100</b>A according to the second embodiment. In <figref idrefs="DRAWINGS">FIG. 7</figref>, the same components as those illustrated and described in relation to <figref idrefs="DRAWINGS">FIG. 1</figref> are designated by the same reference numerals, and detailed descriptions thereof are not included hereinafter.

Unlike the image signal processing apparatus <b>100</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref>, the image signal processing apparatus <b>100</b>A has no teacher image memory <b>104</b>. In the image signal processing apparatus <b>100</b>A, an image signal Va input to the input terminal <b>101</b> is supplied to a coefficient data generation unit <b>105</b> as a teacher signal ST. Other components of the image signal processing apparatus <b>100</b>A are identical to those of the image signal processing apparatus <b>100</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref>.

In the image signal processing apparatus <b>100</b>A, a student image generation unit <b>111</b> (refer to <figref idrefs="DRAWINGS">FIG. 6</figref>) of the coefficient data generation unit <b>105</b> carries out vertical and horizontal decimation on the teacher signal ST to generate a student signal SS. Thus, the coefficient data generation unit <b>105</b> can use the teacher signal ST and the student signal SS, which have the same relationship as described in the description of the image signal processing apparatus <b>100</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref>, so as to generate coefficient data Wi in the same manner as the image signal processing apparatus <b>100</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref>.

According to the image signal processing apparatus <b>100</b>A, a teacher image memory <b>104</b> for storing a teacher signal ST can be removed, and therefore, the image signal processing apparatus <b>100</b>A can be configured simply and at low cost. Furthermore, since the image signal processing apparatus <b>100</b>A uses the image signal Va as a teacher signal ST, the coefficient data generation unit <b>105</b> can determine the coefficient seed data w_ijassociated with coefficient data Wi for obtaining pixel data items y₁to y₄at the pixel position of interest in the image signal Vb by using many training data items whose student signal part resembles the part of the image signal Va corresponding to the pixel position of interest, that is, by using many training data items that have high similarity R. Accordingly, more accurate coefficient data Wi corresponding to the pixel position of interest can be obtained.

Additionally, the processes carried out by the image signal processing apparatuses <b>100</b> and <b>100</b>A shown in <figref idrefs="DRAWINGS">FIGS. 1 and 7</figref> can be achieved by software, for example, in an image signal processing apparatus <b>500</b> shown in <figref idrefs="DRAWINGS">FIG. 8</figref>.

First, the image signal processing apparatus <b>500</b> will be described with reference to <figref idrefs="DRAWINGS">FIG. 8</figref>. The image signal processing apparatus <b>500</b> includes a central processing unit (CPU) <b>501</b> for controlling the operation of the whole apparatus, a read only memory (ROM) <b>502</b> for storing a control program for the CPU <b>501</b>, and a random access memory (RAM) <b>503</b> for serving as a work area of the CPU <b>501</b>. The CPU <b>501</b>, the ROM <b>502</b>, and the RAM <b>503</b> are connected to a bus <b>504</b>.

The image signal processing apparatus <b>500</b> also includes a hard disk drive (HDD) <b>505</b> serving as an external storage device, and a drive <b>506</b> for handling a removable recording medium, such as an optical disk, a magnetic disk, and a memory card. The drives <b>505</b> and <b>506</b> are connected to the bus <b>504</b>. For example, the teacher signal ST is stored in the HDD <b>505</b> in advance.

The image signal processing apparatus <b>500</b> also includes a communication unit <b>508</b> for connecting to a communication network <b>507</b>, such as the Internet, either wired or wirelessly. The communication unit <b>508</b> is connected to the bus <b>504</b> via an interface <b>509</b>.

The image signal processing apparatus <b>500</b> also includes a user interface unit. The user interface unit includes a remote-control signal receiving circuit <b>511</b> for receiving a remote-control signal RM from a remote-control transmitter <b>510</b> and a display <b>513</b>, such as a liquid crystal display (LCD) and a plasma display panel (PDP). The remote-control signal receiving circuit <b>511</b> is connected to the bus <b>504</b> via an interface <b>512</b>. Similarly, the display <b>513</b> is connected to the bus <b>504</b> via an interface <b>514</b>.

The image signal processing apparatus <b>500</b> also includes an input terminal <b>515</b> for inputting an image signal Va, which is an SD signal, and an output terminal <b>517</b> for outputting an image signal Vb, which is an HD signal. The input terminal <b>515</b> is connected to the bus <b>504</b> via an interface <b>516</b>. Similarly, the output terminal <b>517</b> is connected to the bus <b>504</b> via an interface <b>518</b>.

Instead of storing the control program in the ROM <b>502</b> in advance, as described above, the control program may be downloaded via the communication unit <b>508</b> over the communication network <b>507</b>, for example, over the Internet, and may be stored in the HDD <b>505</b> and the RAM <b>503</b>. Alternatively, the control program may be provided by a removable recording medium.

Additionally, instead of inputting an image signal Va to be processed from the input terminal <b>515</b>, the image signal Va may be stored in the HDD <b>505</b> in advance or may be downloaded via the communication unit <b>508</b> over the communication network <b>507</b>, for example, over the Internet. Furthermore, instead of or in addition to outputting a processed image signal Vb to the output terminal <b>517</b>, the image signal Vb may be supplied to the display <b>513</b> to display an image, may be stored in the HDD <b>505</b>, and may be transmitted to the communication network <b>507</b>, such as the Internet, via the communication unit <b>508</b>.

The operation procedure of the image signal processing apparatus <b>500</b> shown in <figref idrefs="DRAWINGS">FIG. 8</figref> when obtaining an image signal Vb from an image signal Va will be described with reference to a flow chart of <figref idrefs="DRAWINGS">FIG. 9</figref>.

First, the process starts at step S<b>10</b>. At step S<b>11</b>, an image signal Va for one frame or one field is input to the apparatus from, for example, the input terminal <b>515</b>. Such image signal Va is temporarily stored in the RAM <b>503</b>. In the case where the image signal Va is stored in the HDD <b>505</b> in advance, the image signal Va is read out from the HDD <b>505</b> and is temporarily stored in the RAM <b>503</b>.

At step S<b>12</b>, it is determined whether the process of the image signal Va for the entire frame or entire field is completed. If completed, the entire process is completed at step S<b>13</b>. If not completed, the process proceeds to step S<b>14</b>.

At step S<b>14</b>, based on the image signal Va, a plurality of pixel data items xci around a pixel position of interest in the image signal Vb is acquired as comparison tap data. At step S<b>15</b>, coefficient data Wi of an estimating equation (refer to equation (1)) are generated for determining pixel data items y₁to y₄at the pixel position of interest.

<figref idrefs="DRAWINGS">FIG. 10</figref> shows the operation procedure of the coefficient data generation process at step S<b>15</b>. That is, the process starts at step S<b>20</b>. At step S<b>21</b>, a teacher signal ST for one frame or one field is input to the apparatus. In this case, the teacher signal ST, for example, stored in the HDD <b>505</b> is read out and is temporarily stored in the RAM <b>503</b>. When the image signal Va already stored at the above-described step S<b>11</b> is used as the teacher signal ST, this process at step S<b>21</b> is eliminated.

Subsequently, at step S<b>22</b>, it is determined whether the process of the teacher signal ST for the entire frame or entire field is completed. If not completed, vertical and horizontal decimation, at step S<b>23</b>, is carried out on the teacher signal ST input at step S<b>21</b> to generate a student signal SS.

At step S<b>24</b>, based on the student signal SS, a plurality of pixel data items sci around the pixel position of interest in the teacher signal ST is acquired as comparison tap data. At step S<b>25</b>, the similarity R of the plurality of pixel data items sci acquired at step S<b>24</b> with respect to the plurality of pixel data items xci acquired at step S<b>14</b> is computed. For example, a sum of squared differences (refer to equation (10)) is determined using the pixel data items xci and sci and is defined as the similarity R. Alternatively, for example, a cross-correlation coefficient (refer to equation (11)) is determined using the pixel data items xci and sci and is defined as the similarity R.

Thereafter, at step S<b>26</b>, it is determined whether the similarity R satisfies a predetermined condition. That is, it is determined whether the value of the similarity R is within a predetermined range with respect to the value that indicates the highest similarity. For example, when a sum of squared differences is defined as the similarity R, it is determined whether the value of the similarity R is smaller than or equal to a predetermined value, since the value that indicates the highest similarity is 0. When a cross-correlation coefficient is defined as the similarity R, it is determined whether the value of the similarity R is greater than or equal to 0.5, since the similarity R is in the range from −1 to 1 and the value of the similarity R that indicates the highest similarity is 1.

If the similarity R satisfies the condition, the process proceeds to step S<b>27</b>. However, if the similarity R does not satisfy the condition, the process returns to step S<b>24</b>, where the process moves to a process on the next pixel position of interest in the teacher signal ST. The process at step S<b>26</b> is not always necessary. However, as will be described below, this step increases the accuracy of the coefficient seed data w_ijwhen the coefficient data Wi is determined using the value of the similarity R that indicates the highest similarity.

At step S<b>27</b>, a plurality of pixel data items xi around a position of a pixel of interest in the teacher signal ST is acquired as prediction tap data based on the student signal SS. At step S<b>28</b>, pixel data y at a position of a pixel of interest in the teacher signal ST is acquired as teacher tap data based on the teacher signal ST.

At step S<b>29</b>, the summation (refer to equations (7) and (8)) is carried out to obtain a normal equation, shown as equation 9, using the similarity R computed at step S<b>25</b>, a plurality of pixel data items xi acquired at step S<b>27</b>, and the pixel data y acquired at step S<b>28</b>. In this case, normal equations for four pixels (i.e., HD<b>1</b> to HD<b>4</b>, or HD<b>1</b>′ to HD<b>4</b>′ in <figref idrefs="DRAWINGS">FIGS. 5A and 5B</figref>) are generated concurrently.

At step S<b>30</b>, it is determined whether a learning process for the entire region of the pixel data in the teacher signal ST for one frame or one field input at step S<b>21</b> is completed. If the learning process is completed, the process returns to step S<b>21</b>, where a teacher signal ST for the next frame or next field is input. Then, the above-described process is repeated. If the learning process is not completed, the process returns to step S<b>24</b>. Subsequently, the next pixel position of interest in the teacher signal ST is processed.

If, at step S<b>22</b>, the process for the teacher signal ST for the entire frame or field is completed, the process proceeds to step S<b>31</b>. At step S<b>31</b>, the normal equations, which are generated by summation at step S<b>29</b>, are solved to compute coefficient seed data w_ijfor the four pixels. Thereafter, at step S<b>32</b>, the coefficient data Wi is computed based on the generation equation (refer to equation (2)) using the coefficient seed data w_ijfor the four pixels computed at step S<b>31</b> and the value of the similarity R that indicates the highest similarity. At step S<b>33</b>, the process is then completed.

Referring back to <figref idrefs="DRAWINGS">FIG. 9</figref>, after the process of step S<b>15</b> is completed, the process proceeds to step S<b>16</b>. At step S<b>16</b>, a plurality of pixel data items xi around the pixel position of interest in the image signal Vb is acquired as prediction tap data. At step s<b>17</b>, pixel data y₁to y₄in a unit pixel block UB located at the pixel position of interest in the image signal Vb are generated based on the estimating equation (1) using the coefficient data Wi generated at step S<b>15</b> and the plurality of pixel data items xi acquired at step S<b>16</b>.

At step S<b>18</b>, it is determined whether the process for acquiring pixel data for the entire region of the image signal Vb for one frame or field is completed. If completed, the process returns to step S<b>11</b>, where an image signal Va for the next one frame or field is input. If not completed, the process returns to step S<b>14</b>. Subsequently, the next pixel position of interest in the teacher signal ST is processed.

Thus, the operation according to the flow chart of <figref idrefs="DRAWINGS">FIG. 9</figref> can process input pixel data of the image signal Va so as to obtain pixel data of the image signal Vb.

In the above-described embodiments, an information signal is an image signal. However, the present invention is not limited thereto. For example, the present invention can be applied in the same manner to the case where an information signal is an audio signal.

INDUSTRIAL APPLICABILITY

The present invention provides optimal coefficient data of an estimating equation used for converting a first information signal to a second information signal, thus improving the image quality of an output according to the second information signal. For example, the present invention can be applied to the case where an SD signal is converted to an HD signal.