Image processing device

The present invention relates to an image processing technique, and more specifically, to an image processing technique for interpolating and enlarging an image.

When enlarging an image vertically or converting an interlaced image to a progressive image, it is necessary to interpolate the pixels on the horizontal line in the vertical direction. As a method of interpolating the pixels on the horizontal line, a method of repeating the original image of the upper line or the lower line of the line to be interpolated (conventional first interpolation method), or an original image of the upper line and an original image of the lower line A method using the average value of (conventional second interpolation method) is well known.

However, in the conventional first interpolation method, the diagonal contour of the image becomes stepped and rattles. In addition, although the rattling can be suppressed to some extent by the conventional second interpolation method, the image is blurred. As a method of solving these problems, from a plurality of pairs of pixels consisting of the pixels of the original image of the upper line and the pixels of the original image of the lower line located at point-symmetrical positions with the pixel to be interpolated of the line to be interpolated sandwiched between them. A method of determining the pixel pair having the highest correlation and interpolating using the average value of the pixel pair (the average value of the pixels on the upper line and the pixels on the lower line constituting the pixel pair having the highest correlation) (conventional method). (Third interpolation method) has been proposed (see, for example, Patent Document 1).

Here, the conventional third interpolation method will be described with reference to FIG. FIG. 19 is an explanatory diagram for explaining the conventional third interpolation method. As shown in FIG. 19, with reference to pixels A1 to pixel A5 in the original image line n directly above and pixels B1 to B5 in the original image line (n + 1) immediately below, the interpolation target on the interpolation target line m. Interpolate pixel X. First, the correlation with respect to the first pixel pair (pixel A1, pixel B5) located at a point-symmetrical position with the interpolation target pixel X in between is calculated. Examples of the value for indexing the correlation include the absolute difference between the pixel values of the pixel A1 and the pixel B5 (| (pixel value of the pixel A1)-(pixel value of the pixel B5) |). Similarly, a second pixel pair (pixel A2, pixel B4), a third pixel pair (pixel A3, pixel B3), a fourth pixel pair (pixel A4, pixel B2) and a fifth pixel pair (pixel). The correlation with A5 and pixel B1) is calculated in sequence. Next, the correlations for the five pixel pairs are compared, and the pixel pair with the highest correlation is selected. Next, the average value of the pixel values for the two pixels constituting the selected pixel pair is determined as the pixel value of the pixel X to be interpolated.<patcit num="1"><text>Patent No. 2732644</text></patcit>

<p> In the conventional third interpolation method, when an image having a thin linear contour forming a predetermined angle with the image line is included in the frame image, the linear contour can be satisfactorily interpolated. It was difficult.</p><p> Here, this problem will be described with reference to FIGS. 20 and 21. FIG. 20 is a schematic plan view showing a frame image including an image having a thin linear contour. FIG. 21 is a schematic partial plan view showing an enlarged thin linear outline portion of the frame image. Note that FIG. 21 is an enlarged view of a part of the contour portion in FIG. 20.</p><p> FIG. 20 shows an image having a contour consisting of a linear black line L1 inclined with respect to an image line and two linear gray lines L2 and L3 along both sides of the black line. As shown in FIG. 21, pixel A4 and pixel B2 are pixels that display a part of the black line L1, and pixels A3 and pixel B1 are pixels that display a part of the gray line L2. Pixel B3 is a pixel that displays a part of the gray line L3. Further, it is assumed that the pixel A1, the pixel A2, the pixel B4, and the pixel B5 display white.</p><p> As shown in FIG. 21, pixel A4 and pixel B2 have the same pixel value, pixel A3, pixel B1, pixel A5 and pixel B3 have the same pixel value, and pixel A1, pixel A2, pixel. When B4 and pixel B5 have the same pixel value, the second pixel pair (pixel A1, pixel B5), the second pixel pair (pixel A2, pixel B4), and the third pixel pair (pixel A3,) The absolute difference between the five pixel values for the pixel B3), the fourth pixel pair (pixel A4, pixel B2) and the fifth pixel pair (pixel A5, pixel B1) is equal. In this case, in order to obtain the best image, the fourth pixel pair (pixel A4, pixel B2) should be selected, but in the conventional third interpolation method, the fourth pixel pair is dominant. Is not selected.</p><p> Therefore, in the present invention, the image quality of the image processing apparatus is improved by realizing good interpolation even for a frame image including an image having a thin linear contour forming a predetermined angle with the image line. Further, in the present invention, the interpolation accuracy in the image processing method is improved.</p>

<p> In order to solve the above problems, the image processing apparatus according to the present invention is a pixel information holding unit that holds information on a first reference pixel group and information on a second reference pixel group different from the first reference pixel group. A first pixel group consisting of a plurality of pixels partially selected from the first reference pixel group and a second pixel group partially selected from the second reference pixel group in the same arrangement as the first pixel group. A third pixel group consisting of a plurality of pixels, including a pixel at a specific position in the first pixel group and a pixel point-symmetrical to the pixel to be interpolated at a specific position relatively identical to the specific position in the first pixel group. Reference pixel information that extracts information on the first pixel group in the correlation calculation pixel group pair and information on the second pixel group in the correlation calculation pixel group pair from the information holding unit, where the two pixel groups are used as the correlation calculation pixel group pair. Correlation value representing the correlation between the first pixel group and the second pixel group based on the information of the first pixel group and the information of the second pixel group extracted by the extraction unit and the reference pixel information extraction unit. Based on the correlation value calculated by the correlation calculation unit and the correlation calculation unit, the pixel group pair having a high correlation among the correlation calculation pixel group pair and the holding pixel group pair is selected as the interpolation reference pixel group pair. Based on the pixel group pair selection unit, the information of the pixel at a specific position in the first pixel group with respect to the interpolation reference pixel group pair, and the information of the pixel at the specific position in the second pixel group with respect to the interpolation reference pixel group pair. , The reference pixel information extraction unit includes a plurality of sets of correlation calculation pixel group pairs having different pixels selected from the first reference pixel group and the second reference pixel group, including an interpolation unit that generates information of the pixel to be interpolated. One set is sequentially selected, and the interpolation unit outputs the information of the pixel to be interpolated generated for the correlation calculation pixel group pair having the highest correlation among the plurality of sets of correlation calculation pixel group pairs. .. In the following, the image processing device having this configuration will also be referred to as an image processing device A.</p><p> The interpolation unit is characterized in that it outputs the information of the pixel to be interpolated for the correlation calculation pixel group pair having the highest correlation among the plurality of sets of correlation calculation pixel group pairs. In the following, the image processing device having this configuration will also be referred to as an image processing device B.</p>

<p> With the image processing apparatus of the present invention, it is possible to improve the interpolation accuracy when enlarging the image by converting the interlaced image into a progressive image or interpolating the pixels on the horizontal line in the vertical direction. .. As a result, the image processing apparatus of the present invention can improve the image quality of the enlarged image.</p>

As described above, the image processing apparatus A according to the present invention has a configuration including a pixel information holding unit, a reference pixel information extraction unit, a correlation calculation unit, an interpolation reference pixel group pair selection unit, and an interpolation unit. .. In this configuration, the first pixel group is composed of a plurality of pixels having the same arrangement as the first pixel group, and the pixel at a specific position in the first pixel group and the pixel point-symmetrical with respect to the pixel to be interpolated are the first pixel group. Correlation calculation consisting of a second pixel group contained in a specific position relatively identical to the specific position in the pixel group In order to calculate the correlation value between pixels using a pair of pixel groups, point symmetry with respect to the pixel to be interpolated. Correlation between pixels with higher accuracy than that of a conventional image processing device that calculates an interpolation value using one point-symmetric pixel pair can be obtained. That is, in the interpolation search, the high-precision correlation for a plurality of sets of correlation calculation pixel group pairs is compared, so that the accuracy of the interpolation search is improved. Therefore, by using the information of the pixel to be interpolated generated from the pixel at the specific position in the first pixel group and the pixel at the specific position in the second pixel group with respect to the correlation calculation pixel group pair having the highest correlation, It is possible to perform interpolation with higher accuracy than when using the information of the interpolation target pixel generated from the point target pixel pair having the highest correlation in the conventional plurality of types of point symmetric pixel pairs.

In the image processing apparatus A according to the present invention, it is preferable that a plurality of pixels constituting the first pixel group are pixel groups arranged in the image line direction. This is because when the correlation value is obtained, the accuracy weight can be made uniform for a plurality of pixels constituting an arbitrary first pixel group. In general, the closer the pixel is on the original image line to the interpolation line having the interpolation target pixel, the higher the interpolation accuracy. That is, when a plurality of pixels constituting the first pixel group include pixels having different original image lines, the accuracy weights for each pixel are not uniform. Since the first pixel group and the second pixel group have the same arrangement, when a plurality of pixels constituting the first pixel group are arranged in the image line direction, the first pixel group and the second pixel group are arranged in the same arrangement. A plurality of pixels constituting the second pixel group are also arranged in the image line direction.

Further, in the image processing apparatus A according to the present invention, the first reference pixel group is a pixel group arranged on one image line above the image line to which the interpolation target pixel belongs, and the second reference pixel group. It is preferable that the group is a pixel group arranged on one image line below the image line to which the pixel to be interpolated belongs. This is because when the interpolation reference pixel group pair having the highest correlation is selected from the plurality of correlation calculation pixel group pairs, the accuracy weight for each correlation calculation pixel group pair can be made uniform. When a plurality of sets of correlation calculation pixel group pairs include different correlation calculation pixel group pairs on different image lines, the accuracy weights for the correlation calculation pixel group pairs on different image lines are not uniform.

Further, it is preferable that the image line to which the first reference pixel group belongs and the image line to which the second reference pixel group belongs are targets with the image line to which the interpolation target pixel belongs. Further, the image line to which the first reference pixel group belongs is the image line directly above the image line to which the interpolation target pixel belongs, and the image line to which the second reference pixel group belongs is directly above the image line to which the interpolation target pixel belongs. It is more preferably an image line. When all the first pixel groups for a plurality of sets of correlation calculation pixel group pairs are pixel groups arranged on one image line, all the first pixel groups for a plurality of sets of correlation calculation pixel group pairs are The two pixel groups are also arranged on one image line.

In the image processing apparatus A according to the present invention, the plurality of pixels constituting the first pixel group are adjacent to each other, or the plurality of pixels constituting the first pixel group are spaced apart from each other. It can be configured to be a group of pixels separated from each other. When pixel groups separated by a predetermined interval are used, the correlation between the first pixel group and the second pixel group for a spatially wide area can be interpolated and searched at high speed.

It is preferable that the pixels at specific positions in the first pixel group and the second pixel group are pixels near the center. This is because highly accurate interpolation search can be performed. In particular, when the first pixel group and the second pixel group are each composed of three or more pixels, the first pixel group and the second pixel group including the pixels on both sides of the pixel at a specific position are sandwiched between the first pixel group and the second pixel group. Since the correlation with the pixel group can be calculated, the interpolation search can be performed with higher accuracy. When the first pixel group and the second pixel group are each composed of an odd number of pixels, the center pixel located at the center in order is set as the pixel at the specific position, and the first pixel group and the second pixel group are used. When each group is composed of an even number of pixels, it is more preferable that one of the two pixels located at the center in order is a pixel at a specific position. In particular, when each of the first pixel group and the second pixel group is composed of an odd number of pixels, and the pixel at each specific position in the first pixel group and the second pixel group is the center pixel. In the correlation between the first pixel group and the second pixel group, the weight of the correlation can be made uniform on both sides of the pixel pair at a specific position in the first pixel group and the second pixel group.

In the image processing apparatus A according to the present invention, the number of a plurality of pixels constituting the first pixel group is an even number, and the pixel at a specific position in the first pixel group is in the first pixel group. The center pixel, the pixel at a specific position in the second pixel group is the center pixel in the second pixel group, and the information of the pixel to be interpolated is the first of the information of the first pixel group. It is preferable that it is an average value of the pixel value of the central pixel in the pixel group and the pixel value of the central pixel in the second pixel group in the information of the second pixel group. With this configuration, as described above, the correlation between the first pixel group and the second pixel group can be evaluated with high accuracy, and the information of the pixel to be interpolated can be easily derived.

In the image processing apparatus A according to the present invention, the number of a plurality of pixels constituting the first pixel group is an odd number, and the pixel at a specific position in the first pixel group is in the first pixel group. The central pixel, the pixel at a specific position in the second pixel group is the central pixel in the second pixel group, and the information of the pixel to be interpolated is the first of the information of the first pixel group. It is preferable that it is an average value of the pixel value of the central pixel in the pixel group and the pixel value of the central pixel in the second pixel group in the information of the second pixel group. With this configuration, as described above, the correlation between the first pixel group and the second pixel group can be evaluated with high accuracy, and the information of the pixel to be interpolated can be easily derived.

In the image processing apparatus A according to the present invention, when the first pixel group is composed of an odd number of pixels, the plurality of pixels constituting the second pixel group are the first pixels with respect to the pixel to be interpolated. A configuration that is point-symmetrical with a plurality of pixels constituting the group is preferable. In this configuration, since a plurality of pixels constituting the first pixel group are arranged point-symmetrically with respect to the center pixel, the center pixel is arranged in a correlation between the first pixel group and the second pixel group. This is because the weights of the correlations on both sides of are the same. Therefore, the calculated correlation value is a value that satisfactorily indicates the correlation between the first pixel group and the second pixel group.

In the image processing apparatus A according to the present invention, it is preferable that the correlation value calculated by the correlation calculation unit is the sum of the absolute differences based on the information of the first pixel group and the information of the second pixel group. This is because the correlation value can be calculated easily and at high speed. The absolute value sum of the differences calculated by the correlation calculation unit of the image processing device A is the absolute value of the difference between the leftmost pixels in the first pixel group and the second pixel group constituting one set of correlation calculation pixel group pairs. It means the sum of the values to the absolute value of the difference values between the rightmost pixels in order. In the image processing apparatus A according to the present invention, the sum of squared differences and the like may be adopted as the correlation value, but from the viewpoint of performing the calculation at high speed, it is preferable to use the absolute difference value as the interpolated value.

In the image processing apparatus A according to the present invention, the correlation value in the correlation calculation unit is from a pixel at a specific position in the first pixel group based on the information of the first pixel group and the information of the second pixel group. It is preferable that the load difference is the sum of the absolute values weighted according to the relative position. With this configuration, a difference calculation pixel pair consisting of a pixel at a specific position in the first pixel group and a pixel at a specific position in the second pixel group, which is finally used for calculating the information of the pixel to be interpolated, is obtained. This is because the accuracy of interpolation can be improved by weighting the peripheral difference calculation pixel pair. Here, the difference calculation pixel pair means a pair of pixels for obtaining the difference value. Further, the load difference absolute value sum calculated by the correlation calculation unit of the image processing apparatus A means the difference absolute value sum obtained by weighting according to the difference calculation pixel pair constituting the correlation calculation pixel group pair. In the image processing apparatus A according to the present invention, the sum of squares of the load difference may be adopted as the correlation value, but from the viewpoint of performing the calculation at high speed, it is preferable to use the absolute value of the load difference as the interpolation value.

In the image processing apparatus A according to the present invention, a plurality of sets of correlation calculation pixel group pairs are selected from the first reference pixel group, and are distributed in a wide area in the first reference pixel group, and the first A plurality of different sets of wide area correlation calculation pixel group pairs each having a wide area pixel group selected from the two reference pixel groups, and a wide area selected from the first reference pixel group and selected from the first reference pixel group. Narrow-range correlation calculation of a plurality of different sets each having a narrow-range pixel group distributed in a narrow region in the first reference pixel group from the pixel group and a narrow-range pixel group selected from the second reference pixel group. Including the pixel group pair, the interpolation reference pixel group pair selection unit serves as the interpolation reference pixel group pair, and the wide area reference pixel group pair having the highest correlation among the plurality of sets of wide area correlation calculation pixel group pairs and the narrow range correlation of the plurality of sets. The narrow area reference pixel group pair having the highest correlation is selected from the calculated pixel group pairs, and the interpolation unit selects the pixel at a specific position selected from the first reference pixel group with respect to the wide area reference pixel group pair and the second reference. Selected from a specific wide area pixel pair consisting of pixels at a specific position selected from a pixel group, a pixel at a specific position selected from a first reference pixel group for a narrow area reference pixel group pair, and a second reference pixel group. If a specific narrow-range pixel pair consisting of pixels at a specific position is the same pixel pair, information on the pixel to be interpolated is generated based on the information of the same pixel pair, and the specific wide-area pixel pair and the specific narrow-range pixel are generated. If the pair is different, the information of the pixel to be interpolated is generated based on the information of the two pixels closest to the pixel to be interpolated among the first reference pixel group and the second reference pixel group (hereinafter, duplicate interpolation). It can also be referred to as a search-type image processing device A). With this configuration, a double interpolation search is performed: an interpolation search for a plurality of wide area correlation calculation pixel group pairs (wide area interpolation search) and an interpolation search for a plurality of narrow area correlation calculation pixel group pairs (narrow area interpolation search). Thereby, the correlation calculation pixel group pair having the highest correlation (the pixel pair having the highest correlation) can be selected with higher accuracy than in the case of performing a single interpolation search. In the image processing apparatus A of the present invention, not only the double interpolation search but also a plurality of sets of correlation calculation pixel group pairs are distributed in different widths3.

In the image processing apparatus A according to the present invention, a plurality of sets of correlation calculation pixel group pairs are selected from the first reference pixel group, and are distributed in a wide area in the first reference pixel group, and the first A plurality of different sets of wide area correlation calculation pixel group pairs each having a wide area pixel group selected from the two reference pixel groups, and a wide area selected from the first reference pixel group and selected from the first reference pixel group. A plurality of different sets of narrow-range correlation calculation pixels each having a narrow-range pixel group distributed in a narrow region in the first reference pixel group from the pixel group and a narrow-range pixel group selected from the second reference pixel group. Including the group pair, the interpolation reference pixel group pair selection unit serves as the interpolation reference pixel group pair with the wide area reference pixel group pair having the highest correlation among the multiple sets of wide area correlation calculation pixel group pairs and the narrow range correlation of the plurality of sets. Among the calculated pixel group pairs, the narrow area reference pixel group pair having the highest correlation in the area occupied by the wide area reference pixel group pair is selected, and the interpolation unit selects from the first reference pixel group for the narrow area reference pixel group pair. A configuration that generates information on the pixel to be interpolated based on the information on the pixel at the specific position and the information on the pixel at the specific position selected from the second reference pixel group (hereinafter, also referred to as a hierarchical interpolation search type image processing device A). ). With this configuration, a two-layer interpolation search is performed, that is, a wide area interpolation search and a narrow area interpolation search for a plurality of narrow area correlation calculation pixel group pairs within the range of the wide area reference candidate pixel pairs selected by the wide area interpolation search. This makes it possible to select the most correlated correlation calculation pixel group pair (the most correlated pixel pair) with higher accuracy than when performing a single interpolation search, and also compared to when performing a duplicate interpolation search. Information on the pixel to be interpolated can be calculated at high speed. The image processing apparatus A of the present invention is not limited to the two-layer interpolation search, and a plurality of sets of correlation calculation pixel group pairs include three or more types of correlation calculation pixel group pairs distributed in different areas. , Interpolation search of 3 or more layers may be performed.

In the duplicate interpolation search type or hierarchical interpolation search type image processing apparatus A according to the present invention, the number of pixels of the wide area pixel group selected from the first reference pixel group is the narrow area pixel selected from the first reference pixel group. It is preferable that the number of pixels in the group is larger than the number of pixels in the group. In the duplicate interpolation search and the hierarchical interpolation search, the correlation between the first pixel group and the second pixel group in a wide spatial range and the correlation between the first pixel group and the second pixel group in a narrow spatial range are performed. This is because the correlation calculation pixel group pair having the highest correlation should be determined based on. With this configuration, in the wide area interpolation search, it is possible to search for a spatially wide range of correlations as compared with the narrow area interpolation search.

In the duplicate interpolation search type or hierarchical interpolation search type image processing apparatus A according to the present invention, the number of pixels in the wide area pixel group and the number of pixels in the narrow area pixel group may both be an odd number or an even number. There may be an odd number of ones and an even number of the other.

In the duplicate interpolation search type or hierarchical interpolation search type image processing apparatus A according to the present invention, the first reference pixel group is a pixel group arranged on one image line above the image line to which the interpolation target pixel belongs. Therefore, it is preferable that the second reference pixel group is a pixel group arranged on one image line below the image line to which the pixel to be interpolated belongs. As described above, when the correlation value is obtained, the weight of accuracy for a plurality of pixels constituting an arbitrary first pixel group can be made uniform, and the most from a plurality of sets of correlation calculation pixel group pairs. This is because when selecting an interpolated reference pixel group pair having a high correlation, the accuracy weight for each correlation calculation pixel group pair can be made uniform.

In the duplicate interpolation search type or hierarchical interpolation search type image processing apparatus A according to the present invention, the wide area pixel groups selected from the first reference pixel group are pixel groups separated from each other at predetermined intervals, and the first It is preferable that the narrow region pixel group selected from the reference pixel group of is a pixel group adjacent to each other. This is because with this configuration, the correlation between the first pixel group and the second pixel group for a spatially wide area can be interpolated and searched at high speed.

In the duplicate interpolation search type or hierarchical interpolation search type image processing apparatus A according to the present invention, the correlation value calculated by the correlation calculation unit is the information of the wide area pixel group selected from the first reference pixel group and the second reference. Absolute difference sum based on the information of the wide area pixel group selected from the pixel group, or the narrow area selected from the information of the narrow area pixel group selected from the first reference pixel group and the narrow area selected from the second reference pixel group. It can be configured to be the sum of the absolute values of the differences based on the information of the pixel group.

In the duplicate interpolation search type or hierarchical interpolation search type image processing apparatus A according to the present invention, the correlation value in the correlation calculation unit is based on the information on the wide area pixel group selected from the first reference pixel group and the second reference pixel group. Based on the information of the selected wide area pixel group, the load difference absolute value sum weighted according to the relative position from the pixel at the specific position in the wide area pixel group selected from the first reference pixel group, or the first Within the narrow range pixel group selected from the first reference pixel group based on the information of the narrow range pixel group selected from the reference pixel group of and the information of the narrow range pixel group selected from the second reference pixel group. It is possible to configure the load difference absolute value sum weighted according to the relative position from the pixel at the specific position in.

As described above, the image processing apparatus B according to the present invention has a configuration including a pixel information holding unit, a pixel information derivation unit, a correlation calculation unit, an interpolation reference pixel group pair selection unit, and an interpolation unit. In this configuration, it is composed of a first correlation calculation pixel group composed of a plurality of correlation calculation pixels and a plurality of correlation calculation pixels having the same arrangement as the first correlation calculation pixel group, and the first correlation calculation pixel group. A second correlation calculation pixel group having a correlation calculation pixel at a specific position within and a correlation calculation pixel point-symmetrical with respect to the pixel to be interpolated at a specific position relatively identical to the specific position in the first correlation calculation pixel group. Correlation calculation of: In order to calculate the correlation value between pixels using the pixel group pair, the interpolation value is calculated using one point-symmetric pixel pair that is point-symmetric with respect to the pixel to be interpolated. Highly accurate correlation between pixels can be obtained. That is, in the interpolation search, the high-precision correlation for a plurality of sets of correlation calculation pixel group pairs is compared, so that the accuracy of the interpolation search is improved. Therefore, the information of the pixel to be interpolated generated from the pixel at the specific position in the first correlation calculation pixel group and the pixel at the specific position in the second correlation calculation pixel group with respect to the correlation calculation pixel group having the highest correlation is obtained. By using this, it is possible to perform interpolation with higher accuracy than when using the information of the interpolation target pixel generated from the point target pixel pair having the highest correlation in the conventional plurality of types of point symmetric pixel pairs. The first correlation calculation pixel group is composed of a plurality of correlation calculation pixels including an interpolation pixel having information interpolated from the information of a plurality of pixels partially selected from the first reference pixel group. The correlation calculation pixel group of 2 is composed of a plurality of correlation calculation pixels including an interpolation pixel having information interpolated from the information of a plurality of pixels partially selected from the second reference pixel group.

The first correlation calculation pixel group and the second correlation calculation pixel group may have a configuration including only interpolated pixels, or may have a configuration including interpolated pixels and ordinary pixels.

In the image processing apparatus B according to the present invention, the plurality of correlation calculation pixels constituting the first correlation calculation pixel group includes at least one pixel selected from the first reference pixel group, and the second correlation calculation pixel. The plurality of correlation calculation pixels constituting the group can be configured to include at least one pixel selected from the second reference pixel group. With this configuration, a wide range of interpolation searches can be performed at high speed.

In the image processing apparatus B according to the present invention, the information of the interpolated pixels in the first correlation calculation pixel group is the two pixel values in the information of the two pixels adjacent to the interpolated pixels selected from the first reference pixel group. It is a linearly interpolated pixel value, and the information of the interpolated pixel in the second correlation calculation pixel group linearly interpolates the two pixel values in the information of the two pixels adjacent to the interpolated pixel selected from the second reference pixel group. It is possible to configure the pixel value to be the same.

In the image processing apparatus B according to the present invention, the information of the interpolated pixels in the first correlation calculation pixel group is a plurality of pixels in the information of a plurality of pixels located in the vicinity of the interpolated pixels selected from the first reference pixel group. It is a pixel value generated by filtering based on the value, and the information of the interpolated pixel in the second correlation calculation pixel group is a plurality of pixels located in the vicinity of the interpolated pixel selected from the second reference pixel group. It is possible to configure the pixel values to be generated by the filtering process based on the plurality of pixel values in the information of.

In the image processing apparatus B according to the present invention, it is preferable that a plurality of correlation calculation pixels constituting the first correlation calculation pixel group are pixel groups arranged in the image line direction. This is because when the correlation value is obtained, the accuracy weight for a plurality of correlation calculation pixels constituting an arbitrary first correlation calculation pixel group can be made uniform. Since the first correlation calculation pixel group and the second correlation calculation pixel group have the same arrangement, a plurality of correlation calculation pixels constituting the first correlation calculation pixel group are arranged in the image line direction. In the case of the correlation calculation pixel group, a plurality of pixels constituting the second correlation calculation pixel group are also arranged in the image line direction.

Further, in the image processing apparatus B according to the present invention, the first reference pixel group is a pixel group arranged on one image line above the image line to which the interpolation target pixel belongs, and the second reference pixel group. It is preferable that the group is a pixel group arranged on one image line below the image line to which the pixel to be interpolated belongs. This is because when the interpolation reference pixel group pair having the highest correlation is selected from the plurality of correlation calculation pixel group pairs, the accuracy weight for each correlation calculation pixel group pair can be made uniform. When a plurality of sets of correlation calculation pixel group pairs include different correlation calculation pixel group pairs of image lines, the accuracy weights for the correlation calculation pixel group pairs on different image lines are not uniform.

Further, it is preferable that the image line to which the first reference pixel group belongs and the image line to which the second reference pixel group belongs are targets with the image line to which the interpolation target pixel belongs. Further, the image line to which the first reference pixel group belongs is the image line directly above the image line to which the interpolation target pixel belongs, and the image line to which the second reference pixel group belongs is directly above the image line to which the interpolation target pixel belongs. It is more preferably an image line. When all the first correlation calculation pixel groups for a plurality of sets of correlation calculation pixel group pairs are correlation calculation pixel groups arranged on one image line, a plurality of sets of correlation calculation pixel group pairs All the second correlation calculation pixel groups for the above are also arranged on one image line.

In the image processing apparatus B according to the present invention, a plurality of correlation calculation pixels constituting the first correlation calculation pixel group constitute a correlation calculation pixel group adjacent to each other or a first correlation calculation pixel group. A plurality of correlation calculation pixels may be configured as a group of correlation calculation pixels separated from each other at predetermined intervals. When the correlation calculation pixel group separated at a predetermined interval is used, the correlation between the first pixel group and the second pixel group for a spatially wide area can be interpolated and searched at high speed.

It is preferable that the pixels at specific positions in the first correlation calculation pixel group and the second correlation calculation pixel group are correlation calculation pixels near the center. This is because highly accurate interpolation search can be performed. In particular, when the first correlation calculation pixel group and the second correlation calculation pixel group are each composed of three or more pixels, the second correlation calculation pixel on both sides of the correlation calculation pixel at a specific position is included. Since the correlation between the correlation calculation pixel group of 1 and the second correlation calculation pixel group can be calculated, the interpolation search can be performed with higher accuracy. When the first correlation calculation pixel group and the second correlation calculation pixel group are each composed of an odd number of correlation calculation pixels, the center correlation calculation pixel located at the center in order is set as the correlation calculation pixel at a specific position. When the first correlation calculation pixel group and the second correlation calculation pixel group are each composed of an even number of correlation calculation pixels, one of the two central correlation calculation pixels located in the center in order is designated as a specific position. It is more preferable to use the correlation calculation pixel of. In particular, each of the first correlation calculation pixel group and the second correlation calculation pixel group is composed of an odd number of correlation calculation pixels, and each specific position in the first correlation calculation pixel group and the second correlation calculation pixel group. When the correlation calculation pixel of is the central correlation calculation pixel, in the correlation between the first correlation calculation pixel group and the second correlation calculation pixel group, the first correlation calculation pixel group and the second correlation calculation pixel This is because the weight of the correlation can be made uniform on both sides of the correlation calculation pixel pair at a specific position in the group.

In the image processing apparatus B according to the present invention, the number of the plurality of correlation calculation pixels constituting the first correlation calculation pixel group is an even number, and the correlation calculation pixels at specific positions in the first correlation calculation pixel group are , The central correlation calculation pixel in the first correlation calculation pixel group, and the correlation calculation pixel at a specific position in the second correlation calculation pixel group is the central correlation calculation pixel in the second correlation calculation pixel group. Yes, the information of the pixel to be interpolated is the pixel value of the central correlation calculation pixel in the first pixel group of the information of the first pixel group and the second pixel group of the information of the second pixel group. It is preferable that it is an average value with the pixel value of the central pixel in. With this configuration, as described above, the correlation between the first pixel group and the second pixel group can be evaluated with high accuracy, and the information of the pixel to be interpolated can be easily derived.

In the image processing apparatus B according to the present invention, the number of the plurality of correlation calculation pixels constituting the first correlation calculation pixel group is an odd number, and the correlation calculation pixels at specific positions in the first correlation calculation pixel group are , The central correlation calculation pixel in the first pixel group, the correlation calculation pixel at a specific position in the second correlation calculation pixel group is the central correlation calculation pixel in the second correlation calculation pixel group, and the interpolation target. The pixel information is the second correlation between the pixel value of the central correlation calculation pixel in the first correlation calculation pixel group in the information of the first correlation calculation pixel group and the second correlation in the information of the second correlation calculation pixel group. It is preferable that it is an average value with the pixel value of the central correlation calculation pixel in the calculation pixel group. With this configuration, as described above, the correlation between the first correlation calculation pixel group and the second correlation calculation pixel group can be evaluated with high accuracy, and the information of the pixel to be interpolated can be easily derived.

In the image processing apparatus B according to the present invention, when the first correlation calculation pixel group is composed of an odd number of pixels, a plurality of correlation calculation pixels constituting the second correlation calculation pixel group are used as interpolation target pixels. On the other hand, a configuration that is point-symmetrical with a plurality of correlation calculation pixels constituting the first correlation calculation pixel group is preferable. With this configuration, a plurality of correlation calculation pixels constituting the first correlation calculation pixel group are arranged point-symmetrically with respect to the center pixel. Therefore, the first correlation calculation pixel group and the second correlation calculation pixel group are calculated. This is because the weights of the correlations on both sides of the central correlation calculation pixel can be made the same in the correlation with the pixel group. Therefore, the calculated correlation value is a value that satisfactorily indicates the correlation between the first correlation calculation pixel group and the second correlation calculation pixel group.

In the image processing apparatus B according to the present invention, the correlation value calculated by the correlation calculation unit is the sum of the absolute differences based on the information of the first correlation calculation pixel group and the information of the second correlation calculation pixel group. Is preferable. This is because the correlation value can be calculated easily and at high speed. The absolute difference sum calculated by the correlation calculation unit of the image processing device B is the leftmost correlation calculation pixel in the first correlation calculation pixel group and the second correlation calculation pixel group constituting one set of correlation calculation pixel group pairs. Correlation calculation means the sum of the absolute values of the differences between the pixels and the absolute values of the differences between the pixels. In the image processing apparatus B according to the present invention, the sum of squared differences and the like may be adopted as the correlation value, but from the viewpoint of performing the calculation at high speed, it is preferable to use the absolute difference value as the interpolated value.

In the image processing apparatus B according to the present invention, the correlation value in the correlation calculation unit is in the first correlation calculation pixel group based on the information of the first correlation calculation pixel group and the information of the second correlation calculation pixel group. It is preferable that the sum of the absolute values of the load differences is weighted according to the relative position from the correlation calculation pixel at the specific position. With this configuration, the correlation calculation pixel at a specific position in the first correlation calculation pixel group and the correlation calculation pixel at a specific position in the second correlation calculation pixel group are finally used for calculating the information of the pixel to be interpolated. This is because the accuracy of interpolation can be improved by weighting the difference calculation pixel pair consisting of the above from the peripheral difference calculation pixel pair. Here, the difference calculation pixel pair means a pair of correlation calculation pixels for obtaining the difference value. Further, the load difference absolute value sum calculated by the correlation calculation unit of the image processing apparatus B means the difference absolute value sum obtained by weighting according to the difference calculation pixel pair constituting the correlation calculation pixel group pair. In the image processing apparatus B according to the present invention, the sum of squares of the load difference may be adopted as the correlation value, but from the viewpoint of performing the calculation at high speed, it is preferable to use the absolute value of the load difference as the interpolation value.

Further, in the image processing apparatus B according to the present invention, a plurality of sets of correlation calculation pixel group pairs are derived from the information of the first reference pixel group, and are distributed in a wide area, and the wide area correlation calculation pixel group and the second. A wide-area correlation calculation pixel group derived from the information of the reference pixel group of the above, and a plurality of different sets of wide-area correlation calculation pixel group pairs, each of which is derived from the information of the first reference pixel group, and the first wide-area pixel group. Narrow-range correlation calculation pixel groups distributed in a narrower region, and multiple sets of narrow-range correlation calculation pixel groups that are different from each other and each have a narrow-range correlation calculation pixel group derived from the information of the second reference pixel group. In the same manner as in the case of the image processing device A described above, a double interpolation search or a two-layer interpolation search is performed on a plurality of sets of wide area correlation calculation pixel groups and a plurality of sets of narrow area correlation calculation pixel group pairs. You may.

(Embodiment 1) In the first embodiment, one form of the image processing apparatus A for determining the interpolation reference pixel group pair by a single interpolation search and the image processing method thereof are described with reference to FIGS. 1 to 5. explain. FIG. 1 is a block diagram conceptually showing the configuration of the image processing apparatus according to the first embodiment. FIG. 2 is an explanatory diagram for explaining an image processing method of the image processing apparatus according to the first embodiment. FIG. 3 is an explanatory diagram for explaining a process of calculating a correlation value for a first set of correlation calculation pixel group pairs in the image processing method of the image processing apparatus according to the first embodiment. FIG. 4 is an explanatory diagram for explaining a process of calculating a correlation value for a second set of correlation calculation pixel group pairs in the image processing method of the image processing apparatus according to the first embodiment. Further, FIG. 5 is an explanatory diagram for explaining a process of calculating the correlation value for the third set of correlation calculation pixel group pairs in the image processing method of the image processing apparatus according to the first embodiment.

The image processing apparatus shown in FIG. 1 is a reference including an input terminal 11, a pixel information holding unit 5 including a 1H delay circuit 12, a line memory 13 and a line memory 14, an address control circuit 15, a buffer 16 and a buffer 17. It consists of a pixel information extraction unit 1, a correlation calculation unit 2 consisting of a difference circuit 18, an absolute value circuit 19 and an addition circuit 20, an interpolation reference pixel group pair selection unit 3 consisting of a minimum value circuit 21, and an interpolation value calculation circuit 22. The configuration includes an interpolation unit 4 and an output terminal 23.

The input terminal 11 is a terminal for receiving an input signal such as a television image signal for interlaced scanning or a reduced image signal that needs to be spatially expanded in the vertical direction.

The 1H delay circuit 12 is a circuit that delays the image signal supplied to the input terminal 11 by one line in time.

The line memory 13 is a memory that stores data (pixel information) for one line of an image signal delayed by one line. Specifically, the line memory 13 stores data relating to all the pixels constituting the original image line directly above the interpolation line. The data stored in the line memory 13 includes information on the pixels constituting all the first pixel groups (information on the first reference pixel group) for a plurality of sets of correlation calculation pixel group pairs. ..

The line memory 14 is a memory that stores data for one line of the original image line. Specifically, the line memory 13 stores data relating to all the pixels constituting the original image line immediately below the interpolation line. The data stored in the line memory 13 includes information on the pixels constituting all the second pixel groups for the plurality of correlation calculation pixel group pairs (information on the second reference pixel group).

The address control circuit 15 specifies the addresses of a plurality of pixels for extracting reference data for generating the interpolation target pixel from the line memory 13, and also extracts the reference data for generating the interpolation target pixel from the line memory 14. It is a circuit that specifies the address of the pixel to be interpolated.

Here, a specific example of the correlation calculation pixel group pair selected by the address control circuit 15 according to the first embodiment will be described. As shown in FIG. 2, in the address control circuit 15, the original image including the pixel A3 located directly above the pixel X to be interpolated on the interpolation m line as the central pixel (the pixel at the specific position in the first pixel group). An original image (n +) containing a continuous pixel group (pixel A2, pixel A3, pixel A4) on the n-line and pixel B3 immediately below pixel X to be interpolated as a central pixel (pixel at a specific position in the second pixel group). 1) Center correlation calculation pixel group pair (pixel group pair surrounded by solid line in Fig. 2) consisting of continuous pixel groups (pixel B2, pixel B3, pixel B4) on the line and center correlation calculation pixel group Original image for pair The pixel group (pixel A1, pixel A2, pixel A3) in which the pixel group on the n-line is shifted to the left by one pixel and the pixel group for the center correlation calculation pixel group are the original image (n + 1). Correlation calculation pixel group pair (pixel group pair surrounded by dotted line in Fig. 2) consisting of pixel group (pixel B3, pixel B4, pixel B5) in which the pixel group on the line is shifted to the right by one pixel, and the center. Original image for the correlation calculation pixel group pair of Pixel group (pixel A3, pixel A4, pixel A5) in which the pixel group on the n-line is shifted to the right by one pixel, and the original image (n +) for the center correlation calculation pixel group pair 1) Correlation calculation pixel group pairs (pixel group pairs surrounded by a single point chain line in Fig. 2) consisting of pixel groups (B1, B2, B3) in which the pixel groups on the line are shifted to the left by one pixel are sequentially arranged. Is selected. In FIG. 2, the pixel group (A1 to A5) is the first reference pixel group, and the pixel group (B1 to B5) is the second reference pixel group. Further, in each set of correlation calculation pixel group pairs, the pixel group (first pixel group) on the original image line n and the pixel group (second pixel group) on the original image line are the same array. Moreover, the plurality of pixels forming the pixel group on the original image line n and the plurality of pixels forming the pixel group on the original image line are point-symmetrical about the pixel to be interpolated.

In FIG. 2, the case where three sets of correlation calculation pixel pairs are selected is shown, but the pixel group on the original image n line and the original image (n + 1) line with respect to the center correlation calculation pixel group pair are shown. Two sets of correlation calculation pixel group equality in which the pixel group is shifted in the opposite direction to the left and right by two pixels may be further selected.

When extracting the reference data, for example, when the first correlation calculation pixel group pair is selected, specifically, the address of the pixel A2 and the address of the pixel A4 are set as the start address and the end with respect to the line memory 13, respectively. The data of pixels A2 to A4 are extracted by designating them as addresses, and the address of pixel B2 and the address of pixel B4 are designated as the start address and end address for the line memory 14, respectively, of pixels B2 to pixel B4. Extract the data. Alternatively, the address of pixel A2, the address of pixel A3, and the address of pixel A4 are specified to extract the data of pixels A2 to A4 as the first reference data, and the address of pixel B2, the address of pixel B3, and the pixel Specify the address of B4 and extract the data of pixels B2 to B4 as the second reference data. The first reference data includes at least information on the first pixel group, and the second reference data includes information on the second pixel group. The information of the first pixel group includes an address and a pixel value related to each pixel constituting the first pixel group. Similarly, the information of the second pixel group also includes the address and the pixel value related to each pixel constituting the second pixel group. Here, the pixel value is information about each pixel required for display. For example, in the case of monochrome display, it is a luminance value, and in the case of full-color display, it is a luminance component (Y component). (Brightness value), 2 types of color difference components (I component and Q component, or Cr component and Cb component), and 3 types of color component (R component, G component, B component) ..

The buffer 16 and the buffer 17 are buffers for temporarily storing the first reference data and the second reference data extracted from the line memory 13 and the line memory 14 according to the instruction of the address control circuit 15, respectively. The first reference data temporarily stored in the buffer 16 is supplied to the difference circuit 18 and the interpolated value output circuit 22. The second reference data temporarily stored in the buffer 17 is also supplied to the difference circuit 18 and the interpolated value output circuit 22 in the same manner.

The difference circuit 18 is a circuit that calculates the difference between the first reference data from the buffer 16 and the second reference data from the buffer 17.

Here, the difference between the first reference data and the second reference data will be described. First, for the first set of correlation calculation pixel group pairs shown in FIG. 3, the first reference data (data of pixel A1 and data of pixel A2) for the first set of correlation calculation pixel group pairs from the buffer 16 And the first data (address a (-1)) of pixel A3) and the second reference data (data of pixel B3, data of pixel B4 and pixel) for the first set of correlation calculation pixel group pairs from buffer 17. The difference calculation is executed with the first data (address b (-1)) of (B5 data). Subsequently, in the first reference data and the second reference data, the second data (address a (0)) and the second data (address b (0)) shifted to the right by one data, respectively. The difference operation is executed with. Subsequently, a difference operation is executed between the tail data (address a (1)) in the first reference data and the tail data (address b (1)) in the second reference data. As a result, the difference with respect to the first set of correlation calculation pixel group pairs is calculated. The number of calculated differences is the same as the number of data constituting the first reference data (second reference data).

Next, the first reference data (data of pixel A2, data of pixel A3 and data of pixel A4) and the second set of correlation calculation pixel group pairs for the second set of correlation calculation pixel group pairs shown in FIG. The difference from the second reference data (data of pixel B2, data of pixel B3, and data of pixel B4) with respect to the second reference data is calculated in the same manner as in the case of the first set of correlation calculation pixel group pairs.

Finally, the first reference data (data of pixel A3, data of pixel A4 and data of pixel A5) and the third set of correlation calculation pixel group pairs for the third set of correlation calculation pixel group pairs shown in FIG. The difference from the second reference data (data of pixel B1, data of pixel B2, and data of pixel B3) with respect to the second reference data is calculated in the same manner as in the case of the first set of correlation calculation pixel group pairs.

In the above, for each set of correlation calculation pixel group pairs, the difference from the leftmost pixel to the rightmost pixel is calculated in order, but this order may be any order. In addition, the difference for the correlation calculation pixel group pair shifted to the right by one pixel in order from the leftmost correlation calculation pixel group pair (first set of correlation calculation pixel group pair) was calculated, but the order is arbitrary. There may be. Since the calculation of these differences is performed sequentially along the time series of the supplied data, the data arrangement of the pixels A1 to A3 in the first reference data is arranged according to the order in which the differences are calculated. Need to be different.

The absolute value circuit 19 is a circuit that makes each difference value calculated by the difference circuit 18 an absolute value.

The addition circuit 20 adds all the plurality of absolute values associated with each of the plurality of sets of correlation calculation pixel group pairs calculated by the absolute value circuit 19, and sums each of the plurality of sets of correlation calculation pixel group pairs ( It is a circuit that outputs the difference absolute value sum). That is, the sum of the absolute values of the differences of the first set of correlation calculation pixel groups to the sum of the absolute values of the differences of the third set of correlation calculation pixel groups are sequentially output.

The minimum value circuit 21 sequentially compares the holding value held inside and the difference absolute value sum calculated by the addition circuit 20, and if the input difference absolute value sum is smaller than the holding value, holds it. The value is updated and the first signal (signal value 1) is output to the interpolation value calculation circuit 22, and if the input difference absolute value sum is larger than the holding value, the second signal (signal value 0) is output. .. The correlation calculation pixel group pair corresponding to the holding value corresponds to the holding pixel group pair.

More specifically, first, in the minimum value circuit 21, when the difference absolute value sum for the first correlation calculation pixel group pair is input, the difference absolute value sum for the first correlation calculation pixel group pair is set as the minimum value. Assuming that the holding value is updated, the first signal (signal value 1) is output. Next, when the absolute difference sum for the second correlation calculation pixel group pair is input, the holding value (the difference absolute value sum for the first correlation calculation pixel group pair) and the difference for the second correlation calculation pixel group pair. Comparing with the absolute sum, if the difference absolute sum for the second correlation calculation pixel group pair is smaller than the holding value, the holding value is updated to the difference absolute value sum for the second correlation calculation pixel group pair. The first signal (signal value 1) is output, while the second signal (signal value 0) is output if the sum of the absolute values of the differences for the second correlation calculation pixel group pair is larger than the holding value. Next, the sum of the absolute values of the differences with respect to the third correlation calculation pixel group pair is input, and the same processing as in the case of the second correlation calculation pixel group pair is performed. As a result, the correlation calculation pixel group pair having the smallest difference absolute value sum among the three sets of correlation calculation pixel group pairs is selected as the interpolation reference pixel group pair. That is, the finally calculated interpolation reference pixel group pair is the correlation calculation pixel group pair having the highest correlation among the three sets of correlation calculation pixel group pairs.

The interpolation value calculation circuit 22 is a first signal synchronized with the first signal or the second signal output from the minimum value circuit 21 and the first signal or the second signal output from the buffer 16 and the buffer 17, respectively. It is a circuit that calculates an interpolated value based on the reference data of the above and the second reference data. In the interpolation value calculation circuit 22, when the first signal is supplied from the minimum value circuit 21, the center data (information of the center pixel) in the first reference data from the buffer 16 and the second reference data from the buffer 17 are supplied. The center data (information of the center pixel) in the above is extracted, the average value of the center data is calculated, and the interpolation value is updated. Further, the interpolation value calculation circuit 22 supplies the first reference data and the second reference data for the interpolation reference pixel group pair, that is, the first signal (signal value 1) indicating the minimum value from the minimum value circuit 21 at the end. The average value of the center data with respect to the first reference data and the second reference data at that time is supplied to the output terminal 23 as an interpolation value.

Specifically, in the interpolation value calculation circuit 22, first, the first signal for the first set of correlation calculation pixel group pairs from the minimum value circuit 21 and the first set of correlation calculation pixel group pairs from the buffer 16 are obtained. Based on the first reference data and the second reference data for the first set of correlation calculation pixel group pairs from the buffer 17, the average value of the center data for the first set of correlation calculation pixel group pairs is calculated. The interpolated value is updated. Next, along with the first signal or the second signal for the second set of correlation calculation pixel group pairs from the minimum value circuit 21, the first reference data and the second reference for the second set of correlation calculation pixel group pairs. If data is input and the signal from the minimum value circuit 21 is the first signal, the average value of the center data for the second set of correlation calculation pixel group pairs is calculated and the interpolated value is updated, and the minimum value circuit 21 If the signal from is the first signal, the average value of the center data for the second set of correlation calculation pixel group pairs is not calculated and the interpolation value is not updated. Next, along with the first signal or the second signal for the third set of correlation calculation pixel group pairs from the minimum value circuit 21, the first reference data and the second reference for the third set of correlation calculation pixel group pairs. The data is input, and the same processing as in the case of the second correlation calculation pixel group pair is performed. Finally, the interpolated value (information on the pixel to be interpolated) is supplied to the output terminal 23.

Further, the interpolation value calculation circuit 22 outputs a plurality of reference data for a plurality of sets of correlation calculation pixel group pairs for calculating the interpolation value of the next interpolation target pixel after outputting the interpolation value of the interpolation target pixel from the line memory. A signal instructing the buffer to be sequentially supplied is output to the address control circuit 15. This instruction can be performed by designating the address of the leftmost pixel in the leftmost correlation calculation pixel group pair and the number of the correlation calculation pixel group pair. It should be noted that this instruction may be specified in any way as long as a plurality of sets of correlation calculation pixel group pairs for the next interpolation target pixel can be specified.

The output terminal 23 outputs the interpolated value calculated by the interpolated value calculation circuit 22 to an external device.

Here, the image processing method (operation) of the image processing apparatus according to the first embodiment will be described. The original image is continuously input to the input terminal 11 as an image signal for each original image line. When the image signal of the original image n-line and the image signal of the original image (n + 1) line are input, all the pixels constituting the last input original image (n + 1) line are input to the line memory 14. The data of all the pixels constituting the original image n line input immediately before is accumulated in the line memory 13 via the 1H delay circuit. When the image signal of the original image (n + 2) line is continuously input, the data stored in the line memory 13 and the line memory 14 is once erased, and the data is finally input to the line memory 14. The data of all the pixels constituting the original image (n + 2) line is accumulated, and the data of all the pixels constituting the original image (n + 1) line input immediately before is stored in the line memory 13. Accumulated via a 1H delay circuit. These are repeated for the image signals of all original image lines. In the following, the line memory 14 stores the data of all the pixels constituting the original image (n + 1) line, and the line memory 13 stores the data of all the pixels constituting the original image n lines. The case where it is done will be described.

Based on the instruction signal output from the interpolation value calculation circuit 22, the address control circuit 15 sequentially extracts the first reference data corresponding to each of the plurality of sets of correlation calculation pixel group pairs for the line memory 13. Then, the line memory 14 is controlled to sequentially extract the second reference data corresponding to each of the plurality of sets of correlation calculation pixel groups and supply the line memory 14 to the buffer 17. Output a signal.

According to the control signal from the address control circuit 15, the first reference data for each of the plurality of sets of correlation calculation pixel group pairs is supplied from the line memory 13 to the buffer 16 for each of the plurality of sets of correlation calculation pixel group pairs. Further, the second reference data for each of the plurality of sets of correlation calculation pixel group pairs is supplied from the line memory 14 to the buffer 17.

The first reference data for each of the plurality of sets of correlation calculation pixel group pairs supplied to the buffer 16 and the second reference data for each of the plurality of sets of correlation calculation pixel group pairs supplied to the buffer 17 are temporarily accumulated. After that, it is supplied to the difference circuit 18 for each of a plurality of sets of correlation calculation pixel group pairs.

Between the first reference data for each of the plurality of sets of correlation calculation pixel group pairs from the buffer 16 supplied to the difference circuit 18 and the second reference data for each of the plurality of sets of correlation calculation pixel group pairs from the buffer 17. The difference calculation is performed with. At this time, the difference calculation is executed from the first data to the last data for the first reference data and the second reference data according to the data supply time series. The plurality of difference values for each of the plurality of sets of correlation calculation pixel group pairs calculated between the first reference data and the second reference data for each of the plurality of sets of correlation calculation pixel group pairs are the plurality of sets of correlation calculation pixels. It is supplied to the absolute value circuit 19 for each group pair.

A plurality of difference values for each of the plurality of sets of correlation calculation pixel group pairs supplied to the absolute value circuit 19 are converted into absolute values, and are supplied to the addition circuit 20 for each of the plurality of sets of correlation calculation pixel group pairs.

The absolute values of the plurality of difference values for each of the plurality of sets of correlation calculation pixel group pairs supplied to the addition circuit 20 are added for each of the plurality of sets of correlation calculation pixel group pairs. As a result, the sum of the absolute values of the differences for each of the plurality of pairs of correlation calculation pixels is calculated. The calculated difference absolute value sum for each of the plurality of sets of correlation calculation pixel group pairs is supplied to the minimum value circuit 21 for each of the plurality of sets of correlation calculation pixel group pairs.

The sum of the multiple absolute values of the differences with respect to the pair of correlation calculation pixels supplied to the minimum value circuit 21 is sequentially compared with the holding value, and the absolute difference smaller than the holding value is obtained for each pair of correlation calculation pixels of the plurality of sets. If the sum of values is supplied, the holding value is updated to the sum of the absolute values of the differences, and the first signal (signal value 1) indicating the minimum value is interpolated, otherwise the second signal (signal value 0) is interpolated. It is supplied to the value calculation circuit 22.

A first signal or a second signal for each of a plurality of pairs of correlation calculation pixel groups from the minimum value circuit 21 supplied to the interpolation value calculation circuit 22 and a plurality of sets of correlation calculation pixel groups supplied from the buffer 16. Minimum for each set of correlation calculation pixel group pairs based on the first reference data for each pair and the second reference data for each of the plurality of pairs of correlation calculation pixel group pairs supplied from the buffer 17. If the signal for each of the plurality of sets of correlation calculation pixel group pairs from the value circuit 21 is the first signal, the data of the central pixel in the first reference data supplied from the buffer 16 and the second signal supplied from the buffer 17 The average value for the center pixel pair is calculated from the data of the center pixel in the reference data of, and the interpolated value is updated to the average value, while each of the plurality of pairs of correlation calculation pixel group pairs from the minimum value circuit 21. If the signal for is the second signal, the average value for the center pixel pair is not calculated and the interpolated value is not updated. As a result, finally, the average value of the center pixel pair with respect to the interpolation reference pixel group pair having the smallest difference absolute value sum among the plurality of sets of correlation calculation pixel group pairs is output to the output terminal 23 as the interpolation value.

According to the image processing apparatus according to the first embodiment, the interpolation reference pixel group pair having the strongest correlation is selected from the plurality of sets of correlation calculation pixel group pairs, and the original image n-line for the interpolation reference pixel group pair is selected. The conventional interpolation value calculated from the correlation of a plurality of pixel pairs by averaging the pixel value of the center pixel and the pixel value of the center pixel on the original image (n + 1) line to obtain the interpolation value of the pixel to be interpolated. More accurate interpolated values can be obtained.

(Embodiment 2) In the second embodiment, the interpolation reference by the double interpolation search of the interpolation search for a plurality of sets of wide area correlation calculation pixel group pairs and the interpolation search for a plurality of sets of narrow area correlation calculation pixel group pairs. A form of the image processing device A that calculates the interpolated value based on the selection of the pixel group pair and the image processing method thereof will be described with reference to FIGS. 6 to 8. FIG. 6 is a flowchart for explaining the image processing method of the image processing apparatus according to the second embodiment. FIG. 7 is a schematic plan view showing a frame image for explaining the effect of the image processing apparatus according to the second embodiment. FIG. 8 is an explanatory diagram for explaining a wide area correlation calculation pixel group pair in the image processing method of the image processing apparatus according to the second embodiment. Since the conceptual configuration of the image processing apparatus according to the second embodiment is substantially the same as that of the image processing apparatus shown in FIG. 1, FIG. 1 is also referred to for convenience.

The image processing method (operation) of the image processing apparatus according to the second embodiment will be described. As shown in FIG. 6, first, the start of the first interpolation value calculation using the first reference data and the second reference data for each of the plurality of sets of wide area correlation calculation pixel group pairs is declared ( Step 101). As a result, the same processing as that for the plurality of sets of correlation calculation pixel group pairs in the first embodiment is performed for the plurality of sets of wide area correlation calculation pixel group pairs.

From the first reference data and the second reference data for one set of wide area correlation calculation pixel group pairs, the sum of the difference absolute values for one set of wide area correlation calculation pixel group pairs is calculated (step 102). The absolute difference sum for one set of wide-area correlation calculation pixel group pairs is the first reference data for one set of narrow-area correlation calculation pixel group pairs in the same manner as the processing for the correlation calculation pixel group of the first embodiment. And the second reference data is generated by passing through the difference circuit 18, the absolute value circuit 19, and the adder circuit 20 (see FIG. 1).

After calculating the difference absolute value sum for one set of wide area correlation calculation pixel group pairs, it is judged whether the calculated difference absolute value sum is smaller than the holding value, and if it is judged to be small, the holding value is updated ( Step 103). This processing is performed in the minimum value circuit 21 in the same manner as the processing for the correlation calculation pixel group of the first embodiment (see FIG. 1).

The above-mentioned step 102 and the above-mentioned step 103 are sequentially repeated for each set for all the sets of the plurality of sets of wide-area correlation calculation pixel groups. When the processing for all the sets of the plurality of sets of wide area correlation calculation pixel groups is completed, the minimum value among the plurality of absolute difference sums for the plurality of sets of wide area correlation calculation pixel group pairs is determined. That is, the wide area reference candidate pixel group pair having the smallest difference absolute value sum is determined among the plurality of sets of wide area correlation calculation pixel group pairs. The process of determining the wide area reference candidate pixel group pair is performed in the same manner as the process for the correlation calculation pixel group of the first embodiment.

After the wide area reference candidate pixel group pair is determined, the wide area interpolation candidate value for the wide area reference candidate pixel group pair is calculated (step 104). The interpolation candidate value for the wide area reference candidate pixel group pair is performed in the interpolation value calculation circuit 22 by the same processing as the processing for the reference candidate pixel group of the first embodiment (see FIG. 1).

After the wide area interpolation candidate value is calculated, the start of the first interpolation value calculation using the first reference data and the second reference data for each of the plurality of sets of narrow area correlation calculation pixel group pairs is declared ( Step 105). As a result, the same processing as that for the plurality of sets of correlation calculation pixel group pairs in the first embodiment is performed for the plurality of sets of wide area correlation calculation pixel group pairs. The number of pixels constituting the narrow-area correlation calculation pixel group pair is smaller than the number of pixels constituting the wide-area correlation calculation pixel group pair. Further, the number of pairs of the narrow region correlation calculation pixel group of a plurality of sets is the same as the number of pairs of the wide area correlation calculation pixel group pair of a plurality of sets, and the pixel selected as the central pixel is the narrow range correlation calculation of a plurality of sets. It is common to the pixel group pair and the wide area correlation calculation pixel group pair.

From the first reference data and the second reference data for one set of narrow area correlation calculation pixel group pairs, the sum of the difference absolute values for one set of narrow area correlation calculation pixel group pairs is calculated (step 106). The absolute value sum of the differences for one set of narrow-area correlation calculation pixel group pairs is the first reference for one set of narrow-range correlation calculation pixel group pairs in the same manner as the processing for the correlation calculation pixel group of the first embodiment. The data and the second reference data are generated by passing through the difference circuit 18, the absolute value circuit 19, and the adder circuit 20 (see FIG. 1).

After calculating the difference absolute value sum for one set of narrow-range correlation calculation pixel group pairs, it is judged whether the calculated difference absolute value sum is smaller than the holding value, and if it is judged to be small, the holding value is updated. (Step 107). This processing is performed in the minimum value circuit 21 in the same manner as the processing for the correlation calculation pixel group of the first embodiment (see FIG. 1).

The above-mentioned step 106 and the above-mentioned step 107 are sequentially repeated for each set for all the sets of the plurality of sets of narrow-range correlation calculation pixel groups. When the processing for all the sets of the plurality of sets of narrow-range correlation calculation pixel groups is completed, the minimum value among the plurality of absolute difference sums for the plurality of sets of narrow-range correlation calculation pixel group pairs is determined. That is, the narrow area reference candidate pixel group pair having the smallest difference absolute value sum is determined among the plurality of sets of narrow area correlation calculation pixel group pairs. The process of determining the narrow region reference candidate pixel group pair is performed in the same manner as the process for the correlation calculation pixel group of the first embodiment.

After the narrow area reference candidate pixel group pair is determined, the narrow area interpolation candidate value for the narrow area reference candidate pixel group pair is calculated (step 108). The interpolation candidate value for the narrow area reference candidate pixel group pair is performed in the interpolation value calculation circuit 22 by the same processing as the processing for the reference candidate pixel group of the first embodiment (see FIG. 1).

It is determined whether or not the wide area interpolation candidate value calculated in step 104 above and the narrow area interpolation candidate value calculated in step 108 above are the same (step 109). This processing is performed in the interpolation value calculation circuit 22 (see FIG. 1). Note that this process is not performed in the first embodiment, that is, the interpolation value calculation circuit 22 in the first embodiment does not have to include a circuit that performs this process.

If the wide area interpolation candidate value and the narrow area interpolation candidate value are the same, the narrow area interpolation candidate value (or the wide area interpolation candidate value) is output as the interpolation value (step 110). This processing is performed in the interpolation value calculation circuit 22 (see FIG. 1). Note that this process is not performed in the first embodiment, that is, the interpolation value calculation circuit 22 in the first embodiment does not have to include a circuit that performs this process.

On the other hand, if the wide area interpolation candidate value and the narrow area interpolation candidate value are different, the average value of the data of the pixel immediately above the pixel to be interpolated and the data of the pixel immediately below it is calculated. The average value is output as an interpolated value (step 112). The calculation of the average value of the data of the pixel immediately above the pixel to be interpolated and the data of the pixel immediately below is performed by the interpolation value calculation circuit 22 (see FIG. 1). Note that this process is not performed in the first embodiment, that is, the interpolation value calculation circuit 22 in the first embodiment does not have to include a circuit that performs this process.

As described above, the image processing apparatus according to the second embodiment, which calculates the interpolation value by using the double interpolation search, has, for example, a pattern in which a plurality of thin diagonal lines are arranged as shown in FIG. In the image, the interpolation value can be calculated more accurately than when a single interpolation search is used. This will be described with reference to FIG. 8 in which the vicinity of the center of the pattern image shown in FIG. 7 is enlarged and displayed. When the number of pixel pairs constituting the correlation calculation pixel group pair is small (for example, in the case of 3 pixel pairs as shown in FIG. 2), the correlation for a plurality of pairs of correlation calculation pixel group pairs may be equal. , The optimum judgment may not be possible. However, when the first reference data and the second reference data for the wide area correlation calculation pixel group pair consisting of a large number of pixel pairs as shown in FIG. 8 are further used, one correlation having the minimum correlation value. The calculated pixel group pair can be determined. In this way, the wide area interpolation candidate value for the wide area correlation calculation pixel group pair and the narrow area interpolation candidate value for the narrow area correlation calculation pixel group pair are calculated, and when those values are the same, those values are output as the interpolation value. By doing so, the accuracy of interpolation can be further improved.

In the above, the interpolation search for a plurality of sets of wide area correlation calculation pixel group pairs was performed, and then the interpolation search for a plurality of sets of narrow area correlation calculation pixel group pairs was performed. After performing the search, an interpolation search may be performed on a plurality of sets of wide area correlation calculation pixel group pairs.

When the wide area interpolation candidate value and the narrow area interpolation candidate value are different, in the above, the average value of the data of the pixel immediately above the pixel to be interpolated and the data of the pixel immediately below is used as the interpolation value. The narrow range interpolation candidate value may be preferentially adopted as the interpolation value.

(Embodiment 3) In the third embodiment, the interpolation reference is performed by a two-layer interpolation search of an interpolation search for a plurality of sets of wide area correlation calculation pixel group pairs and an interpolation search for a plurality of sets of narrow area correlation calculation pixel group pairs. A form of the image processing device A that calculates the interpolated value based on the selection of the pixel group pair and the image processing method thereof will be described with reference to FIGS. 9 and 10. FIG. 9 is a flowchart for explaining the image processing method of the image processing apparatus according to the third embodiment. FIG. 10 is an explanatory diagram for explaining a two-layer interpolation search in the image processing method of the image processing apparatus according to the third embodiment, and FIG. 10A shows a wide area interpolation search in the first stage. , Fig. 10 (b) shows the narrow area interpolation search of the second stage. Since the conceptual configuration of the image processing apparatus according to the third embodiment is substantially the same as that of the image processing apparatus shown in FIG. 1, FIG. 1 is also referred to for convenience.

The image processing method (operation) of the image processing apparatus according to the third embodiment will be described. As shown in FIG. 9, first, the start of the first interpolation value calculation using the first reference data and the second reference data for each of the plurality of sets of wide area correlation calculation pixel group pairs is declared ( Step 101). As a result, the same processing as that for the plurality of sets of correlation calculation pixel group pairs in the first embodiment is performed for the plurality of sets of wide area correlation calculation pixel group pairs.

From the first reference data and the second reference data for one set of wide area correlation calculation pixel group pairs, the sum of the difference absolute values for one set of wide area correlation calculation pixel group pairs is calculated (step 102). The absolute value sum of the differences for one set of wide-area correlation calculation pixel group pairs is the first reference data for one set of wide-area correlation calculation pixel group pairs and the first reference data for one set of wide-area correlation calculation pixel group pairs in the same manner as the processing for the correlation calculation pixel group of the first embodiment. The second reference data is generated by passing through the difference circuit 18, the absolute value circuit 19, and the adder circuit 20 (see FIG. 1).

After calculating the difference absolute value sum for one set of wide area correlation calculation pixel group pairs, it is judged whether the calculated difference absolute value sum is smaller than the holding value, and if it is judged to be small, the holding value is updated ( Step 103). This processing is performed in the minimum value circuit 21 in the same manner as the processing for the correlation calculation pixel group of the first embodiment (see FIG. 1).

The above-mentioned step 102 and the above-mentioned step 103 are sequentially repeated for each set for all the sets of the plurality of sets of wide-area correlation calculation pixel groups. When the processing for all the sets of the plurality of sets of wide area correlation calculation pixel groups is completed, the minimum value among the plurality of absolute difference sums for the plurality of sets of wide area correlation calculation pixel group pairs is determined. That is, the wide area reference candidate pixel group pair having the smallest difference absolute value sum is determined among the plurality of sets of wide area correlation calculation pixel group pairs. The process of determining the wide area reference candidate pixel group pair is performed in the same manner as the process for the correlation calculation pixel group of the first embodiment.

After the wide area reference candidate pixel group pair is determined, the address of the central pixel located at the center of the first pixel group in the wide area reference candidate pixel group pair is extracted (step 201).

After the wide area interpolation candidate value is calculated, the start of the first interpolation value calculation using the first reference data and the second reference data for each of the plurality of sets of narrow area correlation calculation pixel group pairs is declared ( Step 105). As a result, the same processing as that for the plurality of sets of correlation calculation pixel group pairs in the first embodiment is performed for the plurality of sets of narrow area correlation calculation pixel group pairs. The number of pixels constituting the narrow-area correlation calculation pixel group pair is smaller than the number of pixels constituting the wide-area correlation calculation pixel group pair.

The address of the central pixel extracted in step 201 is set as the initial start address of the interpolation search for the first reference data of the interpolation search for a plurality of sets of narrow-area correlation calculation pixel group pairs (step 202). As a result, a plurality of sets of narrow-range correlation calculation pixel group pairs each composed of only the pixels included in the wide-area reference candidate pixel group pair selected in step 103 are selected, and the narrow-range correlation selected first. The calculated pixel group pair is specified.

Here, a plurality of sets of narrow-range correlation calculation pixel group pairs selected and a narrow-range correlation calculation pixel group pair selected first in an interpolation search for a plurality of sets of narrow-range correlation calculation pixel group pairs will be described. When a wide area reference candidate pixel group pair (pixel group surrounded by a dotted line in FIG. 10 (a)) as shown in FIG. 10 (a) is selected, a plurality of sets of narrow area correlation calculation pixel group pairs are selected. , Narrow range correlation calculation pixel group consisting of pixel a (-5) to pixel a (-3) and pixel b (3) to pixel b (5), pixel a (-4) to pixel a shifted by one pixel Narrow range correlation calculation pixel group consisting of (-2) and pixel b (2) to pixel b (4), ..., Pixel a (3) to pixel a (5) and pixel b (shifted by 8 pixels) It is composed of 9 pairs of narrow-range correlation calculation pixel group consisting of -3) to pixel b (-5).

In step 201, the address of the central pixel (a (0)) located at the center of the pixel group on the original image n-line in the wide area reference candidate pixel group pair is extracted, and as shown in FIG. 10 (b), The narrow area correlation calculation pixel group pair including the central pixel as the central pixel in the wide area reference candidate pixel group pair is the start pixel group pair of the interpolation search. After that, the remaining eight sets of narrow-range correlation calculation pixel group pairs are sequentially selected. The selection order of the remaining eight sets of narrow-range correlation calculation pixel group pairs may be arbitrary.

From the first reference data and the second reference data for one set of narrow area correlation calculation pixel group pairs, the sum of the difference absolute values for one set of narrow area correlation calculation pixel group pairs is calculated (step 106). The absolute value sum of the differences for one set of narrow-area correlation calculation pixel group pairs is the first reference for one set of narrow-range correlation calculation pixel group pairs in the same manner as the processing for the correlation calculation pixel group of the first embodiment. The data and the second reference data are generated by passing through the difference circuit 18, the absolute value circuit 19, and the adder circuit 20 (see FIG. 1).

After calculating the difference absolute value sum for one set of narrow-range correlation calculation pixel group pairs, it is judged whether the calculated difference absolute value sum is smaller than the holding value, and if it is judged to be small, the holding value is updated. (Step 107). This processing is performed in the minimum value circuit 21 in the same manner as the processing for the correlation calculation pixel group of the first embodiment (see FIG. 1).

The above-mentioned step 106 and the above-mentioned step 107 are repeated for all the sets of the plurality of sets of narrow-range correlation calculation pixel groups in each set according to the order specified in the above-mentioned step 202. When the processing for all the sets of the plurality of sets of narrow-range correlation calculation pixel group pairs is completed, the minimum value among the plurality of absolute difference sums for the plurality of sets of narrow-range correlation calculation pixel group pairs is determined. That is, the narrow area reference candidate pixel group pair having the smallest difference absolute value sum is determined among the plurality of sets of narrow area correlation calculation pixel group pairs. The process of determining the narrow region reference candidate pixel group pair is performed in the same manner as the process for the correlation calculation pixel group of the first embodiment.

After the narrow area reference candidate pixel group pair is determined, the narrow area interpolation candidate value for the narrow area reference candidate pixel group pair is calculated (step 108). The interpolation candidate value for the narrow area reference candidate pixel group pair is performed in the interpolation value calculation circuit 22 by the same processing as the processing for the reference candidate pixel group of the first embodiment (see FIG. 1).

The narrow range interpolation candidate value is output as an interpolation value (step 203). This processing is performed in the interpolation value calculation circuit 22 (see FIG. 1).

In this embodiment, for each of the interpolation target pixels, the first-stage interpolation search using the first reference data and the second reference data for the wide area correlation calculation pixel pair and the second-stage interpolation search using the narrow area reference data. However, for a certain number of consecutive interpolation target pixels, the address of the wide area reference candidate pixel in the first-stage interpolation search for the first interpolation target pixel for which the interpolation value has already been calculated is searched for in the second-stage interpolation search. It may be commonly used as the start address of, and the operation of the first-stage interpolation search for the subsequent interpolation target pixels may be omitted.

Further, when the address of the wide area reference candidate pixel with respect to the first interpolation target pixel is commonly used as the start address of the second stage interpolation search for a certain number of continuous interpolation target pixels, the position of the subsequent interpolation target pixel is used. A value considering the difference from the position of the first interpolation target pixel (the number of pixels indicating how much the pixels at the upper and lower interpolation positions are shifted to the left and right from the interpolation target pixel) is extracted in step 201, and the subsequent value is extracted in step 201. It may be applied to the initial start address of the pixel to be interpolated. Specifically, when the position of the subsequent interpolation target pixel is the position where the first interpolation target pixel is shifted to the right by + k pixels (an integer excluding k: 0), a wide area reference is made to the first interpolation target pixel. The address of the pixel obtained by shifting the center pixel of the candidate pixel group pair to the right by + k pixels is used as the start address of the second stage interpolation search.

(Embodiment 4) In the fourth embodiment, with reference to FIGS. 11 to 13, one form of the image processing apparatus A for determining a plurality of non-consecutive interpolation reference pixel group pairs and the image processing method thereof. I will explain while. FIG. 11 is a block diagram conceptually showing the configuration of the image processing apparatus according to the fourth embodiment. FIG. 12 is an explanatory diagram for explaining the thinned-out correlation calculation pixel group pair in the image processing method of the image processing apparatus according to the fourth embodiment. Further, FIG. 13 is an explanatory diagram for explaining a correlation calculation pixel group pair that is quasi-decimated at equal intervals in the second image processing method of the image processing apparatus according to the fourth embodiment.

The image processing apparatus shown in FIG. 11 has an input terminal 11, a 1H delay circuit 12, a line memory 13 and a pixel information holding unit 5 including a line memory 14, an address control circuit 15, and a reference pixel including a buffer 16 and a buffer 17. An information extraction unit 1, a correlation calculation unit 2 consisting of a thinning circuit 30, a difference circuit 18, an absolute value circuit 19 and an addition circuit 20, an interpolation reference pixel group pair selection unit 3 consisting of a minimum value circuit 21, and an interpolation value calculation circuit. It is configured to include an interpolation unit 4 composed of 22 and an output terminal 23. The image processing apparatus shown in FIG. 11 is the same as the image processing apparatus according to the first to third embodiments described above, except that the correlation calculation unit 2 further has a thinning circuit 30 in front of the difference circuit 18. It has the same configuration. Therefore, the differences will be described below.

The thinning circuit 30 is a circuit that selectively outputs predetermined pixel data from each of the reference data from the buffer 16 and the reference data from the buffer 17 to the difference circuit 18. As the thinning circuit, a circuit that thins out intermittently and evenly at a fixed number of pixel data intervals, a circuit that intermittently thins out irregularly at a predetermined unspecified number of pixel data intervals, and a circuit that averages a continuous fixed number of pixel data. A circuit that thins out a predetermined number of pixel data evenly in a pseudo manner, a circuit that thins out a predetermined number of pixel data unevenly by averaging a predetermined number of consecutive pixel data, an intermittent thinning and a pseudo Examples include a circuit for thinning out in combination with thinning out.

Here, the image processing method (operation) of the image processing apparatus according to the fourth embodiment will be described. A case where a circuit for thinning out evenly intermittently at a fixed number of pixel data intervals is used as the thinning out circuit 30 will be described. Only the part different from the above-described first embodiment will be described.

The first reference data once stored in the buffer 16 and the second reference data once stored in the buffer 17 are supplied to the thinning circuit 30. The first reference data and the second reference data supplied to the thinning circuit 30 are thinned out at equal intervals as shown in FIG. 12, respectively. The x mark in FIG. 12 represents the pixel corresponding to the data removed by the thinning. The thinned first reference data and the second reference data are supplied to the difference circuit 18. Subsequent processing is the same as in the first embodiment.

In the above, in the thinning circuit 30, the pixel data of the first reference data and the second reference data are simply thinned out at equal intervals, but as shown in FIG. 13, a plurality of pixel data for consecutive pixels are averaged. By doing so, the data may be thinned out at equal intervals in a pseudo manner. FIG. 13 shows a case where one pixel information is generated from three consecutive pixels. Specifically, the data of the pixel a (-1), the data of the pixel a (0), and the data of the pixel from the pixel a (1) are averaged to generate a pseudo data of the pixel a'(0).

Further, the thinning interval does not have to be equal, for example, the thinning interval is narrow in the center of the first reference data and the second reference data, and goes to the periphery of the first reference data and the second reference data. The interval of thinning may be widened accordingly.

By thinning out the first reference data and the second reference data used for the interpolation search, the calculation of the interpolation search is reduced, and the range constituting the reference data (the first pixel in the first pixel group and the second pixel group). It can be allocated to the calculation load by increasing the distance between the position of and the position of the last pixel).

The technique for obtaining the correlation value based on the thinned out first reference data and the second reference data in the fourth embodiment shall be used in combination with the image processing techniques described in the first to third embodiments described above. Can be done. In particular, it is preferable to apply the thinning out to the first wide area reference data and the second wide area reference data in the second embodiment or the third embodiment described above.

(Embodiment 5) In the fifth embodiment, reference to FIGS. 14 and 15 for one form of the image processing apparatus A for determining a plurality of non-consecutive interpolation reference pixel group pairs and an image processing method thereof. I will explain while. FIG. 14 is a block diagram conceptually showing the configuration of the image processing apparatus according to the fifth embodiment. Further, FIG. 15 is an explanatory diagram for explaining a method of calculating a correlation value with respect to a correlation calculation pixel group pair in the image processing method of the image processing apparatus according to the fifth embodiment.

The image processing apparatus shown in FIG. 14 is a reference including an input terminal 11, a pixel information holding unit 5 including a 1H delay circuit 12, a line memory 13 and a line memory 14, an address control circuit 15, a buffer 16 and a buffer 17. Correlation calculation unit 2 consisting of pixel information extraction unit 1, difference circuit 18, absolute value circuit 19, weighting circuit 40 and addition circuit 20, interpolation reference pixel group pair selection unit 3 consisting of minimum value circuit 21, and interpolation value calculation. The configuration includes an interpolation unit 4 including a circuit 22 and an output terminal 23. The image processing apparatus shown in FIG. 14 has the above-described first to third embodiments, except that the correlation calculation unit 2 further has a weighting circuit 40 between the absolute value circuit 19 and the addition circuit 20. It has the same configuration as the image processing apparatus according to the above. Therefore, the differences will be described below.

The weighting circuit 40 sets the absolute value of the difference between the arbitrary pixel data included in the first reference data and the pixel data included in the second reference data associated with the arbitrary data to the arbitrary pixel and the first. It is a circuit that multiplies a constant determined according to the position relative to the center pixel in one pixel group pair. For example, the weighting circuit 40 sets the absolute value of the difference between the pixel at the relative address a (-1) and the pixel at the relative address b (-1) with respect to the correlation calculation pixel group pair shown in FIG. For this, the weight constant W (-1) is multiplied, and the weight constant W (0) is multiplied for the absolute difference between the pixel at the relative address a (0) and the pixel at the relative address b (0). Then, the weight constant W (0) is multiplied by the absolute value of the difference between the pixel at the relative address a (1) and the pixel at the relative address b (1). Note that the weighting is such that W (0) is smaller than W (-1) and W (1), and the pixel data of the pixel of the relative address a (0) finally used for the calculation of the interpolation value and the relative address b ( The smaller the absolute value of the difference between the pixel data of 0) and the pixel data, the easier it is to select as interpolation data.

Here, the image processing method (operation) of the image processing apparatus shown in FIG. 14 will be described. Only the parts different from the above-described first embodiment will be described.

In the absolute value circuit 19, by taking the absolute value, it is converted into a plurality of difference absolute values for each of the supplied plurality of sets of correlation calculation pixel group pairs, and is converted for each set of multiple sets of correlation calculation pixel group pairs. The absolute difference value is supplied to the weighting circuit 40.

In the weighting circuit 40, the plurality of supplied absolute difference values are multiplied by the weight constants associated with each of the plurality of absolute difference values. At this time, the multiplication of the weight constants is executed from the first data to the last data for the first reference data and the second reference data according to the data supply time series. The plurality of weighted absolute difference values are supplied to the adder circuit 20.

The plurality of absolute difference values for each of the plurality of sets of correlation calculation pixel group pairs supplied to the addition circuit 20 are added for each of the plurality of sets of correlation calculation pixel group pairs. As a result, the sum of the absolute load differences for each of the plurality of sets of correlation calculation pixel group pairs is calculated. The calculated load difference absolute value sum for each of the plurality of sets of correlation calculation pixel group pairs is supplied to the minimum value circuit 21 for each of the plurality of sets of correlation calculation pixel group pairs. Subsequent processing is the same as in the case of the first embodiment.

In the image processing apparatus shown in FIG. 14, the weighting circuit 40 is provided between the absolute value circuit 19 and the addition circuit 20, but the weighting circuit 40 is provided between the difference circuit 18 and the absolute value circuit 19. Even if it is done, it has the same effect. In this case, a plurality of difference values for each of the plurality of sets of correlation calculation pixel group pairs calculated by the difference circuit 18 are supplied to the weighting circuit 40 for each of the plurality of sets of correlation calculation pixel group pairs. In the weighting circuit, each of the correlation calculation pixel pairs constituting each of the plurality of sets of correlation calculation pixel group pairs is multiplied by the weight constant corresponding to the correlation calculation pixel pair. At this time, the multiplication of the weight constants is executed from the first data to the last data for the first reference data and the second reference data according to the data supply time series. The plurality of weighted difference values are supplied to the absolute value circuit 19.

The technique for obtaining the correlation value based on the sum of the absolute load differences of the fifth embodiment can be used in combination with the image processing techniques described in the first to fourth embodiments described above.

(Embodiment 6) In the sixth embodiment, an image processing apparatus and an image processing method for determining an interpolation reference pixel group pair by a single interpolation search will be described with reference to FIGS. 16A to 16C. To do. 16 (a) to 16 (c) are explanatory views for explaining the image processing method of the image processing apparatus according to the sixth embodiment, and are the first set of correlation calculation pixel group pairs and the second set, respectively. It is explanatory drawing for demonstrating the process of calculating the correlation value with respect to the correlation calculation pixel group pair of, and the correlation calculation pixel group pair of the 3rd set. Since the image processing apparatus according to the sixth embodiment has the same configuration as the image processing apparatus according to the first embodiment, the description thereof will be omitted. Further, in the image processing method according to the sixth embodiment, the pixel group (first pixel group) on the original image n line and the original image (n + 1) line in each of the plurality of sets of correlation calculation pixel group pairs. Since the image processing method is the same as that of the first embodiment except that the number of pixel groups (second pixel group) is an even number, only the differences thereof will be described below.

Consecutive pixel groups (pixel A4, pixel A5) and pixel B2 on the original image n-line including pixel A4 as a center pixel (pixel at a specific position in the first pixel group) shown in FIG. 16 (a). The first set of correlation calculation pixel group pairs consisting of continuous pixel groups (pixel B2, pixel B3) on the original image (n + 1) line containing as the center pixel (pixel at a specific position in the second pixel group) (The pixel group pair surrounded by the one-point chain line in FIG. 16 (a)) and the pixel group on the original image n line for the first set of correlation calculation pixel group pairs shown in FIG. 16 (b) are shown on the left. Pixel group shifted by 1 pixel (pixel A3, pixel A4) and pixel group on the original image (n + 1) line for the first set of correlation calculation pixel group pair shifted to the right by 1 pixel The second set of correlation calculation pixel group pairs (pixel group pairs surrounded by solid lines in FIG. 16 (b)) consisting of (pixels B3 and pixel B4) and the second set shown in FIG. 16 (c). Original image for the correlation calculation pixel group pair of Pixel group (pixel A2, pixel A3) in which the pixel group on the n-line is shifted to the left by one pixel and the original image (n + 1) for the second set of correlation calculation pixel group pair. ) Pixel group of the third set of correlation calculation pixel group consisting of pixel group (pixel B4, pixel B5) in which the pixel group on the line is shifted to the right by one pixel (pixel group surrounded by the dotted line in FIG. 16 (c)) Pair) and are selected in sequence. The pixel group (A2 to A5) is the first reference pixel group, and the pixel group (B2 to B5) is the second reference pixel group.

In FIGS. 16A to 16C, the case where three sets of correlation calculation pixel pairs are selected is shown, but the pixel group on the original image n line for the first set of correlation calculation pixel group pairs is on the right. Correlation calculation pixel group pair consisting of a pixel group shifted by 1 pixel and a pixel group on the original image (n + 1) line with respect to the first set of correlation calculation pixel group pair Or, the pixel group on which the pixel group on the n-line of the original image for the third set of correlation calculation pixel group pairs is shifted to the left by one pixel, and the original image (n + 1) line for the third set of correlation calculation pixel group pair. At least one correlation calculation pixel group pair, such as a correlation calculation pixel group pair consisting of a pixel group obtained by shifting the upper pixel group to the right by one pixel, may be further selected.

Here, a method of calculating the difference with respect to the correlation calculation pixel group pair of each set shown in FIGS. 16A to 16C will be described. With the start data (address a (0)) of the first reference data (data of pixel A4 and data of pixel A5) from the buffer 16 for the first set of correlation calculation pixel group pairs shown in FIG. 16 (a). , The difference calculation is executed with the head data (address b (0)) of the second reference data (data of pixel B2 and data of pixel B3) from the buffer 17 for the first set of correlation calculation pixel group pairs. (See Figure 1). Subsequently, in the first reference data and the second reference data, the second data (address a (1)) and the second data (address b (1)) shifted to the right by one data, respectively. The difference operation is executed with. As a result, the difference with respect to the first set of correlation calculation pixel group pairs is calculated. Next, the first reference data (data of pixel A3 and data of pixel A4) for the second set of correlation calculation pixel group pairs and the second set of correlation calculation pixel group pairs shown in FIG. 16 (b). The difference from the reference data of 2 (data of pixel B3 and data of pixel B4) and the first reference data (data of pixel A2) for the third set of correlation calculation pixel group pairs shown in FIG. 16 (b). And the difference between the second reference data (data of pixel B4 and data of pixel B5) for the correlation calculation pixel group pair of the third set (data of pixel A3) and the case of the correlation calculation pixel group pair of the first set. It is calculated in the same way.

According to the image processing apparatus according to the sixth embodiment, the interpolation reference pixel group pair having the strongest correlation is selected from the plurality of sets of correlation calculation pixel group pairs, and the original image n-line for the interpolation reference pixel group pair is selected. The conventional interpolation value calculated from the correlation of a plurality of pixel pairs by averaging the pixel value of the center pixel and the pixel value of the center pixel on the original image (n + 1) line to obtain the interpolation value of the pixel to be interpolated. More accurate interpolated values can be obtained.

The technique of using the pixel group consisting of an even number of pixels as the first pixel group and the second pixel group described in the sixth embodiment is the image processing technique described in the above-described second to fifth embodiments. Can be applied to.

(Embodiment 7) In the seventh embodiment, an image processing device B that performs an independent interpolation search in units smaller than a pixel unit and an image processing method thereof will be described with reference to FIGS. 17 and 18. FIG. 17 is a block diagram conceptually showing the configuration of the image processing device. FIG. 18 is an explanatory diagram for explaining a correlation calculation pixel group pair including interpolated pixels in the image processing method of the image processing apparatus.

The image processing apparatus shown in FIG. 17 includes an input terminal 11, a 1H delay circuit 12, a pixel information holding unit 5 including a line memory 13 and a line memory 14, an address control circuit 15, a filter circuit 56, a filter circuit 57, and a buffer. Interpolation with pixel information derivation unit 51 consisting of 16 and buffer 17, correlation calculation unit 2 consisting of difference circuit 18, absolute value circuit 19 and addition circuit 20, interpolation reference pixel group pair selection unit 3 consisting of minimum value circuit 21. The configuration includes an interpolation unit 4 including a value calculation circuit 22 and an output terminal 23. The image processing apparatus shown in FIG. 17 has the same configuration as the image processing apparatus A according to the first embodiment, except that the filter circuit 56 and the filter circuit 57 are further provided. Therefore, the differences will be described below.

The filter circuit 56 is half between at least two selected pixels based on the information of at least two pixels selected by the address control circuit 15 from the first reference pixel group held in the pixel information holding unit 5. Generates information on interpolated pixels that are virtually inserted in 1 / n pixel units (n is a natural number) such as pixel units, 1/3 pixel units, and 1/4 pixel units. Similar to the filter circuit 56, the filter circuit 57 generates information on interpolated pixels that are virtually inserted in 1 / n pixel units (n is a natural number).

Here, a specific example of a correlation calculation pixel group pair including interpolated pixels will be described. As shown in FIG. 18, a continuous correlation calculation pixel group (pixel A4) on the original image n line including the interpolation pixel A4.5 as a center correlation calculation pixel (pixel at a specific position in the first correlation calculation pixel group). , Interpolated pixel A4.5, Pixel A5) and Pixel B1.5 as central correlation calculation pixels (pixels at specific positions in the second correlation calculation pixel group) Consecutive correlation calculation on the original image (n + 1) line The first set of correlation calculation pixel group pairs (pixel group pair surrounded by a single point chain line in FIG. 18) consisting of pixel groups (pixel B1, interpolating pixel B1.5, pixel B2) and the first set of correlation calculation pixels Correlation calculation pixel group (interpolation pixel A3.5, pixel A4, interpolation pixel A4.5) in which the correlation calculation pixel group on the n-line of the original image for the group pair is shifted to the left by half a pixel and the first set of correlation calculation pixels A second consisting of a correlation calculation pixel group (interpolation pixel B1.5, pixel B2, interpolation pixel B2.5) in which the correlation calculation pixel group on the original image (n + 1) line for the group pair is shifted to the right by half a pixel. Shift the correlation calculation pixel group on the original image n line to the correlation calculation pixel group pair (the pixel group pair surrounded by the solid line in FIG. 18) and the correlation calculation pixel group pair of the first set by one pixel to the left. Correlation calculation pixel group (pixel A3, interpolation pixel A3.5, pixel A4) and correlation calculation pixel group on the original image (n + 1) line for the first set of correlation calculation pixel group pair is one pixel to the right In the same manner as the third set of correlation calculation pixel group pairs (pixel group pairs surrounded by dotted lines in FIG. 18) consisting of shifted correlation calculation pixel groups (pixel B2, interpolating pixel B2.5, pixel B3). The correlation calculation pixel group of the 4th to 6th sets shifted by half a pixel and the correlation calculation pixel group on the original image n line with respect to the correlation calculation pixel group pair of the first set are shifted to the left by 3 pixels. Correlation calculation pixel group (pixel A1, interpolation pixel A1.5, pixel A2) and correlation calculation pixel group on the original image (n + 1) line for the first set of correlation calculation pixel group pair are shifted to the right by 3 pixels. Correlation calculation pixel group (pixel B4, interpolated pixel B4. 5. The 7th set of correlation calculation pixel group pairs (pixel group pairs surrounded by the alternate long and short dash line in FIG. 18) consisting of interpolated pixels B5) are sequentially derived. The set of pixels on the original image (n + 1) line (A1, A2, ..., A5) is the first reference pixel group, and the set of pixels on the original image (n + 1) line (B1, B2, A5). ..., B5) is the second reference pixel group.

In FIG. 18, a case where seven sets of correlation calculation pixel pairs are selected is shown, but the correlation calculation pixel group on the original image n-line with respect to the first set of correlation calculation pixel group pairs is shifted to the right by half a pixel. Correlation calculation pixel group pair consisting of a pixel group in which the correlation calculation pixel group on the original image (n + 1) line is shifted to the left by half a pixel for the correlation calculation pixel group and the first set of correlation calculation pixel group pairs. , Original image for the 7th set of correlation calculation pixel group pairs The original image for the 7th set of correlation calculation pixel group pairs and the correlation calculation pixel group with the correlation calculation pixel group on the n-line shifted to the left by half a pixel (n +) 1) At least one correlation calculation pixel group pair consisting of a correlation calculation pixel group obtained by shifting the correlation calculation pixel group on the line by half a pixel to the right may be further selected.

Here, the process of deriving the information of the correlation calculation pixel group pair of each set shown in FIG. 18 will be described. First, in order to derive the information of the first set of correlation calculation pixel group pairs, the information of pixel A4 and the information of pixel A5 are extracted from the information of the first reference pixel group, and the information of the second reference pixel group is used. The information of pixel B1 and the information of pixel B2 are extracted. In the filter circuit 56, the information of the pixel A4 and the information of the pixel A5 are linearly interpolated (averaged) to generate the information of the interpolated pixel A4.5, and in the filter circuit 57, the information of the pixel B1 and the information of the pixel B2 are linearly interpolated. (Average) to generate the information of the interpolation pixel B1.5. As a result, information on all the correlation calculation pixels constituting the first set of correlation calculation pixel group pairs is derived.

Next, in order to derive the information of the second set of correlation calculation pixel group pairs, the information of pixel A3, the information of pixel A4, and the information of pixel A5 are extracted from the information of the first reference pixel group, and the second Information on pixel B1, information on pixel B2, and information on pixel B3 are extracted from the information of the reference pixel group. In the filter circuit 56, the information of the pixel A3 and the information of the pixel A4 are linearly interpolated (averaged) to generate the information of the interpolated pixel A3.5, and the information of the pixel A4 and the information of the pixel A5 are linearly interpolated (averaged). And generate the information of the interpolation pixel A4.5. Similarly, in the filter circuit 57, the information of the pixel B1 and the information of the pixel B2 are linearly interpolated (averaged) to generate the information of the interpolated pixel B1.5, and the information of the pixel B2 and the information of the pixel B3 are linearly interpolated. (Average) to generate the information of the interpolated pixel B2.5.

Next, in the same manner as the process of deriving the information of the first set of correlation calculation pixel group pairs or the process of deriving the information of the second set of correlation calculation pixel group pairs, the third set to the seventh set Correlation calculation Pixel group pair information is derived.

Here, a method of calculating the difference with respect to the correlation calculation pixel group pair of each set shown in FIG. 18 will be described. For the first set of correlation calculation pixel group pairs, the first reference data for the first set of correlation calculation pixel group pairs from the buffer 16 (data of pixel A4, data of interpolated pixel A4.5 and data of pixel A5). ) First data (data of pixel A4) and second reference data for the first set of correlation calculation pixel group pairs from buffer 17 (data of pixel B1, data of interpolated pixel B1.5 and data of pixel B2). The difference calculation is executed with the first data (data of pixel B1) of. Subsequently, in the first reference data and the second reference data, the second data (data of pixel A4.5) and the second data (data of pixel B1.5) shifted to the right by one data, respectively. The difference operation with the data) is executed. Subsequently, in the first reference data and the second reference data, the third data (data of pixel A5) and the third data (data of pixel B2) further shifted to the right by one data, respectively. The difference operation is performed between. As a result, the difference with respect to the first set of correlation calculation pixel group pairs is calculated. In the same way, the difference with respect to the correlation calculation pixel group of the 2nd group to the 7th group is calculated.

According to the image processing apparatus according to the seventh embodiment, the interpolation reference pixel group pair having the strongest correlation is selected from the plurality of sets of correlation calculation pixel group pairs, and the original image n-line for the interpolation reference pixel group pair is selected. The conventional interpolation value calculated from the correlation of a plurality of pixel pairs by averaging the pixel value of the center pixel and the pixel value of the center pixel on the original image (n + 1) line to obtain the interpolation value of the pixel to be interpolated. More accurate interpolated values can be obtained.

In the above, an example of an image processing method in which interpolation search is performed in half pixel units has been described, but the correlation value is calculated by performing interpolation search in 1 / n pixel units such as 1/3 pixel and 1/4 pixel. May be good. When the correlation value is obtained by performing an interpolation search in units of 1 / n pixels, for example, interpolation pixel A (n + 1) / n and interpolation pixel A (n + 2) / between pixel A1 and pixel A2. (n-1) interpolation pixels consisting of n, ..., Interpolation pixels A (n + n-1) / n are virtually inserted.

Further, in the above, the information of the two adjacent left and right pixels is linearly interpolated to generate the information of the interpolated pixel, but the information of the interpolated pixel is generated by filtering from the information of the plurality of left and right pixels located in the vicinity. You may.

Further, in the above, the case where each of the first correlation calculation pixel group pair and the second correlation calculation pixel group pair is a correlation calculation pixel group pair consisting of an odd number of correlation calculation pixels has been described. In the same manner as in Form 6, each of the first correlation calculation pixel group pair and the second correlation calculation pixel group pair can be configured to include an even number of correlation calculation pixels.

Further, in the above, the case where the single interpolation search is performed has been described, but the multiple interpolation search (duplicate interpolation search) can be performed in the same manner as in the second embodiment, and the above-described embodiment can be performed. A multi-step interpolation search (hierarchical interpolation search) can be performed in the same manner as in the third embodiment.

Further, in the above, the case where the correlation value is calculated using the correlation calculation pixel group pair consisting of a plurality of consecutive correlation calculation pixels as each of the first correlation calculation pixel group pair and the second correlation calculation pixel group pair. As described above, in the same manner as in the above-described embodiment 4 or 5, a correlation calculation pixel group composed of a plurality of discontinuous correlation calculation pixels is used as the first correlation calculation pixel group and the second correlation calculation pixel group. It is also possible to calculate the correlation value.

The present invention can be used to improve the interpolation accuracy when interpolating and enlarging an image in an image processing apparatus and an image processing method. Further, the present invention can be used in an image processing apparatus and an image processing method to improve the image quality of an enlarged image. Further, in a video display device such as a television or a personal computer display monitor, by installing the image processing device of the present invention, an interlaced scanning image can be progressively converted and displayed, or a reduced image can be enlarged and displayed. It can be used to improve the display performance.

<figref num="1">FIG. 1 is a block diagram conceptually showing the configuration of the image processing apparatus according to the first embodiment.</figref><figref num="2">FIG. 2 is an explanatory diagram for explaining an image processing method of the image processing apparatus according to the first embodiment.</figref><figref num="3">FIG. 3 is an explanatory diagram for explaining a process of calculating a correlation value for a first set of correlation calculation pixel group pairs in the image processing method of the image processing apparatus according to the first embodiment.</figref><figref num="4">FIG. 4 is an explanatory diagram for explaining a process of calculating a correlation value for a second set of correlation calculation pixel group pairs in the image processing method of the image processing apparatus according to the first embodiment.</figref><figref num="5">FIG. 5 is an explanatory diagram for explaining a process of calculating a correlation value for a third set of correlation calculation pixel group pairs in the image processing method of the image processing apparatus according to the first embodiment.</figref><figref num="6">FIG. 6 is a flowchart for explaining an image processing method of the image processing apparatus according to the second embodiment.</figref><figref num="7">FIG. 7 is a schematic plan view showing a frame image for explaining the effect of the image processing apparatus according to the second embodiment.</figref><figref num="8">FIG. 8 is an explanatory diagram for explaining a wide area correlation calculation pixel group pair in the image processing method of the image processing apparatus according to the second embodiment.</figref><figref num="9">FIG. 9 is a flowchart for explaining an image processing method of the image processing apparatus according to the third embodiment.</figref><figref num="10">10 (a) and 10 (b) are explanatory views for explaining a two-layer interpolation search in the image processing method of the image processing apparatus according to the third embodiment.</figref><figref num="11">FIG. 11 is a block diagram conceptually showing the configuration of the image processing apparatus according to the fourth embodiment.</figref><figref num="12">FIG. 12 is an explanatory diagram for explaining the thinned-out correlation calculation pixel group pair in the image processing method of the image processing apparatus according to the fourth embodiment.</figref><figref num="13">FIG. 13 is an explanatory diagram for explaining a correlation calculation pixel group pair thinned out at pseudo equal intervals in another image processing method of the image processing apparatus according to the fourth embodiment.</figref><figref num="14">FIG. 14 is a block diagram conceptually showing the configuration of the image processing apparatus according to the fifth embodiment.</figref><figref num="15">FIG. 15 is an explanatory diagram for explaining a method of calculating a correlation value for a pair of correlation calculation pixels in the image processing method of the image processing apparatus according to the fifth embodiment.</figref><figref num="16">16 (a) to 16 (c) are explanatory views for explaining a method of calculating a correlation value with respect to a pair of correlation calculation pixels in the image processing method of the image processing apparatus according to the sixth embodiment.</figref><figref num="17">FIG. 17 is a block diagram conceptually showing the configuration of the image processing apparatus according to the seventh embodiment.</figref><figref num="18">FIG. 18 is an explanatory diagram for explaining a method of calculating a correlation value for a pair of correlation calculation pixels in the image processing method of the image processing apparatus according to the seventh embodiment.</figref><figref num="19">FIG. 19 is an explanatory diagram for explaining the conventional third interpolation method.</figref><figref num="20">FIG. 20 is a schematic plan view showing a frame image including an image having a thin linear contour.</figref><figref num="21">FIG. 21 is a schematic partial plan view showing an enlarged thin linear outline portion of the frame image.</figref>

Code description

1 Reference pixel information extraction unit 2 Correlation calculation unit 3 Interpolation reference pixel group vs. selection unit 4 Interpolation unit 5 Pixel information holding unit 11 Input terminal 12 1H Delay circuit 13,14 Line memory 15 Address control circuit 16,17 Buffer 18 Difference circuit 19 Absolute value circuit 20 Addition circuit 21 Minimum value circuit 22 Interpolation value calculation circuit 23 Output terminal 30 Thinning circuit 40 Weighting circuit 51 Pixel information derivation unit 56, 57 Filter circuit