Interpolating lines of video signals.

0001This invention relates to a method of and apparatus for interpolating missing lines of a video signal obtained by line scanning such as a broadcast television signal.

0002Many television processes require the interpolation of television (TV) pictures, to create a signal like one which would have been generated if the picture had been scanned in a different way. For example digital video effects systems change the size and shape of TV pictures, and standards conversion changes the number of lines per field and/or the number of fields per second. There is also considerable potential benefit to the television viewer from display systems which increase the number of lines and/or fields in the display to reduce the visibility of line structure, flicker and twitter.

0003Spatial interpolation, that is, interpolation within a single field, would be relatively straightforward if it were not for the use of interlace in all current broadcast TV systems. Each field of an interlaced TV picture contains only half of the lines of a complete picture. This makes interpolation difficult because the lines of each field do not contain the full vertical resolution. The missing information is carried by the interlaced lines of the adjacent fields, but these may differ from the current field because of movement.

0004This specification is concerned with a method of interpolating the "missing" lines needed to convert interlaced pictures into sequentially scanned pictures. Once the missing lines have been added any further interpolation is straightforward. Any subsequent interpolation may in practice be combined with the interpolation described in this specification, but the two operations are considered separately for ease of description.

0005Characteristics of known methods are illustrated in Figures 1 and 2. In Figure 1, the various diagrams are plots showing the vertical positions of lines on the Y-axis against time in terms of fields on the X-axis. In Figure 1 each input line is shown by an X and each "missing" line to be generated as an output line by an 0. The "current" output line is assumed to be the 0 with a dot in it. Figure 2 shows the response to vertical detail of the different systems, with the response plotted on the Y-axis in terms of a percentage of perfect (100%) response, and vertical frequency plotted on the X-axis in terms of a percentage of the maximum definition of which the system is capable in the vertical direction, i.e., in the 625 line TV system used for broadcasting in the U.K., 100% is 312.5 cycles per picture height.

0006If the missing lines are interpolated from lines of the current field, as illustrated at (a) in Figure 1 of the drawings, then the vertical resolution is limited, as shown at (a) in Figure 2. If on the other hand the missing lines are interpolated from the adjacent fields, as illustrated at (b) in Figure 1, then although the response at 0 Hz (stationary pictures) is perfect, as shown at (b) in Figure 2, at all other temporal frequencies the response falls off, becoming zero at 25 Hz. These higher temporal frequency components are created by movement and their removal results in serious movement blur, which is roughly equivalent to doubling the integration time of the camera. Combinations of the two methods, as illustrated at (c) in Figure 1, usually give a combination of impairments. This is shown at (c) and (d) in Figure 2, which show the vertical frequency responses at 0 and 25 hz respectively.

0007This has in the past led to the assumption that some form of adaption, to distinguish moving areas of the picture from stationary areas, is essential, so that the most appropriate form of interpolation can be used in each area. We have now appreciated that it is after all possible to devise a combined spatio-temporal interpolator which gives improved vertical resolution, without any subjectively-serious movement blur.

0008The present invention is defined in the appended claims to which reference should now be made.

0009Thus in accordance with this invention we arrange that the total contribution from the adjacent fields contains little or no low frequency components. This is achieved by ensuring that the con­tributions from the adjacent fields are multiplied by coefficents which sum substantially to zero.

0010In this way the low vertical frequencies come solely from the current field and are thus free of movement blur. Higher vertical frequency components come partly from the current field and partly from the adjacent fields. An improved vertical resolution is thus achieved, on stationary pictures, by the incorporation of information from adjacent fields. On moving pictures the contribution from the adjacent fields is out of phase and the vertical resolution is reduced (by the same amount as the increase on stationary pictures) but this loss of vertical detail on moving pictures is subjectively much less serious than the movement blur produced by previous methods.

0011Embodiments of the invention will now be described by way of example with reference to the remaining figures of the drawings, in which:- <ul id="ul0001" list-style="none"><li><u style="single">Figure 3</u> is a plot similar to Figure 1 of an arrangement embodying the invention showing vertical positions of lines on the Y-axis against time in terms of fields on the X-axis;</li><li><u style="single">Figure 4</u> illustrates the vertical frequency response of the system of Figure 3 for different rates of movement, with the response plotted on the Y-axis against the picture content in terms of cycles per picture height on the X-axis;</li><li><u style="single">Figure 5</u> is a plot similar to Figure 3 of an alternative arrangement embodying the invention;</li><li><u style="single">Figure 6</u> illustrates the vertical frequency response of the system of Figure 5 in similar form to Figure 4; and</li><li><u style="single">Figure 7</u> is a block circuit diagram of the embodiment of the invention illustrated in Figure 5 and Figure 6.</li></ul>

0012Figure 3 illustrates a spatio-temporal interpolator embodying this invention. The signals from the adjacent field are multiplied by weighting coefficients which sum to zero. As shown, to form each missing output line contributions are taken in the following proportions: 1/2 from each of the lines above and below in the current field, 1/8 from each of the lines at the same position in the two adajacent fields, and -1/16 from each of the lines displaced by two picture lines in each of the two adjacent fields.

0013Thus the nett contribution from each adjacent field is 1/8 - (2 x 1/16) which is zero. The resulting vertical frequency responses at 0 and 25 Hz are shown at (a) and (b) in Figure 4. Compared with (a) of Figure 2 the response at 0 Hz has been improved, at the expense of the response at 25 Hz. This is because at 0 Hz the vertical frequency response, (c) in Figure 4, of the contributions from the adjacent field is in phase with the response (d) of the contributions from the current field, whereas at 25 Hz they are in antiphase. The vertical bandwidth is thus reduced on moving objects, but this is not a very visible defect because the spatial resolution of the eye is reduced by movement. It is much more important that 100% response has been maintained at low vertical frequencies, so there is no movement blur.

0014Figure 3 is just one of a family of useful spatio-temporal interpolators. Figure 5 illustrates a more complex (but not necessarily the optimum) interpolator, which combines more signals from both the current and adjacent fields, as shown on the figure. In Figure 5 contributions are taken in the following relative proportions:

0015From each of the lines immediately above and below in the current field : + 0.526.

0016From each of the lines displaced by three picture lines in the current field: - 0.026.

0017From each of the lines at the same position in the two adjacent fields : + 0.170.

0018From each of the lines displaced by two picture lines in each of the two adjacent fields: - 0.116.

0019From each of the lines displaced by four picture lines in each of the two adjacent fields: + 0.031.

0020The arrangement of Figure 5 allows the flatter, sharper cut-off responses shown at (a) and (b) in Figure 6 to be achieved. Once again the signals from the adjacent fields are multiplied by coefficents which sum to zero, so that their contribution to the vertical frequency response ((c) in Figure 5) has now low frequency component. The low frequencies all come from the current field, whose contribution to the overall frequency response is shown at (d) in Figure 6. Even more complex interpolators are of course possible, allowing the position and rate of cut-off to be finely controlled, to achieve the optimum performance.

0021Circuits to implement the arrangements of Figures 3 and 5 can be implemented in known manner by an appropriate combination of line delays, field delays and multipliers. A circuit implementing the interpolation arrangement of Figure 5 is shown in Figure 7. This example of the invention is a system for interpolating the missing lines which must be added to the interlaced input to create a 625 line sequential output.

0022The circuit shown in Figure 7 has an input 1 for receiving the 625 line interlaced signal. Two delay devices 5 and 6 are connected in series to the input 1 and provide delays of 313 and 312 lines respectively. The output 2 of the first delay device 5 constitutes the current field and the input 1 and the output of the second delay device 6 constitute the succeeding and preceding adjacent fields respectively. A halving adder 7 is connected to the input 1 and the output of the second delay device 6 to add the two adjacent field signals and have the resultant. Both these signals require the same processing and the total number of components required can thereby be reduced.

0023Three one-line delay devices 8a, 8b and 8c are connected in series to the output of the first delay device 5 to receive the current field signal 2. A halving adder 11 adds to outputs of delay devices 8a and 8b and divides the resultant by two. A halving adder 14 adds the outputs of delay devicees 5 and 8c and divides the resultant by two. A subtractor 15 has its non-­inverting input connected to the output of adder 14 and its inverting input connected to the output of adder 11. A multiplier 16 receives the output of subtractor 15 and multiplies by a fixed factor of - 0.052. An adder 17 adds the output of multiplier 16 to the output of adder 11.

0024Four further one-line delay devices 8e to 8h are connected in series to the output of adder 7 supplying the combined adjacent field signal. A halving adder 18 combines and halves the outputs of delay devices 8e and 8g, and a halving adder 19 combines and halves the outputs of adder 7 and delay device 8h. A subtractor 21 has its non-inverting input connected to the output of adder 18 and its inverting input connected to the output of delay device 8f, and has its output connected to a multiplier 23 which receives a fixed factor of - 0.464. A subtractor 20 has its non-inverting input connected to the output of adder 18 and its inverting input connected to the output of delay 8f, and has its output connected to a multiplier 22 which receives a fixed factor of 0.124. An adder 24 adds the outputs of adder 17 and a multiplier 23 and an adder 25 adds the outputs of adder 24 and multiplier 22. The output 26 of the adder 25 constitutes the output of the circuit.

0025Thus in operation, considering first the current field signal 2, delay devices 8a and 8b will have at their outputs the two nearest lines 9 and 10 to the position that would be adopted by the missing line to be generated, see Figure 5. These are combined in the adder 11. The lines 12 and 13 preceding and following these nearest two lines to the current missing line position are combined in halving adder 14. For low vertical frequencies where the information is effectively unchanged on successive lines the outputs of the two halving adders 11 and 14 will sum to zero in the subtractor 15. The output of this subtractor therefore comprises any high frequency components in the resultant signal and is then put through the multiplier 16 with an appropriate weighting factor. The output of the multiplier 16 is then combined with the output of the halving adder 11 in the further adder 17 to produce a combined output of low and high vertical frequency components from the current field.

0026The weighting factor applied by multiplier 16 is - 0.052. The output of the halving adder 11 goes to the negative input of the subtractor 15. Thus, a signal of magnitude 1 is inverted and then given a weighting of - 0.052 by multiplier 16. This produces a signal of resultant magnitude 0.052 from the two lines 9 and 10 nearest to the current missing line position. When this signal is combined with the output of halving adder 11 in adder 17, the total contribution from lines 9 and 10 is 1.052. This is equivalent to a contribution of 0.526 for each of lines 9 and 10. This corresponds to the weighting given to these lines in Figure 5. Lines 12 and 13 are combined in halving adder 14, the output of which is connected to the positive input of subtractor 15. The weighting factor of - 0.052 is applied by multiplier 16 thus giving a total contribution of - 0.052 for these two lines. This is equivalent to a contribution of - 0.026 per line as indicated on Figure 5.

0027The output of adder 17 can thus be calculated in terms of the contributions made by lines 9, 10, 12 and 13: <tables id="tabl0001" num="0001"><img file="EP0266079A2_D0001.tif" /></tables>

0028This is true for cases where all four lines are identical. The addition of higher frequency components will lead to variations around unity for the total contribution made by the current field.

0029The adjacent field signals 3 combined in adder 7 are processed in a similar manner to the current field signal. The halving adders 18 and 19 are used to combine lines which require the same weighting factors. The total low frequency contribution required from the adjacent fields is zero. Therefore, the outputs from the single line delay devices 8e, 8g, 8h and the output from halving adder 7 are combined in the two halving adders 18 and 19 respectively. These two halving adders provide the positive inputs to two subtractors 21 and 20 respectively. The negative inputs to these two subtractors comes from the outputs of the single line delay device 8f. This is the line in the adjacent fields equivalent in position to the missing line in the current field.

0030For low vertical frequency components, the outputs of these two subtractors 20 and 21 will be zero. If, however, higher vertical frequencies are present a non-zero output will be produced. Weighting factors shown in Figure 7 do not correspond directly with those shown on Figure 5. This is because of the manner in which the fields and lines have been combined. For example, four of the adjacent field lines in Figure 5 have a weighting factor of - 0.116. In figure 7, the two adjacent fields are combined in halving adder 7. The lines to which the weighting factor of - 0.116 must be applied are the outputs of the single line delay devices 8c and 8g. These outputs are combined in halving adder 18. This provides the positive input to subtractor 21 which in turn provides the input to multiplier 23 where a weighting factor of - 0.464 is applied. This is equal to 4X - 0.116 as indicated in Figure 5. This weighting will, of course, only apply to the higher vertical frequencies, the lower frequencies having been removed by the subtractors. Thus, it will be seen that the circuit of Figure 7 exploits the symmetries of the weighting coefficients shown in Figure 5 to reduce the total number of components by combining the two adjacent fields and by combining pairs of lines, before multiplication.

0031The outputs of the two multipliers 22 and 23 are then combined with the output of adder 17 in two further adders 24 and 25 to give the missing lines at the system output 26 which must be added to the interlaced input to create a 625 line sequential output.

0032It is thus possible to interpolate the missing lines of inter­laced TV pictures. This is achieved by taking contributions from the current field together with contributions from other, adjacent fields. The net contribution from these other fields contain no substantial low vertical spatial frequency contributions. The method illustrated gives good vertical resolution on stationary areas, and moving areas are not blurred, although they do suffer a slight loss of vertical resolution.