Picture signal transmitting method and apparatus

RELATED APPLICATION

0001This is a continuation-in-part of U.S. patent application Ser. No. 08/219,472, filed Mar. 29, 1994, now U.S. Pat. No. 5,473,380.

BACKGROUND OF THE INVENTION

0002The present invention relates to coding and decoding of a picture signal for transmission, and, more particularly, is directed to matching the type of predictive coding applied to pictures of the picture signal.

0003In, for example, a teleconferencing system or a video telephone system, moving picture signals are compressed and encoded by taking advantage of intra-frame and inter-frame correlation so that they can be more efficiently transmitted over a communication channel to a remote location.

0004Intra-frame correlation can be utilized by an orthogonal transformation, such as a discrete cosine transformation (DCT).

0005Inter-frame correlation can be utilized by predictive encoding between successive pictures. As used herein, a picture generally refers to an image represented by a frame. When the fields of a frame are coded in a non-interlaced manner, that is, separately, each field may be referred to as a picture.

0006As shown in <figref idref="DRAWINGS">FIG. 1A</figref>, for example, frame pictures PC<b>1</b>, PC<b>2</b> and PC<b>3</b> are generated at time points t<b>1</b>, t<b>2</b> and t<b>3</b>. As shown by shading in <figref idref="DRAWINGS">FIG. 1B</figref>, the difference between the frame pictures PC<b>1</b> and PC<b>2</b> is obtained as difference picture data PC<b>12</b>, and the difference between the frame pictures PC<b>2</b> and PC<b>3</b> is obtained as difference picture data PC<b>23</b>. Since there is a fairly small change between signals of temporally neighboring frames, transmission of only the difference picture data utilizes the transmission channel more efficiently than transmission of the original pictures. That is, using the difference picture data as encoded picture signals reduces the amount of data to be transmitted.

0007However, if only the difference signals are transmitted, the original picture cannot be restored. It is advantageous to occasionally transmit a picture which is not predictively encoded as a reference for difference picture data, and because it is sometimes more efficient than transmitting the picture as a predictively encoded picture.

0008Pictures which are encoded utilizing only intra-frame correlation and not inter-frame correlation, are referred to herein as intra-pictures or I-pictures.

0009Pictures which are encoded with predictive encoding relative to one previously encoded picture are referred to herein as predictive pictures or P-pictures. The previously encoded picture may be an I-picture or a P-picture, and temporally succeeds the P-picture.

0010Pictures which are encoded with predictive encoding relative to at most two pictures, a temporally preceding and a temporally succeeding picture, are referred to herein as bi-directionally predictive coded pictures or B-pictures. The two pictures may each be an I-picture or a P-picture. When both are used, the mean value of the two pictures is obtained and used as a reference picture for the picture to be encoded.

0011A series of pictures may be considered as groups of pictures having a predetermined number of frames such as F<b>1</b> . . . F<b>17</b>. The luminance and chrominance picture signals of the leading frame F<b>1</b> are encoded as an I-picture, the picture signals of the second frame F<b>2</b> are encoded as a B-picture, and the picture signals of the third frame F<b>3</b> are encoded as a P-picture. The fourth and the following frames F<b>4</b> to F<b>17</b> are encoded alternately as B-pictures and P-pictures. <figref idref="DRAWINGS">FIG. 2A</figref> shows the reference pictures used for encoding P-pictures, while <figref idref="DRAWINGS">FIG. 2B</figref> shows the reference pictures used for encoding B-pictures.

0012As shown in <figref idref="DRAWINGS">FIGS. 3A and 3B</figref>, there are four methods for encoding the macro-blocks (discussed below) of a picture. When multiple methods are suitable, the method which will give the smallest amount of encoded data is employed on a macro-block by macro-block basis. Blocks F<b>1</b> to F<b>5</b> in <figref idref="DRAWINGS">FIG. 3A</figref> represent data for frames of moving picture signals, whereas blocks F<b>1</b>X to F<b>5</b>X in <figref idref="DRAWINGS">FIG. 3B</figref> represent data for encoded frames. The solid line arrows in <figref idref="DRAWINGS">FIG. 3A</figref> show the frames to which motion vectors x<b>1</b> . . . x<b>6</b> relate.

0013The first method, shown as SP<b>1</b>, is to not use predictive encoding, that is, to use only intra-frame correlation. This is suitable for any macro-blocks of an I-picture, a P-picture and a B-picture. In other words, if less encoded data is produced without predictive encoding, then this method is selected.

0014The second method, shown as SP<b>2</b>, is to predictively encode relative to a picture which temporally succeeds the current picture, referred to as backward predictive encoding. The third method, shown as SP<b>3</b>, is to predictively encode relative to a picture which temporally precedes the current picture, referred to as forward predictive encoding. The second method is suitable for macro-blocks of only B-pictures. The third method is suitable for macro-blocks of P-pictures and B-pictures.

0015The fourth method, shown as SP<b>4</b>, is to predictively encode relative to the mean value of two pictures, one temporally preceding and one temporally succeeding the current picture. This method is suitable for macro-blocks of only B-pictures.

0016The encoding sequence will now be described.

0017The first frame F<b>1</b> is encoded as an I-picture using the first method SP<b>1</b> so that it is directly transmitted over a transmission channel as encoded data F<b>1</b>X.

0018The third frame F<b>3</b> is encoded as a P-picture. When the third method SP<b>3</b>, forward predictive coding, is used for a macro-block, difference signals from the temporally preceding frame F<b>1</b> used as the reference picture, as indicated by a broken-line arrow SP<b>3</b>, and a motion vector x<b>3</b> between the reference picture F<b>1</b> and the current picture F<b>3</b>, are calculated and encoded as data F<b>3</b>X for that macro-block. Alternatively, in this or another macro-block of the P picture, if a smaller amount of encoded data is produced for a macro-block of the P picture being encoded, the first method SP<b>1</b> can be used wherein the data of the original frame F<b>3</b> are directly utilized as the transmission data F<b>3</b>X for that macro-block.

0019The second frame F<b>2</b> is encoded as a B-picture.

0020When the fourth method SP<b>4</b> is used to encode a macro-block of the frame F<b>2</b>, a difference between the mean value of the temporally preceding frame F<b>1</b> and the temporally succeeding frame F<b>3</b> is calculated, on a pixel by pixel basis. The difference data and the motion vectors x<b>1</b> and x<b>2</b> are encoded as data F<b>2</b>X.

0021When the first processing method SP<b>1</b> is used to encode a macro-block of the frame F<b>2</b>, the data of the original frame F<b>2</b> forms the encoded data F<b>2</b>X.

0022When one of the second or third methods SP<b>2</b>, SP<b>3</b> is used to encode a macro-block of the frame F<b>2</b>, one of the difference between the temporally succeeding frame F<b>3</b> and the current frame F<b>2</b>, and the difference between the temporally preceding frame F<b>1</b> and the current frame F<b>2</b> is calculated. The difference data and one of the motion vectors x<b>1</b>, x<b>2</b> are encoded as the data F<b>2</b>X.

0023The frame F<b>4</b> for the B-picture and the frame F<b>5</b> for the P-picture are processed in a similar manner as described above to generate transmitted data F<b>4</b>X and F<b>5</b>X.

0024<figref idref="DRAWINGS">FIG. 4</figref> illustrates an arrangement for encoding and decoding moving picture signals in accordance with the above-described predictive encoding scheme. As shown in <figref idref="DRAWINGS">FIG. 4</figref>, an encoding device <b>1</b> encodes input picture signals and transmits the encoded signals to a recording medium <b>3</b> as a transmission channel for recording. A decoding device <b>2</b> reproduces the signals recorded on the recording medium <b>3</b> and decodes these as output signals.

0025The encoding device <b>1</b> includes an input terminal <b>10</b>, a pre-processing circuit <b>11</b>, A/D converters <b>12</b> and <b>13</b>, a frame memory <b>14</b> including a luminance signal frame memory <b>15</b> and a color difference signal frame memory <b>16</b>, a format converting circuit <b>17</b> and an encoder <b>18</b>.

0026Input terminal <b>10</b> is adapted to receive a video signal VD and to supply the signal VD to pre-processing circuit <b>11</b> which functions to separate the video signal VD into luminance signals and color signals, herein chrominance or color difference signals, that are applied to analog-to-digital (A/D) converters <b>12</b> and <b>13</b>, respectively. The video signals, digitized by analog-to-digital conversion by the A/D converters <b>12</b> and <b>13</b>, are supplied to frame memory <b>14</b> having memories <b>15</b>, <b>16</b> which function to store the luminance signals and the color difference signals, respectively, and to read out the signals stored therein to format converting circuit <b>17</b>.

0027The converter <b>17</b> is operative to convert frame format signals stored in the frame memory section <b>14</b> into block format signals. As shown in <figref idref="DRAWINGS">FIG. 5A</figref>, pictures are stored in the frame memory section <b>14</b> as frame-format data having V lines each consisting of H dots. The converting circuit <b>17</b> divides each frame into N slices, each slice comprising a multiple of <b>16</b> lines. As shown, in <figref idref="DRAWINGS">FIG. 5B</figref>, the converter <b>17</b> divides each slice into M macro-blocks. As shown in <figref idref="DRAWINGS">FIG. 5C</figref>, each macro-block represents luminance signals Y corresponding to 16×16 pixels or dots, and associated chrominance Cr, Cb signals. These luminance signals are subdivided into blocks Y<b>1</b> to Y<b>4</b>, each consisting of 8×8 dots. The 16×16 dot luminance signals are associated with 8×8 dot Cb signals and 8×8 dot Cr signals. The converter <b>17</b> is also operative to supply the block format signals to the encoder <b>18</b>, which is described in detail below with reference to FIG. <b>6</b>.

0028The encoder <b>18</b> operates to encode the block format signals and to supply the encoded signals as a bitstream over a transmission channel for recording on the recording medium <b>3</b>.

0029The decoding device <b>2</b> includes a decoder <b>31</b>, a format converting circuit <b>32</b>, a frame memory section <b>33</b> including a luminance signal frame memory <b>34</b> and a color difference signal frame memory <b>35</b>, digital-to-analog converters <b>36</b> and <b>37</b>, a post-processing circuit <b>38</b> and an output terminal <b>30</b>.

0030The decoder <b>31</b> is operative to reproduce encoded data from the recording medium <b>3</b> and to decode the encoded data, as described in detail below with reference to <figref idref="DRAWINGS">FIG. 9</figref>, and to supply decoded data signals to format converting circuit <b>32</b> which is operative to convert the decoded data signals into frame format data signals and to supply the frame format data signals as luminance signals and color difference signals to the memory <b>33</b>. The memories <b>34</b>, <b>35</b> of the memory <b>33</b> function to store the luminance and chrominance signals, respectively, and to apply these signals to D/A converters <b>36</b> and <b>37</b>, respectively. The analog signals from converters <b>36</b>, <b>37</b> are synthesized by a post-processing circuit <b>38</b> which functions to form output picture signals and to output them to output terminal <b>30</b>, and thence to a display unit, such as a CRT, not shown, for display.

0031<figref idref="DRAWINGS">FIG. 6</figref> illustrates the encoder <b>18</b> shown in FIG. <b>4</b>.

0032Generally, the encoder <b>18</b> stores three pictures, the current picture and the pictures temporally preceding and succeeding the current picture. Based on the sequential position of the current picture in the group of pictures, the picture coding type (I, P or B) is selected for each picture. The picture type sequence is determined by a user using picture type input device <b>65</b>, independent of the pictures applied to an input terminal <b>49</b>.

0033The encoder <b>18</b> also chooses one of frame-based and field-based predictive encoding as will be explained with reference to <figref idref="DRAWINGS">FIG. 7</figref>, and further chooses one of frame-based and field-based DCT encoding as will be explained with reference to FIG. <b>8</b>. For each picture, appropriate motion vectors are obtained and the picture is predictively encoded relative to zero, one or two previously encoded pictures which have been locally decoded and which are referred to as reference pictures to form a difference data signal. The difference data signal is orthogonally transformed into blocks of coefficient data which are quantized, variable length encoded and transmitted as encoded data.

0034At the encoder <b>18</b>, the quantized data are dequantized, inverse orthogonally transformed, and stored as the reference pictures. The predictive encoding applies the motion vector(s) obtained for the current picture to the reference picture(s) to produce a prediction picture which is subtracted from the current picture to yield the difference data.

0035The elements of the encoder <b>18</b> will now be explained in detail.

0036Picture data for encoding is supplied macro-block by macro-block to the input terminal <b>49</b> and thence to a motion vector detection circuit <b>50</b> which is operative to process the picture data of respective frames as I-pictures, P-pictures or as B-pictures, in accordance with a predetermined sequence for each group of pictures, as shown for example, in <figref idref="DRAWINGS">FIGS. 2A</figref>, <b>2</b>B. The circuit <b>50</b> applies the picture data of the current frame to a frame memory <b>51</b> having frame memories <b>51</b>a, <b>51</b>b, <b>51</b>c used for storing a temporally preceding picture, the current picture and a temporally succeeding picture, respectively.

0037More specifically, the frames F<b>1</b>, F<b>2</b>, F<b>3</b> are stored in the memories <b>51</b>a, <b>51</b>b, <b>51</b>c, respectively. Then the picture stored in memory <b>51</b>c is transferred to memory <b>51</b>a. The frames F<b>4</b>, F<b>5</b> are stored in the memories <b>51</b>b, <b>51</b>c, respectively. The operations of transferring the picture in memory <b>51</b>c to memory <b>51</b>a and storing the next two pictures in memories <b>51</b>b, <b>51</b>c are repeated for the remaining pictures in the group of pictures.

0038After the pictures are read into the memory and temporarily stored, they are read out and supplied to a prediction mode changeover circuit <b>52</b> which is adapted to process the current picture for one of frame based and field based predictive encoding. After processing the first frame picture data in a group of pictures as an I-picture and before processing the second frame picture as a B-picture, the motion vector detection circuit <b>50</b> processes the third frame P-picture. The processing sequence is different from the sequence in which the pictures are supplied because the B-picture may involve backward prediction, so subsequent decoding may require that the P-picture temporally succeeding the B-picture have been previously decoded.

0039The motion vector detection circuit <b>50</b> calculates as an estimated value for intra-coding for each macro-block, the sum of absolute values of prediction errors for the frame prediction mode for each macro-block and the sum of absolute values of prediction errors for the field prediction mode for each macro-block and supplies these sums to the prediction decision circuit <b>54</b> which compares these sums and selects frame prediction mode or field prediction mode in accordance with the smallest of these values and provides the selected mode to the prediction mode changeover circuit <b>52</b>.

0040If the frame prediction mode is selected, the prediction mode changeover circuit <b>52</b> outputs the four luminance blocks Y<b>1</b> to Y<b>4</b> and the two chrominance or color difference blocks Cb, Cr of each macro-block received from the motion vector detection circuit <b>50</b> without processing. As shown in <figref idref="DRAWINGS">FIG. 7A</figref>, odd or first field line data, indicated by solid lines, and even or second field line data, indicated by dashed lines, alternate in each luminance and color difference block as received from the motion vector detection circuit <b>50</b>. In <figref idref="DRAWINGS">FIG. 7A</figref>, a indicates units for motion compensation. In the frame prediction mode, motion compensation is performed with four luminance blocks (macro-blocks) as a unit and a single motion vector is associated with the four luminance blocks Y<b>1</b> to Y<b>4</b>.

0041If the field prediction mode is selected, the prediction mode changeover circuit <b>52</b> processes the signals received from the motion vector detection circuit <b>50</b> so that each of the four luminance blocks comprises data from a single field and the two color difference blocks have non-interlaced odd and even field data. Specifically, as shown in <figref idref="DRAWINGS">FIG. 7B</figref>, the luminance blocks Y<b>1</b> and Y<b>2</b> have odd-field data and the luminance blocks Y<b>3</b> and Y<b>4</b> have even-field data, while the upper halves of the color difference blocks Cb, Cr represent odd field color difference data for the luminance blocks Y<b>1</b> and Y<b>2</b> and the lower halves of the color difference blocks Cb, Cr represent even field color difference data for the luminance blocks Y<b>3</b> and Y<b>4</b>. In <figref idref="DRAWINGS">FIG. 7B</figref>, b indicates units for motion compensation. In the field prediction mode, motion compensation is performed separately for the odd-field blocks and even-field blocks so that one motion vector is associated with the two luminance blocks Y<b>1</b> and Y<b>2</b> and another motion vector is associated with the two luminance blocks Y<b>3</b> and Y<b>4</b>.

0042The prediction mode changeover circuit <b>52</b> supplies the current picture, as processed for frame based or field based predictive encoding, to arithmetic unit <b>53</b> of FIG. <b>6</b>. The arithmetic unit <b>53</b> functions to perform one of intra-picture prediction, forward prediction, backward prediction or bi-directional prediction. A prediction decision circuit <b>54</b> is adapted to select the best type of prediction in dependence upon the prediction error signals associated with the current picture signals.

0043The motion vector detection circuit <b>50</b> calculates, for the current picture, the sum of absolute values of the differences between each Aij and the average value of the Aij in each macro-block Σ|Aij−(average of Aij)| and supplies the sum as an estimated value for intra-coding to the prediction decision circuit <b>54</b>.

0044The motion vector detection circuit <b>50</b> calculates the sum of absolute values (or sum of squares) of the difference (Aij−Bij) between signals Aij of the macro-blocks of the current picture, and signals Bij of the macro-blocks of the prediction picture Σ|Aij−Bij| in each of frame prediction mode and field prediction mode. As explained above, the motion vector(s) for the current picture are applied to the reference picture(s) to generate the prediction picture. When the reference picture temporally precedes the current picture, the quantity Σ|Aij−Bij| is referred to as a forward prediction error signal, and when the reference picture temporally succeeds the current picture, the quantity Σ|Aij−Bij| is referred to as a backward prediction error signal. When the prediction picture is the mean of a temporally preceding and a temporally succeeding reference picture, as motion-compensated, the quantity Σ|Aij−Bij| is referred to as a bi-directional prediction error signal.

0045The circuit <b>50</b> supplies the forward frame prediction, the forward field prediction, the backward frame prediction, the backward field prediction, the bi-directional frame prediction and the bi-directional field prediction error signals to the prediction decision circuit <b>54</b>.

0046The prediction decision circuit <b>54</b> selects one of intra-coding, forward inter-picture prediction, backward inter-picture prediction or bi-directional inter-picture prediction and one of frame and field prediction mode in accordance with the smallest of the estimated value for intra-coding and the forward frame, the forward field, the backward frame, the backward field, the bi-directional frame and the bi-directional field prediction error signals. The arithmetic unit <b>53</b> predictively encodes the current picture, as processed by the frame or field changeover circuit <b>52</b>, in accordance with the prediction mode selected by the prediction decision circuit <b>54</b>.

0047The motion vector detection circuit <b>50</b> serves to calculate and supply the motion vector(s) associated with the selected prediction mode to a variable length encoding circuit <b>58</b> and a motion compensation circuit <b>64</b>, explained later.

0048The sums of the absolute values of the inter-frame differences (prediction errors) on the macro-block basis are supplied from the motion vector detection circuit <b>50</b> to the prediction mode changeover circuit <b>52</b> and to the prediction decision circuit <b>54</b>, in the manner as described above.

0049The arithmetic unit <b>53</b> supplies predictively encoded data, also referred to as difference data, for the current picture to a DCT mode changeover circuit <b>55</b> which is adapted to process the current picture for one of frame based and field based orthogonal transformation.

0050The DCT changeover circuit <b>55</b> functions to compare the encoding efficiency when the DCT operations for the macro-blocks in a picture are performed with the odd field data alternating with the even field data, that is, for frame based orthogonal transformation, as shown in <figref idref="DRAWINGS">FIG. 8A</figref>, with the encoding efficiency when the DCT operations for the macro-blocks in a picture are performed with the odd field data separated from the even field data, that is, for field based orthogonal transformation, as shown in FIG. <b>8</b>B. The circuit <b>55</b> functions to select the mode with the higher encoding efficiency.

0051To evaluate the encoding efficiency for frame based orthogonal transformation, the DCT mode changeover circuit <b>55</b> places the luminance macro-block data into interlaced form, as shown in <figref idref="DRAWINGS">FIG. 8A</figref>, and calculates the differences between the odd field line signals and even field line signals vertically adjacent to each other, and finds the sum of absolute values of the differences EFM, or the sum of squared values of the differences. <maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>EFM</mi><mo>=</mo><mrow><mrow><munderover><mo>∑</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mn>16</mn></munderover><mo>⁢</mo><mstyle><mtext> </mtext></mstyle><mo>⁢</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mn>16</mn></munderover><mo>⁢</mo><mrow><mo></mo><mrow><mrow><mi>o</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow><mo>-</mo><mrow><mi>e</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo></mrow></mrow></mrow><mo>+</mo><mrow><munderover><mo>∑</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mn>16</mn></munderover><mo>⁢</mo><mstyle><mtext> </mtext></mstyle><mo>⁢</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mn>16</mn></munderover><mo>⁢</mo><mrow><mo></mo><mrow><mrow><mi>e</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow><mo>-</mo><mrow><mi>o</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo></mrow></mrow></mrow></mrow></mrow></mtd><mtd><mrow><mi>Eq</mi><mo>.</mo><mstyle><mtext> </mtext></mstyle><mo>⁢</mo><mn>1</mn></mrow></mtd></mtr></mtable></math></maths><img file="USRE40415E_D0001.tif" />

0052To evaluate the encoding efficiency for field based orthogonal transformation, the DCT mode changeover circuit <b>55</b> places the luminance macro-block data into non-interlaced form, as shown in <figref idref="DRAWINGS">FIG. 8B</figref>, and calculates the differences between vertically adjacent odd field line signals and the differences between vertically adjacent even field line signals, and finds the sum of absolute values of the differences EFD, or the sum of squared values of the differences. <maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>EFD</mi><mo>=</mo><mrow><munderover><mo>∑</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mn>16</mn></munderover><mo>⁢</mo><mstyle><mtext> </mtext></mstyle><mo>⁢</mo><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mn>15</mn></munderover><mo>⁢</mo><mrow><mo>(</mo><mrow><mrow><mo></mo><mrow><mrow><mi>o</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow><mo>-</mo><mrow><mi>o</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo></mrow><mo>+</mo><mrow><mo></mo><mrow><mrow><mi>e</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow><mo>-</mo><mrow><mi>e</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mrow><mi>i</mi><mo>+</mo><mn>1</mn></mrow><mo>,</mo><mi>j</mi></mrow><mo>)</mo></mrow></mrow></mrow><mo></mo></mrow></mrow><mo>)</mo></mrow></mrow></mrow></mrow></mtd><mtd><mrow><mi>Eq</mi><mo>.</mo><mstyle><mtext> </mtext></mstyle><mo>⁢</mo><mn>2</mn></mrow></mtd></mtr></mtable></math></maths><img file="USRE40415E_D0002.tif" />

0053The DCT changeover circuit <b>55</b> compares the difference between the frame based and field based sums of the absolute values with a predetermined threshold and selects frame based DCT transformation if the difference EFM−EFD is less than the predetermined threshold.

0054If the frame prediction mode is selected in the prediction mode changeover circuit <b>52</b>, the probability is high that the frame DCT mode will be selected in the DCT mode changeover circuit <b>55</b>. If the field prediction mode is selected in the prediction mode changeover circuit <b>52</b>, the probability is high that the field DCT mode will be selected in the DCT mode changeover circuit <b>55</b>. However, since this is not necessarily the case, the prediction mode changeover circuit <b>52</b> sets the mode which will give the least value of the sum of the absolute values of prediction errors, while the DCT mode changeover circuit <b>55</b> sets the mode which will give the optimum orthogonal transformation encoding efficiency.

0055If frame based orthogonal transformation mode, also referred to as frame DCT mode, is selected, the DCT mode changeover circuit <b>55</b> functions to ensure that the four luminance blocks Y<b>1</b> to Y<b>4</b> and two color difference blocks Cb, Cr represent alternating or interlaced odd and even field lines, as shown in FIG. <b>8</b>A.

0056If field based orthogonal transformation mode, also referred to as field DCT mode, is selected, the DCT mode changeover circuit <b>55</b> functions to ensure that each of the luminance blocks represents only one field, and that each of the color difference blocks has segregated or non-interlaced odd and even field lines, as shown in FIG. <b>8</b>B.

0057The DCT mode changeover circuit <b>55</b> functions to output the data having the configuration associated with the selected DCT mode, and to output a DCT flag indicating the selected DCT mode to the variable length encoding circuit <b>58</b> and the motion compensation circuit <b>64</b>.

0058The DCT mode changeover circuit <b>55</b> supplies appropriately configured difference picture data to a DCT circuit <b>56</b> shown in <figref idref="DRAWINGS">FIG. 6</figref> which is operative to orthogonally transform it using a discrete cosine transformation into DCT coefficients, and to supply the DCT coefficient data to a quantization circuit <b>57</b> that functions to quantize the coefficient data with quantization steps selected in accordance with the volume of data stored in a transmission buffer <b>59</b> and to supply quantized data to a variable length encoding circuit <b>58</b>.

0059The variable length encoding circuit <b>58</b> is also supplied with the quantization step or scale data from the quantization circuit <b>57</b>, prediction mode data from the prediction decision circuit <b>54</b>, that is data indicating which of the intrapicture prediction, forward prediction, backward prediction or bi-directional prediction is used, and motion vector data from the motion vector detection circuit <b>50</b>. The encoding circuit <b>58</b> also receives prediction flag data from the prediction decision circuit <b>54</b> comprising a flag indicating which of the frame prediction mode or the field prediction mode is used, and prediction flag data from the DCT mode changeover circuit <b>55</b> comprising a flag indicating which of the frame DCT mode or the field DCT mode is used. This information is placed into the header portion of the encoded data stream.

0060The variable length encoding circuit <b>58</b> serves to encode the quantized data and the header information using a variable length code such as a Huffman code, in accordance with the quantization step data supplied from the quantization circuit <b>57</b>, and to output the resulting data to a transmission buffer <b>59</b>.

0061The quantized data and quantization step are also supplied to a dequantization circuit <b>60</b> which serves to dequantize the quantized data using the quantization step, and to supply the recovered DCT coefficient data to an inverse DCT circuit <b>61</b> that functions to inverse transform the DCT coefficient data to produce recovered difference data and to supply the recovered difference data to an arithmetic unit <b>62</b>.

0062The arithmetic unit <b>62</b> combines the recovered difference data with a previously encoded and decoded reference picture, as motion compensated, to produce decoded data for a reconstructed picture which will be used as a reference picture and which is read into one of two frame memories <b>63</b>a, <b>63</b>b. The memories <b>63</b>a, <b>63</b>b are adapted to read out the reference picture data stored therein to a motion compensation circuit <b>64</b> that uses the motion vectors from the motion vector detection circuit <b>50</b> to produce a prediction picture from the reference picture. Specifically, the circuit <b>50</b> uses the motion vector to alter the readout address of the reference picture from the memory <b>63</b>a or <b>63</b>b.

0063For a group of pictures, after the first frame I-picture data and the third frame P-picture data are stored in the forward and backward prediction picture memories or units <b>63</b>a, <b>63</b>b, respectively, the second frame B-picture data is processed by the motion vector detection circuit <b>50</b>. The prediction decision circuit <b>54</b> selects the frame or field prediction mode, while setting the prediction mode to one of intra-frame prediction mode, forward prediction mode, backward prediction mode and bi-directional prediction mode in correspondence with the sum of absolute values of predictive errors by macro-block.

0064Since a reconstructed B-picture is not used as a reference picture for other pictures, it is not stored in the frame memory <b>63</b>.

0065It will be appreciated that the frame memory <b>63</b> has its forward and backward prediction picture units <b>63</b>a, <b>63</b>b bank-exchanged as needed so that a picture stored in one of the units <b>63</b>a or <b>63</b>b can be outputted as either a forward or a backward prediction picture.

0066The motion compensation circuit <b>64</b> functions to supply the motion compensated data as a prediction picture to the arithmetic unit <b>62</b> and to the arithmetic unit <b>53</b> which subtracts the prediction picture from the P-picture or the B-picture currently being predictively encoded.

0067More specifically, when the motion vector detection circuit <b>50</b> receives picture data for an I-picture from the forward original picture unit <b>51</b>a, the prediction decision circuit <b>54</b> selects the intra-frame prediction mode and sets a switch <b>53</b>d of the arithmetic unit <b>53</b> to an input contact a. This causes the I-picture data to be inputted directly to the DCT mode changeover circuit <b>55</b>. In this case, no prediction picture is expected from the motion compensation circuit <b>64</b>. The I-picture data is also supplied to the forward prediction picture unit <b>63</b>a.

0068When the forward prediction mode is selected by the prediction decision circuit <b>54</b>, the circuit <b>54</b> also sets the switch <b>53</b>d to an input contact b which causes the arithmetic unit <b>53</b>a to subtract the prediction picture, produced by the motion compensation circuit <b>64</b>, from the picture read out from the memory <b>51</b>, for each macro-block on a pixel by pixel basis, to produce difference data. The P-picture, after encoding and local decoding, is supplied to one of the units <b>63</b>a, <b>63</b>b. For example, if the P-picture immediately follows an I-picture, then the P-picture is stored in the backward prediction picture unit <b>63</b>b.

0069For forward predictive encoding, the prediction picture is a reference I-picture or P-picture read out from the forward prediction picture unit <b>63</b>a of the frame memory <b>63</b> and motion-compensated by the motion compensation circuit <b>64</b> in accordance with the motion vector outputted from the motion vector detection circuit <b>50</b>. More specifically, for each macro-block, the motion compensation circuit <b>64</b> shifts the readout address of the forward prediction picture unit <b>63</b>a in an amount corresponding to the motion vector currently output by the motion vector detection circuit <b>50</b>.

0070When the backward prediction mode is selected by the prediction decision circuit <b>54</b>, the circuit <b>54</b> also sets the switch <b>53</b>d to an input contact c which causes the arithmetic unit <b>53</b>b to subtract the prediction picture, produced by the motion compensation circuit <b>64</b>, from the picture read out from the memory <b>51</b>, on a pixel by pixel basis, to produce difference data.

0071For backward predictive encoding, the prediction picture is a P-picture read out from the backward prediction picture unit <b>63</b>b of the frame memory <b>63</b> and motion-compensated by the motion compensation circuit <b>64</b> in accordance with the motion vector outputted from the motion vector detection circuit <b>50</b>. More specifically, for each macro-block, the motion compensation circuit <b>64</b> shifts the readout address of the backward prediction picture unit <b>63</b>b in an amount corresponding to the motion vector currently output by the motion vector detection circuit <b>50</b>.

0072When the bi-directional prediction mode is selected by the prediction decision circuit <b>54</b>, the circuit <b>54</b> sets the switch <b>53</b>d to an input contact d which causes the arithmetic unit <b>53</b>c to subtract a prediction picture from the picture read out from the memory <b>51</b>, on a pixel by pixel basis, to produce difference data. The prediction picture is the mean value of a forward prediction picture and a backward prediction picture.

0073In the case of bi-directional prediction, the picture stored in the forward prediction picture unit <b>63</b>a, and the picture stored in the backward prediction picture unit <b>63</b>b, are read out and motion-compensated by the motion compensation circuit <b>64</b> in dependence upon the motion vectors outputted from the motion vector detection circuit <b>50</b>. More specifically, for each macro-block, the motion compensation circuit <b>64</b> shifts the readout address of the forward and backward prediction picture units <b>63</b>a, <b>63</b>b in an amount corresponding to the appropriate one of the motion vectors currently output by the motion vector detection circuit <b>50</b>.

0074The transmission buffer <b>59</b> temporarily stores the data supplied thereto, generates control data indicating the volume of data stored therein and supplies the control data to the quantization circuit <b>57</b>. When the volume of data stored in the transmission buffer <b>59</b> reaches a predetermined upper limit value, the control data from the transmission buffer <b>59</b> causes the quantization scale of the quantization circuit <b>57</b> to increase so as to decrease the volume of the quantized data. Similarly, when the volume of data stored in the transmission buffer <b>59</b> reaches a predetermined lower limit value, the control data from the transmission buffer <b>59</b> causes the quantization scale of the quantization circuit <b>57</b> to decrease so as to increase the volume of the quantized data. In this manner, the transmission buffer <b>59</b> prevents the data supplied thereto from overflowing or underflowing its capacity. The data stored in the transmission buffer <b>59</b> are read out at a predetermined timing to an output terminal <b>69</b> and thence to a transmission channel for recording on, for example, the recording medium <b>3</b>.

0075Although the foregoing description has been made with reference mainly to the luminance blocks, the color difference blocks are similarly processed and transmitted using the motion vector which corresponds to the motion vector of the luminance block halved in both the vertical and horizontal directions.

0076<figref idref="DRAWINGS">FIG. 9</figref> illustrates the decoder <b>31</b> shown in FIG. <b>4</b>.

0077The reproduced encoded picture data transmitted from the recording medium <b>3</b> is applied to a reception circuit, not shown, or to an input terminal <b>80</b> which applies the encoded picture data to a reception buffer <b>81</b> that serves to temporarily store the encoded picture data and to supply this data to a variable length decoding circuit <b>82</b> of a decoding circuit <b>90</b>.

0078The variable length decoding circuit <b>82</b> functions to variable length decode the encoded data, to output the recovered motion vector, prediction mode data, prediction flags and DCT flags to the motion compensation circuit <b>87</b>, and to output the quantization step data and variable length decoded picture-data, including the predictive mode, the motion vector, the predictive flag, the DCT flag and the quantized picture data for each macro-block, to an inverse quantization circuit <b>83</b>.

0079The inverse quantization circuit <b>83</b> is adapted to dequantize the picture data supplied from the variable length decoding circuit <b>82</b> in accordance with the quantization step data supplied from the variable length decoding circuit <b>82</b> and to output the thus recovered coefficient data to an inverse transformation IDCT circuit <b>84</b>.

0080The IDCT circuit <b>84</b> is adapted to perform an inverse transformation on the recovered coefficient data to produce recovered difference data, and to supply the recovered difference data to an arithmetic unit <b>85</b>.

0081If the recovered difference data supplied from the IDCT circuit <b>84</b> represents an I-picture, the arithmetic unit <b>85</b> does not process the data and simply supplies it through an output terminal <b>91</b> to the format converting circuit <b>32</b> shown in <figref idref="DRAWINGS">FIG. 4</figref>, and to a forward prediction picture unit <b>86</b>a of a frame memory <b>86</b>.

0082If the recovered difference data supplied from the IDCT circuit <b>84</b> represents a macro-block of a P-picture produced in the forward prediction mode, then the reference picture data of the preceding frame, as stored in the forward prediction picture memory <b>86</b>a of the frame memory <b>86</b>, is read and motion-compensated by a motion compensation circuit <b>87</b> in dependence upon the motion vector outputted from the variable length decoding circuit <b>82</b> to generate a prediction picture. Specifically, the motion compensation circuit <b>87</b> uses the motion vector to alter the read out address supplied to the memory <b>86</b>a. The arithmetic unit <b>85</b> adds the prediction picture to the recovered difference data to produce a decoded or reconstructed picture which is stored in a backward prediction picture memory <b>86</b>b of the frame memory <b>86</b>. The decoded P-picture is retained in the decoder <b>31</b>, and output after the next B-picture is decoded and output, so as to restore the pictures to the order in which they were supplied to the encoder <b>18</b> of FIG. <b>4</b>.

0083Even if the macro-block of the P-picture was encoded as intra-coded mode data, the decoded P-picture is directly stored in the backward prediction picture unit <b>86</b>b, without being output to the output terminal <b>91</b> by the arithmetic unit <b>85</b>.

0084If the recovered difference data supplied from the IDCT circuit <b>84</b> represents a macro-block of a B-picture encoded in the intra-coding mode, as determined from the prediction mode supplied from the variable length decoding circuit <b>82</b> to the motion compensation circuit <b>87</b>, a prediction picture is not generated.

0085If the recovered difference data supplied from the IDCT circuit <b>84</b> represents a macro-block of a B-picture encoded in the forward prediction mode, as determined from the prediction mode supplied from the variable length decoding circuit <b>82</b> to the motion compensation circuit <b>87</b>, the data stored in the forward prediction picture unit <b>86</b>a of the frame memory <b>86</b> is read out and motion compensated by the motion compensation circuit <b>87</b> using the motion vector supplied from the variable length decoding circuit <b>82</b> to form the prediction picture. The arithmetic unit <b>85</b> sums the recovered difference data with the prediction picture to form the recovered B-picture.

0086If the recovered difference data supplied from the IDCT circuit <b>84</b> represents a macro-block of a B-picture encoded in the backward prediction mode, as determined from the prediction mode supplied from the variable length decoding circuit <b>82</b> to the motion compensation circuit <b>87</b>, the data stored in the backward prediction picture unit <b>86</b>b is read out and motion compensated by the motion compensation circuit <b>87</b> using the motion vector supplied from the variable length decoding circuit <b>82</b> to form the prediction picture. The arithmetic unit <b>85</b> sums the recovered difference data with the prediction picture to form the recovered B-picture.

0087If the recovered difference data supplied from the IDCT circuit <b>84</b> represents a macro-block of a B-picture encoded in the bi-directional prediction mode, as determined from the prediction mode supplied from the variable length decoding circuit <b>82</b> to the motion compensation circuit <b>87</b>, the data stored in both the forward and backward prediction picture memories <b>86</b>a, <b>86</b>b are read out and respectively motion compensated by the motion compensation circuit <b>87</b> using the motion vectors supplied from the variable length decoding circuit <b>82</b>, then averaged to form the prediction picture. The arithmetic unit <b>85</b> sums the recovered difference data with the prediction picture to form the recovered B-picture.

0088The recovered B-picture is supplied via the output terminal <b>91</b> to the format converting circuit <b>32</b>. However, since the B-picture is not utilized for generating a prediction picture for other pictures, it is not stored in the frame memory <b>86</b>.

0089After outputting of the B-picture, picture data of the P-picture stored in the backward prediction picture unit <b>86</b>b is read and supplied via the motion compensation circuit <b>87</b> to the arithmetic unit <b>85</b>. Motion compensation is not performed at this time.

0090The counterpart circuits to the prediction mode changeover circuit <b>52</b> and the DCT mode changeover circuit <b>55</b> in the encoder <b>18</b> of <figref idref="DRAWINGS">FIG. 6</figref> are not shown in the decoder <b>31</b>. The processing to be performed by these circuits, that is, the processing for restoring the configuration in which odd-field line signals and even-field line signals are separated from each other to the configuration in which odd and even-field line signals alternate with each other, is performed by the motion compensation circuit <b>87</b>.

0091The processing of the luminance signals has been explained in the foregoing. As will be appreciated by one of ordinary skill in the art, the processing of the color difference signals is carried out in a similar manner. However, the motion vector employed in such case is the motion vector for luminance signals which is halved in both the vertical and horizontal directions.

0092<figref idref="DRAWINGS">FIG. 10</figref> shows the signal to noise ratio (SNR) for pictures transmitted using the above-described technique. As can be seen, the best quality transmission is obtained for I-pictures, good quality transmission is obtained for P-pictures, and the poorest quality transmission is obtained for B-pictures. Thus, if the transmission path has adequate capacity, it is preferable to transmit a picture as an I-picture.

0093If all pictures cannot be transmitted as I-pictures, it is better to transmit a series of pictures as shown in <figref idref="DRAWINGS">FIG. 10</figref>, rather than in a format in which one average picture quality is used for all pictures. The technique shown in <figref idref="DRAWINGS">FIG. 10</figref> takes advantage of the human visual sense characteristic of perceiving a series of changing picture quality, as shown in <figref idref="DRAWINGS">FIG. 10</figref>, as of higher quality than a series of unchanging picture quality, for a predetermined transmission rate.

0094Accordingly, in the configuration of <figref idref="DRAWINGS">FIG. 6</figref>, transmission rate control is carried out by the quantizer <b>57</b> in order to attain the picture quality perceived as better.

0095To dub pictures, two coder-decoder (codec) units are used in series. However, the picture quality obtained from the second codec is substantially worse than the picture quality obtained from the first codec, as explained below.

0096<figref idref="DRAWINGS">FIG. 11</figref> shows a configuration representing two codecs connected by an analog connection, namely, coder <b>201</b>, decoder <b>202</b>, coder <b>203</b> and decoder <b>204</b>, connected in series.

0097In <figref idref="DRAWINGS">FIG. 11</figref>, an analog video signal is supplied to an input terminal <b>200</b> as an input signal a. The input terminal <b>200</b> functions to apply the analog video signal to an A/D converter <b>211</b> of coder <b>201</b>. The converter <b>211</b> is adapted to convert the analog video signal to a digital video signal, and to apply the digital video signal to coding circuit <b>212</b> that serves to encode this signal as previously described to produce a coded digital video signal.

0098The coded digital video signal from coding circuit <b>212</b> of coder <b>201</b> is supplied to a decoding circuit <b>213</b> of decoder <b>202</b> which is adapted to decode the coded digital video signal and to apply the decoded video signal to D/A converter <b>214</b>.

0099The analog video signal produced by D/A converter of decoder <b>202</b> is supplied as an output signal b to the coder <b>203</b>, which functions in a similar manner as the coder <b>201</b>.

0100The digital video signal produced by the coder <b>203</b> is supplied to decoder <b>204</b> which functions in a similar manner as the decoder <b>202</b>. The decoder <b>204</b> delivers an analog video signal as an output signal c to a terminal <b>205</b>, which may transmit the signal c to another coder (not shown) and so on.

0101<figref idref="DRAWINGS">FIG. 12</figref> shows the SNR of the output signals b, c shown in FIG. <b>11</b>. The SNR of the output signal c is seen to be substantially worse than the SNR of the output signal b.

0102The deterioration in picture quality results from a mismatch between the picture type applied in the first codec and the picture type applied in the second codec. Namely, if a picture coded as a B picture in the first coder/decoder pair is coded as, e.g., P picture in the second coder/decoder pair, a great deterioration of picture quality results because the picture quality changes as a function of the picture type.

0103Since the deterioration in picture quality results from the mismatch between picture types of respective stages of codecs, such deterioration similarly takes place when digital connections are used between respective codecs.

0104<figref idref="DRAWINGS">FIG. 13</figref> shows a configuration representing two codecs connected by a digital connection, namely, coder <b>302</b>, decoder <b>303</b>, coder <b>304</b> and decoder <b>305</b>, connected in series.

0105An analog video signal is supplied to terminal <b>300</b>, which supplies the analog video signal as an input signal a to A/D converter <b>301</b> that serves to digitize the signal a, and to apply the digital signal to a digital interface <b>311</b> of coder <b>302</b>. The digital interface <b>311</b> applies the signal supplied thereto to a coding circuit <b>312</b> which encodes or compresses the digital video data to an encoded digital video bit stream.

0106The encoded digital video signal from the coding circuit <b>312</b> is supplied to decoding circuit <b>313</b> of decoder <b>303</b> that decodes the signal supplied thereto, and applies the decoded signal to digital interface <b>314</b>. The interface <b>314</b> functions to output the decoded signal as an output signal b.

0107The output signal b is supplied to coder <b>304</b> which functions in a similar manner as coder <b>302</b> to produce a coded signal that is applied to decoder <b>305</b> which functions in a similar manner as decoder <b>303</b>. The digital signal output from the decoder <b>303</b> is supplied to a D/A converter that serves to convert the signal supplied thereto to an analog video signal and to supply the analog video signal as an output signal c to output terminal <b>307</b>.

0108<figref idref="DRAWINGS">FIG. 12</figref> also generally represents the SNR of the output signals b, c shown in FIG. <b>13</b>.

OBJECTS AND SUMMARY OF THE INVENTION

0109Therefore, an object of the present invention is to provide a method and apparatus for encoding and decoding picture signals which avoid the aforementioned disadvantages of the prior art.

0110Another object of the present invention is to provide a method and apparatus for transmitting and receiving picture signals in serial stages which minimizes the deterioration in picture quality at each stage.

0111Yet another object of the present invention is to match the type of coding applied to pictures of a picture signal in serial processing stages, each stage comprising coding and decoding.

0112In accordance with one embodiment of the present invention, apparatus and method for processing a digital picture signal operate by receiving a digital picture signal which has picture type data included in a data identification area of the digital picture signal and which indicates one of intrapicture coding, predictive coding and bidirectionally predictive coding for respective pictures represented by the digital picture signal. The picture signal is encoded as a function of the picture type data to produce an encoded picture signal.

0113In accordance with another embodiment of the present invention, apparatus and method for processing an encoded digital picture signal operate by decoding the encoded digital picture signal so as to produce picture type data which represents the type of encoding of the encoded digital picture signal and to produce a decoded digital picture signal. The picture type data is added to a data identification area of the decoded digital picture signal to produce an output signal.

0114The above, and other objects, features and advantages of the present invention will be apparent from the following detailed description of the preferred embodiments of the present invention when read in conjunction with the accompanying drawings in which corresponding parts are identified by the same reference numeral.

BRIEF DESCRIPTION OF THE DRAWINGS

0115<figref idref="DRAWINGS">FIGS. 1A and 1B</figref> are pictures illustrating inter-frame correlation;

0116<figref idref="DRAWINGS">FIGS. 2A and 2B</figref> are diagrams illustrating types of pictures used in predictive encoding;

0117<figref idref="DRAWINGS">FIG. 3</figref> is a diagram illustrating how picture signals are converted into encoded data for transmission;

0118<figref idref="DRAWINGS">FIG. 4</figref> is a block diagram showing a conventional device for encoding and decoding picture signals;

0119<figref idref="DRAWINGS">FIG. 5</figref> is a diagram referred to in explaining the operation of the format converting circuit shown in <figref idref="DRAWINGS">FIG. 4</figref>;

0120<figref idref="DRAWINGS">FIG. 6</figref> is a block diagram showing the encoder of the device shown in <figref idref="DRAWINGS">FIG. 4</figref>;

0121<figref idref="DRAWINGS">FIG. 7</figref> is a chart referred to in explaining the predictive encoding operation of the encoder shown in <figref idref="DRAWINGS">FIG. 6</figref>;

0122<figref idref="DRAWINGS">FIG. 8</figref> is a chart referred to in explaining the orthogonal transformation operation of the encoder shown in <figref idref="DRAWINGS">FIG. 6</figref>;

0123<figref idref="DRAWINGS">FIG. 9</figref> is a block diagram showing the decoder of the device shown in <figref idref="DRAWINGS">FIG. 4</figref>;

0124<figref idref="DRAWINGS">FIG. 10</figref> is a graph showing picture quality as a function of picture type in a transmitted signal;

0125<figref idref="DRAWINGS">FIG. 11</figref> is a block diagram showing two conventional video codecs connected in series using an analog connection;

0126<figref idref="DRAWINGS">FIG. 12</figref> is a graph showing picture quality of the signals output by the codecs of <figref idref="DRAWINGS">FIG. 11</figref>;

0127<figref idref="DRAWINGS">FIG. 13</figref> is a block diagram showing two conventional video codecs connected in series using a digital connection;

0128<figref idref="DRAWINGS">FIG. 14</figref> is a block diagram showing two video codecs according to the present invention connected in series using an analog connection;

0129<figref idref="DRAWINGS">FIG. 15</figref> is a graph showing picture quality of the signals output by the codecs of <figref idref="DRAWINGS">FIG. 14</figref>;

0130<figref idref="DRAWINGS">FIG. 16</figref> is a block diagram showing two video codecs according to the present invention connected in series using a digital connection;

0131<figref idref="DRAWINGS">FIGS. 17A</figref>, <b>17</b>B and <b>17</b>C illustrate the data structure of a decoded digital video signal;

0132<figref idref="DRAWINGS">FIGS. 18A</figref>, <b>18</b>B and <b>18</b>C are diagrams illustrating various encoding structures of groups of pictures;

0133<figref idref="DRAWINGS">FIG. 19</figref> is a block diagram showing a coder circuit according to the present invention; and

0134<figref idref="DRAWINGS">FIG. 20</figref> is a block diagram showing a decoding circuit according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

0135The present invention comprises a picture type identifier which is included with the picture signal at all times, namely, when the signal is encoded and when the signal is decoded. The picture type indicates one of intra-picture coding (an I-picture), predictive coding (a P-picture) and bi-directionally predictive coding (a B-picture). An I-picture includes macro-blocks encoded by only intra-coding. A P-picture includes macro-blocks encoded by intra-coding and/or macroblocks encoded by forward predictive coding. A B-picture includes macro-blocks encoded by intra-coding and/or macroblocks encoded by forward predictive coding and/or macroblocks encoded by backward predictive coding and/or macroblocks encoded by bi-directionally predictive coding.

0136Initially, each picture of a picture signal is encoded as a function of a respective picture type, then appropriately decoded. The decoded picture signal includes the respective picture types, preferably in the vertical blanking interval of each decoded picture. The decoded picture signal may be further processed by, for example, dubbing.

0137When the pictures of the decoded picture signal are again encoded, the re-encoding is a function of the picture type included in the decoded picture signal. The re-encoded signal includes the picture type. Subsequently decoding of the re-encoded picture is a function of the picture type. Each picture of the re-decoded picture signal includes its respective picture type.

0138Thus, the present invention matches the type of predictive coding applied to pictures in a picture signal by serially arranged coders which process the picture signal.

0139The present invention promotes optimum picture quality. A picture, previously encoded as an I-picture, P-picture or B-picture is again encoded as an I-picture, P-picture or B-picture, respectively. Also, encoding of a picture, previously encoded as a B-picture, as an I-picture or a P-picture is prevented. Thus, deterioration in signal quality after plural coding and decoding operations is minimized.

0140Referring now to the drawings, and in particular to <figref idref="DRAWINGS">FIG. 14</figref>, there are illustrated coding and decoding units (codecs) according to the present invention having a serial analog connection therebetween. A first codec comprises coder <b>120</b> and decoder <b>121</b>, while a second codec comprises coder <b>122</b> and decoder <b>123</b>. It will be appreciated by one of ordinary skill in the art that additional codecs may be serially connected to those shown in FIG. <b>14</b>.

0141In <figref idref="DRAWINGS">FIG. 14</figref>, an analog video signal is supplied to an input terminal <b>100</b> as an input signal a, and a picture type signal is supplied to an input terminal <b>108</b>. The picture type indicates one of intra-picture coding (an I-picture), predictive coding (a P-picture) and bi-directionally predictive coding (a B-picture).

0142The input terminals <b>100</b>, <b>108</b> function to apply the analog video signal and the picture type signal, respectively, to an A/D converter <b>101</b> and a coding circuit <b>102</b>, respectively, of coder <b>120</b>. The converter <b>101</b> is adapted to convert the analog video signal to a digital video signal, and to apply the digital video signal to the coding circuit <b>102</b>.

0143The coding circuit <b>102</b> serves to encode the digital video signal as a function of the picture type signal to produce a coded digital video signal which includes, for each encoded picture, its picture type as identified by the picture type signal. More specifically, if the picture type for a picture indicates intra-picture coding, then the coding circuit <b>102</b> codes the picture as an I-picture. If the picture type for a picture indicates predictive coding, then the coding circuit <b>102</b> codes the picture as a P-picture. If the picture type for a picture indicates bi-directionally predictive coding, then the coding circuit <b>102</b> codes the pictures as a B-picture.

0144The coding unit <b>120</b> may alternatively have the structure shown in <figref idref="DRAWINGS">FIG. 6</figref>, in which coding is performed without reference to an externally supplied picture type. As used herein, an externally supplied picture type means a picture type supplied from generally the same source as supplies the digital video signal, rather than from a separate source as is shown in <figref idref="DRAWINGS">FIG. 6</figref>, namely, the picture type input device <b>65</b>.

0145The coded digital video signal from coding circuit <b>102</b> of <figref idref="DRAWINGS">FIG. 14</figref> is supplied to a decoding circuit <b>103</b> of decoder <b>121</b> which is adapted to decode the coded digital video signal as a function of the picture type included in the encoded signal and to apply the decoded video signal to D/A converter <b>104</b>. The decoding circuit <b>103</b> is further adapted to apply the picture type decoded from the coded digital video signal to a multiplexer <b>105</b>.

0146The multiplexer <b>105</b> is operative to multiplex the picture type information with the decoded video signal to produce a multiplexed analog video signal as an output signal b in which the picture type information is contained in the decoded video signal. Preferably, the multiplexer <b>105</b> inserts the picture type for a picture of the decoded video signal into the vertical blanking interval of the picture. As mentioned, a picture may be either a frame or a field of the video signal.

0147The output signal b is supplied from the multiplexer <b>105</b> to a separating circuit <b>106</b> of the coder <b>122</b>. The separating circuit is operative to separate or demultiplex the analog video signal and the picture type information from the output signal b, to supply the separated analog video signal to an A/D converter <b>107</b>, and to supply the separated picture type information to a coding circuit <b>108</b>. The converter <b>107</b> is adapted to convert the separated analog video signal to a digital video signal, and to apply the digital video signal to the coding circuit <b>108</b>.

0148The coding circuit <b>108</b> serves to encode the digital video signal as a function of the separated picture type to produce a re-coded digital video signal which includes, for each re-encoded picture, its picture type as identified by the separated picture type signal.

0149The re-coded digital video signal from the coding circuit <b>108</b> is supplied to the decoder <b>123</b>, which operates in a similar manner as the decoder <b>121</b>.

0150Decoding circuit <b>110</b> of decoder <b>123</b> decodes the re-coded digital video signal to produce a re-decoded digital video signal and a corresponding picture type signal. The re-decoded digital video signal is converted to an analog signal by D/A converter <b>109</b>, and applied to a multiplexer <b>111</b> which multiplexes the analog video signal with the picture type signal from decoding circuit <b>110</b> to produce a multiplexed analog video signal as an output signal c. Preferably, the multiplexer <b>111</b> inserts the picture type for a picture into the vertical blanking interval of the picture. The multiplexer <b>111</b> applies its output signal c to an output terminal <b>119</b>.

0151Due to the inclusion of the picture type identifier in the signals b and c, the codecs of <figref idref="DRAWINGS">FIG. 14</figref> process respective pictures of the video signals b and c in the same manner, that is, as the same one of an I-picture, a P-picture or a B-picture.

0152<figref idref="DRAWINGS">FIG. 15</figref> shows the SNR of the output signals b, c shown in FIG. <b>14</b>. The SNR of the output signal c is seen to be only slightly worse than the SNR of the output signal b.

0153That is, since the type of predictive coding applied to each picture is the same in each of the serially arranged codecs, the deterioration in picture quality at each codec is minimized even when the picture quality changes from picture to picture due to the type of predictive coding employed from picture to picture.

0154<figref idref="DRAWINGS">FIG. 16</figref> shows codecs according to the present invention having a serial digital connection therebetween. A first codec comprises coder <b>142</b> and decoder <b>143</b>, while a second codec comprises coder <b>144</b> and decoder <b>145</b>.

0155In <figref idref="DRAWINGS">FIG. 16</figref>, an analog video signal is supplied to an input terminal <b>140</b> as an input signal a, and a picture type signal is supplied to an input terminal <b>148</b>. The input terminals <b>140</b>, <b>148</b> function to apply the analog video signal and the picture type signal, respectively, to an A/D converter <b>141</b> and a coding circuit <b>152</b> of coder <b>142</b>, respectively. The converter <b>141</b> is adapted to convert the analog video signal to a digital video signal, and to apply the digital video signal to a digital interface <b>151</b> of coder <b>142</b>.

0156The coding circuit <b>152</b> serves to encode the digital video signal as a function of the picture type signal to produce a coded digital video signal which includes, for each encoded picture, its picture type as identified by the picture type signal. More specifically, if the picture type for a picture indicates intra-picture coding, then the coding circuit <b>152</b> codes the picture as an I-picture. If the picture type for a picture indicates predictive coding, then the coding circuit <b>152</b> codes the picture as a P-picture. If the picture type for a picture indicates bi-directionally predictive coding, then the coding circuit <b>152</b> codes the picture as a B-picture.

0157The coding unit <b>120</b> may alternatively have the structure shown in <figref idref="DRAWINGS">FIG. 6</figref>, in which coding is performed without reference to an externally supplied picture type.

0158The coded digital video signal from coding circuit <b>152</b> is supplied to a decoding circuit <b>153</b> of decoder <b>143</b> which is adapted to decode the coded digital video signal as a function of the picture type included in the encoded signal and to apply the decoded video signal to a digital interface <b>154</b>. The decoding circuit <b>153</b> is further adapted to apply the picture type decoded from the coded digital video signal to a multiplexer <b>155</b>.

0159The multiplexer <b>155</b> is operative to multiplex the picture type information with the decoded video signal to produce a multiplexed digital video signal as an output signal b in which the picture type information is contained in the decoded video signal. Preferably, multiplexer <b>155</b> multiplexes the picture type for a picture of the decoded video signal as a flag in the respective picture.

0160In a preferred embodiment of the present invention, the picture type information is inserted (or multiplexed) into the decoded video signal at a location therein which precedes the actual video data that represents the video field or frame. <figref idref="DRAWINGS">FIGS. 17A</figref>, <b>17</b>B and <b>17</b>C illustrate the data structure of the decoded (MPEG) video signal. <figref idref="DRAWINGS">FIG. 17A</figref> illustrates the data structure of a decoded video signal having a serial digital interface format as specified in the standard SMPTE—259 (Society of Motion Picture &amp; Television Engineers). As shown, a frame consists of a first vertical blanking area VBK<b>1</b>, a first optional blanking area OBK<b>1</b> and a first active video area ACV<b>1</b>, which constitutes the first field, followed by a second vertical blanking area VBK<b>2</b>, a second optional blanking area OBK<b>2</b> and a second active video area ACV<b>2</b>, which constitutes the second field. In the preferred embodiment, each of the vertical blanking areas consists of <b>9</b> horizontal scanning lines, each of the optional blanking areas consists of <b>10</b> horizontal scanning lines, the first active video area consists of <b>244</b> horizontal scanning lines and the second active video area consists of <b>243</b> horizontal scanning lines, for a total of <b>525</b> horizontal scanning lines for a single frame.

0161<figref idref="DRAWINGS">FIG. 17B</figref> illustrates the data structure of a horizontal scanning line. As shown, a horizontal scanning line includes an end of active video (EAV) area followed by an ancillary (ANC) area, a start of active video (SAV) area and a video area. The ANC area, as shown in <figref idref="DRAWINGS">FIG. 17C</figref>, includes an ancillary data flag (ADF) area followed by a data identification (DID) area, an ancillary number data (DBN) area, an ancillary word data (DC) area, an ancillary data (ANC DATA) area, and a check sum (CS) area. Since the above-noted areas of a horizontal scanning line of digital data are well-known in the art, their descriptions are omitted herein except where necessary for an understanding of the present invention.

0162In accordance with the present invention, the picture type information is inserted into the DID area of the ANC area of each of the horizontal scanning lines in the first and second vertical blanking areas. However, the picture type information also may be inserted into the DID area of the ANC area of other horizontal scanning lines of the decoded video signal, although these DID areas may be used for transmitting other types of data. For example, the DID areas in horizontal lines of non-vertical blanking areas may include other formatting information.

0163The picture type information (or picture type data) may identify the type of encoding of the picture (e.g., intra-picture coding, predictive coding, and bi-directionally predictive coding) in various ways. <figref idref="DRAWINGS">FIGS. 18A</figref> to <b>18</b>C illustrate one method in which the type of encoding is identified by the structure of the group of pictures (GOP). As shown, the structure of a group of pictures (GOP) may be identified by the minimum number of frames “M” between I and P pictures, between P and P pictures, and between I and I pictures, and the total number of frames “N” (pictures) in the group of pictures. For example, <figref idref="DRAWINGS">FIG. 18A</figref> illustrates groups of pictures having an encoding structure of M=3 and N=9 in which there are 9 frames in each group and wherein there are 3 frames from each I or P frame to the respectively succeeding I or P frame. Similarly, <figref idref="DRAWINGS">FIG. 18B</figref> illustrates groups of pictures having an encoding structure of M=2 and N=2, and <figref idref="DRAWINGS">FIG. 18C</figref> illustrates groups of pictures having an encoding structure of M=1 and N=2.

0164When the picture type data identifies the “M” and “N” numbers, the type of encoding for each picture can be determined by the location of a respective picture within the group of pictures, and the location of a respective picture may be identified in the picture type data either by identifying each picture's location within the group of pictures or by identifying only the first picture within the group of pictures. For example, when M=3 and N=9 for a group of frames in the decoded video signal (FIG. <b>18</b>A), the third frame in that group is identified as a decoded I-frame.

0165Returning to <figref idref="DRAWINGS">FIG. 16</figref>, the output signal b (the multiplexed decoded video signal) is supplied from the multiplexer <b>155</b> to a separating circuit <b>156</b> of the coder <b>144</b>. The separating circuit is operative to separate or demultiplex the digital video signal and the picture type data from the output signal b, to supply the separated digital video signal to a digital interface <b>157</b>, and to supply the separated picture type data to a coding circuit <b>158</b>. The interface <b>157</b> is adapted to apply the separated digital video signal to the coding circuit <b>158</b>.

0166The coding circuit <b>158</b> serves to encode the separated digital video signal as a function of the separated picture type to produce a re-coded digital video signal which includes, for each re-encoded picture, its picture type as identified by the separated picture type signal.

0167The re-coded digital video signal from the coding circuit <b>158</b> is supplied to the decoder <b>145</b>, which operates in a similar manner as the decoder <b>143</b>.

0168Decoding circuit <b>160</b> of decoder <b>145</b> decodes the re-coded digital video signal to produce a re-decoded digital video signal and a corresponding picture type signal. The re-decoded digital video signal is supplied to a digital interface <b>159</b> and thence to a multiplexer <b>161</b> which multiplexes the re-decoded digital video signal with the picture type signal from decoding circuit <b>160</b> to produce a multiplexed digital video signal. The multiplexer <b>161</b> multiplexes the picture type signal as a flag (e.g., in the DID area of the ANC area) in the re-decoded digital video signal.

0169The multiplexer <b>161</b> supplies the multiplexed digital video signal to an A/D converter <b>146</b> which serves to convert the multiplexed digital video signal to an analog video signal also referred to as output signal c. The converter <b>146</b> applies the output signal c to an output terminal <b>147</b>.

0170Due to the inclusion of the picture type identifier in the signals b and c, the codecs of <figref idref="DRAWINGS">FIG. 16</figref> process respective pictures of the video signals b and c in the same manner, that is, as the same one of an I-picture, a P-picture or a B-picture. Consequently, the deterioration in picture quality at each codec is minimized even when the picture quality changes from picture to picture due to the type of predictive coding employed from picture to picture.

0171<figref idref="DRAWINGS">FIG. 19</figref> shows the coding circuits of <figref idref="DRAWINGS">FIGS. 14 and 16</figref> in more detail. In <figref idref="DRAWINGS">FIG. 19</figref>, elements similar to those in <figref idref="DRAWINGS">FIG. 6</figref> are indicated by the same reference numerals, and detailed explanations thereof are omitted.

0172In <figref idref="DRAWINGS">FIG. 19</figref>, a picture type signal is supplied to input terminal <b>70</b> which serves to supply the picture type signal to motion vector detector <b>450</b>, predictive judging circuit <b>454</b> and variable length coding circuit <b>458</b>. The processing performed by elements <b>450</b>, <b>454</b> and <b>458</b> is similar to the processing performed by elements <b>50</b>, <b>54</b> and <b>58</b> of <figref idref="DRAWINGS">FIG. 6</figref>, except that the elements of <figref idref="DRAWINGS">FIG. 19</figref> perform in accordance with the picture type identified in the external picture type signal which indicates the picture type used in previous coding. The variable length coding circuit <b>458</b> includes the picture type based on the external picture type signal as part of the header information.

0173<figref idref="DRAWINGS">FIG. 20</figref> shows the decoding circuits of <figref idref="DRAWINGS">FIGS. 14 and 16</figref> in more detail. In <figref idref="DRAWINGS">FIG. 20</figref>, elements similar to those in <figref idref="DRAWINGS">FIG. 9</figref> are indicated by the same reference numerals, and detailed explanations thereof are omitted.

0174Variable length decoding circuit <b>482</b> of <figref idref="DRAWINGS">FIG. 20</figref> is similar to variable length decoding circuit <b>82</b> of <figref idref="DRAWINGS">FIG. 9</figref>, except that circuit <b>482</b> applies the picture type separated from the encoded signal not only to motion compensator <b>487</b>, but also to output terminal <b>92</b>.

0175Although an illustrative embodiment of the present invention, and various modifications thereof, have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to this precise embodiment and the described modifications, and that various changes and further modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims.