Method of transmitting audio and/or video signals

The invention relates to a method of transmitting audio and/or video signals <i>via</i> some transmission medium. More particularly the transmission medium is constituted by an optically readable disc. However, the transmission medium may also be a magnetic tape or disc or a direct connection between a transmitter and a receiver. The invention also relates to the transmission medium on which the audio and/or video signals are recorded, to an encoding apparatus for transmitting the audio and/or video signals, and to a decoding apparatus for receiving these signals.

The interactive compact disc (CDI) on which data for text and graphic images in addition to digital audio is stored has been developed in the last few years. The presentation possibilities of this CDI are considerably enhanced if full motion video pictures are also recorded on it. The known analog recording of the video signal on the compact disc video (CDV) is not suitable for this purpose. For this reason the video signal on CDI is digitized. A full-motion video scene is then considered as a sequence of pictures of which there are, for example, 25 or 30 per second. Each picture comprises, for example, 256 picture lines and 352 pixels per picture line. The sequence of pictures is converted by means of a suitably chosen encoding method into a series of video blocks each comprising so much digital information that each picture can be reconstructed without any noticeable loss of quality. Together with the audio signal and further data, the encoded video signal is recorded optically. A CDI may comprise various video scenes.

The most efficient encoding methods convert successive signal portions into successive code blocks of variable lengths. In the case of a video signal these signal portions are formed, for example, by the pictures or picture pairs of which the signal is composed. Some pictures may be subjected to intraframe coding and are then converted into code blocks from which the picture can be reconstructed completely. Other pictures may be subjected to interframe coding, which means that the pictures can only be reconstructed with the aid of previous pictures. The code blocks of a video signal will hereinafter be referred to as video blocks. Due to their variable length, the successive video blocks are read at irregular instants when the disc is being played. Moreover, the video blocks on the disc may alternate with (or may even be interrupted by) other data signals, for example, the lip-synchronous digital audio signal corresponding to the video scene.

The pictures corresponding to the video blocks should be displayed at a constant frequency of, for example, 25 frames per second. However, the instant when a video block of the disc is being read hardly ever corresponds exactly to the instant when the corresponding picture from the video scene is to be displayed. In the player the video blocks are therefore applied to a memory buffer at a frequency which is entirely determined by the way of "packing" on the disc. Subsequently they are read from the buffer at the picture frequency required for display. On a average, the number of video blocks applied per second from the disc to the buffer equals the number of pictures displayed per second. The video block corresponding to the next picture to be displayed will always have to be stored completely in the buffer. Moreover, the buffer will already comprise a subsequent video block, a portion thereof or even a number of subsequent video blocks. As soon as a picture has been decoded, the corresponding video block may be removed from the buffer. The buffer space then released is written by subsequent video blocks or a portion thereof. The number of video blocks stored in the buffer for later display is thus variable and is greatly dependent on the encoding efficiency as well as on the presence of signals other than the relevant video signal on the disc.

The display of a video scene can seemingly start as soon as the first video block has been received completely and is stored in the buffer. However, this is not true. In fact, it is possible that a subsequent video block is too large to be scanned within the required time (1/25 sec). Then the corresponding picture cannot be displayed in time. The absence of a complete video block in the buffer at the instant when the corresponding picture has to be decoded and displayed is sometimes referred to as underflow of the buffer.

Underflow of the buffer also occurs if a large quantity of other (non-video) data has been packed somewhere in the scene so that the buffer is temporarily not filled with video blocks. The buffer then empties and at a given instant a video block for the next picture to be displayed is not present yet. The display of the video scene then stalls and there is no question of smooth moving pictures.

However, the display of a picture of a video scene may neither start too late after the corresponding video block has been received. In that case it is probable that the buffer fills so that the display also stalls. This is also referred to as overflow of the buffer.

The same problem occurs in audio signals which may also be recorded on the disc in a non-contiguous manner.

A known method of avoiding buffer overflow and underflow is disclosed in Takahashi et al. "Interframe CODEC for NTSC Color Television Signals", Fujitsu Scientific &amp; Technical Journal, Vol.13, No.4, 1977, pp.21-52. In accordance with this prior art method, a control signal is transmitted at the instant of time at which decoding of the currently received frame must start. Such a control signal is necessarily a unique code which can never occur elsewhere in the encoded signal. The prior art method maintains a delay of one frame between encoding and decoding each frame. A delay of more than one frame requires the control signal to also indicate which frame is to be decoded by the receiver.

It is an object of the invention to provide an improved method of transmitting audio and/or video signals with which the occurrence of overflow and underflow of the buffer is prevented so that the display of pictures can proceed in an undisturbed manner.

According to the invention, the control signal is constituted by a parameter accommodated at a predetermined position of a code block, said parameter indicating the quantity of delay with which the code block must be decoded after it has been received. Because its position in the encoded signal is now known, the parameter need not be uniquely coded and may assume a large range of different delay values. Further, the delay between encoding and decoding may exceed more than a single frame period.

If the corresponding picture is decoded with this delay, it can be displayed without the risk of overflow or underflow of the buffer. This ensures an undisturbed display. The transmission of the parameter, sometimes referred to as decoder delay parameter or decoder delay for short, also realises the synchronous display of two signals, for example, a video signal with a time-division multiplexed associated audio signal, or two video signals jointly representing a three-dimensional video scene.

In principle, it is sufficient to associate the parameter with one video block. This will preferably be the first video block of the video scene. All further pictures can then be read at a constant picture frequency without a further risk of overflow or underflow of the buffer.

In a further embodiment of the method the decoder delay parameter is regularly associated with video blocks. The constant picture frequency can then be obtained in a favourable manner. Moreover, this provides the possibility of a so-called "random-access" display. This is understood to mean the display of a fragment of a video scene from the video block with which the decoder delay is associated.

The decoder delay parameter may assume various formats, but it preferably indicates the state of a counter which is regularly loaded with a reference value which is also transmitted and which counter further receives a predetermined clock signal.

Fig. 1 shows diagrammatically an optically readable disc with an audio and/or video signal recorded on this disc.

Fig. 2 shows the general structure of a transmission circuit suitable for performing the method according to the invention.

Fig. 3 shows some time diagrams to explain the circuit of Fig. 2.

Fig. 4 shows some further time diagrams to explain the circuit of Fig. 2.

Fig. 5 shows some time diagrams to explain the transmission of two simultaneously transmitted signals.

Fig. 1A shows diagrammatically an optically readable disc 1 with an information track 2 recorded on this disc. A so-called sector of which there may be about 300,000 is present each time between two successive points A, B, C, D, etc. The structure of such a sector is shown diagrammatically in Fig. 1B. It is divided into a sector header H of 24 bytes and a data field DF of 2324 bytes. The sector header comprises a synchronization word SYN of 12 bytes, a sector number SCTNR of 4 bytes and a service word SW of 8 bytes. The synchronization word SYN marks the start of a sector. The sector number SCTNR indicates the ordinal number of the sector on the disc. The service word SW indicates whether the data field DF of the sector comprises video data, audio data or computer data. In conformity therewith a sector is sometimes referred to as video sector, audio sector or data sector. The conventional rate of revolution of the disc is such that 75 sectors per second are scanned. This corresponds to a bit rate, hereinafter referred to as channel bit rate, of approximately 1.4 Mbit/sec. If a disc comprises video sectors only, the video bit rate is equal to the channel bit rate. If a disc also comprises audio and data sectors, the video bit rate decreases accordingly. Fig. 1C also shows a part of the track in which the sectors are denoted by SCT. More particularly the video sectors therein are shaded. As has been attempted to express, video sectors alternate with other sectors.

There are many encoding methods of reducing the number of bits per picture without deteriorating the picture quality. The most efficient encoding methods convert successive pictures of a video scene into video blocks of distinct numbers of bits. In Fig. 1D it has been attempted to express this in the form of video blocks V1, V2, etc. each having a distinct length. For example, video block V1 has a length of 4 sectors on the disc, video block V2 has a length of 1½ sector, etc. Fig. 1E further shows the data format of a video block. The start of a video block is marked by the presence of a unique start-of-frame word SOF. The video block also comprises a lable LBL in which specific control information is stored for the corresponding video block. Fig. 2 shows the general structure of a circuit for transmitting audio and video signals. The circuit comprises an encoding apparatus 3, a compact disc 4 and a decoding apparatus 5. The encoding apparatus 3 comprises a video encoder 31 to which a video signal v(t) is applied. The video encoder 31 encodes the successive pictures of the video signal, of which there are 25 per second, into video blocks VB of variable lengths. These pictures are applied video block by video block to an encoder buffer 32. To prevent overflow and underflow of the encoder buffer, a buffer status signal BFST is fed back to the video encoder 31 in order that this encoder can adapt the encoding efficiency to the contents of the encoder buffer. If the encoder buffer tends to get filled and if there is a risk of overflow, the video signal is temporarily quantized in a coarser manner. If the encoder buffer tends to get empty, the video signal is quantized more accurately or pseudo-video data are applied possibly temporarily to the buffer. Moreover, the buffer status signal BFST is applied to an encoder control circuit 33. This control circuit is adapted to generate, at a predetermined degree of occupation of the encoder buffer, a clock signal F1 with which the encoder buffer is read. The encoder control circuit also determines the value of a parameter T_d for each video block in a manner to be described hereinafter. This parameter is associated with the video block by means of an inserter 34 and transmitted in the lable LBL of this block (see Fig. 1E). As is shown in Fig. 2, the encoder buffer 32 may be considered to be divided into a first buffer 321 to which the video blocks are applied and which is read at a constant video bit rate C and a sector allocation buffer 322 which converts this constant bit stream into video sectors VS. The video sectors are applied from the encoder buffer 32 to a first input of a multiplexer 35 which receives audio sectors AS at a second input and data sectors DS at a third input. The resultant bit stream is recorded on the compact disc 4.

When the disc is played at a later stage, the sectors are scanned and applied to a demultiplexer 51 in which the video sectors are separated from audio sectors and data sectors with reference to their service word SW (see Fig. 1B). The video sectors VS' are applied to a decoder buffer 52 and a video block detector 53. Video block detector 53 detects the occurrence of a start-of-frame code SOF (see Fig. 1E) and reads from the subsequent lable LBL the parameter T_d transmitted therein. The parameter T_d is applied to a decoder control circuit 54 which is adapted to read the decoder buffer 52 with a clock signal F2. In a manner to be described hereinafter, the parameter T_d determines the instant when each video block is read from decoder buffer 52. As is shown in the Figure, the decoder buffer 52 may be considered to be divided into a data rate equalization buffer 521 to which the video sectors are applied and which is read at a constant video bit rate C' and a second buffer 522 which converts this constant bit stream into successive video blocks VB'. Finally, each video block is applied to a video decoder 55 for conversion into a picture. The successive pictures are applied as a video signal v'(t) to a display device (not shown).

In practice the actual instant when a disc is being played does not play any role at all. It will therefore be assumed hereinafter that the output of the encoding apparatus is directly connected to the input of the decoding apparatus. This is indicated by means of a broken line in the Figure.

It will tentatively be assumed that only video signals are transmitted. This means that the encoder buffer 32 can apply a uninterrupted bit stream to the disc at the full channel bit rate of 1.4 Mbit/sec (75 sectors per second). As regards the circuit shown in Fig. 2 this means that the sector allocation buffer 322 and the multiplexer 35 on the one hand and the demultiplexer 51 and the data rate equalization buffer 521 on the other hand do not play a role in the following considerations.

The operation of the circuit shown in Fig. 2 will be explained with reference to some time diagrams which are shown in Fig. 3. More particularly, Fig. 3B shows a number of successive pictures enumerated 1, 2, 3, etc. These pictures occur at a picture frequency of 25 frames per second and are applied to the video encoder 31 (Fig. 2). As soon as each picture has been encoded, the corresponding video block is written into the encoder buffer 32 (Fig. 2). For the sake of simplicity it is assumed that this is effected timelessly. The quantity of video data cumulatively applied to the encoder buffer and expressed in disc sectors thus increases step-wise for each video block. This is denoted by means of the curve n_e(t) in Fig. 3A.

Reading of the encoder buffer and applying data to the output of the encoding apparatus starts from instant t1. This instant t1 is determined, for example, by reaching a predetermined degree b of occupation of the encoder buffer and may be different for each video scene. Reading is effected at a constant rate of 75 sectors per second, <i>i.e.</i> 1.4 Mbit/sec. The quantity of video data cumulatively withdrawn from the encoder buffer thus increases linearly. This is denoted by means of the curve n_s(t) in Fig. 3A. The curves n_e(t) and n_s(t) must not intersect each other because this would mean that a video block is read from the buffer before it is written into it (underflow of the encoder buffer). As already previously stated, underflow of the encoder buffer is prevented by adapting the quantization of the video scene to the contents of the encoder buffer.

Based on the assumption that the output of the encoding apparatus is directly connected to the input of the decoding apparatus, the curve n_s(t) shown in Fig. 3A also indicates the cumulative quantity of video data applied to the decoder buffer 52 (Fig. 2). The decoder buffer must be read in such a manner that the curve n_s(t) is not exceeded, because underflow of the decoder would then occur. Overflow of the decoder buffer should neither occur. In Fig. 3A this has been expressed by means of the line n₁(t) at a distance B (the buffer size) from the line n_s(t). This line n₁(t) should neither be exceeded by the curve n_s(t).

The decoder buffer is now read video block by video block. For the sake of simplicity it is assumed that a video block is read timelessly. The quantity of video data cumulatively withdrawn from the decoder buffer and expressed in disc sectors thus increases step-wise for each video block. This is denoted by means of the curve n_d(t) in Fig. 3A. As soon as a video block has been read and decoded, the corresponding picture is displayed for 1/25 sec. Fig. 3C shows the display of the decoded pictures 1, 2, 3 etc.

The curve n_d(t) is a replica of the curve n_e(t) shifted over a time interval T_e+T_d. Here T_e, also referred to as encoder delay, is the delay time between the instant when the first video block of the video scene is written into the encoder buffer and the instant t1 when the transmission of this video block begins. In a corresponding manner T_d, also referred to as decoder delay, is the delay time between the instant when the reception of the first video block in the decoder buffer begins and the instant when this video block is read from the decoder buffer. The time interval T_e+T_d represents the delay time occurring between writing a video block into the encoder buffer and reading the same video block from the decoder buffer. This delay time is constant. As already previously noted, t1, and consequently T_e, may be different for each video scene. In conformity therewith, also the decoder delay T_d may be different for each video scene.

The instant when the decoding apparatus starts reading the decoder buffer is of great importance. In fact, if the decoder should start reading too early, there may be underflow of the decoder buffer because the curve n_d(t) intersects the boundary n_s(t). If the decoder starts reading too late, there may be overflow of the decoder buffer because the curve n_d(t) intersects the boundary n₁(t). Both underflow and overflow may become manifest in the video scene at a possibly later stage.

Overflow and underflow of the decoder buffer do not occur if the decoder curve n_d(t) is always as far remote from the boundaries n_s(t) and n₁(t) as is the encoder curve n_e(t) from the boundaries n₂(t) and n_s(t). Here n₂(t) is a line at a distance B (the decoder buffer size) from the line n_e(t). This is realised by choosing a suitable value for the decoder delay T_d, as well as by the previously mentioned mechanism of adaptive quantization in the encoding apparatus, which ensures that the encoder curve n_e(t) remains completely between the two boundaries n₂(t) and n_s(t).

A suitable value for decoder delay T_d follows from the consideration that the previously mentioned delay time T_e+T_d is equal to the time interval T shown in Fig. 3A. This time interval T corresponds to the time required to fully read a completely filled encoder buffer or to completely fill a still empty decoder buffer and is entirely determined by the buffer size B and the video bit rate at which the buffer is read or written. The value of T_d then follows from the computation of the difference T-T_e. Here both T_e and T are known to the encoding apparatus.

The encoding apparatus associates the decoder delay T_d as a parameter with the first video block of the scene so as to enable the decoding apparatus to determine the instant when reading of the decoder buffer should start. After the decoding apparatus has read the video block with the delay T_d from the buffer, the reading process can be continued at the picture frequency in an autonomic manner without the risk of underflow or overflow of the decoder buffer. The picture frequency of 25 Hz can be obtained by dividing the channel bit rate (75 sectors/sec*2324 bytes/sector*8 bits/byte) by 55776. The coupling of the picture frequency to the channel bit rate which is necessary for this purpose is, however, superfluous if the decoder delay parameter has regularly been associated with video blocks. This decoder delay will generally be different for each video block. If a difference between the actual instant when a video block is read and the instant determined by the corresponding decoder delay parameter occurs at a given picture frequency, the picture frequency or the rate of revolution of the disc can be corrected in response to this difference. Fig. 3A shows by way of example a decoder delay T_d7 for picture 7. At the instant denoted by t2 the video block corresponding to this picture is written into the encoder buffer. After encoder delay T_e7 the transmission of the video block from the encoder buffer to the decoder buffer starts and after the subsequent decoder delay T_d7 the video block must be decoded. The encoding apparatus associates the decoder delay T_d7 = T - T_e7 for picture 7 with video block 7 and transmits it in the lable LBL of this block (see Fig. 1E).

The regular transmission of the decoder delay also provides the possibility of a so-called "random-access" playback mode. This is understood to mean playing a fragment of the video scene as from a picture other than the first. For example, the decoder delay T_d7 shown in Fig. 3A can be used to play the video scene as from picture 7. Picture 7 is preferably an intraframe-encoded picture and can therefore be reconstructed without the aid of previous pictures. In principle, it is possible to compute the decoder delay T_d7 for picture 7 from the previous signal. However, such computations are complicated and also require the availability of previous information. In the case of random-access display the previous signal is, however, not available.

The situation will be more complicated if not all disc sectors are allocated to video data. If the disc also comprises sectors with audio and computer data, they will alternate with video sectors on the disc. The average video bit rate for a video scene will now be lower than 75 sectors per second. The average video bit rate will hereinafter be expressed in a number S indicating which part of the channel bit rate is suitable for video transmission. For example, S = 1/2 means that on an average 1 out of 2 sectors is a video sector. This corresponds to a video bit rate of 1/2*75 = 37.5 sectors per second, <i>i.e</i>. 0.7 Mbit/sec.

For the purpose of illustration Fig. 4 shows some time diagrams of the encoding and decoding process if S = 1/2. Fig. 4B shows diagrammatically the successive pictures 1, 2, etc. of a video scene applied to the encoding apparatus. The curve n_e(t) in Fig. 4A indicates the quantity of video data applied for each picture to the encoder buffer and expressed in the number of disc sectors. In comparison with the curve in Fig. 3A the encoding is now such that on an average each picture is encoded in 1.5 sectors instead of in 3 sectors. The contents of the encoder buffer are applied to selected sectors of the disc. This is effected in accordance with a given pattern, an example of which is shown in Fig. 4C. In this pattern the shaded sectors represent video sectors and the other sectors are audio sectors or data sectors. Fig. 4C also shows the signal controlling the multiplexer 35 (Fig. 2). The average bit rate is reached in this example by writing video information on 8 out of each 16 sectors on the disc. The quantity of video data thus cumulatively applied to the disc is indicated in Fig. 4A by means of the curve n_s(t). When the disc is being played, the reverse operations take place. The curve n_s(t) now represents the number of video sectors cumulatively read from the disc. These sectors are applied to the decoder buffer from which they are subsequently read video block by video block in accordance with the curve n_d(t).

In Fig. 4A, n_e(t)-n_s(t) indicates the actual contents of the encoder buffer. To prevent underflow of this buffer, the curves n_e(t) and n_s(t) should not intersect each other. This condition is certainly met if, as already shown in Fig. 2, the encoder buffer 32 is considered to be divided into a first buffer 321 to which the video blocks are applied and which is read at a constant video bit rate, and a sector allocation buffer 322 which converts this constant bit stream into the sector-sequential supply of sectors to the disc. Said constant bit stream is denoted by the straight line c_e(t) in Fig. 4A. The slope of this line is the average bit rate, 0.7 Mbit/sec in this example. As long as the two curves n_e(t) and n_s(t) do not intersect the line c_e(t), there is no encoder buffer underflow.

In a corresponding manner, n_s(t)-n_d(t) represents the actual contents of the decoder buffer. To prevent underflow of the decoder buffer the curve n_e(t) should not intersect the curve n_s(t). This condition is certainly met if, as already shown in Fig. 2, the decoder buffer 52 is considered to be divided into a data rate equalization buffer 521 having a size of B1 to which the video sectors of the disc are applied and which is read at the constant bit rate, and a second buffer 522 of the size B2 which converts this constant bit stream into video blocks. Said constant bit stream is denoted by the straight line c_d(t) in Fig. 4A. The slope of this line corresponds to the average video bit rate of 0.7 Mbit/sec. As long as the two curves n_s(t) and n_d(t) do not intersect the line c_d(t), there is no decoder buffer underflow.

The previously mentioned decoder delay parameter, which indicates when the decoding apparatus can read a video block from the decoder buffer after the first bit of the video scene has been scanned, is now equal to T_cd+T_vd shown in Fig. 4A. Its value is completely determined by the encoding apparatus and transmitted in the lable LBL (see Fig. 1E) of the corresponding video block.

The decoder delay parameter is determined as follows. The value of T_cd+T_vd follows from the consideration that T_ve+T_ce+T_cd+T_vd corresponds to the delay of the video signal if the output of the encoding apparatus is directly coupled to the input of the decoding apparatus. This delay is equal to the interval T shown in Fig. 4A and is dependent on the average video bit rate which is S*75 sectors per second and the size B (= B1+B2) of the decoder buffer. More particularly it can be derived from Fig. 4A that<maths id="math0001"><math display="block"><mrow><mtext>T = </mtext><mfrac><mrow><mtext>B</mtext></mrow><mrow><mtext>S*75</mtext></mrow></mfrac></mrow></math><img file="EP0460751B1_D0001.tif" /></maths> and<maths id="math0002"><math display="block"><mrow><msub><mrow><mtext>T</mtext></mrow><mrow><mtext>cd</mtext></mrow></msub><msub><mrow><mtext>+T</mtext></mrow><mrow><mtext>vd</mtext></mrow></msub><msub><mrow><mtext> = T-(T</mtext></mrow><mrow><mtext>ve</mtext></mrow></msub><msub><mrow><mtext>+T</mtext></mrow><mrow><mtext>ce</mtext></mrow></msub><mtext>).</mtext></mrow></math><img file="EP0460751B1_D0002.tif" /></maths> Here, S is fixed when writing the disc. As for the decoder buffer size B it holds that it may be a predetermined fixed value which is fixed in the encoding apparatus as well as in the decoding apparatus. The encoding apparatus may also be adapted to transmit the buffer size B to be reserved in the decoding apparatus as a further parameter. It is notably sensible to render the decoder buffer size B dependent on S. The decoder delay (T_ve+T_ce), being the time between the encoding of a video picture and its transmission, is determined by the encoding apparatus itself. As already previously stated, the instant when the first video block is transmitted may be determined by reaching a predetermined degree of occupation of the encoder buffer.

The form of the transmitted decoder delay parameter may assume several formats. For example, it is possible to transmit a value indicating at which degree of occupation of the decoder buffer the decoder should start reading. This degree of occupation may be expressed in an absolute value (number of bits) or in a relative value (fraction of the buffer size B). The decoder delay parameter may also indicate the time delay in suitably chosen units with respect to the instant when the video block is written into the decoder buffer. Suitable units are, for example, (m)sec or the number of clock periods of a predetermined clock signal. It has been found useful to have the decoder delay parameter indicate the state of a counter at which the decoding of the video block should start. Said counter is then regularly loaded with a reference value which is also transmitted. The counter is further clocked by a predetermined clock signal. The transmitted reference value may form part of the service word SW (see Fig. 1B). A suitable frequency of the clock signal is 90 kHz. This value is an integral multiple of the picture frequencies 24 Hz, 25 Hz, 30 Hz and 29.97 Hz which are suitable for display.

Finally it is to be noted that in the foregoing the decoder delay parameter is only described for one signal, in this case a video signal. However, the decoder delay parameter also realises the synchronous display of a plurality of signals. These may be, for example, a video signal and an associated lip-synchronous audio signal. However, they may also be two video signals multiplexed on the compact disc, which signals may not only be displayed separately but, if desired, also simultaneously. This is shown diagrammatically in Fig. 5 for two video signals A and B. The Figure shows a video block VA(n) and VB(n) of the two signals, in which n is the ordinal number of the corresponding picture to be displayed. As is apparent from the Figure, the decoder delay parameters T_A(n) and T_B(n) transmitted in the video block have such a value that the corresponding pictures A(n) and B(n) are displayed simultaneously.