Robust mode staggercasting

Robust mode staggered broadcast

This patent application claims the priority of provisional patent application No. 60/443,672 filed on January 28, 2003.

Technical field

The present invention relates to a staggercasting method and equipment.

Background technique

The current U.S. digital television transmission standard recommended by the Advanced Television Systems Committee (ATSC) on September 16, 1995, listed here for reference, uses a single-carrier modulation technique: eight-level vestigial sideband modulation (8-VSB). Since this is a single-carrier modulation technology, it is easy to cause signal degradation in the communication channel, such as fading caused by multipath and other signal attenuation. Although some such fading can be compensated by channel equalization techniques, if the fading is long enough and severe enough, the receiver will lose the signal and the demodulator system will lose synchronization. It may take a few seconds to reacquire the signal and resynchronize the demodulator, which is quite offensive to the audience.

In order to solve this problem, the first ATSC proposal allows the creation of a second communication channel by allowing a more robust channel coding (modulation) technique to be used in a limited time interval, for example, a time interval less than 10%. For example, 2 or 4-VSB modulation techniques can be used for selected frames. The second ATSC proposal allows more robust coding techniques, such as trellis coding, while maintaining the 8-VSB modulation technique. Such a system allows compatible receivers to have improved performance while maintaining backward compatibility with existing receivers.

Another technique that has been used to overcome fading is interleaving. Listed here for reference, the PCT application No. US02/22723 filed by K. Rama-swamy et al. on July 17, 2002, and the PCT application No. US02/ filed on July 19, 2002 by JA Cooper et al. No. 23032 discloses an interleaved broadcast communication system. The interleaved communication system transmits a composite signal consisting of two components representing content signals: one of which is delayed relative to the other. In other words, the composition represents that one of the content signals is ahead of the other. Broadcast a composite signal to one or more receivers through a communication channel. On the receiver, the component whose delay time is advanced through the delay buffer represents the content signal, so that it can be resynchronized in time with the other component's representative content signal. Under normal conditions, the undelayed component representative content signal is used to reproduce the content. However, if signal fading occurs, the previously received and time-advanced component in the delay buffer represents the content signal used to reproduce the content until the fading ends and the composite signal is available again, or the delay buffer is empty. If the delay interval and the associated delay buffer are large enough, most possible fading can be compensated.

Summary of the invention

The inventors have realized that the combination of interleaving and the newly proposed robust mode in the ATSC system can enable user modes that provide improved performance.

According to the principles of the present invention, the method and device for interlaced broadcasting encodes the first representative content signal and uses a relatively more robust encoding than the encoding of the first encoded representative content signal to encode the second representative content signal. A composite signal including at least the first and second coded signals is generated, wherein one of the first and second coded signals is delayed with respect to the other coded signal. If an error is detected in the composite signal, the undelayed encoded signal is decoded to reproduce the content. Otherwise, decode the delayed encoded signal to reproduce the content.

Description of the drawings

Fig. 1 is a block diagram of a part of the interleaved transmitter; Fig. 2 is a block diagram of a part of the interleaved receiver; Fig. 3 is a package that helps to understand the operation of the interleaved communication system as shown in Figs. 1 and 2 ( packet) timing diagram; Figure 4 is a GOP timing diagram that helps to understand the work of improving the interleaved transmission communication system; Figure 5 is a block diagram of the selector that can be used in the receiver shown in Figure 2; Figure 6 is the interleaved transmission A block diagram of a part of another embodiment of the receiver; FIG. 7 is a video frame timing diagram helpful to understand the operation of the interlaced broadcast receiver as shown in FIG. 6; FIG. 8 illustrates the related program map table (PMT) and / Or program and information system protocol-extended syntax and semantics of the virtual channel table (PSIP-VCT); FIG. 9 is a part of another embodiment of an interleaved transmitter that transmits a multi-resolution version of a representative content signal Block diagram; Figure 10 is a block diagram of a part of another embodiment of an interlaced broadcast receiver that receives a multi-resolution version of a transmitted representative content signal; Figure 11 is a block diagram of a part of a transmitter that transmits a dual-interlaced representative content signal ;

FIG. 12 is a block diagram of a part of the receiver that receives the double-interleaved representative content signal; and FIG. 13 is helpful to understand the operation of the double-interleaved transmitter shown in FIG. 11 and the double-interleaved receiver shown in FIG. 12 Show graph.

detailed description

Fig. 1 is a block diagram of a part of an interlaced broadcast transmitter 100 based on the principles of the present invention. Those of ordinary skill in the art should understand that other components are required for a complete transmitter, but they are not shown for the sake of simplifying the figure. Those of ordinary skill in the art should also understand what those components are, and how to select, design, implement, and interconnect those other components with the components shown.

In FIG. 1, in the illustrated embodiment, a content source (not shown) that may be a video image signal, an audio sound image, program data, or any combination of these, supplies a representative content signal to the input terminal of the transmitter 100 105. The input terminal 105 is coupled to the respective input terminals of the robust mode encoder 110 and the normal mode encoder 120. The output terminal of the robust mode encoder 110 is coupled with the first input terminal of the multiplexer 140. The output terminal of the normal mode encoder 120 is coupled with the input terminal of the delay 130. The output terminal of the delay 130 is coupled with the second input terminal of the multiplexer 140. The output terminal of the multiplexer 140 is coupled with the input terminal of the modulator 150. The output terminal of the modulator 150 is coupled to the output terminal 115. The output terminal 115 is coupled with a communication channel (not shown).

In operation, the normal mode encoder 120 uses source coding technology to encode content video, audio and/or data. In the illustrated embodiment, the source coding technology is MPEG (Moving Picture Experts Group) 2 coding technology, but any other such source coding technology can also be used. Use predetermined parameters including resolution, frame rate, quantization level, etc. to perform source encoding processing. Further processing is performed in the normal mode encoder 120, so that the system coded source code represents the content signal. In the illustrated embodiment, the source coded representative content signal is made to form a series of transport packets containing coded video, audio, and/or data. These transport packets are formatted according to the MPEG2 standard, but any other such system encoding can also be used.

The robust mode encoder 110 also uses source coding technology to encode content video, audio and/or data. Compared with the source coding technique of the normal mode encoder 120, the source coding technique used by the robust mode encoder 110 is more robust. In the illustrated embodiment, the robust mode coding used is currently being developed by the Joint Video Group (JVT) of the ISO/IEC MPEG and ITU-T VCEG committees designated as MPEG AVC/H.264 and referred to below as Video coding technology for JVT coding. However, any other such source coding techniques can also be used. For example, other source coding techniques that provide robust coding with respect to the MPEG normal mode encoder 120, such as improved trellis coding, can also be used. It is also possible to perform robust encoding processing using predetermined parameters including resolution, frame rate, and quantization level, but the values of these parameters may be different for robust encoding processing and normal encoding processing. Further processing is also performed in the robust mode encoder 110, so that the system coded source code represents the content signal. In the illustrated embodiment, the source coded representative content signal is also made to form a series of transport packets according to the MPEG2 standard, but any other such system coding can also be used again.

The delayer 130 delays the normal mode coded signal by an amount that is desired to enable the system to work within a range of the expected fading interval. The value of this parameter depends on the characteristics of the communication channel. For example, urban areas where there are many buildings and moving bodies such as airplanes are more common in fading and can last longer than rural plain areas. In the illustrated embodiment, the delay can vary from approximately 0.5 seconds to several seconds.

Fig. 3 is a packet sequence diagram helpful to understand the operation of the interleaved communication system shown in Figs. 1 and 2. FIG. 3 illustrates the system coded transport packet stream on the input end of the multiplexer 140. In FIG. 3, the packets from the robust mode encoder 110 are represented by parallel squares 300 with water, and the squares 300 are marked with lowercase letters "a", "b", "c", and so on. The packets from the normal mode encoder 120 are represented by parallel squares 310 with water, and the squares 310 are marked with numbers "0", "1"... and capital letters "A", "B", "C", etc. Packets marked with the same letter contain data representing content from the same time. That is, the packet "a" from the robust mode encoder 110 contains data representing content corresponding in time to the content represented by the data in the packet "A" from the normal mode encoder 120. Each packet in the normal mode and robust mode packet stream contains data identifying them as belonging to that packet stream in the header. The delay unit 130 delays the packet of the normal mode encoder 120 by Tadv. In other words, the robust mode package is ahead of the corresponding normal mode package by Tadv in time. In the embodiment shown in FIG. 3, Tadv is a 10-packet time interval. As mentioned above, this time interval can vary from approximately 0.5 seconds to several seconds.

In the multiplexer 140, the robust mode and delayed normal mode packet streams are multiplexed together into a composite packet stream. The composite packet stream is time-domain multiplexed, which means generating a single data stream that transmits consecutive packets one at a time. Additional packets containing other data, such as identifiers and control data (not shown), may also be multiplexed into the composite packet stream generated by the multiplexer 140. In addition, it may include both normal mode and robust mode packet streams representing one or more of other content signals, and other packet streams representing other content sources (also not shown) may also be multiplexed in a known manner. To the composite packet stream generated by the multiplexer 140. The packet streams 300 and 310 in FIG. 3 represent that the components in the composite packet stream represent content signals. It can be seen that the packet "A" from the normal mode encoder 120 is transmitted at the same time as the packet "k" from the robust mode encoder 110.

Then, for transmission on the communication channel, the composite packet stream from the multiplexer 140 is channel-encoded. In the illustrated embodiment, channel coding is accomplished by modulating the composite packet stream in the modulator 150. The channel coding for the normal mode packet stream is different from the channel coding for the robust mode packet stream. More specifically, the modulation applied to the robust mode packet stream is more robust than the modulation applied to the normal mode packet stream. In the illustrated embodiment, when the packets in the normal mode packet stream are modulated, the modulation is 8-VSB modulation based on the ATSC standard. When modulating packets in a robust mode packet stream, the modulation is a more robust modulation, for example, 4-VSB or 2-VSB as described above.

In short, in the illustrated embodiment, the normal mode packet stream is source coded using MPEG2 coding technology and is coded using 8-VBS modulation channel. This is fully backward compatible with the existing ATSC standard. In addition, in the illustrated embodiment, the robust mode packet stream is a source coded using JVT coding technology, and is a source coded using 4-VBS and/or 2-VBS modulation channel. Those of ordinary skill in the art should understand that the new ATSC standard cited above only relates to the channel coding of the robust mode packet stream, that is, 4-VBS and/or 2-VBS, and does not specify the source coding technology. Therefore, according to the standard, any such source coding technique can be used, and the JVT coding technique in the illustrated embodiment is an example of such source coding for robust mode packet streams. In the rest of this application, "normal mode packet stream" refers to a packet stream encoded by a source using MPEG2 source coding technology, a packet stream encoded into packets by the system according to the MPEG2 standard, and a packet stream encoded by channel using 8-VSB modulation Packet stream; and "Robust mode packet stream" refers to packets encoded by the source using JVT source coding technology, packets encoded into packets by the system according to the MPEG2 standard, and channels modulated by 4-VBS and/or 2-VBS Encoded package.

Then, the modulated composite signal is supplied to a communication channel (not shown), which may be a wireless RF (Radio Frequency) channel, or a wired channel such as a closed circuit television system. The communication channel may degrade the composite signal. For example, the signal strength of the composite signal may change. In particular, composite signals can be fading due to multipath or other signal attenuation mechanisms. One or more receivers receive composite signals from the communication channel that may be degraded.

Figure 2 is a block diagram of a portion of an interlaced broadcast receiver 200 in accordance with the principles of the present invention. In FIG. 2, the input terminal 205 may be connected to a communication channel (not shown) so as to be able to receive the modulated composite signal generated by the transmitter 100 (FIG. 1). The input terminal 205 is coupled with the input terminal of the demodulator 207. The output terminal of the demodulator 207 is coupled with the input terminal of the demultiplexer 210. The first output terminal of the demultiplexer 210 is coupled with the selector 230. The second output terminal of the demultiplexer 210 is coupled with the delay 220. The output terminal of the delay 220 is coupled with the second input terminal of the selector 230. The output terminal of the selector 230 is coupled with the signal input terminal of the multi-standard decoder 240. The control signal output terminal of the demultiplexer 210 is coupled to the respective corresponding input terminals of the selector 230 and the multi-standard decoder 240. The output terminal of the multi-standard decoder 240 is coupled to the output terminal 215. The output terminal 215 generates a representative content signal, and the representative content signal is supplied to a utility circuit (not shown), such as an image reproduction device that reproduces an image represented by video content and a sound reproduction device that reproduces sound represented by audio content, and A television receiver that may include a user input device that allows the viewer to interact with the received data content.

During operation, the demodulator 207 uses appropriate demodulation techniques required to receive packets from the normal mode packet stream (8-VSB) or robust mode packet stream (4-VSB and/or 2-VSB) to demodulate the reception. Modulated signal. The resulting signal is the received composite packet stream signal. The received composite packet stream signal is demultiplexed by the demultiplexer 210 into respective normal mode source code and robust mode source code component packet streams according to the identification data in the header of each received packet. The received normal mode packet stream is directly supplied to the selector 230. The received robust mode packet stream is passed through the delayer 220, and the delayer 220 delays the received robust mode packet stream by the same length of time as delaying the normal packet stream in the transmitter 100 of FIG. Therefore, the contents represented by the two packet stream signals at the input of the selector 230 are aligned in time.

If a part of the received composite signal is unstable, the demultiplexer 210 also generates an error signal on the control signal output terminal. Any of several techniques can be used, for example, a signal-to-noise ratio detector or a bit error rate detector. In addition, errors in receiving composite signals can be detected by detecting lost packets. Each packet includes both data and a packet sequence number in its header that identify which packet stream the packet belongs to. If the sequence number of the packet stream is lost, the packet is also lost, and an error is detected. In this case, it is possible to record the packet flow in which the packet is lost, and only that packet flow is detected as having errors. These or any other such detectors can be used alone or in combination.

Although the control signal is illustrated as being sent from the demultiplexer 210, those of ordinary skill in the art should understand that different error detectors may require signals from different places in the receiver. No matter what configuration is used, an error signal E that is valid when part of the composite signal is not available is generated. The selector 230 is restricted to transmit one of the two packet stream signals to the multi-standard decoder 240 in response to this error signal E. The multi-standard decoder 240 is restricted to decoding that packet stream signal in a manner described in more detail below.

The multi-standard decoder 240 performs both system decoding (unpacking) and source decoding no matter which packet stream is provided to the selector 230. The multi-standard decoder 240 may be configured to perform source decoding of the packet stream signal according to different encoding standards. For example, when a normal mode encoded packet stream is received from the selector 230, the multi-standard decoder 240 is configured to unpack and source the packets according to the MPEG2 standard and reproduce the representative content signal. Similarly, when receiving the robust mode encoded packet stream from the selector 230, the multi-standard decoder 240 is configured to unpack these packets according to the MPEG2 standard and decode these packets according to the JVT standard source, and reproduce the representative content signal.

The operation of the receiver 200 of FIG. 2 can be understood with reference to FIG. 3 again. Time t0 can represent the time when the receiver is switched on, or the user specifies a new content source to receive. In the time Tadv between t0 and t4, the robust mode packets "a" to "j" are loaded into the delay 220, and the normal mode packets designated as "0" to "9" are received. At time t4, the normal mode packet "A" can be obtained from the demultiplexer 210, and the delayed robust mode packet "a" can be obtained from the delay 220. Under normal conditions, the error signal on the error signal line E is invalid. In response, the selector 230 couples the normal mode packet stream to the multi-standard decoder 240, and as described above, the multi-standard decoder 240 starts to generate a representative content signal from the normal mode packet. This is exemplified by the diagonal hatching 301 in the normal mode packages "A" to "G".

From time t1 to t2, severe fading occurs in the communication channel, and from time t2 to t3, the receiver recovers the modulated signal and resynchronizes with that signal. During this time from t1 to t3, normal mode packets "H" to "M" and robust mode packets "r" to "w" are lost. This is indicated by diagonal hatching 302 and 303 in those packets. However, the robust mode packets "h" to "m" have been successfully received before. Due to the delay 220, these robust mode packets can be obtained from other inputs to the selector 230 from time t1 to t3.

The occurrence of fading is detected and indicated by the effective error signal on the error signal line E. In response to the valid error signal on the error signal line E, the selector 230 couples the previously received robust mode packets "h" to "m" to the multi-standard decoder 240. At the same time, the multi-standard decoder 240 is configured to unpack and decode robust mode packets. Therefore, from time t1 to t3, the packets "h" to "m" from the robust mode packet stream are decoded, and the representative content signal is still available to the utility circuit (not shown). This is exemplified by the diagonal hatching 301 in the robust mode packages "h" to "m".

At time t3, the fading ends and the composite signal becomes available again. Therefore, normal mode packages "N", "O", "P"... become available. The disappearance of fading is detected and indicated by the invalid error signal on the error signal line E. In response, the selector 230 couples the normal mode packet stream to the multi-standard decoder 240. At the same time, the multi-standard decoder 240 is configured to unpack and decode normal mode packets and continue to generate representative content signals.

During the fading and recovery period from time t1 to t3, the robust mode packets "r" to "w" are lost. Therefore, from time t6 to t7, when the normal mode packets "R" to "W" are received, there is no corresponding robust mode packet in the delay 220. During this time, there is no protection against fading. However, once the retarder is refilled, fading protection becomes available again.

As described above, despite the fading from time t1 to t3, the representative content signal remains available to the utility circuit (not shown). In addition, due to robust source coding and channel coding (modulation) technologies, robust mode packets may survive more severe channel degradation, and therefore, can be used when normal mode packets may not be available. The quality of the content signal transmitted by the robust mode packet stream may be different from the quality of the content signal in the normal mode packet stream. In particular, the quality of the content signal in the robust mode packet stream may be lower than the quality of the content signal in the normal mode packet stream. Lower-quality content signals require fewer bits to transmit than high-quality content signals, and such robust mode packet streams require lower throughput than normal mode packet streams. Therefore, at the cost of a secondary, lower throughput packet flow, it is possible to allow a system that allows a moderate degradation in the event of a fading event.

In addition, as described above, the content signal may include video, audio, and/or data. In particular, audio data can be transmitted in both the normal mode packet stream and the robust mode packet stream, so that the audio data is still available despite the occurrence of fading. The audio content signal transmitted by the robust mode packet stream may have different qualities, specifically, a lower quality than the quality of the audio content signal in the normal mode packet stream. Lower quality audio signals can be transmitted with fewer bits and fewer packets, so the requirements for robust mode packet streams are relatively low. In the event of a fading event, this also allows for a moderate deterioration.

For the system as described above, the conversion from the normal mode packet flow to the robust mode packet flow may occur at any time. If the robust mode packet stream transmits the same representative content signal as the representative content signal in the normal mode packet stream lowered to that packet level, there may be no problem. However, if the robust mode packet stream transmits a representative content signal that is different from the representative content signal in the normal mode packet stream, for example, if the content is expressed in a different resolution, quantization level, frame rate, etc., the viewer may notice that it is objectionable The reproduced image changes. In the worst case, if a packet stream conversion occurs in the middle of a decoded picture, the decoding of that picture and other nearby pictures may completely fail, and the video picture may be interrupted for a long time interval until the decoder reconnects with the independently decodable picture So far.

As mentioned above, the normal mode packet stream is transmitted through a combination of source, system, and channel coding. In the illustrated embodiment, the source and system coding are based on the known MPEG2 coding scheme, and the channel coding uses 8-VSB modulation technology. The MPEG source coding scheme encodes the video image signal into a series of independently decoded segments. Independently decoded segments (IDS), also called elementary stream segments, are segments that can be accurately decoded independently of any other independently decoded segments. In the MPEG standard, independently decoded segments include sequences, groups of pictures (GOP), and/or pictures. These independently decoded segments are defined by a unique start code in the compressed bit stream. In other words, the independently decoded segment is regarded as all data starting from the segment start code up to but not including the next segment start code. The pictures under the MPEG2 standard are intra-encoded (I pictures), inter-prediction (P pictures) or bidirectional prediction (B pictures) pictures. There is no need to refer to any other pictures to encode an I picture. A GOP includes a group of pictures that are encoded as a combination of I, P, and/or B pictures. In a closed GOP, it is possible to decode all the pictures in the GOP without referring to the pictures in any other GOP. The beginning of each GOP is clearly identified in the MPEG2 packet stream.

In addition, as described above, the robust mode packet stream is transmitted through a combination of source, system, and channel coding. In the illustrated embodiment, the source coding is based on the JVT coding scheme, the system coding is based on the MPEG2 standard, and the channel coding uses 2-VSB and/or 4-VSB modulation techniques. A picture coded using the JVT source coding standard is composed of coded segments, and a given picture may contain segments of different coding types. Each segment may be an intra-coded (I) segment, an inter-prediction (P) segment, a bi-prediction (B) segment, an SI segment using only spatial prediction, or an SP segment that can be accurately reproduced even using different reference pictures. The JVT source coding standard also includes instantaneous decoding refresh (IDR) pictures. IDR is a special type of JVT coded picture that only contains I segments and marks the beginning of IDS. IDR indicates that the current picture and all subsequent coded pictures can be decoded without requiring reference to the previous picture. Following the GOP under the MPEG2 standard, IDR can be encoded once for every predetermined picture. In the JVT source coding scheme, the independent decoding segment can be defined by the IDR clearly identified in the JVT packet stream.

By imposing some constraints on the normal and robust source coding schemes, it is possible to develop a system that can switch from the normal mode packet stream to the robust mode packet stream while minimizing objectionable artifacts. If the independent decoding segment is coded to start from the same content position in the normal (MPEG2) and robust (JVT) packet streams, it is possible to make the difference between the normal and robust packet streams at the position of the independent decoding segment with the least objectionable artifacts. Conversion. In the illustrated embodiment, the independent decoding segment used in the normal (MPEG2) packet stream is a closed GOP and starts from an I picture. In the corresponding robust (JVT) packet stream, each independently decoded segment starts from the IDR picture. Both the I picture in the normal (MPEG) mode packet stream and the IDR picture in the robust (JVT) mode packet stream encode the same video picture from the content source (not shown). Both source coding schemes allow IDS to be formed and defined in other ways. For example, the MPEG2 source coding scheme also allows segments to be formed to represent pictures. If IDS is inserted in two packet streams at the same content position, any such method can be used.

Referring again to FIG. 1, the input terminal 105 is further coupled with the input terminal of the scene cut detector 160 exemplified by a dashed line. The output terminal of the scene cut detector 160 is coupled to the respective control input terminals of the normal mode encoder 120 and the robust mode encoder 110.

In the working process, the scene cut detector 160 detects the appearance of a new scene in the video content. In response to the detection of the new scene, a control signal is sent to the normal mode encoder 120 and the robust mode encoder 110. Both the normal mode encoder 120 and the robust mode encoder 110 respond to the control signal to start encoding a new independently decoded segment. The normal mode encoder 120 inserts the new I picture and the robust mode encoder 110 inserts the IDR picture into their respective encoded packet streams. The normal mode encoder 120 and the robust mode encoder 110 work to generate corresponding independent decoded segments with the same time length. As mentioned above, the encoding means that the content signal is encoded by the system into respective packet streams.

The delay 130 is set to introduce a delay equal to the length of the independent decoding segment. The multiplexer 140 combines the robust mode coded packet stream and the delayed normal mode coded packet stream into a composite packet stream. The composite packet stream is channel-encoded (modulated) by the modulator 150 in an appropriate manner, and is supplied to the communication channel through the output terminal 115.

Refer to Figure 4 to better understand how the transmitter works in this mode of operation. FIG. 4 illustrates the packet flow to the input of the multiplexer 140. In FIG. 4, a series of independent decoding segments (IDS) from the strong mode encoder 110 are illustrated as a series of rectangles 400, and a series of independent decoding segments from the normal mode encoder 120 are illustrated as a series of rectangles 410. As described above, the time position within the content and the width of the independent decoding segments from the strong mode encoder 110 and the normal mode encoder 120 are the same. Since the delay introduced by the delay 130 is the same as the duration of the IDS, the IDS from the strong mode encoder 110 is aligned with the previous IDS from the normal mode encoder 120.

At time t0, which can represent the scene change detected by the scene cut detector 160, the robust mode encoding IDS N start is not delayed and the normal mode encoding IDS N-1 start is delayed previously. Each robust mode (JVT source coding) IDS is illustrated as a series of rectangles 440 representing each segment, and starts with an independent decoding refresh (IDR) picture. The IDR picture is followed by B, P, SI and/or SP segments. These segments are then coded by the system into a sequence 450 of transmission packets "a", "b", "c", and so on. Similarly, each normal mode (MPEG2 source coding) IDS is illustrated as a series of rectangles 420 representing GOPs starting from the I picture. The I picture is followed by a list of P pictures and B pictures. These I, P and B pictures are then coded by the system into a sequence 430 of transmission packets "A", "B", "C", and so on. The illustrated arrangement is merely an example, and any suitable arrangement can be used.

This composite signal is received by the receiver. Referring again to the receiver 200 in FIG. 2, at time t0, the received robust mode IDS N is loaded into the delayer 220 during the time Tadv. The delayer 230 introduces the same delay (one IDS time interval) introduced by the delayer 130 in the transmitter into the received robust packet stream. Therefore, for the representative content signal, the received normal packet stream and the delay robust packet stream on the input end of the selector 230 are realigned in time.

As described in more detail above, under normal conditions, the selector 230 couples the normal mode packet stream to the multi-standard decoder 240, and the multi-standard decoder 200 is restricted to decode normal mode packets. If, as described above, an error is detected in the composite signal or a part of it, a switch is made between the normal mode packet stream and the robust mode packet stream. In this embodiment, as described in more detail above, at the beginning of the IDS, the selector 230 couples the robust mode packet stream to the multi-standard decoder 240, and the multi-standard decoder 240 is limited to decode the robust mode packets. If no further errors are detected in the composite signal, at the beginning of the next IDS, the selector 230 couples the normal mode packet stream to the multi-standard decoder 240, and the multi-standard decoder 240 is restricted to decode the normal mode packets again.

In the receiver 200 of FIG. 2, the conversion from decoding the normal mode packet stream to decoding the robust mode packet stream and vice versa occurs at the beginning of the IDS. Each IDS is an independent decoding segment starting from an I picture (normal mode) or IDR picture (robust mode) that can be successfully decoded without referring to any other pictures. Moreover, the subsequent pictures can also be decoded without referring to the pictures before the IDS. Therefore, the representative content signal can be decoded and displayed immediately without objectionable artifacts caused by conversion.

In order to further minimize the video artifacts caused by the conversion from decoding the normal mode video packet stream to decoding the robust mode packet stream and vice versa, when the conversion occurs, those that can be in the normal mode video signal and those of the robust mode video signal Gradually change the image characteristics of the resulting video signal. When the robust mode video stream is of lower quality compared to the normal mode video stream, for example, if the spatial resolution, frame rate, etc. of the robust mode video stream are less than the spatial resolution, frame rate, etc. of the normal mode video stream, this is especially desirable .

Fig. 5 is a block diagram of a selector (track) 230" that can be used in the receiver shown in Fig. 3. Such selector 230" can switch between them in the normal mode and the robust mode of the video signal The video characteristics (for example, resolution, frame rate, etc.) of the resulting video signal are gradually changed among those of the video signal. FIG. 5a is a functional diagram illustrating the operation of the selector 230", and FIG. 5b is a structural block diagram illustrating an embodiment of the selector 230" that can be used in the receiver shown in FIG. 2.

In FIG. 5a, the robust mode video signal is coupled to one end of the track 232, and the normal mode video signal is coupled to the other end of the track 232. The slide 234 slides along the track 232 and generates the resultant video signal coupled to the output terminal of the selector 230". The resultant video signal is coupled to the output terminal 215 of the receiver 200 (FIG. 2). The control input terminal is coupled to the output terminal 215 of the receiver 200 (FIG. 2). The path resolver 210 receives the error signal E. The control input is coupled to the input of the controller circuit 231. As indicated by the dotted line, the position of the slide 234 along the track 232 is controlled by the controller circuit 231.

During operation, when the slide 234 is at the upper end of the track 232, the resulting video signal with the characteristics of the robust mode video signal (for example, resolution, frame rate, etc.) is coupled to the output of the selector 230". When When the slide 234 is at the lower end of the track 232, the resulting video signal having the characteristics of the normal mode video signal is coupled to the output terminal of the selector 230". As the slide 234 moves between the upper and lower ends of the track 232, the characteristics of the resulting video signal on the output of the selector 230" are adjusted to be between those of the normal mode video signal and those of the robust mode video signal. The closer the slide 234 is to the upper end of the track 232, the closer the characteristics of the resulting video signal are to those of the robust mode video signal, and the farther away the characteristics of the resulting video signal are from those of the normal mode video signal. The closer the slide 234 is to the lower end of the track 232, the closer the characteristics of the resulting video signal are Those with normal mode video signals, and those with more robust mode video signals.

As described above, the value of the error signal E indicates the time at which the transition occurred. When a transition from one video signal (for example, a normal mode or robust mode video signal) to another video signal occurs, the slide 234 gradually moves from the track within the time interval of one or more video frames near the time when the transition occurs. Move one end of 232 to the other end. For example, during the transition from the normal mode video signal to the robust mode video signal, the slide 234 starts from the bottom of the track. In the few video frames before the conversion, the slide piece gradually moves from the bottom of the track 232 to the top. When switching from the normal mode packet stream to the robust mode packet stream, the slider is at the top of the track 232. Therefore, during the several video pictures before the transition to the robust mode packet stream occurs, the characteristics of the resulting video signal gradually change from those of the normal mode video signal to those of the robust mode video signal. Similarly, when switching from the robust mode packet stream to the normal mode packet stream, the slider is at the top of the track 232. In the few video frames after the conversion, the slide piece gradually moves from the top of the track 232 to the bottom. Therefore, during the several video pictures after the conversion to the normal mode packet stream occurs, the characteristics of the resulting video signal gradually change from those of the robust mode video signal to those of the normal mode video signal.

In FIG. 5b, the video signal from the multi-standard decoder 240 (FIG. 2) is coupled to the first input terminal of the variable video quality filter 236 and the first input terminal of the selector 238. The output terminal of the video quality filter 236 is coupled with the second input terminal of the selector 238. The output terminal of the selector 238 generates the resulting video signal and is coupled to the output terminal 215 (FIG. 2). The error signal E from the demultiplexer 210 is coupled to the controller circuit 231. The first output terminal of the controller circuit 231 is coupled with the control input terminal of the video quality filter 236, and the second output terminal of the controller circuit 231 is coupled with the control input terminal of the selector 238.

During operation, the video quality filter 236 responds to the control signal from the controller circuit 231 to change the video characteristics of the decoded video signal. The control signal from the controller circuit 231 limits the video quality filter 236 to generate video signals whose video characteristics range between those of the normal mode video signal and those of the robust mode video signal. Under normal conditions, when no conversion occurs, the controller circuit 231 restricts the selector 238 from coupling the decoded video signal to the output terminal as the resulting video signal.

In response to the change in the value of the error signal E indicating the transition between the normal mode video signal and the robust mode video signal as described above, the controller circuit 231 limits the selector 238 to the video quality filter within a time interval around the transition time. The video signal of 236 is coupled to the output terminal and the quality limiting filter 236 gradually changes the video characteristics of the resulting video signal. More specifically, if a conversion from a normal mode video signal to a robust mode video signal occurs, the video quality filter 236 is limited to change the video characteristics of the resulting video signal during the interval of several video frames before the conversion occurs. Those of the normal mode video signal gradually change to those of the robust mode video signal. At the beginning of that time interval, the selector 238 is restricted to coupling the filtered video signal to the output as the resulting video signal. When that time interval ends and the decoded video signal is derived from the robust mode packet stream, the selector 238 is restricted to couple the decoded video signal to the output terminal as the resulting video signal. Similarly, if a transition from a robust mode video signal to a normal mode video signal occurs, the video quality filter 236 is restricted to change the video characteristics of the resulting video signal from the robust mode within the time interval of a few video frames after the transition occurs. Those of the video signal gradually change to those of the normal mode video signal. At the beginning of that time interval, the selector 238 is restricted to couple the filtered video signal to the output as the resulting video signal. When that time interval ends and the decoded video signal is derived from the normal mode packet stream, the selector 238 is restricted to couple the decoded video signal to the output terminal as the resulting video signal.

Sudden transitions between video signals with different video qualities (resolution, frame rate, etc.) may cause artifacts that are offensive to viewers. Since the video quality of the resulting video signal gradually decreases before switching from the normal mode video signal to the robust mode video signal, and gradually increases after the conversion from the robust mode video signal to the normal mode video signal, the offending artifacts caused by the conversion can be made Minimize.

Another embodiment of the interleaved communication system can also provide conversion while minimizing objectionable artifacts and does not require the IDS to be located in any special position in the normal and robust mode packet streams. The receiver 200' is shown in FIG. In FIG. 6, components similar to those in the receiver 200 in FIG. 2 are denoted by the same reference numerals, and detailed descriptions are omitted below. In FIG. 6, the first output terminal of the demultiplexer 210 is coupled to the input terminal of the normal mode decoder 240'. The first output terminal of the normal mode decoder 240' is coupled with the first input terminal of the selector 230', and the second output terminal of the normal mode decoder 240' is coupled with the first input terminal of the normal mode frame memory 250'. The output terminal of the delay 220 is coupled with the input terminal of the robust mode decoder 240". The first output terminal of the robust mode decoder 240" is coupled with the second input terminal of the selector 230', and the first output terminal of the robust mode decoder 240" The second output terminal is coupled to the first input terminal of the robust mode frame memory 250". The output terminal of the selector 230' is coupled to the respective second input terminals of the normal mode frame memory 250' and the robust mode frame memory 250". The output terminal of the normal mode frame memory 250' and the second input terminal of the normal mode decoder 240' The output of the robust mode frame memory 250" is coupled to the second input of the robust mode decoder 240".

In the working process, the delayer 220 introduces the delayer 130 in the transmitter 100 (FIG. 1) into the robust mode packet flow with the same delay as the normal mode packet flow. Therefore, for the representative content signal, the packet stream signals on the respective input terminals of the normal mode decoder 240' and the robust mode decoder 240" are aligned in time.

As described in detail above, the system decodes and the source decodes both the normal mode packet stream and the delay robust mode packet stream to generate the corresponding representative content signal stream. In the illustrated embodiment, these representative content signal streams are respective sequences of video pictures. In normal mode decoding and robust mode decoding, the decoding of predicted pictures or segments requires video data representing nearby pictures. The normal mode frame memory 250' saves these nearby pictures for the normal mode decoder 240', and the robust mode frame memory 250" saves these nearby pictures for the robust mode decoder 240".

In the receiver shown in Figure 6, the conversion is performed on a screen-by-screen basis rather than on an IDS basis. The normal mode decoder 240' decodes the normal mode packets into relevant representative content signals including continuous video pictures. At the same time, the robust mode decoder 240" decodes the robust mode packets into relevant representative content signals containing continuous video pictures. As described above, the demultiplexer 210 generates a composite signal from the demodulator 207 or at least some parts of it. Use the error signal on the error signal line E. In the embodiment shown in Figure 6, this error signal can be generated by detecting the missing packets in the demultiplexed packet stream. Therefore, the error signal on the error signal line E It not only indicates that the packet is lost, but also that the packet stream is losing packets. Because the packet transmits part of the data that forms the video picture of the packet stream transmission in the payload, and transmits the data identifying the packet stream to which this packet belongs in the header, it is losing The packet flow of the packet can be marked as erroneous.

The video image may be successfully received in both the normal mode packet stream and the robust mode packet stream; it may be successfully received in the normal mode packet stream, but the video image may be received incorrectly in the robust mode packet stream; it may be in the normal mode packet Errors are received in the stream, but the video images are successfully received in the robust mode packet stream; or the video images may be received in error in both the normal mode packet stream and the robust mode packet stream.

Under normal conditions, that is, when no errors are detected in either the normal mode packet stream or the robust mode packet stream, both the normal mode decoder 240' and the robust mode decoder 240" both successfully decode the corresponding video pictures. The selector 230' couples the representative content video picture derived from the normal mode decoder 240' to the output terminal 215. In addition, under normal conditions, the normal mode decoder 240' supplies the video picture to the normal mode frame memory 250', and The robust mode decoder 240" supplies the video pictures to the robust mode frame memory 250".

If an error is detected in the robust mode packet stream, but no error is detected in the normal mode packet stream, only the normal mode decoder 240' successfully decodes the corresponding video picture. The selector 230' couples the representative content video picture derived from the normal mode decoder 240' to the output terminal 215. In addition, the normal mode decoder 240' supplies the decoded video picture to the normal mode frame memory 250'. However, since the robust mode decoder 240" has not successfully decoded the corresponding video pictures, it does not supply any video pictures to the robust mode frame memory 250". Instead, the successfully decoded video picture from the normal mode decoder 240' is routed from the selector 230' to the robust mode frame memory 250".

If an error is detected in the normal mode packet stream, but no error is detected in the robust mode packet stream, only the robust mode decoder 240" successfully decodes the corresponding video picture. The selector 230' will derive from the robust mode decoder 240" The representative content video picture of is coupled to the output terminal 215. In addition, the robust mode decoder 240" supplies the decoded video picture to the robust mode frame memory 250". However, since the normal mode decoder 240' has not successfully decoded the corresponding video picture, it does not supply any video picture to the normal mode frame memory 250'. Instead, the successfully decoded video pictures from the robust mode decoder 240" are routed from the selector 230' to the normal mode frame memory 250'.

In the above two cases, the video picture stored in the frame memory associated with the decoder that has not successfully decoded the video picture is the video picture from the other decoder. This may make subsequent decoding worse than what would have been if the correct video pictures were stored in the frame memory. This is especially true if the replacement video picture has a lower quality than the error video picture. However, if no video images are stored in the frame memory at all, the accuracy of subsequent decoding will be better.

If errors are detected in the video pictures in both the normal mode packet stream and the robust mode packet stream, the video picture cannot be decoded accurately, and other masking techniques must be used.

The operation of the receiver 200 shown in FIG. 6 can be better understood with reference to FIG. 7. In Figure 7, the top group of rectangles (MPEG) represent the input 420 and output 520 of the normal mode decoder 240', respectively; the middle group of rectangles (JVT) represent the input 440 and output 540 of the robust mode decoder 240", respectively; and The bottom group of rectangles (OUTPUT) respectively represent the video pictures 460 on the output terminal 215 and their source 560. Regarding MPEG decoding: the upper group of rectangles 420 represent the source coded video pictures on the input terminal of the normal mode decoder 240' (I, P, and/or B). The lower set of rectangles 520 represents the resulting video pictures on the output of the normal mode decoder 240'. Similarly, regarding JVT decoding: the upper set of rectangles 440 represents the robust mode decoder 240" The source coded IDR picture (which can include several pure I segments) and the subsequent source coded video segments (I, P, B, SI, and/or SP) on the input end of. The lower set of rectangles 540 represent the resulting video pictures on the output end of the robust mode decoder 240". Regarding the output end 215: the upper group of rectangles 460 represents the output video pictures and the lower group of rectangles 560 represents the source of that particular video picture.

More specifically, in the normal mode (MPEG) packet stream, as indicated by diagonal hatching, each of the video pictures 6, 10, and 13 loses at least one packet. Similarly, in the robust mode (JVT) packet stream, as indicated by the diagonal hatching, video pictures 7 and 10 lose at least one packet. All other video pictures of both the normal mode packet stream and the robust mode packet stream include all packets and can be successfully decoded.

For video pictures 0-5, 8, 9, 11, 12 and 14, as indicated by "M" in Figure 7, the selector 230' will couple the video pictures derived from the normal mode decoder 240' (MPEG) to the outputend215. In addition, for these video pictures, the video pictures from the normal mode decoder 240' are supplied to the normal mode frame memory 250', and the video pictures from the robust mode decoder 240" are supplied to the robust mode frame memory 250".

For pictures 6 and 13, the video picture in the normal mode packet stream is wrong, but the corresponding video picture in the robust mode packet stream is complete and usable. For these pictures, as indicated by "J" in FIG. 7, the selector 230' couples the video pictures from the robust mode decoder 240" (JVT) to the output terminal 215. For these pictures, there is no normal mode video picture The robust mode video pictures from the robust mode decoder 240" are coupled to both the normal mode frame memory 250' and the robust mode frame memory 250".

For picture 7, the video picture in the normal mode packet stream is complete, but the corresponding video picture in the robust mode packet stream is wrong. For this picture, as indicated by "M in FIG. 7, the selector 230 couples the video picture from the normal mode decoder 240 to the output terminal 215. Since there is no robust mode video picture for this picture, the normal mode video picture from the normal mode decoder 240' is coupled to both the normal mode frame memory 250' and the robust mode frame memory 250".

For picture 10, the video pictures in the normal mode packet stream and the robust mode packet stream are both wrong. Since there is no valid video picture, some form of error concealment can be used. This is indicated by "XX" in Figure 7. Since there is no valid video picture from the normal mode decoder 240' or the robust mode decoder 240", the undecoded video picture can be stored in the normal mode frame memory 250' or the robust mode frame memory 250". The data stored in frame memories 250' and 250" can also be derived from some form of error concealment.

By decoding the two packet streams into a stream of video pictures, and switching from one video stream to the other at the beginning of each video picture, the video artifacts caused by the failure to properly decode the packet stream can be minimized . In the receiver shown in FIG. 6, a transition in which the video quality gradually changes as shown in FIG. 5 can be used. However, because in the receiver of Figure 6, conversion occurs on every screen, the artifacts from such conversion will not be as objectionable as when conversion occurs on the IDS boundary as shown in Figure 2.

However, degraded channel conditions may cause frequent transitions between the normal mode packet flow and the robust mode packet flow. This frequent switching may cause artifacts that may be offensive to the audience. This is especially true if the video quality of the robust mode video signal is significantly different from the video quality of the normal mode video signal.

In order to minimize the artifacts caused by excessive frequent switching between the normal mode packet stream and the robust mode packet stream, the selectors 230 (Figure 2) and 230' (Figure 6) are configured to limit the switching to a frequency higher than a predetermined frequency. Frequently. More specifically, the selector 230 or 230' can monitor the frequency desired to be switched, and compare it with a predetermined threshold. If the frequency of the desired conversion exceeds the threshold, the frequency at which the actual conversion occurs is limited below a certain maximum frequency. This is a form of conversion lag.

For example, it is assumed that the normal mode packet stream transmits a high-quality (e.g., high definition (HD)) video signal and the robust mode packet stream transmits a low-quality (e.g., standard definition (SD)) video signal. When the normal mode HD packet stream is not available, the robust mode SD packet stream is processed to generate an image. In order to display on the HD display device, the SD video signal is scaled up to generate a poor quality video image. If the normal mode packet stream frequently fades in strength, but the robust mode packet stream is still available, frequent transitions will occur between the normal mode HD video signal and the robust mode SD video signal. Frequent conversions between HD packet streams and SD packet streams, as well as frequent conversions between high-quality images and low-quality images, will produce artifacts that are offensive to the audience.

Continuing the discussion of this example, if the error signal E indicates that a transition (ie, normal mode packet loss) should occur, for example, more than twice per minute, then the actual transition is limited to minimize the transition artifacts described above. In this example, under these conditions, the selector 230 or 230' selects, for example, a robust mode packet stream of at least one minute for each transition. This will reduce the number of conversions and, therefore, minimize the visible artifacts caused by those conversions. Those of ordinary skill in the art should understand that this is just one embodiment for realizing the conversion hysteresis. The threshold values of the maximum switching frequency and the limited switching frequency that cause hysteresis can be made different from those of this example. Such thresholds can be determined empirically to find those that minimize objectionable visible artifacts. And, the threshold can be dynamically changed during the operation of the receiver. Finally, other lagging algorithms can be developed that limit conversion when conditions that usually result in excessive conversion exist.

Referring again to Figures 3 and 4, at the beginning of any broadcast or channel change, there are normal mode packets (310, 410) filling the interval designated as Tadv of the delay 220 (Figures 2 and 6). In the receiver shown in FIGS. 2 and 6, the receiver only starts to work after the delay circuit 220 is full. However, when the receiver is switched on or the channel is changed, this causes an undue delay. However, within the time interval Tadv, the robust mode packet stream (300, 400) is immediately available.

In FIG. 2, as shown by the dotted line, the undelayed robust mode packet stream is directly coupled from the demultiplexer 210 to the third input terminal of the selector 230. When the receiver is powered on or a new channel is selected, the selector 230 couples the undelayed robust mode packet stream to the multi-standard decoder 240. As described in detail above, the multi-standard decoder 240 is limited to unpacking and decoding robust mode packets, and makes the video signal available on the output terminal 215 for immediate use in utility circuits. When the normal mode packet stream is available, the selector 230 couples the normal mode packet stream signal to the multi-standard decoder 240.

In Figure 6, the undelayed robust mode packet stream is directly coupled from the demultiplexer 210 to the robust mode decoder 240". When the receiver is powered on or a new channel is selected, as described in more detail above, the robust mode decodes The receiver 240" is limited to unpack and decode the robust mode packet stream from the demultiplexer 210 and generate a robust mode video signal, and the selector 230' is limited to convert the robust mode video signal from the robust mode decoder through the output terminal 215" Coupled to the utility circuit. When the normal mode packet stream is available, the normal mode decoder 240' unpacks and decodes it to generate a normal mode video signal. The selector 230' is limited to couple the normal mode video signal to the utility through the output terminal 215 Circuit.

In each case, analyze the data in the normal mode packet stream and the robust mode packet stream to determine when the normal mode packet stream becomes available and the receiver can start working normally. According to the known MPEG2 system (transport packet) encoding, the information related to the system clock (STC) in the transmitter is put into the encoded packet stream through the program clock reference (PCR) data. Further information called the presentation time stamp (PTS), which indicates the time at which a part of the packet stream (referred to as an access unit) must be decoded, is included at least at the beginning of each such access unit. When the multi-standard decoder 240 (Figure 2) or the normal mode decoder 240' and the robust mode decoder 240" (Figure 6) unpack and decode the normal mode packet stream and the robust mode packet stream, the receiver is The system clock (STC) is synchronized with the system clock (STC) in the transmitter. When the value of PTS in the normal mode packet stream is equal to the value of the receiver STC, it means that the normal mode packet stream is synchronized with the robust mode packet stream, and as above As described, the receiver can start to work normally by decoding the normal mode packet stream.

Since many signals representing content can be sent on a multiplexed transport packet stream, known devices have been developed to supply information about different packet streams. Each packet flow is identified by a packet identifier (PID), and the PID is included in the header of each packet in that packet flow. A packet stream with a predetermined known PID contains one or more data tables that contain identifiers and other information about all other packet streams. This known table structure can be used to convey information about robust mode packet flows that are not related to any other normal mode packet flows. However, additional information about the robust mode packet flow related to other normal mode packet flows must be sent from the transmitter to the receiver.

The extended syntax and semantics of these existing tables can convey the necessary information. FIG. 8 is a table illustrating the extended syntax and semantics of the program mapping table (PMT) and/or the program and information system protocol-virtual channel table (PSIP-VCT). Each row in Figure 8 represents a data item in the extended table, or a meta-grammar description in pseudo-code form. The first column is the name or meta-grammatical specification of the data item. The second column is the description of the data item or grammar stipulation. The third column is the size representation of any data item.

The first item 802 in the extended syntax is the number of robust mode packet streams used for interleaving with other normal mode packet streams. Then, as indicated by the meta-grammatical provisions in the next and last rows of the table, the table includes information about each such interleaved robust mode packet stream. Some of this information is required by each robust mode packet flow. For example, data 804 represents the program identifier (PID) of the robust mode packet flow; data 806 represents the type of data transmitted by that packet flow; data 808 represents the PID of the normal mode packet flow associated with this packet flow; and data 810 represents The delay 130 in the transmitter 100 (FIG. 1) introduces the delay in the normal mode packet stream.

However, some of this information is only relevant to the robust mode packet flow of a specific data type. For example, if the robust mode packet stream transmits video data, the information 812 related to the compression format, frame rate, interlace format, horizontal and vertical resolution, and bit rate is sent from the transmitter to the receiver so that it can be decoded and decoded appropriately. Display the video image represented by the robust mode packet stream. Similarly, if the robust mode packet stream transmits audio data, the information 814 related to the compression format, bit rate, sampling rate, and audio mode (surround, stereo, or mono) is sent from the transmitter to the receiver so that Properly decode and reproduce the sound represented by the robust mode packet stream.

A piece of other data is related to the relative quality of the representative content signal transmitted by the robust mode packet stream. As described above, the quality of the representative content signal transmitted by the robust mode packet stream may be different from the quality of the representative content signal transmitted by the normal mode packet stream associated therewith. In the above example, it is stipulated that the quality of the representative content signal transmitted by the robust mode packet stream is lower than the quality of the representative content signal transmitted by the related normal mode packet stream. However, under some conditions, the provider can send a higher quality signal on the robust mode packet stream. Under this condition, it is best for the receiver to use the representative content signal transmitted by the robust mode packet stream instead of the representative content signal transmitted by the related normal mode packet stream. This situation is indicated to the receiver through data 816.

By providing information linking the robust mode packet stream with the normal mode packet stream, the receiver 200 (of FIG. 2) or 200' (FIG. 6) can find the normal mode packet stream and the packet stream in the multiplexed packet stream. The robust mode packet flows both, and, as described above, handles both at the same time. Previous receivers that did not include the capabilities of the receivers of Figures 2 and 6 ignored this information and processed the normal mode packet stream in a known manner.

As described above, in the table shown in FIG. 8, the delay introduced by the delay 130 in the transmitter 100 (FIG. 1) between the robust mode packet flow and the related normal mode packet flow is transmitted as data 810. This allows the transmitter to change the delay interval and the receiver to adjust its delay interval accordingly. For example, under some channel conditions, fading may be more likely than under other channel conditions, or the characteristics of the fading may change (ie, the fading may be prolonged). Under such conditions, the delay interval can be increased. The length of the delay is sent to the receiver, and the receiver adapts the delay 220 (in FIGS. 2 and 6) to the same delay interval. Other conditions may also require different delay intervals.

The concept of interleaved broadcasting as described above can be promoted. Multiple versions of the same representative content signal may be interleaved encoded into different video qualities (for example, resolution, frame rate, etc.). Figure 9 is a block diagram of a portion of another embodiment of an interleaved transmitter that transmits multiple versions of a representative content signal. In FIG. 9, the same components as those in the transmitter shown in FIG. 1 are denoted by the same reference numerals, and detailed descriptions will not be given below. Figure 10 is a block diagram of a part of a corresponding embodiment of an interleaved receiver. In FIG. 10, the same components as those in the receiver shown in FIG. 2 are denoted by the same reference numerals, and detailed descriptions are omitted below.

In FIG. 9a, the input terminal 105 is coupled to the input terminal of the hierarchical encoder 160. The hierarchical encoder 160 source encodes and packs several output packet stream signals. The first (0) of the plurality of output packet stream signals is coupled to the corresponding input terminal of the multiplexer 140'. The remaining parts (1) to (n) of the plurality of output packet stream signals are coupled to the respective input terminals of the corresponding plurality of delays 130(1) to 130(n). The delay interval introduced by the retarder (2) is greater than the delay interval introduced by the retarder 130(1); the delay interval introduced by the retarder 130(3) (not shown) is greater than the delay interval introduced by the retarder 130(2); analogy. As shown in Figure 3, the delay can be specified according to the packet; as shown in Figure 4, the delay can be specified according to the independent decoding segment; or as shown in Figure 7, the delay can be specified according to the video frame interval. The respective output terminals of the plurality of delays 130(1) to 130(n) are coupled to the corresponding input terminals of the multiplexer 140'.

In the working process, the first packet stream signal (0) transmits the source-encoded basic video signal at the lowest video quality. The second packet stream signal (1) conveys additional video information. This additional video information, when combined with the basic video signal (0), generates a video signal with a higher video quality than that of the basic video signal (0) alone. The third packet stream signal (2) conveys further additional video information. When the video information in this signal is combined with the video information in the basic video signal (0) and the second packet stream signal (1), the resulting video quality is better than the combination of the basic signal (0) and the second signal (1) Video signal with high video quality. The video information in the additional packet stream signals from the layered encoder 160 up to the packet stream signal (n) can be combined together to generate a video signal with higher video quality. The multiplexed signal is channel coded (modulated), and supplied to the receiver through the output terminal 115.

Fig. 10a is a receiver corresponding to the transmitter shown in Fig. 9a. The demultiplexer 210 extracts several packet streams (0) to (n). The packet stream (n) is coupled to the corresponding input of the layered decoder 260. The remaining parts (0) to (n-1) of the plurality of packet streams are coupled to the respective input terminals of the corresponding plurality of delays 220. The number of delayers 220 are limited to realign in time all the number of packet streams (0) to (n) on the input of the layered decoder 260. The error signal on the signal line E from the demultiplexer 210 is coupled to the control input of the layered decoder 260. The output terminal of the layered decoder 260 is coupled to the output terminal 215.

During operation, as described in more detail above, the demodulator 207 channel decodes (demodulates) the received signal when appropriate. The demultiplexer 210 extracts a number of packet streams (0) to (n) corresponding to the packet streams (0) to (n) shown in FIG. 9a that convey layered video information. Several delayers 220 align these packet streams in time. The error signal from the demultiplexer 210 indicates which packet streams are not available, for example, lost packets. The layered decoder 260 unpacks several packet streams and generates the highest quality video signal that can be layeredly decoded from the available packet streams. That is, if all fading events except the packet stream (0) that transmits the basic video signal are made unavailable, the layered decoder 260 only unpacks and decodes the packet stream (0). If the packet stream (1) is also available, the layered decoder 260 unpacks and decodes both the packet stream (0) and the packet stream (1) and generates a higher quality video signal, and so on. If all packet streams (0) to (n) are available, the hierarchical decoder 260 unpacks and decodes all packet streams and generates the highest quality video signal. In Figure 9b, the output terminal 105 and several video encoders 170 The respective input terminals are coupled. The output end of the first 170(0) of the plurality of video encoders 170 is coupled to the corresponding input end of the multiplexer 140'. The output ends of the remaining parts 170(1) to 170(n) of the plurality of video encoders 170 are coupled to the respective input ends of the delays 131(1) to 130(n). The delay interval introduced by the retarder 130(2) is greater than the delay interval introduced by the retarder 130(1); the delay interval introduced by the retarder 130(3) (not shown) is greater than the delay interval introduced by the retarder 130(2); And so on. As shown in Figure 3, the delay can be specified according to the packet; as shown in Figure 4, the delay can be specified according to the independent decoding segment; or as shown in Figure 7, the delay can be specified according to the video frame interval. The respective output terminals of the several delays are coupled to the corresponding input terminals of the multiplexer 140'.

In the working process, the first encoder 170(0) source coded the representative content signal and the system coded (packed) the source coded signal to generate a packet stream that conveys the information representing the lowest quality video signal: In the embodiment, a quarter common interface format (QCIF) video signal. The second encoder 170(1) similarly generates a packet stream that conveys information representing a video signal of higher quality than the video signal generated by the first encoder 170(0): in the illustrated embodiment, the common interface format (CIF) video signal. Other video encoders not shown similarly generate packet streams that transmit video signals with successively improved video quality. The SD video encoder 170(n-1) similarly generates a packet stream for transmitting SD quality video signals, and the HD video encoder 170(n) similarly generates a packet stream for transmitting HD quality video signals. These packets are multiplexed by the multiplexer 140', then channel coded (modulated), and sent to the receiver through the output 115.

Fig. 10b is a receiver corresponding to the transmitter shown in Fig. 9b. In FIG. 10b, the demultiplexer 210 extracts several packet streams (0) to (n). The packet stream (n) is coupled to the input of the HD decoder 270 (n). The remaining parts (0) to (n-1) of the packet stream are coupled to the respective input terminals of the corresponding number of delays 220. The respective output terminals of the plurality of delayers 220 are coupled to the corresponding input terminals of the plurality of video decoders 270. The respective output terminals of the plurality of video decoders 270 are coupled to the corresponding input terminals of the selector. The error signal on the error signal line E from the demultiplexer 210 is coupled to the control input of the selector 280.

During operation, as described in more detail above, the demodulator 207 channel decodes (demodulates) the received composite signal when appropriate. The demultiplexer 210 extracts packet streams (0) to (n) corresponding to those generated by the several video encoders 170 as shown in FIG. 9b. The delayers 220 realign in time all the packet streams (0) to (n) on the respective input terminals of the video decoders 270. Each packet stream is coupled to a video decoder suitable for decoding the video signal carried by that packet stream. For example, a packet stream that transmits a QCIF quality video signal is coupled to the QCIF decoder 270(0); a packet stream that transmits a CIF quality video signal is coupled to the CIF decoder 270(1), and so on. Each of the plurality of video decoders 270 unpacks and the source decodes the signal supplied to it to generate a video signal. The error signal from the demultiplexer 210 indicates which of the packet streams (0) to (n) is unavailable due to an error (for example, a lost packet). The selector 280 is restricted to couple the highest quality video signal generated from the available packet stream to the output 215.

Those of ordinary skill in the art should understand that image scaling may be required for some lower quality video image signals in the transmitter system as shown in FIG. 9. The encoder, that is, the hierarchical encoder 160 of FIG. 9a or the several encoders 170 of FIG. 9b includes any such necessary image scaling circuits that are not shown in order to simplify the image.

For the communication system shown in Figures 9 and 10, as described in more detail above, it can be based on a robust source coding scheme (JVT) source coding and through a robust modulation scheme (4-VSB and/or 2-VSB) channel Encode (modulate) any packet stream generated by the hierarchical encoder 160 (Figure 9a) or any one of the several video encoders 170 (Figure 9b). The corresponding demodulation and decoding of that packet stream takes place in the receiver of Figure 10. In addition, the best quality video signal leads the most, so there is the highest fading impedance. Also, the lowest video quality signal may be encoded with the least number of bits, so it takes a small amount of time to transmit. With the continuous improvement of the video quality of the video signal transmitted by the packet stream, the lead time of the packet stream is continuously shortened. Therefore, the fading impedance is continuously reduced. Therefore, when the channel characteristics are not fading, the packet stream that transmits the highest video quality signal is still available. Moderate fading makes available packet streams that transmit lower video quality signals, and severe fading makes available only packet streams that transmit lowest quality video signals. This kind of video quality gradually declines as the channel characteristics deteriorate, which is a desirable characteristic for viewing.

As described above, and as shown in FIGS. 1 and 9b, the same representative content signal can be interleaved into a packet stream that transmits a high-quality video signal and one or more packet streams that transmit a video signal with a reduced video quality. Therefore, in such a communication system, some receivers, for example, a television receiver in a cellular phone or a personal digital assistant (PDA), can extract and decode only the representative content signal of reduced quality. In such a receiver, the display device is low-resolution, and may only display video signals of reduced quality. Moreover, the use of battery power is beneficial to minimize the amount of data processed. These two considerations imply that such receivers only decode packet streams that convey video signals of appropriate video quality and display that kind of image.

Figure 10c illustrates a receiver. In FIG. 10c, the input terminal 205 is coupled to the input terminal of the demodulator 207. The output terminal of the demodulator 207 is coupled with the output terminal of the demultiplexer 210. The output terminal of the demultiplexer 210 is coupled with the input terminal of the decoder 270. The output terminal of the decoder 270 is coupled to the output terminal 215.

During operation, as described in more detail above, the demodulator 270 demodulates the received composite signal in an appropriate manner. The demultiplexer 210 selects only a single packet stream containing the video signal of the desired quality. For example, this may be a QCIF format video signal as generated by the QCIF encoder 170(0) of FIG. 9b and transmitted on the packet stream (0). The demultiplexer 210 extracts and the decoder 270 decodes the packet stream (0) to generate a QCIF format video signal. Such a receiver only needs to receive the table shown in FIG. 8 to determine the PID of the desired low-quality video signal packet stream (0). Based on the resolution data 812 sent in the table, the mobile receiver can choose to transmit the packet stream of the reduced quality video signal that it wants to process.

The communication system shown in Figures 9 and 10 can be further promoted. In the system as described above, the video information conveyed in the additional packet stream can be used to provide moderate degradation under degraded channel conditions. However, such a system can also send additional video information that can improve the quality of the video signal under good channel conditions. By including the packet stream that transmits the amplified video information, in addition to the packet stream that transmits the normal video signal, the amplified video information can also be transmitted.

FIG. 11 is a block diagram of a part of a transmitter that transmits a dual-interlaced video signal, and FIG. 12 is a block diagram of a portion of a receiver that receives a dual-interlaced video signal. FIG. 13 is a display diagram helpful to understand the operation of the double-interleaved transmitter shown in FIG. 11 and the double-interleaved receiver shown in FIG. 12. In FIG. 11, the same components as those shown in FIG. 1 are denoted by the same reference numerals, and the detailed description will not be given below. In FIG. 12, the same components as those shown in FIG. 6 are denoted by the same reference numerals, and the detailed description will not be given below.

Referring to FIG. 13, the content source generates a progressive scan video display that is schematically illustrated as a series of video lines 1310 within a display boundary 1320 at the top of FIG. 13. The normal HD video image screen includes 1080 lines. Such HD video images are sent at a rate of 30 frames per second in an interlaced format. That is, the interleaver generates two fields: the first field includes only odd lines and the second field includes only even lines. These half frames are sent successively at a rate of 60 half frames per second.

In FIG. 11, the output terminal 105 is coupled to the dual output interleaver 102. The first output terminal of the dual output interleaver 102 is coupled to the input terminal of the robust mode encoder 110. The second output terminal of the dual output interleaver 102 is coupled to the input terminal of the normal mode encoder 120.

Referring again to FIG. 13, the frame display image 1330(A) corresponds to the video signal A generated on the first output terminal of the dual output interleaver 102, and the frame display image 1330(B) corresponds to the first output of the dual output interleaver 102 The video signal B generated on the second output terminal corresponds to it. In the frame display images 1330(A) and 1330(B), a solid line is transmitted in one field, and a dotted line is transmitted in the following field. In the frame display image 1330(A), the solid line is an odd line and the broken line is an even line; and in the frame display image 1330(B), the solid line is an even line and the broken line is an odd line. This case is exemplified in more detail in the field display images 1340 (A), 1340 (B), 1350 (A), and 1350 (B) located below the frame display images 1330 (A) and 1330 (B). In field 1, the video signal A sends odd-numbered lines as shown in the field display image 1340(A), and the video signal B sends the even-numbered lines as shown in the field display image 1340(B). In field 2, the video signal A sends even-numbered lines as shown in the field display image 1350(B), and the video signal B sends the odd-numbered lines as shown in the field display image 1350(B).

As described in more detail above, the robust mode encoder 110 uses JVT source encoding to source-encode the video signal A, and then systematically encodes (packs) it. The normal mode encoder uses MPEG2 source coding to source code the video signal B, and then performs system coding (packing) on it. The modulator uses 4-VSB and/or 2-VSB modulation to channel encode (modulate) the robust mode packet stream, and uses 8-VSB modulation to modulate the normal mode packet stream.

In FIG. 12, the first output terminal of the demultiplexer 210 is coupled with the input terminal of the normal mode decoder 240', and the second output terminal of the demultiplexer 210 is coupled with the input terminal of the delay 220. The output of the normal mode decoder 240' is coupled to the first signal input of the dual input deinterleaver 202, and the input of the robust mode decoder 240" is coupled to the second signal input of the dual input deinterleaver 202. From. The error signal of the demultiplexer 210 is coupled to the control input of the dual-input deinterleaver 202. The output of the dual-input deinterleaver 202 is coupled to the output 215.

As described in more detail above, the demodulator 207 uses 4-VSB and/or 2-VSB demodulation channels to decode (demodulate) the robust mode packet stream, and uses 8-VSB demodulation to demodulate the normal mode packet stream. The normal mode decoder 240' systematically decodes (unpacks) and decodes the normal mode packet stream using the JVT decoding source to reproduce the video signal B. The robust mode decoder 240" unpacks and uses MPEG2 decoding to decode the robust mode packet stream from the source to reproduce the video signal A.

The dual input deinterlacer 202 works to combine the interlaced scan lines of the video signal A from the robust mode decoder 240" and the interlaced scan lines of the video signal B from the normal mode decoder 240' to generate progressive fields. For field 1, the odd-numbered lines from the video signal A as shown in the field display image 1340(A) and the even-numbered lines from the video signal B as shown in the field display image 1340(B) are combined together. The progressive scan field is displayed in the field display image 1345. For field 2, the even-numbered lines from the video signal A as shown in the field display image 1350(A) and the field display image 1350(B) The odd-numbered lines from the video signal B shown are combined together. The resulting progressive scan field is displayed in the field display image 1355. Therefore, each field interval can be generated on the output of the dual-input deinterleaver 202 Scan field progressively. For HD signals, this means that the entire 1080-line image is generated 60 times per second.

The dual interleaving technique described above and shown in FIGS. 11, 12, and 13 can also be combined with the technique described above in order to provide a wider range of moderate degradation when the channel conditions deteriorate. If the channel conditions make one of the packet streams carrying the video signal A or B unusable, the error signal E indicates this to the dual input deinterleaver 202. The dual input deinterlacer 202 starts to generate a standard HD interlaced video signal from the available video signal. The display device (not shown) is reconfigured to display the image represented by the standard interlaced video signal before other video signals are available again. If none of the HD video signals are available, as described in detail above with reference to the transmitter in FIG. 9 and the receiver in FIG. 10, the highest quality available video signal can be displayed.

The same technique can also be used to convert any interlaced format video signal, for example, an SD video signal into a progressive scan video signal with twice the frame rate. As shown in FIGS. 11 and 12, the two video signals A and B are not necessarily interleaved. Just broadcast them at the same time. However, as mentioned above, in the presence of fading events, interleaved broadcasting additionally provides moderate degradation.

The communication system as described above can be further extended to cooperate with recording equipment, such as a digital personal video recorder (PVR). As the cost of such PVR equipment continues to drop, such PVR equipment is gradually incorporated into digital TV receivers. In FIG. 9b, as shown by the dotted line, the PVR device 295 includes a video terminal (Vid) that is bidirectionally coupled with the selector 280, and a control terminal (Ctl) that is also bidirectionally coupled with the selector 280. As also shown by the dashed line, the selector 280 is also coupled with a user control source.

The selector 280 is configured to couple any desired video signal from the number of video detectors 270 to the PVR 295 independently of the input video signal coupled to the output terminal 215. The selector 280 may also be configured to couple the input video signal from the PVR 295 to the output terminal 215 for playback. The selector 280 can also supply control data to the PVR 295, and the PVR 295 supplies status data to the selector 280 on the bidirectional control terminal.

The PVR 295 can be controlled in several working modes. In one mode of operation, the best available video signal is coupled to the PVR 295 for recording. In this mode of operation, the selector 280 couples the same input video signal as that coupled to the output terminal 215 to the PVR 295. This will result in the best quality video signal being recorded in the PVR 295, but will cost most of the storage space. This will take advantage of the normal mode packet stream and robust mode packet stream that convey the video signal and provide a moderate degradation. Alternatively, the low-resolution video signal may be coupled to the PVR 295 instead of the output terminal 215. For example, although the selector 280 may couple the best available video signal to the output terminal 215, the selector 280 may also couple the video decoder 270 that generates a poor quality video signal with the PVR 295. This poor quality video signal may be a selected one of the available video signals supplied by the poor quality video decoder with moderate degradation, such as the SD quality video signal from the SD decoder 270 (n-1). Such a signal in PVR 295 requires less storage space than the best available video signal. This helps to save storage space in the PVR295, making the recording time longer. In the case where the selected lower-quality video signal is not available, the higher-quality video signal can be recorded before the lower-quality video signal is available again. The viewer can directly select the option of recording which poor quality video signal (ie, SD, CIF or QCIF) through the user input terminal. Alternatively, the selector 280 may automatically control this selection according to certain criteria. For example, the status signal from PVR 295 may indicate the remaining storage capacity in PVR 295. As the remaining storage capacity continues to decrease, the selector 280 may automatically couple the video decoder 270 with reduced video quality to the PVR 295. Other standards may be introduced by the selector 280 to control which video signal is coupled to the PVR 295.

Similarly, the user may wish to control the selection and display of television programs broadcast by the transmitter. In the existing broadcasting system, one of the transmitted packet streams conveys a user program guide, and the user program guide contains information about all programs currently being broadcast and programs scheduled to be broadcast in the near future. Based on the program guide data, an on-screen display generator (OSD) 282 as shown in FIG. 10b can generate an image of a table listing all such programs, their channels, and times. As an auxiliary tool to use the user interface to find the desired program and select that program to watch, the user can control the display of the program guide information. The user interface displays images that present information to the audience, request input from the audience, and accept input from a controller that may be incorporated in the receiver or remote control. Existing systems allow viewers to request additional information about the program list, such as a more detailed description of the program, level (G, PG, R, etc.), length of time, remaining time, etc.

The additional information related to the interlaced broadcasting system as described above can be added to the displayed program list, or the additional information is displayed. This information can be derived from the PSIP-VCT/PMT table shown in Figure 8. For example, an additional indicator can be added to the displayed program list and/or additional information display to indicate: the program is being interlaced; what video quality is the video signal being interlaced; what is the audio signal being interlaced Audio quality; etc. By displaying this information to the audience, the audience can make basic choices about the above program. More specifically, the viewer can select a program that is being interlaced; or can select a program that contains a video signal of the desired video quality, for example, to match the display device that supplies the signal.

Current receivers also allow viewers to set certain parameters. For example, the user may want to automatically watch all the sent channels, or only the channels reserved by the viewer, or the reserved channels plus pay-per-view channels, etc., instead of manually changing the screen display every time it is displayed. The user interface shows the screen image to the user through the OSD 282, and the user can use the controller to make this selection on the OSD 282. An additional screen image can be generated, or an existing screen image can be modified. As described above, the viewer can set options on the selection and display of the video signal that has been interleaved. For example, viewers can choose to have the program table only show interlaced programs, or show interlaced nodes that transmit video signals with or higher than the lowest video quality.

In addition, as described above, the Robust_Mode_High_Quality flag 816 in the PSIP-VC6/PMT table of FIG. 8 indicates that the robust mode packet stream is transmitting the highest quality video signal and the robust mode packet stream should be used unless that packet stream is not available. This data can also be displayed in the program list, and the viewer can also make a choice from that list based on this flag. In addition, the audience can set parameters according to this flag. For example, viewers can choose to show only channels with this flag set.