CA2387254C

Apparatus for arranging compressed video data for transmission over a noisy communication channel

Abstract

A digital compressed video signal transmission system includes a transport processor (FIG. 7) for segmenting (105,115) compressed data into transport cells for transmission. Particular portions of the compressed data are formatted (110,120,125) into further transport cells, which further transport cells are interspersed with normally occurring transport cells. The further transport cells include redundant video signal data which may be utilized to resynchronize a compressed video signal decoder after loss or corruption of transmitted data.

CA2387254C, drawing sheet 1 — Sheet 1 of 11

Term

No projected expiry on record.

9 claims: 9 independent, 0 dependent

1
CLAIMS:1. In a digital video transmission system for transmitting MPEG compressed video signal including layered data having headers containing data descriptive of respective layers, a method for segmenting said compressed video signal into transport cells, respective transport cells containing three protocol layers;said method comprising: providing said compressed video signal;forming a link layer header including at least a first data field identifying the service to which the video signal is to be applied;forming a video transport layer header including a continuity count which is service specific and which increments in successive transport cells, and a data field which indicates one of a plurality of alternative, data formats selected for the respective service layer;dividing said compressed video signal into payloads of no greater than a predetermined number of bits;forming a service layer from one of said plurality of alternative data formats, which service layer may include at least one of said payloads;and concatenating said link layer header, said transport layer header and said service layer for conveyance.
2
The method set forth in claim 1 wherein the step of forming said link layer header further includes providing two data fields relating to whether data is scrambled, the first of which indicates whether or not data is scrambled and the second of which indicates alternative scrambling keys.
3
The method set forth in claim 2 wherein the step of forming said link layer header further includes forming a header data field, different from a service identification data field and containing a bit which indicates the occurrence of a predetermined data boundary within the corresponding service layer. CA 02387254 2002-06-20 RCA 86755B
4
The method set forth in claim 1 wherein the step of forming a service layer further comprises including a further data field which indicates the number of bytes of a predetermined type of data included within a payload.
5
The method set forth in claim 1 wherein the step of forming a service layer comprises:forming a service layer of one of an auxiliary and a basic service format.
6
The method set forth in claim 1 wherein the step of forming said link layer header further includes forming a header data field containing a bit which exhibits an indication of an occurrence of a predetermined signal boundary within the corresponding service layer.
7
The method set forth in claim 1 wherein the step of forming a service layer comprises:determining the type of compressed video signal contained in respective payloads and the location of decoding entry points within respective payloads;forming a service layer header which includes a data field to identify said type of compressed video signal in a respective pay load, and a data field descriptive of locations of respective said entry points.
8
The method set forth in claim 1 wherein the step of forming said service layer further includes:forming auxiliary pack respectively including auxiliary data and auxiliary data identifiers;and concatenating a variable number of said auxiliary packs to form said service layer.
9
The method set forth in claim 1 wherein the step of forming said transport layer header further comprises including a field having a bit which indicates the occurrence of one of a first sequence header of a GOP, the start of a B-Frame, and the start of a P-Frame within respective service layers. CA 02387254 2002-06-20 CA 02387254 2002-06-20 FIB. 2 VIDEO TRANSPORT LAYER -1 & 2 TIER SYSTEMS S3 CA 02387254 2002-06-20 AUXILIARY DATA GROUP FORMAT CA 02387254 2002-06-20

Independent claims9

199 paragraphs in 19 sections, as filed

(57) Abrégé/Abstract:

Canada

http://opic.gc.ca · Ottawa-Hull KIA 0C9 · http://cipo.gc.ca

OPIC · CIPO 191

CA 02387254 2002-06-20

APPARATUS FOR ARRANGING COMPRESSED VIDEO DATA FOR TRANSMISSION OVER A NOISY COMMUNICATION CHANNEL

Abstract of the Disclosure

I 0 loss or corruption of transmitted data.

CA 02387254 2002-06-20

APPARATUS FOR ARRANGING COMPRESSED VIDEO DATA FOR TRANSMISSION OVERA NOISY COMMUNICATIOKCHANNEL

This is a division of Application Serial No. 2,136,616 filed

May 20, 1993.

The present invention relates to a method and apparatus for segmenting compressed video data into cells or packets for transmission in a manner to allow a receiver to rapidly recover from occurrences of missing data or corrupted data.

Background of the Invention

The Moving Picture Experts Group (MPEG) are establishing a standard for transmission and storage of video data primarily for use by computers. This proposed standard is detailed in the document International Organization for Standardization, ISO-IEC JT(1/SC2/WG1), Coding of Moving Pictures and Associated Audio, MPEG 90/176 Rev. 2, Dec. 18, 1990. The signal protocol is hierarchical or layered. Frames of video data are compressed in groups of, for example, 15 frames. Respective frames are either intraframe encoded (I frames), forward predictive interframe encoded (P frames) or forward/backward predictive interframe encoded (B frames). Each frame is divided into slices corresponding to horizontal image bands (e.g., 16 line stripes). The slices are segmented into macroblocks consisting of matrices of 16 by 16 pixels. The macroblocks are encoded in four 8 by 8 blocks of luminance values, and two 8 by 8 blocks of chrominance values (U and V signal components). Each of the 8 by 8 chrominance blocks are derived by horizontally and vertically subsampling component chrominance values representing respective 16 by 16 macroblocks. The signal protocol calls for a sequence layer for identifying the overall signal type, which layer includes a sequence start code and header information identifying, for example, picture size, pixel aspect ratio, picture rate, bit rate, buffer size, a number of flag bits, etc. Following the sequence layer is a group of pictures, GOP header which include a start code, a time code, a closed GOP flag, a broken link flag and extension data. The next layer includes a picture start code and picture header. The picture (PICT) header includes a temporal

CA 02387254 2002-06-20 reference, picture coding type (I, P, B), buffer fullness, vector and pixel precision flags, variable length code identifiers and extension data. A slice start code follows the picture layer and includes a start code and a header identifying the slice. Following the slice layer is the macroblock layer which includes a start code and header data. The macroblock header data includes identifying indicia, quantizing information, type of encoding etc. The macroblock layer also includes motion vectors which are common to the six blocks of data in each macroblock, and encoded block

0 data on a block by block basis. The compression algorithm involves predicting frames of video signal from prior frames of video signal and transmitting in compressed form, the differences between actual and predicted frames. Successively encoded frames are dependent on the correctness of prior encoded frames.

5 Only one or a small number of frames in a group of pictures is non predictively encoded. It should be immediately recognized that, in a receiver, decoding errors due to data loss or corruption during transmission will propagate through successive frames within a GOP. In order to preclude the propagation of such errors and

0 concomitant image corruption special precautions must be taken.

However such precautions are not included in the MPEG protocol because it was fashioned primarily for noiseless transmission channels.

ADTV is a fully digital simulcast system that delivers

5 high definition television (HDTV) in a single 6-MHz broadcast channel. It is currently being developed by the Advanced Television Research Consortium (ATRC). One of the primary design goals of ADTV is to deliver high-quality and robust digital HDTV service for terrestrial simulcast transmission.* The ADTV

0 system uses MPEG compression to permit transmission of HDTV signals within a 6-Mhz channel. However the ATRC has augmented MPEG by adding a custom higher layer structure (MPEG++Rev 1) to achieve sufficient signal robustness for transmission over noisy terrestrial transmission media. This

5 augmentation includes the prioritization of MPEG data into a two

CA 02387254 2002-06-20 tier high-priority (HP), low priority (LP) transmission scheme, and includes a transport protocol to support multiple data services, and to provide graceful degradation in receiver performance in the presence of transmission errors.

DirecTV is a fully digital system that delivers standard definition NTSC television to the home over a satellite channel. It is currently being developed by Thomson Consumer Electronics (TCE). It is similar to ADTV in that it uses MPEG data compression but it is not HDTV. This is a one tier system,for transmitting NTSC

0 quality television signals

The present invention involves a transport protocol for arranging hierarchically formatted compressed video data for robust transmission in noisy communication channels and apparatus for realizing the transport protocol. The transport

5 protocol presented here defines cells (or packets) of data where each cell includes a Prefix and a Transport Block. In an exemplary embodiment, the Prefix consists of four bits of control information and twelve bits for service channel identification. The Transport Blocks, (typically 128 bytes) consists of either Auxiliary data,

0 Redundant MPEG Headers, or standard MPEG data. Compressed video data is applied to a transport processor which is responsive to the header data to develop transport block headers, and to store particular header data. The transport processor segments the. compressed data into data blocks of predetermined size and

5 appends transport headers thereto to form cells for transmission.

The particular stored header data is formatted into a plurality of cells and these cells are interspersed between regularly occurring successive cells of compressed data.

FIGURE 1 is a schematic representation* of a transport

0 cell (or packet) of the transport layer utilized in the invention.

FIGURE 2 is a schematic representation of a Transport Block for a video service included in a transport cell.

FIGURE 3 is a schematic representation of an example of a format of Auxiliary Data cells .

CA 02387254 2002-06-20

FIGURE 4 is a schematic representation illustrating an entry-point concept utilized for fast re-entry into the compressed data stream.

FIGURE 5 is a schematic representation of the entry5 point data in two-tier transmission systems.

FIGURE 6 is a system level block diagram identifying the transport encoder and decoder in the total system.

FIGURE 7 is a block diagram of a typical transport encoder.

0 FIGURE 8 is a flowchart representing operation of the transport encoder.

FIGURE 9 is a block diagram of a typical transport decoder.

FIGURES 10A and 10B are a flowchart of operation of

5 the transport decoder of a one-tier video system.

Detailed Description

The transport protocol of the present invention includes three data protocol layers; a link layer; a transport layer;

0 and a service layer. The link layer is arranged to be service independent, while the service layers are service specific. A service refers to the type of data being transmitted in a particular transport cell, e.g., audio data, video data, auxiliary data etc.

5 The link layer comprises a Prefix byte (actually two eight bit bytes) which contain several link layer control flags as well as channel identifiers for many different video, audio and data services. FIGURE 1 shows the logical structure of a transport cell indicating the relationship between the Prefix 'and the

0 Transport Block. The designators P, BB, CF, and CS are all one bit designators. The designator, P, is used in a two tier system to identify whether the transport block includes high or low priority data (1=HP, 0=LP), and is used in a one tier system for cell framing by toggling in successive cells. The designator, BB defines

5 a bundle boundary and is set to a 1 value only for the first cell

CA 02387254 2002-06-20 of respective bundles. The designator, CF, is a control flag used to indicate a scrambling state. CS is a control sync bit which toggles with each scramble key change.

The designation SCID is a twelve bit word which is used to identify service types. A SCID value of zero is reserved for null packets, and the value 4095 is reserved for future definition. The remaining 4094 SCID values are available for defining various service types.

FIGURE 2 illustrates the Video Transport Layer which

0 is an example of one of many possible Transport Layer formats. Every service type can have a specific Transport Block format. This description pertains to MPEG encoded Video Transport services. FIGURE 2 shows the logical structure of a transport block. The first field of the video transport layer contains a 4-bit

5 continuity counter (CC). This counter increments by one for each cell transmitted. It is service dependent and priority dependent,

i.e., separate counters are maintained for each service identity and for each transmission priority tier. The value of the continuity count sequences from 0 through 15. The continuity count

0 provides a measure of error detection at respective receivers. A discontinuity in the received count indicates either errors in received data or a loss of continuous data for a particular transport service.

The next field in the video transport layer contains a

5 4-bit Header Designator (HD) which has two-bit subfields of Type and Identity. The subfields identify the form of data transmitted in the respective data field. For the Video Transport Layer, HD types 0, I, 2 and 3 are used to respectively identify Auxiliary Packets, Basic Service Packets, Basic Service Packets with MPEG

0 redundant data, and Basic Service Packets with N0N_MPEG redundant data. The latter two types are non standard forms of transmitting MPEG data, and are included for completeness. The type Basic Service Packets is the only type identified which includes MPEG data in standard form albeit segmented in

5 transport cells The type Auxiliary Packets in general is not an

CA 02387254 2002-10-10

RCA 86755B

MPEG signal though in this application it is used to transmit redundant MPEG header data. Nominally the Auxiliary Packets are used to transmit auxiliary data such as closed captioning data, for example.

The HD identity values define subsets of the HD types. One HD type/identity value combination (0/0) indicates an Auxiliary Data Group cell, and its contents are defined in FIGURE 3. The fields of auxiliary data cells are to be unscrambled, therefore PREFIX bit CF is set to one. Each Data Group is self defined, with a flag-bit indicating whether additional Data Groups exist in the same packet. Data Groups contain such information as Time Code References, and Scramble Keys.

Basic Service packets are used to carry most of the MPEG encoded data. Two-tier basic service packets include entry-point data to synchronize the two data streams. Entry-points allow data blocks to segment across cell boundaries. This concept is illustrated in FIGURE 4. FIGURE 5 shows the entry-point components found in the entry-point data field for twotier transmission schemes. The frame type, slice, and macroblock identities are supplied by the video processor, while the entry pointer and frame number are supplied by the transport processor. The entry pointer is the byte offset to the entry-point position in the transport block. Frame type indicates whether the data refers to an intraframe encoded frame or an interframe encoded frame, or the first cell of a GOP. The frame number is used as a frame continuity counter, incrementing once per frame. Both the frame type and the frame number assist decoder synchronization of the two-tier data streams. The slice and macroblock identities are unique over the frame, and specify the entrypoint position without decoding the MPEG data stream. While having one entry-point per cell is a design goal, there is a wide range of data per slice dependent upon the priority channel and frame type. For further information on entry point processes, see U.S. Patent No. 5,168,356 issued December 1,

1992.

CA 02387254 2002-10-10

RCA 86755B

Two methods of carrying redundant MPEG data in the Transport Block may be utilized. One method uses a specific Auxiliary Packet to carry a copy of the MPEG sequence Header (which could span multiple packets). The second method uses a modification of the Basic Service Transport Block to carry a copy of MPEG Group of Pictures (GOP) Header and Picture Header.

All information contained in the video service layer is supplied by the video encoder (and the priority processor in a two tier system). See U.S. Patent No. 5,168,356, issued December 1, 1992 for a detailed description of a two tier system.

Specific formatting rules are required when encoding the Video Transport Block and are outlines below:

. For HD Types 1, 2, 3 the HD ID bit 1 is toggled on the first sequence header of a GOP, the start of a B-Frame, and the start of a P-Frame.

• A new cell is started at the beginning of a GOP (assuming GOP begins with an I-Frame), and the beginning of respective successive frames.

. A Basic Service transport block format is used on the first cell of a GOP, and the first cell of respective successive frames.

. A Redundant Data transport block format is used instead of the Basic Service format on the second packet of a frame if the frame spans multiple packets. The Redundant Data format is used again at an interval of about 4 to 8 times per frame.

. The redundant transmissions of the MPEG Sequence header are carried as Auxiliary Packets at an interval of 5 to 30 per second.

FIGURE 6 is a block diagram of MPEG encoding apparatus including a transport encoder according to the present invention. The transport encoder takes a MPEG data stream and attaches a protocol which:

CA 02387254 2002-06-20 •Allows a transport decoder to detect missing or invalid data;

•Offers redundant transmission of critical data; and •Indicates data reentry points to restart MPEG decoding.

Input to the transport encoder 12 is either directly from an MPEG encoder 10 (for a one tier transmission system) or from a MPEG priority processor 11 (for a two tier transmission system). In two tier systems, two separate data paths are used, 1 0 one for high-priority (HP) and one for low-priority (LP) data. The priority processor monitors rate buffer fullness, and generates priority-breakpoints which indicate where in the data stream data is split between the HP and LP data paths for each slice of MPEG data. The breakpoint data, along with the MPEG encoded data is

5 the input data to the transport encoder 12. MPEG codewords arrive at the input of the transport encoder tagged with data word length indicia and data type indicia (e.g., header data, motion vectors, discrete cosine transform coefficients etc.). A further input to the transport encoder is provided by a system clock 13.

0 This clock is incorporated to genlock the receiver and transmitter so decoder rate buffers do not overflow or underflow.

Output from the transport encoder 12 is sent to a service multiplexer and rate buffer 15 via a data scrambling mechanism 14. The multiplexer 15 interleaves data from

5 different service sources. The output from the multiplexer 15 is applied to the communications channel via a transmitter 16.

A transport decoder 20 receives cells from the communication channel receiver 17 via the service demultiplexer and rate buffer 18. The demultiplexer responsive Io the data in

0 the service type field of the transport cell header, separates data of different service types, and applies the separated data types to the appropriate processing circuitry. Video output data from the demultiplexer 18 is coupled to a descrambler 19 which performs a descrambling function which is inverse to the scrambling function 3 5 of element 14. Descrambled data is applied to a transport decoder

CA 02387254 2002-06-20

20, which separates header data from service data and applies the service data to a decoder 22.

Output from the transport decoder 20 provides both a system clock (21) for synchronizing the receiver to the transmitting service, and a data path to the MPEG decoder 22. Within the transport decoder, error checks are performed to determine whether a cell has been lost or includes errors. For example, the CC code is monitored to determine if respective transport cells occur in proper sequence. Only transport cells for 1 0 which no errors are detected are delivered to the MPEG decoder.

The transport decoder strips off the Entry-Point Data from the transport block, decodes this header, and presents data to the MPEG decoder in a suitable format. If there is a cell discontinuity, the video transport decoder is programmed to initiate a sequence

5 of resynchronizing tasks, as discussed below.

FIGURE 7 is a block diagram of a typical transport encoder. For one-tier transmission systems, components 145-170 are not included. For two-tier systems, all components in FIGURE 7 are used.

One-tier systems:

Encoded video codewords, and corresponding codeword identifying indicia related to codeword type, and codeword length arrive at the transport encoder from the video

5 encoder 100. Element 105, responsive to the identifying indicia captures and stores certain of the header information in a memory element 110. Data stored in element 110 will be included in the transmitted data a plurality of times to provide a degree of information redundancy. The data selected for

0 redundant transmission generally includes sequence header data,

GOP header data, and Picture header (PICT) data. At a minimum the data selected as redundant data is that data necessary to condition an MPEG decoder to begin decoding a data stream which has been entered at other than the beginning of a data sequence.

CA 02387254 2002-06-20 ο

Nominally a sequence may include a large number of GOP’s. Decoding of transmitted MPEG data requires use of the sequence header data. If the user tunes into the data after the occurrence of the sequence header, he may not be able to decode 5 the subsequent data. The transport protocol described herein repeatedly provides sequence and other needed header data for decoding shortly after entering the transmitted data stream no matter where it is entered.

Element 105 also extracts user data and applies this data to a memory 115. User data may be of many different types such as time stamps; whether the images are in color or not; the form of chrominance preprocessing; whether the original source material was film mode or video mode etc.. The MPEG protocol does not support inclusion of these types of information. However

5 inclusion of such information permits the receiver designer to incorporate special processing for particular signal types and thereby enhance the overall reproduction of images. The user data is included in auxiliary transport cells, when convenient.

Element 105 provides header identifying indicia to a

0 clock formatter 130. Clock formatter 130 includes a clock which is sampled on the occurrence of certain header data to generate time stamps associated with the corresponding header data. These time stamps are used in receiver apparatus to provide a measure of signal synchronization.

5 Data from elements 105, 110, 115 and 130 are coupled to a cell formatter 120. Nominally formatter 120 receives data from element 105, parses such data into cell length packages, develops the appropriate video service transport headers according to the protocol indicated in FIGURES 1 and 2,

0 concatenates the transport headers and the cell data, and couples the transport cells to a cell buffer 140. However, at the start of a sequence of data, and periodically during the transmission of data, the formatter is conditioned by the controller 125 to form and transmit other data. This other data includes auxiliary

5 information such as time stamps from the clock formatter 130 for

CA 02387254 2002-06-20

1 signal synchronization, and redundant header data stored in memory element 110.

Auxiliary data cells are generated as needed and included in the data stream when space is available. That is, 5 auxiliary data cells may be interleaved with video data in any of the I, P or B field data. On the other hand redundant data is for the most part interleaved only with I field video data. This is because decoding of video data must start with an I field. All other fields of MPEG data are predictive and depend from I fields.

0 The redundant data cells may be included at regularly spaced intervals or as data space is available but with at least a certain minimum inclusion of data to provide enough information to indicate decoding.

The cell formatter includes a continuity count in each

5 transport cell regardless of type. The continuity count, CC, is incremented by one unit in successive cells and repeats modulo N, where N may be a convenient binary number such as 16.

Two-tier systems:

0 Operation of the transport encoder in a two-tier system includes all functions described for a one-tier system, with the one-tier functions applied to the high priority or HP channel. In addition to the encoded video data provided at block 100, the priority processor (FIGURE 6), provides priority breakpoint data

5 which is stored in element 145. The breakpoint data is constant over a slice of MPEG data and indicates a threshold of what data (codewords) are placed on the HP channel and what data are placed on the LP channel. The priority breakpoint data is applied to a switch 165 which compares the breakpoint information

0 stored in element 145 with the current codeword identity provided by element 100, and supplies data codewords to either cell formatter 120 or cell formatter 160 for generation of either HP or LP transport cells respectively.

The element 105 provides header data to a functional 3 5 element 150 which develops entry-point definitions for both HP

CA 02387254 2002-06-20

2 and LP data, These entry point definitions are stored in entrypoint data memory 155. Cell formatters 120 and 160 create entry-point data for each cell generated. The entry-point is used by the decoder to resume decoding of the variable length data after a packet loss due to transmission errors. Completely formatted packets are sent from the LP cell formatter 160 to a cell buffer 170 for output.

Referring to the flowchart of FIGURE 8, the controller 125 initializes the system (850) by resetting the continuity I 0 counters (CC) and a cell count. It then checks (852) for an auxiliary data interrupt. These interrupts allow the user to interpose special information (if desired) for transmission. If an auxiliary interrupt has occurred, an auxiliary cell is created (854) and coupled to the rate buffer, and the auxiliary CC is

5 incremented. If there is currently no channel space available, the system is directed to access MPEG data (856).

The MPEG data is checked for the occurrence of a sequence header (858). If a sequence header is available, a basic cell type is created (860) using the sequence header data. The

0 basic cell continuity counter is incremented (862) and the cell is output (864) to the rate buffer. Following creation of the basic cell type with the sequence header data, N auxiliary type cells are created using the sequence header data. Here N is a small integer such as four. Each of the N auxiliary type cells are output, and the

5 auxiliary continuity count is incremented with the production of each cell.

Alternatively, if sequence header data is not available, a test is performed to determine the occurrence of group of picture (GOP) or picture (PICT) header data (870). 'If GOP/PICT

0 header data is available the cell count is reset (872) and a redundant type cell is created with the GOP/PICT header data (873). If space is available within the cell further MPEG data is included. The cell is output and the redundant cell continuity count and the cell count are incremented. Note at tests 858 and

5 870 if sequence headers or GOP or PICT headers are available,

CA 02387254 2002-06-20

3 they are stored in memory element 110 for use in forming redundant cells of the same data.

If at test 870 the current MPEG data is not GOP/PICT header data, the cell count is tested. If the cell count is not, for example, 2, 4 or 8 then a basic type cell is created with the current MPEG data. Alternatively, if the cell count is 2, 4 or 8 then a redundant type cell is created with the last occurring GOP/PICT header data.

0 Table I shows an exemplary sequence of transport cells.

CA 02387254 2002-06-20

<td></td><td> TABLE I</td>

<td> RHD class</td><td> Packet Contents</td>

<td> Basic</td><td> Sequence Header or GOP Header with MPEG dat</td>

<td> Aux</td><td> Redundant Sequence Header Transmission #1</td>

<td> Aux</td><td> Redundant Sequence Header Transmission #2</td>

<td> Aux</td><td> Redundant Sequence Header Transmission #N</td>

<td> Redundant</td><td> Redundant GOP/PICT Header + MPEG Data</td>