Signalling characteristics of multiview video coding (mvc) operation point

[0001] The present application for patent claims priority to US provisional applications № 61/232272, filed August 7, 2009, № 61/248738, filed October 5, 2009 and № 61/266861, filed December 4, 2009, the entire contents of which are incorporated herein by reference in their respective entirety.

TECHNICAL FIELD

[0002] This disclosure relates to the transportation of the encoded video.

BACKGROUND

[0003] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast, wireless broadcast systems, personal digital assistants (PDA), laptop or desktop computers, digital cameras, digital recording devices, players of digital Media, a device video games, video game consoles, cellular or satellite radio telephones, video conferencing device, and the like. Digital video devices implement video compression techniques, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, or ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC ), and extensions of these standards, to transmit and receive digital video more efficiently.

[0004] Video compression methods perform spatial prediction and / or temporal prediction to reduce or remove redundancy inherent in video sequences. For video block coding, a video frame or a slice (cut) can be divided into macroblocks. Each macroblock can be further partitioned. Macroblocks encoded by intra-coding (I) frame or slice (slice) are coded using spatial prediction with respect to neighboring macroblocks. Macroblocks are encoded in an external coding (P or B) frame or slice may use spatial prediction with respect to neighboring macroblocks in the same frame or slice or temporal prediction based on other reference frames.

[0005] Once the video data has been encoded, video data may be packetized multiplexer for transmission or storage. The MPEG-2 includes the "system", which sets the transport layer for a variety of video coding standards. Systems transport layer of MPEG-2 can be used in video encoders MPEG-2, or other video encoder consistent with different video coding standards. For example, MPEG-4 prescribes other encoding and decoding methods, rather than MPEG-2 encoders but implementing the method of MPEG-4, can still use the methods of the transport of MPEG-2.

[0006] In general, references to "MPEG-2 Systems," in this disclosure of the invention relate to a video transport layer prescribed by MPEG-2. The transport layer prescribed by MPEG-2, in this disclosure the invention is also referred to as "transport stream MPEG-2" or simply "transport stream". Similarly, the transport layer of MPEG-2 system also includes a program streams. Transport streams and program streams, in general, include a variety of formats for the delivery of such data, wherein the transport stream contains one or more "programs" that include both audio and video data, while the program streams include one program that includes both audio and video data.

[0007] Efforts have been made to develop new video coding standards-based H.264 / AVC. One such standard is the scalable video coding (SVC), which is a scalable extension to H.264 / AVC. Another standard is to encode multiple video images, which becomes an extension of a few images for H.264 / AVC. System Specification MPEG-2 describes how compressed media streams (audio and video) data may be multiplexed together with other data to form a single data stream suitable for transmission or storage. The latest specification of MPEG-2 systems listed in the "Information Technology-Generic Coding of Moving Pictures and Associated Audio: Systems, Recommendation H.222.0; International Organisation for Standardisation, ISO / IEC JTC1 / SC29 / WG11; Coding of Moving Pictures and Associated Audio "May 2006. The newly designed by MPEG standard transport MVC via MPEG-2 system and the latest version of this specification is" Study of ISO / IEC 13818-1: 2007 / FPDAM4 Transport of MVC ", MPEG doc. N10572, MPEG of ISO / IEC JTC1 / SC29 / WG11, Maui, Hawaii, USA, April 2009.

SUMMARY OF THE INVENTION

[0008] In general, this disclosure describes techniques for improving the encoding of multiple video images in MPEG-2 system (Expert Group on Film). In particular, the methods of this disclosure are directed to a data structure for the working point of the bit stream of MPEG-2 systems, where the structure data playback capability according to the receiving device, the ability to decode the receiving device, and in some examples, the bit rate for the operation point. The data structure descriptor may correspond to a working point, which is included in the bit stream of MPEG-2 systems.

[0009] To correctly decode and display the image data of the working point, the receiving device must satisfy the properties described reproducibility and the ability to decode those reported in the data structure. Bit streams MPEG-2 system can include a plurality of working points, which correspond to different program images. The use of different operating points for the program allows various client devices to perform adaptation. That is, the client devices with different capabilities of decoding and playback can retrieve the images from the same program for displaying three-dimensional or two-dimensional video data. Client devices may also be consistent with the server device to retrieve data and variable bit rate, to adapt for the transport of media having different characteristics bandwidth.

[0010] In one embodiment, the method comprises constructing by the starting device, the data structure corresponding to the working point encoding multiple video images (MVC) bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which must be met by the receiver, to the working point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC and wherein the structure It included as part of the data bit stream and outputting a bitstream comprising data structure.

[0011] In another example, the apparatus includes a multiplexer, which constructs a data structure corresponding to the operating point MVC bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which is necessary to satisfy the receiving device, for Use working point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC and which includes a data structure as part of the bitstream, and an output interface that outputs a bitstream comprising data structure.

[0012] In another example, the device includes means for constructing a data structure corresponding to the working point MVC bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which is necessary to satisfy the receiving device, to operating point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC, and wherein the data structure is included as part of the bitstream, and means to output a bitstream comprising data structure.

[0013] In another example, the computer readable medium comprises instructions that cause the processor of the source device to construct a data structure corresponding to the operating point MVC bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which is necessary satisfy receiving device, to the working point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC, and wherein the data structure includes as part of a bit stream and make the output interface to output a bitstream comprising data structure.

[0014] In another example, the method includes the steps of receiving, by the target device data structure corresponding to the operating point MVC bitstream standard MPEG-2 system (Picture Experts Group), wherein the data structure reported value reproducibility which describes the reproducibility which is necessary to satisfy the receiving device, to the working point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC, defines the way whether a video decoder of the target device to decode the image corresponding to the operating point of MVC, based on the ability to decode the communicated data structure defines the ability of whether the target device to reproduce the image corresponding to the operating point of MVC, based on the ability of reproducing the reported data structure, and send images corresponding MVC operation point, the video decoder of the target device, when it is determined that the target device video decoder can decode and reproduce the image corresponding to the operating point of MVC.

[0015] In another example, the device comprises an input interface adapted to receive a data structure corresponding to the working point MVC bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which is necessary to satisfy the receiving device, for MVC operation point, meaning the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, the bit rate that describes the operating point of MVC, a video decoder operable to decode video data; and a demultiplexer operable to determine capable whether a video decoder decoding the image corresponding to the operating point of MVC, based on the ability to decode the communicated data structure determining capable whether a device to reproduce the image corresponding to the operating point of MVC, based on the ability of reproducing the reported structure data, and image sending corresponding operating point MVC, a video decoder, when it is determined that the decoder is able to decode and display images corresponding to the operating point of MVC.

[0016] In another example, the apparatus includes means for receiving a data structure corresponding to the working point MVC bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which is necessary to satisfy the receiving device, to operating point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC, means for determining of whether, decoder unit decode an image, corresponding operating point MVC, based on the ability to decode the communicated data structure, means for determining, can the device to reproduce the image corresponding to the operating point of MVC, based on the ability of reproducing the reported data structure, and means for sending the images corresponding to operating point MVC, a video decoder device, when it is determined that the decoder device is able to decode and display images corresponding to the operating point of MVC.

[0017] In another example, the computer readable medium comprises instructions that cause the processor of the target device to accept data structure corresponding to the operating point MVC bitstream standard MPEG-2 system, wherein the data structure reported value reproducibility, which describes the reproducibility which is necessary satisfy receiving device, to the working point MVC, of ​​the ability of decoding, which describes the ability of decoding to satisfy the receiving device for the operating point of MVC, and the value of the bit rate, which describes a bit rate operation point MVC, determine capable whether decoder target device The decoded image corresponding to the operating point of MVC, based on the ability to decode the communicated data structure defines the ability of whether the target device to reproduce the image corresponding to the operating point of MVC, based on the ability of reproducing the reported data structure, and send the images corresponding to operating point MVC, a video decoder target device when it is determined that the target device video decoder can decode and reproduce the image corresponding to the operating point of MVC.

[0018] The details of one or more examples are set out in the accompanying drawings and the description below. Other features, objects and advantages will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG. 1 is a block diagram illustrating an exemplary system in which the original audio / video (A / V) device transports the audio and video data to the target A / V device.

[0020] FIG. 2 is a block diagram illustrating an exemplary arrangement of components of the multiplexer in accordance with this disclosure.

[0021] FIG. 3 is a block diagram illustrating an exemplary set of tables of special information about the programs in accordance with this disclosure.

[0022] FIG. 4-6 is a conceptual diagram illustrating various examples of data sets that may be incorporated in a handle operating point.

[0023] Fig. 7 is a conceptual diagram illustrating an exemplary sample predictions MVC.

[0024] FIG. 8 shows the strip chart illustrating an exemplary method for using data structures which informs the operating point characteristics.

DETAILED DESCRIPTION

[0025] The methods of this disclosure, in general, aimed at improving the encoding of multiple video images (MVC) in the systems of the MPEG-2 (Moving Picture Experts Group), that is, systems that are coordinated with the MPEG-2 based on the information of the transport layer. MPEG-4, for example, provides standards for video coding, but generally assumes encoders agreed with the MPEG-4, the system will use the transport layer of MPEG-2. Consequently, methods of this disclosure are applicable to video encoders that are consistent with MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 or another video coding standard that uses traffic flows and / or program streams (also referred to as a "program stream") MPEG-2.

[0026] In particular, the methods of this disclosure may be modified syntax elements in the transport layer for transport streams and program streams MPEG-2. For example, the methods of this disclosure include a handle that is transmitted in the transport stream to describe the characteristics of the working point. Server device, for example, can provide different operating points in the bit stream of the transport layer of MPEG-2, each of which corresponds to a corresponding subset of specific video image encoding multiple video images. That is, the operating point generally corresponds to a subset of images in the bitstream. In some examples, the operating point of each image includes image data of the same frame rate.

[0027] The target device may use descriptors operating points included in the bit streams, to select one of the operating points to be decoded, and eventually presentation (e.g., display) by the user. Instead of transmitting the reception data for all images in the video decoder, the target device may only send the video decoder of the selected operating point. Thus, the target device can reject the data for images that can not be decoded. The target device may select an operating point on the basis of the highest quality, supported by one of the operating points for the bit stream.

[0028] The server device may send a plurality of substreams of bits (each of which may correspond to the operating point) in one transport stream or program stream. Despite the fact that different sections of this disclosure may reference individually on the "transport stream" or "program stream" it will be understood that the methods of this disclosure are generally applicable to any or both of the transport MPEG-2 stream and program streams. In general, this disclosure describes the use of descriptors in an exemplary data structures to perform the techniques of this disclosure. Descriptors are used to extend the functionality of the flow. Handles of this disclosure may be used as a transport stream and program stream for implementing the methods of this disclosure. Despite the fact that this disclosure will focus primarily on the handle as the exemplary data structure that can be used to report the values ​​playback capability for the working point of the ability of decoding for the operating point and the value of the bit rate for the working point, it should be understood that perform these methods may be used other data structures.

[0029] In accordance with the methods of this disclosure, the source device 20 may be designed operating point descriptor which describes the characteristics of the operating point. Features can include, for example, what kind of images are included in the operating point and the frame rate of images to the operating point. Handle operating point may indicate playback capability that should be supported by the video decoder to receive and decode the working point, the ability to decode the video decoder which must be maintained in order to receive and decode the operating point and the bit rate for the operation point.

[0030] The methods of this disclosure may, in general, represent each operating point, as if the operating point has its own program, communicated program map table in a transport stream or a program stream map in a program stream. Alternatively, when a program includes a plurality of operating points, the methods of this disclosure provide information about how to re-assemble the working point descriptors operating points. Descriptors operating points may additionally communicate according to the operating points, which can save bits.

[0031] FIG. 1 is a block diagram illustrating an example system 10 in which the original audio / video (A / V) device 20 conveys the audio and video data to the target A / V device 40. The system 10 of FIG. 1 may correspond to the system video teleconference, a server / client system shirokoveschayuschey / receiving system or any other system in which the video data are sent from the source device, such as the original A / V device 20 to the target device, such as the target A / V device 40. In some examples, the original A / V device 20 and the target A / V device 40 may perform two-way communication. That is, the original A / V device 20 and the target A / V device 40 may be able to both encode and decode (and transmit and receive) video and audio data. In some examples, an audio encoder 26 may include a voice encoder, also called a vocoder.

[0032] Initial A / V device 20 in the example of FIG. 1 comprises an audio source 22 and the video source 24. The audio source 22 may comprise, for example, a microphone, that produces electrical signals representative of the captured audio data to be decoded audio encoder 26. Alternatively, audio source 22 may comprise a storage medium storing a previously recorded audio data, the audio data generator, such as computerized synthesizer, or any other audio source. Video source 24 may comprise a video camera that produces video data to be decoded video encoder 28, a storage medium encoded previously recorded video data, video data generating unit, or any other video source.

[0033] The raw audio and video data may comprise analog or digital data. Analog data can be digitized before encoding an audio encoder 26 and / or video encoder 28. The audio source 22 may also receive audio from the speaker's party, while speaking as a participant said, and the video source 24 may simultaneously receive video speaking participant. In other examples, the audio source 22 may comprise a computer readable medium having stored audio data and the video source 24 may comprise a computer readable medium having stored video data. Thus, as described in this disclosure, the inventive methods can be applied to live, streaming audio and video data in real-time and archived, pre-recorded audio and video data.

[0034] The audio frames that correspond to video frames, in general, are the audio frames containing audio data that have been captured by the audio source 22, along with video data captured by video source 24 which is contained within a video frame. For example, while talking party usually makes the audio through conversation, audio source 22 captures audio and video source 24 captures video speaker at the same time, ie during that time, as the audio source 22 captures audio. Consequently, the audio frame may temporarily satisfy one or more of the specific video frame. Consequently, audio frame corresponding to the video frame, in general, corresponds to the situation in which audio data and video data have been captured at the same time, and for which the audio frame and the video frame comprise, respectively, the audio and video data that were captured in the same time .

[0035] In some examples, an audio encoder 26 may encode the time stamp for each coded audio frame which is the time that has been recorded audio data to encoded audio frame, and similarly, the video encoder 28 may encode the time stamp of each encoded video frame, which is the time that was saved video data for the encoded video frame. In such instances audio frame corresponding to the video frame may comprise audio frame containing a timestamp and a video frame having the same time stamp. Initial A / V device 20 may include an internal clock from which the audio encoder 26 and / or encoder 28 may generate timestamps or which audio source 22 and the video source 24 may be used to associate audio and video data, respectively, labeled with time.

[0036] In some embodiments, the audio source 22 may send data to the audio encoder 26, corresponding to the time at which the audio was recorded, and the video source 24 can send data to the video encoder 28, corresponding to the time at which the video was recorded. In some examples, an audio encoder 26 may encode the sequence identifier in the coded audio data to indicate the relative temporal order of the encoded audio data, but without the need to indicate the absolute time at which were recorded audio data, and likewise the video encoder 28 can also use the sequence identifiers for indicating the relative temporal order of the coded video. Similarly, in some instances, the sequence identity can be mapped to or otherwise correlated with the timestamp.

[0037] The methods of this disclosure are generally directed to transportation of encoded multimedia (e.g. audio and video) data, and receiving and subsequent interpretation and decoding the multimedia data transported. The methods of this disclosure are particularly useful for transporting data encoding multiple video images, that is video data having a plurality of images. As shown in the example of FIG. 1, video source 24 may provide a plurality of images of the scene in the video encoder 28. MVC can be useful for generating three-dimensional image data for three-dimensional display, such as stereoscopic and autostereoscopic three-dimensional display.

[0038] The original A / V device 20 may provide a "service" to the target A / V device 40. In general, the service corresponds to a subset of the available image data MVC. For example, data can be accessed MVC eight images, numbered from zero to seven. One service can match the stereo video having two images, while other service can match the four images and one service can meet all eight images. In general, the service corresponds to any combination (ie any subset) of available images. The service can also match the combination of available images as well as audio. Operating point may correspond to a service, so that the original A / V device 20 may further provide a handle to the operating point for each service provided by the source A / V device 20.

[0039] Initial A / V device 20 in accordance with the methods of this disclosure are able to provide services which correspond to a subset of images. In general, the image is represented by the image identifier, also referred to as the "view_id". In general, the image identifier containing the syntax elements that may be used to identify images. MVC encoder provides view_id image when the image is encoded. view_id can be used to predict the MVC decoder changes between images, or other blocks for other purposes, e.g., for playback.

[0040] Prediction of changes between images is MVC encoding method for video frame with reference to one or more of the frames for a total time location as different coded frame image. FIG. 7, which is discussed in more detail below, provides an exemplary encoding scheme for predicting changes between images. In general, the MVC video encoded frame may be encoded with a spatial prediction, temporal and / or with reference to other image frames for a total time location. Therefore, the reference image on the basis of which predict other pictures, in general, to decode the image for which an image model act as a sample so that the decoded image can be used to decode the sample at exemplary images. The order of decoding not necessarily correspond to the order view_id. Consequently, the order of decoding images is described using the order of indexes of images. Indexes order images are indexes that indicate the decoding order of image components in the access unit.

[0041] Each individual data stream (or audio, or video) is called the elementary stream. An elementary stream is a single digitally encoded (possibly compressed) component of the program. For example, encoded video or audio portion of the program may be an elementary stream. An elementary stream can be converted to a packetized elementary stream (PES) to the multiplexing program stream or transport stream. Within one and the same program flow ID is used to distinguish the PES-packets belonging to an elementary stream from the other. The basic unit of data is an elementary stream PES packet (PES). Thus, each image corresponding to the respective MVC video elementary streams. Similarly, the audio data corresponding to one or more of the relevant elementary streams.

[0042] A video sequence encoded with the MVC can be divided into several sub-streams of bits, each of which is an elementary stream. Each sub-stream of bits may be identified by a subset MVC view_id. Based on the concept of each subset MVC view_id set bits MVC video sub-stream. Subflow bits MVC NAL units comprises video images of said subset in MVC view_id. Program stream generally contains only NAL units, NAL units that are elementary streams. It is also designed so that any two elementary stream may not contain an identical image.

[0043] In the example of FIG. 1, the multiplexer 30 receives the elementary streams containing the video data from the video encoder 28 and the elementary streams containing audio data from the audio encoder 26. In some examples, video encoder 28 and audio encoder 26, each may include paketizatory to form PES packets of the encoded data. In other examples, video encoder 28 and audio encoder 26, each can interact with relevant paketizatorami to form PES packets of the encoded data. In still other examples, the multiplexer 30 may include paketizatory PES packets to generate encoded video and audio data.

[0044] "program" as used in this disclosure, the invention may comprise a combination of audio and video data, such as audio elementary stream and a subset of the available images delivered service source A / V device 20. Each PES packet includes a stream_id, which identifies the elementary stream , PES packet belongs. The multiplexer 30 is responsible for assembling the elementary streams into multiple program streams or transport streams. Program Stream and Transport Stream multiplexes are two alternative aimed at different applications.

[0045] In general, the program stream includes audio data for the program, while the transport stream may include data for one or more programs. Multiplexer 30 may encode either or both of the program stream or transport stream based on the service provided, the environment in which will be streamed, the number of programs for sending, or other reasons. For example, when the video data must be encoded in an information storage medium, the multiplexer 30 may be more likely to form the program stream at the time when the video data be streamed over a network, broadcast or sent as part of a videophone, a multiplexer 30 may be more likely to use the transport stream.

[0046] The multiplexer 30 may be inclined toward the use of the program stream for storage and display of a single program of digital storage service. Program Stream is designed for use in error-free environments or environments that are less prone to errors occurring because the program streams are sensitive to errors. Program flow simply contains elementary streams belonging to it, and generally contains variable length packets. The program stream PES-packets that are received from the provided elementary streams are organized into "packaging". The package contains a header package, an optional system header and any number of PES-packet taken from any given elementary streams in any order. System header contains a summary of the characteristics of the program stream, such as a maximum data transfer rate, the number provided by the elementary video and audio streams, additional information about time alignment, or other information. The decoder can use the information contained in the system header to determine whether or not the decoder is able to decode the program stream.

[0047] The multiplexer 30 may use a transport stream for simultaneous delivery of multiple programs through potentially error-prone channels. The transport stream is a multiplex, conceived for multi-program applications, such as broadcast, so the only transport stream can contain a lot of independent programs. The transport stream may contain a number of transport packets, wherein the length of each transport packet is 188 bytes. The use of short packets with constant length leads to the fact that traffic is less sensitive to errors than the program stream. Moreover, each transport packet length 188 bytes can be given extra protection against errors due to packet processing by a standard process of error protection, such as Reed-Solomon coding. Enhanced robustness traffic means that it has a better chance of survival in error-prone channels, which are, for example, in a broadcast environment.

[0048] It may seem that the traffic flow is better than the program stream because of its increased fault tolerance and the ability to carry a lot of concurrent programs. However, the transport stream is a multiplex more complex than the program stream, and therefore is more difficult to create and more difficult for demultiplexing than the program stream. The first byte of the transport packet may be the sync byte, having a value of 0 × 47 (hexadecimal 47, a binary '01000111', 71 decimal). A single transport stream can carry a lot of different programs, each program contains a lot of packetized elementary streams. The multiplexer 30 may use field trinadtsatibitnoe packet identifier (PID), to distinguish the transport packets containing the data for one elementary stream from those that carry data from other elementary streams. It is the responsibility of the multiplexer is to ensure that each elementary stream is assigned a unique value PID. The last byte of the transport packet can be continuity counter field. Multiplexer 30 increases the value of the counter continuity between successive transport packets belonging to the same elementary stream. This enables the decoder or another unit of the target device, such as a target A / V device 40 to detect the loss or gain of the transport packets and, hopefully, to hide the errors that might otherwise arise from such an event.

[0049] The multiplexer 30 receives PES packets of elementary streams of the program from the audio encoder 26 and video encoder 28 and generates the corresponding NAL units (NAL) of PES packets. In the example, H.264 / AVC (Advanced Video Coding) encoded video segments organized into blocks of NAL, which provide a "friendly networks", addressing the use of video representations, such as video telephony, storage, broadcast or streaming. NAL units may be classified into levels VCL NAL units (VCL) NAL units and not-VCL. VCL units comprise a compression mechanism of the base and may contain levels of blocks, macroblocks and / or bands. Other NAL units are NAL units not-VCL.

[0050] The multiplexer 30 may generate NAL units, comprising a header which identifies the program to which NAL belongs, as well as useful information, for example audio data, video data go, which describe the transport or program stream, which correspond NAL units. For example, in H.264 / AVC, a NAL unit includes a 1-byte header and useful data of variable size. In one example, the header of the NAL unit contains an element priority_id, element temporal_id, element anchor_pic_flag, element view_id, non_idr_flag element and an element inter_view_flag. In traditional MVC NAL unit is held, by a predetermined H.264, except the prefix NAL unit and the NAL units of slice coded using MVC, which contain a 4-byte NAL unit header MVC and useful information NAL unit.

[0051] The element priority_id NAL header can be used to process a simple one-pass adaptation of the bit stream. Temporal_id element can be used to indicate the level of the corresponding temporary block NAL, where different time levels correspond to different speeds.

[0052] The element anchor_pic_flag can indicate whether the picture is a reference picture or a reference picture. The reference picture and all pictures that follow it in output order (i.e. display order) can be correctly decoded without decoding previous pictures in decoding order (i.e. the order of the bitstream), and thus can be used as a random access point . The reference images and reference images may have different dependencies, both are reported in the sequence parameter set. Other flags should be discussed and used in the following sections of this chapter. This reference image may also be referred to as an access point open GOP (Group of pictures), while the access point is the closed GOP is also maintained when the element non_idr_flag zero. Non_idr_flag element indicates whether the picture is a picture instant updates decoder (IDR) picture or IDR (V-IDR) picture. In general, the IDR picture and all the subsequent pictures in output order, or the order of the bit stream can be correctly decoded without decoding previous picture in decoding order and the display order.

[0053] The element may comprise a view_id syntax information that can be used to identify images that can be used for interactive data within the MVC decoder, for example for predicting the change between images, and is a decoder, for example for playback. Inter_view_flag element may indicate whether the NAL unit corresponding to other images to predict changes between images. To transfer information 4-byte header of the NAL unit for the base image, which can be compatible with AVC, MVC is set in the prefix NAL unit. In the context of MVC, an access unit of the base image includes image blocks VCL NAL current time, as well as its prefix NAL unit, which contains only the NAL unit header. H.264 / AVC decoder can ignore the prefix unit NAL.

[0054] The NAL unit, comprising video data in its payload may include various granularity video. For example, the NAL unit may comprise a video macroblock, a plurality of macroblocks, a slice of video data or an entire frame of video data. The multiplexer 30 may receive the encoded video data from the video encoder 28 in the form of PES packets the elementary streams. The multiplexer 30 can communicate with each elementary stream corresponding to the program by relating stream_id with the relevant programs, for example, in a database or other data structures, such as a program map table (PMT) and a program stream map (PSM).

[0055] The multiplexer 30 may also collect the access units of the plurality of NAL units. In general, an access unit may comprise one or more NAL units to represent a frame of video as well as audio data corresponding to the frame when the audio data are available. Access unit generally includes all NAL units for a time-points, for example, all audio and video data for one point in time. For example, if each image has a frame rate of 20 fps (frame / sec), each instant of time may correspond to a time interval of 0.05 seconds. During this time interval, characteristic for all image frames of the same access unit (the same point in time) may be played simultaneously. In the example corresponding to H.264 / AVC, an access unit may comprise a coded image at one time, which can be represented as a picture, the encoded first. Therefore, an access unit may comprise all audio and video frames of the overall time points, for example, all images corresponding to the time X. This disclosure also relates to a coded picture of a particular image, as a "component ratio". That is, the image component may contain coded picture (or frame) for a particular image at a particular time. Consequently, the access unit can be defined as comprising all the components of the image overall time point. The decoding order of access units need not be the same as the display order or display.

[0056] The multiplexer 30 may also incorporate data relating to the program, in the NAL unit. For example, the multiplexer 30 may create flow NAL, containing the program map table (PMT) or program stream map (PSM). Generally, PMT is used to describe a transport stream, while PSM is used to describe the program stream. As described in more detail below following the example of FIG. 2, the multiplexer 30 may include or interact with a data storage unit that associates the elementary streams received from audio encoder 26 and video encoder 28 with the programs and, therefore, with the respective traffic flows and / or the program stream.

[0057] As in the case of most video coding standards, H.264 / AVC specifies the syntax and semantics of the decoding process for error-free bit streams, each of which is consistent with a specific profile or level. H.264 / AVC encoder does not specify, but the encoder are aiming to ensure that the generated bit streams are compatible with the standard decoder. In the context of video coding standard "profile" corresponds to a subset of algorithms, tools, and features or limitations that apply to them. How to set the standard H.264, for example, the "profile" is a subset of the entire bitstream syntax that is specified standard H.264. "Level" corresponds to the decoder resource consumption limitations, such as, for example, memory and computing resources decoder which relate to image resolution, bit rate and a speed of processing the macroblock (MB).

[0058] The H.264 standard shows that within the bounds imposed by the syntax of a given profile, it is still possible to require a big change in the performance of encoders and decoders depending upon the values ​​taken syntax elements in the bitstream, such as a specified amount of decoded images. The H.264 standard additionally shows that in many applications such as impractical and not cost-effective to implement a decoder capable of dealing with all hypothetical uses of the syntax within a particular profile. Therefore, the H.264 standard defines a "level" as a specified set of restrictions imposed on the values ​​of syntax elements in the bitstream. These limitations can be simple restrictions on the values. Alternatively, these restrictions can take the form of restrictions on the combinations of arithmetic values ​​(e.g., image width multiplied by the height of the image multiplied by the number of decoded pictures per second). The H.264 standard further provides that individual implementations may support a different level for each supported profile.

[0059] The decoder, agreeing with the profile, generally supports all the features defined in the profile. For example, as a feature coding encoding B-pictures is not supported in the base profile H.264 / AVC, but other profiles supported in H.264 / AVC. A decoder according the level should be able to decode any bit stream, which does not require resources beyond the raised level. Assignments profiles and levels can be useful for interpretability. For example, during a video transmission, a pair of reference profile and level can be negotiated and established for the session transfer. More specifically, in H.264 / AVC standard may specify, for example, restrictions on the number of macroblocks to be processed, the size of the decoded picture buffer (DPB), the size of the coded picture buffer (CPB), the range of vertical motion vectors, the maximum number of motion vectors the next two MB, and whether B-blocks have sub-macro division less than 8x8 pixels. Thus, the decoder can determine whether the decoder is able to correctly decode the bit stream.

[0060] The parameter set generally comprises a sequence level header information in a sequence parameter set (SPS) and rarely change picture level header information in picture parameter sets (PPS). With a set of parameters, this is rarely variable data does not need to be repeated for each sequence or images; consequently, it can be effectively improved coding. Furthermore, the use of parameter sets may enable outband transmission header information, avoiding the need for redundant transmission to achieve fault tolerance. When NAL units-of-band transmission of a set of parameters are transmitted on a different channel than the other blocks NAL.

[0061] The standard MPEG-2 system allows you to expand the system by means of "tags." As PMT, and PSM include descriptors cycles, in which can be inserted into one or more descriptors. In general, the descriptor may comprise a data structure that can be used to extend the task programs and / or program elements. This disclosure describes the descriptors operating point to perform the methods of this disclosure. In general, the operating point of the handle of the disclosure enhances the traditional MVC extension descriptor by describing playback capability, the ability to decode and bit rate for the operation point. The target device, such as the target A / V device 40 may use descriptors operating points for each operating point, to select one of the operating points of the bit stream for decoding.

[0062] Each PMT or PSM may include operating point descriptor which describes the characteristics of the operating point. For example, source device 20 may provide a handle for the operating point of the values ​​of reproducibility that describes playback capability for client device 40. In order to correct the client device 40 of the playback (e.g., display) the working point of the video data, the client device 40 must satisfy the abilities playback playback capability reported value. The value can describe the reproduction capability, for example, the number of images to be displayed (e.g. number of images targeted for playback) and / or the frame rate for video images. Thus, the client device 40 may determine that the ability to playback satisfied when vydeovyhod 44 client device 40 is capable of displaying a number of images of the operating point with the frame rate, the operating point of said handle.

[0063] In the examples in which the source device 20 transmits MVC bitstream, using protocols multicast or broadcast, source device 20 may be packaged whole MVC bitstream into transport streams that may be taken by client devices having different reproduction capabilities. For example, some three-dimensional applications can have a different number of images (for example two images, the four images or six images of eight images) and various devices may be configured to use any number between one and four pairs of images. Thus, each client device can determine which operating point based on the use of supported number of images that can be displayed by the client device. For example, the client device 40 can determine which of the operating points used by determining the number of images that can be displayed video out 44, and the frame rate at which vydeovyhod 44 is capable of displaying video data and to determine which of the operating points must be used on the basis of capacity playback video output 44.

[0064] In the examples in which the source device transmits MVC bitstream, using a unicast protocol, the client device 40 may establish a session corresponding to the program with the allowable number of images, by checking whether reproduction capacity specified in respective operating point descriptors. Similarly, in the examples in which the MVC bitstream coded in a computer readable medium for local playback, the client device 40 may select an appropriate program by testing the ability of reproduction of said descriptors operating points PMT or PSM.

[0065] Source device 20 may also provide the ability to decode the value of the operating point in the deployment descriptor. The number of images to be decoded, may optionally be the same as the number of images to be displayed. Therefore, the operating point can handle separately report the number of images to be displayed and the number of images to be decoded to the operating point. Moreover, the operating point can handle specifically identify the image corresponding to the operating point. Certain client devices may prefer specific images for various purposes, for example, based on viewing angle. Therefore, the client device 40 may be adapted to select a working point on the basis of what images are available at the operating point.

[0066] In some embodiments, the decoding capability, reported at the operating point, may additionally or alternatively indicate the profile and level, which corresponds to the operating point. In the examples in which the source device 20 transmits a bit stream, using protocols multicast or broadcast, various client devices with different capabilities of decoding may take bitstream. For example, some decoders can only be able to decode the two images with 30 frames / sec, while some may be capable of decoding four images of 60 frames / sec. In the examples in which the source device 20 transmits a bit stream, using a unicast protocol, the client device 40 can establish the appropriate session (for a special three-dimensional software) after checking the decoding ability, said descriptor into PMT. Similarly, for local playback client device 40 can select a suitable program by testing the ability of decoding of said descriptors of operating points PMT or PSM.

[0067] Source device 20 may further report information about the bit rate at the operating point descriptor. Information on the bit rate can describe any average bit rate, maximum bit rate of either or both the speed of the operating point. For example, when the source device 20 transmits a bit stream, using a unicast protocol is used for transmission channel may be limited in terms of bandwidth. Therefore, the client device 40 may select operating point having an acceptable maximum or average bit rate for the communication channel.

[0068] In some embodiments, source device 20 may further indicate the frame rate of the point descriptor operating point. Certain operating point of the image may be a frame rate that does not match the frame rate of the operating point. Thus, the client device 40 may determine the frame rate of the working point and the frame rate of the image to facilitate the reassembly of the decoded video data for video display purposes. In various embodiments, when the frame rate of the two working points do not coincide, the client device 40 can be disposed of pictures from the operating point of the image having a higher frame rate, or the interpolation frame image of the operating point having a lower frame rate.

[0069] Typically, the elementary stream includes a flag "no_sei_nal_unit_present" and "no_prefix_nal_unit_present", which describe, respectively, does an elementary stream SEI message and blocks prefix NAL. This disclosure proposes that client devices such as client device 40, concluded whether there is an SEI message and the prefix NAL units in the operating point, rather than to explicitly report these values ​​to the operating point. To determine whether there is an SEI message at the operating point, the client device 40 may determine whether one is equal to the maximum value of the values ​​no_sei_nal_unit_present elementary streams for the operating point. Similarly, to determine whether there is a prefix NAL units at the operating point, the client device 40 may determine whether one is equal to the maximum value of the values ​​no_prefix_nal_unit_present elementary streams for the operating point.

[0070] The above examples focus on the descriptors of operating points contained for each operating point of the bit stream MVC. Alternatively, source device 20 may provide descriptions MVC extension, which report similar findings. For example, source device 20 may communicate more than one descriptor from the MVC extension MVC video substreams of bits which corresponds to an elementary stream. Source device 20 may indicate the descriptor for MVC extension bit substream framerate view_id subset of images to be displayed and the number of images to be decoded. Source device 20 may further communicate the correspondence between the MVC extension descriptors and the corresponding operating point.

[0071] The video compression standards such as H.261, H.262, H.263, MPEG-1, MPEG-2 and H.264 / MPEG-4 part 10, using temporal prediction motion compensation to reduce the temporal redundancy . The encoder uses the motion compensation prediction based on the previously coded picture (also referred to herein as frames) to predict the image encoded at present, according to the motion vector. There are three basic types of pictures in conventional video coding. These are intra-coded pictures ("I-picture" or "I-frames"), the image prediction ("P-picture" or "P-frames") and bi-directional predictive pictures ("B-picture" or "B- frames "). P-pictures use only the reference image before the current picture in the temporal order. The B-picture, each block B-picture can be predicted from one or two reference pictures. These reference pictures may be located before and after the current picture in time order.

[0072] In accordance with a coding standard H.264 as Example B-pictures use two lists previously encoded reference picture list 0 and list 1. Each of these two lists contains the encoded past and future pictures in time order. Blocks in the B-picture can be predicted by one of several methods: motion compensated prediction based on the reference picture list 0, the motion compensation prediction based on the reference picture list 1, or motion compensation prediction based on a combination of reference pictures as the list 0 and List 1. To obtain a combination of reference pictures as the list 0 and list 1, two anchor zones motion compensation is obtained from the reference picture list 0 and list 1 respectively. Their combination will be used to predict the current block.

[0073] Standard ITU-T H.264 supports intra prediction in various block sizes, such as 16 by 16, 8 by 8, or 4 by 4 luminance components, and 8x8 for chroma components, as well as the inter-prediction in various block sizes, such as 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8 and 4 × 4 for the luminance components and appropriately scaled sizes for chroma components. This disclosure of the invention "×" and "on" may be used interchangeably to refer to the pixel dimensions of the block in terms of vertical and horizontal dimensions, for example 16 × 16 pixels or 16 by 16. In general, a 16 × 16 block will have 16 pixels in the vertical direction (y = 16) and 16 pixels in the horizontal direction (x = 16). Similarly, block N × N, generally has N pixels vertically and N pixels in the horizontal direction, where N is a nonnegative integer value. The pixels in a block may be arranged in rows and columns.

[0074] Block sizes that are less than 16 by 16 may be referred to as a division of the macroblock 16 at 16. The video blocks may comprise blocks of pixel data in pixels or blocks of transform coefficients in the transform domain, e.g., following application of the transformation such as discrete cosine transformation (DCT), integer transform, wavelet transform or a conceptually similar transformation in the residual video block data representing the difference between the coded video blocks of pixels and the predicted video blocks. In some cases, video block may comprise blocks of quantized transform coefficients in the transform domain.

[0075] Smaller video blocks can provide better resolution, and may be used for locations of video frames that include high levels of detail. In general, macroblocks and the various separation, sometimes referred to as sub-blocks may be considered video blocks. Moreover, the slice can be considered as the set of video blocks, such as macroblocks and / or subblocks. Each slice may be an independently decodable unit of a video frame. Alternatively, frames themselves may be decodable units, or other portions of the frame can be defined as decodable units. The term "coded unit" and "encoding unit" may refer to any independently decodable unit of a video frame such as an entire frame, slice frame group of pictures (GOP) also known as a sequence, or another independently decodable unit defined according to the coding method is applied.

[0076] The term refers to a macroblock data structure for coding pictures and / or video data according to the two-dimensional array of pixels which comprises 16x16 pixels. Each pixel contains a chrominance component and a luminance component. Therefore, the macroblock may define four luma blocks, each comprising a two-dimensional array of 8x8 pixels and two chroma block, wherein each comprises a two-dimensional array of 16x16 pixels, and a header containing syntactic information such as sample coded block (SVR), coding mode (e.g. inner (I) or external (P or B) coding modes), the size separation for separating coded by intra-coded block (e.g., 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8 or 4 × 4), or one or more motion vectors coded macroblocks outwardly.

[0077] Video encoder 28, video decoder 48, audio encoder 26, the audio decoder 46, multiplexer 30 and demultiplexer 38 may each be implemented in any of a variety of suitable coding and decoding, under appropriate conditions, such as one or more microprocessors, processors, digital processing signals (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), discrete logic, software, hardware, firmware or any combination thereof. Each video encoder 28 and video decoder 48 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (codec). Similarly, each audio encoder 26 and audio decoder 46 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined codec. The apparatus includes a video encoder 28, video decoder 48, audio encoder 26, the audio decoder 46, multiplexer 30 and / or demultiplexer 38 can comprise an integrated circuit, a microprocessor and / or wireless communication device, such as a cellular phone.

[0078] The methods of this disclosure may offer certain advantages in addition to the traditional methods for the sub stream bits MVC, which do not provide a message characteristics of the operating points. Each substream bits may include one or more images of the corresponding bitstream. And some cases, the operating point may correspond to images of different bitstreams. The methods of this disclosure provide operating point descriptor that identifies the image corresponding to the operating point.

[0079] When the multiplexer 30 assembled NAL unit, and / or access unit from the received data, the multiplexer 30 passes the output interface unit 32 for output. The output interface 32 may comprise, for example, a transmitter, a transceiver, an apparatus for recording data on a computer readable medium such as, for example, an optical drive, a drive for magnetic media (e.g., floppy disk drives), universal serial bus, a network interface or other output interface. The output interface 32 outputs the NAL unit or access unit, the computer readable medium 34 such as, for example, signal transmission, magnetic media, optical media, memory, flash drive or other computer readable media.

[0080] Ultimately, the input interface 36 retrieves the data from the computer readable medium 34. The input interface 36 may comprise, for example, an optical disk drive, a magnetic media drive, USB port, receiver, transceiver or other interface of computer readable media. The input interface 36 may provide the NAL unit or block access to the demultiplexer 38. The demultiplexer 38 can demultiplex a transport stream or program stream into component PES streams, depaketirovat PES streams for fetching data and send the coded data or to the audio decoder 46 or the video decoder 48 depending on whether whether the coded data part of video or audio, for example, as indicated PES packet header flow. The audio decoder 46 decodes the audio data and sends the decoded audio data to the audio output 42, while video decoder 48 decodes the coded video data and sends the decoded video data, which may include a plurality of flow images at video output 44. The video output 44 can include a display, which uses a plurality of images of the scene such stereoscopic or autostereoscopic display that represents each image scene simultaneously.

[0081] In particular, the demultiplexer 38 can select the operating point of the received bit stream. For example, the demultiplexer 38 can compare the characteristics of the operating points of the bitstream to select the proper operating point, to be used, the target A / V device 40. In general, the demultiplexer 38 may attempt to select one of the operating points, which will provide the highest quality viewing sensation for the user, who may be decoded by the video decoder 48. For example, the demultiplexer 38 may compare the ability of play and ability to decode video decoder 48 with the proposed powers of reproduction and decoding signaled descriptors of operating points of the bitstream. From the operating point which determines both the demultiplexer 38 can be correctly decoded by the video decoder 48, the demultiplexer 38 can select an operating point that provides the highest quality video, for example the highest frame rate and / or bit rate. In other examples, the demultiplexer 38 can select one of the supported operating points based on other considerations such as power consumption.

[0082] FIG. 2 is a block diagram illustrating an exemplary arrangement of components of the multiplexer 30 (FIG. 1). In the example of FIG. 2 multiplexer 30 includes a flow control unit 60, the video input interface 80, audio interface 82, output interface 84 of the multiplexed stream and the table 88 of specific information about the programs. The flow control unit 60 includes a constructor NAL unit 62, PMT constructor 64, the search unit 66 a flow identifier (Flow ID) and the identifier designation unit 68, a program (PID).

[0083] In the example of FIG. 2 interface 80 and video input interface 82 include audio paketizatory PES to form blocks of encoded video data and encoded audio data. In other examples, video and / or audiopaketizatory may be incorporated into a block or module that is external to the multiplexer 30. In view of the example of FIG. Two video input interface 80 may generate PES packets of the encoded data received from the video encoder 28 and the interface 82 may generate audio PES packets from the encoded audio data received from audio encoder 26.

[0084] The block 60 receives the flow control PES packets from the interface 80 video inputs and 82 audio interface. Each PES packet includes a stream ID, which identifies the elementary stream, PES packet belongs. Block 66 Search thread ID may define a program which corresponds to PES packet by performing a query on tables 88 specific information about the programs. That is, the search unit 66 can determine the flow ID, a program corresponding to the received packet PES. Each program may include a plurality of elementary streams, while generally corresponds to one elementary stream of only one program. However, in some examples, the elementary stream may be incorporated into a variety of programs. Each PES packet may be included in a plurality of streams outputted from the multiplexer 30, as well as various services may each include different subsets of the available audio and video streams. Consequently, the search unit 66 may determine the stream ID whether the PES packet to be included in one or more output streams (e.g., one or more transport or program streams), and specifically, in which of the output stream includes a PES packet.

[0085] In one example, each elementary stream corresponding to the program. The multiplexer 30 may be responsible for ensuring that each elementary stream is associated with a particular program and, consequently, with the program ID (PID). When the PES packet is received, including the flow ID that is not recognized by the multiplexer 30 (e.g., a flow ID is not stored in table 88 specific program information), the control unit 68 destination PID creates one or more entries in tables 88 special program information to tie the new thread ID is not used with the PID.

[0086] After determining the program that corresponds to the packet PES, designer of the NAL unit 62 generates NAL unit, comprising a package of PES, for example, by encapsulating the PES packet header of the NAL unit, including the PID of the program, which corresponds to the flow ID packet PES. In some examples, the constructor 62 NAL unit, or another sub-block unit 60, flow control may generate an access unit containing a plurality of NAL units.

[0087] The constructor creates 64 PMT program map table (PMT) for the corresponding output stream multiplexer 30 using information from tables 88 specific information about the programs. In another example, the flow control unit 60 may include a constructor to create maps PSM program stream output by multiplexer 30. In some embodiments, the multiplexer 30 may comprise a PMT constructor 64, and a constructor PSM, and output the transport stream and / or the program stream. In the example of FIG. 2 designer can design the 64 PMT PMT, including a new descriptors described by this disclosure of the invention, such as a handle operating point, as well as any other necessary data and descriptors for PMT PMT. PMT constructor 64 can periodically, for example after a certain period of time or after transmitting a certain amount of data to send subsequent PMT transport stream. The constructor 64 can drop by PMT PMT constructor 62 for forming the NAL unit of the NAL unit, comprising a PMT, such as by encapsulation PMT corresponding heading NAL unit, comprising a respective PID.

[0088] Designer PMT 64 may create a data structure, such as a handle operating point, the operating point for each program. The data structure created by the designer 64 PMT, may report the value of reproducibility that describes playback capability that must be met by the receiver, to the working point of the ability of decoding, which describes the ability of decoding to satisfy the receiving device, to the working point, and the value of the bit rate, which describes the operating point of the bit rate.

[0089] For example, the designer 64 PMT can specify the number of images to be displayed to the operating point, and the frame rate of the image of the working point on the basis of information stored tables 88 specific information about programs or information received from the video encoder 28 through an interface 80 the video input. Constructor PMT 64 may report the number of images and / or the frame rate of the image using the value of the operating point of the ability of reproducing the data structure.

[0090] PMT constructor 64 may also determine the number of images to be decoded for the operating point and significance level for the profile that correspond to the operating point. For example, the designer 64 PMT may determine the number of macroblocks that need to be processed, the buffer size of the decoded picture buffer size of encoded images, the range of the vectors of the vertical movement, the maximum number of motion vectors for two consecutive macroblock, and / or whether the B-block have a separation sub-macro less than 8 × 8 pixels, and using these definitions to determine a level for the operating point. PMT constructor 64 may receive this information from the encoder 28 through the interface 80 video inputs. Constructor PMT 64 can then represent the number of images to be decoded, and / or the value of the level profile using the value of the decoding capability for operating point.

[0091] The PMT constructor 64 may further determine the value of the bit rate for the operating point and encode the value of the bit rate in a data structure. Bit rate value may correspond to an average bit rate or the maximum bit rate for the operation point. PMT constructor 64 may calculate the bit rate for the operating point or receive an indication of the bit rate of the video encoder 28.

[0092] The output interface 84 can receive the multiplexed stream one or more NAL units and / or blocks access from the flow control unit 60, such as NAL units, packets containing PES (e.g., audio and video), and / or NAL units, containing PMT . In some examples, the output interface 84 may generate a multiplexed stream of access units of one or more NAL units, corresponding to the common temporary location after receiving NAL units from the flow control unit 60. The output interface 84 transmits the multiplexed stream NAL units or blocks access to the corresponding output as a transport stream and program stream. The output interface 84 of the multiplexed stream may also receive a data structure of PMT constructor 64 and include the data structure as part of the bitstream.

[0093] FIG. 3 is a block diagram illustrating an exemplary set of table 88 of specific information about the programs. The elementary stream to which a transport packet belongs, may be determined based on the PID value of a transport packet. To the decoder correctly decodes the received data, the decoder must be able to determine which elementary streams each program belongs. Special information programs as listed in Table 88 specific information about the programs can explicitly specify the relationship between programs and elementary streams of components. In the example of FIG. 3 Table 88 specific information about programs include network information table 100, table 102 conditional access software access table 104 and the table 106 card programs. For example, in FIG. 3 that the output stream comprises a transport stream of MPEG-2. In an alternative example, the output stream may contain a program stream, in this case, the program map table 106 may be replaced with a program stream map.

[0094] The specification of MPEG-2 Systems specifies that each program carried in the transport stream, a program map table such as table 106 card programs associated with it. Table 106 card programs can include information about the program and elementary streams, which includes the program. As one example, the program identified by program number 3, may contain a video elementary stream with PID 33, the English audio stream with PID 57 and Chinese audio stream with PID 60. PMT allowed to include more than one program.

[0095] The base table card program, said the system specification of MPEG-2, can be decorated with a number of the plurality of tags, such as tags 108 specifications in the MPEG-2 system. Descriptors 108 may include any or all of the descriptor system specification of MPEG-2. In general, handles, such as handles 108, carry additional information about the program or its component elementary streams or substreams of bits. Descriptors may include encoding parameters of the video, audio encoding settings, language identification information "pan-and-scan", conditional access information, copyright information and other similar information. Specialist broadcasting or other user can specify additional private descriptors.

[0096] This disclosure provides a handle to the operating point to characterize the operating point in the bit stream consistent with the MPEG-2 system. Descriptors 108 may include descriptions of operating points for each operating point of the corresponding bitstream. As shown in FIG. 3, 108 descriptors include descriptors MVC extension 110, handle 112 and the hierarchy descriptor 114 operating points. Each of the descriptors 114 operating point may correspond to a particular operating point of the bitstream and to communicate to the operating point of the ability playback that describes playback capability that must be met by the receiver, to the working point of the ability of decoding, which describes the ability of decoding to satisfy receiving means, to the working point and the value of the bit rate, which describes the operating point of the bit rate. The elementary streams of components associated with the video, and there is a hierarchy descriptor that provides information to identify program elements comprising the components of hierarchically encoded video, audio and private streams.

[0097] The following Table 1 provides one example of data included in the descriptors 110 MVC extension. Various fields and bit fields shown in Table 1 are only one example. In one example, each bit MVC video substream is associated with a respective one of the descriptors MVC extension 110, which indicates characteristics corresponding substream MVC video bits. MVC video sub-stream of bits may be a need to gather other sub-streams of bits MVC video. That is, in order to decode and present a specific bit substream, the client device may have a need to retrieve and decode video data bit substreams from other common bit stream that includes two substreams of bits.

Table 1Deskriptor expansion MVCSintaksisChislo bitovSimvolikaMVC_extension_descriptor () { descriptor_tag8uimsbfdescriptor_length8uimsbfaverage_bit_rate16uimsbfmaximum_bitrate16uimsbfreserved4bslbfview_order_index_min10bslbfView_order_index_max10bslbftemporal_id_start3bslbftemporal_id_end3bslbfno_sei_nal_unit_present1bslbfreserved1bslbf}

[0098] In the example of Table 1 label field descriptor can correspond to an eight-bit field descriptor tags, which is included in each descriptor as set the standard MPEG-2 system, the identification of specific descriptors. Standard MPEG-2 system sets certain label descriptors and notes other label values ​​of descriptors such as values ​​from 36 to 63, as "reserved". However, Amendment 4 to the standard MPEG-2 system offers to install the MVC extension descriptor is set to "49", which corresponds to one of the reserved label tag, as described in the specification of MPEG-2 system. Thus, this disclosure provides a set of descriptors value descriptor_tag MVC extension 110 to the value "49".

[0099] Again, the length field descriptor can correspond to an eight-bit field length of the handle, which is also included in each descriptor as set the standard MPEG-2 system. The multiplexer 30 may set the value of a descriptor length field equal to the number of bytes of a respective one of the descriptors MVC extension 110, directly following field length descriptor. Since the length of MVC extension descriptor is not changed, the multiplexer 30 may set the value of the length field descriptor for each descriptor MVC extension 110 to a value to represent the presence of eight eight bytes of information following the length field descriptor.

[0100] Field of the average bit rate can contain shestnadtsatibitnoe field that indicates the average bit rate, in kilobits per second, reassemble videopoto ka AVC. That is, the average bit rate field describes the average bit rate for the video stream when the video stream is assembled from the component parts of the transport stream or the program stream, which corresponds to one of the descriptors 110 MVC extension. In some examples, the multiplexer 30 may set the value of the average bit rate to zero to indicate that the average bit rate is not available one of the descriptors 110 MVC extension.

[0101] Field of the maximum bit rate can contain shestnadtsatibitnoe field that indicates the maximum bit rate in kilobits per second, the video stream re-assembled AVC. That is, the maximum bit rate field describes the maximum bit rate for the video stream when the video stream is assembled from the component parts of the transport stream or the program stream, which corresponds to one of the descriptors 110 MVC extension. In some examples, the multiplexer 30 may set the value of the maximum bit rate to zero to indicate that the maximum bit rate is not available one of the descriptors 110 MVC extension.

[0102] Field of the minimum index of the order of images may comprise desyatibitnoe field that indicates the minimum value of the index picture order of NAL units, contained in the associated bit substream MVC video. Similarly, the maximum field of the order of index images is desyatibitnym field that specifies the maximum value of the index picture order of NAL units, contained in the associated bit substream MVC video.

[0103] The field of the initial temporary ID may comprise a three-bit field that indicates the minimum value of temporal_id syntax element NAL unit header of NAL units, contained in the associated bit substream MVC video. That is the value of temporary ID included in the header for each block of NAL. In general, the value of the temporary ID corresponds to a specific frame rate, where a relatively large temporary ID values ​​correspond to higher frame rates. For example, a value of '0' for the temporary ID can match the frame rate of 15 fps (frames / second), '1' for the temporary ID can match the frame rate of 30 frames / sec. Thus, the collection of images having temporary ID, equal to 0 in this example, the kit may be used to form a video segment having a frame rate of 15 frames / sec, while collecting all images having a temporary ID, equal to 0, and of all images having temporal ID, equal to 1 in another set may be used to form a video segment having a frame rate of 30 frames / sec. The multiplexer 30 determines the smallest temporary ID of all NAL units MVC video sub-streams of bits and sets the value of the initial temporary ID, equal to the smallest value of this particular temporary ID.

[0104] End time field may comprise three-bit ID field which indicates the maximum value of the temporary ID syntax element NAL unit header of NAL units, contained in the associated bit substream MVC video. Therefore, the multiplexer 30 determines the largest temporary ID of all NAL units MVC video sub-streams of bits and sets the initial field of temporary ID, equal to the highest value of certain temporary ID.

[0105] Field of the lack of block NAL SEI may include a one-bit flag, which when set to '1' indicates that the associated video substream bits MVC NAL units are no more information to improve. The multiplexer 30 can determine whether there were placed into a bit stream, one or more NAL units of additional information to improve and to set the value of the absence of the NAL unit SEI set to '1' when the bitstream is no NAL unit SEI, but can set the value of the absence of the NAL unit SEI is set to '0', when at least one block of SEI NAL is present in the bitstream.

[0106] The following Table 2 provides one example of data included in the handle 112 of the hierarchy. In MPEG-2 Systems hierarchy descriptor may be defined for the video program stream that contains an embedded video program stream. Various fields and bit fields shown in Table 2, given as one example. Meaning hierarchy_layer_index sets the index level of the current program stream, and the value hierarchy_embedded_layer_index identifies dependent layer. The draft MVC program stream may depend on another software thread using the hierarchy descriptor. That is, the relationship between the program stream may be determined based on data included in the hierarchy descriptor.

Table 2Deskriptor ierarhiiSintaksisChislo bitovSimvolikahierarchy_descriptor () { Descriptor_tag8uimsbfDescriptor_length8uimsbfreserved1bslbftemporal_scalability_flag1bslbfspatial_scalability_flag1bslbfquality_scalability_flag1bslbfHierarchy_type4Uimsbfreserved2bslbfHierarchy_layer_index6uimsbfTref_present_flag1bslbfreserved1bslbfhierarchy_embedded_layer_index6uimsbfreserved2bslbfhierarchy_channel}6uimsbf

[0107] As noted above, the specification of MPEG-2 Systems specifies that each descriptor includes a descriptor tag field and field length descriptor. Hence, the hierarchy descriptor 112 includes a descriptor tag field and field length descriptor. In accordance with the system specification of MPEG-2 multiplexer 30 may set the value of the descriptor tag is set to "4" for the descriptor 112 hierarchy.

[0108] The length of the handle 112 of the hierarchy can be defined in advance, as each instance of the hierarchy descriptor 112 must include the same amount of data. In one example, the multiplexer 30 may set the value in the length field descriptor value four indicating four bytes of the instance 112 of the hierarchy descriptor following the end of the field length descriptor.

[0109] The field type hierarchy describes a hierarchical relationship between the related levels of hierarchy and levels of nested hierarchy. In one example, the multiplexer 30 sets the value of the type field hierarchy based on a hierarchical relationship, for example, as described in Table 3 below. As one example, when scalability is used in more than one direction, the multiplexer 30 may set a field of type hierarchy is "8" ("combined scalability" as shown in Table 3), and a multiplexer 30 sets the values ​​of the flags field temporal scalability field Flag spatial scalability and quality scalability flag field according to the data extracted from the PES packet header and PES packet corresponding flows. In general, the multiplexer 30 may define relationships between different streams corresponding to different images and / or audio data streams. The multiplexer 30 may also determine whether the dependent stream that contains the level of improvement, spatial level, the level of improvement of signal to noise ratio (SNR), the level of improvement of the quality or level of other types of improvements.

[0110] As another example for substreams of bits MVC video multiplexer 30 may set a type field hierarchy to '9' ("MVC", as shown in Table 3) and may set the value of each field flag scalability field flag spatial scalability and Quality scalability flag field to '1'. As another example of the bit substreams for the base image MVC multiplexer 30 may establish a hierarchy type field to '15' and may set the flag field values ​​are scalability, spatial scalability flag field and quality scalability flag field to '1'. As another example for the prefix bit substreams MVC multiplexer 30 may establish a hierarchy type field to '14' and may set the flag field scalability flag field spatial scalability and quality scalability flag field is set to '1'.

[0111] The field of the index level in the hierarchy can contain six-bit field that defines a unique index of the associated program in the hierarchy of coding. Indexes can be unique within the definition of a single program. For the sub-streams video bit streams AVC, consistent with one or more profiles defined in Annex G ITU-T Rec. H.264 | ISO / IEC 14496-10, this program element is an index which is assigned so that the order of the bitstream will be correct if the associated representation according SVC video substreams of bits of the same access unit reassembled in ascending hierarchy_layer_index. For sub stream bits MVC video streams AVC, consistent with one or more profiles specified in Appendix H ITU-T Rec. H.264 | ISO / IEC 14496-10, this program element is an index which is assigned in such a way that any one of these values ​​is greater than the value hierarchy_layer_index, said hierarchy descriptor for MVC prefix sub stream bits.

[0112] Field of the nested hierarchy level of the index may contain the six-bit field that specifies the index of the table hierarchy of program element to which you want to access to the decoding of the elementary stream associated with a corresponding instance handle 112 hierarchy. This disclosure leaves the index field value to a nested hierarchy level are not given for the case when the field type of hierarchy has a value of 15 (ie, the value corresponding to the baseline).

[0113] The field of the channel hierarchy may contain the six-bit field that indicates the number of dedicated channel for the associated program element in an ordered set of transmission channels. The most robust transmission channel is defined by a lower value of the channel field with the definition of the hierarchy of the overall transmission hierarchy. Note that the channel in the hierarchy can be one and the same time assigned to multiple program elements.

[0114] The reserved field in Tables 1 and 2 are reserved for future use through the development of future standards. The methods of this disclosure are not being offered the appointment of the semantic meaning of the reserved fields.

[0115] The following table 3 shows the potential values ​​for the type of hierarchy are described below:

Table 3Znacheniya field type masshtabiruemost2SNR masshtabiruemost3Vremennaya masshtabiruemost4Razdelenie dannyh5Potok bit rasshireniya6Chastny potok7Profil several izobrazheniyami8Kombinirovannaya masshtabiruemost9Podpotok bits MVC video10-13Zarezervirovano14Prefiksny MVC15Bazovy bit sub-stream level, sub-stream of bits MVC base image, or a sub-stream of bits AVC video MVC

[0116] In some embodiments the hierarchy descriptor 112 may be used to report bit substream MVC, reported substreams incremental bits and nested substreams of bits. Nested sub-streams include directly dependent substreams bits corresponding hierarchy_embedded_layer_index, and all nested sub-streams of bits directly dependent sub stream bits. This disclosure of the invention the image contained explicitly called enhanced images, while the images are embedded, called dependent images.

[0117] In the example in which the output of the multiplexer 30 contains a program stream, a table 88 of specific information about the programs may include a program stream structure (PSM). PSM may provide a description of the elementary streams in the respective program stream and elementary streams relationship with each other. In some examples, the structure of a program stream may also correspond to the transport stream. When carrying in the respective transport stream structure PSM must not be modified. Multiplexer 30 may indicate that the PSM is present in the PES packet by setting the stream_id 0xBC in the PES packet, that is, the hexadecimal value BC, which corresponds to the binary value of 10.1111 million, or 188 decimal.

[0118] The multiplexer 30 contains a complete list of all the programs available in the transport stream, the table 104 communications programs. The multiplexer 30 can also embed a table relations programs and blocks NAL. Multiplexer 30 may indicate that the NAL unit includes a table relations program through the appointment of NAL unit values ​​PID 0. The multiplexer 30 may make the list table 104 communications programs each program with PID value of transport packets that contain the appropriate table card program. Using the same example above, the exemplary program map table, which indicates the number of elementary streams of the program 3 PID is 1001 and the other has a different PMT PID 1002. These sets of information or the like may be included in table 104 links programs.

[0119] Table 88 specific information about the programs also include the 100 network information table (NIT) and the conditional access table 102 (CAT). Program number zero, as indicated in the PAT, has a special meaning. In particular, the program number zero can be used to specify the path to the table 100 network information. The table is optional, and when present, the table can provide information about the physical network that carries traffic flow, such as the frequency channel information from the satellite transponder, modulation characteristics, the sender service, the service name and information about available alternative networks.

[0120] If any of the elementary streams within the transport stream are scrambled, there must be a conditional access table 102. Table 102 provides conditional access information using the system (s) and provides a scrambling PID values ​​of transport packets that contain information management and conditional access authorizing information. The format of this information is not specified in the standard MPEG-2 system.

[0121] FIG. 4 is a block diagram illustrating an exemplary set of data that can be included in one of the descriptors operating point 114 (FIG. 3). In the example of FIG. 4 handle 118 operating point includes a field 120 tags descriptor field 122 Length descriptor field 124 of the frame rate, the field 126 the number of images to be displayed, a field 128 the number of images to be decoded field 130 image identifier field 132 average bit rate field 134 maximum bit rate, a temporary identifier field 136 and reserved field 138 of tail bits.

[0122] In the example of FIG. 4 field 124, and the frame rate field 126 the number of images for playback corresponds approximately to the value of the ability to play, the field of 128 of the image decoding corresponds to the approximate value of the ability to decode and field 132 average bit rate and a maximum field of 134 bit rate correspond to the approximate value of the bit rate. Handle operating point 118 is only one example of a data structure that can be used to report the operating point characteristics such as reproducibility, the ability to decode and bit rate. FIG. 5 and 6 below in alternative examples of descriptors operating points that report these characteristics.

[0123] As described above, the specification of MPEG-2 system indicates that each descriptor has a field descriptor tag and a descriptor length field, each of 8 bits. Thus, the multiplexer 30 (FIG. 1) may assign the value of the field descriptor tag 120 indicating the operating point of MVC descriptor. The multiplexer 30 may also determine the number of images for the operating point and the number of reserved bits to handle the operating point, and then calculate the length of the working point descriptor 118 bytes following the length field 122 descriptor. Multiplexer 30 may assign this calculated value of the length field 122 length descriptor when an instance descriptor 118 operating point.

[0124] The field 124 may include a frame rate of 16-bit field that specifies the maximum frame rate in frames / 256 seconds to re-assemble video AVC. That is, the multiplexer 30 may calculate the maximum frame rate of 256 second period to set the value of the field 124 of the frame rate. In some examples, the division by 256 can lead to the conversion of a floating point value to an integer value. In another example, may be used other than 256 seconds periods. 256 second time period described in view of the field 124 of the frame rate is just one example of the potential for which can be calculated from the maximum frame rate of the operating point.

[0125] The field of 126 images for display may include desyatibitnoe field that indicates the value of the number of images intended to display video stream re-assembled AVC. In general, a field 126 for displaying the number of images represents the number of images to be displayed for the respective operating point. Since different displays may be capable of displaying different number of images, the client device can use the value field 126 for displaying the number of images to select a working point which has many images to be displayed as possible on the screen to the client device. For example, if a client device capable of displaying four images, the client device may select the operating point of the field of the image for display having a value indicating that will be displayed four images for the respective operating point. Consequently, the field of 126 images to be included as part of the value of the ability to play. Similarly, the multiplexer 30 may set the value of the field 126 to display images according to the number of images to be displayed to the operating point.

[0126] The field 128 for decoding the number of images may comprise desyatibitnoe field which indicates the value of the number of images required for decoding the re-assembled AVC video stream. This value can differ from the number of images to be displayed, the specified number field 126 for displaying images. This can occur because certain images required for decoding the resulting dependency of images, but that is not actually displayed.

[0127] Briefly referring to FIG. 7, as an example of image S0 and S1 may be images that are to be displayed to the operating point. Image S0 may be decoded directly without decoding of any other pictures. However, to decode the image S1, must also be decoded image S2, S1 as the image data includes a prediction related to an image S2. Consequently, in this example, the field 126 for number of image reproduction will have a value of "two", but the field 128 for decoding the number of images will have a value of "three". In some examples, the image to be displayed may be interpolated from one or more images, so the number of images to be displayed may be higher than the number of images to be decoded. That is, using a basic image and depth information, the video decoder 48 (FIG. 1) may be interpolated a second image. Video decoder 48 may use two or more images to calculate the depth information to interpolate new image or video decoder 48 may receive the depth information for the image from the source device 20.

[0128] The field 128 for decoding the number of images may match the capabilities of decoding that the decoder of the client device (such as video decoder 48 of the target device 40) must be able to decode the number of images equal to the value of the number of image 128 for decoding. Therefore, the client device may select an operating point having a field of image for decoding, representing the number of pictures that the video decoder can decode the client device.

[0129] Handle 118 operating point in FIG. 4 also includes field identifiers 130 images. Each of the identifier fields 130 may comprise images desyatibitnoe field which indicates the value of view_id NAL units, contained in the re-assembled AVC video stream. Thus, an image identifier of each displayed image to inform the working point, using field identifiers 130 images. That is the image identifier field identifiers 130 images match the displayed image. Thus, images that are decoded but not displayed, is not reported by the image identifier field 130, in the example of FIG. 4.

[0130] Field 132 average bit rate can hold a 16-bit field that indicates the average rate of bits in kilobits per second, reassembled video AVC. When set to 0, the average bit rate is not specified. That is, the value of field 132 to zero average bit rate implies that the field 132 average bit rate should not be used to determine the average bit rate reassembled AVC video stream.

[0131] Field 134 maximum bit rate can hold a 16-bit field that indicates the maximum bit rate in kilobits per second, the video stream re-assembled AVC. When set to 0, the maximum bit rate is not specified. That is, when the value of the maximum bit rate 134 is set to zero, the field 134 the maximum bit rate must not be used to determine the maximum bit rate reassembled AVC video stream.

[0132] Field 136 temporary ID can contain three-bit field that indicates the value of temporal_id, corresponding to the frame rate of the video stream re-assembled AVC. That is temporal_id can be used to determine the frame rate reassembled AVC video stream, as discussed above.

[0133] An exemplary operating point descriptor 118 also includes a reserved field 138 of tail bits. In one embodiment, for example, as shown in Table 4 below, the number of reserved tail bits may be used for additional messages and to fill the working point descriptor 118, a handle 118 so that the operating point to an end on a byte boundary. For example, as discussed above, handle 118 may utilize the operating point ten identifier bits to represent each image of the displayed image. Static number of bits in addition to the bits used for image identifiers reserved and tail bits, in this example, is 87. Thus, to ensure that the operating point descriptor 118 ends on a byte boundary (i.e., it has a number of bits which is divisible by eight ), the multiplexer 30 may be added a number of tail bits in accordance with the following formula:

tail bits = (1 + 6 * num_display_views)% 8,

where '%' is the mathematical remainder operator of integer division. That is A% B results in a residue A, divided by the B, so that the balance is in the range of whole numbers between 0 and B-1.

[0134] Table 4 summarizes an exemplary set of data that can be included in the example operating point descriptor 118 in FIG. 4.

Table 4Deskriptor operating point MVCSintaksisChislo bitovSimvolikaMVC_op_descriptor (3 { (i = 0; i <num_display_views; i ++) {} view_id10uimsbf {Reserved_bit1bslbf}}

[0135] FIG. 5 is a block diagram illustrating an alternative exemplary set of data that can be included in one of the descriptors operating point 114 (FIG. 3). In general, each of the descriptors 114 operating points must have a common format so that the client device could be adapted to receive operating point descriptors common format. Thus, each of the descriptors 114 operating point can have a format similar to the descriptor operation point in FIG. 4, FIG. 5 or FIG. 6, or other common format that includes similar data for the message.

[0136] In the example of FIG. 5 handle 140 operating point includes a field 142 tags descriptor field 144 Length descriptor field 146 profile_IDC field 148 level_IDC field 149 of the frame rate, the field 150 the number of images to be displayed, a field 152 the number of images to be decoded field 154 average bit rate field 156 of the maximum bit rate temporary identifier field 158, field 160 reserved bits field 162 indexes the order of images, image ID field 164 and reserved field 166 of tail bits. IDC is a "pointer". As explained below, an example of operating point descriptor 140 explicitly informs the values ​​and profile_idc level_idc for the operating point, as well as the information being collected by the operating point.

[0137] The field of 150 images for display and frame rate field 149 correspond to the value the ability to play, Descriptor 140 operating point. Profile_IDC field 146, field 148 and the field 152 level_IDC number of images to be decoded in the example of FIG. 5 are examples of data that can match the ability of decoding Descriptor 140 operating point. Field 154 and an average bit rate field 156 correspond to the maximum bit rate value of the bit rate Descriptor 140 operating point.

[0138] As described above, the specification of MPEG-2 system indicates that each descriptor has a field descriptor tag and a descriptor length field, each of which may be 8 bits in length. Thus, the multiplexer 30 (FIG. 1) may assign the value of the field descriptor tag 142 indicating the operating point of MVC descriptor. The multiplexer 30 may also determine the number of images for the operating point and the number of reserved bits to handle the operating point, and then calculate the length of the working point descriptor 140 bytes following the length field 144 of the descriptor. Multiplexer 30 may assign this calculated value of the length field 144 length descriptor when an instance descriptor 140 operating point.

[0139] The field 146 may comprise profile_IDC octet field that indicates profile_IDC operating point re-collected by the information given in the descriptor 140 operating point. The field 148 may comprise level_IDC octet field that indicates level_IDC operating point re-collected by the information given in the descriptor 140 operating point.

[0140] The field 149 may include a frame rate of 16-bit field that specifies the maximum frame rate in frames / 256 seconds to re-assemble video AVC. That is, the multiplexer 30 may calculate the maximum frame rate of 256 second period to set the value of the field 149 of the frame rate. As in the case of a field frequency of 124 frames, in other examples, the field 149 for the frame rate could be used other periods except for 256 seconds.

[0141] The field of 150 images for display may include desyatibitnoe field that indicates the value of the number of images intended to display video stream re-assembled AVC. In general, a field 150 for displaying the number of images represents the number of images to be displayed for the respective operating point. The field 152 for decoding the number of images may comprise desyatibitnoe field which indicates the value of the number of images required for decoding the re-assembled AVC video stream. This value can differ from the number of images to be displayed, the specified number field 150 for displaying images. This can occur because certain images required for decoding the resulting dependency of images, but that is not actually displayed, for example, as described above, taking into account the field of image 128 for decoding.

[0142] Field 154 average bit rate can hold a 16-bit field that indicates the average rate of bits in kilobits per second, reassembled video AVC. When set to 0, the average bit rate is not specified. That is, the value of field 154 to zero average bit rate implies that the field 154 average bit rate should not be used to determine the average bit rate reassembled AVC video stream. Field of the maximum bit rate 156 may comprise a 16-bit field that indicates the maximum bit rate in kilobits per second, re-assembled AVC video stream. When set to 0, the maximum bit rate is not specified. That is, when the value of the maximum bit rate 156 is set to zero, the field 156 of the maximum bit rate must not be used to determine the maximum bit rate reassembled AVC video stream.

[0143] Field 158 temporary ID can contain three-bit field that indicates the value of temporal_id, corresponding to the frame rate of the video stream re-assembled AVC. That is temporal_id can be used to determine the frame rate reassembled AVC video stream, as discussed above.

[0144] Handle 140 also includes an operating point of an index field 162 and a field picture order identifier 164 images. Each of the fields 162 index of order may contain images desyatibitnoe field that indicates the index of the order of image blocks NAL, contained in the operating point. The client device can reassemble blocks NAL, corresponding to all the reported values ​​view_order_index, reported in the descriptor 140 operating point through the field about 162 index images. Fields 162 indexes the order of images include field picture order index for each of the images to be decoded. This value view_order_index client device may retrieve the corresponding NAL units of elementary streams as MVC extension descriptor index values ​​reported range in the order of image elementary stream, and the range covers the value view_order_index, reported in the descriptor operating point.

[0145] Each of the fields 164 may include an image identifier desyatibitnoe field which indicates the value of view_id NAL units, contained in the re-assembled AVC video stream. Thus, an image identifier of each displayed image to inform the working point, using field identifiers 164 images. That is the image identifier field identifiers 164 images match the displayed image. Thus, images that are decoded but not displayed, is not reported by the image identifier field 164, in the example of FIG. 5.

[0146] Handle 140 also includes an operating point of a reserved field 166 of tail bits. Handle 140 operating point may include tail bits as a filling so that the number of bits in the descriptor 140 operating point divide evenly into eight. Since the number of fields of the order of index images and an image identifier fields may vary, the number of tail bits, which multiplexer 30 includes a handle 140 operating point varies accordingly. For example, the number of tail bits can be determined according to the following formula:

tail bits = (6 * (num_display_views + num_decode_views))% 8,

where '%' is the remainder of integer division operator.

[0147] Table 5 summarizes an exemplary set of data that can be included in the exemplary operating point descriptor 140 in FIG. 5.

Table 5Deskriptor operating point MVCSintaksisChislo bitovSimvolikaMVC_op_descriptorQ { maximum_bitrate16uimsbftemporal_id3uimsbfreserved_bit1bslbffor (i = 0; i <num_decode_views; i ++) {view_order_index10uimsbf} for (i = 0; i <num_display_views; i ++) {view_id10uimsbf} for (i = 0; i <6 * (num_display_views + num_decode_views)% 8; i ++) {reserved_bit1bslbf}}

[0148] FIG. 6 is a block diagram illustrating another alternative exemplary set of data that can be included in one of the descriptors operating point 114 (FIG. 3). In the example of FIG. 6, handle 170 operating point includes a field 172 tags descriptor field 174 Length descriptor field 176 profile_IDC field 178 level_IDC field 180 frame rate field 182 the number of images to be displayed, a field 184 the number of images to be decoded field 186 average bit rate field 188 maximum bit rate field 190 temporary identifier field 192 reserved bits field 194 identifier operating point field 196 dependent flag operating point, an optional field 198 an identifier dependent operating point field 200 index picture order fields 202 of identifiers of images, and fields 204 tail bits reserved. As described below, the handle 170 provides an exemplary operating point descriptor operating point for the operating point that depends on the operating point and other reports that additional images required for decoding.

[0149] The field of 182 images for display and frame rate field 180 correspond to the value the ability to play, Descriptor 140 operating point. Profile_IDC field 176, field 178 and the field 184 level_IDC number of images to be decoded in the example of FIG. 6 are examples of data that can match the ability of decoding Descriptor 140 operating point. Field 154 and an average bit rate field 156 correspond to the maximum bit rate value of the bit rate Descriptor 140 operating point.

[0150] As described above, the specification of MPEG-2 system indicates that each descriptor has a field descriptor tag and a descriptor length field, each of 8 bits. Thus, the multiplexer 30 (FIG. 1) may assign the value of the field descriptor tag 172 indicating the operating point of MVC descriptor. The multiplexer 30 may also determine the number of images for the operating point and the number of reserved bits to handle the operating point, and then calculate the length of the working point descriptor 170 bytes following the length field 174 descriptor. The multiplexer 30 assigns it calculated the length of the field 174 of length descriptor when an instance descriptor 140 operating point.

[0151] The field 176 may comprise profile_IDC octet field that indicates profile_idc operating point re-collected by the information given in the descriptor 170 operating point. The field 178 may comprise level_IDC octet field that indicates level_idc operating point re-collected by the information given in the descriptor 170 operating point.

[0152] The field 180 may include a frame rate of 16-bit field that specifies the maximum frame rate in frames / 256 seconds to re-assemble video AVC. That is, the multiplexer 30 may calculate the maximum frame rate of 256 second period to set the value of the field 149 of the frame rate. As in the case of a field frequency of 124 frames, in other examples, the field 180 for the frame rate could be used other periods except for 256 seconds.

[0153] The field of 182 images for display may include desyatibitnoe field that indicates the value of the number of images intended to display video stream re-assembled AVC. In general, a field 182 for displaying the number of images represents the number of images to be displayed for the respective operating point. The field 184 for decoding the number of images may comprise desyatibitnoe field which indicates the value of the number of images required for decoding the re-assembled AVC video stream. This value can differ from the number of images to be displayed, the specified number field 182 for displaying images. This can occur because certain images required for decoding the resulting dependency of images, but that is not actually displayed, for example, as described above, taking into account the field of image 128 for decoding.

[0154] Field 186 average bit rate can hold a 16-bit field that indicates the average rate of bits in kilobits per second, reassembled video AVC. When set to 0, the average bit rate is not specified. That is, the value of field 186 to zero average bit rate implies that the field 186 average bit rate should not be used to determine the average bit rate reassembled AVC video stream. Field of the maximum bit rate 188 may comprise a 16-bit field that indicates the maximum bit rate in kilobits per second, re-assembled AVC video stream. When set to 0, the maximum bit rate is not specified. In particular, when the value of the maximum bit rate 188 is set to zero, the field 188 the maximum bit rate must not be used to determine the maximum bit rate reassembled AVC video stream.

[0155] Field 190 temporary ID can contain three-bit field that indicates the value of temporal_id, corresponding to the frame rate of the video stream re-assembled AVC. That is temporal_id can be used to determine the frame rate reassembled AVC video stream, as discussed above. Field 192 reserved bit corresponds to a single bit, which is reserved for future use.

[0156] Handle 170 also operating point includes an identifier field 194 and the operating point dependent flag field 196 of the working point. Field identifier 194 can comprise the operating point desyatibitnoe field that specifies the identity of the operating point, the described handle 170 operating point. Field 196 dependent operating point flag is a single bit flag that indicates whether the reported dependence on the current operating point of the other operating point. If the flag is dependent operating point 196 has a value of "one" (or "true"), the relationship is communicated; if the value of the flag 196 dependent operating point "zero" (or "false"), the relationship is not reported.

[0157] When the value of flag 196 dependent operating point of "true" or "unit", the handle 170 the operating point further includes a field identifier 198 operating point dependent. When present, the field of 198 operating point identifier may contain desyatibitnoe field that specifies the identity of the operating point, which depends on the current descriptor. That is, when the multiplexer 30 determines that the handle 170 operating point corresponds to an operating point that depends on the other operating point, the multiplexer 30 sets the value of the dependent flag operating point in "true" or "unit" and then communicates the identifier of the working point from which depends operating point corresponding to the descriptor 170 operating point.

[0158] Handle 170 also includes an operating point of an index field 200 and a field picture order identifier 202 images. Each of the fields 202 indexes the order of images may comprise desyatibitnoe field that indicates the index value of the order of NAL units of images, contained in the current operating point identifier operation_point_id, but not contained in the operating point identifier dependent_operation_point_id. The client device can reassemble blocks NAL, corresponding to all the reported values ​​view_order_index, reported in the descriptor 170 operating point by means of the field indexes about 200 images. Fields 200 indexes the order of images include field picture order index for each of the images to be decoded. This value view_order_index client device may retrieve the corresponding NAL units of elementary streams as MVC extension descriptor index values ​​reported range in the order of image elementary stream, and the range covers the value view_order_index, reported in the descriptor operating point. Working point, reported in the descriptor 170 operating point re-collected by blocks NAL, corresponding to all the values ​​reported view_order_index fields 200 indexes the order of images and blocks of NAL, receiving an operating point with the identifier dependent_operation_point_id.

[0159] Each of the fields 202 may include an image identifier desyatibitnoe field which indicates the value of view_id NAL units, contained in the re-assembled AVC video stream. Thus, an image identifier of each displayed image to inform the working point, using field identifiers 202 images. That is the image identifier field identifiers 164 images match the displayed image. Thus, images that are decoded but not displayed, is not reported by the image identifier field 202 in the example of FIG. 5.

[0160] Handle 170 also includes an operating point of a reserved field 204, tail bits. Handle 170 operating point may include tail bits as a filling so that the number of bits in the descriptor 170 operating point divide evenly into eight. Since the number of fields of the order of index images and an image identifier fields may vary, the number of tail bits, which multiplexer 30 includes a handle 170 operating point may vary accordingly. For example, the number of tail bits can be determined according to the following formula:

tail bits = (6 * (num_display_views + num_decode_views))% 8,

where '%' is the remainder of integer division operator.

[0161] Table 6 below summarizes an exemplary set of data that can be included in the exemplary operating point descriptor 170 in FIG. 6.

Table 6Deskriptor operating point MVCSintaksisChislo bitovSimvolikaMVC_op_descriptor () { (op_dependent_flag) dependent_operation_point_id1010for (i = 0; i <num_decode_views; i ++) {} view_order_index10uimsbf

for (i = 0; i <num_display_views; i ++) {view id10uimsbf} for (i = 0; i <6 * (num_display_views + num_decode_views)% 8; i ++) {}} reserved_bit1bslbf

[0162] As a further alternative, the source device 20 (FIG. 1) reports the characteristics of the operating point using the data structure descriptor is different from the operating point. For example, source device 20 may report the value of reproducibility that describes playback capability that satisfies the receiving device for the operating point MVC, of ​​the ability of decoding, which describes the ability of decoding, satisfying the receiving device for the operating point of MVC, and the value of the bit rate, which describes the bit rate of the working point of MV using a modified handle extension MVC.

Below, Table 7 illustrates an example of such a modified MVC extension descriptor.

Table 7Deskriptor expansion MVCSintaksisChislo bitovSimvolikaMVC_extension_descriptor () { (i = 0; i <num display views; i ++) {} view_id10uimsbf

[0163] The multiplexer 30 (FIG. 2) may be designed MVC extension descriptor 110 according to the syntax specified in Table 7. In general, semantics, syntax elements of Table 7 is the same as that of elements with common names described in Table 1 given above. Example Table 7 includes additional components in addition to the elements of Table 1, namely the frame rate field, a field of image for the display of image field decoding and field identifiers of images for each image a working point, which corresponds to a handle extension MVC.

[0164] Field of the frame rate can contain shestnadtsatibitnoe field, which specifies the maximum frame rate in frames / 256 seconds to re-assemble video AVC. Field number of images to display "num_display_views" may contain desyatibitnoe field that indicates the value of the number of images intended to display video stream re-assembled AVC. Field of the image decoding "num_decode_views" desyatibitnoe may comprise a field which indicates the value of the number of images required for decoding the re-assembled AVC video stream. Each field image identifier "view_id" may contain desyatibitnoe field that indicates the value of view_id NAL units corresponding to the image contained in the video stream re-assembled AVC.

[0165] In some examples, one or more descriptors operating points may include a value that indicates the maximum value of the temporary identifier, and the maximum frame rate all operating points MVC bitstream. In some examples, the maximum value of the temporary identifier, and the maximum frame rate of all the operating points MVC bitstream may be communicated to the operating point MVC descriptor.

[0166] Fig. 7 is a conceptual diagram illustrating an exemplary sample predictions MVC. In the example of FIG. 7 illustrates eight image (image having the ID with "S0" for "S7"), and to each of twelve images illustrated temporary location (from "T0" on "T11"). That is, each row in FIG. 7 corresponds to the image, while each column indicates a temporary location.

[0167] Although what has so-called MVC base image which is decoded by decoders H.264 / AVC, and images of the stereopair, should also be supported by MVC, MVC advantage is that it can maintain an example which uses a than two images as an input and decodes a 3D video is a 3D video, represented by a plurality of images. Reproducing means customer having MVC decoder may expect with a plurality of 3D video images.

[0168] The frames in FIG. 7 are indicated with arrows beside each row and each column in FIG. 7 with the hatched block including a letter indicating whether the corresponding frame is intra-coded (i.e., I-frame) or an encoded outwardly in the same direction (i.e. as a P-frame) or several directions (i.e. both B- frame ). In general, the predictions indicated by arrows, where the frame to which they are directed, using as a model for predicting the object from which they are directed. For example, P-picture frame S2 temporary location T0 predict on the basis of the I-frame image S0 in a temporary location T0.

[0169] As in the case of the image coding video frames of a video sequence encoding multiple video images can be encoded with prediction based time frames in different locations. For example, b-frame image S0 in a temporary location T1 has aimed an arrow at him from the I-frame image to a temporary location S0 T0, indicating that the b-frame is predicted from the I-frame. Additionally, however, in the context of encoding of video images of several frames can be predicted by predicting changes between images. That is, an image component can utilize other components of the image to the image of the sample. In MVC, for example, the prediction of change between images is performed as if the component images in the other picture is a model between the predictions. Potential samples to predict changes between images are reported in the sequence parameter set MVC extension, and can be modified by constructing a list of reference pictures, which provides a flexible sequencing the samples between the samples or prediction prediction between pictures.

Below, Table 8 provides an exemplary definition for the sequence parameter set MVC extension.

Table 8

<IMG>

[0170] FIG. 7 provided various examples of predicting changes between images. S1 image frames in the example of FIG. 7 illustrates how to predict the outcome of frames in different time location of the image S1, as well as predicted by predicting changes between images based on the image frames S0 and S2 in the same temporary location. For example, b-frame image S1 in a temporary location T1 predicted on the basis of each of the B-frame image S1 in temporary locations T0 and T2, as well as b-frame image S0 and S2 in the temporary location T1.

[0171] In the example of FIG. 7 capital letter "B" and lowercase "b" are used to indicate different hierarchical relationships between shots, instead of different encoding methods. In general, images with capital letter "B" relatively higher in the hierarchy of predictions than footage with lowercase "b". FIG. 7 also illustrates the variation in the prediction hierarchy using different degrees of shading, where the frames with larger magnitude shading (i.e. relatively dark) are higher in the hierarchy prediction than those frames which have a lower hatch (i.e. relatively light). For example, all I-frames in FIG. 7 are illustrated with full shading, while P-frames have a slightly lighter shading and B-frames (frames with a lower case "b") have different degrees of shading relative to each other, but always lighter than hatching P-frames and I-frames.

[0172] In general, the hierarchy of the prediction relates to indexes of the order of images in the sense that relatively higher in the hierarchy, the prediction frames are to be decoded before decoding of frames which are relatively lower in the hierarchy, so that those relatively higher in the hierarchy frames may be used as the reference frames during decoding relatively lower in the hierarchy of predictive frames. Index picture order is an index that indicates the decoding order of image components in the access unit. Indexes order the images are assumed to be in the SPS extension MVC, as specified in Appendix H of the standard H.264 / AVC (Amendment MVC). In the SPS for each index i corresponding report view_id. Decoding components image should follow the increasing order of the indices of the order of images. If all the images presented, the index order of the images are in sequential order from 0 to num_views_minus_1.

[0173] Thus, the frames are used as reference frames may be decoded before the decoding of frames which are coded with reference to the reference frames. Index picture order is an index that indicates the decoding order of image components in the access unit. For each index of the order corresponding to the image i The report view_id. Decoding component images to be the order of increasing index order images. If all the images presented, the set of indices of the order may contain images sequentially ordered set from zero to the total number of images of minus one.

[0174] For certain frame at the same level of hierarchy decoding order may not be important with respect to each other. For example, I-picture frame in a temporary location S0 T0 used as a reference frame for a P-picture frame in a temporary location S2 T0, which in turn is used as a frame for a P-picture frame in a temporary location in S4 T0. Hence, I-picture frame in a temporary location S0 T0 to be decoded P-picture frame in a temporary location S2 T0, which must be decoded before the P-picture frame in a temporary location in S4 T0. However, among the images S1 and S3, the decoding order does not matter, since the images S1 and S3 are independent from each other for the prediction, but only predicted on the basis of the images that are higher in the hierarchy prediction. Moreover, the image can be decoded S1 to S4 image, provided that the image after the image is decoded S1 S0 and S2.

[0175] Thus, for the description of image S0 to S7 can be used hierarchical ordering. Let the record SA> SB means that the image of SA to be decoded to the image SB. Using this recording, S0> S2> S4> S6> S7, in the example of FIG. 7. Also, following the example of FIG. 7 S0> S1, S2> S1, S2> S3, S4> S3, S4> S5 and S6> S5. Any possible decoding order for images that do not violate these requirements. Therefore, there may be many different orders of decoding only with certain restrictions. Below are two exemplary decoding order, though it should be understood that many other possible decoding orders. In one example illustrated in Table 9 below, the image is decoded as soon as possible.

Table 9ID izobrazheniyaS0S1S2S3S4S5S6S7Indeks order izobrazheniya02143657

[0176] Example Table 9 shows that the image S1 can be decoded immediately after having been decoded images S0 and S2, the image S3 can be decoded immediately after having been decoded image S2 and S4, and the image S5 can be decoded immediately after having been decoded image S4 and S6.

[0177] Table 10 below provides another exemplary decoding order in which the order of decoding is such that any image that is used as a reference for another image are decoded to the images that are not used as a reference to any other image.

Table 10ID izobrazheniyaS0S1S2S3S4S5S6S7Indeks order izobrazheniya05162734

[0178] Example of Table 10 shows that image frames S1, S3, S5, and S7 do not act as reference frames for frames of any other pictures, and therefore, the image S1, S3, S5, and S7 can be decoded after the frame of the image which is used as the reference frames, i.e. image S0, S2, S4 and S6 in the example of FIG. 7. respect to each other image S1, S3, S5 and S7 can be decoded in any order. Therefore, in the example of Table 10, the image S7 to encode each of the images S1, S3 and S5.

[0179] To be clear, there may be a hierarchical relationship between frames of each image, as well as between the temporary location of personnel in each frame. Considering the example in FIG. 7 frames in a temporary location T0 are either predicted by the intra prediction or predicted by predicting changes between images based on the staff of other images in a temporary location T0. Similar images to a temporary location T8 are either predicted posreds Twomey predicting changes within the image, or predicted by predicting changes between images based on the staff of other images in a temporary location T8. Therefore, considering the time hierarchy temporal location T0 and T8 are on top of time hierarchy.

[0180] The frames in the temporary location of T4 in the example of FIG. 7 are lower in the time hierarchy than frames temporary location T0 and T8, as the time frame arrangement T4 are B-encoded with reference to the time frame locations T0 and T8. Personnel temporary location T2 and T6 are lower in the time hierarchy than frames in the location T4. Finally, the staff at the temporary location T1, T3, T5 and T7 are lower in the time hierarchy than the shots to a temporary location T2 and T6.

[0181] The MVC bitstream subset of all can be recovered to form substreams of bits that is still consistent with MVC. There are many possible sub stream bits which may be required for specific applications, based, for example, the service provided by the server performance support and a decoder to one or more clients and / or advantages of one or more clients. For example, a customer may require only three images, and there may be two scenarios. In one example one client may require soft feeling and may prefer watching images S0, S1 and S2 the values ​​view_id, while another client may require the scaled image and prefer images S0, S2 and S4 to the values ​​view_id. If initially view_id arranged following the example of Table 9, the index values ​​of the order of images constitute {0, 1, 2} and {0, 1, 4} in the two examples respectively. Note that both of these substreams bits may be encoded as independent and MVC bitstreams can be supported simultaneously.

[0182] There may be many sub-streams of bits MVC, which are decoded MVC decoder. In theory, any combination of images, which satisfies the following two properties can be decoded MVC decoder compatible with a specific profile or level: (1) the image components in each access unit in ascending order of the index picture order, and (2) for each image, combinations of its dependent images are also included in the combination.

[0183] FIG. 8 shows the strip chart illustrating an exemplary method for using data structures which informs the operating point characteristics. That is, the method of FIG. 8 includes the construction of data structures for each operating point the bit stream of MPEG-2 Systems source device, such as the source device 20 (FIG. 1). The method of FIG. 8 also includes the use of received data structures to select a working point from which the multimedia data is recovered to decode and display the target device, such as target device 40 (FIG. 1).

[0184] Initially, in the example of FIG. 8, the source device 20 determines the working point of the program (210). For example, source device 20 may select different subsets of the image of the program to create different operating points that represent client devices having different features, such as playback and decoding. The administrator may interact with the source device 20, for example, to select images and employment points that represent client devices that have varying ability to playback and decoding, or other working points, which can be created by the source device 20 automatically.

[0185] After determining the operating points for the program source device 20 can generate a data structure for each of the operating points in the program map table (212), for example, when a bitstream to be broadcast as a transport stream of the MPEG-2 systems. Alternatively, source device 20 may generate a data structure in the program stream map, for example, when a bitstream to be broadcast as a program stream of MPEG-2 system. In any case, the source device 20 can generate the operating point for each data structure that represents the characteristics of the corresponding operating point. The data structure descriptor may include a working point corresponding to one of the examples in Fig. 4-6, for example. Thus, the data structure may inform the playback, decoding performance and bit rate for the respective operating point.

[0186] Then, the original A / V device 20 may output data structure (214), for example within the PMT in the example of FIG. 8, the client device, such as target device 40 (FIG. 1). Thus, source device 20 may output data structure as part of the bitstream. Source device 20 may output the bit stream as a broadcast, unicast, multicast transmission to any address, or other communication protocol over a network, such as wireless or wired network, or to broadcast on the television frequency, for example, according to the signals, consistent with the standards of the improved television system (ATSC) standards or National Television Systems Committee (NTSC). Alternatively, source device 20 may encode the bit stream into a computer-readable storage medium such as a DVD-ROM, disc Blu-ray, flash memory, magnetic disk, or other storage medium, in this case, source device 20 may form PSM, which It includes a data structure for operating points and encoding PSM in computer readable media.

[0187] The target device 40 may eventually take the PMT (or PSM) from the source device 20 (216). Then, the target device 40 can select one of the operating points on the basis of characteristics of operating points reported data structures included in the PMT or PSM (218). In general, the target device 40 can select the operating point for which the target device 40 satisfies playback capability and decoding a message matches the data structure. For example, the target device 40 may be able to determine whether the video output 44 perform reproduction of images of said structure of data as the number of images to be displayed, a frame rate in accordance with the value of the ability of reproducing a data structure communicated to the operating point. Similarly, the target device 40 may be able to determine whether the video decoder 48 to decode the number of images to be decoded for the operating point that reported data structure values ​​the ability to decode the operating point. Moreover, in some instances, the target device 40 may use a bit rate that is provided in the data structure to select a working point which is suitable for medium transport, for example, based on the bandwidth limitations of the medium transport, from which the target device 40 receives a bitstream.

[0188] When the target device 40 determines that the target device 40 is capable of decoding and playing of more than one operating point, the target unit 40 may select the operating point of the highest quality for decoding and playback. For example, the target device 40 may select operating point having the largest number of images, the highest bit rate, the highest frame rate, or other quality indicator for the working point to determine which operating point chosen.

[0189] After selecting the operating point of the target device 40 can fetch data for the operating point of the bit stream (220). That is, the target device 40 can extract the data for each image corresponding to the operating point of the program included in the bitstream. In some examples, the target device 40 selects data from one or more substreams of bits of the bitstream to extract the data for the operating point. After extracting the data destination device can decode and display data for the selected operating point (222). Video decoder 48 may decode each of the images that are to be decoded to the operating point, whereas the video output 44 can display each of the images that are to be displayed to the operating point. The displayed image may optionally be images which are decoded as described above.

[0190] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer-readable medium. Computer readable media may include computer-readable media, such as a storage medium and communication medium, including any medium that facilitates transfer of a computer program from one place to another. A data storage medium may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementing the methods described in this disclosure. By way of example, and not limitation, such computer-readable media can comprise RAM, read only memory, electrically erasable programmable read only memory, CD-ROM or other storage of the optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other medium which can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection may be termed a computer-readable medium. For example, if the instruction is transmitted from Web-site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber link (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be appreciated that computer-readable media, and storage media do not include compounds carrier waves, the signals intervening medium. An optical disc and a magnetic disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while optical discs usually reproduce data optically with lasers. Combinations of the above it should also be included within the scope of computer readable media.

[0191] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSP), general purpose microprocessors, application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other equivalent circuits integrated or discrete logic. Hence, the term "processor" as used herein, may refer to any of the foregoing structures or any other structure suitable for implementation of the methods described herein. Moreover, in some aspects described herein, the functionality may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined CODEC. Also, methods can be fully implemented in one or more circuits or logic elements.

[0192] The methods of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless headset, integrated circuit (IC) or a set of IC (eg, chipset). The different components, modules or units described in this disclosure as to emphasize functional aspects of the devices designed to perform the disclosed methods, but not necessarily require realization by different hardware units. Rather, as described above, various units may be combined into a codec hardware unit or provided by a group of interacting hardware blocks including one or more processors as described above, together with a suitable software and / or firmware.