Image encoding method, image decoding method, image encoding device, image decoding device, and image encoding/decoding device

Technical Field The present invention relates to a method for coding pictures for image coding and a method for decoding images for image decoding.

[Background of the Invention] [0002] Most of the current normalized video coding algorithms are based on hybrid video coding. In hybrid video coding methods, a series of lossless compression and loss compression schemes are used to achieve the desired compression increase. Hybrid video coding is also the basis for ITU-T standards (H.26x standards such as H.261, H.263) as well as ISO / IEC standards (MPEG-X standards such as MPEG-1, MPEG-2 and MPEG-standards 4).

[0003] The latest and most advanced video coding standard is currently the standard referred to as advanced video coding (AVC) H.264 / MPEG-4. This is the result of standardization efforts undertaken by the Joint Video Group (JVT, joint video team), a combined team of ITU-T and ISO / IEC MPEG groups.

[0004] In addition, the video encoding standard known as High Efficiency Video Coding (HEVC) is considered by the Common Collaborative Team on Video Coding (JCT-VC), with particular emphasis on improving efficiency in the field of high definition video coding.

[List of quoted items] [Non-patent literature] [0005]

Non-patent literature 1: C. Gordon, et al., "Wavefront Parallel Processing for HEVC Encoding and Decoding", JCTVC-F274-v2, from the meeting in Turin, July 2011, Internet <URL: <a href="http://phenix">http: // Phenix</a>. int-evry. fr.>

Non-patent literature 2: A. Fuldseth, et al., "Tiles", JCTVC-F355-v1, from the meeting in Turin, July 2011, July 2011, Internet <URL: <a href="http://phenix">http: // Phenix</a>. int-evry. fr.>

Non-patent literature 3: JCTVC-J1003_d7, "High efficiency video coding (HEVC) text specification draft 8", July 2012, page 73, "dependent_slice_flag", Internet <URL: <a href="http://phenix">http: // Phenix</a>. ITsudparis. eu / jct />

Non-patent literature 4: T. Lee, et al., "He dependent segment", JCTVC-J0217, from meeting in

Stockholm, July 2012, Internet <URL: <a href="http://wftp3">http: // wftp3</a>. itu. int / av-arch / ictvc-site>

Non-patent literature 5: T. Schierl, et al., "Dependent segments", JCTVC-10229, from the meeting in

Gene knows, April 2012, Internet <URL:<a href="http://wftp3.itu.int/av-arch/jctvc-site">http://wftp3.itu.int/av-arch/jctvc-site</a>>

[0006] Documents "Non-patent literature 4" and "Non-patent literature 5" introduce the concept of dependent segments. In particular, the "dependency segment permit marker" is placed in the parameter set of the syntactic part that is common to the segments, and the "dependent segment marker" is placed in the segment header, after the segment address parameter.

[Summary of invention] [Technical problem] [0007] There is, however, the problem that the image coding method, the image decoding method, etc. do not have sufficient processing efficiency.

[0008] Therefore, the present invention provides a method for coding images and a method for decoding images that are able to increase the processing efficiency.

[Solution]

This is achieved due to the characteristic features of the independent claims.

[0009] A picture coding method according to one aspect of the present invention is a picture coding method of performing coding processing through a sub-division of an image into a plurality of segments, the image coding method comprising a bit stream that includes: a dependent segment permission flag indicating whether the image contains or does not contain a dependent segment on which the coding processing is performed depending on the result of the encoding processing on a segment other than the current segment; segment address indicating the starting position of the current segment; and an indication of the relationship indicating whether the current segment is or is not a dependent segment, where the dependency marker on the dependent segment is placed in the parameter set common to the segments,

the segment's address is placed in the segment header of the current segment, and the dependency indication is placed in the segment header, and it is placed before the segment's address, and after the syntax element that identifies the set of parameters. [0011] The general and specific aspects disclosed above may be implemented by means of a system, method, integrated circuit, computer program or a readable computer recording medium, such as a CD-ROM, or any combination of systems, methods, integrated circuits, programs computer or readable computer storage media.

[Advantageous effects of the invention] [0012] The image coding method and the image decoding method according to the present invention is able to increase the coding efficiency.

[Brief Description of the Drawings] [0013] These and other objects, advantages and characteristics of the disclosure will become better apparent in the light of the following description taken in conjunction with the accompanying drawings that illustrate a particular embodiment of the present invention.

[FIG. 1]

FIG. 1 is a block diagram showing an example of a HEVC compliant encoder.

[FIG. 2]

FIG. 2 is a block diagram showing an example HEVC decoder.

[FIG. 3]

FIG. 3 is a diagram showing an exemplary image configuration in parallel wavefront parallel processing (WPP).

[FIG. 4]

FIG. 4 is a diagram showing an exemplary relationship between a normal segment and a dependent segment in parallel wave front processing.

[FIG. 5]

FIG. 5 is a diagram showing an exemplary header of a packet.

[FIG. 6]

FIG. 6 is a diagram showing an example of an entropy or dependent segment header.

[FIG. 7]

FIG. 7 is a diagram showing dependencies and signal transmission when using a regular segment.

[FIG. 8]

FIG. 8 is a schematic view showing dependencies and signaling when using a dependent segment and an entropy segment.

[FIG. 9A]

FIG. 9A is a schematic view showing an example depicting a syntactic implementation of inter-layer relationships, time dependencies, and intersegmental relationships in HM8.0.

[FIG. 9B]

FIG. 9B is a diagram explaining the steps of parsing to perform for parsing the inter-tiered relationships in HM8.0.

[FIG. 9C]

FIG. 9C shows a diagram explaining the steps of parsing to perform for parsing the inter-layer dependencies in HM8.0.

[FIG. 10]

FIG. 10 is a diagram showing an example of the location of the dependent_slice_flag flag.

[FIG. 11]

FIG. 11 is a diagram showing a syntax example when the parsing condition related to dependent_slice_enabled_flag in FIG. 10.

[FIG. 12]

FIG. 12 is a diagram showing a syntax example when dependent_slice_flag is moved before first_slice_in_pic_flag.

[FIG. 13]

FIG. FIG. 13 is a diagram showing a syntax example when dependent_slice_flag is moved before the slice_address syntax element.

[FIG. 14]

FIG. FIG. 14 is a diagram showing a syntax example when dependent_slice_flag is moved within a NAL header.

[FIG. 15]

FIG. 15 is a diagram showing an example of the dependent segment header syntax when a new type is added to the types of NAL units used for dependent segments.

[FIG. 16]

FIG. 16 is a diagram showing an example of the syntax of a segment header and a NAL entity header when it is assumed that the dependent_slice_flag is set to 1 for certain NALU types.

[FIG. 17]

FIG. 17 shows the general configuration of the content delivery system for the deployment of content distribution services;

[FIG. 18]

FIG. 18 shows the general configuration of the digital broadcasting system;

[FIG. 19]

FIG. 19 is a block diagram illustrating an example of TV configuration.

[FIG. 20]

FIG. 20 is a block diagram illustrating an example of a configuration of an information reproducing / recording module that reads and writes information from and onto a recording medium that is an optical disk;

[FIG. 21]

FIG. 21 is an example of a configuration of a recording medium that is an optical disk;

[FIG. 22A]

FIG. 22A illustrates an example of a cellular telephone;

[FIG. 22B]

FIG. 22B is a block diagram showing an example of a mobile phone configuration;

[FIG. 23]

FIG. 23 shows a structure of multiplexed data;

[FIG. 24]

FIG. 24 schematically shows a method for multiplexing each stream in multiplexed data;

[FIG. 25]

FIG. 25 shows a more detailed way of memorizing a video stream in a PES packet stream;

[FIG. 26]

FIG. 26 shows the structure of TS packets and host packets in multiplexed data; [FIG. 27]

FIG. 27 shows a PMT data structure;

[FIG. 28]

FIG. 28 shows the internal structure of multiplexed data information;

[FIG. 29]

FIG. 29 shows the internal structure of stream information attributes;

[FIG. thirty]

FIG. 30 shows the steps to identify video data;

[FIG. 31]

FIG. 31 is an example of a integrated circuit configuration for implementing a method of encoding moving pictures and a method for decoding moving images according to each of the embodiments;

[FIG. 32]

FIG. 32 is a configuration for switching between control frequencies;

[FIG. 33]

FIG. 33 shows the steps for identifying video data and switching between control frequencies;

[FIG. 34]

FIG. 34 is an example of a look-up table in which video data standards are associated with control frequencies;

[FIG. 35A]

FIG. 35A is a diagram showing an example of a configuration for sharing a signal processor module, [FIG. 35B]

FIG. 35B is a diagram showing another example of a configuration for sharing a signal processor module.

[Description of the form] (Background of the knowledge forming the basis of the present disclosure) [0014] Regarding the method of coding images and the image decoding method described in the background section of the invention, the inventors have discovered the following problem.

[0015] First, an image coding apparatus and an HEVC image decoding apparatus will be described.

[0016] The video signal inserted into the image coding apparatus is a sequence of images called frames (pictures). Each frame contains a two-dimensional pixel matrix. All of the aforementioned standards based on hybrid video encoding include dividing each single video frame into smaller blocks containing many pixels. The block size may vary, e.g. according to the content of the image. The coding method may typically vary with respect to the block. The largest possible size of such a block, eg in HEVC, is 64 x 64 pixels. It is called the largest coding unit (LCU). LCU can be recursively divided into 4 CU.

[0017] In H.264 / MPEG-4 AVC, a macroblock (usually denoting a block of 16 x 16 pixels) was the basic element of the image for which the coding is performed. The macroblock can also be divided into smaller subblocks. The coding steps included in the coding method and / or the decoding steps included in the decoding method are performed with respect to the sub-block.

[1-1. Hybrid video coding] [0018] Hereinafter, hybrid video coding is described.

[0019] Typically, the steps of encoding hybrid video encoding include spatial and / or temporal prediction (prediction in space and / or prediction over time). Accordingly, each coding block is first predicted either by using blocks in its vicinity, or blocks in its vicinity, i.e. from pre-encoded video frames. Next, the residual block is calculated as the difference between the coding block and its prediction result. Then the residual block is transformed from the spatial (pixel) domain into the frequency domain. The conversion aims to reduce the correlation of the input block. [0020] Furthermore, the coefficients of the transformation result obtained from the transformation are subject to quantization. This quantization is compression with losses (irreversible). Usually, the compressed values ​​of the transformation result coefficient are further losslessly compressed by entropy coding. In addition, the auxiliary information needed to reproduce the encoded video signal is encoded and supplied with the coded video signal. This is e.g. information about spatial prediction, temporal prediction and / or quantization.

[1-2. Configuration of the image coding device] [0021] FIG. 1 shows an example of a typical H.264 / MPEG-4 AVC and / or HEVC image encoder (encoder 100).

[0022] As shown in FIG. 1, the encoder 100 includes a subtracting circuit 105, a transformation module 110, a quantization module 120, a reverse transformation module 130, an adder 140, a filter 150, an adaptive loop 160, a frame 170, a prediction module 180 and an entropy coder 190.

[0023] The prediction module 180 controls the prediction signal s2 by time prediction or spatial prediction. The prediction type used in the prediction module 180 may be changed with respect to the frame or with respect to the block. Time-based prediction is called inter-prediction, and spatial prediction is called intra-prediction. The coding using the prediction signal s2 by time prediction is called inter-coding, and the coding using the prediction signal s2 by spatial prediction is called intra-coding. In the derivation of the prediction signal using time prediction, coded images stored in the frame's memory are used. In the derivation of the prediction signal using spatial prediction, the limit pixels of the encoded or decoded neighboring blocks stored in the memory are used. The number of prediction directions in the intraprediction depends on the size of the coding unit (CU). It should be noted that the details of the prediction will be described later.

[0024] The subtraction circuit 105 first determines the difference (prediction error e) between the current block to encode the input image (= input signal s1) and the corresponding prediction block (= prediction signal s2). The difference is used to predict the current coding block. It should be noted that the prediction error e is also called the residual prediction signal.

[0025] The transformer module 110 converts prediction error signal e into coefficients. Typically, the transformer module 110 uses an orthogonal transformation, such as a two-dimensional discrete cosine transformation (DCT) or a complete version thereof. Orthogonal transformation can reduce effectively the correlation of the input signal s1 (video signal before encoding). After conversion, lower frequency components are usually more important for image quality than high frequency components, so you can use more bits to encode low frequency components than high frequency components.

[0026] The quantization module 120 quantizes the coefficients and derives the quantized coefficients.

[0027] The entropy encoder 190 performs entropy coding on quantized coefficients. Quantized coefficients are losslessly compressed through entropy coding. In addition, through entropy coding, the amount of data stored in the memory and the amount of data (bit stream) to be transferred can be further reduced. The entropy coding is done mainly by using coding using a variable-length code word. The length of the codeword is chosen based on the probability of its occurrence.

[0028] The entropy coder 190 converts a two-dimensional matrix of quantized coefficients into a one-dimensional matrix. Typically, the entropy coder 190 performs the conversion through so-called zigzag scan. Zigzag scan begins with the DC factor in the top left corner of the two-dimensional matrix and scans the two-dimensional matrix in the order of preset, ending with the AC coefficient in the lower right corner. Energy is typically concentrated in the upper left part of the two-dimensional matrix of coefficients. Usually, when the coefficients are located in the upper left corner, they are the coefficients of the low frequency components. When the coefficients are located in the lower right corner, they are the coefficients of the high frequency components. Therefore, zigzag scan results in a matrix in which usually the last values ​​are successively many ones or zeros.

[0029] H.264 / MPEG-4 AVC and HEVC use different types of entropy coding. Although some syntactic elements are coded with a fixed length, most syntax elements are coded with variable length codes. In particular, among the syntax, context-adaptive variable length codes (CABACs) are used to encode prediction error signals (residual prediction signals). Typically, other integer codes, non-adaptive variable length codes, are used to encode other syntax elements. However, you can use adaptive contextually binary arithmetic coding.

[0030] Variable length codes enable lossless compression of the coded bit stream. However, since code words have a variable length, decoding must be performed sequentially on code words. In other words, code words can not be coded or decoded before encoding or decoding the previous code words without restarting (initializing) the entropy coding or without a separate indication of the position of the codeword (starting point) from which the decoding should begin.

[0031] Arithmetic coding encodes a sequence of bits into a single code word based on a predetermined probability model. This predetermined probability model is determined according to the content of the video sequence in the case of CABAC.

The arithmetic coding, and thus also the CABAC, is more efficient when the length of the bit stream to encode is greater. In other words, CABAC applied to bit strings is efficient for larger blocks. At the beginning of each string, CABAC restarts. In other words, at the beginning of each video sequence its probability model is initialized by means of certain predetermined or predetermined values.

[0032] The entropy coder 109 transmits, to the decoder side, a bitstream including encoded quantized coefficients (encoded video signals) and coded auxiliary information. [0033] H.264 / MPEG-4 and H.264 / MPEG-4 AVC, as well as HEVC, contain two functional layers, a video coding layer (Video Coding Layer, VCL) and a network abstraction layer (NAL). . VCL provides coding functions as described earlier. NAL encapsulates information elements in normalized units called NAL units according to their further use, such as channel transmission or storage in a memory device. The information elements closed by NAL are e.g. (1) an encoded prediction error signal (compressed video data) or (2) another information needed to decode a video signal, e.g.

[0034] Some entities that are not NAL VCL entities comprise e.g. parameter sets. A set of parameters is a set of parameters for encoding and decoding certain parts of a video string. For example, there is a sequence parameter set (SPS) that contains a parameter that is important for encoding and decoding a whole sequence of images. In particular, the set of string parameters is a syntactic structure containing syntactic elements. In particular, syntactic elements apply to zero or more entire encoded video strings, as determined by the seq_parameter_set_id content. seq_parameter_set_id is a syntax element in the image parameter set (described later) to which pic_parameter_set_id applies. pic_parameter_set_id is a syntax element contained in each segment header.

[0035] The picture parameter set (PPS) is a set of parameters that defines parameters used to encode and decode an image from a sequence of images (video sequence). In particular, PPS is a syntactic structure containing syntactic elements. Syntactic elements apply to zero or more entire coded images, as specified by pic_parameter_set_id, which is a syntax element in each segment header.

[0036] Accordingly, it is easier to track SPS than PPS. This is because the PPS changes for each image, while the SPS is constant for the whole video sequence, which can be as long as several minutes or hours.

[0037] The encoder 100 includes a reproduction module (so-called decoding module) which outputs the reconstructed signal (the so-called decoded signal) s3. By means of a reproduction module, a reproduced image obtained by reproducing (decoding) the encoded image is generated and stored in the memory 170 of the frame.

[0038] The reproduction module includes an inverse transformation module 130, an adder 140, a filter 150 and an adaptive loop filter 160.

[0039] The reverse transformation module 130, according to the coding steps described above, performs inverse quantization and inverse transformation. It should be noted that the prediction error e 'output from the reverse transformation module 130 differs from the prediction error e due to the quantization error, also called the quantization noise.

[0040] The adder 140 outputs the reconstructed signal s 'by adding the reconstructed prediction error signal e' reproduced by the reverse transformation module 130 to the prediction signal s2.

[0041] The non-blocking filter 150 performs the processing of the filter to reduce the quantizing noise that overlaps the reproduced signal s' due to quantization. Here, since the coding steps described above are performed with respect to the block, there is a case in which the block boundary is visible when noise is applied (noise blocking properties). The applied noise is called blocking noise. Especially when the quantization module 120 performs strong quantization, there are more visible block boundaries in the reproduced image (decoded image). Such blocking noise has a negative effect on visual perception by a human being, which means that the person experiences a deterioration of the image quality. To reduce blocking noise,

[0042] For example, in processing the H.264 / MPEG-4 AVC filter, corresponding filter processing is selected for each area. In the case of a high level of blocking noise, a strong (narrowband) low pass filter is used, while in the case of a low level of blocking noise a weaker (broadband) low pass filter is used. The strength of the low-pass filter is determined by the prediction signal e2 and the prediction error e 'signal. Processing of the filter unblocking usually smoothes the edges of the blocks. This leads to a better subjective picture quality of the decoded signals. The filtered image is used for motion-compensated prediction of the next image. Because filter processing also reduces prediction errors, you can improve coding efficiency.

The Adaptive Adaptive Adaptive Filter 160 uses adaptive offset processing (SAO) and / or adaptive loop filter (ALF) processing to the reproduced image s "after processing the unblocking filter in the filter 150 to output reconstructed signal (decoded signal) s3.

[0044] The processing of the unblocking filter in the unblocking filter 150 is intended to improve the subjective quality. At the same time, ALF processing and SAO processing in the adaptive loop filter 160 is intended to improve pixel fidelity ("objective" quality). SAO processing is used to add an offset value to a pixel value for each pixel using the pixel value of a directly adjacent pixel. ALF processing is used to compensate for image distortion caused by compression. The filter used in the ALF processing is typically a Wiener filter with filter coefficients set so that the mean square error (MSE) between the reproduced signal s' and the input signal s1 is kept to a minimum. The filter coefficients in the ALF processing are calculated and transmitted e.g. in relation to the frame. ALF processing can be applied to the entire frame (image) or to local areas (blocks). The help information indicating which areas are to be filtered can be sent on a block by block basis or on a quad tree basis after a quad tree.

[0045] The frame memory 170 (a frame buffer) stores part of the encoded and reproduced (decoded) image (the reproduced signal s3). The memorized reconstructed image is used to decode the inter-coded block.

[0046] The prediction module 180 derives the prediction signal s2 using the (same) signal that can be used on both the encoder side and the decoder side to maintain compatibility between the encoder side and the decoder side. The signal that can be used both on the encoder side and on the decoder side is the reproduced signal s3 (video signal after the filter is processed by the adaptive loop filter 160) on the encoder side, which is coded, then reproduced (decoded), and the reproduced signal s4 (video signal after the filter is processed by the adaptive loop filter in FIG. 2) at the decoder side, which is decoded from the bit stream.

[0047] When the prediction module 180 generates the prediction signal s2 by inter-coding, it performs the prediction using the motion compensation prediction. The prediction module's motion estimator 180 (not shown) finds the best-matching block for the current block among blocks within previously coded and reconstructed video frames. The best matching block then becomes a prediction signal. Relative displacement (movement) between the current block and its best matching block is then signaled as motion data contained in the auxiliary information in the form of three-dimensional motion vectors. The signal is transmitted together with the encoded video data. The three-dimensional motion vector contains two motion vectors with a spatial dimension and one motion vector with a time dimension. To optimize the accuracy of predictions, motion vectors can be determined with spatial sub-pixel resolution, e.g. with a resolution of half or quarter of a pixel. A motion vector with sub-pixel spatial resolution may point to a space in space within an already recreated frame, where no real pixel value is available, i.e. to a sub-pixel place. Therefore, you need spatial interpolation of such pixel values ​​to perform prediction with motion compensation. This can be achieved by means of an interpolation filter (integrated in the prediction module 180 in FIG. 1). to the sub-pixel place. Therefore, you need spatial interpolation of such pixel values ​​to perform prediction with motion compensation. This can be achieved by means of an interpolation filter (integrated in the prediction module 180 in FIG. 1). to the sub-pixel place. Therefore, you need spatial interpolation of such pixel values ​​to perform prediction with motion compensation. This can be achieved by means of an interpolation filter (integrated in the prediction module 180 in FIG. 1).

[1-3. Configuration of the image decoding apparatus] [0048] The configuration of the decoder (image decoding apparatus) will be described with reference to FIG. 2.

[0049] FIG. 2 is a block diagram showing an example decoder 200 according to the video coding standard H.264 / MPEG-4 AVC or HEVC.

[0050] As shown in FIG. 2, decoder 200 includes entropy decoder 290, reverse transform module 230, combiner 240, filter 250, adaptive loop filter 260, frame memory 270 and prediction module 280.

[0051] The bit stream inserted into the decoder 200 (coded video signal) is first sent to the entropy decoder 290.

[0052] The entropy decoder 290 extracts the encoded quantized coefficients and the coded auxiliary information from the bit stream, and decodes the encoded quantized coefficients and coded auxiliary information. As described above, the auxiliary information includes information needed for decoding, such as traffic data (motion vector) and prediction mode (prediction type).

[0053] The entropy decoder 290 converts the decoded quantized coefficients in a one-dimensional matrix into coefficients in a two-dimensional matrix by inverse scanning. The entropy decoder 290 introduces into the module 230 inverse transformation quantized coefficients after their transformation into coefficients in a two-dimensional matrix.

[0054] Reverse transformer module 230 performs inverse quantization and inverse transformation on quantized coefficients transformed into coefficients in a two-dimensional matrix to output prediction error e '. The prediction error e 'error corresponds to the differences obtained by subtracting the prediction signal from the signal input to the encoder in the case where no quantization noise has been introduced and an error has not occurred.

[0055] The prediction module 280 controls the prediction signal s2 by means of time prediction or spatial prediction. Information, such as the type of prediction included in the auxiliary information, is used in the case of intra-prediction (spatial prediction). In addition, information such as traffic data contained in the auxiliary information is used in the case of predictions with motion compensation (inter-prediction, time prediction).

[0056] The adder 240 adds the prediction error e 'signal received from the inverse transform module 230 and the prediction signal e2 obtained from the prediction module 280 to output the reproduced signal s'.

[0057] The non-blocking filter 250 performs the processing of the filter on the reconstructed signal s'. The adaptive loop filter 260 uses SAO processing and ALF processing to the reconstructed signal s "to which the filtering operation is applied by the filter 250. The decoded signal S4 obtained by using SAO processing and ALF processing in the adaptive loop filter 260 is stored in the memory 270 The decoded signal S4 stored in the frame memory 270 is used in the prediction module 280 to predict the next current decoding block or the current image to be decoded.

[1-4. Processing efficiency] [0058] Parallel processing is usually considered to improve processing efficiency in the coding and decoding field.

[0059] Compared to H.264 / MPEG-4 AVC, HEVC has the function to support high-level parallel processing (parallel processing) of coding and decoding. In HEVC it ​​is possible to divide the frame into segments, similar to H.264 / MPEG-4 AVC. Here, the segments are LCU groups in the order of scanning. In H.264 / MPEG-4 AVC, segments can be decoded independently and spatial prediction between segments is not used. Therefore, parallel processing can be carried out on a segment by segment basis.

[0060] However, the segments have much larger headers and there is no relationship between the segments, and the efficiency of compression decreases. In addition, CABAC coding loses efficiency when applied to small blocks.

[0061] To enable more efficient parallel processing, parallel wavefront processing (WPP) is proposed. WPP maintains a constant dependency that is different from parallel processing in which each segment is independent.

[0062] A further description will be given with reference to the case where the image comprises an LCU, in each of which the images are arranged in a matrix, and each line of the LCU comprises one segment (see FIG 3). In WPP, among LCUs containing current LCU row 23, as the CABAC probability model for CABAC state reset of the first LCU (front LCU), the CABAC probability model is applied immediately after processing on the second LCU of the previous row 31 LCU. All inter-block dependencies are maintained. This enables parallel decoding of LCU rows. The timing of each LCU line start processing is delayed by two LCUs compared to the previous one. The start point information for decoding the LCU line is included in the segment header. WPP is described in detail in non-patent literature 1.

[0063] Another approach to improving the parallelism of processing is called "tiles." So the frame (picture) is divided into tiles. Tiles are rectangular LCU groups. The boundaries between the tiles are set so that the whole picture is divided into a matrix. Tiles are processed in the order of raster scanning.

[0064] All dependencies are interrupted at the tile boundaries. The entropy coding such as CABAC is also reset at the beginning of each tile. Only the processing of the filter and the adaptive sample shifting can be applied across the borders of the tile. Thus, tiles can be coded and decoded in parallel. Tiles are described in detail in non-patent literature 2 and non-patent literature 3.

[0065] Furthermore, to improve the concept of segments and make it suitable for parallel processing rather than for fault tolerance, which was the primary purpose of the segments in H.264 / MPEG-4 AVC, the concept of dependent segments and entropy segments is proposed.

[0066] In other words, in HEVC, three types of segments are supported: (1) ordinary segments, (2) entropy segments, and (3) dependent segments.

[0067] Normal segments mean segments already known from H.264 / MPEG-4 AVC. Spatial prediction is not allowed between normal segments. In other words, prediction across segment boundaries is not allowed. This means that the regular segment is coded without reference to any other segment. To enable independent decoding of such segments, CABAC restarts at the beginning of each segment.

[0068] When the segment to be processed is a regular segment, the CABAC restart includes post processing (termination processing) of arithmetic coding or arithmetic decoding processing at the end of the previous segment and processing of context table (probability table) initiation to the default value at the beginning of the regular segment.

[0069] Normal segments are used at the beginning of each frame. In other words, each frame should start with a regular segment. The regular segment has a header containing the parameters needed to decode the segment data.

[0070] The term "entropy segments" means segments in which spatial prediction is permissible between a parent segment and an entropy segment. The analysis of the parent segment and the entropy segment is performed independently.

[0071] However, the parent segment is e.g. a normal segment immediately before the entropy segment. The parent segment is required to reproduce the pixel values ​​of the entropy segment. To enable the independent analysis of entropy segments, CABAC also restarts at the beginning of the segment. As segment header for entropy segments, you can use a segment header that is shorter than the segment header. The entropy segments segment header contains a subset of coding parameters in relation to information sent in the segment header. Missing elements in the header of the entropy segment are copied from the header of the parent segment.

When the segment to be processed is an entropy segment, the CABAC restart, similar to the ordinary segment, includes post processing (end of processing) at the end of the previous segment and processing of the context table (probability table) to the default value at the beginning of the current segment.

[0073] (3) The dependent segment is similar to the entropy segment, but differs partially in terms of processing in which CABAC is restarted.

[0074] When the processing segment is a dependent segment and the WPP does not work, the CABAC restart includes post-processing in the previous segment (termination of processing), and the context table initiation processing to the end-state value of the previous segment. When the segment to be processed is a dependent segment and the WPP does not work, the CABAC restart includes final processing in the previous segment (processing completion), and the context table initialization processing to the status value after processing the LCU that belongs to the previous segment and is the second from the left end on beginning of the current segment. [0075] As described before, the CABAC restart always includes termination of processing. Conversely, in the CABAC restart, the state of CABAC is often transferred.

[0076] Dependent segments can not be analyzed without a home segment. Therefore, dependent segments can not be decoded if the parent segment is not received. The parent segment is usually the segment preceding the dependent segments in the coding order, and the segment containing the complete segment header. The same applies to the home segment of the entropy segment.

[0077] As described before, dependent entropy segments use a segment header (especially segment header information that is missing in the dependent segment header) of the immediately preceding segment according to the coding order of the segments. This rule is applied recursively. The parent segment of the current dependent segment depends on whether it is recognized as available for reference. The reference includes the use of spatial prediction between segments, sharing of CABAC states, etc. The dependent segment uses CABAC context tables that are generated at the end of the immediately preceding segment. Thus, the dependent segment does not initialize CABAC tables to default values, but instead continues using already developed context tables.

[0078] HEVC provides a series of profiles. The profile includes some settings of the image coding apparatus and an image decoding apparatus suitable for a particular application. For example, the "main profile" contains only normal and dependent segments, but not entropy segments.

[0079] As described before, the encoded segments are further encapsulated in NAL units, which in turn are encapsulated eg in the Real Time Protocol (RTP) and finally in Internet Protocol (IP) packets. Either this or other protocol stacks make it possible to transmit encoded video in packet-oriented networks, such as the Internet or some private networks.

[0080] Networks typically include at least one or more routers using special hardware that operates very quickly. The router's function is to receive IP packets, analyze their IP packet headers and, according to the analysis result, send IP packets to their respective destinations. Because routers must support traffic from multiple sources, the logic of packet support must be as simple as possible. The minimum requirement for the router is to check the destination address field in the IP header to determine the path to send it. In addition to providing quality of service (QoS) support, intelligent (media-aware) routers check additional fields in the network protocol headers, such as in the IP header, RTP header, and even in the NALU header.

[0081] As can be seen from the above video encoding description, the different segment types defined for parallel processing purposes, such as dependent segments and entropy segments, have different validity with respect to the quality of the distortion after their failure. In particular, dependent segments can not be analyzed and decoded without a home segment. This is because the encoder or entropy decoder can not be restarted at the beginning of the dependent segment. Thus, the parent segment is more important for image or video playback.

[0082] In HEVC, the dependent and entropy segments introduce an additional dimension of the relationship, namely the intersegmental relationship (the dependence within the frame). This type of dependency is not taken into account by routers.

[0083] The relationships described above, in particular the inter-segment relationship, are not taken into account at the network level. However, it would be desirable to take into account the relationship described above at the network level to provide better support for the quality of services. Thus, there is a need to improve the flexibility of packet manipulation at the network level by taking into account segment dependencies.

(Problem details) [1-5. WPP and dependent segment] [0084] Dependent segments can be used together with parallel processing tools, such as wave front wave (WPP) processing and tiles. In particular, dependent segments allow the wavehead (sub-stream) to reduce the transmission delay without causing a loss of coding.

[0085] In addition, the dependent segments serve as starting points for the CABAC sub-streams because CABAC is not restarted in the dependent segments. In addition, the information indicating the starting points can be transmitted in the bit stream to provide starting points for an optional independent analysis. Especially when more than two sub-streams of CABAC are encapsulated in a regular or dependent segment, the starting points are signaled explicitly in the form of a number of bytes per substream. Here, the substream means the part of the flow that is analytically independent thanks to the starting points. In addition, dependent segments can be used as "tags" starting points, because each dependent segment must have a NAL header. This means that starting points can be signaled in relation to such markers.

[0086] These two approaches, namely explicit signaling of starting points and marking of starting points through dependent segments, are used together.

[0087] As a rule, the starting point of each NAL unit (the beginning of each NAL header) must be identifiable. There is no requirement for a precise identification operation. For example, the following two methods can be used.

[0088] The first method consists in placing the initial code (e.g., 3 bytes long) at the beginning of each NAL header. The second way is to place each NAL unit in a separate package. Thanks to segment dependencies, you can reduce the size of the segment header.

[0089] As regards the entropy segments, the method allows parallel CABAC analysis. This is because CABAC is actually restarted at the beginning of entropy segments. In the case of parallel CABAC processing, CABAC presents a bottleneck that can be overcome by a parallel CABAC analysis followed by sequential pixel reproduction operations. In particular, the WPP parallel implementation tool enables the decoding of each LCU line by one processing core (intellectual property code (IP core), function block). It should be noted that assigning LCU rows to the cores may be different. For example, you can assign two lines to one core and one row can be assigned to two cores.

[0090] FIG. 3 is a diagram showing an exemplary image configuration 300. In FIG. 3, the image 300 is divided into 31 to 3m (m is the ordinal number LCU) of the largest coding units (LCU) rows. Each of the lines LCU 3i (I = 1 to m) contains LCU 3i1 to 3in (n is the ordinal number of the LCU column), which are arranged in a row. Line 3i LCU corresponds to "Wave Corp". Parallel processing can be performed for the wave fronts. The CABAC status arrow in FIG. 3 means the relationship between the LCU relating to the CABAC state and the reference purpose. [0091] More specifically, in FIG. 3, first, from the LCUs included in row 31 of the LCU, processing (coding or decoding) starts for the leading LCU 311. Processing on the LCU is performed in order from LCU 311 to 31n. After the processing of the first two LCUs 311 and 312, in line 31 of the LCU, processing is started on row 32 of the LCU. In processing the first LCU 321 in a row from LCU column 32, as indicated by the CABAC status arrow in FIG. 3, the CABAC state immediately after processing on LCU 312 in line 31 LCU in the first row is used as the initial state of CABAC. In other words, there is a delay of two LCUs between two parallel processing.

[0092] FIG. 4 is a diagram showing an exemplary case when using a dependent segment using WPP. Rows 41 to 43 of the LCU correspond respectively to "wave front 1", "wave front 2" and "wave front 3". Rows 41 to 43 of the LCU are processed by their respective independent cores. In FIG. 4, line 41 LCU is a regular segment, and rows 42 to 4m LCU are dependent segments.

[0093] Dependent segments make WPP capable of reducing latency. Dependent segments do not have a complete segment header. In addition, dependent segments can be decoded independently of other segments, as long as only the starting points are known (or the starting point of the dependent segments, which is known according to the principle described above). In particular, dependent segments can make WPP suitable also for low latency applications without causing coding losses.

[0094] In the ordinary case of sub-masks (LCU rows) in segments, it is mandatory to insert explicit starting points into the segment header to ensure parallel entropy coding and decoding. As a result, the segment is ready to send only after full encoding of the last sub-stream of the segment. The segment header is complete only after encoding all sub-streams in the segment. This means that the forwarding of the beginning of the segment can not begin by fragmenting the packet in the RTP / IP layer until the entire segment is completed.

[0095] However, since dependent segments can be used as starting point markers, explicit signaling of starting points is not required. Therefore, you can divide a regular segment into many dependent segments without coding loss. Dependent segments can be sent as soon as the encapsulation of the sub-mask has been completed (or even earlier in the case of fragmentation of the packet).

[0096] The dependent segments do not interrupt spatial prediction dependencies. Dependent segments do not even break the analytical dependency. This is because usually the analysis of the current dependent segment requires CABAC states from the previous segment.

[0097] When dependent segments are not allowed, then each LCU line can be configured to be a segment. Such a configuration reduces the transmission delay, but at the same time leads to rather high coding losses, as discussed in the previous section on the background of the invention.

[0098] Alternatively, the entire frame (picture) encapsulates in a single segment. In this case, the starting points for the sub-streams (LCU rows) must be signaled in the segment header to allow their parallel analysis. As a result, there is a transmission delay at the frame level. In other words, the header must be modified after encoding the entire frame. The encapsulation of the whole image in a single segment does not increase the transmission delay itself. For example, the transmission of some parts of the segment may start already before the completion of the entire coding. However, if WPP is used, then the segment header should then be modified to save the starting points. Therefore, this entire segment must be delayed for transmission.

[0099] The use of dependent segments therefore allows reducing the delay. As shown in FIG. 4, the image 400 is divided into line 41 LCU, which is a regular segment, and rows 42 to 4m LCU, which are dependent segments. When each LCU line is one dependent segment, one LCU transmission delay can be obtained without any coding loss. This is due to the fact that the dependent segments do not interrupt any spatial dependence and do not restart the CABAC engine.

[1-6. Package Configuration] [0100] As described above, network routers need to analyze packet headers to allow quality of service. The quality of the service varies depending on the type of application and / or service priority and / or the importance of the packet to the distortion caused by its packet loss.

[0101] FIG. 5 is a diagram showing an example of encapsulation (packet) of a bit stream.

[0102] In general, a real-time protocol (RTP) is used for packaging. RTP is usually used for real-time media transmission. The length of the headers of the respective protocols used is substantially constant. The protocol headers have extension fields. The extension fields can increase the length of the headers by 4 bytes. For example, an IP header can be extended to 20 bytes. The syntactic elements of IP headers, User Datagram Protocol (UDP) and RTP also have a fixed length.

[0103] FIG. 5 shows the packet header 500 included in the IP packet. The packet header shown in FIG. 5 includes an IP 510 header, UDP header 530, a 540 RTP header, a RTP content header H264, and a NIF header 570. The 510 IP header is a 20 byte long header with an extension field of 520, 4 bytes long. The content of the IP packet is a UDP packet. The UDP packet contains a UDP header 530 with a length of 8 bytes and UDP content. The UDP content is created by the RTP packet. The RTP packet includes a 12-byte 540 RTP header and a 550 byte extension field of 4 bytes. The RTP package can be selectively expanded using the extension field. The content of the RTP packet contains a special header 560 RTP content H264 with a length from 0 to 3 bytes followed by a header 570 NAL HEVC, which is 2 bytes long.

[0104] Routers that are capable of providing a higher quality of service are called media network elements (Media Aware Network Elements, MANE). The media serving network elements check some of the header fields of the packets shown in FIG. 5. For example, MANE is called "temporal_id" and contains in header 570 NAL or the decode order number contained in the Router header 540 can be checked to detect losses and the order in which content of received packets is presented. Routers (network elements) support packets as quickly as possible to allow high bandwidth in the network. Logic is required to access package headers quickly and simply to keep the processing complexity of the network elements low.

[0105] The NALU is closed by a header 500. The NALU may include segment data when the segment header is present.

[0106] FIG. 6 is a diagram showing an example of the syntax 600 of the segment header. The syntax element, dependent_slice_flag 601 is a syntax element that indicates whether the segment is a dependent segment or not. This syntax element can be used to identify intersegment dependencies. However, the segment header is the content of NALU. Analyzing syntax elements before dependent_slice_flag 601 requires rather complex logic. This is a level that can not effectively take into account ordinary routers, as will be shown later.

[0107] As described before, the NALU includes information common to several segments, such as parameter sets, or includes directly encoded segments with information necessary for decoding included in the segment header. The syntax of the segment header used for the entropy or dependent segment is shown, for example, in FIG. 6. FIG. 6 shows the table with the segment header structure. When the "dependent_slice_flag" syntax element is set to 1, all segments are required up to the first regular segment (a segment that is not an entropy segment or a dependent segment) preceding the current segment in the decoding order. When segments are not decoded, you can usually not decode the current dependent segment. For example, in some specific cases, a dependent segment can be decoded, when some other side, signal or derivative information is available. The dependent_slice_flag 601 syntax element is contained approximately in the middle of the segment header. In addition, the segment header includes the number of CABAC sub streams within the current segment indicated by the num_entry_point_offsets 602 information element and the number of bytes in the substream indicated by the entry_point_offset syntax element [i] 603. Here, the num_entry_point_offsets 602 information element corresponds to the number of entry points. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream. The dependent_slice_flag 601 syntax element is contained approximately in the middle of the segment header. In addition, the segment header includes the number of CABAC sub streams within the current segment indicated by the num_entry_point_offsets 602 information element and the number of bytes in the substream indicated by the entry_point_offset syntax element [i] 603. Here, the num_entry_point_offsets 602 information element corresponds to the number of entry points. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream. The dependent_slice_flag 601 syntax element is contained approximately in the middle of the segment header. In addition, the segment header includes the number of CABAC sub streams within the current segment indicated by the num_entry_point_offsets 602 information element and the number of bytes in the substream indicated by the entry_point_offset syntax element [i] 603. Here, the num_entry_point_offsets 602 information element corresponds to the number of entry points. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream. In addition, the segment header includes the number of CABAC sub streams within the current segment indicated by the num_entry_point_offsets 602 information element and the number of bytes in the substream indicated by the entry_point_offset syntax element [i] 603. Here, the num_entry_point_offsets 602 information element corresponds to the number of entry points. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream. In addition, the segment header includes the number of CABAC sub streams within the current segment indicated by the num_entry_point_offsets 602 information element and the number of bytes in the substream indicated by the entry_point_offset syntax element [i] 603. Here, the num_entry_point_offsets 602 information element corresponds to the number of entry points. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream. In addition, i is an integer and an index denoting special entry points (offsets of entry points). The number of bytes in the substream indicated by entry_point_offset [i] 603 allows easy navigation in the bit stream.

[1-7. Image dependency] [0108] As described earlier, there are several types of dependencies resulting from the HEVC coding approach.

[0109] FIG. 7 is a diagram showing dependencies and their indication when only ordinary segments are used, i.e. there are no dependent or entropy segments. FIG. 7 shows three images 710, 720 and 730.

[0110] Image 710 is an image of a base layer carried in two VCL NALUs, namely unit 1 of VCL NAL and unit 2 of VCL NAL. POC indicates the order in which the images are to be rendered. VCL NALU contains a syntax element indicating whether the image belongs to the base layer or to the enhancement layer, and the syntax element temporal_id. A syntax element indicating whether the image belongs to the base layer or to the enhancement layer is transmitted in a state within the header Nos. 570 of the header 500 of the packet shown in FIG. The syntax element "temporal_id" is also sent in the state inside the 570 NAL header. The syntax element "temporal_id" indicates the degree of dependence of other images. For example, images or segments coded with temporal_id = 0 can be decoded independently of other images / segments having a higher temporal_id. It should be noted that in HEVC, temporal_id is indicated in the NAL header as nuh_temporal_id_plus1 (see FIG 9A). In particular, the following expression 1 can be applied to the relationship between the temporal_id used in these examples and the syntax element nuh_temporal_id_plus1.

[Equation 1] temporal_id = nuh_temporal_iD_plus1 - 1 (expression 1) [0111] Segments with temporal_id = 1 depend on segments having a lower temporal_id value. In other words, the temporal_id value in this case is 0. In particular, syntax temporal_id syntax refers to the image prediction structure. In general, segments with a specific temporal_id value depend only on segments having a lower or equal temporal_id value.

[0112] Thus image 710 and FIG. 7 can be decoded first.

[0113] The image 720 is an enhancement layer for the image layer 710. There is therefore a dependence that requires image decoding 720 after decoding the image 710. Image 720 contains two NALUs, namely the 3 VCL NAL unit and the 4 VCL NAL unit. Both images 710 and 720 have their POC values ​​equal to 0. This means that images 710 and 720 belong to the same image to be displayed immediately. The images contain a base layer and an improvement layer.

[0114] Image 730 is a base layer that comprises two NALUs, namely a 5 VCL NAL unit and a 6 VCL NAL unit. The image 730 has a POC value of 1. This means that the image (part) 730 is to be displayed after the pictures 720 and 710. In addition, the image 730 has the value temporal_id = 1. This means that the image 730 temporally depends on the image having temporal_id = 0. Thus, based on the relationship indicated in the NAL header, the image 730 depends on the image 710.

[0115] FIG. 8 is a diagram showing dependencies (degree of dependence) and their indication in the case when dependent and entropy segments are used. FIG. 8 shows three images 810, 820 and 830. FIG. 8 differs from the FIG. 7 in that add dependencies of entrapped and sub segments are indicated in the segment header.

[0116] In FIG. 7 shows the interlayer dependence on the example of images 710 and 720. In addition, the time dependence is shown on the example of images 710 and 730. Both of these dependencies are indicated in the NAL header.

[0117] An intersegment dependency such as shown in FIG. 8, is inherent for dependent and entropy segments. In particular, both frames, the base layer frame 810 and the improvement layer frame 820 have two segments. Of these two segments, one is the parent segment (regular segment) and the other is the child segment (dependent segment). In box 810, the unit segment 1 VCL NAL is the parent segment of the 2 VCL NAL unit. In Box 820, the 3 VCL NAL unit segment is the parent segment of the 4 VCL NAL unit. As described earlier, the term "parent segment" referring to a dependent segment refers to the segment on which the dependent segment depends, i.e. to the segment whose information from the segment header is used by the dependent segment. It is a rule that the first preceding segment is a segment, which has a complete header. For example, a segment that has a complete header is a regular segment, not a dependent segment.

[0118] A suitable syntax for the header of the NAL unit and segment header currently used in HEVC, and in particular in HM8.0, will be described with reference to FIG. 9A.

[0119] FIG. 9A is a diagram showing the syntax of a 910 header of a NAL unit and the syntax of a segment header 920. In particular, it is planned to indicate inter-layer relationships (in current standardization) in the header of the NAL unit using the syntax element nuh_reserved_zero_6bits. Time dependencies are indicated by the syntactic element nuh_temporal_id_plus1. Segment header 920 contains a signal indicating the intersegment dependency ratio. The intersegment dependency index is an element of the syntax dependent_slice_flag. In other words, the intersegment dependency (eg time dependency) is signaled in the segment header, somewhere in the segment header.

[0120] To parse this syntax element, all syntactic elements preceding the dependent_slice_flag must be parsed as well as the syntactic elements of the parameter set needed to analyze the segment header elements preceding the dependent_slice_flag.

[1-8. Router processing] [0121] As described earlier, in determining motion shaping it is desirable to take into account the dependencies introduced by dependent and entropy segments, in addition to the relationships signaled in the NAL header. For example, the router may be implemented as a mobile telephony base station of the MANE type. The bandwidth in the connection between the satellite and the station is very limited and must be managed very carefully. It can be assumed that the following example case. It can be assumed that the packet is randomly omitted earlier by an ordinary router. In this case, the media serving network element (MANE) discovers the packet loss by checking the package number. After checking the loss of the packet, MANE skips all packets that depend on the packet that is skipped and follow it. This is a characteristic feature desired for network elements that support media. In this way, packages can be skipped more intelligently. When the router decides to omit the NAL unit, it can immediately conclude that the next dependent segments should also be omitted. In the current syntax introduced in FIG. 9A access to the dependent_slice_flag requires the analysis of a significant amount of information. This is not important for routing packets or shaping traffic operations on routers. All the information needed to discover inter-layer and temporal relationships can be found in the video parameter set. The set of video parameters is the highest set in the hierarchy of parameter sets. When the router decides to omit the NAL unit, it can immediately conclude that the next dependent segments should also be omitted. In the current syntax introduced in FIG. 9A access to the dependent_slice_flag requires the analysis of a significant amount of information. This is not important for routing packets or shaping traffic operations on routers. All the information needed to discover inter-layer and temporal relationships can be found in the video parameter set. The set of video parameters is the highest set in the hierarchy of parameter sets. When the router decides to omit the NAL unit, it can immediately conclude that the next dependent segments should also be omitted. In the current syntax introduced in FIG. 9A access to the dependent_slice_flag requires the analysis of a significant amount of information. This is not important for routing packets or shaping traffic operations on routers. All the information needed to discover inter-layer and temporal relationships can be found in the video parameter set. The set of video parameters is the highest set in the hierarchy of parameter sets. All the information needed to discover inter-layer and temporal relationships can be found in the video parameter set. The set of video parameters is the highest set in the hierarchy of parameter sets. All the information needed to discover inter-layer and temporal relationships can be found in the video parameter set. The set of video parameters is the highest set in the hierarchy of parameter sets.

[0122] Thus, the information described above is signaled in the NAL header 570. However, for the NAL header and segment header shown in FIG. 9A access to segment dependency information requires tracking additional parameter sets, such as PPS and SPS. This, again, uses the capabilities of network gateways or routers that support media. As can be seen from FIG. 9A, segment header 920 must be parsed to the dependent_slice_flag and the parameters analyzed are useless for network activity.

[0123] To be able to parse the address of the segment preceding the dependent_slice_flag, the following syntactic elements are required from among the syntax elements contained in SPS930, as shown in FIG. 9B. FIG. 9B is a diagram showing the syntax example contained in SPS.

• pic_width_in_luma_samples (reference 931 in FIG 9B) • pic_height_in_luma_samples (reference 932 in FIG 9B) • log2_min_code_block_size_minus 3 (reference 933 in FIG 9B) • log2_diff_max_min_code_block_size (reference 934 in FIG 9B) [0124] These parameters are shown in the right table from FIG. 9B and they are needed to get the slice_address parameter. The slice_adress syntax element is encoded with a variable length (as can be seen by looking at the length "v" in the descriptor, the second column, slice_address and segment header 920 in FIG 9A). To know the length of this parameter encoded with variable length, these syntactic elements from SPS are needed. In fact, to be able to analyze dependent_slice_flag, the actual value of the slice_address syntax element is not needed. You only need to know the length of the syntactic element,

[0125] Therefore, the SPS should be analyzed up to the point 935 from the syntax elements in SPS 930, as shown in FIG. 9B. These four syntactic elements should be remembered. They are later used in the formula to calculate the length of the slice_address syntax element.

[0126] In addition, to access the dependent_slice_enabled_flag also preceding the prior_slice_flag, the PPS should be parsed to the point 945 from the PPS syntax elements shown in FIG. 9C. FIG. 9C is a diagram showing an example of the syntax contained in the PPS. It should be noted that syntactic elements whose methods of analysis have been described with reference to FIG. 9A to 9C, which are located in the segment header as well as SPS and PPS, are not required for ordinary routine operations. In addition, some of the syntax elements can not simply be omitted because some of the syntax elements are coded with variable-length codes. Thus, even if a jump is made by a predefined number of bits in the bitstream, no jump is possible until the dependent_slice_enabled_flag is possible.

[0127] In other words, to read the dependent_slice_flag (dependency display), MANE must go further in dividing the header (see segment header 920), whose analysis is rather complicated.

[0128] In particular, the first_slice_in_pic_flag flag should be analyzed. The first_slice_in_pic_flag tag is a marker that indicates whether the segment is the first segment in the image or not.

[0129] Next, no_output_of_prior_pics_flag, whose presence depends on the type of NALU to be analyzed, should be analyzed.

[0130] Furthermore, the coded variable length pic_parameter_set_id has to be decoded. The syntax element pic_parameter_set_id is a syntax element indicating which of the parameter sets is used (the element of the syntax identifying the set of parameters). By analyzing pic_parameter_set_id, you can identify the set of parameters to use.

[0131] Finally, a slice_address syntax element is required. The slice_address syntax element is a syntax element that indicates the starting position of the segment. This syntax element also requires analyzing PPS and SPS as well as additional calculations.

[0132] As a last step, the value of the dependent_slice_enabled_flag (dependency tag) should be obtained from the PPS to know whether the dependent_slice_flag is present in the bit stream or not. When dependent_slice_enabled_flag = 0, this means that the current segment is a regular segment, because dependent segments are not allowed. To get the value of dependent_slice_enabled_flag, it is required to analyze PPS approximately to its center.

[0133] Unfortunately, the syntax elements before the dependent_slice_flag can not be omitted and should be parsed in contrast to the RTP and NAL header data in which the data location is predefined. This is due to the fact that syntactic elements in the segment header are coded with variable length. Therefore, the presence and length of elements must be calculated for each VCL NAL unit. In addition, additional session data must be saved because they are needed later (see PPS and SPS). In addition, the presence of some elements of syntax depends on the presence or value of other syntax elements possibly contained in other parameter structures (syntactic elements are conditionally coded).

[0134] In current standardization, there is a proposal to signal the structure of a video string relationship in a Video Parameter Set (VPS) that describes how many layers are contained in a bitstream and in dependency indicators to indicate particular inter-layer relationships. The VPS is signaled at the very beginning of the video, before the first SPS. Many SPS can refer to a single VPS. This means that VPS carries information that is important for many video strings. The main purpose of the VPS is to inform the router or decoder of the video content containing the information. How many video strings exist and how they are related to each other. SPS is valid only within the video string, while VPS carries information about multiple video strings.

[0135] Furthermore, the characteristics of the information carried in the VPS are particularly informative for routers. For example, VPS can carry information that is required to set up a streaming session because the project is not being finalized. The router analyzes information in VPS. Router, without the need to use other parameter sets (by browsing only NAL headers), can specify which data packets to send to the decoder and which to skip.

[0136] However, in order to discover the currently active VPS, the following steps must be followed:

analyzing PPS_id in the segment header;

analyzing SPS_id in the active PPS determined by PPS_id, and analyzing VPS_id in the active SPS determined by SPS_id.

[0137] To solve the above problem, a picture coding method according to one aspect of the present invention is a picture coding method of performing coding processing by dividing an image into a plurality of segments, the method of encoding the images comprising transmitting a bit stream that includes: a dependent segment permission flag whether the image contains or does not include a dependent segment on which the coding processing is performed depending on the result of the encoding processing on a segment other than the current segment; segment address indicating the starting position of the current segment; and indication of dependency (dependent_slice_flag) informing whether the current segment is or is not a dependent segment,

[0138] In the above image coding method, the dependency indication regarding the intersegment dependency is located in a position suitable for analysis by the router. Thanks to this, you can encode a syntax element indicating the dependence independently of other syntax elements, in other words unconditionally.

[0139] For example, the dependency display may be included in the bit stream when the dependent segment enable flag indicates the inclusion of the dependent segment.

[0140] For example, the dependent segment permission flag may be placed at the beginning of a set of parameters.

[0141] For example, each of the segments may comprise a plurality of macroblocks, and the coding processing on the current segment may start after performing the coding processing on two of the macroblocks included in the segment immediately preceding the current segment.

[0142] For example, an indication of a dependency may not be included in the segment header that is processed first for the image from other segments.

[0143] To solve the above problem, a picture decoding method according to one aspect of the present invention is a picture decoding method of performing decoding processing by dividing the image into a plurality of segments, which image decoding method comprises extracting from the coded bit stream the flag for the dependent segment indication, whether the image contains, or does not contain, a dependent segment on which the decoding processing is performed depending on the result of the decoding processing on the segment other than the current segment, the address of the segment indicating the starting position of the current segment, and indication of the relationship indicating whether the current segment is or not is a dependent segment, where the dependency marker on the dependent segment is placed in a set of parameters common to segments,the segment's address is placed in the segment header of the current segment, and the dependency indication is placed in the segment header, and it is placed before the segment's address, and after the syntax element that identifies the set of parameters.

[0144] For example, a dependency display may be extracted from a bitstream when the dependent segment permission flag indicates the inclusion of a dependent segment.

[0145] For example, the dependent segment permission flag may be located at the beginning of a set of parameters.

[0146] For example, each of the segments may comprise a plurality of macroblocks, and the decoding processing on the current segment may start after performing the decoding processing on two of the macroblocks included in the segment immediately preceding the current segment.

[0147] For example, an indication of a dependency may not be included in the segment header that is processed first for the image from other segments.

[0148] To solve the problem, the picture coding apparatus in accordance with an aspect of the present invention is a picture coding apparatus that performs coding processing dividing the image into a plurality of segments, which image coding apparatus comprises an encoder which transmits a bitstream including: allowing a dependent segment to indicate whether the image contains or does not contain a dependent segment on which the coding processing is performed depending on the result of the coding on the non-current segment, the segment address indicating the starting position of the current segment, and an indication of the dependency indicating whether the current the segment is or is not a dependent segment, where the dependency marker on the dependent segment is placed in the parameter set common to the segments,and the address of the segment is placed in the segment header of the current segment, and the indication of the dependence is placed in the segment header, and it is placed before the segment's address, and after the syntax element identifying the set of parameters.

[0149] To solve the problem, an image decoding apparatus according to an aspect of the present invention is an image decoding apparatus that performs decoding processing, dividing the image into a plurality of segments, which image decoding apparatus comprises a decoder that extracts from the bitstream an authorization tag a dependent segment indicating whether the image contains or does not contain a dependent segment on which the decoding processing is performed depending on the result of the decoding processing on the non-current segment, the segment address indicating the starting position of the current segment, and an indication of the dependency indicating whether the current segment is if it is not, a dependent segment, where the dependency marker on the dependent segment is placed in the parameter set common to the segments,and the address of the segment is placed in the segment header of the current segment, and the indication of the dependence is placed in the segment header, and it is placed before the segment's address, and after the syntax element identifying the set of parameters.

[0150] To solve the above problem, the apparatus for coding and decoding images according to an aspect of the present invention includes a previously described image coding apparatus and an image decoding apparatus.

[0151] According to the image coding method, the image decoding method, etc., which have been previously configured, the indication of the intersegment dependency is located in the bit stream syntax relating to the segment independently of the other elements. Indication of dependence is located, without unnecessary analysis of other elements, separately from other elements. In the above examples, the HEVC indicator of intersegment dependency dependent_slice_flag - is signaled in a place where there is no need to analyze elements of syntax unrelated to the operation of the network.

[0152] More specifically, the present invention provides a device for analyzing a bit stream of a sequence of video images coded at least partially with a variable code length comprising data units carrying coded segments of a video string. The device contains a parser to extract dependencies from the display stream, which is an element of the syntax indicating for a given segment, whether variable length coding or segment analysis depends on whether it depends on other segments, where the dependency is extracted from the bitstream independently of the other syntax elements and without the need to extract these other elements first.

[0153] Such a device may be included e.g. in the entropy decoder 290 in FIG. 2. When talking about mining from a stream of bits, it is about extracting, and when needed to extract, entropy decoding. Entropy coding is a variable length coding, e.g. arithmetic coding such as CABAC. In HEVC, this is used to encode image data. The data units here refer to NAL units or access units. The expression "without the need to extract other syntax elements" refers to the situation in which the indication of the dependence is preceded only by elements whose length is known and whose presence is known, or conditioned by elements already analyzed or not even uncoded.

[0154] The present invention further provides an apparatus for generating a bit stream of a video sequence coded at least partially with a variable code length comprising data units carrying coded segments of video images. The apparatus includes a bit stream generator for inserting a dependency indicator into the bitstream, which is an element of the syntax indicating for a given segment, whether segment decoding with variable segment length depends on whether it depends on other segments, wherein the dependency ratio is inserted into the bitstream regardless of other syntax elements and without the need to first insert these other elements.

[0155] Such a device may be included e.g. in the entropy encoder 190 in FIG. 1.

[0156] According to the image coding method, the image decoding method, etc. that have been configured before, the bit stream comprises coded segment data and segment-related segment data, and the dependency ratio is located at the beginning of the segment header. This means that the segment header starts with syntax elements indicating the segment's dependence.

[0157] It should be noted that the indication of the dependence does not have to be at the very beginning of the segment header. However, it is advantageous if no other conditionally coded and / or variable-length coded syntax elements precede the dependency index in the segment header.

[0158] For example, the current location of dependent_slice_flag changes from the previously described known solutions to be at the beginning of a segment header. This change results in a reduction in the number of syntax elements that require analysis. It avoids complicated syntactic analysis operations in routers, such as variable length decoding and information analysis, which requires additional calculations and / or memorizing additional parameters for future use and / or analysis of other parameter sets. In addition, the number of parameter sets that need to be tracked is reduced.

[0159] Hereinafter, the embodiments have been specifically described with reference to the drawings. Each of the forms described in the following is a general or specific example. Numerical values, shapes, materials, structural elements, arrangement and connection of structural elements, steps, order of processing of steps etc. shown in the following forms are only examples and thus do not limit the scope of the present invention. Therefore, among the structural elements in the embodiments described hereinafter, the structural elements not mentioned in any of the independent claims have been described as random structural elements.

(Form 1) [0160] FIG. 10 shows an example of the bit stream syntax according to the present embodiment. NAL 1010A header shown in FIG. 10 is the same as the NAL headline 910 shown in FIG. 9A. In other words, there is no change.

[0161] However, the syntactic structure of the segment header 1020 differs from the structure of the syntactic segment header 920 in FIG. 9A. In particular, in the segment header 1020, the dependent_slice_flag element moves in the segment header in such a way that there is no syntactic element preceding the dependent_slice_flag. The dependent_slice_flag element is conditionally coded, is encoded with a variable code length, or receives a syntax analysis that requires additional calculations.

[0162] Both syntactic elements, first_slice_in_pic_flag and dependent_slice_flag, actually determine spatial dependencies. The syntactic elements are coded just after the NAL header in such a way that you do not need to parse syntactically any other syntactic element. Because the first_slice_in_pic_flag element also carries information that is related to intersegmental relationships, it can precede dependent_slice_flag. The syntax element first_slice_in_pic_flag is a tag that sets up according to the rule that each frame must start with a regular segment. Thus, when the first_slice_in_pic_flag flag is set, it means that the segment is an ordinary segment, and thus independent.

[0163] In other words, the dependency ratio may be defined to include an indication of the first segment indicating whether the segment is or is not the first segment in the image, and the dependent segment flag indicates whether decoding with variable segment length depends or not, from other segments. The first segment in the image is always a segment for which the encoding with variable length does not depend on other segments.

[0164] Preferably, the bit stream includes a dependent segment permission flag indicating whether the dependent segments may be included or not in the bit stream. The dependency ratio is included in the bitstream only if the allow flag of the dependent segment indicates that dependent segments can be included in the bit stream. The dependent segment permission flag is located in the bit stream in the parameter set common to many segments and is located at the beginning of the parameter set. The set of parameters can be, for example, a set of image parameters that contains parameters for a single image. Alternatively, the dependent segment approval flag is located in a string parameter set that contains parameters for the entire image sequence (video).

[0165] However, in the present invention, the dependent_slice_flag (indication of dependency) is coded without conditioning on the syntax element dependent_slice_enabled_flag (dependency tag on the dependent segment). In this embodiment, since the identifier of the image parameter set is located after the indication of the relationship, it is preferable to avoid a possible parsing error in case the identifier of the image parameter set is signaled in the segment header.

[0166] This change can also be seen and / or interpolated by changing the location of other required syntax elements in parameter sets or headers to reduce the number of syntax elements that should be analyzed to determine the relationship between segments.

[0167] For example, the dependent_slice_flag syntax element in the segment header of the present HM8.0 syntax only occurs when the value of the syntax element "dependent_slice_enabled_flag" indicates that the use of dependent segments in the bit stream is allowed. The permission for dependent segments, and thus also the syntax element "dependent_slice_enabled_flag", is included in the PPS, as shown in FIG. 9C. Thus, the syntax element "dependent_slice_enabled_flag" in PPS moves within PPS syntax to simplify its analysis required to analyze dependent_slice_flag (eg at the beginning of a set of parameters). This can also be useful if the dependent_slice_flag is coded after the pic_parameter_set_id (element of the syntax that identifies the set of parameters). This is due to the fact that

[0168] Instead of moving the "dependent_slice_enabled_flag" in the PPS, "dependent_slice_enabled_flag" can be moved from PPS to SPS and / or VPS, so that parameter sets that are lower in the hierarchy do not have to be tracked.

[0169] In other words, according to the present embodiment, the location of required elements of the syntax is changed to reduce the number of sets of parameters to be followed. It also reduces the complexity of the syntax analysis. In the present context, the term "required parameters" means parameters that contribute to determining if a segment is or is not intersegmentally dependent. The first option applicable directly to the HEVC is to provide an indication of the dependence at the beginning of the dependent segment header without being dependent on the dependent segment permission flag, which is included in the parameter set other than the segment header. The second option applicable directly to the HEVC is to provide an indication of the dependence in the header of the dependent segment after indicating the set of parameters identifying the set of parameters, in which the authorization flag for the dependent segment is included. Indication of dependence may be conditioned by the permission marker for the dependent segment. Moving the permission flag to a dependent segment in the PPS or moving the approval flag to a SPS dependent segment may be beneficial for either of these possibilities. This is particularly advantageous for the second possibility in which the dependency tag on the dependent segment is needed to analyze the indication of dependency.

[0170] As can be seen in FIG. 10, the header of the NAL unit with the key portion of the segment header has 18 bits (14 bits of the NALU header and 2 bits of the segment header). According to this example, the media serving network element can work for the current segment packet as follows. If the previous segment is omitted, which is a normal, entropy or dependent segment, the network element checks the first two bits of the current segment header, which are the first_slice_in_pic_flag and (in the case of allowing dependent segments in the bit stream) dependent_slice_flag.

[0171] When the type of NAL unit is the VCL NAL unit type and the last two bits out of 18 checked are "01", the NAL unit is omitted. Especially when the first bit of the segment header is "1", then it is the first segment in the image that is not (according to the rules) dependent segment. When the first bit of the segment header is "0" and the next bit of the segment header is also "0", the segment is not dependent. Therefore, only if the first two bits of the segment header are "01", the segment is dependent. In addition, the segment should be omitted because it can not be decoded when the parent segment has already been omitted. So the first_slice_in_pic_flag and dependent_slice_flag tags can be treated like the extension of the NAL header, even if they belong to the segment header syntax.

[0172] Thus, the present form also provides, as one of its aspects, a network router for receiving, analyzing and forwarding network packets to their destinations. The router includes a receiving module for receiving a network packet comprising a packet destination address and a bitstream portion with encoded video data; a parser analyzer comprising a device for parsing a bit stream of an encoded video sequence according to any of the forms cited earlier or hereinafter, to determine the dependence of the encoded video data on other packets; and the packet analyzer to analyze the destination address of the received packet and the determined dependency, and to evaluate how to handle the network packet.

(Form 2) [0173] According to embodiment 2, dependent_slice_enabled_flag is omitted from PPS. Note that this dependent_slice_enabled_flag can be moved to SPS instead of omitting it.

[0174] FIG. 11 shows an example in which the dependent_slice_enabled_flag does not need to be parsed before going to the first_slice_in_pic_flag and the dependent_slice_flag [0175] In this example, the dependent_slice_enabled_flag is not used because it is not conditioned by the presence of the dependency indication. This example provides the ability to place a dependency indication at the beginning of a segment header, without causing analysis problems due to an unknown identification of the current PPS file.

(Effect of form 2, etc.) [0176] In form 1, to analyze dependent_slice_flag, the dependent_slice_enabled_flag element must be parsed. The element dependent_slice_enabled_flag is signaled in the PPS. This may result in some excess analysis, as discussed earlier, when the dependent_slice_enabled_flag is located far from the beginning of the PPS, and the preceding syntactic elements are conditionally coded.

[0177] Furthermore, signaling the syntax element dependent_slice_flag before parsing the syntactic element pic_parameter_set_id in the PPS can cause syntax error parsing as follows. The presence of dependent_slice_flag depends on the dependent_slice_enabled_flag, which is signaled in the PPS. However, the identifier of the currently active PPS is signaled after dependent_slice_flag. Therefore, it is not possible to analyze the dependent_slice_flag before accessing the previous items.

[0178] It is therefore preferable to delete the analysis condition related to dependent_slice_enabled_flag. It may be more beneficial if you apply the following limitation. Namely, if the dependent_slice_enabled_flag in PPS is zero, then the dependent_slice_flag should be equal to zero.

[0179] However, these preferred implementations are not intended to limit the scope of the present invention.

(Modification 1 form 1 and 2) [0180] Alternatively, or as a complement to remove the condition related to the dependent_slice_enabled_flag, the dependent_slice_enabled_flag element can be moved from the PPS to any of SPS and / or VPS.

[0181] Moreover, instead of just moving the dependent_slice_enabled_flag, the dependent_slice_enabled_flag can be duplicated in the SPS. In this case, you can force the pointer in SPS and PPS to have the same value. You can also allow the PPS overwrite the SPS indicator.

[0182] For example, when sps_dependent_slice_enabled_flag is 1, pps_dependent_slice_enabled_flag may be 0 or 1. Then sps_dependent_slice_enabled_flag is an indication of allowing dependent segments for the sequence of images signaled in the SPS, and pps_dependent_slice_enabled_flag is an indication of the permission for segments dependent on the image signaled in the PPS. However, when the value of dependent_slice_enabled_flag can change in PPS, it means that PPS parsing is still needed and the benefit of less frequent PPS tracking and analysis is prevented.

[0183] These modifications provide the advantage that VPS and SPS carry dependency structures. The deployment of VPS and SPS dependence structures allows network elements to shape bit streams, i.e. to decide on rejecting dependent packets that can not be decoded in any way, or about rejecting dependent segments rather than independent segments. Therefore, the dependent_slice_enabled_flag in VPS will (or will not) initiate the router to check additionally the segment header.

[0184] It should be noted that these modifications do not further reduce the complexity of the analysis if the example of FIG. 10 and 11. However, they provide a more favorable syntactic structure for transferring dependency structures. In summary, according to this example, an indicator to indicate whether dependent segments are or are not allowed in the bit stream is signaled in a set of video parameters. A set of video parameters is a set of parameters applicable to more than one segment in more than one image.

[0185] There are two different benefits of signaling the dependent_slice_enabled_flag in VPS and / or SPS. When the tag only moves or duplicates, it is not required to analyze the PPS, reducing the cost of analysis. The second benefit is enabling the routers to learn about the structure of video string prediction. This advantage occurs all the time. Typically, the router can check the contents of the VPS / SPS to know what it will receive.

[0186] VPS is the highest parameter in the hierarchy. VPS can contain information about multiple video strings, while SPS and PPS are appropriate for a single video and image sequence. Information in VPS includes transmission speed, temporal_layering structure of video sequences, etc. They also include information on inter-layer relationships (dependencies between different video strings). Thus, VPS can be treated as a container for many video strings and gives a general overview of each string.

[0187] In the current version of HEVC, the relationship between segments in the frame is established by the dependent_slice_flag and first_slice_in_pic_flag. According to current technical requirements, network objects can not use intersegment dependencies without using highly complex syntax analysis. The immediate solution would be to discover the packet loss by missing the packet number, skipping all packets until you encounter the first_slice_in_pic_flag, which value equals 1. This is due to the fact that the first segment in the image is always a regular segment.

[0188] However, this solution leads to a reduction in the coding efficiency. Therefore, as described earlier, you can use the signaling of the intersegmental dependence allowing efficient analysis. This is achieved by signaling the dependent_slice_flag and the first_slice_in_pic_flag in the segment header directly after the NAL header.

[0189] Alternatively or additionally, syntactic elements associated with intersegmental relationships are unconditionally coded, i.e. independent of other syntax elements that may be in the segment header or in the PPS.

(Modification of form 1 and 2) [0190] FIG. 12 depicts modification 2 as an alternative to the previously discussed modification 1. In particular, the header 1210 of the NAL unit is the same as the header of the unit

The NAL shown in FIG. 10 (header 910 of the NAL unit shown in FIGURE 9A). However, the headline

1220 of the segment and segment 1020 header 1020 shown in FIG. 10 differ in that the syntactic elements of the header of the dependent_slice_flag and first_slice_in_pic_flag segment are in reverse order. In particular, segment header 1220 contains the dependent_slice_flag as the first syntax element, and the syntax element first_slice_in_pic_flag as the second syntax element, conditioned by the presence of dependent_slice_flag.

[0191] As can be seen from this example, the first segment indication indicating whether the segment is or is not the first segment in the image is included in the syntax. The first segment in the image is always a segment for which decoding with variable length does not depend on other segments. Furthermore, the dependent segment flag is contained in the bitstream before the first segment is indicated. The indication of the first segment is included in the bitstream only if the dependent segment flag does not indicate a dependent segment. This system provides the same benefits as the conditioning. In other words, the dependency marker is conditioned by the indication of the first segment. As can be seen in FIG. 12, both elements can be understood as an indication of dependencies and they are included at the beginning of the segment header.

(Form 3) [0192] In embodiment 3, compared to embodiments 1 and 2, the method of arranging the syntax elements has been changed to limit parsing of unneeded syntax elements.

[0193] In the above described embodiments, dependent_slice_flag is described in the case where the first_slice_in_pic_flag is included as a condition for the presence of dependent_slice_flag. However, you can include both elements first_slice_in_pic_flag and dependent_slice_flag into the bit stream, without making the one dependent on the presence of the other. For example, the dependent_slice_flag coding method is changed to be independent of the syntax element dependent_slice_enabled_flag according to one of the previously described modifications.

[0194] FIG. 13 is a diagram of an exemplary segment header according to the present embodiment. FIG. 13 presents a case that still contains a condition for indicating the dependence on the license marker for a dependent segment.

[0195] More specifically, in the segment header according to the present embodiment, dependent_slice_flag is placed before slice_address, in comparison with the existing segment header shown in FIG. 6. Furthermore, in the segment header according to the present embodiment, compared to the examples in FIG. 10 to 12, dependent_slice_flag is placed after pic_parameter_set_id.

[0196] In this embodiment, since the dependent_slice_flag is placed before the slice_address, at least the SPS does not need to be parsed to parse the dependent_slice_flag. As described earlier, slice_address is a syntax element that indicates the beginning of the segment. In addition, slice_address can only be parsed using syntax elements signaled in SPS (pic_parameter_set_id).

[0197] Alternatively or additionally, dependent_slice_enabled_flag either moves up in PPS or moves to SPS and / or VPS. If the enable flag is in VPS and / or SPS, you may not need to analyze and track PPS and SPS.

(Modification of form 3, effect, etc.) [0198] (1) The example of FIG. 13 may provide a device for analyzing a bit stream of the video string coded at least in part with a variable length code comprising data units carrying the coded segments of the video images. In this case, the device is configured to include a parser that extracts the following syntactic elements from the bit stream:

indication of the dependence, which is an element of the syntax indicating for the segment in the segment header, whether the decoding with variable segment length depends or not on other segments;

a dependent segment tag included in a parameter set for multiple segments indicating whether the dependent segments may or may not be included in the bit stream, and a segment address indicating the position in the bitstream in which the segment starts.

(2) In addition, in the present form the indication of the dependence is signaled in the segment header before the address of the segment and after the syntax element identifying the set of parameters.

In this form, it is possible, without making analysis errors, to configure in such a way that the dependency indication is included in the bitstream only if the dependent segment enable flag indicates that the dependent segments may be included in the bit stream.

(3) In the present form, the dependent segment permission flag is located in the bit stream in a parameter set (PPS) common to many segments forming the same picture frame and is located at the beginning of the parameter set. However, it is not limited to this. Alternatively (or additionally), the dependent segment permission flag is located in the bit stream in the parameter set (SPS) common to many segments forming the same sequence of images. Still alternatively (or additionally), the dependent segment permission flag is located in the bit stream in a parameter set (VPS) common to many segments forming a plurality of image frame strings.

(4) Furthermore, in the present embodiment, VPS_id and SPS_id can be signaled explicitly in the SEI message. When dependent_slice_enabled_flag is signaled in SPS, dependent_slice_flag must still follow pic_parameter_set_id.

[0199] Otherwise, a dependency analysis is introduced because SPS_id is signaled in the PPS. When the identification of the current SPS or VPS that signals the dependent_slice_enabled_flag is signaled, the dependency indication may also be included before pic_parameter_set_id, because the analysis of the image parameter set is not needed. In addition, such a SEI message bearing the VPS_id or SPS_id is not needed for the decoding operation, because these IDs are also determined by analyzing the PPS. The SEI message can thus be rejected without affecting the decoding operation after being used by the network elements.

(Form 4) [0200] In form 4, the intersegmental dependency information is reproduced (in addition to information signaled in the segment header and / or in the parameter set) in another NAL entity, such as a SEI message.

[0201] For example, it is possible to specify a SEI message that carries the intersegmental dependency information in each access entity or in front of each dependent segment. The term "access unit" refers to a data unit that is constructed from a set of NAL units. The access unit includes encoded image segments, i.e. VCL NALU. In particular, the access units may define random access points and may comprise a single image NALU. However, the access unit is not necessarily a random access point.

[0202] In the current HEVC technical requirements, an access unit is defined as a set of NAL units that are consecutive in the decoding order and contain exactly one coded picture. In addition to the NAL units of the coded segment of the encoded image, the access units may also include other NAL units not comprising coded image segments. Decoding the access unit always gives a decoded image. However, in a future HEVC extension (such as Multi-View Coding (MVC) or Scalable Video Coding (SVC)), the access unit definition can be relaxed or changed. According to the current technical requirements, the access unit consists of: access unit limiter, SEI and VAL NALU messages.

[0203] According to the present embodiment, the relationship indication is located in the bitstream except the segment header to which the dependency indication relates. In addition, it may be advantageous if the indication of the relationship is located in the bit stream in an additional enrichment message contained in the bit stream before the dependent segment or once per access unit.

(Form 5) [0204] According to embodiment 5, the intersegmental dependency information is signaled in the NAL header as the tag implicitly as the type of NAL unit to which it is associated.

[0205] As a rule, the analysis of the syntax elements in the NAL header does not depend on any other syntax elements. The header of each NAL unit can be analyzed independently. The NAL header is the usual place to signal dependency information. Thus, according to the present embodiment, an intersegmental relationship is also signaled there.

[0206] In other words, the evaluation device can be adapted in the router or in the decoder. The analysis device further comprises a network adaptation module for adding to the segment of coded video data and to the header of the adaptation layer network segment, and the NAL header. Preferably, the dependency indication is located in the bit stream in the NAL header and is coded independently of other syntax elements.

[0207] The dependency ratio may be located in the NAL header, because the NAL header in the current HEVC technical requirements includes several reserved bits that can be used for this purpose. A single bit is enough to signal the dependency indication. Alternatively, the dependency indication is indicated by the type of NAL unit and the predetermined type of NAL unit is reserved for transferring dependency information.

(Form 6) [0209] It should be noted that the above five forms can be arbitrarily combined to allow efficient analysis of dependency information in network elements. Even when their use is redundant, characters can be combined. Therefore, the replication of a dependency display can be used even when the dependency display is also signaled at the beginning of the segment header.

[0210] FIG. 14 is an exemplary headline 1410 of a NAL unit in which the header 910 of the NAL unit shown in FIG. 9A has been modified. The header 1410 of the NAL unit contains dependent_slice_flag.

[0211] In addition, to move the dependent_slice_flag to the NAL header and keep the constant size of the NAL header due to downlink compatibility, this one bit needed for dependent_slice_flag comes from the syntax element nuh_reserved_zero_6bits of the header of the unit. Thus, the syntax element nuh_reserved_zero_6bits currently has only 5 bits. The syntax element nuh_reserved_zero_6bits contains bits reserved for future use, so the reduction does not cause any problems and does not require any further modification.

[0212] In general, the current VCL NAL unit depends on the previous VCL NAL unit that has the same temporal_layer_id. When the dependent_slice_flag is signaled in the NAL header, one bit will be used for both the VCL units and the non-VCL NAL units, because each data unit, such as a picture segment or parameter set, has the same NAL header. Thus, although dependent_slice_flag appears to be also signaled for a set of parameters or SEI messages, this is unnecessary. In addition, dependent_slice_flag must always be signaled even if dependent segments are excluded in the string parameter set. This leads to unnecessary costs.

[0213] In all of the above embodiments, the dependence indication may be a one-bit flag.

(Form 7) [0214] According to embodiment 7, the relationship indication is indicated by the type of NAL unit and the predetermined type of NAL unit is reserved for transferring dependency information.

[0215] Thus, the new (separate) VCL NAL type is defined with similar semiotics as existing VCL NAL units. For example, when NAL_unit_type is equal to 15 (or another predefined type or NALU, which is not reserved for another specific type of NALU), then the current VCL NAL depends on the previous VCL NAL unit that has the same temporal_layer_id. The dependence concerns the dependence of the current segment on the segment header of the previous segment, as described earlier, i.e. dependencies in the analysis.

[0216] In these cases, it may be preferable to include the bit in the NAL header to additional types of NAL units. This can be used to indicate if the current segment is or is not a dependent segment.

[0217] When dependency information is signaled in the segment header in addition to the NAL header, signaling in the NAL header becomes optional. In particular, when the NAL_unit_type field in the NAL header is configured to indicate that the current segment is a dependent segment, then it is not possible to signal information of any other type. For example, in some cases, it may be more advantageous to transfer information that the current segment is the "first image in a sequence" (NAL_unit_type equal to 10 or 11). When the intersegment dependency information in the NAL header is optional (since it is duplicated in the segment header), it can be selected to signal more valuable information.

[0218] It may further be advantageous to add two or more types of VCL NAL units, such as a "dependent RAP table image" (required for analysis) or "non-RAP non-RAP image" of a dependent segment. "RAP" means an accidental access image. An accidental image is an image coded independently (in the sense of prediction) from other images, so this image can be used as the starting point for coding and decoding. Therefore, it is suitable as a random access point.

[0219] In the dependent segment header, the RapPicFlag syntax element is used in the analysis process. More specifically, the RapPicFlag syntax element is an indication of whether the current image is or is not a random access image.

[0220] The value of RapPicFlag depends on the type of NAL unit, as in the following expression 2.

[Pattern 2]

RapPicFlag = (nal_unit_type> 7 && nal_unit_type <12)

Expression 2 [0221] In other words, in the example shown in FIG. 15, the random access images are carried by the NALU type NALU between 7 and 12. To enable correct parsing and provide segment dependencies for random access images, the present invention defines two different types of NAL units to ensure correct segment header analysis. .

[0222] As a general rule, even when a new type of VCL NAL unit is defined, segment header parsing should still be possible without any problem. Each of the many NAL types is defined as above or the dependent segment heading changes in such a way that there are no problems with the analysis.

[0223] When a new type of VCL NAL is defined to indicate a dependent segment, the syntactic structure of the segment header can be changed as follows.

[0224] In the above example, the unit type NAL "DS_NUT" is used to indicate that the current VCL NAL unit is a dependent segment. Compared with the most recent syntactic structure of the segment header, which is described in non-patent literature 3, the following changes are made in this embodiment.

(1) no_output_of_prior_pics_flag is not signaled in the header of the dependent segment. In other words, the presence of no_output_of_prior_pics_flag is based on the condition that the current segment is not a dependent segment (no_output_of_prior_pics_flag can occur in the segment header when the current segment is not a dependent segment).

(2) first_slice_in_pic_flag is conditionally signaled depending on the nal_unit_type value. When nal_unit_type indicates that the current segment is a dependent segment, the syntax element first_slice_in_pic_flag is not signaled explicitly and is considered equal to 0. This maintains the transmission speed with the same quality.

[0225] According to the example, no_output_of_prior_pics_flag is not signaled when the current segment is a dependent segment. Thus, the value of RapPicFlag is not required to estimate when the current segment is a dependent segment. Therefore, the segment header in the dependent segment can be parsed with no problem. More specifically, you can analyze the segment header in a dependent segment without referring to the NAL header of the previous NAL unit. The problem occurs when the header of the previous NAL unit is not present at the time of decoding.

[0226] Second, first_slice_in_pic_flag is signaled based on the NAL_unit_type value. This change is the same as the change in the example described in FIG. 12. In FIG. 12 first_slice_in_pic_flag is signaled in the segment header only if the current segment is not a dependent segment (which is indicated by dependent_slice_flag). Similarly, in the above example, first_slice_in_pic_flag is signaled only when nal_unit_type is not equal to "DS_NUT", which means that the current segment is not a dependent segment.

[0227] Both of the above-mentioned changes do not have to be entered together. You can also enter only one of the changes in the segment header. The benefit of each change is related to the cost of checking whether the segment is or is not a dependent segment. However, when both changes are made together, the benefits of both changes can be obtained at the same expense as the benefit of each single change in the case that both syntactic elements first_slice_in_pic_flag and no_output_of_prior_pics_flag are coded sequentially. Thus, the use of both changes in combination with the subsequent coding of the two elements of syntax is an advantage over the direct application of each change individually.

[0228] Full explanation of the form can also remove the dependent_slice_enabled_flag from the bitstream when the indication of the dependent segment is not conditionally coded therein. In other words, when e.g. a new type of NAL unit is used to indicate if the current segment is a dependent segment, then dependent_slice_enabled_flag can be removed from the bit stream. [0229] FIG. 15 shows a header 1510 of a NAL unit that is the same as the header 910 of the NAL unit shown in FIG. 9A, and segment header 1520, which was converted from the segment header 920 shown in FIG. 9A. Segment header 1520 includes the termination of the value of dependent_slice_flag according to the NALU type. In particular, the syntax element NAL_unit_type with the values ​​15 and 16 defines dependent segments. When NAL_unit_type is 15, segment type is a dependent segment of the random access image. If, on the other hand, NAL_unit_type is equal to 16, the segment is a dependent segment of the image, not of random access. Therefore, the following expression 3 is introduced.

[Pattern 3]

RcipPicFlag = (nal_unit_type> 7 && nal_unit_type <] 2 || nal_unit_type = 15) Expression 3 [0230] It should be noted that the values ​​of 15 and 16 were chosen only as an example. As it is clear to those skilled in the art, any number that is not used in any other way may be taken. In particular, the first type of NALU should be defined as identifying the dependent segment being the content of the random access image, and the second type of NALU should be defined as identifying the dependent segment being the content of the image, not about random access.

[0231] In addition, a limitation may be applied that dependent segments are used only in RAP or used only in non-RAP. In such cases, only one new type of NALU is needed.

(Form 8) [0232] FIG. 16 is a diagram showing an alternative solution. The 1610 NAL header is the same as the 910 header of the NAL unit. Segment header 1620 assumes the definition of NAL_unit_type with the values ​​15 and 16 signaling dependent segments as described earlier.

[0233] However, the NAL unit type is not used in the dependent segment flag analysis. This allows you to use NAL_unit_type as an optional encoder. Thus, the advantages of this character are obtained only when the encoder is to adopt new NALU types.

[0234] The router then only needs to look at the NALU type. However, when the encoder does not use the new NALU types, the router will treat the dependent segments as in the latest known solutions.

[0235] In summary, the indication of the relationship can be indicated by the type of NAL unit.

The redefined NAL unit type can be reserved for moving encoded segments whose segment header depends on the segment header of the previous segment. Preferably, a separate type of NAL unit is provided that the relationship is provided for the random access images and the images without such access.

[0236] In summary, the embodiments described above relate to the structure of a syntactic bit stream carrying coded video strings. In particular, the forms described above refer to the syntactic structure associated with dependent and entropy segments whose segment header depends on the segment header of the previous segment. To allow a media network element to take into account such dependencies without substantially increasing the complexity and delay due to parsing, the dependency indication is signaled at the beginning of the packets or, in other words, near the headers or parameters to be analyzed. This is achieved, for example, by including the indication of the dependence at the beginning of the segment header (FIGURES 10 to 12), possibly after the identifier of the parameter set, and before the address of the segment,

(Modifications of form 1 to 8, effect, etc.) [0237] Various changes are possible without being limited by the forms 1 to 8, and of course they are within the scope of the present invention.

[0238] Each of the structural elements in each of the embodiments described above may be configured as a special hardware product (processing system) or may be implemented by executing a program suitable for the structural element. Each of the structural elements may be implemented using a program execution module, such as a CPU and a processor, reading and executing a program stored on a recording medium, such as a hard disk or semiconductor memory.

[0239] Although in descriptions from 1 to 8 the description assumes the wave front, this is not a limitation.

[0240] However, in the case of the wave front, all sub-streams can not start at the same time. As described earlier, in connection with each substream except for the sub-stream at the beginning, the start of processing (coding or decoding) is delayed by two LCUs from the previous substream. Therefore, further processing shortening is required in the wave front. In this embodiment, by placing a dependency indication (dependent_slice_flag) over a syntax element that identifies the PPS and before the segment address, the number of syntax elements to be analyzed can be reduced, thereby reducing the amount of processing.

[0241] Furthermore, in the above-described embodiments from 1 to 8, by placing an indication of the relationship up in the segment header (especially at the beginning), it can e.g. be checked whether each segment is or is not a segment dependent on the early image processing stage.

[0242] In other words, at the start of image processing (coding or decoding), when the step of checking whether each of the segments is or is not a dependent segment, one can extract the starting point of the parallel processing at the moment of image processing. In other words, when the image contains a plurality of ordinary segments, one can extract the starting point of parallel processing at the time of image processing or early processing.

[0243] Here, classically, when the dependency indication is placed after the segment address, it can not be checked whether the segment is or is not a dependent segment or a regular segment until the address of the segment is analyzed. In this case, the start of processing on the ordinary segment in the middle of the image is significantly delayed compared to the start of processing on the regular segment at the beginning of the image.

[0244] Conversely, in the above-described embodiments from 1 to 8, since it is possible to check whether each of the segments is or is not a dependent segment at an early stage of image processing, the start of processing on the ordinary segment in the center of the image can be accelerated. In other words, you can start processing on a regular segment in the center of the image at the same time as on the regular segment at the beginning of the image.

(Form 9) [0245] The processing described in each form can be easily implemented on an independent computer system by writing to the program recording medium to implement the configuration of the moving picture coding method (image coding method) and the moving picture decoding method (image decoding method). ) described in each character. The recording medium can be any medium, as long as it is possible to save the program, such as, for example, a magnetic disk, an optical disk, a microprocessor card and semiconductor memory.

[0246] Hereinafter, the applications of a method for encoding moving images (a picture coding method) and a method for decoding moving pictures (the image decoding method) described in each form and the systems in which they are used are described. The system has the feature that it includes a picture coding and decoding apparatus that includes a picture coding apparatus using an image coding method and a device for decoding images using an image decoding method. Other configurations in the system can be changed as appropriate.

[0247] FIG. 17 shows the general configuration of the ex100 content delivery system for implementing content delivery services. The area of ​​provision of communication services is divided into cells of the requested size, and base stations ex106, ex107, ex108, ex109 and ex110, which are permanent wireless stations, are placed in each of the cells.

[0248] The ex100 content delivery system is connected to devices such as ex111 computer, palmtop (PDA) ex112, ex113 camera, ex114 mobile phone and ex115 gaming console, via ex101 Internet, ex102 service providers, ex104 telephone network, as well as also respectively base stations ex106 to ex110.

[0249] The configuration of the ex100 content delivery system is however not limited to the configuration shown in FIG. 17 and a combination is acceptable in which any of the elements are combined. In addition, each device can be directly connected to the ex104 telephone network instead of via ex106 to ex110 base stations which are fixed wireless stations. In addition, the devices may be connected to each other on short-range wireless communication or by other means.

[0250] An ex113 camera, such as a digital camera, can record movies. An ex116 camera, such as a digital camera, can record both movies and digital photos. In addition, the mobile phone ex114 can be a phone compatible with any of the standards, such as Global

System for Mobile Communications (GSM) ™, Code Division Multiple Access (CDMA), WidebandCode Division Multiple Access (W-CDMA), Long Term Evolution (LTE) and High Speed ​​Packet Access (HSPA). Alternatively, the mobile phone ex114 can be the phone of the Personal Handyphone System (PHS).

[0251] In the ex100 content delivery system, the ex103 streaming server is connected to the camera ex113 and other devices via the ex104 telephone network and the ex109 base station, which allows the dissemination of images from live and other spectacles. In such dissemination, content (e.g., video from live musical representation) recorded by the user using the camera ex113 is coded in the manner described earlier in each form (i.e., the camera acts as an image coding device according to the present invention) and the encoded content is sent to streaming server ex103. On the other hand, the ex103 streaming server performs the streaming distribution of transmitted content to clients at their request. Clients include ex111 computer, ex112 PDA, ex113 camera, ex114 mobile phone and ex115 game console, which devices can decode the above encoded data. Each of the devices that received the transmitted data decodes and plays back the encoded data (i.e. each device acts as an image decoding device according to an aspect of the present invention).

[0252] The recorded data may be encoded by the camera ex113 or the streaming server ex103, which transmits the data, or the coding processes may be separated between the camera ex113 and the streaming server ex103. Similarly, distributed data can be decoded by clients or an ex103 streaming server, or decoding processes can be distributed between clients and an ex103 streaming server. In addition, data of non- moving images (photos) and video recorded not only by the camera ex113, but also by the camera ex116, can be sent to the streaming server ex103 by the computer ex111. Coding processes can be performed by the camera ex116, the computer ex111 or by the streaming server ex103, or they can be shared between them.

[0253] Furthermore, the coding and decoding processes may be performed by the LSI ex500 normally found in each of the ex111 computers and devices. The LSI ex500 can be configured as a single chip or multiple chips. The video encoding and decoding software can be stored on a certain type of recording medium (such as a CD-ROM, floppy disk and hard disk) that can be read by the ex111 computer and other devices, and the coding and decoding processes can be performed using using software. In addition, if the ex114 mobile phone is equipped with a camera, the image data obtained with the camera can be transmitted. Video data is encoded by LSI ex500 found on the ex114 mobile phone.

[0254] Furthermore, the ex103 streaming server may consist of servers and computers, and may disperse data and process distributed data, write or distribute data.

[0255] As described earlier, customers can receive and reproduce coded data in the content delivery system ex100. In other words, clients can receive and decode information sent by the user and play back the decoded data in real time in the ex100 content delivery system, so a user who does not have any special rights or equipment can implement personal distribution.

[0256] In addition to the example of an ex100 content delivery system, at least one moving picture coding apparatus (picture coding device) and a moving picture decoding apparatus (image decoding apparatus) described in each embodiment may be implemented in the digital broadcast system ex200 shown in FIG. 18. More specifically, ex201 transmitting station transmits or transmits on radio waves to ex202 satellite sending, multiplexed data obtained by multiplexing audio and other data into video data. The video data is data encoded using the method of encoding the moving images described in each form (i.e., the data encoded by the picture coding apparatus according to an aspect of the present invention). After receiving the multiplexed data, the ex202 transmit satellite transmits radio waves for transmission. Next, the ex204 home aerial with satellite reception function receives radio waves. Thereafter, the apparatus, such as ex300 television (receiver) and satellite decoder (STB) ex217, decodes the received multiplexed data and reconstructs the decoded data (i.e., the device acts as a picture decoding device according to an aspect of the present invention).

[0257] Furthermore, the ex218 player / recorder (i) reads and decodes the multiplexed data recorded on the write ex215 medium, such as DVD and BD, or (i) encodes the video signals on the ex215 medium for recording, and in some cases records the data received via multiplexing audio signal to encoded data. The ex218 player / recorder may include a moving picture decoding apparatus or a device for encoding moving images such as shown in each form. In this case, the reproduced video signals are displayed on the monitor ex219 and can be played by another device or system using an ex215 recording medium on which the multiplexed data is stored. It is also possible to implement a device for decoding moving pictures in an ex217 decoder connected to an ex203 cable television cable or to an ex204 antenna for receiving satellite and / or terrestrial broadcasts to display video signals on the ex300 monitor of the ex300 television. The device for decoding moving images can be implemented not in the decoder but in the ex300 TV set.

[0258] FIG. 19 shows a television (receiver) ex300, which uses the method of encoding moving images and the method of decoding moving images described in each form. The ex300 television includes: an ex301 tuner that receives or provides multiplexed data received by multiplexing audio data into video data, via an ex204 antenna or an ex203 cable, etc., which (s) receives an emission; an ex302 modulator / demodulator that demodulates the received multiplexed data or modulates data into multiplexed data for outward delivery; and an ex303 multiplexer / demultiplexer, which demultiplexes modulated data multiplexed to video data and audio data, or multiplexes video data and audio data encoded by the ex306 signal processor into data.

[0259] The ex300 television also includes an ex306 signal processor including an ex304 audio signal processor and an ex305 video signal processor that respectively decodes audio data and video data and encodes audio data and video data (which acts as an image encoding apparatus and an image decoding apparatus according to aspects of the present invention), and an ex309 output module including an ex307 loudspeaker that provides the decoded audio signal, and an ex308 display module that displays a decoded video signal, e.g., a display. In addition, the ex300 TV contains an ex317 interface module containing the ex312 input programming module that receives user input data. In addition, the ex300 TV contains the ex310 control module, which controls in general every component of the ex300 TV set, as well as the ex311 power module, which provides energy for each element. In contrast to the ex312 input programming module, the interface module ex317 can contain: ex313 bridge, which is connected to an external device, such as ex218 player / recorder; an ex314 socket for connecting an ex216 recording medium, such as an SD card; an ex315 driver for connection to an external recording device, such as a hard disk, and an ex316 modem for connection to the telephone network. In this case, the ex216 storage medium can electrically write information using non-volatile / volatile semiconductor memory. The ex300 TV components are connected to each other via a synchronous bus. which is connected to an external device, such as ex218 player / recorder; an ex314 socket for connecting an ex216 recording medium, such as an SD card; an ex315 driver for connection to an external recording device, such as a hard disk, and an ex316 modem for connection to the telephone network. In this case, the ex216 storage medium can electrically write information using non-volatile / volatile semiconductor memory. The ex300 TV components are connected to each other via a synchronous bus. which is connected to an external device, such as ex218 player / recorder; an ex314 socket for connecting an ex216 recording medium, such as an SD card; an ex315 driver for connection to an external recording device, such as a hard disk, and an ex316 modem for connection to the telephone network. In this case, the ex216 storage medium can electrically write information using non-volatile / volatile semiconductor memory. The ex300 TV components are connected to each other via a synchronous bus. and an ex316 modem to connect to the telephone network. In this case, the ex216 storage medium can electrically write information using non-volatile / volatile semiconductor memory. The ex300 TV components are connected to each other via a synchronous bus. and an ex316 modem to connect to the telephone network. In this case, the ex216 storage medium can electrically write information using non-volatile / volatile semiconductor memory. The ex300 TV components are connected to each other via a synchronous bus.

[0260] First, a configuration will be described in which the ex300 television decodes the multiplexed data received externally by the ex204 antenna and other devices and restores the decoded data. On the ex300 TV, after the user's operation with the ex220 remote control and others, the ex303 demultiplexer demultiplexes the multiplexed data demodulated by the ex302 modulator / demodulator, under the control of the ex310 control module containing the CPU. In addition, the audio signal ex304 processor decodes the demultiplexed audio data, and the ex305 video signal processor decodes the demultiplexed video data using the decoding method described in each form on the ex300 television. The ex309 output module provides a decoded video signal and an audio signal to the outside, respectively. When the ex309 output module provides a video signal and an audio signal, signals can be temporarily stored in ex318 and ex319 and other buffers, so signals are played back in synchronization with each other. In addition, the ex300 TV can read multiplexed data not broadcast and other, but ex215 and ex216 media for recording, such as magnetic disk, optical disc and SD card. In the following, a configuration will be described in which the ex300 television encodes the audio signal and video signal, and transmits data to the outside or writes data to the recording medium. On the ex300 TV, after the user operation using the ex220 remote control and the other, the ex304 audio signal processor encodes the audio signal, and the ex305 video signal processor encodes the video signal under the control of the ex310 control module using the encoding method described in each form. The ex303 demultiplexer / multiplexer multiplexes the encoded video signal and the audio signal, and provides the resulting signal to the outside. When the multiplexer / demultiplexer ex303 multiplexes the video signal and the audio signal, the signals can be temporarily stored in buffers ex320 and ex321 and others, so the signals are reproduced in synchronization with each other. Here the ex318, ex319, ex320 and ex321 buffers can be multiple as shown, or at least one buffer can be shared on an ex300 TV set. In addition, the data can be stored in the buffer, so overflow and underflow can be avoided, e.g. between the ex302 modulator / demodulator and the ex303 demuxer / demultulator. signals can be temporarily stored in buffers ex320 and ex321 and others, so signals are played back in synchronization with each other. Here the ex318, ex319, ex320 and ex321 buffers can be multiple as shown, or at least one buffer can be shared on an ex300 TV set. In addition, the data can be stored in the buffer, so overflow and underflow can be avoided, e.g. between the ex302 modulator / demodulator and the ex303 demuxer / demultulator. signals can be temporarily stored in buffers ex320 and ex321 and others, so signals are played back in synchronization with each other. Here the ex318, ex319, ex320 and ex321 buffers can be multiple as shown, or at least one buffer can be shared on an ex300 TV set. In addition, the data can be stored in the buffer, so overflow and underflow can be avoided, e.g. between the ex302 modulator / demodulator and the ex303 demuxer / demultulator.

[0261] Also, the ex300 television may include a configuration for receiving an AV input signal from a microphone or camera other than a configuration for receiving audio and video data from broadcast or recordable media, and may encode the received data. Although the ex300 TV can encode, multiplex, and provide data externally in the description, it may only be able to receive, decode and provide data outside, but not to encode, multiplex, and provide data outside.

[0262] Furthermore, when the ex218 player / writer reads or writes data multiplexed from or to to the recording medium, one device from the ex300 television and the ex218 player / recorder can decode or encode the multiplexed data, and the ex300 television and the ex218 player / recorder can share decoding or coding.

[0263] As an example, FIG. 20 shows the configuration of an ex400 player / recorder when data is being restored from or burned on an optical disk. The ex400 player / recorder contains components ex401, ex402, ex403, ex404, ex405, ex406 and ex407 described later. The ex401 optical head radiates a laser spot in the recording surface of the ex215 recording medium, which is an optical disk to record information, and detects the light reflected from the surface to record an ex215 recording medium to read information. The ex402 modulator / demodulator electrically controls the semiconductor laser placed in the optical head ex401 and modulates the laser light according to the recorded data. The reproducing ex403 demodulation module amplifies the reproduced signal obtained by electric detection of light reflected from the surface for recording using a photodetector located in the optical head ex401, and demodulates the reconstructed signal, separating the signal component stored on the ex215 medium for recording to reproduce the necessary information. The ex404 cache temporarily stores the information to be written to the ex215 storage medium and the information to be read from the ex215 storage medium. The ex405 disk drive puts ex215 media into rotation. The ex406 servo controller moves the ex401 optical head, moving it to a pre-determined information path, while simultaneously controlling the ex405 motor rotary drive so that it follows the laser spot. The ex407 control module controls the entire ex400 player / recorder module. Read and write processes can be implemented using the ex407 system control module, using various information stored in the ex404 buffer and generating and adding new information as needed, and using the ex402 modulation and writing module, ex403 demodulation and demodulation module and ex406 server which saves and reproduces the information using the optical head ex401, while operating in a coordinated manner. The ex407 system control module contains, for example, a microprocessor and performs processing, causing the computer to execute a read and write program. using various information stored in the ex404 buffer and generating and adding new information as required, and using the ex402 modulating and saving module, ex403 demo module - ex406 servo controller, which records and restores information with the ex401 optical head, while simultaneously acting in a way coordinated. The ex407 system control module contains, for example, a microprocessor and performs processing, causing the computer to execute a read and write program. using various information stored in the ex404 buffer and generating and adding new information as required, and using the ex402 modulating and saving module, ex403 demo module - ex406 servo controller, which records and restores information with the ex401 optical head, while simultaneously acting in a way coordinated. The ex407 system control module contains, for example, a microprocessor and performs processing, causing the computer to execute a read and write program.

[0264] Although in the description the ex401 optical head radiates a laser spot, it can perform a high density recording using a near field lighting technique.

[0265] FIG. 21 shows an ex215 storage medium, which is an optical disk. On the recording surface of the ex215 record carrier, the guide grooves are spirally shaped, and the information path ex230 saves in advance the address information indicating the absolute position on the disk in accordance with the change in the shape of the guide grooves. The address information contains information needed to determine the location of the saved blocks ex231, which are modules for data storage. Restoring the ex230 information path and reading the address information on the device that saves and restores the data may lead to the location of the recording blocks. In addition, the ex215 storage medium contains an ex233 data storage area, an internal perimeter area ex232, and an external perimeter area ex234. The ex233 data storage area is an area to be used in the user data record. The inner perimeter area ex232 and the outer perimeter area ex234, which are respectively located inside and outside the data storage area ex233, are intended for special use, with the exception of user data recording. The player / recorder 400 reads and writes encoded audio data, encoded video data or multiplexed data obtained by multiplexing the encoded audio and video data, from and within the ex233 area of ​​writing the ex215 data to record data.

[0266] Although an optical disc having a layer such as a DVD and BD is described as an example in the description, the optical disk is not limited to this variant and may be an optical disk having a multilayer structure and which may be recorded on a part other than a surface. In addition, the optical disk may have a structure for multidimensional recording / playback, such as for storing information using colored light of different wavelengths in the same part of the optical disk and for storing information in different layers at different angles.

[0267] Furthermore, an ex210 car having an ex205 antenna can receive data from the ex202 satellite and the other and play the video on a display device, such as the ex210 car navigation system mounted in the ex210 car, in the ex200 digital radio system. In this case, the configuration of the automotive navigation system ex211 will e.g. be a configuration including a GPS receiver from the configuration shown in FIG. 19. The same will apply to the configuration of computer ex111, mobile phone ex114 and other devices.

[0268] FIG. 22A depicts an ex114 mobile telephone that uses a moving picture coding method and a method for decoding moving images described in the embodiments. Mobile phone ex114 has: ex350 antenna for transmitting and receiving radio waves via base station ex110, ex365 camera capable of recording moving and still images and ex358 display, such as liquid crystal display, for displaying data such as decoded video saved by the camera ex365 or received via the ex350 antenna. The ex114 mobile phone also has: a main body housing the keyboard ex366; ex357 audio output module, such as an audio output loudspeaker; ex356 audio input module, such as a microphone, for audio input; memory module ex367 for memorizing stored video or still images, recorded audio, encoded or decoded data of received video, still images, e-mails or other; and an ex364 socket, which is an interface module for a recording medium that stores data in the same way as memory module ex367.

[0269] Hereinafter, the configuration of the cellphone ex114 will be described with reference to FIG. 22B. In the ex114 mobile phone, the main control module ex360 designed to control each main body module containing the ex358 display and the ex366 keyboard is interconnected, via the ex370 synchronous bus, to the power circuit module ex361, the input ex362 control module, the ex355 video signal processor, the ex363 interface module camera, ex359 control module for liquid crystal display (LCD), modulator / demodulator ex352, multiplexer / demultiplexer ex353, ex354 processor for audio signals, socket ex364 and memory module ex367.

[0270] When the end-of-end or power key is turned ON (ON) by the user, the power circuit ex361 feeds the respective modules with battery power to activate the mobile phone ex114.

[0271] In the ex114 mobile phone, the audio signal ex354 processor processes the audio signals collected by the audio input module ex356 in voice chat mode to digital audio signals under the control of the ex360 main control module including CPU, ROM and RAM. The ex352 modulator / demodulator then performs spread spectrum processing on the digital audio signals, and the ex351 transceiver module performs digital-to-analog conversion and frequency conversion on the data to transmit the resulting data via the ex350 antenna. What's more, on the ex114 mobile phone, the ex351 transceiver module amplifies the data received by the ex350 antenna in voice chat mode and performs frequency conversion and analogue data conversion on the data. Then the ex352 modulator / demodulator performs the inverse processing of the spread spectrum on the data,

[0272] Furthermore, when an e-mail is sent in the data transmission mode, the text data of the e-mail entered using the keypad ex366 and other main body modules are sent to the main control module ex360 by the input control module ex362. The main ex360 control module causes the ex352 modulator / demodulator to perform distributed spectrum processing on text data, and the ex351 transceiver module performs digital-to-analog conversion and frequency conversion on the resulting data to send data to the ex110 base station via the ex350 antenna. When the e-mail is received, processing approximately the opposite of processing to send an e-mail is performed on the received data, and the resulting data appear on the display ex358.

[0273] When the video, still images or video and audio are either transmitted in a data transfer mode, the ex355 video signal processor compresses and encodes the video signals supplied with the ex365 camera using the moving picture coding method shown in each form (i.e. like a moving picture coding apparatus according to an aspect of the present invention), and transmits encoded video data to an ex353 multiplexer / demultiplexer. However, when the ex365 camcorder records video, still images and other, the ex354 audio signal processor encodes the audio signals collected by the audio input module ex356 and sends the encoded data to the ex353 multiplexer / demultiplexer.

[0274] The multiplexer / demultiplexer ex353 multiplexes encoded video data provided from the ex355 video signal processor and encoded audio data provided from the audio signal ex354 processor using a predetermined method. Then, the modulator / demodulator (modulative-demodulation system) ex352 performs the spread spectrum processing on the multiplexed data, and the ex351 transceiver module performs the digital-to-analog conversion and frequency conversion on the data to send the resulting data via the ex350 antenna.

[0275] When data of a video file that is associated with a web page is received, and others in a data transmission mode, or when an e-mail with a video and / or audio attachment is received to decode the multiplexed data received by the ex350 antenna, multiplexer / the ex353 demultiplexer demultiplexes the data multiplexed into a video data bit stream and the audio data bit stream, and provides encoded video data to the ex355 video signal processor, and encoded audio data to the audio signal processor ex354, via the synchronous ex370 bus. The ex355 video signal processor decodes a video signal using a decoding method corresponding to the moving picture decoding method shown in each form (i.e. acts as a device for decoding moving pictures according to an aspect of the present invention), and then the ex358 display, for example, displays video and still images contained in the video file associated with the website via the ex359 control module with LCD display. In addition, the audio signal ex354 processor decodes the audio signal, and the audio output module ex357 provides audio.

[0276] Also, similar to an ex300 television, a terminal such as an ex114 mobile phone probably has three types of implementation configurations including not only (i) an transceiver terminal including both a coding device and a decoding apparatus, but also (ii) a broadcast terminal containing only a coding device and (iii) a receiving terminal comprising only a decoding device. Although the digital broadcast system ex200 receives and transmits multiplexed data obtained by multiplexing audio data into video data in the description, the multiplexed data may be data received by multiplexing non-audio data, but text data associated with video into video data, and may not be multiplexed data, but with the video data itself.

[0277] As such, the method of encoding moving images and the method of decoding moving images in each form can be used in any of the devices and systems described. Thus, the advantages described in each of the forms can be obtained.

[0278] Furthermore, the present invention is not limited to the form, and various modifications and versions are possible without departing from the scope of the present invention.

(Figure 10) [0279] Video data may be generated by switching between (i) the moving picture coding method or a moving picture coding apparatus as shown in each embodiment, and (ii) a moving picture coding method or a moving picture coding apparatus compatible with each other. with another standard, such as MPEG-2, MPEG4-AVC and VC-1.

[0280] Here, when a plurality of video data that conforms to different standards is generated and then decoded, the decoding methods should be chosen to be compliant with different standards. However, since it is not possible to detect which standard each of the many video data to be decoded corresponds to, there is the problem of not being able to select the correct decoding method.

[0281] To solve the problem, the multiplexed data obtained as a result of multiplexing audio and other data into video data has a structure containing identification information indicating the standard in which the video data matches. In the following, a special structure of multiplexed data will be described comprising video data generated in the method of encoding moving pictures and a device for coding moving images shown in each form. The data is multiplexed with a digital stream in the MPEG2-Transport Stream format. [0282] FIG. 23 shows a structure of multiplexed data. As shown in FIG. 23, the multiplexed data may be obtained by multiplexing at least one of a video stream, an audio stream, a graphic presentation (PG) stream, and an interactive graphic stream. The video stream shows the main video and secondary video of the film, the audio stream (IG) shows the main part of the audio and the secondary part of the audio to blend with the main part of the audio, and the presentation graphics stream shows the subtitles of the movie, the main video is a normal video for display on the screen, and the secondary video is a video to display in a small window in the main video. In addition, the stream of interactive graphics presents an interactive screen for generation by arranging the GUI components on the screen. The video stream is encoded in the method of encoding moving images or by a moving picture coding apparatus shown in each form, or in a moving picture coding method or a moving picture coding apparatus in accordance with a classical standard such as e.g. MPEG-2, MPEG4-AVC. and VC-1.

[0283] Each stream included in the multiplexed data is identified by the PID. For example, 0x1011 is allocated to the video stream to be used for the video of the movie, 0x1100 to 0x111F are allocated to audio streams, 0x1200 to 0x121F are allocated to presentation graphic streams, 0x1400 to 0x141F are allocated to interactive graphics streams, 0x1B00 to 0x1B1F are allocated to streams video for use as secondary video film, and 0x1A00 to 0x1A1F are allocated to audio streams to be used for secondary video, to be mixed with the main audio.

[0284] FIG. 24 schematically shows a method for multiplexing data. First, a video ex235 stream consisting of video frames and an audio ex238 stream consisting of audio frames is converted into an ex236 stream of PES packets and an ex239 stream of PES packets, and then into ex237 TS packets and ex240 TS packets, respectively. Similarly, ex241 presentation graphics data and ex244 interactive graphic stream data is transformed into an ex242 stream of PES packets and an ex245 stream of PES packets, and then into ex243 TS packets and ex246 TS packets, respectively. These TS packets are multiplexed into a stream to obtain ex multip477 multiplexed data.

[0285] FIG. 25 illustrates a more detailed way of memorizing a video stream in a PES packet stream. The first strip in FIG. 20 shows a stream of video frames in a video stream. The second bar represents the stream of PES packets. As indicated by the arrows marked as yy1, yy2, yy3 and yy4 in FIG. 20, the video stream is divided into images as I pictures, B pictures and P images, each of which is a video presentation unit, and the images are stored in the contents of each PES packet. Each of the PES packets has a PES header, and the PES header includes a Presentation Time-Stamp (PTS) indicating the display time, and a decode time stamp (Time-Stamp decoding) indicating the decode time of the image.

[0286] FIG. 26 shows the format of the TS packets to be ultimately stored in the multiplexed data. Each of the TS packets is a fixed-length 188-byte packet containing a 4-byte TS header including information such as PID for identifying the stream and 184-byte TS data for storing data. PES packets are appropriately divided and stored in TS data. When using BD ROM, each TS packet receives an additional 4-byte TP_Extra_Header label, which gives 192-byte native packets. These parent packages are stored in multiplexed data. The TP_Extra_Header label stores information such as arrival time stamp (Arrival_Time_Stamp, ATS). ATS shows the start time of transmission in which each TS packet is to be forwarded to the PID filter. The parent packets are arranged in multiplexed data as shown in the bottom portion of FIG. 21. Numbers increasing from the beginning of multiplexed data are called the parent package numbers (SPN).

[0287] Each of the TS packets included in the multiplexed data includes not only the audio, video, subtitle and other streams, but also the Program Association Table (PAT), the Table Map Matrix (PMT). , and Program Time Stamp (PCR). PAT shows what indicates the PMT PID used in the multiplexed data, and the PID of the PAT itself is recorded as zero. PMT remembers PID of video, audio, subtitle and other streams contained in multiplexed data, and information about attributes of streams corresponding to PID. PMT also has various descriptors related to multiplexed data. The descriptors contain information such as copy control information showing whether copying of multiplexed data is allowed or not.

[0288] FIG. 27 shows in detail the structure of PMT data. The PMT header is placed at the top of the PMT. The PMT header describes the length of the data contained in PMT and others. Behind the PMT header, there are many multiplexed data descriptors. Information, such as information on copy control, is described in the descriptors. Behind the descriptors, there are a plurality of stream information units relating to the streams included in the multiplexed data. Each stream information unit includes stream descriptors, each of which describes information such as the stream type to identify the stream compression codec, the PID stream, and stream attribute information (such as frame rate or aspect ratio).

[0289] When the multiplexed data is recorded on the recording medium and the other, they are stored together with the data files of the multiplexed data.

[0290] Each of the multiplexed data information files is information about managing the multiplexed data, as shown in FIG. 28. The multiplexed data information files correspond one-to-one with multiplexed data, and each of the files contains information about multiplexed data, stream attribute information, and an entry map.

[0291] As shown in FIG. 28, the multiplexed data includes system rate, start time, and end time. The system rate indicates the maximum transmission rate at which the target system decoder described hereinafter transmits the multiplexed data to the PID filter. The ATS intervals contained in the multiplexed data are set not greater than the system speed. The start playback time indicates the PTS in the video frame at the beginning of the multiplexed data. The interval of one frame is added to the PTS in the video frame at the end of the multiplexed data, and the PTS is set as the end time of the reproduction.

[0292] As shown in FIG. 29, the attribute information unit is stored in the stream attribute information, for each PID of each stream contained in the multiplexed data. Each unit of attribute information contains various information depending on whether the respective stream is a video stream, an audio stream, a presentation graphics stream, or a stream of interactive graphics. Each unit of information about the attributes of the video stream carries information about what compression codec has been used to compress the video stream, and the resolution, aspect ratio and frame rate of the image data contained in the video stream. Each unit of information about the attributes of the audio stream carries information about which compression codec was used to compress the audio stream, how many channels contains the audio stream, which language supports the audio stream, and how high the sampling rate is. Information about the attributes of the video stream and information about the attributes of the audio stream are used to initialize the decoder before the player starts playing information.

[0293] In the present embodiment, the multiplexed data to be used is of the streaming type included in the PMT. Furthermore, when the multiplexed data is recorded on the recording medium, information about the video stream attributes included in the multiplexed data is used. More specifically, the moving picture coding method or moving picture coding apparatus described in each embodiment includes a step or module for allocating unambiguous information indicating video data generated by the moving picture coding method or a device for encoding moving images in each form, to the type of stream included in the picture. PMT or in information about the attributes of the stream.

[0294] Furthermore, FIG. 30 shows the steps of a method for decoding moving images according to the present embodiment. In step exS100, the stream type included in the PMT or information about the attributes of the video stream is obtained from the multiplexed data. Then, in step exS101, it is determined whether the stream type or information about the attributes of the video stream indicates whether it does not indicate that the multiplexed data is generated by the coding method of the moving images or by the apparatus for encoding the moving images in each form. When it is determined that the stream type or information about the stream attributes indicates that the multiplexed data is generated by the method of encoding the moving images or by the apparatus for encoding the moving images in each form, in step exS102, the decoding is performed by the method of decoding moving images in each form. In addition, when the stream type or information about the stream attributes indicates compliance with classical standards such as MPEG-2, MPEG4-AVC and VC-1, in step exS103 decoding is performed by a method of decoding moving images conforming to classical standards.

[0295] As such, assigning a new unique value to the stream type or to the attribute of the stream attributes allows determining whether the moving picture decoding method or the moving picture decoding apparatus that is described in each form can perform decoding or not he can do it. Even if it is multiplexed data compliant with another standard, an appropriate decoding method or device can be chosen. In this way, it becomes possible to decode information without errors. In addition, a method or device for encoding moving images or a method or apparatus for decoding moving images in the present embodiment may be used in the devices and systems described above.

(Form 11) [0296] Each of the coding method of moving images, a moving picture coding apparatus, a moving picture decoding method, and a moving picture decoding apparatus in each form typically is obtained in the form of an integrated circuit or a large integration scale chip ( LSI). As an example of LSI, FIG. 31 shows the configuration of LSI ex500, which is made on one chip. LSI ex500 contains elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508 and ex509 described later, while the elements are connected with each other by bus ex510. The ex505 power supply system is activated by powering each element with energy when the power ex505 is on.

[0297] For example, when encoding is performed, the LSI ex500 receives the AV signal from the microphone ex117, the camera ex113, and the other via the ex 509 IO AV under control ex501 containing the ex502 CPU control module, ex503 memory controller, ex504 stream controller and control frequency ex512 control module . The received AV signal is temporarily stored in the external memory ex511, such as SDRAM. Under the control of the ex501 control module, the stored data is segmented into portions of data according to the amount of processing and the speed of sending signals to the ex507 processor. Then, the ex507 signal processor encodes the audio signal and / or the video signal. Here, the video signal encoding is the encoding described in each form. In addition, the ex507 signal processor sometimes multiplexes encoded audio data and encoded video data, and the IO0 ex 506 stream provides the externally multiplexed data. The delivered multiplexed data is sent to the ex-base base station or copied to the ex215 storage medium. When data sets are multiplexed, the data should be temporarily stored in the ex508 buffer so that the data sets are synchronized with each other.

[0298] Although the ex511 memory is an element outside of the LSI ex500, it may be in the LSI ex500. Ex508 buffer is not limited to one buffer, but may consist of buffers. In addition, the LSI ex500 can be made in one chip or multiple chips.

[0299] In addition, although the ex501 control module includes an ex502 CPU, ex503 memory controller, ex504 stream controller, control module ex512 of the control frequency, the configuration of the ex501 control module is not limited to those mentioned. For example, the ex507 signal processor may further comprise a CPU. Inclusion of another CPU to the ex507 signal processor can improve the processing speed. Furthermore, as another example, the ex502 CPU may serve as or be part of the ex507 signal processor, and may e.g. include an audio signal processor. In this case, the ex501 control module contains an ex507 signal processor or the ex502 CPU contains a part of the ex507 signal processor.

[0300] In the present context, the LSI name is used, but you can also use the IC name, LSI system, super LSI or ultra LSI depending on the integration scale.

[0301] Moreover, the ways of achieving integration are not limited to LSI and the special system or general purpose processor etc. also can achieve integration. A directly programmable gate array (FPGA) that can be programmed after LSI is produced or a reconfigurable processor for reconfiguration of the connection or LSI configuration can be used for the same purpose. Such a programmable logic device may typically perform a method for encoding moving images and / or a method of decoding moving pictures according to any of the above embodiments, loading or reading from memory etc. one or more programs that are included in the software or micro-programming.

[0302] In the future, with the advances in semiconductor technology, a completely new technology can replace LSI. Functional blocks can be integrated using such technology. It is possible that the present invention will be used in biotechnology.

(Figure 12) [0303] When the video data generated in the method of encoding the moving images or the moving picture coding apparatus described in each of the embodiments is decoded, compared to a situation in which video data conforming to a classical standard such as e.g. MPEG-2, MPEG4-AVC and VC-1, the amount of processing is likely to increase. Thus, the LSI ex500 must be set to a control frequency higher than that of the ex502 CPU to be used when video data conforming to the classic standard is decoded. However, when you set a higher control frequency, there is a problem with increased power consumption.

[0304] To solve the problem, a moving picture decoding apparatus, such as an ex300 and LSI ex500 television, is configured to determine what standard the video data is compatible with and to switch between control frequencies according to a particular standard. FIG. 32 shows the ex800 configuration in this embodiment. The ex803 module switches the control frequency to a higher control frequency when the video data is generated by the coding method of the moving images or the moving picture coding apparatus described in each form. Next, the ex803 switching frequency control module instructs the ex801 decoding processor that performs the encoding method of the moving images described in each form to decode the video data. When the video data conforms to the classic standard, the ex803 switching frequency control module sets the control frequency to a lower control frequency than for the video data generated by the moving picture coding method or the moving picture coding apparatus described in each form. Then, the ex803 switching frequency control module instructs an ex802 decoding processor that conforms to the classic standard to decode video data.

[0305] More specifically, the control frequency switching ex803 module comprises an ex502 CPU and an ex 512 control frequency control module in FIG. 26. Here, each of the decoding ex801 processors that performs the moving picture decoding method described in each of the embodiments, and an ex802 decoding processor that conforms to the classic standard, corresponds to the ex507 processor of the signals in FIG. 31. The ex502 CPU determines the standard with which the video data is compatible. Then the control module ex512 of the control frequency determines the control frequency based on the signal from the CPU ex502. In addition, the ex507 signal processor decodes video data based on the ex502 CPU signal. For example, it is possible that the identification information described in form 10 is used to identify video data. The identification information is not limited to that described in form 10, but may be any information provided that the information indicates what standard the compatible video data is. For example, when the standard with which the video data is compatible can be determined from an external signal to determine whether the video data is used for a television or disk, etc., the determination can be made based on such an external signal. In addition, the ex502 CPU selects a control frequency e.g. based on a look-up table in which video data standards are associated with control frequencies as shown in FIG. 34. The control frequency can be selected by storing the look-up table in the ex508 buffer and in the LSI internal memory, and in relation to the look-up table by the ex502 CPU. what standard the compatible video data is. For example, when the standard with which the video data is compatible can be determined from an external signal to determine whether the video data is used for a television or disk, etc., the determination can be made based on such an external signal. In addition, the ex502 CPU selects a control frequency e.g. based on a look-up table in which video data standards are associated with control frequencies as shown in FIG. 34. The control frequency can be selected by storing the look-up table in the ex508 buffer and in the LSI internal memory, and in relation to the look-up table by the ex502 CPU. what standard the compatible video data is. For example, when the standard with which the video data is compatible can be determined from an external signal to determine whether the video data is used for a television or disk, etc., the determination can be made based on such an external signal. In addition, the ex502 CPU selects a control frequency e.g. based on a look-up table in which video data standards are associated with control frequencies as shown in FIG. 34. The control frequency can be selected by storing the look-up table in the ex508 buffer and in the LSI internal memory, and in relation to the look-up table by the ex502 CPU. whether the video data is used for a television or disk, etc., the determination can be made on the basis of such an external signal. In addition, the ex502 CPU selects a control frequency e.g. based on a look-up table in which video data standards are associated with control frequencies as shown in FIG. 34. The control frequency can be selected by storing the look-up table in the ex508 buffer and in the LSI internal memory, and in relation to the look-up table by the ex502 CPU. whether the video data is used for a television or disk, etc., the determination can be made on the basis of such an external signal. In addition, the ex502 CPU selects a control frequency e.g. based on a look-up table in which video data standards are associated with control frequencies as shown in FIG. 34. The control frequency can be selected by storing the look-up table in the ex508 buffer and in the LSI internal memory, and in relation to the look-up table by the ex502 CPU.

[0306] FIG. 33 shows the steps to carry out the method in the present form. First, in step exS200, the ex507 signal processor receives identification information from the multiplexed data. Then, in step exS201, the CPU ex502 determines whether the video data is or are not generating the coding method and coding device described in each form, based on the identification information. When the video data is generated by the method of encoding moving pictures and by the moving picture coding apparatus described in each form, in step exS202, the CPU ex502 sends a signal to set the control frequency to a higher control frequency to the control frequency control module ex512. Then the control module ex512 of the control frequency sets the control frequency to a higher control frequency. On the other hand, when the identification information indicates that the video data conforms to a classic standard, e.g. MPEG-2, MPEG4-AVC and VC-1, in step exS203, the CPU ex502 sends a signal to set the control frequency to a lower control frequency. to the ex512 control frequency control module. Next, the control frequency control module ex512 sets the control frequency to a lower control frequency than when the video data is generated by the coding method of the moving images and the moving picture coding apparatus described in each form. The ex502 CPU sends a signal to set the control frequency to a lower control frequency to the control frequency control module ex512. Next, the control frequency control module ex512 sets the control frequency to a lower control frequency than when the video data is generated by the coding method of the moving images and the moving picture coding apparatus described in each form. The ex502 CPU sends a signal to set the control frequency to a lower control frequency to the control frequency control module ex512. Next, the control frequency control module ex512 sets the control frequency to a lower control frequency than when the video data is generated by the coding method of the moving images and the moving picture coding apparatus described in each form.

[0307] Furthermore, along with the switching of the control frequencies, the effect of the power behavior can be improved by changing the voltage supplied to the LSI ex500 or to a device containing LSI ex500. For example, when a lower control frequency is set, the voltage supplied to the LSI ex500 or to the device containing the LSI ex500 can optionally be set as a lower voltage than if the higher control frequency is set.

[0308] Furthermore, when the amount of processing to be decoded is larger, a higher control frequency can be set, and when the amount of processing to be decoded is smaller, a lower control frequency can be set as a method for setting the control frequency. Thus, the method of setting is not limited to those described earlier. For example, when the amount of processing for decoding MPEG 4 -AVC video data is greater than the amount of processing to decode the video data generated by the moving picture coding method and the moving picture coding apparatus described in each form, the control frequency is probably set inversely to the setting described above.

[0309] Furthermore, the method for setting the control frequency is not limited to a method for setting a lower control frequency. For example, when the identification information indicates that the video data is generated by the method of encoding the moving images and the moving picture coding apparatus described in each of the embodiments, the voltage supplied to the LSI ex500 or to the device containing the LSI ex500 is probably set higher. When the identification information indicates that the video data conforms to a classic standard such as MPEG-2, MPEG4-AVC and VC-1, the voltage supplied to the LSI ex500 or to the device containing the LSI ex500 is probably set to be lower. As another example, when the identification information indicates that the video data is generated by the method of encoding the moving images and the device for encoding the moving images described in each of the embodiments, the ex502 CPU control probably does not have to be suspended. When the identification information indicates that the video data conforms to a classic standard such as MPEG-2, MPEG4-AVC and VC-1, the ex502 CPU control is probably suspended at a given time because the ex502 CPU has additional processing capability. Even when the identification information indicates that the video data is generated by the moving picture coding method and the moving picture coding apparatus described in each form, in the case where the ex502 CPU has additional processing capability, the ex502 CPU control is probably suspended at a given time .

[0310] Therefore, the power conservation effect can be improved by switching between the control frequencies according to the standard with which the video data matches. In addition, when the LSI ex500 or the device containing the LSI ex500 is battery powered, the battery life can be extended by the power conservation effect.

(Form 13) [0311] There are cases where a plurality of video data compliant with different standards is provided to devices and systems such as television and cell phones. To enable the decoding of multiple video data compliant with different standards, the ex507 ex-L700 ex500 processor must comply with various standards. Nevertheless, the problems associated with increasing the scale of the LSI ex500 system and the increase in costs occur in the case of individual application of ex507 processors that are compliant with the relevant standards.

[0312] To solve the problem, the idea considered is a configuration in which a decoding processor to implement the moving picture decoding method described in each form and a decoding processor that complies with the classic standard, e.g. MPEG-2, MPEG4-AVC. and VC-1, are partially shared. Ex900 in FIG. 35A shows an example of a configuration. For example, the moving picture decoding method described in each of the embodiments and the MPEG4-AVC compliant picture decoding method have, in part together, processing fragments, such as entropy coding, inverse quantization, unblocking filtering, and motion compensation prediction.

It is possible with the ex902 module for MPEG-4 AVC compliant processing to be shared by common processing operations, and in the case of ex901 module for special decoding processing, processing use that is exclusive to the aspect of the present invention and is not MPEG-4 AVC compliant. It is possible that the ex902 module for decoding processing which is compatible with MPEG-4 AVC is shared by common processing operations, and that the ex901 module for decoding processing serves for processing that is exclusive to the aspect of the present invention and is not MPEG compliant -4 AVC. The decoding processing module for implementing the moving picture decoding method described in each form may be shared for processing intended for sharing,

[0313] Furthermore, ex1000 in FIG. 35B illustrates another example in which the processing is partially shared. This example uses a configuration including a dedicated decoder processor ex1001 that handles the processing exclusive of the present invention, a dedicated ex1002 decoding processor that supports processing unique to another classical standard, and an ex1003 decoding processor that handles shared processing between the moving picture decoding method the present invention and the classic method for decoding moving images. In this case, dedicated decoding processors ex1001 and ex1002 are not necessarily specialized for the processing according to the present invention and processing according to the classic standard, and may be processors that can implement general processing. In addition, the configuration of this embodiment can be implemented using LSI ex500.

[0314] As such, reducing the scale of the LSI system and reducing costs is possible by sharing a decoding processor for shared processing between the moving image decoding method according to the present invention and a method for decoding moving images conforming to the classic standard.

[Possibility of industrial application] [0315] The method of encoding moving pictures and the method for decoding moving images according to the present invention can be applied to any multimedia data. The image coding method and the image decoding method according to the present invention is useful as a method for coding images and a method for decoding moving pictures in a collection, transmission, telecommunications, etc. process using cell phones, DVD devices, personal computers, etc.

[List of reference numbers] [0316]

100 coder

105 subtraction system

110 transformation module

120 quantization module

130, 230 inverse quantization module

140, 240 combiner

150, 250 unblocking filter

160, 260 adaptive loop filter

170, 270 frame memory

180, 280 prediction module

190 entropy coder

200 decoder

290 entropy decoder

300, 400, 710 picture

31, 32, 3i, 41, 42 LCU

311, 312, 3, 1, 321 LCU

500 package header

510 IP header

520, 550 extension field

530 UDP header

540 RTP header

560 content header

570 NAL header s1 input signal s2 prediction signal e, e 'prediction error signal s', s ", s3 signal reproduced

Sun Patent Trust Plenipotentiary:

PL-PAT-2012-930

EP 2 903 267 B1