Image decoding method, image encoding method, image decoding device, image encoding device, and image encoding/decoding device.
Abstract
The image decoding method of an embodiment of the present invention includes: a context control step (S204) for determining the context to be used for a block to be processed, among a plurality of contexts; and an arithmetic decoding step (S210) for using the determined context to arithmetically decode a bit sequence corresponding to the block to be processed. In the context control step (S204), if a signal type for control parameters is a first type, the context is determined by using a condition that utilizes control parameters for both a left block and an above block, which are adjacent to the block to be processed (S206), and if the signal type for the control parameters is a second type, the context is determined by using a condition that does not utilize the control parameter for the above block (S207).

Term
7.2 yearsleft in the term
Expires 19 November 2033.
- Priority
- Filed
- Granted
- Today
- Expires
8 claims: 4 independent, 4 dependent
- 1REIVINDICACIONES Habiéndose descrito la invención como antecede se reclama como propiedad lo contenido en las siguientes reivindicaciones:5 1. Un método de codificación de imagen utilizando codificación aritmética, caracterizado porque comprende: binarizar un parámetro de control de un bloque actual en una imagen para generar una secuencia binaria;determinar un contexto para uso en el bloque 10 actual, de entre una pluralidad de contextos;y realizar codificación aritmética sobre la secuencia binaria utilizando el contexto determinado para generar una secuencia de bits que corresponde al bloque actual, en donde la determinación de un contexto incluye: 15 determinar un tipo de señal del parámetro de control del bloque actual;determinar el contexto bajo una primera condición de que los parámetros de control codificados de bloques aledaños del bloque actual se utilizan, cuando el tipo de 20 señal es un primer tipo, los bloques aledaños son un bloque izquierdo y un bloque superior del bloque actual;y determinar el contexto bajo una segunda condición de que se utilice un valor fijo predeterminado y los parámetros de control codificados del bloque izquierdo y el 25 bloque superior no se utilizan, cuando el tipo de señal es un 126 segundo tipo diferente del primer tipo, en donde el primer tipo es uno de (i) un indicador que indica si el bloque actual está dividido o no en una pluralidad de bloques, y (ii) un indicador que indica si el 5 bloque actual debe ser saltado o no, y el segundo tipo es un indicador que indica si se utiliza el modo de fusión o no para el bloque actual.
- 2El método de codificación de imagen de conformidad con la reivindicación 1, caracterizado porque la 10 determinación de un contexto incluye además:determinar si el parámetro de control codificado del bloque superior está disponible en la codificación o no, con base en una posición del bloque actual;y determinar el contexto bajo la segunda condición, 15 cuando el parámetro de control codificado del bloque superior no está disponible.
- 3El método de codificación de imagen de conformidad con la reivindicación 2, caracterizado porque en la determinación de un contexto, se determina que el 20 parámetro de control codificado del bloque superior no está disponible en la codificación, cuando el bloque actual está en un límite de corte.
- 4El método de codificación de imagen de conformidad con la reivindicación 2, caracterizado porque en 25 la determinación de un contexto, se determina si el parámetro 127 ' ÍNS’!V:·.. · . · / τ G P«U». FGTG-T de control codificado del bloque superior está disponible en la codificación o no, de acuerdo con una profundidad jerárquica de una unidad de datos a la cual pertenece el parámetro de control del bloque actual.
- 5El método de codificación de imagen de conformidad con una de las reivindicaciones 1 a 4, caracterizado porque la determinación de un contexto incluye además determinar un contexto de un parámetro de control de una segunda unidad más pequeña que una primera unidad por conmutación entre la primera condición y la segunda condición, con base en un parámetro de control de la primera unidad.
- 6Un aparato de codificación de imagen que utiliza codificación aritmética, caracterizado porque comprende:una unidad de binarización configurada para binarizar un parámetro de control de un bloque actual en una imagen para generar una secuencia binaria;una unidad de control de contexto configurada para determinar un contexto para uso en el bloque actual, de entre < una pluralidad de contextos;y una unidad de codificación aritmética configurada para realizar codificación aritmética en la secuencia binaria utilizando el contexto determinado para generar una secuencia de bits que corresponde al bloque actual, en donde la unidad de control de contexto está configurada para: 128 determinar un tipo de señal del páTáfftST'TO........de control del bloque actual;determinar el contexto bajo una primera condición en la que se utilizan parámetros de control codificados de 5 bloques aledaños del bloque actual, cuando el tipo de señal es un primer tipo, los bloques aledaños son un bloque izquierdo y un bloque superior del bloque actual;y determinar el contexto bajo una segunda condición de que se utilice un valor fijo predeterminado y los
- 710 parámetros de control codificados del bloque izquierdo y el bloque superior no se utilizan, cuando el tipo de señal es un segundo tipo diferente del primer tipo, en donde el primer tipo es uno de (i) un indicador que indica si el bloque actual está dividido o no en una
- 815 pluralidad de bloques, y (ii) un indicador que indica si el bloque actual debe ser saltado o no, y el segundo tipo es un indicador que indica si se utiliza el modo fusionado o no para el bloque actual. 129
Independent claims8
862 paragraphs in 24 sections, as filed
(54) Title: IMAGE CODING METHOD AND IMAGE CODING DEVICE.
(54) Title: IMAGE DECODING METHOD, IMAGE ENCODING METHOD, IMAGE DECODING DEVICE, IMAGE ENCODING DEVICE, AND IMAGE ENCODING / DECODING DEVICE.
(57) Summary
The image decoding method of an embodiment of the present invention comprises: a context control step (S204) to determine the context that is used for a block that is processed, among a plurality of contexts; and an arithmetic decoding step (S210) to use the determined context to arithmetically decode a bit stream corresponding to the block being processed. In the context control step (S204), if a signal type for control parameters is a first type, the context is determined by using a condition that uses control parameters for both a left and a top block, which are adjacent to the block being processed (S206), and if the signal type for the control parameters is a second type, the context is determined by using a condition that does not use the control parameter for the upper block (S207).
(57) Abstract
The image decoding method of an embodiment of the present invention ineludes: a context control step (S204) for determining the context to be used for a block to be processed, among a plurality of contexts; and an arithmetic decoding step (S210) for using the determined context to arithmetically decode a bit sequence corresponding to the block to be processed. In the context control step (S204), if a signal type for control parameters is a first type, the context is determined by using a condition that uses control parameters for both a left block and an above block, which are adjacent to the block to be processed (S206), and if the signal type for the control parameters is a second type, the context is determined by using a condition that does not use the control parameter for the above block (S207).
I KNOW
KCRfiMÍA Of. tCOXDMÍA
Λ,
V '.. >> 3
X ·· .. *
Institute
Mexican Industrial Property
PATENT TITLE NO. 337291
Headlines):
Home:
Denomination:
Classification:
Inventor (s):
PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA
20000 Mariner Avenue, Suite 200, Torrante, California, 90503, USA
IMAGE CODING METHOD AND CODING DEVICE
DE IMAGENES lnt.CI.8: H04N19 / 00; H04N7 / 12
HISAO SASAI; TAKAHIRO NISHI; YOUJI SHIBAHARA; TOSHIYASU SUGIO; KYOKO TANIKAWA; TORU MATSUNOBU with an international presentation:
June 2012 Patent Number: 328738
<img file="MX337291B_D0001.tif" />
I i
*
<img file="MX337291B_D0002.tif" />
Country:
US
PRIORITY
Date:
June 2011
Validity: Twenty high
Reference Expiration Date »is entered with fi
Informity with artice co ada from that of chos.
date
I have signed this Industrial Property (Diario
<img file="MX337291B_D0003.tif" />
nflacn
Number:
61/500,163
<img file="MX337291B_D0004.tif" />
Ugh «· .-» / · i with fundarm Federación (D.
de la Pi ': ¡ón de la S rticles 1 2nd fraction V, 6th fraction III, and 59 of the Industrial Property Law.
This patent has a validity of twenty non-extendable years, and it will be subject to the payment of the fee to keep the ey of the 05/1999, valid as the rganic valid.
Í7); 1 °, 3 ° (it is of the Property τ articles 6 ° fractions lll and 7 ° bis 2 of the I U2 / 08 199 * 25/10/19 *, 25/12/1997, 1 12); articles 1, 3 i (DOF 14/12/1999, reí ictions I and lll and 30 of the iftustrial Opportunity Statute (DOF 27/12/1999, amended on 10/10/2002, 07/29/2004, 04 / 05/2004 and 09/13/21.. -------- 1/26/2004, 06/16/2005, 06/25/2006, 0f 15 / 2009.05 / 01/2010, n < ¡A), 4 ° and 12 ° fraccionr ly lll d <Regulations of the
7/2002, 07/15/2004, 28 / I of Mexican Institute of the delega fa
72004 and 7> 9/2007); articles 1, 3, 4,
<img file="MX337291B_D0005.tif" />
Regional offices. Divisional Deputy Directors, Departmental Coordinators and other subordinates of the Mexican Industrial Institute. (DOF 12/15/1999, amended on 02/04/2000, 07/29/2004, 08/04/2004 and 09/13/2007).
<img file="MX337291B_D0006.tif" />
Issue Date: February 24, 2016
THE DIVISIONAL DIRECTOR OF PATENTS
<img file="MX337291B_D0007.tif" />
• Ί '
NAHANNY CANAL REYES
Arenal No. 550, Floor 1,
Pueblo Santa María Tepepan, Xochímiico, CP 16020,
Mexico City 'ei (55i 53 34 07 00' V'v \ vjmpj_.2pb mx
<img file="MX337291B_D0008.tif" />
MX / 2016/15108
<img file="MX337291B_D0009.tif" />
IMAGE CODING METHOD AND CODING DEVICE
<img file="MX337291B_D0010.tif" />
OF PICTURES
FIELD OF THE INVENTION
The present invention relates to an image decoding method, an image encoding method, an image decoding apparatus, an image encoding apparatus, and an image encoding and decoding apparatus, and in particular to a decoding method. of images, an image encoding method, an image decoding apparatus, an image encoding apparatus and an image encoding and decoding apparatus which utilize arithmetic encoding or arithmetic decoding.
BACKGROUND OF THE INVENTION
Natural image signals have statistical variations that show non-stationary behavior. One of the entropic coding methods that uses non-stationary statistical variations is Context Based Adaptive Arithmetic, Binary Coding (CABAC) (see NPL 1). CABAC is used as the ITU-T / ISOIEC standard for
<img file="MX337291B_D0011.tif" />
video encoding, H.264 / AVC.
The meaning of the terms used in the CABAC scheme will be described below.
(1) Context-Based Adaptive means adapting encoding and decoding methods to statistical variations. In other words, Context-Based Adaptive means predicting an appropriate probability as a probability of a symbol occurring along with an event of surrounding conditions, when the symbol is encoded or decoded. In coding, when determining a probability of occurrence p (x) of each value of a symbol S, a conditional probability of occurrence is applied using either a real event or a sequence of events F (z) as a condition.
(2) Binary means the representation of a symbol using a binary sequence. A symbol represented by a multi-value is mapped once in a binary sequence referred to as a binary series. A predicted probability (conditional probability) is switched and used for each of the sequence elements and the occurrence of one of the events of the two values is represented by a sequence of bits. Therefore, the probability of a value can be managed (initialized and updated) using a unit (unit of element
<img file="MX337291B_D0012.tif" />
binary) smaller than a unit of ^ mu-Xipo signal (see Figure 2 and others in NPL 1).
(3) Arithmetic means that the bit stream is generated not with reference to the mappings in a table but by calculation. In the encoding scheme using variable length code tables such as H.263, MPEG-4, and H.264, even each value of a symbol with a probability of occurrence greater than 0.5 (50%) needs to be associated with a binary stream (bit stream). Thus, a value with the largest probability needs to be associated with a bit for at least one symbol. In contrast, arithmetic coding can represent the occurrence of an event with a higher probability of an integer equal to or smaller than a bit. When (i) there is a type of signal in which the probability of occurrence of having the first binary value as 0 exceeds 0.9 (90%) and (ii) an event that has the first binary value as 0 successfully occurs N times, no there is a need to send 1 bit data N times for each value of 0.
List of References
Non-Patent Bibliography [NPL 1] Detlev Marpe et al., Context-Based
Adaptive Binary Arithmetic Coding in the H.264 / AVC Video Compression Standard, IEEE Transaction on circuits and Systems for video technology, Vol. 13, No. 7, July 2003.
<img file="MX337291B_D0013.tif" />
[NPL 2] Joint Collaborative Team on Vide<sup>1</sup>?
of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 4th Meeting: Daegu, KR, 20-28 January 2011, WD2: Working Draft 2 of High-Efficiency Video Coding
JCTVC-D503 http://wftp3.itu.int/av-arch/jctvc-site/20ll_01_D_Daegu/
JCTVC-D503.doc [NPL 3] Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 4th Meeting: Daegu, KR, 20-28 January 2011, Common test conditions and software reference configurations, JCTVC-E700 [NPL 4] Gisle Bjontegaard, Improvements of the BD-PSNR model,
ITU-T SG16 Q.6 Document, VCEG-AI11, Berlin, July 2008
BRIEF DESCRIPTION OF THE INVENTION
Technical problem
In this image encoding method and image decoding method, it is desired that the memory usage (used memory capacity) be reduced.
In this document, the present invention aims to provide an image encoding method or an image decoding method that can reduce memory usage.
Solution to the problem
In order to achieve the goal, the image decoding method according to one aspect of the
<img file="MX337291B_D0014.tif" />
The present invention is an image decoding method using arithmetic decoding, and the method includes: determining a context for use in a current block, from among a plurality of contexts; performing an arithmetic decoding on a bit sequence that corresponds to the current block, using the given context to decode a binary sequence, the bit sequence is obtained by performing an arithmetic encoding on a control parameter of the current block; and reverse binarizing the binary sequence to decode the current block control parameter, wherein determining a context includes: determining a signal type of the current block control parameter; determining the context under a first condition that the decoded control parameters of adjacent blocks of the current block are used, when the signal type is a first type, the adjacent blocks are a left block and an upper block of the current block; and determining the context under a second condition that the decoded control parameter of the upper block is not used, when the signal type is a second type different from the first type.
Advantageous Effects of the Invention
The present invention can provide an image encoding method or a '··' - Λ decoding method
Zj images that can reduce memory usage.
BRIEF DESCRIPTION OF THE FIGURES
FIGURE 1 is a functional block diagram of an image coding apparatus according to
Mode 1.
FIGltfÓ ^ 2 is a functional block diagram of a variable length encoding unit according to Mode IZ
FIGURE 3 is a table of a context model of a control parameter according to Mode 1.
FIGXJíóí 4 is a flow chart indicating an arithmetic coding method according to Modality
1.
/
FIGURE 5 is a functional block diagram of an image decoding apparatus according to
Mode 2.
FIGURE 6 is a functional block diagram of a variable length decoding unit according to Mode 2.
<td>Lci</td><td>FIGURE 7 is a</td><td>diagram of</td><td>flow</td><td>which indicates</td><td>a</td>
<td>method of</td><td>decoding</td><td>arithmetic</td><td colspan="2">in accordance with</td><td>the</td>
<td>Mode 2.</td><td></td><td></td><td></td><td></td><td></td>
<td>The</td><td>FIGURE 8 is a</td><td>diagram of</td><td>flow</td><td>which indicates</td><td>a</td>
<td>modification</td><td colspan="3">the decoding method</td><td>arithmetic</td><td>of</td>
according to Mode 2.
<img file="MX337291B_D0015.tif" />
illustrates partitioned blocks (a gomase
<img file="MX337291B_D0016.tif" />
FIGURE 9 tree structure) according to HEVC according to Mode 2.
FIGURE 10 illustrates a multi-layer block structure according to Mode 2.
FIGiptA 11 illustrates an arithmetic decoding method for split_coding_unit_flag according to
Mode 3.
FIGÜRA 12A is a table indicating a result of the verification in split_coding_unit_flag according to Mode 3.
FIGURE 12B is a table indicating a result of the verification in split_coding_unit_flag according to Mode 3.
FIGURE 13 illustrates an arithmetic decoding method for skip_flag according to Mode 3.
FIGURE 14A is a table indicating a result of verification in skip_flag according to Mode 3.
FIGURE 14B is a table indicating a verification result in skip_flag according to Mode 3.
FIGURE 15 is a table indicating an arithmetic decoding method for merge_flag according to the
Mode 3.
FIGURE 16A is a table indicating a result of the verification in merge_flag according to Mode 3.
H rM i · i XA; / FIGURE 16B is a table indicating a result of the mergef lag verification according to Mode 3.
FIGURE 17 illustrates context models using control parameter values that correspond to two adjacent blocks according to Modalities.
FIGURE 18 illustrates an increase in memory usage when using an upper block according to the Modalities.
FIGURE 19 illustrates a complete configuration of a content delivery system for implementing content delivery services.
FIG TIBA 20 illustrates a complete configuration of a digital broadcast system.
FIGURE 21 is a block diagram illustrating an example of a television configuration.
FIGURE 22 is a block diagram illustrating an example of an information playback / recording unit configuration that reads and writes information from or to a recording medium that is an optical disc.
FIGURE 23 illustrates an example of a configuration of a recording medium that is an optical disc.
FIGURE 24A illustrates an example of a cell phone.
FIGURE 24B illustrates a block diagram showing an example of a cell phone configuration.
'ΛΑ a
J STJ. X.1X ~ U.'KI. TJ · ΑΪ
FIG0RA 25 illustrates a multiplexed data structure.
FIGUR / 26 schematically illustrates how each stream is multiplexed into multiplexed data.
FIGURE 7 illustrates in more detail how a video stream is stored in a PES packet stream.
FIGURE ^ / 8 illustrates a structure of TS packets and source packets in multiplexed data.
FIGURE ^ illustrates a data structure of an FMT.
FIGURE / 30 illustrates an internal structure of multiplexed data information.
FIGURE ^ 31 illustrates an internal structure of current attribute information.
FIGURE / 32 illustrates steps to identify video data.
FIGURE 33<sup>z</sup> it is a block diagram illustrating an example of a configuration of an integrated circuit to implement the motion picture encoding method and the motion picture decoding method according to each of the Modalities.
FIGURE 34 illustrates a configuration for switching between drive frequencies.
FIGURE 3 ^ illustrates steps to identify video data and switch between drive frequencies.
FIGURE 3 / illustrates an example of an institu table. rc
L'r. LA i is queried in which the video data standards associate with the driving frequencies.
FIGURE 37A illustrates an example configuration for sharing a module of a signal processing unit and
FIGURE 37B illustrates another configuration example for sharing a module of a signal processing unit.
DETAILED DESCRIPTION OF THE INVENTION one of one of (Knowledge on which the present invention is based)
The present inventors have discovered the following problems.
<td>In</td><td>the</td><td>Coding</td><td>Registration Video</td><td>Efficiency</td>
<td>(HEVC, for</td><td>their</td><td>groin acronym</td><td>s) what is a</td><td>schedule of</td>
<td>coding</td><td>of</td><td>next video</td><td>generation the</td><td>model of</td>
<td>context in</td><td>the</td><td>coding and</td><td>decoding</td><td>several</td>
Control parameters are being studied (NPL 2). The control parameter is included in an encoded bitstream and is a parameter (flag, etc.) used in encoding or decoding processing. More specifically, the control parameter is a syntax element.
The context model is information indicating (i) what condition is considered for (ii) a signal of which unit (each element of a multiple value, a binary value, a binary sequence (binary series). In this document, that. one':
condition indicates which condition with the number of conditional elements is applied or what type of signal of a control parameter that is considered as a condition is appropriate.
As the conditions are divided into smaller categories, that is, as the number of conditions τ increases, the number of cases that remain true for the conditions decreases. As a result, since the number of workouts decreases, the precision of the predicted probability decreases (for example, see the dilution effect in NPL 1).
Additionally, a decrease in the number of conditions indicates that it is not considered a context (surrounding conditions) and that it is not adaptable to statistical variations.
In designing a context model, after determining a guide to designing the model, it is necessary to consider the validity of the model when conducting specialized checks for an image, such as checks for statistical variations in the details of a picture and in a control parameter to control the encoding and decoding of an image.
In H.264, the use of advanced events of a limited number to encode a symbol is a criterion of a rule, and context models are classified into four basic design types.
The first type and the second type refer to the encoding and decoding of a control parameter. ~
The first context model uses encoded values of up to two adjacent encoded values (see NPL 1). Although the definition of the two adjacent encoded values depends on each signal type, normally corresponding control parameter values are used which are included in adjacent blocks to the left and above the current block.
The second type of context models is a type to determine a context based on a binary tree block as a probability of occurrence. More specifically, the second type of context models applies to the mb_type and sub_mb_type control parameters.
The third type and the fourth type of context models refer to the encoding and decoding of residual values (residual data), such as image data. The third type uses only the past encoded or decoded values in the scanning order of frequency coefficients (or quantized coefficients). The fourth type determines a context according to decoded and accumulated values (levels).
The advantages of the design principle and the implementation of the probability transition model in
H.264, such as the first type, have already been studied and are i
fj apply to the HEVC being studied (see 'la ·. NPL. 2). For example, the first type (mode 1 cr-de · context that uses adjacent syntax elements) is being studied to be used for the control parameters alf__cu_f lag, split_coding_unit_flag, skip_flag, merge_flag, intra_chroma_pred_mode, inter_pred_flag, ref_idx_lc, ref_id_id_id, ref_id mvd_l0, mvd_ll, mvd_lc, no_residual_data_flag, cbf_luma, cbf_cb and cbf_cr (see 9.3.3.1.1 of NPL 2).
However, the present inventors have discovered that there is a problem in memory usage in encoding that uses the context model that uses the two adjacent blocks of the first type.
FIGURE 17 illustrates context models using control parameter values that correspond to the two adjacent blocks. Additionally, FIGURE 17 illustrates context models using adjacent blocks in H. 264.
Tree block C in FIGURE 17 includes a
<td>value of a</td><td>parameter of</td><td>control</td><td>I know that</td><td>is</td><td>encoded</td><td>and</td>
<td>decoded</td><td>at present.</td><td>When the</td><td>value</td><td>of the</td><td>parameter</td><td>of</td>
<td>control SE</td><td>encode it</td><td>they use</td><td>values</td><td>of</td><td>parameters</td><td>of</td>
SE control of the same type included in upper block A and left block B that are already coded. More specifically, the probability p (x) indicating whether the value
<img file="MX337291B_D0017.tif" />
! i
(.
x of the control parameter SE of block C (or the first binary vaTor ^ of the binary series of control parameter SET or 0 is predicted based on a conditional probability p (xl (condition A (value of the upper block) and condition B (value of the left block)) using, as conditions, the value of the control parameter SE of the upper block A and the value of the control parameter SE of the left block B.
FIGURE 18 illustrates an increase in memory usage when using an upper block.
In FIGURE 18, (xP, yP) is a position of an upper left pixel of a prediction unit (PU, motion prediction unit) that includes tree block C. In this document, tree block C is a block which includes a control parameter (for example, skip_flag) that is currently encoded. Additionally, (xP, yA) in FIGURE 18 is a pixel position that is included in tree block B and is used as a condition A (value of the control parameter skip_flag of the upper block). Additionally, (xL, yP) in FIGURE 18 is a pixel position that is included in tree block A and is used as a condition B (value of the control parameter skip_flag of the left tree block).
For the purpose of encoding or decoding the value of the block's skip_flag control parameter
<img file="MX337291B_D0018.tif" />
tree C, the encoding apparatus or the decoding apparatus needs to maintain the value of PU skip_flag (or a result of the determination of a condition) corresponding to the position (xP, yA) included in the upper tree block B and the position (xL, yP) included in the left tree block A. Assuming that the image has a horizontal width of 4 0 96 pixels, in order to encode a skip_flag control parameter, it is necessary to keep all the determination values included in the top row (Line L in FIGURE 18). In other words, a control parameter needs the memory capacity obtained by 4096 pixels / block size.
In this document, the tree block C that is encoded has variable sizes, for example, 64 x 64, 16 x 16 or 4 x 4. Additionally, the block size of the tree block C that is subsequently encoded or decoded cannot be predicted when the blocks in the top row (Line L) that include (xP, yA) are encoded or decoded. This is because the size of each of the blocks in the bottom row (row that includes tree block C) is not known when the top row (row that includes tree block A) is encoded or decoded. Thus, the coding apparatus or the decoding apparatus needs to maintain a value of a control parameter (or determination value) for each block size.
Τ '»
I Λ.
* Τ · minimum, assuming the smallest block size of all the sizes applied to the control parameters ”is used as the block size of the bottom row. The positions of the black circles in FIGURE 18 indicate conditions that have to be maintained, although the conditional values are not really necessary when the bottom row (row including tree block C) is encoded and decoded.
Additionally, the two adjacent blocks in FIGURE 18 (left tree block A and upper tree block B) follow the concept of adjacent blocks in H.264 and no new perspective is introduced on hierarchical block division. As described later, there are cases where these conditional values that are referred to in FIGURE 18 do not always make sense for control parameters adapted to the quadruple tree block recursive partition that is entered in the HEVC, because the control parameters follow recursive order of execution, hierarchical depth, or block positions.
As such, the present inventors have discovered that memory usage increases by utilizing the conditional values of the upper blocks in performing arithmetic encoding or decoding in control parameters. Additionally, the present inventors have
<img file="MX337291B_D0019.tif" />
discovered that memory usage further increases in HEVC.
In contrast, the image decoding method according to one aspect of the present invention is an image decoding method that uses arithmetic decoding, and the method includes: determining a context for use in a current block, from among a plurality of contexts; performing an arithmetic decoding on a bit sequence that corresponds to the current block, using the given context to decode a binary sequence, the bit sequence is obtained by performing an arithmetic encoding on a control parameter of the current block; and reverse binarizing the binary sequence to decode the current block control parameter, wherein determining a context includes: determining a signal type of the current block control parameter; determining the context under a first condition that the decoded control parameters of adjacent blocks of the current block are used, when the signal type is a first type, the adjacent blocks are a left block and an upper block of the current block; and determining the context under a second condition that the decoded control parameter of the upper block is not used, when the signal type is a second type different from the first type.
<img file="MX337291B_D0020.tif" />
With structure, the -ifi ςηcion imaging method can reduce memory usage. More specifically, in the image decoding method, since the control parameter of the upper block is not used for a control parameter of the second type, there is no need to maintain the control parameter of the second type of the upper block. With the structure, compared to the case where the left block and the top block are used as uniformly using a context model based on control parameter values of adjacent blocks, memory usage can be reduced according to the method of decoding images.
Additionally, according to the image decoding method, the appropriate context can be used for a hierarchical tree structure that is a data structure that is not considered in conventional H.264 and is unique to the new HEVC standard. Alternatively, a memory reference can be made.
Additionally, the second condition may be a condition that the decoded control parameters of the left and upper blocks are not used.
With the structure, the image decoding method can reduce memory usage by not using the left block control parameter in addition to the top block control parameter.
in:
<img file="MX337291B_D0021.tif" />
Additionally, in determining üTT context, a predetermined context can be determined under the second condition, as the context for use in arithmetic decoding of the current block, when the signal type is the second type.
With the structure, the image decoding method can reduce the amount of processing.
Additionally, the context can be determined under the second condition according to a hierarchical depth of a data unit to which the control parameter of the current block belongs, when the signal type is the second type.
With structure, the image decoding method can select an appropriate context while reducing memory usage.
Additionally, determining a context may further include: determining whether or not the decoded control parameter of the upper block is available in decoding, based on a position of the current block; and determining the context under the second condition, when the decoded control parameter of the upper block is not available.
With the structure, the image decoding method can reduce the amount of processing.
Additionally, in determining a context,
<img file="MX337291B_D0022.tif" />
<img file="MX337291B_D0023.tif" />
It can be determined that the control parameter decodlírcádCT ~~~ —— of the upper block is not available in the decoding, when the current block is at a cutoff limit.
Additionally, in determining a context, it can be determined whether the decoded control parameter of the upper block is available or not in decoding, according to a hierarchical depth of a data unit to which the block control parameter belongs. current.
Additionally, the second type can be a control parameter that has a predetermined data structure.
Additionally, determining a context may further include determining a context of a second unit control parameter smaller than a first unit by switching between the first condition and the second condition, based on a first unit control parameter. .
Additionally, the first type can be one of split_coding_unit_flag and skip_flag and the second type can be merge_flag.
With the structure, the image decoding method can appropriately reduce the memory usage of the second type control parameter without, for example, ceasing to evaluate a BD rate of an image.
Tu You'—
P '£ LA liAdditionally, the split codinq_un ± fc - é4 «' U<sup><</sup>* - ^ ueHe indicate whether or not the current block is partitioned into a plurality of blocks, the skip_flag can indicate whether or not the current block should be skipped, and the merge_flag can indicate whether or not a merge mode is used for the block current.
Additionally, decoding processes according to a first standard and decoding processes according to a second standard can be switched according to an identifier indicating one of the first standard and the second standard, the identifier is included in an encoded signal and context determination, implementation and reverse binarization can be performed as the decoding processes according to the first standard, when the identifier indicates the first standard.
Additionally, the image encoding method according to one aspect of the present invention is an image encoding method that uses arithmetic encoding, and the method includes: binarizing a control parameter of a current block to generate a binary sequence; determining a context for use in the current block, from among a plurality of contexts; and perform arithmetic encoding on the binary sequence using the given context to generate a sequence
<img file="MX337291B_D0024.tif" />
bit, where the determination of a determine a signal type of the current block control parameter; determining the context under a first condition that control parameters of adjacent blocks of the current block are used, when the signal type is a first type, the adjacent blocks are a left block and an upper block of the current block; and determining the context under a second condition that the upper block control parameter is not used, when the signal type is a second type different from the first type.
With structure, the image encoding method can reduce memory usage. More specifically, in the image encoding method, since the control parameter of the upper block is not used for a control parameter of the second type, there is no need to maintain the control parameter of the second type of the upper block. With the structure, compared to the case where the left block and the top block are used as uniformly using a context model based on control parameter values of adjacent blocks, the memory usage can be reduced according to the method image encoding.
Additionally, according to the image coding method, the appropriate context can be used for a hierarchical tree structure which is a structure of
<img file="MX337291B_D0025.tif" />
Data that is not considered in the H.264 is suitable for X. ~ and „. ,, - ias. ·, ico for the new HEVC standard. Alternatively, a memory reference can be made.
Additionally, the image decoding apparatus in accordance with one aspect of the present invention is an image decoding apparatus that uses arithmetic decoding, and the apparatus includes: a context control unit configured to determine a context for use in a current block, from among a plurality of contexts; an arithmetic decoding unit configured to perform arithmetic decoding on a bit stream corresponding to the current block, using the given context to decode a binary sequence, the bit stream is obtained by performing arithmetic encoding on a block control parameter current; and a reverse binarization unit configured to reverse binarize the binary sequence to decode the current block control parameter, wherein the context control unit is configured to: determine a signal type of the current block control parameter; determine the context under a first condition that decoded control parameters of adjacent blocks of the current block are used, when the signal type is a first type, the adjacent blocks are a left block and an upper block of the block
<img file="MX337291B_D0026.tif" />
i; ·.
<img file="MX337291B_D0027.tif" />
current; and determining the context under a second condition that the decoded control parameter of the upper block is not used, when the signal type is a second type different from the first type.
With the configuration, the image decoding apparatus can reduce memory usage.
Additionally, the image coding apparatus according to one aspect of the present invention is an image coding apparatus that uses arithmetic coding, and the apparatus includes: a binarization unit configured to binarize a control parameter of a current block to generate a binary sequence; a context control unit configured to determine a context for use in the current block, from among a plurality of contexts; and an arithmetic encoding unit configured to perform arithmetic encoding on the binary sequence using the determined context to generate a bit sequence, wherein the context control unit is configured to: determine a signal type of the block control parameter current; determining the context under a first condition that control parameters of adjacent blocks of the current block are used, when the signal type is a first type, the adjacent blocks are a left block and an upper block of the current block; and determine the
<img file="MX337291B_D0028.tif" />
context under a second condition that no upper block control parameter, when the signal type is a second type different from the first type.
With configuration, the image encoding apparatus can reduce memory usage.
Additionally, the image encoding and decoding apparatus in accordance with an aspect of the present invention is an image encoding and decoding apparatus including the image decoding apparatus and the image encoding apparatus.
The general or specific aspects can be implemented by a system, a method, an integrated circuit, a computer program or a recording medium, or by an arbitrary combination of the system, the method, the integrated circuit, the computer program and the recording medium.
The image decoding apparatus and the image coding apparatus in accordance with one aspect of the present invention will be specifically described with reference to the figures.
The embodiments described hereinafter indicate specific examples of the present invention. The values, shapes, materials, constituent elements, positions and connections of the constituent elements, steps and orders of the steps indicated in the Modalities are examples and do not limit the present invention. The constituent elements in the Modalities that are not described in the
Independent claims describing the more generic concept of the present invention are described as arbitrary constituent elements.
<img file="MX337291B_D0029.tif" />
<td rowspan="2">Modality</td><td colspan="4"> 1</td>
<td>I know</td><td>will describe</td><td>a device d</td><td>e coding</td>
<td>images</td><td>of</td><td colspan="2">according to the Modality</td><td>1 hereof</td>
<td>invention,</td><td>. The</td><td>apparatus</td><td>coding</td><td>images okay</td>
with Mode 1 it determines a context by switching between (1) the use of the upper block and (2) without the use of the upper block, according to a signal type of a control parameter in arithmetic coding. With the structure, deterioration in image quality can be suppressed and memory usage can be reduced.
First, a configuration of the image coding apparatus according to the
Mode 1.
FIGURE 1 is a block diagram illustrating an image coding apparatus 10 0 in accordance with
Mode 1.
The image coding apparatus 100 shown in FIGURE 1 is an image coding apparatus that uses arithmetic coding and encodes an input image signal 121 to generate a bit stream 124.
<img file="MX337291B_D0030.tif" />
JÁ1.
<img file="MX337291B_D0031.tif" />
The image encoding apparatus 100 iñ'clu5? É “arrar-w» idad. control 101, subtraction unit 102, transformation and quantization unit 103, variable length encoding unit 104, inverse quantization and inverse transformation unit 105, addition unit 106, intra-prediction unit 107, inter-prediction unit 108 and switch 109.
The control unit 101 calculates a control parameter 130 based on the input image signal 121 that is encoded. For example, control parameter 130 includes information about an image type of input image signal 121 that is encoded, a size of a motion prediction unit (prediction unit, PU) of the current block that is encoded, and information on the control of the movement prediction unit. In this document, the control parameter 130 (control data) itself must be encoded. In this way, control unit 101 sends control parameter 130 to variable length encoding unit 104.
The subtraction unit 102 calculates a residual signal 122 which is a difference (residual value) between the input image signal 121 and an image prediction signal 129 on a block unit basis.
The transformation and quantization unit 103 transforms the residual signal 122 into coefficient values
<img file="MX337291B_D0032.tif" />
INSTITUTE ¡y i> E LA ~ ¡
I λ
<img file="MX337291B_D0033.tif" />
frequency and quantifies the values of coefficients of «» ·· ΓΙΓ ι ·· ι> ι »ιι thousand JJj.3i_j_ · At the frequency obtained in quantized transformation coefficients 123 (residual data).
The inverse quantization and inverse transformation unit 105 inverse quantizes the quantized transformation coefficients 123 into frequency coefficient values and inversely transforms the obtained frequency coefficient values into a reconstructed residual signal 125.
The addition unit 106 adds the residual signal
125 to the image prediction signal 129 and sends a reconstructed image signal 126.
Intra prediction unit 107 performs intra prediction using the reconstructed image signal
126 to generate an image prediction signal 127. The inter-prediction unit 108 performs an inter-prediction using the reconstructed image signal 126 to generate an image prediction signal 128.
Switch 109 selects one of image prediction signal 127 and image prediction signal 128 and outputs the selected signal as image prediction signal 129.
The variable length encoding unit 104 encodes, using CABAC, the quantized transformation coefficients 123 and the control parameter
<img file="MX337291B_D0034.tif" />
Of AH
IN,
<img file="MX337291B_D0035.tif" />
0 for each input block to generate ^ T5r ~ T.'OL-ri? enfee..de · bits 124.
Next, the configuration of the variable length encoding unit 104 will be described.
FIGURE 2 is a functional block diagram of the variable length coding unit 104. The variable length coding unit 104 includes a binarization unit 141, a context control unit 142 and a binary arithmetic coding unit 143. The following describes the variable length encoding process in control parameter 130. Although the description about the variable length encoding process in the quantized transformation coefficients
123 omitted, the process can be implemented, for example, using a known technique.
The binarization unit 141 binarizes the control parameter 130 to generate a binary sequence 151. More specifically, the binarization unit 141 is a processing unit that performs II.1) binarization processing in accordance with NPL 1. Binarization 141 transforms control parameter 130 into binary sequence 151 referred to as a binary series for each signal type, according to a predetermined binarization method. The correspondence between signal types and binarization methods will be described later.
<img file="MX337291B_D0036.tif" />
When the input control parameter 13 0 is a binary value, such as an indicator, the binar unit 141 outputs the control parameter 130 as the binary sequence
151 How is it going.
Context control unit 142 determines a context for use in arithmetic encoding of control parameter 130 included in a current block that is processed, from among a plurality of contexts (a probability state table). Additionally, the context control unit 142 sends a context index 152 specifying the determined context to the binary arithmetic coding unit 143.
More specifically, context control unit 142 is a processing unit that performs 2) context modeling in accordance with NPL 1. Context control unit 142 sequentially receives a plurality of elements included in binary sequence 151 sent from the binary arithmetic coding unit 143. Context control unit 142 selects one of the contexts that are used for the binary value of control parameter 130, according to the signal type of control parameter 130 and an element position of the binary value in binary sequence 151 and sends, to binary arithmetic encoding unit 143, the context index
152 which is an index indicating the selected context.
<img file="MX337291B_D0037.tif" />
Additionally, the control unit Τίδ 'óólYCéxtó' 142 maintains the table of probability state values (context index values) obtained by dividing the elements in the binary sequence of control parameter 130 under conditions of conditional probabilities, such as context, and initializes and updates the probability state table.
Additionally, the context control unit 142 maintains a state (probability state index) for each occurrence condition τ (for each context), as an additional division of a signal type (for each element number in the binary sequence of control parameter 130 when the number of elements in the binary sequence is two or more; the same will apply later). The state is represented by the total value of 7 bits by combining the probability of occurrence P (internal relationship, typically a 6-bit value) which is the lowest probability of one of two values 0 and 1 and a value of 1 bit indicating which of the values has the highest probability. Additionally, maintaining a state means initializing and updating the state. For example, the update corresponds to changing the indexing indicating a current probability state (i.e. a probability) as a transition between 64 finite states as in H.264.
When an X event occurs on the most likely side
Τ
Ο λ t. '. «Τ. ·.
<img file="MX337291B_D0038.tif" />
Having the highest probability between the two values, a ratio of the probability on the most probable side increases slightly. For example, the context control unit 142 may slightly increase the probability ratio on the most probable side by increasing or decreasing, by 1, the value of the probability state index corresponding to 64 tables. On the other hand, when a Not-X event occurs that has the lowest probability (versus the predicted probability), the context control unit 142 greatly decreases the ratio of the most possible probability maintained based on a scale coefficient default to (for example, ~ 0.95) (see
FIGURE 6 in the NPL 1). The context control unit 142 according to Mode 1 makes a transition and maintains a state, based on a corresponding table index change value in order to be associated with the change in consideration of a as in H .264.
Binary arithmetic coding unit 143 performs arithmetic coding on the binary sequence
151 using the context determined by context control unit 142 to generate bit stream 124 (bit stream).
More specifically, binary arithmetic coding unit 143 is a processing unit that performs 3) binary arithmetic coding in accordance with
<img file="MX337291B_D0039.tif" />
NPL 1. Binary arithmetic coding unit 143 performs arithmetic coding on the binary sequence
151 using the context specified by context index 152 to generate bit stream 124. In this document, arithmetic encoding is to handle events that occur for control parameters 13 0 of various signal types as a cumulative sum of probabilities and determine matches between events by reducing the range to a predetermined range on a number line.
First, the binary arithmetic coding unit 143 divides the number one line into two half sections, according to the probabilities of occurrence of two possible values of the supplied binary value from the context control unit 142. When the current value that occurs for the binary value (for example, 0) is a value with a higher probability (exceeding 0.5 (for example, 0.75)), the binary arithmetic coding unit 143 maintains the lowest limit Low at the range on the number line without change and set a value that corresponds to a result of multiplying a 0.95 scale coefficient once by the probability of 0.75 this time, to a new range. On the other hand, when the generated binary value is actually a predicted value with a lower probability, the binary arithmetic coding unit 143 changes the lower Low limit for the higher probability and changes the range according to the lower probability. The sections 'TnálitTeñen' agree with a cumulative sum of the results of multiplications of the probability ranges. When a value with a lower probability successfully occurs, the range length precision soon becomes lower than the precision that can be assured by a calculation. In this document, the binary arithmetic coding unit 143 enlarges (normalizes again) the range to maintain precision and sends the bit sequence indicating the current range. Conversely, when a value with a higher probability (0.95, etc.) occurs successfully, the probability values can take a number of calculations (state transitions in the case of an implementation by a table) until the length of the range becomes shorter than a predetermined length even with multiplication of values. In this way, the number of symbols that can accumulate until the bit is sent is large.
The figure. 3 is a table into which the control parameters 130 each using a context model based on a value of the control parameter 13 0 of an adjacent block are classified.
The meaning of each column will be described from the left of the table.
ι.
<img file="MX337291B_D0040.tif" />
(c2) The signal type (syntax element) indicates a specific name of a signal type of control parameter 130. The meaning of each of the signal types will be described later.
(c3) The binarization scheme indicates a binarization scheme that is applied to control parameter 130 (SE) specified in the column immediately to the left. The binarization unit 141 performs the binarization process. In the column, Fixed length means that the binar unit 141 outputs the value of control parameter 130 in the section immediately to the left as a binary sequence (binary series) of a fixed length. In HEVC, a signal type of control parameter 13 0 whose name ends with a flag is a binary value of either 0 or 1. Thus, binarization unit 141 sends only the first element (binldx = 0) as the element of the binary sequence 151 and does not send the elements after the second element (binldx> = 1). In other words, binarization unit 141 outputs the value of control parameter 130 as binary sequence 151 as is.
Additionally, Variable Length in Column means that binarization unit 141 maps, in a binary sequence, the value of control parameter 130 using binary sequences with respective variable lengths whose values are associated to have lengths. • .A. · ;:
<img file="MX337291B_D0041.tif" />
binary in ascending order of the frequencies of occurrence (binary series or binary sequences each with the number of elements> 1) and sends the binary sequence. For example, binarization unit 141 employs and outputs a scheme according to the signal type, such as a unary (truncated) scheme and a combination of the unary scheme and other exponential Golomb schemes (see A. Binarization of NPL 1) . In the case of Variable Length, the number of elements of the binary sequence 151 is sometimes limited to 1 or equal to or greater than 2. A reverse binarization unit in an image decoding apparatus described below performs the transformation inverse to the binarization scheme to reconstruct the input binary sequence into a multiple value or indicator value.
Regarding (c4) Context index of the first element (binldx = 0), the context control unit 142 indicates the choice of a context index (increment) that is applied to the first element included in a binary sequence generated according with the binarization scheme specified in the c3 column. In the column, 0, 1, 2 indicates that the context control unit 142 selects and applies one of three probability state tables (contexts). For example, three context indexes with detailed conditions are prepared for signal type one skip_flag, that is, three contexts are prepared and the
Wicked
MEXICAN INSTITUTE A «• WfsraAi in 'Tos'“' íñaiGt'S —- column c4 indicates element (binldx = 0) arithmetic coding is done context.
Similarly, 0, 1, 2, 3 in which the context that applies to the first included in binary sequence 151 is selected from one of four values, either 0, 1, 2, or 3. Binary sequence 151 is obtained by map, in a binary sequence, the value of control parameter 13 0 of the signal type specified in the c2 column, according to the binarization scheme in the c3 column. The conditional expressions in the column will be described later.
With respect to (c5) Left Block Condition L (condL), the context control unit 142 indicates the left block condition to select one of 0, 1 and 2 in column c4. The left block condition L has a true or false value determined according to the value of the left block control parameter that corresponds to the control parameter that is encoded (or decoded).
For example, in the case where the control parameter (SE) is skip_flag, the left block condition L has the true value when skip_flag [xL] [yL] indicates true (for example, 1) and has the false value when it indicates false (for example, 0).
With respect to (c6) Upper block condition i
<img file="MX337291B_D0042.tif" />
A, the context control unit 142 indicates the upper block condition for selecting one of 0, 1 and 2 in the encoding or decoding elements of a sequence specified in column c4. The upper block condition A has a true or false value determined according to the value of the control parameter of the upper block that corresponds to the control parameter that is encoded (or decoded). For example, in the case where the control parameter (SE) is skip_flag, the upper block condition A has the true value when skip_f lag [xA] [yA] indicates true (for example, 1) and has the false value when indicates false (for example, 0).
Although not illustrated, the signal type of more than two bits is associated with (c7) The context increment that is applied to binldx> = 1. This (c7) indicates the context model applied by the context control unit 142 to a binary value after the second element in the binary sequence (binary value of a binary sequence element that includes a binldx index value> = 1).
In the Mode 1 encoding method, the following operations are switched according to the signal type of control parameter 13 0 for the left block condition L and the upper block condition A (operated using different patterns):
(Pattern 1) Using two adjacent blocks (one
<img file="MX337291B_D0043.tif" />
ΠΟΙ il -. 'ST.'Tírr :; μξ. \,? -: oi Λ í RG / 'ü'D.'. u í'íózíL.
left block condition determination value L and an upper block condition determination value A);
(Pattern 2) Using an adjacent block (only one left block condition determination value L); and (Pattern 3) Using zero adjacent blocks (not using a left block condition determination value L or an upper block condition determination value A).
FIGURE 4 is a flow chart indicating an image encoding method according to Mode 1 which is performed by the variable length encoding unit 104 shown in FIGURE 2.
First, the binarization unit 141 maps the value of the control parameter 130 in a binary sequence according to a scheme corresponding to the signal type of the control parameter 130 (S101).
The context control unit 142 then obtains a basic value of a context for use in an arithmetic encoding of control parameter 130 (S102).
For example, the context control unit 142 determines the basic value according to the type of image (I, Ρ or B).
The context control unit 142 then determines a context value using one of the patterns
C-ε LA i ϋίΛϋ INrUiT ^ JAL
<img file="MX337291B_D0044.tif" />
to 3, based on the signal type of the ront-roi parameter
130 (S103). In this document, determining a context value is equivalent to determining an adjustment value (increment value Ctxldxlnc) for the basic value of the context.
First, context control unit 142 determines the signal type of control parameter 130 (S103). When the signal type of the control parameter 130 is the first type that corresponds to pattern 1 (the first type in S104), the context control unit 142 determines a context value using a determination value deduced from values of control parameters of two adjacent blocks (block A and block B) (S105). In other words, the context control unit 142 determines a context under a condition that the control parameters of the two adjacent blocks of the left block and the top block are used. In this document, the context control unit 142 uses both a result of the determination of (c5) condL and a result of the determination of (c6) condA in FIGURE 3. Accordingly, data from one row of images is maintained for control parameters of the first type.
On the other hand, when the signal type of the control parameter 13 0 is the second type corresponding to pattern 2 (the second type in S104), the context control unit 142 determines a context value value of a control parameter of a block '·' · ¿ϊ <sup>r</sup>'. V<sup>1</sup> utxií Éíasan
Adjacent INDUSTRIAL (a
<img file="MX337291B_D0045.tif" />
immediately adjacent block in encoding order) (S106). In other words, the context control unit 142 determines the context value under a condition that the upper block control parameter is not used.
On the other hand, when the signal type of the control parameter 130 is the third type that corresponds to pattern 3 (the third type in S104), the context control unit 142 fixedly determines a context value without using the control parameters both the upper block and the left block (S107).
Then, the context control unit 142 adds the increment determined in Step S103 to the base value of the context index determined in Step S102 to determine a context index value (S108).
Finally, binary arithmetic encoding unit 143 performs arithmetic encoding on the binary value of the first element using the context value specified by the context index value determined in Step S108 to generate the bit stream (bit stream 124) (S109).
Then, when the processes in Steps S102 to S109 are not executed on all the elements included in the binary sequence (not in S110), the encoding unit of
<img file="MX337291B_D0046.tif" />
<img file="MX337291B_D0047.tif" />
variable length 104 performs the processes of Steps S102 to S109 on the next element included in the binary sequence. On the other hand, when the processes in Steps S102 to S109 are completed in all the elements included in the binary sequence (If in S110), the variable length encoding unit 104 terminates the encoding processing in the block control parameter current.
As described above, the image encoding apparatus 100 according to Mode 1 determines a context using the upper block in performing arithmetic encoding on the control parameter of the first type and determines a context without using the upper block to the control parameters of the second and third type.
Compared to the case where the left block and upper block are used as uniformly using a context model based on adjacent block control parameter values, the image encoding apparatus 100 can reduce memory usage with the configuration . In this way, the image encoding apparatus 100 can suppress deterioration in image quality and reduce memory usage.
Mode 2
Mode 2 will describe an image decoding apparatus that decodes the stream of
INSi Ιϊϋϊό í'j £ LA í'l'Oi'lH i ?? '. O
INDUSTRIAL bits 124 generated by the encoding apparatus
<img file="MX337291B_D0048.tif" />
images
FIGURE 5 is a block diagram illustrating an image decoding apparatus 200 in accordance with Mode 2. Image decoding apparatus 200 is an image decoding apparatus using arithmetic decoding and decoding the bitstream 124 for generating an image signal 229. In this document, the bit stream 124 is, for example, generated by the image encoding apparatus 100.
The image decoding apparatus 200 includes a control unit 201, variable length decoding unit 202, inverse quantization unit 204, inverse transformation unit 205, addition unit 206, intra-prediction unit 207 and inter-prediction unit 208 .
The image decoding apparatus 200 performs a decoding process for each bit stream of a predetermined processing unit. The processing unit is, for example, a cutting unit or a block unit.
The variable length decoding unit
202 performs arithmetic decoding on bit stream 124 to generate control parameter 230 (control data syntax element) and quantized transform coefficients 223 (values of residual data syntax elements)
The control unit 2 01 receives the generated control parameter 230.
Control unit 201 controls each of the processing units included in image decoding apparatus 200, in accordance with control parameter 230.
The inverse quantization unit 204 inverse quantizes the quantized transformation coefficients 223 into orthogonal coefficients 224.
The inverse transformation unit 205 inversely transforms the orthogonal transformation coefficients 224 to reconstruct a residual signal 225. The addition unit 206 adds the residual signal 225 to an image prediction signal (image signal 229) to generate a signal of decoded image 226.
Intra prediction unit 207 performs intra prediction using decoded image signal 226 to generate image prediction signal 227. Inter prediction unit 208 performs inter prediction using decoded image signal 226 to generate an image prediction signal 228.
Switch 2 09 selects one of the image prediction signal 227 and the transformation image prediction signal
<img file="MX337291B_D0049.tif" />
228 and outputs the selected signal as image signal 229 (image prediction signal).
Next, the configuration of the variable length decoding unit 202 will be described.
FIGURE 6 is a functional block diagram illustrating a configuration of the variable length decoding unit 202. The variable length decoding unit 202 includes a binary arithmetic decoding unit 243, a context control unit 242 and a unit reverse binarization 241. The following describes the variable length decoding process in control parameter 230. Although the description about the variable length decoding process in the quantized transform coefficients 223 is omitted, the process can be implemented, for example, using a known technique.
Context control unit 242 determines a context for the use in arithmetic decoding of control parameter 230 of the current block, from among a plurality of contexts. Additionally, the context control unit 242 sends a context index 252 that specifies the determined context to the binary arithmetic decoding unit 243.
More specifically, context control unit 242 uses the same context model as that of
<img file="MX337291B_D0050.tif" />
the context control unit 142 shown in FIGURE 2 as a maintained probability transition model.
When the arithmetic encoding unit 143 uses 64 probability states, the binary arithmetic decoding unit 243 also maintains the 64 probability states. This is because both the encoder and decoder need to interpret a range of the number line that is encoded in exactly the same way. In this way, the decoder uses the same pattern as the pattern selected by the encoder from the three patterns 1 to 3.
The arithmetic decoding unit 243 performs arithmetic decoding in the bit stream (bit stream 124) using the context determined by the context control unit 242 to reconstruct the binary stream 251. More specifically, the arithmetic decoding unit 243 reconstructs the input bit sequence in binary sequence 251, according to the context (probability state table) specified by the provided context index of the context control unit 242.
Reverse binarization unit 241 rebuilds binary sequence 251 into a control parameter 230 if necessary through the reverse binarization process. As such, the 142 context control unit included in the £
INS '
Γ / π / ϊο μξλ: ca? Jo
OF THE F «\ O?<sub>4</sub>'h'0Aí / ¡NDUSTRÍAl
<img file="MX337291B_D0051.tif" />
image encoding apparatus 100 and the 'Lili ida d do · context control 242 included in image decoding apparatus 200 use the same context model in both arithmetic encoding and arithmetic decoding of a certain control parameter signal type.
FIGURE 7 is a flowchart indicating an image decoding method according to Mode 2 which is performed by the variable length decoding unit 202.
First, the variable length decoding unit 202 obtains the bit stream 124 (S201).
Then, the context control unit 242 determines a signal type of a control parameter that is decoded, according to the data structure of bit stream 124 (S202).
The context control unit 242 then determines a basic value of a context for use in an arithmetic decoding of the control parameter that is decoded (S203). For example, the context control unit 242 determines the basic value according to the type of image (I, P or B).
The context control unit 242 then determines a context value using one of the patterns
<img file="MX337291B_D0052.tif" />
F
Hee
INSTITUTO IvILX:,. ·,; · :. ') U £ LA ΡΕΟγ7ΓΓ · λΩ industrial
<img file="MX337291B_D0053.tif" />
to 3, based on the signal type of the Hp control (S2 04). In this document, determining a context value is equivalent to determining an adjustment value (increment value Ctxldxlnc) for the basic value of the context. For example, context control unit 242 statically determines one of patterns 1 to 3 based on the signal type of the control parameter by following a predetermined table.
Context control unit 242 switches between adjacent blocks for use in determining a context to obtain a binary value of the first element included in binary sequence 251 using arithmetic decoding, according to the signal type of the control parameter .
First, context control unit 242 determines the signal type of control parameter 230 (S205). When the signal type is the first type that corresponds to pattern 1 (the first type in S205), the context control unit 242 determines a context value using control parameters from two adjacent blocks (S206). In other words, the context control unit 242 determines the context value under a condition that the decoded control parameters of the two adjacent blocks of the left block and the top block are used.
p-—
<img file="MX337291B_D0054.tif" />
· *> «Λ- v jj. νΛINSTITUTO MEXICANA 'V>
FROM THE FRCricDAi) \
INDUSTRIAL
On the other hand, when tino. . signal is the second type corresponding to pattern 2 (the second type in S205), the context control unit 242 determines a context value using a value of a control parameter of an adjacent block (an immediately adjacent block in the encoding order) (S207). In other words, the context control unit 242 determines the context value under a condition that the decoded control parameter of the upper block is not used.
On the other hand, when the signal type is the third type corresponding to pattern 3 (the third type in S205), the context control unit 242 fixedly determines a context value (S208). In other words, the context control unit 242 determines the context value under a condition that the decoded control parameters of the upper block and the left block are not used.
Then, the context control unit 242 adds the increment determined in Step S204 to the basic value of the context index determined in Step S203 to determine a context index value (S209).
Thereafter, the binary arithmetic decoding unit 243 determines one of the elements of the binary sequence through a decoding using the context value indicated by the context index value provided by the context control unit 242 (S210).
'O ir f; i R *
He passed'#<sup>1</sup>- S
Then, when the processes of the
S210 does not execute in all elements '± rTCÍuidos ^ eTr' ”binary sequence (Not in S211), the variable length decoding unit 202 performs the processes of Steps S203 to S210 in the next element included in the binary sequence.
Moreover, when the Steps processes
S203 to S210 are completed in all the elements included in the binary sequence (If in S211), the reverse binarization unit 241 changes one or more of the elements of the binary sequence 251 obtained by repeating the processes of Steps S203 to S210 more all at once to generate control parameter 230 (S212).
As described above, the image decoding apparatus 200 according to Mode 2 determines a context using the upper block in performing an arithmetic decoding on the control parameter of the first type and determines a context without using the upper block to the control parameters of the second and third type.
Compared to the case where the left block and top block are used as uniformly using a context model based on adjacent block control parameter values, the image decoding apparatus 200 can reduce memory usage with the configuration . In this way, the i Ni image decoding apparatus 200 can suppress deterioration in image quality and can reduce memory usage.
For example, when binary sequence 251 is an indicator and has only one element, i.e. binary sequence 251 is made up of 1 binary value, the reverse binar unit 241 may send the binary sequence
251 How is it going.
In addition to the above description, the control unit 101 or 201 can control each of the processing units or can refer to a value in a memory, through a signal line that is not illustrated.
Although the context control unit 14 2 or 24 2 switches between the three patterns 1 to 3 according to a signal type of a control parameter in the above description, it can switch between two of the patterns 1 to 3 according with the type of signal. In other words, the context control unit 142 or 242 can toggle between using and not using the upper block condition, according to a signal type of a control parameter.
Additionally, the context control unit
142 or 242 can change a method to switch between the selected context models in this way (including a case where the context model increment is changed, · the same will apply later) according to predetermined image information. For example, the / Vi / .1 context control unit 142 or 242 may also switch the switching policy itself, according '' to 'l'a “' Sárit ict ^ T ^ 'memory or memory size. horizontal width or a sample format of an image that affects the number of workouts in each context.
Although the context control unit 142 or 242 toggles between using and not using the upper block condition as the simplified description, the context control unit 142 or 242 can combine a case where the upper block is not available for switching and you can apply the combined case. For example, context control unit 142 or 242 can change the switching policy itself, depending on whether or not a cut that is processed is an entropic cut (entropy_slice_flag indicates 1 or 0).
Similarly, when the availability of the upper adjacent block cannot be assured, the context control unit 142 or 242 may change the switching policy so as not to use the upper block.
For example, as illustrated in FIGURE 8, context control unit 142 or 242 may switch the context model determination policy between the first determination criterion (S302) and the second determination criterion (S303), according to a parameter value of a predetermined unit. In this document, according to a parameter value of one unit
v-CJI & í default means according to whether or not a cut is an entropic cut as described above.
Additionally, the first determination criterion is a criterion based on which of the processes shown in FIGURE 7 is performed. The second determination criterion is a criterion that excludes Step S204 shown in FIGURE 7, and is, for example, a conventional criterion. This is equivalent to determining the context index increment, using a parameter of a default local unit and a parameter value of a unit greater than the default local unit.
In other words, the context control unit
142 or 242 may switch from a determination criterion that is applied to a unit smaller than the first unit, to another determination criterion based on a value of a control parameter of the first unit.
Additionally, the context control unit
142 or 242 can change the determination criteria that is used, according to the characteristics of an imaging system. For example, the context control unit 142 or 242 may change the determination criteria that is used, according to I-image ranges (IntraPeriod setting values).
Although the context control unit 142 or 242 toggles between the determination criteria according to. 1C A ./; í>: j C. l ·? · .'Ο V '
INSTITUTE · Λ · Λ.-ν. · ι, ·<sub>;</sub>Using the above conditions, you can switch if ^ eeüsÜÉS.l'MSá ^ fciHo the upper block.
Additionally, the context control unit
142 or 242 can determine whether an upper block control parameter is used or not, according to whether or not the upper block control parameter is available in encoding or decoding based on a position of the control parameter. In other words, the context control unit 142 or 242 can determine whether or not the upper block control parameter is available in decoding, based on a current block position, and can determine a context using one of the patterns 2 and 3 when the upper block control parameter is not available. Additionally, the context control unit 142 or 242 can determine whether or not an upper block reference value is available based on a tree structure for the TU, CU or PU block partition. In other words, the context control unit 142 or 242 can determine whether or not the upper block control parameter is available in decoding, according to the hierarchical depth of a data unit to which each of the data belongs. control parameters that are processed.
FIGURE 9 illustrates a relationship between an image, cuts and blocks according to the HEVC standard. An image is partitioned into one or more slices. In the example of
<img file="MX337291B_D0055.tif" />
FIGURE 9,,. . . , UMS7 image is partitioned into two co
PART 2) . One of the cuts includes the _301 ~ blocks (for example, tree blocks). In this document, block 301 is the largest unit as a certain control unit when a cut is partitioned to a predetermined size and is one root size when the unit is at the root in the hierarchically partitioned structure.
In the example of FIGURE 9, PART 2 starts from a block 301A and is made up of a sequence that includes blocks to the bottom right corner of the image through the shaded blocks 3 0 IB and 3 01C. One of the shaded blocks in FIGURE 9 is a tree block (TreeBlock) that is currently processed.
Each of blocks 301 includes N x M pixels.
One of blocks 3 01 is recursively partitioned inside (typically four). In other words, a Tree Block conceptually composes a Quad Tree Block. In tree block 301B shown in FIGURE 9, the upper right tree block obtained by partitioning shaded tree block 301B into four is recursively partitioned into four blocks twice. In other words, tree block 301B includes 10 logical drives from the top left zero drive to the ninth bottom right drive that are partitioned with a certain perspective.
In this document, the perspective <sub>nL</sub> indrca ^. ^ iy<sup>1 </sup>concept of a plurality of tree blocks having different depths with a root as the base point, such as a tree block with respect to a coding unit (CU) and a tree block with respect to residual_data. In this document, a value of each control parameter belongs to one of the leaf nodes.
In this document, whether or not a control parameter value of a certain type of signal included in a superior tree block is actually available depends on a type of tree block to which the control parameter belongs. In this way, the context control unit 142 or 242 can change a determination criterion according to a type of tree block to which the control parameter belongs. This change is equivalent to the change to a syntax unit. For example, context control unit 142 or 242 may use pattern 2 or 3 in which the top tree block is not used for data from an adaptive filter with a data structure such as alf_param, while it may use the policy of context model (pattern 1) for the other syntaxes as conventionally used. In other words, the second type or the third type can be a control parameter that has a predetermined data structure. Additionally, this means that the determination criteria can be changed
Á • A'l according to the type of tree block of an adjacent tree block.
Additionally, whether the value of the control parameter can actually be used or not or produces the advantage of reduced memory usage differs depending on a position of a tree block in the hierarchical relationship. In other words, the context control unit 142 or 242 can toggle between using or not using the upper tree block, according to a depth of a tree block and a hierarchical position of the block.
For example, in FIGURE 9, the numbers 0 to 9 in tree block 301B are in a decoding order. In this case, the control parameters of blocks 1 and 2 are available when tree block 4 is encoded or decoded.
Additionally, for the purpose of reducing memory usage, the context control unit 142 or 242 can select pattern 1 using the top tree block, when the tree block is not at depth 0 and the possessed position is one of the second element to subsequent elements in the vertical partition. In this document, depth indicates the depth of the root. In other words, when a certain tree block is defined as block [xn], [yO] [depth], the determination criterion can be changed according to whether the
<img file="MX337291B_D0056.tif" />
current tree block satisfies or not block [xn] [(yO) + 1] [depth]. In other words, the upper blocks are used for blocks 4 to 9 in FIGURE 9. When the tree block is encoded or decoded in the order as numbered (starting from 0 and ending at 9), it is clear that blocks 4 a 9 can use the control parameters included in the upper blocks. Additionally, there is the advantage that those blocks have only temporarily maintained data. Additionally, this indicates that the context value is determined according to the 3D position that includes the depth in addition to the x and y coordinates. Also, a conditional value of a tree block in the highest layer can be used (followed) as a conditional value of a tree block in the lowest layer.
Additionally, the context control unit
142 o 242 can change these criteria in view of the relationship between the current tree block and the other cuts. Hereinafter, the three shaded blocks 301A, 301B and 301C shown in FIGURE will be described.
In this document, the tree block 301A is a starting tree block, and both the left tree block and the upper tree block of the tree block 301A are included in another section. The upper tree block of tree block 30IB is τ 7ΰ -τ><sup>;</sup>5 and ; »Includes in another cut. Both the left tree block and the upper tree block of the tree block 301C are included in the same cut as the tree block 301C includes. The context control unit 142 or 242 can switch the criteria according to this condition. In other words, the context control unit 142 or 242 can switch criterion (i) according to whether or not the upper tree block is included in another cut, (ii) according to whether the left tree block is included or not in another court or (iii) according to both (i) and (ii). In other words, the context control unit 142 or 24 2 can determine that the top tree block control parameter is not available in decoding when the current tree block is at the cutoff limit. Accordingly, when the decoding processing at the top part 1 is not completed, for example, it is possible to perform the decoding processing considering whether or not the part 2 can obtain information by itself.
Next, the hierarchical processing unit (multi-layer block structure) will be described. FIGURE 10 illustrates the hierarchical processing unit (multi-layer block structure).
The image encoding apparatus 100 encodes moving images in a processing unit
<img file="MX337291B_D0057.tif" />
complete and the image encoding apparatus 200 decodes an encoded stream in a complete processing unit. The processing unit is stratified by the processing unit partition into small processing units and the additional partition
<td>of the units of</td><td colspan="2">small processing</td><td>in</td><td>units</td><td>of</td>
<td>processing more</td><td>little.</td><td>According</td><td>the</td><td>Unit</td><td>of</td>
<td>processing is over</td><td>small the</td><td>depth</td><td>of</td><td>unit</td><td>of</td>
Processing is larger and hierarchically lower, and the value indicating depth is larger. Conversely, as the processing unit is larger, the depth of the processing unit is less and is hierarchically greater, and the value indicating the depth is smaller.
The processing unit includes a coding unit (CU), a prediction unit (PU) and a transformation unit (TUj. A CU is a tree block of maximum 128 x 128 pixels and is a unit corresponding to a macroblock Conventional A PU is a basic unit for inter-prediction A TU is a basic unit for orthogonal transformation and is identical in size to or much smaller than PU. A CU is partitioned, for example, into 4 sub-CUs and one of the sub-CUs includes a PU and a TU that is the same size as that of the sub-CU (in this document, PU and TU overlap each other ). For example, i • τ * ”* - '» · »jja.v.-fi-gk' ¡: -í :: ái. i. ·. · r TU is} ~ ~ /
INS ;; (. .'7 ... ι-E: .. i the PU is further partitioned into 4 sub-PUs further partitioned into 4 sub-CUs. When the processing unit is partitioned into smaller processing units, each of the smaller processing units is referred to as a processing sub-unit For example, when the processing unit is a CU, the processing sub-unit is a sub-CU. When the processing unit is a PU, the sub-processing unit is a sub-PU. Additionally, when the processing unit is a TU, the processing sub-unit is a sub-TU.
More specifically, the following indicates the details.
An image is partitioned into one or more slices. A cut is a sequence of the largest encoding unit. The position of the largest encoding unit is indicated by an address of the largest encoding unit lcuAddr.
Each of the encoding units including the respective larger encoding units is divided into four encoding units.
As a result, a quadruple tree block is constructed that is the size of a CU. The position of the CU is indicated by a culdx encoding unit index that has a sample (pixel or coefficients) in the upper left corner of the larger encoding unit as a starting point.
mm nravUHraue'62
<img file="MX337291B_D0058.tif" />
<img file="MX337291B_D0059.tif" />
When partitioning of a CU is not allowed, the CU is handled like a PU. Similar to CU, the position of a PU is indicated by a prediction unit index puldx that has a sample in the upper left corner of the larger encoding unit as a starting point.
The PU can include partitions (PU partitions or sub-PUs). The PU partition is indicated by a puPartldx prediction unit partition index that has a sample in the upper left corner of the PU as a starting point.
The PU can include TUs. Similar to the CU, the TU can be partitioned into four smaller TUs (subTUs). This indicates the permission of the quad tree block partition of a residual signal. The position of the TU is indicated by a tuldx transformation unit index that has a sample in the upper left corner of the PU as a starting point.
In this document, the definition of each of the processing units is as follows:
CTB (tree coding block): Basic unit to identify the quad tree block partition of a square region. Which has various frame sizes;
LCTB (largest coding tree block): The largest CTB allowed in a cut. A cut includes a plurality of LCTBs that do not overlap each other;
><sup>Λ</sup> ·«:
lJl u · »V.
coding more
<img file="MX337291B_D0060.tif" />
SCTB (tree block of
<img file="MX337291B_D0061.tif" />
small): The smallest CTB allowed in ”a 'cor'Ee. ™ Ta partitioning of an SCTB into smaller CTBS is not allowed;
PU (prediction unit): Basic unit to identify a prediction processing. A PU is as big as a CU in which partition is not allowed. Although partitioning of a CU into four square regions is allowed, a PU can be partitioned into a plurality of partitions having any shape;
TU (transformation unit): Basic unit to identify a transformation and a quantification;
CU (coding unit): Same as CTB;
LCU (Largest Coding Unit): Same as Largest CTB; and
SCU (smallest encoding unit): Same as smallest CTB.
Additionally, the quantization parameters include at least one of a delta quantization scale parameter (delta QP or QP delta), a quantization compensation parameter, an index (Q matrix select idc) and a dead zone compensation parameter of quantification. The index is to select one of the quantized scaling matrices.
The delta quantization scale parameter (delta QP or QP delta) is a difference between a quantization scale parameter that is applied to "transform coefficient and a scale parameter - specified by a sequence header or a cutoff header ( or a quantization scale parameter immediately before in the scan order Z).
The quantization offset parameter is also referred to as a quantization offset and is an adjustment value (offset value) to round off a signal in the quantization performance. Thus, when the image coding apparatus 100 performs quantization, it encodes the quantization offset. Then, the image decoding apparatus 200 decodes the encoded quantization compensation. Then, the image decoding apparatus 200 performs a correction using the quantization compensation when the transform coefficients are inversely quantized.
An index (Qmatrix select idc) is referred to as an adaptive quantization matrix and indicates which quantization scale matrix is used from among a plurality of quantization scale matrices. Additionally, when there is only one quantization scaling matrix, Qmatrix select idc indicates whether the quantization scaling matrix is used or not. The adaptive quantization matrix can be controlled per unit of
<img file="MX337291B_D0062.tif" />
[NDL'S i'ÍJ / .L
<img file="MX337291B_D0063.tif" />
block (processing unit).
»—J irtWW. '^ R' <sup>Item,</sup>« —
The quantization dead zone compensation parameter is referred to as an adaptive dead zone and is control information for adaptively changing a dead zone per block. The dead zone is a width whose frequency coefficients are made 0 by quantization (last width that becomes +1 or -1 after quantization).
Although a case where pattern 3 with which a fixed value is used, predetermined as a context value is described earlier in this document, the case can be done under the condition that the control parameters of the upper tree block are not used and the left tree block, and additionally under the condition without the use of control parameters of the upper tree block and the left tree block as pattern 3. For example, context control unit 142 or 242 can determine a context according to the hierarchical depth of a data unit to which each of the control parameters belongs, such as pattern 3.
Mode 3
Mode 3 will describe which signal type should be used as the first type and the second type (or the third type).
More specifically, the present inventors have verified each of the subsequent signal types between
<img file="MX337291B_D0064.tif" />
signal types as indicated in FIGURE 3 (Section 9.3.3.1.1.1 of NPL 2). Each of the signal types has been verified, because there are several parameters and it is difficult to predict whether or not each pattern of the other signal types satisfies the validity, based on a result of the verification in one of the signal types. signal (which of patterns 1 to 3 is appropriate).
Verification is in accordance with the structure (setting parameter and HM3.0 software version) described in JCTVC-E700, Common Test Conditions and Software Reference Configurations (see NPL 3). Additionally, each of the test images is limited in length to 49 frames.
The image encoding method and the image decoding method according to Mode 3 are related to CABAC. In this way, verification has been conducted using the following four test patterns which are a set of adjustment values each indicating 1 as the Symbol Mode value (# 0: LCEC,
High efficiency fit, intra;
4.3
High efficiency random access setting;
Low delay high efficiency setting; already
4.7 High efficiency adjustment of
<img file="MX337291B_D0065.tif" />
(P cuts only).
The evaluation is made based on an evaluation value called a BD rate that is used as an evaluation standard used uniformly for an implementation evaluation at the HEVC. BD Y Rate, BD U Rate and BD V Rate are BD rates for a YUV color space and are standard evaluation values. According to VCEG-AI11 (NPL 4), the BD rate is an evaluation value obtained by integrating two pairs of code quantities with a PSNR result and representing the coding efficiency according to the area ratio.
Additionally, the BD rate indicating a less value means that the encoding efficiency has been improved. The comparison criteria are based on a result of the product of a reference program which implements pattern 1. The results of patterns 2 and 3 are shown with respect to the result of pattern 1.
The following describes a verification result on each of the signal types:
(First check) split_coding_unit_flag;
(Second verification) skip_flag; and (Third check) merge_flag.
(First check) split_coding_unit_flag
FIGURE 11 illustrates a decoding method * U · WWVCX-ΛΛΧ.ΐχ;
arithmetic for split_coding_unit_flag.
Verification is conducted by changing the context pattern from pattern 1 to pattern 2 or 3 only for one signal type that is verified, without changing the context model for the other signal types and the verification parameter specified in NPL 3. In the column shown in FIGURE 11, the value of Fixed indicates that the condition (the left tree block condition or the top tree block condition) of the column specified by Fixed is not used when a context value (or increment) it follows. In other words, when only one of the left tree block condition and the top tree block condition is Fixed, only the other condition is used. Additionally, when both the left tree block condition and the upper tree block condition are Fixed, a default value (for example, 0) is used as a context value (or increment).
The meaning of the split_coding_unit_flag signal type is defined as follows.
split_coding_unit_flag [xO] [yO] specifies whether an encoding unit is divided into encoding units with half horizontal and vertical size. The matrix indices xO, yO specify the location (xO, yO) of the upper left luma sample of the encoding block «ΜΓΜίΐ · ι» ι ai μ *. · INSTITUT,
L't LA Π · .ΟΓ INDUS top luma specimen
In other words, the CU target is considered relative to the left of the image.
split_coding_unit_flag indicates whether or not it partitions in four. More specifically, the target CU is partitioned when split_coding_unit_flag indicates 1, while the target CU is not partitioned when split_coding_unit_flag indicates 0.
The split_coding_unit_flag data is structured in a tree encoding syntax as a syntax. The image decoding apparatus analyzes a bit stream according to the syntax of this data structure.
FIGURES 12A and 12B are tables to describe verification results in split_coding_unit_flag.
FIGURE 12A indicates the result of the verification using an adjacent block (only a left block condition determination value L) of pattern 2. FIGURE 12B indicates the result of the verification using a zero adjacent block (not using the condition upper block L or left block condition L) of pattern 3.
The result of the verification in each of the
FIGURES 12A and 12B indicate the increase and decrease of the BD rate according to the four test patterns.
Additionally, the evaluation value is
<img file="MX337291B_D0066.tif" />
evaluation indicating an evaluation in the case of both the left block represented by the value standard in relation to a pattern value 1 in which they are used as the upper block. More specifically, when the evaluation value is positive, the result is less than the evaluation value (BD rate) in the case of pattern 1. Additionally, when the evaluation value is negative, the result is more improved than the value of evaluation in the case of pattern 1.
The result clarifies that pattern 1 is superior as a context model pattern for split_coding_unit_flag. In other words, the evaluation values obtained by patterns 2 and 3 are lower than that of pattern 1.
Thus, when the signal type of a .control parameter is split_coding_unit_flag, the context control unit 142 or 242 determines a context value using pattern 1 which is a conventional pattern of a context model, in terms of the BD rate.
(Second verification) skip_flag
FIGURE 13 illustrates an arithmetic decoding method for skip_flag. In this document, the verification method is the same as that of the first verification.
The meaning of the skip_flag signal type is ιΐ; ί í'S; Ιι b> Ό ζΖΓϊ ', Ζ · ..' · * · /;
L »¿tz .'Tv. '?! LL', \ D) defined as follows. »Wsrsui skip_flag [xO] [yO] equal to 1 specT Current encoding unit, when decoding a cut
P or B, no further syntax elements are analyzed except for motion vector predictor indices after skip_f lag [xO] [yO]. skip_flag [xO] [yO] equal to 1 specifies that the encoding unit should not be omitted. The matrix indices xO, yO specify the location (xO, yO) of the upper left luma sample of the coding block considered in relation to the upper left luma sample of the image. In other words, skip_flag indicates whether the target CU should be skipped or not (handled as a skipped block).
The skip_flag data is structured in a coding unit syntax as a syntax. In other words, skip_flag is set for each CU. The image decoding apparatus analyzes a bit stream according to the syntax of this data structure.
FIGURES 14A and 14B are tables to describe verification results in skip_flag.
FIGURE 14A indicates the result of verification using an adjacent block (only a left block condition determination value L) of pattern 2. FIGURE 14B indicates the result of the
<img file="MX337291B_D0067.tif" />
an adjacent block zero (not upper block L nor pattern condition 3.
verification using the left block condition L) of the
The result of the verification in each of the
FIGURES 14A and 14B indicate the increase and decrease of the BD rate according to four test patterns as described for the first verification. Additionally, the meaning of the evaluation value is the same as that of the first verification.
The result clarifies that pattern 1 is superior as a pattern of a context model for skip_flag. In other words, the evaluation values obtained by patterns 2 and 3 are lower than that of pattern 1.
Thus, when the signal type of a control parameter is skip_flag, the context control unit 142 or 242 determines a context value using pattern 1 which is a conventional pattern of a context model, in terms of the BD rate.
(Third check) merge_flag
FIGURE 15 is a table indicating an arithmetic decoding method for merge_flag. In this document, the verification method is the same as those in the first verification and the second verification.
The meaning of the merge_flag signal type is defined as follows.
ΊΓ Ά / Τ ΤΊ) Τ
Á y.vli <rh uvsrm.-ro μ. · Χιγ ·. ·<sub>ν</sub>7 -*$
- -'T Lí.'ULij i ¡j ^ 'l'<sup>s</sup>»^ Í ~ 'eS¡y merge_f lag [xO] [yO] specifies whether the inter-prediction tr & sr-tie parameters for the current prediction unit gon— inferred from an adjacent inter-predicted partition. The matrix indices xO, yO specify the location (xO, yO) of the upper left luma sample of the prediction block considered in relation to the upper left luma sample of the image. When merge_flag [xO] [yO] is not present (InferredMergeFlag equals), it is inferred that it equals 1. In other words, merge_flag [xO] [yO] indicates whether or not a merge mode is used. In this document, the blending mode is a mode in which a motion vector and a reference image index are copied from an adjacent block of the current block that is encoded and the current block is encoded.
The merge_flag data is structured in a prediction unit as a syntax. In other words, merge_flag is set for each PU. The image decoding apparatus analyzes a bit stream according to the syntax of this data structure.
FIGURES 16A and 16B are tables to describe verification results in merge_flag.
FIGURE 16A indicates the result of verification using an adjacent block (only a left block condition determination value L) of pattern 2. FIGURE 16B indicates the result of the
<img file="MX337291B_D0068.tif" />
verification using a zero adjacent block (not using the upper block condition L or the left block condition L) of pattern 3.
The result of the verification in each of the
FIGURES 16A and 16B indicate the increase and decrease of the BD rate according to the four test patterns as described for the first verification. Additionally, the meaning of the evaluation value is the same as that of the first verification.
The result is different from those of the first verification of split_coding__unit_f lag and the second verification of skip_flag. There is no significant difference in BD rate between patterns 1 and 2 or 3 as a pattern from a context model for merge_flag.
In this way, under an environment mixed with a plurality of signal type control parameters, the context control unit 142 or 242 determines a context value without using the top block as an adjacent block particularly when the signal type of the control parameter is merge_flag. In other words, Context Control Unit 142 or 242 determines a context value using pattern 2 or 3 when the signal type of the control parameter is merge_flag. In other words, the first type includes split_coding_unit_flag or skip_flag, and the second type or third type includes merge_flag. By
<img file="MX337291B_D0069.tif" />
Accordingly, the image encoding apparatus and the image decoding apparatus according to Mode 3 can reduce memory usage while suppressing the decrease in BD rate.
When pattern 2 is compared to pattern 3 for merge_flag, these BD rates do not make a significant difference. Thus, it is preferred to use pattern 3 for merge_flag. Accordingly, it is possible to further reduce memory usage and amount of processing.
In this document, compared to merge_flag and skip_flag, although motion vector residual data is not transmitted in a skip mode, motion vector residual data is transmitted in a merge mode. Therefore, even when the context that is temporarily used is not optimal for merge_flag, the deterioration in image quality caused by not using the optimal context can be compensated to some extent by the processing that uses the residual data. Consequently, the deterioration in image quality is suppressed.
Although the image encoding apparatus and the image decoding apparatus according to Modes 1 to 3 of the present invention are described, the present invention is not limited to those Modes.
il
INSTITUTO MEXk'ι LA PROPI «j> *;
• NDVSTíual
<img file="MX337291B_D0070.tif" />
For example, at least part of the '~ ctS' ·· image encoding apparatus, the image decoding apparatus and functions of modifying those apparatus according to Modes 1 to 3 can be combined.
Additionally, all of the values and logical values described above are exemplifications to specifically describe the present invention and the present invention is not limited by the exemplified values.
Additionally, the divisions of the functional blocks in the block diagrams are examples. In this way, the functional blocks can be implemented as a functional block, a functional block can be divided into a plurality of functional blocks and a part of the functions can be switched to another functional block. Additionally, a plurality of functional blocks having similar functions can be processed by individual hardware or software in parallel or time division.
The orders of the steps of the image encoding method performed by the image encoding apparatus and the image decoding method performed by the image decoding apparatus are to specifically describe the present invention and may be a different order from the orders. previous. Additionally, part of the steps can be performed simultaneously (in
Λ '. ·' * -'- »'!; -» jl .ht / á fp Ú «4.! L. "2\. j .'i *. H NS7? TU'iC AícXícaná?
'ΊέΥί parallel) with the other steps.
Mode 4
The processing described in each of the modes can be simply implemented on a separate computer system, by recording, on a recording medium, a program for implementing the motion picture encoding method settings (picture encoding method). and the moving image decoding method (image decoding method) described in each of the modalities. The recording media can be any recording medium as long as the program can be recorded, such as a magnetic disk, an optical disk, a magnetic optical disk, an IC card and a semiconductor memory.
Hereinafter, the applications for the motion picture encoding method (picture encoding method) and the motion picture decoding method (picture decoding method) described in each of the modalities and systems using the same. The system has a feature that is to have an image encoding and decoding apparatus including an image encoding apparatus using the image encoding method and an image decoding apparatus using the image decoding method.
<img file="MX337291B_D0071.tif" />
Other settings in the system can be changed appropriately depending on the situations.
FIGURE 19 illustrates a complete configuration of an exOO content delivery system for implementing content delivery services. The area to provide communication services is divided into cells of the desired size and the base stations exl06, exl07, exl08, exl09 and exllO which are fixed wireless stations are placed in each of the cells.
The exlOO content delivery system connects to devices, such as an exlll computer, an exll2 personal digital assistant (PDA), an exll3 camera, an exll4 cell phone, and an exll5 video game console, by via exlOl Internet, an exl02 Internet service provider, an exl04 telephone network, as well as base stations exl06 to exllO, respectively.
However, the configuration of the exlOO content delivery system is not limited to the configuration shown in FIGURE 19 and a combination in which any of the elements is connected is acceptable. In addition, each device can be connected directly to the exl04 telephone network, preferably via the exl06 to exllO base stations which are the fixed wireless stations. Additionally, the
<img file="MX337291B_D0072.tif" />
IWSTr<sup>1</sup>'! G '.' G devices can be interconnected with each other via wireless communication in short time and 'ÓErásr ™ “' ^ '· · - ~
The exll3 camera, just like a digital video camera, is capable of capturing video. An exll6 camera, such as a digital camera, is capable of capturing both still images and video. Additionally, the exll4 cell phone may be one that complies with any of the standards such as the Global System for Mobile Communications (GSM) (registered trademark), Code Division Multiple Access (CDMA). Broadband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access (HSPA) for its acronym in English) . Alternatively, the exll4 cell phone can be a Personal Telephone System (PHS).
In the exIOO content delivery system, an exl03 streaming server connects to the exll3 camera and others via the exl04 telephone network and the exl09 base station, making it possible to distribute images of a live show and others. In this distribution, content (for example, video of a live music show) captured by the user using the exlL3 camera is encoded as described above in each of the modes (i.e., the
<img file="MX337291B_D0073.tif" />
camera functions as the encoding apparatus-i-cdclt5n ~ aSTmagenes in accordance with an aspect of the present invention), and the encoded content is transmitted to the exl03 streaming server. On the other hand, the exl03 uninterrupted transmission server performs the uninterrupted distribution of the content data transmitted to the clients when they request it. Clients include the exlll computer, the exll2 PDA, the exll3 camera, the exll4 cell phone, and the exll5 video game console that are capable of decoding the encoded data mentioned above. Each of the devices that has received the distributed data decodes and reproduces the encoded data (ie, functions as the image decoding apparatus in accordance with one aspect of the present invention).
The captured data can be encrypted by the exll3 camera or the exl03 streaming server that transmits the data, or the encryption processes can be shared between the exll3 camera and the exl03 streaming server. Similarly, the distributed data can be decoded by the clients or the exl03 streaming server, or the decoding processes can be shared between the clients and the exl03 streaming server. Additionally, the data of still images and video
<img file="MX337291B_D0074.tif" />
captured not only by exll3 sIjlq camera. .also. -1.a rame exll6 can be transmitted to the exl03 streaming server through exlll computer. The encoding processes can be performed by the exll6 camera, the exlll computer or the exl03 streaming server, or they can be shared between them.
Additionally, the encoding and decoding processes can be performed by an LSI ex500 generally included in each of the exlll computer and devices. The LSI ex500 can be configured from a single chip or a plurality of chips. The software for encoding and decoding video can be integrated into some type of recording medium (such as a CD-ROM, floppy disk, and hard drive) that is readable by exlll and others, and the encoding and decoding processes can be perform using the software. Additionally, when the exll4 cell phone is equipped with a camera, the video data obtained by the camera can be transmitted. Video data is data encoded by the LSI ex500 included in the exll4 cell phone.
Additionally, the exl03 streaming server can be comprised of servers and computers and can decentralize data and process decentralized data, write or distribute data.
<img file="MX337291B_D0075.tif" />
As previously described “,” ~~ ± OT --- © iAeíites__ can receive and reproduce the encoded data in the exlOO content provisioning system. In other words, the clients can receive and decode information transmitted by the user and can reproduce the decoded data in real time in the exlOO content provisioning system, so that the user who does not have any rights and particular equipment can implement a broadcast personal.
In addition to the example of the exlOO content delivery system, at least one of the moving image encoding apparatus (image encoding apparatus) and the moving image decoding apparatus (image decoding apparatus) described in each one of the modalities can be implemented in an ex200 digital broadcast system illustrated in FIGURE 20. More specifically, an ex201 broadcast station communicates or transmits, via radio waves to an ex202 broadcast satellite, multiplexed data obtained by multiplexing audio and other data into video data. The video data is data encoded by means of the moving image encoding method described in each of the modalities (ie, the data encoded by the image encoding apparatus according to an aspect of the present invention). With the reception of the data
<img file="MX337291B_D0076.tif" />
multiplexed, the broadcast satellite ex202<sup>stz</sup>'<sup>l</sup>| ^^ gShi<sup>,</sup>tea_<sub>:</sub>jtt
<img file="MX337291B_D0077.tif" />
<sup>r</sup>Radio s for broadcasting. Then, an ex204 homemade PfpQ ant-pna with a satellite broadcast reception function receives the radio waves. Thereafter, a device such as an ex300 television (receiver) and an external tuner (STB) ex217 decodes the received multiplexed data and reproduces the decoded data (i.e. functions as the image decoding apparatus according to with an aspect of the present invention).
Additionally, an ex218 (i) reader / writer reads and decodes multiplexed data that is recorded on an ex215 recording medium, such as a DVD and a BD, or (ii) encodes video signals on the ex215 recording medium, and In some cases, it writes data obtained by multiplexing an audio signal into the encoded data. The ex218 reader / writer may include the motion picture decoding apparatus or the motion picture encoding apparatus as shown in each of the modalities. In this case, the reproduced video signals are displayed on the ex219 monitor and can be reproduced by another device or system using the ex215 recording medium on which the multiplexed data is recorded. It is also possible to implement the motion picture decoding apparatus in the external tuner ex217 connected to the ex203 cable for a cable television or to the ex204 antenna
<img file="MX337291B_D0078.tif" />
INSTI for satellite and / or terrestrial broadcast ^ _ ^ coa --- ^ X «» fixed -— * 4 »display the video signals on the ex219 monitor of the ex300 television. The motion picture decoding apparatus may not be implemented on the external tuner but on the ex300 television.
FIGURE 21 illustrates the television (receiver) ex300
<td>who uses the</td><td>method</td><td>coding</td><td>of</td><td>images</td><td>in</td>
<td>movement and the</td><td>method</td><td>decoding</td><td>of</td><td>images</td><td>in</td>
<td colspan="2">movement described in</td><td>each of the</td><td colspan="2">modalities.</td><td>The</td>
<td>ex300 television</td><td>It includes</td><td>: a device</td><td>of</td><td colspan="2">tuning</td>
ex301 that obtains or provides multiplexed data obtained by multiplexing audio data into video data, through the ex204 antenna or the ex203 cable, etc. that receives a broadcast; an ex302 modulation / demodulation unit that demodulates the received multiplexed data or modulates data into multiplexed data to be supplied to the outside; and an multiplexing / demultiplexing unit ex303 that demultiplexes the modulated multiplexed data into video and audio data or multiplexes video and audio data encoded by an ex306 signal processing unit into data.
The ex300 television further includes: an ex306 signal processing unit including an ex304 audio signal processing unit and an ex305 video signal processing unit decoding data
K. <P Τ <sup>1</sup> · 'Ά
VAT .1 Α. tAT><sup>;</sup> ..<<sup>;</sup>Α □ ^ 51 H'U'í'Q ΑίΣΧΙίΛΝΟ \ · <<
C'¿ LA PÁGFírDAi f ίςί; '> * audio and video data and encode audio data and “video ciatos *, respectively (which function as the image encoding apparatus and the image decoding apparatus according to aspects of the present invention); and an ex309 output unit including an ex307 speaker that provides the decoded audio signal and an ex308 display unit that displays the decoded video signal, such as a display. Additionally, the ex300 television includes an interconnection unit ex317 that includes an operation input unit ex312 that receives an input from a user operation. Additionally, the ex300 television includes an ex310 control unit that collectively controls each constituent element of the ex300 television and an ex311 power supply circuit unit that supplies power to each of the elements. Unlike the ex312 operation input unit, the ex317 interconnect unit can include: an ex313 jumper that connects to an external device, such as the ex218 reader / writer; an ex314 slot unit to enable bonding of the ex216 recording medium, such as an SD card; an ex315 controller that is connected to an external recording medium, such as a hard drive; and an ex316 modem that is connected to a telephone network. At this point, the ex216 recording medium can electrically record information
I saw ΡΙ
MtXJCAND INSTITUTE l »£ ¿λ ρκοϊ'ικολ:»
INDEX
<img file="MX337291B_D0079.tif" />
using a nonvolatile / volatile aemiconductor memory element for storage. The constituent elements of the ex3 00 television are connected to each other through a synchronous busbar.
First, the configuration in which the ex300 television decodes the multiplexed data obtained from the outside through the ex204 antenna and others and reproduces the decoded data will be described. In television ex300, with the operation of a user through a remote controller ex220 and others, the multiplexing / demultiplexing unit ex303 demultiplexes the multiplexed data that is demodulated by the modulation / demodulation unit ex302, under the control of the ex310 control that includes a CPU. Additionally, the audio signal processing unit ex304 decodes the demultiplexed audio data and the video signal processing unit ex305 decodes the demultiplexed video data, using the decoding method described in each of the modalities, on television ex300. The ex309 output unit provides the decoded video signal and audio signal to the outdoors, respectively. When the ex309 output unit provides the video signal and the audio signal, the signals can be temporarily stored in the ex318 and ex319 buffers and others so that the signals are reproduced on
<img file="MX337291B_D0080.tif" />
synchronization with each other. Adi ci ona lment evi can read multiplexed data not through a broadcast and others but from the ex215 and ex216 recording media, such as a magnetic disk, an optical disk and an SD card. Next, a configuration will be described in which the ex300 television encodes an audio signal and a video signal and transmits the data to the outside or writes the data to a recording medium. In the ex300 television, with the operation of a user through the remote controller ex220 and others, the audio signal processing unit ex3 04 encodes an audio signal and the video signal processing unit ex305 encodes a video signal , under control of the ex310 control unit using the coding method described in each of the modalities. The multiplexing / demultiplexing unit ex303 multiplexes the encoded video signal and audio signal and supplies the resulting signal to the outside. When the multiplexing / demultiplexing unit ex303 multiplexes the video signal and the audio signal, the signals can be temporarily stored in the ex320 and ex321 buffers and others so that the signals are reproduced in synchronization with each other. At this point, the ex318, ex319, ex320, and ex321 buffers can be plural as illustrated, or at least one buffer can be shared on the ex300 television. Additionally,
MEXICAN INSTITUTE OF PROPERTY
INDUSTRY!.
they can store data in a buffer so that system overflow and underflow between the ex302 modulation / demodulation unit and the ex303 multiplex / demultiplex unit, for example, can be avoided.
Additionally, the ex300 television may include a configuration to receive an AV input from a microphone or a different camera from the configuration to obtain audio and video data from a broadcast or recording medium and may encode the data obtained. Although the ex300 television can encode, multiplex, and provide data to the outside in the description, it may only be able to receive, decode, and provide data to the outside but not encode, multiplex, and provide data to the outside.
Additionally, when the ex218 reader / writer reads or writes multiplexed data from or to a recording medium, one of the ex300 television and the ex218 reader / recorder can decode or encode the multiplexed data and the ex300 television and the ex218 reader / recorder can share decoding or encoding.
As an example, FIGURE 22 illustrates a configuration of an ex400 information recording / reproducing unit when data is read from or written to or from an optical disc. The information reproduction / recording unit ex400 includes the constituent elements ex401, ex402, ex403, ex404, ex405, ex406 and ex407 that are described by the Mexican institute LA PROPIEDAD
INDUSTRIAL later in this document. The head-opt-i-Ga. „Ax401 radiates a laser focus onto a recording surface of the ex215 recording medium which is an optical disc for writing information and detects the reflected light from the recording surface of the ex215 recording medium to read the information. The ex402 modulation recording unit electrically drives a semiconductor laser included in the ex401 optical head and modulates the laser light according to recorded data. The ex403 reproduction demodulation unit amplifies a reproduction signal obtained by electrically detecting the reflected light from the recording surface using a photodetector included in the ex401 optical head and demodulates the reproduction signal by separating a signal component, recorded in the medium recording ex215 to reproduce the necessary information. The buffer ex4G4 temporarily holds the information to be recorded on the recording medium ex215 and the information reproduced from the recording medium ex215. The ex405 disk motor spins the ex215 recording medium. The ex406 servo drive moves the optical head ex401 to a predetermined information track while controlling the rotary drive of the ex405 disc motor in order to follow the laser focus. The ex4 07 system control unit together controls the ex400 information recording / reproducing unit. The reading and writing processes can
<img file="MX337291B_D0081.tif" />
be implemented by the ex407 system control unit using various information stored in the ex404 buffer and generating and adding new information as necessary and by means of the ex402 modulation recording unit, the ex403 playback demodulation unit and the unit ex406 servo controls that record and reproduce information through the ex401 optical head while being operated in a coordinated manner. The ex407 system control unit includes, for example, a microprocessor and executes processing by causing a computer to run a program to read and write.
Although the ex401 optical head radiates a laser focus in the description, it can perform high-density recording using near-field light.
FIGURE 23 schematically illustrates the ex215 recording medium which is the optical disc. On the recording surface of the ex215 recording medium, the guide grooves are spirally formed and an ex230 information track, in advance, records direction information indicating an absolute position on the disc according to a change in the shape of the grooves guide. The address information includes information for determining ex231 recording block positions that are a unit for recording data. Playing the information track ex230 and reading the address information on a recording device and
<img file="MX337291B_D0082.tif" />
Playing back data can lead to the determination of the positions of the recording blocks. Additionally, the recording medium ex215 includes a data recording area ex233, an inner circumference area ex232 and an outer circumference area ex234. The ex233 data recording area is an area for use in recording user data. The inside circumference area ex232 and the outside circumference area ex234 which are the inside and outside the data recording area ex233, respectively, are for specific use except for recording user data. The information recording / reproducing unit 400 reads and writes encoded audio data, encoded video data, or multiplexed data obtained by multiplexing the encoded audio and video data from and onto the data recording area ex233 of the recording medium. recording ex215.
Although an optical disc having one layer, such as a DVD and BD is provided as an example in the description, the optical disc is not limited to that type and may be an optical disc having a multi-layer structure and capable of be engraved on a different part of the surface. Additionally, the optical disc may have a structure for multi-dimensional recording / reproduction, such as recording information using colored light with different wavelengths.
<img file="MX337291B_D0083.tif" />
on the same portion of the optical disc and for recording— information that has different layers from various angles.
Additionally, an ex210 car having an ex205 antenna can receive data from the ex202 satellite and others, and can play video on a display device such as an ex211 car navigation system set up on the ex210 car, in the ex200 digital broadcast system. . At this point, a configuration of the ex211 car navigation system will be a configuration, for example, that includes a GPS receiver unit of the configuration illustrated in FIGURE 21. The same will be true for exlll computer setup, exll4 cell phone and others.
FIGURE 24A illustrates the exll4 cell phone utilizing the motion picture encoding method and the motion picture decoding method described in the embodiments. The exll4 cell phone includes: an ex350 antenna for transmitting and receiving radio waves through the exllO base station; an ex365 camera unit capable of capturing moving and still images; and an ex358 display unit such as a liquid crystal display for displaying data such as decoded video that is captured by the ex365 camera unit or received by the ex350 antenna. The exll4 cell phone further includes: a main body unit including an ex366 key operating unit; an output unit of
<img file="MX337291B_D0084.tif" />
ex357 audio such as a speaker for audio output; an ex356 audio input unit such as a microphone for the audio input; an ex367 memory unit for storing captured video or still images, recorded audio, encoded or decoded data of the received video, still images, emails or others; and an ex364 slot unit which is an interconnect unit for a recording medium that stores data in the same manner as the ex367 memory unit.
Next, an example of an exll4 cell phone configuration will be described with reference to the
FIGURE 24B. On the exll4 cell phone, an ex360 main control unit designed to jointly control each main body unit including the ex358 display unit as well as the ex366 key operation unit connect to each other via a synchronous busbar ex370, to an ex361 power supply circuit unit, an ex362 operation input control unit, an ex355 video signal processing unit, an ex363 camera interconnect unit, an ex359 liquid crystal display (LCD) control unit, an ex352 modulation / demodulation unit, an ex353 multiplex / demultiplex unit, an ex354 audio signal processing unit, the slot drive ex364 and memory drive ex367.
Τ / 4 Ό V fi'AU Ι.Γ A
MEXICAN INSTITUTE OF THE '.-ROI-'Ei-'AD
INDUSTRIAL
<img file="MX337291B_D0085.tif" />
When an end of call key or a power key is turned on by a user's operation, the ex361 power supply circuit unit supplies the respective units with power from a battery pack in order to activate the cell phone exll4.
In the exll4 cell phone, the audio signal processing unit ex354 converts the audio signals collected by the audio input unit ex356 in voice talk mode into digital audio signals under the control of the main control unit ex3 60 which includes a CPU, ROM and RAM. The ex352 modulation / demodulation unit then performs spread spectrum processing on the digital audio signals, and the transmitting and receiving unit ex351 performs the digital-to-analog conversion and the frequency conversion on the data in order to transmit the resulting data via the ex350 antenna Also, on the exll4 cell phone, The EX351 transmit and receive unit amplifies the data received by the EX350 antenna in voice talk mode and performs frequency conversion and analog to digital conversion on the data.
The ex352 modulation / demodulation unit then performs the inverse spread spectrum processing on the data and the ex3 54 audio signal processing unit converts them to analog audio signals, for the purpose of impx eio
INSTITUI OIL
CE or J.
, „H L j U- --- send them via the audio output unit ex357. Additionally, when an email in data communication mode is transmitted, the email text data entered through the operation of the key operation unit ex366 and others in the main body is sent to the main control unit ex360 via the ex362 operation input control unit. The ex360 main control unit causes the ex352 modulation / demodulation unit to perform spread spectrum processing on the text data and the ex351 transmit and receive unit to perform digital-to-analog conversion and frequency conversion on the resulting data to transmit the data to the exllO base station via the ex350 antenna. When an email is received, the processing which is roughly the reverse of the processing to transmit an email is performed on the received data and the resulting data is provided to the display unit ex358.
When transmitting video, still images, or video and audio in the data communication mode, the ex355 video signal processing unit compresses and encodes the supplied video signals from the ex365 camera unit using the motion picture encoding method. shown in each of the modes (i.e. works as the image encoding apparatus of
<img file="MX337291B_D0086.tif" />
V
Λ.Λ according to the aspect of the present invention). and Trancmi ^ ,.
the encoded video data to the ex353 multiplex / demultiplex unit. In contrast, when the ex365 camera unit captures video, still images, and others, the ex354 audio signal processing unit encodes audio signals collected by the ex356 audio input unit and transmits the encoded audio data to the audio unit. multiplexing / demultiplexing ex3 53.
The multiplexing / demultiplexing unit ex353 multiplexes the data of
<td>supplied</td><td>of</td><td>the</td><td>Unit</td>
<td>video ex355</td><td>and</td><td>the</td><td>data</td>
<td>supplied</td><td>of</td><td>the</td><td>Unit</td>
encoded video that is audio signal processing encoded that is audio signal processing ex354, using a predetermined method. Then, the modulation / demodulation unit (modulation / decomposition circuit unit) ex352 performs the spread spectrum processing on the multiplexed data and the transmitting and receiving unit ex351 performs the digital-to-analog conversion and the frequency conversion on the data in order to transmit the resulting data via the ex350 antenna.
When data is received from a video file which is linked to a web page and others in the data communication mode or when an email with video and / or audio is received, for the purpose of
! Μ ΡI
MEXICAN INSTITUTE eg THE industrial PROf'ZOAD
<img file="MX337291B_D0087.tif" />
decode the multiplexed data that is received via the ex350 antenna, the multiplexing / demultiplexing unit ex353 demultiplexes the multiplexed data into a video data bitstream and an audio data bitstream and provides the unit with ex355 video signal processing with encoded video data and ex354 audio signal processing unit with encoded audio data, via synchronous busbar ex370. The ex355 video signal processing unit decodes the video signal using a motion picture decoding method that corresponds to the motion picture encoding method shown in each of the modes (i.e., it functions as the decoding apparatus of images according to the aspect of the present invention), and then the display unit ex358 displays, for example, the video and still images that are included in the video file linked to the website via the ex359 LCD control unit. Additionally, the ex354 audio signal processing unit decodes the audio signal and the ex357 audio output unit provides the audio.
Additionally, similar to ex300 television, a terminal such as the exll4 cell phone may have 3 types of deployment configurations that include not only (i) a
<img file="MX337291B_D0088.tif" />
transmission and reception including both a coding apparatus and a decoding apparatus, but also (ii) a transmission terminal including only a coding apparatus and (iii) a receiving terminal including only a decoding apparatus. Although the ex200 digital broadcast system receives and transmits the multiplexed data obtained by multiplexing audio data into video data in the description, the multiplexed data may be data obtained by multiplexing not audio data but video-related character data in video data and may not be multiplexed data but video data itself.
As such, the motion picture encoding method and the motion picture decoding method in each of the modalities can be used in any of the described devices and systems. In this way, the advantages described in each of the modalities can be obtained.
Additionally, the present invention is not limited to the embodiments and various modifications and revisions are possible without departing from the scope of the present invention.
Mode 5
Video data can be generated by switching, as necessary, between (i) the encoding method of
<img file="MX337291B_D0089.tif" />
moving images or the apparatus -day- -godif icaclón, rie¡. moving pictures displayed in each of the modalities and (ii) a moving picture coding method or a moving picture coding apparatus in accordance with a different standard, such as
MPEG-2, MPEG-4 AVC and VC-1.
At this point, when a plurality of video data that meets the different standards is generated and then decoded, the decoding methods need to be selected to meet the different standards. However, since it cannot be detected which standard each of the plurality of video data that is decoded meets, there is a problem that an appropriate decoding method cannot be selected.
In order to solve the problem, the multiplexed data obtained in multiplexing audio data and others in video data has a structure that includes identification information indicating which standard the video data complies with. The specific structure of the multiplexed data including the video data generated in the motion picture encoding method and by the motion picture encoding apparatus shown in each of the modalities will be described below.
The multiplexed data is a digital stream in the MPEG-2 Transport Stream format.
100
<img file="MX337291B_D0090.tif" />
FIGURE 25 illustrates a structure of multiplexed data. As illustrated in FIGURE 25, multiplexed data can be obtained by multiplexing at least one of a video stream, an audio stream, a presentation graphics stream (PG), and a stream of interactive graphics. The video stream represents primary video and secondary video of a movie, the audio stream (IG) represents a primary audio part and a secondary audio part that mixes with the primary audio part and the presentation graphics stream represents movie subtitles. At this point, the primary video is normal video that is displayed on a screen and the secondary video is video that is displayed in a smaller window on the primary video. Additionally, the interactive graphics stream represents an interactive screen that is generated by ordering the GUI components on one screen.
The video stream is encoded in the motion picture encoding method or by the motion picture encoding apparatus shown in each of the modalities, or in a motion picture encoding method or by a motion picture encoding apparatus. moving images in accordance with a conventional standard, such as MPEG-2, MPEG-4 AVC and VC-1. The audio stream is encoded according to a standard,
101
<img file="MX337291B_D0091.tif" />
such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD and
Linear PCM.
Each stream included in the multiplexed data is identified by a PID. For example, 0x1011 is assigned to the video stream that is used for movie video, 0x1100 to OxlllF is assigned to audio streams, 0x1200 to 0xl21F is assigned to presentation graphics streams, 0x1400 to 0xl41F are assigned to interactive graphics streams, OxlBOO to OxlBlF are assigned to the video streams that are used for the secondary video of the movie and OxlAOO to OxlAlF are assigned to the audio streams that are used for the secondary audio that is mixed with the primary audio.
FIGURE 26 schematically illustrates how data is multiplexed. First, an ex235 video stream composed of video frames and an ex238 audio stream composed of audio frames are transformed into a PES ex236 packet stream and a PES ex239 packet stream and furthermore into TS ex237 packets and TS packets ex240, respectively. Similarly, the data from an ex241 display graphics stream and the data from an ex244 interactive graphics stream are transformed into a PES ex24 2 packet stream and a PES ex245 packet stream and furthermore into TS ex243 packets and TS ex246 packets, respectively. These TS packages are
102
<img file="MX337291B_D0092.tif" />
multiplexed into a stream for ohteuex - ^ —- da toe multiplexed ex247.
FIGURE 27 illustrates in greater detail how a video stream is stored in a PES packet stream. The first bar in FIGURE 27 shows a video frame stream in a video stream. The second bar shows the PES packet stream. As indicated by the arrows designated yyl, yy2, yy3, and yy4 in FIGURE 27, the video stream is divided into images such as I images, B images, and P images, each of which is a video display unit and the images are stored in a payload of each of the PES packages. Each of the PES packets has a PES header and the PES header stores a Presentation Time Record (PTS) that indicates an image display time and a Decode Time Record (DTS). , which indicates a decoding time of the image.
FIGURE 28 illustrates a format of TS packets that are ultimately written to the multiplexed data. Each of the TS packets is a 188-byte fixed-length packet, which includes a 4-byte TS header that has information, such as a PID to identify a stream and a 184-byte TS payload to store data. PES packages are split and stored in payloads
103
<img file="MX337291B_D0093.tif" />
of TS, respectively. When A.A. ^ L ^ JD was used<sub>m</sub>RQM ^^ L · ™ Cafia »one of the TS packets is provided with a 4 byte TP_Extra_Header (Additional Header TP), thus resulting in 192 byte source packets. The source packets are written to the multiplexed data. The TP_Extra_Header stores information such as an Arrival_Time_Stamp (Record of Arrival Times) (ATS). The ATS displays a transfer start time at which each of the TS packets must be transferred to a PID filter. The source packets are arranged in the multiplexed data as shown in the background of FIGURE 28. The numbers that increment from the head of the multiplexed data are called source packet numbers (SPNs).
Each of the TS packages included in the multiplexed data includes not only audio, video, subtitle, and other streams, but also a Program Association Table (PAT), a Program Map Table (PMT) , and a Program Clock Reference (PCR). The PAT shows what a PID indicates in a PMT used in multiplexed data and a PID from the PAT itself is recorded as zero. PMT stores PIDs of video, audio, subtitle and other streams included in multiplexed data and stream attribute information
104 corresponding to the PIDs. The PMT tátt ± iíéri '' has ^ several descriptors that refer to multiplexed data. Descriptors have information such as copy control information that shows whether or not copying of multiplexed data is allowed. The PCR stores the STC time information that corresponds to an ATS that shows when the PCR packet is transferred to a decoder, in order to achieve a synchronization between an Arrival Time Clock (ATC) that is a time axis. of ATSs and a System Time Clock (STC) which is a time axis of
PTSs and DTSs.
FIGURE 29 illustrates the PMT data structure in detail. A PMT header is placed at the top of the PMT. The PMT header describes the length of data included in the PMT and others. A plurality of descriptors that refer to the multiplexed data is placed after the PMT header. Information such as copy control information is described in the descriptors. After the descriptors, a plurality of pieces of current information is placed that refers to the currents included in the multiplexed data. Each piece of stream information includes stream descriptors each describing information, such as a stream type to identify a stream compression codee, a stream PID and
105 stream attribute information (such as * a 'frame rate or dimensional relationship). ^ Loe- ^ etescTipt'ors of current are equal in number to the number of currents in the multiplexed data.
When multiplexed data is written to a recording medium and others, it is written together with
<td>information of</td><td colspan="2">multiplexed data.</td><td></td><td></td><td></td>
<td>Every</td><td>one</td><td>of the files</td><td>of information</td><td>of</td><td>data</td>
<td>multiplexed</td><td>is</td><td>information of</td><td colspan="2">management of</td><td>data</td>
<td>multiplexed</td><td>how</td><td>It is shown in</td><td>FIGURE 30. The</td><td colspan="2">records</td>
The multiplexed data information paths are in one-to-one correspondence with the multiplexed data and each of the files includes multiplexed data information, current attribute information, and an input map.
As illustrated in FIGURE 30, the multiplexed data information includes a system rate, a playback start time, and a playback end time. The system rate indicates the maximum transfer rate at which a target system decoder described below transfers the multiplexed data to a PID filter. The ranges of the ATSs included in the multiplexed data are set at a level no higher than a system rate. Playback start time indicates a PTS in a video frame on the
106
<img file="MX337291B_D0094.tif" />
head of the multiplexed data. An interval of one frame is added to a PTS in a video frame at the end of the multiplexed data and the PTS is set at the end of playback time.
<td>How</td><td>it shows</td><td>in FIGURE</td><td> 31,</td><td>a</td><td>piece</td><td>of</td>
<td>information of</td><td>attributes</td><td>register in</td><td>the</td><td colspan="2">information</td><td>of</td>
<td>attributes of</td><td>stream,</td><td>for each PID</td><td>of</td><td>every</td><td colspan="2">stream</td>
<td>included in</td><td>the data</td><td>multiplexed.</td><td colspan="2">Every</td><td>piece</td><td>of</td>
Attribute information has different information depending on whether the corresponding stream is a video stream, an audio stream, a presentation graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including what kind of compression codee is used for compression of the video stream and the resolution, dimensional ratio, and frame rate of the pieces of image data that are included in the video stream. Each piece of audio stream attribute information carries information including what kind of compression codee is used to compress the audio stream, how many channels are included in the audio stream, what language the audio stream supports, and how high is the sampling frequency. Video stream attribute information and audio stream attribute information are used for initialization of a
ΜΡΪ θ '
107
INSTITUTO MEX'U. '. N'j <sub>ν</sub><sup>lHS</sup> £ LA? KG? 1EDM>
INDUSTRIAL decoder before the repjc © 4aet «T -'-“ * TSprE'á ““ Ta ”information.
In the present embodiment, the multiplexed data used is of a stream type included in the PMT. Additionally, when the multiplexed data is recorded on a recording medium, the video stream attribute information included in the multiplexed data information is used. More specifically, the motion picture encoding method or motion picture encoding apparatus described in each of the modalities includes a step or unit for assigning unique information indicating the video data generated by the video encoding method. moving pictures or the moving picture coding apparatus in each of the modes, to the stream type included in the PMT or the video stream attribute information. With the configuration, the video data generated by the motion picture encoding method or the motion picture encoding apparatus described in each of the modalities can be distinguished from the video data that complies with another standard.
Additionally, FIGURE 32 illustrates steps of the moving image decoding method according to the present embodiment. In Step exSlOO, the stream type included in the PMT or the attribute information of ^ ¿3
108
Λ :.
<img file="MX337291B_D0095.tif" />
INSTITUTO MEXICANO DE LA ΙΤ '··· ί ··! £ · χ * .Ο video stream included in the multiplexed information is obtained from the multiplexed data. Then, in Step exSlOl, it is determined whether or not the stream type or video stream attribute information indicates that the multiplexed data is generated by the motion picture encoding method or the motion picture encoding apparatus. in each of the modalities. When it is determined that the stream type or video stream attribute information indicates that the multiplexed data is generated by the motion picture encoding method or the motion picture encoding apparatus in each of the modes, in In Step exS102, decoding is performed by the motion picture decoding method in each of the modes. Additionally, when the stream type or video stream attribute information indicates compliance with conventional standards, such as MPEG-2, MPEG-4 AVC and VC-1, in Step exS103, decoding is performed by a decoding method of moving images in accordance with conventional standards.
As such, assigning a new unique value to the stream type or video stream attribute information makes it possible to determine whether the motion picture decoding method or the
109
<img file="MX337291B_D0096.tif" />
<img file="MX337291B_D0097.tif" />
decoding of moving images that are dfisnrihsn in.
each of the modalities may or may not perform decoding. Even when entering multiplexed data that conforms to a different standard, an appropriate decoding method or apparatus can be selected. In this way, it becomes possible to decode information without any error. Additionally, the moving image encoding method or apparatus, or the moving image decoding method or apparatus in the present embodiment can be used in the devices and systems described above.
Mode 6
Each of the image encoding method in
<td>movement</td><td>the</td><td>apparatus</td><td>coding</td><td>of</td><td>images</td><td>in</td>
<td>movement,</td><td>the</td><td colspan="2">decoding method</td><td>of</td><td>images</td><td>in</td>
<td>movement</td><td>and the</td><td>apparatus</td><td>decoding</td><td>of</td><td>images</td><td>in</td>
<td>movement</td><td>in</td><td>each</td><td colspan="4">of the modalities is achieved</td>
<td>typically</td><td>in</td><td>the shape</td><td>of a circuit</td><td colspan="2">integrated or</td><td>a</td>
Large Scale Integrated Circuit (LSI). As an example of the LSI, FIGURE 33 illustrates a LSI ex500 configuration that is done on a chip. The LSI ex500 includes the ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 elements described below, and the elements are connected to each other via an ex510 busbar. The ex505 power supply circuit unit is activated by
110
supply each of the elements with power when the ex505 power supply circuit unit is turned on.
For example, when encoding is performed, the LSI ex500 receives an AV signal from an exll7 microphone, an exll3 camera, and others through an AV 10 ex5 09 under the control of an ex501 control unit that includes an ex502 CPU, an ex503 memory controller, an ex504 current controller, and an ex512 drive frequency control unit. The received AV signal is temporarily stored in external memory ex511, such as a
SDRAM. Under the control of the ex5Cl control unit, the stored data is segmented into data portions according to the amount and speed of processing that is transmitted to an ex507 signal processing unit. The ex507 signal processing unit then encodes an audio signal and / or a video signal. At this point, the encoding of the video signal is the encoding described in each of the modalities. Additionally, the signal processing unit ex507 sometimes multiplexes the encoded audio data and the encoded video data, and an ex506 stream 10 provides the multiplexed data to the outside. The provided multiplexed data is transmitted to the base station exl07, or written to the recording medium ex215. When
111
T multiplex the data sets, the data should be readejraiwTD »- '. *.
temporarily store in the ex508 buffer so that the data sets are synchronized with each other.
<td>Although memory ex511</td><td>is</td><td>a</td><td>element</td><td>outside the</td>
<td>LSI ex500, can be included in</td><td>the</td><td>LSI</td><td>ex500.</td><td>The memory</td>
<td>intermediate ex508 no. is limited</td><td>to</td><td>a</td><td>memory</td><td>intermediate.</td>
it can be made up of buffers. Additionally, the LSI ex500 can be made on one chip or a plurality of chips.
Additionally, although the ex501 control unit includes the ex502 CPU, ex503 memory controller, ex504 current controller, and ex512 drive frequency control unit, the configuration of the ex501 control unit is not limited to these. For example, the ex507 signal processing unit may also include a CPU. Including another CPU in the ex507 signal processing unit can improve processing speed. Additionally, as another example, the ex502 CPU may serve as or may be a part of the ex507 signal processing unit and, for example, may include an audio signal processing unit. In this case, the ex501 control unit includes the ex507 signal processing unit or the ex502 CPU which includes a part of the ex507 signal processing unit.
The name used in this document is LSI, but
112 ¿> Íf. ', SS7 can also be called IC, LSI system,
LSI depending on the degree of integration.
On the other hand, the ways to achieve integration are not limited to the LSI and a special circuit or a general-purpose processor and thus can also achieve integration. Field Programmable Gate Matrix (FPGA) that can be programmed after the manufacture of LSIs or a reconfigurable processor that allows reconfiguration of the connection or configuration of an LSI can be used for the same purpose .
In the future, with the advancement in semiconductor technology, a new technology may replace]. ' the LSI. Function blocks can be integrated using this technology. The possibility is that the present invention is applied to biotechnology.
Mode 7
When the video data generated in the motion picture encoding method or by the motion picture encoding apparatus described in each of the modalities is decoded, compared to when the video data complying with a conventional standard, such as MPEG-2, MPEG-4 AVC and VC-1 are decoded, the amount of processing may increase. In this way, the LSI ex500 needs to be set to a
113
'.sr l'h
<img file="MX337291B_D0098.tif" />
drive frequency higher than that of ex 5 n CPU is used when decoding video data in accordance with the conventional standard. However, when the drive frequency is set higher, there is a problem with increasing power consumption.
For the purpose of solving the problem, the moving image decoding apparatus such as the ex300 television and the ex500 LSI are configured to determine which standard the video data meets and to switch between the driving frequencies according to the determined standard. FIGURE 34 illustrates an ex800 configuration in the present embodiment. An ex803 drive frequency switching unit sets a drive frequency to a higher drive frequency when the video data is generated by the motion picture encoding method or the motion picture encoding apparatus described in each of the modalities. . Then, the drive frequency switching unit ex803 instructs a decoding processing unit ex801 that executes the motion picture decoding method described in each of the modes for decoding the video data. When the video data complies with the conventional standard, the drive frequency switching unit ex803 sets a drive frequency to a
114
<img file="MX337291B_D0099.tif" />
lower driving frequency than that 1st.,. of —the— video generated by the moving image coding method or the moving image coding apparatus described in each of the modalities. The ex803 driver frequency switching unit then instructs the ex802 decoding processing unit that complies with the conventional standard for decoding video data.
More specifically, the ex803 boost frequency switching unit includes the ex502 CPU and the ex512 drive frequency control unit in FIGURE 33. At this point, each ex801 decoding processing unit that executes the motion picture decoding method described in each of the modalities and the ex802 decoding processing unit that complies with the conventional standard corresponds to the signal processing unit ex507 in FIGURE 33. The ex502 CPU determines which standard the video data meets. The drive frequency control unit ex512 then determines a drive frequency based on a signal from the ex502 CPU. Additionally, the ex507 signal processing unit decodes the video data based on the ex502 CPU signal. For example, the identification information described in Mode 5 may be used to identify the video data. The information of
115
<img file="MX337291B_D0100.tif" />
Identification is not limited to that described in Mode 5 but can be any information as long as the information indicates which standard the video data meets. For example, when it can be determined which standard the video data meets based on an external signal to determine that the video data is used for a television or a disc, etc., the determination can be made based on this external signal. Additionally, the ex502 CPU selects a drive frequency based on, for example, a lookup table in which the video data standards are associated with the drive frequencies as shown in FIGURE 36. The drive frequency can be selected by storing the lookup table in the ex508 buffer and in an internal memory of an LSI and with reference to the lookup table by the ex502 CPU.
FIGURE 35 illustrates steps for executing a method in the present embodiment. First, in Step exS200, the signal processing unit ex507 obtains identifying information from the multiplexed data. Then, in Step exS201, CPU ex502 determines whether or not the video data is generated by means of the encoding method and the encoding apparatus described in each of the modes, based on the identification information. When the video data is generated by
116
<img file="MX337291B_D0101.tif" />
By means of the ^ motion image encoding method-fe-Q ^ and the moving image encoding apparatus described in each of the modes, in Step exS202, the CPU ex502 transmits a signal to set the drive frequency at a higher boost frequency to the boost frequency control unit ex512. Then the drive frequency control unit ex512 sets the drive frequency to the highest drive frequency. On the other hand, when the identifying information indicates that the video data complies with the conventional standard, such as MPEG-2, MPEG-4 AVC and VC-1, in Step exS203, CPU ex502 transmits a signal to establish the drive frequency at a lower drive frequency to the drive frequency control unit ex512. Then, the drive frequency control unit ex512 sets the drive frequency to the lower drive frequency than that in the case where the video data is generated by the motion picture encoding method and the picture encoding apparatus in movement described in each of the modalities.
Additionally, along with switching drive frequencies, the energy conservation effect can be enhanced by changing the voltage that is applied to the LSI ex500 or an appliance that includes the LSI ex500. For example, when the driving frequency is set lower, it is
117
<img file="MX337291B_D0102.tif" />
The voltage applied to the L-IóSI, ex5QQ ^ CL-> fill · apparatus including the LSI ex500 may be set to a lower voltage than that in the case where the drive frequency is set higher.
Additionally, when the amount of processing for decoding is larger, the driving frequency can be set higher, and when the amount of processing for decoding is smaller, the driving frequency can be set lower as the method of setting the driving frequency. Thus, the method of establishment is not limited to those described above. For example, when the amount of processing for decoding video data in accordance with MPEG-4 AVC is larger than the amount of processing for decoding video data generated by the motion picture encoding method and the video apparatus. With the encoding of moving images described in each of the modalities, the driving frequency may be set in the reverse order of the setting described above.
Additionally, the method of setting the drive frequency is not limited to the method of setting the lowest drive frequency. For example, when the identifying information indicates that the video data is generated by the encryption method of
118
<img file="MX337291B_D0103.tif" />
Moving images and the moving image encoding apparatus described in each of the modalities, the voltage applied to the LSI ex500 or the apparatus including the LSI ex500 may be set higher. When the identifying information indicates that the video data complies with the conventional standard, such as MPEG-2, MPEG-4 AVC, and VC-1, it is possible that the voltage applied to the LSI ex500 or the device that includes the LSI ex500 is set lower. As another example, it is possible that when the identification information indicates that the video data is generated by the motion picture encoding method and the motion picture encoding apparatus described in each of the modalities, the driving of the ex502 CPU does not have to suspend. When the identifying information indicates that the video data complies with the conventional standard, such as MPEG-2, MPEG-4 AVC and VC-1, it is possible that the ex502 CPU drive is suspended in a certain time because the ex502 CPU has additional processing capacity. Even when the identifying information indicates that the video data is generated by the motion picture encoding method and the motion picture encoding apparatus described in each of the modes, it is possible that in the case where the CPU ex502 has additional processing power, the ex502 CPU drive is suspended in a
119 set time. In this case, the suspension time may be set shorter than that in the case when the identifying information indicates that the video data complies with the conventional standard, such as MPEG-
<img file="MX337291B_D0104.tif" />
1Ν5Ί π ul *.? r.ir.Aiv.i, '.. „__
CELA PROPitCAD V,
I STR i AL
2, MPEG-4 AVC and VC-1.
Accordingly, the energy conservation effect can be improved by switching between the driving frequencies according to the standard that the video data meets. Additionally, when the LSI ex500 or the device that includes the LSI ex500 is powered using a battery, the life of the battery can be extended with the effect of conserving energy.
Mode 8
There are cases where a plurality of video data that meets different standards is provided to devices and systems, such as a television and a cell phone. In order to make it possible to decode the plurality of video data that meet the different standards, the LSI ex500's signal processing unit ex507 needs to meet the different standards. However, the problems of increasing the scale of the LSI ex500 circuit and increasing the cost rise with the individual use of the ex507 signal processing units that comply with the respective standards.
120
<img file="MX337291B_D0105.tif" />
For the purpose of solving the probl.g.ma,. „J. ^„ That ..- ^ e _—— idea is a configuration in which the decoding processing unit to implement the described motion picture decoding method in each of the modalities and the standard compliant decoding processing unit such as MPEG-2, MPEG-4 AVC and VC-1 are partially shared. Ex900 in FIGURE 37A shows an example of the configuration. For example, the motion picture decoding method described in each of the modalities and the motion picture decoding method that complies with MPEG-4 AVC have processing details, such as entropic encoding, partially in common. , inverse quantization, unlock filtering, and motion compensation prediction. Shared processing details may include the use of an ex902 decoding processing unit that complies with MPEG-4 AVC. In contrast, it is possible for a specialized ex901 decoding processing unit to be used for other processing that is unique to one aspect of the present invention and that is not MPEG-4 AVC compliant. Since the aspect of the present invention is characterized by arithmetic decoding in particular, for example, the specialized decoding processing unit ex901 is used for
121
<img file="MX337291B_D0106.tif" />
ι; .5 ·. '·.'; 'υνο LE ι ..'
I.
V? 1 arithmetic decoding. Moreover, it is pos.ibla ^ uaue ^ Jpa · ^ - decoding processing unit is shared for one of the arithmetic decoding, unlock filtering and motion compensation or the entire processing. The decoding processing unit to implement the motion picture decoding method described in each of the modalities can be shared so that the processing is shared and a specialized decoding processing unit can be used for single processing for that of
MPEG-4 AVC.
Additionally, exlOOO in FIGURE 37B shows another example where processing is partially shared. This example uses a configuration that includes a specialized exlOOl decoding processing unit that supports single processing for one aspect of the present invention, a specialized decoding processing unit exl002 supporting unique processing for another conventional standard and a decoding processing unit exl003 supporting processing that is shared between the motion picture decoding method in accordance with the aspect of the present invention and the conventional motion picture decoding method. At this point, the decoding processing units
122
<img file="MX337291B_D0107.tif" />
ExlOOl and exl002 are not necessarily specialized for processing in accordance with the aspect of the present invention and conventional standard processing, respectively, and may be those capable of implementing general processing. Additionally, the configuration of the present modality can be implemented by the LSI ex500.
As such, reducing the scale of the circuit of an LSI and reducing the cost are possible by sharing the decoding processing unit so that the processing is shared between the motion picture decoding method according to the aspect of the present invention and the method of decoding moving images in accordance with the conventional standard.
Industrial Applicability
The present invention is applicable to an image encoding method, an image decoding method, an image encoding apparatus, and an image decoding apparatus, and in particular, is applicable to an image encoding method, a method of image decoding, an image encoding apparatus and an image decoding apparatus which use arithmetic encoding and arithmetic decoding.
<img file="MX337291B_D0108.tif" />
123
<img file="MX337291B_D0109.tif" />
List of Reference Signs · --100 Image coding apparatus
101 Control unit
102 Subtraction unit
103 Transformation and quantification unit
104 Variable length coding unit
105 Inverse quantification and inverse transformation unit
106,
107,
108,
109,
121
122,
123,
124
126
127,
130,
141
142,
143
151,
152,
206 Addition unit
207 Intra-prediction unit
208 Inter-prediction unit
209 Switch
Input image signal
125, 225 Residual signal
223 Quantified transformation coefficients
Bit stream
Reconstructed image signal
128, 129, 227, 228 Image prediction signal
230 Control parameter
Binarization unit
242 Context control unit
Binary arithmetic coding unit
251 Binary sequence
252 Context Index
200 Image decoding apparatus
124
<img file="MX337291B_D0110.tif" />
..i- '.
<img file="MX337291B_D0111.tif" />
mixed institute:;. '. FROM THE ΚϋίυΊΑ-i
INDUSl lit .'- S201 Control unit
202 Variable length decoding unit
204 Inverse quantification unit
205 Inverse transformation unit
224 Orthogonal transformation coefficients
226 Decoded image signal
229 Picture signal
241 Reverse binarization unit
243 Binary arithmetic decoding unit
It is noted that in relation to this date, the best method known by the applicant to put the aforementioned invention into practice is the one that is clear from the present description of the invention.
125
T &, / '
J- JA i? Cr / :.
Contents24
148 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40 Sheet 41 Sheet 42 Sheet 43 Sheet 44 Sheet 45 Sheet 46 Sheet 47 Sheet 48 Sheet 49 Sheet 50 Sheet 51 Sheet 52 Sheet 53 Sheet 54 Sheet 55 Sheet 56 Sheet 57 Sheet 58 Sheet 59 Sheet 60 Sheet 61 Sheet 62 Sheet 63 Sheet 64 Sheet 65 Sheet 66 Sheet 67 Sheet 68 Sheet 69 Sheet 70 Sheet 71 Sheet 72 Sheet 73 Sheet 74 Sheet 75 Sheet 76 Sheet 77 Sheet 78 Sheet 79 Sheet 80 Sheet 81 Sheet 82 Sheet 83 Sheet 84 Sheet 85 Sheet 86 Sheet 87 Sheet 88 Sheet 89 Sheet 90 Sheet 91 Sheet 92 Sheet 93 Sheet 94 Sheet 95 Sheet 96 Sheet 97 Sheet 98 Sheet 99 Sheet 100 Sheet 101 Sheet 102 Sheet 103 Sheet 104 Sheet 105 Sheet 106 Sheet 107 Sheet 108 Sheet 109 Sheet 110 Sheet 111 Sheet 112 Sheet 113 Sheet 114 Sheet 115 Sheet 116 Sheet 117 Sheet 118 Sheet 119 Sheet 120 Sheet 121 Sheet 122 Sheet 123 Sheet 124 Sheet 125 Sheet 126 Sheet 127 Sheet 128 Sheet 129 Sheet 130 Sheet 131 Sheet 132 Sheet 133 Sheet 134 Sheet 135 Sheet 136 Sheet 137 Sheet 138 Sheet 139 Sheet 140 Sheet 141 Sheet 142 Sheet 143 Sheet 144 Sheet 145 Sheet 146 Sheet 147 Sheet 148
53 members in 18 offices
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 61500163 | United States of America | – | |
| 201161500163 | United States of America | P |
Members53
| Document | Office | Kind | |
|---|---|---|---|
| CA2837532A1 | Canada | A1 | |
| US2012328010A1 | United States of America | A1 | |
| WO2012176463A1 | World Intellectual Property Organization (WIPO) | A1 | |
| TW201322772A | Taiwan Province of China | A | |
| JP5318292B2 | Japan | B2 | |
| JP2013225926A | Japan | A | |
| JP5342084B1 | Japan | B1 | |
| JP2013232966A | Japan | A | |
| AU2012274778A1 | Australia | A1 | |
| SG195163A1 | Singapore | A1 | |
| PH12013502458A1 | Philippines | A1 | |
| CN103609113A | China | A | |
| MX2013013508A | Mexico | A | |
| KR20140028033A | Republic of Korea | A | |
| EP2725791A1 | European Patent Office (EPO) | A1 | |
| US8743969B2 | United States of America | B2 | |
| EP2725791A4 | European Patent Office (EPO) | A4 | |
| JPWO2012176463A1 | Japan | A1 | |
| RU2013152627A | Russian Federation | A | |
| ZA201308890B | South Africa | B | |
| MX337291BThis record | Mexico | B | |
| AU2012274778B2 | Australia | B2 | |
| BR112013030469A2 | Brazil | A2 | |
| JP6016033B2 | Japan | B2 | |
| RU2602672C2 | Russian Federation | C2 | |
| TWI562592B | Taiwan Province of China | B | |
| JP2017022758A | Japan | A | |
| CN103609113B | China | B | |
| CN106878731A | China | A | |
| JP6225405B2 | Japan | B2 | |
| JP2018026839A | Japan | A | |
| MY165357A | Malaysia | A | |
| CA2837532C | Canada | C | |
| JP6471989B2 | Japan | B2 | |
| JP2019054557A | Japan | A | |
| USRE47366E | United States of America | E | |
| CN106878731B | China | B | |
| USRE47537E | United States of America | E | |
| USRE47547E | United States of America | E | |
| KR102008030B1 | Republic of Korea | B1 | |
| JP6685021B2 | Japan | B2 | |
| USRE48810E | United States of America | E | |
| BR112013030469B1 | Brazil | B1 | |
| EP4228264A1 | European Patent Office (EPO) | A1 | |
| EP2725791B1 | European Patent Office (EPO) | B1 | |
| PL2725791T3 | Poland | T3 | |
| ES2962519T3 | Spain | T3 | |
| USRE49906E | United States of America | E | |
| EP4404558A2 | European Patent Office (EPO) | A2 | |
| EP4228264B1 | European Patent Office (EPO) | B1 | |
| EP4404558A3 | European Patent Office (EPO) | A3 | |
| PL4228264T3 | Poland | T3 | |
| ES2991675T3 | Spain | T3 |
Numbers
- Publication
- 337291
- Application
- 2015003467
Titles2
- Spanish
- METODO DE DECODIFICACION DE IMAGENES, METODO DE CODIFICACION DE IMAGENES, APARATO DE DECODIFICACION DE IMAGENES, APARATO DE CODIFICACION DE IMAGENES Y APARATO DE CODIFICACION Y DECODIFICACION DE IMAGENES.
- English
- IMAGE DECODING METHOD, IMAGE ENCODING METHOD, IMAGE DECODING DEVICE, IMAGE ENCODING DEVICE, AND IMAGE ENCODING/DECODING DEVICE.
Classification
- CPC, 7
- H04N19/157
- H04N19/176
- H04N19/463
- H04N19/91
- H04N19/13
- H04N19/197
- H04N19/196
- IPC, 2
- H04N7 12
- H04N19 00