MX358071B

Motion-constrained tile set for region of interest coding.

Abstract

Control data for a motion-constrained tile set ("MCTS") indicates that inter-picture prediction processes within a specified set of tiles are constrained to reference only regions within the same set of tiles in previous pictures in decoding (or encoding) order. For example, a video encoder encodes multiple pictures partitioned into tiles to produce encoded data. The encoder outputs the encoded data along with control data (e.g., in a supplemental enhancement information message) that indicates that inter-picture prediction dependencies across tile set boundaries are constrained for a given tile set of one or more of the tiles. A video decoder or other tool receives the encoded data and MCTS control data, and processes the encoded data. Signaling and use of MCTS control data can facilitate region-of-interest decoding and display, transcoding to limit encoded data to a selected set of tiles, loss robustness, parallelism in encoding and/or decoding, and other video processing.

MX358071B, drawing sheet 1 — Sheet 1 of 112

Term

7 yearsleft in the term

Expires 20 September 2033.

35 claims: 8 independent, 27 dependent

1
REIVINDICACIONES X 1 ' iU ' / 1.- Un sistema de computadora que comprende una o más unidades de procesamiento y memoria, en donde el sistema de computadora implementa un sistema codificador configurado para realizar un método que comprende:codificar múltiples imágenes para producir datos codificados, en donde cada una de las múltiples imágenes se divide en múltiples mosaicos;y / dar salida a los datos codificados junto con datos de control que indican que dependencias de predicción de ínter-imagen a través de límites específicos se restringen para un conjunto de mosaicos dado de uno o más mosaicos de los múltiples mosaicos, en donde el conjunto de mosaicos dado es parametrizado en los datos de control como una o más regiones de mosaico que cubren el uno o más mosaicos de los múltiples mosaicos, y en donde los datos de control incluyen: un parámetro de conteo que Índica un conteo de regiones de mosaico en el conjunto de mosaicos dado;y para cada una de las una o más regiones de mosaico en el conjunto de mosaicos dado, elementos de sintaxis que indican la ubicación de la región de mosaico dentro de las múltiples imágenes.
2
2- El sistema de computadora de conformidad con la reivindicación 1, en donde la una o más regiones de mosaico son uno o más rectángulos de mosaico, y en donde, para un rectángulo IMPI INSTITUTO MEXICANO DE LA MONEDAD INDUSTRIAL de mosaico dado del uno o más rectángulos—ele—rn-esa-i^e—en—eX. conjunto de mosaicos dado, los elementos de sintaxis que indican la ubicación de la región de mosaico son elementos de sintaxis que identifican dos esquinas del rectángulo de mosaico dado.
3
3, - El sistema de computadora de conformidad con la reivindicación 2, en donde las dos esquinas son una esquina izquierda superior del rectángulo de mosaico dado y una esquina derecha inferior del rectángulo de mosaico dado.
4
4, - El sistema de computadora de conformidad con la reivindicación 1, en donde los datos de control además incluyen:un identificador del conjunto de mosaicos dado.
5
5, - El sistema de computadora de conformidad con la reivindicación 1, en donde los datos de control son un mensaje de información de mejora suplementaria (“SEI”) que indica que dependencias de predicción de inter-imagen a través de límites de conjunto de mosaicos se restringen para el conjunto de mosaicos dado. δ.- El sistema de computadora de conformidad con la reivindicación 1, en donde el sistema codificador además está configurado para:decidir si dependencias de predicción de inter-imagen a través de límites específicos se van a restringir para el conjunto de mosaicos dado;y si es así, restringir la estimación de movimiento durante la codificación de manera que las dependencias de predicción de IMPI INSTITUTO MEXICANO Oí IA ntONEMC INDUSTRIA! inter-imagen a través de límites específicos s a.. »v ¡tora.. conjunto de mosaicos dado.
6
7, - El sistema de computadora de conformidad con la reivindicación 1, en donde los datos de control además indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno de uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde la codificación utiliza procesamiento paralelo para al menos algunas etapas de codificación para el conjunto de mosaicos dado y el uno o más de otros conjuntos de mosaicos.
7
8, - El sistema de computadora de conformidad con la reivindicación 1, en donde los datos de control además indican que dependencias de predicción de inter-imagen a través de límites específicos están restringidos para cada uno de uno o más de otros conjuntos de mosaicos de los mosaicos múltiples, y en donde la codificación configura el conjunto de mosaicos dados y uno o más de otros conjuntos de mosaicos para proporcionar funcionalidad de actualización de decodificador gradual. ,
8
9, - En un sistema de computadora, un método que comprende:recibir datos codificados para múltiples imágenes, en donde cada una de las múltiples Imágenes se divide en múltiples mosaicos;recibir datos de control que indican que dependencias de predicción de inter-imagen a través de límites específicos se INSTITUTG ΜΞΧ’.CANO D£ LA ?ROí’!tDAD INDUSTRIAL restringen para un conjunto de mosaicos dado de uno n más mosaicos de los múltiples mosaicos, en donde el conjunto de mosaicos dado es parametrizado en los datos de control como una o más regiones de mosaico que 5 cubren el uno o más mosaicos de los múltiples mosaicos, y en donde los datos de control incluyen: un parámetro de conteo que indica un conteo de regiones de mosaico en el conjunto de mosaicos dado;y para cada una de las una o más regiones de mosaico en el
9
10 conjunto de mosaico dado, elementos de sintaxis que indican la ubicación de 1a región de mosaico dentro de las múltiples imágenes;y procesar los datos codificados. 10. - El método de conformidad con la reivindicación 9, en 15 donde la una o más regiones de mosaico son uno o más rectángulos de mosaico, y en donde, para un rectángulo de mosaico dado del uno o más rectángulos de mosaico en el conjunto de mosaicos dado, los elementos de sintaxis que indican la ubicación de la región de mosaicos son elementos de sintaxis que 20 identifican dos esquinas del rectángulo de mosaico dado.
10
11, - El método de conformidad con la reivindicación 10, en donde las dos esquinas son una esquina izquierda superior del rectángulo de mosaico dado y una esquina derecha inferior del rectángulo de mosaicos dado. z
11
12,- El método de conformidad con la reivindicación 9. en IMPI INSTITUTO MEXICANO DI LA FROPIEDAD INDUSTRIAL donde los datos de control además incluyen:..un,-Ldenti.f.iea.cío-r—d-ej· conjunto de mosaicos dado.
12
13, - El método de conformidad con la reivindicación 9, en donde los datos de control son un mensaje de información de mejora suplementaria (“SEI”) que indica que dependencias de predicción de ¡nter-imagen a través de limites del conjunto de mosaicos se restringen para el conjunto de mosaicos dado.
13
14, - El método de conformidad con la reivindicación 9. en donde los datos de control además indican que dependencias de predicción de ínter-imagen a través de límites específicos están restringidos para cada uno de uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde el procesamiento incluye la decodificación que utiliza procesamiento paralelo para al menos algunas etapas de decodificación para el conjunto de mosaicos dado y el uno o más de otros conjuntos de mosaicos. t
14
15, - El método de conformidad con la reivindicación 9, en donde el procesamiento incluye:después de la detección de pérdida de al menos algunos de los datos codificados diferentes al conjunto de mosaicos dado, la decodificación del conjunto de mosaicos dado como parte de la recuperación de pérdida.
15
16, - Un medio legible por computadora que provoca que un sistema de computadora realice un método, en donde el medio legible por computadora se selecciona del grupo que consiste en memoria volátil, memoria no volátil, disco magnético, un CD-ROM, y un DVD, y en donde el método comprende:recibir datos codificados para múltiples imagené¿, Sil dund!·? cada una de las múltiples imágenes se divide en múltiples mosaicos;recibir datos de control que indican que dependencias de predicción de ínter-imagen a través de límites específicos se restringen para un conjunto de mosaicos dado de uno o más mosaicos de los múltiples mosaicos, en donde los datos de control incluyen: un identificador del conjunto de mosaicos dado;un parámetro de conteo que indica un conteo de regiones de mosaico en el conjunto de mosaicos dado;y para cada una de las regiones de mosaico en el conjunto mosaicos dado, elementos de sintaxis que indican la ubicación de la región de mosaico dentro de las múltiples imágenes;y procesar los datos codificados.
16
17, - El medio legible por computadora de conformidad con la reivindicación 16, en donde los datos de control son un mensaje de Información de mejora suplementaria (“SEI”) que indica que dependencias de predicción de inter-imagen a través de límites de conjunto de mosaicos se restringen para el conjunto de mosaicos dado, y en donde el mensaje de SEI incluye un elemento de sintaxis que es utilizado por un sistema decodificador para evaluar si la calidad puede ser afectada adversamente si solo el conjunto de mosaicos dado es decodiflcado.
17
18, - El medio legible por computadora de conformidad con la IMPI INSTITUTO MEXICANO DE LA PP.OPÍSDAO INDUSTRIAL reivindicación 16, en donde los datos de control además indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno de uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde los datos de control además incluyen:un parámetro que indica un conteo de conjuntos de mosaicos;para cada uno del uno o más de otros conjuntos de mosaicos: un ¡dentificador del otro conjunto de mosaicos;un parámetro de conteo que indica un conteo de regiones de mosaico en el otro conjunto de mosaicos;y para cada una de las regiones de mosaico en el otro conjunto de mosaicos, elementos de sintaxis que indican la ubicación de la región de mosaico dentro de las múltiples imágenes.
18
19,- El medio legible por computadora de conformidad con la reivindicación 16, en donde los datos de control además indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno del uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y , en donde procesar los datos codificados incluye utilizar los datos de control para controlar:(a) decodificar el conjunto de mosaicos dado como una región-de-interés dentro de las múltiples imágenes sin decodificar porciones de las múltiples imágenes fuera del conjunto de mosaicos dado, (b) transcodiflcar los datos codificados, (c) procesamiento paralelo para al menos algunas IMPI INSTITUTO MEXICANO DE LA PROPIEDAD INDUSTRIAL etapas de decodificación para el conjunto denvusjíluíí dudu y u.l uno o más de otros conjuntos de mosaicos, o (d) después de la detección de pérdida de al menos algunos de los datos codificados, decodificar el conjunto de mosaicos dado como parte de la recuperación de pérdida.
19
20, - El sistema de computadora de conformidad con la reivindicación 5, en donde el mensaje de SEI incluye un elemento de sintaxis que es utilizable por un sistema decodificador para evaluar si la calidad puede ser afectada adversamente si solo el conjunto de mosaicos dado es decodiflcado.
20
21, - El sistema de computadora de conformidad con la reivindicación 1, en donde los datos de control además indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno de uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde los datos de control además incluyen:un parámetro que indica un conteo de conjuntos de mosaicos;y para cada uno del conjunto de mosaicos dado y uno o más de otros conjuntos de mosaicos, un identificador del conjunto de mosaicos. .
21
22, - El sistema de computadora de conformidad con la reivindicación 1, en donde ios múltiples mosaicos son organizados de acuerdo a filas de mosaicos y columnas de mosaicos que, con límites de imágenes, definen límites horizontales y límites INSTITUTO MCXICa. · :DE LA PROHEO/· INOl.'STVfj verticales de tos mosaicos múltiples H ° n dg —loe m ú 11' p i ? ? imágenes.
22
23,- El método de conformidad con la reivindicación 13, en donde el mensaje de SEI incluye un elemento de sintaxis que es utilizable por un sistema decodificador para evaluar si la calidad puede ser afectada adversamente si solo el conjunto de mosaicos dado es decodificado.
23
24 - El método de conformidad con la reivindicación 9, en donde los datos de control además indican que dependencias de predicción de ínter-imagen a través de límites específicos se restringen para cada uno del uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde los datos de control además incluyen:un parámetro que indica un conteo de conjuntos de mosaicos;para cada uno del conjunto de mosaicos dado y uno o más de otros conjuntos de mosaicos, un identificador del conjunto de mosaicos. z
24
25. - El método de conformidad con la reivindicación 9, en donde los múltiples mosaicos son organizados de acuerdo a filas de mosaicos y columnas de mosaicos que, con límites de imagen, definen límites horizontales y límites verticales de los múltiples mosaicos dentro de las múltiples imágenes.
25
26, - El método de conformidad con la reivindicación 9, en donde los datos de control además indican que dependencias de IMPI INSTITUTO MEXICANO DE LA PROPIEDAD INDUSTMM. predicción de ínter-imagen a través de límites—eeperrrftrtr5~5F restringen para cada uno del uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde el procesar los datos codificados incluye utilizar los datos de control para controlar:(a) la decodificación del conjunto de mosaicos dado como una región-de-interés dentro de las múltiples imágenes sin decodificar porciones de las múltiples imágenes fuera del conjunto de mosaicos dado, (b) transcodificar los datos codificados, (c) procesamiento paralelo para al menos algunas etapas de decodificación para el conjunto de mosaicos dado y el uno o más de otros conjuntos de mosaicos, o (d) después de la detección de pérdida de al menos algunos de los datos codificados, decodificar el conjunto de mosaicos dado como parte de la recuperación de pérdida.
26
27,- Un medio legible por computadora que provoca que un sistema de computadora realice un método, en donde el medio legible por computadora se selecciona del grupo que consiste en memoria volátil, memoria no volátil, disco magnético, un CD-ROM, y un DVD, y en donde el método comprende:codificar múltiples imágenes para producir datos codificados, en donde cada una de las múltiples imágenes se divide en múltiples mosaicos;y dar salida a los datos codificados junto con datos de control que indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para un conjunto de IMPI INSTITUTO MEXICANO DE LA PROPIEDAD industrial mosaicos dado de uno o más mosaicos de los múltiples mosaicos, en donde el conjunto de mosaicos dado es parametrizado en los datos de control como una o más regiones de mosaico que cubren el uno o más mosaicos de los múltiples mosaicos, y en donde los datos de control incluyen: un parámetro de conteo que indica un conteo de regiones de mosaico en el conjunto de mosaicos dado;y para cada una de las una o más regiones de mosaico en el conjunto de mosaicos dado, elementos de sintaxis que indican la ubicación de la región de mosaico dentro de múltiples imágenes.
27
28, - El medio legible por computadora de conformidad con la reivindicación 27, en donde la una o más regiones de mosaico son uno o más rectángulos de mosaico, y en donde, para un rectángulo de mosaico dado del uno o más rectángulos de mosaico en el conjunto de mosaicos dado, los elementos de sintaxis que indican la ubicación de la región de mosaico son elementos de sintaxis que identifican dos esquinas del rectángulo de mosaico dado.
28
29. - El medio legible por computadora de conformidad con la reivindicación 27, en donde los datos de control son un mensaje de información de mejora suplementaria (“SEI”) que indica que dependencias de predicción de inter-imagen a través de límites de conjunto de mosaicos se restringen para el conjunto de mosaicos dado, y en donde el mensaje de SEI incluye un elemento de sintaxis que es utilizable por un sistema decodificador para evaluar si la calidad puede ser afectada adversamente si solo el INSTITUTO MEXICANO DE LA PROPIEDAD INDUSTRIAL conjunto de mosaicos dado es decodificado.
29
30, - El medio legible por computadora de conformidad con la reivindicación 27, en donde los datos de control además Indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno de uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde los datos de control además incluyen:un parámetro que indica un conteo de conjuntos de mosaico;y para cada uno del conjunto de mosaicos dado y uno o más de otros conjuntos de mosaico, un identificador del conjunto de mosaicos.
30
31, - Un sistema de computadora que comprende una o más unidades de procesamiento y memoria, en donde el sistema de computadora implementa un sistema decodificador configurado para realizar un método que comprende;recibir datos codificados para múltiples imágenes, en donde cada una de las múltiples Imágenes se divide en múltiples mosaicos;recibir datos de control que indican que dependencias de predicción de inter-imagen a través de limites específicos se restringen para un conjunto de mosaicos dado de uno o más mosaicos de los múltiples mosaicos, en donde el conjunto de mosaicos dado es parametrizado en los datos de control como una o más regiones de mosaico que cubren el uno o más mosaicos de 100 los múltiples mosaicos, y en donde los datos d’g”U'UH Li UT'i llllu y un. un parámetro de conteo que indica un conteo de regiones de mosaico en el conjunto de mosaicos dado;y para cada una de la una o más regiones de mosaico en el conjunto de mosaico dado, elementos de sintaxis que indican la ubicación de la región de mosaico dentro de las múltiples imágenes;y procesar los datos codificados.
31
32, - El sistema de computadora de conformidad con la reivindicación 31, en donde la una o más regiones de mosaico son uno o más rectángulos de mosaico, y en donde, para un rectángulo de mosaico dado del uno o más rectángulos de mosaico en el conjunto de mosaicos dado, los elementos de sintaxis que indican la ubicación de la región de mosaico son elementos de sintaxis que identifican dos esquinas del rectángulo de mosaico dado.
32
33, - El sistema de computadora de conformidad con la reivindicación 31, en donde los datos de control son un mensaje de información de mejora suplementaria (“SEI) que indica que dependencias de predicción de inter-imagen a través de límites de conjunto de mosaicos se restringen para el conjunto de mosaicos dado, y en donde el mensaje de SEI incluye un elemento de sintaxis que es utilizado por un sistema decodificador para evaluar si la calidad puede ser afectada adversamente si solo el conjunto de mosaicos dado es decodificado.
33
34, - El sistema de computadora de conformidad con la 101 INSTITUTO MEXICANO DE LA PROPIEDAD INDUSTRIAL reivindicación 31, en donde los datos de conlTUr”adtímas IllUiCdiT que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno de uno o más conjuntos de mosaicos de los múltiples mosaicos, y en donde los datos de control además incluyen:un parámetro que indica un conteo de conjuntos de mosaicos;y para cada uno del conjunto de mosaicos dado y uno o más de otros conjuntos de mosaicos, un identificador del otro conjunto de mosaicos.
34
35,- El sistema de computadora de conformidad con la reivindicación 31, en donde ios datos de control además indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para cada uno del uno o más de otros conjuntos de mosaicos de los múltiples mosaicos, y en donde procesar los datos codificados incluye utilizar los datos de control para controlar:(a) la decodificación del conjunto de mosaicos dado como una región-de-interés dentro de las múltiples imágenes sin decodificar porciones de las múltiples imágenes fuera del conjunto de mosaicos dado, (b) transcodificación de los datos codificados, (c) procesamiento paralelo para ai menos algunas etapas de decodificación para el conjunto de mosaicos dado y el uno o más de otros conjuntos de mosaicos, o (d) después de la detección de pérdida de al menos algunos de los datos codificados, decodificar el conjunto de mosaicos dado como parte de la recuperación de 102 INSTITUTO MEXICAN!' DE LA ΡΧΟΡΙΕΓΜΤ INTIUST»!*! pérdida.
35
36,- Un medio legible por computadora que almacena datos codificados para múltiples imágenes y datos de control, en donde el medio legible por computadora se selecciona del grupo que consiste en memoria volátil, memoria no volátil, disco magnético, un CD-ROM, y un DVD, en donde los datos codificados y los datos de control son organizados para facilitar etapas que comprenden, recibir los datos codificados, recibir los datos de control, y procesar los datos codificados, y en donde:cada una de las múltiples imágenes se divide en múltiples mosaicos: los datos de control indican que dependencias de predicción de inter-imagen a través de límites específicos se restringen para un conjunto de mosaicos dado de uno o más mosaicos de los múltiples mosaicos;el conjunto de mosaicos dado es parametrizado en los datos de control como una o más regiones de mosaico que cubren el uno o más mosaicos de los múltiples mosaicos, y los datos de control incluyen: un parámetro de conteo que Indica un conteo de regiones de mosaico en el conjunto de mosaicos dado;y para cada una de la una o más regiones de mosaico en el conjunto de mosaicos dado, elementos de sintaxis que indican la ubicación de la región de mosaico dentro de las múltiples imágenes. 1 03

Independent claims35

505 paragraphs in 95 sections, as filed

(54) Title: RESTRICTED MOVEMENT MOSAIC ASSEMBLY FOR INTEREST CODING REGION. (54) Title: MOTION-CONSTRAINED TILE SET FOR REGION OF INTEREST CODING.

(57) Summary

Control data for a restricted motion mosaic set (MCTS) indicates that inter-image prediction procedures within a specified set of mosaics are restricted to referencing only regions within the same mosaic set in previous images in decoding order (or coding). For example, a video encoder encodes multiple images divided into tiles to produce encoded data. The encoder outputs the encoded data along with control data (for example, in a supplemental enhancement information message) indicating that inter-image prediction dependencies across tile set boundaries are restricted to a set of given mosaic of one or more of the mosaics. A video decoder or other tool receives the encoded data and MCTS control data, and processes the encoded data. Signaling and use of MCTS control data can facilitate decoding and display of region of interest, transcoding to limit encoded data to a selected set of tiles, lost robustness, parallelism to encode and / or decode, and other video processing.

(57) Abstract

Control data for a motion-constrained tile set (MCTS) indicates that inter-picture prediction processes within a specified set of tiles are constrained to reference only regions within the same set of tiles in previous pictures in decoding (or encoding) order. For example, a video encoder encodes multiple pictures partitioned into tiles to produce encoded data. The encoder outputs the encoded data along with control data (eg, in a supplemental enhancement Information message) that indicates that inter-picture prediction dependencies across tile set boundaries are constrained for a given tile set of one or more of the tiles. A video decoder or other tool receives the encoded data and MCTS control data, and processes the encoded data. Signaling and use of MCTS control data can facilitate region-of-interest decoding and display, transcoding to limit encoded data to a selected set of tiles, loss robustness, parallelism in encoding and / or decoding, and other video Processing.

PATENT TITLE No. 358071

<td>Headlines):</td><td colspan="2">MICROSOFT TECHNOLOGY LICENSING, LLC</td>

<td>Home:</td><td>One Microsoft Way, Redmond, Washington, 98052, USA</td><td></td>

<td>Denomination:</td><td>RESTRICTED MOVEMENT MOSAIC SET FOR INTERESTING CODING.</td><td>REGION OF</td>

<td>Classification:</td><td>CIP: H04N19 / 55; H04N19 / 40; H04N19 / 70; H04N19 / 436;</td><td>H04N19 / 577</td>

<td>Inventor (s):</td><td>YONGJUN WU: GARY J SULLIVAN; YIFU ZHANG</td><td></td>

REQUEST

Number: International Presentation Date:

MX / a / 2015/014202 September 20, 2013

<td></td><td colspan="2">PRIORITY</td>

<td>Country:</td><td>Date:</td><td>Number:</td>

<td>US</td><td>April 8, 2013</td><td> 61/809,427</td>

<td>US</td><td>September 9, 2013</td><td> 14/021,883</td>

Validity: Twenty years

Expiration Date: September 20, 2033 Issue Date: August 3, 2018

The referenced patent is granted based on articles 1, 2 · section V, 6 ° section (H, and 59 of the Industrial Property Law,

In accordance with article 23 of the Industrial Property Law, this patent has a non-extendable term of twenty years, counted from the date of filing of the international application and I will be subject to the payment of the fee to maintain the rights in force.

Whoever signs this title does so based on the provisions of articles 6 fractions III and 7 bis 2 of the Industrial Property Law (Official Gazette of the Federation (DO.F) 06/27/1991, amended 06/02/1994, 10/25/1996, 12/26/1997, 05/17/1999, 01/26/2004, 06/16/2005, 01/25/2006, 06/05/2009 / 06 / 01/2010, 06/18/2010, 06/28/2010, 01/27/2012 and 04/09/2012); Articles 1, 3, section V, subsection a), 4, and 12 »sections I and lll of the Regulations of the Mexican Institute of Industrial Property (DOF 14/12/1999, amended on 01 / Q7 / 20G2, 15/07 / 2004, 07/28/2004 and 09/07/2007); Articles 1, 3, 4, 5, section V, subsection a), 16 sections t and lll and 30 of the Organic Statute of the Institute · Mexican of Industrial Property DO.F, 12/27/1999, amended on 10/10 / 2002, 07/29/2004, 08/04/2004 and 09/13/2007), 1, 3 * and 5 »subsection a) of the Agreement that delegates powers to the Deputy Directors General, Coordinator, Divisional Directors, Holders of the Regional Offices, Divisional Deputy Directors, Departmental Coordinators and other subordinates of the Mexican Institute of Industrial Property (DOF 12/15/1999, amended on 02/04/2000, 07/29/2004, 08/04/2004 and 09/13/2007).

This letter is signed with an advanced electronic signature (FIEL), based on articles 7 BIS 2 of the Industrial Property Law; 3rd of its Regulations, and 1 section III, 2 section V, 26 BIS and 26 TER of the Agreement establishing the guidelines for the use of the Electronic Payment and Services Portal (PASE) of the Mexican Institute of Industrial Property, in the procedures that are indicated,

DIVISIONAL DIRECTOR OF PATENTS NAHANNY CANAL REYES

Original string:

NAHANNY MARISOL CANAL REYES (000010000004032527S3¡Tax Administration Service¡1695 | ¡MX / 2018 / 65268jMX / a / 2015/014202 | Patent title PCT¡1027 [RGZ | Pág (s) 1 | + lajdAe8Li7TLMGeYRHomeM

Digital stamp:

SKJS7GV / SHP \ A / 28WEJ8up0lVSUXvLtlgi9i7cn9Omj5ds5xtwendFHC6SOgKcM7188 + ip + AL3nAqS8PooM5BMw2CpTy vOnRX7WJAXZeKPQtLlem3MV6ELMGICYSzvlsLceSTPgQ5r7T65puÍGFx1pbmTgxD + tKGbu5K / U1kRegxBPvq7FlmSP dfoDar9HiDzAKOgT47DLSBa3os / uYAf09u3VliqoA7Qjw4wlX5KwrUPs2w7vQXvevT¡8PRm2MwQcTxgZrcNSQPgNrD glHzHye584w4ybJoe4d61y9Wsy¡A4xrltKUTK3jQhnNxSMa7kqrdMU3 / == X5tAkld5V3zG82qA

RESTRICTED MOVEMENT MOSAIC SET FOR

INTEREST CODING REGION

IMPI

MEXICAN INSTITUTE

DE 1.A T'.ÍOPiEL'AC ind: ι ·;? ;;, A,

BACKGROUND

Engineers use compression (also called source encoding) to reduce the bit rate of digital video. Compression lowers the cost of storing and transmitting video information by converting the information into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original information in the compressed form. A "codec" is an encoder / decoder system.

Over the past two decades, various video codec standards have been adopted, including ITU-T H.261, H.262 (MPEG-2 or ISO / IEC 13818-2), H.263, and H.264 standards. (AVC or ISO / IEC 14496-10), the MPEG-1 (ISO / IEC 11172-2) and MPEG-4 Visual (ISO / IEC 14496-2) standards, and the SMPTE 421M standard. More recently, the HEVC standard (ITU-T H.265 or ISO / IEC 23008-2) has been approved. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular encoding and decoding features are used. In many cases, a video codec standard also provides details on decoding operations that

you must perform a decoder to adapt to decoding results. In addition to codec standards, several proprietary codec formats define other options for the syntax of an encoded video bitstream and corresponding decoding operations.

In the January 2013 version of the HEVC standard (see Bross et al., “High Efficiency Video Coding (HEVC) Text Specification Draft 8, JCTVC-L1003_v34, January 2013), an image can be divided into multiple tiles, which are rectangular regions. Where the flag_ mosaic syntax element equals 1, mosaic images are built. Tiles define horizontal and vertical boundaries within an image and are made within the image according to tile columns and tile rows. When mosaics are used, the HEVC bitstream syntax and HEVC decoding processes are structured to eliminate intra-image prediction dependencies across mosaic boundaries within the same image, and to eliminate entropy decoding dependencies through of mosaic boundaries within the same image. The inter-image prediction dependencies are not restricted, however, with respect to mosaic limits according to the January 2013 version of the HEVC standard.

BRIEF DESCRIPTION OF THE INVENTION

IMPI

MEXICAN INSTITUTE OE THE PROPERTY

INDUSTRIAL

In summary, the detailed description introduces innovations in signaling and use of control data for a restricted motion mosaic set (“MCTS”). For example, the innovations support signaling and the use of control data to indicate that inter-image prediction processes within one or more specified mosaic sets are restricted to reference-only regions within each corresponding mosaic set in other images. This can facilitate region of interest encoding, decoding and presentation, transcoding to limit encoded data to a selected mosaic set, missing resistance, and encoding and / or decoding parallelism.

In accordance with one aspect of the innovations described herein, a video encoder or other tool encodes multiple images to produce encoded data, where each of the images is divided into multiple tiles. For example, the tool decides whether inter-image prediction dependencies across specific boundaries are to be constrained for a mosaic set and, if so, restricts motion estimation during encoding so that inter-image prediction dependencies through the specific limits are avoided for the mosaic set. In some implementations, the specific limits are the limits of the same image-to-image mosaic set, but

M £ X1C / INSTITUTE. ί-: ·-DE LA PXOf *; ¿? AL. «

INDUG'rr.UL r; ^, · In other implementations the specific tiles may be boundaries of another tile set or another tile region used for inter-image prediction. The tool sends the encoded data along with control data indicating that inter-image prediction dependencies across specific boundaries are constrained on a mosaic set of one or more mosaics from the multiple mosaics. Restricting inter-image prison dependencies for multiple sets of tiles can facilitate the use of parallel processing in encoding and can also help to provide region of interest decoding functionality of gradual decoder update functionality.

In accordance with another aspect of the innovations described herein, a video decoder or other tool receives encoded data for multiple images, where each of the multiple images is divided into multiple tiles. The tool also receives control data indicating that inter-image prediction dependencies across specific limits are constrained for a given mosaic set of one or more mosaics from multiple mosaics. The tool then processes the encoded data, for example, by decoding the given mosaic set as a region of interest within the images without decoding portions of the images outside the given mosaic set. Or, as part of the encoded data processing, the tool transcodes the encoded data, removing data

encoded for portions of the images outside the given mosaic set, and organizes encoded data for the given mosaic set as a new bit stream. Or, as part of the processing of the encoded data, upon loss detection of at least some of the other encodings other than the given mosaic set, the decoder decodes the given mosaic set as loss recovery part. Also, restricting inter-image prediction dependencies for multiple tile sets can facilitate the use of parallel processing in decoding.

In illustrative implementations, a given tile set in the control data is parameterized as one or more tile rectangles that include the one or more tiles in the tile set. For example, for a given mosaic rectangle in the mosaic set, the control data includes syntax elements that identify two corners of the mosaic rectangle (such as an upper left corner of the mosaic rectangle and lower right corner of the mosaic rectangle). mosaics). Control data can also include an identifier for the mosaic set, a count parameter that indicates a count of mosaic rectangles in the mosaic set, and, for each of the mosaic rectangles in the mosaic set, syntax elements. indicating location of the mosaic rectangle.

In illustrative implementations, the multiple images are divided identically to produce tiles within each of

MEXICAN INSTITUTE OF THE ΡΛΟΝΙΠ.ίΑΏ

INDUSTXíaI

the images. Typically, a given mosaic set is identical for each of the images. In some cases, however, tile sets may differ by at least some of the images. Alternatively, different images can be tiled into different shapes.

In illustrative implementations, the control data is a supplemental enhanced information ("SEI") message indicating that inter-image prediction dependencies across mosaic set boundaries are constrained for one mosaic set. One SEI message directs inter-image prediction dependencies for an individual mosaic set, and different SEI messages can direct different mosaic sets. Alternatively, a single SEI message directs inter-image prediction dependencies for each of multiple mosaic sets. Or, instead of SEI messages, the control data may be an indicator whose value indicates whether inter-image prediction dependencies across mosaic set boundaries are constrained for a mosaic set. Or, the control data may take some other form.

Signaling and use of MCTS control data can be implemented as part of a method, as part of a computing device adapted to perform the method, or as part of a tangible computer-readable medium that stores computer-executable instructions to cause a device to of computation perform the method.

The foregoing and other objects, features, and advantages of the invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.

INSTITUTO MEZíCZ -. '.' L DE LA FRGPIr.íW

INDUSTRY!

Í> 'ZS'

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a diagram of an illustrative computing system in which some described modalities can be implemented.

Figures 2a and 2b are illustrative network environment diagrams in which some described modalities can be implemented.

Figure 3 is a diagram of an illustrative encoder system in conjunction with which some described modalities can be implemented.

Figure 4 is a diagram of an illustrative decoder system in conjunction with which some described modalities can be implemented.

Figures 5a and 5b are diagrams illustrating an illustrative video encoder 20 in conjunction with which some described modalities can be implemented.

Figure 6 is a diagram illustrating an illustrative video decoder in conjunction with which some described modalities can be implemented.

Figures 7a-7g are diagrams illustrating examples of

MEXICAN INSTITUTE OF PROnt'tWO

INDUSTRIAL tiles divided into mosaics, which can be arranged-ee-e-onj u-ntes of mosaic.

Figure 8 is a diagram illustrating motion estimation and motion prediction compensated by a mosaic set prediction unit without movement constraints at mosaic set boundaries.

Figure 9 is a diagram illustrating motion estimation and motion compensated prediction for a mosaic set prediction unit with movement constraints at mosaic set boundaries.

Figure 10 is a diagram illustrating an example of parallel encoding and parallel decoding for images with

MCTS.

Figure 11 is a diagram illustrating an example of a region of interest decoding for images with an MCTS.

Figure 12 is a diagram illustrating an example of transcoding for Images with an MCTS.

Figure 13 is a diagram illustrating an example of gradual decoder update functionality for MCTS images.

Figures 14a-14c are tables illustrating SEI message syntax for an MCTS in illustrative implementations.

Figure 15 is a flow chart illustrating a generalized technique for signaling MCTS control data.

Figure 16 is a flow chart illustrating a technique.

INSTITUTO mexjCaA ·;

D £ la ί · κ.Γ · Ρ «ιη ... .

INDUSTAI.M. > * · Λ * - · »j £« r

i? «F · <, Τλ «9 illustrative for selective use coding of M ^ 3TS7 ——

Figure 17 is a flow chart illustrating a generalized technique for processing flagged encoded data in conjunction with MCTS control data.

DETAILED DESCRIPTION

The detailed description presents aspects for signaling and / or using control data for a restricted motion mosaic set ("MCTS"). In particular, the detailed description presents Innovations for signaling and use of control data that indicates that inter-image prediction processes within a specified mosaic set are constrained to refer only to center rulers of the same mosaic set in other images. . In several examples, syntax and semantics of a supplemental enhancement information message (“SEI”) for MCTS control data are presented. MCTS control data can facilitate complexity scalability for decoding and display of the region of interest, transcoding to limit encoded data to a selected mosaic set, resistance to loss, encoding and / or parallel decoding.

Although operations described herein are in places described as being performed by an encoder (eg, video encoder), decoder (eg, video decoder), or transcoding tool (eg, transcoder).

INSTITUTO A4 £ X} CaNG DE LA F * ROF'ÍclOAO «NOUSTRIAL

video), in many cases the operations can be performed alternatively by another type of media processing tool (for example, video processor for up-sampling, video processing for down-sampling).

Some of the innovations described here are illustrated with reference to specific syntax elements and operations for the HEVC standard. For example, reference is made to the draft version JCTVC-L1003 of the HEVC standard- “High Efficiency Video Coding (HEVC) Text Specification Draft 8”, JCTVC-L1003_v34, January 2013, the description of which is incorporated herein by reference. The images described here can also be implemented for other standards or formats.

More generally, various alternatives to the examples described herein are possible. For example, some of the methods described here can be altered by changing the order of the method acts described, dividing, repeating, or omitting certain method acts, etc. The various aspects of the described technology can be used in combination or separately. Different modalities use one or more of the innovations described. Some of the innovations described here address one or more of the problems noted in the background. Typically, a given technique / tool does not solve all of those problems.

I. Illustrative Computer Systems.

Figure 1 illustrates a generalized example of a

MEXICAN INSTITUTE OF THE PROf'lt'OAD iNoumu adequate computation (100) where several of the innovations described can be implemented. The computer system (100) is not intended to suggest any limitations regarding the scope of use or functionality, since the innovations can be implemented in general-purpose or special-purpose computer systems.

Referring to Figure 1, the computing system (100) includes one or more processing units (110, 115) and memory (120, 125). The processing units (110, 115) execute instructions executable by computer. A processing unit may be a general-purpose processing unit ("CPU"), processor on an application-specific integrated circuit ("ASIC"), or any other type of processor. In a multiple processing system, multiple processing units execute computer executable instructions to increase processing power. For example, Figure 1 shows a central processing unit (110) as well as a graphics processing unit or co-processing unit (115). Tangible memory (120, 125) can be volatile memory (eg registers, cache memory, RAM), non-volatile memory (eg

ROM, EEPROM, flash memory, etc.), or some combination of the two, accessible by the processing unit (s). Memory (120, 125) stores software (180) that implements one or more innovations for signaling and / or use of MCTS control data, in the form of computer executable instructions suitable for execution by the processing unit (s).

IMPI

MEXICAN INSTITUTE ΡΪ UA TROPlEflAO

INDUSTRIAL 'n

Can a computer system have an adTcToiTaTei feature? For example, the computing system (100) includes storage (140), one or more input devices (150), one or more output devices (160), and one or more communication connections (170). An interconnection mechanism (not shown) such as a common conductor, controller, or network interconnects the computer system components (100). Typically, the operating system software (not shown) provides an operating environment for other software that runs on the computer system (100), and coordinates activities of the computer system components (100).

Tangible storage (140) can be removable or non-removable, and includes magnetic discs, magnetic tapes or cassettes, CD-ROMs, DVDs, or any other means that can be used to store information and that can be accessed within the computer system (100 ). The storage (140) stores instructions for the software (180) that implements one or more innovations for signaling and / or use of MCTS control data.

The input device (s) 150 may be a tactile input device such as a keyboard, mouse, pen, or tracker, a voice input device, a scanning device, or other device that provides input to the input system. computation (100). For video, the input device (s) 150 may be a camera, video card, television tuner card, a similar device that accepts video input in analog or digital form, or a

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

CD-ROM or CD-RW that reads video samples within the computer system (100). The output device (s) (160) may be a display, printer, speaker, CD writer, or other device that provides output from the computer system (100).

The communication connection (s) 170 allows communication over one communication medium to another computing entity. The communication medium transmits such information with instructions executable by computer, audio and video input or output, or other data in a modulated data signal, modulated data signal is a signal that has one or more of its characteristics established or changed in such a way. way to encode information in the signal. By way of example, and not limitation, the communication media may use an electrical, optical, RF, or other carrier.

Innovations can be described in the general context of computer readable media, computer readable media is any available tangible media that can be accessed without a computing environment. By way of example, and not limitation, with the computing system 100, computer readable media include memory 120, 125, storage 140, and combinations of any of the above.

Innovations can be written in the general context of computer executable instructions, such as those included in program modules, that are executed in a computer system on a real or virtual target processor. Generally, program modules include routines, programs, libraries,

IMPI

MEXICAN INSTITUTE OF LA MONEDAD

INDUSTRIAL

objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules can be combined or divided among the program modules as desired in various modes. Computer-executable instructions for program modules can be executed within a local or distributed computer system.

The terms "system" and "device" are usually interchangeable here. Unless the context clearly indicates otherwise, no term implies any limitation on a type of computing system or computing device. In general, a computer system or computing device may be local or distributed, and may include any combination of special-purpose hardware and / or general-purpose hardware with software that implements the functionality described herein.

The described methods can also be implemented using specialized computing hardware configured to perform any of the described methods, for example, the described methods can be implemented by an integrated circuit (for example, an ASIC (such as an ASIC digital signal processing unit). ("DSP"), a graphics processing unit (GPU "), or a programmable logic device (" PLD "), such as a Field Programmable Gate Array ("FPGA")) specifically designed and configured to implement any of the methods described.

IMPI

MEXICAN INSTITUTE OF Industrial PROPERTY

For presentation search, the detailed description uses terms like "determine" and "use" to describe computer operations on a computer system. These terms are high-level abstractions for operations performed by a computer, and should not be confused with acts performed by a human being. The actual computer operations for these terms vary depending on the implementation.

II. Illustrative Network Environments.

Figures 2a and 2b show illustrative network environments (201, 202) including video encoders (220) and video decoders (270). Encoders 220 and decoders 270 are connected through a network 250 using an appropriate communication protocol. Network 250 may include the Internet or another computer network.

In the network environment (201) shown in Figure 2A, each real-time communication tool ("RTC") (210) includes both an encoder (220) and a decoder (270) for communication bidirectional. A given encoder (220) can produce condescending output with the HEVC standard, SMPTE 421M standard, ISO-IEC 14496-10 standard (also known as H.264 or AVC), another standard, or a proprietary format, with a corresponding decoder (270) which accepts encoded data from the encoder (220). The communication

INSTITUTO MMCMO DILAROPIEOAD

Two-way INDUSTRIAt may be part of a video reference, video phone call, or other two-party communication scenario. Although the network environment (201) in Figure 2a includes two real-time communication tools (210), the network environment (201) in turn can include three or more real-time communication tools (210) that participate in multi-part communication.

The real-time communication tool (210) handles the encoding by an encoder (220). Figure 3 shows an illustrative encoder system (300) that can be included in the real-time communication tool (210). Alternatively, the real-time communication tool 210 uses another encoder system. A real-time communication tool (210) also handles decoding using a decoder (270). The

Figure 4 shows an illustrative decoder system (400), which can be included in the real-time communication tool (210). Alternatively, the real-time communication tool 210 uses another decoder system.

In the network environment (202) shown in Figure 2b, an encoding tool (212) includes an encoder (220) that encodes video for supply to multiple playback tools (214), including decoders (270). One-way communication can be provided for a video surveillance system, webcam verification system, remote desktop conference presentation or other setting in

which is video encoded and sent from one location to one or more of other locations. Although the network environment (202) in Figure 2b includes two playback tools (214), the network environment (202) may include more or less playback tools (214). In general, a playback tool (214) communicates with the encoding tool (212) to determine a video stream for the playback tool (214) to receive. The playback tool 214 receives the stream, caches the encoded data received for an appropriate period, and begins decoding and playback.

Figure 3 shows an illustrative encoding system (300) that can be included in the encoding tool (212). Alternatively, the coding tool 212 uses another coding system. The encoding tool (212) may also include server-side controller logic to handle connections to one or more replay tools (214). Figure 4 shows an illustrative decoder system (400), which can be included in the playback tool (214). Alternatively, the playback tool 214 uses another decoder system. A replay tool (214) may also include client-side controller logic to handle connections to the encoding tool (212).

ili. Illustrative Coding Systems.

Figure 3 is a block diagram of a system

INST'TUTó ME7i; C ... 'O OF THE INDUSTRIAL PROPERTY illustrative decoder (300) together with which some described modalities can be implemented. The encoder system 300 can be a general-purpose encoding tool capable of operating in any of multiple encoding modes such as a low-latency encoding mode for real-time communication, transcoding mode, and regular encoding mode for playback of media from a stream file, or it may be a special-purpose encoding tool adapted for one such encoding mode. Encoder system 300 can be implemented as an operating system module, as part of an application library, or as a standalone application. In general, the encoder system (300) receives a sequence of source video frames (311) from a video source (310) and produces encoded data as output to a channel (390). The encoded data output to the channel may include MCTS control data (eg, SEI messages for MCTS).

The video source 310 may be a camera, tuner card, storage media, or other digital video source. The video source 310 produces a sequence of video frames at a frame rate of, for example, 30 frames per second. As used herein, the term "frame" generally refers to source, encoded, or reconstructed image data. For progressive video, a frame is a progressive video frame. For interlaced video, in illustrative modalities, a video frame

IMPI

MEXICAN INSTITUTE OF LA prüfieda .: industrial

before interlaced video is de-interlaced

Alternatively, two complementary fields are coded as an interlaced video frame or separate fields. In addition to indicating a progressive video frame, the term "frame" or "image" may indicate an individual unpaired video field, a complementary pair of video fields, a video object plane representing a video object on a given time, or region of interest in a larger image. The video object plane or region can be part of a larger image that includes multiple objects or regions in a scene.

An arrival source box (311) is stored in a source box temporary memory storage area (320) that includes multiple box cache storage areas (321, 322, ..., 32n). A box cache (321, 322, etc.) holds a font box in the font box storage area (320). After one or more of the source frames (311) have been stored in the frame caches (321, 322, etc.), a frame selector (330) periodically selects an individual source frame from the memory storage area. source box (320). The order in which frames are selected by frame selector (330) for input to encoder (340) may differ from the order in which frames are produced by video source (310), eg, encoding.

Interlaced box can be forward in order, to facilitate temporarily backward prediction. Before the encoder (340), the system

IMPI

MEXICAN INSTITUTE OE LA HROfÍEÜAD

INDUSTRIAL

Encoder (300) may include a pre-processor (not shown) that performs pre-processing (eg, filtering) of the selected frame (331) prior to encoding. Pre-processing may also include converting color space into primary and secondary components for encoding.

Encoder 340 encodes selected frame 331 to produce encoded frame 341 and also produces memory management control ("MMCO") operation signals (342) or group information Reference Image (“RPS”). If the current frame is not the same frame that has been encoded, then the embodiment is the encoding process, the encoder (340) may use one or more previously encoded / decoded frames (369) that have been stored in a storage area decoded frame temporary memory (360). Such stored decoded frames (369) are used as reference frames for inter-frame prediction of the content of the current source frame (331). Generally, encoder 340 includes multiple encoding modules that perform encoding tasks such as tiling, motion estimation and compensation, frequency transformations, quantization, and entropy encoding. The exact operations performed by encoder 340 may vary depending on the compression format. The output encoded data format can be HEVC format, Windows media video format, VC-1 format, MPEG-x format (for

I ΚΙ ΡI Mexican institute

EU LA ¡KCi'IEUAD INDUSTRIAL example, MPEG-1, MPEG-2, or MPEG-4), H.26x format (for example, H.261, H.262, H.263, H.264), or other Format.

Encoder 340 can divide a frame into multiple tiles of the same size or different sizes. For example, encoder (340) divides the box along mosaic rows and mosaic columns that, box boundaries, define horizontal and vertical boundaries of tiles within the box, where each tile is a rectangular region. Encoder 340 can then group the tiles into one or more tile sets, where a tile set is a group of one or more of the tiles. The mosaic (s) in a mosaic set can be contiguous in a box. Or, a tile set can include tiles that are not contiguous in the box. Typically, the mosaic set (s) defined for one frame are the same mosaic set (s) as defined for other frames in a series of frames (for example, for a group of frames, for an entire sequence).

Encoder 340 represents a predicted frame intercoded in terms of reference frame prediction. A motion estimator estimates the motion of blocks or other sets of samples from a source frame (331) with respect to one or more reference frames (369). When multiple reference frames are used, the multiple reference frames can be from different time directions or the same time direction. As part of the motion estimation, the encoder

MtXICAM INSTITUTE>

OBLA PR 'he; Ai

INHUSTRU !.

(340) can constrain block motion vectors within a mosaic set of a current frame so that the compensated motion prediction reference regions fall within the same mosaic set in the reference frame (s). A compensated motion prediction reference region is a region of samples in the reference frame (s) that is used to generate compensated motion prediction values for a prediction unit (eg, blocks) of samples from a current frame . Motion compensated prediction may involve processes such as fractional position interpolation that applies filtration to samples of somewhat larger regions in the reference frame (s) compared to the size of the prediction unit. In other words, the compensated motion prediction reference region used to compute compensated motion prediction values for a prediction unit of a current frame may be larger than the prediction unit, due to the use of interpolation filters whose Support extends beyond the edges of the nominal prediction unit size. By using such

MCTS can provide functionality for region of interest decoding for mosaic set, transcoding, and parallel decoding. The motion estimator outputs motion information such as motion vector information, which is encoded by entropy. A motion compensator applies motion vectors to reference frames

(369) for determining compensated motion prediction values.

The encoder determines the differences (if any) between block offset motion prediction values and corresponding original values. These prediction residuals are further encoded using frequency transformation, quantization, and entropy coding. For example, encoder 340 sets values for quantization parameter ("QP") for an image, mosaic, clip, and / or other video portion, and quantizes transform coefficients, accordingly. Similarly, for intra-prediction, encoder 340 can determine intra-prediction values for a block, determine prediction residuals, and encode prediction residuals (with frequency transformation, quantization, and entropy coding). In particular, the entropy encoder of encoder 340 comprises quantized transform coefficient values as well as some lateral information (eg, motion vector information, QP values, mode decisions, parameter choices). Typical entropy coding techniques include Exp-Golomb coding, arithmetic coding, differential coding, Hoffman coding, operating length coding, variable length to variable length coding ("V2V"), length to variable fixed length coding (" V2F ”), LZ encoding, dictionary encoding, interval division entropy encoding of

probability (“PIPE”), and combinations of the above. The entropy encoder can use different encoding techniques for different kinds of information, and can choose from multiple code tables within a particular encoding technique.

The encoded frames (341) and MMCO / RPS information (342) are processed by a decoding process emulator (350). The decoding process emulator (350) implements some of the functionality of a decoder, eg, decoding tasks to reconstruct reference frames that are used by the encoder (340) in motion estimation and compensation. The decoding process emulator (350) uses the MMCO / RPS information (342) to determine if a given encoded frame (341) needs to be reconstructed and stored to be used as a reference frame in interframe prediction of subsequent frames to go. to encode. If the MMCO / RPS information (342) indicates that an encoded frame (341) needs to be stored, the decoding process emulator (350) models the decoding process that would be driven by a decoder that receives the encoded frame (341) and produces a corresponding decoded frame (351). In doing this, if the encoder (340) has used the decoded frame (s) (369) that has been stored in the decoded frame storage area (360), the decoding process emulator (350) also uses the frame decoded (s) (369) of the

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL storage (360) as part of the decoding process.

The decoded frame temporary memory storage area (360) includes multiple frame buffer storage areas (361, 362, ..., 36n). The decoding process emulator (350) uses the MMCO / RPS information (342) to handle the contents of the storage area (360) in order to identify any frame cache (361, 362, etc.) with frames that They are no longer required by encoder 340 to be used as reference frames. After modeling the decoding process, the decoding process emulator (350) stores a newly encoded frame (351) in a frame cache (361, 362, etc.) that has been identified in this way.

The encoded frames (341) and MMCO / RPS information 342) are cached in a temporary encoded data area (370). The encoded data that is aggregated in the encoded data area 370 contains, as part of the syntax of an elementary encoded video bitstream, encoded data for one or more images. The encoded data that is aggregated in the encoded data area (370) may also include media metadata that relates to the encoded video data (for example, as one or more parameters in one or more SEI messages or useful information of video (“VUI”) of messages). Such media metadata may include syntax elements indicating MCTS control data (for example, SEI messages for

IMPI

industrial

MCTS).

The aggregated data (371) from the temporary encoded data area (370) is processed by a channel encoder (380). The channel encoder 380 can pack the aggregated data for transmission as a media stream (eg encoding to a media container format such as ISO / IEC

14496-12), in which case channel encoder 380 may add syntax elements as part of the media stream stream syntax. Such syntax can include syntax elements indicating MCTS control data. Or, the channel encoder (380) can organize the aggregated data for storage as a file (for example, according to a media container format such as ISO / IEC 14496-12), in which case the channel encoder ( 380) You can add syntax elements as part of the media archive file syntax. Such syntax can include syntax elements indicating MCTS control data. Or, more generally, the channel encoder (380) may implement one or more media system multiplexing protocols or transport protocols, in which case the channel encoder (380) may add syntax elements as part of the syntax of the protocol (s). Again, tai syntax can include syntax elements indicating MCTS control data. Channel encoder 380 provides output to a channel 390, which represents storage, a communications connection, or another channel for output.

MEXICAN INSTITUTE OF LA FROFIEPAi:

IMPUSTAIAl

IV. Illustrative Decoder Systems. - —---------------------------------------- Figure 4 is a block diagram of an illustrative decoder system (400) in conjunction with which some described modalities can be implemented. The decoder system 400 can be a general-purpose encoding tool capable of operating in any of multiple decoding modes such as low latency decoding mode for real-time communication and regular decoding mode for media production from a file. stream, or it may be a special-purpose decoding tool adapted for one such decoding mode. The decoder system 400 can be implemented as an operating system module, as part of an application library, or as a standalone application. In general, the decoder system (400) receives encoded data from a channel (410) and produces reconstructed frames as output for an output destination (490). Encoded data can include syntax elements that indicate control data

MCTS.

Decoder system 400 includes a channel 410, which may represent storage, a communications connection, or another channel for data encoded as input. Channel 410 produces encoded data that has been encoded per channel. A channel decoder 420 can process the encoded data. For example, the channel decoder 420 unpacks data that has been aggregated for transmission with a

media stream (for example, according to a media container format such as ISO / IEC 14496-12), in which case the channel decoder (420) can parse aggregated syntax elements as part of the stream syntax media transmission. Such syntax can include syntax elements indicating MCTS control data. Or, the channel decoder (420) separates encrypted lock data that has been aggregated for storage as a file (for example, according to a media container format such as ISO / IEC 14496-12, in which case the decoder Channel 420 can parse aggregated syntax elements as part of the media archive file syntax, such syntax may include syntax elements indicating MCTS control data. Or, more generally, the channel decoder (420) can implement one or more media system demultiplexing protocols or transport protocols, in which case the channel decoder (420) can parse aggregated syntax elements as part of the syntax. of the protocol (s). Again, such syntax can include syntax elements indicating MCTS control data.

The encrypted data 421 that is sent from the channel decoder 420 is stored in a temporary encoded data area 430 until a sufficient amount of such data has been received. The encoded data (421) includes encoded frames (431) and MMCO / RPS information (432). The encoded data (421) in the encoded data area (430) contains,

IMPI

MEXICAN INSTITUTE OF FROMEDAP

INDUIRHAl

as part of the syntax of an elementary encoded video bitstream, encoded data for one or more images. The encoded data (421) in the encoded data area (430) can also include media metadata that relates to the encoded video data (eg, as one or more parameters in one or more SEI messages or VUI messages). Such media metadata may include syntax elements indicating MCTS control data (eg as part of SEI messages).

In general, the encoded data area (430) temporarily stores encoded data (421) until such encoded data (421) is used by the decoder (450). At that point, encoded data for an encoded frame (431) and MMCO / RPS information (432) are transferred from the encoded data area (430) to the decoder (450). As decoding continues, new encoded data is added to the encoded data area (430) and the oldest encoded data remaining in the encoded data area (430) is transferred to the decoder (450).

The decoder (450) periodically encodes an encoded frame (431) to produce a corresponding decoded frame (451). As appropriate, when performing its decoding process, the decoder 450 may use one or more previously decoded frames 469 as reference frames for interframe prediction. The decoder (450) reads such previously decoded frames (469) from an area of

ΙΜΡΪ

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

decoded frame temporary memory storage (460). Generally, the decoder 450 includes multiple decoding modules that perform decoding tasks such as entropy decoding, inverse quantization, inverse frequency transformations, motion compensation, and tile blending. The exact operations performed by the decoder (450) may vary depending on the compression format.

For example, the decoder (450) receives encoded data for a compressed frame or frame sequence and produces output including decoded frame (451). At decoder 450, a cache receives encoded data for a compressed frame and, at an appropriate time, makes the received encoded data available to an entropy decoder. The entropy decoder decodes entropy-encoded quantized data as well as information from the entropy-encoded lateral information, typically applying the intropy-encoding inverse performed on the encoder. A motion compensator applies motion information to one or more reference frames to form sub-block and / or block (generally, block) compensated motion predictions of the frame being reconstructed. An intra-prediction module can spatially predict sample values from a current block of neighboring, previously reconstructed sample values. Decoder 450 also rebuilds prediction residuals. An inverse quantizer quantizes the inverse data decoded by

IMPI

MEXICAN INSTITUTE OF PROHEDAO

INDUSTRIAL

entropy. For example, decoder 450 sets values for QP for an image, mosaic, clip, and / or other video portion based on syntax elements in the bitstream, and quantizes the inverse transform coefficients accordingly.

An inverse frequency transformer converts the quantized frequency domain data into spatial domain information. For a predicted frame, decoder 450 combines reconstructed prediction residuals with motion compensated predictions to form a reconstructed frame. Decoder 450 can similarly combine prediction residuals with intra-prediction spatial predictions. A motion compensation circuit in the video decoder (450) includes an adaptive unlock filter to smooth discontinuities across rows and / or columns of block boundary in the decoded frame (451).

Decoder 450 can use MCTS control data in various ways, depending on implementation. For example, decoder 450 can use MCTS control data when it decides to decode different sets of tiles in parallel. Or, the decoder 450 can use MCTS control data when it decides to decode only a selected mosaic set for presentation as a region of interest, without decoding portions of the frames outside the mosaic set.

The decoded frame temporary memory storage area (460) includes multiple storage areas of

MEXICAN INSTITUTE OF THE FMEIESAC

INOUSTRIAl.

frame damper (461, 462, ..., 46n). The decoded frame storage area 460 is an example of a decoded image cache. The decoder (450) uses the MMCO / RPS information (432) to identify a frame cache (461, 462, etc.) where it can store a decoded frame (451). Decoder 450 stores the decoded frame 451 in that frame cache.

An output sequencer (480) uses the MMCO / RPS information (432) to identify when the next frame to be produced in output order is available in the decoded frame storage area (460). When the next frame (481) to be produced in output order is available in the decoded frame storage area (460), it is read by the output sequencer (480) and output to the output destination (490) (by example, presentation). In general, the order in which frames are sent from the decoded frame storage area (460) by the output sequencer (480) may differ from the order in which the frames are decoded by the decoder (450).

V. Illustrative Video Encoders.

Figures 5a and 5b are a block diagram of a generalized video encoder (500) in conjunction with which some described modalities can be implemented. Encoder 500 receives a sequence of video images including a

current image as an input video signal (505) and produces encoded data in an encoded video bit stream (595) as output.

Encoder (500) is block based and uses an implementation dependent block format. The blocks can also be sub-divided into different stages, for example, in the frequency transformation and entropy coding stages. For example, an image can be divided into 64 x 64 blocks, 32 x 32 blocks, or 16 x 16 blocks, which in turn can be divided into smaller blocks and sub-blocks of pixel values for encoding and decoding.

Encoder 500 comprises images using intra-image encoding and / or inter-image encoding. Many of the encoder components (500) can be used for intra-image coding and inter-image coding. The exact operations performed by those components may vary depending on the type of information that is compressed.

A tile module (510) optionally divides an image into multiple tiles of the same size or different sizes. For example, the mosaic module (510) divides the image along mosaic rows and mosaic columns that, with Image boundaries, define horizontal and vertical boundaries of tiles within the image, where each tile is a region rectangular. The mosaic module 510 can then group the mosaics into one or more mosaic sets, where one mosaic set is a

IMPI

MEXICAN INSTITUTE OE THE PROPERTY

INDUSTRIAL

group of one or more of the tiles. The mosalco (s) in a mosaic ΈδΈμπτΓο may be contiguous in an Image. Or, a tile set can include tiles that are not contiguous in the image. Typically, the mosaic set (s) defined for an Image are the same mosaic set (s) as defined for other images in a series of Images (for example, for a group of Images, for an entire sequence).

The general encoding control (520) receives Images for the input video signal (505) as well as feedback (not shown) from various encoder modules (500). In general, the general encoding control (520) provides control signals (not shown) to other modules (such as the mosaic module (510), transformer / scaler / quantizer (530), scaler / reverse transformer (535), Intra-Imaging Estimator (540), Motion Estimator (550) and In / Out Switch) to set and change encoding parameters during encoding. The general encoding control (520) can also evaluate Intermediate results during encoding, for example, performing velocity distortion analysis. The general encoding control (520) produces general control data (522) that indicates decisions made during encoding, so that a corresponding decoder can make conscious decisions. General control data (522) is provided to the entropy encoder / header formatter (590). The general encoding control (520) can decide whether to use MCTS during

IMPI

MEXICAN INSTITUTE OF THE PROFIFCMC

INDUSTRIAL

coding. '

If the current image is predicted using inter-image prediction, a motion estimator (520) estimates movement of blocks, sub-blocks, or other sets of pixel values of the current image of the input video signal (505) with respect to a or more reference images. The decoded image cache (570) caches one or more reconstructed pre-encoded images for use as reference images. When multiple reference images are used, the multiple reference images can be from different time directions or the same time direction. For an MCTS of a current image, as part of motion estimation, motion estimator 550 can constrain motion vectors to blocks within the mosaic set such that regions referenced by offset motion prediction processes fall within of the same mosaic set in the reference image (s).

The motion estimator (550) produces as lateral information motion data (552) such as motion vector data and reference image selection data. The motion data (552) is provided to the entropy encoder / header formatter (590) as well as the motion compensator (555).

The motion compensator (555) applies motion vectors to the reconstructed reference image (s) of the

IMPI

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

decoded image damper (5 7 0 ')' .h I ó ó rf) but motion motions (555) produces predictions of compensated motion for the current image.

In a separate path of the encoder (500), an intra-image estimator (540) determines how to perform intra-image prediction for blocks, sub-blocks, or other sets of pixel values of a current Image of the input video signal ( 505). The current image can be fully or partially encoded using intra-image encoding. Using values from a reconstruction (538) of the current image, the intra-image estimator (540) determines how to spatially predict pixel values of a current block, sub-block, etc., of the current image from neighboring pixel values, previously reconstructed from the current image. The intraprediction estimator (540) produces intraprediction data (542) as lateral information as predictive mode data. Intra-prediction data (542) is provided to the entropy encoder / header formatter (590) as well as the intra-image predictor (545). In accordance with prediction mode data, the intra-image predictor 545 spatially predicts pixel values of a current block or sub-block of the current image from neighboring pixel values, previously reconstructed from the current image.

The intra-switch selects values from a motion compensated prediction or intra-image prediction for use as the prediction (558) for a given block, sub-block, or other set

IMPI

MEXICAN INSTITUTE OF LA PROPÍLÜa ·,>

? NDUSTtf.V '.;

pixel values. The difference (if any) between a subblock, block, etc. of the prediction (558) and corresponding part of the original current image of the input video signal (505) is the remainder (518) for the sub-block, block. During reconstruction of the current image, the reconstructed residuals are combined with the prediction (558) to produce a reconstruction (538) of the original content of the video signal (505). In lossy compression, however, some information is still lost from the video signal (505).

In the transformer / scaler / quantizer (530), a frequency transformer converts video information from the spatial domain to the frequency domain (ie, spectral, transformation). For block-based video encoding, the frequency transformer applies a discrete cosine transformer, an integer approximation thereof, or another type of forward block transformation to prediction residual data blocks or sub-blocks (or data pixel value if the prediction (558) is null), producing blocks / sub-blocks of frequency transformation coefficients. The scaler / quantizer then scales and quantifies the transform coefficients. For example, the quantizer applies non-uniform quantization, scaling to frequency domain data with a step size that varies on a frame-by-frame basis, mosaic-by-mosaic base, fragment-by-fragment base, block-by-block base or another base. The coefficient data of

MEXICAN INSTITUTE OF PSOflEDAD

INDUSTP.IaL quantized transformation (532) are provided to the entropy encoder / header formatter) 590.

In the inverse scaler / transformer (535), an inverse scaler / quantizer performs inverse scaling and inverse quantization on the quantized transformation coefficients. An inverse frequency transformer performs an inverse frequency transformation, producing reconstructed prediction remainder blocks / subblocks or pixel values. Encoder (500) combines reconstructed residuals with prediction values (558) (eg, motion-compensated prediction values, intra-image prediction values) to form the reconstruction (538).

For intra-image prediction, the reconstruction values (538) can be fed back to the intra-image estimator (540) and intra-image predictor (545). For inter-image prediction, the reconstruction values (538) can be further filtered. A filter control (560) determines how to perform the unlock filter and sample adaptive shift filter ("SAO") on reconstruction values (538), for a given image of the video signal (505). The filter control (560) produces filter control data (562), which is provided to the entropy encoder / header formatter (590) and merger / filter (s) (565).

In the merger / filter (s) (565), the encoder (500) works content from different tiles in a rebuilt version of the

IMPI

PROI'iF.no Mexican INSTITUTE;

INDU5 IR'a;

image. Encoder 500 selectively performs deblocking filtering and ODS filtering in accordance with filter control data 562, to adaptively smooth discontinuities across boundaries in the tables. Mosaic limits can be selectively filtered to absolute filtration, depending on encoder settings (500). The decoded image cache (570) caches the reconstructed current image for use in subsequent motion compensated prediction.

The entropy encoder / header formatter (590) formats and / or entropy encodes the general control data (522), quantized transform coefficient data (532), intra-prediction data (542), motion data ( 552) and filter control data (562). For example, the Entropy Encoder / Header Formatter (590) uses context-adaptive binary arithmetic encoding for entropy encoding of various syntax elements. The entropy encoder / header formatter (590) provides the encoded video bitstream encoded data (595). The format of the encoded video bitstream (595) can be HEVC format, Windows media video format, VC-1 format, MPEG-x format (for example, MPEG-1, MPEG-2, or MPEG- 4), H.26x format (for example, H.261, H.262, H.263, H.264), or other format.

Depending on the desired implementation and compression type, encoder modules can be added, omitted,

IMPI

MEXICAN INSTITUTE OF THE PKOF1EDM. ·

INDU ^ nn

be divided into multiple modules, combined with other modules, and / or replaced with similar modules. In alternative embodiments, encoders with different modules and / or other module configurations perform one or more of the techniques described.

Specific encoder modes typically utilize a variation or supplemented version of the encoder (500). The relationship shown between modules within the encoder (500) indicates general information flows in the encoder, other relationships are not shown for the sake of simplicity.

SAW. Illustrative Video Decoders.

Figure 6 is a block diagram of a generalized decoder (600) together with which various described modalities can be implemented. The decoder (600) receives encoded data in an encoded video bitstream (605) and produces output that includes images for reconstructed video (695). The encoded video bitstream format (605) can be HEVC format, Windows media video format, VC-1 format, MPEG-x format (for example, MPEG-1, MPEG-2, or MPEG-4 ), Format

H.26x (for example, H.261, H.262, H.263, H.264), or other format.

Decoder 600 is block-based and uses a block format that depends on implementation. For example, an image can be divided into 64 x 64 blocks, 32 x 32 blocks, or 16 x 16 blocks, which in turn can be divided into smaller blocks and sub-blocks of pixel values for decoding.

V.EX¡CANO Ji INSTITUTE

OF THE FRCntCAD ^ Λγ

INDUSTRIAL

Decoder 600 decompresses images using intra-image decoding and / or inter-image decoding. Many of the decoder components (600) are used for intra-picture decoding and inter-picture decoding. The exact operations performed by those components can vary depending on the type of information that is decompressed.

A cache memory receives encoded data in the encoded video bitstream (605) and makes the encoded data received available to the entropy analyzer / decoder (610).

The entropy analyzer / decoder (610) entropy decodes entropy encoded data, which typically applies the intropy encoding inverse performed in encoder (500) (eg, contact adaptive binary arithmetic decoding). With an entropy analysis and decoding result, the entropy analyzer / decoder (610) produces general control data (622), quantized transform coefficient data (632), intra-prediction data (642), motion data (652 ) and filter control data (662).

The general decoding control 620 receives the general control data (622) and provides control signals (not shown) to other modules (such as the inverse transformer / scaler (635), intra-image predictor (645), motion compensator (655) and intra / inter switch) to set and change decoding parameters during decoding. Based on MCTS control data, the general decoding control

Mexican INSTITUTE OF PROPERTY

INDUSTRIAL

(620) can decide how to take advantage of MCTS during decoding (for example, for decoding region of interest for selected mosaic set, for parallel decoding of different mosaic sets).

If the current image is predicted using interimage prediction, a motion compensator (655) receives the motion data (652), such as motion vector data and reference image selection data. The motion compensator (655) applies motion vectors to the reconstructed reference image (s) of the decoded image cache (670). Motion offset (655) produces motion offset predictions for sub-blocks and / or blocks of the current image. The decoded image buffer (670) stores one or more previously reconstructed images for use as reference images.

In a separate path within the decoder (600), the intra-prediction predictor (645) receives the intra-prediction data (642), such as the prediction mode data. By using reconstruction values (638) of the current image, according to prediction mode data, the intra-image predictor (645) spatially predicts pixel values of a current image current block or sub-block of neighboring pixel values , previously reconstructed from the current Image.

The intra / inter switch selects values from a motion compensated prediction or intra-image prediction to use

IMPI

MEXICAN INSTITUTE-> OF INDUSTRIAL PROPERTY

such as the prediction (658) for a given block, sub-block, or other set of pixel values. Decoder 600 combines prediction 658 with reconstructed residuals to produce reconstruction 638 of the video signal content.

Reconstruction to reconstruct the residue, the inverse scaler / transformer (635) receives and processes the quantized transform coefficient data (632). In the inverse scaler / transformer (635), an inverse scaler / quantizer performs inverse scaling and inverse quantization on the quantized transformation coefficients. An inverse frequency transformer performs an inverse frequency transformation, producing reconstructed prediction residual blocks / sub-blocks or pixel values. For example, the inverse frequency transformer applies an inverse block transformation to frequency transformation coefficients, producing pixel value data or prediction residual data. The inverse frequency transformation may be an inverse discrete cosine transformation, an integer approximation thereof, or another type of inverse frequency transformation.

For intra-image prediction, the reconstruction values (638) can be fed back to the intra-image predictor (645). For inter-image prediction, the reconstruction values (638) can be further filtered. If the merger / filter (s) (665), the decoder (600) works to contents of different tiles in a reconstructed version of the image. The decoder (600)

IMPI

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

selectively performs filtering by SAO filtering unlocking in accordance with filter control data (662) and filter adaptation rules, to adaptively smooth discontinuities or across limits in the tables. Mosaic boundaries can be selectively filtered to absolute filtration, depending on decoder configurations (600). The decoded image cache (570) caches the reconstructed current image for use in subsequent motion compensated prediction.

Decoder 600 may also include a post-processing unlock filter. The post-processing unlock filter optionally evens out discontinuities in reconstructed images. Another filtration (such as ring elimination filtration) can also be applied as part of the post-processing filtration.

Depending on the implementation and the type of decompression desired, decoder modules can be added, omitted, divided into multiple modules, combined with other modules, and / or replaced with similar modules. In alternative embodiments, decoders with different modules and / or other module configurations perform one or more of the techniques described. Specific decoder modes typically utilize a variation or supplemented version of the decoder (600). The relationships shown between the modules within the decoder (600) indicate general flows of information in the

IMPI

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

decoder, no other relationships are shown for simplicity search.

Vile. Signaling and Use of Control Data for Sets of

Restricted Motion Mosaics.

This section presents several innovations for signaling and use of control data for a restricted motion mosaic set (“MCTS). In general, MCTS control data indicates that inter-image prediction processes within one or more specified mosaic sets (MCTS) are constrained to refer only to specific regions (eg, regions within each corresponding mosaic set in other images). Innovations can enable a decoder to correctly decode a specified MCTS within the images of an encoded video stream without the need to decode the entire content of each image. By providing an explicit indication of when inter-image prediction dependencies in encoded video are constrained across specific limits (eg, mosaic set limits), MCTS control data can facilitate complexity scalability for decoding and region display of interest, allow simple transcoding, provide improved loss resistance, and allow improved decoder parallelism.

Various examples are provided for MCTS control data

IMPI '

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

as noted in supplemental enhancement information (“SEI”) messages in accordance with a version of the HEVC standard. Such MCTS control data SEI messages can be easily incorporated into the HEVC format.

A. Illustrative Mosaics and Mosaic Sets

In general, tiles are rectangular regions of an image. The tiles are arranged within the image according to tile columns and tile rows. In this way, the tiles define horizontal and vertical limits within the image. The tiles within an image can be uniformly sized, or tiles within an image can vary in size.

In the January 2013 version of the HEVC standard, for example, an image can be split into multiple tiles. The mosaic / enabled / flag syntax element is signaled in an image parameter set (“PPS”). When flag_enabled_ mosaic is 1, an image is tiled, and the number of mosaic columns, number of mosaic rows, or size information are flagged. Size information can indicate a uniform size for all tiles, or a specific size can be indicated by mosaic. See Bross et al., "High Efficiency Video Coding (HEVC) Text Specification Draft 8", JCTVC-L1 003_v34, January 2013.

In general, a mosaic is coded independently of others

tiles for some encoding processes. According to the January 2013 version of the HEVC standard, when mosaics are used, HEVC bitstream syntax and HEVC decoding processes are structured to eliminate (1) intra-image prediction dependencies across mosaic boundaries within the same image, and (2) entropy encoding / decoding dependencies across mosaic boundaries within the same image. Circuit filtering is selectively disabled across mosaic boundaries, but some cases are allowed. The inter-image prediction dependencies are not restricted, however, with respect to mosaic boundaries. A prediction unit in a mosaic can reference regions in a reference image that is outside the spatial limits of an image placed in the reference image. Thus, for mosaics in the January 2013 version of the HEVC standard, no independence relationship is required for mosaics relative to other mosaics within other images that are used as references for inter-image prediction.

A tile set is an arrangement of one or more tiles in an image. A tile set can be specified as one or more tile ranges within the image. As explained in the next section, a restricted motion mosaic set (“MCTS”) is a mosaic set for which inter-image prediction dependencies are limited to regions within the image mosaic set to

INSTITUTE

DS LA DiC'i'ffnAO IN: V - '= aI.'. L

image. In simple cases, serial images have the same mosaic set configuration, so the mosaic set in a current image has a mosaic set placed in its reference image (s).

Figure 7a shows a box (701) divided into 16 uniformly dimensioned tiles. The tile count is implementation dependent and may have some other value (for example, 9, 20, or 25 tiles). In Figures 7b-7g, the tiles are grouped into tile sets in different ways.

A mosaic set can include multiple mosaics. For example, Figure 7b shows a box (702) in which four tiles are arranged in the center of box (702) as a tile set-tile set A. Alternatively, a tile set can include a single tile. In an extreme case, each tile in an image can be defined as its own tile set (for example, 16 tile sets for the 16 tiles, respectively, in the box in Figure 7a).

The count of tiles in a tile set, and tile settings within a tile set, can be arbitrarily specified from the available tiles. For example, Figure 7c shows a box (703) in which six tiles from box (703) are arranged as a tile set-tile set A. The remaining tiles are not in any tile set. A given mosaic of a painting can be distributed to a mosaic set or left outside the

IMPI

INSTITUTO MEX1CAN, DE LA ΡΚΟΓΙΕΟΑ ·

INDUSTRIAL

mosaic sets.

Figure 7d shows a table (704) in which the 16 tiles are distributed to tile sets. Tile set A includes four tiles in the center of frame (704) and Tile set B includes the remaining 12 tiles that surround the tiles of tile set A in box (704).

In Figures 7b-7d, the tiles in a given tile set are contiguous, that the tiles in a tile set do not need to be contiguous. For example, Figure 7e shows a box (705) in which 8 mosaics are distributed to mosaic set A and 8 mosaics are distributed to mosaic set B. The 8 mosaics of mosaic set B are separated into two regions on sides opposites of mosaic set A in table (705).

In many cases, a tile set includes one or more tiles in the center of a frame, as in Figures 7b-7e. This tile setting can be useful for decoding Region of Interest (for example, when the desired focal point is in the center or when an identified region contains a talking head for video conferencing). The configuration shown in Figures 7b and 7d offers an additional advantage that the aspect ratio does not change between the central mosaic set (mosaic set A) and square.

On the other hand, Figure 7f shows a table (706) in which mosaics are distributed to four mosaic sets A, B, C and D

IMPI

INSTITUTO MtXICAN 'DE LA «OWEDA.

INDUmi '

covering the entire frame (706). Each mosaic set has four mosaics. This mosaic set configuration can facilitate parallel encoding and decoding. In particular, for MCTS, motion estimation (during encoding) and motion compensation (during encoding or decoding) can be performed in parallel for tile sets A, B, C, and D.

In Figures 7b-7f, a mosaic is a maximum part of a mosaic set. In some implementations, however, a tile can be part of multiple tile sets. Figure 7g shows three different views of a frame (707) where some of the tiles are part of multiple tile sets, some of the tiles are part of a single tile set, and some of the tiles are part of no tile set. mosaic. In the box (707) shown in Figure 7g, the tiles in the top row of tiles are part of tile set A, tile set B (overlapping tile set A), and tile set C (which is overlap with mosaic set A and mosaic set B). The tiles in the second row are part of the second tile B and tile set C. The tiles in the third row are part of tile set C, and the tiles in the fourth row are part of no tile set. Such a mosaic configuration can facilitate functionality such as gradual decoder update, when the mosaic set used for encoding and decoding a given image may change image to

image within a sequence, or when the size, shape, and / or location of regions that can be referenced for tile sets are allowed to change from image to image within a sequence.

B. Restricted Motion Mosaic Set,

Usually

A restricted motion mosaic set (“MCTS) is a mosaic set for which inter-image prediction dependencies are limited to specific region or regions. In many cases, the specific regions are within the same image-by-image mosaic set. In other cases, however, the specific regions are within another mosaic set or some other region or regions of the reference images that are used for inter-image prediction. In general, it is possible to perform motion compensation for a separate MCTS for a decoding of other mosaic sets or regions outside the MCTS. This is possible because inter-image prediction is restricted to not reference any of the regions outside the MCTS in reference images (ie, outside the mosaic set placed in the reference images).

Coding for an MCTS can be implemented through constraints looking for motion estimation motion vectors. The search range for a motion vector is

bounded by mosaic set boundaries.

Figure 8 shows motion estimation and motion compensated prediction for a prediction unit for a mosaic set with no motion restrictions and mosaic set limits. Current frame 820 includes mosaic set A 822 with a prediction unit 824 which is a sample block or sub-block. The motion vector (826) for the prediction unit (824) is associated with a region (814) with a reference frame (810) that is used to generate the compensated motion prediction values for the prediction unit (824). ). The region (814) is either partially within mosaic set A (812) in reference frame (810), and partially outside mosaic set A (812) in reference frame (810). Mosaic set A is not in an MCTS, so there is no restriction on inter-image prediction processes for prediction units in mosaic set A referencing sample value locations outside mosaic set A. As a result, the correct decoding of the prediction unit (824) of the current frame (820) depends on the reconstruction of values outside the mosaic set A (812) in the reference frame (810).

In contrast, Figure 9 shows motion estimation and compensated motion prediction for a mosaic set prediction unit with mosaic set boundary movement constraints. The current table (920) includes a

MEXICAN INSTITUTE OF RAOLILLOAD

INDUSTRIAL mosaic set A (922) with a prediction unit (924) that is a sample block or sub-block. A motion vector (926) for the prediction unit (924) is associated with a region (914) in a reference frame (910) that is used to generate the compensated motion prediction values for the prediction unit (924). ). Even if a region partially or completely outside the mosaic set A (912) can give a better prediction for the prediction unit (924), due to restrictions on range of motion estimation, the encoder uses a region (914) that lies completely within the mosaic set placed A (912) in reference frame (910). A mosaic set A is an MCTS, so that none of the inter-image prediction processes for a prediction unit in mosaic set A can reference locations of sample values outside mosaic set A. As a As a result, the correct decoding of the prediction unit (924) of the current frame (920) does not depend on the reconstruction of values outside the mosaic set A (912) in the reference frame (910).

Thus, with MCTS, inter-image prediction dependencies are constrained across mosaic set boundaries. Movement is still allowed across mosaic boundaries within a mosaic set, however. The restrictions on intra-image prediction dependencies and arithmetic coding dependencies for mosaics still apply.

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

When performing filtering operations (for example, for unlock filtering) across mosaic boundaries, some of the mosaic set boundaries may be affected. As a result, the sample values from a reference frame used during encoding may not exactly match the sample values from a reference frame used during decoding. Specifically, if only the MCTS is decoded during decoding, the sample values at the MCTS mosaic set limits may be different in the reference table since circuit filtering through such mosaic set limits is not performed. This may have a minor negative effect on MCTS only decoding quality compared to full image decoding.

Decoding for an MCTS does not involve changes to core decoding processes. A decoder can use MCTS control data, however, to decide how to perform decoding for separate tile sets for different parts of images, or decide to perform ROI decoding, as explained below.

C. Illustrative Uses of MCTS and MCTS Control Data

This section describes various uses of MCTS and MCTS control data, including parallel encoding and decoding, region of interest decoding and display, simplified transcoding, and loss recovery. The data

MCTS controllers can enable useful functionality for regular video encoding / decoding, it can also be seen as a scalability extension for complexity scalability.

one. Parallel Coding and / or Parallel Decoding

An encoder can encode separate MCTSs in parallel for many encoding operations. The encoder segments its encoding processes in a region-specific way for the regions defined by mosaic sets. By using MCTS control data, a corresponding decoder can decode the separate MCTS in parallel for many decoding operations. The decoder segments its decoding processes in a region-specific way for the regions defined by the mosaic sets. In particular, for motion compensation for a given mosaic set, the encoder (or decoder) does not need to access reference image sample values for regions outside the given mosaic set. In this way, different MCTSs can be encoded or decoded in parallel, without the need to wait for reconstruction of all the reference images.

Figure 10 shows an example (1000) of parallel coding and parallel coding for images with MCTS. In the

Figure 10, the encoder (1010) receives the input video signal (1005), tiles it in four tile sets A, B,

MEXICAN INSTITUTE OF THE PROt'JEOAO

ΙΝΓ / ΧΤΚΙΑΙ

C and D (as in Figure 7f), and decodes the ~ with respective mosaic coughs in parallel. (Some encoder (1010) encoding processes, for example, circuit filtering, are not performed in parallel for different tile sets.) E l encoder (1010) produces a coded bit stream video (1015) with coded data sets for mosaic A, B, C and D. The bitstream encoded video (1015) also includes data MCTS control.

The encoded video bitstream (1015) is transported through the network (1020) to the decoder (1030). Using the MCTS control data to identify an opportunity for parallel decoding, the decoder (1030) decodes the respective mosaic sets in parallel, merges the reconstructed content for the mosaic sets, and produces reconstructed video (1035). (Some decoder processes of the decoder (1030), eg circuit filtering, are not performed in parallel for different sets of tiles).

Although Figure 10 shows both parallel encoding and parallel decoding, alternatively, only parallel encoding is implemented or only parallel decoding is implemented. Also, although Figure 10 shows an encoding and decoding in which the number of parallel process cases matches the number of mosaic sets (i.e. 4), alternatively the number of parallel process cases is less than the number of mosaic sets.

IMPI

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

2. ROI decoding

When inter-image prediction is constrained over mosaic set boundaries, a mosaic set can be decoded and presented independently for decoding and region of interest display (“ROI”). A decoder can use MCTS control data to decode ROI and display one or more selected tile sets. In this case, only the mosaic subset is specified by the mosaic set (s), instead of all images, it is decoded and rendered. For example, the decoder only decodes the subset of an encoded video bitstream containing the encoded data for the selected tile set (s), rather than decoding the encoded data for the entire images.

Figure 11 shows an example (1100) of ROI decoding for Images with an MCTS. In Figure 11, the encoder (1110) receives the input video signal (1105), tiles it to include a set of tile A (as in Figures 7b), and encodes the video. Encoder 1110 encodes mosaic set A common to an MCTS. Encoder 1110 produces an encoded video bit stream 1115 with encoded data for the entire image, including mosaic set A as an MCTS. The encoded video bitstream (1115) also includes MCTS control data.

The encoded video bitstream (1115) is transported to

IMPI • MEXICAN INSTITUTE

DE LA FROl'lEf ',,)? INDUSTR:, ·.!

through the network (1120) to the decoder (1130). By using MCTS control loTdata to identify an opportunity for ROI decoding, the decoder (1130) decodes the encoded data for mosaic set A and produces reconstructed video (1135) for mosaic set A.

ROI decoding is especially useful when the mosaic set selected for ROI decoding is a single rectangular area, which can be a single mosaic or contiguous rectangular area of tiles as in Figures 7b or 7d. For example, the individual rectangular area can be decoded for display on a small display device. Or, the individual rectangular area can be decoded for display as a picture-in-picture display window. Or, the individual rectangular area can be encoded for presentation as a part of a composite with small regions created from other bit streams (for example, for a multi-part conference).

In addition, in limited bandwidth scenarios such as real-time communication, signaling, and MCTS usage, they enable a new dimension of ROI scalability, with different continuous transmission bit rates for different decoding / presentation resolutions. This could be useful for scenarios in which video content is delivered for different devices through heterogeneous channels. For example, a bit stream can be organized as MCTS

IMPI

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

configured as one or more concentric "ring" regions around a central MCTS, such that (a) the central MCTS provides a lower bit rate and image size, (b) the central MCTS plus the first ring region concentric provide higher bit rate and higher image, (c) the central MCTS plus first two concentric ring regions provide higher or even bit rate and image size, and so on. Or, MCTS can be arranged for combination in other ways.

MCTS control data that specifies one or more regions for ROI decoding can be used in conjunction with panoramic scan metadata. For example, panoramic scan SEI messages allow the specification of rectangles for ROI presentation. With MCTS SEI messages that control ROI decoding, panning SEI messages can also allow ROI presentation.

3. Transcoding

In some cases, a transcoder performs simple low-delay transcoding operations to extract encoded data for one or more tile sets from a video encoded video bitstream that has a larger image size, producing a new bitstream Video encoded for video that has a smaller image size. For example, for HEVC transcoding, when a

MEXICAN INSTITUTE OF PROKTDAD

INDUSTRIAL

MCTS is a rectangular area, the transcoder can produce the new encoded bitstream for the MCTS by modifying only high level syntax elements, without the need to fully decode and / or recode lower level data (such as data at the level coding tree unit and below).

Figure 12 shows an example (1200) of transcoding for images with an MCTS. In Figure 12, the encoder (1210) receives the input video signal (1205), tiles it to include a set of tile A (as in Figure 7b), and encodes the video. Encoder 1210 encodes mosaic set A as an MCTS. Encoder 1210 produces an encoded video bit stream 1215 with encoded data for the entire image, including mosaic set A as an MCTS. The encoded video bitstream (1215) also includes MCTS control data.

The encoded bit video stream (1215) is transported through the network (1220) to the transcoder (1230). Using the MCTS control data to identify an opportunity for transcoding, transcoder (1230) discards encoded data for regions of the image outside mosaic set A, and produces an encoded video bit stream (1235) with data encoded only for the mosaic set

TO.

In HEVC implementations, even when the MCTS is not

rectangular, or is rectangular but is not transcoded, in some cases the subset of the bitstream required for decoding the MCTS may be extracted before sending the data to a decoder that is capable of operating on such a subset of the MCTS bitstream .

Four. Resistance to Loss and Recovery

Signaling and use of MCTS control data can also improve resilience to data loss and recovery from data loss. By providing a decoder with an explicit indication of region-by-region dependency ratios with decoded images, the decoder may be able to complete the decoding of some regions (tile sets) when encoded data has been corrupted for other regions (tile sets) ) or otherwise lost.

5. Gradual Decoder Update

An encoder may implement gradual decoder update functionality using MCTS control data in some implementations. For example, when a tile can be part of multiple data sets (as in the example in Figure 7g), the top row of tiles can define one MCTS, with the top two rows of tiles defining a second MCTS, the three upper rows of tiles defining a third MCTS, and so on. The encoder can use such

V Τ). <Τ '

11VI i '.a

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

MCTS for gradual decoder update functionality.

Figure 13 shows an example (1300) of gradual decoder functionality with MCTS. For an image (1301) in a series (image 30 in Figure 13), the encoder updates the region for MCTS A. The encoder encodes the top row of tiles (to be encoded as MCTS A in a later table) using intra encoding -image. Coding of other image tile rows (1301) is not restricted.

For the next image (1302) in the series (image 31 of Figure 13), the encoder updates the region for MCTS B using in-image prediction with dependencies on regions in MCTS A and intra-image encoding. The encoder encodes the top row of tiles as MCTS. This MCTS (as MCTS A) can be encoded using inter-image prediction relative to the mosaic set placed in the previous image (the top row of mosaics in image 30). The encoder encodes the second row of tiles in the image (1302) using intra-image encoding. The encoding of other rows of image tiles (1302) is not restricted.

For the next image (1303) in the series (image 32 in Figure 13), the encoder updates the region for MCTS using inter-image prediction with dependencies on regions in MCTS B and intra-image encoding. The encoder encodes the top two rows of tiles as an MCTS. This MCTS (MCTS B) can be encoded using inter-image prediction relative to the mosaic set placed in the previous image (the top two rows of

INSTITUTO McX 'CANO D £ THE PROPERTY

INDUSTRIAL

mosaics in image 31). The encoder encodes the third row of tiles in the image (1303) using intra-image encoding. Coding of the other row of image tiles (1303) is not restricted.

For the last image (1304) in the series (image 33 in the

Figure 13), the encoder updates the image using inter-image prediction with dependencies on regions in MCTS C and intra-image coding. The encoder encodes the top three rows of tiles as an MCTS. This MCTS (MCTS C) can be encoded using inter-image prediction relative to the mosaic set placed in the previous image (the top three rows of mosaics in image 32). The encoder encodes the last row of tiles in the image (1304) using intra-image encoding. At that point, the tile rows in the images have been gradually updated.

Alternatively, an encoder can implement gradual decoder update functionality by allowing shaded regions to be entered in Figure 13 (intra-image encoded regions) to be encoded using either intra-image encoding or inter-image encoding relative to a Non-corresponding "subordinate" region in an updated reference image. The encoding can decide between intra-image encoding and such inter-image encoding on a block basis whereby. For example, for the shaded region in the third image (1303), blocks can be encoded using intra-encoding.

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

image or inter-image encoding relative to the region of the second image (1302) just updated (top two rows of tiles). This extra flexibility can improve compression performance.

Figure 13 shows a special case of a more general scenario in which the relationship changes dynamically from image to image between (a) the region or regions that are indicated in the reference image and (b) the region or regions of a current image. who depend on them. In such a scenario, the size, shape, and / or location of regions that can be referenced for tile sets are allowed to change from image to image within a sequence.

One way to implement such dynamic changes is to signal MCTS control data per image. The MCTS control data for an image can identify an MCTS that is active for encoding and decoding for that image, where inter-image prediction dependencies are constrained to fall within a mosaic set placed on any reference image that is used. for the identified MCTS. For example, if MCTS B is identified for a current image, then inter-image prediction dependencies are constrained to fall within the MCTS B region on any reference image (even if MCTS was not identified for that reference image ).

When MCTS control data can be signaled by image, one aspect is to explicitly specify tiles

IMPI

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

in the Identified MCTS for that image. Another aspect is to use a common set of MCTS for all images in a coded video sequence (or group of images), then identify the active MCTS for an image using an identifier value within the set with an MCTS. For example, the common set of MCTS includes 4, 5, or 6 MCTS (possibly overlapping), in the MCTS control data for a given image MCTS 2 is identified as the active MCTS for encoding and decoding for that image.

Another way to implement such dynamic changes is to signal signal-by-image MCTS control data that identifies an active MCTS for the image and also identifies one or more reference image tile array reference regions. For example, the MCTS control data identifies an MCTS for a given current image identifies a tile set reference region and a reference image. For the current image, reference regions of different tile assemblies can be identified for different reference images. An identified mosaic set reference region can be placed on the

MCTS identified for the current image (as assumed in many examples described here), or may have a different size, location, shape. For the current image, the mosaic set reference region (s) can be explicitly linked (as a mosaic rectangle or as an arbitrary region) or identified by an identifier value of a common set of

MCTS that apply to the respective reference images. For example, a reference image may be associated with one or more MCTS defined when that Image was encoded, and subsequent images (in coding order) may specify mosaic set reference strands in the reference image by values of identifier of the one or more MCTS that were previously defined for the Reference Image.

D. Illustrative Signaling of MCTS Control Data

This section describes examples of syntax and semantics for MCTS control data.

one. First Example Syntax and Semantics for

MCTS SEI Messages

Figure 14a shows the syntax (1401) for an MCTS SEI message in an illustrative implementation. In Figure 14a, the set_mosalco_set_movement SEI message includes several syntax elements encoded using unsigned integer 0 °, Exp-Golomb order encoding with the first left bit (ue (v)), as well as some elements of syntaxes indicated with Indicators. The MCTS SEI message syntax (1401) is consistent with the HEVC standard, and this section includes references to various syntax elements defined in the HEVC standard.

For the MCTS SEI message shown in Figure 14a, the scope of the MCTS SEI message is the video stream

fully coded. When an MCTS SEI message is present in any access unit of an encoded video sequence, it is present for the first access unit of the encoded video sequence in decoding order. The MCTS SEI message may also be present for other access units of the encoded video stream.

The MCTS SEI message does not display for an encoded video stream if the flag_enable_ mosaic is 0 for any image parameter set (“PPS”) that is active in the encoded video stream. In this case (flag_ enabled_ mosaic is 0), mosaic is not enabled at least for some images. Even when tiles are enabled for images in the encoded video stream, the images in the encoded video stream must be tiled identically. That is, the MCTS SEI message is not present for an encoded video stream unless each PPS is active for the encoded video stream that have the same values for syntax elements num_mosaico_columnas_menos1, num_mosaico_filas_menos1, uninforme_separación_indicador, column_width_less1 [i] , and row_height_less1 [i], which specify how images are divided into tiles. This constraint is similar to the constraint associated with indicator_fix_structure_fixes that equals 1. (If the indicator_fix_structure_fixed mosaics noted in the video utility information is 1, then all

the active PPS for the encoded video stream have the same number of mosaic columns, the same number of mosaic rows, and the same size information; if 0, then it can vary in different PPS).

The MCTS SEI message identifies a mosaic set, the MCTS. The presence of the MCTS SEI message indicates that inter-image prediction is restricted so that no sample value outside the identified MCTS, and no sample value at a sample position per fraction that is derived using one or more sample values outside of the Identified MCTS, it is used for the inter-prediction of any sample within the identified MCTS. The syntax elements that identify the MCTS are defined as follows.

The syntax element num_mosaico_rect_en_conjunto_ less1, with the addition of 1, specifies the number of rectangular tile regions (tile rectangle examples) in the identified MCTS. The value of num_mosa¡co_rects_en_conjunto_menos1 is in the range of 0 to (num_mosaico_columnas_menos1 + 1) * (num_mosaico_filas_menos1 + 1) -1, inclusive.

The left_mosaic_column [i] and top_mosaic_ row [i] syntax elements identify the mosaic column and mosaic row, respectively, of the left and top mosaic of a rectangular region (mosaic rectangle example) of the MCTS. The syntax element width_in_mosaic_columns_less1 [i], with the

MEXtCAHU INSTITUTE OF PROPERTY

INDUSTRIAL

addition of 1 indicates the width of the rectangular region (mosaic rectangle examples) of the MCTS in mosaic column units. The value of width_in_mosaic_columns_less1 [I] is in the range of 0 to num_mosa¡co_columns_less1 left_mosaic_column [i], inclusive. The syntax element height_in_mosaic_files_less1 [i], with the addition of 1, indicates the height of the rectangular region (mosaic rectangle example) of the MCTS in mosaic row units. The value of height_in_mosaico_filas_menos1 [i] is in the range of 0 to num_mosaico_filas_menos1 - superior_mosa¡co_columna [I], inclusive.

Thus, the MCTS is the combination of one or more rectangular rulers (examples of mosaic rectangles) of mosaics identified in the MCTS SEI message.

In Figure 14a, the MCTS SEI message includes another syntax element that can be used by a decoder to assess whether the quality can be adversely affected in decoding only MCTS. When the exact syntax_sample_indication_match_value_indicator equals 0, within the encoded video stream, when (a) the encoding tree blocks outside the MCTS are not decoded and (b) the MCTS boundaries are treated as Image limits For purposes of the decoding process, the value of each sample in the identified MCTS may not be exactly the same as the value of the same sample when all the tree blocks of

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

Image encoding is decoded. On the other hand, when exact_sample_value_indicator_indicator equals 1, within the encoded video sequence, when (a) the encoding tree blocks do not belong to the MCTS they are not decoded and (b) the MCTS limits are treated as limits image for decoding process purposes, the value of each sample in the MCTS is exactly the same as the value of ia sample that would be obtained when all the encoding tree blocks of all the images in the encoded video sequence are encoded. Exacto_muestra_valor_coincidencia_indicador set equal to 1 may be possible with certain combinations of values for syntax elements circu ito_f i través_mosaicos_habilitados_indicador ltro_a, través_fragmentos_habilitado_indicador pps_circuito_filtro_a, pps_desbloqueo_filtro_deshabilitado_indicador, través_fragmentos_habilitado_indicador fragmento_circuito_filtro_a, fragmento_desbloqueo_filtro_deshabilitado_indicador, sample_adaptable_scroll_enabled_indicator_fragment_sao_luma_indicator, and fragment_sao_chrom_indicator.

In Figure 14a, the MCTS SEI message includes other syntax elements that can be used for ROI presentation in conjunction with ROI decoding. When pan_escaneo_rect_ind¡cador is 0, the mets_psr_id element is not present in the MCTS SEI message. When pan_escaneo_rect_ind¡cador is 1, mets_psr_¡d is present. The

FROM INDUSTRIAL PROPERTY syntax element mets_psr_id indicates that the identified MCTS covers at least the panoramic scan rectangle with pan_escaneo_rect_id equal to mets_psr_id within the encoded video stream. When pan_escaneo_rect_indicador is 1, at least one panoramic scan rectangle with pan_escaneo_rect_id equal to mets_psr_id is present in the encoded video stream.

For the MCTS SEI message syntax 1401 in Figure 14a, multiple MCTS SEI messages may be associated with the encoded video sequence, each identifying an MCTS. Consequently, more than one different MCTS may be active within an encoded video stream.

2, Second Illustrative Syntax and Semantics for

SEI Messages from MCTS

Figure 14b shows the syntax (1402) for a message of

MCTS SEI in another illustrative implementation. As in Figure 14a, the SEI message_restricted_mosaic_group_set_group includes several syntax elements encoded using ue (v) encoding, as well as some syntax elements flagged. The MCTS SEI message syntax (1402) is consistent with the HEVC standard, and this section includes references to various syntax elements defined in the HEVC standard.

For the MCTS SEI message shown in Figure 14b to be present, the flag_enabled_ mosaics equals 1

instituto müx; can; i DELA PRO ñ EDA O INDUSTRIAL for all PPS active in the encoded video sequence (indicating images that have mosaics) and the mosaic_fix_structures_indicator equals 1 in the encoded video sequence. This indicates that all active PPS for the encoded video stream specify the same number of mosaic columns, the same number of mosaic rows, and the same size information for images in the encoded video stream.

When present, the MCTS SEI message only appears if it is associated with the first primary image of an encoded video stream, a split link access image (“BLA”), an instant decoding update image (“IDR”) ). The target image set for the MCTS SEI message contains all consecutive primary encoded images in decoding order starting with the first associated primary encoded image (inclusive) and ending with (a) the next primary encoded BLA or IDR image (exclusive ) or (b) the last primary encoded image in the encoded (inclusive) video sequence in decoding order when there is no next primary encoded BLA or IDR image.

The MCTS SEI message identifies a mosaic set, where MCTS, which is a collection of one or more mosaics. A group (example of mosaic rectangle) of one or more mosaics for the MCTS is identified by the syntax elements top_left [i] and bottom_right [i]. When

INSTITUTO MEXICAI <0 DE LA PROPIEDAD INDUSTRIAL separated_color_plano_indicador is 1 with the term "primary coded images" represents the parts of the corresponding primary coded images that correspond to the NAL units that have the same color_plano_id. The MCTS SEI message indicates that for each image in the target image set, inter-image prediction is restricted as below. No sample value outside the MCTS, and no sample value in a sample at a fraction sample position that is derived using one or more sample values outside the MCTS, is used for inter-image prediction of any sample within the MCTS.

MCTS is the combination of one or more rectangular mosaic regions (mosaic groups) that are examples of mosaic rectangles) Identified in the MCTS SEI message. The element num_mosaicos_grupos_en_conjunto_menos1, with addition of 1, specifies the number of mosaic groups (mosaic rectangle examples) in the MCTS. The averaged range of num_mosaico_grupo_en_conjunto_menos1 in 0 to (num_mosaico_columnas_-1 + 1) by num_mosaico_filas_menos1 + 1) -1, inclusive.

The top_left [i] and bottom_right [i] syntax elements specify the upper left corner and lower right corner, respectively, of a mosaic group (example mosaic rectangle) with restricted inter-image prediction, in tree block units coding. The

values of left_or [i] and right_fer [i] are with mosaic group unit positions in a raster scan of the image. For each rectangle i, the following restrictions are obeyed by the values of upper_left [i] and lower _right [i];

• upper_left [i] is less than or equal to lower_right [i];

• lower_right [i] is less than PicSizeEnCtbsY;

• (upper_left [i]% PicAnchoEnCtbsY) is less than or equal to the value of (lower_right [i]% PicAnchoEnCtbsY); and • the rectangle specified by top_left [i] bottom_right [i] contains one or more complete tiles.

In Figure 14b, the MCTS SEI message includes other syntax elements that can be used for ROI presentation in conjunction with ROI decoding. When the pan_escaneo_rect_indicator syntax element is 0, pan_escaneo_rect_id is not present. When pan_escaneo_rect_indicador is 1, pan_escaneo_rect_¡d is present. The pan_escaneo_rect_id syntax element indicates that the specified MCTS covers at least the panoramic scan rectangle identified by the pan_escaneo_rect_id within the target image set.

For the MCTS SEI message syntax (1402) shown in Figure 14b, multiple MCTS SEI messages can be associated with the same target image set. Consequently, more than one MCTS may be active within a target image set.

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL

3. Third Example Syntax and Semantics For

SEI Messages from MCTS

Figure 14c shows syntax (1403) for a MCTS SEI message in another illustrative implementation. The presence of the MCTS SEI message indicates that the inter-prediction process is constrained so that no sample value outside of each identified mosaic set, and no sample value at a friction sample position that is lived using one or more sample values outside the identified mosaic set is used for inter-prediction of any sample within the identified mosaic set. Except as indicated in this section, the MCTS SEI message syntax (1403) shown in Figure 14c is the same as the MCTS SEI message syntax (1401) shown in Figure 14a.

When more than one MCTS SEI message is present within the access units of an encoded video stream, they must contain identical content. The number of SEI messages from

MCTS in each access unit must not exceed 5.

The num_conjunto_en_mensaje_menos1, with addition of 1, specifies the MCTS number identified in the SEI message. The value of num_conjunto_en_mensaje_menos1 is in the range of 0 to 255, Inclusive.

The mets_id syntax element [¡] contains an identification number that can be used to identify the purpose of the ¡e5imo identified mosaic set. For example, the element of

Mexican INSTITUTE OF PROPERTY

INDUSTRIAL

syntax mets_id [i] can be used to identify an area to be extracted from the encoded video stream for a particular purpose. The value of mets_id [¡] must be in the range of 0 to 232 2, inclusive. The values of mets_id [i] from 0 to 255 and from 512 to 231 They can be used as determined by the application. Values of mets_id [¡] from 256 to 511 and from 231 to 232 - 2 are reserved for future use. Decoders that find the value of mets_id [i] in the range of 256 to 511 or in the range of 231 to 232 - 2 Ignore it (removing it from the bit stream and discarding it).

The remaining syntax elements num_mosa¡co_rects_en_conjunto_less 1 [I], left_mosalco_column [i] [j], superior_mosaico_flla [i] [j], ancho_en_mosa¡co_columnas_menos1 [i] [j], height_in_mosalco_fllas_ less1 [I] [] ], pan_escaneo_rect_lnd¡cador [1], and mets_psr_ld [¡] generally have the meaning explained with reference to the syntax (1401) of the Illustrative MCTS SEI message of Figure 14a. For each syntax element, however, the circuit counter variable indicates the value of the syntax element for the |nes, mo MCTS specified in the MCTS SEI message, and the circuit counter variable j indicates value for jth mosaic rectangle in a given MCTS. Alternatively, instead of using left_mosaico_column [i] [j], superlor_mosaico_fila [¡] [j], width_in_mosaico_columns_less1 [I] [j], and

height_in_mosa¡co_f¡las_less1 [i] [j], two syntax elements for a given mosaic rectangle can identify the mosaic position of the upper left mosaic in the mosaic rectangle and the mosaic position of the lower right mosaic in the rectangle of mosaic, respectively, in order of scanning mosaic grid.

Four. Syntax and Alternative Semantics for Data

MCTS control

In the preceding two sections, an MCTS SEI message specifies an MCTS and identifies the tile (s) in that MCTS. For this aspect, there may be multiple SEI messages for MCTS control data when there are multiple MCTSs for an individual encoded video stream, with each MCTS SEI message specifying a different MCTS within the same encoded video stream.

Alternatively, a single MCTS SEI message can specify multiple MCTSs. For example, an external circuit in the MCTS SEI message syntax is repeated for the respective MCTS. For a given MCTS, syntax and semantics can follow the example of one of the two preceding sections to identify regions (mosaic rectangles) of mosaics for the MCTS, an associated panoramic scan rectangle, etc.

In the preceding two sections, an MCTS SEI message implies that the mosaic set is an MCTS. Alternatively,

X .k. _L

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL

an MCTS SEI message can decompose an image of multiple mosaic sets, and one indicator per mosaic set with the MCTS SEI message indicates whether the mosaic set is an MCTS or not an MCTS.

In the two sections present, the scope of an MCTS SEI message may be an encoded video stream (as in the example in Figure 14a) or possibly a group of images between a BLA or IDR (inclusive) image and another BLA image or IDR (as in the example in Figure 14b). Alternatively, a MCTS SEI message may be image signaled or have some other scope.

In the preceding two sections, images affected by an MCTS SEI message have the same mosaic set configuration, so mosaic to mosaic sets do not switch from image to image within an encoded video stream (or group of images). Alternatively, the size, shape and / or location of the regions that can be referenced for an MCTS can change from image to image within the encoded video sequence (or group of images).

In the preceding two sections, the MCTS control data is a SEI message. Alternatively, the MCTS control data may be some other form of metadata or an elementary video bitstream syntax element indicating that inter-picture prediction dependencies across mosaic set boundaries are constrained for a given mosaic set.

ΐ7 "?!

you

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

F. Techniques for Signaling and Using Control Data d

MCTS

Figure 15 shows a generalized technique (1500) for signaling MCTS control data. A video encoder such as one described above with reference to Figure 3 or 5 or another performs the technique (1500).

The tool encodes (1510) multiple images to produce encoded data. Each of the multiple images is divided into multiple tiles. For example, each of the multiple images is divided into mosaic rows and mosaic columns that define the multiple mosaics for the image, and each of the multiple mosaics is a rectangular region. In illustrative implementations, each of the multiple images is divided identically to produce the multiple tiles within each of the multiple Images. Alternatively, different images can be tiled differently.

The tool sends (1520) the encoded data along with control data indicating that inter-image prediction dependencies across specific limits (eg, mosaic set limits) are constrained for a given mosaic set (the MCTS ) of one or more tiles from multiple tiles. Control data can include one or more syntax elements that identify when multiple tiles are provided in the

MCTS given.

In illustrative implementations, a set is parameterized

MEXICAN INSTITUTE OF PROPERTY

INDUSTRIAL mosaic given in the control data as one or more mosaic rectangles that include the one or more mosaics in the mosaic set. For example, for a given mosaic rectangle in the mosaic set, the control data includes syntax elements that identify two corners of the mosaic rectangle (such as a top left corner of the mosaic rectangle and bottom right corner of the mosaic rectangle. ). Control data may also include a mosaic set identifier, a count parameter indicating a count of mosaic rectangles in the mosaic set, and, for each of the mosaic rectangle in the mosaic set, syntax elements that indicate the location of the mosaic rectangle (for example, the position, width, and height of the mosaic rectangle).

More generally, the syntax element (s) may include a count parameter indicating a count of tile regions in the given MCTS, where each tile region covers one or more tiles of the multiple tiles. The syntax element (s) may also include, for each of the mosaic rulers in the given mosaic set, one or more location parameters indicating location of the mosaic region (for example, position, width, and height of the mosaic region).

Control data can include other syntax elements.

For example, the control data includes an indicator indicating whether (a) samples reconstructed for the given MCTS if portions of the multiple images outside the given MCTS are not decoded,

τ ¿

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY exactly match, (b) with reconstructed samples for the given MCTS if portions of the multiple images outside the given MCTS are decoded. Or, the control data includes an identifier of a panoramic scan rectangle covered by the given MCTS.

In illustrative implementations, the control data is a SEI message for an individual MCTS, indicating inter-image prediction dependencies across mosaic set boundaries that are constrained for the given MCTS. In this case, the control data may include a different SEI message for each of the given MCTS and one or more of other MCTS. Alternatively, the control data is a single SEI message for multiple MCTSs, including the given MCTS and one or more other MCTSs. Or, the control data may be an indicator whose value indicates whether inter-image prediction dependencies across mosaic set boundaries are constrained for the given mosaic set. Or, the control data may take some other form.

In illustrative implementations, the given mosaic set is identical for each of the multiple images. Alternatively, the given mosaic set differs between at least some of the multiple images.

Control data may also include inter-image prediction dependencies across specific boundaries that are constrained for each of one or more tile sets of the multiple tiles. This may be the case, for example, when the

Coding (1510) has used parallel processing for at least some coding steps for the given MCTS and one or more other MCTS.

The tool can repeat the technique (1500) on a unit-by-unit basis (eg, sequence-by-sequence basis), (group-by-group basis). For simplicity's sake, Figure 15 does not show how the technique (1500) works in conjunction with other coding processes.

Figure 16 shows an illustrative technique (1600) for selective use encoding of MCTS. A video encoder such as one described above with reference to Figures 3 or 5 or another tool performs the technique (1600).

The tool breaks (1610) a current image into tiles for encoding. The tool decides (1620) whether to constrain movement for a given set of one or more of the tiles. If so, the tool encodes (1630) the MCTS mosaic (s) with motion constraints at tile set boundaries, constraining motion estimation during encoding so that inter-image prediction dependencies across boundaries mosaic set are avoided for the MCTS. The tool sends (1640) the encoded data along with control data indicating that interimage prediction dependencies across mosaic set boundaries are restricted for the mosaic set. Otherwise (unrestricted movement for tiles), the tool encodes (1650) the tile (s) without

institute ι «κ: τ.? <ο DE LA? RC» ') vÍ? aD IWDUSTRJáL movement restrictions at mosaic set boundaries, and sends (1660) the encoded data. The tool reviews (1670) and continues to encode for any of the other tiles in the image and, if so, decides (1620) whether or not to encode one or more remaining tiles as an MCTS. After encoding the current image, the tool decides (1680) whether to continue with the next image in a series.

Figure 17 shows a generalized technique (1700) for processing flagged encoded data along with data control data.

MCTS. A video decoder such as one described above with reference to Figure 4 or 6 or another tool performs the technique (1700).

The tool receives (1710) encoded data for multiple images. Each of the multiple images is divided into multiple tiles. For example, each of the multiple images is divided into mosaic rows and mosaic columns that define the multiple mosaics for the image, and each of the multiple mosaics is a rectangular region. In illustrative implementations, each of the multiple images is divided identically to produce the multiple mosaics within each of the multiple images. Alternatively, different images can be tiled differently.

The tool also receives (1720) control data indicating that inter-image prediction dependencies across specific limits (for example, mosaic set limits)

ΙΝ5 η τ ui ό ív; : cλ no OF THE PROPERTY

INDUSTRIAL

they are restricted to a given mosaic set (the MCTS) of one or more mosaics of the multiple mosaics. Control data may include one or more syntax elements that identify which of the multiple tiles are in the given MCTS.

In illustrative implementations, a given tile set is parameterized in the control data as one or more tile rectangles that include the one or more tiles in the tile set. For example, for a given mosaic rectangle in the mosaic set, the control data includes syntax elements that identify two corners of the mosaic rectangle (such as a top left corner of the mosaic rectangle and a bottom right corner of the mosaic rectangle. ). Control data can also include an identifier for the mosaic set, a count parameter that indicates a count of mosaic rectangles in the mosaic set, and, for each of the mosaic rectangles in the mosaic set, syntax elements that indicate the location of the mosaic rectangle (for example, the position, width, and height of the mosaic rectangle).

More generally, the syntax element (s) may include a count parameter indicating a count of tile regions in the given MCTS, where each tile region covers one or more tiles of the multiple tiles. The syntax element (s) may also include, for each of the mosaic regions with the given mosaic set, one or more location parameters indicating location of the mosaic region (for

example, the position, width and height of the mosaic region).

Control data can include other syntax elements. For example, the control data includes an indicator indicating whether (a) samples reconstructed for the given MCTS if portions of the multiple images outside the given MCTS are not decoded, exactly match (b) with samples reconstructed for the given MCTS if the Portions of the multiple images outside the given MCTS are decoded. Or, the control data includes an identifier of a panoramic scan rectangle covered by the given MCTS.

In illustrative implementations, the control data is a SEI message for an individual MCTS, indicating inter-image prediction dependencies across mosaic set boundaries that are constrained for the given MCTS. In this case, the control data may include a different SEI message for each of the given MCTS and one or more other MCTS. Alternatively, the control data is a single SEI message for multiple MCTSs, including the given MCTS and one or more other MCTSs. Or, the control data may be an indicator whose value indicates whether inter-image prediction dependencies across mosaic set boundaries are constrained for the given mosaic set. Or, the control data may take some other form.

In illustrative implementations, the given mosaic set is identical for each of the multiple images.

Alternatively, the given mosaic set differs between at least some of the multiple images.

IMPI

MEXICAN INSTITUTE OE THE PROPERTY

INOUSTRIAL

The tool processes (1730) the encoded data. For example, as part of the processing of the encoded data, the tool decodes the given MCTS as a region of interest within the multiple images without decoding portions of the multiple images outside the given MCTS. Or, as part of the encoded data processing, the tool transcodes the encoded data by removing encoded data for portions of the multiple images outside of the given MCTS, and arranging encoded data for the given MCTS as a new bit stream.

Control data may also indicate inter-image prediction dependencies across specific limits that are constrained for each of one or more other MCTSs. In this case, the processing of the encoded data may include decoding that uses parallel processing for at least some decoding steps for the given MCTS and the one or more other MCTS.

The tool can repeat the technique (1700) on a unit-by-unit basis (eg, sequence-by-sequence basis, group-by-group basis). For the sake of simplicity, Figure 17 does not show how the technique (1700) works in conjunction with other decoding processes.

In view of the many possible embodiments to which the principles of the disclosed invention may apply, it should be recognized that the illustrative embodiments are only preferred examples of the invention and should not be taken as limiting the scope of the invention. Rather, the scope of the invention is defined by the following claims. Therefore we claim as our invention everything that comes within the scope and spirit of these claims.

INSTITUTO MtXICAN '·

FROM THE FROPIEDa:. SAW

INDUSTRI «i

Contents95

28 members in 11 offices

This recordMember, by publication datePriority claim1 more office is in the list below

Priority claims14

Priority claims
Document	Office	Kind	Date
201361809427	United States of America	P	2013-04-08
201361809427	United States of America	P	2013-04-08
61809427	United States of America	–	2013-04-08
14021883	United States of America	–	2013-09-09
201314021883	United States of America	A	2013-09-09
201314021883	United States of America	A	2013-09-09
2013061057	United States of America	W	2013-09-20
2013061057	United States of America	W	2013-09-20
14021883	–	–	–
61809427	–	–	–
PCTUS2013061057	–	–	–
US201314021883	–	–	–
US201361809427P	–	–	–
WO2013US61057	–	–	–

Members28

Family members, oldest first
Document	Office	Kind
US2014301464A1	United States of America	A1	2014-10-09
CA2908007A1	Canada	A1	2014-10-16
WO2014168650A1	World Intellectual Property Organization (WIPO)	A1	2014-10-16
AU2013385808A1	Australia	A1	2015-10-22
KR20150140360A	Republic of Korea	A	2015-12-15
EP2984823A1	European Patent Office (EPO)	A1	2016-02-17
CN105432082A	China	A	2016-03-23
MX2015014202A	Mexico	A	2016-05-05
JP2016519516A	Japan	A	2016-06-30
RU2015142850A	Russian Federation	A	2017-04-12
BR112015025393A2	Brazil	A2	2017-08-29
US9749627B2	United States of America	B2	2017-08-29
AU2013385808B2	Australia	B2	2017-09-21
US2017318288A1	United States of America	A1	2017-11-02
RU2648592C2	Russian Federation	C2	2018-03-26
MX358071BThis record	Mexico	B	2018-08-03
JP6449852B2	Japan	B2	2019-01-09
CN105432082B	China	B	2019-10-18
EP2984823B1	European Patent Office (EPO)	B1	2019-12-04
US10523933B2	United States of America	B2	2019-12-31
EP3606068A1	European Patent Office (EPO)	A1	2020-02-05
CA2908007C	Canada	C	2020-07-28
KR102178769B1	Republic of Korea	B1	2020-11-13
BR112015025393B1	Brazil	B1	2022-12-20
BR122022001588B1	Brazil	B1	2022-12-20
EP3606068B1	European Patent Office (EPO)	B1	2024-12-18
EP4482149A2	European Patent Office (EPO)	A2	2024-12-25
EP4482149A3	European Patent Office (EPO)	A3	2025-03-05