Adaptive processing with multiple media processing nodes.
19 claims: 4 independent, 15 dependent
- 1REIVINDICACIONES 1. Un método para decodificación de audio, que comprende:obtener, mediante uit decodificador de audio, un flujo de bits codificado, el flujo de bits codificado incluyendo datos de audio y datos de señalización, los datos de señalización indicando que un valor de volumen de ruido se encuentra incluido en el flujo de bits codificado;obtener, mediante el decodificador de audio, el valor de volumen de ruido del flujo de bits codificado;y procesar, mediante el decodificador de audio, los datos de audio de acuerdo con el valor de volumen de ruido.
- 2El método de la reivindicación 1, en donde procesar los datos de audio incluye normalizar el ruido de los datos de audio de acuerdo con el valor de volumen de ruido. 3. El método de la reivindicación 1, que :además comprende: determinar a partir de la señalización que los . datos de audio incluyen datos de audio de diálogo y que el valor de volumen de ruido es un valor de normalización de diálogo;y normalizar los datos de audio de diálogo de acuerdo con el valor de normalización de dialogo. < 148
- 34. El método de la reivindicación.1, que JLléhiá'g' comprende:determinar a partir de la señalización que un dato pico de muestra se encuentra incluido en el flujo de bits codificado;obtener los datos pico de muestra del flujo de bits codificado;y procesar los datos de audio de acuerdo con los datos pico de muestra.
- 45. El método de la reivindicación 1, que además comprende:determinar a partir de la señalización que un dato pico verdadero esté incluido en el flujo de bits codificado;obtener el dato pico verdadero del flujo de bits codificado;y procesar los datos de audio de acuerdo con el dato pico verdadero.
- 56. El método de la reivindicación 1, que además comprende:determinar a partir de la señalización que datos de control de rango dinámico estén incluidos en el flujo de bits codificado;IMPI 149 obtener los datos de control de rango dinámico HaI de bits codificado;y procesar los datos de audio de acuerdo con los datos de control de rango dinámico.
- 67. El método de la reivindicación 6, que además comprende:determinar a partir de la señalización que los datos de audio incluyan datos de audio de diálogo y que los datos de control de rango dinámico sean datos de control de rango dinámico de diálogo;y procesar los datos de audio de diálogo de acuerdo con los datos de control de rango dinámico de diálogo.
- 78. El método de la reivindicación 1, en donde el flujo de bits codificado incluye un primer flujo de bits que incluye los datos de audio y un segundo flujo de bits que incluye el valor de volumen de ruido.
- 89. El método de la reivindicación 1, en donde los datos de audio están codificados en cuadros de datos de audio, el valor de volumen de ruido está incluido en un campo auxiliar o reservado de un cuadro de datos de audio, y el valor de volumen de ruido está codificado por separado de los datos de audio en el cuadro de datos de audio.
- 910. Un dispositivo decodificador dé audio, que 150 comprende:un procesador;y una memoria acoplada al procesador y configurada para 5 implementar un método que comprenden: obtener un flujo de bits codificado, el flujo de bits codificado incluyendo un dato de audio y datos de señalización, los datos de señalización indicando que un valor de volumen de ruido se encuentra incluido en el flujo 10 de bits codificado;obtener el valor de volumen de ruido del flujo de bits codificado;y procesar los datos de audio de acuerdo con el valor de volumen de ruido. 15
- 1011. El dispositivo decodificador de audio de la reivindicación 10, en donde procesar los datos de audio incluye normalizar el ruido de los datos de audio de acuerdo con el valor de volumen de ruido.
- 1112. El dispositivo decodificador de audio de la 20 reivindicación 10, que además comprende:determinar a partir de la señalización que los datos de audio incluyen datos de audio de diálogo y gue el valor de volumen de ruido es un valor de normalización de diálogo;y IMPI 151 normalizar los datos de audio de diálogo’‘de acuerdo''con el valor de normalización de diálogo.
- 1213. El dispositivo decodificador de audio de la reivindicación 10, que además comprende:determinar a partir de la señalización que un dato pico de muestra se encuentra incluido en el flujo de bits codificado;obtener los datos pico de muestra del flujo de bits codificado;y procesar los datos de audio de acuerdo con los datos pico de muestra.
- 1314. El dispositivo decodificador de audio de la reivindicación 10, que además comprende:determinar a partir de la señalización que un dato pico verdadero esté incluido en el flujo de bits codificado;obtener el dato pico verdadero del flujo de bits codificado;y procesar los datos de audio de acuerdo con el dato pico verdadero.
- 1415. El dispositivo decodificador de audio de la reivindicación 10, que además comprende:152 IMPI instituto mrxicano US LA PROPISDAD INDUSTRIAL determinar a partir de la señalización que—daTUs de control de rango dinámico estén incluidos en el flujo de bits codificado;obtener los datos de control de rango dinámico del flujo de bits codificado;y procesar los datos de audio de acuerdo con los datos de control de rango dinámico.
- 1516. El dispositivo decodificador de audio de la reivindicación 15, que además comprende:determinar a partir de la señalización que los datos de audio incluyan datos de audio de diálogo y que los datos de control de rango dinámico sean datos de control de rango dinámico de diálogo;y normalizar los datos de audio de diálogo de acuerdo con los datos de control de rango dinámico de diálogo.
- 1617. El dispositivo decodificador de audio de la reivindicación 10, en donde el flujo de bits codificado incluye un primer flujo de bits gue incluye los datos de audio y un segundo flujo de bits que incluye el valor de volumen de ruido.
- 1718. El dispositivo decodificador de audio de la reivindicación 10, en donde los datos de audio están codificados en cuadros de datos de audio, el valor de volumen 153 IMPI de ruido está incluido en un campo auxiliar tr'Téservado de un cuadro de datos de audio, y el valor de volumen de ruido está codificado en separado de los datos de audio en el cuadro de datos de audio.
- 1819. Un método para codificación de audio, que comprende:obtener datos de audio;procesar los datos de audio de acuerdo con un valor de volumen de ruido y datos de control de rango dinámico;generar señales que incluyan el valor de volumen de ruido y los datos de control de rango dinámico;codificar los datos de audio y las señales en un flujo de bits codificado;y almacenar el flujo de bits codificado como uno o más archivos en un medio de almacenamiento o enviar el flujo de bits codificado hacia un decodificador de audio sobre un canal de comunicaciones.
- 1920. El método de la reivindicación 19, en donde los datos de audio incluyen datos de diálogo, el valor de volumen de ruido es un valor pico de muestra o verdadero y los datos de control de rango dinámico están asociados con los datos de diálogo. / * 154
Independent claims19
834 paragraphs in 174 sections, as filed
(54) Title: ADAPTIVE PROCESSING WITH MULTIPLE MEDIA PROCESSING NODES.
(54) Title: ADAPTIVE PROCESSING WITH MULTIPLE MEDIA PROCESSING NODES.
(57) Summary
Techniques for adaptive processing of media data based on separate data that specify a status of the media data are provided. A device in a media processing chain can determine if a media processing type has already been performed on an input version of the media data. If so, the device can tailor its processing of the media data to disable the performance of the media processing type. If not, the device performs the media processing type. The device can create a state of the media data that specifies the type of media processing. The device may communicate the status of the media data and an output version of the media data to a canister device in the media processing chain, for the purpose of supporting adaptive processing of the canister data device. media.
(57) Abstract
The present invention relates generally to media processing systems, and in particular, to adaptively processing media data based on media processing states of the media data.
PATENT TITLE No. 359652 ί Μ Ρ1 | Γ **
Say i./·- í<sup>;</sup>'D' · Say '«***; í ·} '
Headlines):
Home:
DOLBY LABORATORIES LICENSING CORPORATION
100 Potrero Avenue, San Francisco, California, 94103-4813, USA
Denomination:
Classification:
Inventor (s):
Number:
MX / a / 2016/004510
ADAPTIVE PROCESSING WITH MULTIPLE MEDIA PROCESSING NODES.
CIP: G10L19 / 008; G10L19 / 16; G10L21 / 00
CPC: G1G # _19 / 008; G10L19 / 167; G1OL21 / Q0
JEFFREY RIEDMILLER; REGUNATHAN RADHAKRISHNAN; MARVIN PRIBADI; FARHAD FARAHANI; MICHAEL. SNMTHERS
International REQUEST:
j 2011
388238
Divijtfupftl
<img file="MX359652B_D0001.tif" />
Number:
61/419,747
61 / 558,286. '3 4e dk¿H1il3r®' 4b go 1Q., 1Q de noviúmbte.'4e .20i1
Country:
US US
Validity: Twenty years
Expiration Date: December 1, 2031
Issue Date: October 5 4e 2ÍI18
The reference patent is granted on the basis of eAfej ártíojjjos T * 2 ° fríicciíh V, 6 'fraction #f, and 89 of the Law of the Industrial Brdpiwted.
In accordance with the article of the Industrial Property Law, the present term is valid for a non-extendable period, counted from the date of filing of the. international application and will be subject to the payment of the fee to keep the rights in force
Whoever subscribes to this title did so on the basis of Jo provided by articles 6 "sections III and 7" bis 2 of the Industrial Property Law (Official Gazette of the Federation (391 amended on 08/02/1994, 10/25/1986 . 12/26/1987; 08/17/1999, 01/26/2004, 06/16/2005,
01/25/2006, 05/06/2009, 06/01/2010 06/2010, 01/27/2612, flg / Q4 / 2Q 12, 06/01 / 2Q16 and 03/13/2018), articles Γ 3 ° fraction V subsection a), 4 ° and 12 “sections i and III of the Regulations of the MtituKrMkMcarW ¢ 8 <sup>the</sup> Prwedad Industrial (O; Q, Fv-14 / l2 / 1999, amended on 07/01/2002, 07/15/2004. 07/28/2004 and 09/07/2007); articles 1, 3 ", 4, Sfiecaton V incise-S), 16 'fraíctofw» I and * and JOntfel Organic Statute of the Mexican Institute of Industrial Property (DOF 12/27/1999, amended on 10/1/2 ^ 02, 29/07/2004 04/08/2004-¼ 13 / <J9 / 2O07 | 1 °, 3 “and 5 'clause a) of the Agreement that delegates powers to the Deputy Directors General. Coordinator, DirectqBlS DtVísidtwles, TitJteres'of the Regional Offices, Divisional Deputy Directors, Departmental Coordinators and other subordinates of the Instituto Mexicana given Industrial Property. (DOF 12/15/1999, amended on 02/04/2000, 07/29/2004, 08/04/2004 and 09/13/2007).
This letter is signed with an advanced electronic signature (FIEL), based on articles 7 BIS 2 of the Industrial Property Law; 3 of its Regulations, and 1 fraction III, 2 fraction V, 26 BIS and 26 TER of the Agreement establishing the guidelines for the use of the Electronic Payment and Services Portal (PASE) of the Mexican Institute of Industrial Property, in the procedures indicated.
<img file="MX359652B_D0002.tif" />
THE DIVISIONAL DIRECTOR OF PATENTS
NAHANNY CANAL REYES
<img file="MX359652B_D0003.tif" />
Original string:
NAHANNY MARISOL CANAL REYES | 0000100Ci000403252793 | Administration Service
Tax | 1695 || MX / 2018/84927 | MX / a / 2016/004510 | Normal patent title with divisional PCT | 1027 | RGZ | Page (s) | ur99NAMXZ2f6VH70i399ir62vTU =
Digital stamp:
Mlbot3n97uth34rAYrRknJ 11 sA557J4jNHtTOHGe5SDDy8mRS3qOf9 / uyXs7yJfqd + LcvvxlBmaGP526mE64bhDdnl / 9tOnpqxRP9K97fkAkrr> 7bEOwRf1uH8kzxQcZXXxAzS9rhUTuPW9g3kOy7r4quMS2ZA4uBLmDmypnwCLILmQpfnm7Str CF4BvwlfAA8xBlfWpa2QXYVVzpoiaSOG6uQebePBZucy + DuPJR / 2uqxw2h5cRM3f93R + KjrdA + GliCucu3Pv9UoEBVO 9uTlk1Z wCUbB1BSrlZr9L66dm + + + TVDSS kmH6qXAzGrftDY + yP0Zit + / g36rAmBV¡2FKoXIFg == DFLC: i: ta Τιψβρ.ιη Χ, ιχίmniliro. lriJZü
<img file="MX359652B_D0004.tif" />
MX / 2018/84927
SlOlU ^ / Q
------ M.lMIIMIIt --- 35 (£ 65
ADAPTIVE PROCESSING WITH MULTIPLE
IMPI MEXICAN INSTITUTE OF INDUSTRIAL HtORITY
<img file="MX359652B_D0005.tif" />
MEDIA PROCESSING NODES ** '
CROSS REFERENCE TO RELATED REQUESTS AND
CLAIM OF PRIORITY
This application claims priority of US Provisional Patent Application Serial No. 61 / 419,747, filed on December 3, 2010 and US Provisional Patent Application Serial No. 61 / 558,286, filed on November 10, 2011 , both of which are incorporated herein by reference for all purposes.
TECHNOLOGY
The present invention relates generally to media processing systems, and in particular to adaptively processing media data, based on media processing states of media data.
BACKGROUND
Media processing units typically operate in a blind manner and pay no attention to the history of media data processing that occurs before the media data is received. This can work in a media processing structure where a single entity does all the media processing and encoding for a variety of target media rendering devices, while one target media rendering device does all the decoding and rendering
IMPI
INSTITUTE Μ «ΚΑΝΟ OF THE INDUSTRIAL COMPANY
<img file="MX359652B_D0006.tif" />
of the encoded media data. However, this blind processing does not work well (or entirely) in situations where a plurality of media processing units are dispersed across a diverse network or placed in tandem (i.e. chain) and are expected to perform smoothly. Optimum their respective types of media processing. For example, some media data may be encoded for high-performance media systems and may have to be converted to a reduced form, suitable for a mobile device on a media processing chain. Accordingly, a media processing unit may unnecessarily perform a type of processing on the media data that has already been performed. For example, a volume leveling unit performs processing on an input audio trim, regardless of whether or not volume leveling has previously been performed on the audio data trim. As a result, the volume leveling unit performs leveling, even when it is not necessary. This unnecessary processing can also cause degradation and / or deletion of specific features while rendering media content in media data.
The approaches' or approaches described in this section are those that can be sought, but not necessarily approaches that have been previously conceived or
IMPI
ΚΦΠτυΤΟ MEXICAN iNVUynUAL
<img file="MX359652B_D0007.tif" />
Wanted. Therefore, unless <; p._ inriigiip do otu forma, none of the approaches described in this section should be considered to qualify as prior art simply by virtue of their inclusion in this section. Similarly, aspects identified with respect to one or more approaches should not be considered to have been recognized in any prior art based on this section, unless otherwise indicated.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention is illustrated by way of example, and not by way of limitation, in the accompanying drawing figures and where like reference numbers refer to like elements and wherein:
FIGURE 1 illustrates an exemplary media processing chain in accordance with some possible embodiments of the present invention;
FIGURE 2 illustrates an exemplary improved media processing chain, in accordance with some possible embodiments of the present invention;
FIGURE '3 illustrates an exemplary encoder / transcoder in accordance with some possible embodiments of the present invention;
FIGURE 4 illustrates an exemplary decoder in accordance with some possible embodiments of the present invention;
<img file="MX359652B_D0008.tif" />
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL MONEDAD
<img file="MX359652B_D0009.tif" />
<td>The</td><td>FIGURE 5</td><td>illustrates a</td><td>post unit</td>
<td>processing</td><td>copy,</td><td>in accordance with</td><td>some modalities</td>
<td>possible of the</td><td colspan="2">present invention;</td><td></td>
FIGURE 6 illustrates an exemplary implementation of an encoder / transcoder, in accordance with some possible embodiments of the present invention;
FIGURE 7 illustrates an example of an evolution decoder, which controls modes of operation of a volume leveling unit based on the validity of noise metadata in and / or associated with processing status metadata, according to some possible modalities. of the present invention; ,
FIGURE 8 illustrates exemplary configuration of using data concealment to pass processing information, in accordance with some possible embodiments of the present invention;
FIGURE 9A and FIGURE 9B illustrate exemplary processing flows in accordance with a possible embodiment of the present invention;
FIGURE 10 illustrates an exemplary hardware platform in which a computer or computing device as described herein can be implemented, in accordance with a possible embodiment of the present invention;
FIGURE 11 illustrates frames of media with which processing status metadata associated with
IMPI
<img file="MX359652B_D0010.tif" />
media data in the med-ius tables, jJLWUén to be transmitted according to an exemplary modality; and
FIGURES 12A through 12L-2 illustrate block diagrams of some exemplary media processing nodes / devices in accordance with some embodiments of the present invention.
DESCRIPTION OF POSSIBLE EXEMPLARY MODALITIES
Possible exemplary modalities, which refer to adaptive processing of media data based on media processing states of the media data, are described herein. In the following description, for purposes of explanation, numerous specific details are set forth to provide a complete understanding of the present invention. It will be apparent, however, that the present invention can be practiced without these specific details. In other cases, well known structures and devices are not described in exhaustive detail, in order to unnecessarily avoid occlusion, darkening or obfuscation of the present invention.
Exemplary modalities are described here according to the following profile:
one. GENERAL DESCRIPTION
2. MEDIA PROCESSING CHAINS
3. MEDIA PROCESSING DEVICES OR UNITS
Four. EXAMPLE ADAPTIVE PROCESSING OF MEDIA DATA
<img file="MX359652B_D0011.tif" />
5. DATA HIDING
6. EXEMPLARY PROCESS FLOW
7. IMPLEMENTATION MECHANISMS
PHYSICAL
EQUIPMENT OVERVIEW
8. LISTED EXEMPLARY MODALITIES
9. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS
one. GENERAL DESCRIPTION
This generality presents a basic description of some aspects of a possible embodiment of the present invention. It should be noted that this generality is not an extensive or exhaustive compendium of aspects of the possible modality. Furthermore, it should be noted that this generality is not intended to be understood as identifying any particularly significant aspects or elements of the possible embodiment, nor as delineating any scope of the possible embodiment in particular, or the invention in general.
This
<td>generality</td><td>it just presents some concepts that</td>
<td>refer to</td><td>exemplary possible modality in a format</td>
<td>condensed</td><td>and simplified and will have to be understood</td>
simply as a conceptual prelude to a more detailed description of possible exemplary modalities given below.
Techniques for adaptive processing of media data based on media processing states of media data are described. In some modalities
IMPI
<img file="MX359652B_D0012.tif" />
Possible medlTTb processing units in an enhanced media processing gadiÍi'it * are automatically enabled to retrieve and validate media status metadata and / or media processing signaling, determine the status of media data, with based on the processing status metadata and / or media processing signaling, tailoring their respective processing based on the status of the media data. Media processing units in the enhanced media processing chain may include, but are not limited to encoders, transcoders, decoders, pre-processing units, 'post-processing units, bitstream processing tools, codees of the Advanced Television Systems Committee (ATSC), codes of the Moving Picture Experts Group (MPEG), etc. A media processing unit can be a media processing system or a part of the media processing system.
As used herein, the term processing status metadata refers to separate and different data from media data, whereas media data (i.e. video frames, perceptually encoded audio frames or PCM audio samples that have media content) refer to sample data from
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0013.tif" />
media that represent media content and are used to represent media content as audio or video output. Processing status metadata is associated with the media data and specifies what type of processing has already been performed on the media data.
This association of the processing state metadata with the media data is synchronous in time. In this way, the present processing status metadata
<td colspan="2">indicate that the present data</td><td>of</td><td>media</td><td colspan="2">understand in form</td>
<td>contemporary</td><td>results</td><td>of</td><td>the</td><td>indicated types</td><td>of</td>
<td>processing of</td><td>media</td><td>me</td><td>a</td><td>description</td><td>of</td>
media characteristics in media data. In some possible embodiments, processing status metadata may include processing history and / or some, or all, of the parameters that are used in and / or derived from the indicated types of media processing. Additionally and / or optionally, the processing status metadata may include media characteristics of one or more different types calculated / extracted from the media data. Media characteristics as described here provide a semantic description of the media data and may comprise one or more structural properties, tonality including harmony and melody, timbre, rhythm, reference volume, stereo mix, or a number of source sources. sound of media data, absence or
IMPI
<img file="MX359652B_D0014.tif" />
voice presence, repeating characteristics, melody, harmonies, lyrics, timbre, perceptual characteristics, digital media characteristics, stereo parameters, voice recognition (for example, what a caller is saying), etc. / Processing status metadata may also include other metadata that is not related to or derived from any media data processing. For example, third party data, tracking information, identifiers, proprietary or standard information, user annotation data, user preference data, may be added to a particular media processing unit to pass to other data processing units. media.
These independent types of metadata can be distributed to or from, validated, and used by a media processing component in the media processing chain. The term "media processing signaling" refers to relatively light control or status data (which may be small in volume relative to the processing status metadata) that is communicated between media processing units in a stream. media bits. The media processing signaling may comprise a subset or a summary of processing status metadata.
Processing status metadata and / or
IMPI tNSTmn »μεχιΛνο OF INDUST1UAL PROPERTY
<img file="MX359652B_D0015.tif" />
Media processing signaling may be embedded in one or more reserved fields (eg, which may be, but are not limited to, currently unused), carried in a substream in a stream of media bits, hidden with data media or provided with a separate media processing database ^ J In some possible embodiments, the volume of processing status metadata noise and / or media processing signaling, they can be small enough to be transported (for example, in reserved fields, or hidden in media samples using reversible data concealment techniques, or store detailed processing status information in an external database, while fingerprints or digital media identification codes of media data or retrieve media fingerprints from media data, etc.), without affecting the bit rate assigned to transport the media data. Communicating processing status metadata and / or media processing signaling in an improved media processing chain is particularly useful when two or more media processing units require to work in tandem with each other through the media processing chain (or content life cycle). No processing status metadata and / or media processing signaling, probably
<img file="MX359652B_D0016.tif" />
IMPI
MEXICAN INSTITUTE
DtLAHOHIOAO IKUCTUÍAL »
Severe media processing issues such as quality, level, and spatial degradation can occur, for example when two or more audio codecs are used in the chain and single-ended volume leveling is applied more than once during the workday. media content to a media consuming device (or a point of representation of media content in media data).
In contrast, the present techniques elevate the intelligence of any or all of the media processing units in an enhanced media processing chain (content life cycle). Under the present techniques, · any of the media processing units can both listen and adapt as well as announce the status of the media data to downstream media processing units. In this way, under the present techniques, a downstream media processing unit can optimize its media data processing, based on past knowledge of media data processing, as performed by one or more units. upstream media processing. Under the present techniques, media processing by the media processing chain as a whole in the media data becomes more efficient, more adaptive, and more predictable than otherwise. How
<img file="MX359652B_D0017.tif" />
i
<img file="MX359652B_D0018.tif" />
IMPI
MSX1CANO INSTITUTE
DlUnOFllOAD INDUSTRIAL result, total rendering and handling of media content in media data is greatly improved.
Importantly, under the present techniques, the presence of the status of the media data as indicated by the processing status metadata and / or media processing signaling, it does not negatively impact legacy or legacy media processing units that may be present in the enhanced media processing chain and may not proactively use the status of · media data to adaptively process media data . Furthermore, even if a legacy media processing unit in the media processing chain may have a tendency to mishandle with the processing results of other upstream media processing devices, the processing status metadata here may pass in securely to downstream media processing devices through secure communication methods that make use of · cryptographic values, encryption, authentication and data concealment. Examples of data concealment include both reversible and irreversible data concealment.
In some possible embodiments, in order to convey a state of media data to downstream media processing units, the techniques
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0019.tif" />
Presents wrap and / or embed one or more processing subunits in the forms of hardware, hardware, or both in a media processing unit to allow the media processing unit to read, write, and / or validate metadata from processing status supplied with the media data.
In some possible embodiments, a media processing unit (eg encoder, decoder, leveler, etc.) may receive data from media on which one or more types of media processing have previously been performed yet: 1) do not exist Processing status metadata to indicate these types of previously performed media processing and / or 2) Processing status metadata may be incorrect or incomplete. The types of media processing that were previously performed include operations (eg volume leveling) that can alter media samples as well as operations (eg fingerprint extraction and / or feature extraction based on media samples), which they may not alter the media samples. The media processing unit can be configured to automatically create correct processing status metadata that reflects the actual status of the media data and associate this status of the media data with the media data by communicating the metadata from safaras. 'Λ ··' · »''
<img file="MX359652B_D0020.tif" />
processing status created to one or tháq -units — 35<sup>1 </sup>downstream media processing. Furthermore, the association of the media data and the processing status metadata can be performed such that a resulting media bitstream is backward compatible with legacy media processing units such as legacy decoders. As a result, legacy decoders that do not implement the present techniques may still be able to decode the media data correctly as the legacy decoders are designed to do, while ignoring the associated processing status metadata indicating the status of the data. media. 'In some possible modalities, the media processing unit here can be configured concurrently with an ability to validate the processing status metadata with the media (source) data by forensic analysis and / or validation of one or more hash values embedded (for example, signatures).
Under the techniques described here, adaptive processing of the media data based on a contemporary state of the media data as indicated by the received processing status metadata 'can be performed at various points in a media processing chain. For example, if the noise volume metadata in the processing status metadata is valid,
IMPI
<img file="MX359652B_D0021.tif" />
then a volume release unit subsequent to a decoder can be notified by the decoder with processing status metadata and / or media processing signaling, such that the volume release unit can pass the media data such as audio without change ^ J
In some embodiments, processing status metadata includes media characteristics that are extracted from underlying media samples. Media features can provide a semantic description of the media samples and can be provided as part of the processing status metadata to indicate, for example if the media samples comprise speech, music, if someone is singing in quiet or noisy conditions , if the song is about a talking crowd, if a dialogue occurs, if it is spoken on a background with noise, a combination of two or more of the above, etc. Adaptive media data processing can be performed at various points in a media processing chain, based on the description of the media characteristics contained in the media status metadata.
Under techniques described here, processing state metadata embedded in a media bitstream with media data can be authenticated and validated. For example, the present techniques may be
IMPI
<img file="MX359652B_D0022.tif" />
Useful for noise regulatory entities - to vorify or noise a particular program is already within a specified range and that the media data itself has not been altered (thus ensuring compliance with regulations). A noise value includes a data block that comprises the processing status metadata can be read to verify this, instead of recalculating the noise.
Under the techniques described here, a data block comprising processing status metadata may include additional reserved bytes to transport third-party metadata securely. This feature can be used to allow a variety of applications. For example, a rating agency (for example, Nielsen Media Research) may choose to include a content identification tag that can then be used to identify a particular program that is viewed or heard for the purpose of calculating computational rates, statistics of spectator levels and auditorium levels.
Significantly, the techniques described above, and variations of the techniques described here, can ensure that the processing status metadata associated with the media data is preserved throughout the media processing chain from content creation to consumption.
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0023.tif" />
content.
In some possible embodiments, mechanisms such as those described herein form a part of a media processing system, including but not limited to a portable device, game machine, television, laptop, netbook computer, cellular radiotelephone, electronic book reader, terminal point of sale, desktop computer, computer workstation, computer kiosk, and various other types of terminals media processing units.
Various modifications, preferred embodiments, and the principles and characteristics described will be readily apparent to those skilled in the art as generic herein. Thus, the description is not intended to be limited to the modalities shown, if it should not be given the widest scope consistent with the principles and characteristics described here.
2. MEDIA PROCESSING CHAINS
Figure 1 illustrates an exemplary media processing chain in accordance with some possible embodiments of the present invention. The media processing chain may, but is not limited to, comprises encoders, decoders, pre / post processing units, transcoders, and metadata correction units and
IMPI MEXICAN INSTITUTE PE THE INDUSTRIAL NEWS
<img file="MX359652B_D0024.tif" />
signal analysis. These units in the media processing chain can be comprised in the same system or in different systems. In modalities where the media processing chain spans across multiple different systems, these systems can be geocallocated or geographically distributed.
In some possible embodiments, a preprocessing unit of Figure 1 can accept PCM (time domain) samples that comprise media contents such as PCM input and output processed samples. An encoder can accept PCM samples as input and output an encoded (eg compressed) media stream of the media content.
As used herein, data (eg carried in a main stream of bitstream) comprising the media content is referred to as media data, while data separate from the media data indicating types of processing performed in media data at any given point in the media processing chain is referred to as processing status metadata.
A Metadata and Signal Analysis correction unit can accept one or more encoded media bit streams as power or input and validate whether the processing status metadata included in the
<img file="MX359652B_D0025.tif" />
IMPI
MEXICAN INSTITUTE
IA PROPERTY iKDUSTtlAL
<img file="MX359652B_D0026.tif" />
encoded media bit streams are —C'üil ecLU's—. perform signal analysis. If the signal analysis metadata correction unit finds that the included metadata is invalid, the signal analysis and metadata correction unit replaces the incorrect value with the correct value obtained from the signal analysis.
A transcoder can accept media bit streams as input and output a modified media bit stream. A decoder can accept compressed media bit streams as input and output a stream of decoded PCM samples as output. A post-processing unit can accept a stream of decoded PCM samples, perform any post-processing such as volume leveling of the media content there, and render the media content in the decoded PCM samples on one or more speakers and / or panels. exhibition. All media processing units may not be able to tailor their processing to apply to the media data using 'processing state' metadata.
Techniques are provided here which provide an improved media processing chain wherein media processing units such as encoders, decoders, transcoders, pre and post processing units, etc., adapt their respective processing.
<img file="MX359652B_D0027.tif" />
IMPI '“-SE to apply to media data of arn ^ rdo c.nn ¿L .h'Htt contemporary of the media data as indicated by the processing status metadata and / or signaling of received media processing respectively by these media processing units.
The Figure illustrates an exemplary improved media processing chain comprising encoders, decoders, pre / post processing units, transcoders, and metadata correction and signal analysis units, in accordance with some possible embodiments of the present invention. In order to adapt the processing of the media data based on the status of the media data, some or all of the units in Figure 2 may be modified. In some possible embodiments, each of the media processing units in the exemplary enhanced media processing chain is configured to cooperatively perform non-redundant media processing and avoid unnecessary and erroneous repetition of processing that has been performed. per units upstream. In some possible embodiments, the state of the media data at any point in the enhanced media processing chain from content creation to content consumption is understood by a current media processing unit at that point in the content chain. improved media processing.
IMPI
<img file="MX359652B_D0028.tif" />
• ·
3. MEDIA PROCESSING DEVICES OR UNITS
Figure 3 illustrates an exemplary (modified) encoder / transcoder in accordance with some possible embodiments of the present invention. Unlike the encoders of Figure 1, the encoder / transcoder of Figure 3 can be configured to receive processing status metadata associated with power supply data and to determine pre-processing (pre / post-) performed by a or more units upstream of the encoder / transcoder or power media data (eg input audio), that the modified encoder / transcoder logically receives from an upstream unit (eg the last upstream unit that has performed its processing on the input audio).
As used herein, the term receive logically may mean that an intermediate unit may or may not be involved in communicating the input media data from an upstream unit (eg the last upstream unit) to a recipient unit, such as the encoder / transcoder unit in the present example.
In one example, the upstream unit that performs pre / post processing on the input or feed media data may be in a different system than the system of which the recipient unit is part. The data
<img file="MX359652B_D0029.tif" />
IMPI MEXICAN INSTITUTE OF INDUSTRIAL FBOPTEDAD
<img file="MX359652B_D0030.tif" />
Input media can be a stream of media bits that is sent out by the upstream unit and communicated through an intermediate transmission unit such as a network connection, a USB, a wide area network connection , a wireless connection, an optical connection, etc.
In another example, the upstream unit that performs pre / post processing on the input media data may be in the same system that a recipient unit is part of. The input media data can be output from the upstream unit and communicated through an internal connection through one or more internal system units. For example, data can be physically supplied through an internal busbar, a crossbar connection, a series connection, etc. In any case, under the present techniques, the receiving unit can logically receive the input media data from the upstream unit.
In some possible embodiments, the encoder / transcoder is configured to create or modify processing status metadata associated with the media data, which may be a revision of the input media data. New or changed processing status metadata created or modified by
IMPI MEXICAN INSTITUTE £> £ INDLISTRIAL PROPERTY
<img file="MX359652B_D0031.tif" />
the encoder / transcoder can capture an. F<sub>or rma</sub>automatically and accurately states the status of the media data to be output by the encoder / transcoder later in the media processing chain. For example, the processing status metadata may or may not include some processing (eg Dolby volume, mixed, commercially available from Dolby Laboratories) performed on the media data.
(~ Additionally and / or optionally, the processing status metadata can include the parameters used in and / or derived from certain processing operations or any constituent in the processing.
Additionally and / or optionally, the processing status metadata may include one or more footprints calculated / extracted from the media data. Additionally and / or optionally, the processing status metadata may include media characteristics of one or more different types calculated / extracted from the media data. Media characteristics as described here provide a semantic description of the media data and may comprise one or more structural properties, key including harmony and melody, timbre, rhythm, reference noise, stereo mix, or a number of sound sources of media data, absence or presence of voice, repetition characteristics,
IMPI
MEXICAN INSTITUTE OF THE FSOMEDa ΙΜΗβηίΑΙ
<img file="MX359652B_D0032.tif" />
melody, harmonies, lyrics, timbre, perceptual characteristics, digital media characteristics, stereo parameters, voice recognition (for example, what a caller says), etc.i In some modes, the extracted media characteristics are used to classify data of underlying media in one or more of a plurality of kinds of media data. The one or more media data classes may include, but are not limited to any one single 'total / dominant class (eg, a class type) for the entire media piece and / or a single class representing a smallest period of time (for example, a class sub-type for a whole part subset / subinterval) such as a single media box, a media data block, multiple media data, multiple blocks of media data, a fraction of a second, one second, multiple seconds, etc. For example, a class tag can be computed and inserted into the bitstream and / or hidden (using reversible or irreversible data concealment techniques) every 32 msec of the bitstream. A class label can be used to indicate one or more class types and / or one or more class sub-types. In a media data box, the class tag can be inserted into a preceding metadata structure, or alternatively follows a block of media data with which the class tag is
INSTITUTO MEXICANO DE LANIOS industrial has associated, as illustrated in FIGURE 11. Media classes may include, but are not limited to, any one-of-a-kind types such as music, speech, noise, silence, applause. A media processing device as described herein can also be configured to classify media data comprising mixes of media class types such as talking about music, etc. Additionally, alternately and optionally, a media processing device as described here can be configured to carry an independent probability or probability value for a media class type or sub-type indicated by a calculated media class label. . One or more of these probability values can be transmitted with the media class tag in the same metadata structure. A probability value indicates the level of confidence that a calculated media class label has in relation to the media block / segment for which the media class type or sub-type is indicated by the calculated media class label. The one or more probability values in combination with associated middle class tag can be used by a recipient media processing device to adapt to media processing in a way to enhance any wide variety of operations throughout an entire processing chain. means such as mixing, encoding,
IMPI
<img file="MX359652B_D0033.tif" />
decoding, transcoding, vi rtnai -¡x<sub>n</sub>--- headphones, etc. Processing status metadata can include, but is not limited to any of the media class types or subtypes, probability or probability values. Additionally, optionally or alternately, instead of passing media class types / sub-types and likelihood / probability values in a metadata structure, inserted between media (audio) data blocks, some or all types / Media class sub-types and probability values can be embedded and passed to a recipient media processing node / device in media data (or samples) as hidden metadata. In some embodiments, the content analysis results of the media data included in the processing status metadata may comprise one or more indications of whether certain system-defined or user- defined keywords are mentioned in any time slot. the media data. Such indications may be used by one or more applications to trigger the performance of related operations (for example, to present contextual ads for products and services related to keywords).
In some embodiments, while media data is processed with a first processor, a device as described here may run a second processor in
IMPI
<img file="MX359652B_D0034.tif" />
parallel to classify / extract media features from media data. Media features can be extracted from a segment that lasts for a period of time (one frame, multiple frames, one second, multiple seconds, one minute, multiple minutes, a user-defined period of time, etc.), or alternately for a scene (based on characteristic detectable signal changes). Media characteristics as described by the processing status metadata can be used throughout the entire media processing chain. A downstream device can tailor its own media processing of the media data, based on one or more of the media characteristics. Alternately, a downstream device may select to ignore the presence of any or all of the media characteristics as described in the processing state metadata.
An application on a device in the media processing chain can influence media characteristics in one or more of a variety of ways. For example, this application can index the underlying media data using the media features. For a user who wants to go to the sections where judges talk about performance, the app can skip other preceding sections. Media characteristics as described in the processing status metadata
<img file="MX359652B_D0035.tif" />
I
INSTITUTO MEXICANO 0ΪΙΛ «Ο« ΙΟ * θ INCUSTUAL
<img file="MX359652B_D0036.tif" />
<img file="MX359652B_D0037.tif" />
<img file="MX359652B_D0038.tif" />
they provide contextual device information downstream of the media data as an intrinsic part of the media data.
More than one device in the media processing chain can perform analyzes to extract media features from media data content. This allows downstream devices to not have to analyze the content of the media data.
In some possible embodiment, the modified or generated processing state metadata may be transmitted as part of a media bit stream (eg audio bit stream with metadata in the audio state) and quantity at a transmission rate in the 3-10 kbps order.
In some embodiments, the processing status metadata can be transmitted within the media data (eg, PCM media samples) based on data concealment.
A wide variety of data concealment techniques, which can alter media data reversibly or irreversibly, can be used to hide part, or all, of the processing status metadata (including but not limited to data related to authentication) in the media samples. Data concealment can be implemented with imperceptible or perceivable secure communication channel. Data concealment can
IMPI
<img file="MX359652B_D0039.tif" />
achieved by altering / manipulating / modulating signal characteristics (phase and / or amplitude in a frequency or time domain) of a signal in the underlying media samples. Data concealment can be implemented based on FSK, spread spectrum, or other available methods.
In some possible embodiments, a pre / post processing unit 'can perform media data processing in a cooperative manner with the encoder / transcoder. Processing performed by cooperating pre-post processing unit is also specified in the processing status metadata that is communicated (eg, by the audio bit stream) to a downstream media processing unit.
In some possible ways, once a piece of processing status metadata (which may include media footprints and any parameters used in or derived from one or more types of media processing) is derived, this piece of processing status metadata Processing can be retained by the media processing units in the media processing chain and communicated to all downstream units. In this way, in some possible modalities, a piece of processing status metadata can be created by the first media processing unit and passed to the last media processing unit - as data embedded within a
<img file="MX359652B_D0040.tif" />
INSTITUTO ΜΗΚΛΝΟ delamedao t ^ DUSTWAL
<img file="MX359652B_D0041.tif" />
bit stream / media stream · σ ~ υυπισ dátóá '^ uesé derive from an external data source or media processing database, in the media processing chain (full life cycle).
FIGURE 4 illustrates an exemplary decoder (eg, an evolution decoder implementing the present according to some possible embodiments of the present invention.
A decoder in possible embodiments of the present invention can be configured (1) to analyze and validate the processing status metadata (eg, a processing history, a description of media characteristics, etc.) associated with metadata data (eg example, data entry and other data processing independent of any media such as third party data, tracking information, identifiers, property or standard information, user annotation data, user preference data, etc.) that has been entered, and (2) to determine, based on validated processing status metadata, the processing status of media of the media data. For example, when parsing and validating processing state metadata in a media bitstream (for example, audio bitstream with metadata in audio state) that transports input media data and
IMPI
<img file="MX359652B_D0042.tif" />
Processing status metadata, the decoder can determine that the .noise metadata (or media characteristics metadata) is valid and reliable, and was created by one of an improved content provider subunits that implements the techniques described here (eg, Dolby Media Generator (DMG = commercially available from Dolby Laboratories). In some possible embodiments, in response to determining that the received processing status metadata is valid and reliable, the 'decoder may then be configured to generate, based on at least part of the received processing status metadata, processing signaling of media regarding the status of media data using a reversible or irreversible data concealment technique. The decoder can then be configured to provide the media processing signaling to a downstream media processing unit (eg, a post-processing unit) in the media processing chain. This type of signaling can be used, for example when there is no dedicated (and synchronous) metadata path between the decoder and the downstream media processing unit. This situation may arise in some possible modes where the decoder and the downstream media processing unit exist as
IMPI
MEXICAN INSTITUTE OF INDUS1KIAL PROPERTY
<img file="MX359652B_D0043.tif" />
Separate entities in an éTé'CTÍóñico '' ct'e consumer device (for example, PCs, mobile phones, connection boxes, audio video recorders, or in a different subsystem or different systems where the synchronous control and data path between the decoder and subsequent processing unit are not available. In some possible embodiments, media processing signaling under the data concealment technique here can be transmitted as part of a quantity and media bitstream at a transmission rate on the order of 16 bps. A wide variety of data concealment techniques, which can reversibly or irreversibly alter media data, can be employed to conceal some or all of the processing status metadata in media samples, including but not limited to any of, perceptible or imperceptible secure communication channels, alterations / manipulations / modulations of narrow-spectrum or broad-spectrum signal characteristics (phase and / or amplitude in a frequency or time domain) of one or more signals in the underlying media samples, or other available methods.
In some possible modes, the decoder may not attempt to pass all received processing status metadata; conversely, the decoder can
<img file="MX359652B_D0044.tif" />
MEXICAN INSTITUTE OE EA INDUSTRIAL PROPERTY
<img file="MX359652B_D0045.tif" />
just embed enough fpOf 'éJ'éWgTcg information within the hiding capacity limits of to change the mode of operation of the downstream media processing unit, based on the status of the media data.
In some possible embodiments, redundancy in audio or video signal can be exploited in the media data to convey the state of the media data. In some possible modes, without causing any audible or visible artifacts, some or all of the processing status metadata and / or media processing signaling may be hidden in the least significant bits (LSBs = Least Significant Bits) of a plurality of bytes in the media data or hide in a secure communication channel carried within the media data. The plurality of bytes can be selected based on one or more criteria factors including whether LSBs can cause perceptible or visible artifacts when media samples with hidden data are represented by a legacy or older media processing unit. Other data concealment techniques (for example, noticeable or imperceptible secure communication channels, FSK-based data concealment techniques, etc.), which can alter media data in a reversible or irreversible way, can be used to hide a part wave
<img file="MX359652B_D0046.tif" />
IKJTÍTUTOMíWCAWO
OEUMMUtM * '«NDLSTttót.
all of the processing status metadata in the media samples.
In some possible embodiments, data concealment technology may be optional and may not be required, for example, if the downstream media processing unit is implemented as part of the decoder. For example, two or more media processing units may share a busbar and other communication mechanisms that allow metadata to be passed as out-of-band signals without hiding data in the media samples from one processing unit. media.
FIGURE 5 illustrates an example of a post-processing unit (eg, a Dolby evolution post-processing unit), in accordance with some possible embodiments of the present invention. The post-processing unit can be configured to first extract the media processing signaling hidden in the media data (eg, PCM audio samples with embedded information) to determine the status of the media data as indicated by the processing signaling media. This can be done, for example, with an auxiliary processing unit (for example, an audio restoration and information extraction sub-unit in some possible modalities where the media data comprises
<img file="MX359652B_D0047.tif" />
Audio). In modes where media processing signaling is hidden using one. reversible data concealment technique, previous modifications made to the media data by the data concealment technique (eg, the decoder) to embed the media processing signaling can be undone. In embodiments where media processing signaling is hidden using an irreversible data concealment technique, prior modifications made to the media data by the data concealment technique - (eg, the decoder) to embed the processing signaling media may not be completely undone but rather side effects on media rendering quality can be minimized (eg, minimal audio or visual artifacts). Subsequently, based on the status of the media data as indicated by the media processing signaling, the post processing unit can be configured to tailor its processing to be applied to the media data. In one example, volume processing may be disabled in response to a determination (from the media processing signaling) that the noise metadata was valid and that volume processing was performed by an upstream unit. In another example, a contextual message or advertisement
<img file="MX359652B_D0048.tif" />
IMPI
INÍTtTlrtO MEXICANO DE LA r «<On« OAO IHDUSniM.
<td>can be presented or activated</td><td>by a</td><td colspan="3">keyword</td>
<td>recognized by voice.</td><td></td><td></td><td></td><td></td>
<td>In some modalities</td><td>possible,</td><td>a</td><td>Unit</td><td>of</td>
<td>metadata correction and analysis</td><td>signal</td><td>in a</td><td>system</td><td>of</td>
Media processing described here can be configured to accept encoded media bit streams as feed and validate whether the metadata embedded in a media bit stream is correct when performing signal analysis. After validating that the embedded metadata is or is not valid within the media bitstream, correction can be applied on a base as required basis.
In some possible embodiments, the signal analysis and metadata correction unit can be configured to perform analysis on media data or samples encoded on the power media streams in time and / or frequency domain (s) to determine media characteristics of the media data.
After determining the media characteristics, processing status metadata (eg a descriptor of one or more media characteristics) can be generated and provided to downstream devices relative to the signal analysis and metadata correction unit. In some possible embodiments, the signal analysis and metadata correction unit may be integrated with one or more other media processing units in
IMPI MEXICAN INSTITUTE OF PROPERTY INBUSTRIAL
<img file="MX359652B_D0049.tif" />
one or more media processing systems.
Additionally and / or optionally, the signal analysis and metadata correction unit can be configured to hide media processing signaling in the media data and to signal to a downstream unit that the metadata embedded in the media data is valid and have been successfully verified. In some possible embodiments, the signaling data and / or processing status metadata associated with the media data can be generated and inserted into a compressed media bitstream that carries the media data.
Therefore, techniques as described here ensure that different processing blocks or media processing units in an improved media processing chain (eg encoders, transcoders, decoders, pre / post processing units, etc.), they are able to determine the status of the media data. Therefore, each of the media processing units can tailor its processing according to the state of the media data as indicated by upstream units. Furthermore, one or more units for reversible or irreversible data concealment can be employed to ensure that signal information regarding the status of the media data can be provided to units.
IMPI
<img file="MX359652B_D0050.tif" />
downstream media processing units in an efficient manner with the minimum amount of bit rate required to transmit the signal information to the downstream media processing units. This is especially useful when there is no metadata path between an upstream unit such as a decoder and a downstream unit such as a post-processing unit, for example when the post-processing unit is not part of the decoder.
In some possible embodiments, an encoder may be enhanced with or may comprise a pre-processing and metadata validation subunit. In some possible embodiments, the pre-processing and metadata validation subunit can be configured to ensure that the encoder performs adaptive processing of media data based on the status of the media data as indicated by the processing status metadata. and / or signaling media processing. In some possible embodiments, through the preprocessing and metadata validation subunit, the encoder can be configured to validate the processing status metadata associated with (eg embedded in a media bitstream with) the data from media. For example, if metadata is validated to be trustworthy, then results from a media processing type
ΙΜΡΙ ^>
mexican institute
OF THE INDUSTRIAL RROHETY performed can be reused and subsequent performance of the media processing type can be avoided. On the other hand, if the metadata are mishandled, then the type of media processing intentionally performed previously can be repeated by the encoder. In some possible embodiments, additional types of media processing may be performed by the encoder on the metadata once the processing status metadata (including footprint-based metadata retrieval and media processing signaling) is found to be unreliable.
If the processing status metadata is determined to be valid (for example based on a mapping of a cryptographic value being extracted and a cryptographic reference value), the encoder can also be configured to point to other downstream media processing units in an improved media processing string, that the processing status metadata, for example present in the media bitstream, is valid. Any, some or all of a variety of approaches can be implemented by the encoder.
Under a first approach, the encoder can insert a flag into an encoded media bitstream (eg an evolution flag) to
IMPI
<img file="MX359652B_D0051.tif" />
indicate that validation of the metadata of this processing has already been performed on this encoded media bitstream. The flag can be inserted such that the presence of the flag does not affect a legacy media processing unit such as a decoder that is not configured to process and make use of processing status metadata as described here. In an exemplary embodiment, an Audio-3 Compression encoder (AC-3 = Audio Compression-3) can be enhanced with a pre-processing and metadata validation subunit to fit an evolution flag in the xbsi2 fields of a stream AC-3 media bits, as set forth in the ATSC specifications (eg ATSC A / 52b). This bit may be present in any encoded frame carried in the AC-3 media bit stream and may not be used. In some possible modes, the presence of this flag in the xbsi2 field does not affect already deployed legacy decoders that are not configured to process and use the processing status metadata as described here.
Under the first approach, there may be an aspect with information authentication in xbsi2 fields. For example, an upstream (eg malicious) drive may be able to activate the xbsi2 field without actually validating the processing status metadata and may point
IMPI “Incorrect tñjsasss to other units downstream '^ üe the processing status metadata is“ ”valid.
In order to resolve this aspect, some embodiments of the present invention may use a second approach. A secure data concealment method (including but not limited to any of a number of data concealment methods for creating a secure communications channel within the media data itself such as spread spectrum based methods, FSK based methods , and other methods based on secure communication channel, etc.) can be used to embed the evolution flag. This secure method is configured to prevent the evolution flag from being passed in plain text, thereby intentionally or accidentally being easily attacked by a unit or intruder. By contrast, under this second approach, a downstream drive can retrieve hidden data in an encrypted form. Through a decryption and authentication thread, the downstream unit can verify the correctness of the hidden data and trust the evolution flag on the hidden data. As a result, the downstream unit can determine that the processing status metadata in the media bitstream has been previously validated successfully. In various modalities, any portion of the processing status metadata such as flag of
IMPI
<img file="MX359652B_D0052.tif" />
evolution can be supplied by a device <sup>1 </sup>up to downstream devices in any one or more cryptographic methods (HMAC-based, or non-HMAC-based).
In some possible embodiments, media data may initially be simply legacy media bit streams, for example comprising PCM samples. However, once the media data is processed by one or more processing units as described herein, the processing status metadata generated by the one or more media processing units comprises the status of media data as well as Relatively detailed information (including but not limited to any one or more of certain media characteristics of the media data) that can be used to decode the media data. ^ In some possible embodiments, the generated processing status metadata may include media footprints such as media footprints, such as video footprints, noise metadata, dynamic range metadata, one or more hash-based message authentication codes (HMACs = Hash Message Authentication Codes), one or more dialog channels, audio footprints, enumerated processing history, audio noise, dialogue noise, actual peak values, Peak values displays and / or any user-specified metadata (3rd part). The
IMPI
<img file="MX359652B_D0053.tif" />
Processing status metadata can coifipr ehdéf tlft evolution data block.]
As used herein, the term "improved" refers to an ability for a media processing unit under techniques described herein to function in such a manner with other media processing units or other media processing systems under the techniques described herein, that they can perform adaptive processing based on the state of the media data as set by the upstream units. The term evolution refers to a capacity for media processing units under techniques described herein to operate in a manner compatible with legacy media processing units or legacy media processing systems as well as an ability for media processing units to under the present techniques work so that with other media processing units or other media processing systems under the techniques here described, perform adaptive processing based on the status of media data as set by upstream units.
In some possible embodiments, a media processing unit described herein may receive data from media on which one or more types of media processing have been performed, but there may be no metadata or
<img file="MX359652B_D0054.tif" />
ΐΝΐΤΙΤΙΓΓΟ MDüCMMC DE I * <sup>T #OF</sup>ET¿ INOUST ^ IAI insufficient metadata associated with the measurement data. to indicate one or more types of media processing. In some possible embodiments, this media processing unit may be configured to create processing status metadata to indicate media processing that units one or more types upstream have been performed by others relative to the media processing unit. Extracting features that have not been performed by upstream devices can also be performed and carried forward from processing status metadata to downstream devices. In some possible embodiments, the media processing unit (eg, an evolution encoder / transcoder) may comprise a media forensic analysis subunit. The media forensic sub unit such as an audio forensic sub unit can be configured to determine (without any metadata received) whether a certain type of processing has been performed on a piece of media content or on the media data. The analysis sub unit can be configured to search for specific signal processing artifacts / traces entered and remaining for a certain type of processing. The media forensic subunit can also be configured to determine whether a certain type of feature extraction has been performed on a piece of media content or on the media data.
IMPI
<img file="MX359652B_D0055.tif" />
The analysis sub-unit can configure cfl y'é to search for specific presence of characteristics based metadata. For the purpose of the present invention, the media forensic analysis sub unit as described herein, can be implemented by any media processing unit in a media processing chain. Furthermore, processing status metadata created by a media processing unit by the media forensic analysis sub-unit may be supplied here to a unit downstream in the media processing chain.
In some possible modalities, processing state metadata as described here may include additional reserved bytes to support third party applications. Additional reserved bytes can be secured by assigning a separate encryption key to randomize any simple text to be carried in one or more fields in the reserved bytes. Modalities of the present invention support novel applications including identification and content tracking. In one example, Nielsen-rated media can carry a unique identifier for a program in a media (medium) bit stream. Nielsen Ratings can then use that unique identifier to calculate audience level or audience level statistics for the program. In another example, the bytes reserved here can
IMPI
<img file="MX359652B_D0056.tif" />
transport keywords to search engines such as Google. Google can then associate ads based on keywords included in one or more fields in the reserved bytes that carry keywords. For the purpose of the present invention, in applications such as here discussed, the present techniques can be employed to ensure that reserved bytes are secured and not decrypted by anyone other than the third party designated to use one or more fields in the bytes. reserved.
Processing status metadata as described here can be associated with media data in any of a number of different ways. In some possible modalities ,. the processing status metadata can be inserted into the output compressed media bitstream that carries the media data. In some embodiments, the metadata is inserted in a way that maintains backward compatibility with legacy decoders that are not configured to perform adaptive processing based on the present processing state metadata.
Four. EXAMPLE ADAPTIVE PROCESSING OF MEDIA DATA
Figure 6 illustrates an exemplary implementation of an encoder / transcoder, in accordance with some possible embodiments of the present invention. Any of the illustrated components can be implemented as one or more
Κ ·
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0057.tif" />
processes and / or one or more IC circuits (including ASICs, FPGAs, etc.), in hardware, software, or a combination of hardware and software. The encoder / transcoder may comprise a number of legacy subunits such as Front Decoding (FED = Front-End Decode), Post Decoding (Full Mode) which does not select to perform Dialogue Standard Processing / Dynamic Range Control (DRC / Dialnorm) with based on whether this processing has already been performed, a DRC generator (DRC Gen), a rear-end encoding (BEE), a filler structure, a CRC regeneration unit, etc.
With these legacy sub units, the encoder / transcoder will be able to convert a bit stream (which for example can but is not limited to
AC-3) with another bit stream comprising results of one or more types of media processing (which for example may be but are not limited to E AC-3 with adaptive and automated noise processing).
However, media processing (eg, noise processing) can be performed regardless of whether noise processing has previously been performed and / or whether media data in the input bitstream comprises the results of this input bitstream noise. In this way, a previous and / or if processing status metadata is in
IMPI MEXICAN INSTITUTE OF MONEDAD INDUSTUAL
<img file="MX359652B_D0058.tif" />
encoder / transcoder with subunits' inherited alone will perform erroneous or unnecessary media processing.
Under the techniques described here, in some possible embodiments, as illustrated in Figure 6, the encoder / transcoder may comprise any one of a plurality of new subunits such as data analyzer / media validator (which for example may be but is not limited to an AC-3 flag parser and validator), accompanying media processing (eg real-time noise in adaptive transformation domain and dynamic range controller, signal analysis, feature extraction, etc.), media footprint generation (eg audio footprint generation), metadata generator (eg evolution data generator and / or other metadata generator), signaling insertion processing media (eg insert add_bsi or insert to auxiliary data fields), HMAC generator (which can digitally sign one or more, up to all tables to prevent tampering by malicious or legacy entities), one or more other types of cryptographic processing units, one or more switches that operate on the basis of processing status metadata and / or processing status signaling ( for example noise flag status received from validator and analyzer
<img file="MX359652B_D0059.tif" />
IMPI
INSTITUTO MKICANO w LA F »on» n * L> INBUSTUIAI for characteristics of the medium), etc.
user (for example target noise and / or other feed (for example from a video fingerprint application process) and / or other feed or metadata input (for example one or more flag or flags
Furthermore, user / dialnorm feed) third party data types, tracking information, identifiers, proprietary or standard information, user annotation data, user preference data, etc.) can be received by the encoder / transcoder.
As illustrated, measured dialogue, controlled and uncontrolled noise, and dynamic range values can also be inserted into evolution data generator. Other information related to media characteristics can also be injected into a processing unit as described here, to generate a portion of processing status metadata, j
In one or more of some possible modalities, processing status metadata as described here is carried out in the add_bsi fields specified in the Enhanced AC-3 (E AC-3) syntax in accordance with ATSC A / 52b, or in one or more auxiliary data fields in the media bitstream as described here. In some possible embodiments, dragging processing state metadata into these fields has an adverse impact on the size of the compressed media bitstream box and / or
<img file="MX359652B_D0060.tif" />
bit rate.
In some possible embodiments, processing state metadata can be included in a separate or dependent substream associated with a main program media bitstream. The advantage of this approach is that the allocated bit rate for encoding media data (ie performed by the main program media bit stream) is not affected. If the processing status metadata is carried out as a part of encoded frames, then the allocated bits for encoding audio information may be reduced such that the compressed media bitstream frame size and / or bit rate may be unchanged. For example, the processing status metadata may comprise a reduced data rate representation and occupy a low data rate in the order of 10 kbps to transmit between media processing units. Therefore, media data such as audio samples can be encoded at a slower rate by 10 kbps in order to accommodate the processing status metadata.
In some possible embodiments, at least a portion of the processing status metadata can be embedded with media data (or samples) by reversible or irreversible data concealment techniques. The advantage
IMPI
MEXICAN INSTITUTE
OF THE MONEDAD
INDUSTRIAL
<img file="MX359652B_D0061.tif" />
of this approach is that the media samples and metadata can be received by di.spos
Ltives downstream in the same bit stream.
In some possible modes, processing status metadata may be stored in a fingerprint-bound processing database. A downstream media processing unit to an upstream unit such as an encoder / transcoder that creates the processing status metadata, can create a footprint of received media data and then use the footprint as media processing. D query 'after the database of processing status data in the database is located, a data block comprising the processing status data associated with (or for) the received media data , can be retrieved from the media processing atos base and can be made available to the media processing unit implemented here, footprints may include downstream. As I know but are not limited to any one or more media footprints generated to indicate media characteristics.
possible ages, a data block
In some modal comprising processing state metadata it comprises a cryptographic hash (HMAC) for the processing state metadata and / or the underlying media data.
Since the data block is supposed to be digitally signed in
IMPI MEXICAN INSTITUTE OF INDUSTRY PROPERTY !.
<img file="MX359652B_D0062.tif" />
In these modalities, a downstream J.tfUChrpy-downstream process unit can relatively easily authenticate and validate processing state metadata. Other cryptographic methods including but not limited to any one or more of non-HMAC cryptographic methods may be employed for secure transmission and reception of the processing status metadata and / or the underlying media data.
As previously described, a media processing unit such as an encoder / transcoder as described here can be configured to accept legacy media bit streams and PCM samples. If the feed media bitstream is a legacy media bitstream, the media processing unit can check an evolution flag that can be in the media bitstream or hidden in the media data by one of the enhanced legacy encoders that comprise preprocessing and metadata validation logic as described above. In the absence of an evolution flag the encoder is configured to perform adaptive processing and generate processing status metadata as appropriate in an output media bitstream or in a data block comprising the processing status metadata. By
IMPI
<img file="MX359652B_D0063.tif" />
For example, as illustrated in Figure 6, an exemplary unit such as the transform noise and dynamic range domain real-time controller can adaptively process the audio content in the input media data that the unit receives. and automatically adjusts noise and dynamic range if an evolution flag is missing from the input media data or source media bit stream. Additionally, optionally or alternately, another unit can make use of feature-based metadata to perform adaptive processing.
In exemplary embodiments as illustrated in Figure 6, the encoder may be aware of the post / pre-processing unit that has performed a type of media processing · (eg noise domain processing) and therefore may create processing status metadata in a data block that includes the specific parameters used in and / or derived from noise domain processing. In some possible modalities, the encoder can create processing status metadata that reflects processing history on the content of the media data as long as the encoder is aware of the types of processing that has been performed (for example -processing of noise domain) in content in media data. In addition, optional or alternate, the encoder can perform
IMPI
<img file="MX359652B_D0064.tif" />
adaptive processing with base ΞΤΪ üñá <sup>1</sup> or —— TESTS media characteristics described by the processing status metadata. Additionally, optionally or alternately, the encoder may perform analysis of the media data to generate a description of media characteristics, as part of the processing status metadata to be provided to any of the other processing units.
In some possible embodiments, a decoder using the techniques mentioned here is capable of understanding the state of the media data in the following scenarios.
[Under a first scenario, if the decoder receives a media bitstream with the evolution flag set to indicate the validity of processing status metadata in the media bitstream, the decoder can parse and / or retrieve the metadata processing status and point to a downstream media processing unit such as an appropriate post-processing unit. On the other hand, if an evolution flag is absent, then the decoder can point the volume's processing unit
Γ leveling processing will still need downstream means of noise to be performed such as noise metadata - for example that would have been included
IMPI
<img file="MX359652B_D0065.tif" />
in the processing status metadata in some possible modalities, if the volume leveling processing has already been performed, it is absent or cannot be trusted as valid.
Under a second scenario, if the decoder receives a media bitstream generated and encoded by an upstream media processing unit such as an evolution encoder with cryptographic hash, then the decoder can parse and retrieve the cryptographic hash from a block data comprising processing state metadata, and using the cryptographic hash to validate the received media bit stream and associated metadata. For example, if the decoder finds the associated metadata (for example noise metadata in the processing status metadata) to be valid based on a match between a reference cryptographic hash and the cryptographic hash retrieved from the data block, then the decoder can point to the downstream media processing unit such as a volume leveling unit that passes media data such as audio without change. Additionally, optionally, or alternately, other types of cryptographic techniques can be used in place of a cryptographic hash-based method. Optionally or alternatively, operations other than volume leveling
IMPI
INSTITUTO MEXICANA) OE LA NiOPIEDAO INDUSTRIAL
<img file="MX359652B_D0066.tif" />
can also be done based on<sup>1</sup> 'one or more media characteristics of the media data as described in the processing status metadata.
Under a third scenario, if the decoder receives a media bit stream generated by an upstream media processing unit such as an evolution encoder, but a data block comprising processing status metadata is not included in the stream media bit; rather, the data block is stored in a media processing database. The decoder is configured to create a footprint of the media data in the media bitstream such as audio, and then use the footprint to query the media processing database. The media processing database may return the appropriate data block associated with the received media data based on footprint mapping. In some possible modalities, the encoded media bitstream contains a simple universal resource locator (URL = Universal Resource Locator) to direct the decoder to send the footprint based query as previously discussed to the media processing database .
In all of these scenarios, the decoder is configured to understand the state of the media and signal
IMPI
<img file="MX359652B_D0067.tif" />
a downstream media processing unit to adapt this latter processing of the media data in compliance. In some possible embodiments, the present media data may be re-encoded after being decoded. In some possible embodiments, a data block comprising contemporary processing status information corresponding to the recoding can be passed to a downstream media processing unit such as an encoder / converter subsequent to the decoder. For example, the data block may be included as associated metadata in the decoder output media bitstream.
Figure 7 illustrates an exemplary evolution decoder that controls modes of operation of a volume leveling unit based on the validity of noise metadata in and / or associated with processing status metadata in accordance with some possible embodiments herein. invention. Other operations such as feature-based processing can also be handled. Any of the illustrated components can be implemented as one more process and / or one or more IC circuits (including ASICs and FPGAs), in hardware, software, or a combination of hardware and software.
The decoder may comprise a number of legacy subunits such as information module of
IMPI
<img file="MX359652B_D0068.tif" />
frame (for example a frame information module in AC3, MPEG AAC, MPEG HE AAC, E AC-3, etc. ), a front decode (for example EDF in AC-3, MPEG AAC, MPEG HE AAC, E AC-3, etc.), synchronization and conversion (for example a synchronization and conversion module in AC-3, MPEG AAC, MPEG HE AAC, E AC-3, etc.), frameset buffer, post decoding (for example, a BED on AC-3, MPEG AAC, MPEG HE AAC, E AC-3, etc.), post encoded (for example, a BEE on AC-3, MPEG AAC, MPEG HE AAC, E AC-3, etc.), CRC regeneration, media rendering (for example, Dolby volume) etc. With these legacy subunits, the decoder will be able to transport media content in media data to a downstream media processing unit and / or render the media content. However, the decoder will not be able to convey the status of the media data or provide processing status metadata and / or media processing signaling in the output bitstream.
Under the present techniques, in some possible embodiments, as illustrated in FIGURE 7, the decoder may comprise any one of a plurality of new subunits such as metadata handling (evolution data and / or other metadata feed including one or more types third-party data, tracking information,
IMPI
<img file="MX359652B_D0069.tif" />
identifiers, standard or 'pfó ^ iedad information,' user annotation data, user preference data, feature extraction, feature management, etc.), secure communication (eg tamper-proof) to process status information (HMAC generator and signature validator, other cryptographic techniques), media footprint excerpt (eg audio and video fingerprint excerpt), auxiliary media processing (eg, · speech volume / channel (s) information, other types of media characteristics), data concealment (eg, PCM data concealment that may be destructive / irreversible or
<td>insertion</td><td>of</td>
<td>generator</td><td>HMAC</td>
<td>add bsi,</td><td>or</td>
<td>auxiliaries;</td><td>), or-</td>
<td>validation</td><td>of</td>
processing signaling validator (which may for example include inserts into one or more cryptographic technical ras fields, hidden data (eg hidden PCM data), undo media, data insertion recovery recovery data concealment, one or more switches that operate based on processing status signaling and / or processing status metadata (for example, valid evolution data and data concealment insertion control from a signature validator and HMAC generator), etc. As illustrated, information extracted by the HMAC signature validator and generator and the video and audio fingerprint extract
IMPI
<img file="MX359652B_D0070.tif" />
can be outputted to, or used for, audio and video sync correction, indexes, media rights, quality control, media placement processes, feature-based processing, etc.
In some possible modalities, a post / pre-processing unit 'in a media processing chain does not operate independently. Conversely, the post / pre-processing unit may interact with an encoder or decoder in the media processing chain. In the case of interacting with an encoder, the post / pre-processing unit can help to create at least a part of the processing status metadata regarding the status of the media data in a data block. In the case of interacting with a decoder, the post / pre-processing unit is configured to determine the status of the media data and to adapt its processing of the media data accordingly. In an example in FIGURE 7, an exemplary post / preprocessing unit such as a volume leveling unit can retrieve the hidden data in the PCM samples that are sent by an upstream decoder and to determine, based on the hidden data , whether or not the noise volume metadata is valid. If noise volume metadata is valid, data from input media such as audio can be passed unchanged through
IMPI II MEXICAN INDUSTRIAL PROPERTY GUIDELINE
<img file="MX359652B_D0071.tif" />
from the dG ruttRT volume leveling unit: —m another example, an exemplary post / pre-processing unit can retrieve the hidden data in the PCM samples that are sent by an upstream decoder and to determine based on the hidden data , one or more types of media characteristics previously determined from the content of the media samples. If a voice recognized keyword is indicated, the post-processing unit may perform one or more specific operations related to the voice recognized keyword.
5. DATA HIDING
FIGURE 8 illustrates an exemplary configuration that uses data concealment to pass media processing information, in accordance with some possible embodiments of the present invention. In some possible embodiments, data concealment may be employed to allow signaling between an upstream media processing unit such as an evolution encoder or decoder (eg, audio processing # 1) and a downstream media processing unit such as a post / pre-processing unit (for example, audio processing # 2) when there is no metadata path between the upstream and downstream media processing units.
In some possible modalities, concealment of
IMPI
<img file="MX359652B_D0072.tif" />
Reversible media data (eg, reversible audio data concealment) can be used to modify media data samples (eg, X) in the media data into modified media data samples (eg, X ') that carry processing status metadata and / or media processing signaling between the two media processing units. In some possible embodiments, the modification to the media data samples described herein is performed such that there is no perceptual degradation as a result of the modification. In this way, even if there may be no other media processing unit subsequent to the media processing unit 1, no audible or visible artifacts can be perceived with the modified media data samples. In other words, concealment of the processing status metadata and / or media processing signaling in a perceptually transparent manner will not cause any audible or visible artifacts when rendering audio and video in the modified media data samples.
In some possible embodiments, a media processing unit (eg, audio processing unit # 2 in FIGURE 8) retrieves embedded processing status and / or signaling media processing metadata from the media data samples. modified, and .X
INSTITUTO MEXIONO DE LA PROPIEDAD INDUSTRIAL restores the modified media data samples into the original media data samples by undoing the modifications. This can be done, for example, through a sub-unit (for example, information extraction and audio restoration). The recovered embedded information can then serve as a signaling mechanism between the two media processing units (eg, audio processing units # 1 and # 2 in FIGURE 8). The robustness of the data concealment technique here may depend on what types of processing can be performed by the media processing units. An example of media processing unit # 1 may be a digital decoder in a connection module or decoder, while an example of media processing unit # 2 may be a volume leveling unit in the same decoder or module. Connection. If the decoder determines that the noise volume metadata is valid, the decoder can use a stealth technique. reversible data controller to signal the subsequent volume leveling unit not to apply leveling.
In some possible embodiments, irreversible media data concealment (eg, an irreversible secure communications channel-based data concealment technique) may be employed to modify samples
<img file="MX359652B_D0073.tif" />
IMPI
INDUSTRIAL media data (eg X) on the Hat-ns Hp modified media data samples (eg X 'carrying processing status metadata and / or media processing signaling between the two processing units media.
In some possible embodiments, modification to the media data samples described herein is performed such that there is minimal perceptual degradation as a result of the modification. In this way, minimal visible or audible artifacts can be perceived with the modified media data samples. In other words, concealment of the processing status metadata and / or media processing signaling in a perceptually transparent manner will cause minimal audible and visible artifacts when rendering audio and video in the modified media data samples.
In some possible embodiments, modifications to the media data samples modified through irreversible data concealment may not be undone to recover the original media data samples.
6. EXEMPLARY PROCESS FLOW
FIGURE 9A and FIGURE 9B illustrate exemplary process flows in accordance with a possible embodiment of the present invention. In some possible embodiments, one or more computing units or devices in a media processing system can perform this flow of
IMPI
MEXICAN INSTITUTE OF NEWS
INDUSTRIAL
<img file="MX359652B_D0074.tif" />
process. - —...
In block 910 of FIGURE 9A, a first device in a media processing chain (eg, an improved media processing chain as described herein) determines whether a type of media processing has been performed in a media data output version. The first device may be part or all of a media processing unit. At block 920, in response to determining that the type of media processing has been performed on the output version of the media data, the first device may create a state of the media data. In some possible embodiments, the state of the media data may specify the type of media processing, the result of which is included in the output version of the media data. The first device may communicate, to a second device downstream in the media processing chain, the output version of the media data and the status of the media data, for example in an output media bitstream, or in an auxiliary metadata bitstream associated with a separate media bitstream that carries the output version of the media data.
In some possible embodiments, media data comprises media content such as one or more of: only
<img file="MX359652B_D0075.tif" />
INSTITUTO MEXICANO Di LA FROÍISDAD INDUSTRIAL
<img file="MX359652B_D0076.tif" />
audio content, video content only, or both audio content and video content.
In some possible embodiments, the first device may provide the second device with the status of the media data as one or more of: (a) media footprints, (b) processing status metadata, or (c) signaling of media processing.
In some possible embodiments, the first device may store a block of media processing data 'in a media processing database. The media processing data block may comprise media processing metadata and wherein the media processing data block is retrieved based on one or more media footprints that are associated with the media processing data block.
In some possible embodiments, the media data state comprises a cryptographic hash value encrypted with credential information. The cryptographic hash value can be authenticated by a canister device.
In some embodiments, at least a portion of the media data state comprises one or more secure communication channels hidden in the media data, and wherein the one or more secure communication channels must be authenticated by a recipient device. In an exemplary embodiment, the one or more secure communication channels can
IMPI
<img file="MX359652B_D0077.tif" />
understand at least one secure spread spectrum communications channel. In an exemplary embodiment, the one or more secure communication channels comprise at least one channel
<td>communications</td><td>sure with</td><td>modulation</td><td>for change</td><td>of</td>
<td>frequency.</td><td></td><td></td><td></td><td></td>
<td>In some</td><td>modalities</td><td>possible,</td><td>the state of</td><td>the</td>
<td colspan="2">media data comprises one or</td><td>more games</td><td>of parameters</td><td>than</td>
<td>are used in and / or</td><td colspan="2">derive from the type of</td><td>processing</td><td>of</td>
<td>media.</td><td></td><td></td><td></td><td></td>
<td>In some</td><td>modalities</td><td>possible,</td><td>at least one</td><td>of the</td>
<td>first device or</td><td>the second</td><td colspan="3">device comprises one or</td>
plus preprocessing units, encoders, decoder processing subunits, exemplary mode rendering unit units, the 'first media, transcoders, postprocessing or media subcontent. On a device it is an encoder
<td>(for example,</td><td colspan="2">an AVC encoder),</td><td>While</td><td colspan="2">that the second</td>
<td>device</td><td>is a</td><td colspan="2">decoder (for</td><td>example,</td><td>a</td>
<td>decoder</td><td>AVC).</td><td></td><td></td><td></td><td></td>
<td>In</td><td>some</td><td>modalities</td><td>possible,</td><td>the type</td><td>of</td>
Processing is performed by the first device, whereas in some other possible embodiments, the type of processing is instead performed by an upstream device, relative to the first device, in the media processing chain.
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0078.tif" />
In some possible embodiments, the first device may receive an input version of the media data. The input version of the media data comprises any state of the media data indicating the type of media processing. In these embodiments, the first device can analyze the input version of the media data to determine the type of media processing that has already been performed on the input version of the media data.
In some possible embodiments, the first device encodes noise volume and dynamic range in the media data state.
In some possible embodiments, the first device can adaptively avoid performing the type of media processing that has been performed by an upstream device. However, even when the type of media processing has been performed, the first device may receive a command to overcome the type of media processing performed by the upstream device. Rather, the first device may be directed to still perform the type of media processing, for example with either the same or different parameters. The media data state that the first device communicates to a second device downstream in the media processing chain,
<img file="MX359652B_D0079.tif" />
IMPI may comprise an output version of lo's'datuJ di · iiiudi®! * Including the result of the media processing type performed by the first device under command and a status of the media data indicating that the processing type of Media has already been made in the output version of the media data. In various possible embodiments, the first device may receive command from one of: (a) a user feed, (b) a system configuration of the first device, (c) pointing from an external device to the first device, or (d ) point from a sub-unit within the first device.
In some embodiments, the state of the media data comprises at least a portion of hidden state metadata on one or more secure communication channels.
In some embodiments, the first device alters a plurality of bytes in the media data to store at least a portion of the state of the media data.
In some embodiments, at least one of the first device and the second device comprise one or more codes of the Advanced Television Systems Committee (ATSC), codes of the Moving Image Expert Group (MPEG = Moving). Picture Experts Group), Codee Audio 3 (AC-3), and Code AC-3
Improved.
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL MONEDAD
<img file="MX359652B_D0080.tif" />
In some embodiments, the media processing chain comprises: · a pre-processing unit configured to accept time domain samples comprising media content as feed and to output processed time domain samples; an encoder configured to output compressed media bitstreams of the media content based on the processed time domain samples; a signal analysis and metadata correction unit configured to validate processing state metadata in the compressed media bitstream; a transcoder configured to modify the compressed media bitstream; a decoder configured to output decoded time domain samples based on the compressed media bit stream; and a post-processing unit configured to perform post-processing of the media content on the decoded time domain samples. In some embodiments, at least one of the first device and the second device comprise - one or more of the preprocessing unit, the metadata correction and signal analysis unit, the transcoder, the decoder, and the post processing unit. In some embodiments, at least one of the pre-processing unit, the metadata correction and signal analysis unit, the
IMPI
INSTITUTO MÍTICA NO D £ LA INGNTDAD INTOUSNUAL
<img file="MX359652B_D0081.tif" />
The transcoder, the decoder and the post-processing unit perform adaptive processing of the media content based on processing metadata received from an upstream device.
In some embodiments, the first device determines one or more media characteristics of the media data, and includes a description of the one or more media characteristics in the state of the media data. The one or more media features can comprise at least one media feature determined from one or more frames, seconds, minutes, user-defined time intervals, scenes, songs, pieces of music, and recordings. The one or more media features comprise a semantic description of the media data. In various modes, the one or more media characteristics comprise one or more structural properties, key including harmony and melody, timbre, rhythm, volume, stereo mix, a number of sound sources of the media data, absence or presence of voice, repeating characteristics, melody, harmonies, lyrics, timbre, perceptual characteristics, digital media characteristics, stereo parameters, one or more portions of speech content.
In block 950 of FIGURE 9B, a first device in a media processing chain (by 'Wl ·. »- ··
<img file="MX359652B_D0082.tif" />
example, an enhanced media processing string as described here) determines whether a media processing type has already been performed on a media data entry version.
In block '960, in response to determining that the media processing type has already been performed on the input version of the media data, the first device adapts media data processing to disable the performance of the processing type. of media on the first device. In some possible embodiments, the first device may disable one or more types of media processing based on the input state of the media data.
In some possible embodiments, the first device communicates, with a second device downstream in the media processing chain, an output version of the media data and a status of the media data indicating that the type of media processing means has been performed in the output version of the m data
<img file="MX359652B_D0083.tif" />
In some possible embodiments, the first device may encode dynamic noise volume in the media data state.
and interval
In some possible modes, 'the first device may automatically perform one or more dynamic or corrective noise volume processing based at least in part on itself.
IMPI
MEXICAN INSTITUTE OF THE INDUSTRIAL HOUSE
<img file="MX359652B_D0084.tif" />
the type of processing has already been performed on the input version of the media data.
In some possible embodiments, the first device may perform a second different type of media processing on the media data. The first device may communicate, with a second device downstream in the media processing chain, an output version of the media data and a status of the media data indicating that the media processing type and second different type Media processing has already been done on the output version of the media data.
In some possible embodiments, the first device may retrieve an input state of the media data that is associated with an input version of the media data. In some possible embodiments, the input state of the media data is conveyed with the input version of the media data in a bit stream of the input media. In some possible embodiments, the first device can extract the input state of the media data from the data units in the media data encoding media content. The input status of the media data may be hidden in one or more of the data units.
In some possible modalities, the first
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0085.tif" />
The device can retrieve a version of the data units that does not understand the input status of the media data and represent the media content based on the version of the data unit that was recovered.
In some possible modes, the first device can authenticate the input state of the media data by validating a cryptographic hash value
<td colspan="3">associated with the input status of</td><td colspan="3">the media data.</td>
<td></td><td>In</td><td>some modalities,</td><td>the</td><td>first</td><td>device</td>
<td>authentic</td><td>the</td><td>input status of</td><td>the</td><td>Data of</td><td>means to</td>
<td colspan="3">validate one or more associated fingerprints</td><td>with</td><td>the state</td><td>input</td>
of the media data, wherein at least one or more footprints is generated based on at least a portion of the media data.
In some embodiments, the first device validates the media data by validating one or more fingerprints associated with the media data entry state, where at least the one or more fingerprints are generated based on at least a portion of the data. media.
In some possible embodiments, the first device may receive the media data entry status as described with processing status metadata. The first device can create media processing signaling based at least in part on the processing status metadata. The
IMPI MEXICAN INSTITUTE OF THE PROPIWAD INBUfTMAL
<img file="MX359652B_D0086.tif" />
media processing signaling may indicate the input status of the media data, even though the media processing signaling may be smaller in volume and / or require a lower bit rate than that of the processing status metadata . The first device may transmit the media processing signaling to a downstream media processing device to the first device in the media processing chain. In some possible embodiments, the media processing signaling is hidden in one or more data units in an output version of the media data using a reversible data concealment technique such that one or more modifications to the media data is removed by a container device. In some embodiments, media processing signaling is hidden in one or more data units in an output version of the media data using an irreversible data concealment technique such that at least one of the one or more modifications to the media data is not removed by a container device.
In some embodiments, the first device determines one or more media characteristics based on a description of the one or more media characteristics in the media data state. The one or more media features may comprise at least one media feature.
IMPI tNsmvTO mmicamo M LA INDUSTRIAL MIQPtKMD
<img file="MX359652B_D0087.tif" />
determined from one or more tables, segundus, · mifÍuEJs,<sup>,</sup>'User definable time intervals, scenes, songs, pieces of music and recordings. The one or more media features comprise a semantic description of the media data. In some embodiments, the first device performs one or more specific operations in response to determining the one or more media characteristics.
In some possible embodiments, a method is provided, comprising: calculating, with a first device in a media processing chain, one or more reduced representations of data rate from a media data source table; and transporting the one or more reduced data rate representations simultaneously and securely, within a state of the data media itself, to a second device in the media processing chain; where the method is performed by one or more computing devices.
In some possible embodiments, the one or more reduced data rate representations are carried in at least one of a sub-stream, one or more reserved fields, an add_bsi field, one or more auxiliary fields, or one or more transformation coefficients .
In some possible embodiments, the one or more reduced data rate representations comprise
IMPI
<img file="MX359652B_D0088.tif" />
Synchronization data used for audio and video synchronization supplied within the media data.
In some possible embodiments, the one or more reduced data rate representations comprise media footprints (a) generated by a media processing unit and (b) embedded with the media data for one or more quality monitoring indicators media, media tracking, or content search.
In some possible embodiments, the method further comprises calculating and transmitting, by at least one of the one or more computing devices in the media processing chain, a cryptographic hash value based on media data and / or the state of the data. of media within one or more encoded bitstreams that carry the media data.
In some possible embodiments, the method further comprises: authenticating, by a canister device, the cryptographic hash value; signaling, by the container device to one or more downstream media processing units, a determination of whether the status of the media data is valid; signaling, by the container device to the one or more downstream media processing units, the status of the media data in response to determining that the status of the media data is valid.
IMPI
<img file="MX359652B_D0089.tif" />
In some possible embodiments, the cryptographic hash value representing the state of the media and / or media data is held in at least one of a substream, one or more reserved fields, and an add bsi field, one or more auxiliary data fields, or one or more transformation coefficients.
/ In some possible modalities, a method is provided comprising: Adaptively process, with one or more computing devices, in a media processing chain comprising one or more transformed psychoacoustic units, special audio / waveform encoding units, encoders, decoders, transcoders or current processors , a media data entry version based on past history of volume processing of the media data by one or more upstream media processing units as indicated by a status of the media data; Noise and / or dynamic volume range of an output version of the media data at one end of the media processing chain for consistent dynamic range and / or noise volume values.
In some possible embodiments, the consistent noise volume value comprises a noise volume value of (1) controlled or selected by a user or (2) adaptively reported by a state in the version of
<img file="MX359652B_D0090.tif" />
input of media data .J ___
In some possible embodiments, the noise volume value is calculated in the dialogue (speech) portions of the media data.
<td></td><td>In</td><td>some</td><td>modalities</td><td>possible the</td><td>value</td><td>of</td>
<td>volume</td><td>of</td><td>noise</td><td>calculated in</td><td>the portions</td><td colspan="2">absolute,</td>
<td>relative</td><td>me</td><td colspan="3">no control of media data.</td><td></td><td></td>
<td></td><td>In</td><td>some</td><td>modalities</td><td>possible the</td><td>value</td><td>of</td>
<td>interval</td><td colspan="2">dynamic</td><td>consistent</td><td>comprises a</td><td>value</td><td>of</td>
<td>interval</td><td colspan="2">dynamic of</td><td colspan="3">(1) controlled or selected by</td><td>a</td>
<td>user or</td><td> (2)</td><td colspan="3">adaptively signaled by a</td><td>. state</td><td>in</td>
<td colspan="2">the version of</td><td>entry</td><td>of the data</td><td>media.</td><td></td><td></td>
<td></td><td>In</td><td>some</td><td>modalities</td><td>possible the</td><td>value</td><td>of</td>
<td>interval</td><td colspan="2">dynamic it</td><td>calculated in</td><td>the portions</td><td colspan="2">dialogue</td>
<td colspan="4">(speaks) of the 'media' data.</td><td></td><td></td><td></td>
<td></td><td>In</td><td>some</td><td>modalities</td><td>possible the</td><td>value</td><td>of</td>
<td>interval</td><td colspan="2">dynamic</td><td>is calculated</td><td>in portions</td><td colspan="2">absolute,</td>
Relative and / or without control of the media data.
<img file="MX359652B_D0091.tif" />
In some possible embodiments, the method further comprises: calculating one or more dynamic range gain and / or noise volume control values to normalize the output version of the media data to one. value of consistent noise volume and consistent dynamic range; simultaneously carry the one or more dynamic range gain control values and / or
IMPI
<img file="MX359652B_D0092.tif" />
volume 'of noise within a state of salicTa' version 'of media data at the end of the media processing chain, wherein the one or more dynamic range gain and / or noise volume control values are usable by another device to apply inversely to one or more dynamic range gain and / or noise volume control values to retrieve an original noise volume value and original dynamic range in the input version of the media data.
In some possible embodiments, one or more dynamic range and / or noise volume control values that represent the status of the output version of the media data, are carried in at least one of an undercurrent, one or more reserved fields, an add_bsi field, one or more auxiliary data fields, or one or more transformation coefficients.
In some possible embodiments, a method is provided comprising performing one of inserting, extracting or editing related and unrelated media data locations and / or a status of related and unrelated media data locations within one or more streams of bits encoded by one or more computing devices in a media processing chain comprising one or more transformed psychoacoustic units, spatial audio / waveform encoding units, encoders,. «ift, τ ~ y
IMPI ^
MEXICAN INSTITUTE Df LA reOPlEDAO industrial 'decoders, transcoders or current processors.
In some possible modes, the one or more data locations of. related and unrelated media and / or the status of related and unrelated media data locations within encoded bitstreams are carried in at least one substream, one or more reserved fields, an add_bsi field, one or more fields auxiliary data, or one or more transformation coefficients.
In some possible embodiments, a method is provided which comprises performing one or more of inserting, extracting or editing related and unrelated media data and / or a state of related and unrelated media data within one or more bit streams. encoded by one or more computing devices in a media processing chain comprising one or more of transformed psychoacoustic units, spatial audio / waveform encoding units, current encoders, decoders, transcoders, or processors.
media stream data
In some possible embodiments, the one or related and unrelated encoded bit means plus related and / or unrelated data is transported into the state within less than one of a sub-stream, one or more reserved fields, an add field bsi, one or more auxiliary data fields, or one or more
IMPI
<img file="MX359652B_D0093.tif" />
transformation coefficients. ——-------——
In some possible embodiments, a media processing system is configured to compute and transport cryptographic hash values based on media data and / or a state of the media data within one or more bit streams encoded by one or more devices. computing in a media processing chain, comprising one or more of transformed psychoacoustic units, spatial audio / waveform encoding units, encoders, decoders, current transcoders or processors.
As used herein, the term related and unrelated media data locations may refer to information that may include a media resource locator such as an absolute path, relative path, and / or URL indicating the location of related media (for example , a copy of media in a different bitstream format) or an absolute path, relative path, and / or URL that · indicates the location of media unrelated to other, unrelated information
<td colspan="4">directly to essence or bit stream</td><td>in which</td><td>I know</td>
<td>find</td><td>the data location</td><td>of</td><td>means (for</td><td>example,</td><td>the</td>
<td>Location</td><td>of a new piece</td><td>of</td><td colspan="2">means such as</td><td>a</td>
<td>commercial,</td><td>warning page</td><td>the</td><td>network, etc.).</td><td></td><td></td>
<td></td><td>As used here,</td><td>the</td><td>expression</td><td>state</td><td>of</td>
IMPI
<img file="MX359652B_D0094.tif" />
Related and non-related media data locations may refer to the validity of related and unrelated media locations (as they can be edited / updated throughout the life cycle of the bitstreams in which they are carried).
As used herein, related media data may refer to related media data transport in the form of secondary media data bit streams highly correlated with the primary media representing the bit stream, (eg, the transport of a copy of media data in a second (independent) bitstream format. In the context of unrelated media data, this information may refer to the transport of bit streams of secondary media data that are independent of the primary media data.
As used herein, status for related media data can refer to any signaling information (processing history, updated target noise volume, etc.) and / or metadata as well as the validity of related media data. Status for unrelated media data may refer to independent signaling information and / or metadata including validity information which can be transporter separately in (independent) form of the data status of
IMPI
<img file="MX359652B_D0095.tif" />
related media. The unrelated media data state represents media data that is unrelated to the media data bitstream, where this information is located (since this information can be independently edited / updated throughout the life cycle of the bit streams in which they are transported).
As used herein, the term "absolute, relative and / or uncontrolled portions of the media data" refers to the control of noise level and / or volume measurements that is performed on the media data. Control refers to a specific noise volume threshold level where the calculated value that exceeds the threshold is included in the final measurement, (for example ignoring the short-term noise volume value below -60 dBFS at the final measured value). Control at an absolute value refers to a fixed level or volume of noise where control at a relative value refers to a value that is dependent on the measurement value without current control.
FIGURE 12A through FIGURE 12L-2 further illustrate block diagrams of some exemplary media processing devices / nodes, in accordance with some embodiments of the present invention.
As illustrated in FIGURE 12A, a signal processor (which may be N-node Node 1) is
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY configures to receive an input signal / 'that' can comprise PCM audio samples. PCM audio samples may or may not contain processing status metadata (or media status metadata) hidden between PCM audio samples. The signal processor of FIGURE 12A may comprise a media state metadata extractor that is configured to decode, extract and / or interpret the processing state metadata from the PCM audio samples, as provided by one or more media processing units before signal processor of FIGURE 12A. At least a part of the processing status metadata can be provided to an audio encoder in the signal processor of FIGURE 12A, to tailor processing parameters for the audio encoder. In parallel, an audio analysis unit in the signal processor of FIGURE 12A can analyze the media content that is passed in the input signal. Feature extraction, media classification, volume estimation, fingerprint generation, etc. can be implemented as part of the analysis performed by the audio analysis unit. At least a part of the results of this analysis can be provided to the audio encoder in the signal processor of FIGURE 12A to tailor processing parameters for the audio encoder. The audio encoder encodes the PCM audio samples into the audio signal.
IMPI
<img file="MX359652B_D0096.tif" />
input into a bit stream encoded into an output signal based on the processing parameters. A coded bitstream analysis unit in the signal processor of FIGURE 12A can be configured to determine whether media data or samples in the coded bitstream are to be transmitted in the signal processor output signal of the FIGURE 12A has space to store at least a portion of the processing status metadata. The new processing status metadata to be transmitted by the signal processor of the
FIGURE 12A comprise some or all of the processing status metadata that is extracted by the processing status metadata extractor, the processing status metadata that is generated by the audio analysis unit, and the media status metadata generator. of the signal processor of FIGURE 12A, and / or any data from 3<sup>was</sup> part.
It is determined that the media or sample data in the encoded bitstream has space to store at least a portion of the processing status metadata, some or all of the new processing status metadata may be stored as hidden data in the media data or samples in the output signal.
Additionally, optionally or alternately, some or all of the new processing status metadata can be stored in the
<img file="MX359652B_D0097.tif" />
<sup>87</sup> IMPI
MEXICAN INSTITUTE
OF THE PROPERTY
INDUSTRIAL data separate from media data and samples at the output signal. In this way, the output signal may comprise an encoded bitstream containing the metadata of the new processing state (or media state) that is transported within and / or between the media samples (essence) over a communications channel hidden or not hidden insurance.
As illustrated in FIGURE 12B, a signal processor (which can be N-node Node 1) is configured to receive an input signal that can comprise PCM audio samples. The PCM audio samples may or may not contain processing status metadata (or media status metadata) hidden between the PCM audio samples. The signal processor of FIGURE 12B may comprise a media state metadata state that is configured to decode, extract, and / or interpret the processing state metadata from the PCM audio samples, as provided by one or more media processing units before signal processor of FIGURE 12B. At least a portion of the processing status metadata can be provided to a PCM audio sample processor in the signal processor of FIGURE 12B to tailor processing parameters for the PCM audio sample processor. In parallel, an audio analysis unit in the signal processor of FIGURE 12B can analyze the content of
IMPI
<img file="MX359652B_D0098.tif" />
means that are passed in the input signal. Feature extraction, media classification, volume estimation, footprint generation, etc. can be implemented as part of the analysis performed by the audio analysis unit. At least a part of the results of this analysis can be provided to the audio encoder in the signal processor of FIGURE 12B to tailor processing parameters for the PCM audio sample processor. The PCM audio sample processor processes the PCM audio samples on the input signal into a PCM audio bit stream (samples) on an output signal based on the processing parameters. A PCM audio analysis unit in the signal processor of FIGURE 12B can be configured to determine whether media data or samples in the PCM audio stream to be transmitted on the output signal of the signal processor of FIGURE 12B have room for store at least a portion of the processing status metadata. The new processing state metadata to be transmitted by the signal processor of FIGURE 12B comprises some or all of the processing state metadata that is extracted by the media state metadata extractor, the processing state metadata that is generated by the audio analysis unit and a media state metadata generator of the signal processor of FIGURE 12B, and / or any data
IMPI MEXICAN INSTITUTE OF LA MONEDAD INDUSTRIAL
<img file="MX359652B_D0099.tif" />
of 3<sup>was</sup> part. If it is determined that the media or sample data in the PCM audio bitstream has room to store at least a portion of the processing status metadata, some or all of the new processing status metadata may be stored as hidden data in the media data or samples in the output signal. Additionally, optionally or alternately, some or all of the metadata from the new processing status metadata can be stored in a separate metadata structure apart from the sample and media data in the output signal. In this way, the output signal may comprise a PCM audio bitstream containing the new processing state (or media state) metadata that is transported within and / or between the media (essence) samples via a Secure hidden or not hidden communication.
As illustrated in FIGURE 12C, a signal processor (which may be N-node Node 1) is configured to receive an input signal, which may comprise a PCM audio bitstream (samples). The PCM audio stream may contain processing status metadata (or media status metadata) that is transported between or between media samples (essence) in the PCM audio bit stream via a secure or hidden communication channel. hidden. The signal processor in FIGURE 12C • ”1- * 'k, <v — v ♦
IMPI
MEXICAN INSTITUTE
OF THE PROPERTY
INDUSTRIAL
<img file="MX359652B_D0100.tif" />
it may comprise a media state metadata extractor that is configured to decode, extract and / or interpret the processing state metadata of the PCM audio bitstream. At least a portion of the processing status metadata can be provided to a PCM audio sample processor in the signal processor of FIGURE 12C to tailor processing parameters for the 'PCM audio sample processor. Processing status metadata may include a descriptor of media characteristics, media class types or sub-types, or probability / probability values, as determined by one or more media processing units before the signal processor. FIGURE 12C, which the signal processor of FIGURE 12C can be configured to use without performing its own media content analysis. Additionally, optionally or alternately, the Media State Metadata Extractor can be configured to extract data from 3<sup>was</sup> part of the input signal and transmit 3 data<sup>was</sup> part to a downstream processing node / entity / device. In one embodiment, the PCM audio sample processor processes the PCM audio bitstream into audio PCM samples an output signal based on the set of processing parameters based on the processing status metadata provided by the one or more
IMPI
<img file="MX359652B_D0101.tif" />
media processing units before processor 05 'signal of FIGURE 12C.
As illustrated in FIGURE 12D, a signal processor (which may be N-node Node 1) is configured to receive an input signal, which may comprise an encoded audio bitstream containing processing status metadata (or media status metadata) that are transported within and / or hidden between media samples via a secure hidden or non-hidden communication channel. The signal processor of FIGURE 12D may comprise a media state metadata extractor that is configured to decode, extract and / or interpret the encoded bitstream processing status metadata, as provided by one or more units. of media processing before signal processor of FIGURE 12D. At least a portion of the processing status metadata can be provided to an audio decoder in the signal processor of FIGURE 12D to tailor processing parameters for the audio decoder.
<td>In</td><td>parallel,</td><td>the</td><td>Unit</td><td colspan="2">audio analysis</td><td>in</td><td>processor</td>
<td>of</td><td>signal of</td><td>the</td><td>FIGURE</td><td>12D</td><td>can analyze</td><td>the</td><td>content of</td>
<td colspan="2">means that</td><td>I know</td><td>pass</td><td>in</td><td>the sign of</td><td colspan="2">entry. Can</td>
implement feature extraction, media classification, volume estimation, fingerprint generation, etc., as part of the analysis performed by the analysis unit
IMPI
<img file="MX359652B_D0102.tif" />
audio. At least a portion of the analysis rujifl'LartlÓé may be provided to the audio decoder in the signal processor of FIGURE 12D to tailor processing parameters for the audio decoder. The audio decoder transforms the encoded audio bitstream into the. input signal into a PCM audio bitstream into an output signal based on the processing parameters. A PCM audio analysis unit in the signal processor of FIGURE 12D can be configured to determine if the media or sample data in the PCM audio bitstream has room to store at least a portion of the processing status metadata . The new processing state metadata to be transmitted by the signal processor of FIGURE 12D comprises some or all of the processing state metadata that is extracted by the media state metadata extractor, the processing state metadata that is generated by the audio analysis unit and a medium state metadata generator of the signal processor of FIGURE 12D, and / or any data from 3<sup>was</sup> part.
If it is determined that the media or sample data in the PCM audio bitstream has the space to store at least a portion of the processing status metadata, some or all of the new processing status metadata can be stored how
IMPI
<img file="MX359652B_D0103.tif" />
hidden data in the media data or samples in the output signal. Additionally, optionally or alternately, some or all of the new processing status metadata may be stored in separate metadata structure apart from the sample and media data in the output signal. In this way, the output signal may comprise a PCM audio stream (samples) containing processing state (or media state) metadata that is transported within and / or between the data / media samples (essence) through a hidden or non-hidden secure communications channel.
As illustrated in FIGURE 12E, a signal processor (which may be N-node Node 1) is configured to receive an input signal, which may comprise an encoded audio bitstream. The encoded audio bitstream may contain processing status metadata (or media status metadata) carried within and / or between media samples (essence) in the encoded audio bitstream, via a hidden secure communications channel. or not hidden. The signal processor of FIGURE.12E may comprise a media state metadata extractor that is configured to decode, extract and / or interpret the processing state metadata from the encoded audio bitstream. At least part of the status metadata for
IMPI
<img file="MX359652B_D0104.tif" />
Processing can be provided to an audio decoder in the signal processor of FIGURE 12E to tailor processing parameters for the audio decoder. Processing status metadata may include a description of media characteristics, media class types or subtypes, or probability / probability values, as determined by one or more media processing units before the signal processor. FIGURE 12E, the signal processor of FIGURE 12E can be configured for use without performing its own media content analysis. Additionally, optionally or alternately, the Media State Metadata Extractor can be configured to extract data from 3<sup>was</sup> part of the input signal and transmit 3 data<sup>was</sup> part to a downstream processing node / entity / device. In one embodiment, the audio decoder processes the audio bitstream encoded in PCM audio samples an output signal · based on the processing parameter set based on the processing status metadata provided by the or more media processing units before the signal processor of FIGURE 12E.
As illustrated in FIGURE 12F, a signal processor (which may be N-node Node 1) is configured to receive a power signal, which may comprise a
IMPI MEXICAN INSTITUTE PE THE INDUSTRIAL PROPERTY
<img file="MX359652B_D0105.tif" />
encoded audio bitstream containing processing status metadata (or media status metadata) that is transported within and / or hidden between the media samples via a secure hidden or non-hidden communication channel. The signal processor of FIGURE 12F may comprise a media state metadata extractor that is configured to decode, extract and / or interpret the processing state metadata from the encoded bitstream, as provided by one or more media processing units before the signal processor of FIGURE 12F. At least a portion of the processing status metadata may be provided to a bitstream transcoder (or encoded audio bitstream processor) in the signal processor of FIGURE-12F to tailor processing parameters for the transcoder of bit stream. In parallel, an audio analysis unit in the signal processor of FIGURE 12F can analyze the media content passed in the input signal. Character extraction, media classification, volume estimation, fingerprint generation, etc. can be implemented as part of the analysis performed by the audio analysis unit. At least a part of the results of this analysis can be provided to the bitstream transcoder in the signal processor of FIGURE 12F
IMPI
<img file="MX359652B_D0106.tif" />
to adapt processing parameters for the bitstream transcoder. The bitstream transcoder transforms the encoded audio bitstream in the input signal into an encoded audio bitstream in an output signal based on the processing parameters.
A bitstream analysis unit encoded in the signal processor of the
FIGURE 12F can be configured to determine if the media or sample data in the encoded audio bitstream has room to store at least a portion of processing status metadata. The new processing state metadata to be transmitted by the signal processor of FIGURE 12F comprises some or all of the processing state metadata that is extracted by the media state metadata extractor, the processing state metadata that was generated by the audio analysis unit and a media state metadata generator of the signal processor of FIGURE
12F, and / or any data from 3<sup>was</sup> part.
If it is determined that the media or sample data in the encoded audio bitstream has room to store at least a portion of the processing status metadata, some or all of the new processing status metadata may be stored as hidden data in the media data or samples in the output signal. Additionally, optional
<img file="MX359652B_D0107.tif" />
IMPI
Mexican Institute of Industrial Property
<img file="MX359652B_D0108.tif" />
or alternately, some or all of the new processing status metadata may be stored in a separate metadata structure apart from the media data in the output signal. In this way, the output signal may comprise an encoded audio bitstream containing processing state (or media state) metadata that is transported within and / or between the media (essence) data / samples via a channel hidden or not hidden secure communications.
FIGURE 12G illustrates an exemplary configuration similar in part to that of FIGURE 12A. Additionally, optionally or alternately, a signal processor of FIGURE 12G may comprise a media state metadata extractor that is configured to query a local and / or external media state metadata database, which may link operationally with the signal processor of FIGURE 12G via intranet and / or internet. A query sent by the signal processor of FIGURE 12G to the database may include one or more fingerprints associated with the media data, one or more names associated with the media data (for example, the title of a song, the title of a film), or any other types of identifying information associated with media data. Based on the information in the query, coupled media status metadata stored in the database can be located and
IMPI
INSTITUTO HKXICAMO CE THE INDUSTRIAL PROPERTY
<img file="MX359652B_D0109.tif" />
be provided to the signal processor
1'3 — FIGURE 1'2Π: --- The media state metadata can the processing state metadata that is provided by the media state metadata extractor nodes / entities an alternate encoder, the understanding is configured of downstream processing such as audio. Additionally, optionally, or the signal processor of FIGURE 12G may generate state metadata to provide any associated media states such media information as generated metadata and / or identification information such as fingerprints, identification status metadata means illustrated in FIGURE 12G. In names and / or other types to a local and / or external database, as an additional, optional alternate form, one or more portions of the media's state metadata stored in the database may be provided to the data processor. FIGURE 12G signal to communicate to a processing node / device within and / or between samples of media channel of similar optional downstream media (essence) via secure hidden or non-hidden communications.
FIGURE in part a or alternates,
12H illustrates FIGURE 12B.
exemplary configuration processor
Additionally, a signal from the
FIGURE
12H may comprise a media state metadata extractor that is configured to query a database of
IMPI
<img file="MX359652B_D0110.tif" />
Local and / or external media state metadata, which can be linked in an operational way to the signal processor of FIGURE 12H through intranet and / or internet. A query sent by the signal processor of FIGURE 12H to the database may include one or more fingerprints associated with the media data, one or more names associated with the media data (for example, the title of a song, the title of a film), or any other types of identifying information associated with media data. Based on the information in the query, corresponding media state metadata stored in the database can be located and provided in the signal process of FIGURE 12H. Media status metadata can be included in processing status metadata that is provided by the media status metadata extractor to downstream processing nodes / entities such as a PCM audio sample processor. Additionally, optionally or alternately, the signal processor of FIGURE 12H may comprise a media state metadata generator that is configured to provide any generated media state metadata and / or associated identification information such as fingerprints, names and / or other types of identifying information to a local and / or external media state metadata database, as illustrated in FIGURE 12H. Additionally, optional or
100
IMPI MEXICAN INSTITUTE OF INDUfTRIAL PROPERTY
<img file="MX359652B_D0111.tif" />
Alternately, one or more portions of the media metadata stored in the database may be provided to the signal process of FIGURE 12H to communicate to a downstream media processing node / device within and / or between samples of means (essence) through a hidden or non-hidden secure communications channel.
FIGURE 121 illustrates an exemplary configuration similar to that of FIGURE 12C in part. Additionally, optionally or alternately, a signal processor of FIGURE 121 may comprise a media state metadata extractor that is configured to query a linkable local and / or external media state metadata database. operatively with the signal processor of FIGURE 121 through intranet and / or internet. A query sent by the signal processor of FIGURE 121 to the database may include one or more fingerprints associated with the media data, one or more names associated with the media data (for example, the title of a song , the title of a movie), or any other types of identifying information associated with media data. Based on the query information, corresponding media state metadata stored in the database can be located and provided to the signal process of FIGURE 121. Media state metadata can be provided to current processing nodes / entities
<img file="MX359652B_D0112.tif" />
101
IMPI
<img file="MX359652B_D0113.tif" />
below such as a sample PCM audio processor. ”
FIGURE 12 J illustrates an exemplary configuration similar to that of FIGURE 12D in part. Additionally, optionally or alternately, a signal processor of FIGURE 12J may comprise a media state metadata extractor that is configured to query a local and / or external media state metadata database, which may operatively link to the signal processor of FIGURE 12J via intranet and / or internet. A query sent by the signal processor of FIGURE 12J to the database may include one or more fingerprints associated with the media data, one or more names associated with the media data (for example, the title of a song, the title of a film), or any other types of identifying information associated with media data.
Based on the information in the query, corresponding media state metadata stored in the database can be located and provided to the signal process of FIGURE L2J. The media state metadata from the database can be included in processing state metadata that is provided to downstream processing nodes / entities such as an audio decoder. Additionally, optionally or alternately, the signal processor of FIGURE 12J may comprise an audio analysis unit that is configured to provide
102
<img file="MX359652B_D0114.tif" />
IMPI
INSTITUTO MEXICO DE LA MONEDAD INDUSTRIAL any media state metadata generated and / or associated identifying information such as fingerprints, names and / or other types of identification information to a local and / or external media state metadata database, as illustrated in FIGURE 12J. Additionally, optionally or alternately, one or more portions of media state metadata stored in the database may be provided to the signal process of FIGURE 12J to communicate to a downstream media processing node / device within and / or between media samples (essence) through a hidden or non-hidden secure communication channel.
FIGURE 12K illustrates an exemplary configuration similar to that of FIGURE 12F in part. Additionally, optionally or alternately, a signal processor of FIGURE 12K may comprise a media state metadata extractor that is configured to query a linkable local and / or external media state metadata database. operatively to the signal processor of FIGURE 12K via intranet and / or internet. A query sent by the signal processor of FIGURE 12K to the database may include one or more fingerprints associated with the media data, one or more names associated with the media data (for example, the title of a song, the title of a movie), or any other types of information from
<img file="MX359652B_D0115.tif" />
103
IMPI identification associated with the media data. Based on the information in the query, corresponding media state metadata stored in the database can be located and provided to the signal process of FIGURE 12K. The media state metadata from the database can be included in processing state metadata that is provided to downstream processing nodes / entities such as a bitstream transcoder or encoded audio bitstream processor. Additionally, optionally or alternately, one or more portions of the media state metadata stored in the database may be provided in FIG. 12K signal processing to communicate to a downstream media processing node / device within and / or between media samples (essence) through a hidden or non-hidden secure communication channel.
FIGURE 12L-1 and 12L-2 illustrate a signal processor node 1 and a signal processor node 2, in accordance with an exemplary embodiment. Signal processor node 1 and signal processor node 2 can be part of a total media processing chain. In some embodiments, signal processor node 1 adapts media processing based on processing state metadata that is received by signal processor node 2, while signal processor node 2 adapts
104
IMPI
<img file="MX359652B_D0116.tif" />
media processing based on processing status metadata received by signal processor node 2. Processing status metadata received by signal processor node 2 may comprise processing status metadata and / or media status metadata added by signal processor node 1 after content processor node 1 parses the content media data; As a result, the signal processor node 2 can directly use the metadata that is provided by the signal processor node 1 in media processing without repeating some or all of the analysis previously performed by the signal processor node 1.
7. IMPLEMENTATION MECHANISMS - EQUIPMENT OVERVIEW
PHYSICAL
According to one embodiment, the techniques described herein are implemented by one or more special purpose computing devices. Special purpose computing devices may be wired to perform the techniques, or may include digital electronic devices such as one or more application specific integrated circuits (ASICs) or field programmable gate arrays. (FPGAs = field programmable gate arrays) that are persistently programmed to perform the techniques, or may include one or more • ei
105
IMPI MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0117.tif" />
general-purpose hardware processors, programmed to perform the techniques in accordance with program instructions in firmware, memory, other storage, or a combination thereof. These special-purpose computing devices can also combine custom physical wiring logic, ASICs, or FPGAs with custom programming to accomplish the techniques. Special purpose computing devices may be desktop computer systems, laptop computer systems, portable devices, network devices, or any other device that incorporates physical wiring and / or program logic to implement the techniques.
For example, FIGURE 10 is a block diagram illustrating a computer system 1000 in which an embodiment of the invention can be implemented. Computer system 1000 includes a busbar 1002 or other communication mechanism to communicate information, and a hardware processor 1004 coupled with busbar 1002 to process information. Physical equipment processor 1004 may be, for example, a general-purpose microprocessor.
Computer system 1000 also includes a main memory 1006, such as a random access memory (RAM) or other storage device
106
IMPI
<img file="MX359652B_D0118.tif" />
dynamic, coupled with the busbar iO & g-'paiu ulmuceiiai 'information and instructions to be executed by processor 1004. Main memory 1006 can also be used to store temporary variables or other intermediate information during instruction execution by processor 1004. These instructions, when stored in non-transient storage medium accessible to processor 1004, convert computer system 1000 into a special purpose machine that is custom tailored to perform the operations specified in the instructions.
Computer system 1000 further includes a read-only memory (ROM) 1008 or other static storage device coupled to busbar 1002, for storing static information and instructions for processor 1004. A storage device 1010, such as a disk Magnetic or optical disc, it is provided and attached to the busbar 1002 to store information and instructions.
Computer system 1000 may be coupled by busbar 1002 to a display 1012, such as a cathode ray tube (CRT), to display information to a computer user. An input device 1014, including alphanumeric and other keys, is coupled to busbar 1002 to communicate information and
107
COMMAND INSTITUTE OF INDUSTRIAL PROPERTY command selections to processor 1004. Another type of user power device is cursor control 1016, such as a mouse, a footstool, or cursor arrow keys to communicate address information and command selection to the processor 1004 and to control cursor movement in display 1012. This feeding device typically has two degrees of freedom on two axes, a first axis (for example, x) and a second axis (eg,
y), which allow the device to specify positions on a plane.
Computer system 1000 can implement the techniques described herein using custom-tuned physical wiring logic, one or more ASICs or FPGAs, firmware, and / or program logic that in combination with the computer system causes or programs that the 1000 computer system is a special purpose machine. According to one embodiment, the present techniques are performed by a computer system 1000 in response to processor 1004 that executes one or more sequences of one or more instructions contained in main memory 1006. These instructions may be read from main memory 1006. from another storage medium, such as the storage device
1010.
Executing the sequences of instructions contained in main memory 1006 causes the
108
IMPI MEXICAN INSTITUTE OF INDUSTRIAL DEVELOPMENT
<img file="MX359652B_D0119.tif" />
1004 processor perform the process steps described here'7
In alternate modes, the physical wiring circuits
<td>they can</td><td colspan="2">be used instead of</td><td>or</td><td>in combination with</td><td>the</td>
<td colspan="2">support instructions</td><td>logical.</td><td></td><td></td><td></td>
<td></td><td>The expression</td><td>means, medium</td><td>of</td><td>storage like</td><td>I know</td>
<td>employs</td><td>here refers</td><td colspan="2">to someone</td><td>non-transitory medium</td><td>than</td>
stores data and / or instructions that cause a machine to operate in a specific way. These storage media may comprise non-volatile media and / or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 1010. Volatile media includes dynamic memory, such as main memory 1006. Common forms of storage media include eg floppy disk drive, floppy disk, hard drive, solid state drive, magnetic tape or any other magnetic data storage medium, a CD-ROM or any other optical data storage medium , any physical media with hole patterns, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, or any other memory cartridge or chip.
Storage media are different from, but can be used in conjunction with, transmission media. The transmission media participate in transferring information between storage media. For example, media
109
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0120.tif" />
transmission include coaxial cables, —gjramtritj ·<sup>1</sup> ele vwbi'e-and optical fibers, including the cables that comprise the busbar 1002. Transmission media can also take the form of light or acoustic waves, such as those generated during radio-wave and infra-red data communications ace.
Various forms of media may be involved in bringing one or more sequences of one or more instructions to processor 1004 for execution. For example, instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a phone line using a modem. A local modem to computer system 1000 can receive the data on the phone line and use an Infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried on the infra-red signal, and appropriate circuits can put the data on busbar 1002. Busbar 1002 transports the data to main memory 1006, from where processor 1004 retrieves and execute the instructions. Instructions received by main memory 1006 can optionally be stored in storage device 1010 either before or after execution by the processor.
110
IMPI
MEXICAN INSTITUTE OE INDUSTRIAL PROPERTY
<img file="MX359652B_D0121.tif" />
1004 .
Computer system 1000 also includes a communication interface 1018 coupled to busbar 1002. Communication interface 1018 provides two-way data communication that couples to a network link 1020 that connects to a local network 1022. For example, the communication interface 1018 may be an integrated services digital network card (ISDN), cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of phone line. As another example, the communication interface 1018 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links can also be implemented. In either of these implementations, communication interface 1018 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams that represent various types of information.
Network link 1020 typically provides data communication over one or more networks to other data devices. For example, network link 1020 may provide a connection over local network 1022 to a host or host computer 1024 or to a computer
111
IMPI MEXIGANQ INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0122.tif" />
data operated by an Internet Service Provider (ISP) 1026. ISP 1026 in turn provides data communication services through the global packet data communication network now commonly referred to as Internet 1028. Local network 1022 and Internet 1028 both use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the network link signals 1020 and through the communication interface 1018, which carry the digital data to and from the computer system 1000, are exemplary forms of transmission media.
Computer system 1000 can send messages and receive data, including program code, through the network (s), network link 1020, and communication interface 1018. In the example of the Internet, a server 1030 can transmit a code requested for an application program over the Internet 1028, ISP 1026, the local network 1022 and the communication interface 1018.
The received code may be executed by processor 1004 as it is received and / or stored in storage device 1010, or other non-volatile storage for later execution.
LISTED EXEMPLARY MODALITIES
In this way, modalities of the present
112
IMPI
<img file="MX359652B_D0123.tif" />
Invention may relate to one or more of the examples listed below, each of which are examples, and as with any other related discussion provided above, are not to be construed as limiting any claim or claims which are however provided in addition to
<td>continuation</td><td>how</td><td>I know</td><td>find</td><td>Now whatever</td>
<td>later</td><td colspan="2">amended,</td><td>replaced or</td><td>added.</td>
<td colspan="2">Equally,</td><td colspan="2">these examples</td><td>they should not be</td>
considered as limiting with respect to any claim or claims of any patents and / or related patent applications (including any applications and / or patents against foreign or international as well as divisional, continuations, re-expeditions,
The exemplary embodiment listed 1 is a method comprising: determining, by a first device in a media processing chain, whether a type of media processing has been performed on an output version of media data; in response to determining, by the first device, that · the type of media processing has been performed on the output version of the media data, perform: (a) create, by the first device, a status of the media data media, the state specifies the type of media processing performed in the output version of
113
<img file="MX359652B_D0124.tif" />
MEXICAN INSTITUTE tXLAPROTMDAS INDUSTRIAL
<img file="MX359652B_D0125.tif" />
the media data, and (b) communicating, from the first device to a second device downstream in the media processing chain, the output version of the media data and the status of the media data.
The numbered exemplary mode 2 is a method as described in the numbered exemplary mode 1, wherein the media data comprises media content such as one or more of: audio content only, video content only, or both audio content and video content.
The listed exemplary mode 3 is a method as described in the listed exemplary mode 1, which further comprises providing, to the second device, the status of the media data as one or more of: (a) media footprints, (b) processing status metadata, (c) characteristic values of extracted media, (d) description (s) and / or values of media class type or sub-type, (e) probability characteristics of class and / or sub-class of media characteristics, (f) cryptographic hash values or (f) signaling of media processing.
The listed exemplary mode 4 is a method as described in the listed exemplary mode 1, further comprising: storing a media processing data block in a media processing database, wherein the media processing data block media comprises media processing metadata, and where the block of t
114
IMPI
<img file="MX359652B_D0126.tif" />
Media processing data is retrieved based on one or more media footprints that are associated with the media processing data block.
Enumerated exemplary mode 5 is a method as described in enumerated exemplary mode 1, wherein the state of the media data 'comprises a cryptographic hash value encrypted with credential information, and wherein the cryptographic hash value will be authenticated by container device.
The exemplary mode numbered 6 is a method as described in the exemplary mode numbered 1, wherein at least a portion of the state of the. media data comprises one or more secure communication channels hidden in the media data, and wherein the one or more secure communication channels' will be authenticated by a recipient device.
The listed exemplary mode 7 is a method as described in the listed exemplary mode 6, wherein the one or more secure communication channels comprises at least one broad spectrum secure communications channel.
The listed exemplary mode 8 is a method as described in the listed exemplary mode 6, wherein the one or more secure communication channels comprise at least one frequency shift modulated secure communication channel.
IMPI
<img file="MX359652B_D0127.tif" />
115
The exemplary mode numbered 9 Itiétg'gíT CSflTg is described in the exemplary mode numbered 1, where the state of the media data is conveyed with the output version of the media data in a bit stream of
<td rowspan="2">means of</td><td colspan="6">departure.</td>
<td colspan="2">The modality</td><td>numbered copy</td><td>10 is a</td><td colspan="2">method</td>
<td>how I know ,</td><td colspan="3">describes in exemplary mode</td><td>numbered</td><td> 1,</td><td>in</td>
<td>where he</td><td>state</td><td>the</td><td>media data is</td><td>transports</td><td>in</td><td>a</td>
<td>stream</td><td>bit</td><td>of</td><td>auxiliary metadata</td><td colspan="2">associated with</td><td>a</td>
<td>stream</td><td>bit</td><td>of</td><td colspan="3">separate means transporting</td><td>the</td>
output version of the media data.
The numbered exemplary mode 11 is a method as described in the numbered exemplary mode, wherein the state of the media data comprises one or more sets of parameters that relate to the type of media processing.
The listed exemplary mode 12 is a method as described in the listed exemplary mode 1, wherein at least one of the first device or the second device comprises one or more of: preprocessing units, encoders, media processing sub-units, transcoders, decoders, post-processing units, or media content rendering subunits.
The exemplary modality listed 13 is a method
115
IMPI INSTITUTO MEJICANO Di LA NtCFIEDAD INDUSTMAL
<img file="MX359652B_D0128.tif" />
as described in the exemplary embodiment listed 1, wherein the first device is an encoder, and where the second device is a decoder.
The numbered exemplary mode 14 is a method as described in numbered exemplary mode 1, which further comprises performing, by the first device, the type of media processing.
The listed exemplary mode 15 is a method as described in the listed exemplary mode 1, wherein the type of media processing is performed by a device upstream, relative to the first device, in the media processing chain; and which also includes: receiving, by. the first device, an input or feed version of the media data, wherein the input version of the media data comprises any media data state indicating the type of media processing; analyze the input version of the media data to determine the type of media processing that has already been performed on the input version of the media data.
The listed exemplary mode 16 is a method as described in the listed exemplary mode 1, further comprising: encoding dynamic range and noise volume values in the media data state.
The exemplary modality listed 17 is a method
117
IMPI
<img file="MX359652B_D0129.tif" />
as described in the exemplary embodiment listed 1, wherein the type of media processing was previously performed by an upstream device, relative to the first device, in the media processing chain; and further comprises: receiving, by the first device, a command to overcome the type of media processing previously performed; perform by the first device, the media processing type; communicate, from the first device to a second device downstream in the media processing chain, an output version of the media data and a status of the media data indicating that the type of media processing has already been performed in the output version of the media data.
The listed exemplary mode 18 is a method as described in the listed exemplary mode 17, which further comprises receiving the command from one of: (a) user power, (b) a system configuration of the first device, (c) signaling from a device external to the first device, or (d) signaling a subunit within the first device.
The listed exemplary mode 19 is a method as described in the listed exemplary mode 1, which further comprises communicating, from the first device to the second device downstream in the media chain, one or more types of metadata
118
IMPI MEXICAN INSTITUTE OF MONEDAD INDUjntlAL
<img file="MX359652B_D0130.tif" />
independent of the state of the media data.
The listed exemplary mode 20 is a method as described in the listed exemplary mode 1, wherein the state of the media data comprises at least a portion of the hidden state metadata on one or more secure communication channels.
The numbered instance mode 21 is a method as described in the numbered exemplary mode 1, which further comprises altering a plurality of bytes in media data to store at least a portion of the state of the media data.
The exemplary modality listed 22 is a method as described in the exemplary modality listed 1, wherein at least one of the first device and the second device comprise one or more of the Advanced Television Systems Committee (ATSC) codes, Motion Picture Experts Group (MPEG), Audio Code 3 Code (AC-3), and Enhanced AC-3 Code.
The numbered exemplary mode 23 is a method as described in the numbered exemplary mode 1, wherein the media processing chain comprises: a pre-processing unit configured to accept time domain samples comprising media content as input and to output processed time domain samples; an encoder configured to send out
119
IMPI
INSTITUTO MEXICANO DEUNiOHEDAD INDUSTRIAL media bit streams rnmpr-j mi do <+ ...- ^ to media content based on time domain samples processed; a signal analysis and metadata correction unit configured to validate processing state metadata in the compressed media bitstream; a transcoder configured to modify the compressed media bit stream; a decoder configured to output decoded time domain samples based on the compressed media bit stream; and a post-processing unit configured to perform post-processing of the media content on the decoded time domain samples.
Enumerated exemplary mode 24 is a method as described in numbered exemplary mode 23, wherein at least one of the first device and the second device comprise one or more of the preprocessing unit, the metadata correction and analysis unit. signal, the transcoder, the decoder and the post-processing unit.
The listed exemplary mode 25 is a method as described in the listed exemplary mode 23, wherein at least one of the pre-processing unit, the metadata correction and signal analysis unit, the transcoder, the decoder and the unit post-processing performs adaptive content processing
120
<img file="MX359652B_D0131.tif" />
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL MONEDAD
<img file="MX359652B_D0132.tif" />
of media based on metadata processing received from an upstream device.
The listed exemplary mode 26 is a method as described in the listed exemplary mode 1, further comprising determining one or more media characteristics of the media data; include a description of the one or more media features in the media data state.
The numbered exemplary mode 27 is a method as described in the numbered exemplary mode 26, wherein the one or more media characteristics comprise at least one particular media characteristic of one or more frames, seconds, minutes, intervals of weather
<td>user definable,</td><td>scenes,</td><td>songs,</td><td>pieces</td><td>of</td><td>music,</td>
<td>and recordings.</td><td></td><td></td><td></td><td></td><td></td>
<td>The modality</td><td>copy</td><td>numbered</td><td>28 is</td><td>a</td><td>method</td>
<td>as described in</td><td colspan="2">exemplary modality</td><td colspan="2">numbered</td><td>26, the</td>
one or more media features comprises a semantic description of the media data.
The numbered exemplary embodiment 29 is a method as described in the numbered exemplary embodiment 26, the one or more characteristics of the media comprises one or more of structural properties, tonality including harmony and melody, timbre, rhythm, volume, stereo mixing, a number of sound sources of the media data,>
121
Industrial IMPI
<img file="MX359652B_D0133.tif" />
absence or presence of voice, characteristics — rupetlci'óñT melody, harmonies, lyrics, timbre, perceptual characteristics, digital medium characteristics, stereo parameters, one or more portions of speech content.
The numbered exemplary mode 30 is a method as described in the numbered exemplary mode 26, which further comprises using the one or more media features to classify the media data into one or more media data classes into a plurality of classes of media data.
The numbered exemplary mode 31 is a method as described in the numbered exemplary mode 30, wherein the one or more kinds of media data comprises one or more than one single dominant / total media data class for a whole piece of media, or a single class that represents a smaller period of time than the entire media piece.
The numbered exemplary mode 32 is a method as described in the numbered exemplary mode 31, wherein the smallest period of time represents one or more than a single media frame, a single media data block, multiple media frames, multiple media data blocks, a fraction of a second, one second, or multiple seconds.
The modality 'numbered instance 33 is a method
122
<img file="MX359652B_D0134.tif" />
IMPI as described in the j ^ mpi? .R_ modality. «Numbered —— gTT where one or more media data class labels representing the one or more media data classes are computed and inserted into a bit stream.
The exemplary mode numbered as described in the exemplary mode where a represent compute and recipient is an enumerated method
30, in or more media data class labels the one or more data classes point to a processing node as hidden data embedded with media media data media data.
The enumerated exemplary mode is a method as described in enumerated exemplary mode 30, wherein one or more media data class labels representing the one or more media data classes are computed and pointed to a data processing node. container media in a separate metadata structure between blocks of media data.
The numbered exemplary mode 36 is a method as described in the numbered exemplary mode
31, where the single dominant / total media data class presents one or more than one single class type such as music, speech, noise, silence, applause, or a mix class type such as talk about music, conversation over noise, or other mixes of media data types.
123
IMPI
<img file="MX359652B_D0135.tif" />
The exemplary modality listed 37 * <sup>r</sup>is '' a method as described in the exemplary mode listed 30, further comprising associating one or more probability or possibility values with the one or more media data class labels, where a-probability or possibility value represents a confidence level that a calculated media class label has with respect to a media block / segment with which the calculated media class label is associated.
The numbered exemplary mode 38 is a method as described in the numbered exemplary mode 37, wherein the probability or chance value is used by a recipient media processing node in the media processing chain to tailor processing in a manner to enhance one or more operations such as headphone mixing, encoding, decoding, transcoding, or virtualization.
Enumerated Exemplary Mode 39 is a method as described in Enumerated Exemplary Mode 38, where at least one of one or more operations eliminates a need for predetermined processing parameters, reduces the complexity of processing units across the chain of media ,, or increases battery life, such as complex analysis operations to classify media data by the media processing node
<img file="MX359652B_D0136.tif" />
124 container, are avoided.
The listed exemplary embodiment 40 is a method comprising: determining by a first device in a media processing chain, whether a type of media processing has already been performed in a media data feed version; in response to determining, by the first device, that the type of media processing has already been performed on the input version of the media data, performing adaptive processing of the media data to disable the performance of the media processing type. media in the first-device; where the method is performed by one or more computational processors.
The listed exemplary mode 41 is a method as described in the listed exemplary mode 40, further comprising: communicating, from the first device to a second device downstream in the media processing chain, an output version of the data from media and a media data status indicating that the type of media processing has been performed on the output version of the media data.
The listed exemplary mode 42 is a method as described in the listed exemplary mode 41, which further comprises encoding dynamic range and noise volume values in the media data state.
The exemplary mode listed 43 is a method
125
IMPI
<img file="MX359652B_D0137.tif" />
as described in the exemplary embodiment listed 40, further comprising: performing, by the first device, a second type of media processing on the media data, the second type of media processing different from the type of media processing; communicate from the first device to a second device downstream in the media processing chain, an output version of the media data and a status of the media data indicating that the type of media processing and the second type of Media processing has already been done in the output version to the media data.
The listed exemplary mode 44 is a method as described in the listed exemplary mode 40, further comprising: automatically performing one or more dynamic audio processing or adaptive corrective noise volume based at least in part on whether the type of Processing has been done previously input or feed version of the media data.
The numbered exemplary mode 45 is a method as described in the numbered exemplary mode 40, further comprising: extracting an input state of the media data from data units in the media data encoding media content, wherein the state Media data entry is hidden in one or more of the data drives.
126
IMPI
<img file="MX359652B_D0138.tif" />
The numbered exemplary mode 46 is a method as described in the numbered exemplary mode 45, further comprising retrieving a version of the data units that does not comprise the input state of the media data and rendering the media content based on the version of the data units that have been recovered.
The numbered exemplary mode 47 is a method as described in the numbered exemplary mode 46, further comprising retrieving an input state of the media data that is associated with the input version of the media data.
Enumerated exemplary mode 48 is a method as described in enumerated exemplary mode 47, further comprising authenticating the media data entry state by validating a cryptographic hash value associated with the entry state of the media data.
The numbered exemplary mode 49 is a method as described in the numbered exemplary mode 47, further comprising authenticating the input status of the media data by validating one or more traces associated with the input status of media data, wherein the At least one of one or more footprints is generated based on at least a portion of the media data.
The numbered exemplary mode 50 is a method as described in the numbered exemplary mode 47,
127
<img file="MX359652B_D0139.tif" />
It also comprises validating the medieg- ^ ri? · ^ ral-idar · ΒΠ'Β · -ο plus footprints associated with the input status of the media data, where at least one of the one or more footprints is generated based on at least a portion of the media data.
The numbered exemplary mode 51 is a method as described in the numbered exemplary mode 47, wherein the input state of the media data is conveyed with the input version of the media data in an input media bitstream .
The numbered exemplary mode 52 is a method as described in the numbered exemplary mode 47, further comprising: disabling one or more types of media processing based on the media data entry state.
Enumerated exemplary mode 53 is a method as described in enumerated exemplary mode 47, wherein the input state of the media data is described with processing status metadata; and further comprises: creating media processing signaling based at least in part on the processing status metadata, wherein the media processing signaling indicates the input state of the media data; transmit the media processing signaling to a downstream media processing device to the first device
128
IMPI
<img file="MX359652B_D0140.tif" />
in the media processing chain.
The numbered exemplary mode 54 is a method as described in the numbered exemplary mode 53, wherein the media processing signaling is hidden in one or more data units in an output version of the media data.
The numbered exemplary mode 55 is a method as described in the numbered exemplary mode 54, wherein the media processing signaling is performed using a reversible data concealment technique such that one or more modifications to the media data are removed by a container device.
The numbered exemplary mode 56 is a method as described in the numbered exemplary mode 54, wherein the media processing signaling is performed using an irreversible data concealment technique such that at least one of one or more modifications to the data of media is not removed by a container device.
The numbered exemplary mode 57 is a method as described in the numbered exemplary mode 46, further comprising receiving, from a device upstream in the media processing chain, one or more types of metadata independent of any past media processing that is performed on the media data.
The exemplary mode listed 58 is a method
129
IMPI RWUCANO INSTITUTE OF INDUSTRIAL PROPERTY
<img file="MX359652B_D0141.tif" />
as described in the exemplary embodiment listed 47, wherein the media data state comprises at least a portion of the hidden state metadata on one or more secure communication channels.
The numbered exemplary mode 59 is a method as described in the numbered exemplary mode 46, further comprising altering a plurality of bytes in the media data to store at least a portion of a state of the media data.
The exemplary mode numbered 60 is a method as described in the exemplary mode numbered 46, wherein the first device comprises one or more codes of the Committee on Advanced Television Systems (ATSC), codes of the Expert Group of Motion Images (MPEG ), Audio Codee 3 (AC-3) code, and Enhanced AC-3 code.
The numbered exemplary mode 61 is a method as described in the numbered exemplary mode 46, wherein the media processing chain comprises: a pre-processing unit configured to accept time domain samples comprising media content as input and to output processed time domain samples; an encoder configured to output compressed medium bitstream output of the media content based on the processed time domain samples; a unit for metadata correction and signal analysis, ce
130
IMPI Mexican INSTrRuro DE LA PROREDAD industrial
<img file="MX359652B_D0142.tif" />
configured to validate processing status metadata on the compressed media bitstream; a transcoder configured to modify the compressed media bit stream; a decoder configured to output decoded time domain samples based on the compressed media bit stream; and a post-processing unit configured to post-process the media content in the decoded time domain samples.
The listed exemplary mode 62 is a method as described in the listed exemplary mode 61, wherein the first device comprises one or more of the pre-processing unit, the metadata correction and signal analysis unit, the transcoder, the decoder, and post-processing unit.
The listed exemplary mode 63 is a method as described in the listed exemplary mode 61, wherein at least one of the pre-processing unit, the signal analysis and metadata correction unit, the transcoder, the decoder, and the post-processing unit performs adaptive processing of media content based on processing metadata received from an upstream device.
The numbered exemplary mode 64 is a method as described in the numbered exemplary mode 47,
131
IMPI ΜίΧΙΟβΟ INSTITUTE OF THE INDUSTRIAL PROPERTY
<img file="MX359652B_D0143.tif" />
it further comprises determining one or more media features based on a description of the one or more media features in the media data state.
The numbered exemplary mode 65 is a method as described in the numbered exemplary mode 64, wherein the one or more media characteristics comprise at least one media characteristic determined from one or more frames, seconds, minutes, time intervals user-defined scenes, songs, pieces of music and recordings.
The numbered exemplary mode 66 is a method as described in the numbered exemplary mode 64, the one or more media characteristics comprise a semantic description of the media data.
The numbered exemplary mode 67 is a method as described in the numbered exemplary mode 64, further comprising performing one or more specific operations in response to determining the one or more media characteristics.
Enumerated exemplary mode 68 is a method as described in 'numbered exemplary mode 43, further comprising providing, to the second device in the media processing chain, the status of the media data as one or more of: (a) media footprints, (b) processing status metadata, (c) values of
132
IMPI
<img file="MX359652B_D0144.tif" />
extracted media characteristics, (dj désuiiperón - cr descriptions and / or values of types or sub-type of media class, (e) probability values of class and / or sub-class of media characteristics, (f) value cryptographic hash or (f) media processing signaling.
The listed exemplary embodiment 69 is a method comprising: calculating, with a first device in a media processing chain, one or more reduced data rate representations of a media data source frame; and transporting the one or more reduced data rate representations simultaneously and securely, within a state of the media data itself, to a second device in the media processing chain; where the method is performed by one or more computing devices.
The numbered exemplary mode 70 is a method as described in the numbered exemplary mode 69, wherein the one or more reduced data rate representations are carried in at least one of a sub-stream, one or more reserved fields, a field add_bsi, one or more auxiliary data fields, or one or more transformation coefficients.
The numbered exemplary mode 71 is a method as described in the numbered exemplary mode 69, wherein the one or more reduced speed representations of
133
IMPI
<img file="MX359652B_D0145.tif" />
data includes synchronization data ...... ~ Sm ^ Tfe'adC> S - paTS<sup>1</sup> sync supplied audios and videos within media data.
The numbered exemplary mode 72 is a method as described in the numbered exemplary mode 69, wherein the one or more reduced data rate representations comprise media footprints (a) generated by a media processing unit and (b) embedded with media data for one or more quality monitoring, media metrics, media tracking, or content search.
The numbered exemplary embodiment 73 is a method as described in the numbered exemplary embodiment 69, wherein at least one of the one or more reduced representations
<td>in speed</td><td>of data</td><td>understands</td><td>at least one</td><td>portion</td><td>of</td>
<td>the metadata</td><td colspan="2">hidden state s</td><td>in one or more</td><td>channels</td><td>of</td>
<td>communication</td><td>safe.</td><td></td><td></td><td></td><td></td>
<td>The</td><td>modality</td><td>copy</td><td>numbered 74 is</td><td colspan="2">a method</td>
as described in the exemplary embodiment listed 69, it further comprises altering a plurality of bytes in the media data to store at least a portion of the one or more reduced data rate representations.
The numbered exemplary mode 75 is a method as described in the numbered exemplary mode 69, wherein at least one of the first device and the second device
134
IMPI
<img file="MX359652B_D0146.tif" />
device comprise one or more of codaivs · -te! · eeimiÍbér-tfe · Advanced Television Systems (ATSC), codes of the Expert Group of Moving Images (MPEG), codes of Audio Codee 3 (AC-3), and codes AC-3 improved.
The numbered exemplary mode 76 is a method as described in the numbered exemplary mode 69, wherein the media processing chain comprises: a pre-processing unit configured to accept time domain samples comprising media content such as feed and to output processed time domain samples; an encoder configured to output compressed media bit streams of media content based on the processed time domain samples; a signal analysis and metadata correction unit configured to validate processing state metadata in the compressed media bitstream; a transcoder configured to modify the compressed media bit stream; a decoder configured to output decoded time domain samples based on the compressed media bit stream; and a post-processing unit configured to perform post-processing of the media content on the decoded time domain samples.
The numbered exemplary mode 77 is a method as described in the numbered exemplary mode 76, in
135
IMPI
<img file="MX359652B_D0147.tif" />
wherein at least one of the first device and the second device comprise one or more of the preprocessing unit, the metadata correction and signal analysis unit, the .transcoder, the decoder and the post-processing unit.
The numbered exemplary mode 78 is a method as described in the numbered exemplary mode 76, wherein at least one of the pre-processing unit, the metadata correction and signal analysis unit, the transcoder, the decoder, and the post-processing unit performs adaptive processing of media content based on processing metadata received from an upstream device.
The numbered exemplary mode 79 is a method as described in the numbered exemplary mode 69, further comprising providing, to the second device, the status of the media data as one or more of: (a) media footprints, (b) processing status metadata, (c) values of extracted media characteristics, (d) description (s) and / or values of types or sub-types of media classes, (e ) probability values of class and / or sub-class of media characteristics, (f) cryptographic hash value or (f) signaling of media processing.
The exemplary modality listed 80 is a method comprising: adaptive processing, with one more
136
IMPI
MifcCAWO INSTITUTE
OF THE PROPERTY
INDUSTRIAL
<img file="MX359652B_D0148.tif" />
computing devices in a media processing chain comprising one or more of transformed psychoacoustic units, spatial audio / waveform encoding units, encoders, decoders, transcoders or current processors, a media data entry version based on a past history of volume processing of the media data by one or more upstream media processing units as indicated by a status of the media data; normalize noise volume and / or dynamic range of an output version of the media data at one end of the media processing chain for consistent values of noise volume and / or dynamic range.
The listed exemplary mode 81 is a method as described in the listed exemplary mode 80, wherein the consistent noise volume value comprises a noise volume value of (1) controlled or selected by a user or (2) noted in Adaptive form by a state of the input version of the media data.
The numbered exemplary mode 82 is a method as described in the numbered exemplary mode 80, wherein the noise volume value is calculated in the dialogue (speech) portions of the media data.
The numbered exemplary mode 83 is a method as described in the numbered exemplary mode 80, in
<img file="MX359652B_D0149.tif" />
137
IMPI
MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY
<td>where he</td><td>value of</td><td>volume of</td><td>noise</td><td>calculate</td><td>in the</td>
<td>servings</td><td>absolute,</td><td>relative and / or</td><td colspan="2">without control of</td><td>Data of</td>
<td>media.</td><td></td><td></td><td></td><td></td><td></td>
<td></td><td colspan="2">The exemplary modality</td><td>numbered</td><td colspan="2">84 is a method</td>
as described in the exemplary mode listed 80, wherein the consistent dynamic range value comprises a dynamic range value of (1) controlled or selected by a user or (2) adaptively reported by a state in the power version of the media data.
The numbered exemplary mode 85 is a method as described in the numbered exemplary mode 84, wherein the dynamic range value is calculated in the dialog (speech) portions of the media data.
The numbered exemplary mode 86 is a method as described in the numbered exemplary mode 84, wherein the dynamic range value is calculated in absolute, relative, and / or uncontrolled portions of the media data.
The listed exemplary mode 87 is a method as described in the listed exemplary mode 80, further comprising: calculating one or more dynamic range gain and / or noise volume control values to normalize the output version of the media data at a consistent noise volume value and a consistent dynamic range; simultaneously carry the one or more dynamic range and / or volume gain control values of
138
<img file="MX359652B_D0150.tif" />
IMPI
MÍWCAJW INSTITUTE ocuprohídac tNDURTMAi.
<td>noise</td><td>within</td><td>a</td><td colspan="2">state</td><td>the version</td><td>departure of</td>
<td>data</td><td>media</td><td>to the</td><td>final</td><td>of</td><td>The chain of</td><td>processing of</td>
<td>media</td><td>, where</td><td>the</td><td>one or</td><td>plus</td><td colspan="2">gain control values</td>
of dynamic range and / or noise volume are used by another device to inversely apply the one or more dynamic range gain and / or noise volume cohtrol values to retrieve an original noise volume value and original dynamic range in the input version of the media data.
The listed exemplary mode 88 is a method as described in the listed exemplary mode 87, wherein the one or more dynamic range and / or noise volume control values representing the status of the output version of the media data , are carried in at least one of a sub-stream, one or more reserved fields, an add_bsi field, one or more auxiliary data fields, or one or more transformation coefficients.
Enumerated exemplary mode 89 is a method as described in enumerated exemplary mode 80, further comprising calculating and transmitting, by at least one of the one or more computing devices in the media processing chain ', a cryptographic hash value based on in the media data and / or the status of the media data within one or more encoded bitstreams that carry the media data.
139
IMPI
INDUSTRIAL
<img file="MX359652B_D0151.tif" />
The numbered exemplary mode 90 "is a method as described in the numbered exemplary mode 89, further comprising: authenticating, by a canister device, the cryptographic hash value; signaling, by the container device to one or more downstream media processing units, a determination of whether the status of the media data is valid; signaling, by the container device to the one or more downstream media processing units, the status of the media data in response to determining that the status of the media data is valid.
Enumerated exemplary mode 91 is a method as described in enumerated exemplary mode 89, wherein the cryptographic hash value representing the state of the media and / or media data is carried in at least one of a substream, one or more reserved fields, an add_bsi field, one or more auxiliary data fields, or one or more transform coefficients.
The numbered exemplary mode 92 is a method as described in the numbered exemplary mode 80, wherein the state of the media data comprises one or more of: (a) media gaps, (b) processing status metadata, ( c) values of characteristics of extracted media, (d) description (s) and / or values of types or subtypes of media classes, (e) probability values of
140
IMPI
<img file="MX359652B_D0152.tif" />
class and / or sub-class of media characteristics, (f) cryptographic hash value or, (f) media processing signaling.
The listed exemplary embodiment 93 is a method comprising performing one of inserting, extracting or editing related and unrelated media data locations and / or a status of related and unrelated media data locations within one or more bit streams encoded by one or more computing devices in a media processing chain comprising one or more of transformed psychoacoustic units, spatial audio / waveform encoding units, current encoders, decoders, transcoders, or processors.
The listed exemplary mode 94 is a method as described in the listed exemplary mode 93, wherein the one or more related and unrelated media data locations and / or the status of related and unrelated media data locations within encoded bit streams are carried in at least one of a sub-stream, one or more reserved fields, an add_bsi field, one or more auxiliary data fields, or one or more transform coefficients.
The exemplary mode listed 95 is a method comprising performing one or more of inserting, extracting or editing
141
IMPI
INSTITUTO MEXICANO DE LA INDUSTRIAL related and unrelated media data and / or a state of related and unrelated media data within one or more encoded bitstreams by one or more computing devices in a media processing chain that it comprises one or more of transformed psychoacoustic units, spatial audio / waveform encoding units, encoders, decoders, transcoders, or current processors.
The numbered exemplary mode 96 is a method as described in the numbered exemplary mode 95, wherein the one or more related and unrelated media data and / or the state of related and unrelated data within encoded bitstream is they carry in at least one of a sub-stream, one or more reserved fields, an add_bsi field, one or more auxiliary data fields, or one or more transform coefficients.
The numbered exemplary mode 97 is a method as described in numbered exemplary mode 93, further comprising providing, from an upstream media processing device to a downstream media processing device, a status of the media data as one or more: (a) media footprints, (b) processing status metadata, (c) extracted media feature values, (d) description (s) and / or values of media class types or sub-types (e) values of
142
IMPI
<img file="MX359652B_D0153.tif" />
probability of class and / or sub-class of media characteristics, (f) cryptographic value has or (f) media processing signaling.
Enumerated exemplary mode 98 is a media processing system configured to compute and transport cryptographic hash values based on media data and / or a status of media data within one or more bitstreams encoded by one or more devices. computing in a media processing chain comprising one or more transformed psychoacoustic units, spatial audio / waveform encoding units, encoders, decoders, current transcoders or processors.
The numbered exemplary mode 99 is a media processing system as described in the numbered exemplary mode 98, wherein the state of the media data comprises one or more of: (a) media footprints, (b) processing status metadata, (c) extracted media feature values, (d) description (s) and / or media class type or sub-type values, (e ) probability values of class and / or sub-class of media characteristics, (f) cryptographic hash value or (f) signaling of media processing ·.
Enumerated Exemplary Mode 100 is a media processing system configured to process in the form
143
<img file="MX359652B_D0154.tif" />
IMPI
INSTITUTE
DF LA PUOHIiMD iNnWHHM adaptive media data based on a <sup>Qqho /</sup>io 1L<sup>1</sup>* 'dal'oT of media received from one or more secure communication channels.
The numbered exemplary mode 101 is a media processing system as described in the numbered exemplary mode 100, wherein the media processing system comprising one or more of processing nodes, and wherein the processing nodes comprise data processing systems. media supply, media distribution systems, and media representation systems.
Listed exemplary mode 102 is a media processing system as described in numbered exemplary mode 101, wherein the one or more secure communications channels comprise at least one secure communications channel through two or more of the streams. of compressed / encoded bits and PCM processing nodes.
The listed exemplary mode 103 is a media processing system as described in the listed exemplary mode 101, wherein the one or more secure communication channels' comprise at least one secure communication channel through two media processing devices. separated.
The listed exemplary mode 104 is a media processing system as described in the listed exemplary mode 101, wherein the one or more channels of
IMPI
<img file="MX359652B_D0155.tif" />
144 Secure communications comprise at least ...... one Ud'iTd'l * 'of secure communication through two media processing nodes in a single media processing device.
The numbered exemplary mode 105 is a media processing system as described in the numbered exemplary mode 100, wherein the media processing system is configured to perform autonomous media processing operations independent of how the media processing systems are ordered. media in a media processing chain of which the media processing system is part.
The numbered exemplary mode 106 is a media processing system as described in the numbered exemplary mode 100, wherein the state of the media data comprises one or more of: (a) media footprints, (b) processing status metadata, (c) extracted media feature values, (d) description (s) and / or values of media class types or sub-type, (e ) probability values of class and / or sub-class of media characteristics, (f) cryptographic hash value or (f) signaling of media processing.
Enumerated exemplary mode 107 is a media processing system configured to perform any one of the methods described in numbered exemplary modes 1-99.
IMPI
<img file="MX359652B_D0156.tif" />
145
The listed exemplary mode 108 is an apparatus comprising a processor and configured to perform any of the methods as described in the listed exemplary modes 1-99.
Enumerated exemplary mode 107 is a computer readable storage medium comprising software instructions, which when executed by one or more processors, cause any of the methods to perform as described in numbered exemplary modes 1-99.
9. EQUIVALENTS, EXTENSIONS, ALTERNATIVES AND MISCELLANEOUS
In the above specification, possible embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what the invention is and is intended by the applicants as the invention, is the set of claims that arise from this application, in the specific way in which these claims arise, including any correction. subsequent .. Any definitions expressly set forth herein for terms contained in these claims shall regulate the meaning of these terms as used in the claims. Therefore, no limitation, element, property, characteristic, advantage or attribute that
146
IMPI
MEXICAN INSTITUTE M LA MONEDAD INDUSTRIAL
<img file="MX359652B_D0157.tif" />
It is not expressly described in a claim and should limit the scope of this claim in any way.
The specification and drawings are accordingly to be considered in an illustrative rather than restrictive manner.
147
t] GÑI
IMPIñ INSTITUTO MMICANO V, DE LA PROPIEDAD V INDUSTRIAL
Contents174
174 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40 Sheet 41 Sheet 42 Sheet 43 Sheet 44 Sheet 45 Sheet 46 Sheet 47 Sheet 48 Sheet 49 Sheet 50 Sheet 51 Sheet 52 Sheet 53 Sheet 54 Sheet 55 Sheet 56 Sheet 57 Sheet 58 Sheet 59 Sheet 60 Sheet 61 Sheet 62 Sheet 63 Sheet 64 Sheet 65 Sheet 66 Sheet 67 Sheet 68 Sheet 69 Sheet 70 Sheet 71 Sheet 72 Sheet 73 Sheet 74 Sheet 75 Sheet 76 Sheet 77 Sheet 78 Sheet 79 Sheet 80 Sheet 81 Sheet 82 Sheet 83 Sheet 84 Sheet 85 Sheet 86 Sheet 87 Sheet 88 Sheet 89 Sheet 90 Sheet 91 Sheet 92 Sheet 93 Sheet 94 Sheet 95 Sheet 96 Sheet 97 Sheet 98 Sheet 99 Sheet 100 Sheet 101 Sheet 102 Sheet 103 Sheet 104 Sheet 105 Sheet 106 Sheet 107 Sheet 108 Sheet 109 Sheet 110 Sheet 111 Sheet 112 Sheet 113 Sheet 114 Sheet 115 Sheet 116 Sheet 117 Sheet 118 Sheet 119 Sheet 120 Sheet 121 Sheet 122 Sheet 123 Sheet 124 Sheet 125 Sheet 126 Sheet 127 Sheet 128 Sheet 129 Sheet 130 Sheet 131 Sheet 132 Sheet 133 Sheet 134 Sheet 135 Sheet 136 Sheet 137 Sheet 138 Sheet 139 Sheet 140 Sheet 141 Sheet 142 Sheet 143 Sheet 144 Sheet 145 Sheet 146 Sheet 147 Sheet 148 Sheet 149 Sheet 150 Sheet 151 Sheet 152 Sheet 153 Sheet 154 Sheet 155 Sheet 156 Sheet 157 Sheet 158 Sheet 159 Sheet 160 Sheet 161 Sheet 162 Sheet 163 Sheet 164 Sheet 165 Sheet 166 Sheet 167 Sheet 168 Sheet 169 Sheet 170 Sheet 171 Sheet 172 Sheet 173 Sheet 174
607 members in 28 offices
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 61419747 | United States of America | – | |
| 41974710 | United States of America | P | |
| 61558286 | United States of America | – | |
| 201161558286 | United States of America | P | |
| 2011062828 | United States of America | W |
Members607
| Document | Office | Kind | |
|---|---|---|---|
| CA2816889A1 | Canada | A1 | |
| CA2998405A1 | Canada | A1 | |
| CA3216692A1 | Canada | A1 | |
| WO2012075246A2 | World Intellectual Property Organization (WIPO) | A2 | |
| TW201236446A | Taiwan Province of China | A | |
| AR084086A1 | Argentina | A1 | |
| DE202013001075U1 | Germany | U1 | |
| AU2011336566A1 | Australia | A1 | |
| JP3183637U | Japan | U | |
| MX2013005898A | Mexico | A | |
| IL226100A0 | Israel | A0 | |
| IL226100D0 | Israel | D0 | |
| SG190164A1 | Singapore | A1 | |
| DE202013006242U1 | Germany | U1 | |
| CN203134365U | China | U | |
| US2013246077A1 | United States of America | A1 | |
| EP2647006A1 | European Patent Office (EPO) | A1 | |
| JP3186472U | Japan | U | |
| KR20130111601A | Republic of Korea | A | |
| CL2013001571A1 | Chile | A1 | |
| CN103392204A | China | A | |
| TWM467148U | Taiwan Province of China | U | |
| CN203415228U | China | U | |
| JP2014505898A | Japan | A | |
| CN103943112A | China | A | |
| CA2888350A1 | Canada | A1 | |
| WO2014113465A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2014113471A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2014113478A1 | World Intellectual Property Organization (WIPO) | A1 | |
| FR3001325A3 | France | A3 | |
| UA106163C2 | Ukraine | C2 | |
| WO2014124377A2 | World Intellectual Property Organization (WIPO) | A2 | |
| KR20140106760A | Republic of Korea | A | |
| KR101438386B1 | Republic of Korea | B1 | |
| TWM487509U | Taiwan Province of China | U | |
| TW201442020A | Taiwan Province of China | A | |
| WO2014124377A3 | World Intellectual Property Organization (WIPO) | A3 | |
| CA2898891A1 | Canada | A1 | |
| CN104240709A | China | A | |
| WO2014204783A1 | World Intellectual Property Organization (WIPO) | A1 | |
| FR3007564A3 | France | A3 | |
| KR20140006469U | Republic of Korea | U | |
| RU2013130293A | Russian Federation | A | |
| TW201506911A | Taiwan Province of China | A | |
| SG11201502405RA | Singapore | A | |
| IL237561A0 | Israel | A0 | |
| IL237561D0 | Israel | D0 | |
| KR20150047633A | Republic of Korea | A | |
| AU2014207590A1 | Australia | A1 | |
| HK1198674A | Hong Kong, China | A | |
| HK1198674A1 | Hong Kong, China | A1 | |
| CN104737228A | China | A | |
| FR3001325B3 | France | B3 | |
| MX2015004468A | Mexico | A | |
| AU2014281794A1 | Australia | A1 | |
| EP2901449A1 | European Patent Office (EPO) | A1 | |
| TWI496461B | Taiwan Province of China | B | |
| AU2014207590B2 | Australia | B2 | |
| AU2014281794B2 | Australia | B2 | |
| IN1633MUN2015A | India | A | |
| IN1765MUN2015A | India | A | |
| IN1766MUN2015A | India | A | |
| SG11201505426XA | Singapore | A | |
| IL239687A0 | Israel | A0 | |
| IL239687D0 | Israel | D0 | |
| KR20150099586A | Republic of Korea | A | |
| KR20150099615A | Republic of Korea | A | |
| KR20150099709A | Republic of Korea | A | |
| KR200478147Y1 | Republic of Korea | Y1 | |
| AU2014281794B9 | Australia | B9 | |
| KR20150105955A | Republic of Korea | A | |
| CN104937844A | China | A | |
| CN104995677A | China | A | |
| MX2015010477A | Mexico | A | |
| JP2015531498A | Japan | A | |
| CN105027478A | China | A | |
| HK1204135A | Hong Kong, China | A | |
| HK1204135A1 | Hong Kong, China | A1 | |
| US2015325243A1 | United States of America | A1 | |
| FR3007564B3 | France | B3 | |
| TW201543469A | Taiwan Province of China | A | |
| RU2568372C2 | Russian Federation | C2 | |
| EP2946469A1 | European Patent Office (EPO) | A1 | |
| EP2946495A1 | European Patent Office (EPO) | A1 | |
| US2015348558A1 | United States of America | A1 | |
| EP2954515A1 | European Patent Office (EPO) | A1 | |
| US2015363160A1 | United States of America | A1 | |
| US2015372820A1 | United States of America | A1 | |
| IL239687A | Israel | A | |
| TWI524329B | Taiwan Province of China | B | |
| JP2016507088A | Japan | A | |
| JP5879362B2 | Japan | B2 | |
| JP2016507779A | Japan | A | |
| TW201610984A | Taiwan Province of China | A | |
| KR20160032252A | Republic of Korea | A | |
| JP2016510544A | Japan | A | |
| MX338238B | Mexico | B | |
| CA2888350C | Canada | C | |
| CA2898891C | Canada | C | |
| CN103392204B | China | B |
Numbers
- Publication
- 359652
- Application
- 2016004510
Titles2
- Spanish
- PROCESAMIENTO ADAPTATIVO CON MULTIPLES NODOS DE PROCESAMIENTO DE MEDIOS.
- English
- ADAPTIVE PROCESSING WITH MULTIPLE MEDIA PROCESSING NODES.
Classification
- CPC, 7
- G10L19/008
- G10L19/16
- G10L21/00
- G10L19/167
- G10L19/26
- G10L21/0316
- H03M7/30
- IPC, 3
- G10L19 008
- G10L19 16
- G10L21 00
