Power aware video decoding and streaming.

(54) Title: DECODING AND EMISSION IN REAL TIME OF VIDEO SENSITIVE TO ENERGY.

(54) Title: POWER AWARE VIDEO DECODING AND STREAMING.

(57) Summary

The present invention relates to methods and systems for a mobile device to decode video based on available power. For example, the mobile device may receive a Media Description File (MDF Media Description File) for a video stream from a video server. The MDF can include complexity information associated with a plurality of video segments. The complexity information can be related to the amount of processing power to be used to decode the segment on the mobile device. The mobile device can determine at least one power indicator for the mobile device. The mobile device can determine a first level of complexity that will be requested for a first video segment based on the complexity information derived from the MDF and the power indicator. The mobile device can dynamically modify the decoding process to save power based on the detected power level.

(57) Abstract

Methods and systems are disclosed for a mobile device to decode video based on available power and / or energy. For example, the mobile device may receive a media description file (MDF) from for a video stream from a video server. The MDF may include complexity Information associated with a plurality of video segments. The complexity Information may be related to the amount of Processing power to be utilized for decoding the segment at the mobile device. The mobile device may determine at least one power metric for the mobile device. The mobile device may determine a first complexity level to be requested for a first video segment based on the complexity information from the MDF and the power metric. The mobile device may dynamically alter the decoding process to save energy based on the detected power / energy level.

Institute Π

Mexican Q Γ of Property Q LJM |

Industrial Π

PATENT TITLE NO. 345571

Holders): VIO SCALE, INC

Address: 200 Bellevue Parkway, Suite 300, Wilmington, Delaware, 19809-3727, USA

Denomination: ENERGY SENSITIVE VIDEO DECODING AND REAL-TIME EMISSION.

Classification: lnt.CI.8: H04L29 / 06; H04N21 / 2343; H04N21 / 414; H04N21 / 442; H04N21 / 845; H04N21 / 854: H04W52 / 00

Inventors): YUWEN HE, YAN YE; YONG HE; GEORGE W. MCCLELLAN; EUN RYU

REQUEST

Number: International filing date:

MX / a / 2015/000266 July 09, 2013

PRIORITY

Country: Date: Number:

US July 9, 2012 61 / 669,581

Validity: Twenty years ....... ........

Expiration Date: July 9, 2033

The reference patent is granted based on articles 1, 2nd section V, 6th section III, and 59 of the Industrial Property Law.

In accordance with article 23 of the Industrial Property Law, this patent has a validity of twenty years, non-extendable, counted from the filing date of the international application and will be subject to payment (the fee to keep the rights

Whoever signs the present title does so based on the provisions of articles β * fractions lll and 7 bis 2 of the Industrial Property Law (Official Gazette of the Federation (DOF) 08/27/1991, amended on 02 / 08/1994, 25 * 10/1996, 12/26/1997, 05/17/1999, 01/26/2004, 06/16/2005, 01/25/2006, 05/06/2009, 06/01 / 2010, 1WQ6 / 201Q, 08/28/2010, 2701/2012 and 04/09/2012); articles 1 «, 3^or Section V subsection a), 4th and 12th sections I and III of the Regulations of the Mexican Institute of Industrial Property (DOF 12/14/1999, amended on 07/91/2002, 07/15/2004, 07/28/1999 2004 and 9/7/2007); Articles 1, 3, 4, 8 'section V subsection a), 16 sections I and III and 30 of the Organic Statute of the Mexican Institute of Industrial Property (DOF 12/27/1999, amended on 10/10/2002, 07/29/2004, 08/04/2004 and 09/13/2007); 1, 3 and 5 * Subsection a) of the Agreement that delegates powers to the Deputy General Directors. Coordinator, Divisional Directors, Heads of Regional Offices, Divisional Deputy Directors, Departmental Coordinators and other subordinates of the Mexican Institute of Industrial Property. (DOF 12/15/1999, amended on 02/04/2000, 07/29/2004, 08/04/2004 and 09/13/2007).

<img file="MX345571B_D0001.tif" />

MX / 2017/11623

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0002.tif" />

DECODING AND REAL-TIME BROADCAST OF SENSITIVE VIDEO

TO ENERGY

FIELD OF THE INVENTION

As the proliferation of mobile devices continues to increase, they are being used for a greater variety of applications. Also, mobile devices, such as smartphones, are using more and more complex functionality. Such increases in processing can increase power consumption, which can adversely affect the user experience in power-limited situations, such as when a battery is the power source for the device.

BACKGROUND OF THE INVENTION

Methods and systems are described for a mobile device to decode video based on available energy. For example, the mobile device may receive a Media Description File (MDF) for a video stream from a video server. The MDF may include information on the complexity associated with a plurality of video segments. The complexity information can be related to the amount of processing power to be used to decode the segment in the device.

<img file="MX345571B_D0003.tif" />

IMPI

MEXICAN INSTITUTE DE LA nOFlEDAR mobile industrial. The mobile device can determine at least one power indicator for the mobile device. For example, the power indicator can be an amount of power remaining for a current power source such as a battery. The mobile device may determine a first level of complexity to be requested for a first video segment based on the complexity information derived from the MDF and the power indicator. The mobile device can request the first video segment at the first complexity level from the video server.

For example, the complexity information in the MDF can be provided per video segment. Some examples of power indicators that can be used to determine the appropriate level of complexity to request may include the amount of power to allocate to decode the rest of the video, the amount of power used for a previous decoding, the statistics collected on the mobile device. relating to previously decoded segments, cost information in the form of a ratio of the energy used to decode to the quality of the segment, cost information in the form of a ratio of energy used to segment complexity, and / or the like. The device, mobile can determine a duration of the rest of the flow of

IΜ ΡI

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL

<img file="MX345571B_D0004.tif" />

video, and you can determine a duration of the first video segment. The mobile device can then determine an amount of power to be allocated to the first video segment based on the amount of power to be allocated to decode the rest of the video stream, the duration of the remainder of the video stream, and the duration of the first video stream. video segment. The mobile device may make the determination based on the assumption that each segment remaining in the video stream will be allocated an equal amount of power for decoding. The sum of the energies allocated to each of the remaining segments may be less than or equal to the total energy allocated to complete decoding of the video.

BRIEF DESCRIPTION OF THE INVENTION

The mobile device can determine the first level of complexity to be requested for the first video segment based on the complexity information derived from the MDF and the power indicator by determining a level of complexity that will cause a decoding of the first video segment. video using approximately the amount of power to be allocated to the first video segment. For example, the levels of complexity can be quantified, and the mobile device can select that more complex level that meets a desired goal of energy use. The device

WICKED

MEXICAN INSTITUTE

OF THE PROPERTY &lt;

INDUSTRIAL mobile can determine the first level of complexity to be requested for the first video segment based on the complexity information derived from the MDF and the energy indicator by selecting the first level of complexity based on an assigned energy for a segment above, a level of complexity used to decode the previous segment, and an allocated power to decode the first video segment.

The mobile device can adjust the decoding process autonomously or jointly with the network in order to save energy, while simultaneously limiting the introduction of decoding errors. For example, the mobile device can determine if a region of the first video segment includes high-frequency components that exceed a high-frequency threshold. The mobile device can apply a first interpolation filter during motion compensation for the region on the condition that the high-frequency components exceed the high-frequency threshold, or the mobile device can apply a second interpolation filter for motion compensation. movement for the region on the condition that high-frequency components do not exceed the high-frequency threshold. The second interpolation filter can be associated with shorter taps (for example, a shorter

IMPI

MEXICAN INSTITUTE »e LA PRONEDA» INDUSTRIAL

<img file="MX345571B_D0005.tif" />

cutoff frequency) than the first interpolation filter, consequently limiting memory access and saving power. The mobile device may determine not to apply the filter with shorter taps (for example, a lower cutoff frequency), although there are few high-frequency components based on the determination that the region is a reference for other regions, which means that the possibility of propagating an error is greater. The mobile device can perform unlocking for reference images on the first video segment, and may refrain from unlocking for non-reference images.

A mobile device or other wireless receiving transmission unit may include one or more functional components to dynamically modify video decoding based on power conditions. For example, the mobile device may include a transceiver that can be configured to request an MDF for a video file from a video server and to receive the MDF from the video server, wherein the MDF may include information on the complexity for a plurality of video segments. The mobile device may include a complexity control and statistics unit that may be configured to store statistics for decoding the previous segment. For example,

<img file="MX345571B_D0006.tif" />

⁶ IMPI

INSTITUTO MbXICANC

Dt LA PROFIERAC

INDUSTRIAL statistics for the decoding of the previous segment may include energy information associated with the decoding of at least one previous segment and information on the complexity for at least the previous segment. The mobile device may include an energy detector that can be configured to determine current energy level information. The mobile device may include an energy sensitive adaptive controller that can be configured to determine a level of complexity to request for a subsequent segment based on the complexity information derived from the MDF, the statistics for the decoding of the previous segment, and current energy information. The mobile device may include a decoder configured to determine a parameter or method to apply to decode the subsequent segment based on the information determined by the energy sensitive matching controller.

For example, the decoder is configured to determine an interpolation filter to apply for motion compensation of a certain region based on a frequency analysis of a sample of pixels in the certain region. The decoder can be configured to use a first interpolation filter with shorter taps (for example, a

IMPI

MEXICAN INSTITUTE

I heard THE HOniTY Co. '- <>

INDUSTRIAL lower cutoff frequency) than a second interpolation filter for an upper layer temporal prediction for regions with high-frequency components below a specified threshold, and use the second interpolation filter for a lower layer temporal prediction, even if the high-frequency components of the lower layer are below the threshold. The decoder can be configured to use a first interpolation filter with shorter taps (for example, a lower cutoff frequency) than a second interpolation filter for regions with high-frequency components below a specified threshold, except for at least interblocks in which a non-causal neighbor of the interblock comprises one or more intracoded blocks. Exemption of interblocks in which a non-causal neighbor of the interblock comprises one or more intracoded blocks can help prevent error propagation in the presence of intracoding. The power matching controller can be configured to determine a resolution to request for the subsequent segment based on the complexity information derived from the MDF, the statistics for decoding the previous segment, and the current power information. Resolution can be requested at a specified level of complexity. The level of complexity may correspond to⁸ IMPI ^ b

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL level of detail at which the encoder performs the encoding.

A method is described for a wireless transmit receive unit (WTRU) to interact with a video server in order to dynamically save power during video decoding. For example, you can dynamically update the determinations regarding the amount of power remaining to decode and determine the appropriate level of complexity to select for a video segment based on the updated determinations. As an example, the WTRU can request an MDF from the video server. The WTRU can receive the MDF from the video server. The MDF can include information on the complexity to decode one or more segments of a video stream. The WTRU may determine an amount of power to be allocated to decode video segments of the video stream based at least in part on an amount of remaining power allocated to decode a remainder of the video stream. The WTRU may determine a first level of decoding complexity to request for a first video segment of the video stream based on the amount of power to be allocated to decode the video segments of the video stream. The WTRU may request the first level of decoding complexity for the first 'IMPI

MEXICAN INSTITUTE 'tx LA PROPIEDAD L —L_ J and industrial video segment. The WTRU can re &amp; j¿a_Luax_l ^ - ^ QajxtidacL ,, from remaining energy allocated to decode a remainder of the video stream and may determine that the WTRU lacks sufficient energy to complete the decoding of the video stream at a current energy level at each video segment.

The WTRU can determine a new amount of power to be allocated to decode the remaining video segments of the video stream based on a new amount of remaining power allocated to decode the remainder of the video stream. The WTRU can determine a second level of decoding complexity to request a second video segment of the video stream based on the new amount of energy to be allocated for decoding the remaining video segments of the video stream. Each of the remaining video segments can be assigned an equal amount of power based on the new amount of remaining power allocated to decode the rest of the video stream. A corresponding level of complexity may be selected for a respective video segment based on an amount of energy allocated to decode video segments, a duration of the respective video segment, and the statistics associated with the energy used for decoding at least one anterior segment, the level of complexity for at least one segment

<img file="MX345571B_D0007.tif" />

previous, and the duration of at least one previous segment. The WTRU can report to the server the power usage statistics for various decoded segments, and the WTRU can receive a video segment of a certain complexity, based on the report.

BRIEF DESCRIPTION OF THE FIGURES

A more detailed understanding can be obtained from the following description, provided by way of example, in conjunction with the accompanying figures in which: /

Figure 1A is a system diagram of an exemplary communication system in which one or more of the described embodiments may be implemented.

Figure IB is a system diagram of an exemplary wireless transmission / reception unit (WTRU) that may be used in the communication system illustrated in Figure 1A.

Figure 1C is a system diagram of an exemplary radio access network and an exemplary core network that may be used in the communication system illustrated in Figure 1A.

Figure ID is a system diagram of another exemplary radio access network and radio core network that may be used in the communication system illustrated in Figure 1A.

MEXICAN INSTITUTE Λ

From THE CURRENCY »

INDUSTRIAL

Figure 1E is a system diagram of another exemplary radio access network and an exemplary core network that may be used in the communication system illustrated in Figure 1A.

Figure '2 illustrates the trend of the growing number of mobile users.

Fig. 3 illustrates an exemplary insert mode video real-time broadcast system.

Figurq / 4 illustrates an exemplary HTTP-based video real-time broadcast system. /

Figure / 5 illustrates an example / architecture of a smartphone platform that can be implemented on a mobile device.

Figure '6Ά illustrates an example of energy use in a video playback scenario.

Figure 6B illustrates dynamic voltage and frequency scaling.

FIG. 7 illustrates an exemplary energy sensitive real-time video broadcasting system.

Figure 8 illustrates possible points of switching operation by way of example on the client side.

Figure 9 illustrates exemplary HEVC decoding processes.

Figure 10 illustrates examples of different pixel positions for luminance MC processing.

¹² WICKED

INSTITUTO .MEXICANO f '

Dt LA «OTIEDAD Osr..2SíS> JíSí / INDUSTRIAL

Figures 11A and 11B illustrate examples of time profiles for HEVC using the HM6.1 decoder and encoded bit streams using the Random Access (RA) configuration.

Figure 12A illustrates an exemplary interpolation filter waveform that can be used for regions with a relatively small portion of the frequency components in the high-frequency range.

FIG. 12B illustrates an exemplary frequency response for the interpolation filter of FIG. 12A.

Fig. 13 illustrates an example of a hierarchical coding structure in HEVC. z

Fig. 14 illustrates exemplary addresses used in HEVC intracoding.

DETAILED DESCRIPTION OF THE INVENTION

In recent years, mobile devices have become the computing platform of choice for a wide variety of users and applications. For example, the rapid evolution of the integrated circuits of the practical Monochip System (SoC - System on a Chip) has allowed mobile devices to increase their available functionality while maintaining a size that makes them practical for handheld use.

IAI ΡI

MEXICAN INSTITUTE 1 3 \

Ut THE PROPERTY CTea ^ íh I Π3

Recently, ICs (Integrated Circuits) have greatly increased the computing power of mobile devices, for example in terms of CPU frequency and / or the number of available processing cores. In addition, the increase in bandwidth and / or the total data speed of wireless networking technologies (eg, 4G LTE, HSPA +, Wi-Fi, etc.) have allowed mobile devices to obtain media at speeds comparable to traditional broadband Internet access.

Such advancements are among the reasons for the high rate of mobile device adoption, leading to an ever-increasing number of deployment devices. Figure 2 illustrates how the number of mobile Internet users continues to increase over time. For example, as illustrated in Figure 2, it is estimated that the number of mobile Internet users may soon exceed the number of desktop Internet users. Such a trend may indicate that previous content delivery and processing systems that were optimized for a desktop environment should be reviewed in order to optimize performance for mobile Internet traffic. For example, a feature of mobile computing that may have contributed to its tremendous growth in recent years may¹⁴ IMPI MEXICAN INSTITUTE DE LA PROPIEDAD INDUSTRIAL be that mobile users can access services and / or content practically at any time and / or anywhere using their mobile device.

Distributing and / or viewing media content to / from mobile devices can cause or create complexities that can be a problem on a traditional desktop device. For example, despite recent advancements, some mobile devices may still lack the resources to run complex processing in a timely manner compared to desktop environments. Additionally, size restrictions may produce some mobile devices that have less processing capabilities and / or fewer hardware modules than are available in traditional devices. In addition, many types of mobile devices may have power restrictions as they can operate using a battery power source during normal operation, compared to a fixed power source using an electrical network. In order to facilitate the delivery and / or display of media content to mobile devices of varying complexity and / or with varying capacities, techniques are being developed to improve the distribution of content on mobile platforms. As an example, the Trademark Language of

<img file="MX345571B_D0008.tif" />

Hypertext 5 (HTML5

HyperText Markup Language 5) is a · '“' ·»,

OF PROPERTY Γ industrial brand language for web pages that has been designed in part to try to make it easier to access media for mobile devices. For example, HTML5 is designed to use dynamic page layout design support, which can help mobile devices retrieve and interpret media content.

An example use case in which mobile devices have experienced a large increase in Internet traffic is in the generation and delivery of mobile video content. In order to effectively and efficiently generate, process, distribute and / or display mobile video traffic, techniques can be specified to optimize video processing on mobile devices. For example, as the hardware included in the mobile device becomes more powerful, the WTRUs can be configured to perform increasingly complex processing in order to broadcast in real time and / or decode the video traffic.

Additionally, advancements in mobile displays have also influenced the high rate of adoption of mobile devices for video viewing. For example, 720p video quality is relatively widespread for mobile devices (such as smartphones and tablets) for viewing content.

<img file="MX345571B_D0009.tif" />

MEXICAN INSTITUTE A

OF THE PROPIgDAL 'Ο ^ ιΤΖΚίΣΛϊ INDUSTRIAL VIDEO. Some higher performance tablets may be - ——————— ——_M--—----- . _ .

even capable of receiving and displaying 1080p video and more. Additionally, LCD enhancements have enabled mobile devices to improve color levels and / or contrast levels that can be used for video display. Other advanced display techniques may include the introduction of three-dimensional (3d) displays (eg, auto-stereoscopic 3D) for mobile devices. When used in conjunction with advanced wireless communications networks (e.g. 4G LTE, Wi-Fi wireless networks, etc.), mobile device users may be able to access high-quality video services with relative ease. .

Furthermore, mobile devices can be configured to integrate one or more functional modules together on a single platform. For example, a typical mobile device may include one or more of a touch screen, a digital camera, a global positioning system (GPS), a gravity sensor, and / or other features or technologies. Such elements can compete with each other for mobile processing resources, which can complicate the display of video content. The use of one more of these characteristics

<img file="MX345571B_D0010.tif" />

IMPI

MEXICAN INSTITUTE oe LA PKOFIEDAÍ) INDUSTRIAL can affect almost simultaneously · ι i ·· »· _B unfavorable use of energy in the mobile device, causing a decrease in the amount of video that can be viewed per battery charge. Such power usage is an important factor for mobile devices as they often have limited power due to the limited amount of power available in the battery.

Since mobile devices are configured to perform more processing intensive methods for streaming and / or video decoding, there may be differences in terms of power and use of mobile resources. The methods and systems described herein take into account energy use and / or energy level (for example, energy use and / or energy level) during video decoding processes and / or streaming video in order to display adequate quality video in power-constrained environments. For example, in order to deliver high-quality mobile video services on heterogeneous mobile devices with limited resources, factors such as one or more screen sizes, processing capabilities, network conditions, battery levels, and / or the like may be considered and used to affect encoding processing

<img file="MX345571B_D0011.tif" />

<img file="MX345571B_D0012.tif" />

INSTITUTE .MÜXICANi; ra LA «raiEDAÜ INDUSTRIAL and / or broadcast in real time of video on the mobile device and / or in an advanced communications network.

For example, streaming video is an exemplary method of providing video services to one or more mobile devices over communication networks. Some examples of video streaming modes may include push modes and / or pull modes. For example, a real-time transmission system using a video push mode can deliver video data using the real-time transport protocol (RTP). Push-mode video streaming can apply the Real-Time Control Protocol (RTCP) functionality to monitor and / or control the Quality of Service (QoS) associated with the video.

Methods and systems are described for providing client and / or server-based techniques to improve the energy efficiency of video decoding. The methods and systems may include the use of energy-sensitive real-time broadcasting and / or energy-sensitive decoding. For example, decoding a video may include receiving a media description file (MDF) and / or other in-band or out-of-band information indicating

<img file="MX345571B_D0013.tif" />

complexity information for a video stream from a video server. The complexity information can refer to the relative amount of processing resources and / or the relative amount of processing power that a decoder would use to decode a certain video or segment. For example, a more complex segment may use more processing resources and / or more power to decode than a less complex video or video segment. The complexity information for the video stream can be determined based on the MDF. In one example, the complexity information can be embedded in the video stream, for example, using Supplemental Enhancement Information (SEI) messaging in a video bitstream and / or using other embedded metadata. For example, complexity information can be signaled using RTP Control Protocol (RTCP). The mobile device can decode the video stream using the complexity information. The complexity information can be provided by video segment. Decoding the video stream using the complexity information may include decoding the video stream using the complexity information while simultaneously staying within a certain level of power consumption.

<img file="MX345571B_D0014.tif" />

For example, the MDF may be a Media Presentation Description (MPD) file. The MDF can include one or more of a quality level, a bit rate, and / or complexity information. The complexity information can be applicable to a plurality of levels of complexity. The decoding of the video stream may include performing the decoding using different parameters depending on a certain complexity for a certain segment. The method may include determining the power allocation for future decoding of the video stream based on past power dissipation statistics. Circuit filtering can be done, for example, depending on the power available in the mobile device. The circuit filtering may include one or more of unblocking, using adaptive sample offset (SAO), and / or adaptive loop filtering (ALF).

In one example, shortcut decoding can be performed. Performing shortcut decoding can include one or more of applying different interpolation filters in different characteristic areas, omitting unblocking operations in one or more areas where blocking impairments are less visible or where

<img file="MX345571B_D0015.tif" />

the propagation of the error is less problematic, and / or the application of a smaller transform size according to the non-zero coefficient distribution. By taking into account the relative power level remaining and / or the power level available in the mobile device during real-time video broadcasting and / or video decoding, devices capable of performing complex real-time broadcasting and high power consumption can receive high-quality video signals, intensive processing, while devices with power limitations can still receive video of a specified minimum quality while simultaneously limiting the total power consumption.

Figure 3 illustrates an exemplary architecture for an insert mode video real-time broadcast system. For example, as shown in Figure 3, the management server 320 can be configured to manage administration based on user requests. Users can make the request using mobile devices such as one or more of the WTRU 302, the WTRU 304, and / or the WTRU 306. The management server 320 can be configured to assign a user request to one of the streaming servers (e.g., the streaming server 310, the streaming server 312, the

<img file="MX345571B_D0016.tif" />

MEXICAN INSTITUTE OF INDUSTRIAL FRONITY

<img file="MX345571B_D0017.tif" />

streaming server 314, etc.). Multiple requests can be distributed to one or more streaming servers in order to balance system load (eg, admission control). In one example, one or more streaming servers can initiate a streaming session.

A streaming server providing a streaming video session can control one or more aspects or parameters associated with the streaming session, for example, in terms of bit rate, resolution, switching of the flow, and / or the like. The client (s) (for example, WTRU 302, WTRU 304, WTRU 306, etc.) can decode the video data after receiving the video data from the broadcast server on time real. The client (s) (for example, the WTRU 302, the WTRU 304, the WTRU 306, etc.) may periodically report statistics such as packet loss, delays, and / or the like to the broadcast server at real time that is providing the video stream.

In many insert video systems, the adaptation logic for a streaming video session (for example, the functionality used to specify bit rate, resolution, stream switching, etc. based on the actual conditions)

<img file="MX345571B_D0018.tif" />

they can be located on the server side (eg, real-time broadcast server 310, real-time broadcast server 312, real-time broadcast server 314, etc.). For example, a streaming server can be configured to perform bandwidth adaptation using bitstream (and / or layer) switching according to available bandwidth reported by a client {for example, the WTRU 302, WTRU 304, WTRU 306, etc.). Such insert mode video streaming in which adaptation is done on the server side can allow for lower transmission loads and / or processing loads on the client, facilitating the use of a relatively small size of buffer. on the client that since the server can send data according to the determined bit rate relatively continuously and regularly.

In one example, one or more client devices (eg, WTRU 302, WTRU 304, WTRU 306, etc.) can send a feedback report to the server. The server can configure and / or modify the video parameters based on the feedback report (s) received from the client device (s). However, during periods of increased network congestion, the server may stop receiving the feedback report (s). Further,²⁴Mexican institute DE LA PROPIEDAD OaesySi / iií 'INDUSTRIAL RTP protocols, which are frequently used for insert-based video delivery, can be difficult to implement in conjunction with transmission control protocol-based delivery systems ( TCP), and some firewalls can block traffic flows using RTP. In order to provide video of a specified quality, push video streaming systems may implement or utilize streaming servers that are relatively close to users, for example to provide a suitable service. Such implementations make it more difficult to provide the service on a large scale and can make it even more difficult to implement with mobile devices, since, by definition, they can move during the course of the session.

An example of a streaming mode streaming system can be played while downloading. For example, a client device such as a WTRU can download a large media file, for example, using the Hypertext Transfer Protocol (HTTP). The client can decode a portion of the file, while the download process is in progress. However, from the perspective of the video or service provider (e.g. Netflix, Amazon, Google, etc.), it may be undesirable to do the adaptation in a

<img file="MX345571B_D0019.tif" />

per user basis; Instead, such content sources may prefer to use the existing Internet infrastructure as-is, and may implement their over-the-top (OTT) service, which can reduce implementation time and cost. .

An example of an extraction video streaming technique may include HTTP-based streaming video. Some examples of HTTP-based video streaming techniques may include Microsoft's Smooth Streaming, Adobe's Dynamic HTTP and HTTP Streaming, and Apple's HTTP Live Streaming (HLS). Adaptive sampling methods can be used in HTTP streaming. Consortiums such as the Moving Picture Experts Group (MPEG) of the International Organization for Standardization (ISO - International Organization for Standardization) / International Electrotechnical Commission (IEC - International Electrotechnical Commission) and / or the Telecommunication Standardization of the International Telecommunication Union (ITU-T - International Telecommunication Union Telecommunication Standardization Sector) of the Third Generation Sector Partnership Project (3GPP - 3rd Generation Partnership Project) are in the process of standardizing the various

<img file="MX345571B_D0020.tif" />

IMPI MEXICAN INSTITUTE DI LA HLONIDAD INDUSTRIAL adaptive sampling aspects to allow operational communications between devices.

For example, Dynamic Adaptive Streaming over HTTP (DASH) can be an adaptive approach to streaming video using HTTP-based streaming methods. DASH has received a great deal of attention, for example, due to the fact that it can handle various bandwidth conditions. One concept implemented in DASH can be to divide a media file or stream into segments that are independently decodable. Afterwards, a piece of content can be encoded in different qualities or resolutions and separated into segments of equal length. The way of segmentation and / or other descriptions about how the video was divided and / or the information regarding the relative quality of the segments can be included in an XML-based manifest file (MF - manifest file) that must be provided to the client . An example of an MF file can include a Media Presentation Description (MPD) file. An MPD can be an example of a Media Description File (MDF). Based on the MF file, the customer can access the video content using HTTP and can select the segments that are most suitable based on their bandwidth and / or resolution requirements.

MtXICANO INSTITUTE

M THE PROPERTY

INMJJTRIAL 'Sa-ü-í ^

Figure 4 illustrates an exemplary HTTP-based video streaming system. For example, one or more media creation devices 430 may generate a media file such as a video file. Media content may be compressed and cut into small segments, for example, one or more media creation devices 430 and / or on one or more HTTP source servers (for example, HTTP source server 420, server HTTP 422 origin server, HTTP 424 origin server, etc.). For example, the segment period can vary between 2 and 10 seconds in many live broadcast systems. Segments can be stored on one or more origin servers from

HTTP (for example, HTTP source server 420, HTTP source server 422, HTTP source server

HTTP 424, etc.) and can be distributed via a content delivery network (CDN).

For example, at the start of a streaming session, one or more client devices (for example, WTRU 402, WTRU 404, WTRU 406, WTRU 408, etc.) can request the MPD file to media content and can determine which segments to use. The decision regarding which segments to use may be based on the capabilities of the customer (for example, such as one or more of the resolution, width of

IMPI Mexican institute

OF THE PROPERTY

INDUSTRIAL

<img file="MX345571B_D0021.tif" />

band available, and / or the like). One or more HTTP origin servers (for example, HTTP origin server 420, HTTP origin server 422, HTTP origin server 424, etc.) can send the requested segments to one or more devices of the client (e.g. WTRU 402, WTRU 404, WTRU 406, WTRU 408, etc.) according to the request, e.g. by one or more HTTP cache (e.g. HTTP cache 410, the associated memory of

HTTP 412, the HTTP 414 cache, etc.). By using one or more HTTP cache servers to store and / or distribute the media segments, the video can be distributed and used by other users such that the system can provide real-time broadcast service on a large scale.

Compared to push-mode real-time broadcast systems, pull-mode real-time broadcast systems often include adaptation logic to select the appropriate segments on the client side (for example, the WTRU 402 , WTRU 404, WTRU 406, WTRU 408, etc.). In some cases, the cache storage process used for distributing the video segments may be different depending on whether insert mode or extract mode is used for the

IMPIAS

MEXICAN INSTITUTE

OWNED BY CjSfflFiM®

INDUSTRIAL content distribution. For example, since different transport protocols can be used for content delivery depending on which mode is used (e.g. HTTP for pull mode, RTP for push mode, etc.) the memory storage process used The associated protocol may vary based on the type of transport protocol used. For example, HTTP was not initially designed for real-time media delivery; rather, HTTP was designed for the best effort delivery of files and other data. Consequently, HTTP-related infrastructure, for example, which includes one or more of memory sticks, CDN (s), and / or proxies, may be quite well compatible with such a transfer, but may be less optimized for content delivery. video in real time.

Since network address translation (NAT) and / or firewall traversal may or may not apply for HTTP, RTP can be difficult to implement using the existing HTTP infrastructure. Additionally, HTTP can add significant overhead to a streaming session compared to RTP. In order to use HTTP for streaming, the client part (for example, WTRU 402, WTRU 404, WTRU 406, WTRU ³⁰ IΜ ΡI.

ΙΝΪΤΤΠΠΤ »MEXICAN i

Dt LA MtOHEDAD V «erÑiMeS¿Z 'INDUSTRIAL' -T!>,

408, etc.) can use relatively large buffer sizes, for example, since the transmission speed of HTTP may not be uniform over time. For example, the transmission speed of HTTP can vary greatly when networks become congested.

In real-time broadcast systems, unscaled coding can be used to generate different bit streams with one or more different bit rates and / or different resolutions in order to accommodate varying bandwidth and / or bit sizes. variable display. In addition, scalable encoding techniques can be used in order to save transmission bandwidth and / or limit the amount of storage used during transmission and / or reception. For example, scalable video coding (SVC) can refer to techniques for encoding a relatively high quality video bitstream that can also include one or more subset bitstreams. Said one or more subset video bitstreams can be determined using a subset of the packets derived from the main bitstream during the decoding process. By dropping packets from the largest video bit set, the subset bit sets can be used to reduce the bandwidth associated with receiving and displaying video. In one example, the subset bitstreams may represent lower spatial resolution (for example, smaller screens), lower temporal resolution (for example, lower frame rate), and / or a lower quality video signal. than the main bitstream.

The application running on the client device can determine which of the main SVC bitstream and / or one or more subset bitstreams should be decoded based on the rate and / or representation desired by the application. SVC can save bandwidth and storage compared to non-scalable solutions. The international video standards MPEG-2 Video, H.263, MPEG4 Visual, and / or H.264 may have the tools and / or profiles that support some scalability modes. Recently, the requirements and use cases for the High Efficiency Video Coding (HEVC) scalable extension have been approved. The HEVC can currently be developed jointly by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Image Experts Group (MPEG). With scalable video encoding technology, bandwidth can be saved for DASH and / or cases of

<img file="MX345571B_D0022.tif" />

³² IMPI Mexican institute OF INDUSTRIAL PROPERTY multicast. In DASH systems, an HTTP cache server can cache the base layer of video viewed by many users, for example, rather than cache all or some versions with different bit rates.

Due to factors such as the tremendous increase in the number of applications running on mobile devices, power endurance and / or power efficiency of mobile devices has become a fundamental concern in managing mobile device performance. Various industry research has been conducted on energy efficiency in relation to mobile device hardware and power supply design. For example, the power consumption for an exemplary mobile platform can be analyzed under various conditions. Figure 5 illustrates an exemplary architecture of a mobile smartphone device that can be used to implement one or more of the methods described herein. For example, mobile device 500 may include processor / CPU 502, an internal and / or external display, such as LCD 504, memory 520 (e.g., internal / external memory; examples may include NAND flash memory 522 and / or a synchronous dynamic random access memory (SDRAM - Synchronous ³³ WICKED

MEXICAN INSTITUTE DE LA PHOPIíDA · INDUSTRIAL

Dynamic Random Access-Memory) 522), a 506 cellular radio (for example, GSM / 3G / 4G, LTE, HSPA +, CDMA2000, etc.), WiFi Radio 508, a 510 graphics card (for example, with memory card graphics 514, and / or secure digital card (SD - Secure Digital) 512), Bluetooth Radio 530 (for example, via a universal serial bus (USB - universal serial bus)), internal and / or external GPS 540, codec 550 , amplifier 560, and / or various other components for receiving / decoding video streams and / or communicating via a wireless network.

Depending on the applications, the power consumption used by the various components of the mobile device 500 may vary. For example, Figure 6 illustrates an example of power consumption with a mobile device during a video playback scenario. In Figure 6 it can be seen that CPU processing, display aspects (eg, backlight, graphics card, etc.), and memory access may be the predominant sources of power consumption during playback. Of video. Therefore, video decoding can be characterized by applications with relatively high power consumption, for example, because video decoding can involve both intensive calculations, frequent memory accesses, and / or screen usage. relatively

<img file="MX345571B_D0023.tif" />

<img file="MX345571B_D0024.tif" />

constant. Also, video playback applications can be configured to display the image in relative luminance at the screen output, which can lead to increased power consumption.

During a video encoding and / or decoding process, motion compensation may be applied. For example, video encoding standards such as MPEG-1, MPEG-2, H.263, H.264 / MPEG-4 Advanced Video Coding (AVC), HEVC, and / or the like may use motion compensation to limit the amount of signaling used to communicate video information to a device. Motion compensation can be thought of as a video compression technique that describes an image in terms of a transformation or difference from a reference image. The reference image can be a previous screen and / or a screen to be used more later. When images can be accurately synthesized from previously transmitted / stored images, it can improve compression efficiency.

Motion compensation can be implemented using a linear filtering process. Although a larger filter size can achieve better compression efficiency, larger filters can also

<img file="MX345571B_D0025.tif" />

⁵ IMPI

ΙΝίΤΓΠ, ΙΤΟ MEXICAN

D € EROFIITY

INDUSTRIAL increase the memory access bandwidth used during decoding, which can increase power consumption (for example, increases in memory access bandwidth during video decoding can increase the power dissipation in mobile devices). Therefore, a relatively lower power decoder chip can be designed to reduce the memory access bandwidth by compression of the frame buffer, for example, by compressing the reconstructed pixels losslessly before storing the reconstructed pixels in the frame buffer. When used for motion compensation, reconstructed pixels can be extracted from frame buffer and decompressed for motion compensation.

Exemplary methods of managing power for video decoding may include adaptively changing system state and / or adaptively switching processor frequency (e.g., DVFS dynamic voltage and frequency scaling)). Figure 6B illustrates the power consumption in the presence and absence of DVFS. DVFS can be used to reduce the frequency of a processor in order to conserve power or reduce processor power consumption. When using

IMPI

INSTITUTE MiXiCAN

OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0026.tif" />

With a lower processing frequency, the processor can execute a certain task for a certain processing time (for example, decode a frame before it is displayed) while simultaneously executing the task using less power than would have been used if processing had been run at a higher processor frequency. For example, when a mobile device is idle, the DVFS can be used to transition to a low power state, while simultaneously keeping certain modules running. Furthermore, when the frequency of the processor decreases, the voltage corresponding to the supplied power may also decrease accordingly. Therefore, the power consumption can be reduced. For example, the energy dissipated by a chip or processor can be expressed as:

P = CV² f Equation (1) where C can be the capacitance (eg capacitance toggles per clock cycle), V can be the voltage, and f can be the switching frequency. Many CPUs / processors can provide multiple frequencies that the CPU can operate on, and the application can configure the frequency on the fly as desired.

Decoding complexity may be different for each image, and the level of complexity may

IMPI MEXICAN INSTITUTE DE LA NtOfflDAD INDUSTRIAL Can be used to modify the format for example, if the image is relatively simple, the frequency can be reduced in order to save energy. In one example, the image size can be used to calculate the decoding time associated with decoding the image. When the image decoding time is less than the image duration (for example, 1 / f), then the processor frequency can decrease without affecting the display. The dynamic power range may be smaller with recent advances in memory and processor technology, and the use of power in idle / sleep mode may be more efficient. These technological changes can limit the effect of DVFS, so DVFS power savings may not be as prominent on new mobile platforms compared to older platforms.

Improving energy use efficiency to extend battery life is becoming an increasingly critical point for mobile platform design and mobile application design. Software and hardware video decoding is widely used in mobile multimedia applications, and it can be computationally intensive to process with high power consumption. The complexity of decoding the

IMPI

MEXICAN INSTITUTE

Say THE INDUSTRIAL MONTY

<img file="MX345571B_D0027.tif" />

video compression standards too¹ 'bridge¹ inegemenfraT · -— gradually in order to achieve better compression efficiency.

Real-time broadcast systems like DASH focus on network bandwidth variation, but may fail to address mobile platform power use from a system point of view. In addition, adaptive decoding techniques can be used to save power from the customer side, but additional power savings may be desirable, particularly during periods when additional power is limited (for example, the power level of the battery is close to zero). Additionally, the power consumption for different parts of an application can be balanced to ensure complete playback. Methods that focus on energy savings on the client side may be limited and / or the user experience may degrade due to image irregularities when there is not enough power for full playback. As described herein, energy use can be targeted and modified based on the concept of energy-sensitive computing. Exemplary methods and systems may include collaboration between the server and the client. For example, the customer (for example, the

<img file="MX345571B_D0028.tif" />

mobile device) can request flows with di, fpr ° n between— complexities according to the available bandwidth and / or current power status.

Clients can be configured to try their best to decode and play video once a suitable bit rate has been determined according to the currently available bandwidth. For many video decoders, decoding and playback can occupy significant percentages of processor resources in order to meet real-time processing requirements. Pure client-based systems may be unable to play smoothly, for example, because the processor power level may be insufficient to decode in real time. Asynchronous audio and video and frame skipping may be observed in situations where the processor is unable to perform dull decoding during playback. In addition, client-based systems can reduce the response speed of the system, which can affect the quality of the user experience in multi-tasking environments. For example, the system response time for user input / output may be reduced due to the processor load being full. In such scenarios, switching tasks can be slower.

<img file="MX345571B_D0029.tif" />

ίο iMPI

MEXICAN INSTITUTE

M THE MOMITY

INDUSTRIAL

In order to conserve energy dnrAnta ... 1st video decoding, energy-sensitive real-time broadcast, energy-sensitive decoding, and / or shortcut decoding ways can be used to achieve a better balance of processor load and / or power savings. For example, bitstream switching logic or smart logic on the client side of a real-time broadcast system can be configured to consider available network bandwidth, current processor load, and / or remaining energy together, for example, to ensure a minimum quality of user experience. For mobile devices that primarily rely on dedicated hardware acceleration for video streaming and decoding applications, bandwidth and power status can be important factors to monitor and optimize, whereas if the mobile device relies on mainly in the software to decode and broadcast real-time video, then, in addition to the use of bandwidth and power level, Processor load and usage due to video decoding can also be considered.

As described herein, energy-sensitive technologies such as real-time energy emission and / or energy-sensitive decoding

IMPI

MUÍ1CAHO INSTITUTE

Rt INDUSTRIAL FROFIDITY

<img file="MX345571B_D0030.tif" />

they can be used to achieve mobile devices, such as energy savings for WTRUs that use streaming video. For example, one or more of the energy-sensitive real-time broadcast and / or energy-sensitive decoding can be used to adjust the energy consumption per segment and / or per image, for example, according to the energy remaining and / or power consumption statistics derived from previous image decoding. For example, the mobile device may use one or more power indicators to determine an amount of power to allocate to decode the rest of the video stream and / or one or more segments of the rest of the video stream. The energy indicator can be any measure of energy consumption, energy location, energy use, energy allocation, complexity information by energy consumption, energy consumption by quality of video, and / or similar that allows the WTRU to calculate the energy resources to be used for a future decoding process. As can be seen, although energy may be a term used herein, the term energy may be substituted for energy where appropriate as those skilled in the art will observe.

⁴² ΙΜΡΙ »>

MEXICAN KSTHVTO

OF THE REHABILITY

INDVSTMAL

Energy-sensitive real-time broadcast technology can be implemented through collaboration between a server and a client. Energy sensitive decoding can be used as a client-side adaptive technology. For example, the customer may request one or more videos or video segments at a certain level of complexity based on a determination regarding the customer's energy use and / or the remaining energy level that can be used to decode at the customer. client. In one example, the energy sensitive decoding can be a collaboration between the video server. For example, the client can provide the server with one or more parameters related to its current power uses {for example, current power level, power level previously used during decoding, amount of time a power level current can continue without draining a current power source such as a battery, the power level previously used to decode a segment of a specified complexity, the power statistics for one or more threads or components of the decoding process, etc.), current power levels {for example, the amount of power remaining in a power source such as a battery, the amount of power used with prior to decoding, the amount of energy previously used to decode a segment of a specific complexity, the energy use statistics for one or more sub-processes or components of the decoding process, etc.), the bandwidth information, and / or the like. The server may use this information in order to selectively encode one or more segments for use by the client and / or it may select a segment of an appropriate level of complexity based on the feedback information. In one example, the server can add statistics from multiple clients to derive more accurate calculations for the amount of power used to decode segments of the specified complexity levels.

Energy sensitive decoding and / or real-time broadcast technologies can be integrated into existing DASH real-time broadcast systems and / or other OTT real-time broadcast methods. Low decoding complexity displays can be achieved by complexity sensitive encoding, encoding tool limitations, bit rate reduction, and / or resolution reduction. Energy sensitive decoding can be used to achieve an adaptive decoding method, of

IMPI

INITITUTE MWCANQ

Df LA «OtttOAO industrial

<img file="MX345571B_D0031.tif" />

fine granularity. Power sensitive decoding can be used with software decoding techniques (for example, a specialized or general purpose CPU and / or the processor is configured to run video decoding) and / or a hardware decoding chip (for For example, dedicated hardware modules are configured to perform decoding). Energy-sensitive real-time broadcasting can be used in combination with energy-saving technologies such as DVFS; for example, the mobile client device can lower the frequency to save power for low-complexity video segments.

Figure 7 illustrates an exemplary energy sensitive video real-time broadcast system. For example, distributing video to a client device (e.g., a WTRU such as a mobile device) may include preparing video 710, distributing video 730, and / or consuming video 750 (e.g., decoding and display). The video encoder (eg, a complexity sensitive encoder 712) can encode the video file or broadcast in real time using a plurality of levels of complexity. For example, a low complexity level may consume less power to decode in the video decoder, but it may generate a version

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0032.tif" />

relatively more distorted of the images included in the video file. A high complexity level may require more power, but can produce a more accurate recreation of the original video. An intermediate complexity level can decode with less power than the high complexity version, but may require more power than the low complexity version. The level of distortion associated with the intermediate complexity level can be found between the distortion level of the high complexity level and the low complexity level.

For example, during video preparation 710, a source video (for example, an MP4 or some other media file (MP) 720) may be generated. For example, the complexity sensitive encoder 712 can generate and encode the MF 720. The MF 720 can be compressed and divided into small segments with relatively short durations, eg, fragment (s) 722, 724, ..., N, etc. The address and property information, for example, including the complexity information for one or more chunk (s) 722, 724, ..., N, can be described in a Complexity Sensitive Media Display Description (MPD) file. ) 728. The MF 720 can be encoded using a plurality of complexity levels (eg, a high complexity level, an intermediate complexity level, a low complexity level, etc.)

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0033.tif" />

and the fragments can be generated by encoded versions of MF 720 at each level of complexity. The complexity-sensitive MPD file 728 may include the complexity information for each of the complexity levels used to encode the MF 720, and / or there may be a corresponding complexity-sensitive MPD file 728 for each level of complexity. complexity used to encode the MF 720.

Once fragments 722, 724, ..., N and the complexity-sensitive MPD file 728 are ready for distribution, fragments 722, 724, ..., N and the complexity-sensitive MPD file the complexity 728 can be sent to a distribution server for distribution 730. For example, one or more media files 723 can be sent to the HTTP Server 740 for distribution. The media files 732 may include one or more clips 722, 724, ..., N, and / or other descriptions of the video. Media files 732 can include encoded versions of MF 720 using one or more levels of complexity. The MPD file 738 may be a copy of the complexity-sensitive MPD file 728 that is stored on the HTTP server 740. The HTTP server 740 may provide one or more HTTP memories 742 with copies of the media files 732 and / or the MPD file 738 for distribution to client devices.

⁴⁷ WICKED

MEXICAN INSTITUTE J: «

M THE PROPERTY C ''; s-ot · -

INDUSTRIAL

During consumption 750, the power-sensitive client device 760 (for example, a WTRU mobile device, and / or another device that includes a video decoder) may request the MPD file 738 from one or more servers of the HTTP 740 and / or HTTP associates 742. The Power Sensitive Client 760 may determine the description information for fragments 722, 724, ..., N based on the received MPD file 738. The 760 energy sensitive client can determine the complexity levels that are available for the file based on the MPD File 738 file. The 760 energy sensitive client can submit a request for one or more video segments ( eg, chunks 722, 724, ..., N) based relatively continuously on available bandwidth and / or your current power state. The 760 energy sensitive customer may order the segments at a particular complexity level, for example, based on the current energy level and / or the remaining available energy on the 760 energy sensitive customer (e.g. the amount battery power remaining, amount of processing time remaining given current power usage, etc.). For example, based on the current battery power level that is above a first threshold, the 760 power sensitive client

IMPI

INSTITUT · MEXICANO OE LA MONEDAD INDUSTRIAL

<img file="MX345571B_D0034.tif" />

you can request a first segment at a high complexity level. If the energy level falls below the first threshold (for example, but is above a second threshold), the energy sensitive client 760 may request a subsequent segment at an intermediate complexity level. If the remaining power level in the battery falls below the second threshold, the 760 power sensitive customer can request another subsequent segment at the low complexity level.

For example, the energy sensitive client 760 may include one or more of the energy detector 762, the bandwidth sensor 764, the transceiver 766, the decoder 768, the request 770, the statistics and control unit of complexity 772, the Power Sensitive Match Controller 774, and / or other components for receiving and processing video streams. The energy detector 762 can be configured to determine the current energy use of the energy sensitive client 760 and / or the energy use of one or more components of the energy sensitive client 760 (for example, the 768 decoder, a display, a CPU, etc.) · The 762 power detector can be configured to determine the amount of remaining power available to the 760 power sensitive client. For example, if a battery is the power source for the 760 energy sensitive client,

MEXICAN INSTITUTE

OF THE PROPERTY .'

INDUSTRIAL> “* then the energy detector 7 62 I can oonfigmf'a ^ & e — p ^ ya ------- determine the amount of battery power remaining at a given time. The power detector 762 can be configured to determine the amount of time the power sensitive client 760 can continue to operate under current decoding conditions before the battery is depleted. The energy detector 760 can be configured to determine the amount of time that the energy sensitive client 760 can operate under the presumed or selected decoding conditions before the battery is depleted.

The bandwidth sensor 7 64 may be configured to determine information related to the communications link between the power sensitive client 760 and the source of a video stream (e.g., the HTTP server 740 and / or the ( s) HTTP cache (s) 742). The bandwidth sensor 764 may be configured to determine the bandwidth available to the energy sensitive client 760 (for example, based on said one or more radio access technologies associated with the transceiver 766), the amount of bandwidth available band for the communication link between the power-sensitive client 760 and the source of a video stream (for example, the HTTP server 740 and / or the associated HTTP memory (s) 742), the bit rate ⁵⁰ IMPI ^

MEXICAN INSTITUTE ne la monedad &lt;Α ».ζΖΒΐί

INDUSTRIAL 2 effective communication link between the energy sensitive client 760 and the source of a video stream (for example, the HTTP 740 server and / or the associated HTTP 742 memory (s) ), the information related to the previous bit rates or bandwidths associated with the communication link between the power sensitive client 760 and the source of a video stream (for example, the HTTP server 740 and / or the ( HTTP cache (s) 742), and / or other information related to the communication channel between the energy sensitive client 760 and the source of a video stream (for example, the HTTP server 740 and / or the associated memory (s) ) of HTTP 742).

The complexity control and statistics unit 772 can be configured to store information determined by the bandwidth sensor 7 64 and / or the energy detector 762. For example, the complexity control and statistics unit 772 can be configured to store the energy usage statistic and associate the energy usage statistic with the type of decoding that was being performed when the statistic was determined by the 762 energy detector. The complexity control and statistics unit 772 may be configured to maintain statistics associated with the communications link between the power-sensitive client 760 and the video source (e.g., server

<img file="MX345571B_D0035.tif" />

of HTTP 740 and / or the associated memory (s) of HTTP 742) as observed by the bandwidth sensor 7 64. The stored statistics can be used during the determination of an appropriate level of complexity of the an encoded video to be requested.

The energy sensitive adaptive controller 774 can be configured to use the statistics determined by the 7 bandwidth sensor 64 and / or the energy detector 762 and / or the statistics stored by the complexity statistics and control unit 772 in order to dynamically adapt the decoding process that is executed by decoder 768. Power sensitive matching controller 774 may interface with application 770 in order to consider application requirements and / or application properties to tailor the decoding process. The 774 Power Sensitive Adaptive Controller can be configured to select an appropriate level of complexity for a given file or file segment based on the current power level, current bandwidth, and / or past statistics related to the energy use, and / or bandwidth.

The complexity sensitive encoder 712 can be configured to compress source videos during preparation 710. For example, the complex sensitive encoder

IMPI

MEXICAN IMTITUTE or. THE t-KOmOAl 'INDUSTRY !.

<img file="MX345571B_D0036.tif" />

the complexity 712 can be configured to encode the video using rate-distortion optimization (RDO). RDO may refer to a method of improving video quality in the presence of video compression in which the amount of distortion (e.g. loss of video quality, loss of video information) is balanced or optimized against the amount of data bits used to encode the video (for example, the rate). The complexity sensitive encoder 712 may attempt to achieve the highest encoding quality of encoding (eg, providing the maximum amount of information for decoding) given bit rate constraints without regard to decoding complexity. Therefore, the encoder may attempt to maximize a composite indicator of the derived video source drift due to encoding losses (eg, distortion) against the bit cost for a possible decision result. Equation (2) can be an exemplary cost indicator used to evaluate a cost for a given encoding mode when running rate distortion optimization.

Cost_mo¿_or DAY_mode + lf_handle X Rmodo

Equation (2)

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0037.tif" />

cost_mOdo can represent the HpI_mode — de-encoding. Olst_mode it can represent the level of distortion associated with the encoding mode. R_mAll may represent the number of encoding bits associated with the encoding mode. Atasa can represent the slope of the rate distortion curve, which can be related to and / or roughly proportional to the encoding bit rate.

The complexity-sensitive encoder 712 may select a suitable encoding mode in order to minimize the cost indicator (for example, given complexity and bit rate constraints, the complexity-sensitive encoder 712 may select the mode of coding that achieves the lowest cost relative to CDO). For example, C_moc¡_or it may be mode complexity measured in CPU cycles, memory access, and / or the like. TO_CLeanness can be the slope of the complexity against the distortion curve. Equation (3) can be used to evaluate cost mode cost in rate distortion optimization when such parameters are taken into consideration.

Cost_mode - Dist_mode + λ {_α5α XR_mode + ^ complexity * ^ Equation mode (3)

A higher a_conplEfficiency may correspond to lower decoding complexity. Consequently, it can⁵⁴

ΙΜΤΠΤυΤΟ MEXICAN Λ

OF MIOMIOAD

INDUSTRIAL there are a plurality of possible bit streams with various complexities that can be generated with different values ​​of the respective ^_complehee_dad. The encoder can make each of the bitstreams with varying levels of complexity available to potential clients. Customers can select the appropriate bitstream based on their local power conditions and / or their local bandwidth conditions.

In one example, the complexity sensitive encoder 712 can be configured to compress video streams while simultaneously adhering to one or more limits with respect to the amount of encoding information provided for different encoding tools for different levels of respective complexity. For example, Table 1 illustrates examples of encoding tools and how the level at which the encoding tool can be applied depending on the desired level of complexity of the encoded bit stream. Based on the parameters selected for each encoding tool, the encoded bit stream can be more complex or less complex to decode. In order to generate bitstreams of various levels of complexity, the parameters for each of the encoding tools can be selected, for example, as indicated in Table 1.

IMPI

MEXICAN INSTITUTE

OF THE MOLD '_^Y

INDUSTRIAL

Complexity can refer to the amount of processing resources used to encode / decode the video segment, and can be indicated to the mobile device in an MDF. The complexity information can be designated as a specific complexity value (for example, based on a predefined method for determining the value), and can indicate an approximate amount of processing resources in the decoder to be used to decode a segment. . The complexity information can include specific parameters used to carry out the encoding, such as one or more values ​​for the encoding tools listed in Table 1.

Some examples of encoding tools may include the level of precision used to encode motion information, the size of the motion compensation block, the size of the encoding transform, the type of in-circuit filters used, the threshold for skip coefficient encoding for the block, and / or the like. As illustrated in Table 1, the encoding parameters and / or the application of one or more encoding tools can be restricted according to the desired level of complexity of the encoded bit stream.

<img file="MX345571B_D0038.tif" />

IMPI

MEXICAN INSTITUTE

M THE PROPERTY

INDUSTRIAL

<td rowspan="2">Complexity</td><td colspan="5">Coding tools</td>

<td>Information accuracy / motion vector</td><td>Size of compensation block movement</td><td>Transform size</td><td>Filter (s) in circuit</td><td>The cost threshold to skip the coefficient encoding for the block</td>

<td>low</td><td>integer pixel position</td><td>big</td><td>smaller size</td><td>intra segment</td><td>high</td>

<td>intermediate</td><td>integer pixel position, pixel position fractional only in one dimension</td><td>means, medium</td><td>size small or medium</td><td>intra segment and reference p / b segment</td><td>medium</td>

<td>high</td><td>without limitations</td><td>without limitations</td><td>without limitations</td><td>without limitations</td><td>yes in cuts</td>

Table 1: Example coding tools applied according to a desired level of complexity

In one example, the precision of the motion information and / or the precision of the motion vector to be used for video decoding can be modified. By varying the accuracy of the motion information, the complexity of the decoding scheme can be increased (eg, scenarios without power restrictions) and / or reduced (eg, scenarios with power restrictions). The accuracy of motion information in existing coding standards (eg HEVC, H.264) can be an example of motion information / vector accuracy. The precision of the information / motion vector can be an integer value (for example, specified to a certain

IMPI

ΠΓΠΤυΤΟ MEXICAN INDUSTRIAL PROPERTY

<img file="MX345571B_D0039.tif" />

pixel), a value of half a pixel, a, .valmg .da-.fourth— pixel, and / or some other pixel fraction. Fractional pixel precision may be more complex than whole pixel information due to the application of the interpolation filter and / or due to a greater number of reference pixels being used {for example, producing a higher bandwidth of memory access and / or an increase in the number of memory access requests). Therefore, when encoding a less complex video stream, the precision of the motion information can be specified at an entire pixel location and when encoding a more complex video stream, the precision of the motion information can be specified in a fractional pixel location. In one example, for the highest complexity level, there may be no limitation on the accuracy of the motion information, and the encoder can select the level of motion information accuracy based on which level will cause the least distortion. .

In one example, the size of the motion compensation block to be used for video decoding can be modified. By varying the size of the motion compensation block, the complexity of the decoding scheme can increase or decrease the complexity of the video stream.

<img file="MX345571B_D0040.tif" />

IMPI

INSTITUTE MSXJCAN0

OF THE INBUSTRIAL FMÍIIDAD to decode. By varying the size of the motion compensation block, the complexity of the decoding scheme can be increased (eg, without power restrictions) and / or reduced (eg, power-restricted scenarios). For example, the size of the motion compensation block can affect the efficiency of memory access. For example, a larger block motion compensation block size may reduce the memory access frequency, but may result in greater distortion. Thus, when encoding a less complex video stream a relatively large motion compensation block size can be used and when encoding a more complex video stream the precision of the motion information can be specified at a fractional pixel location. . In one example, for a higher level of complexity, there may be no limitation on the size of the motion compensation block, and the encoder may select the size of the motion compensation block based on what size will cause the least distortion.

In one example, the size of the transform block to be used for video decoding can be varied in order to achieve a desired level of complexity. Variation of the transform block size

IMPIAS

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL can increase or reduce the complexity of the video stream to decode. For example, HEVC may allow different transform block sizes (eg up to a transform block size of 32x32). Larger transform block sizes can improve compression efficiency, simultaneously increasing decoding complexity as well. Therefore, during brief scenarios (for example, the remaining energy level is below a certain threshold), an energy sensitive client may request a level of complexity corresponding to a smaller size of the transform block. Table 1 identifies the exemplary values ​​that can be used for the transform block size, depending on the level of complexity requested. For example, if the energy level is above a first threshold, a high decoding complexity may be requested (eg a transform block size of 16 * 16) and / or there may be no limitation on the block size. transform used. If the energy level is below the first threshold, but above a second threshold, an intermediate complexity level may be requested, and an intermediate transform block size may be used (eg, 8 * 8). If the energy level is below

IMPI

MUCAMO INSTITUTE

Dt THE PROHCDAD

INDUSTRIAL

<img file="MX345571B_D0041.tif" />

below both the first threshold and the second threshold, a low level of decoding complexity can be used, eg, using transform block sizes that are relatively small (eg, such as 4 * 4).

In one example, the in-circuit filters to be used for video decoding may vary in order to achieve different levels of complexity for video segments. By varying the parameters and / or types of in-loop filters used, different power levels can be achieved during video decoding. For example, in-circuit filter (s) may include different restoration tools, such as unblocking filter (s), adaptive sample compensation (s), adaptive circuit filter (s) , and / or the like. The unblocking filter (s), sample adaptive compensation (s), adaptive circuit filter (s), and / or the like may be in a motion compensation circuit. The encoder can apply these filters in circuit in various combinations in order to reach levels of complexity that affect the power consumption to be used for decoding. For example, unblocking filter (s), sample adaptive compensation (s), adaptive circuit filter (s) and / or the like may be applied in the

IMPI

INJTITUTC MMICaNO M LA FKOHSiJAD INDUSTRIAL

<img file="MX345571B_D0042.tif" />

encoder such that a first -Élaj p'-produces-a decoding with low power consumption (for example, a lower level of complexity) and a second stream produces a decoding with high power consumption (for example, a higher level of complexity).

By way of example, in order to achieve a relatively less complex video segment, predicative coding can be performed using intra-segment coding. For example, intra-segment coding can be performed without the use of intra-segment coding. For example, if a low complexity level is desired, the segment can be encoded using intra-segment encoding. An intra-segment (for example, it can be called segment I or frame I) can be decoded without reference to other frames, and thus can be decodable at lower power levels than frames or segments that refer to other frames or segments. If a relatively higher complexity is to be encoded, the segment can be encoded using intra-segment coding and intersegment coding. Some examples of intra-segment coding may include the use of P segments (eg, they may be referred to as P segments or P frames) and / or B segments (for example, they may be referred to as B segments or B frames) as reference frames. A plot of

IMPI

MEXICAN INSTITUTE • t INDUSTRIAL PROPERTY

<img file="MX345571B_D0043.tif" />

P-frame reference may refer to data from previous frames being used in order to decompress and / or decode the current P-frame. A B frame can refer to a frame that uses data from both previous and next frames (eg, future frames) to decompress or decode the current B frame. The use of intersegment coding (eg, P / B segments) can increase processing complexity due to references to other frames, which means that power use during decoding can be increased. Therefore, if a lower energy level is achieved, the use of P-frames and / or B-frames for intersegment coding can be reduced or stopped.

The encoder can adjust a cost threshold to determine if it can skip the coefficient encoding for a given block. For a low complexity level, the cost threshold can be a relatively high value, while for a higher complexity level, the cost threshold can be set relatively low. In one example, for the highest level of complexity, clipping may not be performed, (eg, skipping of coefficient encoding is not performed for the highest level of complexity).

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0044.tif" />

The encoder can also determine whether it will encode some or all of the received non-zero coefficients of the transform and quantization for each video block (e.g. video segment or chunk), for example, by considering the quality and complexity of the decoding. . The cost of the block can be measured based on the human visual system (HVS). The cost of the block can be a weighted sum of nonzero coefficients, for example, as illustrated in Equation (4).

(l, if Coefij Ψ 0

CostObigqug ^ (ij) bioque ^ ij X $ i, p $ í, j (θ Coef- · = 0 Equation. (4)

Me,! it can be a weight matrix related to HVS. The weight in the low frequency position may be greater than in the high frequency position. If the cost of the block, CostObioquer, is less than a threshold that is adjusted to correspond to a certain level of complexity, then the encoder can skip the non-zero coefficient encoding. The threshold for controlling whether the encoder skips the coefficient encoding can be adjusted given the level of complexity.

The complexity information such as λ ¹ complexity / and / or an indication of the level of complexity associated with an encoded stream (e.g. low, intermediate, high, etc.) may be added in the description

IMPI

MEXICAN INSTITUTE

OF THE INDUSTRIAL PRIORITY

<img file="MX345571B_D0045.tif" />

that is requested by the power sensitive nHpntp. In one example, the mobile client device may consider bandwidth information and / or power status information to determine the bit stream to request (eg, high complexity, intermediate complexity, low complexity, etc.). In order to determine an appropriate level of complexity to request for a subsequent video segment, the customer's mobile device can determine the calculated power to be allocated for decoding the subsequent segment. The client mobile device can then determine the appropriate complexity level to request based on an energy allocation for a previous decoding process for a segment, the complexity level for the previous segment, and the energy level associated with the decoding. subsequent. The mobile client device can determine the power allocation information for future video decoding, for example, based on the previous power dissipation statistics. If the energy is allocated uniformly, then the energy for the next segment can be determined, for example, using Equation (5).

^ next (T_s/ D_r) * P_r Equation (5)

Next can be the energy allocated for the next segment, T_s can be the duration for the following ⁶⁵ IMPIAS

MEXICAN INSTITUTE

BE THE PROPERTY

INDUSTRIAL segment, and D_r It can be the remaining length of the video (for example, or the length of a current or previous segment). P_r It can be the total energy allocated to decode the remaining video (for example, the amount of energy to be allocated for decoding the remaining r segments). Based on the energy allocated for a subsequent segment, the energy used to decode a previous segment, and / or a decoding complexity of the previous segment, the mobile device can determine an appropriate level of complexity to request for a subsequent segment, for example , using Equation (6).

Cnext ^— (Psig / Pant) ^x C_ant Equation (6)

P_ant can be the energy used for the previous segment. C_yesg_client can be the level of complexity to be requested for a subsequent segment, and C_ant it can be the complexity for a previous segment. If the complexity information is signaled by a discrete complexity level, the complexity level of the next segment can be determined, for example, according to equation (7).

{£ q YES P next / P prev &lt;

Li yes next / P_ant&lt;Thi Equation (7)

L_n other

IMPI

MEXICAN INJTTTUTO OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0046.tif" />

T_hl can be the threshold for each complexity level i. Based on the calculated complexity level, the customer can request the segment that has the closest complexity level as C_next. Quantifying the complexity levels can reduce the overhead associated with indicating complexity and / or can simplify the encoding / decoding process (s).

Figure 12 illustrates an example of different levels of complexity and resolutions that can be selected by a client device (eg, mobile device, WTRU, etc.) when performing energy-sensitive real-time broadcasting. For example, generally speaking, as the quality level increases, the energy used to decode the video segment may increase. A client device may receive one or more of the quality information for a segment, the bit rate information for the segment, the complexity information for the segment, and / or other information related to the encoding of the segment from from the MPD metadata file. Based on current power restrictions (for example, and potentially based on current bandwidth), the client device can select a resolution and / or complexity level

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0047.tif" />

suitable for a subsequent segment. In a —example ··; —the · client device can provide the server with information related to its current power level, current power usage, amount of power remaining (for example, battery level), amount of power that can be allocated to complete decoding, current bandwidth, statistics regarding previous decoded segments (for example, the complexity of the previous segment and the energy used to decode the segment, etc.), and / or other information and the server can select the appropriate segment to be sent in the stream.

For example, a mobile device can request an MPD file for a certain video. The MPD file can provide complexity information applicable to one or more segments for the video. For example, the MPD file can indicate levels of complexity that can be selected by the mobile device and / or that can indicate resolutions that can be selected by the mobile device. In one example, at the start of the session, the mobile device may select quality level 802, which may correspond to a high resolution segment with a high complexity mode. For example, the mobile device may select quality level 802 based on the remaining power level for⁶⁸ IMPI ^

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL * = ¿uYES_ the mobile device that is above a first threshold. In one example, the mobile device may select quality level 802 based on a determination that it will be able to decode video to quality level 802 without depleting power, for example, assuming that each subsequent segment is also used. it also sends quality level 802.

The mobile device can continue to monitor its decoding power statistics and / or its remaining power level during the video decoding process. For example, after a period of time, the mobile device's remaining power level may drop below the first threshold (for example, but may be above a second threshold). Based on the energy level that is below the threshold, the mobile device can request that the next segment be sent using quality level 804. Quality level 804 can correspond to a high resolution segment with a mode of intermediate complexity. By switching to intermediate complexity mode, energy savings can be achieved during segment decoding. If the remaining power level falls below a second threshold, the mobile device may switch to quality level 806, which may correspond to a high resolution segment with a low mode. ⁶⁹ IMPI 0¾

MEXICAN INSTITUTE

OF THE PROPERTY VSb- ^ Β ·? J.

INDUSTRIAL complexity. If the remaining power level increases above certain thresholds, and / or if the mobile device is connected to a fixed power source (for example, a charger or is connected to a new power source), the mobile device it can be activated to request higher quality segments (for example, at quality level 802).

In order to achieve additional energy savings, in addition to selecting a suitable complexity mode based on a power level of the mobile device, the resolution can be selected based on the power level. For example, if your mobile device has requested quality level 806, but you want additional power savings (for example, current power usage may cause the remaining power to be fully used during video playback), the device mobile may change to a lower resolution. For example, based on the remaining energy that falls below a third threshold, the mobile device can request quality level 812, it can correspond to an intermediate resolution segment with a high complexity mode. Similar threshold-based analysis can be performed for intermediate resolution segments to select between 812, 814 and / or 816 quality levels. Likewise, if additional savings of

IMPI iwrnvrmmkano

MLAFMmiDAD iNDurnuAí.

<img file="MX345571B_D0048.tif" />

power after switching to intermediate resolution, the mobile device can select low resolution, and can perform threshold energy level analysis in order to select a suitable level of complexity in low resolution (for example, between quality levels 822 , 824, and / or 826).

The mobile device may select a suitable resolution and / or level of complexity based on remaining power, previous statistics regarding power usage, and / or other power indicators. Although the low, medium, and high resolutions are shown as examples, there can be many of these resolution levels and the various combinations of resolution and complexity can produce various levels of power savings. Therefore, the mobile device can calculate the power use for various combinations of resolution and complexity, and can select the appropriate combination in order to achieve a desired level of power use during decoding. In one example, if the resolution is quantized between a few different levels (for example, high, medium, low), switching between the resolutions can achieve greater energy savings than changing the complexity levels (for example, although this may not always may be the case). In these scenarios, the mobile device can switch between

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX345571B_D0049.tif" />

resolutions to achieve larger amounts of energy savings and to be able to toggle between complexity levels in a resolution in order to achieve finer granularity of energy savings.

Energy sensitive decoding can include identifying one or more sub-processes or modules in the decoding process and optimizing, or changing the parameters and / or mode of operation of one or more of the sub-processes with in order to minimize energy use while maintaining acceptable video quality and / or user experience. As an example, consider the HEVC video compression standard (for example, although the techniques, methods, and systems described may be applicable to other video compression techniques).

Figure 9 illustrates an exemplary process for performing energy sensitive decoding using HEVC. For example, HEVC decoding may include 902 entropy decoding, 904 dequantization, 906 inverse transform, 908 circuit filter, 910 reference image store, 912 spatial prediction, 914 temporal prediction (e.g. motion compensation (MC - motion compensation)), and / or the like. In one example, one or more of the components of the decoding process may ⁷² IMPI

MEXICAN INSTITUTE DE LA PROPIEDAD INDUSTRIAL modify the parameters and / or methods used in the component based on the energy use and / or the remaining energy level information. For example, the temporal prediction 914 and / or the circuit filter 908 can be varied in order to strike a balance between quality and power consumption. During periods when additional energy savings are desired, the time forecast 914 and / or the circuit filter 908 can be modified in order to save energy.

For example, encoded bitstream 900 can be decompressed and / or its entropy decoded in entropy decoding unit 902. Entropy decoding complexity can be highly related to the size of each compressed image. The more bits that are used to compress the image, the more energy the entropy decoding process can use. The mobile device including the energy sensitive decoder may request a bit stream that is encoded using fewer bits in order to save energy during entropy decoding. The encoding mode, prediction information, motion information, and / or other information determined in entropy decoding unit 902 may be sent to spatial prediction unit 912 (eg, if intracoded) and / or to temporal prediction unit 914 ⁷³ IMPI ^

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL {for example, if intercoded) to form the prediction block. If intercoded, the prediction information may comprise prediction block sizes, one or more motion vectors (for example, which may indicate direction and amount of motion), and / or one or more reference indices (e.g. For example, which can indicate from which reference image the prediction signal is to be obtained). The motion compensation prediction can be applied by the temporal prediction unit 914 to form the temporal prediction block. Temporal prediction (eg, motion compensation) can account for a relatively large portion of decoder power usage, for example, because temporal prediction can use intensive external memory access for filtering.

The residual transform coefficients can be sent to the dequantization unit 904 and the inverse transform unit 906 to reconstruct the residual block. The prediction block and the residual block can be added together to form a reconstructed block. The rebuilt block can be sent to circuit filter 908 before being stored in reference image store 910. Reconstructed video in reference image store 910 can be used to drive a display device

IMPI

MEXICAN INSTITUTE Dt LA INDUSTRIAL PROPERTY

<img file="MX345571B_D0050.tif" />

(eg decoded video 920) and / or is used to predict future video blocks.

A single layer video encoder can have a single video stream input and generate a single compressed bit stream transmitted to the single layer decoder. A video codec can be designed for digital video services (for example, such as, but not limited to, sending satellite TV signals, cable and terrestrial broadcast channels). With video-centric applications deployed in heterogeneous environments, multi-layer video coding technologies can be developed as an extension of video coding standards to enable various applications. For example, scalable video coding technologies can be designed to handle more than one video layer, where each layer can be decoded to reconstruct a video signal of a particular spatial resolution, temporal resolution, fidelity, and / or view. Aungue describes a single layer decoder with reference to Figure 9, the concepts described herein can utilize a multi-layer decoder, for example, for scalable multi-layer encoding technologies.

IMPI MEXICAN INSTITUTE O £ LA PROPERTY industrial

In the video code as an example suchlike HEVC, in circuit filter 908 can be usedan 8-lead filter for interpolation at the half-pixel position, and 7-lead non-zero filters can be used for and for pixel position. If the size of the prediction block is W * H (for example, where W can represent the width of the prediction block and H can represent the height of the prediction block), the pixels extracted from the external reference image buffer can be (W + 7) * (H + 7) for the half-pixel position in both the vertical and horizontal directions. Figure 10 illustrates examples of different pixel positions for the luminance motion compensation process that can be performed for temporal prediction.

For example, pixel positions (eg fractional pixel positions) can be grouped into a plurality of classes (eg 6) shaded as shown in Figure 10. Pixel positions can be grouped into classes based on the number of pixels used for interpolation filtering at respective pixel position (eg fractional). For example, Table 2 identifies examples of the classes, the respective pixel positions associated with the classes, the memory size used for filtering

<img file="MX345571B_D0051.tif" />

ΙΜΡΓ μμκ institute>_νο of industrial PROPERTY

<img file="MX345571B_D0052.tif" />

interpolation during rn ^ ponniAr— will murimine the different classes, and the number of times the filter operation is applied for each class. For example, the most complex value might be position (^ í, (for example, position 10 in Figure 10), since position (^ i, ^) can use the largest memory size to refer to the reference frame and may involve the application of both a horizontal and vertical filter operation.

If the memory address is configured horizontally (for example, which may normally be the case), the memory access efficiency for horizontal interpolation may be higher than vertical interpolation. Loop filtering may include one or more of unblocking, using adaptive sample offset (SAO), and / or adaptive loop filtering (ALF). Unblocking can be used to reduce discontinuity at transform block boundaries, and can have many comparison and conditional operations. SAO can be used to correct for discontinuity at edge points. ALF can be a linear filtering process and can focus on the appearance of some or all of the pixels. These circuit filters can be very power consuming as they can use a significant number of pixel based operations.

<img file="MX345571B_D0053.tif" />

IMPI

MEXICAN INSTITUTE

OF INDUSTRIAL MOEIITY

<td>Lessons</td><td>Position in Fig. 10</td><td>Memory size</td><td>Filter operation times</td>

<td> 1</td><td> 0</td><td>W * H</td><td> 0</td>

<td rowspan="2"> 2</td><td> 1, 3</td><td>(W + 6) * H</td><td>1 (horizontal)</td>

<td> 4, 12</td><td>W * (H + 6)</td><td>1 (vertical)</td>

<td rowspan="2"> 3</td><td> 2</td><td>(W + 7) * H</td><td>1 (horizontal)</td>

<td> 8</td><td>W * (H + 7)</td><td>1 (vertical)</td>

<td> 4</td><td> 5, 7, 13, 15</td><td>(W + 6) * (H + 6)</td><td>2 (horizontal and vertical)</td>

<td rowspan="2"> 5</td><td> 6, 14</td><td>(W + 7) * (H + 6)</td><td>2 (horizontal and vertical</td>

<td> 9, 11</td><td>(W + 6) * (H + 7)</td><td>2 (horizontal and vertical</td>

<td> 6</td><td> 10</td><td>(W + 7) * (H + 7)</td><td>2 (horizontal and vertical</td>

Table 2: Example of memory size queried by for W * H block MC

Given the complexity of motion compensation it can vary based on the fractional pixel position (for example, based on the size of the memory access and the number of filter operations dependent on the fractional pixel location class), using a subset of the classes can result in less complex decoding. For example, if power savings are desired, the encoder can encode a less complex video segment (eg, a segment that can be decoded using relatively less power), refraining from using one or more classes. For example, the encoder can refrain from using classes where excluding the use of the class during encoding does not affect

IMPI

MEXICAN INSTITUTE

OF THE CURRENCY

INDUSTRIAL rate distortion. By

<img file="MX345571B_D0054.tif" />

significantly the example level, if the omission of two classes 5 and 6) causes a class change (for example, those in the distortion rate that is less than a certain threshold, the encoder can encode unused less complex segments of the classes omitted. '

The threads of the decoding process can occupy different percentages of the processor resources and / or take a long time to complete. For example, Figures, 11A and 11B illustrate examples of time profiles for HEVC using an HM6.1 decoder and encoded bit streams using a random access setting (RA). Without the use of single instruction multiple data (Figure 11A), motion compensation can account for about 61% of decoding time, unlocking and SAO together can take about 14% (for example, about 9% and 5 %, respectively), the entropy decoding can account for about 7%, for example, since the bitrate can be relatively low in this example. SIMDs (single instruction multiple data multiple single instruction data) can greatly speed up the MC, for example up to 2 * -3 * times. However, the CM can still represent 38% of the

<img file="MX345571B_D0055.tif" />

IMPI

MEXICAN INSTITUTE DE LA NIOLEDAD INDUSTRIAL decoding time (for example, Figure 11B).

In one example, the motion compensation and / or operation of the loop filter can be modified in order to dynamically reduce power consumption based on current power conditions and / or previous power statistics. For example, Figure 12A illustrates an exemplary interpolation filter waveform for power saving, and Figure 12B illustrates an exemplary interpolation filter waveform for power saving. The filter can be a low pass filter. The interpolation filtering process can be implemented in the frequency domain, for example, using equation (8), where X can be the input signal, F can be the filter, and Y can be the output signal in the frequency domain.

Y = X® F Equation (8)

In the spatial domain, if the input x does not have a large high frequency signal, the filter F can be reduced without generating large errors. Therefore, the motion compensation used can be based on the frequency analysis of the signal. For example, if the region to be interpolated does not have strong edges or strong contrasts (for example, the frequency response lacks a large proportion of frequency components

IMPI

MEXICAN INSTITUTE relatively high), then the energy-sensitive decoder disclosed in the mobile device can determine the application of a filter with a lower cut-off frequency in order to save time and / or energy during the decoding processes. For example, a shorter filter (for example, a lower cutoff frequency) can reduce the memory access bandwidth. Analysis of the characteristics of the region to determine whether the shorter filter can be applied without significantly affecting the decoding processes (for example, determining whether the frequency components of the region are primarily of lower frequencies) can be performed using relatively low energy. low. For example, if additional energy savings are desired and a certain given percentage of the frequency components associated with the region are less than a specified frequency threshold, filtering can be done using a relatively lower frequency filter.

In one example, the determination as to whether the lower frequency filter should be applied can be made based on low resolution images (eg analysis is performed on sampled pixels in order to save energy). For example, the determination as to whether the lower frequency filter is applied can be made based on an analysis of sampled pixels, for example, rather than based on all pixels associated with the image or a portion of the image. For example, when analyzing a 16 * 16 block, the pixels at the positions (4n, 4m) can be analyzed and considered, where n and m are integers (for example, the analysis can be performed on 16 pixels to determine the filter to apply for the 256 pixel block).

There can be two issues exposed by the energy movement compensation process. First, an error propagation to future images can occur due to an error in the temporal prediction process. For example, if an error is introduced into a reference image due to the use of the lower frequency filter, the error can spread to future images that use the reference for temporal prediction. In order to limit propagation error, in one example, lower frequency filtering can be applied to higher layer images, but not to lower layer images. From the example, in the HEVC and video codes, a hierarchical coding structure like the one illustrated in Figure 17 can be used. For example, there may be four layers and images in lower layers can refrain from using images in higher layers for temporal prediction. Therefore, in one example, by using the lower frequency filter for higher layers (eg higher layers) and / or layers that are not based on temporal prediction by other layers, energy savings can be achieved simultaneously limiting the propagation effect of the error.

A second issue that may be exposed due to the use of the lower frequency filter may be the propagation of error due to intraprediction in the same image. HEVC, H.264, and / or other codees may request directional intraprediction in order to improve intracoding efficiency. For example, HEVC can use directional intraprediction in a plurality of directions. Figure 14 illustrates the exemplary addresses used in HEVC intracoding. If an error is generated in the current encoding block due to the use of the lower frequency filter, and if a neighboring block to the current block is intracoded, then the error in the current block can propagate to the pixels along the address prediction in its neighboring block. The error can continue to propagate if there are several intrablocks spatially adjacent to each other, and the impairments can be extended. Therefore, in one example, the energy sensitive encoder in the mobile device may refrain from using the lowest frequency filter of an interblock if there are blocks of

IMPI Mexican Institution of Industrial Property

<img file="MX345571B_D0056.tif" />

intracoding on your non-case neighbors (for example, your right neighbor block, your bottom neighbor block, and / or your bottom right neighbor block).

Consequently, the mobile device, even if the mobile device determines that lower-energy decoding should be used, One or more coding blocks can still be decoded using a regular motion compensation filter if one or more intracoding blocks are a non-random neighbor to the respective coding block and / or one or more other coding blocks if a neighboring block is intracodes based on the information encoded in the respective encoding block.

Unblocking can be useful to improve the quality of relatively flat and / or relatively smooth areas in terms of visual quality, since blocking degradations are usually more pronounced in those areas due to characteristics of the human visual system. For areas with high-frequency components such as high-texture areas, a phenomenon called texture masking can make blocking degradations effectively invisible to the human eye. For highly textured and / or small debris interlock areas, skipping unlocking can also save some energy. The error caused by

<img file="MX345571B_D0057.tif" />

IMPI

MEXICAN INSTITUTE

OF THE 'ROPIETY

INDUSTRIAL can be propagated through but the error cannot be intrablock. Therefore, omit unlocking also motion compensation, propagate by prediction of in an example, if the mobile device's power sensitive decoder determines that energy savings should be achieved, unlocking can be omitted for non-reference images , but it can still be done for reference images.

Energy-sensitive decoding and / or energy-sensitive real-time emission can be used separately or in combination to achieve energy savings using customer-based techniques {for example, energy-sensitive decoding such as usage low pass filter with a lower cutoff frequency during motion compensation) and / or techniques based on common customer networks (for example, the server provides information regarding the complexity of different potential flows and the client dynamically requests a suitable flow based on the energy level information and / or energy level information).

The energy-sensitive real-time broadcast can be a collaboration between a server and a client (eg, a WTRU such as a mobile device). For example, content can be generated with multiple

IMPI

MEXICAN INSTITUTE DE LA PROPISBao INDUSTRIAL versions, and each version can be associated with a different complexity and / or a different resolution. The complexity information can be requested by the client and sent from the server in a media description file for the content. The customer can select a suitable media segment based on information such as the power level status for the client, the customer power level status, the expected playback time, the processor usage status, the preferences of the user, an indication received from the user, available bandwidth, and / or the like.

For example, a user configures the customer's mobile device to operate in a high-quality mode. In high quality mode, the user can prefer quality over energy reduction. When operating in high quality mode, the mobile device can be configured to take into account the levels of decoding complexity and to determine the resolution and / or the level of complexity that maximizes quality, while ensuring that the mobile device has enough remaining power to complete all video playback. The mobile device can calculate the amount of energy that will be used for the rest of the video based on the previous energy statistics, the information of the complexity associated with the

<img file="MX345571B_D0058.tif" />

⁸⁶ IMPI

MEXICAN INSTITUTE r * LA INDUSTRIAL PROPERTY energy statistics above, and information on the complexity associated with the remaining video segments.

In one example, the user may configure the client mobile device to operate in a power saving mode, in which the user may prefer less power dissipation over better quality. In power saving mode, the mobile device can be configured to use energy-sensitive real-time broadcast and / or energy-sensitive decoding in order to minimize energy use. Utilize the energy consumption statistics collected by simultaneously decoding associated previous images

<img file="MX345571B_D0059.tif" />

At one or more levels of complexity, the customer can allocate the power for the next segment according to the remaining power. Based on the assigned power and / or the complexity of the previous segment, the customer can calculate the complexity of the current segment. The customer can then find out what level of complexity should be requested in subsequent segments based on the amount of power remaining.

Energy sensitive decoding technology can be based on content analysis of one or more blocks to be decoded. Power sensitive decoding can attempt to strike an acceptable balance between decoding complexity / power usage and

<img file="MX345571B_D0060.tif" />

⁸⁷ IMPI

MEXICAN INSTITUTE

OF LA RROHEDA »

INDUSTRIAL quality. Power sensitive decoding may be able to save power consumption by introducing barely noticeable errors. There may be several methods for energy sensitive decoding. For example, the decoder can apply different interpolation filters associated with different characteristic areas for low priority images. For regions with many high-frequency signals, such as sharp edges or texture, the decoder can use a suitable or normal interpolation filter to avoid introducing errors. For relatively flat regions with fewer components / high frequency signals, the decoder can reduce the cutoff frequency of the low pass filter in order to reduce the memory access bandwidth. In one example, the power sensitive decoder can save power consumption by omitting unlocking operations in areas where lock impairments are less visible and / or in areas where error propagation is less problematic (e.g. , such as in non-reference images or reference images in higher temporal layers). In one example, for an inverse transform, the decoder may apply a smaller transform size according to the nonzero coefficient distribution, for example, because most of the

WICKED

MEXICAN INSTITUTE JS of the property

INDUSTRIAL i »nonzero coefficients can be distributed in the upper left corner corresponding to a low frequency region.

A detailed description of illustrative embodiments will now be described with reference to the various figures. Although this description provides a detailed example of possible implementations, it should be noted that the details are intended to be exemplary and in no way limit the scope of the application.

Figure 1A is a diagram of an exemplary communication system 100 in which one or more of the described embodiments may be implemented. Communication system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. Communication system 100 may allow multiple wireless users to access such content through the allocation of system resources, including wireless bandwidth. For example, communication systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA). , multiple access by

<img file="MX345571B_D0061.tif" />

⁸⁹ IMPI Mexican nwrnrro

I THE PROPERTY

INDUSTRIAL frequency division (FDMA - freauencv, multiple division access), orthogonal FDMA (OFDMA - orthogonal FDMA), single carrier FDMA (SC-FDMA), and the like.

As shown in Figure 1A, communication system 100 may include wireless transmission / reception units (WTRUs) 102a, 102b, 102c, and / or 102d (which may generally or collectively be referred to as WTRU 102), an access network radio network (RAN) 103/104/105, a central network 106/107/109, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, although it will be observed that the described modalities consider any number of base stations, WTRUs, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d can be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d can be configured to transmit and / or receive wireless signals and can include user equipment (UE - user equipment), a mobile station, a fixed or mobile subscriber unit, a pager, cell phone, personal digital assistant (PDA), smartphone, laptop, mini-laptop, personal computer, wireless sensor, consumer electronics, and the like.

⁹⁰ IMPIAS

MEXICAN INSTITUTE

OF THE MOHEDA »

INDUSTRIAL

The communication system 100 may also include a base station 114a and a base station 114b. Each of the base stations 114a, 114b can be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d in order to facilitate access to one or more communication networks, such such as the 106/107/109 core network, the Internet 110, and / or the 112 networks. As an example, base stations 114a, 114b can be a base transceiver station (BTS), a Node B, an e-Node B, a local Node B, a local Node B, a site controller , an access point (AP - access point), a wireless router, and the like. Although each of the base stations 114a, 114b is represented as a single element, it will be appreciated that the base stations 114a, 114b can include any number of interconnected base stations and / or network elements.

Base station 114a may be part of RAN 103/104/105, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. Base station 114a and / or base station 114b can be configured to transmit and / or receive signals

IMPI iNrnryro mexicana

OS THE PROMITY

INDUSTRIAL

<img file="MX345571B_D0062.tif" />

within a particular geographic region, which may be called a cell (not shown). The cell can also be divided into cell sectors. For example, the cell associated with base station 114a can be divided into three sectors. Consequently, in one embodiment, base station 114a may include three transceivers, ie, one for each sector of the cell. In another embodiment, the base station 114a can employ multiple input multiple output (MIMO) technology and therefore can use multiple transceivers for each sector of the cell.

Base stations 114a, 114b can communicate with one or more WTRUs 102a, 102b, 102c, 102d via an air interface 115/116/117, which can be any suitable wireless communications link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 115/116/117 can be established using any suitable radio access technology (RAT).

More specifically, as noted above, communication system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. . For example, the base station⁹² κπττυτο Mexican i

DE LA FROFICDA industrial t * 9 ^ 114a in RAN 103/104/105 and WTRUs 102a, 102b, 102c can implement a radio technology such as the Terrestrial Radio Access (OTRA - UMTS Terrestrial Radio) of the Universal System of Mobile Telecommunications (UMTS Universal Mobile Telecommunications System), which can establish the 115/116/117 air interface using broadband CDMA (WCDMA - Wideband CDMA). WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA +). The HSPA may include High-Speed ​​Downlink Packet Access (HSDPA) and / or High-Speed ​​Uplink Packet Access (HSUPA).

In another embodiment, the base station 114a and the WTRUs 102a, 102b, 102c can implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can interface 115/116/117 using long-term evolution (LTE - Long Term Evolution) and / or LTE-advanced (LTE-Advanced).

In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies, such as IEEE 802.16 (i.e.

Worldwide interoperability for microwave access

<img file="MX345571B_D0063.tif" />

IMPI

I HEARD INDUSTRIAL CURRENCY (WiMAX)), CDMA2000, CDMA2000 IX, cnMMrw fv-dq, Norma

Provisional 2000 (IS-2000), Provisional Standard 95 (IS-95),

Provisional Standard 856 (IS-856), Global System for

Mobile Communications (GSM - Global System for Mobile Communications), Enhanced Data rates for GSM Evolution (EDGE - Enhanced Data rates for GSM Evolution), EDGE GSM (GERAN), and the like.

Base station 114b in Figure 1A can be a wireless router, local node B, local e-Node B, or access point, for example, and can use any suitable RAT to facilitate wireless connectivity in a localized area, such as a business, a house, a vehicle, a college campus, and the like. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technology such as the IEEE 802.11 standard in order to establish a wireless local area network (WLAN). In another embodiment, the base station 114b and the WTRUs 102c, 102d can implement a radio technology such as the IEEE 802.15 standard in order to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may use a cellular-based RAT (eg, WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.) in order to establish a picocell or femtocell. . As shown in Figure 1A, base station 114b may have a

MEXICAN INSTITUTE

DE LA TROPIEDA »industrial direct connection to the Internet 110. Therefore, In base 114b may not be necessary to access the Internet 110 through the central network 106/107/109.

The RAN 103/104/105 can be in communication 5 with the core network 106/107/109, which can be any type of network configured to provide voice, data, application, and / or voice over Internet protocol (VoIP) services. - voice over internet protocol) to one or more WTRUs 102a, 102b, 102c, 102d. For example, the core network

106/107/109 can provide call control, billing services, mobile location based services, prepaid calls, Internet connectivity, video distribution, etc., and / or execution of high-level security functions, such as 15 user authentication. Although not shown in Figure 1A, it will be noted that RAN 103/104/105 and / or core network 106/107/109 may be in direct or indirect communication with other RANs that use the same RAT as RAN 103 / 104/105 or a different RAT. For example, in addition to connecting to RAN 103/104/105, which may be using E-UTRA radio technology, core 106/107/109 may also be in communication with another RAN (not shown) using a GSM radio technology.

The 106/107/109 core network can also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d

<img file="MX345571B_D0064.tif" />

⁹⁵ IMPI

MEXICAN INSTITUTE DE LA PROPIEDAD INDUSTRIAL in order to access PSTN 108, Internet 110, and / or other 112 networks. PSTN 108 may include circuit-switched telephone networks that provide traditional telephone service (POTS - plain old telephone service) . The Internet 110 may include a worldwide system of interconnected computer networks and devices that utilize common communication protocols, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP). ) and the Internet protocol (IP - internet protocol) in the TCP / IP internet protocol suite. Networks 112 may include wired or wireless communication networks owned and / or operated by other service providers. For example, networks 112 may include a core network connected to one or more RANs, which may employ the same RAT as RAN 103/104/105 or a different RAT.

Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities, i.e. the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers to communicate with different wireless networks over different links. wireless. For example, the WTRU 102c shown in Figure 1A can be configured to communicate with the base station 114a, which can "IMPI ^

MEXICAN INSTITUTE • E LA erofiedad tSZjSYes

INDUSTRIAL ^ »^ 8Βγ” »» employ a network-based radio technology-i * 4 - = ^ y with base station 114b, which can employ IEEE 802 radio technology.

Figure IB is a system diagram of an exemplary WTRU 102. As shown in Figure IB, the WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keyboard 126, a screen / touch surface 128, a non-removable memory 130, a removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and other peripherals 138. It will be appreciated that the WTRU 102 may include any subcombination of the above elements as long as it is consistent with one embodiment. Also, the modalities consider that the base stations 114a and 114b, and / or the nodes that the base stations 114a and 114b can represent, such as, but not limited to the transceiver station (BTS), a Node-B, a controller of site, an access point (AP), a local node B, an evolved local node B (eNodeB), a local evolved node B (HeNB), a local evolved node B network gateway, and intermediary nodes, among others , it may include some or each of the elements graphically represented in Figure IB and described herein.

<img file="MX345571B_D0065.tif" />

IMPI

MEXICAN INSTITUTE

M THE FROWtOAD

INDU1TMAL

Processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC - integrated circuit), a state machine, and the like. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other functionality that allows the WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, which can be coupled to transmit / receive element 122. Although Figure IB graphically depicts processor 118 and transceiver 120 as separate components, it will be appreciated that processor 118 and transceiver 120 can be integrated. together in an electronic package or chip.

The transmitting / receiving element 122 can be configured to transmit signals to, or receive signals ⁹⁸ IMPI ^ h

ΙΝΓΠΤυΤΟ MEXICAN

Dt THE FROFIEBY A

INDUSTRIAL from a base station (eg, base station 114a) via air interface 115/116/117. For example, in one embodiment, the transmitting / receiving element 122 may be an antenna configured to transmit and / or receive RF signals. In another embodiment, the transmitting / receiving element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmitting / receiving element 122 may be configured to transmit and receive both RF and light signals. It will be noted that the transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

Furthermore, although the transmit / receive element 122 is depicted in Figure IB as an individual element, the WTRU 102 can include any number of transmit / receive elements 122. More specifically, the WTRU 102 can employ MIMO technology. Therefore, in one embodiment, the WTRU 102 may include two or more transmitting / receiving elements 122 (eg, multiple antennas) to transmit and receive wireless signals over the air interface.

115/116/117.

<img file="MX345571B_D0066.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

The transceiver 120 may be configured to modulate the signals to be transmitted by the transmit / receive element 122 and to demodulate the signals that are received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Therefore, the transceiver 120 can include multiple transceivers to allow the WTRU 102 to communicate via multiple RATs, such as the OTRA and the IEEE 802.11 standard, for example.

The processor 118 of the WTRU 102 can be coupled to, and can receive user input data from, the speaker / microphone 124, the keyboard 126, and / or the screen / touch surface 128 (e.g., a display unit of Liquid crystal display (LCD) or organic light-emitting diode (OLED) display unit. Processor 118 can also output user data to speaker / microphone 124, keyboard 126, and / or screen / touch surface 128. Additionally, processor 118 can access information from, and store data in, any suitable type of memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 may include random-access RAM

100

<img file="MX345571B_D0067.tif" />

IMPI

MEXICAN INSTITUTE DE LA PROPIEDAD INDUSTRIAL memory), read-only memory (ROM read-only memory), a hard disk, or any other type of memory storage device. Removable memory 132 may include a subscriber identity module (SIM) card, a portable memory, a secure digital memory (SD-Secure Digital) card, and the like. In other embodiments, processor 118 can access information from, and store data in, memory that is not physically located in the WTRU 102, such as on a server or home computer (not shown).

Processor 118 can receive power from power supply 134, and can be configured to distribute and / or control power to the other components in WTRU 102. Power supply 134 can be any suitable device for powering WTRU 102 . For example, power source 134 may include one or more dry cell batteries (for example, nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium ion (Li -ion), etc.), solar cells, fuel cells, and the like.

Processor 118 can also be coupled to GPS chipset 136, which can be configured to provide location information (e.g., longitude and latitude) relative to the current location of ¹¹ IMPI

MEXICAN INSTITUTE PE LA MtOMBDA · INDUSTRIAL the WTRU 102. In addition to, or instead of, the information from the GPS 136 chipset, the WTRU 102 can receive location information via the air interface 115/116/117 from a station base stations (eg, base stations 114a, 114b) and / or determine their location based on the timing of the received signals from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location determination method while maintaining compatibility with one mode.

Processor 118 may also be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional wired or wireless features, functionality, and / or connectivity. For example, peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB - universal serial bus) port, a vibration device, a transceiver television, hands-free headphones, a Bluetooth® module, a radio frequency modulated (FM) unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.

102

<img file="MX345571B_D0068.tif" />

IMPI

INSTITUTE M1XKAN ·

Say THE INDUSTRIAL FROTTITY

Figure 1C is a system diagram of the RAN

103 and the core network 106 according to one embodiment. As noted above, RAN 103 may employ UTRA radio technology to communicate with WTRUs 102a, 102b, 102c via air interface 115. RAN 103 may also be in communication with core network 106. As shown in Figure 13C, RAN 103 may include Node Bs 140a, 140b, 140c, which may include one or more transceivers for communicating with WTRUs 102a, 102b, 102c via air interface 115. Node Bs 140a, 140b, 140c can each be associated with a particular cell (not shown) within RAN 103. RAN 103 can also include RNCs 142a, 142b. It will be appreciated that the RAN 103 can include any number of Node Bs and RNCs while maintaining compatibility with one mode.

As shown in Figure 13C, Node Bs 140a, 140b may be in communication with RNC 142a. Additionally, Node-B 140c may be in communication with RNC 142b. Node Bs 140a, 140b, 140c can communicate with respective RNCs 142a, 142b via a lub interface. The RNCs 142a, 142b may be in communication with each other via a lub interface. Each of the RNCs 142a, 142b can be configured to control the respective Node Bs 140a, 140b, 140c to which it connects. In addition, each of the RNCs 142a, 142b can be configured to perform

103

<img file="MX345571B_D0069.tif" />

IMPI

MEXICAN INSTITUTE M LA MONEDAD INDUSTRIAL or support other functionality, such as external circuit power control, load control, admission control, packet scheduling, transfer control, macro diversity, security functions, data encryption, and the like.

The core network 106 shown in Figure 1C may include a media gateway (MGW) 144, a mobile switching center (MSC) 14 6, a GPRS support node (SGSN 10 serving GPRS support node) in service 148, and / or a network access GPRS support node (GGSN - gateway GPRS support node) 150. Although each of the above elements is represented graphically as part of the core network 106, it will be appreciated that any of these elements 15 may be owned and / or operated by an entity other than the core network.

RNC 142a in RAN 103 can connect to MSG 146 in core network 106 via a luCS interface. MSC 146 can connect to MGW 144. MSC 146 and MGW 144 can provide WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as PSTN 108, in order to facilitate communications between stations. WTRUs 102a, 102b, 102c and traditional wireline communication devices.

104

IMPI

INSTITUTE MRXJCANO

OF THE RWOniTY

INDUSTRIAL

<img file="MX345571B_D0070.tif" />

RNC 142a in RAN 103 can also connect to SGSN 148 in core network 106 via a luPS interface. SGSN 148 can connect to GGSN 150. SGSN 148 and GGSN 150 can provide WTRUs 102a, 102b, 102c with access to packet-switched networks, such as Internet 110, in order to facilitate communications between WTRUs 102a, 102b, 102c and IP-enabled devices.

As noted above, core network 106 may also connect to networks 112, which may include other wired or wireless networks that are owned and / or operated by other service providers.

Figure ID is a stem diagram of RAN 104 and core network 107 in accordance with one embodiment. As noted above, RAN 104 may employ E-UTRA radio technology to communicate with WTRUs 102a, 102b, 102c via air interface 116. RAN 104 may also be in communication with core network 107.

RAN 104 may include e-Node Bs 160a, 160b, 160c, although it will be appreciated that RAN 104 may include any number of e-Node Bs while still being compatible with one modality. The e-Nodes 160a, 160b, 160c may include one or more transceivers to communicate with the WTRUs 102a, 102b, 102c via the air interface 116. In one embodiment, the e-Nodes 160a

160b, 160c can

105 implement MIMO technology. So,

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROMITY

<img file="MX345571B_D0071.tif" />

e-Node B 160a, for example, can use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

Each e-Node B 160a, 160b, 160c can be associated with a particular cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling on the link uplink and / or downlink, and the like. As shown in Figure ID, the e-Nodes 160a, 160b, 160c can communicate with each other via an X2 interface.

The core network 107 shown in Figure ID may include a mobility management gateway (MME) 162, a service gateway 164, and a packet data network gateway (PDN). packet data network) 166. Although each of the above elements is represented as part of the core network 107, it will be noted that any of these elements may be owned and / or operated by an entity other than the core network operator.

The MME 162 can connect to each of the e-Node Bs 160a, 160b, 160c in the RAN 104 via an SI interface and can serve as a control node. For example, the MME 162 may be sensitive to authenticate

106

<img file="MX345571B_D0072.tif" />

WTRU users

IMPI

MEXICAN INSTITUTE M LA MONEDAD inouítxjajl

102a, 102b, 102c, carrier activation / deactivation, selecting a particular service gateway during an initial connection of the WTRUs 102a, 102b, 102c, and the like. The MME 162 can also provide a control plane function to toggle between the RAN 104 and other

RANs (not shown) that use other radio technologies, such as GSM or WCDMA.

The service gateway 164 may connect to each of the e-Node Bs 160a, 160b, 160c in the RAN 104 via the SI interface. The service gateway 164 can generally route and send the user data packets to / from the WTRUs 102a, 102b, 102c. The service gateway 164 can also perform other functions, such as setting user planes during inter-Node-B handoffs, activating paging when downlink data is available for WTRUs 102a, 102b, 102c, manage and store the contexts of the WTRUs 102a, 102b, 102c, and the like,

Service gateway 164 can also connect to PDN gateway 166, which can provide WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between

107

<img file="MX345571B_D0073.tif" />

the 102nd WTRUs

102b,

IMPI

INSTITUTE ΜΕΙΚΤΟ • t INDUSTRIAL PROPERTY

102c and devices enabled for

IP.

The core network 107 can facilitate communications with other networks. For example, core network 107 may provide WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as PSTN 108, in order to facilitate communications between WTRUs 102a, 102b, 102c and traditional communication devices. wireline communications. For example, the core network 107 may include, or may communicate with, an IP gateway (for example, an IP multimedia subsystem server (IMS - IP multimedia subsystem)) that serves as an interface between the core network 107. and PSTN 108. In addition, core network 107 may provide WTRUs 102a, 102b, 102c, access to networks 112, which may include other wired or wireless networks owned and / or operated by other service providers. .

Figure 1E is a system diagram of RAN 105 and core network 109 in accordance with one embodiment. RAN 105 may be an ASN access service network that employs IEEE 802.16 radio technology to communicate with WTRUs 102a, 102b, 102c via air interface 117. As will be described later, the communication links between the different

IMPI

MEXICAN INSTITUTE t »LA PROHIDAD

IN »WTWAL

<img file="MX345571B_D0074.tif" />

108 functional entities of WTRUs 102a, 102b, 102c, RAN

105, and core network 109 can be defined as reference points.

As shown in Figure 1E, RAN 105 can include base stations 180a, 180b, 180c, and an ASN 182 gateway, although it will be appreciated that RAN 105 can include any number of base stations and ASN gateways. while maintaining compatibility with a modality. Each of the base stations 180a,

180b, 180c may be associated with a particular cell (not shown) in RAN 105 and each may include one or more transceivers for communicating with WTRUs 102a, 102b, 102e via air interface 117. In one embodiment, the stations base 180a, 180b, 180c can implement MIMO technology. Thus, base station 180a, for example, can use multiple antennas to transmit wireless signals to, and receive wireless signals from, WTRU 102a. Base stations 180a, 180b, 180c can also provide mobility management functions, such as handoff activation, tunnel establishment, radio resource management, traffic classification, application of quality of service (QoS - Quality of Service) policies. Service), and the like. The ASN 182 gateway can serve as a traffic accumulation point and

IMPI

MEXICAN INSTITUTE

Ot THE HtOfUGMD

INDUSTRIAL

109

<img file="MX345571B_D0075.tif" />

it may be responsible for paging calls, buffering subscriber profiles, routing to core network 109, and the like.

Air interface 117 between WTRUs 102a, 102b,

102c and RAN 105 can be defined as an R1 reference point that implements the IEEE 802.16 specification. Also, each of the WTRUs 102a, 102b, 102c can establish a logical interface (not shown) with the core network 109. The logical interface between the WTRUs 102a, 102b, 102e and the core network 109 can be defined as an R2 reference point, which can be used for authentication, authorization, IP host configuration management, and / or mobility management. .

The communication link between each of the base stations 180a, 180b, 180c can be defined as an R8 reference point that includes protocols to facilitate WTRU handoffs and data transfer between the base stations. The communication link between the base stations 180a, 180b, 180c and the ASN gateway 182 can be defined as a reference point R6. The R6 reference point may include protocols to facilitate mobility management based on the mobility events associated with each of the 102nd WTRUs.

102b

102c.

<img file="MX345571B_D0076.tif" />

¹¹⁰ IMPI MEXICAN INSTITUTE Dt LA HIOPIIDA »INDUSTRIAL

As shown in Figure 1E, RAN 105 can be connected to core network 109. The communication link between RAN 105 and core network 109 can be defined as a reference point R3 that includes protocols to facilitate data transfer and mobility management capabilities, for example. The core network 109 may include a mobile IP home agent (MIPHA) 184, an authentication, authorization, accounting server 186, and a gateway 188. Although each of the elements above is represented graphically as part of the core network 109, it will be noted that any of these elements may be owned and / or operated by an entity other than the core network operator.

The MIP-HA may be responsible for IP address management, and may allow the WTRUs 102a, 102b, 102c to roam between different ASNs and / or different core networks. The MIP-HA 184 can provide WTRUs 102a, 102b, 102c with access to packet switched networks, such as the Internet 110, in order to facilitate communications between WTRUs 102a, 102b, 102c and IP-enabled devices. . ΆΑΆ 186 server may be responsible for user authentication and supporting lll services

<img file="MX345571B_D0077.tif" />

IMPI

INSTITUTO MCXICAN · »t la momo * · industxml user. Gateway 188 can facilitate interaction with other networks. For example, gateway 188 may provide WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as PSTN 108, in order to facilitate communications between WTRUs 102a, 102b, 102c and traditional network devices. wireline communications. Additionally, the gateway 188 may provide the WTRU 102a, 102b, 102c with access to the networks 112, which may include other wired or wireless networks that are owned and / or operated by other service providers.

Although not shown in Figure 1E, it will be noted that RAN 105 can connect to other ASNs and core network 109 can connect to other core networks. The communication link between RAN 105 and the others

ASNs can be defined as an R4 reference point, which can include protocols to coordinate the mobility of WTRUs 102a, 102b, 102c between RAN 105 and the other ASNs. The communication link between core network 109 and the other core networks can be defined as a reference R5, which may include protocols to facilitate interaction between local core networks and visited core networks.

Although features and elements in particular combinations have been described previously,

112 Those skilled in the art will appreciate that each feature or element can be used individually or in any combination with the other features and elements. Furthermore, the methods described herein may be implemented in a computer program, software, or firmware embedded in a computer-readable medium for execution by a computer or processor. Some examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Some examples of computer-readable storage media include, but are not limited to, a read-only memory (ROM), a random access memory (RAM), a register, a cache memory, semiconductor memory devices, media magnetic such as internal hard drives and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital versatile disks (DVDs). A processor may be used in association with software to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

113

<img file="MX345571B_D0078.tif" />

IMPI

INSTITUTO mwcano OtUWWtDA »iNDurrtaAL