Reverse channel of user input for wireless display

UKRAINE ₍₁₉₎ iA (11) 107151 (s) C2

(51) IPC

H04b 29/06 (2006.01)

(12) DESCRIPTION TO THE INVENTORY PATENT

<tr><td><p>(21) Application number:</p></td><td><p>and 2013 10266</p></td></tr><tr><td><p>(22) Date of application:</p></td><td><p>January 20, 2012</p></td></tr><tr><td><p>(24) Date from which it is valid</p></td><td><p>November 25, 2014</p></td></tr><tr><td><p>rights to invention:</p></td></tr><tr><td><p>(31) The number of the previous one</p></td><td><p>61 / 435,194,</p></td></tr><tr><td><p>applications according to</p></td><td><p>61 / 447,592,</p></td></tr><tr><td><p>Paris Convention:</p></td><td><p>61 / 448,312,</p></td></tr><tr><td><p>(32) Date of submission</p></td><td><p>61 / 450,101,</p><p>61 / 467,535,</p><p>61 / 467,543,</p><p>61 / 514,863,</p><p>61 / 544,470,</p><p>13 / 344,253</p><p>Jan 21, 2011</p></td></tr><tr><td><p>previous application</p></td><td><p>Feb 28, 2011</p></td></tr><tr><td><p>according to</p></td><td><p>03/02/2011</p></td></tr><tr><td><p>Paris Convention:</p></td><td><p>03/07/2011</p></td></tr><tr><td><p>(33) Code of the State Party</p></td><td><p>March 25, 2011,</p><p>March 25, 2011,</p><p>08/03/2011</p><p>10/07/2011</p><p>01/05/2012</p><p>υδ,</p></td></tr><tr><td><p>The Paris Convention</p></td><td><p>from</p></td></tr><tr><td><p>to which is filed</p></td><td><p>υδ,</p></td></tr><tr><td><p>preliminary application:</p></td><td><p>from</p></td></tr><tr><td><p>(41) Publication of information</p></td><td><p>υδ,</p><p>υδ,</p><p>υδ,</p><p>υδ,</p><p>υδ</p><p>Nov 25, 2013, No. 22</p></td></tr><tr><td><p>about the application:</p></td></tr><tr><td><p>(46) Publication of information</p></td><td><p>Nov 25, 2014, No. 22</p></td></tr><tr><td><p>about the issuance of a patent:</p></td></tr><tr><td><p>(86) Number and date</p></td><td><p>Ρ ^ / υδ2012 / 022072,</p></td></tr><tr><td><p>international representation</p></td><td><p>January 20, 2012</p></td></tr>

applications filed in accordance with the PCT Agreement

(72) The inventor (s):

Huang Xiaolong (υδ)

Ravindran Vijajalakshmi R. (υδ),

Wang Xiaodong (υδ),

Shaukat Fawad (υδ)

(73) Owner (s):

QUALCOM INCORPORATEID,

Institute for the Protection of Human Rights and Human Rights, 5775 Moghioiye, Zhen, Ioei, Saiiyogpia92121, TsIpiiyeS Ziyayye oi Ategis (South Ossetia)

(74) Representative:

Moshynska Nina Nikolaevna, registry number 115

(56) List of documents taken into account by examination:

UZ 2008129879 A1.05.06.2008YUZ 2009133122 A1.21.05.2009YUZ 2010146143 A1, 10.06.2010with 2009109974 A1.30.04.2009

iA 107151 C2

(54) REVERSE USER CHANNEL FOR FREE DISPLAYS

(57) Summary:

As part of a communication session, a wireless source device can transmit audio and video data to the receiver's wireless device, and the wireless device of the receiver can transmit the user input data received on the wireless device of the receiver back to the non-wireless source device. Therefore, the user of the wireless device receiver

iA 107151 C2

can control a wireless source device and control the content that is transferred from the wireless source device to the wireless device of the receiver. Data provided by the wireless user of the recipient may be input data received from a third-party device and sent to a wireless source device.

iA 107151 C2

This application claims priority:

US Preliminary Application No. 61 / 435,194 filed January 21, 2011; US ​​Preliminary Application No. 61 / 447,592 filed Feb. 28, 2011, US Preliminary Application No. 61 / 448,312 filed March 2, 2011, US Preliminary Application No. 61 / 450,101 filed March 7 2011; US ​​Preliminary Application No. 61 / 467,535 filed March 25, 2011; US ​​Preliminary Application No. 61/467, 543, filed March 25, 2011; United States Preliminary Application No. 61 / 514,863 filed Aug. 3, 2011; and US Preliminary Application No. 61 / 544,475 filed Oct. 7, 2011, each of which is fully incorporated herein by reference.

THE FIELD OF TECHNOLOGY TO WHICH THIS REFERENCE ISSUES

This disclosure relates to methods for transmitting data between a wireless device source and the wireless device of the recipient.

PREVIOUS TECHNOLOGY LEVEL

The wireless display (4В) or the system 4І-ЕI display (4ГЮ) include a non-wire device of the source and one or more wireless devices of the receiver. Device sources and each of the recipient devices can be either mobile devices or wireless devices with wireless capabilities. One or more recipient devices and devices may, for example, include mobile phones, portable computers with wireless cards, personal digital assistants (PII assistants), portable media players or other such devices with wireless communication capabilities that include so-called "smartphones" and "smartpads" or tablets, electronicbooks, or any type of wireless display, video game devices, or other types of wireless devices.

The source device sends media data such as audio / video (Αν) data to one or more recipient devices that participate in a particular media sharing session. Media data can be played back both on the local display of the source device and on any of the device displays. the recipient More specifically, each of the participating receiver devices reproduces received media data on its screen and audio equipment.

HOW TO FIND

This disclosure as a whole describes a system in which the wireless device of the receiver can bind to the wireless device of the recipient. As part of the communication, the wireless device source can transmit audio and video data to the wireless device of the receiver, and the receiver's wireless device can transmit custom input received to the receiver's wireless device back to the wireless source device. Thus, the recipient's wireless device user can control the wireless device's resources and control the content that is transmitted from the wireless source device to the receiver's wireless device.

In one example, a method for transmitting user input from a wireless device to a wireless source device includes receiving data from a user input from an external device; generation of the data packet header, and the data packet header identifies the user input data as directed data of the user input; generating useful data containing user-generated data; generating a data packet containing a data packet header and useful data, transferring this data packet to a wireless source device.

In another example, the wireless device of the receiver is configured to transmit data to a user's input on a wireless source device. The wireless device of the recipient includes the memory that stores the commands; one or more processors configured to execute commands, and when executing commands, one or more processors are called: obtaining user input from an external device; generating the header of the data packet, and the header of the data packet identifies the user input data as directional data of the user input; the generation of useful data containing user input data; generation of a data packet containing the data packet header and useful data. The wireless device of the receiver also includes a transport unit for transmitting this data packet to a wireless source device.

In another example, a computer-readable storage medium stores commands that, when using one or more processors, force one or more processors to execute a user input method from the wireless device of the recipient to

1

iA 107151 C2

cordless source device. The method includes obtaining data from the user's command from an external device; generating the header of the data packet, and the header of the data packet identifies the data of the user input as directed user-generated data; generating useful data containing user input; generating a data packet containing the data packet header and useful data, and transferring this data packet to a wireless source device.

In another example, the wireless device of the receiver is configured to transmit data to a user's input on a wireless source device. Wireless receiver includes a means for obtaining user input from an external device; means for generating a data packet header, and the data packet header identifies the user input data as directed data of the user input; a means for generating useful data containing user input; means for generating a data packet comprising a data packet header and useful data, and means for transmitting said data packet to a wireless source device.

In another example, a method for receiving user input data from a wireless device of a receiver in a wireless source device includes receiving a data packet containing a data packet header and useful data from the wireless device of the recipient; performing a parser analysis of the data packet header to determine that the useful data contains a directed command user input; performing a parsing analysis of useful data for identifying third-party identification information, and processing these useful data based on the identification information of a third-party device.

In another example, the wireless source device is configured to receive data from the user's input from the wireless device of the recipient. The wireless device source includes a transport block configured to receive a data packet containing the data packet header and useful data from the wireless device of the receiver. The wireless source device also includes the memory storing commands and one or more processors configured to execute of these commands, and when executing a command, one or more processors call the execution of parser header parser data analysis to determine that the useful data contains a directed user command about the introduction; execution of parsing analysis of useful data for identifying identification information device of a third party,

In another example, a computer-readable storage medium stores commands that, when using one or more processors, force one or more processors to execute a method for receiving user input from a wireless device of the receiver to a non-originating source device. The method includes receiving a data packet containing the data packet header and useful data from the wireless device of the recipient, executing the parsing analysis of the data packet header to determine that the useful data contains a directional user-command; execution of parsing analysis of useful data for the identification of the identification information of a third-party device, and processing of these useful data based on the identification information of a third-party device.

In another example, the wireless source device is configured to receive data from the user's input from the wireless device of the recipient. The wireless device source includes a means for receiving a data packet containing the data packet header and useful data from the wireless device of the recipient, a means for performing a syntax analysis of the data packet header to determine that the useful data contains a directed user input command; a tool for performing a parsing analysis of useful data for identifying third-party device identification, and a tool for processing these useful data based on the identification information of a third-party device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram illustrating an example of a source / receiver system that can implement the methods of this disclosure.

FIG. 1B is a block diagram illustrating an example of a source / receiver system with two device receivers.

FIG. 2 shows an example of a source device that can implement the methods of this disclosure.

FIG. 3 shows an example of a recipient device that can implement the methods of this disclosure.

FIG. 4 shows a block diagram of the transmitter system and receiver systems that can realize the means of this disclosure.

2

iA 107151 C2

FIG. 5A and 5B show exemplary message transfer sequences for executing option matching in accordance with the methods of this disclosure.

FIG. 6 shows an exemplary data packet that can be used to deliver user-entered data received in the receiver device to the source device.

FIG. 7A and 7B are flowcharts illustrating the methods of this disclosure, which may be used to coordinate the capabilities between the source device and the receiver device.

FIG. 8A and 8B are flowcharts illustrating the methods of this disclosure that may be used to transmit and receive packets of data with user input data.

FIG. 9A and 9B are flowcharts illustrating the methods of this disclosure, which may be used to transmit and receive packets of data with user input data.

FIG. 10A and 10B are flowcharts illustrating the methods of this disclosure that can be used to transmit and receive packets of data with timestamp information and user input data.

FIG. 11A and 11B are flowcharts illustrating the methods of this disclosure that can be used to transmit and receive data packets with timestamp information and user input data.

FIG. 12A and 12B are flowcharts illustrating the methods of this disclosure that may be used to transmit and receive data packets that include voice commands.

FIG. 13A and 13B are flowcharts illustrating the methods of this disclosure, which can be used to transmit and receive data packets with user input commands with the help of multiple touches.

FIG. 14A and 14B are flowcharts illustrating the methods of this disclosure that may be used to transmit and receive data packets with user input data sent from a third party device.

FIG. 15A and 15B are flowcharts illustrating the methods of this disclosure that can be used to transmit and receive data packets.

REFERENCE DESCRIPTION

This disclosure as a whole describes a system in which the wireless device of the receiver can bind to the wireless device of the recipient. As part of a communication session, a wireless source device can transmit audio and video data to the wireless device of the receiver, and the receiver's wireless device can transmit user inputs received to the recipient's wireless device, back to the wireless source device. In this way, the user of the wireless receiver device can control the wireless device's source and control the content that is transmitted from the wireless source device to the receiver's wireless device.

FIG. 1A is a block diagram illustrating an exemplary source / receiver system 100 that can realize one or more methods of this disclosure. As shown in FIG. 1a, system 100 includes a source device 120 that communicates with the recipient device 160 via the communication channel 150. The source device 120 may include the memory that stores the audio / video (A / ν) data 121, the display 122, the speaker 123, the audio / video encoder 124 (also referred to as the encoder 124), the audio / video control module 125, and the block 126 transmitter / receiver (TX / RX). The recipient device 160 may include a display 162, a speaker 163, an audio / video decoder 164 (also referred to as a decoder 164), a transmitter / receiver unit 166, a user input device 167, and a user input processing unit 168. The illustrated components are just one exemplary configuration for the source / recipient system. Other configurations may include fewer components than those that are illustrated, or may include additional components than those illustrated.

In the example of FIG. 1A, the source device 120 may display part of the video video / data 121 on display 122 and may output some audio audio / video data 121 to dynamics 123. Audio / video data 121 may be stored locally on the source device 120 to which access is obtained from an external storage medium such as a file server, a hard disk, an external memory, a CD-ROM, a DVD, or another physical storage medium, or may be transmitted to a source device 120 via a network connection such as the Internet. In some cases, the audio / video data 121 can be captured in real time using the camera and microphone of the source device 120. Audio / video data 121 may include multimedia content such as movies, television shows or music, but may also include real-time content generated by the source device 120. Such real-time content, for example, may be made by applications working on a source 120 source or captured video data, for example, as part of a video telephony session. As will be described in more detail, such real-time content may be in some

3

iA 107151 C2

cases include a video frame of user-input options available to the user for selection. In some cases, audio / video data 121 may include video frames that are a combination of different types of content, such as a video movie frame or television program that has user-input options overlayed on a video frame.

In addition to playing audio / video data 121 locally using the display 122 and the speech 123, the source audio / video device 120 encoder 124 can encode the audio / video data 121, and the transmitter / receiver unit 126 can transmit the encoded data on the communication channel 150 to the terminal 160 the recipient The transmitter / receiver unit 166 of the receiver device 160 receives the coded data, and the audio / video decoder 164 decodes the encoded data and outputs the decoded data using the display 162 and the speaker 163. Thus, the audio and video data reproduced with the help of the display 122 and the speaker 123 can be simultaneously reproduced by the display 162 and the speaker 163. Audio data and video data can be arranged in frames, and audio cadres can be synchronized in time with video frames during playback.

An audio / video encoder 124 and an audio / video decoder 164 can implement any number of audio and video compression stanzas such as the 1Ti-T H.264 standard, alternatively called MER-4, Part 10, enhanced video encoding (AUC), or video encoding standard Performance (NO), which has recently appeared, sometimes called the H.265 standard. Also, many other types of ownership or standardized compression methods can be used. In general, the audio / video decoder 164 is configured to execute mutually reverse encoding operations of the audio / video encoder 124. Although not shown on the fiberglass. 1A, in some aspects, an A / B encoder 124 and an A / B decoder 164 can be combined with an audio encoder and may include appropriate blocks of the MIC-SEMIX or other hardware and software,

As will be described in more detail below, the A / B encoder 124 can also perform other functions of encoding additionally to the implementation of the video compression standard as described above. For example, an A / S encoder 124 can add different types of metadata to A / D data 121 to transmit A / D data 121 to a recipient device 160. In some cases, the A / D data 121 may be stored on or received in the source device 120 in the encoded form and thus do not require additional reduction using the A / V encoder 124.

Although FIG. 1A shows a communication channel 150 that transmits the useful audio data and useful data of the video separately, it should be understood that in some cases, useful video data and useful data of the audio may be part of the overall data stream. If applicable, the MiX-YEEM blocks may correspond to the protocol of the multiplexer IT No.223 or other protocols such as the user datagram protocol (UDP). The audio / video encoder 124 and the audio / video decoder 164 can be implemented as one or more microprocessors, digital signal processors (DDR processors), specialized integrated circuits (AZIS circuits), user programmable users, ELISA matrices, discrete logic, software, hardware provision , software, or any combination of them. Each of the audio / video encoder 124 and the audio / video decoder 164 may be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (codec). Thus, each of the recipient device 120 and the device 160 can provide specialized machines configured to perform one or more of the methods of disclosure.

The display 122 and the display 162 may include any plurality of video output devices such as an electron beam tube (CPT), a liquid crystal display (LCD), a plasma display, an LED display (LED), an organic LED display (OLE), or another display device . In these or other examples, displays 122 and 162 may be emitting displays or permeable displays. Display 122 and display 162 may also be input-touch displays, so that they are both input devices and display devices at the same time. Such touch-screen displays may be capacitive, resistive or other type of input panel, which allows the user to provide a user-friendly input to the device.

The speaker 123 may include any plurality of audio output devices such as headphones, a single speaker system, a speaker system, or a surround sound system. Additionally, although the display 122 and the speaker 123 are shown as part of the source device 120, and the display 162 and the speaker 163 are shown as part of the receiver device 160, the source device 120 and the receiver device 160 may in fact be a system of devices. As one example, the display 162 may be a television, the speaker 163 may be a surround system surround sound, and the decoder 164 may be part of an external unit connected or wired

4

iA 107151 C2

or wirelessly with a display 162 and a speaker 163. In other cases, the recipient device 160 may be the only device, such as a tablet or smartphone. In other cases, the source device 120 and the receiver device 160 are similar devices, for example, both are smartphones, tablet PCs, and the like. In this case, one device can work as a source, and the other can work as a recipient. These lists may even be reversed in subsequent communication sessions. In other cases, the device source may contain a mobile device such as a smartphone, laptop or tablet computer, and the receiver device may contain more stationary devices (for example, AC power connector),

The transmitter / receiver unit 126 and the transmitter / receiver unit 166 may include intercoms, filters, amplifiers and other components designed to modulate the signal, as well as one or more antennas and other components designed to transmit and receive data. The communication channel 150 generally represents any suitable communication medium or a collection of different communication media for transmitting video data from the source device 120 to the receiver terminal 160. The communication channel 150 is, of course, a communication channel with respect to minority, similar to NI-E, VIAI, and others. However, the communication channel 150 is not necessarily limited in this respect and may contain any wireless or wiredcommuni cation media such as radio frequency (PE) (RF) spectrum or one or more physical lines of transmission, or any combination of wireless and wired media. In other examples, the communication channel 150 may even be part of a packet network, such as a wired or wireless LAN, a large-scale network or a global network such as the Internet. Additional, the communication channel 150 may be used by the source device 120 and the receiver device 160 to create peer-to-peer communication line. The 120 source and receiver device 160 can communicate over the communication channel 150 using a transmission protocol, such as the standard of the IEEE 802.11 standard group. The 120 source and recipient device 160 may, for example, communicate in accordance with the E-mail and E-mail standard so that the source device 120 and the recipient device 160 communicate directly with each other without the use of the intermediary, such as wireless access points or the so-called hot spot. The source device 120 and the receiver device 160 may also set up a tuned direct line (TUZ) to avoid or reduce network overload. The means of this disclosure may from time to time be described with respect to NI-E, but it is considered that aspects of these methods may also be compatible with other data transfer protocols. By way of example and not limitation, the wireless communication between the source device 120 and the receiver device may employ methods for orthogonal multiplexing with frequency division multiplexing (ΟΕΜΜ). Also, a large variety of other ways of wireless communication can be used, including, but not limited to, multiple access with time division of channels (TIMA), multiple access with frequency division of the channels (ΕΜΜΑ), multiple access with code division of channels (ΟΜΜΑ) or any combination of ΕΕΜΜ, ΕΜΜΑ, ΤΜΜΑ and / or ΟΜΜΑ. Eu Eigens and TOES are designed to establish communication circuits at relatively short distances. Relatively short distance in this context maybe, for example, less than 70 meters, although in noisy or creating barrier in the environment, the distance between the devices may be shorter, for example, less than 35 meters.

In addition to decoding and reproducing data received from the source device 120, the recipient device 160 may also accept user inputs from the user input device 167. The user input device 167 may, for example, be a keyboard, mouse, trackball or touchpad, touch input screen, voice recognition module, or any other such user input device. The UE 168 formats user input commands received by the user input device 167 into a data packet structure that the source device 120 is capable of interpreting. Such data packets are transmitted by the transmitter / receiver 166 to the receiver 120 on the communication channel 150. The transmitter / receiver unit 126 receives packet data, and the A / ν control module 125 performs a parsing analysis of data packets, to interpret a user input command received by the user input device 167. Based on the command received in the data packet, the A / ν control module 125 can change the content encoded and transmitted. Thus, the user of the recipient device 160 can manage the useful audio data and useful video data transmitted by the source device 120 remotely and without direct interaction with the source device. Examples of the types of commands that the user of the recipient device 160 can transmit remotely and without direct interaction with the source device. Examples of the types of commands that the user of the recipient device 160 can transmit remotely and without direct interaction with the source device. Examples of the types of commands that the user of the recipient device 160 can transmit

5

iA 107151 C2

the source device 120 includes commands for rewinding, accelerating rewind, interrupting and playing audio and video data, as well as commands for scaling the image, rotation, scrolling, etc. Users can also make selections from the options menu, for example, and pass the selection back to the source device 120.

Additionally, users of the recipient device 160 may be able to run and manage applications on the source device 120. For example, the user of the recipient device 160 may be able to start the photo editing application stored on the source device 120 and use this application to edit a photo locally stored by the source device 120. The recipient device 160 may provide the user with a custom experience that looks and is perceived as a photo that is edited locally on the receiver device 160, while the actual photo is edited on the source device 120. Using such a configuration, the user of the device may be able to efficiently use the capabilities of one device for use with For example, a source device 120 may be a smartphone with a large amount of memory ' and the processing possibilities at a high level. The user of the source device 120 can use the smartphone in all settings and situations in which smartphones are commonly used. However, when watching a movie, a user may want to watch a device cinema with a large display screen; in this case, the recipient device 160 may be a tablet computer or even a larger display device or television. To send or reply to an e-mail message, the user may want to use a keyboard device, in this case, the recipient device 160 may be a laptop. In both cases, most of the processing may still be done by the device 120 source (a smartphone in this example), even though the user interacts with the device receiver. In this particular operational context, because of the large amount of processing that is achieved by the source device 120, the receiver device 160 may be a cheaper device with fewer resources than if the receiver 160 was asked to do the work done by the source device 120. Both the source device and the recipient device may be able to accept user input (for example, the input screen command input) in some examples, and the methods of this disclosure can facilitate a two-way interaction by matching and / or identifying the capabilities of devices in any given session.

In some configuration, the A / S control module 125 may be a process of an operating system executed by the operating system of the source device 125. [ However, in other configurations, the A / S management module 125 may be an application software process running on the source device 120. In such a configuration, the user input command can be interpreted by the software process so that the user of the recipient device 160 interacts directly with the application running on the source device 120, as opposed to the operating system operating on the source device 120. By directly interacting with the application, as opposed to the operating system, the user of the recipient device 160 can have access to the library of commands that are not "native" to the operating system of the source device 120. Additionally,

The source device 120 may respond to user inputs applied to the receiver's wireless device 160. In such an interactive application installation, the user inputs applied to the wireless device 160 of the recipient may be sent back to the wireless source of the display through the communication channel 150. In one example, the architecture of the return channel, also referred to as the return interface of the user interface (IIS), may be implemented to allow the receiver device 160 to transmit the user inputs applied to the receiver device 160 to the source device 120. The architecture of the return channel may include a top-level message for the transport of user inputs and a lower-level frame to reconcile the features of the user interface in the recipient device 160 and the source device 120. iiVs can be in the transport layer of the Internet Protocol (IP) between the recipient device 160 and the source device 120. Thus, iiVS can be higher than the transport level in the communication model of the interaction of open systems (OZI). In one example, the O5I connection includes seven levels (1 - physical, 2 data lines, 3 - network, 4 - transport 5 - session, 6 - presentation and 7 - applications). In this example, the location above the transport layer refers to levels 5, 6 and 7. To facilitate reliable transmission and consistent delivery of data packets containing user input data, iiVs can be configured to work on top of others. interaction between open systems (OII). In one example, the O5I connection includes seven levels (1 - physical, 2 data lines, 3 - network, 4 - transport 5 - session, 6 - presentation and 7 - applications). In this example, the location above the transport layer refers to levels 5, 6 and 7. To facilitate reliable transmission and consistent delivery of data packets containing user input data, iiVs can be configured to work on top of others. interaction between open systems (OII). In one example, the O5I connection includes seven levels (1 - physical, 2 data lines, 3 - network, 4 - transport 5 - session, 6 - presentation and 7 - applications). In this example, the location above the transport layer refers to levels 5, 6 and 7. To facilitate reliable transmission and consistent delivery of data packets containing user input data, iiVs can be configured to work on top of others.

6

iA 107151 C2

packet data protocols, such as transmission control protocol / Internet Protocol (TCP / IP) or custom datagram protocol (UDP). Win and TCP can work in parallel in the O8I architecture. The TCP / IP can allow the recipient device 160 and the source device 120 to implement retransmission methods in the event of loss of the packet.

In some cases, there may be a discrepancy between the user input interfaces located in the source device 120 and the receiver device 160. To solve the potential problems caused by such a discrepancy and to promote good user experience in such circumstances, the coordination of the features of the user input interface can take place between the device 120 sources and the receiver device 1b0 prior to the establishment of the communication session or the different times during the communication session. As part of this alignment process, the source device 120 and the recipient device 160 can agree on a consistent screen resolution. When the recipient device 160 transmits the coordinate data associated with the user input, the recipient device 160 can scale the coordinate data received from the display 162 to match the coordinated screen resolution. In one example, if the recipient device 160 has a resolution of 1280 x 720, and the source device 120 has a resolution of 1600 x 900, the devices can, for example, use a 1280 x 720 as their own coherent distinction. A coherent differentiation may be selected based on the distinction between the receiver device 160, although may also be used to distinguish the source device 120 or some other distinction. In an example using a receiver device with a resolution of 1280 x 720, the recipient device 160 can scale the resulting x coordinates by using the 1600/1280 ratio before transmitting the coordinates to the source device 120, and similarly, the recipient device 160 can scale the received y-coordinates by a factor of 900 / 720 before transmitting the coordinates to the source device 120. In other configurations, the source device 120 can scale the resulting coordinates to a coherent distinction. Zooming may either increase or decrease the range of coordinates based on whether the recipient device 160 uses a higher resolution display than the source device 120, or vice versa.

Additionally, in some cases, the distinction in the recipient device 160 may change during the communication session, potentially creating a discrepancy between the display 122 and the display 162. In order to improve the user experience and ensure proper functionality, the source / recipient system 100 may implement methods for reducing or preventing the heterogeneity of the user interaction. by implementing methods for normalizing the screen. The display 122 of the source device 120 and the display 162 of the recipient device 160 may have different distinctions and / or different aspect ratios. Additionally, in some configuration settings, the user of the recipient device 160 may have the ability to resize the screen display for video data taken from the source device 120 so that the video data received from the source device 120 is played back in the window, which covers less than the entire display 162 of the receiver device 160. In other exemplary settings, the user of the recipient device 160 may have a content view option either in the horizontal orientation mode or in the portrait mode, each of which has unique coordinates and different aspect ratios. In such situations, the coordinates associated with the user input received by the receiver 160, such as the coordinate of the location of the mouse click or the touch event, may not be able to be processed by the source device 120 without changing the coordinates. Accordingly, the methods of this disclosure may include displaying the user input coordinate received in the recipient device 160 in coordinates associated with the source device 120. This reflection is also called normalization in the given description, and,

Custom inputs received by the recipient device 160 may be adopted by the module 167 and at the driver level, for example, and sent to the operating system of the recipient device 160. The operating system in the recipient device 160 may receive the coordinates (hzimk.Uzimk) associated with where the surface of the display has a user input. In this example (hdd, k, k) can be the coordinates of the display 162, in which there was a click on the mouse or an event of the touch. The screen window of the display reproduced on display 162 may have the length of the x coordinates of Whats) and the width of the y coordinates (OT_0ТО) that describe the size of the screen display screen. The screen window of the display may also have the coordinate of the upper left corner (a_oot, B ^), which describes the location of the screen of the display screen. Based on b_oot, OT_0ТО and the upper left coordinate (a_oot, B_oh), a portion of the display 162 covered by the screen of the display can be defined. For example, the right upper corner of the screen window can be located in the coordinate (as_ATfrom + b_ATfrom b_oot), the lower left corner of the screen of the display screen can be located in the coordinate (a_oot, B_0ТО+ OT_0ТО), and the right bottom corner of the screen of the display may be located in

7

iA 107151 C2

coordinate (a_0U+ B_0U, B_uh+ U_uh) The recipient device 160 can process the input as input and / or AV, if this input is taken in coordinate within the display window. In other words, the introduction with associated coordinates (Χδ_ΝΝΚ, Υθινκ) can be processed as the introduction of iiVS if the following conditions are met:

3 "-CH2-NO-3 MY + THYM

LouboutinWhite-Luo + Wooou

After determining that the user input is the input of iiVs, the coordinates associated with this input can be normalized using iRiM 168 prior to transmission to the source device 120. Inputs, which are defined as outside of the screen of the display screen, can be processed locally by the recipient device 160 as a non-I / V input.

As mentioned above, the normalization of the input coordinates may be based either on a source or based on the recipient. In the implementation of a normalized device based on the receiver, the source 120 can send the display distinction (b_3КС, Y₃^_WITH) that is supported for display 122, or with video data, or regardless of video data, to the recipient device 160. The distinction of a supported display, for example, may be transmitted as part of a session of matching capabilities, or may be transmitted at another time during a communication session. The recipient device 160 can determine the resolution of the display φ_3YNC, Y₃Yes) for the display 162, the distinction from the screen display screen (b_uh, Woo_IN) for a window that displays the content received from the source device 120 and the coordinate of the left upper corner (and_uh, B_uh) for the screen of the display. As a descriptive note, when the coordinate (χ_3YNC, Y3_ΝΝΚ) which corresponds to the data input by the user, is determined within the limits of the screen of the display screen, the operating system of the recipient device 160 may reflect the coordinate (χ_3YNC, at_3I^ in coordinates (x₃p_WITH, at_33C) source using the conversion function. Exemplary transformation functions for transformation (χ_3YNC, at₃^^ in (X₃p_WITH, at₃to_WITH) can be as follows:

^X3КС^{= (x}3 ^ K^-andOW⁾* Ф3ЯсФоУ⁾

y3RS⁼(y3Zh-ou7) * (OT3rs / OTu'7)

Thus, when transmitting a coordinate corresponding to a user input passed by the user, the recipient device 160 can transmit the coordinate (x_3КС, at_33C) for the user input taken in (χ_3YNC, at_3ІЖ) As will be described in more detail below, the coordinate (x_33C, at₃^_WITH), for example, may be transmitted as part of a data packet that is used to transmit a user input received by the recipient device 160 to the source device 120 of the I / V. Throughout the other parts of this disclosure, in which the input coordinates are described as included in the data packet, these coordinates can be converted to the output coordinates as described above in cases where the source / recipient system 100 implements the recipient-based normalization.

When the source / receiver system 100 implements a source-based normalization, for the user inputs defined by the inputs of the IIS, in contrast to the local input (that is, within the display screen window as opposed to the outer side of the display screen window), the calculations described above may be performed in device 120 sources instead of recipient device 160. To facilitate such calculations, the recipient device 160 can transmit to a value of 120 source values ​​for b_uh, Y_uh and location information for the screen display (for example, a_uh, B_uh), and also the coordinates for (Χδ_ΝΝΚ, at_3Ig) Using these transmitted values, the source 120 can determine the value for (x_3КС, at_33C) according to the equations 3 and 4 presented above.

In other implementations based on the recipient of normalization, the device 160 of the recipient may transfer coordinates (x_uh, at_uh) for user input that describes where there is a custom input event within the window screen of the display, as opposed to where there is a user input on display162. In this implementation, the coordinates (x_uh, at_uh) may be transmitted to the source device 120 along with the values ​​for (b_uh, Y_uh) Based on these accepted meanings, the source 120 can determine (x_3КС, at_33C) according to the following conversion functions:

H3RS⁼Hou * Sh3Khou)

y33s⁼wow * (w3x / wow)

The recipient device 160 can define x_uh and y_0U based on the following functions:

^X0U^{= X}3ІЖ^-and0U

UU⁼U3UZH^-Bouw

When this disclosure describes the transmission of coordinates associated with the user input, in a data packet, for example, the transmission of these coordinates may include a payee based or source-based normalization as described above and / or may include

8

iA 107151 C2

itself any additional information necessary for the implementation based on the recipient orbased on the source of normalization.

ІІВС can be designed for the transport of various types of user-generated data, including cross-platform data of user input. For example, the source device 120 may operate the ίδδ® operating system, while the receiver 160 is controlled by another operating system such as Apbogio® or OTipBase®. Regardless of the platform, the ipI 168 can encapsulate the received custom input in a form that is understandable to the control unit A / ν. Several different types of custom formats supported via vvedennyamozhut yiVS so as to allow many different typamprystroyiv source and destination protocol used regardless of whether pratsyuyutprystroyi source and destination on different platforms. Formatter input can be defined, and the format-specific input formats for the platform can be supported,

In the example of FIG. 1A, the source device 120 may comprise a smartphone, tablet PC, laptop, desktop computer, a RT-RI TV or any other device capable of transmitting audio and video data. The recipient device 160 can similarly contain a smartphone, a tablet PC, a laptop, a desktop computer, a TV with the support of RT-RIAO, or any other device capable of receiving audio and video data and accept data from the user's input. In some cases, the receiver device 160 may include in the system devices such as the display 162, the speaker 163, the device 167 and the encoder 164 A / ν of all parts of the individual but interacting devices. Device 120 source can similarly be a system of devices, and not the only device.

In this disclosure, the term "source device" generally is used to vidnosytysyado device that transmits audio / video data, and the term "receiving device" in tsilomuvykorystovuyetsya to refer to a device that accepts audio / video data from prystroyudzherela. In many cases, the source device 120 and the receiver device 160 may be analogous or identical devices, with one device operating as a source, and the other operating as the receiver. In addition, these lists can be changed to the inverse in different communication sessions. Thus, the recipient's device in one communication session may become the device of the source in the subsequent communication session, or vice versa.

FIG. 1B is a block diagram illustrating a sample source / receiver system 101 that can realize the methods of this disclosure. The source / receiver system 101 includes a source device 120 and a receiver device 160, each of which may operate and work as described above for FIG. 1A. The source / recipient complement system 101 includes a recipient device 180. Similar to the recipient device 160, as described above, the receiver device 180 can receive audio and video data from the source device 120 and transmit the user commands to the source device 120 via the ipI set. In some configurations, the recipient device 160 and the recipient device 180 may operate independently, and the output of the audio and video data in the source device 120 may be simultaneously output to the receiver 160 and the receiver device 180. In additional configurations, the recipient device 160 may be the primary receiver device and the receiver device may be a secondary recipient device. In such an exemplary configuration, the recipient device 160 and the receiver device 180 can be connected and the receiverthe third recipient may display the video data, while the recipient device 180 outputs the corresponding audio data. Additionally, in some configurations, the recipient device 160 may output the transmitted video data only when the recipient device 180 outputs the transmitted audio data.

FIG. 2 is a block diagram illustrating one example of a source device 220. The source 220 may be a device similar to the source device 120 of FIG. 1A, and can work in the same way as the source device 120. The source device 220 includes a local display 222, a loudspeaker 223, processors 231, memory 232, a transport unit 233, and a wireless modem 234. As shown in FIG. 2, the source device 220 may include one or more processors (i.e. processor 231) that encode and / or decode A / ν data for transport, storage, and display. A / ν data can, for example, be stored in memory232. The memory 232 can store the entire A / ν file or may contain a smaller buffer that simply stores part of the A / ν file, for example, transmitted by the stream from another device or source. The transport unit 233 can process the coded A / ν data for the network transport. For example, the encoded A / ν data can be processed by processor 231 and encapsulated by transport unit 233 into network access layer blocks (NABs) for transmission

9

iA 107151 C2

through the network. NAI blocks can be sent by wireless modem 234 to the wireless device of the receiver using a network connection. A wireless modem 234 may, for example, be a Siemens modem configured to implement one of the IEEE802.11 standards group.

The source 220 can also process locally and display A / ν data. In particular, the display processor 235 can process the video data that should be displayed on the local display 222, the audio processor 236 can process the audio data for outputting to the speaker 223.

As described above with references to the source θ2θ device in FIG. 1A, the source device 220 may also receive user input commands from the receiver device. Thus, the wireless modem 234 of the source device 220 receives packets of encapsulated data, such as the blocks NAB, and sends the blocks of encapsulated data to the encoded transport block 233. For example, the transport unit 233 may pull data packets from the NIB blocks and the processor 231 can perform parsing data packets to extract user input commands. On the basis of custom input commands, the processor 231 can configure the encoded A / ν data transmitted by the source device 220 to the device of the receiver. Thus, the functionality described above with respect to the control unit A / ν in FIG. 1 A

Processor 231 according to FIG. 2 as a whole represents any large variety of processors, including, but not limited to, one or more digital signal processors (processors δδ), general-purpose microprocessors, specialized integrated circuits (ABC schemes), user-programmed gate matrices (matrix PROA), another an equivalent integrated or discrete logic circuit, or some combination of them. Memory232 according to FIG. 2 may include any large variety of energy-independent or non-energy-dependent memory, including, but not limited to, an operational memory device (RAM) such as a synchronous dynamic operational memory (RAM), a permanent storage device (ROM ), non-volatile operative storage device (K ^ RAM), Electrically erased permanent memory (EERRO), flash memory, and so on. Memory 232 may comprise a computer-readable storage medium for storing audio / video data, as well as other types of data. Memory 232 can further store the commands and program code that are executed by the processor 231 as part of the execution of the various methods described in this disclosure.

FIG. 3 shows an example of a recipient device 360. The recipient device 360 ​​may be a device similar to the receiver device 160 in FIG. 1A, and may work in the same way as receiver device 160. The recipient device 360 ​​includes one or more processors (i.e. processor 331), memory 332, transport unit 333, wireless modem 334, display processor335, local display 362, audio processor 336, speaker 363, and user interface interface 376. The recipient device 360 ​​receives 334 blocks of encapsulated data sent from the source device in a wireless modem. The wireless modem 334, for example, may be a SHI-Ei modem configured to implement another standard from the IEEE 802.11 Group Bandwidth. The transport unit 333 can decapitate blocks of encapsulated data. Example, the transport unit 333 can extract the encoded video data from the blocks of the encapsulated data and send the encoded A / ν data to the processor 331, which should be decoded and reproduced for output. The display processor 335 can process the decoded video data that must be displayed on the local display 362, and the audio processor 336 may process the decoded audio data for outputting to the dynamics 363.

In addition to playing audio and video data, the recipient's wireless device 360 ​​may also receive user input data via the user input interface 376. The user input interface 376 may represent any of a number of user input devices including, but not limited to, the interface of the touch display, the keyboard, the mouse , a module for voice commands, a gesture capture device (for example, based on the capture capability of data input) or any other of a number of device users ack input. The custom input received through the user input interface 376 may be processed by the processor 331. This processing may include generating data packets that include a custom output command, according to the methods described in this disclosure.

Processor 331 according to FIG. 3 may contain one or more wide range of processors, such as one or more digital signal processors (DDR processors), microprocessor general purpose, specialized integrated circuits (ABC schemes), programmable

10

iA 107151 C2

the user's gate matrix (matrix PPCA), other equivalent integrated or discrete logic circuits or some combination of them. Memory 332 according to FIG. 3 may include any large variety of energy-dependent or non-volatile memory, including, but not limited to, operational memory (RAM) such as a synchronous dynamic storage memory device (RAM), a permanent memory device (ROM ), non-volatile operating memory (ΝνΡΑΜ), electrically erased programmable permanent memory (EERRO), flash memory, etc. The memory 232 may include a computer-readable storage medium for storing audio / video data as well as other types of data. Memory 332 can additionally save commands and program code,

FIG. 4 shows a block diagram of an exemplary transmitter system 410 and receiver system 450 that can be used by the transmitter / receiver 126 and the transmitter / receiver 166 of FIG. 1 for communication on the communication channel 150. In transmitter system 410, traffic data for a number of data streams is output from a data source 412 to a data transfer processor 414 (TX). Each data stream can be transmitted on the appropriate transmission antenna. The TX data processing processor 414 TX formats, encodes and routines traffic data for each data stream based on a specific schema encoding selected for this data stream.

Encoded data for each data stream can be multiplexed with pilot data using methods of multiplexing with orthogonal frequency division of the mnals (OR0M). A wide variety of other wireless methods may also be used, including, but not limited to, multiple channel time access (TOMA), multiple frequency channel access (ROMA), multiple channel code access (COMA), or any combination of OROM, ROMA, TOMA and / or SOMA.

According to FIG. 4, pilot data is usually a known data pattern, which is processed in a known way and can be used in the receiver system to assess the response of the channel. Multiplexed pilot data and encoded data for each stream are then modulated (for example, displayed in a symbol) based on a specific modulation scheme (eg binary phase manipulation (BP5K), quadrature phase manipulation (OP5K), M-R5K or M-OAM (quadrature amplitude modulation), where M may be a degree of duplication) selected for this data stream to give modulation symbols. The data transfer rate, coding, and modulation for each data stream can be determined by the commands executed by the processor 430 which can be connected to memory 432.

Then, the modulation symbols for the data streams are output to the MIMO TX processor 420, which can further process the modulation symbols (for example, for OROM). Then, the processor 420 MIMO TX data transmission may produce N_τ symbolic flows of modulation in N_τtransmitters (TMTP) 422a-4221. In some aspects, the MIMO 420 processor transmits data from weighting factors for generating a forward-point chart to the data stream symbols and the antenna from which the symbol is transmitted.

Each transmitter 422 may receive and process an appropriate symbol stream to output one or more analog signals and additionally cause analog signals to be made (for example, amplify, filter, and convert with increasing frequency) analog signals to produce a modulated signal suitable for transmission over the MIMO channel. Then N._τ Modulated signals from transmitters 422a-4221 are transmitted from N_τ antennas 424a-4241, respectively.

Modulated signals transmitted to the receiver system 450 are received by N_κ antennas 452a-452g, and the received signal from each antenna 452 is output to the corresponding receiver (P ^ P) 454a-454g. Receiver 454 leads to the necessary conditions (for example, it filters, amplifies and converts with a decrease in frequency), the corresponding received signal converts it to the required conditioned in a digital form to provide sampling, and further processes the sampling for issuing an appropriate "adopted" character stream.

Then the 460 RX processor receives and processes the data reception N_κ Accepted symbolic streams from N_κ receivers 454 based on a specific method of processing the receiver for issuing N_τ"detected" symbol streams.After this, the processor 460 PC receiving data demodulates, performs turning and decoding each detected symbol stream to restore data traffic for the data stream.processing the processor 460 PC reception of data may becomplementary to that performed by the processor 420 MIMO TX data transmission andprocessor 414 TX data transmission in the transmitter system 410.

11

iA 107151 C2

The processor 470, which can be connected to memory 472, periodically determines which use the matrix of the previous coding. Reverse link messages can provide various types of information relative to the communication link and / or received data stream. Then the feedback link is processed by the TX data processor 438, which also receives traffic data for a number of data streams from the data source 436, modulated by the modulator 480, is reduced to the required conditions by transmitters 454a-454g and transmitted back to the transmitter system 410.

In the transmitter system 410, the modulated signals from the receiver system 450 are received by antennas 424, are brought to the required conditions by receivers 422, demodulated by the modulator 440, and processed by the PC receive processor 442 to receive the feedback received by the receiver system 450. Processor 430 then determines what to use a matrix of pre-coding to determine the weight coefficients of the direction diagram, then handles the elongated message.

FIG. 5A is a block diagram illustrating an exemplary sequence of messages transmitted between the source device 520 and the receiver device 560 as part of the negotiation session of the capabilities. Matching capabilities may take place as part of a larger process of establishing a communication session between the source device 520 and the receiver device 560. This session, for example, can be established using OTi-Rig Beigesi or TBI8 as the standard connection underlying. After the OTiRi Beiges or TBY8 session is established, the recipient device 560 can initiate a TCP connection with the source device 520. As part of establishing a TCP connection, a control port implementing a live streaming protocol (PTTP) can be installed to control a communication session between the source device 520 and the receiver device 560.

The source 520 may generally work in the same way as described above for the source device 120 in FIG. 1A, and the receiver device 560 may generally work in the same manner as described above for the receiver device 160 in FIG. 1A. Once the source device 520 and the receiver device 560 establish connectivity, the source device 520 and the receiver device 560 can define a set of parameters that should be used for their subsequent communication session as part of the exchange of matching capabilities.

The 520 source and the receiver device 560 can match the possibilities through the sequence of messages. Messages, for example, may be messages in the real-time streaming protocol (PTTP). At any stage of the agreement, the recipient of the request for a request PTPP can respond using the response PT8R, which includes a status code PTPR, except PT8R OK, in this case, the message exchange can be repeat with another set of parameters, or may be completed session of the negotiation opportunities.

The source device 520 may send a first message (a message requesting options for the PT8R (PT8R ORT1OY8)) to the recipient device 560 to determine a set of methods of PT8R that supports the receiver device 560. Upon receiving the first message from the device 520, the receiver device 560 can respond to a second message (a PTR message message ORT1OY8) that lists the PTRs supported by the recipient device 560. The second message may also include the status code of PT8P OK.

After sending the second message to the source device 520, the receiver device 560 can send a third message (request message PT8R ORT1OY8) to determine the selection of PTRs that support the source device 520. After receiving a third message from the receiver device 560, the source device 520 may respond to a fourth message (a response message PT8R ORT1OY8) that lists the PT8R methods supported by the source device 520. The fourth message may also include the status code of the TRPROC.

After the fourth message link, the source device 520 may send a fifth message (a message requesting the SET_RAPAMETPPTPP request) to determine the list of options that are of interest to the source device 520. The recipient device 560 can respond to the sixth message (message of the answer SET_RAPAMETEP PTPP). The latter message may contain the status code of the PTZP. If the PTQP status code is OK, then the sixth message may also include the response parameters for the parameter determined by the fifth message supported by the recipient device 560. The recipient device 560 may ignore the parameters in the fifth message, which does not support the receiver device 560.

Based on the sixth message, the source 520 may determine an optimal set of parameters to be used for the communication session and may send the seventh message (message of the 8ET_RAPAMETPPTPP request) to the receiver device 560. Seventh

12

iA 107151 C2

the message may include a set of parameters to be used during a session communication between the source device 520 and the receiver device 560. The seventh message may include a myrib-rgezepiaioiop-igi that describes the universal resource identifier (IRI) that should be used in the request for setting the PT5R to establish a communication session. Mb-rgezepiaiiiop-igi defines the IRI that can use the device 560 recipient for later messages when exchanging session setup. The values ​​of the myBIU and myBir11, defined in this parameter, may correspond to the value of the Hierarchy and the significance of the mountain rhythm in the Meyb-Sieep-Hirsch-Rogis in the seventh message. In this case, the RTR refers to real-time protocols that can work on top of the device.

After receiving the seventh message, the recipient device 560 can respond to eighth message with the status code RT5P, indicating whether the setting of the parameters was successful, as defined in the seventh message. As mentioned above, the roles or device of the source and device of the receiver may be reversed or varied in different sessions. The order of messages that establish a communication session, in some cases, can determine the device that works as a source, and determine the device that acts as the receiver.

FIG. 5B is a block diagram illustrating another exemplary sequence of message transmission between the source device 560 and the receiver device 520 as part of the session of matching capabilities. The sequence of message transmission in FIG. 5B is intended to provide a more detailed view of the sequence of transmission described above for FIG. 5A. In FIG. 5 The message "1B .NET RARAMETER" shows an example message that identifies the list of supported categories of input categories (for example, generic and NUUs) for a list of supported input types. Each of the input categories supported, from the supported input category list, has an associated list of types (for example, deleģis_saar_1i5i and iibs_saar_1i5i) that are supported. In FIG. 5In the message "2a.5ET_RAMAMETER REOiEZT" is an example of a second message, which identifies the second category of input categories (for example, generic and NUS), and a set of other subscripts of supported types. Each of the supported input categories from the second subset of supported input categories has an associated second list of types (e.g., Delegation_ServerIi5I and Ibs_Sar_1i5i) supported. The message "1b.SET_RAMAMETER ΡΕδΡΟNδΕ" identifies the input categories and input types supported by the recipient device 560. The message "2a. 5ET_RAMARETER REOiEZT" identifies the input categories and input types supported by the source device 520, but may not be a complete list of all input categories and input types supported by the source device 520. Instead, the message "2a. 5ET_RAMAMETER REOiEZT" can only identify the categories of input and the types of input identified in the message "1b.SET_RAMAMETER PPEδΡΟNδΕ" as those supported by the recipient device 560. So, the input categories and types of input identified in the message "2a.5ET_RAMAMETER REOiEZT" can be a subset of input categories and input types identified in the message "1b. SET_RARAMETER ΡΕδΡΟNδΕ".

FIG. 6 is a conceptual diagram illustrating one example of a data packet that can be generated by the receiver device and transmitted to the source device. Aspects of the package 600 will be explained with reference to FIGS. 1A, but the methods discussed may apply to additional types of source / recipient systems. The data packet 600 may include a data packet header 610 with further useful data 650. Useful data 650 may further include one or more useful data headers (e.g., useful data header 630). The data packet 600 may, for example, be transmitted from the recipient device 160 of FIG. 1 to the source device 120 so that the recipient device 160 can control the audio / video data transmitted by the source device 120. In this case, useful data 650 may include user input data, received at receiver device 160. Useful data 650 can, for example, identify one or more custom commands. The recipient device 160 may receive one or more user commands and may, based on received instructions, generate a packet header 610 and useful data 650. Based on the contents of the data packet header 610 of the data packet 600, the source device 120 can perform a parsing analysis of the useful data 650 to identify the user input data taken in recipient device 160. Based on the user inputs contained in the useful data 650, the source device 120 may somehow alter the audio and video data transmitted from the source device 120 to the receiver device 160. The recipient device 160 may receive one or more user commands and may, based on received instructions, generate a packet header 610 and useful data 650. Based on the contents of the data packet header 610 of the data packet 600, the source device 120 can perform a parsing analysis of the useful data 650 to identify the user input data taken in recipient device 160. Based on the user inputs contained in the useful data 650, the source device 120 may somehow alter the audio and video data transmitted from the source device 120 to the receiver device 160. The recipient device 160 may receive one or more user commands and may, based on received instructions, generate a packet header 610 and useful data 650. Based on the contents of the data packet header 610 of the data packet 600, the source device 120 can perform a parsing analysis of the useful data 650 to identify the user input data taken in recipient device 160. Based on the user inputs contained in the useful data 650, the source device 120 may somehow alter the audio and video data transmitted from the source device 120 to the receiver device 160. Based on the contents of the data packet header 610 of the data packet 600, the source device 120 can perform a parsing analysis of the useful data 650 to identify the user input data received in the recipient device 160. Based on the user inputs contained in the useful data 650, the source device 120 may somehow alter the audio and video data transmitted from the source device 120 to the receiver device 160. Based on the contents of the data packet header 610 of the data packet 600, the source device 120 can perform a parsing analysis of the useful data 650 to identify the user input data received in the recipient device 160. Based on the user inputs contained in the useful data 650, the source device 120 may somehow alter the audio and video data transmitted from the source device 120 to the receiver device 160.

The terms "perform parsing analysis" and "execution of parsing analysis", which are used in this disclosure, generally relate to the bitstream analysis process,

13

iA 107151 C2

to extract data from a bit stream. After extraction, the data can be processed by the source device 120, for example. Extracting data can, for example, include identification as formatted information in a bitstream. As will be described in more detail below, the data packer header 610 may define a standardized format known as the device 120 and the receiver device 160. Useful data 650, however, may have been formatted in one of many possible ways. By performing a syntax analysis of the data packet header 610, the source device 120 can determine how the formatted useful data 650, and thus the source device 120 can perform a parsing analysis of useful data to extract one or more custom instruction commands from the useful data 650. This can provide flexibility with respect to the different types of useful data that may be maintained in the context of the recipient source. As will be described in more detail below, usefuldata 650 may also include one or more useful data headers, such as a header 630 of useful data. In such cases, the source device 120 can perform a parsing analysis of the data packet header 610 to determine the format for the useful data header 630 and then perform a parsing analysis of the useful data header 630 to determine the format for the remainder of the useful data 650.

Diagram 620 is a conceptual description of how a data packet header 610 can be formatted. Numbers 0-15 in row 615 are intended to identify the bits location of the header 610 of the data packet and are not intended to actually represent the information contained in the data packet header 610. The data packet header 610 includes a field 621version, a timestamp flag 622, a reserved field 623, an input category 624, a field

625 length and optional field 626 timestamp.

In the example of FIG. The 6th field 621 version is a 3-bit field that can specify a version of a specific data protocol implemented by the recipient device 160. The value in the field 621 of the version can inform the source device 120 of how to perform a parsing analysis of the remainder of the data packet header 610, as well as about how to perform a parsing analysis of useful data 650. In the example of FIG. 6 field 621 version is a 3-bit field that allows a unique identifier for eight different versions. In other examples, the larger or less number of bits can be allocated to the field 621 version.

In the example of FIG. 6 flag (T) 622 timestamp is a 1-bit field indicating whether there is a field

626 timestamp in the data packet header 610. The time marker field 626 is a 16-bit field containing a timestamp based on the multimedia data generated by the source device 120 and transmitted to the receiver device 160. The timestamp may, for example, be a serial number assigned to video frames using a source device 120 before the frames are transmitted to the recipient device 160. Flag 622 of the timestamp may, for example, include "1" to indicate that there is a time marker field 626, and may include "0" to indicate that there is no time period field 626. After performing a parsimon analysis of the data packet header 610 and determining that the time marker field 626 is being used, the source device 120 may process the timestamp included in the time marker field 626.

If available, the time marker field 626 may include a timestamp for identifying the video data frame that was displayed on the wireless device 160 of the recipient when the user input data of the useful data 650 was received. The timestamp, for example, may be added to the video frame using the device 120 the source before the source device 120 will transmit the video frame to the receiver device 160. FIG. Accordingly, the source device 120 can generate a video frame and include in the video data of the frame, such as metadata, for example, timestamp. The 120 source can transmit a video frame with a timestamp to the recipient device 160, and the receiver unit 160 may display a video frame. While the frame of the video is displayed by the recipient device 160, the recipient device 160 may receive a user command from the user.

Upon receipt of the data packet 600 with the time marker field 626, which is in the header, the source wireless source 120 can identify the video frame displayed in the device 160 of the recipient while the user input data of the useful data 650 was received and to process the user input data based on frame content

14

iA 107151 C2

identified by a timestamp. For example, if the user input data is a touch command applied to the touch screen display or mouse pointer, the source device 120 can determine the frame content displayed while the user applied the touch command to the display or clicked the mouse. In some cases, the frame content may be necessary to properly process useful data. For example, custom input based on user touches or mouse clicks may depend on what was shown on the display during touch or tapping. Touch or push can, for example, match a character or menu option. In cases where the content of the display is changed, the timestamp, which is in the time field 626, may be used by the device 120 of the source,

The device 120 sources can additionally or alternatively compare the timestamp in field 626 of the time marker with the timestamp applied to the frame of the video that is played at this time. By comparing the timestamp of the time field 626 with the current time mark, the source device 120 can determine the time for the "there and the back" signal to pass. The transit time of the "back and forth" signal in general corresponds to the amount of time, the time expiring from the moment when the frame is transmitted by the source device 120, to the moment when the user input based on this frame is taken back to the source device 120 from the receiver device 160. The passage of the "back and forth" signal can provide the source device 120 indicating the system timeout, and if the passage of the "there and the back" signal is greater than the threshold value, then the source device 120 may ignore the user input data contained in the useful data 650, according to the assumption that the input command was applied to the obsolete display frame. When the passage of the "back and down" signal is less than a threshold, the source device 120 can process the data of the user input and adapt the audio / video content transmitted in response to the user input provided. Thresholds can be programmable, and different types of devices (or different combinations of the receiver source) can be configured to determine the various thresholds during the passage of the "back and forth" signal that is acceptable. that the input command was applied to the obsolete display frame. When the passage of the "back and down" signal is less than a threshold, the source device 120 can process the data of the user input and adapt the audio / video content transmitted in response to the user input provided. Thresholds can be programmable, and different types of devices (or different combinations of the receiver source) can be configured to determine the various thresholds during the passage of the "back and forth" signal that is acceptable. that the input command was applied to the obsolete display frame. When the passage of the "back and down" signal is less than a threshold, the source device 120 can process the data of the user input and adapt the audio / video content transmitted in response to the user input provided. Thresholds can be programmable, and different types of devices (or different combinations of the receiver source) can be configured to determine the various thresholds during the passage of the "back and forth" signal that is acceptable.

In the example of FIG. 6, the reserved field 623 is an 8-bit field that does not include the information used by source 120 when performing syntactic analysis of the data packet header 610 and useful data 650. Future versions of a specific protocol (as identified in the field 621 version), however, may use reserved field 623 when the source device 120 can use the information in the reserved field 623 to perform parsing analysis of the data packet header 610 and / or to perform a parsing analysis of the useful data 650. Reserve andnot field 623 with 621 field versiyipotentsiyno provides opportunities to enhance and add features to the format of packet danyhbez significant change in the format and signs that are used.

In the example of FIG. 6, the input category 624 is a 4-bit field for identifying the input category for the user input data contained in the useful data 650. The recipient device 160 may divide the user input data into categories to define the input category. The categorization of user input data may, for example, be based on the device from which the command was adopted or based on the properties of the team itself. The value category 624 of the input category, possibly together with the other information of the data packet header 610, identifies the source device 120 as formatted useful data 650. Based on this formatting, the source device 120 may perform a syntax analysis of the useful data 650 to determine the custom input that was received in the device 160 the recipient

Since the introduction category 624 in the example of FIG. 6 is 4 bits, sixteen different types of input may probably be identified. One such input category may be a generic input format to indicate that the user input data of useful data 650 is formatted using generic information elements defined in the protocol performed both by the source device 120 and the recipient device 160. The formatted input, which will be described in more detail below, may use generic information elements that allow the user of the recipient device 160 to interact with an application-level source device 120.

Another such input category can be the format of the interface unit command with the person (NIUS) to indicate that the user input data of the useful data 650 formatted based on the type of input device used to receive data input. Examples of device types include a keyboard, mouse, touch device input, joystick, camera, gesture capture device (such as camera-based device

15

iA 107151 C2

input) and the remote control tool. Other types of input categories that may be identified in the input category field 624 include an input format direction to indicate that the user data in the useful data 650 did not begin at the recipient device 160, or the operating system specific format and format of the voice command to indicate that Useful data 650 include a voice command.

A field 625 of length can contain a 16-bit field to indicate the length of the data packet 600. Length, for example, may be specified in blocks of 8 bits. Since the syntactic analysis of the data packet 600 via the source device 120 in the words of 16 bits is performed, the data packet 600 can be filled to an integer of 16 bits. Based on the length contained in the field 625, the source 120 can identify the end of the useful data 650 (that is, the end of the 600 data packet) and the beginning of a new further packet of data.

The different sizes of the fields provided in the example of FIG. 6 are simply intended to be explanatory and are intended to allow fields to be implemented using different bits than the number shown in FIG. 6. Additionally, it is also considered that the data packet header 610 may include less than all the fields discussed above, or may use additional fields not considered above. Indeed, the methods of this disclosure maybe flexible in relation to the actual format used for different fields of data packets.

After performing a parsing analysis of the data packet header 610 to determine the formatting of the useful data 650, the source device 120 can perform a parsing analysis of the useful data 650 to define a custom input command contained in useful data 650. Useful data 650 may have its own useful data header (header 630 useful data) indicating the content of the useful data 650. Thus, the source device 120 can perform a parsing analysis of the useful data header 630 based on the syntactic analysis of the header 610 data packet, and then perform a syntactic analysis of the remaining data useful 650 based on the syntactic analysis of the useful data header 630.

For example, if the input category 624 of the data packet header 610 indicates that the input is useful in data 650, then useful data 650 may have a generic output format. The device 120 sources can thus perform a parsing analysis of useful data 650 according to the generic input format. As part of the generic input format, useful data 650 may include a sequence of one or more input events with each input event having its own title of the input event. Table 1, below, identifies the fields that can be included in the title of the input.

Table 1

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Value</p></td></tr><tr><td><p>Ye genitive IE</p></td><td><p>1</p></td><td><p>See Table 2</p></td></tr><tr><td><p>Long</p></td><td><p>2</p></td><td><p>The length of the following fields in octets</p></td></tr><tr><td><p>Description</p></td><td><p>variable</p></td><td><p>Details of user inputs. See Tables</p></td></tr>

Identification field (Y) generic events (IE) identifies the event of the birth input to identify the type of input. The generic IE field may, for example, be one octet in length and may include the identification selected from Table 2 below. If, as in this example, the field of the generic IE is 8 bits, then 256different types of entries (identified by 0-255) can be identified, although not all 256 identities necessarily require an associated type of input. Some of the 256 may be reserved for future use with future versions regardless of the protocol implemented by the recipient device 160 and the source device 120. In Table 2, for example, identifications of the 9-255 generic IE do not have associated typing types, but they may be intended to be introduced in the future.

The length field in the entry event header identifies the length of the description field, while the field description includes information elements that describe the user input. The formatting of the description field may depend on the type of input identified in the Yurodovo IE field. Thus, the source device 120 can perform a syntactic analysis of the field of the text contained on the basis of the type of input identified in the field of generic IE. Based on the field of the input header length field, the source device 120 can determine the end of one event of input in useful data 650 and the beginning of a new input event. As will be explained more

16

iA 107151 C2

In detail below, one custom command can be described in useful data 650 as one or more input events.

Table 2 provides an example of type of administration with a generic U genetic element that can be used to identify the type of administration.

5

Table 2

<tr><td><p>u</p><p>generic</p><p>IE</p></td><td><p>Type of input</p></td></tr><tr><td><p>0</p></td><td><p>Left mouse button down / touch down</p></td></tr><tr><td><p>1</p></td><td><p>Left mouse button up / upside down</p></td></tr><tr><td><p>2</p></td><td><p>Mouse movement / touch movement</p></td></tr><tr><td><p>3</p></td><td><p>Down key</p></td></tr><tr><td><p>4</p></td><td><p>Down key</p></td></tr><tr><td><p>5</p></td><td><p>Changing the scale</p></td></tr><tr><td><p>6</p></td><td><p>Vertical scrolling</p></td></tr><tr><td><p>7</p></td><td><p>Horizontal scrolling</p></td></tr><tr><td><p>8</p></td><td><p>Rotation</p></td></tr><tr><td><p>9-255</p></td><td><p>Reserved</p></td></tr>

Description fields associated with each input type may have a different format. Description Fields "Left Down / Down Button" events, "Left Mouse Up / Up" button events and "Move / Touch Move" events can for example include information items identified

10 in Table 3 below, although other formats can also be used in other examples.

Table 3

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Notes</p></td></tr><tr><td><p>Number of pointer (Ν)</p></td><td><p>1</p></td><td><p>The number of pointers to the traffic events with multiple contacts. When set to 1, the event is indicated by a single touch</p></td></tr><tr><td><p>For t = 1: N {</p></td><td><p></p></td><td><p></p></td></tr><tr><td><p>Yu pointer</p></td><td><p>1</p></td><td><p>Identification number of this pointer. Value is in the range [0,1, ...]</p></td></tr><tr><td><p>X coordinate</p></td><td><p>2</p></td><td><p>X coordinate for the event, normalized relativecongregate video stream separation between the device receiver and the source device</p></td></tr><tr><td><p>Y coordinate}</p></td><td><p>2</p></td><td><p>Y coordinate for an event normalized relative coherent discretization of the video stream between the device receiver and the source device</p></td></tr>

The number of pointers can identify the number of touches or mouse clicks,

15 associated with the event of the introduction. Each pointer can have a unique Y pointer. If, for example, the multi-touch event includes three fingers, then the input event may have three pointers, each with a unique Y pointer. Each pointer (that is, each finger with a finger) can have an appropriate x-coordinate and y-coordinate, which corresponds to where the place had a touch.

20 A single custom command can be described as a sequence of input events.

For example, if three-finger slides are the command to close the application, slipping with triplets can be described in useful data 650 as a touch event down three pointers, an event touch movement of three pointer and an event touch up three pointers. Three pointers of the touchdown may have the same identifiers of the Y pointer as the three pointer

25 events touch and touch events up. The device 120 sources can interpret the combination of these three input events as slipping with three fingers.

For example, the description of the "Down key" or "Up key" events fields may include all the information items identified in Table 4 below.

17

iA 107151 C2

Table 4

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Notes</p></td></tr><tr><td><p>Reserved</p></td><td><p>1</p></td><td><p>reserved</p></td></tr><tr><td><p>Key Code 1 (A8CII)</p></td><td><p>2</p></td><td><p>Code of the first event key "down or up". The main / advanced A8CII code is used by a smaller one byte. Senior one byte is reserved for the future of A8CII-compliant codecs</p></td></tr><tr><td><p>Key code 2 (A8CII)</p></td><td><p>2</p></td><td><p>The key code of the second event is "down or up". The main / advanced A8CII code is used by a smaller one byte. Senior one byte is reserved for the future of A8CII-compliant codecs</p></td></tr>

The zoom-change event description field may, for example, include the information elements identified in Table 5 below.

Table 5

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Notes</p></td></tr><tr><td><p>X</p></td><td><p>2</p></td><td><p>The X-coordinate support for the zoom operation normalized with respect to the agreed-upon differentiation of the video stream between the recipient device and the device source</p></td></tr><tr><td><p>Υ</p></td><td><p>2</p></td><td><p>The Υ-coordinate support for the scale-normalization operation, normalized for a coherent differentiation of the video stream between the receiving device and the device source</p></td></tr><tr><td><p>An integer multiplied by a scale change</p></td><td><p>1</p></td><td><p>Significant part of the number of times to change the scale</p></td></tr><tr><td><p>A fractional number is required to change the scale</p></td><td><p>1</p></td><td><p>A fractional amount of times to scale</p></td></tr>

5

The description field for a horizontal scroll event or vertical scroll event may, for example, include the information elements identified in Table 6 below.

Table 6

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Note</p></td></tr><tr><td><p>Size for scrolling</p></td><td><p>2</p></td><td><p>The number of pixels for scrolling, normalized relative to the agreed discrepancy between the stream of video between the recipient device and the source device. A negative number may indicate scrolling to the right, and a positive number may indicate scrolling to the left.</p></td></tr>

10

The examples above have shown some exemplary ways that may have been formatted for useful data for generic input categories. If the input category field 624 of the data packet header 610 indicates an excellent input category, such as a directed user input, then the useful data 650 may have an excellent input format. WITH

The 15-way user-input receiver 160 may receive data from the user's input from a third-party device and direct the input to the source device 120 without interpreting the user input data. The device 120 of the source, in this way, can perform a parsing analysis of the useful data 650 according to the directional user input format. For example, the header 630 of helpful data can be 650 useful data

20 include a field for identifying the third party device from which the user input was received. The field may, for example, include an Internet protocol address

18

iA 107151 C2

(IP) of a third-party device, an MAC address, a domain name, or some other such identifier. The device 120 sources can perform a parsing analysis of the remaining useful data based on the identifier of the third party device.

The recipient device 160 can reconcile the capabilities of a third-party device by means of a sequence of messages. The recipient device 160 can then transmit the unique identifier of the third party device to the source device 120 as part of establishing a communication session with the source device 120 as part of the process of reconciling the capabilities. Alternatively, the recipient device 160 may transmit information describing the third party device to the source device 120, and based on this information, the source device 120 may identify a unique identifier for a third party device. Information that describes a third-party device may include, for example, information for identifying a third-party device and / or information for identifying the capabilities of a third-party device. No matter what

If the input data field 624 of the data packet header 610 still indicates a different input category, such as a voice command, then the useful data 650 may still have a great input format. For a voice command, useful data 650 may include encoded audio. The codec for coding and decoding the audio of the voice command can be arranged between the source device 120 and the receiver device 160 using a message sequence. For voice command, the time marker field 626 may include the time value of the speech sampling. In this case, the timestamp flag 622 may be set to indicate that there is a time stamp, but instead of the timestamp as described above, the timestamp field 626 may include the meaning of the speech sampling time for the encoded audio data 650.

In some examples, a voice command can be passed as a generic command as a descriptor; in this case, the input category field 624 can be set to identify the generic command format, and one of the reserved generic IDs may be assigned to voice commands. If the voice command is transmitted as a generic command, the frequency sampling rate of the speech signal may be present in the field 626 of the time marker of the data packet header 610 or may be present in useful data 650.

For captured data about a voice command, voice data can be encapsulated in a variety of ways. For example, voice command data can be encapsulated using RTR, which can provide a type of useful data to identify the timing codec and the timestamp used to identify the frequency sampling. The RTR data can be encapsulated using the generic user input format described above, or with or without an optional timestamp. The receiver device 160 can transmit generic input data that transmits voice command data to the source device 120 using the TRS / IP.

As discussed earlier, when the coordinates are included as part of a data packet, such as a data packet600, useful data 650, for example, coordinates may correspond to coordinates scaled on the basis of a coherent distinction, coordinates of the display screen, normalized coordinates or coordinates associated with the receiver display. In some cases, additional information may be included either in a data packet or transmitted separately for use with a source device to normalize the coordinates taken in the data packet.

Regardless of the input category for a particular data packet, the data packet header maybe an application level packet header, and the data packet can be transmitted over the TCP / IP. The TCP / IP can allow the receiver device 160 and the source device 120 to perform a retransmission in case of loss of the packet. The data packet may be sent from the receiver unit 160 to the source device 120 to control the audio data or video data of the source device 120, or for other purposes, for example, to operate the application operating on the source device 120.

FIG. 7A is a block diagram of an exemplary method for reconciling capabilities between a receiver device and a source device. The exemplary exemplary method may be implemented by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable storage medium (e.g., memory 332) can store

19th

iA 107151 C2

commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in one or more of the flowcharts described herein.

The method of FIG. 7A includes a receiver device 160 that receives a first message from the source device 120 (step 701). The message may, for example, contain a hold-up of the parameter. In response to the first message, the recipient device 160 may send a second message to the source device 120 (step 703). The second message may, for example, contain an answer for obtaining a parameter identifying the first list of supported input categories and a plurality of first supported types of lists, each of the supported input categories from the first list of input categories supported, has an associated first list of types that are supported. Supported input categories may, for example, correspond to the same categories used for the input category field 624 in FIG. 6. Table 2, presented above, represents one example of supported types for a particular category of input (generic input in this example). The receiver device 160 may receive from the third message source source 120 (step 705). The third message may, for example, include a parameter setup request, and the parameter setup request identifies the communication port, the second list of supported input categories, and a plurality of other supported types of lists, some of the supported input categories, from the second list of input categories , which is supported, has an associated second list of supported types, and each of the supported types from other lists includes a subset of types from the first lists. The recipient device 160 may transmit a fourth message to the source device 120 (step 707). A fourth message may, for example, contain parameter setup response to confirm that the types of other lists have been resolved. The recipient device 160 may receive a fifth message from the source device 120 (step 709). The fifth message may, for example, contain a second parameter setup request indicating that the communication channel between the source device 120 and the receiver device 160 has been enabled. The communication channel can, for example, contain a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to step 120 (step 711). The sixth message may, for example, contain a second parameter setting response, which acknowledges receipt of a second request by setting the parameter using the recipient device 160. The recipient device 160 may receive a fifth message from the source device 120 (step 709). The fifth message may, for example, contain a second parameter setup request indicating that the communication channel between the source device 120 and the receiver device 160 has been enabled. The communication channel can, for example, contain a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to step 120 (step 711). The sixth message may, for example, contain a second parameter setting response, which acknowledges receipt of a second request by setting the parameter using the recipient device 160. The recipient device 160 may receive a fifth message from the source device 120 (step 709). The fifth message may, for example, contain a second parameter setup request indicating that the communication channel between the source device 120 and the receiver device 160 has been enabled. The communication channel can, for example, contain a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to step 120 (step 711). The sixth message may, for example, contain a second parameter setting response, which acknowledges receipt of a second request by setting the parameter using the recipient device 160. The connection between the source device 120 and the receiver device 160 was allowed. The communication channel can, for example, contain a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to step 120 (step 711). The sixth message may, for example, contain a second parameter setting response, which acknowledges receipt of a second request by setting the parameter using the recipient device 160. The connection between the source device 120 and the receiver device 160 was allowed. The communication channel can, for example, contain a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to step 120 (step 711). The sixth message may, for example, contain a second parameter setting response, which acknowledges receipt of a second request by setting the parameter using the recipient device 160.

FIG. 7B is a block diagram of an exemplary method for reconciling capabilities between a device and a source device. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable storage medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in this flowchart .

The method of FIG. 7B includes a source device 120 that transmits the first message (702) to the recipient device 160. The first message may, for example, contain the hold-hold of the parameter. The source device 120 may receive a second message from the receiver device (604). The second message may, for example, contain an answer for obtaining a parameter that identifies the first list of supported input categories and a plurality of first supported list types, each of the input categories supported from the first list of supported input categories has an associated first type list, which are supported. The source device 120 may transmit the third message (706) to the recipient device 160. The third message may, for example, include a request for setting a parameter that identifies a port for data transmission, a second list of supported input categories, and a plurality of other supported types of lists, each of the supported input categories from the second supported subset of the input categories has an associated second list of supported types, and each of the supported types is different Lists includes a subset of types from the first lists. A device 120 may receive a fourth message from the receiver 160 (708). The fourth message can, for example, contain a parameter setting response to confirm that the types of other listings have been allowed. The device 120 can be transmitted to the fifth message receiver 160 (710). The fifth message may, for example, contain a second parameter setup request that indicates that the communication channel between the source device 120 and the recipient device 160 has been resolved. The communication channel, for example, may include a reverse channel of user input (IIS). The source device 120 may receive a sixth message from the recipient device (712). A sixth message may contain, for example, a second one

20

iA 107151 C2

the response setting parameter confirming receipt of the second parameter setup request using the recipient device 160.

FIG. 8 A is a block diagram of an exemplary way of transferring user input from the wireless device of the receiver to a wireless source device according to this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 8A includes receiving custom input data in a recipient's wireless device, such as the receiver wireless device 160 (801). The user input data can be obtained via the user input component of the wireless device 160 of the receiver, such as, for example, the user interface interface 376 shown with respect to the wireless device 360 ​​of the receiver. Additionally, the recipient device 160 can be divided into categories of user input data such as generic, directed, or specific operating systems. The recipient device 160 can then generate the data packet header based on the user input data (803). The header of the data packet may be the header of the application level packet. The header of the data packet can be, among other fields, the field for identifying the input category, which corresponds to the data of the user input. The input category can contain, for example, a generic output format or a human interface device command. A recipient device 160 may additionally generate a data packet (805), wherein the data packet contains a generated data packet header for the data. In one example, the useful data may include received user input data and can identify one or more user commands. The recipient device 160 can then transmit the generated data packet (807) to a wireless source device (e.g., the source device 120 according to FIG. 1A or device 220 a source according to FIG. 2). The recipient device 160 may include components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334 as shown in FIG. 3 for example.

FIG. 8B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 8B includes receiving a data packet (step 802), where the data packet can be, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data. The 120 device source can provide communications components that allow the transmission of data packets included in the interconnect unit 233 and the wireless modem 234, for example, as shown with reference to FIG. The source device 120 may then perform a parser analysis of the data packet header (804) included in the data packet to determine the input category associated with the user input data contained in the useful data. A device of 120 sources canproduce useful data based on a specific input category (806). Data packets described with links in FIG. 8A and 8B, as a whole, can take the form of data packets described with links in FIG. 6, and can be used to control audio / video data and applications in source devices.

FIG. 9A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless device source in accordance with this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 9A includes receiving custom input data in a receiver's wireless device, such as a receiver wireless device 160 (901). The user input data can be obtained through the user input component of the wireless device 160 of the recipient, such as, for example, the user-interface interface 376 shown with reference to FIGS. 3. The receiver device 160 can then be generated

21

iA 107151 C2

useful data (903), where useful data can describe user input data. In one example, useful data may include accepted user input data and can identify one or more user commands. A recipient device 160 may additionally generate a data packet (905), wherein the data packet contains a data packet header with generated useful data. The receiver device 160 can then transfer the generated packet data (907) to a wireless source device (e.g., a source device 120 according to FIG. 1A or source device 220 according to FIG. The receiver device 160 may include components that allow the transmission of data packets, such as the transport unit 333 and the wireless modem 334, for example. The data packet can be transmitted to a wireless source device via TCP / IP.

FIG. 9B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 9B includes receiving a data packet from the recipient device (902), wherein the data packet may include, among other things, the data packet header and useful data. In the aqueous example, useful data may include, for example, data describing the details of the user input, for example, the value of the input type. The 120 source can provide communications components that allow the transmission of data packets including the interconnect unit 233 and the wireless modem 234, for example, as shown with links to FIG. 2. The source device 120 can then parse the data packet (904) to determine the input type value in the input type field in the useful data. The device 120 can process the data describing the details of the user input on the basis of the defined value of the input type (906). Data packets described with reference to FIGS. 9A and 9B, as a whole, may take the form of data packets described with reference to FIGS. 6

FIG. 10A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in this flowchart .

The method of FIG. 10A includes receiving custom input data in a recipient wireless device such as a recipient wireless device 160 (1001). The user input data can be obtained via the user-generated component of the wireless device 160 of the recipient, such as, for example, the user input interface 376, as shown with the links to FIG. 3. The recipient device 160 may then generate a header of the data packet based on the user input (1003). The header of the data packet may contain, among other fields, a time stamp flag (for example, a 1-bit field) to indicate if there is a time marker field in the data packet header. Flag of the timestamp may, for example, include "1" to indicate that there is a time marker field, and mayinclude "0" to indicate that there is no time marker field. The time marker field maybe, for example, a 16-bit field containing a timestamp generated by the source device 120 and added to the video data prior to transmission. The receiver device 160 may additionally generate a data packet (1005), wherein the data packet contains a generated data packet header and useful data. In one example, the useful data may include the received data of the user input and can identify one or more user commands. The recipient device 160 can then transmit the generated data packet (1007) to the non-originating source device (e.g., the source device 120 in FIG. 1A or the source device 220 on FIG. 2). The recipient device 160 may include components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown with reference to FIG. 3. The data packet can be transmitted to the wireless device of the source on the TCP / IP.

FIG. 10B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms,

22

iA 107151 C2

which, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 10B includes receiving a data packet from the wireless recipient device (1002), wherein the data packet may include, among other things, the data packet header and useful data. Useful data may include, for example, user-input data. The source device 120 may include communication components that allow the transmission of data packets including the transport unit 233 and the wireless modem 234, for example, as shown with reference to FIGS. 2. The source device 120 may then perform a parsing analysis of the data packet header (1004) included in the data packet. A 120 source can determine whether a time marker field in the data packet header (1006). In one example, the source device 120 can make a determination based on the value of the time flag flag included in the header of the data packet. If the data packet header includes a timestamp, the source device 120 can process useful data based on the timestamp, which is in the timestamp field (1008). The data packets described with reference to FIGS. 10A and 10B, as a whole, may take the form of data packets described with reference to FIGS. 6, and can be used to control the audio / video data in the source device.

FIG. 11 A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless device source in accordance with this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 11A includes receiving custom input data for a recipient wireless device such as a receiver wireless device (1101). The user input data can be obtained via the user-generated component of the wireless receiver device 160, such as, for example, the user input interface 376 shown with references to FIG. . 3. The recipient device 160 may then generate a header of the data packet based on the user input (1103). The header of the data packet may contain, among other fields, a time marker field. A timestamp field may include, for example, a 16-bit field containing a timestamp based on multimedia data generated by the source wireless source 120 and transmitted to the receiver outside the wireless device 160. The timestamp could be added to the video data frame by using the wireless source 120 device before transmitting to the wireless device in the receiver. A timestamp field may, for example, identify a timestamp associated with a video data frame displayed on the wireless recipient device 160 at a time when user input data was captured. The receiver device 160 may additionally generate a data packet (1105), wherein the data packet contains a generic header of the data packet and useful data. In one example, useful data may include accepted user input data and can identify one or more custom commands. The recipient device 160 can then transfer the generated data packet (1107) to the wireless source device (e.g., the source device 120 according to FIG. 1A or source device 220 according to FIG. 2). The recipient device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown with reference to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP.

FIG. 11B is a block diagram of an exemplary method for receiving user input from the wireless device of the recipient in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 11B includes receiving a data packet from a wireless receiver device, such as a receiver wireless device 160 (1102), where the data packet can be, among other things, the data packet header and useful data. Useful data may include, for example, user input data. The 120 device source can provide communications components that allow the transmission of data packets included in the interconnect unit 233 and the wireless modem 234, for example, as shown with reference to FIG. The source device 120 can then identify the timestamp field in the packet header

23

iA 107151 C2

data (1104). The device 120 sources can process useful data based on the timestamp, which is in the field of time marking (1106). As part of the processing of useful data, based on the timestamp, the source device 120 can identify the video data frame that is displayed on the wireless device of the recipient at the time when the user input data was received and to interpret the useful data based on the content of the frame. As part of the processing of useful data, based on the timestamp, the source device 120 can compare the time marker with the current time marker for the current video frame transmitted by the source device 120 and may execute the user input command described in useful data in response to the time difference between the timestamp and the current timestamp, which is less than the threshold value, or do not execute a command-line command described in useful data in response to a time difference between a timestamp and a current time marker that is greater than a threshold value. Data packets described with links in FIG. 11A and 11B, as a whole, may take the form of data packets described with links in FIGS. 6, and can be used to control the audio / video data in the device source.

FIG. 12A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 12A includes receiving user-entered data in a recipient's wireless device, such as the receiver wireless device 160 (1201). In the aqueous example, the user input data may be data of a voice command which can be obtained through the user-input component of the wireless receiver device 160, such as, for example, the voice command recognition module, the user-interface interface 376 of the user input in FIG. 3. A recipient device 160 can generate a data packet header based on user input (1203). The recipient device 160 can also generate useful data (1205), where useful data may contain voice data. In one example, the useful data may also include the received data of the user input and can identify one or more user commands. The recipient device 160 may additionally generate a data packet (1207) where the data packet contains the generated data packet header and useful data. The recipient device 160 may then transfer the generated data packet (1209) to a wireless source device (e.g., a source device 120 according to FIG. 1A or a source device 220 according to FIGURE 2). The receiver device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in the link in FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. where the data packet contains the generated data packet header and useful data. The recipient device 160 may then transfer the generated data packet (1209) to a wireless source device (e.g., a source device 120 according to FIG. 1A or a source device 220 according to FIGURE 2). The receiver device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in the link in FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. where the data packet contains the generated data packet header and useful data. The recipient device 160 may then transfer the generated data packet (1209) to a wireless source device (e.g., a source device 120 according to FIG. 1A or a source device 220 according to FIGURE 2). The receiver device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in the link in FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. The receiver device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in the link in FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. The receiver device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in the link in FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP.

FIG. 12B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 12B includes the reception of a data packet (1202), where the data packet can be, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data such as voice command data. The source device 120 may include communication components that allow the transmission of data packets including the transport unit 233 and the wireless modem 234, for example, as shown with reference to FIG. 2. The source device 120 may then perform a parsing analysis of useful data (1204) included in the data packet to determine whether the data is useful in the voice of the voice command. The data packets described with reference to FIGS. 12A and 12B, in general, may take the form of data packets described with reference to FIGS. 6, and can be used to control audio / video data in the source device.

FIG. 13 A is a block diagram of an example method for the transfer of user input from the wireless device of the receiver to a wireless device source in accordance with this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples it is read by a computer

24

iA 107151 C2

a storage medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 13A includes receiving user-entered data in a recipient's wireless device, such as the receiver wireless device 160 (1301). In the aqueous example, the user input data can be a multiple-touch gesture that can be obtained through the user-input component of the wireless recipient device 160, such as, for example, ii 167 or the user-input interface 376. 3. In an aqueous example, a plural touch gesture may include the first input of the data by touch and typing the data by touch. The recipient device 160 can generate the header of the data packet on the basis of the user input (1303). The recipient device 160 may also generate useful data (1305), where the useful data can associate the user input data for the first touch input event with the first identifier of the pointer and the user input for the second input event by the touch with a second pointer identifier. The recipient device 160 may additionally generate a data packet (1307), wherein the data packet contains a generated data packet header and useful data. The recipient device 160 may then transmit the generated data packet (1309) to the wireless source device (e.g., the source device 120 according to Figure 1A, the source device 220 according to Figure 2). The receiver device 160 may comprise components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in the link in FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP.

FIG. 13B is a block diagram of an exemplary method for receiving user input from the wireless device of the recipient in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 13B includes receiving a data packet (1302), wherein the data packet can be, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data such as multiple-touch gestures. The source device 120 may include communication components that allow the transmission of data packets including the transport unit 233 and the wireless modem 234, for example, as shown FIG. 2. The 120 source can then parse the useful data (1304) included in the data packet to identify the user input data, useful data included. In one example, the identifiable data may include the data of the user input for the first input event by the first identifier of the index and the data of the user input for the second event of the introduction by the touch with the second identification of the index. The source device 120 can then interpret the user input for the first touch input event and the user input data for the second touch input event as a multiple-touch gesture (1306). Data packets described with links in FIG. 13A and 13B, as a whole, may take the form of data packets described with links in FIGS. 6, and can be used to control the audio / video data in the device source. The source device 120 can then interpret the user input for the first touch input event and the user input data for the second touch input event as a multiple-touch gesture (1306). Data packets described with links in FIG. 13A and 13B, as a whole, may take the form of data packets described with links in FIGS. 6, and can be used to control the audio / video data in the device source. The source device 120 can then interpret the user input for the first touch input event and the user input data for the second touch input event as a multiple-touch gesture (1306). Data packets described with links in FIG. 13A and 13B, as a whole, may take the form of data packets described with links in FIGS. 6, and can be used to control the audio / video data in the device source.

FIG. 14A is a block diagram of an exemplary method for transferring user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. The illustrative exemplary method may be performed by the recipient device 160 (Figure 1A) or the recipient device 360 ​​(Figure 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 14A includes receiving custom input data in the recipient's external wireless device 360 ​​from an external device (1401). In one example, an external device may be a third party device coupled to a receiver device. The recipient device 160 can generate a data packet header based on a user guide (1403). In one example, the header of the data packet can identify the data of the user input as directed user-input data. The recipient device 160 can also generate useful data (1405), where useful data may contain user input data. The recipient device 160 may additionally generate a data packet (1407), wherein the data packet may comprise a generated data packet header and useful data.

25

iA 107151 C2

The recipient device 160 can then transmit the generated data packet (1409) to the non-originating source device (e.g., the source device 120 according to FIG. 1A of the source device 220 according to FIG. 2). The recipient device 160 may include components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown with reference to FIGS. 3. The data packet can be transmitted to the wireless device of the source on the TCP / IP.

FIG. 14B is a block diagram of an exemplary method for receiving user input data from a wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 14B includes receiving a data packet (1402), wherein the data packet can be, among other things, the header of the data packet and useful data. Useful data may include, for example, user-input data, such as a directed user input command, indicating that user-input data was sent from a third-party device. The source device 120 may include communication components, such as allowing the transmission of data packets including the transport unit 233 and the wireless mode 234, for example, as shown with reference to FIG. 2. The source device 120 may then perform a parser analysis of the data packet header and may determine that the useful data contains a directed user input command (1404). The device 120 sources can then perform parsing analysis of useful data (1406), included in the data packet to identify the identity associated with a third-party device in accordance with a command-specific, user-friendly input. The device 120 of the source can then process the useful data on the basis of the identified identification of the third party device (1408). The data packets described with reference to FIGS. 14A and 14B generally can take the form of data packets described with reference to FIGS. 6, and can be used to control the audio / video data in the source device. in general, may take the form of data packets described with reference to FIGS. 6, and can be used to control the audio / video data in the source device. in general, may take the form of data packets described with reference to FIGS. 6, and can be used to control the audio / video data in the source device.

FIG. 15A is a block diagram of an exemplary method for transmitting user data from a wireless recipient device to a wireless source device according to this disclosure. The illustrated exemplary method can be implemented by the receiver device 160 (FIG. 1A) or receiver device 360 ​​(FIG. In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

The method of FIG. 15A includes receiving user-entered data in the recipient's wireless device (1501). Custom input data may have associate coordinate data. Associated coordinate data may, for example, correspond to the location of the mouse click event or location of the touch event. The recipient device 160 can then normalize the associated coordinate data to generate the coordinate data (1503). The receiver device 160 may then generate packet data, which includes normalized coordinate data (1505). Normalization of coordinate data can include the scaling of associated coordinate data based on the ratio of the screen display screen and the distinction of the source display, such as the display device 22 of the source 120. The screen display may be distinguished by the receiver device 160, and the source separation display may be taken from the device 120 of the source. The recipient device 160 may then transmit a packet of data with normalized coordinates to the wireless source 120 (1507). As part of the method according to Fig. 15A, the recipient device 160 may also determine if there is an associated data coordinate within the display screen window for content received from a wireless device source and, for example, processing a custom input locally, if the associated data coordinate is outside the display screen window, or otherwise normalize the coordinates, is written if the input is within the window of the display screen. The recipient device 160 may then transmit a packet of data with normalized coordinates to the wireless source 120 (1507). As part of the method according to Fig. 15A, the recipient device 160 may also determine if there is an associated data coordinate within the display screen window for content received from a wireless device source and, for example, processing a custom input locally, if the associated data coordinate is outside the display screen window, or otherwise normalize the coordinates, is written if the input is within the window of the display screen. The recipient device 160 may then transmit a packet of data with normalized coordinates to the wireless source 120 (1507). As part of the method according to Fig. 15A, the recipient device 160 may also determine if there is an associated data coordinate within the display screen window for content received from a wireless device source and, for example, processing a custom input locally, if the associated data coordinate is outside the display screen window, or otherwise normalize the coordinates, is written if the input is within the window of the display screen.

FIG. 15B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (Figure 1A) or a source device 220 (Figure 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms,

26

iA 107151 C2

which, when executed, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

The method of FIG. 15B includes the reception of a data packet in a wireless device of the source, where the data packet contains data from the user input with associated data coordinates (1502). Associated coordinate data may, for example, match the location of the mouse click event or location of the touch event in the recipient's device. The device 120 source can then normalize the associated coordinate data to generate a normalized data coordinate (1504). The device 120 sources can normalize the coordinate data by scaling the associated coordinate data based on the ratio of the distinguishing of the screen display display and the differentiation of the source display. The device 120 sources can determine the distinction of the display device of the source and can accept the distinguishing of the screen display screen from the wireless device recipient. The source device can then process a data packet based on the conventional coordinate data (1506). The data packets described with reference to FIGS. 15A and 15B, as a whole, may take the form of data packets described with reference to FIGS. 6, and can be used to control the audio / video data in the source device.

For the sake of simplicity, the aspects of this disclosure have been described separately with references to FIG. 7-15. However, it is considered that these different aspects can be combined and used together with one, but not just individually. In general, the functionality and / or modules described in this description can be implemented in either one or both of the: wireless device source and the wireless device of the recipient. Thus, the capabilities of the user interface described in the current example can be used interchangeably with the source device interconnected device and the wireless device of the receiver.

The methods of this disclosure can be implemented in a large variety of devices or devices, which include a wireless telephone and integrated circuit (IC) or set of IC circuits (ie, chipset). Any components, modules or blocks that have been described are provided to emphasize the functional aspects, and do not necessarily require the implementation of various hardware units.

Accordingly, the methods described herein can be implemented in hardware, software, hardware, or any combination of these. If implemented in hardware, any features described as modules, blocks or components can be implemented together in an integrated logic device or separately as discrete but interactive logical devices. If implemented in softwareprotection, methods can be implemented at least partially with computer-readable media containing commands that, when executed in the processor, perform one or more of the methods described above. Computer-readable media may contain a material and non-time-consuming computer-readable storage media and may be part of a computer software product, which may include packaging materials. A computer-readable storage medium may comprise an operational storage device (RAM), such as a synchronous dynamic memory storage device (RAM), a permanent storage device (ROM), non-volatile operating memory Switching device (K ^ RAM), electrically erased programmable permanent memory device (EERRO), flash memory, magnetic or optical storage media, etc. Methods, in addition or alternatively, may be implemented at least partially by means of a computer-readable media carrier that transmits or transmits a code in the form of commands or data structures, and which may be available, read, and / or executed by the computer. The storage medium may comprise an operational storage device (RAM), such as a synchronous dynamic storage RAM (RAM), a permanent storage device (ROM), a non-volatile operational memory device (K ^ RAM), an electrically erased programmable, charger (EERRO), flash memory, magnetic or optical storage media, etc. Methods, in addition or alternatively, may be implemented at least partially by means of a computer-readable media carrier that transmits or transmits a code in the form of commands or data structures, and which may be available, read, and / or executed by the computer. The storage medium may comprise an operational storage device (RAM), such as a synchronous dynamic storage RAM (RAM), a permanent storage device (ROM), a non-volatile operational memory device (K ^ RAM), an electrically erased programmable, charger (EERRO), flash memory, magnetic or optical storage media, etc. Methods, in addition or alternatively, may be implemented at least partially by means of a computer-readable media carrier that transmits or transmits a code in the form of commands or data structures, and which may be available, read, and / or executed by the computer. permanent memory (ROM), non-volatile operative memory device (K ^ RAM), electrically erased programmable permanent memory device (EERROM), flash memory, magnetic or optical storage media, etc. Methods, in addition or alternatively, may be implemented at least partially by means of a computer-readable media carrier that transmits or transmits a code in the form of commands or data structures, and which may be available, read, and / or executed by the computer. permanent memory (ROM), non-volatile operative memory device (K ^ RAM), electrically erased programmable permanent memory device (EERROM), flash memory, magnetic or optical storage media, etc. Methods, in addition or alternatively, may be implemented at least partially by means of a computer-readable media carrier that transmits or transmits a code in the form of commands or data structures, and which may be available, read, and / or executed by the computer.

The code can be executed by one or more processors such as one or more digital signal processors (δδ processors), general-purpose microprocessors, specialized integrated circuits (ABC schemes), user-programmed gate matrices (PROA matrices) or other equivalent integral or discrete logic. Accordingly, the term "processor" used in this specification may refer to any pre-existing structure or any other structure suitable for implementing the methods described herein. Additionally, in some aspects, the functionality described herein may be provided within the allocated software modules or hardware modules configured for encoding and decoding, or incorporated into a video codec. In addition,

Various aspects of this disclosure were described. These and other aspects are within the scope of the following claims.

Reference positions

27

iA 107151 C2

100 source / recipient system

120 device source

121 memory

122 display

123 speaker

124 audio / video encoder

125 audio / video control module

126 transmitter / receiver unit (TX / RH)

150 communication channel

160, 180, 360, 560 receiver device

162 display

163 speaker

164 audio / video decoder

166 transmitter / receiver unit

167 user input device

168 user input processing module220, 520 source device

222, 362 local display

223 local speaker

231, 331 processors

232, 332 memories

233, 333 transport block

234, 334 wireless modem

235, 335 display processor

236, 336 audio processor

363 speaker

376 user input interface

410 transmitter system

412 data source

414 processor (TX) data transfer

420 processor MIMO TX data transfer

422 transmitter

424 transmit antenna

430, 470 processor

432, 472 memories

436 data source

438 TX data transfer processor

450 receiver system

452 receiving antenna

454 receiver

460 RX data processor

480 modulator

600 data packet

610 header of the data packet

Chart 620

621 version field

622 flag of time mark

623 reserved field

624 input category field

625 field length

626 optional time field field630 useful data header

650 useful data

FORMULA INSTRUCTIONS

A method for transmitting user input data from a wireless receiver device to a non-wireless source device, the method comprising:

obtaining in the wireless device of the recipient of data from the user input from the device of the third party;

28

iA 107151 C2

generation in the wireless device of the recipient of the data packet header, and the header of the data packet contains a field for identifying the user input as directed data user input;

generating in a wireless device a recipient of useful data that contains data for user input;

generating in a wireless device the recipient of a data packet that contains the packet header and useful data, and

transferring this data packet from the wireless device of the recipient to the wireless device source.

2. The method of claim 1, further comprising:

Matching capabilities of a third-party device, with the wireless device of the recipient, through a sequence of messages.

3. The method of claim 1, further comprising:

as part of establishing a communication session between a wireless source device and a wirelessdevice device, the transmission of the third party device ID from the wirelessdevice device to the wireless source device.

4. The method of claim 1, further comprising:

as part of establishing a communication session between a wireless source device and a wirelessdevice device receives the device ID of the third party from the wirelessdevice device.

5. The method of claim 1, wherein the value of said field is set to indicate that the useful data contains the referenced user input data.

6. The method of claim 1, wherein the useful data comprises an identifier of the third party device.

7. The method of claim 1, wherein the third party device identifier is selected from the group consisting of: IP address of the third party device, domain name of the third party device.

8. The method of claim 1, wherein the identifier is generated by a wireless source device and transmitted to the wireless device of the receiver.

9. The method of claim 1, wherein the third party device is a different wireless receiver device.

10. The method of claim 1, wherein the third-party device is an input device communicatively coupled to the wireless device of the receiver.

11. The method of claim 1, wherein the data packet header is an application-level packet header.

12. A wireless device of the receiver configured to transmit the user's data to a wireless source device, and the receiver's wireless device contains:

means for receiving user input from a third-party device;

a tool for generating a data packet header, and the data packet header contains a field

to identify user-entered data as directed user data

introduction;

a tool for generating useful data containing user input;

means for generating a data packet containing a data packet header and useful data, and means for transmitting this data packet to a wireless source device.

13. A method of receiving user input data from a wireless device of a receiver to a non-wireless source device, the method comprising:

reception in the wireless device of the recipient of a data packet containing the packet header and useful data from the wireless device of the recipient;

executing a data packet header parser in a wireless device to determine that the useful data contains a directed user input command; executing a useful parsing parsing in the wireless device of the recipient for identifying the third-party device identification information; and

handling of these useful data in the wireless device on the basis of the identification information of the third-party device.

14. Wireless source device configured to receive user-generated data from the wireless device of the receiver, with the wireless source device containing:

a means for receiving a data packet that contains a data packet header and useful data from the recipient's wireless device,

a means for performing parsing analysis of the data packet header to determine that useful data contains a directed user input command;

a tool for performing parsing analysis of useful data for identifying the identity information of a third-party device,

29

iA 107151 C2

A tool for processing these useful data on the basis of the identification information of the third-party device.

15. A computer readable storage medium that stores instructions that, when executed by the processor, make the said processor execute a method for transmitting user-defined data

5 from the wireless device of the receiver to the wireless device of the source according to any of the claims. 1-11.

16. A computer readable storage medium that stores instructions that, when executed by the processor, compel said processor to perform a method for receiving user input data from the wireless device of the receiver in a wireless source device according to claim 13.

30

iA 107151 C2

31

iA 107151 C2

<tr><td><p>Source device</p></td><td><p>The first message</p></td><td><p>The recipient's device</p></td></tr><tr><td><p>520</p></td><td><p>The second message</p></td><td><p>560</p></td></tr><tr><td><p>The third message</p><p>Fourth message</p></td></tr><tr><td><p></p></td><td><p> Fifth message</p></td><td><p></p></td></tr><tr><td><p>■ ------</p><p>Sixth message _</p></td></tr><tr><td><p></p></td><td><p>Seventh message</p></td><td><p></p></td></tr><tr><td><p>■ ·</p><p>Eighth message</p></td></tr><tr><td><p></p></td><td><p></p></td><td><p></p></td></tr>

FIG. 5A

32

iA 107151 C2

<tr><td><p>Source device</p></td><td><p></p></td><td><p>Device</p></td></tr><tr><td><p>520</p></td><td><p></p></td><td><p>the recipient</p><p>560</p></td></tr>

1a. SET_RAKAMETEK^Request

»Kyyyts_saraIIiKu

1b.SET_RAKAMETEK ^Answer("I_iІссaaІІііу: Іпри1_саїдегу_ііз1 = СЕЛЕРІС; депагис_арАРІІі $ 1 = Моїв. ЗіпДІвТоізгі; ІІРс_ар_Ііз1 = Міт роти = N1) 11.)

OK

МР_ииссaарІІіііу: іпри (_са1едогу_ІІ8 (= НЮС; двпагіс_арарIІзи1 = ΝΙΙΙ_Ι_;

Iisis_sar_Ii5 (= Moiw / BT, KetoIeopsihi / Ipiğagha &lt;1; horns = N1) 11.)

FIGURE 5.B

33

iA 107151 C2

34

iA 107151 C2

35

iA 107151 C2

36

iA 107151 C2

37

iA 107151 C2

38

iA 107151 C2

Computer layout L. Tsikhanovska

State Service of Intellectual Property of Ukraine, st. Uritskogo, 45, Kyiv, SME, 03680, Ukraine

State Enterprise "Ukrainian Institute of Industrial Property", st. Glazunova, 1, Kyiv - 42, 01601

39