Reverse channel of user input for wireless display

UKRAINE ₍₁₉₎ iA (11) 109176 (13) C2

(51) IPC

O06P 3/03 (2006.01)

H04M 1/72 (2006.01)

STATE SERVICE BANITELECTUAL PROPERTY IN UKRAINE

(12) DESCRIPTION TO THE INVENTORY PATENT

<tr><td><p>(21) Application number:</p></td><td><p>and 2013 10237</p></td></tr><tr><td><p>(22) Date of application:</p></td><td><p>January 20, 2012</p></td></tr><tr><td><p>(24) Date from which the valid law for the invention:</p></td><td><p>07/27/2015</p></td></tr><tr><td><p>(31) The number of the previous one</p></td><td><p>61 / 435,194,</p></td></tr><tr><td><p>applications according to</p></td><td><p>61 / 447,592,</p></td></tr><tr><td><p>Paris Convention:</p></td><td><p>61 / 448,312,</p><p>61 / 450,101,</p><p>61 / 467,535,</p><p>61 / 467,543,</p><p>61 / 514,863,</p><p>61 / 544,440,</p><p>13 / 344,424</p></td></tr><tr><td><p>(32) Date of submission</p></td><td><p>Jan 21, 2011</p></td></tr><tr><td><p>previous application</p></td><td><p>Feb 28, 2011</p></td></tr><tr><td><p>according to</p></td><td><p>03/02/2011</p></td></tr><tr><td><p>Paris Convention:</p></td><td><p>03/07/2011</p><p>March 25, 2011,</p><p>March 25, 2011,</p><p>08/03/2011</p><p>10/07/2011</p><p>01/05/2012</p></td></tr><tr><td><p>(33) Code of the State Party</p></td><td><p>υδ,</p></td></tr><tr><td><p>The Paris Convention</p></td><td><p>from</p></td></tr><tr><td><p>to which is filed</p></td><td><p>υδ,</p></td></tr><tr><td><p>preliminary application:</p></td><td><p>from</p><p>υδ,</p><p>υδ,</p><p>υδ,</p><p>υδ,</p><p>υδ</p></td></tr>

(41) Publication of the information dated November 25, 2013, No. 22 on the application:

(46) Publication of the data on 07/27/2015, Bulletin No. 14 on the issuance of a patent:

(72) The inventor (s):

Ravindran Vijayalakshmi R. (υδ), Huang Xiaolong (υδ),

Wang Xiaodong (υδ),

Shaukat Fawad (υδ)

(73) Owner (s):

QUALCOM INCORPORATEID,

Institute of Physiology, Department of Biology, 5775 Mohyiouz Yugiuye, Hall of YujiDo, Saiiyogpia92121, Ipiyev Ziayesh otegi Ategis (South Ossetia)

(74) Representative:

Moshynska Nina Nikolaevna, registry number 115

(56) List of documents taken into account by examination:

OTO 01/84291 A1, November 8, 2001, and 2004160967 A1, 19.08.2004 and 7696980 B1, April 13, 2010

iA 109176 C2

(86) Number and date of PCT / 52001/022080,

submission of the international application dated January 20, 2012 submitted

in accordance with the PCT Agreement

(54) OPEN DIRECTION DATABASE CHANNEL FOR UNSWEIGHT DISPLAYS

(57) Summary:

As part of a communication session, the wireless source device can transmit audio and video data to the receiver's wireless device, and the wireless device of the receiver can transmit the user input data taken on the wireless device of the receiver back to the wireless device of the source. Therefore, the user of the wireless device receiver

iA 109176 C2

can manage a wireless source device and manage content that is transferred from the wireless source device to the wireless device of the recipient. The input data received by the wireless device of the receiver may have associated coordinate information, which is scaled or normalized with either the wireless device of the receiver, or a non-wireless source device.

iA 109176 C2

This application requires priority:

US Preliminary Application No. 61/435194 filed January 21, 2011, US Preliminary Application No. 61/447592 filed Feb. 28, 2011, US Preliminary Application No. 61/448312, filed March 2, 2011, US Preliminary Application No. 61 / 450101 filed March 7, 2011, United States Preliminary Application No. 61/467535 filed March 25, 2011, US Preliminary Application No. 61/467543 filed March 25, 2011, US Preliminary Application No. 61/514863 filed 3 August 2011; and a US Preliminary Application No. 61/544440 filed on October 7, 2011, each of which is fully incorporated herein by reference.

The field of technology to which this invention belongs

This disclosure relates to methods for transmitting data between a wireless source device and the receiver's wireless device.

The prior art

The Wireless Display (MIMO) or the Mi-Gi (MGU) display system includes a non-wireless source device and one or more wireless devices of the receiver. Device sources and each of the recipient devices can be either mobile devices or wireless devices with wireless capabilities. One or more recipient devices and devices may, for example, include mobile phones, portable computers with wireless cards, personal digital assistants (PII assistants), portable media players or other such devices with wireless communication capabilities that include so-called "smartphones" and "smartpads" or tablets, electronicbooks, or any type of wireless display, video game devices, or other types of wireless devices.

The source device sends media such as audio / video data (Α / ν) to one or more receiver devices that participate in a particular media sharing session. Media can be played on either the local display device of the source device, or on any of the displays. receiver devices. More specifically, each of the participating devices of the receiver reproduces the received media on its screen and audio equipment.

The essence of the invention

This disclosure generally describes a system in which the wireless device of the receiver can be associated with the wireless device of the recipient. As part of a communication session, a wireless source device can transmit audio and video data to the wireless device of the receiver, and the receiver's wireless device can transmit the data input by the user, received to the recipient's wireless device, back to the wireless device of the source. Thus, the recipient's wireless device user can control the wireless device's resources and control the content that is transmitted from the wireless source device to the receiver's wireless device.

In one example, the method for transmitting user data from a wireless device to a receiver on a wireless source device includes receiving data about the input of data by the user in the wireless device of the recipient, and the data on the data input user has associated data of the coordinates; normalization of associated coordinate data for generating normalized coordinate data; generation of a data packet containing the coordinated data; Transfer this data packet to a wireless source device.

In another example, the receiver's wireless device for transmitting user data to a non-originating source device includes memory that stores commands; one or more processors configured to execute commands, and after executing commands, one or more processors are called to receive data on user data input in the recipient's wireless device, and the user data input data is associated with coordinate data, the normalization of the associated coordinate data for generating the normalized coordinate data, generation of the data packet, containing normalized data coordinate; and a transport block for transmitting this data packet to a wireless device source.

In another example, a computer-readable storage medium stores commands that after one or more processors are executed, one or more processors are forced to execute a method for transmitting user data from the wireless device of the receiver to a non-shipping source device. The method includes the acquisition of data entry data user in the wireless device recipient, and data entry data

1

iA 109176 C2

the user has associated coordinate data; normalization of associated coordinate data for generating normalized coordinate data; generation of a data packet containing the coordinated data; Transfer this data packet to a wireless source device.

In another example, the wireless device of the receiver for the transfer of user data to a non-shipping source device includes a means for obtaining data on data input user in the wireless device receiver, and data entry data user has associated data coordinates; a means for the normalization of associated data coordinate for generating normalized coordinate data; means for generating packet data containing normalized coordinate data; A tool for transferring this data packet to a non-shipping source device.

In another example, the method of receiving user data from a wireless device receiver in a wireless source device includes receiving a data packet in a non-wireless source device, and the data packet contains data about data input by the user with the associated coordinate data; normalization of associated coordinate data for generating normalized coordinate data; processing a data packet based onormalized coordinate data.

In another example, the wireless device of the receiver for the transfer of user data to the external source device includes a transport block for receiving a data packet in a non-originating source device, and the data packet contains data about data input by the user with the associated coordinate data; memory that stores commands; one or more processors configured to execute commands, and after executing a command of one or more processors, cause the normalization of associated coordinate data for generating normalized coordinate data and processing a data packet based on normalized data coordinates.

In another example, a computer-readable storage medium stores commands that, once executed by one or more processors, force one or more processors to execute the method of receiving user data from the wireless device of the recipient to the non-wireless source device. The method includes receiving a data packet in a wireless device device, the data packet comprising data for entering user data associated data coordinates; normalization of associated coordinate data for generating normalized coordinate data; processing a data packet based on normalized data coordinates.

In another example, the wireless device of the receiver for transmitting user data to a non-originating source device includes a means for receiving a data packet in a wireless device device, the data packet comprising data for entering data by the user by the associated data coordinates; a means for the normalization of associated coordinate data for generating normalized coordinate data; a tool for processing a data packet based onormalized coordinate data.

Brief description of the drawings

FIG 1A is a block diagram illustrating an example of a source / receiver system that can implement the methods of this disclosure.

FIG. 1B is a block diagram illustrating an example of a source / receiver system with two device receivers.

FIG 2 shows an example of a source device that can implement the methods of this disclosure.

FIG 3 shows an example of a recipient device that can implement the methods of this disclosure.

FIG 4 shows a block diagram of the transmitter system and receiver systems that can realize the means of this disclosure.

FIG 5A and 5B show exemplary sequences of message transmission for executing option matching in accordance with the methods of this disclosure.

FIG 6 shows an exemplary data packet that can be used to deliver data from the user's data input data to the device of the receiver.

FIG 7A and 7B are flowcharts illustrating the methods of this disclosure that may be used to reconcile the capabilities between the source device and the receiver device.

FIG 8A and 8B are flowcharts illustrating the methods of this disclosure, which may be used to transmit and receive data packets with user data input data.

FIG 9A and 9B are flowcharts illustrating the methods of this disclosure, which may be used to transmit and receive data packets with data input data by the user.

2

iA 109176 C2

FIG 10A and 10B are flowcharts illustrating the methods of this disclosure, which can be used to transmit and receive data packets with timestamp information and data for data entry by the user.

FIG 11A and 11B are flowcharts illustrating the methods of this disclosure, which can be used to transmit and receive packets of data with information about the timestamp and data data input by the user.

FIG 12A and 12B are flowcharts illustrating the methods of this disclosure, which may be used to transmit and receive data packets that include voice commands.

FIG. 13A and 13B are flowcharts illustrating methods of this disclosure that can be used to transmit and receive data packets with user input commands by means of multiple taps.

FIG 14A and 14B are flowcharts illustrating methods of this disclosure that may be used to transmit and receive data packets with user input data sent from a third party device.

FIG 15A and 15B are flowcharts illustrating the methods of this disclosure, which may be used to transmit and receive data packets.

Detailed description

This disclosure generally describes a system in which the wireless device of the receiver can be associated with the wireless device of the recipient. As part of a communication session, a wireless source device can transmit audio and video data to the wireless device of the receiver, and the receiver's wireless device can transmit user input received in the recipient's wireless device to the wireless source device. Thus, the recipient's wireless device user can control the wireless device's resources and control the content that is transmitted from the wireless source device to the receiver's wireless device.

FIG 1A is a block diagram illustrating an exemplary source / receiver system 100 that can realize one or more methods of this disclosure. As shown in FIG. 1a, system 100 includes a source device 120 that communicates with the recipient device 160 via the communication channel 150. The source device 120 may include the memory that stores the audio / video data (A / ν) 121, the display 122, the speaker 123, the audio / video encoder 124 (also referred to as encoder 124), the audio / video control module 125, and the transmitter unit 126 / Receiver (TX / RX). The receiver device 160 may include a display 162, a speaker 163, an audio / video decoder 164 (also called decoder 164), a transmitter / receiver unit 166, a user input device 167, and a user input processing unit 168. The illustrated components are just one exemplary configuration for the source / recipient system. Other configurations may include fewer components than those that are illustrated, or may include additional components than those illustrated.

In the example of FIG. 1A, the source device 120 may display part of the audio / video video 121 in the display 122 and may output part of the audio audio / video data 121 to the speaker 123. Audio / video data 121 may be stored locally on the source device 120, which may receive access from an external storage medium, such as a file server, a hard drive, an external memory, a disk drive, a drive, or other physical storage medium, or may be transmitted as a stream to the source device 120 via a network connection such as the Internet. In some cases, audio / video data 121 can be captured in real time using the camera and microphone of the source device 120. Audio / video data 121 may include multimedia content such as movies, television shows or music, but may also include real-time content generated by the device's 120 source. Such real-time content, for example, may be formed by applications operating on the source device 120, or captured video data, for example, as part of a video telephony session. As will be described in more detail, such content in real time can in some cases include a video frame of user input options available to a user to select. In some cases, audio / video data 121 may include video frames that are a combination of different types of content, such as a movie footage or a program that has user-input options imposed on a video frame. or captured video data, for example, as part of a video telephony session. As will be described in more detail, such content in real time can in some cases include a video frame of user input options available to a user to select. In some cases, audio / video data 121 may include video frames that are a combination of different types of content, such as a movie footage or a program that has user-input options imposed on a video frame. or captured video data, for example, as part of a video telephony session. As will be described in more detail, such content in real time can in some cases include a video frame of user input options available to a user to select. In some cases, audio / video data 121 may include video frames that are a combination of different types of content, such as a movie footage or a program that has user-input options imposed on a video frame.

In addition to playing audio / video data 121 locally with the display 122 and the sound 123, the source audio / video source encoder 124 can encode the audio / video data 121, the transmitter / receiver unit 126 can transmit the encoded data on the communication channel 150 to the receiver terminal 160. The transmitter / receiver unit 166 of the receiver device 160 receives the coded data, and the audio / video decoder 164 decodes the encoded data and outputs the decoded data with the help of the display 162 and the speaker 163. Thus, audio and video data reproduced using

3

iA 109176 C2

the display 122 and the speaker 123 can be simultaneously played back by the display 162 and the speaker 163. The audio and video data can be arranged in frames, and the audio frames can be synchronized in time with video frames during playback.

An audio / video encoder 124 and an audio / video decoder 164 can implement any number of audio and video compression stanzas such as the IT-TN.264 standard, alternatively called MPE-4, Part 10, enhanced video encoding (AUC), or encoding standard High-Efficiency Video (NEUS), which has recently appeared, sometimes called the H.265 standard. You can also use many other types of compression methods that make up property or standardized. In general, the audio / video decoder 164 is configured to perform reciprocal audio / video encoder encoding operations. Though not shown in FIG. 1A, in some aspects, a 124 A / B encoder and an A4 / A decoder 164 can be combined with an encoder and audio decoder and may include appropriate blocks of the MiH-YuMih or other hardware and software provisioning to control the encoding as audio,

As will be described in more detail below, the A / S encoder 124 may also perform other functions of encoding to complement the implementation of the video compression standard as described above. For example, an A / S encoder 124 can add different types of metadata to A / D data 121 before transmission. / In-data 121 on the recipient device 160. In some cases, the A / D data 121 may be stored on or received in the source device 120 in a coded form, and thus do not require additional compression using the A / V encoder 124.

Though FIG 1A shows a communication channel 150 that transports useful audio data and useful video data separately, it should be understood that in some cases, useful video data and useful audio data can be part of a common data stream. If applicable, the MiX-UEMIH blocks may correspond to the protocol of the multiplexer ITI-H.223 or other protocols such as the protocol for user datagrams (IEURs). Audio / video encoder 124 and audio / video decoder 164 may be implemented as one or more of the microprocessors, digital signal processors (processorsδδ), specialized integrated circuits (A8IS circuits), user-programmable gate arrays (ERC matrices), discrete logic, software, hardware provision , software hardware or any combination of them. Each audio / video decoder 124 and an audio / video decoder 164 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODECA). Thus, each source device 120 and recipient device 160 may comprise specialized machines configured to perform one or more methods of this disclosure.

The display 122 and the display 162 may include any plurality of video output devices such as an electron beam tube (CPT), a liquid crystal display (LCD), a plasma display, an LED display (LED), an organic LED display (OLE), or another display device . In these or other examples, displays 122 and 162 may be emitting displays or skip displays. The display 122 and the display 162 may also be touch screens, so that they are both input devices and display devices at the same time. Such touch screens can be capacitive, resistive or other type of touch panel, which allows the user to provide custom data to the device.

The speaker 123 may include any plurality of audio output devices such as headphones, a single speaker system, a speaker system, or a surround sound system. Additionally, although the display 122 and the speaker 123 are shown as part of the source device 120, and the display 162 and the speaker 163 shown as part of the recipient device 160, the source device 120 and the receiver device 160 may actually be a system of devices. As an example, the display 162 may be a television receiver, the speaker 163 may be an ambient audio system, and the decoder 164 may be part of an external unit, connected or wired, or wirelessly, with a display 162 and a speaker 163. In other cases, the receiver device 160 may be the only device, such as a tablet or smartphone. In still other cases, the source device 120 and the receiver device 160 are similar devices, for example, both are smartphones, tablet PCs, etc. In this case, one device can operate as a source, and the other can operate as a receiver. These lists maybe even completely changed in subsequent communication sessions. In other cases, the device source may contain a mobile device such as a smartphone, laptop or tablet computer, and the recipient device may contain a more stationary device (for example, a power connector), in which case the source device can provide audio and video data for presentations to a large group people using the recipient's device. and the other may work as a recipient. These lists maybe even completely changed in subsequent communication sessions. In other cases, the device source may contain a mobile device such as a smartphone, laptop or tablet computer, and the recipient device may contain a more stationary device (for example, a power connector), in which case the source device can provide audio and video data for presentations to a large group people using the recipient's device. and the other may work as a recipient. These lists maybe even completely changed in subsequent communication sessions. In other cases, the device source may contain a mobile device such as a smartphone, laptop or tablet computer, and the recipient device may contain a more stationary device (for example, a power connector), in which case the source device can provide audio and video data for presentations to a large group people using the recipient's device.

4

iA 109176 C2

The transmitter / receiver unit 126 and the transmitter / receiver unit 166 may include intercoms, filters, amplifiers and other components designed to modulate the signal, as well as one or more antennas and other components designed for transmitting and receiving data. The communication channel 150 in general, represents any suitable communication media or collection of various communication media for transmitting video data from the source device 120 to the receiver device 160. The communication channel 150 is, of course, a communication channel with respect to a short range, similar to Shri R, BIA, and so on. However, the communication channel 150 is not necessarily limited in this respect and may contain any wireless or wired communication media, such a radio frequency spectrum (RF) or one or more physical transmission lines, or any combination of wireless and wired media. In other examples, the communication channel 150 may even be part of a packet network, such as a wired or wireless LAN, a large-scale network or a global network, such as the Internet. Additionally, the communication channel 150 may be used by the source device 120 and the receiver device 160 to form a peer-to-peer communication link. The 120 source and receiver device 160 may be connected to the communication channel 150 using a data transmission protocol, such as the standard of the standards group, and the IEEE 802.11. The 120 source and recipient device 160 may, for example, communicate in accordance with the Shi Rai Uyghsya standard so that the source device 120 and the receiver device 160 communicate directly with each other without the use of an intermediary, such as wireless access points or the so-called "hot spot". The device 120 of the source and the receiver unit 160 may also set the tunneling of the direct link (TCH5) to avoid or reduce the overload of the network. A custom disclosure may from time to time be described in relation to Shire, but it is considered that aspects of these methods may also be compatible with other data transmission protocols. By way of example, and not limitation, wireless communication between the source device 120 and the receiver device can use methods for orthogonal multiplexing with frequency division multiplexing (OBMs). Also, a large variety of other wireless methods may be used, including, but not limited to, multiple access time-division channels (TOMA), Multi-frequency Frequency Division (MIMO), Multiple Code Distribution (COMA) or any combination of OBOM, ROMA, TOMA and / or COMA. Shi Rie Oigesi and T08 are intended for establishing communication circuits at relatively short distances. A relatively short distance in this context maybe, for example, less than 70 meters, although in a noisy or creating barrier in the surrounding environment, the distance between the devices may be shorter, for example, less than 35 meters.

In addition to decoding and reproducing data received from the source device 120, the recipient device 160 may also accept the input of user data from the user input device167. The user input device 167 can, for example, be a keyboard, mouse, trackball or touch pad, a touch screen, a voice recognition module, or any other such user-driven device. The iRiM 168 formats the user input commands received by the user input device 167 into a data packet structure that the source device 120 can interpret. Such data packets are transmitted by the transmitter / receiver 166 to the receiver 120 on the communication channel 150. The transmitter / receiver unit 126 receives data packets, the A / ν control module 125 controls the packet data parsing, to interpret the user input command that was received by the user input device 167. Based on the command received in the data packet, the A / ν control module 125 may change the content encoded and transmitted. Thus, the user of the recipient device 160 can manage useful audio data and useful video data transmitted by the source device 120 remotely and without direct interaction with the source device 120. Examples of the types of commands that the recipient device 160 of the receiver can transmit to the source device 120 include commands for rewinding, fast rewinding, termination and playback of audio and video data, as well as commands for resizing the image, rotation, scrolling, etc. Users can also make selections from the options menu, for example, and pass the selection back to the source device 120.

Additionally, users of the recipient device 160 have the ability to run and manage the attachments on the source device 120. For example, the user of the recipient device 160 has the ability to start the photo editing application stored on the source device 120, and use this application to edit the photo locally stored on the source device. The recipient device 160 may provide the user with user experience that is viewed and perceived as a photo that is edited locally on the receiver device 160,

5

iA 109176 C2

while in fact the photo is edited on the source device 120. Using such a configuration, the user of the device is able to effectively use the capabilities of one device for use with several devices. For example, the 120 device source may be a smartphone with high memory and processing capabilities at a high level. The user of the 120 device source can use the smartphone in all settings and situations in which smartphones are commonly used. However, when viewed, the movie user may want to watch a movie on a device with a large display screen; in this case, the recipient device 160 may be a tablet computer or even a larger display device or TV. If you want to send or reply to an e-mail message, the user may want to use a keyboard device, in this case, the recipient device 160 may be a laptop. In both cases, most of the processing can still be performed by the source device 120 (a smartphone in this example) even though the user interacts with the recipient's device. In this particular operating context, due to the large amount of processing performed by the source device 120, the receiver device 160 may be a cheaper device with fewer resources than if the recipient device 160 was requested to perform processing performed by the source device 120. Like the source device and the receiver device, be able to accept user input (for example, touch screen commands) in some examples,

In some configuration, the A / ν control module 125 may be a process of an operating system executed by the operating system of the source device 125. However, in other configurations, the A / ν control module 125 may be an application software process running on a source device 120. In this configuration, the user input command can be interpreted by the software process so that the user of the recipient device 160 interacts directly with the application operating on the source device 120 in comparison with the operating system operating on the source device. By directly communicating with the application, compared to the operating system, the user of the recipient device 160 may have access to a library of commands that are not specific to the operating system of the source device 120. Additionally,

The source device 120 may respond to user inputs applied to the receiver's wireless device 160. In such an interactive setup, the user inputs applied to the wireless device 160 of the recipient may be sent back to the wireless source of the display through the communication link 150. In one example, the architecture of the reverse channel, also called the user interface back channel, can be implemented to allow the receiver device 160 to transmit the user inputs applied at the receiver device 160 to the source device 120. The reverse-channel architecture may include an upper-level message for the transport user inputs, and lower-level frames for reconciling the capabilities of the user interface in the recipient device 160 and the source device 120. iiVs may be located above the transport layer of the Internet Protocol (IP) between the recipient device 160 and the source device 120. Thus, iiVS can be higher than the transport level in the communication model of the interaction of open systems (OZI). In one example, the connection O8I includes in these levels (1 - physical, 2 - data transmission line, 3 - network, 4 - transport, 5 - session, 6 - representations, 7 - applications). In this example, the location above the transport layer uses levels 5, 6 and 7. To facilitate the reliable transmission and consistent delivery of data packets containing data on the input of user data, iiVs can be configured working on other packet data protocols such as transmission control / protocol (TCP / IP) or custom datagram protocol (UDP). Both the ANDSD can work in parallel in the O8I architecture. The TCP / IP can allow the recipient device 160 and the source device 120 to implement retransmission methods in the event of packet loss.

In some cases there may be a discrepancy between the user input interfaces located in the source device 120 and the receiver device 160. To solve the potential problems caused by such a discrepancy and to improve the good user experience in such circumstances, the coordination of the features of the user input interface may have a location between the source device 120 and the recipient device 160 prior to the establishment of the communication session at different times during the communication session. As part of this alignment process,

6

iA 109176 C2

the source device 120 and the receiver device 160 may agree on a coherent resolution of the screen. When the recipient device 160 transmits coordinate data associated with the user input, the recipient device 160 can scale the coordinate data received from the display 162 to match the agreed screen resolution. In one example, if the receiver device 160 has a resolution of 1280x720 and the source device 120 has a resolution of 1600x900, the devices may, for example, use 1280x720 as their coherent resolution. A harmonized distinction may be selected based on the distinction between the receiver 160, although the distinction of the source device 120 may also be used. or some other distinction. In an example that uses a recipient device with a resolution of 1280x720, the recipient device 160 can scale the received x-coordinates by using the coefficient 1600/1280 prior to transmitting coordinates to the source device 120, and, similarly, the receiver 160 can scale the received co-coordinates with 900/720 before transmitting the coordinate to the source device 120. In other configurations, the source device 120 can scale received coordinates to a coherent differentiation. Zooming may eitherincrease or reduce the range of coordinates based on whether the recipient device 160 uses a display with a higher resolution than the source device 120, or vice versa. In other configurations, the source device 120 can scale received coordinates to a coherent differentiation. Zooming may eitherincrease or reduce the range of coordinates based on whether the recipient device 160 uses a display with a higher resolution than the source device 120, or vice versa. In other configurations, the source device 120 can scale received coordinates to a coherent differentiation. Zooming may eitherincrease or reduce the range of coordinates based on whether the recipient device 160 uses a display with a higher resolution than the source device 120, or vice versa.

Additionally, in some cases, the distinction in the recipient device 160 may change during the communication session, potentially creating a discrepancy between the display 122 and the display 162. To enhance user experience and ensure proper functionality, the source / destination system 100 may implement methods for reducing or preventing non-matching user interaction. by implementing methods for normalizing the screen. The display 122 of the source device 120 and the display 162 of the recipient device 160 may have different distinctions and / or different aspect ratios. Additionally, in some configuration settings, the user of the recipient device 160 may have the ability to resize the screen display for the video data received from the source device 120 so that the video data received from the source device 120 is played back in the window, which covers less than the entire display 162 of the receiver device 160. In other exemplary settings, the user of the recipient device 160 may have a content view option either in the horizontal orientation mode or in the portrait mode, each of which has unique coordinates and different aspect ratios. In such situations, the coordinates associated with the user input received by the receiver 160, such as the coordinate of the location of the mouse click or the touch event, may not be able to be processed by the source device 120 without modifying the coordinates. Accordingly, the methods of this disclosure may include displaying the user input coordinate received in the recipient device 160 in coordinates associated with the source device 120. This reflection is also called normalization in the given description, and,

Custom inputs received by the recipient device 160 may be adopted by the module 167 and at the driver level, for example, and sent to the operating system of the recipient device 160. The operating system in the recipient device 160 may receive coordinates (Χδ_ΝΝΚ, Uzim) associated with where the user has been placed on the surface of the display. In this example (χ_3YNC, Uz ^ k) can be the coordinates of the display 162, in which there was a click on the mouse or a touch event. The screen window of the display reproduced on display 162 may have a longevity of the x coordinates (b_oot) and the width of the y coordinate (M_οnν) that describe the size of the screen display screen. The screen window of the display may also have the coordinate of the upper left corner (a_oot, B ^), which describes the location of the screen of the display screen. Based on b_oot, OT_0ТО and the upper left coordinate (a_oot, B_oh), a portion of the display 162 covered by the screen of the display can be defined. For example, the right upper corner of the screen window can be located in the coordinate (as_ATfrom + b_ATfrom b_oot), the lower left corner of the screen of the display screen can be located in the coordinate (a_oot, B_0ТО+ OT_0ТО), and the right bottom corner of the screen of the display can be located in the coordinate (a_ATfrom + b_ATfrom b_0ТО+ OT_0ТО) The recipient device 160 can process the input as input and / or AV, if this input is taken in coordinate within the display window. In other words, the introduction with associate coordinates (χ_3YNC, at₃₁you_K) can be processed as an input of iiVs if the following conditions are met:

Νν- ^ ΙΝΚ-Ζνν + ίονν ⁽¹⁾

^Bote-Uzicek-loto + OT_oh⁽²⁾

After determining that the user input is the input of iiVs, the coordinates associated with this input can be normalized with iRiM 168 prior to being transmitted to device 120

7

iA 109176 C2

sources. Inputs, which are defined as outside of the screen of the display screen, can be processed locally by the recipient device 160 as the input of the non-WSV.

As mentioned above, the normalization of the input coordinates can be based either on the source or on the recipient. When implementing a normalized device based on the receiver, the source device 120 can send the supported display resolution (L_3КС, OT_3P&gt;_WITH) for the display 122, or with the video data, or irrespective of the video data, to the receiver device 160. Support non-distinction of a display, for example, can be transmitted as part of a session of coordination options or may be transmitted at another time during a communication session. The recipient device 160 can determine the resolution of the display (1_3YNC, Μ_3YNC) for display 1b2, distinguishing from the screen of the display (b_at^, OTH_0OT) for a window that displays the content received from the source device 120 and the coordinate of the left upper corner (and_oot, B_oot) for the screen of the display. As a descriptive note, when the coordinate (χ_3YNC, γ_3YNC) corresponding to the user input, is determined in conjunction with the screen of the display screen, the operating system of the recipient device 160 may reflect the coordinate (χ_3YNC, at₃₁ık) at output coordinates (x_with^_WITH, at₃^_WITH) using transformation functions. Exemplary transformation functions for the transformation (χ_3YNC, Y_ZI-|_K) in (X₃K_WITH, at₃to_WITH) can befollowing:

Khzx⁼(Hazyak-aoot) * (bzks / bost) (3)

UX⁼(United States) * (OTzks / OToots) (4)

Thus, when transmitting a coordinate corresponding to a custom input received, the recipient device 160 can transmit the coordinate (x_with^_WITH, at₃^_WITH) for the user input taken in (χ_3YNC, at_WITHand_K) · As will be described in more detail below, the coordinate (x_with^_WITH, at₃^_WITH), for example, may be transmitted as part of a data packet used to transmit the user input received by the recipient device 160 to the source POI AID device 120. In other parts of this disclosure, in which the input coordinates are described as included in the data packet, these coordinates can be converted to the output coordinates as described above in cases where the source / receiver system 100 implements the base of the recipient of normalization.

When the source / receiver system 100 implements a source-based normalization, for user inputs defined by inputs iiVs, compared to local inputs (that is, within the display screen window opposite the outside of the screen window), the calculations described above may be performed on the device 120 sources instead of receiver device 160. To facilitate such calculations, the recipient device 160 can transmit to a value of 120 source values ​​for b_0OT, and location information for the window

display screen (for example, a_0OT, B_0OT), and coordinates for (χ_3YNC, u ^ iU- By using these transmitted values, the source device 120 can determine the value for (χ_{3 &amp;} in₃^_WITH) according to Equations 3 and 4 presented above.

In other implementations based on the recipient of normalization, the recipient device 160 can transfer the coordinates (χ_0№, at_oot) for user input that describes where there is a custom input event within the window screen of the display, as opposed to where the display 162 has a user input. In such a realization of the coordinates (χ_0№, at_0OT) can be transmitted to the source device 120 along with the values ​​for the OT_0OT) Based on these assumed values, the source 120 can be determined (χ_{3 &amp;} in₃^_WITH) according to the following conversion functions:

^xfrom^_ho '&gt; /' / fzts ^ here) ⁽⁵⁾

ⁱⁿ3КС^{= y}0OT *^(OT3КС^{/ OT}0OT^{) (6)}

The recipient device 160 can determine χ_0№ and at_oot based on the following functions:

Θθνν⁼Χ3ΙΝΚ-3θ \ / ν (7)

(8)

When this disclosure describes the transfer of coordinates associated with the user input, in a data packet, for example, the transmission of these coordinates may include a payee based or source-based normalization as described above and / or may include any additional information required for performance based on the recipient orbased on the source of normalization.

8

iA 109176 C2

iiVs can be designed to transport various types of user-generated data, including cross-platform data input of user data. For example, the source device 120 may operate the ίδδ® operating system, while the receiver 160 is controlled by another operating system such as Apbogio® or OTipBase®. Regardless of the platform, ipI 168 can encapsulate the custom input received in a form that is understood by the A / ν control module 125. A number of different types of user input formats can be supported by the IIS in such a way as to allow many different types of source devices and recipients to use the protocol, regardless of whether the device and source of the receiver on different platforms. Common format formats can be defined, and platform-specific input formats can be supported,

In an example on FIG. 1A, the source device 120 may comprise a smartphone, tablet PC, laptop, desktop computer, an OTi-Ei TV or any other device capable of transmitting audio and video data. The recipient device 160 may similarly contain a smartphone, a tablet PC, a laptop, a desktop computer, a TV with the support of OTi-Ei, or any other device capable of receiving audio and video data and accept data from the user's input. In some cases, the receiver device 160 may include in the system devices such as the display 162, the speaker 163, the device 167 and the encoder 164 A / ν of all parts of the individual but interacting devices. Device 120 source can similarly be a system of devices, and not the only device.

In the present disclosure, the term "source device" is generally used to refer to an audio / video device and the term "receiver device" is generally used to refer to a device that receives audio / video data from a device source. In many cases, the source device 120 and the receiver device 160 may be analogous or identical devices, with one device operating as a source, and the other operating as the receiver. In addition, these lists can be reversed in different sessions of communication. Thus, the recipient's device in one communication session may become the device's sources in the subsequent communication session or vice versa.

FIG 1B is a block diagram illustrating a sample source / receiver system 101 that can realize the methods of this disclosure. The source / recipient system 101 includes a source device 120 and a receiver device 160, each of which may function and work as described above for the FID. 1A. The source / recipient complement system 101 includes a recipient device 180. Similar to the recipient device 160, as described above, the receiver device 180 can receive audio and video data from the source device 120 and transmit the user commands to the source device 120 via the iiVs installed. In some configurations, the recipient device 160 and the recipient device 180 can operate independently of each other, and the output of the audio and video data in the source device 120 may be simultaneously output to the receiver 160 and the receiver device 180. In additional configurations, the recipient device 160 may be the primary receiver device and the receiver device may be a secondary recipient device. In such an exemplary configuration, the recipient device 160 and the recipient device 180 can be connected, and the recipient device 160 may display video data, while the receiver 180 outputs corresponding audio data. Additionally, in some configurations, the recipient device 160 can output the transmitted video data only when the recipient device 180 outputs transmitted aids. while the receiver 180 outputs the corresponding audio data. Additionally, in some configurations, the recipient device 160 can output the transmitted video data only when the recipient device 180 outputs transmitted aids. while the receiver 180 outputs the corresponding audio data. Additionally, in some configurations, the recipient device 160 can output the transmitted video data only when the recipient device 180 outputs transmitted aids.

FIG 2 is a block diagram illustrating one example of a source device 220. The device 220 of the source may be a device similar to the source device 120 on FIG. 1A, and can work in the same way as the source device 120. The source device 220 includes a local display 222, a loudspeaker 223, processors 231, memory 232, a transport unit 233, and a wireless modem 234. As shown in FIG. 2, the source device 220 may include one or more processors (i.e. processor 231) that encode and / or decode A / V data for transport, storage, and display. The Α / ν data can, for example, be stored in memory232. The memory 232 can store the entire Α / ν file or may contain a smaller buffer, which simply stores the part of the Α / ν file, for example, transmitted by a stream from another device or source. The transport unit 233 can process encoded Α / ν data for the network transport. For example, the encoded A / V data can be processed by the processor231 and encapsulated by the transport unit 233 into the network access level blocks (AAAs) for network communication. The KIAI_ blocks can be sent by the wireless modem 234 to the receiver's wireless device using a network connection. Wireless modem

9

iA 109176 C2

234 can, for example, be a modem OTi-R, configured to implement one of the group standards IEEE 802.11.

The source 220 can also process locally and display Α / ν-data. In particular, the display processor 235 can process the video data that should be displayed on the local display 222, the audio processor 236 can process the audio data for outputting to the speaker 223.

As described above with references to the source device 120 according to FIG. 1A, the source device 220 may also receive user input commands from the receiver device. Thus, the wireless modem 234 of the source device 220 receives packets of encapsulated data, such as blocks NΑ6, and sends blocks of encapsulated data to the transport unit 233 to decapulate. For example, the transport unit 233 may pull data packets from blocks NAΑ6 and the processor 231 can parse the data packets to extract user input commands. On the basis of user-input commands, processor 231 can configure encoded Α / ν-data transmitted by the source device 220 to the device of the receiver. Thus, the functionality described above with respect to the control module A / ν according to FIG. 1A may be implemented either completely

Processor 231 according to FIG. 2 generally represents any large variety of processors, including, but not limited to, one or more digital signal processors (processors δδ), general purpose microprocessors, specialized integrated circuits (ΑδΙΟ schemes), user-programmed gate matrices (matrices RRSA), other equivalent Integral or discrete logic circuits or some combination of them. Memory 232 according to FIG. 2 may comprise any large variety of energy-independent or non-autonomous memory, including, but not limited to, an operational memory device (RAM) such as a synchronous dynamic operational memory device ^ URAM), a permanent storage device (ROM), non-volatile operational storage device (NVRAM), electrically erased permanent storage device (EERRO), flash memory, etc. Memory 232 may include a computer-readable storage medium for storing audio / video data, as well as other types of data. Memory 232 can further store the commands and program code executed by the processor 231 as part of the execution of the various methods described in this disclosure.

FIG 3 shows an example of a recipient device 360. The recipient device 360 ​​may be a device similar to the receiver device 160 on the FIG. 1A, and may work in the same way as receiver device 160. The recipient device 360 ​​includes one or more processors (i.e., processor 331), memory 332, a transport unit 333, a wireless modem 334, a display processor335, a local display 362, an audio processor 336, a speaker 363, and a user input interface 376. The recipient device 360 ​​receives 334 blocks of encapsulated data sent from the source device in a wireless modem. A wireless modem 334, for example, may be an OTiRi modem configured to implement another standard from the IEEE 802.11 grouping standards. The transport unit 333 can decapitate blocks of encapsulated data. Example, the transport unit 333 can extract encoded video data from blocks of encapsulated data and send encoded Α / ν data to processor 331, which should be decoded and reproduced for output. The display processor 335 can process the decoded video data that must be displayed on the local display 362, and the audio processor 336 may process the decoded audio data for outputting to the speaker 363.

In addition to playing audio and video data, the recipient's wireless device 360 ​​may also accept user input data via the user interface 376. The user input interface 376 can represent any of a number of custom input devices included, but not limited to, the interface display, the keyboard, the mouse, the voice command module, the gesture capture device (for example, the camera-based capture capability) or any -any other than a number of custom input devices. The custom input received from the user input interface 376 may be processed by the processor 331. This processing may include the generation of data packets that include an accepted user input command according to the methods described in this disclosure.

Processor 331 according to FIG. 3 may contain one or more broadband processors such as one or more digital signal processors (DDR processors), general-purpose microprocessors, specialized integrated circuits (ΔδIs circuits), user-programmed gate matrices (matrix PPCA), other equivalent integral

10

iA 109176 C2

or a discrete logic circuit or some combination of them. Memory 332 according to FIG. 3 may include any large variety of energy-independent or non-volatile memory, including, but not limited to, an operational memory (RAM) such as a synchronous dynamic memory storage device (RAM), a permanent memory device (ROM ), a non-volatile operational memory device (ΝνΡΑΜ), an electrically erased programmable permanent memory (EERRO), a flash memory, etc. Memory 232 may include a computer-readable memory storage device for audio storage / you deodan, as well as other types of data. Memory 332 can further store the commands and program code executed by the processor 331 as part of the implementation of various methods,

FIG 4 shows a block diagram of an exemplary transmitter system 410 and a receiver system 450 that can be used by the transmitter / receiver 126 and the transmitter / receiver 166 according to the FIG. 1A for communication over the communication channel 150. In the transmitter system 410, traffic data for a number of data streams is output from the data source 412 to the data transfer processor 414 (TX). Each data stream can be transmitted over an appropriate transmission antenna. The TX Transmitter 414 TX format, encode, and rotate traffic data for each data stream based on the particular encoding scheme selected for this data stream.

Encoded data for each data stream can be multiplexed to pilot data using methods of multiplexing with orthogonal frequency division of the multinationals (ΟΕΜΜ). A large variety of other methods of non-wire communication may also be used, including, but not limited to, multiple access with time division of channels (TIMA), multiple frequency channel access (EYMB), multiple code-division channel access (UMBA) or any combination of EOI, EYMA, TYMA and / or EO.

According to FIG. 4, pilot data is generally a known data template that is processed in a known manner and can be used in the receiver system to evaluate the channel response. Multiplexed pilot data and encoded data for each data stream are thenmodulated (eg, displayed in symbols) based on a specific Modulation schemes (for example, binary phase manipulation (BP5K), quadrature phase manipulation (OP5K), M-P5K or M-APM (quadrature amplitude modulation), where M can be a power of two) selected for this data stream to give modulation symbols. Data rates, coding and modulation for each data stream can be determined by commands executed by the processor 430 which can be connected to memory 432.

Subsequently, the modulation symbols for the data streams are output to the 420 MIMO TX processor, which can further process the modulation symbols (for example, for EOI). Then, the processor 420 MIO TX can transmit data N_τ symbolic flows of modulation in N_τtransmitters (ТМФР) 422а-4221. In some aspects, the data processor 420 MIMO TX transmits the weight of the formation of the diagram to the data stream symbol and to the antenna from which the symbol is transmitted.

Each transmitter 422 can receive and process an appropriate symbol stream to output one or more analog signals, and additionally cause analog signals to be made (for example, amplify, filter, and convert with increasing frequency) analog signals to produce a modulated signal suitable for transmission over the MIMO channel. Then N._τ Modulated signals from transmitters 422a-4221 are transmitted from N_τ antennas 424a-4241, respectively.

Modulated signals transmitted to the receiver system 450 are received by N_κ antennas 452a-452g, and the received signal from each antenna 452 is given to the corresponding receiver (ΡΟνΡ) 454a-454g. Receiver 454 leads to the necessary conditions (for example, filters, amplifies and converts with a decrease in frequency), the corresponding received signal, translates the signal brought to the necessary conditions a digital form to provide sampling, and further processes the samples for the output of the corresponding "adopted" character stream.

Then the 460 RX processor receives and processes the data reception N_κ Accepted symbolic streams from N_κ receivers 454 based on a specific method of processing the receiver for issuing N_τ"detected" character streams. Then, the 460 PC processor accepts data demodulating, executing reverse alternating, and decoding each detected symbol stream to restore data traffic for the data stream. Processing by the 460 PC processor is complementary to the one that is completed by the 420 MIMO TX processor and the TX processor 414 TX in the transmitter system 410.

The processor 470, which can be connected to memory 472, periodically determines which use the matrix of the previous coding. Message backlink can

11

iA 109176 C2

contain different types of information regarding the link and / or received data stream. Then the feedback link is processed by the TX data processor 438, which also receives traffic data for a number of data streams from the data source 436, modulated by the modulator 480, is reduced to the required conditions by transmitters 454a-454g and transmitted back to the transmitter system 410.

In the transmitter system 410, the modulated signals from the receiver system 450 are received by antennas 424, are brought to the required conditions by receivers 422, demodulated by the modulator 440, and processed by the PC receive processor 442 to receive the feedback received by the receiver system 450. Processor 430 then determines which one to use a matrix of pre-coding to determine the weight of the chart-orientation, then handles the elongated message.

FIG 5A is a block diagram illustrating an exemplary sequence of message transmission between the source device 520 and the receiver device 560 as part of the negotiation session of the capabilities. Matching capabilities may take place as part of a larger process of establishing a communication session between the source device 520 and the receiver device 560. This session, for example, can be established with the help of OTi-Gi Yuigesi or T0b8 as the standard connection underlying. After the OTi-Gi Yuigesi or T0b8 session is established, the recipient device 5b may initiate the TCP connection to the source device 520. As part of establishing a TCP connection, a control port can be set up that manages the real-time streaming protocol (PTTP) to control the communication session between the source device 520 and the receiver device 560.

The 520 source can generally work in the same way as described above for the source device 120 according to FIG. 1A, and the receiver device 560 may generally work in the same manner as described above for the receiver device 160 according to FIG. 1A. Once the receiver 520 and receiver device 560 is able to connect, the source device 520 and the recipient device 560 can define a set of parameters that should be used for their subsequent communication session as part of the exchange of matching capabilities.

The 520 source and the recipient device 560 can match the possibilities through the sequence of messages. Messages, for example, may be messages in the real-time streaming protocol (PTTP). At any stage of the negotiation, the recipient of the request for reply PTPP can respond using the response PT8P, which includes a status code PTPP, different from PT8P OK, when the messaging can be repeated with another set of parameters, or may be completed session matching capabilities.

The source device 520 may send a first message (OPTIONPT8P request message (PT8R ORT1OY8)) to the receiver device 560 to determine a set of PTRs that support the receiver device 560. Upon receipt of the first message from the device 520, the receiver device 560 may respond to a second message (message responses PT8R ORT1OY8) that lists the PT8R methods supported by the receiver device 560. The second message may also include the status code of PT8P OK.

After sending a second message to the source device 520, the receiver device 560 may send a third message (request message PT8R ORT1OY8) to determine the selection of PTRs that support the source device 520. After receiving the third message from the recipient device 560, the source device 520 may respond to a fourth message (a response message PT8R ORT1OY8) that lists the methods of PT8R supported by the source device 520. The fourth message may also include the status code of the TRPROC.

After sending the fourth message, the source device 520 may send a fifth message (the message of the SET_RAPAMETPPTPP request) to determine the list of options that are of interest to the source device 520. The recipient device 560 can respond to the sixth message (message of the answer SET_RAPAMETEP PTPP). The latter message may contain the status code of the PTZP. If the PTQP status code is OK, then the sixth message may also include the response parameters for the parameter determined by the fifth message supported by the recipient device 560. The recipient device 560 may ignore the parameters in the fifth message, which does not support the receiver device 560.

Based on the sixth message, the source 520 may determine an optimal set of parameters to be used for the communication session and may send the seventh message (message of the 8ET_RAPAMETPPTPP request) to the receiver device 560. The seventh message may include a set of parameters to be used during a session communication between the source device 520 and the receiver device 560. The seventh message may include a myrib-pseudoportal-game1 that describes a universal resource identifier (SPI),

12

iA 109176 C2

which should be used in the request for setting the PT5R to establish a communication session. This myrib-rgezepia-iop-igi identifies the IRI that can use the recipient device 560 for later communications when exchanging session setup. The values ​​of the ibI-iRi and the iB-ij11 defined in this parameter may correspond to the value of the gGr-rGiO and the value of the gIR-roGPv-i-cIiPi-gIr-roGIZ in the seventh message. RTR in this case, in general, belongs to a real-time protocol that can work on top of the device.

After receiving the seventh message, the recipient device 560 can respond to eighth message with the status code RT-5R, indicating whether the setting of the parameters was successful, as defined in the seventh message. As mentioned above, the roles or device of the source and device of the receiver can be reversed or changed in different sessions. The order of the messages that the communication session is set up may in some cases define a device that works as a source and determine the device that works as the recipient.

FIG 5B is a block diagram illustrating another exemplary message transmission sequence between the device 560 of the source and the receiver device 520 as part of the negotiation session of the capabilities. The sequence of message transfer according to FIG. 5B is intended to provide a more detailed view of the sequence of transmission described above for FIG. 5A. PIFIG 5 The message "1b. OET_RAMAMETER ΕΕδΡΟΝδΕ" shows an example message that identifies a list of supported input categories (for example, common and NUUs), and multiple sets of supported input types. Each of the supported entry categories from the list of supported input categories has an associated list of supported types (e.g., depegis_sar_Іvi5I and iibs_sar_Іi5i). At FIG 5V message "2a. 5ET_RAMAMETER REOiEZT" is an example of the second message, which identifies the second list of supported categories of input (for example, general and NUS) and a set of other lists of supported types. Each of the supported input categories from the second list of supported input categories has an associated second list of supported types (e.g., depegis_sar_Ii5I and iibs_sar_IzI). The message "1b. OET_RAMAMETER ΡΕδΡΟNδΕ" identifies the input categories and input types supported by the recipient device 560. The message "2a. 5ET_RARAMETERREOiEZT" identifies the input categories and input types supported by the source device 520, but may not be a complete list of all input categories and input types supported by the source device 520. Instead, the message "2a. 5ET_RARAMETERREOiEZT" can identify only those categories of input and type of input,

FIG 6 is a conceptual diagram illustrating one example of a data packet that can be generated by the receiver device and transmitted to the source device. Aspects of the package of 600 data will be explained with references to FIG. 1A, but the methods discussed may apply to additional types of source / recipient systems. The data packet 600 may include a data packet header 610 with further data of 650 useful data. Useful dates 650 may additionally include one or more useful data headers (e.g., header 630 of useful data). The data packet 600 may, for example, be transmitted from the recipient device 160 to the FIG. 1A to the source device 120 so that the user of the recipient device 160 can control the audio / video data transmitted by the source device 120. In this case, the useful data 650 may include the user manual data received in the recipient device 160. Useful data 650 can, for example, identify one or more user commands. The receiver device 160 may accept one or more user commands and may, based on received commands, generate a data packet header 610 and useful data 650. Based on the content of the data packet header 610, data source 600, the source device 120 can perform a parsing analysis of useful data 650 to identify user input data taken in recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160. Useful data 650 can, for example, identify one or more user commands. The receiver device 160 may accept one or more user commands and may, based on received commands, generate a data packet header 610 and useful data 650. Based on the content of the data packet header 610, data source 600, the source device 120 can perform a parsing analysis of useful data 650 to identify user input data taken in recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160. Useful data 650 can, for example, identify one or more user commands. The receiver device 160 may accept one or more user commands and may, based on received commands, generate a data packet header 610 and useful data 650. Based on the content of the data packet header 610, data source 600, the source device 120 can perform a parsing analysis of useful data 650 to identify user input data taken in recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160. The receiver device 160 may accept one or more user commands and may, based on received commands, generate a data packet header 610 and useful data 650. Based on the content of the data packet header 610, data source 600, the source device 120 can perform a parsing analysis of useful data 650 to identify user input data taken in recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160. The receiver device 160 may accept one or more user commands and may, based on received commands, generate a data packet header 610 and useful data 650. Based on the content of the data packet header 610, data source 600, the source device 120 can perform a parsing analysis of useful data 650 to identify user input data taken in recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160. to identify the user input data received in the recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160. to identify the user input data received in the recipient device 160. Based on the user input data contained in the useful data 650, the source device 120 may in some way modify the audio and video data transmitted from the device 120 to the receiver device 160.

Used in this disclosure, the terms "perform syntactical analysis" and "conducting the parsing analysis" generally relate to the bitstream analysis process to extract data from the bit stream. After extraction, the data can be processed by the source device 120, for example. Extracting data may, for example, include an identification of how the formatted information in a bitstream. As will be described in more detail below, the data packet header 610 may define a standardized format known as device 120

13

iA 109176 C2

sources, and recipient device 160. Useful data 650, however, can be formatted one of many possible ways. By performing a parsing analysis of the data packet header 610, the source device 120 can determine how the useful data formats 650 are formatted, and thus the source device 120 can perform a parsing analysis of 650 data useful data to extract one or more custom instruction commands from the useful data 650. This can provide flexibility with respect to various types of useful data that can be maintained in the context of the recipient source. As will be described in more detail below, usefuldata 650 may also include one or more useful data headers, such as a header 630 of useful data. In such cases, the source device 120 may carry out a parsing analysis of the data packet header 610,

Diagram 620 is a conceptual description of how a data packet header 610 can be formatted. Numbers 0-15 in row 615 are intended to identify the bit position in the header 610 of the data packet and are not intended to actually represent the information contained in the data packet header 610. The data packet header 610 includes a field 621version, a timestamp flag 622, a reserved field 623, an input category 624, a length field625, and an additional time period field 626.

In an example on FIG. 6 The 621 version of the version is a 3-bit field that can specify a version of a particular data transfer protocol implemented by the recipient device 160. The value in the field 621version may inform the source device 120 of how to perform a parser analysis of the left side of the data packet header 610, as well as how to perform a parsing analysis of useful data 650. In an example to FIG. 6 field 621 version is a 3-bit field, which allows a unique identifier for eight different versions. In other examples, a larger or smaller number of bits can be allocated to the field 621 version.

In the example according to FIG. 6, the flag (T) 622 of the timestamp is a 1-bit field indicating whether the time field 626 is a time marker in the data packet header 610. The time marker field 626 is a 16-bit field containing a timestamp based on the multimedia data generated by the source device 120 and transmitted to the receiver device 160. The timestamp may, for example, be a sequential value assigned to video frames using the source device 120 before the frames are transmitted to the recipient device 160. Flag 622 of the timestamp may, for example, include "1" to indicate that there is a time marker field 626, and may include "0" to indicate that there is no time period field 626. After performing a parsing analysis of the data packet header 610 and determining what time field 626 is being used,

If available, the timestamp field 626 may include a timestamp for identifying the video frame that was displayed on the wireless device 160 of the recipient when the user input data 650 was received. The timestamp, for example, may be added to the video frame using the source device 120 before the source device 120 transmits the video frame to the receiver device 160. Accordingly, the source device 120 can generate a video frame and include a video frame as a metadata, for example, a timestamp. The 120 source can transmit a timed video frame to the recipient device 160, and the receiver unit 160 may display a video frame. While the frame of the video is displayed by the recipient device 160, the recipient device 160 may receive the user command from the user.

Upon receipt of a packet 600 of data with a timestamp field 626, which is in the header, the source wireless source 120 can identify the video frame displayed in the device 160 of the recipient, while the custom data input of the useful data 650 was received and to process the user input data based on frame content, identified by the time mark. For example, if the user input data is a touch command applied to the touch screen display or a mouse pointer, the source device 120 can determine the frame content that is displayed at the time the user applied the touch command to the display or clicked on the mouse. In some cases, the content of the frame can

14

iA 109176 C2

be necessary to properly process useful data. For example, custom input based on a user's touch or mouse clicks may depend on whether it is shown on the display when tapping or tapping. Tapping or pressing may for example correspond to a character or menu option. In cases in which the content of the display changes, the timestamp, which is in the timestamp field 626, may be used by the source device 120 to match touch or clicks with the correct symbol or menu option.

The 120 source may additionally or alternatively compare the timestamp in the field 626 of the timestamp with the time marking applied to the currently playing video frame. By comparing the time marker 626 of the time marker with the current time marker, the source device 120 can determine the time of propagation of the "there and irreversible" signal. The time for signal propagation there and vice versa corresponds to the amount of time that ends with the moment when the frame is transmitted by the source device 120, until the user input on the basis of this frame is taken back in the source device 120 from the receiver device 160. The time of signal propagation there and back can provide the source device 120 the indication of the system standby time, and, if the time of propagation of the signal there andmore greater than the threshold value, then the source device 120 may ignore the user input data contained in the data 650 useful data, according to the assumption that the input command was applied to the outdated frame of the display. When the signal frequency is there and inversely smaller than the threshold, the source device 120 can process the user input data and adapt the audio / video content transmitted in response to the user input data. Thresholds can be programmable, and different types of devices (or different recipient source combinations) can be configured to determine the various thresholds for the transmission and confirmation time that is acceptable. the device 120 of the source can process the user input data and adapt the audio / video content transmitted in response to the user's input data. Thresholds can be programmable, and different types of devices (or different recipient source combinations) can be configured to determine the various thresholds for the transmission and confirmation time that is acceptable. the device 120 of the source can process the user input data and adapt the audio / video content transmitted in response to the user's input data. Thresholds can be programmable, and different types of devices (or different recipient source combinations) can be configured to determine the various thresholds for the transmission and confirmation time that is acceptable.

In an example on FIG. 6, the reserved field 623 is an 8-bit field, which does not include the information used by source 120 during the syntax analysis header 610 of the data packet and useful data 650. Future versions of a particular protocol (as identified in the field 621 version), however, can use the reserved field 623 when the source device 120 can use the information in the reserved field 623 for conducting a parsing analysis of the data packet header 610 and / or for performing a parsing analysis of the useful data 650. Reserved field 623 together with the 621 version potentially provides the ability to expand and add features to the format of the data packet without significantly altering the format and features that are already being used.

In an example on FIG. 6, the input category 624 is a 4-bit field for identifying the input category for the user input data contained in the useful data 650. The recipient device 160 may divide the user input data into categories to define the input category. The categorization of user input data may, for example, be based on the device from which the command was adopted or based on the properties of the team itself. The value category 624 of the input category, possibly together with the other information of the data packet header 610, identifies the source device 120 as useful data 650 is formatted. Based on this formatting, the source device 120 can execute a parsing analysis of the useful data 650 to determine the custom input that was received in the device 160 the recipient

Since Category 624 is an introduction to the FIG in the example. 6 is 4 bits, sixteen different categories of input may possibly be identified. One such input category may be a generic input format to indicate that the user inputs of useful data 650 are formatted using common informational elements defined in the protocol performed both by the source device 120 and the receiver device 160. The format-fed input, which will be described in more detail below, may use general information elements that take into account the user of the recipient device 160 to interact with the application-level source device.

Another such input category may be the format of the User Interface User Interface (NUU) command to indicate that the user input data of the useful data 650 is formatted based on the type of input device used to receive the input. Examples of device types include a keyboard, mouse, touch device input, joystick, camera, gesture capture device (such as device based camera) and a remote control. Other types of input categories that may be identified in the input category field 624 include an input format direction to indicate that the user data in the useful data 650 did not occur in the recipient device 160, or the operating system specific format and format of the voice command to indicate that Useful data 650 include a voice command.

15

iA 109176 C2

A field 625 of length can contain a 16-bit field to indicate the length of the data packet 600. Length, for example, may be specified in blocks of 8 bits. Since the syntax analysis of the data packet 600 via the source device 120 in the words of 16 bits, the data packet 600 can be filled in an integer of 16 bits. Based on the length contained in the field 625, the source 120 can identify the end of the useful data 650 (that is, the end of the 600 data packet) and the beginning of a new further packet of data.

Different size fields provided in the example of FIG. 6 are simply intended to be explanatory, and it is assumed that the fields can be implemented using other number of bits than the number shown on the FIG. 6. Additionally, it is also considered that the data packet header 610 may include less than all the fields discussed above, or may use additional fields not considered above. Indeed, the methods of this disclosurecan be flexible in relation to the actual format used for the various fields of data packets.

After performing the parsing analysis of the data packet header 610 to determine the formatting of the useful data 650, the source device 120 can perform a parsing analysis of the useful data 650 to define the user input command contained in the useful data 650. Useful data 650 may have its own header of useful data (heading 630 useful data) indicating the content of the useful data 650. Thus, the source device 120 can perform a parsing analysis of the useful data header 630 based on the syntactic analysis of the header 610 p data actuator, and then perform a syntactic analysis of the remainder of the useful data 650 based on the syntactic analysis of the header 630 useful data.

If, for example, the input category 624 of the data packet header 610 indicates that there is general input in the useful data 650, then the useful data 650 may have a generic output format. The device 120 of the source can thus perform a parsing analysis of useful data 650 according to the general input format. As part of the general input format, useful data 650 may include a sequence of one or more input events with each input event having its own title of the input event. Table 1, below, identifies the fields that can be included in the title of the input.

Table 1

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Value</p></td></tr><tr><td><p>Rodovaya Yu</p></td><td><p>1</p></td><td><p>See Table 2</p></td></tr><tr><td><p>Length</p></td><td><p>2</p></td><td><p>The length of the following fields in octets</p></td></tr><tr><td><p>Description</p></td><td><p>Variable</p></td><td><p>Details of user inputs. See Tables</p></td></tr>

The Identification field (UI) of the general input event (IE) identifies the identification of the general input event to identify the type of input. The field Y of the general IE can, for example, be one octet in length and may include the identifier selected from Table 2 below. If, as in this example, the field U of the general IE is 8 bits, then 256different types of entries (identified by 0-255) can be identified, although not all 256 identifications necessarily require an associated type of input. Some of the 256 may be reserved for future use with future versions regardless of the protocol implemented by the recipient device 160 and the source device 120. In Table 2, for example, identifiers I0 9-255 of the general IE do not have associated typing types, but they may be prescribed for future reference types.

The length field in the entry event header identifies the length of the description field, while the field description includes information elements that describe the user input. The formatting of the description field may depend on the type of input identified in the General IE field. Thus, the source device 120 can perform a syntactic analysis of the content of the description field based on the type of input identified in the field U of the general IE. On the basis of the field of the header of the input event, the source device 120 can determine the end of one event in a useful data 650 and the beginning of a new input event. As will be explained more detail below, one custom command can be described in useful data 650 as one or more input events. Table 2 provides an example of input types with a corresponding General IE that can be used to identify the type of administration.

16

iA 109176 C2

Table 2

<tr><td><p>Generalized IE</p></td><td><p>Type of input</p></td></tr><tr><td><p>0</p></td><td><p>Left mouse button down / tap down</p></td></tr><tr><td><p>1</p></td><td><p>Left mouse button up / tap up</p></td></tr><tr><td><p>2</p></td><td><p>Mouse movement / touch movement</p></td></tr><tr><td><p>3</p></td><td><p>Down key</p></td></tr><tr><td><p>4</p></td><td><p>Down key</p></td></tr><tr><td><p>5</p></td><td><p>Zoom the image</p></td></tr><tr><td><p>6</p></td><td><p>Vertical scrolling</p></td></tr><tr><td><p>7</p></td><td><p>Horizontal scrolling</p></td></tr><tr><td><p>8</p></td><td><p>Rotation</p></td></tr><tr><td><p>9-255</p></td><td><p>Reserved</p></td></tr>

Description fields associated with each input type may have a different format. Description Fields "Left mouse button down / tap down" events, "Left mouse up / touch up" events and events

5 "Mouse motion / touch motion" may, for example, include the information elements identified in Table 3 below, although other formats may also be used in other examples.

Table 3

<tr><td><p>Field</p></td><td><p>Size</p><p>(octet)</p></td><td><p>Notes</p></td></tr><tr><td><p>Number of pointer (Ν)</p></td><td><p>1</p></td><td><p>The number of indicators of the event of motion with the help of multipleindices. When set to 1, indicates the event of movement with the help of a single touch</p></td></tr><tr><td><p>For t = 1: N {</p></td><td><p></p></td><td><p></p></td></tr><tr><td><p>Yu pointer</p></td><td><p>1</p></td><td><p>Identification number of this pointer. Value is in the interval [0,1, ...]</p></td></tr><tr><td><p>X coordinate</p></td><td><p>2</p></td><td><p>X coordinate for an event normalized relative to a consistent video stream separation between the device receiver and the source device</p></td></tr><tr><td><p>Y coordinate}</p></td><td><p>2</p></td><td><p>Y-coordinate for an event normalized relative to a consistent video stream separation between the device receiver and the source device</p></td></tr>

10 Number of pointers can identify the number of touches or mouse clicks,

associated with the event of the introduction. Each pointer can have a unique Y pointer. If, for example, the multi-touch event includes three touches, then the input event may have three pointers, each with a unique Y pointer. Each pointer (that is, every finger with a finger) may have an appropriate x-coordinate and y-coordinate, corresponding to where little

15th touching place.

For example, if three-finger sliding is the command to close the application, slipping the triplets can be described in useful data 650 as a touch-down event with three pointers, an event triggered by three pointers, and an event touching up three pointers. Three

20 pointers of the tap-down event may have the same identifiers of the Y pointer as the triggers of the touch-up event and touch-up events. A device of 120 sources can interpret the combination of these three input events as slipping with three fingers.

For example, the "Down key" or "Up key" event description fields may include all the information items identified in Table 4 below.

25

17

iA 109176 C2

Table 4

<tr><td><p>Field</p></td><td><p>Size (octet)</p></td><td><p>Notes</p></td></tr><tr><td><p>Reserved</p></td><td><p>1</p></td><td><p>reserved</p></td></tr><tr><td><p>Key Code 1 (A8CII)</p></td><td><p>2</p></td><td><p>Code of the first event key "Down key". The main / advanced A8CII code uses the younger one byte. Senior</p><p>One byte is reserved for the future compatible code of the A8CII key</p></td></tr><tr><td><p>Key code 2 (A8CII)</p></td><td><p>2</p></td><td><p>Code of the second event key "down arrow". The main / advanced A8CII code uses the younger one byte. The highest one byte is reserved for the future compatible code of the A8CII key</p></td></tr>

For example, an enlargement scale event description field may include the information elements identified in Table 5 below.

5

Table 5

<tr><td><p>Field</p></td><td><p>Size</p><p>(octet)</p></td><td><p>Notes</p></td></tr><tr><td><p>X</p></td><td><p>2</p></td><td><p>Support X-coordinate for the operation of increasing the scale of the image normalized relative to the agreed differentiation of the video stream between the device receiver and the source device</p></td></tr><tr><td><p>Υ</p></td><td><p>2</p></td><td><p>Support Υ-coordinate for the operation of increasing the scale of the image normalized relative to the agreed differentiation of the video stream between the device and receiver source device</p></td></tr><tr><td><p>The whole number multiplies zoom magnification</p></td><td><p>1</p></td><td><p>Unsigned integer part of the number of times to increase the scale of the image</p></td></tr><tr><td><p>Fractional number multiplies zoom magnification</p></td><td><p>1</p></td><td><p>Fractional part of the number of times to increasescale of the image</p></td></tr>

The description field for a horizontal scroll event or vertical scroll event may, for example, include the information elements identified in Table 6 below.

10

Table 6

<tr><td><p>Field</p></td><td><p>Size</p><p>(octet)</p></td><td><p>Note</p></td></tr><tr><td><p>Size for scrolling</p></td><td><p>2</p></td><td><p>The number of pixels for scrolling is normalized in relation to the agreed reconciliation of the video stream between the device and the receiver source device. A negative number may indicate scrolling to the right, and a positive number may indicate scrolling to the left.</p></td></tr>

The above examples showed some exemplary ways that may be formatted for useful data for the general introduction category . If the category 624 of the category marker of the data packet header 610 indicates a different input category, such as a directed

15 user inputs, the useful data 650 may have an excellent input format. A well-known custom input device receiver 160 may receive data from a user's input from a third party device and direct the input on the device 120 of the source without interpreting the user input. The source 120 can, in this way, perform a parsing analysis of useful data 650 according to a directed format

18

iA 109176 C2

custom input. For example, the header 630 of the useful data 650 may include a field for identifying the third person device from which the user input was received. The field may, for example, include the Internet Protocol (IP) address of a device of a third person, an MAC address, a domain name, or some other such identifier. The device 120 of the source can conduct a parsing analysis of the balance of useful data data based on the identifier of the third person device.

The recipient device 160 can reconcile the capabilities of a third party device with a sequence of messages. The recipient device 160 can then transmit the unique identifier of the third person device to the source device 120 as part of establishing a communication session with the source device 120 as part of the process of reconciling the capabilities. Alternatively, the recipient device 160 may transmit information describing the third person device to the source device 120 and, based on this information, the source device 120 may identify a unique identifier for the third person device. Information that describes a third-party device may for example include information for identifying a third-party device and / or information for identifying the capabilities of a third-party device. No matter what

If the input data field 624 of the data packet header 610 still indicates a different input category, such as a voice command, then the useful data 650 may still have a different input format. For a voice command, useful data 650 may include legitimized audio. The codec for coding and decoding the audio of the voice command may be co-ordinated between the source device 120 and the recipient device 160 using a message sequence. To transmit a voice command, the time field 626 may include the meaning of the speech sampling time of the speech signal. In this case, the time flag flag 622 may be set to indicate that there is a time stamp, but instead of the time marker described above, the time marker field 626 may include the meaning of the speech signal sampling time for the encoded audio data 650.

In some examples, a voice command can be transmitted as a general command as a descriptor, when the input category field 624 can be set to identify the format of the general command, and one of the reserved IDs of the common IE can be assigned to the voice commands. If a voice command is transmitted as a general command, the speech sampling rate may be present in the time field 626 of the data packet header 610 or may be present in useful data 650.

For captured voice command data, voice data can be encapsulated in a variety of ways. For example, voice command data can be encapsulated using a RTR that can provide a type of useful data to identify the timestamp and timestamp, with a timestamp used to identify the speed of the search engine. The RTR data can be encapsulated using the general user input format described above, with or without an optional timestamp. The recipient device 160 can transmit general input data that transmits the voice command to the source device 120 using the TRS / IP.

As discussed earlier, when the coordinates are included as part of a data packet, such as a data packet600, useful data 650, for example, coordinates may correspond to coordinates scaled on the basis of a coherent distinction, coordinates of the display screen, normalized coordinates or coordinates associated with the receiver display. In some cases, additional information may either be included in a data packet or transmitted separately for use with a source device to normalize the coordinates taken in the data packet.

Regardless of the input category for a particular data packet, the data packet header maybe an application packet header and the data packet can be transmitted over the TCP / IP. The TCP / IP can allow the receiver device 160 and the source device 120 to perform the retransmission in the event of a packet loss. A data packet may be sent from the recipient device 160 to the source device 120 to control the audio data or video data of the source device 120, or for other purposes, for example, to control the application operating on the source device.

19th

iA 109176 C2

FIG 7A is a block diagram of an exemplary method for reconciling capabilities between a device's receiver and a source device. An exemplary exemplary method may be implemented by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable storage medium (e.g., memory 332) can store commands, modules, or algorithms that, when executed, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a water circuit or more flowcharts disclosed in this disclosure.

Method according to FIG. 7A includes a receiver device 160 that receives a first message from the source device 120 (step 701). The message may, for example, contain a request for the receipt of the parameter. In response to the first message, the recipient device 160 may send a second message to the source device 120 (step 703). The second message may, for example, contain an answer to obtain a parameter that identifies the first list of supported input categories and the set of first lists of supported types, and each of the supported input categories from the first list of supported input categories is an associated first list of supported types. Supported input categories may, for example, correspond to the same categories used for field 624 categorization for FIG. 6. Table 2, presented above, represents one example of supported types for a particular category of input (general introduction in this example). The receiver device 160 may receive a third message from the source device 120 (step 705). The third message can, for example, contain a parameter set request, and the parameter setup request identifies the communication port, the second list of supported categories, the input, and the set of other supported types of lists, with each of the supported categories from the second list of supported input categories, with an associated second list of supported types, and each of the supported types from other lists includes the typing of types from the first lists. The recipient device 160 can transmit a fourth message to the source device 120 (step 707). A fourth message may, for example, be contains a response to setting the parameter to confirm that the types of other lists were allowed. The receiver device 160 may receive a fifth message from the source device 120 (step 709). The fifth message may, for example, contain a second parameter request request indicating that the communication channel between the source device 120 and the receiver device 160 has been enabled. The communication channel, for example, may comprise a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to the source unit 120 (step 711). The sixth message may, for example, contain a second response to the setting of a parameter that confirms receiving a second parameter request using the recipient device 160. The receiver device 160 may receive a fifth message from the source device 120 (step 709). The fifth message may, for example, contain a second parameter request request indicating that the communication channel between the source device 120 and the receiver device 160 has been enabled. The communication channel, for example, may comprise a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to the source unit 120 (step 711). The sixth message may, for example, contain a second response to the setting of a parameter that confirms receiving a second parameter request using the recipient device 160. The receiver device 160 may receive a fifth message from the source device 120 (step 709). The fifth message may, for example, contain a second parameter request request indicating that the communication channel between the source device 120 and the receiver device 160 has been enabled. The communication channel, for example, may comprise a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to the source unit 120 (step 711). The sixth message may, for example, contain a second response to the setting of a parameter that confirms receiving a second parameter request using the recipient device 160. The link between source device 120 and receiver device 160 was allowed. The communication channel, for example, may comprise a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to the source unit 120 (step 711). The sixth message may, for example, contain a second response to the setting of a parameter that confirms receiving a second parameter request using the recipient device 160. The link between source device 120 and receiver device 160 was allowed. The communication channel, for example, may comprise a reverse user input channel (IIS). The recipient device 160 may transmit the sixth message source to the source unit 120 (step 711). The sixth message may, for example, contain a second response to the setting of a parameter that confirms receiving a second parameter request using the recipient device 160.

FIG 7B is a block diagram of an exemplary method for coordinating the capabilities between the device receiver and the source device. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable storage medium (e.g., memory 232) can store commands, modules, or algorithms that, when executed, force one or more processors (for example, processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 7B includes a source device 120 that transmits the first message to the receiver device 160 (step 702). The first message may, for example, contain a request for a parameter. The 120 source may receive a second message from the receiver device 160 (step 704). The second message may, for example, contain a response to obtain a parameter that identifies the first list of supported categories of input in the first list of supported types, with each of the supported categories entering from the first list of supported input categories has an associated first list of supported types. The device 120 may transmit the source to the third message recipient 160 (step 706). The third message may, for example, include a request to install a parameter that identifies the port for communication, the second list of supported categories for the introduction of a number of other supported types of lists, where each of the supported categories for entering a well-organized list of supported input categories has an associated second list of supported types, and each of the supported types from other lists includes sub-lists from the first lists. The source device 120 may receive a fourth message from the recipient device 160 (step 708). A fourth message may, for example, contain an appropriate parameter setting to confirm that the types of other lists have been allowed. The source device 120 may transmit a fifth message to the recipient device 160 (710). The fifth message may, for example, contain a second request for setting a parameter indicating that the communication channel between the source device 120 and the recipient device 160 has been resolved.

20

iA 109176 C2

The communication channel, for example, may comprise a reverse user input channel (IIVS). The source device 120 may receive a sixth message from the recipient device 160 (step 712). The sixth message may, for example, contain a second response to the setting of a parameter that confirms receiving a second parameter request using the recipient device 160.

FIG 8A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 8A includes receiving user-entered data in a recipient's wireless device, such as the receiver wireless device 160 (step 801). The user input data can be obtained through the user-generated component of the wireless receiver device 160, such as, for example, the user input interface 376 shown with respect to the wireless device 360 ​​of the receiver. Additionally, the receiver device 160 can be divided into categories of user-generated data such as general, directed or specific for the operating system. The receiver device 160 may then generate a data packet header based on the user input data (step 803). The header of the data packet may be the packet header of the application layer. The header of the data packet may contain, among other fields, field for identifying the type of input corresponding to the data of the user input. The input category can be, for example, the general input format or the command of the user interface device. The recipient device 160 may additionally generate the data packet (step 805), the data packet contains a generated data packet header and useful data. In one example, useful data may include accepted user input data and can identify one or more user commands. The recipient device 160 may then transfer the generated data packet (step 807) to a wireless source device (e.g., a source device 120 at FIG 1A or a source device 220 at FIG. 2). The receiver device 160 may contain components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, as shown in FIG. 3 for example. The receiver device 160 can transmit a data packet over a TCP / IP.

FIG 8B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 8B includes receiving a data packet (step 802), where the data packet can be, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data. The 120 source can provide communication components that allow the transmission of data packets including the interconnect unit 233 and the wireless modem 234, for example, as shown with respect to FIG. 2. The source device 120 may then perform a parsing analysis of the data packet header (step 804) included in the data packet to determine the input category associated with the user input that is contained in the useful data. A device of 120 sources canproduce useful data based on a specific input category (step 806). Data packets described with links to FGI. 8A and 8B, in general, can take the form of data packets described with links to FIG. 6, and can be used to control audio / video data and applications in source devices.

FIG 9A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless device source in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

21

iA 109176 C2

Method according to FIG. 9A includes receiving custom input data in a receiver's wireless device, such as a receiver wireless device 160 (step 901). The user input data can be obtained through the user-generated component of the wireless device 160 of the receiver, such as, for example, the user input interface 376, shown with links to the FIG. 3. The recipient device 160 may then generate useful data (step 903), where the useful data may describe the user's data. In one example, the useful data may include the received data of the user input and can identify one or more user commands. The recipient device 160 may additionally generate a data packet (step 905) where the data packet contains the data packet header and generated useful data. The recipient device 160 may then transfer the generated data packet (step 907) to a wireless source device (e.g., a source device 120 on FIG 1A or a source device 220 on FIG. 2). The receiver device 160 may contain components that allow the transmission of data packets, such as the transport unit 333 and the wireless modem 334, for example. The data packet can be transmitted to a wireless source device via TCP / IP.

FIG 9B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 9B includes receiving a data packet from the recipient device 360 ​​(step 902), wherein the data packet may include, among other things, the data packet header and useful data. In one example, useful data may include, for example, data describing the details of the user input, such as the value of the input type. The 120 source can provide communication components that allow the transmission of data packets included in the network unit 233 and the wireless modem 234, for example, as shown with references to FIG. 2. The source 120 can then parse the data packet (step 904) to determine the value of the input type in the input type field of the useful data. The device 120 can process the data describing the details of the user input, based on a certain input type value (step 906). Data packets described with links to FIG. 9А and 9В, can generally take the form of data packets described with links to FIG. 6

FIG 10A is a block diagram of an exemplary way of transferring user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 10A includes receiving user-entered data in a recipient's wireless device, such as a receiver wireless device 160 (step 1001). The user input data can be obtained through the user-generated component of the wireless device 160 of the recipient, such as, for example, the user input interface 376, as shown with links to the FIG. 3. The recipient device 160 may then generate a header of the data packet based on the user input (step 1003). The data packet header may contain, among other fields, a time stamp flag (for example, a 1-bit field) to indicate if the time marker field is in data packet header. The flag of the timestamp may, for example, include "1" to indicate that there is a time marker field and may include "0" to indicate, that there is no time marker field. The time marker field maybe, for example, a 16-bit field containing a timestamp generated by the source device 120 and added to the video data prior to transmission. The recipient device 160 may additionally generate a data packet (step 1005), wherein the data packet contains a generated header of the packet data and useful data. In one example, the useful data may include the received data of the user input and can identify one or more user commands. The recipient device 160 can then transmit the generated data packet (step 1007) to the non-originating source device (e.g., the source device 120 according to FIG 1A or device 220 sources according to FIG. 2). The receiver device 160 may include components that allow the transmission of data packets including the transport unit 333 and the wireless

22

iA 109176 C2

modem 334, for example, as shown with respect to FIG. 3. The data packet can be transmitted to the wireless device of the source on the TCP / IP.

FIG 10B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 10B includes receiving a data packet from a wireless recipient device 160 (step 1002), wherein the data packet may contain, among other things, a header of the data packet and useful data. Useful data may include, for example, user-entered data. The source device 120 may include communication components, such as allowing the transmission of data packets including a transport unit 233 and a wireless modem 234, for example, as shown in relation to FIG. 2. The source device 120 may then perform a parsing analysis of the data packet header (step 1004) included in the data packet. The source device 120 can determine if there is a timestamp field in the data packet header (step 1006). In one example, the source device 120 can make a determination based on the value of the time-delay flag included in the header of the data packet. If the data packet header includes all the time marker field, the source device 120 can process useful data based on the timestamp in the timestamp field (step 1008). Data packets described with links to FGI. 10A and 10B, can generally take the form of data packets described with links to FIG. 6, and can be used to control the audio / video data in the device source.

FIG 11A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 11A includes receiving custom input data in a recipient's wireless device, such as a recipient wireless device 160 (step 1101). The user input data can be obtained through the user-generated component of the wireless device 160 of the receiver, such as, for example, the user input interface 376 shown with respect to the FIG. 3. The recipient device 160 can then generate the user data caption header (step 1103). The header of the data packet may contain, among other fields, a time marker field. A timestamp field may include, for example, a 16-bit field containing a timestamp based on multimedia data generated by the source wireless source 120 and transmitted to the receiver outside the wireless device 160. Time marker could be added to the frame of video data using a wireless device 120 source to transmit to the wireless device receiver. The timestamp field may, for example, identify the timestamp associated with the video frame displayed on the wireless recipient device 160 while the user input data was captured. The recipient device 160 may additionally generate a data packet (step 1105), wherein the data packet contains a generated data packet header and useful data. In one example, useful data may include custom data inputs and can identify one or more custom commands. The recipient device 160 can then transfer the generated data packet (step 1107) to the wireless source device (e.g., the source device 120 according to FIG. 1A or source 220 device according to FIG. 2). The recipient device 160 can provide components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in relation to the FIG. 3. Packet data can be transmitted to a wireless source device via TCP / IP.

FIG 11B is a block diagram of an exemplary method for receiving user input from the wireless device of the recipient in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms,

23

iA 109176 C2

which, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 11B includes receiving a data packet from a wireless receiver device such as a recipient wireless device 160 (step 1102), where the data packet can be, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data. The 120 source can provide communication components that allow the transmission of data packets including the interconnect unit 233 and the wireless modem 234, for example, as shown with respect to FIG. 2. The source device 120 may then identify the timestamp field in the packet header header (step 1104). The 120 source can process useful data based on the timestamp, which is in the timestamp field (step 1106). As part of the processing of useful data data, based on the timestamp, the source device 120 can identify the frame of the video data that is displayed on the wireless device of the recipient, at the time when user data was received and to interpret the useful data based on the content of the frame. As part of the processing of useful data, based on the time marker, the source device 120 can compare the timestamp with the current time marker for the current video frame transmitted by the source device 120 and may execute a user input command described in useful data in response to the time difference between the time marker and the current timestamp, which is smaller than the threshold value, or not execute the command command, described in the useful data, in response to the time difference between the timestamp and the current time a mark which is more than a threshold value. Data packets described with links to FGI. 11A and 11B, can generally take the form of data packets described with links to FIGs. 6, and can be used to control the audio / video data in the device source.

FIG 12A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless device source in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 12A includes receiving user-entered data in a recipient's wireless device, such as a receiver wireless device 160 (step 1201). In one example, user input data may be voice command data that can be obtained through the user-input component of the wireless device 160 of the recipient, such as, for example, a voice command recognition module included in the user interface 376 of the FIG interface. 3. The recipient device 160 can generate the header of the data packet based on the user input (step 1203). The receiver device 160 may also generate useful data (step 1205), where the useful data may contain a voice command. In one example, useful data may also include accepted custom input and can identify one or more user commands. The recipient device 160 may additionally generate a data packet (step 1207), wherein the packet data comprises a generated data packet header and useful data. The recipient device 160 can then transfer the generated data packet (step 1209) to a wireless source device (e.g., source device 120 according to FIG 1A or source device 220 according to FIG 2). The receiver device 160 may include components that allow the transmission of data packets , which include a transport unit 333 and a wireless modem 334, for example, as shown in relation to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. The recipient device 160 may additionally generate a data packet (step 1207), wherein the packet data comprises a generated data packet header and useful data. The recipient device 160 can then transfer the generated data packet (step 1209) to a wireless source device (e.g., source device 120 according to FIG 1A or source device 220 according to FIG 2). The receiver device 160 may include components that allow the transmission of data packets , which include a transport unit 333 and a wireless modem 334, for example, as shown in relation to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. The recipient device 160 may additionally generate a data packet (step 1207), wherein the packet data comprises a generated data packet header and useful data. The recipient device 160 can then transfer the generated data packet (step 1209) to a wireless source device (e.g., source device 120 according to FIG 1A or source device 220 according to FIG 2). The receiver device 160 may include components that allow the transmission of data packets , which include a transport unit 333 and a wireless modem 334, for example, as shown in relation to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. source device 120 according to FIG. 1A or source 220 device according to FIG. 2). The recipient device 160 may comprise components that allow the transmission of data packets including a transport unit 333 and a wireless modem 334, for example, as shown in relation to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP. source device 120 according to FIG. 1A or source 220 device according to FIG. 2). The recipient device 160 may comprise components that allow the transmission of data packets including a transport unit 333 and a wireless modem 334, for example, as shown in relation to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP.

FIG 12B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 12B includes receiving a data packet (step 1202), where the data packet may contain, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data such as voice command data. The source device 120 may include communication components that allow the transmission of data packets including the transport unit 233 and the wireless modem 234, for example, as shown

24

iA 109176 C2

in relation to FIG. 2. The source device 120 may then perform a parsing analysis of the data useful data (step 1204) included in the data packet to determine whether there is useful data for the voice command data. Data packets described with links to FIG. 12A and 12B, in general, may take the form of data packets described with references to FIG. 6, and can be used to control audio / video data in the source device.

FIG 13A is a block diagram of an exemplary method for the transfer of user input from the wireless device of the receiver to a wireless device source in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 13A includes receiving user-entered data in a recipient's wireless device, such as the receiver wireless device 160 (step 1301). In one example, user-input data can be a multi-touch gesture that can be obtained through the user-input component of the wireless device 160 of the recipient, such as, for example, ii 167 or user interface 376 interface FIG. 3. In one example, the multiple-gesture gesture may include the first input of the touching data and the second touch input. The recipient device 160 can generate a header of the data packet based on the user input (step 1303). The receiver device 160 may also generate useful data (step 1305), where useful data can be associated with the user input for the first event of input by touching the first identifier of the index and the data of the user input for the second event of the input touching the second identification of the pointer. The recipient device 160 may additionally generate a data packet (step 1307), wherein the data packet contains a generated data packet header and useful data. The recipient device 160 may then transmit the generated data packet (step 1309) to the non-originating source device (e.g., the source device 120 according to FIG 1A or device220 source according to FIGURE 2). The recipient device 160 may include components that allow the transmission of data packets including the transport unit 333 and the wireless modem 334, for example, as shown in relation to the FIG. 3. The data packet can be transmitted to the wireless device of the source on the TCP / IP.

FIG 13B is a block diagram of an exemplary method for receiving user input from the wireless device of the recipient in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 13B includes the reception of a data packet (step 1302), where the data packet may contain, among other things, the header of the data packet and useful data. Useful data may include, for example, user input data such as a multiple-gesture gesture. The source device 120 may include communication components that allow the transmission of data packets including the transport unit 233 and the wireless modem 234, for example, as shown FIG 2. The device 120 of the source can then perform a parsing analysis of useful data data (step 1304) included in the data packet to identify the user-generated data included in the useful data. In one example, the identifiable data may include all the user input data for the first event input, touching the first identifier of the index, and the user input for the second hit entry event with a second identifier of the pointer. The source device 120 may then interpret the user input for the first touch input event and the user input for the second touch input event as a multiple-touch gesture (step 1306). Packages data are described with links to FIG. 13А and 13В, can generally take the form of data packets described with references to FIG. 6, and can be used to control the audio / video data of the source device. The source device 120 may then interpret the user input for the first touch input event and the user input for the second touch input event as a multiple-touch gesture (step 1306). Packages data are described with links to FIG. 13А and 13В, can generally take the form of data packets described with references to FIG. 6, and can be used to control the audio / video data of the source device. The source device 120 may then interpret the user input for the first touch input event and the user input for the second touch input event as a multiple-touch gesture (step 1306). Packages data are described with links to FIG. 13А and 13В, can generally take the form of data packets described with references to FIG. 6, and can be used to control the audio / video data of the source device.

FIG 14A is a block diagram of an exemplary method for transferring user input from the wireless device of the receiver to a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by the recipient device 160 (FIG. 1A) or the recipient device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms,

25

iA 109176 C2

which, when executed, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 14A includes receiving user-entered data in the recipient's external wireless device 360 ​​from an external device (step 1401). In one example, an external device may be a device of a third person connected to the device receiver. The receiver device 160 can generate a user data caption header (step 1403). In one example, the data packet header may identify the user input data as directed user input data. The recipient device 160 may also generate useful data (step 1405), where useful data may contain user input data. The recipient device 160 may additionally generate a data packet (step 1407), wherein the data packet may contain a generated header of the data packet and useful data. The recipient device 160 can then transfer the generated packet data (step 1409) to a wireless source device (for example, a source device 120 according to FIG 1A or a source device 220 according to FIG. 2). The recipient device 160 may comprise components that allow the transmission of data packets including a transport unit 333 and a wireless modem 334, for example, as shown with links to FIG. 3. The data packet can be transmitted to the wireless device of the source via TCP / IP.

FIG 14B is a block diagram of an exemplary method for receiving user input data from a wireless device of the receiver in a wireless source device in accordance with this disclosure. An illustrative exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 14B includes receiving a data packet (step 1402), where the data packet may contain, among other things, the data packet header and useful data. Useful data may include, for example, user-input data, such as a directed user-input command that indicates that the user-input data has been sent from a third-party device. The source device 120 may include communication components, such as allowing the transmission of data packets including a transport unit 233 and a wireless modem 234, for example, as shown in relation to FIG. 2. The source device 120 may then perform a parsing analysis of the data packet header and may determine that the useful data contains a directional user-input command (step 1404). The device 120 sources can then perform a parsing analysis of useful data (step 1406), included in the data packet to identify the identity associated with the third-party device in accordance with the direction of the user-input command. The device 120 of the source can then process the useful data based on the identification of the third party device (step 1408). Packages data are described with links to FIG. 14A and 14B, generally can take the form of data packets described with references to FIGs. 6, and can be used to control the audio / video data of the source device. Generally, they can take the form of data packets described with links to FIGs. 6, and can be used to control the audio / video data of the source device. Generally, they can take the form of data packets described with links to FIGs. 6, and can be used to control the audio / video data of the source device.

FIG 15A is a block diagram of an exemplary method for transmitting user data from a wireless recipient device to a wireless source device according to this disclosure. Illustrated exemplary method can be implemented by the recipient device 160 (FIG. 1A) or receiver device 360 ​​(FIG. 3). In some examples, a computer-readable memory medium (e.g., memory 332) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 331) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 15A includes receiving user-entered data in the recipient's wireless device (step 1501). Custom input data may have associate coordinate data. Associated coordinate data may, for example, correspond to the location of the mouse click event or the location of the touch event. The receiver device 160 can then normalize the associated coordinate data to generate the normalized coordinate data (step 1503). The recipient device 160 can then generate a data packet that includes normalized coordinate data (step 1505). Normalization of the data coordinate may include scaling of associated coordinate data on the basis of the relationship between the resolution of the display screen window and the differentiation of the source display, such as the display 22 of the source device 120. The difference in the display screen may be determined by the receiver device 160, and the differentiation of the display of the source device can be taken from the device 120 sources. The recipient device 160 may then transmit the data packet with the coordinates to the wireless source 120 (step 1507). As part of

26

iA 109176 C2

method according to FIG. 15A, the recipient device 160 may also determine whether the associated coordinate data is within the display window, for content received from the wireless source device, and, for example, to process the user input locally if the associated coordinate data is outside the screen display window or to otherwise co-ordinate the coordinates, as described, if the input is within the screen display.

FIG 15B is a block diagram of an exemplary method for receiving user input from the wireless device of the receiver in a wireless source device in accordance with this disclosure. The illustrated exemplary method may be performed by a source device 120 (FIG. 1A) or a source device 220 (FIG. 2). In some examples, a computer-readable memory medium (e.g., memory 232) can store commands, modules, or algorithms that, in execution, force one or more processors (e.g., processor 231) to perform one or more illustrated steps in a flowchart.

Method according to FIG. 15B includes the reception of a data packet in a wireless device, where the data packet contains user input data with associated data coordinates (step 1502). Associated coordinate data may, for example, correspond to the location of the mouse click event or location of the touch event in the receiver device. The source device 120 can then normalize the associated coordinate data to generate normalized coordinate data (step 1504). The device 120 sources canoren the data of coordinates by scaling the associated coordinate data on the basis of the ratio of the distinguishing of the screen display screen and the differentiation of the source display. Device120 sources can determine the distinction of the display device source and can takedisplay the screen display screen from the wireless device receiver. A source device can then process a data packet based on normalized coordinate data (step 1506). Data packets described with links to FIG. 15A and 15B, in general, can accept packets of data, described with references to FIG. 6, and can be used to control the audio / video data in the source device.

For ease of explanation, aspects of this disclosure have been described separately with references to the FIG. 7-15. However, it is considered that these various aspects can be combined and used by the modem with one, and not simply separately. In general, the functionality and / or modules disclosed in this description can be implemented in either one or both of the wireless device's source and the wireless device of the receiver. Thus, the capabilities of the user interface, described in the current example, can be used alternately between the wireless source device and the wireless device of the recipient.

Methods of this disclosure can be implemented in a large variety of devices or devices, including a wireless telephone and integrated circuit (IC) or set of IC circuits (ie, set of chips). Any components, modules or blocks that have been described are provided to emphasize the functional aspects, and do not necessarily require the implementation of various hardware units.

Accordingly, the methods disclosed herein may be implemented in hardware, software, hardware, or any combination thereof. If implemented in hardware, any features described as modules, blocks, or components can be implemented together in an integrated logical device or separately as discrete but interactive logical devices. If implemented in softwareprotection, methods can be implemented, at least partially, with computer-readable media containing commands that, when executed in the processor perform one or more of the methods described above. Computer-readable media may contain a material and permanent computer-readable storage media and may be part of a computer software product, which may include packaging materials. The computer memory memory may comprise an operational memory device (RMA) such as a synchronous dynamic memory storage device (RAM), a permanent storage device (RME), a non-volatile memory device An electronic device (ΝνΡΑΜ), an electrically erased progromovoj permanent memory device (EEPROM), flash memory, magnetic or optical storage media data, etc. Methods can be implemented, in addition or alternatively, at least Enhanced, by means of a computer-readable communication medium that transmits or transmits a code in the form of commands or data structures and which can be accessed, read and / or executed by the computer. The storage medium may comprise an operational storage device (RMA), such as a synchronous dynamic storage memory device (SDMM), a permanent storage device (PZM), a non-volatile operational memory device (NNPMA), an electrically erased progrommovoj permanent memory device (EEPROM), flash memory, magnetic or optical storage media, etc. Alternatively, alternatively, methods may be implemented, at least partially, by means of a computer-readable communication medium, i ground transfers aboperedaye code in the form of commands or data structures and that can be accessed, the read and / abovykonanyy computer. The storage medium may comprise an operational storage device (RMA), such as a synchronous dynamic storage memory device (SDMM), a permanent storage device (PZM), a non-volatile operational memory device (NNPMA), an electrically erased progrommovoj permanent memory device (EEPROM), flash memory, magnetic or optical storage media, etc. Alternatively, alternatively, methods may be implemented, at least partially, by means of a computer-readable communication medium, i ground transfers aboperedaye code in the form of commands or data structures and that can be accessed, the read and / abovykonanyy computer.

The code can be executed by one or more processors such as one or more digital signal processors (DDR processors), general-purpose microprocessors,

27

iA 109176 C2

specialized integrated circuits (ASEE schemes), programmable users, vent matrices (matrixes of RRCA) or other equivalent integrated ordiscrete logic circuits. Accordingly, the term "processor" used in this description may relate to any prior art structure or any other structure suitable for implementing the methods disclosed herein. Additionally, in some aspects, the functional capabilities disclosed herein may be provided in dedicated software modules or hardware modules configured for encoding and decoding or incorporated into a video codec. In addition, the methods can be fully implemented in one or more schemes or logical elements.

Various aspects of this disclosure were described. These and other aspects are within the scope of the following formula of the invention.

FORMULA INSTRUCTIONS

A method for transmitting user data from a wireless receiver device to a non-wireless source device, the method comprising:

obtaining user-entered data in the wireless device of the receiver, andthis user input has associated data of the coordinates;

the normalization of the associated coordinate data for generating normalized coordinate data, with the normalization being the agreement between the wireless source device and the wireless device of the recipient of the interface interface of the user input of the devices;

generating a data packet containing normalized coordinate data;

Transfer the data packet to a wireless source device.

2. The method of claim 1, further comprising:

Determining whether the associated coordinate data is within the display window, for the content taken from the wireless device of the source.

3. The method of claim 1, further comprising:

definition of the distinguishing of the window screen display for content received from the wireless device source;

receiving from the device a source of indication of the differentiation of the display of the device of the source.

4. The method of claim 3, wherein the step of normalizing the coordinate data includes the scaling of the associated coordinate data based on the ratio of the distinguishing of the display window and the differentiation of the display of the source.

5. The method of claim 1, wherein the associated coordinate data corresponds to the location of the event of the mouse click.

6. The method of claim 1, wherein the associated coordinate data corresponds to the location of the event of intervention.

7. The wireless device of the receiver for transmitting user data to a wireless source device, wherein the wireless device of the receiver comprises:

memory that stores commands;

one or more processors configured to execute commands, and after executing the commands, one or more processors call:

obtaining user-entered data in the wireless device of the receiver, andthis user input has associated data of the coordinates;

the normalization of the associated coordinate data for generating normalized coordinate data, with the normalization being the agreement between the wireless source device and the wireless device of the recipient of the interface interface of the user input of the devices;

generating a data packet containing normalized coordinate data;

a transport block for transmitting a data packet to a wireless source device.

8. The wireless device of the recipient of item 7, which, after executing commands, one or more processors additionally cause:

Determining whether the associated coordinate data is within the display window, for the content taken from the wireless device of the source.

9. The wireless device of the receiver according to paragraph 7, which, after executing commands, one or more processors additionally cause:

definition of the distinguishing of the window screen display for content received from the wireless device source;

receiving from the device a source of indication of the differentiation of the display of the device of the source.

28

iA 109176 C2

10. The wireless device of the receiver according to item 9, wherein the step of normalizing the coordinate data includes scaling of the associated coordinate data based on the ratio of the difference in the window of the display screen and the differentiation of the display of the source.

11. The wireless device of the recipient according to item 7, in which the associated coordinate data correspond to the location of the mouse click event.

12. The wireless device of the recipient according to item 7, in which the associated coordinate data correspond to the location of the touch event.

13. A computer-readable storage medium that stores commands that, after performing one or more processors, force one or more processors to execute a method for transmitting user data from the wireless device of the receiver to a wireless source device, the method comprising:

obtaining user-entered data in the wireless device of the receiver, andthis user input has associated data of the coordinates;

the normalization of the associated coordinate data for generating normalized coordinate data, with the normalization being the agreement between the wireless source device and the wireless device of the recipient of the interface interface of the user input of the devices;

generating a data packet containing normalized coordinate data;

Transfer the data packet to a wireless source device.

14. The wireless device of the receiver for transmitting user data to a wireless source device, wherein the wireless device of the receiver comprises:

a means for obtaining user input data in a wireless device of the recipient, and the data of the user input have associated data coordinates, a means for normalizing the associated coordinate data for generating normalized data coordinates, with the normalization is the agreement between the wireless source device and the wireless device receiver capabilities of the interface user input of the devices;

means for generating a data packet containing normalized coordinate data;

A tool for transmitting a data packet to a wireless source device.

15. A method for receiving user data from a wireless device of a receiver to a non-wireless source device, the method comprising:

receiving a data packet in a wireless device of the source, the data packet comprising the data of the user input with the associated data of the coordinates;

the normalization of the associated coordinate data for generating normalized coordinate data, with the normalization being the agreement between the wireless source device and the wireless device of the recipient of the interface interface of the user input of the devices;

processing a data packet based on normalized coordinate data.

16. The method of claim 15, further comprising:

Receiving from the wireless device of the recipient a distinction between the display screen and the content received from the wireless device of the source, and the location information for the window display screen;

Determine the resolution of the display of the source device.

17. The method of claim 16, wherein the step of normalizing the coordinate data includes scaling of the associated coordinate data based on the ratio of the distinguishing between the display screen and the source display differentiation.

18. The method of claim 15, wherein the associated coordinate data corresponds to the location of the mouse click event.

19. The method of claim 15, wherein the associated coordinate data corresponds to the location of the touch event.

20. A wireless source device for receiving user data from a wireless receiver device, wherein the wireless source device comprises:

a transport unit for receiving a data packet in a wireless source device, wherein the packet data comprises user input data with associated coordinate data, memory that stores commands;

one or more processors configured to execute commands, and after executing the commands, one or more processors call:

the normalization of the associated coordinate data for the generation of normalized coordinate data, with the normalization being the harmonization between the wireless device of the source and

29

iA 109176 C2

wireless device recipient features user interface interface for the mentioned devices;

processing a data packet based on normalized coordinate data.

21. Wireless source device according to paragraph 20, which, after executing commands, one or more processors additionally cause:

Receiving from the wireless device of the recipient a distinction between the display screen and the content received from the wireless device of the source, and the location information for the window display screen;

Determine the resolution of the display of the source device.

22. The wireless device of the source according to item 21, in which the step of normalizing the coordinate datacontains the scaling of the associated coordinate data based on the ratio of the difference in the window display screen and the differentiation of the source display.

23. The wireless source device according to item 20, wherein the associated coordinate data corresponds to the location of the mouse click event.

24. The wireless device of the source according to item 20, in which the associated coordinate data correspond to the location of the touch event.

25. A computer-readable storage medium that stores commands that, after performing one or more processors, force one or more processors to execute a method for receiving user data from a wireless device of the receiver to a wireless source device, the method comprising:

receiving a data packet in a wireless device of the source, the data packet comprising the data of the user input with the associated data of the coordinates;

the normalization of the associated coordinate data for generating normalized coordinate data, with the normalization being the agreement between the wireless source device and the wireless device of the recipient of the interface interface of the user input of the devices;

processing a data packet based on normalized coordinate data.

26. A wireless source device for receiving user data from a wireless device of the receiver, wherein the wireless source device comprises:

means for receiving a data packet in a wireless source device, and the data packet contains user input data with associated coordinate data, a means for normalizing the associated coordinate data for generating normalized data coordinates, with the normalization is the agreement between the wireless device source and the wireless device recipient capabilities of the interface to the user input of the devices;

a tool for processing a data packet based on normalized coordinate data.

30

iA 109176 C2

31

iA 109176 C2

32

iA 109176 C2

The first message

The second message

The third message

Fourth message

Fifth message

Sixth message

Sixth message

Eighth message

FIG. 5A

FIG. 4

Source device520

The recipient's device

560

412

410

Source

data

450

424A

452A

Pilot signal

460

t

454A

Processor

Processor TX

Receiver

Transmitter

Transmitter

Data acquisition processor

MICHAEL TH

data transmission

Receiver

data transmission

Memory

Memory

Processor

And Irocessor

424T

452V

her

Processor TX

CP processor

Transmitter

Receiver

receiving data demodulator

Modulator

data transmission

And the interrogator

And Iryamach

442

422T

454K

Source

data

436

33

iA 109176 C2

Source device

520

The recipient's device

560

1a. SAT-RACAMETEK REQUESTS FOR SUGGESTIONS

1b OET_RAACAMETEK ANSWER ("М_и! Бс_сараиИі4у: Іпри1саіедогу-Ііз (= ΟΕΝΕΒίΟ;

Depechex "sr_Ii5I = Μοιίδβ, ZipDiToIs; LitSi-Sar-IivI = N1111 .; horns = ΝΙΛ.Ι.)

■ OH

(ννίά_ιιίΙχ; SARANIUU: Ipri1_sa1vdogo_ |) з1 = NIOS;

delvgis_sar_IizI = ΝΙΙΙΧ;

MbS-Sar-IizI = Moizlav / BT, NetOiSopIoLiP / HgvB; horns = Nv_A_I_)

2a, 5ET_RACAMETER REQUEST (UDGiIsSaRIiIiIiUiI: IpriI_saIeDoUGhIІ5I = ΟΕΝΕΡίΟ;

Dvlvgis_sar_I! зI = Моизл, ЗипдиіоосЬ; Либс_араріІіІІ = ΝΙΙΙΧ; horns = 1000)

- HE

("/ І_ІІсс ~ сараяІІІіу: іпри1_саїдегу_ | ізі = НЮС;

depag-SariI = NMI;

Ibid sari Iivi = Moizv / BT, RVOiSopiGiIiPiGagVB, ROP = 1000)

_ 2b. ZETRANAMETEN ANSWER

OK

_3a. 5ET_RARAMETEN REQUEST _

uyub_iиsс_5eІ (іпд: епаЬіе

Z. 8ET-SENTENCE REPLY

FIG. 5V

34

iA 109176 C2

35

iA 109176 C2

36

iA 109176 C2

37

iA 109176 C2

38

iA 109176 C2

39

iA 109176 C2

40

iA 109176 C2

41

iA 109176 C2

42

iA 109176 C2

Computer layout by I. Mironenko

State Service of Intellectual Property of Ukraine, st. Vasyl Lipkivsky, 45, Kyiv, Ukraine, 03680, Ukraine

State Enterprise "Ukrainian Institute of Intellectual Property", st. Glazunova, 1, Kyiv - 42, 01601

43