User input back channel for wireless displays
54 claims: 12 independent, 42 dependent
- 1ワイヤレスシンクデバイスからワイヤレスソースデバイスにユーザ入力データを送信する方法であって、前記方法は、 外部デバイスからユーザ入力データを取得することと、 データパケットヘッダを生成することであって、前記データパケットヘッダが、前記ユーザ入力データをフォワーディングされたユーザ入力データとして識別する、データパケットヘッダを生成することと、 前記ユーザ入力データを備えるペイロードデータを生成することと、 前記データパケットヘッダと前記ペイロードデータとを備えるデータパケットを生成することと、 前記ワイヤレスソースデバイスに前記データパケットを送信することとを備える、方法。
- 2一連のメッセージを介して、前記ワイヤレスシンクデバイスと、サードパーティデバイスの機能をネゴシエートすることをさらに備える、請求項1に記載の方法。
- 3前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスシンクデバイスから前記ワイヤレスソースデバイス にサ ードパーティデバイスの識別子を送信することをさらに備える、請求項1に記載の方法。
- 4前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスソースデバイスか らサ ードパーティデバイスの識別子を受信することをさらに備える、請求項1に記載の方法。
- 5前記データパケットヘッダが、ユーザ入力カテゴリーを識別するためのフィールドを備え、前記フィールドの値は、前記ペイロードデータが前記フォワーディングされたユーザ入力データを備えることを示すように設定される、請求項1に記載の方法。
- 6前記ペイロードデータ がサ ードパーティデバイスの識別子を備える、請求項1に記載の方法。
- 7サ ードパーティデバイス の識 別子が、前記サードパーティデバイスのIPアドレスと、前記サードパーティデバイスのドメイン名とからなるグループから選択される、請求項1に記載の方法。
- 8前記識別子が、前記ワイヤレスソースデバイスによって生成され、前記ワイヤレスシンクデバイスに送信される、請求項1に記載の方法。
- 9前記外部デバイスが別のワイヤレスシンクデバイスである、請求項1に記載の方法。
- 10前記外部デバイスが、前記ワイヤレスシンクデバイスに通信可能に結合された入力デバイスである、請求項1に記載の方法。
- 11前記データパケットヘッダがアプリケーションレイヤパケットヘッダである、請求項1に記載の方法。
- 12前記データパケットが、前記ワイヤレスソースデバイスのオーディオデータまたはビデオデータを制御するためのものである、請求項1に記載の方法。
- 13前記データパケットがTCP/IPを介して送信される、請求項1に記載の方法。
- 14ワイヤレスソースデバイスにユーザ入力データを送信するように構成されたワイヤレスシンクデバイスであって、前記ワイヤレスシンクデバイスは、 命令を記憶するメモリと、 前記命令を実行するように構成された1つまたは複数のプロセッサであって、前記命令の実行時に、前記1つまたは複数のプロセッサが、 外部デバイスからユーザ入力データを取得することと、 データパケットヘッダを生成することであって、前記データパケットヘッダが、前記ユーザ入力データをフォワーディングされたユーザ入力データとして識別する、データパケットヘッダを生成することと、 前記ユーザ入力データを備えるペイロードデータを生成することと、 前記データパケットヘッダと前記ペイロードデータとを備えるデータパケットを生成することとを行わせる、1つまたは複数のプロセッサと、 前記ワイヤレスソースデバイスに前記データパケットを送信するためのトランスポートユニットとを備える、ワイヤレスシンクデバイス。
- 15前記命令の実行時に、前記1つまたは複数のプロセッサが、 一連のメッセージを介して、前記ワイヤレスシンクデバイスと、サードパーティデバイスの機能をネゴシエートすることをさらに行わせる、請求項14に記載のワイヤレスシンクデバイス。
- 16前記命令の実行時に、前記1つまたは複数のプロセッサが、 前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスシンクデバイスから前記ワイヤレスソースデバイス にサ ードパーティデバイスの識別子を送信することをさらに行わせる、請求項14に記載のワイヤレスシンクデバイス。
- 17前記命令の実行時に、前記1つまたは複数のプロセッサが、 前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスソースデバイスか らサ ードパーティデバイスの識別子を受信することをさらに行わせる、請求項14に記載のワイヤレスシンクデバイス。
- 18前記データパケットヘッダが、ユーザ入力カテゴリーを識別するためのフィールドを備え、前記フィールドの値は、前記ペイロードデータが前記フォワーディングされたユーザ入力データを備えることを示すように設定される、請求項14に記載のワイヤレスシンクデバイス。
- 19前記ペイロードデータ がサ ードパーティデバイスの識別子を備える、請求項14に記載のワイヤレスシンクデバイス。
- 20サ ードパーティデバイス の識 別子が、前記サードパーティデバイスのIPアドレスと、前記サードパーティデバイスのドメイン名とからなるグループから選択される、請求項14に記載のワイヤレスシンクデバイス。
- 21サードパーティデバイスの 識別子が、前記ワイヤレスソースデバイスによって生成され、前記ワイヤレスシンクデバイスに送信される、請求項14に記載のワイヤレスシンクデバイス。
- 22前記外部デバイスが別のワイヤレスシンクデバイスである、請求項14に記載のワイヤレスシンクデバイス。
- 23前記外部デバイスが、前記ワイヤレスシンクデバイスに通信可能に結合された入力デバイスである、請求項14に記載のワイヤレスシンクデバイス。
- 24前記データパケットヘッダがアプリケーションレイヤパケットヘッダである、請求項14に記載のワイヤレスシンクデバイス。
- 25前記データパケットが、前記ワイヤレスソースデバイスのオーディオデータまたはビデオデータを制御するためのものである、請求項14に記載のワイヤレスシンクデバイス。
- 26前記データパケットがTCP/IPを介して送信される、請求項14に記載のワイヤレスシンクデバイス。
- 271つまたは複数のプロセッサによって実行されると、ワイヤレスシンクデバイスからワイヤレスソースデバイスにユーザ入力データを送信する方法を実行することを前記1つまたは複数のプロセッサに行わせる、命令を記憶するコンピュータ可読記憶媒体であって、前記方法は、 外部デバイスからユーザ入力データを取得することと、 データパケットヘッダを生成することであって、前記データパケットヘッダが、前記ユーザ入力データをフォワーディングされたユーザ入力データとして識別する、データパケットヘッダを生成することと、 前記ユーザ入力データを備えるペイロードデータを生成することと、 前記データパケットヘッダと前記ペイロードデータとを備えるデータパケットを生成することと、 前記ワイヤレスソースデバイスに前記データパケットを送信することとを備える、コンピュータ可読記憶媒体。
- 28ワイヤレスソースデバイスにユーザ入力データを送信するように構成されたワイヤレスシンクデバイスであって、前記ワイヤレスシンクデバイスは、 外部デバイスからユーザ入力データを取得するための手段と、 データパケットヘッダを生成するための手段であって、前記データパケットヘッダが、前記ユーザ入力データをフォワーディングされたユーザ入力データとして識別する、データパケットヘッダを生成するための手段と、 前記ユーザ入力データを備えるペイロードデータを生成するための手段と、 前記データパケットヘッダと前記ペイロードデータとを備えるデータパケットを生成するための手段と、 前記ワイヤレスソースデバイスに前記データパケットを送信するための手段とを備える、ワイヤレスシンクデバイス。
- 29ワイヤレスソースデバイスにおいてワイヤレスシンクデバイスからユーザ入力データを受信する方法であって、前記方法は、 ワイヤレスシンクデバイスからデータパケットヘッダとペイロードデータとを備えるデータパケットを受信することと、 前記ペイロードデータがフォワーディングされたユーザ入力コマンドを備えることを判断するために前記データパケットヘッダをパースすることと、 サードパーティデバイスの識別情報を識別するために前記ペイロードデータをパースすることと、 前記サードパーティデバイスの前記識別情報に基づいて前記ペイロードデータを処理することとを備える、方法。
- 30一連のメッセージを介して、前記ワイヤレスソースデバイスと、前記サードパーティデバイスの機能をネゴシエートすることをさらに備える、請求項29に記載の方法。
- 31前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスソースデバイスに前記サードパーティデバイスの識別子を送信することをさらに備える、請求項29に記載の方法。
- 32前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスソースデバイスから前記サードパーティデバイスの識別子を受信することをさらに備える、請求項29に記載の方法。
- 33前記データパケットヘッダが、ユーザ入力カテゴリーを識別するためのフィールドを備え、前記フィールドの値は、前記ペイロードデータが前記フォワーディングされたユーザ入力データを備えることを示すように設定される、請求項29に記載の方法。
- 34前記サードパーティデバイス の識 別子が、前記サードパーティデバイスのIPアドレスと、前記サードパーティデバイスのドメイン名とからなるグループから選択される、請求項29に記載の方法。
- 35前記 サードパーティデバイスの 識別子が、前記ワイヤレスソースデバイスによって生成され、前記ワイヤレスシンクデバイスに送信される、請求項29に記載の方法。
- 36前記サードパーティ デバイスが別のワイヤレスシンクデバイスである、請求項29に記載の方法。
- 37前記 サードパーティ デバイスが、前記ワイヤレスシンクデバイスに通信可能に結合された入力デバイスである、請求項29に記載の方法。
- 38前記データパケットヘッダがアプリケーションレイヤパケットヘッダである、請求項29に記載の方法。
- 39前記データパケットが、前記ワイヤレスソースデバイスのオーディオデータまたはビデオデータを制御するためのものである、請求項29に記載の方法。
- 40前記データパケットがTCP/IPを介して送信される、請求項29に記載の方法。
- 41ワイヤレスシンクデバイスからユーザ入力データを受信するように構成されたワイヤレスソースデバイスであって、前記ワイヤレスソースデバイスは、 ワイヤレスシンクデバイスからデータパケットヘッダとペイロードデータとを備えるデータパケットを受信するように構成されたトランスポートユニットと、 命令を記憶するメモリと、 前記命令を実行するように構成された1つまたは複数のプロセッサであって、前記命令の実行時に、前記1つまたは複数のプロセッサが、 前記ペイロードデータがフォワーディングされたユーザ入力コマンドを備えることを判断するために前記データパケットヘッダをパースすることと、 サードパーティデバイスの識別情報を識別するために前記ペイロードデータをパースすることと、 前記サードパーティデバイスの前記識別情報に基づいて前記ペイロードデータを処理することとを行わせる、1つまたは複数のプロセッサとを備える、ワイヤレスソースデバイス。
- 42前記命令の実行時に、前記1つまたは複数のプロセッサが、 一連のメッセージを介して、前記ワイヤレスソースデバイスと、前記サードパーティデバイスの機能をネゴシエートすることを行わせる、請求項41に記載のワイヤレスソースデバイス。
- 43前記命令の実行時に、前記1つまたは複数のプロセッサが、 前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスソースデバイスに対する前記サードパーティデバイスの識別子を送信することを行わせる、請求項41に記載のワイヤレスソースデバイス。
- 44前記命令の実行時に、前記1つまたは複数のプロセッサが、 前記ワイヤレスソースデバイスと前記ワイヤレスシンクデバイスとの間の通信セッションを確立することの一部として、前記ワイヤレスソースデバイスから前記サードパーティデバイスの識別子を受信することをさらに行わせる、請求項41に記載のワイヤレスソースデバイス。
- 45前記データパケットヘッダが、ユーザ入力カテゴリーを識別するためのフィールドを備え、前記フィールドの値は、前記ペイロードデータが前記フォワーディングされたユーザ入力データを備えることを示すように設定される、請求項41に記載のワイヤレスソースデバイス。
- 46前記サードパーティデバイス の識 別子が、前記サードパーティデバイスのIPアドレスと、前記サードパーティデバイスのドメイン名とからなるグループから選択される、請求項41に記載のワイヤレスソースデバイス。
- 47前記 サードパーティデバイスの 識別子が、前記ワイヤレスソースデバイスによって生成され、前記ワイヤレスシンクデバイスに送信される、請求項41に記載のワイヤレスソースデバイス。
- 48前記サードパーティ デバイスが別のワイヤレスシンクデバイスである、請求項41に記載のワイヤレスソースデバイス。
- 49前記 サードパーティ デバイスが、前記ワイヤレスシンクデバイスに通信可能に結合された入力デバイスである、請求項41に記載のワイヤレスソースデバイス。
- 50前記データパケットヘッダがアプリケーションレイヤパケットヘッダである、請求項41に記載のワイヤレスソースデバイス。
- 51前記データパケットが、前記ワイヤレスソースデバイスのオーディオデータまたはビデオデータを制御するためのものである、請求項41に記載のワイヤレスソースデバイス。
- 52前記データパケットがTCP/IPを介して送信される、請求項41に記載のワイヤレスソースデバイス。
- 531つまたは複数のプロセッサによって実行されると、ワイヤレスソースデバイスにおいてワイヤレスシンクデバイスからユーザ入力データを受信する方法を実行することを前記1つまたは複数のプロセッサに行わせる、命令を記憶するコンピュータ可読記憶媒体であって、前記方法は、 ワイヤレスシンクデバイスからデータパケットヘッダとペイロードデータとを備えるデータパケットを受信することと、 前記ペイロードデータがフォワーディングされたユーザ入力コマンドを備えることを判断するために前記データパケットヘッダをパースすることと、 サードパーティデバイスの識別情報を識別するために前記ペイロードデータをパースすることと、 前記サードパーティデバイスの前記識別情報に基づいて前記ペイロードデータを処理することとを備える、コンピュータ可読記憶媒体。
- 54ワイヤレスシンクデバイスからユーザ入力データを受信するように構成されたワイヤレスソースデバイスであって、前記ワイヤレスソースデバイスは、 ワイヤレスシンクデバイスからデータパケットヘッダとペイロードデータとを備えるデータパケットを受信するための手段と、 前記ペイロードデータがフォワーディングされたユーザ入力コマンドを備えることを判断するために前記データパケットヘッダをパースするための手段と、 サードパーティデバイスの識別情報を識別するために前記ペイロードデータをパースするための手段と、 前記サードパーティデバイスの前記識別情報に基づいて前記ペイロードデータを処理するための手段とを備える、ワイヤレスソースデバイス。
Independent claims54
140 paragraphs, as filed
0001The entire contents of this application are incorporated herein by reference. US Provisional Application No. 61 / 435,194, filed January 21, 2011, US Provisional Application No. 61 / 447,592, filed February 28, 2011, US Provisional Application No. 61 / 448,312, filed March 2, 2011, US Provisional Application No. 61 / 450,101, filed March 7, 2011, US Provisional Application No. 61 / 467,535, filed March 25, 2011, US Provisional Application No. 61 / 467,543, filed March 25, 2011, US Provisional Application No. 61 / 514,863, filed August 3, 2011, and US Provisional Application No. 61 / 544,470 filed on October 7, 2011 Claim the interests of.
0002The present disclosure relates to techniques for transmitting data between a wireless source device and a wireless sink device.
0003A wireless display (WD) or Wi-Fi Display (WFD) system includes a wireless source device and one or more wireless sink devices. Each of the source device and the sink device can be either a mobile device or a wired device having wireless communication capability. One or more of the source and sink devices are, for example, mobile phones, portable computers with wireless communication cards, personal digital assistants (PDAs), portable media players, or so-called "smart" phones and "smart" pads. Alternatively, it may include tablets, electronic readers, or other such devices with wireless communication capabilities, including any type of wireless display, video game device, or other types of wireless communication devices. One or more of the source and sink devices may also include wired devices such as televisions, desktop computers, monitors, projectors, etc. that include communication capabilities.
0004The source device sends media data, such as audio-video (AV) data, to one or more of the sink devices participating in a particular media sharing session. Media data can be played on both the local display of the source device and each of the displays of the sink device. More specifically, each of the participating sink devices renders the received media data on its screen and audio equipment.
0005The present disclosure generally describes a system in which a wireless sink device can communicate with the wireless sink device. As part of a communication session, the wireless source device can send audio and video data to the wireless sink device, which can send the user input received by the wireless sink device to the wireless source device. it can. In this way, the user of the wireless sink device can control the wireless source device and control the content being transmitted from the wireless source device to the wireless sink device.
0006In one example, the method of sending user input data from a wireless sink device to a wireless source device is to obtain user input data from an external device and generate a data packet header, where the data packet header is the user input. Generating data packet headers that identify data as forwarded user input data, generating payload data with user input data, and generating data packets with data packet headers and payload data. Includes sending data packets to wireless source devices.
0007In another example, a wireless sink device is configured to send user input data to a wireless source device. The wireless sync device is a memory that stores instructions and one or more processors configured to execute the instructions, and when the instructions are executed, the one or more processors input user data from an external device. Acquiring data and generating a data packet header, the data packet header identifies the user input data as forwarded user input data, generates a data packet header, and generates the user input data. Includes one or more processors that allow the generation of payload data to be provided and the generation of data packets with data packet headers and payload data. The wireless sink device also includes a transport unit for sending data packets to wireless source devices.
0008In another example, when a computer-readable storage medium is run by one or more processors, it performs the method of sending user input data from a wireless sink device to a wireless source device to one or more processors. Memorize the instructions to be made. The above method is to acquire user input data from an external device and generate a data packet header, wherein the data packet header identifies the user input data as forwarded user input data. It includes generating, generating payload data with user input data, generating a data packet with a data packet header and payload data, and sending the data packet to a wireless source device.
0009In another example, a wireless sink device is configured to send user input data to a wireless source device. The wireless sync device is a means for acquiring user input data from an external device and a means for generating a data packet header, and the data packet header identifies the user input data as forwarded user input data. Means for generating data packet headers, means for generating payload data with user input data, means for generating data packets with data packet headers and payload data, and wireless source devices. Includes means for sending data packets to.
0010In another example, the method of receiving user input data from a wireless sync device on a wireless source device is to receive a data packet with a data packet header and payload data from the wireless sync device and the user to whom the payload data has been forwarded. Parsing the data packet header to determine that it has an input command, parsing the payload data to identify the identity of the third-party device, and parsing the payload data based on the identity of the third-party device. Includes processing.
0011In another example, the wireless source device is configured to receive user input data from the wireless sink device. The wireless source device includes a transport unit configured to receive a data packet with a data packet header and payload data from a wireless sink device. The wireless source device is a memory that stores instructions and one or more processors that are configured to execute the instructions, and when the instructions are executed, one or more processors forward the payload data. Parsing the data packet header to determine that it has a user-entered command, parsing the payload data to identify the identity of the third-party device, and the payload based on the identity of the third-party device. It also includes one or more processors that allow it to process and do data.
0012In another example, when a computer-readable storage medium is run by one or more processors, the wireless source device performs the method of receiving user input data from a wireless sink device on one or more processors. Memorize the instructions to be made. The above method is to receive a data packet with a data packet header and payload data from a wireless sink device and to parse the data packet header to determine that the payload data has forwarded user input commands. Includes parsing the payload data to identify the identity of the third party device and processing the payload data based on the identity of the third party device.
0013In another example, the wireless source device is configured to receive user input data from the wireless sink device. The wireless source device provides a means for receiving a data packet with a data packet header and payload data from a wireless sink device and a data packet header to determine that the payload data comprises a forwarded user input command. It includes means for parsing, means for parsing the payload data to identify the identity of the third party device, and means for processing the payload data based on the identity of the third party device.
0014<figref num="1A">The block diagram which shows an example of the source / sink system which can implement the technique of this disclosure.</figref><figref num="1B">A block diagram showing an example of a source / sink system with two sink devices.</figref><figref num="2">The figure which shows an example of the source device which can implement the technique of this disclosure.</figref><figref num="3">The figure which shows an example of the sink device which can implement the technique of this disclosure.</figref><figref num="4">The block diagram of the transmitter system and the receiver system which can implement the technique of this disclosure.</figref><figref num="5A">The figure which shows the exemplary message forwarding sequence for performing functional negotiation according to the technique of this disclosure.</figref><figref num="5B">The figure which shows the exemplary message forwarding sequence for performing functional negotiation according to the technique of this disclosure.</figref><figref num="6">The figure which shows the exemplary data packet which can be used to deliver the user input data acquired in a sink device to a source device.</figref><figref num="7A">A flowchart illustrating the techniques of the present disclosure that can be used for functional negotiation between a source device and a sink device.</figref><figref num="7B">A flowchart illustrating the techniques of the present disclosure that can be used for functional negotiation between a source device and a sink device.</figref><figref num="8A">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with user input data.</figref><figref num="8B">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with user input data.</figref><figref num="9A">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with user input data.</figref><figref num="9B">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with user input data.</figref><figref num="10A">A flowchart illustrating a technique of the present disclosure that can be used to send and receive data packets with time stamp information and user input data.</figref><figref num="10B">A flowchart illustrating a technique of the present disclosure that can be used to send and receive data packets with time stamp information and user input data.</figref><figref num="11A">A flowchart illustrating a technique of the present disclosure that can be used to send and receive data packets with time stamp information and user input data.</figref><figref num="11B">A flowchart illustrating a technique of the present disclosure that can be used to send and receive data packets with time stamp information and user input data.</figref><figref num="12A">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets containing voice commands.</figref><figref num="12B">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets containing voice commands.</figref><figref num="13A">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with multi-touch user input commands.</figref><figref num="13B">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with multi-touch user input commands.</figref><figref num="14A">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with user-input data forwarded from a third party device.</figref><figref num="14B">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets with user-input data forwarded from a third party device.</figref><figref num="15A">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets.</figref><figref num="15B">A flowchart illustrating the techniques of the present disclosure that can be used to send and receive data packets.</figref>
0015The present disclosure generally describes a system in which a wireless sink device can communicate with the wireless sink device. As part of a communication session, the wireless source device can send audio and video data to the wireless sink device, which can send the user input received by the wireless sink device to the wireless source device. it can. In this way, the user of the wireless sink device can control the wireless source device and control the content being transmitted from the wireless source device to the wireless sink device.
0016FIG. 1A is a block diagram showing an exemplary source / sync system 100 that may implement one or more of the techniques of the present disclosure. As shown in FIG. 1A, the system 100 includes a source device 120 that communicates with the sink device 160 over the communication channel 150. The source device 120 includes a memory for storing audio / video (A / V) data 121, a display 122, a speaker 123, an audio / video encoder 124 (also called an encoder 124), an audio / video control module 125, and the like. It may include a transmitter / receiver (TX / RX) unit 126. The sink device 160 includes a display 162, a speaker 163, an audio / video decoder 164 (also called a decoder 164), a transmitter / receiver unit 166, a user input (UI) device 167, and a user input processing module (UIPM). : user input processing It can include module) 168 and. The illustrated components form only one exemplary configuration for the Source / Sync System 100. Other components may contain fewer components than the components shown, or may include additional components other than those shown.
0017In the example of FIG. 1A, the source device 120 can display the video portion of the audio / video data 121 on the display 122 and output the audio portion of the audio / video data 121 on the speaker 123. Can the audio / video data 121 be stored locally on the source device 120 and accessed from an external storage medium such as a file server, hard drive, external memory, Blu-ray Disc, DVD, or other physical storage medium? Or it can be streamed to the source device 120 over a network connection such as the Internet. In some cases, audio / video data 121 may be captured in real time via the camera and microphone of the source device 120. The audio / video data 121 may include multimedia content such as video, television programs, or music, but may also include real-time content generated by the source device 120. Such real-time content can be, for example, video data generated by an application running on the source device 120 or captured, for example, as part of a video telephony session. As described in more detail, such real-time content may, in some cases, include video frames of user input options available for the user to select. In some cases, the audio / video data 121 may include video frames that are a combination of different types of content, such as video frames for videos or TV programs with user input options overlaid on the frames of the video.
0018In addition to locally rendering the audio / video data 121 through the display 122 and the speaker 123, the audio / video encoder 124 of the source device 120 can encode the audio / video data 121 and the transmitter / The receiver unit 126 can transmit the encoded data to the sink device 160 via the communication channel 150. The transmitter / receiver unit 166 of the sink device 160 receives the encoded data, the audio / video decoder 164 decodes the encoded data, and the decoded data is passed through the display 162 and the speaker 163. And output. In this way, the audio and video data rendered by the display 122 and the speaker 123 can be rendered simultaneously by the display 162 and the speaker 163. Audio data and video data can be composed of frames, which can be time synchronized with the video frame when rendered.
0019The audio / video encoder 124 and audio / video decoder 164 are alternatives to the new ITU-T H.264 standard called MPEG-4, Part10, Advanced Video Coding (AVC), or the H.265 standard. Any number of audio and video compression standards can be implemented, including high efficiency video coding (HEVC) standards. Many other types of proprietary or standardized compression techniques can also be used. Generally, the audio / video decoder 164 is configured to perform the reverse coding operation of the audio / video encoder 124. Although not shown in FIG. 1A, in some embodiments, the A / V encoder 124 and A / V decoder 164 may be integrated with the audio encoder and decoder, respectively, and are suitable MUX-DEMUX units, or other hardware. It can handle both audio and video encoding in a common data stream or separate data streams, including hardware and software.
0020As described in more detail below, the A / V encoder 124 may perform other coding functions in addition to implementing the video compression standard as described above. For example, the A / V encoder 124 may add various types of metadata to the A / V data 121 before the A / V data 121 is transmitted to the sink device 160. In some cases, the A / V data 121 may be stored in or received in the source device 120 in encoded form and therefore does not require further compression by the A / V encoder 124. is there.
0021Figure 1A shows that the communication channel 150 carries the audio payload data and the video payload data separately, but in some cases the video payload data and the audio payload data are part of a common data stream. Please understand that it can be. If applicable, the MUX-DEMUX unit is ITU It may comply with the H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP). The audio / video encoder 124 and audio / video decoder 164 are each one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software. , Hardware, firmware, or any combination thereof. Each of the audio / video encoder 124 and the audio / video decoder 164 may be included in one or more encoders or decoders, both of which may be integrated as part of a composite encoder / decoder (codec). Thus, each of the source device 120 and the sink device 160 may be equipped with a dedicated machine configured to perform one or more of the techniques of the present disclosure.
0022Display 122 and Display 162 have a variety of video outputs, including cathode ray tubes (CRTs), liquid crystal displays (LCDs), plasma displays, light emitting diode (LED) displays, organic light emitting diode (OLED) displays, or other types of display devices. It may be equipped with any of the devices. In these or other examples, the displays 122 and 162 can be emissive displays or transmissive displays, respectively. The display 122 and display 162 can also be touch displays, such that they are both input and display devices at the same time. Such a touch display can be a capacitive, resistive, or other type of touch panel that allows the user to provide user input to each device.
0023Speaker 123 may include any of a variety of audio output devices, such as headphones, single speaker systems, multi-speaker systems, or surround sound systems. Further, the display 122 and the speaker 123 are shown as part of the source device 120, and the display 162 and the speaker 163 are shown as part of the sink device 160, but the source device 120 and the sink device 160 are actually plural. It can be a system of devices. As an example, the display 162 can be a television, the speaker 163 can be a surround sound system, and the decoder 164 can be part of an external box that is wired or wirelessly connected to the display 162 and the speaker 163. In another example, the sink device 160 can be a single device, such as a tablet computer or smartphone. In yet other cases, the source device 120 and the sink device 160 are similar devices, for example, both are smartphones, tablet computers, and so on. In this case, one device can act as a source and the other can act as a sink. These roles can even be reversed in subsequent communication sessions. In yet other cases, the source device may include a mobile device, such as a smartphone, laptop or tablet computer, and the sink device may include a more fixed device (eg, using an AC power cord), in which case the source. The device may deliver audio and video data for presentation to a large crowd via a sync device.
0024The transmitter / receiver unit 126 and the transmitter / receiver unit 166 are each a variety of mixers, filters, amplifiers, and other components designed for signal modulation, as well as one or more antennas. And may include other components designed to transmit and receive data. The communication channel 150 generally represents any communication medium suitable for transmitting video data from the source device 120 to the sink device 160, or a collection of various communication media. The communication channel 150 is usually a relatively short-range communication channel similar to Wi-Fi, Bluetooth (registered trademark), and the like. However, the communication channel 150 is not necessarily limited in this regard, and may be any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines, or a wireless medium and a wired medium. Can be any combination with. In another example, the communication channel 150 may even form part of a packet-based network, such as a wired or wireless local area network, a wide area network, or a global network such as the Internet. In addition, communication channel 150 may be used by source device 120 and sink device 160 to create peer-to-peer links. The source device 120 and sink device 160 may communicate over the communication channel 150 using communication protocols such as standards from the IEEE 802.11 standard family. The source device 120 and sink device 160 comply with the Wi-Fi Direct standard so that, for example, the source device 120 and sink device 160 communicate directly with each other without the use of intermediates such as wireless access points or so-called hotspots. Can communicate. Source device 120 and sink device 160 are also tunneled direct link setups (TLDS:) to avoid or reduce network congestion. tunneled direct link setup) can be established. Although the techniques of the present disclosure are sometimes described with respect to Wi-Fi, it is contemplated that aspects of these techniques may be compatible with other communication protocols. As an example, but not a limitation, wireless communication between the source device 120 and the sink device may utilize orthogonal frequency division multiplexing (OFDM) techniques. A wide variety of others, including, but not limited to, time division multiple access (TDMA), frequency division multiple connection (FDMA), code division multiple connection (CDMA), or any combination of OFDM, FDMA, TDMA and / or CDMA. Wireless communication techniques can also be used. WiFi Direct and TDLS aim to set up relatively short-range communication sessions. In this context, relatively short distances can refer, for example, less than 70 meters, but in noisy or obstructive environments, distances between devices can be even shorter, such as less than 35 meters. Direct and TDLS aim to set up relatively short-range communication sessions. In this context, relatively short distances can refer, for example, less than 70 meters, but in noisy or obstructive environments, distances between devices can be even shorter, such as less than 35 meters. Direct and TDLS aim to set up relatively short-range communication sessions. In this context, relatively short distances can refer, for example, less than 70 meters, but in noisy or obstructive environments, distances between devices can be even shorter, such as less than 35 meters.
0025In addition to decoding and rendering the data received from the source device 120, the sink device 160 can also receive user input from the user input device 167. The user input device 167 can be, for example, a keyboard, mouse, trackball or trackpad, touch screen, voice command recognition module, or other such user input device. The UIPM168 formats the user input command received by the user input device 167 into a data packet structure that can be interpreted by the source device 120. Such data packets are transmitted by the transmitter / receiver 166 to the source device 120 over the communication channel 150. The transmitter / receiver unit 126 receives the data packet, and the A / V control module 125 parses the data packet to interpret the user input command received by the user input device 167. Based on the commands received in the data packet, the A / V control module 125 can modify the encoded and transmitted content. In this way, the user of the sink device 160 can control the audio payload data and the video payload data transmitted by the source device 120 remotely and without interacting directly with the source device 120. Examples of the types of commands that a user of sink device 160 can send to source device 120 are commands for rewinding, fast-forwarding, interrupting, and playing audio and video data, as well as for zooming, rotating, and scrolling. Includes commands and more. The user may also make a selection, for example from a menu of options, and send the selection to the source device 120.
0026In addition, the user of sink device 160 may be able to launch and control applications on source device 120. For example, a user of sync device 160 may be able to launch a photo editing application stored on source device 120 and use that application to edit photos stored locally on source device 120. The sink device 160 can provide the user with a user experience in which the photo appears and feels locally edited on the sink device 160 while the photo is actually being edited on the source device 120. With such a configuration, device users may be able to take advantage of the capabilities of one device for use with several devices. For example, the source device 120 can be a smartphone with a large amount of memory and high-end processing power. The user of the source device 120 may use the smartphone in all settings and situations in which the smartphone is commonly used. However, when watching a video, the user may wish to watch the video on a device with a larger display screen, in which case the sink device 160 may be a tablet computer or a larger display device or television. When wishing to send or respond to e-mail, the user may wish to use a device with a keyboard, in which case the sink device 160 may be a laptop. In both cases, the user is interacting with the sink device, but most of the processing can still be performed by the source device 120 (smartphone in this example). In this particular operating context, most of the processing is done by the source device 120, so the sink device 160 is more than if the sink device 160 is asked to do the processing that is being done by the source device 120. , Using a few resources, yo It can be a low cost device. In some examples, both source and sink devices may be able to receive user input (such as touch screen commands), and the techniques of the present disclosure show the functionality of those devices in a given session. Two-way dialogue can be enabled by negotiating and / or identifying.
0027<u style="single">A few</u>In this configuration, the A / V control module 125 is the source device.<u style="single">120</u>It can be an operating system process running by the operating system of. However, in other configurations, the A / V control module 125 can be the software process of the application running on the source device 120. In such a configuration, user input commands can be interpreted by software processes so that the user of sink device 160 runs on source device 120 as opposed to the operating system running on source device 120. You are interacting directly with the application you are using. By interacting directly with the application as opposed to the operating system, users of sink device 160 may have access to a library of commands that are not native to the operating system of source device 120. In addition, by interacting directly with the application, commands may be more easily transmitted and processed by devices running on different platforms.
0028The source device 120 can respond to user input applied in the wireless sink device 160. In such an interactive application configuration, the user input applied in the wireless sink device 160 may be sent to the wireless display source via the communication channel 150. In one example, the user to allow the sink device 160 to send the user input applied at the sink device 160 to the source device 120.<u style="single">input</u>Back channel (UIB)<u style="single">C)</u>A reverse channel architecture, also known as, can be implemented. The reverse channel architecture may include a higher layer message for transporting user input and a lower layer frame for negotiating user interface functionality on the sink device 160 and the source device 120. UIBC may reside on top of the Internet Protocol (IP) transport layer between sink device 160 and source device 120. In this way, UIBC can be above the transport layer in the Open Systems Interconnection (OSI) communication model. In one example, OSI communication includes seven layers: 1-physical, 2-datalink, 3-network, 4-transport, 5-session, 6-presentation, and 7-application. In this example, being above the transport layer refers to layers 5, 6, and 7. To facilitate the reliable transmission and sequential delivery of data packets containing user-entered data, UIBC uses other packets, such as Transmission Control Protocol / Internet Protocol (TCP / IP) or User Datagram Protocol (UDP). It can be configured to run on top of the base communication protocol. UDP and TCP can operate in parallel in the OSI layer architecture. TCP / IP may allow sink device 160 and source device 120 to implement retransmission techniques in the event of packet loss.
0029In some cases, there may be a discrepancy between the user input interface located on the source device 120 and the user input interface located on the sink device 160. In order to resolve the potential problems posed by such discrepancies and promote a good user experience under such circumstances, user input interface feature negotiations may be performed prior to establishing a communication session or in communication. It can occur between the source device 120 and the sink device 160 at various times throughout the session. As part of this negotiation process, the source device 120 and the sink device 160 can agree on the negotiated screen resolution. When the sync device 160 transmits the coordinate data associated with the user input, the sync device 160 can scale the coordinate data obtained from the display 162 to match the negotiated screen resolution. In one example, if the sink device 160 has a resolution of 1280 x 720 and the source device 120 has a resolution of 1600 x 900, then those devices may use, for example, 1280 x 720 as their negotiated resolution. The negotiated resolution may be selected based on the resolution of the sink device 160, but the resolution of the source device 120 or some other resolution may also be used. In the example where a 1280x720 sink device is used, the sink device 160 can scale the obtained x-coordinates at a ratio of 1600/1280 before sending them to the source device 120, as well. In addition, the sink device 160 can scale the obtained y-coordinates by 900/720 before sending them to the source device 120. In other configurations, the source device 120 can scale the acquired coordinates to the negotiated resolution. Its scaling is sink device 1
0030Moreover, in some cases, the resolution at the sink device 160 can fluctuate during the communication session, potentially resulting in a discrepancy between the display 122 and the display 162. To improve the user experience and ensure proper functionality, Source / Sync System 100 implements techniques to reduce or prevent user interaction discrepancies by implementing techniques for screen normalization. obtain. The display 122 of the source device 120 and the display 162 of the sink device 160 may have different resolutions and / or different aspect ratios. In addition, in some settings, the user of the sync device 160 can source the video data received from the source device 120 so that it is rendered in a window that covers a smaller portion of the sink device 160's display 162. It may have the ability to resize the display window for video data received from device 120. In another exemplary setting, the user of the sink device 160 may have the option to view content in either landscape mode or portrait mode, each of which has unique coordinates and different aspect ratios. Have. In such situations, the coordinates associated with the user input received on the sink device 160, such as the coordinates of where the mouse click or touch event occurs, are processed by the source device 120 without modification to those coordinates. It may not be possible. Thus, the techniques of the present disclosure may include mapping the coordinates of user input received on the sink device 160 to the coordinates associated with the source device 120. This mapping, also referred to herein as normalization, can be either sink-based or source-based, as described in more detail below.
0031User input received by the sink device 160 may be received by the UI module 167 and passed to the operating system of the sink device 160, for example at the driver level. The operating system on the sink device 160 has the coordinates (x) associated with where the user input took place on the display surface.<sub>SINK</sub>, y<sub>SINK</sub>) Can be received. In this example, (x<sub>SINK</sub>, y<sub>SINK</sub>) Can be the coordinates of the display 162 where the mouse click or touch event took place. The display window rendered on display 162 has an x-coordinate length (L) that describes the size of the display window.<sub>DW</sub>) And y coordinate width (W)<sub>DW</sub>) And may have. The display window is the upper left corner coordinate (a) that describes the location of the display window.<sub>DW</sub>, b<sub>DW</sub>) May also have. L<sub>DW</sub>, W<sub>DW</sub>, And upper left coordinates (a<sub>DW</sub>, b<sub>DW</sub>), The portion of the display 162 covered by the display window can be determined. For example, the upper right corner of the display window is the coordinates (a<sub>DW</sub>+ L<sub>DW</sub>, b<sub>DW</sub>), The lower left corner of the display window is the coordinates (a)<sub>DW</sub>, b<sub>DW</sub>+ W<sub>DW</sub>), The lower right corner of the display window is the coordinates (a)<sub>DW</sub>+ L<sub>DW</sub>, b<sub>DW</sub>+ W<sub>DW</sub>) Can be located. When the input is received at the coordinates in the display window, the sink device 160 can process the input as a UIBC input. In other words, the associated coordinates (x)<sub>SINK</sub>, y<sub>SINK</sub>The input in) can be processed as a UIBC input if the following conditions are met:<maths num="1"><img id="000002" he="28" wi="158" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths><maths num="2"><img id="000003" he="26" wi="158" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths>
0032After determining that the user input is a UIBC input, the coordinates associated with that input can be normalized by UIPM168 before being sent to the source device 120. Input determined to be outside the display window may be processed locally by the sink device 160 as non-UIBC input.
0033As mentioned above, the input coordinate normalization can be either source-based or sync-based. When implementing sync-based normalization, the source device 120 has a supported display resolution (L) for display 122.<sub>SRC</sub>, W<sub>SRC</sub>) Can be sent to the sink device 160 with or without video data. The supported display resolutions may be transmitted, for example, as part of a functional negotiation session or at another time during the communication session. The sink device 160 has a display resolution (L) for the display 162.<sub>SINK</sub>, W<sub>SINK</sub>) And the display window resolution (L) for the window displaying the content received from the source device 120<sub>DW</sub>, W<sub>DW</sub>) And the upper left corner coordinates for the display window (a)<sub>DW</sub>, b<sub>DW</sub>) And can be judged. As explained above, the coordinates (x) corresponding to the user input<sub>SINK</sub>, y<sub>SINK</sub>When) is determined to be in the display window, the operating system of the sink device 160 uses a transform function to determine the coordinates (x).<sub>SINK</sub>, y<sub>SINK</sub>) To source coordinates (x<sub>SRC</sub>, y<sub>SRC</sub>) Can be mapped. (x<sub>SINK</sub>, y<sub>SINK</sub>) To (x<sub>SRC</sub>, y<sub>SRC</sub>An exemplary conversion function for converting to) can be:<maths num="3"><img id="000004" he="27" wi="158" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths><maths num="4"><img id="000005" he="27" wi="158" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths>
0034Therefore, when transmitting the coordinates corresponding to the received user input, the sink device 160 is (x).<sub>SINK</sub>, y<sub>SINK</sub>) Coordinates for user input received in (x)<sub>SRC</sub>, y<sub>SRC</sub>) Can be sent. Coordinates (x), as described in more detail below.<sub>SRC</sub>, y<sub>SRC</sub>) May be transmitted, for example, as part of a data packet used to transmit user input received at sink device 160 to source device 120 via UIBC. Throughout the rest of the disclosure, where the input coordinates are described as being contained within the data packet, those coordinates are as described above in the case where the source / sync system 100 implements sink-based normalization. Can be converted to source coordinates.
0035When the source / sync system 100 implements source-based normalization, user input determined to be UIBC input as opposed to local input (ie, inside the display window as opposed to outside the display window). For, the above calculation can be performed on the source device 120 instead of the sink device 160. To enable such calculations, the sink device 160 is L.<sub>DW</sub>, W<sub>DW</sub>Values for, as well as location information about the display window (eg a)<sub>DW</sub>, b<sub>DW</sub>), And (x<sub>SINK</sub>, y<sub>SINK</sub>) Can be sent to the source device 120. Using these transmitted values, the source device 120 follows Equations 3 and 4 above (x).<sub>SRC</sub>, y<sub>SRC</sub>) Can be determined.
0036In another implementation of sync-based normalization, the sink device 160 describes where in the display window the user input event occurs, as opposed to where the user input event occurs on the display 162. Coordinates for user input (x<sub>DW</sub>, y<sub>DW</sub>) Can be sent. In such an implementation, the coordinates (x)<sub>DW</sub>, y<sub>DW</sub>) Is (L<sub>DW</sub>, W<sub>DW</sub>Can be sent 120 to the source device with a value for). Based on these received values, the source device 120 follows the following conversion function (x)<sub>SRC</sub>, y<sub>SRC</sub>) Can be judged.<maths num="5"><img id="000006" he="12" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths><maths num="6"><img id="000007" he="27" wi="158" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths>
0037The sink device 160 is x based on the following function<sub>DW</sub>And y<sub>DW</sub>Can be judged.<maths num="7"><img id="000008" he="26" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths><maths num="8"><img id="000009" he="11" wi="158" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></maths>
0038In the present disclosure, for example, when describing transmitting the coordinates associated with a user input in a data packet, the transmission of these coordinates may include sink-based or source-based normalization as described above. , And / or may contain any additional information needed to perform sink-based or source-based normalization.
0039UIBC can be designed to transport various types of user input data, including cross-platform user input data. For example, the source device 120 may run an iOS® operating system, and the sink device 160 may run another operating system, such as Android® or Windows®. Regardless of the platform, the UIPM168 can encapsulate the received user input in a form understandable to the A / V control module 125. Several different types of users allow their source and sink devices to take advantage of the protocol, regardless of whether many different types of source and sink devices operate on different platforms. Input formats may be supported by UIBC. General input formats can be defined and both platform-specific input formats can be supported, thus providing flexibility in the way user input can be communicated between the source device 120 and the sink device 160 by UIBC.
0040In the example of Figure 1A, the source device 120 may include a smartphone, tablet computer, laptop computer, desktop computer, Wi-Fi capable television, or other device capable of transmitting audio and video data. The sync device 160 can also be a smartphone, tablet computer, laptop computer, desktop computer, Wi-Fi enabled television, or any other device that can receive audio and video data and receive user input data. Can be prepared. In some cases, the sink device 160 is such that the display 162, speaker 163, UI device 167, and A / V encoder 164 are all separate but interoperable devices. It may include a system of devices. Similarly, the source device 120 can be a system of multiple devices rather than a single device.
0041In this disclosure, the term source device is used generally to refer to a device that is transmitting audio / video data, and the term sink device is generally used to refer to a device that is receiving audio / video data from a source device. Used to point to. In many cases, the source device 120 and the sink device 160 can be similar or equivalent devices, with one device acting as the source and the other acting as the sink. Moreover, these roles can be reversed in different communication sessions. Thus, a sink device in one communication session can be a source device in subsequent communication sessions, and vice versa.
0042FIG. 1B is a block diagram showing an exemplary source / sync system 101 that may implement the techniques of the present disclosure. The source / sink system 101 includes a source device 120 and a sink device 160, each of which may function and operate in the manner described above for FIG. 1A. The source / sink system 101 further includes a sink device 180. In a manner similar to the sink device 160 described above, the sink device 180 may receive audio and video data from the source device 120 and send user commands to the source device 120 via the established UIBC. In some configurations, the sink device 160 and the sink device 180 can operate independently of each other, and the audio and video data output by the source device 120 is output simultaneously by the sink device 160 and the sink device 180. obtain. In an alternative configuration, the sink device 160 can be the primary sink device and the sink device 180 can be the secondary sink device. In such an exemplary configuration, the sink device 160 and the sink device 180 may be combined, the sink device 160 may display video data, and the sink device 180 may output the corresponding audio data. Further, in some configurations, the sink device 160 may output only the transmitted video data and the sink device 180 may output only the transmitted audio data.
0043FIG. 2 is a block diagram showing an example of the source device 220. The source device 220 can be a device similar to the source device 120 in FIG. 1A and can operate in the same way as the source device 120. The source device 220 includes a local display 222, a local speaker 223, a processor 231, a memory 232, a transport unit 233, and a wireless modem 234. As shown in FIG. 2, the source device 220 may include one or more processors (ie, processor 231) that encode and / or decode A / V data for transport, storage, and display. A / V data can be stored, for example, in memory 232. Memory 232 may include the entire A / V file or may include a smaller buffer that may store, for example, only a portion of the A / V file streamed from another device or source. Transport unit 233 may process encoded A / V data for network transport. For example, encoded A / V data can be processed by processor 231 and encapsulated in a network access layer (NAL) unit by transport unit 233 for communication over the network. The NAL unit can be sent by the wireless modem 234 to the wireless sink device over a network connection. The wireless modem 234 can be, for example, a Wi-Fi modem configured to implement one of the IEEE 802.11 standard families.
0044The source device 220 may also process and display A / V data locally. In particular, the display processor 235 may process the video data to be displayed on the local display 222, and the audio processor 236 may process the audio data for output on the speaker 223.
0045As described above for the source device 120 of FIG. 1A, the source device 220 may also receive user input commands from the sink device. In this way, the wireless modem 234 of the source device 220 receives the encapsulated data packet, such as the NAL unit, and sends the encapsulated data unit to the transport unit 233 for decapsulation. For example, transport unit 233 can extract data packets from the NAL unit, and processor 231 can parse the data packets to extract user input commands. Based on user input commands, processor 231 can adjust the encoded A / V data transmitted by the source device 220 to the sink device. In this way, the functionality described above for the A / V control module 125 of FIG. 1A may be fully or partially implemented by processor 231.
0046Processor 231 in Figure 2 is generally, but not limited to, one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), and field programmable logic arrays (FPs).<u style="single">L</u>Represents any of a wide variety of processors, including A), other equivalent integrated circuits or discrete logic circuits, or any combination thereof. The memory 232 in FIG. 2 is, but is not limited to, random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), and electrically erasable programmable. It can have either a wide variety of volatile or non-volatile memory, including read-only memory (EEPROM), flash memory, and so on. The memory 232 may include a computer-readable storage medium for storing audio / video data as well as other types of data. Memory 232 may further store instructions and program code executed by processor 231 as part of performing the various techniques described herein.
0047Figure 3 shows an example of the sink device 360. The sink device 360 can be a device similar to the sink device 160 in FIG. 1A and can operate in the same way as the sink device 160. The sink device 360 includes one or more processors (ie, processor 331), memory 332, transport unit 333, wireless modem 334, display processor 335, local display 362, audio processor 336, and speaker 363. And the user input interface 376. The sink device 360 receives the encapsulated data unit sent from the source device in the wireless modem 334. The wireless modem 334 can be, for example, a Wi-Fi modem configured to implement one or more standards from the IEEE 802.11 standard family. The transport unit 333 can decapsulate the encapsulated data unit. For example, transport unit 333 may extract encoded video data from the encapsulated data unit and send the encoded A / V data to processor 331 for decoding and rendering for output. The display processor 335 may process the decoded video data as it appears on the local display 362, and the audio processor 336 may process the decoded audio data for output on the speaker 363.
0048In addition to rendering audio and video data, the wireless sync device 360 can also receive user input data through user input interface 376. The user input interface 376 is one of several user input devices, including, but not limited to, a touch display interface, keyboard, mouse, voice command module, and gesture capture device (eg, with camera-based input capture capabilities). , Or other user input devices of some user input devices. User input received through user input interface 376 may be processed by processor 331. This process may include generating a data packet containing the received user input command according to the techniques described herein. Once generated, transport unit 333 may process the data packet for network transport to a wireless source device via UIBC.
0049Processor 331 in Figure 3 is one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), other equivalent integrated circuits or discrete logic circuits. It may have one or more of a wide range of processors, or any combination thereof. The memory 332 of FIG. 3 is, but is not limited to, random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), and electrically erasable programmable. It can have either a wide variety of volatile or non-volatile memory, including read-only memory (EEPROM), flash memory, and so on. memory<u style="single">332</u>May include computer-readable storage media for storing audio / video data as well as other types of data. Memory 332 may further store instructions and program code executed by processor 331 as part of performing the various techniques described herein.
0050FIG. 4 shows an exemplary transmitter system 410 and receiver system 450 that can be used by transmitter / receiver 126 and transmitter / receiver 166 of FIG. 1A to communicate over communication channel 150. A block diagram is shown. In transmitter system 410, traffic data for several data streams is fed from data source 412 to transmit (TX) data processor 414. Each data stream may be transmitted via its own transmitting antenna. The TX data processor 414 formats, codes, and interleaves the traffic data for each data stream based on the particular coding scheme selected for that data stream.
0051The encoded data in each data stream can be multiplexed with pilot data using Orthogonal Frequency Division Multiplexing (OFDM) techniques. A wide variety of others, including, but not limited to, time-division multiple access (TDMA), frequency division multiple connection (FDMA), code division multiple connection (CDMA), or any combination of OFDM, FDMA, TDMA and / or CDMA. Wireless communication techniques can also be used.
0052According to FIG. 4, the pilot data is typically a known data pattern processed in a known manner and can be used in the receiver system to estimate the channel response. The multiplexed pilot and coded data of each data stream is then given a particular modulation scheme (eg, 2-phase shift keying (BPSK), 4-phase shift) selected for that data stream to give a modulation symbol. Modulated (eg, symbol mapping) based on keying (QPSK), M-PSK, or M-QAM (quadrature keying), where M can be a power of 2. The data rate, coding, and modulation of each data stream can be determined by instructions executed by processor 430, which can be coupled to memory 432.
0053A modulation symbol for the data stream is then given to the TX MIMO processor 420, which may further process that modulation symbol (for example, for OFDM). The TX MIMO processor 420 is then N<sub>T</sub>N modulation symbol streams<sub>T</sub>It can be given to multiple transmitters (TMTR) 422a ~ 422t. In some embodiments, the TX MIMO processor 420 applies beamforming weights to the symbols of the data stream and to the antenna at the source of the symbols.
0054Each transmitter 422 receives and processes its own symbol stream to give it one or more analog signals, and then tunes (eg, amplifies, filters, and upconverts) those analog signals. , A modulated signal suitable for transmission via a MIMO channel can be provided. Then N from transmitters 422a ~ 422t<sub>T</sub>Each modulated signal is N<sub>T</sub>It is transmitted from the antennas 424a to 424t.
0055In the receiver system 450, the transmitted modulated signal is N<sub>R</sub>Received by the antennas 452a to 452r, the received signal from each antenna 452 is given to the respective receiver (RCVR) 454a to 454r. The receiver 454 tunes (eg, filters, amplifies, and down-converts) each received signal, digitizes the tuned signal, gives a sample, and further processes those samples to accommodate the corresponding " Gives a "receive" symbol stream.
0056The receive (RX) data processor 460 is then N<sub>R</sub>Receivers 454 to N<sub>R</sub>Receives a received symbol stream and processes it based on a particular receiver processing technique, N<sub>T</sub>Gives a "detection" symbol stream. The RX data processor 460 then demodulates, deinterleaves, and decodes each detected symbol stream to restore the traffic data in the data stream. The processing by the RX data processor 460 complements the processing performed by the TX MIMO processor 420 and the TX data processor 414 in the transmitter system 410.
0057Processor 470, which can be combined with memory 472, periodically determines which precoding matrix should be used. A reverse link message can contain various types of information about a communication link and / or a received data stream. The reverse link message is then processed by the TX data processor 438, which also receives traffic data for some data streams from the data source 436, modulated by modulator 480, tuned by transmitters 454a-454r, and transmitter. Returned to system 410.
0058In transmitter system 410, the modulated signal from receiver system 450 is received by antenna 424, tuned by receiver 422, demodulated by demodulator 440, processed by RX data processor 442, and received by receiver system 450. The reverse link message sent by is extracted. Processor 430 then determines which precoding matrix should be used to determine the beamforming weights, and then processes the extracted message.
0059FIG. 5A is a block diagram showing an exemplary message forwarding sequence between the source device 520 and the sink device 560 as part of a functional negotiation session. Functional negotiation can occur as part of the larger communication session establishment process between the source device 520 and the sink device 560. This session can be established, for example, with Wi-Fi Direct or TDLS as the underlying connectivity standard. After establishing a Wi-Fi Direct or TDLS session, the sink device 560 can initiate a TCP connection with the source device 520. As part of establishing a TCP connection, a control port can be established running a real time streaming protocol (RTSP) to manage communication sessions between the source device 520 and the sink device 560. ..
0060The source device 520 may generally operate in the same manner as described above for the source device 120 in FIG. 1A, and the sink device 560 may generally operate in the same manner as described above for the sink device 160 in FIG. 1A. Can work. After the source device 520 and sink device 560 establish connectivity, the source device 520 and sink device 560 are parameters that should be used for subsequent communication sessions of those devices as part of a functional negotiation exchange. You can judge the set.
0061The source device 520 and sink device 560 can negotiate functionality through a sequence of messages. Those messages can be, for example, Real Time Streaming Protocol (RTSP) messages. At any stage of the negotiation, the recipient of the RTSP request message may respond with an RTSP response containing an RTSP status code other than RTSP OK, in which case the message exchange may be retried with a different set of parameters. Alternatively, the functional negotiation session can be terminated.
0062The source device 520 can send a first message (RTSP OPTIONS request message) to the sink device 560 to determine the set of RTSP methods supported by the sink device 560. Upon receiving the first message from the source device 520, the sink device 560 can respond with a second message (RTSP OPTIONS response message) listing the RTSP methods supported by the sink 560. Also, the second message may contain the RTSP OK status code.
0063After sending the second message to the source device 520, the sink device 560 can send a third message (RTSP OPTIONS request message) to determine the set of RTSP methods supported by the source device 520. Upon receiving the third message from the sink device 560, the source device 520 can respond with a fourth message (RTSP OPTIONS response message) listing the RTSP methods supported by the source device 520. Also, the fourth message can include the RTSP OK status code.
0064After sending the fourth message, the source device 520 can send a fifth message (RTSP GET_PARAMETER request message) to specify a list of features related to the source device 520. The sink device 560 can respond with a sixth message (RTSP GET_PARAMETER response message). The sixth message may contain the RTSP status code. If the RTSP status code is OK, the sixth message can also include response parameters to the parameters specified in the fifth message, supported by sink device 560. The sink device 560 can ignore the parameters in the fifth message that the sink device 560 does not support.
0065Based on the sixth message, the source 520 can determine the optimal set of parameters to be used for the communication session and can send a seventh message (RTSP SET_PARAMETER request message) to the sink device 560. it can. The seventh message may contain a set of parameters that should be used during the communication session between the source device 520 and the sink device 560. The seventh message should be used in an RTSP Setup request to set up a communication session<u style="single">uniform</u>Resource identifier (UR<u style="single">I)</u>Can include wfd-presentation-url to describe. wfd-presentation-url specifies a URI that sink device 560 can use for later messages during a session establishment exchange. The wfd-url0 and wfd-url1 values specified in this parameter can correspond to the rtp-port0 and rtp-port1 values in wfd-client-rtp-ports in the seventh message. .. RTP in this case generally refers to a real-time protocol that can run on top of UDP.
0066Upon receiving the seventh message, the sink device 560 can respond with an eighth message with an RTSP status code indicating whether the parameters specified in the seventh message were successfully set. .. As mentioned above, the roles of the source device and the sink device can be reversed or changed in different sessions. The order of messages that set up a communication session can, in some cases, define a device that acts as a source and a device that acts as a sink.
0067Figure 5B shows the source device as part of a functional negotiation session.<u style="single">520</u>And sink device<u style="single">560</u>It is a block diagram which shows another exemplary message forwarding sequence between and. The message forwarding sequence of FIG. 5B is intended to provide a more detailed diagram of the forwarding sequence described above for FIG. 5A. In Figure 5B, the message "1b.GET_PARAMETER response" shows an example of a message that identifies a list of supported input categories (eg, general and HIDC) from a list of supported input types. List of Supported Input Categories Each of the supported input categories has an associated list of supported types (eg generic_cap_list and hidc_cap_list). In Figure 5B, the message "2a.SET_PARAMETER request" identifies a second list of supported input categories (eg, general and HIDC) from a second list of supported types. This is an example of a message. Second List of Supported Input Categories Each of the supported input categories has an associated second list of supported types (eg generic_cap_list and hidc_cap_list). The message "1b.GET_PARAMETER response" identifies the input categories and input types supported by the sink device 560. The message "2a.SET_PARAMETER request" identifies the input categories and input types supported by source device 520, but it is not a comprehensive list of all input categories and input types supported by source device 520. Sometimes. Instead, the message "2a.SET_PARAMETER request" may only identify the input category and input type identified in the message "1b.GET_PARAMETER response" as supported by the sink device 560. In this way, the message "2a.
0068FIG. 6 is a conceptual diagram showing an example of a data packet that can be generated by a sink device and transmitted to a source device. Although aspects of the data packet 600 will be described with reference to FIG. 1A, the techniques described may be applicable to additional types of source / sink systems. The data packet 600 may include a data packet header 610, followed by payload data 650. Payload data 650 may further include one or more payload headers (eg, payload header 630). The data packet 600 may be transmitted from the sink device 160 of FIG. 1A to the source device 120 so that, for example, the user of the sink device 160 can control the audio / video data transmitted by the source device 120. In such cases, the payload data 650 may include user input data received at the sink device 160. Payload data 650 may identify, for example, one or more user commands. The sink device 160 can receive one or more user commands and can generate data packet headers 610 and payload data 650 based on the received commands. Based on the content of the data packet header 610 of the data packet 600, the source device 120 can parse the payload data 650 to identify the user input data received by the sink device 160. Based on the user input data contained in the payload data 650, the source device 120 may somehow modify the audio and video data transmitted from the source device 120 to the sink device 160.
0069As used in this disclosure, the terms "parse" and "persing" generally refer to the process of analyzing a bitstream to extract data from the bitstream. Once extracted, the data can be processed, for example, by the source device 120. Extracting the data can include, for example, identifying how the information in the bitstream is formatted. As described in more detail below, the data packet header 610 may define a standardized format known to both the source device 120 and the sink device 160. However, the payload data 650 can be formatted in one of many possible ways. By parsing the data packet header 610, the source device 120 can determine how the payload data 650 is formatted, so the source device 120 can be one or more users from the payload data 650. Payload data 650 can be parsed to extract input commands. This can provide flexibility in terms of different types of payload data that may be supported in source sink communication. As described in more detail below, the payload data 650, one or more payload header such as a payload header 630 may also include a da. In such a case, the source device 120 parses the data packet header 610 to determine the format for the payload header 630 and then the payload header 630 to determine the format for the rest of the payload data 650. obtain.
0070FIG. 620 is a conceptual diagram of how the data packet header 610 can be formatted. The numbers 0 to 15 in line 615 are intended to identify the bit locations within the data packet header 610 and are not intended to actually represent the information contained within the data packet header 610. The data packet header 610 includes a version field 621, a time stamp flag 622, a reserved field 623, an input category field 624, a length field 625, and an optional time stamp field 626.
0071In the example of FIG. 6, version field 621 is a 3-bit field that may indicate the version of a particular communication protocol implemented by sink device 160. The value in the version field 621 may inform the source device 120 how to parse the rest of the data packet header 610 and how to parse the payload data 650. In the example in Figure 6, version field 621 is a 3-bit field that will allow unique identifiers for eight different versions. In other examples, more or less bits may be dedicated to version field 621.
0072In the example of FIG. 6, the time stamp flag (T) 622 is a 1-bit field indicating whether or not the time stamp field 626 is present in the data packet header 610. Timestamp field 626 is a 16-bit field that contains a time stamp based on multimedia data generated by the source device 120 and sent to the sink device 160. The time stamp can be, for example, a continuous value assigned by the source device 120 to the frame of the video before it is transmitted to the sink device 160. The time stamp flag 622 may include, for example, a "1" to indicate that the time stamp field 626 exists and a "0" to indicate that the time stamp field 626 does not exist. If the data packet header 610 is parsed and it is determined that the time stamp field 626 is present, the source device 120 can process the time stamp contained in the time stamp field 626. If it parses the data packet header 610 and determines that the timestamp field 626 does not exist, the source device 120 will parse the length field 625 and then the payload data 650 because the timestamp field does not exist in the data packet header 610. You can start parsing.
0073If present, the time stamp field 626 can include a time stamp to identify a frame of video data that was displayed on the wireless sink device 160 when the user input data of the payload data 650 was retrieved. The time stamp may be added to the video frame by the source device 120, for example, before the source device 120 sends the video frame to the sink device 160. Thus, the source device 120 may generate a frame of video and embed a time stamp in the video data of that frame, for example as metadata. The source device 120 can send the video frame to the sink device 160 along with the time stamp, and the sink device 160 can display the frame of the video. The sink device 160 can receive user commands from the user while the frame of the video is displayed by the sink device 160. When the sink device 160 generates a data packet to forward the user command to the source device 120, the sink device 160 time stamps the time stamp of the frame displayed by the sink device 160 when the user command is received. Can be included in field 626.
0074When the timestamp field 626 receives the data packet 600 present in the header, the wireless source device 120 identifies the frame of the video displayed on the sink device 160 when the user input data for the payload data 650 is retrieved. , User input data can be processed based on the content of the frame identified by the time stamp. For example, if the user input data is a touch command applied to the touch display or a mouse pointer click, the source device 120 is displayed when the user applies the touch command to the display or clicks the mouse. You can determine the content of the frame you are in. In some cases, frame content may be required to properly process the payload data. For example, user input based on user touch or mouse click may depend on what was shown on the display at the time of touch or click. Touches or clicks may correspond to, for example, icons or menu options. In the case where the content of the display is changing, the time stamp present in the time stamp field 626 can be used by the source device 120 to match the touch or click to the correct icon or menu option.
0075The source device 120 may, in addition or as an alternative, compare the time stamp in the time stamp field 626 with the time stamp applied to the currently rendered frame of the video. By comparing the time stamp in the time stamp field 626 with the current time stamp, the source device 120 can determine the round trip time. Round-trip time generally corresponds to the amount of time elapsed from the time a frame was transmitted by the source device 120 to the time user input based on that frame was received back from the sink device 160 to the source device 120. .. The round trip time can give the source device 120 an indication of system latency, and if the round trip time is greater than the threshold, the source device 120 will assume that the input command was applied to the old display frame. Therefore, the user input data contained in the payload data 650 can be ignored. When the round trip time is less than the threshold, the source device 120 may process the user input data and adjust the audio / video content being transmitted in response to the user input data. The thresholds can be programmable and different types of devices (or different source sink combinations) can be configured to define different thresholds for acceptable round trip times.
0076In the example of FIG. 6, reserved field 623 is an 8-bit field that does not contain the information used by source 120 when parsing data packet header 610 and payload data 650. However, a future version of a particular protocol (identified in version field 621) may utilize reserved field 623, in which case the source device 120 is to parse the data packet header 610 and / or The information in the reserved field 623 can be used to parse the payload data 650. Reserved field 623, along with version field 621, has the potential to extend features and add features to the data packet format without essentially changing the format and features already in use. Give to.
0077In the example of FIG. 6, the input category field 624 is a 4-bit field for identifying the input category for the user input data contained in the payload data 650. The sink device 160 may categorize user input data to determine the input category. Categorizing user input data can be based, for example, on the device on which the command was received, or on the nature of the command itself. The value of the input category field 624 identifies to the source device 120 how the payload data 650 is formatted, optionally along with other information in the data packet header 610. Based on this formatting, the source device 120 can parse the payload data 650 to determine the user input received by the sink device 160.
0078In the example of FIG. 6, since the input category 624 is 4 bits, 16 different input categories can be identified in some cases. One such input category is that the user input data in payload data 650 is formatted using the general information elements defined in the protocol running by both the source device 120 and the sink device 160. Can be a general input format to indicate. The general input format may utilize general information elements that allow the user of the sink device 160 to interact with the source device 120 at the application level, as described in more detail below.
0079Another such input category is the human interface device command (to indicate that the user input data in payload data 650 is formatted based on the type of input device used to receive the input data. HIDC: human interface device command) format is possible. Examples of device types include keyboards, mice, touch input devices, joysticks, cameras, gesture capture devices (such as camera-based input devices), and remote control devices. Other types of input categories that can be identified in the input category field 624 are the forwarding input format, or operating system specific format, and payload to indicate that the user data in the payload data 650 did not originate on the sink device 160. Includes a voice command format to indicate that data 650 contains voice commands.
0080The length field 625 may include a 16-bit field to indicate the length of the data packet 600. The length can be expressed in units of 8 bits, for example. When the data packet 600 is parsed by the source device 120 with a 16-bit word, the data packet 600 can be padded to a 16-bit integer. Based on the length contained in the length field 625, the source device 120 can distinguish between the end of payload data 650 (ie, the end of data packet 600) and the start of a new subsequent data packet.
0081The various sizes of the fields given in the example of Figure 6 are for illustration purposes only, and those fields may be implemented using a different number of bits than those shown in Figure 6. It is also contemplated that the data packet header 610 may contain fewer fields than all the fields described above, or may use additional fields not described above. In fact, the techniques of the present disclosure can be flexible in terms of the actual format used for the various data fields of the packet.
0082After parsing the data packet header 610 to determine the formatting of the payload data 650, the source device 120 can parse the payload data 650 to determine the user input commands contained in the payload data 650. .. The payload data 650 may have its own payload header (payload header 630) indicating the contents of the payload data 650. In this way, the source device 120 may parse the payload header 630 based on the parsing of the data packet header 610 and then the remaining payload data 650 based on the parsing of the payload header 630.
0083For example, if the input category field 624 of the data packet header 610 indicates that a general input is present in the payload data 650, the payload data 650 may have a general input format. Therefore, the source device 120 can parse the payload data 650 according to the general input format. As part of the general input format, the payload data 650 can include a series of one or more input events, where each input event has its own input event header. Table 1 below identifies the fields that can be included in the input header.<tables num="1"><img id="000010" he="59" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0084The General Input Event (IE) Identification Field identifies the general input event identification information for identifying the input type. The general IE ID field can be, for example, one octet in length and may contain identification information selected from Table 2 below. If the general IE ID field is 8-bit, as in this example, 256 different types of inputs (identified as 0-255) can be identifiable, but not all 256 identities are necessarily identifiable. It does not always require an associated input type. Some of the above 256 may be reserved for future use of future versions of any protocol implemented by the sink device 160 and the source device 120. In Table 2, for example, general IE IDs 9-255 do not have an associated input type, but may be assigned an input type in the future.
0085The length field in the input event header identifies the length of the descriptive field, and the descriptive field contains an information element that describes the user input. The formatting of the descriptive field can depend on the type of input identified in the general IE ID field. Therefore, the source device 120 may parse the contents of the descriptive field based on the input type identified in the general IE ID field. Based on the length field of the input event header, the source device 120 can determine the end of one input event and the start of a new input event in the payload data 650. As described in more detail below, a user command can be described as one or more input events in payload data 650.
0086Table 2 gives an example of an input type, each with a corresponding general IE ID that can be used to identify the input type.<tables num="2"><img id="000011" he="100" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0087The descriptive fields associated with each input type can have different formats. The left mouse down / touchdown event, left mouse up / touch up event, and mouse move / touch move event descriptive fields may contain, for example, the information elements identified in Table 3 below, but other Other formats may be used in the example.<tables num="3"><img id="000012" he="145" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0088The number of pointers can identify the number of touches or mouse clicks associated with an input event. Each pointer can have a unique pointer ID. For example, if the multi-touch event contains a three-finger touch, each input event may have three pointers, each with a unique pointer ID. Each pointer (ie, each finger touch) may have a corresponding x-coordinate and y-coordinate corresponding to where the touch was made.
0089A single user command can be described as a series of input events. For example, if a three-finger swipe is a command to close an application, a three-finger swipe uses a touchdown event with three pointers, a touch movement event with three pointers, and a three pointers. It can be described in the payload data 650 as the touchup event that was there. The three pointers for a touchdown event can have the same pointer ID as the three pointers for a touch movement event and a touchup event. The source device 120 can interpret the combination of those three input events as a three-finger swipe.
0090The descriptive fields for key-down or key-up events can contain, for example, the information elements identified in Table 4 below.<tables num="4"><img id="000013" he="105" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0091The zoom event description field may contain, for example, the information elements identified in Table 5 below.<tables num="5"><img id="000014" he="109" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0092The description fields for horizontal or vertical scrolling events may include, for example, the information elements identified in Table 6 below.<tables num="6"><img id="000015" he="78" wi="159" file="JP5714726B2_D0001.tif" img-format="tif" img-content="drawing" /></tables>
0093The above example shows some exemplary ways in which payload data can be formatted in the case of general input categories. Payload data 650 may have different input formats if the input category field 624 of the data packet header 610 indicates a different input category, such as forwarded user input. For forwarded user input, the sink device 160 may receive user input data from a third party device and forward the input to the source device 120 without interpreting the user input data. Therefore, the source device 120 can parse the payload data 650 according to the forwarded user input format. For example, the payload header 630 of the payload data 650 may include a field for the user input to identify a third party device obtained from it. The field may include, for example, the Internet Protocol (IP) address, MAC address, domain name, or any other such identifier of a third party device. The source device 120 can parse the rest of the payload data based on the identifier of the third party device.
0094The sink device 160 can negotiate features with a third party device via a series of messages. The sink device 160 can then send a unique identifier for the third party device to the source device 120 as part of establishing a communication session with the source device 120 as part of the functional negotiation process. Alternatively, the sink device 160 may send information describing the third party device to the source device 120, from which the source device 120 can determine a unique identifier for the third party device. The information describing the third party device may include, for example, information for identifying the third party device and / or information for identifying the function of the third party device. When the sink device 160 sends a data packet with user input obtained from a third party device, whether the unique identifier is determined by the source device 120 or the sink device 160, the sink device 160 Can include a unique identifier in the data packet, for example in the payload header, so that the source device 120 can identify the origin of the user input.
0095If the input category field 624 of the data packet header 610 indicates a different input category, such as a voice command, the payload data 650 may have a different input format. For voice commands, the payload data 650 may include coded audio. A codec for encoding and decoding the audio of a voice command can be negotiated between the source device 120 and the sink device 160 via a series of messages. To send a voice command, the timestamp field 626 may include a voice sampling time value. In such cases, the time stamp flag 622 may be set to indicate that a time stamp is present, but instead of the time stamp described above, the time stamp field 626 is for the encoded audio of the payload data 650. Can include audio sampling time values of.
0096In some examples, the voice command may be sent as a general command as described above, in which case the input category field 624 may be set to identify the general command format and of the reserved general IE ID. One of them can be assigned to a voice command. If the voice command is sent as a general command, the voice sampling rate can be present in the timestamp field 626 of the data packet header 610 or in the payload data 650.
0097For captured voice command data, the voice data can be encapsulated in multiple ways. For example, voice command data can be encapsulated using RTP, which can be given a payload type to identify the codec and timestamp, which timestamp is used to identify the sampling rate. RTP data can be encapsulated using the general user input format described above with or without arbitrary time stamps. The sink device 160 can use TPC / IP to send general input data that carries voice command data to the source device 120.
0098As described earlier, when coordinates are included as part of a data packet, such as data packet 600, in, for example, payload data 650, the coordinates are negotiated resolution, display window coordinates, normalized coordinates, or It may correspond to coordinates scaled based on the coordinates associated with the sync display. In some cases, additional information may be included in the data packet or transmitted separately for use by the source device to normalize the coordinates received in the data packet.
0099Regardless of the input category for a particular data packet, the data packet header can be an application layer packet header and the data packet can be sent over TCP / IP. TCP / IP may allow sink device 160 and source device 120 to perform retransmission techniques in the event of packet loss. Data packets are sent from sink device 160 to source device 120 for other purposes, such as to control audio or video data on source device 120, or to control applications running on source device 120. Can be sent.
0100FIG. 7A is a flowchart of an exemplary method of negotiating functionality between a sink device and a source device. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more of the illustrated steps in one or more of the flowcharts described herein. It may store instructions, modules, or algorithms that cause one or more processors (eg, processor 331) to do what they want to do.
0101The method of FIG. 7A involves the sink device 160 receiving a first message from the source device 120 (701). The message may include, for example, a get parameter request. In response to the first message, sink device 160 sends a second message to source device 120 (703). The second message is, for example, a get parameter that identifies the first list of supported input categories from the first list of supported types. Response) may be provided and each of the supported input categories in the first list of supported input categories has an associated first list of supported types. The supported input categories may correspond to, for example, the same categories used for the input category field 624 in FIG. Table 2 above represents an example of the supported types for a particular input category (general input in this example). The sink device 160 receives a third message from the source device 120 (705). The third message is, for example, a set parameter A request) can be provided, the parameter setting request identifies the port for communication, the second list of supported input categories, and multiple second lists of supported types, and the supported inputs. Each of the supported input categories in the second list of categories has an associated second list of supported types, and each of the supported types in the second list is in the first list. Contains a subset of types. The sink device 160 sends a fourth message to the source device 120 (707). The fourth message may include, for example, a set parameter response to confirm that the type in the second list has been enabled. The sink device 160 receives a fifth message from the source device 120 (709). The fifth message may include, for example, a second parameter setting request indicating that the communication channel between the source device 120 and the sink device 160 has been enabled. The communication channel is, for example, the user input back (UIBC). channel) can be provided. The sink device 160 sends a sixth message to the source device 120 (711). The sixth message may include, for example, a second parameter setting response confirming receipt of the second parameter setting request by the sink device 160.
0102FIG. 7B is a flowchart of an exemplary method of negotiating functionality between a sink device and a source device. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0103The method of FIG. 7B involves the source device 120 sending a first message to the sink device 160 (702). The first message may include, for example, a parameter acquisition request. The source device 120 receives a second message from the sink device 160 (704). The second message may include, for example, a parameter acquisition response that identifies the first list of supported input categories from the plurality of first lists of supported types, and the first of the supported input categories. Each of the supported input categories of the list has an associated first list of supported types. The source device 120 sends a third message to the sink device 160 (706). The third message may include, for example, a parameter setting request that identifies a port for communication, a second list of supported input categories, and multiple second lists of supported types. Each of the supported input categories in the second list of input categories to be made has an associated second list of supported types, and each of the supported types in the second list is the first. Contains a subset of the types of lists in. The source device 120 receives a fourth message from the sink device 160 (708). The fourth message may include, for example, a parameter setting response to confirm that the type in the second list has been enabled. The source device 120 sends a fifth message to the sink device 160 (710). The fifth message may include, for example, a second parameter setting request indicating that the communication channel between the source device 120 and the sink device 160 has been enabled. The communication channel may include, for example, a user input back channel (UIBC). Source device 120 receives a sixth message from sink device 160 (712). The sixth message confirms, for example, that the sink device 160 has received a second parameter setting request.
0104FIG. 8A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0105The method of FIG. 8A involves acquiring user input data in a wireless sink device such as the wireless sink device 160 (801). User input data can be obtained through the user input component of the wireless sync device 160, for example, the user input interface 376 shown for the wireless sync device 360. In addition, the sink device 160 may categorize user input data as, for example, general, forwarded, or operating system specific. The sink device 160 then generates a data packet header based on user input data (803). The data packet header can be an application layer packet header. The data packet header may include, among the fields, a field for identifying an input category corresponding to user input data. The input category may include, for example, a general input format or a human interface device command. The sink device 160 further generates a data packet (805), which comprises the generated data packet header and payload data. In one example, the payload data may include received user input data and may identify one or more user commands. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in Figure 1A or source device 220 in Figure 2) (807). The sink device 160 may include components that allow the transfer of data packets, including, for example, a transport unit 333 and a wireless modem 334 as shown in FIG. The sink device 160 may forward data packets over TCP / IP.
0106FIG. 8B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0107The method of FIG. 8B involves receiving a data packet (802), which may include, in particular, a data packet header and payload data. The payload data may include, for example, user input data. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown with reference to FIG. The source device 120 then parses the data packet header contained in the data packet to determine the input category associated with the user input data contained in the payload data (804). Source device 120 processes payload data based on the determined input category (806). The data packets described with reference to FIGS. 8A and 8B can generally take the form of the data packets described with reference to FIG. 6 and are used to control audio / video data and applications in the source device. obtain.
0108FIG. 9A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0109The method of FIG. 9A involves acquiring user input data in a wireless sink device such as the wireless sink device 160 (901). User input data can be obtained through the user input component of the wireless sink device 160, for example, the user input interface 376 shown with reference to FIG. The sink device 160 then generates payload data (903), which may describe user input data. In one example, the payload data may include received user input data and may identify one or more user commands. The sink device 160 further generates a data packet (905), which comprises a data packet header and the generated payload data. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in Figure 1A or source device 220 in Figure 2) (907). The sink device 160 may include components that allow the transfer of data packets, such as transport unit 333 and wireless modem 334. Data packets can be sent to wireless source devices over TCP / IP.
0110FIG. 9B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0111The method of FIG. 9B involves receiving a data packet from the sink device 360 (902), which may include, in particular, a data packet header and payload data. In one example, the payload data may include data that describes user input details, such as input type values. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown with reference to FIG. The source device 120 then parses the data packet to determine the input type value in the input type field in the payload data (904). The source device 120 processes data describing the details of the user input based on the determined input type value (906). The data packet described with reference to FIGS. 9A and 9B may generally take the form of the data packet described with reference to FIG.
0112FIG. 10A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0113The method of FIG. 10A involves acquiring user input data (1001) on a wireless sink device such as the wireless sink device 160. The user input data can be obtained through the user input component of the wireless sink device 160, for example, the user input interface 376 as shown with reference to FIG. The sink device 160 then generates a data packet header based on user input (1003). The data packet header may include, among other fields, a timestamp flag (eg, a 1-bit field) to indicate whether the timestamp field is present in the data packet header. The time stamp flag may include, for example, a "1" to indicate that the time stamp field exists and a "0" to indicate that the time stamp field does not exist. The time stamp field is, for example, a 16-bit field containing a time stamp generated by the source device 120 and can be added to the video data prior to transmission. The sink device 160 further generates a data packet (1005), which comprises the generated data packet header and payload data. In one example, the payload data may include received user input data and may identify one or more user commands. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in Figure 1A or source device 220 in Figure 2) (1007). The sink device 160 may include components that allow the transfer of data packets, including, for example, transport unit 333 and wireless modem 334, as shown with reference to FIG. Data packets can be sent to wireless source devices over TCP / IP.
0114FIG. 10B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0115The method of FIG. 10B involves receiving a data packet from the wireless sync device 160 (1002), which may specifically include a data packet header and payload data. The payload data may include, for example, user input data. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown with reference to FIG. The source device 120 then parses the data packet header contained in the data packet (1004). The source device 120 determines if the timestamp field is present in the data packet header (1006). In one example, the source device 120 may make a decision based on the time stamp flag value contained in the data packet header. If the data packet header contains a timestamp field, the source device 120 processes the payload data based on the timestamp in the timestamp field (1008). The data packet described with reference to FIGS. 10A and 10B can generally take the form of the data packet described with reference to FIG. 6 and can be used to control audio / video data in the source device.
0116FIG. 11A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0117The method of FIG. 11A involves acquiring user input data (1101) on a wireless sink device such as the wireless sink device 160. User input data can be obtained through the user input component of the wireless sink device 160, for example, the user input interface 376 shown with reference to FIG. The sink device 160 then generates a data packet header based on user input (1103). The data packet header may include a time stamp field among the fields. The time stamp field may include, for example, a 16-bit field containing a time stamp based on multimedia data generated by the wireless source device 120 and transmitted to the wireless sync device 160. The time stamp may have been added to the frame of the video data by the wireless source device 120 before it was sent to the wireless sink device. The time stamp field may identify, for example, the time stamp associated with a frame of video data displayed on the wireless sync device 160 when user input data is captured. The sink device 160 further generates a data packet (1105), which comprises the generated data packet header and payload data. In one example, the payload data may include received user input data and may identify one or more user commands. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in FIG. 1A or source device 220 in FIG. 2) (1107). The sink device 160 may include components that allow the transfer of data packets, including, for example, transport unit 333 and wireless modem 334, as shown with reference to FIG. Data packets can be sent to wireless source devices over TCP / IP.
0118FIG. 11B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0119The method of FIG. 11B involves receiving a data packet from a wireless sync device such as the wireless sync device 160 (1102), the data packet may specifically include a data packet header and payload data. The payload data may include, for example, user input data. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown with reference to FIG. Source device 120 then identifies the timestamp field in the data packet header (1104). Source device 120 processes payload data based on the time stamp in the time stamp field (1106). Based on the above time stamp, as part of processing the payload data, the source device 120 identifies a frame of video data that is displayed on the wireless sync device when the user input data is retrieved and that frame. The payload data can be interpreted based on the contents of. As part of processing the payload data based on the time stamp, the source device 120 compares the time stamp with the current time stamp for the current frame of video being transmitted by the source device 120. Obtained, in response to the time difference between the above time stamp and the current time stamp being less than the threshold, execute the user input command described in the payload data, or execute the above time stamp and the current time stamp. The user input command described in the payload data may not be executed in response to the time difference from the time stamp being greater than the threshold. The data packet described with reference to FIGS. 11A and 11B can generally take the form of the data packet described with reference to FIG. 6 and can be used to control audio / video data in the source device.
0120FIG. 12A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0121The method of FIG. 12A involves acquiring user input data (1201) on a wireless sink device such as the wireless sink device 160. In one example, the user input data can be voice command data, which is obtained through the user input component of the wireless sync device 160, for example, the voice command recognition module included in the user input interface 376 in FIG. Can be done. The sink device 160 generates a data packet header based on user input (1203). The sink device 160 also generates payload data (1205), which may include voice command data. In one example, the payload data may also include received user input data and may identify one or more user commands. The sink device 160 further generates a data packet (1207), which comprises the generated data packet header and payload data. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in Figure 1A or source device 220 in Figure 2) (1209). The sink device 160 may include components that allow the transfer of data packets, including, for example, transport unit 333 and wireless modem 334, as shown with reference to FIG. Data packets can be sent to wireless source devices over TCP / IP.
0122FIG. 12B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0123The method of FIG. 12B involves receiving a data packet (1202), which may include, in particular, a data packet header and payload data. The payload data may include user input data such as voice command data. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown with reference to FIG. The source device 120 then parses the payload data contained in the data packet to determine if the payload data comprises voice command data (1204). The data packet described with reference to FIGS. 12A and 12B can generally take the form of the data packet described with reference to FIG. 6 and can be used to control audio / video data in the source device.
0124FIG. 13A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0125The method of FIG. 13A involves acquiring user input data (1301) in a wireless sink device such as the wireless sink device 160. In one example, the user input data can be a multi-touch gesture, which can be obtained through a user input component of the wireless sync device 160, for example, UI167 or the user input interface 376 of FIG. In one example, a multi-touch gesture may include a first touch input and a second touch input. The sink device 160 generates a data packet header based on user input (1303). The sink device 160 also generates payload data (1305), which associates user input data for the first touch input event with the first pointer identification information and user input for the second touch input event. The data can be associated with a second pointer identification. The sink device 160 further generates a data packet (1307), which comprises the generated data packet header and payload data. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in Figure 1A or source device 220 in Figure 2) (1309). The sink device 160 may include components that allow the transfer of data packets, including, for example, transport unit 333 and wireless modem 334, as shown with reference to FIG. Data packets can be sent to wireless source devices over TCP / IP.
0126FIG. 13B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0127The method of FIG. 13B includes receiving a data packet (1302), which may include, in particular, a data packet header and payload data. The payload data may include user input data such as multi-touch gestures. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown in FIG. The source device 120 then parses the payload data contained in the data packet to identify the user input data contained in the payload data (1304). In one example, the identified data is the user input data for the first touch input event using the first pointer identification information and the user input for the second touch input event using the second pointer identification information. May include data. The source device 120 then interprets the user input data for the first touch input event and the user input data for the second touch input event as multi-touch gestures (1306). The data packet described with reference to FIGS. 13A and 13B can generally take the form of the data packet described with reference to FIG. 6 and can be used to control audio / video data in the source device.
0128FIG. 14A is a flowchart of an exemplary method of transmitting user input data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0129The method of FIG. 14A involves retrieving user input data from an external device on the wireless sink device 360 (1401). In one example, the external device can be a third-party device connected to the sink device. The sink device 160 generates a data packet header based on user input (1403). In one example, the data packet header may identify the user input data as forwarded user input data. The sink device 160 also generates payload data (1405), which may include user input data. The sink device 160 further generates a data packet (1407), which may include the generated data packet header and payload data. The sink device 160 then sends the generated data packet to a wireless source device (eg, source device 120 in Figure 1A or source device 220 in Figure 2) (1409). The sink device 160 may include components that allow the transfer of data packets, including, for example, transport unit 333 and wireless modem 334, as shown with reference to FIG. Data packets can be sent to wireless source devices over TCP / IP.
0130FIG. 14B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0131The method of FIG. 14B comprises receiving a data packet (1402), which may include, in particular, a data packet header and payload data. The payload data may include user input data, for example, a forwarded user input command indicating that the user input data has been forwarded from a third party device. The source device 120 may include communication components that allow the transfer of data packets, including, for example, transport unit 233 and wireless modem 234, as shown with reference to FIG. The source device 120 then parses the data packet header and determines that the payload data comprises a forwarded user input command (1404). Source device 120 then parses the payload data contained in the data packet to identify the identity associated with the third-party device that corresponds to the forwarded user input command (1406). Source device 120 then processes payload data based on the identified identity of the third party device (1408). The data packet described with reference to FIGS. 14A and 14B can generally take the form of the data packet described with reference to FIG. 6 and can be used to control audio / video data in the source device.
0132FIG. 15A is a flowchart of an exemplary method of transmitting user data from a wireless sink device to a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by a sink device 160 (FIG. 1A) or 360 (FIG. 3). In some examples, when a computer-readable storage medium (eg, memory 332) is executed, one or more processors (eg, memory 332) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 331).
0133The method of FIG. 15A involves acquiring user input data (1501) in a wireless sink device. User input data may have associated coordinate data. The associated coordinate data may correspond to, for example, the location of a mouse click event or the location of a touch event. The sink device 160 then normalizes the associated coordinate data to generate the normalized coordinate data (1503). The sink device 160 then generates a data packet containing normalized coordinate data (1505). Normalizing the coordinate data is the resolution of the display window and the display of the source device 120.<u style="single">122</u>It can include scaling the associated coordinate data based on the ratio to the display resolution of the source, such as. The resolution of the display window can be determined by the sink device 160 and the display resolution of the source device can be received from the source device 120. The sink device 160 then sends a data packet with normalized coordinates to the wireless source device 120 (1507). As part of the method in Figure 15A, the sync device 160 also determines if the associated coordinate data is in the display window for the content being received from the wireless source device, eg, associated. If the coordinate data is outside the display window, the user input can be processed locally, or if the input is inside the display window, the coordinates can be normalized as described.
0134FIG. 15B is a flow chart of an exemplary method of receiving user input data from a wireless sink device in a wireless source device in accordance with the present disclosure. The illustrated exemplary method can be performed by source device 120 (FIG. 1A) or 220 (FIG. 2). In some examples, when a computer-readable storage medium (eg, memory 232) is executed, one or more processors (eg, memory 232) perform one or more of the illustrated steps in the flowchart. It may store instructions, modules, or algorithms to be performed by the processor 231).
0135The method of FIG. 15B involves receiving a data packet on a wireless source device, which comprises user input data with associated coordinate data (1502). The associated coordinate data may correspond to, for example, the location of a mouse click event or the location of a touch event on the sink device. The source device 120 then normalizes the associated coordinate data to generate the normalized coordinate data (1504). The source device 120 can normalize the coordinate data by scaling the associated coordinate data based on the ratio of the resolution of the display window to the resolution of the source display. The source device 120 can determine the display resolution of the source device and can receive the resolution of the display window from the wireless sink device. The source device then processes the data packet based on the normalized coordinate data (1506). The data packet described with reference to FIGS. 15A and 15B can generally take the form of the data packet described with reference to FIG. 6 and can be used to control audio / video data in the source device.
0136For the sake of brevity, aspects of the present disclosure have been described separately with reference to FIGS. 7-15. However, it is contemplated that these various aspects may be combined and used in relation to each other, not just separately. In general, the features and / or modules described herein can be implemented in either or both wireless source devices and wireless sink devices. In this way, the user interface features described in this example can be used interchangeably between wireless source devices and wireless sink devices.
0137The techniques of the present disclosure can be implemented in a wide variety of devices or devices, including wireless handsets and integrated circuits (ICs) or sets of ICs (ie, chipsets). Although any component, module or unit has been described and given to emphasize its functional aspects, those optional components, modules or units do not necessarily require implementation by different hardware units. ..
0138Therefore, the techniques described herein can be implemented in hardware, software, firmware, or any combination thereof. When implemented in hardware, the features described as modules, units or components can be implemented together in an integrated logical device or separately as a separate but interoperable logical device. When implemented in software, these techniques may be at least partially realized by a computer-readable medium with instructions that, when executed on a processor, performs one or more of the methods described above. The computer-readable medium may comprise a tangible, non-transitory computer-readable storage medium and may form part of a computer program product that may contain packaging material. Computer-readable storage media include random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), and electrically erasable programmable read-only memory (EEPROM). , Flash memory, magnetic or optical data storage medium, etc. may be provided. The technique may, in addition or as an alternative, be at least partially realized by a computer-readable communication medium that carries or communicates code in the form of instructions or data structures and can be accessed, read, and / or executed by a computer.
0139The code is one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrations. It can be performed by a circuit or a discrete logic circuit. Thus, the term "processor" as used herein may refer to either the structure described above or any other structure suitable for implementing the techniques described herein. Moreover, in some embodiments, the functionality described herein may be provided within a dedicated software or hardware module configured for encoding and decoding, or may be incorporated into a composite video codec. .. Also, the technique may be fully implemented in one or more circuits or logic elements.
0140Various aspects of the present disclosure have been described. These and other aspects fall within the scope of the following claims.<u style="single">In addition, the invention described in the claims at the time of filing is added below.</u><u style="single">[C1] A method of transmitting user input data from a wireless sink device to a wireless source device, wherein the method is</u><u style="single">Obtaining user input data from an external device and</u><u style="single">Generating a data packet header, wherein the data packet header identifies the user input data as forwarded user input data.</u><u style="single">Generating payload data with the user input data</u><u style="single">To generate a data packet including the data packet header and the payload data,</u><u style="single">To send the data packet to the wireless source device</u><u style="single">A method.</u><u style="single">[C2] Negotiating the functions of the wireless sink device and the third party device through a series of messages.</u><u style="single">The method described in C1 further comprising.</u><u style="single">[C3] Sending the identifier of the third party device from the wireless sink device to the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The method described in C1 further comprising.</u><u style="single">[C4] Receiving the identifier of the third party device from the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The method described in C1 further comprising.</u><u style="single">[C5] The data packet header comprises a field for identifying a user input category, and the value of the field is set to indicate that the payload data comprises the forwarded user input data, C1. The method described in.</u><u style="single">[C6] The method of C1, wherein the payload data comprises an identifier of the third party device.</u><u style="single">[C7] The method according to C1, wherein the identifier of the third party device is selected from a group consisting of the IP address of the third party device and the domain name of the third party device.</u><u style="single">[C8] The method of C1, wherein the identifier is generated by the wireless source device and transmitted to the wireless sink device.</u><u style="single">[C9] The method of C1, wherein the external device is another wireless sink device.</u><u style="single">[C10] The method of C1, wherein the external device is an input device communicatively coupled to the wireless sink device.</u><u style="single">[C11] The method according to C1, wherein the data packet header is an application layer packet header.</u><u style="single">[C12] The method of C1, wherein the data packet is for controlling audio or video data of the wireless source device.</u><u style="single">[C13] The method of C1, wherein the data packet is transmitted over TCP / IP.</u><u style="single">[C14] A wireless sink device configured to transmit user input data to a wireless source device, said wireless sink device.</u><u style="single">Memory to store instructions and</u><u style="single">One or more processors configured to execute the instruction, and when the instruction is executed, the one or more processors</u><u style="single">Obtaining user input data from an external device and</u><u style="single">Generating a data packet header, wherein the data packet header identifies the user input data as forwarded user input data.</u><u style="single">Generating payload data with the user input data</u><u style="single">To generate a data packet including the data packet header and the payload data.</u><u style="single">With one or more processors,</u><u style="single">With a transport unit for transmitting the data packet to the wireless source device</u><u style="single">A wireless sink device.</u><u style="single">[C15] At the time of executing the instruction, the one or more processors</u><u style="single">Negotiating the functionality of the wireless sink device and the third party device through a series of messages.</u><u style="single">The wireless sink device described in C14 that allows you to do more.</u><u style="single">[C16] At the time of executing the instruction, the one or more processors</u><u style="single">Sending the identifier of the third party device from the wireless sink device to the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The wireless sink device described in C14 that allows you to do more.</u><u style="single">[C17] At the time of executing the instruction, the one or more processors</u><u style="single">Receiving the identifier of the third party device from the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The wireless sink device described in C14 that allows you to do more.</u><u style="single">[C18] The data packet header comprises a field for identifying a user input category, and the value of the field is set to indicate that the payload data comprises the forwarded user input data, C14. Wireless sync device described in.</u><u style="single">[C19] The wireless sink device according to C14, wherein the payload data comprises an identifier for the third party device.</u><u style="single">[C20] The wireless sink device according to C14, wherein the identifier of the third party device is selected from a group consisting of the IP address of the third party device and the domain name of the third party device.</u><u style="single">[C21] The wireless sink device according to C14, wherein the identifier is generated by the wireless source device and transmitted to the wireless sink device.</u><u style="single">[C22] The wireless sink device according to C14, wherein the external device is another wireless sink device.</u><u style="single">[C23] The wireless sink device according to C14, wherein the external device is an input device communicatively coupled to the wireless sink device.</u><u style="single">[C24] The wireless sink device according to C14, wherein the data packet header is an application layer packet header.</u><u style="single">[C25] The wireless sink device according to C14, wherein the data packet is for controlling audio or video data of the wireless source device.</u><u style="single">[C26] The wireless sink device according to C14, wherein the data packet is transmitted over TCP / IP.</u><u style="single">[C27] Store instructions that, when executed by one or more processors, cause the one or more processors to perform a method of transmitting user input data from a wireless sink device to a wireless source device. A computer-readable storage medium, wherein the method</u><u style="single">Obtaining user input data from an external device and</u><u style="single">Generating a data packet header, wherein the data packet header identifies the user input data as forwarded user input data.</u><u style="single">Generating payload data with the user input data</u><u style="single">To generate a data packet including the data packet header and the payload data,</u><u style="single">To send the data packet to the wireless source device</u><u style="single">A computer-readable storage medium.</u><u style="single">[C28] A wireless sink device configured to transmit user input data to a wireless source device, said wireless sink device.</u><u style="single">A means to obtain user input data from an external device,</u><u style="single">A means for generating a data packet header, wherein the data packet header identifies the user input data as forwarded user input data, and a means for generating a data packet header.</u><u style="single">A means for generating payload data including the user input data, and</u><u style="single">A means for generating a data packet including the data packet header and the payload data, and</u><u style="single">With means for transmitting the data packet to the wireless source device</u><u style="single">A wireless sink device.</u><u style="single">[C29] A method of receiving user input data from a wireless sink device in a wireless source device, wherein the method is:</u><u style="single">Receiving a data packet with a data packet header and payload data from a wireless sink device</u><u style="single">Parsing the data packet header to determine that the payload data comprises a forwarded user input command.</u><u style="single">Parsing the payload data to identify the identity of the third party device,</u><u style="single">Processing the payload data based on the identification information of the third party device</u><u style="single">A method.</u><u style="single">[C30] Negotiating the functionality of the wireless source device and the third party device through a series of messages.</u><u style="single">The method described in C29, further comprising.</u><u style="single">[C31] Sending the identifier of the third party device to the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The method described in C29, further comprising.</u><u style="single">[C32] Receiving an identifier of the third party device from the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The method described in C29, further comprising.</u><u style="single">[C33] The data packet header comprises a field for identifying a user input category, and the value of the field is set to indicate that the payload data comprises the forwarded user input data, C29. The method described in.</u><u style="single">[C34] The method according to C29, wherein the identifier of the third party device is selected from a group consisting of the IP address of the third party device and the domain name of the third party device.</u><u style="single">[C35] The method of C29, wherein the identifier is generated by the wireless source device and transmitted to the wireless sink device.</u><u style="single">[C36] The method described in C29, wherein the external device is another wireless sink device.</u><u style="single">[C37] The method of C29, wherein the external device is an input device communicatively coupled to the wireless sink device.</u><u style="single">[C38] The method according to C29, wherein the data packet header is an application layer packet header.</u><u style="single">[C39] The method of C29, wherein the data packet is for controlling audio or video data of the wireless source device.</u><u style="single">[C40] The method of C29, wherein the data packet is transmitted over TCP / IP.</u><u style="single">[C41] A wireless source device configured to receive user input data from a wireless sink device, said wireless source device.</u><u style="single">A transport unit configured to receive data packets with data packet headers and payload data from wireless sink devices, and</u><u style="single">Memory to store instructions and</u><u style="single">One or more processors configured to execute the instruction, and when the instruction is executed, the one or more processors</u><u style="single">Parsing the data packet header to determine that the payload data comprises a forwarded user input command.</u><u style="single">Parsing the payload data to identify the identity of the third party device,</u><u style="single">Processing the payload data based on the identification information of the third party device</u><u style="single">With one or more processors</u><u style="single">A wireless source device that features.</u><u style="single">[C42] When executing the instruction, the one or more processors</u><u style="single">Negotiating the functionality of the wireless source device with the third party device through a series of messages.</u><u style="single">The wireless source device described in C41 that lets you do.</u><u style="single">[C43] At the time of executing the instruction, the one or more processors</u><u style="single">Receiving the identifier of the third party device for the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The wireless source device described in C41 that lets you do.</u><u style="single">[C44] When executing the instruction, the one or more processors</u><u style="single">Receiving the identifier of the third party device from the wireless source device as part of establishing a communication session between the wireless source device and the wireless sink device.</u><u style="single">The wireless sink device described in C41 that allows you to do more.</u><u style="single">[C45] The data packet header comprises a field for identifying a user input category, and the value of the field is set to indicate that the payload data comprises the forwarded user input data, C41. The wireless source device described in.</u><u style="single">[C46] The wireless source device according to C41, wherein the identifier of the third party device is selected from a group consisting of the IP address of the third party device and the domain name of the third party device.</u><u style="single">[C47] The wireless source device according to C41, wherein the identifier is generated by the wireless source device and transmitted to the wireless sink device.</u><u style="single">[C48] The wireless source device described in C41, wherein the external device is another wireless sink device.</u><u style="single">[C49] The wireless source device according to C41, wherein the external device is an input device communicatively coupled to the wireless sink device.</u><u style="single">[C50] The wireless source device according to C41, wherein the data packet header is an application layer packet header.</u><u style="single">[C51] The wireless source device according to C41, wherein the data packet is for controlling audio or video data of the wireless source device.</u><u style="single">[C52] The wireless source device according to C41, wherein the data packet is transmitted over TCP / IP.</u><u style="single">[C53] When executed by one or more processors, it stores an instruction that causes the one or more processors to perform a method of receiving user input data from a wireless sink device in a wireless source device. A computer-readable storage medium, wherein the method</u><u style="single">Receiving a data packet with a data packet header and payload data from a wireless sink device</u><u style="single">Parsing the data packet header to determine that the payload data comprises a forwarded user input command.</u><u style="single">Parsing the payload data to identify the identity of the third party device,</u><u style="single">Processing the payload data based on the identification information of the third party device</u><u style="single">A computer-readable storage medium.</u><u style="single">[C54] A wireless source device configured to receive user input data from a wireless sink device, said wireless source device.</u><u style="single">A means for receiving a data packet with a data packet header and payload data from a wireless sink device,</u><u style="single">A means for parsing the data packet header to determine that the payload data comprises a forwarded user input command.</u><u style="single">A means for parsing the payload data to identify the identity of a third party device, and</u><u style="single">With means for processing the payload data based on the identification information of the third party device</u><u style="single">A wireless source device that features.</u>
40 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| WO2012096546A2 | Cites | World Intellectual Property Organization (WIPO) |
| JP2008301249A | Cites | Japan |
| US20100257238A1 | Cites | United States of America |
168 members in 20 offices
Priority claims49
| Document | Office | Kind | Date |
|---|---|---|---|
| 201161435194 | United States of America | P | |
| 201161435194 | United States of America | P | |
| 61435194 | United States of America | – | |
| 201161447592 | United States of America | P | |
| 201161447592 | United States of America | P | |
| 61447592 | United States of America | – | |
| 201161448312 | United States of America | P | |
| 201161448312 | United States of America | P | |
| 61448312 | United States of America | – | |
| 201161450101 | United States of America | P | |
| 201161450101 | United States of America | P | |
| 61450101 | United States of America | – | |
| 201161467535 | United States of America | P | |
| 201161467535 | United States of America | P | |
| 201161467543 | United States of America | P | |
| 201161467543 | United States of America | P | |
| 61467535 | United States of America | – | |
| 61467543 | United States of America | – | |
| 201161514863 | United States of America | P | |
| 201161514863 | United States of America | P | |
| 61514863 | United States of America | – | |
| 201161544470 | United States of America | P | |
| 201161544470 | United States of America | P | |
| 61544470 | United States of America | – | |
| 13344253 | United States of America | – | |
| 201213344253 | United States of America | A | |
| 201213344253 | United States of America | A | |
| 2012022072 | United States of America | W | |
| 2012022072 | United States of America | W | |
| 13344253 | – | – | – |
| 61435194 | – | – | – |
| 61447592 | – | – | – |
| 61448312 | – | – | – |
| 61450101 | – | – | – |
| 61467535 | – | – | – |
| 61467543 | – | – | – |
| 61514863 | – | – | – |
| 61544470 | – | – | – |
| US201161435194P | – | – | – |
| US201161447592P | – | – | – |
| US201161448312P | – | – | – |
| US201161450101P | – | – | – |
| US201161467535P | – | – | – |
| US201161467543P | – | – | – |
| US201161514863P | – | – | – |
| US201161544470P | – | – | – |
| US2012022072 | – | – | – |
| US201213344253 | – | – | – |
| WO2012US22072 | – | – | – |
Members168
| Document | Office | Kind | |
|---|---|---|---|
| CA2824287A1 | Canada | A1 | |
| CA2824559A1 | Canada | A1 | |
| CA2824563A1 | Canada | A1 | |
| CA2824567A1 | Canada | A1 | |
| WO2012100186A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100189A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100191A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100193A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100197A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100201A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100204A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100214A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2012100218A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2013002949A1 | United States of America | A1 | |
| US2013003621A1 | United States of America | A1 | |
| US2013003622A1 | United States of America | A1 | |
| US2013003623A1 | United States of America | A1 | |
| US2013003624A1 | United States of America | A1 | |
| US2013009873A1 | United States of America | A1 | |
| US2013009887A1 | United States of America | A1 | |
| US2013009996A1 | United States of America | A1 | |
| US2013013318A1 | United States of America | A1 | |
| AU2012207073A1 | Australia | A1 | |
| AU2012207127A1 | Australia | A1 | |
| AU2012207129A1 | Australia | A1 | |
| AU2012207133A1 | Australia | A1 | |
| PH12013501451A1 | Philippines | A1 | |
| PH12013501482A1 | Philippines | A1 | |
| PH12013501483A1 | Philippines | A1 | |
| SG191367A1 | Singapore | A1 | |
| SG191377A1 | Singapore | A1 | |
| SG191763A1 | Singapore | A1 | |
| SG191765A1 | Singapore | A1 | |
| KR20130115370A | Republic of Korea | A | |
| KR20130115371A | Republic of Korea | A | |
| KR20130118958A | Republic of Korea | A | |
| CN103384995A | China | A | |
| CN103392160A | China | A | |
| CN103392161A | China | A | |
| CN103392325A | China | A | |
| CN103392326A | China | A | |
| CN103392359A | China | A | |
| CN103403649A | China | A | |
| CN103404104A | China | A | |
| CN103404114A | China | A | |
| KR20130126968A | Republic of Korea | A | |
| KR20130126969A | Republic of Korea | A | |
| KR20130126970A | Republic of Korea | A | |
| KR20130126971A | Republic of Korea | A | |
| KR20130126972A | Republic of Korea | A | |
| KR20130126973A | Republic of Korea | A | |
| EP2666069A1 | European Patent Office (EPO) | A1 | |
| EP2666073A1 | European Patent Office (EPO) | A1 | |
| EP2666074A1 | European Patent Office (EPO) | A1 | |
| EP2666274A1 | European Patent Office (EPO) | A1 | |
| EP2666275A1 | European Patent Office (EPO) | A1 | |
| EP2666276A1 | European Patent Office (EPO) | A1 | |
| EP2666277A1 | European Patent Office (EPO) | A1 | |
| EP2666278A1 | European Patent Office (EPO) | A1 | |
| EP2666323A1 | European Patent Office (EPO) | A1 | |
| JP2014506082A | Japan | A | |
| US8677029B2 | United States of America | B2 | |
| JP2014508995A | Japan | A | |
| HK1188005A | Hong Kong, China | A | |
| HK1188005A1 | Hong Kong, China | A1 | |
| JP2014509475A | Japan | A | |
| JP2014509476A | Japan | A | |
| JP2014510434A | Japan | A | |
| JP2014510435A | Japan | A | |
| ZA201305997B | South Africa | B | |
| JP2014510961A | Japan | A | |
| JP2014511582A | Japan | A | |
| JP2014511583A | Japan | A | |
| ZA201306271B | South Africa | B | |
| UA107151C2 | Ukraine | C2 | |
| US8964783B2 | United States of America | B2 | |
| RU2013138718A | Russian Federation | A | |
| RU2013138723A | Russian Federation | A | |
| RU2013138748A | Russian Federation | A | |
| RU2013138750A | Russian Federation | A | |
| KR101503386B1 | Republic of Korea | B1 | |
| ZA201305995B | South Africa | B | |
| JP5694568B2 | Japan | B2 | |
| AU2012207073B2 | Australia | B2 | |
| JP5714726B2This record | Japan | B2 | |
| AU2012207127B2 | Australia | B2 | |
| US9065876B2 | United States of America | B2 | |
| KR101533753B1 | Republic of Korea | B1 | |
| UA109176C2 | Ukraine | C2 | |
| AU2012207129B2 | Australia | B2 | |
| AU2012207133B2 | Australia | B2 | |
| UA109928C2 | Ukraine | C2 | |
| RU2567378C2 | Russian Federation | C2 | |
| JP5815741B2 | Japan | B2 | |
| ZA201404660B | South Africa | B | |
| JP5826860B2 | Japan | B2 | |
| JP5826861B2 | Japan | B2 | |
| JP2015222953A | Japan | A | |
| KR101572977B1 | Republic of Korea | B1 | |
| RU2571595C2 | Russian Federation | C2 |
18 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written permission of extension of timeJAPANESE INTERMEDIATE CODE: A602A602 | A602 | |
| Written request for extension of timeJAPANESE INTERMEDIATE CODE: A601A601 | A601 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 |
Numbers
- Publication
- 5714726
- Publication, DOCDB
- 5714726
- Publication, EPODOC
- JP5714726B
- Application
- 2013550629
- Application, DOCDB
- 2013550629
- Application, EPODOC
- JP20130550629
Titles2
- Japanese
- ワイヤレスディスプレイのためのユーザ入力バックチャネル
- English
- User input back channel for wireless display
Classification
- CPC, 6
- H04L65/00
- H04L69/24
- H04W99/00
- H04L2012/5603
- H04W4/00
- H04W88/00
- IPC, 3
- H04L47 43
- H04Q9 00
- H04L29 06
