Wireless display using multi-screen service
Abstract
Problem to be solved.To transmit the contents of a first wireless computing device to a second wireless computing device. A first wireless computing device initiates a WI-FI display (WFD) connection, sends data to a second wireless computing device over the WFD connection, and a playlist. Sharing media items with wireless client computing devices, sending information describing media items in playlists to wireless client computing devices, and sending wireless client computing devices to a second wireless computing device. Includes determining if it is possible to output a media item and sending the media item to a wireless client computing device. [Selection diagram] Fig. 8

Term
Projected expiry 13 February 2038.
- Priority
- Filed
- Published
- Today
- Projected expiry
32 claims: 8 independent, 24 dependent
- 1第2のワイヤレスコンピューティングデバイスに第1のワイヤレスコンピューティングデバイスのコンテンツを送信する方法であって、前記方法は、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記第2のワイヤレスコンピューティングデバイスとのWI-FIディスプレイ(WFD)接続を開始することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記第1のワイヤレスコンピューティングデバイスから前記WFD接続を介して前記第2のワイヤレスコンピューティングデバイスにデータを送信することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記第1のワイヤレスコンピューティングデバイスがプレイリストのメディア項目をワイヤレスクライアントコンピューティングデバイスと共有することを可能にするメディア共有アプリケーションを実行することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストの前記メディア項目を記述する情報を送信することであって、前記メディア項目を記述する前記情報を送信することが、前記ワイヤレスクライアントコンピューティングデバイスに、前記ワイヤレスクライアントコンピューティングデバイスが前記メディア項目を出力することが可能であるかどうかを判断することを行わせる、送信することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記ワイヤレスクライアントコンピューティングデバイスに前記メディア項目を送信することとを備える、方法。
- 2前記第1のワイヤレスコンピューティングデバイスから前記第2のワイヤレスコンピューティングデバイスに前記データを送信することが、前記第2のワイヤレスコンピューティングデバイスに、前記第1のワイヤレスコンピューティングデバイスのディスプレイ出力デバイスをミラーリングすることを行わせる、請求項1に記載の方法。
- 3前記メディア項目のフォーマットを記述する前記情報が、拡張可能マークアップ言語(XML)、2進数、ハイパーテキストマークアップ言語(HTML)、およびカンマ区切り値(CSV)のうちの少なくとも1つを備える、請求項1に記載の方法。
- 4前記ワイヤレスクライアントコンピューティングデバイスからメディア再生コマンドを受信することをさらに備える、請求項1に記載の方法。
- 5前記メディア再生コマンドがリアルタイムストリーミングプロトコル(RTSP)指示を備える、請求項4に記載の方法。
- 6前記メディア項目を送信することが、前記ワイヤレスクライアントコンピューティングデバイスから前記メディア再生コマンドを受信したことに応答して行われる、請求項4に記載の方法。
- 7前記第1のワイヤレスコンピューティングデバイスを用いて、前記第2のワイヤレスコンピューティングデバイスからユーザ入力を受信することをさらに備える、請求項1に記載の方法。
- 8前記WFD接続を介した前記データ送信と、前記メディア共有アプリケーションの前記実行が同時に行われる、請求項1に記載の方法。
- 9前記メディア項目を記述する前記情報が、前記メディア項目のビットレート、レベル、解像度、ファイルタイプ、およびファイル名のうちの少なくとも1つを含む、請求項1に記載の方法。
- 10前記第1のワイヤレスコンピューティングデバイスを用いて、前記ワイヤレスクライアントコンピューティングデバイスから認証情報を受信することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記認証情報に基づいて前記ワイヤレスクライアントデバイスを認証することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記ワイヤレスクライアントデバイスを認証したことに応答して前記ワイヤレスクライアントデバイスに前記プレイリストへのアクセスを許可することとをさらに備える、請求項1に記載の方法。
- 11前記メディア項目を送信することが、リアルタイムトランスポートプロトコル(RTP)を使用して前記メディア項目をストリーミングすることを備える、請求項1に記載の方法。
- 12前記WFD接続が第1のWFD接続を備え、前記方法が、 前記第1のワイヤレスコンピューティングデバイスを用いて、第2のWFD接続を介して前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストと前記プレイリストの前記メディア項目とのうちの少なくとも1つに関係するWFDデータを送信することと、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記第2のWFD接続を介して前記ワイヤレスクライアントコンピューティングデバイスからユーザ入力バックチャネル(UIBC)入力コマンドを受信することとをさらに備える、請求項1に記載の方法。
- 13前記第1のワイヤレスコンピューティングデバイスを用いて、前記ワイヤレスクライアントコンピューティングデバイスから前記UIBC入力コマンドを受信したことに応答して前記第2のWFD接続を終了することをさらに備える、請求項12に記載の方法。
- 14前記第1のワイヤレスコンピューティングデバイスが前記プレイリストの前記メディア項目を前記ワイヤレスクライアントコンピューティングデバイスと共有することを可能にする前記メディア共有アプリケーションを実行することが、 前記第1のワイヤレスコンピューティングデバイスを用いて、前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストを送信することをさらに備える、請求項1に記載の方法。
- 15前記第2のワイヤレスコンピューティングデバイスと前記ワイヤレスクライアントコンピューティングデバイスが同じデバイスである、請求項1に記載の方法。
- 16第1のワイヤレスコンピューティングデバイスであって、 第2のワイヤレスコンピューティングデバイスとのWI-FIディスプレイ(WFD)接続を開始することと、 前記第1のワイヤレスコンピューティングデバイスから前記WFD接続を介して前記第2のワイヤレスコンピューティングデバイスにデータを送信することとを行うように構成されたWI-FIディスプレイ(WFD)モジュールと、 前記第1のワイヤレスコンピューティングデバイスがプレイリストのメディア項目をワイヤレスクライアントコンピューティングデバイスと共有することを可能にするメディア共有アプリケーションを実行することと、 前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストの前記メディア項目を記述する情報を送信することであって、前記メディア項目を記述する前記情報の前記送信が、前記ワイヤレスクライアントコンピューティングデバイスに、前記ワイヤレスクライアントコンピューティングデバイスが前記メディア項目を出力することが可能であるかどうかを判断することを行わせる、送信することと、 前記ワイヤレスクライアントコンピューティングデバイスに前記メディア項目を送信することとを行うように構成されたメディア共有モジュールとを備える第1のワイヤレスコンピューティングデバイス。
- 17前記第1のワイヤレスコンピューティングデバイスから前記第2のワイヤレスコンピューティングデバイスへの前記データの前記送信が、前記第2のワイヤレスコンピューティングデバイスに、前記第1のワイヤレスコンピューティングデバイスのディスプレイ出力デバイスをミラーリングすることを行わせる、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 18前記メディア項目のフォーマットを記述する前記情報が、拡張可能マークアップ言語(XML)、2進数、ハイパーテキストマークアップ言語(HTML)、およびカンマ区切り値(CSV)のうちの少なくとも1つを備える、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 19前記メディア共有モジュールが、前記ワイヤレスクライアントコンピューティングデバイスからメディア再生コマンドを受信するようにさらに構成された、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 20前記メディア再生コマンドがリアルタイムストリーミングプロトコル(RTSP)指示を備える、請求項19に記載の第1のワイヤレスコンピューティングデバイス。
- 21前記メディア共有モジュールが、前記ワイヤレスクライアントコンピューティングデバイスから前記メディア再生コマンドを受信したことに応答して前記メディア項目を送信するように構成された、請求項19に記載の第1のワイヤレスコンピューティングデバイス。
- 22前記WFDモジュールが、前記第2のワイヤレスコンピューティングデバイスからユーザ入力を受信するようにさらに構成された、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 23前記WFD接続を介して送信される前記データと、前記メディア共有アプリケーションの前記実行が同時に行われる、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 24前記メディア項目の前記フォーマットを記述する前記情報が、前記メディア項目のビットレート、レベル、解像度、ファイルタイプ、およびファイル名のうちの少なくとも1つを含む、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 25前記メディア共有モジュールが、 前記ワイヤレスクライアントコンピューティングデバイスから認証情報を受信することと、 前記認証情報に基づいて前記ワイヤレスクライアントコンピューティングデバイスを認証することと、 前記ワイヤレスクライアントコンピューティングデバイスの前記認証に応答して前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストへのアクセスを許可することとを行うようにさらに構成された、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 26前記メディア項目を送信するために、前記WFDモジュールが、リアルタイムトランスポートプロトコル(RTP)を使用して前記メディア項目をストリーミングするように構成された、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 27前記WFD接続が第1のWFD接続を備え、前記WFDモジュールが、 第2のWFD接続を介して前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストと前記プレイリストの前記メディア項目とのうちの少なくとも1つに関係するWFDデータを送信することと、 前記第2のWFD接続を介して前記ワイヤレスクライアントコンピューティングデバイスからユーザ入力バックチャネル(UIBC)入力コマンドを受信することとを行うようにさらに構成された、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 28前記WFDモジュールが、 前記ワイヤレスクライアントコンピューティングデバイスから前記UIBC入力コマンドを受信したことに応答して前記第2のWFD接続を終了するようにさらに構成された、請求項27に記載の第1のワイヤレスコンピューティングデバイス。
- 29前記第1のワイヤレスコンピューティングデバイスが前記プレイリストの前記メディア項目を前記ワイヤレスクライアントコンピューティングデバイスと共有することを可能にする前記メディア共有アプリケーションを実行するために、前記メディア共有モジュールが、 前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストを送信するようにさらに構成された、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 30前記第2のワイヤレスコンピューティングデバイスと前記ワイヤレスクライアントコンピューティングデバイスが同じデバイスである、請求項16に記載の第1のワイヤレスコンピューティングデバイス。
- 31第1のワイヤレスコンピューティングデバイスであって、 第2のワイヤレスコンピューティングデバイスとのWI-FIディスプレイ(WFD)接続を開始するための手段と、 前記WFD接続を介して前記第2のワイヤレスコンピューティングデバイスにデータを送信するための手段と、 前記第1のワイヤレスコンピューティングデバイスがプレイリストのメディア項目をワイヤレスクライアントコンピューティングデバイスと共有することを可能にするメディア共有アプリケーションを実行するための手段と、 前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストの前記メディア項目を記述する情報を送信するための手段であって、前記メディア項目を記述する前記情報を送信するための前記手段が、前記ワイヤレスクライアントコンピューティングデバイスに、前記ワイヤレスクライアントコンピューティングデバイスが前記メディア項目を出力することが可能であるかどうかを判断することを行わせる、送信するための手段と、 前記ワイヤレスクライアントコンピューティングデバイスに前記メディア項目を送信するための手段とを備える第1のワイヤレスコンピューティングデバイス。
- 32実行されたとき、1つまたは複数のプロセッサに、 第2のワイヤレスコンピューティングデバイスとのWI-FIディスプレイ(WFD)接続を開始することと、 前記第1のワイヤレスコンピューティングデバイスから前記WFD接続を介して前記第2のワイヤレスコンピューティングデバイスにデータを送信することと、 前記第1のワイヤレスコンピューティングデバイスがプレイリストのメディア項目をワイヤレスクライアントコンピューティングデバイスと共有することを可能にするメディア共有アプリケーションを実行することと、 前記ワイヤレスクライアントコンピューティングデバイスに前記プレイリストの前記メディア項目のフォーマットを記述する情報を送信することであって、前記メディア項目を記述する前記情報の前記送信が、前記ワイヤレスクライアントコンピューティングデバイスに、前記ワイヤレスクライアントコンピューティングデバイスが前記メディア項目を出力することが可能であるかどうかを判断することを行わせる、送信することと、 前記ワイヤレスクライアントコンピューティングデバイスに前記メディア項目を送信することとを行わせる、命令を記憶したコンピュータ可読記憶媒体。
Independent claims32
148 paragraphs, as filed
0001This application is incorporated by reference in its entirety, US Provisional Application No. 61 / 583,987 filed January 6, 2012, and US Provisional Application No. 61 / 599,564 filed February 16, 2012. Claim the interests of the issue.
0002The present disclosure relates to techniques for transmitting data between a wireless source device and other wireless devices, and more particularly to the transmission of media data from a wireless source device to a wireless sink device and a wireless client device.
0003A wireless display (WD) or WI-FI (registered trademark) display (WFD) system includes a wireless source device and one or more wireless sink devices. Each of the source device and the sink device can be either a mobile device or a wired device having wireless communication capability. One or more of the source and sink devices are, for example, mobile phones, portable computers with wireless communication cards, personal digital assistants (PDAs), portable media players, or so-called "smart" phones and "smart" pads. Or it may include other such devices with wireless communication capabilities, including tablets, or other types of wireless communication devices. One or more of the source and sink devices may also include wired devices such as televisions, desktop computers, monitors and projectors that include communication capabilities.
0004The source device sends media data, such as audio video (AV) data, to one or more of the sink devices participating in a particular media sharing session. Media data can be played on both the local display of the source device and each of the displays of the sink device. More specifically, each of the participating sync devices renders the received media data on its screen and audio equipment.
0005Server computing devices can also use various media sharing protocols to give media items to client devices. The client device may issue a replay command to the server computing device. In response to receiving a play command, the server may send media items to the client device, for example using streaming.
0006In the present disclosure, generally, a first wireless computing device configured as a wireless source device can communicate with a second wireless computing device configured as a wireless sink device, and a wireless client computing device. theory about the system to Akira. As part of a communication session, the wireless source device can send audio and video data to the wireless sync device, which can send back user input received on the wireless sync device to the wireless source device. it can. The wireless source device may also run a media sharing application that allows a playlist of media items to be shared between the first wireless computing device and the wireless client computing device. The wireless client computing device can determine which media items the wireless client computing device can output. The first wireless computing device may receive media playback commands from the wireless client computing device and may send media items to the wireless client computing device in response to the playback commands.
0007As an example, in the present disclosure, a first wireless computing device is used to initiate a WI-FI display (WFD) connection, and a first wireless computing device is used to initiate a first wireless computing device. Sending data from to a second wireless computing device over a WFD connection, and using the first wireless computing device, the first wireless computing device wirelessly client-computes the media items in the playlist. To run a media sharing application that allows sharing with the device, and to use the first wireless computing device to send information describing media items in the playlist to the wireless client computing device. Sending information that describes a media item causes the wireless client computing device to determine if the wireless client computing device is capable of outputting the media item. And how to send the content of the first wireless computing device to the second wireless computing device, including sending media items to the wireless client computing device using the first wireless computing device. Will be described.
0008In another example, in the present disclosure, initiating a WI-FI display (WFD) connection with a second wireless computing device and a second wireless computing from the first wireless computing device over the WFD connection. A WI-FI Display (WFD) module configured to send data to the ing device and the first wireless computing device to share playlist media items with the wireless client computing device. Allowing the execution of media sharing applications and the transmission of information describing media items in playlists to wireless client computing devices, which is the transmission of information describing media items to wireless client computing devices. To determine if a wireless client computing device is capable of outputting media items, to send, and to send media items to a wireless client computing device. A first wireless computing device, including a configured media sharing module, will be described.
0009In another example, in the present disclosure, a means for initiating a WI-FI display (WFD) connection with a second wireless computing device and sending data to the second wireless computing device over the WFD connection. And a means to run a media sharing application that allows the first wireless computing device to share the media items of the playlist with the wireless client computing device, and to the wireless client computing device. A means for transmitting information that describes a media item in a playlist, a means for transmitting information that describes a media item, to a wireless client computing device, and a wireless client computing device to output a media item. Describes a first wireless computing device, including means for transmitting and means for transmitting media items to a wireless client computing device, which makes it possible to determine if it is possible. To do.
0010In another example, the present disclosure describes a computer-readable storage medium. The computer-readable storage medium initiates a WI-FI display (WFD) connection to one or more processors at runtime with a second wireless computing device and WFD from the first wireless computing device. A media sharing application that allows the first wireless computing device to share playlist media items with wireless client computing devices by sending data to a second wireless computing device over a connection. To perform and to send information that describes the format of the playlist's media items to the wireless client computing device, and to send information that describes the media items to the wireless client computing device. Memorize instructions to let the ing device determine if it is possible to output media items, to send and to let the wireless client computing device send media items. There is.
0011<figref num="1A">A block diagram showing an example of a system including a source / server device and a sink device system in which the technique of the present disclosure can be implemented.</figref><figref num="1B">A block diagram showing an example of a system including a source device and a client device.</figref><figref num="1C">A block diagram showing an example of a system including a source device, a sink device, and a client device.</figref><figref num="2A">A conceptual diagram showing a playlist of media items.</figref><figref num="2B">A conceptual diagram showing a playlist containing media items.</figref><figref num="2C">A conceptual diagram showing examples of attributes and values associated with media items.</figref><figref num="2D">A conceptual diagram showing examples of attributes and values associated with media items.</figref><figref num="3">A conceptual diagram showing an example of a communication reference model.</figref><figref num="4">A block diagram showing an example of a source device that can implement the technique of sending video and / or application data to a sink device.</figref><figref num="5">A block diagram showing an example of a sink device that implements a technique for receiving video and / or other information from a source device.</figref><figref num="6">The block diagram which shows the transmitter system and the receiver system which can implement the technique of this disclosure.</figref><figref num="7A">Diagram showing an exemplary message forwarding sequence for performing WI-FI Display (WFD) feature negotiation.</figref><figref num="7B">Diagram showing an exemplary message forwarding sequence for performing WI-FI Display (WFD) feature negotiation.</figref><figref num="8">A flowchart showing how to execute WFD and transmit media items by the technique of the present disclosure.</figref>
0012WI-FI displays (WFDs) can be used in various applications to support the wireless transmission of content. As an example, a mobile device (called a "source") is WFD-enabled ("sink" (s) from a mobile computing device, such as a phone, tablet, smartphone, or personal digital assistant (PDA). And can be used to wirelessly transmit video content or other application data to other devices (called "clients"). Video content or other application data can be transmitted from the source, received by the sink, and output by one or more output devices of the sink.
0013In the present disclosure, the term source device generally refers to a device that is transmitting media data to either a sink device or a client device. As described in more detail below, the term sink device generally refers to a device that is receiving media data from the source device and at the same time rendering the same media content as the source device. The term client device generally refers to a device that is receiving media data from a source device, but unlike sink devices, client devices do not always render the same media content as the source device at the same time. .. For example, a source device may stream video or audio data to a client device, but the source device is not itself rendering video or audio data. The terms source device, sink device, and client device generally refer to the operating state for a particular device. Thus, one device can be either a source device, a sink device, or a client device, and in some cases it can even function as more than one type of device at the same time. For example, a particular device can be a client device for one device, but a source device for another.
0014In one example, a user of a mobile source device may run a media sharing application on the source device when the user enters a neighborhood to support WI-FI communication. A media sharing application allows one or more users of a WI-FI-equipped client device to view, listen, and / or view media items shared on the source device via WI-FI streaming. It may be possible to select content. The source device's media sharing application may also use WFD to connect with a WFD-compatible sink device to share contacts, or other application data on the source device, with a WFD-compatible sync device.
0015A media sharing application may present one or more playlists of media items, such as audio, video, and pictures for streaming, to the user of the client device, which allows the client device to communicate with the media sharing application. Can be executed. The user of the client device may select the media item to play from the playlist. In some examples, the client device and the media sharing application may negotiate with each other and show only the media items in the playlist that the device can output. The user of the client device may select one or more media items for playback.
0016Source computing devices may share playlists using one or more protocols, such as the Universal Plug and Play (UPnP) protocol. The client device may use a protocol, such as Real Time Streaming Protocol (RTSP), to request a stream of one or more selected media items in a playlist. In response to receiving a request for one or more media items, the source device may use a protocol, such as Real-Time Transport Protocol (RTP), to stream the requested item to the requesting client device. ..
0017When the source device is in close enough proximity to support wireless communication over WI-FI, the user of the source device may launch a media sharing application. The application can initiate a WFD session that can configure a mobile device as a WFD source. The source device may connect to the WFD-compatible sink device by wirelessly communicating with the WFD-compatible device (called a "sink" or "sink device"). The WFD sink device may use some authentication mechanism, such as a pre-shared key, or a certificate system, to ensure that the user of the sink device is allowed to connect to the sink device.
0018Source devices, client devices, and sink devices can include devices such as DVD players, TVs, MP3 players, laptops, tablets, netbooks, and / or other devices that are WI-FI capable. In some examples, client devices and / or sink devices can be incorporated into the vehicle. In another example, the client device and / or sink device can belong to the user and can be portable.
0019In one exemplary user environment, a smartphone can act as a wireless source device and send media data to passengers in the car. The vehicle may include, for example, a wireless sink device in a dashboard or control panel that allows the driver to safely browse map applications or other such content while driving. The vehicle may further include one or more client devices. For example, a client device on the backseat may allow passengers on the backseat to view a video stored on a smartphone or listen to music stored on a smartphone. For illustration and examples, some aspects of the disclosure may be described with respect to the in-vehicle user environment, but the techniques of the present disclosure are not limited to any particular user environment.
0020A WFD connection between the source device and the sink device can allow sharing of application data. In various examples, application data may include contacts, calendar appointments, music stored on the source device, navigation data, or other application data that users of the source device may wish to access. In addition to feeding application data to automotive WFD-enabled devices, the source device may also perform screen mirroring with the sink device according to the WFD draft specification currently under development. When performing mirroring, the display of the source device can be sent to the sink device in real time, so that the sink device and the source device are in sync.
0021WFD mirroring refers to the device transmitting the image data at the source to a sink device that displays the transmitted image data in real time. In addition to sending image data between the source and the sink, WFD also uses a user input back channel (UIBC), as described below, to allow the source device to be the sink device. Allows the sink device to send input commands from the sink to the source. In one example, the sync device may also receive playlists of media items via WFD. The sink device may receive a user input command to select a media item for playback and send it to the source device. In response to receiving a media command, the source device may send the requested media item to the sink device.
0022In the context of the present disclosure, the WFD sink device may include one or more processors, memory, one or more storage devices, input and / or output devices, wireless modules capable of WI-FI communication. As mentioned above, when the source device connects to the sink device, the sink device may display the interface of the source device. In an example where the sink device comprises a car device and the source device comprises a car driver's device, the sink device may include a larger screen than the source device. This can be beneficial from a safety standpoint for the driver. By using the vehicle's built-in output device as the wireless sync device, the driver can avoid having to take his eyes off the road to see the display of the source device.
0023In various examples, the user of the sink device may issue user input commands to the source device. In some examples, input commands may include mouse clicks, scroll actions, keyboard input, or other types of user input. The sink input device may include a touch screen and / or voice command system, for example via BLUETOOTH®, in some examples.
0024In response to receiving user input, the sink device may return user input to the source device via the UIBC of the data connection between the source and sink. In response to receiving user input from the sink device, the source device takes actions such as accepting and scrolling user input, accepting mouse and / or keyboard input, or acting in response to voice commands. It can be taken.
0025In a car user environment, the driver may utilize the sink device in various ways to interact with the content of the source device. In various examples, the user can interact with the sink device, which interacts with the source device to make calls, access contact information, change music selections, access calendar and / or scheduling data. You may be forced to access the Internet, access navigation data and / or services, or perform other actions. Moreover, in some examples, the various inputs and outputs of the source device can be redirected to the various devices connected to the sink device. For example, if the driver is making a call, the driver may speak into one or more microphones that may be connected to the sink device to facilitate the ease of making the call. In addition, the audio of the call can be redirected from the source device through the car speaker connected to the sink device to give the driver better audio call quality and audibility.
0026As mentioned above, users of the client device can also access the content of the source device through the media sharing application. In one example, a user of a source device (eg, a driver) may set up a playlist for a user of the client device before a user of the client device (eg, a passenger on the backsheet) can access content on the source device. .. The playlist may contain various media from which the user of the client device may select to view and / or listen. The playlist may include, for example, various compressed video files, audio files, images, or other content that may be displayed by the output device of the client device. Multiple output devices, such as multiple WI-FI-equipped client devices, may be able to connect to the source device's media sharing application at the same time, and the source device sends multiple simultaneous media streams to each of the client devices. obtain. In this way, each user of the client device may be able to access different media items at the same time according to each user's preference.
0027Using one or more I / O devices on the client device, such as a touch screen display, mouse, keyboard, users of the wireless client device can use WI-FI to media sharing applications running on the source device. Can connect. In some examples, the WI-FI connection can be established using WI-FI Direct. In another example, a car can provide a wireless network, source and client devices can connect to that wireless network, and data between the source and client devices can be transferred over that wireless network. In some examples, the client device may include a WI-FI module, processor, memory, storage, and one or more additional input devices, allowing the user to select media from the source device's playlist. Can be.
0028When the user of the client device selects one or more media items from the playlist, the source device may start streaming the selected media item to the client device over the WI-FI connection. After a media item completes playback, the next media item from the playlist may be streamed to the output device until the playlist is complete. In some examples, the user of the client device may utilize user input using the client device to select different media items from the media server of the source device. The user may also execute additional playback commands such as "start", "stop", and "fast forward" to control the playback of the media on the client device.
0029FIG. 1A is a block diagram showing an exemplary system 100 that may implement the techniques of the present disclosure. As shown in FIG. 1A, the system 100 includes a source device 120 that communicates with the sink device 160 over the communication channel 150. The source device 120 may include devices such as netbooks, tablets, smartphones, PDAs, or any similar mobile device capable of supporting WFD. The source device 120 includes a memory for storing audio / video (A / V) data 121, a display 122, a speaker 123, an audio / video encoder 124 (also called an encoder 124), an audio / video control module 125, and the like. It may include a transmitter / receiver (TX / RX) unit 126. The sink device 160 includes a display 162, a speaker 163, an audio / video decoder 164 (also called a decoder 164), a transmitter / receiver unit 166, a user input (UI) device 167, and a user input processing module (UIPM). : user input processing can include module) 168 and. The illustrated components form only one exemplary configuration for System 100. Other components may contain fewer components than the components shown, or may include additional components other than those shown.
0030In the example of FIG. 1A, the source device 120 can display the video portion of the audio / video data 121 on the display 122 and output the audio portion of the audio / video data 121 on the speaker 123. The audio / video data 121 is stored locally on the source device 120 and can be accessed from an external storage medium such as a file server, Blu-ray® disc, or DVD. In some cases, audio / video data 121 can be captured in real time via the camera and microphone of the source device 120, so that the audio or video of the phone is captured or played back through the audio system of the car. Can be done. The audio / video data 121 may include multimedia content such as video, television programs, or music, but may also include real-time content generated by the source device 120. Such real-time content can be generated, for example, by an application running on the source device 120. As described in more detail, such real-time content may, in some cases, include video frames of user input options available for the user to select. In some cases, the audio / video data 121 may include a video frame that is a combination of different types of content, such as a video frame of a video or TV show with a user input option overlaid on the frame of the video.
0031In addition to locally rendering the audio / video data 121 through the display 122 and the speaker 123, the audio / video encoder 124 of the source device 120 can encode the audio / video data 121 and the transmitter / The receiver unit 126 can transmit the encoded data to the sink device 160 via the communication channel 150. The sink device 160 may include a device including a touch screen display that can be mounted at a convenient location in the vehicle for driver dialogue. The transmitter / receiver unit 166 of the driver sync device 160 receives the encoded data, the audio / video decoder 164 decodes the encoded data, and the decoded data is combined with the display 162 and the speaker 163. Output via. In this way, the audio and video data rendered by the display 122 and the speaker 123 can be rendered simultaneously by the display 162 and the speaker 163. Audio data and video data can consist of frames, which can be time synchronized with the video frame when rendered.
0032The audio / video encoder 124 and audio / video decoder 164 are alternatives to the new ITU-T H.264 standard called MPEG-4, Part10, Advanced Video Coding (AVC), or the H.265 standard. Any number of audio and video compression standards can be implemented, including high efficiency video coding (HEVC) standards. Generally, the audio / video decoder 164 is configured to perform the reverse coding operation of the audio / video encoder 124. Although not shown in FIG. 1A, in some embodiments, the A / V encoder 124 and the A / V decoder 164 may be integrated with the audio encoder and decoder, respectively, in a common or separate data stream. It may include a suitable MUX-DEMUX unit, or other hardware and software, to handle both audio and video encoding.
0033As described in more detail below, the A / V encoder 124 may perform other coding functions in addition to implementing the video compression standard as described above. For example, the A / V encoder 124 may add various types of metadata to the A / V data 121 before the A / V data 121 is transmitted to the sink device 160. In some cases, the A / V data 121 may be stored in or received in the source device 120 in encoded form and therefore does not require further compression by the A / V encoder 124. is there.
0034Figure 1A shows that the communication channel 150 carries the audio payload data and the video payload data separately, but in some cases the video payload data and the audio payload data are part of a common data stream. Please understand that it can be. Where applicable, the MUX-DEMUX unit is an ITU H.223 multiplexer protocol, or user datagram protocol (UDP). Can comply with other protocols such as protocol). The audio / video encoder 124 and audio / video decoder 164 each have one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, and software. , Hardware, firmware, or any combination thereof. Each of the audio / video encoder 124 and the audio / video decoder 164 may be included in one or more encoders or decoders, both of which may be integrated as part of a composite encoder / decoder (codec).
0035The displays 122 and 162 may comprise any of a variety of video output devices, such as cathode ray tubes (CRTs), liquid crystal displays (LCDs), plasma displays, organic light emitting diode (OLED) displays, or other types of display devices. .. Speaker 123 may include any of a variety of audio output devices, such as headphones, single-speaker systems, multi-speaker systems, or surround sound systems. Further, although the display 122 and the speaker 123 are shown as part of the source device 120 and the display 162 and the speaker 163 are shown as part of the sink device 160, the source device 120 and the sink device 160 are actually plural. It can be a system of devices. As an example, the display 162 can be a television, the speaker 163 can be a surround sound system, and the decoder 164 can be part of an external box that is wired or wirelessly connected to the display 162 and the speaker 163. In another example, the sink device 160 can be a single device, such as a tablet computer or smartphone. In yet other cases, the driver device 160 and the sink device 120 are similar devices, for example, both are smartphones, tablet computers, and so on. In this case, one device can act as a source and the other as a sink. These roles may even be reversed in subsequent communication sessions.
0036The transmitter / receiver unit 126 and the transmitter / receiver unit 166, respectively, include various mixers, filters, amplifiers, and other components designed for signal modulation, as well as one or more antennas. And may include other components designed to transmit and receive data. The communication channel 150 generally represents any communication medium suitable for transmitting video data from the source device 120 to the sink device 160, or a collection of various communication media. The communication channel 150 is usually a relatively short-range communication channel similar to WI-FI, BLUETOOTH, and the like. However, the communication channel 150 is not necessarily limited in this regard, and may be any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines, or a wireless medium and a wired medium. Can have any combination with. In another example, the communication channel 150 may even form part of a packet-based network, such as a wired or wireless local area network, a wide area network, or a global network such as the Internet. In addition, the communication channel 150 can be used by the source device 120 and the sink device 160 to create peer-to-peer links. The source device 120 and sink device 160 may communicate over the communication channel 150 using communication protocols such as standards from the IEEE 802.11 standard family. Although the techniques of the present disclosure are sometimes described with respect to WI-FI, it is contemplated that aspects of these techniques may also be compatible with other communication protocols.
0037In addition to decoding and rendering the data received from the source device 120, the sink device 160 can also receive user input from the user input device 167. The user input device 167 can be, for example, a keyboard, mouse, trackball or trackpad, touch screen, voice command recognition module, or other such user input device. The UIPM168 formats the user input commands received by the user input device 167 into a data packet structure that can be interpreted by the source device 120. Such data packets are transmitted by the transmitter / receiver 166 to the source device 120 over the communication channel 150. The transmitter / receiver unit 126 receives the data packet, and the A / V control module 125 parses the data packet to interpret the user input command received by the user input device 167.
0038In addition, a user of sync device 160, such as a passenger or driver, may be able to launch and control applications on source device 120. For example, a user of sync device 160 can launch a photo editing or navigation application stored on source device 120 and use that application to edit photos stored locally on source device 120. obtain. The sink device 160 can provide the user with a user experience in which the photo appears and feels locally edited on the sink device 160 while the photo is actually being edited on the source device 120. With such a configuration, device users may be able to take advantage of the capabilities of one device for use with several devices. For example, the source device 120 can be a smartphone with a large amount of memory and high-end processing capabilities, and users of the source device 120 can use the smartphone in all settings and situations in which the smartphone is commonly used. When watching a video, the user may wish to watch the video on a device with a larger display screen, in which case the sync device 160 may be a tablet computer. When wishing to send or respond to e-mail, the user may wish to use a device with a keyboard, in which case the sync device 160 may be a laptop. In both situations, the user is interacting with a tablet computer or laptop, but most of the processing can still be performed by the source device 120 (smartphone in this example). Since most of the processing is performed by the source device 120, the sink device 160 uses fewer resources than if the sink device 160 was asked to do the processing that is being done by the source device 120. , Can be a lower cost device.
0039In some configurations, the A / V control module 125 can be an operating system process running by the operating system of the source device 120. However, in other configurations, the A / V control module 125 can be the software process of the application running on the source device 120. In one example, the A / V control module 125 may include a media server capable of WIFI media streaming, and a WFD module. In such a configuration, user input commands can be interpreted by a software process so that the user of sink device 160 runs on source device 120 as opposed to the operating system running on source device 120. You are interacting directly with the application you are using. By interacting directly with the application as opposed to the operating system, users of sink device 160 may have access to a library of commands that are not native to the operating system of source device 120. In addition, by interacting directly with the application, commands may be more easily transmitted and processed by devices running on different platforms.
0040The source device 120 can respond to user input applied in the wireless sync device 160. In such an interactive application configuration, the user input applied in the wireless sync device 160 may be sent to the wireless display source via the communication channel 150. In one example, the user interface back (UIBC) is used to allow the sink device 160 to send the user input applied at the sink device 160 to the source device 120. A reverse channel architecture, also known as channel), can be implemented. The reverse channel architecture may include a higher layer message for transporting user input and a lower layer frame for negotiating user interface functionality on the sink device 160 and the source device 120. The UIBC may reside on top of the Internet Protocol (IP) transport layer between the sink device 160 and the source device 120. In this way, UIBC can be above the transport layer in the Open Systems Interconnection (OSI) communication model. In one example, OSI communication includes seven layers: 1-Physical, 2-Datalink, 3-Network, 4-Transport, 5-Session, 6-Presentation, and 7-Application. In this example, above the transport layer refers to layers 5, 6, and 7. To facilitate the reliable transmission and sequential delivery of data packets containing user-entered data, UIBC uses other packets, such as Transmission Control Protocol / Internet Protocol (TCP / IP) or User Datagram Protocol (UDP). It can be configured to run on top of the base communication protocol.
0041In some cases, there may be a discrepancy between the user input interface located on the source device 120 and the user input interface located on the sink device 160. In order to resolve the potential problems posed by such discrepancies and promote a good user experience under such circumstances, the source device before user input interface feature negotiation establishes a communication session. It can be done between 120 and the sink device 160.
0042UIBC can be designed to transport various types of user input data, including cross-platform user input data. For example, the source device 120 may run an iOS® operating system, and the sink device 160 may run another operating system, such as Android® or Windows®. Regardless of the platform, the UIPM168 can encapsulate the received user input in a form understandable to the A / V control module 125. UIBC may support several different types of user input formats so that many different types of source devices and automotive sink devices can take advantage of the protocol. General input formats can be defined and both platform-specific input formats can be supported, thus giving the flexibility of how user input can be communicated between the source device 120 and the sink device 160 by UIBC.
0043In one example, the sink device 160 may establish a WFD connection with the source device 120, and the source device 120 may send the sink device 160 information describing one or more media items in the playlist. Playlists and media items, for example FIGS. 2A-2D, will be described in more detail below. The sink device 160 can determine which media items the sink device 160 can output. The sync device 160 may output playlists and receive user input selections for one or more media items. The sink device 160 may transmit the above selection of media items to the source device 120, and the source device 120 may transmit (eg, stream) the selected media item to the sink device 120.
0044In the example of Figure 1A, the source device 120 may include a smartphone, tablet computer, laptop computer, desktop computer, WI-FI capable television, or other device capable of transmitting audio and video data. The sync device 160 can also be a smartphone, tablet computer, laptop computer, desktop computer, WI-FI capable television, or any other device capable of receiving audio and video data and receiving user input data. Can be prepared. In some cases, the sink device 160 is such that the display 162, speaker 163, UI device 167, and A / V encoder 164 are all separate but interoperable devices. It may include a system of devices. Similarly, the source device 120 can be a system of multiple devices rather than a single device.
0045In the present disclosure, the term source device is used generally to refer to a device transmitting audio / video data, and the term sink device is generally used to refer to a device receiving audio / video data from a source device. Used to point to. In many cases, the source device 120 and the sink device 160 can be similar or equivalent devices, one device acting as the source and the other acting as the sink. Moreover, these roles can be reversed in different communication sessions. Thus, a sink device in one communication session can be a source device in subsequent communication sessions, and vice versa.
0046FIG. 1B is a block diagram showing an example of a system including a source device and a client device that can implement the techniques of the present disclosure. In the system of FIG. 1B, the source device 120 may communicate with the client device 180 via the communication channel 152. The client device 180 and the sink device 160 can be the same device or different devices. The communication channel 152 may include a wireless communication channel similar to WI-FI or the like. The source device 120 may communicate with the client device 180 over the communication channel 152 using one or more protocols, such as UPnP set of protocols, UDP, RTSP and / or RTP. In some examples, streaming data from the source device 120 to the client device 180 using UPnP, UDP, RTSP, and / or RTP is content with reduced power consumption on the source device 120 and client device 180. It may be possible to send items and require less computational complexity compared to sending media items using different protocols such as WFD.
0047The source device 120 may share one or more available playlists with one or more devices, such as the client device 180. The source device 120 may further transmit information describing at least one media item in the playlist to the client device 180. The transmission of the above information may cause the client device 180 to determine whether the wireless client device 180 is capable of outputting at least one media item. A user of client device 180 may request driver source device 180 for one or more media items in a playlist. In response to receiving a request for one or more media items, the source device 120 may stream or transmit the requested media item to the client device 180, which is the display 182, and /. Alternatively, it may output the requested media item on the output device, such as speaker 183.
0048The source device 120 of FIG. 1B can be equivalent to the driver source 120 of FIG. 1A. The source device 120 may include a display 122, audio-video data 121, speakers 123, audio / video control 125, audio / video encoder 124, and transmit / receive unit 126. The client device 180 can be similar or equivalent to the sink device 160 of FIG. 1A. The client device 180 may include an audio / video decoder 184, a display 182, a speaker 183, a user input device 187, and a transmit / receive unit 186.
0049The audio / video control unit 125 may be configured to run the media sharing application 128 using one or more processors of the source device 120. Media application 128 may, in some cases, be part of the operating system or stand-alone application of source device 120. The media sharing application 128 may determine one or more playlists to be shared with a client computing device, such as the client computing device 180. Playlist media items may be stored in local storage, including hard drives, flash memory, and / or peripherals connected to the source device 120. In addition, playlist media items can be remotely accessed by the source device 120. Examples of such remotely accessible ones may be media items stored in the cloud, streaming video, or media items stored on a file server.
0050In response to a user of the driver source device launching the media sharing application 128, the media sharing application 128 may broadcast the playlist to one or more client devices, such as the client device 180. In some examples, the media sharing application 128 may broadcast playlists using one or more of the protocols in the UPnP set of protocols. As described for UPnP, the media sharing application 128 may use any mechanism compatible with wireless communication protocols for broadcasting playlists to client devices, such as client device 180. The source device 120 is a Simple Service Discovery Protocol (SSDP), which is a UPnP protocol that provides service discovery for devices on the network. Protocol) may be used to announce the services of the source device 120 (ie, the driver source device uses RTSP and RTP to provide streaming services). The use of SSDP as a discovery protocol is only an example and should be considered non-limiting. Universal Datagram Protocol (UDP), BONJOUR, Service Location Protocol (SLP), Web Services Dynamic (WS-Discovery) Other protocols and sets of protocols, such as Discovery) and Zero Configuration Networking (zeroconf), can also allow client devices to discover the streaming services provided by the source device 120. In the example where the source device 120 uses UDP to send playlists, the source device 120 may allow one or more client devices, such as the client device 180, to listen for playlists sent by the source device 120. As such, playlists can be sent using a specific port, broadcast address or well-known multicast address.
0051The client device 180 may be similar to the sink device 160 of FIG. 1B and may include a mobile computing device such as a tablet, PDA, laptop, netbook, DVD player, or another computing device. A user on client device 180 may launch client application 185. Client application 185 may receive one or more announcements from media sharing application 128 via communication channel 152. In response to receiving the above notification, the media sharing application 128 may parse the notification message and determine the service provided by the media sharing application 128. In some examples, the client device 180 may use, for example, RTSP and / or RTP to determine that the media sharing application 128 provides media streaming and playlist sharing capabilities.
0052The service announcement received by the client application 185 provides a link to one or more playlists of media items or a resource location that contains one or more playlists of media items such as URLs, network paths, etc. Can also be included. If a link to a playlist is included in the service announcement, client application 185 may retrieve the playlist shared by media sharing application 128 from that location.
0053Each of the playlists may contain a list of media items that may be streamed from the driver device 120 to the client device 180. Each playlist may also include a user identifier that can be used to restrict access to one or more users for a particular playlist. Each playlist may contain one or more properties for each of one or more media items. Properties can generally include information such as name, length, resolution, frame rate, profile level, bit rate, and / or file format for each media item, as some examples. The properties of playlists and media items will be described in more detail below with respect to FIGS. 2A-2D.
0054Upon receiving one or more playlists, client application 185 may use display 182 to output playlists to users on client device 180. The display 182 may include any of a variety of video output devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device. The client device 180 may receive user input from user input device 187 to select one of the playlists. The user input device 187 can be, for example, a keyboard, mouse, trackball or trackpad, touch screen, voice command recognition module, or other such user input device.
0055In response to receiving the playlist, the client application 185 determines which media items in the playlist can be output by the client device 180 and uses the display 182 to determine which media item in the playlist can be output by the client device 180. It may present the user with media items that can be output by the client device 180. To determine which of the media items the client device 180 is capable of playing, the client application 185 uses the client device 180's installed codecs, digital rights management (DRM) capabilities, and /. Alternatively, you can contact the operating system for hardware features and compare the features of the client device 180 with the attribute information contained in the playlist and the media items in the playlist.
0056Client application 185 may display media items from selected playlists that client device 180 can play to users of client device 180 using display 182. A user of client device 180 may select one or more media items from a playlist for playback using user input device 187. In response to receiving a selection for one or more media items, the client application 185 makes a play request for one of the selected media items in the transmit / receive unit 186. Can be done. In some examples, the playback request may request that the source device 120 play, pause, stop, record, etc. the selected media item. The transmit / receive unit 186 may transmit a request for the selected media item to the transmit / receive unit 126 via the communication channel 152.
0057If the user of client device 180 selects multiple media items from the playlist, client application 185 may issue a play request for the first media item of the selected media items and the selected media. When the request for the rest of the item can be enqueued and as a result the playback of the first media item is complete, the client application 185 requests and is requested to play one of the enqueued media items. Stream the media items that have been created and repeat the process of requesting and streaming the enqueued media items until all the enqueued media items have been requested and streamed.
0058In one example, the client device 180 may be connected to the source device 120 using a WFD connection. The client device 180 may receive display information (eg, a graphical representation) indicating one or more playlists further containing one or more media items when rendered by the client device 180. The client device 180 may use the display 182 to output display information to the user of the client device 180, and the user of the client device 180 uses UI187 to select one or more media items for playback. obtain. In one example, the client device 180 may receive a playlist with information describing one or more media items using a WFD connection. UI187 receives a user input command to select one or more media items and sends it to the source device 120 via UIBC.
0059In response to receiving a request or media playback command, such as an RTSP playback request or UIBC user input command, from the client device 180 for one or more media items, the media sharing application 128 sends the transmit / receive unit 126 to Allows you to configure a stream of requested media items. In some examples, the transmit / receive unit 126 may configure an RTP session for a stream of media items. The transmit / receive unit 126 may establish an RTP session with the client device 180 and transmit a stream for media items to the transmit / receive unit 186 over the communication channel 152. In the example described above, the client device 180 receives information about the playlist from the WFD connection and uses the UIBC to send the media item selection, the source device 120 and the client device 180 are the media items and / or play. The WFD connection may be terminated in response to receiving a UIBC input command to select a list. Upon termination of the UIBC connection, the source device 120 and client device 180 may continue to communicate using RTSP and / or RTP.
0060In some examples, the media sharing application 128 may need to transcode the selected media item to a different format before sending the media item to the client device 180. In such cases, Media Application 128 will use Audio / to re-encoder the selected media format from one format to another, for example from the MPEG Layer 3 Audio (MP3) format to the Windows Media Audio (WMA) format. Video encoder 124 can be used.
0061The transmit / receive unit 186 may receive an RTP stream of media items requested by the transmit / receive unit 126. If some of the packets in the requested media item stream are out of order, the transmit / receive unit 186 may reassemble and / or reorder the stream in the correct order. The transmit / receive unit 186 can also determine if there is something wrong with the received RTP stream, such as a missing packet, and can reclaim the missing packet to the source device 120.
0062The client application 185 can analyze the stream and use the display 182 and the speaker 183 to output the audio and / or video portion of the stream. In this way, the audio and video data rendered by the display 182 and the speaker 183 can be rendered simultaneously by the display 182 and the speaker 183. Audio data and video data can consist of frames, which can be time synchronized with the video frame when rendered. If client application 185 determines that a stream of media items needs to be decrypted, client application 185 uses an audio / video decoder 184 to decode the encoded stream before outputting the stream. Can be used.
0063FIG. 1C is a block diagram showing an exemplary system 101 that may implement the techniques of the present disclosure. System 101 includes a source device 120 and a sink device 160, each of which may function and operate in the manner described above for FIG. 1A. System 101 further includes client device 180. As described above, the client device 180 can be any device capable of wirelessly connecting to the source device 120 and streaming media from the media sharing application 128 of the source device 120. Sync device 160 may receive audio and video data from source device 120 over WIFI using streaming protocols such as RTSP and / or RTP. In some configurations, the sink device 160 and the client device 180 can operate independently of each other, and the audio and video data output on the source device 120 is output simultaneously on the sink device 160 and the client device 180. obtain. Although the sink device 160 and the client device 180 are shown as separate devices, they can be the same device. System 101 is shown as having only a single client device 180, but this is just an example and should not be limited. Additional devices similar to client device 180 may also be present in system 101.
0064FIG. 2A is a conceptual diagram showing a playlist of media items according to the technique of the present disclosure. FIG. 2A shows three exemplary playlists, playlists 200, 202, and 204. Each playlist may include a playlist of media items as described above in the examples of FIGS. 1A-1C. Playlists 200, 202, and 204 may also contain one or more related attributes that describe the properties of the associated playlist. Each of playlists 200, 202, and 204 may contain one or more media items, such as audio, video, and picture media items. A media item may also contain attributes or metadata that describe the properties of the media item.
0065Each of one or more attributes of playlists 200, 202, and 204 can generally have an identifier. The identifier can be associated with a list of one or more values. As an example, playlists 200, 202, 204 may include a "number" attribute. The number attribute can indicate the number of media items associated with each playlist that a client device can stream from source device 120 to client device 180. As another example, playlists 200, 202, 204 may also include a "user" attribute. User attributes can relate to a list of one or more users, groups of users, and / or devices that are allowed to stream media items related to a particular playlist.
0066In some examples, the value associated with the access attribute may comprise the username of a particular user who is allowed to stream media items in the playlist. As an example, in FIG. 2A, playlist 200 may include the usernames "Jesse" and "Bob", and those usernames should be accessible to the users "Jesse" and "Bob". Show that. In some other examples, the value associated with the access attribute may include the device's identifier. For example, a device identifier can include an IP address, a Machine Access Control (MAC) address, or other hardware that identifies a particular device. In some other examples, the access attribute may have a group identifier that corresponds to a group of one or more users.
0067The media sharing application 128 can authenticate the client device identified by the hardware identifier by comparing the hardware identifier (eg, MAC address) given to the media sharing application 128 by the client device. In the example of Figure 2A, the user attribute of playlist 202 is related to the MAC address ("BF: 54: 51: 7E: 30: B6"). The media sharing application 128 may compare the MAC address of a client device, eg, client device 180, with the MAC address (s) associated with playlist 202. If the given identifier matches one of the identifiers associated with the user attribute (eg, MAC address), the media sharing application 128 may grant access to the client device 180.
0068Media sharing application 128 can authenticate users of client devices, such as client device 180, using a variety of different authentication mechanisms. In some examples, the client application 128 may request credentials from the client device 180, such as a username and password. In response to receiving the username and password from the client device 180, the media sharing application 128 may compare the received username and password with the username associated with the user attribute. In one example, the user attributes of the playlist 200 include the associated users "Jesse" and "Bob". Media sharing application 128 may request a username and password from client application 185 and may receive a response containing the username "Bob" and the password for user Bob. The media sharing application 128 may determine that the username Bob is included in the user attributes. The media sharing application 128 can then compare the stored password with the supplied password to determine if the password provided by the client device 180 matches the locally stored password. If the provided password matches the stored password, the media sharing application 128 may authenticate the client device 180 and allow the client device 180 to access the media item in the playlist 200 (ie, that media item). Can enable streaming). The media sharing application 128 may store the user's password in the database or local storage of the source device 120. In some examples, the media sharing application 128 may utilize authentication techniques, such as a certificate system for authenticating a user or device. In another example In the example of FIG. 2A, the user attribute of playlist user 204 includes the identifier "adult", which may indicate that a group of one or more users is allowed to access the media items of playlist 204. In this example, the group "adults" may indicate that a group of users corresponding to adults and excluding children is allowed to stream media items in playlist 204. Excluding users, such as children, can prevent people from streaming content that is not suitable for the user's excluded group, or it can be tricky or access to playlists of privileged media items. It can be useful in limiting access to only the users who should have it.
0069The user attributes of playlists 200, 202, 204 are shown as a list of users or devices that are allowed access to the media items of playlists 200, 202, 204, but the user attributes of a particular playlist are alternate. May include a list of users and / or devices excluded from accessing the playlist's media items. In some examples, a playlist may include a list of users who are allowed access to the playlist's media items and a list of users who are denied access to a particular playlist.
0070FIG. 2B is a conceptual diagram showing a playlist containing media items according to the technique of the present disclosure. Figure 2B shows the media items associated with Playlist 200. Figure 2B includes three media items 220, 222, 224. Media items, such as playlist 200 media items, may include any type of media that can be transmitted wirelessly. In the example of FIG. 2B, media item 220 can be video media, such as H.264 video, Motion Picture Experts Group (MPEG) video, or other video formats. Media item 222 can be audio media, such as MP3, WMA, OGG Vorbis, FLAC (Free Lossless Audio Codec), or other compressed or uncompressed media format. Media item 224 is a raw JPEG (Joint Picture Experts Group), BMP (bitmap), or TIFF (Tagged Image File). It can be an image file, such as an image format (Format), or another image media format. Although not shown in Figure 2B, other media formats, such as document, web page, and drawing media formats, may also be included as media items in the playlist.
0071FIG. 2B may also represent an exemplary interface of a playlist 200 that may be presented to a user of client device 180, eg, client application 185. In response to receiving user input from one of the user input devices 187 (FIGS. 1A, 1B), the client device is that the source device 120 has media items 220, 222, and / or in playlist 200. It may be required to execute a playlist related to one or more of the 224. In one example, a replay command may include one or more RTSP commands, such as replay, stop, pause, record, and / or other RTSP commands or requests. With respect to RTSP, client device 185 may utilize other protocols for controlling media playback from source device 120.
0072In response to receiving one or more playback commands, the media sharing application 128 may take action according to the requested playback command. As an example, if client application 185 sends an RTSP playback command to media sharing application 128 that requires media sharing application 128 to play media item 200, then media sharing application 128 goes to requested media item 202. It can respond by sending the corresponding stream to client application 185. As another example, if the media sharing application 128 receives the RTSP stop command, the media sharing application 128 may stop streaming the currently playing media item, such as media item 222. In addition to using RTSP to control the playback of media items, the stream of media items sent from the source device 120 to the client device 180 is generally actually using RTSP to control the RTP stream. Different protocols such as RTP can be used to stream media items.
00732C and 2D are conceptual diagrams showing two examples of attributes and values associated with media items according to the techniques of the present disclosure. Figures 2C and 2D show some of the attributes and values associated with media items 220 and 222. Each of the media items 220, 222, 224 may have one or more related attributes. Each of the related attributes can have one or more related values. The format of the attributes of media items 220, 222, and 224 can be generally similar to the format of the attributes of playlists 200, 202, 204, in which case the attribute is an identifier associated with a list of one or more values. Has.
0074In general, media items can have file name, file type, resolution, bit rate, length, and / or profile attributes. The filename attribute can indicate the filename or title of the media item. The file type attribute can indicate whether the media item is a video media item, an audio media item, or a media item in another file format. In some examples, the file type attribute may also indicate more specific information, such as a particular type of media item such as audio or video (eg H.264 or MP3).
0075The resolution attribute can indicate the horizontal and vertical resolution of the media item. The source device 120 and the client device 180 may negotiate a set of one or more resolutions that the client device 180 can output based on the resolution attributes of one or more media items. As part of this negotiation process, the source device 120 and the client device 180 can agree on the negotiated screen resolution. When the client device 180 streams data related to a media item, such as streaming video, the source device 120 can scale or transcode the video data of the media item to match the negotiated screen resolution. In this way, the client device 180 may receive video data of the agreed resolution. By feeding the video to the client device 180 at the agreed resolution, the client device 180 may not have to transcode the video data of the media item, which may consume additional power and / or processing cycles. is there.
0076In Figure 2C, the resolution of media item 220 is 1920 x 1080 pixels. In one example, if the client device 180 has a resolution of 1280x720 and the source device 120 has a resolution of 1600x900, then those devices may use, for example, 1280x720 as their negotiated resolution. The negotiated resolution may be selected based on the resolution of the client device 180, but the resolution of the source device 120 or some other resolution may also be used. In the example where a 1280x720 sink device is used, the client device 180 can scale the obtained x-coordinates at a ratio of 1600/1280 before sending them to the source device 120, as well. In addition, the client device 180 can scale the obtained y-coordinates by 900/720 before sending them to the source device 120. In other configurations, the source device 120 can scale the acquired coordinates to the negotiated resolution. The scaling can increase or decrease the coordinate range based on whether the client device 180 uses a higher resolution display than the source device 120 or vice versa. In FIG. 2D, the resolution attribute of media item 222 has no value because media item 222 is an MP3 audio file with no resolution (not applicable). The media item may also include a bit rate attribute that may indicate the streaming bit rate of the media item. In Figure 2C, media item 220 has a bit rate of megabits / second (12 Mbit / s). Media item 222 has a bit rate of 320 kilobits / second (320 kbit / s). If a media item, such as a still image or document, does not have a bitrate, there may be no value associated with the bitrate attribute.
0077Media items 220 and 222 may also have a length attribute indicating the playback time of the media item. Media item 220 has a length of 1 hour and 40 minutes (1 hour 40 minutes). Media item 222 has a length of 2 minutes and 22 seconds (2 minutes 22 seconds). In addition to the length attribute, the media item may have associated profile attributes that may indicate the function or coding characteristics of the media item. As an example, the value of the profile attribute value of media item 220 is "main", which may correspond to a particular profile of MPEG4 video. Media item 222 has no value associated with the profile attribute because the MP3 media does not have a profile. Although the H.264 video profile has been described in Figure 2C, other profile values are possible.
0078Client application 185 may use the values of attributes associated with media items 220, 222, 224 of playlist 200 to determine a subset of media items that client device 180 can play. When the client application 185 determines the media items that the client device 180 can play, the client application 185 determines those media items in a subset of the media items that the client device 185 can play. Can only be output to the user on client device 185. The user may select from the subset only the media items that the client device 180 can play for playback.
0079To determine which media items the client device 180 is capable of playing, the client application 185 may query the client device 180 to determine the hardware capabilities of the client device 180. Client application 185 is concerned with the amount of RAM, storage space, output device resolution, sound output capabilities, processor speed, installed libraries, codecs (codec decoders), or playback of media items on client device 180. You can contact the operating system for other information. When the client application 185 requests access to a playlist, such as the playlist 200, the client application 185 determines which media item the client device 180 is capable of playing the playlist 200. And the attributes of playlist 200 media items 220, 222, 224 can be compared to the capabilities of client device 185.
0080As an example, the client device 180 may have an output device capable of displaying only 1280 x 720 pixel video resolution. The client application 185 may determine that the media item 220 has a 1920 × 1080 resolution and may exclude the media item 220 from the media items of the playlist 200 available for playback by the client device 180. As an alternative, the client application 185 may determine that the source device 120 can scale down the video of media item 220 to 1280 x 720 resolution and media in the media items of playlist 200 available for playback. May include item 220.
0081In another example, the client application 185 may determine the connection speed of the communication link 152 between the client device 180 and the source device 120, eg, the bandwidth of 10 Mbit / s. Based on the connection speed, the client application 185 can determine if there is enough bandwidth to stream a particular media item without excessive buffering. Since the media item 220 has a bit rate of 12 Mbit / s, which is greater than the available 10 Mbit / s bandwidth, the client application 185 can use the media item 220 from the list of media items in the playlist 220 available for playback. Can be excluded. The client application 185 can inspect the bitrate attribute of media item 222 and has a value of 320 kbit / s less than the bandwidth of 10 Mbit / s, so it is in the list of media items in playlist 220 available for playback. May include media item 222.
0082The ability of the client device 180 to determine which media items it is capable of playing and present only those media items to the user can be useful in situations where the client device 180 has limited functionality. As an example, in a car setting, the client device 180 may be hardware built into the car, such as a seatback media player, which may not receive updates containing newer codecs or media profiles. Therefore, the client device 180 may not be able to display significantly different media items, and the media items that the client device 180 may not be able to play are playlists of media items that are ultimately presented to the user. Should be excluded from.
0083The attributes of playlists 200, 202 and media items 220, 222, and 224 can be stored in various formats. In some examples, attributes and their associated values are XML (extensible markup language), binary numbers, CSV (comma-separated values), HTML (hypertext markup language), or other records. Can be stored in format. In some examples, the attributes of a playlist and media items may be stored in the playlist itself. In some cases, the attributes and their associated values may be stored in a separate database that the media sharing application 125 can index based on a unique identifier associated with each playlist and / or media item.
0084FIG. 3 is a block diagram showing an example of a data communication model or protocol stack for a WD system. The data communication model 300 illustrates the interaction between the data and control protocols used to transmit data between the source device and the sink device in the implemented WD system. In one example, the WD system 100 may use the data communication model 300. The data communication model 300 includes a physical (PHY) layer 302, a media access control (MAC) layer (304), an Internet protocol (IP) 306, a user datagram protocol (UDP) 308, and a real-time protocol (RTP) 310. , MPEG2 Transport Stream (MPEG2-TS) 312, Content Protection 314, Packetized Elementary Stream (PES) Packetized 316, Video Codec 318, Audio Codec 320, and Transport Control Protocol (TCP) 322. It includes Real Time Streaming Protocol (RTSP) 324, feedback packetization 328, human interface device constant 330, general user input 332, performance analysis 334, and operating system (OS) 336.
0085Physical Layer 302 and MAC Layer 304 may define the physical signaling, addressing and channel access control used for communication in the WD system. The physical layer 302 and MAC layer 304 are the frequency band structures used for communication, such as the Federal Communications Commission's band or ultra-wideband (UWB) frequencies defined at 700MHz, 2.4GHz, 3.6GHz, 5GHz, 60GHz. Bandwidth structure can be defined. Physical layers 302 and MAC 304 may also define data modulation techniques such as analog and digital amplitude modulation, frequency modulation, phase modulation techniques, and combinations thereof. Physical Layers 302 and MAC304 are also any of the multiplexing techniques, such as Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Code Division Multiple Access (CDMA), or OFDM, FDMA, TDMA and / or CDMA. Combinations can be defined. In one example, the physical layer 302 and media access control layer 304 can be defined by Wi-Fi (eg, IEEE 802.11-3007 and 802.11n-3009x) standards, such as the Wi-Fi standard provided by WFD. In another example, the physical layer 302 and media access control layer 304 can be defined by any of WirelessHD, Wireless Home Digital Interface (WHDI), WiGig, and Wireless USB.
0086Internet Protocol (IP) 306, User Datagram Protocol (UDP) 308, Real Time Protocol (RTP) 310, Transport Control Protocol (TCP) 322, and Real Time Streaming Protocol (RTSP) 324 are packet structures used in WD systems. And encapsulation can be defined according to standards maintained by the Internet Engineering Task Force (IETF).
0087RTSP324 is used by source device 120 and sink device 160 to negotiate features and establish sessions and session maintenance and management, and to transmit media items in accordance with the techniques disclosed. Can be used by 120 and sink device 160. For example, the source device 120 may send a feature request message (eg, an RTSP GET_PARAMETER request message) to the sink device 160 that specifies a list of features related to the source device 120. The sink device 160 may respond to the source device 120 with a feature response message (eg, RTSP GET_PARAMETER response message) that declares the functionality of the sink device 160 that supports that feature. As an example, if the sync device 160 supports the above features, the feature response message may indicate "yes". The source device 120 then receives an acknowledgment request message (eg, RTSP) indicating that the above features are supported. SET_PARAMETER request message) can be sent to sync device 160. The sink device 160 may respond to the source device 120 with an acknowledgment message (eg, RTSP SET_PARAMETER response message) that acknowledges that the above features will be used during a media sharing session.
0088Video codec 318 may define video data coding techniques that may be used by the WD system. Video codec 318 is ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, Any number of video compression standards can be implemented, including ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), VP8 and High Efficiency Video Coding (HEVC). Note that in some cases the WD system can be either compressed video data or uncompressed video data.
0089Audio codec 320 may define audio data coding techniques that can be used by the WD system. Audio data can be coded using a multi-channel format, such as the multi-channel format developed by Dolby and Digital Theater Systems. Audio data can be coded using a compressed or uncompressed format. Examples of compressed audio formats are MPEG-1, 2 Audio Layers II and III, AC-3, and AAC. An example of an uncompressed audio format is the pulse code modulation (PCM) audio format.
0090Packetized Elementary Streams (PES) Packetized 316 and MPEG2 Transport Streams (MPEG2-TS) 312 can define how coded audio and video data is packetized and transmitted. Packetized Elementary Streams (PES) Packetized 316 and MPEG-TS312 can be defined according to MPEG-2 Part 1. In another example, audio and video data can be packetized and transmitted according to other packetizing and transport stream protocols. Content protection 314 may provide protection against unauthorized copying of audio or video data. In one example, content protection 314 can be defined according to the High bandwidth Digital Content Protection 2.0 specification.
0091FIG. 4 is a block diagram showing an example of a source device that may implement a technique for sending video and / or application data to a sink device. The source device 400 may be part of a WD system that incorporates the data communication model given in FIG. The source device 400 may be configured to encode and / or decode media data for transport, storage, and / or display. The source device 400 includes a memory 402, a display processor 404, a local display 406, an audio processor 408, a speaker 410, a video encoder 412, a video packetizer 414, an audio encoder 416, and an audio packetizer 418. A / V It includes a mux 420, a transport module 422, a modem 424, a control module 426, a feedback depacketizer 428, and a feedback module 430. Source device 400 components include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, etc. , Any of a variety of suitable circuits, or any combination thereof.
0092Memory 402 may store A / V visual data in the form of media data in compressed or uncompressed format. Memory 402 may include an entire media data file or may include, for example, a smaller buffer that stores only a portion of the media data file streamed from another device or source. Memory 402 includes, but is not limited to, random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), and electrically erasable programmable read-only memory. (EEPROM®), may include any of a wide variety of volatile or non-volatile memories, including flash memory and the like. The memory 402 may include a computer-readable storage medium for storing media data as well as other types of data. Memory 402 may further store instructions and program code executed by the processor as part of performing the various techniques described herein.
0093The display processor 404 may acquire the captured video frame and process the video data for display on the local display 406. The display 406 is of a variety of display devices, including liquid crystal displays (LCDs), plasma displays, organic light emitting diode (OLED) displays, or other types of display devices capable of presenting video data to users of the source device 400. Equipped with one of them.
0094The audio processor 408 may take an audio-captured audio sample and process the audio data for output to the speaker 410. Speaker 410 may include any of a variety of audio output devices, such as headphones, single-speaker systems, multi-speaker systems, or surround sound systems.
0095The video encoder 412 can acquire video data from memory 402 and encode the video data into a desired video format. The video encoder 412 can be a combination of hardware and software used to implement the aspects of video codec 318 described above with respect to FIG. The video encoder 412 is ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, Video can be encoded according to any number of video compression standards, including ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), VP8 and High Efficiency Video Coding (HEVC). Note that in some cases, the video encoder 412 may encode the video so that the video data is compressed using lossy or lossy compression techniques.
0096The video packetizer 414 can packetize the encoded video data. In one example, the video packetizer 414 may packetize the encoded video data as defined according to MPEG-2 Part 1. In another example, the video data can be packetized according to other packetization protocols. The video packetizer 414 can be a combination of hardware and software used to implement the packetization Elementary Stream (PES) packetization 216 aspect described above with respect to FIG.
0097The audio encoder 416 can acquire audio data from memory 402 and encode the audio data into a desired audio format. The audio encoder 416 can be a combination of hardware and software used to implement aspects of the audio codec 320 described above with respect to FIG. Audio data can be coded using a multi-channel format, such as the multi-channel format developed by Dolby and Digital Theater Systems. Audio data can be coded using a compressed or uncompressed format. Examples of compressed audio formats are MPEG-1, 2 Audio Layers II and III, AC-3, and AAC. An example of an uncompressed audio format is a pulse code modulated (PCM) audio format.
0098The audio packetizer 418 can packetize the encoded audio data. In one example, the audio packetizer 418 may packetize the encoded audio data as defined according to MPEG-2 Part 1. In another example, the audio data can be packetized according to other packetization protocols. The audio packetizer 418 can be a combination of hardware and software used to implement the packetization Elementary Stream (PES) packetization 316 aspect described above with respect to FIG.
0099The A / V mux 420 may apply multiplexing techniques to combine video and audio payload data as part of a common data stream. In one example, the A / V mux420 may encapsulate a packetized elementary video stream and an audio stream as an MPEG2 transport stream defined according to MPEG-2 Part 1. The A / V mux420 may provide synchronization for audio and video packets, as well as error correction techniques.
0100Transport module 422 may process media data for transport to sink devices. In addition, the transport module 422 may process the packets received from the sink device so that they can be further processed. For example, transport module 422 may be configured to communicate using IP, TCP, UDP, RTP, and RTSP. For example, transport module 422 may also encapsulate MPEG2-TS for communication to or over the network to sink devices.
0101Modem 424 may be configured to perform physical and MAC layer processing according to the physical and MAC layers utilized in the WD system. As explained with reference to Figure 3. The physical and MAC layers can define the physical signaling, addressing and channel access control used for communication in WD systems. In one example, modem 424 is to perform physical and MAC layer processing for the physical and MAC layers defined by the Wi-Fi (eg, IEEE802.11x) standard, such as the Wi-Fi standard provided by WFD. Can be configured. In another example, modem 424 may be configured to perform physical and MAC layer processing for any of WirelessHD, WiMedia, Wireless Home Digital Interface (WHDI), WiGig, and Wireless USB.
0102The control module 426 may be configured to perform the source device 400 communication control function. Communication control functions can relate to negotiating functions with sink devices, establishing sessions with sink devices, and maintaining and managing sessions. Control module 426 may use RTSP to communicate with the sink device. In addition, control module 426 may use RTSP message transactions to negotiate the functionality of the source device 400 and the sink device to support the functionality of UIBC.
0103The feedback depacketizer 428 can parse human interface device commands (HIDCs) from feedback packets, general user inputs, OS-specific user inputs, and performance information. The feedback category field may identify a general input category to indicate that the feedback packet payload data is formatted using general information elements. As another example, the feedback category field can identify the Human Interface Device Command (HIDC) input category. As another example, the Feedback Category field identifies an OS-specific input category to indicate that the payload data is formatted based on the type of operating system (OS) used by either the source device or the sink device. obtain.
0104The feedback module 430 receives performance information from the feedback depacketizer and processes the performance information so that the source device 400 can coordinate the transmission of media data based on the performance information message.
0105Source device 400 provides an example of a source device configured to send content to a second wireless computing device. The source device 400 initiates a WI-FI display (WFD) connection, sends data from the first wireless computing device to the second wireless computing device over the WFD connection, and the first wireless computing device. Can run media sharing applications that allow a playlist media item to be shared with a wireless client computing device and send information describing the playlist media item to the wireless client computing device. Sending information that describes a media item causes the wireless client computing device to determine if the wireless client computing device is capable of outputting media items, and the wireless client computing device. Can send media items to.
0106FIG. 5 is a block diagram showing an example of a sink or client device that implements a technique for receiving video and / or other information from a source device. The sink or client device 500 may be part of a WD system that incorporates the data communication model given in FIG. In one example, the sink or client device 500 may form a WD system with the source device 400. The sink or client device 500 includes a modem 502, a transport module 504, and an A / V. demux506, video depacketizer 508, video decoder 510, display processor 512, display 514, audio depacketizer 516, audio decoder 518, audio processor 520, speaker 522, user input module 524 Includes a performance analysis module 526, a feedback processor 528, and a control module 530. Each sink or client device 500 consists of one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, and hardware. , Firmware, etc., can be implemented as any of a variety of suitable circuits, or any combination thereof.
0107Modem 502 may be configured to perform physical and MAC layer processing according to the physical and MAC layers utilized in the WD system. As explained with reference to Figure 3. The physical and MAC layers can define the physical signaling, addressing and channel access control used for communication in WD systems. In one example, modem 502 is to perform physical and MAC layer processing for the physical and MAC layers defined by the Wi-Fi (eg, IEEE802.11x) standard, such as the Wi-Fi standard provided by WFD. Can be configured. In another example, the modem 502 may be configured to perform physical and MAC layer processing for any of WirelessHD, WiMedia, Wireless Home Digital Interface (WHDI), WiGig, and Wireless USB.
0108Transport module 504 may process the received media data from the source device. In addition, transport module 504 may process feedback packets for transport to the source device. For example, transport module 504 may be configured to communicate using IP, TCP, UDP, RTP, and RSTP. In addition, transport module 504 may include a time stamp value in any combination of IP, TCP, UDP, RTP, and RSTP packets. The time stamp value may allow the source device to identify which media data packet experienced the reported performance degradation and to calculate the round trip delay in the WD system.
0109The A / V demux506 may apply demultiplexing techniques to separate the video and audio payload data from the data stream. In one example, the A / V mux506 may separate the packetized elementary video stream and the audio stream of the MPEG2 transport stream defined according to MPEG-2 Part 1.
0110The video depacketizer 508 and video decoder 510 perform the reverse of the video packetizer and video encoder that implement the packetization and coding techniques described herein and output the video output video data to the display processor 512. obtain.
0111The display processor 512 may acquire the captured video frame and process the video data for display on the display 514. The display 514 may include one of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display.
0112The audio depacketizer 516 and the audio decoder 518 may perform the reverse process of the audio packetizer and audio encoder implementing the packetizing and coding techniques described herein and output audio data to the display processor 520.
0113The audio processor 520 may obtain audio data from the audio decoder and process the audio data for output to speaker 522. Speaker 522 may include any of a variety of audio output devices, such as headphones, single speaker systems, multi-speaker systems, or surround sound systems.
0114The user input module 524 formats user input commands received by a user input device, such as a keyboard, mouse, trackball or trackpad, touch screen, voice command recognition module, or other such user input device. obtain. In one example, the user input module 524 formats the user input commands according to the formats defined according to the human interface device command (HIDC) 330, general user input 332, and OS specific user input 336 described above with respect to FIG. obtain.
0115The performance analysis module 526 can determine performance information based on media data packets received from the source device. Performance information may include delay jitter, packet loss, error distribution over time, packet error rate, and RSSI distribution over time, as well as other examples described herein. The performance analysis module 526 may calculate performance information according to any of the techniques described herein.
0116The feedback packetizer 528 can packetize and process user input information from the user input module 524 and the performance analysis module generator 526 to create a feedback packet. In one example, the feedback packet may use the message format described with respect to FIG. In addition, the feedback packetizer 528 may include a time stamp value in each of the feedback packets. The time stamp value may allow the source device to identify which media data packet experienced the reported performance degradation and to calculate the round trip delay in the WD system.
0117Control module 530 may be configured to perform sink or client device 500 communication control functions. Communication control features can relate to negotiating features with the source device, establishing a session with the source device, and maintaining and managing the session. Control module 530 may use RTSP to communicate with the source device. In addition, control module 530 may negotiate the functionality of the sink or client device 500 with the source device to support the features of UIBC.
0118In one example, the sink device 500 initiates a WFD connection with a wireless source device, such as the source device 400 (Figure 4), and receives data from the source device 400 to the source device 400 over the WFD connection. Gives an example of a source device configured to do this. A wireless client computing device, which may be similar to or equivalent to the sync device 500, may run a media client application that allows the wireless client device to receive shared media items in playlists on the source device 400. The wireless client computing device may receive information describing the media items of the playlist from the source device 400. Receiving information describing a media item may cause the wireless client computing device to determine whether the wireless client computing device is capable of outputting the media item. The wireless client computing device may receive media items from the source device 400.
0119FIG. 6 shows an exemplary transmitter system 610 and receiver system 650 that can be used by transmitter / receiver 126 and transmitter / receiver 166 of FIG. 1A to communicate over communication channel 150. A block diagram is shown. In transmitter system 610, traffic data for several data streams is fed from data source 612 to transmit (TX) data processor 614. Each data stream may be transmitted via its own transmitting antenna. The TX data processor 614 formats, codes, and interleaves the traffic data for each data stream based on the particular coding scheme selected for that data stream.
0120The coded data in each data stream can be multiplexed with pilot data using Orthogonal Frequency Division Multiplexing (OFDM) techniques. A wide variety of others, including, but not limited to, time division multiple access (TDMA), frequency division multiple connection (FDMA), code division multiple connection (CDMA), or any combination of OFDM, FDMA, TDMA and / or CDMA. Wireless communication techniques can also be used.
0121According to FIG. 3, the pilot data is typically a known data pattern processed in a known manner and can be used in a receiver system to estimate the channel response. The multiplexed pilot and coded data in each data stream is then given a particular modulation scheme (eg, 2-phase shift keying (BPSK), 4-phase shift) selected for that data stream to give a modulation symbol. Modulated (eg, symbol mapping) based on keying (QPSK), M-PSK, or M-QAM (quadrature keying), where M can be a power of 2. The data rate, coding, and modulation of each data stream can be determined by instructions executed by processor 630, which can be coupled to memory 632.
0122A modulation symbol for the data stream is then given to the TX MIMO processor 620, which may further process that modulation symbol (for example, for OFDM). The TX MIMO processor 620 is then N<sub>T</sub>N modulated symbol streams<sub>T</sub>It can be given to multiple transmitters (TMTR) 622a ~ 622t. In some embodiments, the TX MIMO processor 620 applies beamforming weights to a symbol in the data stream and to the antenna from which the symbol is transmitted.
0123Each transmitter 622 receives and processes its own symbol stream to give one or more analog signals, and then tunes (eg, amplifies, filters, and upconverts) those analog signals. , A modulated signal suitable for transmission via a MIMO channel can be provided. Then N from transmitters 622a ~ 622t<sub>T</sub>Each modulated signal is N<sub>T</sub>It is transmitted from the antennas 624a to 624t.
0124In the receiver system 650, the transmitted modulated signal is N<sub>R</sub>It is received by the antennas 652a to 652r, and the received signal from each antenna 652 is given to the respective receivers (RCVR) 654a to 654r. The receiver 654 tunes (eg, filters, amplifies, and downconverts) each received signal, digitizes the tuned signal, gives a sample, and further processes the sample to make a corresponding " Gives a "receive" symbol stream.
0125The receive (RX) data processor 660 is then N<sub>R</sub>Receivers 654 to N<sub>R</sub>Receives a received symbol stream, processes it based on a particular receiver processing technique, and N<sub>T</sub>Gives a "detection" symbol stream. The RX data processor 660 then demodulates, deinterleaves, and decodes each detected symbol stream to restore the traffic data in the data stream. The processing by the RX data processor 660 complements the processing performed by the TX MIMO processor 620 and the TX data processor 614 in the transmitter system 610.
0126Processor 670, which can be combined with memory 672, periodically determines which precoding matrix should be used. The reverse link message can contain various types of information about the communication link and / or the received data stream. The reverse link message is then processed by the TX data processor 638, which also receives the traffic data of some data streams from the data source 636, modulated by the modulator 680, tuned by the transmitters 654a-654r, and the transmitter. Reply to system 610.
0127In transmitter system 610, the modulated signal from receiver system 650 is received by antenna 624, tuned by receiver 622, demodulated by demodulator 640, processed by RX data processor 642, and received by receiver system 650. The reverse link message sent by is extracted. Processor 630 then determines which precoding matrix should be used to determine the beamforming weights, and then processes the extracted message.
0128FIG. 7A is a block diagram showing an exemplary message forwarding sequence between the source device 120 and the sink device 160 as part of functional negotiation for a WFD session. Functional negotiation can take place as part of the larger WFD communication session establishment process between the source device 120 and the sink device 160. This session can be established, for example, with WI-FI Direct or TDLS as the underlying connectivity standard. After establishing a WI-FI Direct or TDLS session, the sink device 160 can initiate a TCP connection with the source device 120. As part of establishing a TCP connection, a control port running a real-time streaming protocol can be established to manage the communication session between the source device 120 and the sink device 160.
0129The source device 120 may generally operate in the same manner as described above for the source device 120 of FIG. 1A, and the sink device 160 may generally operate in the same manner as described above for the sink device 160 of FIG. 1A. Can work. After the source device 120 and sink device 160 establish connectivity, the source device 120 and sink device 160 are parameters that should be used for subsequent communication sessions of those devices as part of a functional negotiation exchange. You can judge the set.
0130The source device 120 and the sink device 160 can negotiate functionality through a sequence of messages. Those messages can be, for example, Real Time Streaming Protocol (RTSP) messages. At any stage of that negotiation, the recipient of the RTSP request message may respond with an RTSP response that contains an RTSP status code other than RTSP OK, in which case the message exchange may be retried with a different set of parameters. Alternatively, the functional negotiation session can be terminated.
0131The source device 120 can send a first message (RTSP OPTIONS request message) to the sink device 160 to determine the set of RTSP methods supported by the sink device 160. Upon receiving the first message from the source device 120, the sink device 160 can respond with a second message (RTSP OPTIONS response message) listing the RTSP methods supported by the sink 160. Also, the second message may contain the RTSP OK status code.
0132After sending the second message to the source device 120, the sink device 160 can send a third message (RTSP OPTIONS request message) to determine the set of RTSP methods supported by the source device 120. Upon receiving the third message from the sink device 160, the source device 120 can respond with a fourth message (RTSP OPTIONS response message) listing the RTSP methods supported by the source device 120. Also, the fourth message can include the RTSP OK status code.
0133After sending the fourth message, the source device 120 can send a fifth message (RTSP GET_PARAMETER request message) to specify a list of features related to the source device 120. The sink device 160 can respond with a sixth message (RTSP GET_PARAMETER response message). The sixth message may contain the RTSP status code. If the RTSP status code is OK, the sixth message can also include response parameters to the parameters specified in the fifth message, supported by sync device 160. The sink device 160 can ignore the parameters in the fifth message that the sink device 160 does not support.
0134Based on the sixth message, the source 120 can determine the optimal set of parameters to be used for the communication session and can send a seventh message (RTSP SET_PARAMETER request message) to the sink device 160. it can. The seventh message may contain a set of parameters that should be used during a communication session between the source device 120 and the sink device 160. The seventh message is the Universal Resource Identifier (URI) that should be used in the RTSP Setup request to set up the communication session. Can include a wfd-presentation-url that describes the Identifier). wfd-presentation-url specifies a URI that the sink device 160 can use for later messages during a session establishment exchange. The wfd-url0 and wfd-url1 values specified in this parameter can correspond to the rtp-port0 and rtp-port1 values in wfd-client-rtp-ports in the seventh message. .. RTP in this case generally refers to a real-time protocol that can run on top of UDP.
0135Upon receiving the seventh message, the sink device 160 can respond with an eighth message with an RTSP status code indicating whether the parameters specified in the seventh message were successfully set. .. As mentioned above, the roles of the source device and the sink device can be reversed or changed in different sessions. The order of messages that set up a communication session can, in some cases, define a device that acts as a source and a device that acts as a sink.
0136FIG. 7B is a block diagram showing another exemplary message forwarding sequence between the source device 120 and the sink device 160 as part of a functional negotiation session. The message forwarding sequence of FIG. 7B is intended to provide a more detailed diagram of the forwarding sequence described above for FIG. 7A. In Figure 7B, the message "1b.GET_PARAMETER response" shows an example of a message that identifies a list of supported input categories (eg, general and HIDC) from a list of supported input types. List of Supported Input Categories Each of the supported input categories has an associated list of supported types (eg generic_cap_list and hidc_cap_list). In Figure 7B, the message "2a.SET_PARAMETER request" identifies a second list of supported input categories (eg general and HIDC) from a second list of supported types. This is an example of a message. Second List of Supported Input Categories Each of the supported input categories has an associated second list of supported types (eg generic_cap_list and hidc_cap_list). The message "1b.GET_PARAMETER response" identifies the input categories and input types supported by the sink device 160. The message "2a.SET_PARAMETER request" identifies the input categories and input types supported by the source device 120, but it is not a comprehensive list of all input categories and input types supported by the source device 120. Sometimes. Instead, the message "2a.SET_PARAMETER request" is assumed to be supported by the sink device 160 and the message "1b. Only the input category and input type identified in the GET_PARAMETER response can be identified. In this way, the input category and input type identified in the message "2a.SET_PARAMETER request" may constitute a subset of the input category and input type identified in the message "1b.GET_PARAMETER response".
0137FIG. 8 is a flowchart showing a method of executing WFD and transmitting media items by the technique of the present disclosure. The method of FIG. 8 can be performed by a device, such as the source device 120 of FIGS. 1A and 1B, as an example. Although FIG. 8 describes the source device 120, other source devices, including the source device 400 of FIG. 4, may also perform the technique of FIG. In the method of FIG. 8, the source device 120 may initiate a WFD connection to the sink device 160 (FIG. 1A) (800). In some examples, the sink device 160 and the client device 180 can be the same device.
0138The source device 120 may initiate a WI-FI display (WFD) connection to the sink device 160 (800). The source device 120 may also transmit data from the source device 120 to the sink device 160 (FIG. 1A) over a WFD connection (802). In one example, transmitting data from the source device 120 to the sink device 160 may cause the sink device 160 to mirror the display output device of the source device 120. The source device 120 may also receive user input from the sink device 160.
0139Source device 120 may run a media sharing application that allows source device 120 to share playlist media items with client device 180 (Figure 1B) (804). The source device 120 may transmit information to the client device 180, describing the media item and the format of the playlist media item (806). In some examples, the information that describes a media item has at least one of extensible markup language (XML), binary, hypertext markup language (HTML), and comma-separated values (CSV). .. Information describing a media item can include at least one of the media item's bit rate, level, resolution, file type, and filename.
0140The source device 120 may also transmit WFD data relating to the playlist and at least one of the playlist's media items to the wireless client device 180 via a second WFD connection. The source device 120 may also receive user input backchannel input commands from the client device 180 via a second WFD connection. In response to receiving the UIBC input command, the first wireless computing device may terminate the second WFD connection.
0141Sending information describing a media item in a playlist may cause the client device 180 to determine if the client device 180 is capable of outputting the media item (808). Source device 120 may send media items to client device 180 (810). The source device 120 may also receive media playback commands from the client device 180, and the media playback commands may include RTSP instructions in some examples. In some cases, the source device 120 may use RTP to transmit media items. In some cases, transmitting media items using the source device 120 may occur after receiving a media playback command from the client device 180.
0142The source device 120 may also receive credentials from the client device 180. The source device 120 may authenticate the wireless client device based on the credentials and allow the client device 180 to access the playlist in response to authenticating the client device 180.
0143Although shown in a specific order for example, the method in Figure 8 can be performed in any order or in parallel, and the method in Figure 8 is to send data using a WFD connection and at the same time media. It may even be prepared to run a shared application.
0144In one or more examples, the functionality described in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, a function may be stored on a computer-readable medium as one or more instructions or codes, or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium that corresponds to a tangible medium such as a data storage medium, or any medium that allows the transfer of a computer program from one location to another, for example, according to a communication protocol. Can include communication media including. In this way, the computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier. The data storage medium is any available that can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for the implementation of the techniques described in this disclosure. It can be a medium. Computer program products may include computer-readable media.
0145As an example, but not limited to, such computer-readable storage media can be RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage device, flash memory, or instruction or data structure. Any other medium that can be used to store the desired program code of the form and can be accessed by a computer can be provided. Also, any connection is properly referred to as a computer-readable medium. For example, instructions are sent from a website, server, or other remote source using coaxial cable, fiber optic cable, twist pair, digital subscriber line (DSL), or wireless technology such as infrared, wireless, and microwave. Where so, coaxial cables, fiber optic cables, twisted pairs, DSL, or wireless technologies such as infrared, wireless, and microwave are included in the definition of medium. However, it should be understood that computer-readable and data storage media do not include connections, carriers, signals, or other temporary media, but instead target non-temporary tangible storage media. The discs and discs used herein are compact discs (CDs), laser discs (registered trademarks) (discs), optical discs, and digital versatile discs (DVDs). ), Flop (registered trademark) discs and Blu-ray discs, discs typically play data magnetically, and discs optically laser data data. Play to. The above combinations should also be included within the scope of computer readable media.
0146Instructions can be one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrations. It can be performed by a circuit or a discrete logic circuit. Thus, the term "processor" as used herein may refer to either the aforementioned structure or any other structure suitable for implementing the techniques described herein. Moreover, in some embodiments, the functionality described herein may be provided within dedicated hardware and / or software modules configured for coding and decoding, or may be incorporated into a composite codec. Also, the technique may be fully implemented in one or more circuits or logic elements.
0147The techniques of the present disclosure can be implemented in a wide variety of devices or devices, including wireless handsets, integrated circuits (ICs) or sets of ICs (eg, chipsets). Although various components, modules, or units have been described herein to emphasize the functional aspects of devices configured to perform the disclosed techniques, these components, modules, or units are described. It does not necessarily have to be achieved by different hardware units. Rather, as described above, various units, along with suitable software and / or firmware, are combined or interacting hardware in the codec hardware unit, including one or more processors described above. It can be given by a set of wear units.
0148Various examples have been described. These and other examples fall within the scope of the following claims. The inventions described in the claims of the original application of the present application are described below. [C1] A method of transmitting the content of the first wireless computing device to the second wireless computing device, wherein the method is: Using the first wireless computing device to initiate a WI-FI display (WFD) connection with the second wireless computing device. Using the first wireless computing device to transmit data from the first wireless computing device to the second wireless computing device over the WFD connection. Using the first wireless computing device to run a media sharing application that allows the first wireless computing device to share playlist media items with a wireless client computing device. Using the first wireless computing device to transmit information describing the media item of the playlist to the wireless client computing device, and transmitting the information describing the media item. To cause, transmit, and cause the wireless client computing device to determine whether the wireless client computing device is capable of outputting the media item. Using the first wireless computing device to transmit the media item to the wireless client computing device. A method. [C2] Sending the data from the first wireless computing device to the second wireless computing device mirrors the display output device of the first wireless computing device to the second wireless computing device. The method described in [C1] that allows you to do what you want to do. [C3] The information describing the format of the media item comprises at least one of extensible markup language (XML), binary, hypertext markup language (HTML), and comma-separated values (CSV). The method described in C1]. [C4] The method according to [C1], further comprising receiving a media playback command from the wireless client computing device. [C5] The method according to [C4], wherein the media playback command comprises a Real Time Streaming Protocol (RTSP) instruction. [C6] The method according to [C4], wherein the transmission of the media item is performed in response to receiving the media playback command from the wireless client computing device. [C7] The method according to [C1], further comprising receiving user input from the second wireless computing device using the first wireless computing device. [C8] The method according to [C1], wherein the data transmission via the WFD connection and the execution of the media sharing application are performed at the same time. [C9] The method according to [C1], wherein the information describing the media item comprises at least one of the bit rate, level, resolution, file type, and file name of the media item. [C10] Using the first wireless computing device to receive authentication information from the wireless client computing device, Using the first wireless computing device to authenticate the wireless client device based on the authentication information, Using the first wireless computing device to allow the wireless client device to access the playlist in response to authenticating the wireless client device. The method described in [C1], further comprising. [C11] The method according to [C1], wherein transmitting the media item comprises streaming the media item using Real Time Transport Protocol (RTP). [C12] The WFD connection comprises a first WFD connection, the method said. WFD data relating to at least one of the playlist and the media item of the playlist to the wireless client computing device via a second WFD connection using the first wireless computing device. To send and Using the first wireless computing device to receive user input back channel (UIBC) input commands from the wireless client computing device over the second WFD connection. The method described in [C1], further comprising. [C13] Using the first wireless computing device to terminate the second WFD connection in response to receiving the UIBC input command from the wireless client computing device. The method described in [C12], further comprising. [C14] It is possible to execute the media sharing application that allows the first wireless computing device to share the media item in the playlist with the wireless client computing device. Sending the playlist to the wireless client computing device using the first wireless computing device. The method described in [C1], further comprising. [C15] The method according to [C1], wherein the second wireless computing device and the wireless client computing device are the same device. [C16] The first wireless computing device, Initiating a WI-FI display (WFD) connection with a second wireless computing device To transmit data from the first wireless computing device to the second wireless computing device via the WFD connection. With a WI-FI display (WFD) module configured to do To run a media sharing application that allows the first wireless computing device to share playlist media items with wireless client computing devices. The transmission of the information describing the media item of the playlist to the wireless client computing device, the transmission of the information describing the media item to the wireless client computing device, said the wireless client. Letting the computing device determine if it is possible to output the media item, transmitting and transmitting. Sending the media item to the wireless client computing device With a media sharing module configured to do The first wireless computing device with. [C17] The transmission of the data from the first wireless computing device to the second wireless computing device mirrors the display output device of the first wireless computing device to the second wireless computing device. The first wireless computing device described in [C16] that allows you to do so. [C18] The information describing the format of the media item comprises at least one of extensible markup language (XML), binary, hypertext markup language (HTML), and comma-separated values (CSV). The first wireless computing device described in C16]. [C19] The first wireless computing device according to [C16], wherein the media sharing module is further configured to receive media playback commands from the wireless client computing device. [C20] The first wireless computing device according to [C19], wherein the media playback command comprises a Real Time Streaming Protocol (RTSP) instruction. [C21] The first wireless computing device according to [C19], wherein the media sharing module is configured to transmit the media item in response to receiving the media playback command from the wireless client computing device. .. [C22] The first wireless computing device according to [C16], wherein the WFD module is further configured to receive user input from the second wireless computing device. [C23] The first wireless computing device according to [C16], wherein the data transmitted via the WFD connection and the execution of the media sharing application are performed at the same time. [C24] The first wireless computing according to [C16], wherein the information describing the format of the media item comprises at least one of the bit rate, level, resolution, file type, and file name of the media item. Ing device. [C25] The media sharing module Receiving authentication information from the wireless client computing device To authenticate the wireless client computing device based on the authentication information, To allow the wireless client computing device to access the playlist in response to the authentication of the wireless client computing device. The first wireless computing device described in [C16], further configured to do so. [C26] The first wireless computing device according to [C16], wherein the WFD module is configured to stream the media item using Real Time Transport Protocol (RTP) to transmit the media item. .. [C27] The WFD connection comprises a first WFD connection and the WFD module To transmit WFD data related to at least one of the playlist and the media item of the playlist to the wireless client computing device via a second WFD connection. To receive a user input back channel (UIBC) input command from the wireless client computing device via the second WFD connection. The first wireless computing device described in [C16], further configured to do so. [C28] The WFD module Terminate the second WFD connection in response to receiving the UIBC input command from the wireless client computing device. The first wireless computing device described in [C27], further configured as described above. [C29] To execute the media sharing application that allows the first wireless computing device to share the media item in the playlist with the wireless client computing device, the media sharing module Send the playlist to the wireless client computing device The first wireless computing device described in [C16], further configured as described above. [C30] The first wireless computing device according to [C16], wherein the second wireless computing device and the wireless client computing device are the same device. [C31] The first wireless computing device, A means to initiate a WI-FI display (WFD) connection with a second wireless computing device, A means for transmitting data to the second wireless computing device over the WFD connection and A means for running a media sharing application that allows the first wireless computing device to share playlist media items with a wireless client computing device. The means for transmitting the information describing the media item of the playlist to the wireless client computing device, and the means for transmitting the information describing the media item is the wireless client computing. A means for transmitting, causing the device to determine whether the wireless client computing device is capable of outputting the media item. As a means for transmitting the media item to the wireless client computing device The first wireless computing device with. [C32] When executed, on one or more processors, Initiating a WI-FI display (WFD) connection with a second wireless computing device Sending data from the first wireless computing device to the second wireless computing device via the WFD connection, and To run a media sharing application that allows the first wireless computing device to share playlist media items with wireless client computing devices. By transmitting information describing the format of the media item of the playlist to the wireless client computing device, the transmission of the information describing the media item to the wireless client computing device. Let the wireless client computing device determine if it is possible to output the media item, transmit and transmit. Sending the media item to the wireless client computing device A computer-readable storage medium that stores instructions.
15 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15
Every citation, both ways
| Document | Relation | Office | Category | Cited during | Relevant claims |
|---|---|---|---|---|---|
| JP2004341657A | Cites | Japan | Y | Search report | 9,23 |
| JP2004341657A | Cites | Japan | Y | Search report | 9,23 |
| JP2008176320A | Cites | Japan | A | Search report | – |
| JP2008176320A | Cites | Japan | A | Search report | – |
| JP2009087065A | Cites | Japan | A | Search report | – |
| JP2009087065A | Cites | Japan | A | Search report | – |
| JP2010218146A | Cites | Japan | Y | Search report | 1-30 |
| JP2010510696A | Cites | Japan | A | Search report | – |
| JP2010510696A | Cites | Japan | A | Search report | – |
| US2011107388A1 | Cites | United States of America | Y | Search report | 1-30 |
| US2011107388A1 | Cites | United States of America | Y | Search report | 1-30 |
| 松元 英樹: "ニュース&トレンド", 日経パソコン, vol. 第619号, JPN6017001423, 14 February 2011 (2011-02-14), JP, pages 14 - 15, ISSN: 0003905662 | Non-patent | – | – | Search report | – |
10 members in 7 offices
Priority claims6
| Document | Office | Kind | Date |
|---|---|---|---|
| 61583987 | United States of America | – | |
| 201261583987 | United States of America | P | |
| 61599564 | United States of America | – | |
| 201261599564 | United States of America | P | |
| 13606839 | United States of America | – | |
| 201213606839 | United States of America | A |
Members10
| Document | Office | Kind | |
|---|---|---|---|
| WO2013103726A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2013238702A1 | United States of America | A1 | |
| KR20140110047A | Republic of Korea | A | |
| CN104115466A | China | A | |
| EP2801180A1 | European Patent Office (EPO) | A1 | |
| JP2015510306A | Japan | A | |
| IN4461CHN2014A | India | A | |
| US9525998B2 | United States of America | B2 | |
| CN104115466B | China | B | |
| JP2018113696AThis record | Japan | A |
5 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Written measure of dismissal of application [lapsed due to lack of payment]LapsedJAPANESE INTERMEDIATE CODE: A045A045 | A045 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 |
Numbers
- Publication
- 2018113696
- Application
- 23157
Titles2
- Japanese
- マルチスクリーンサービスを用いたワイヤレスディスプレイ
- English
- Wireless display with multi-screen service
Classification
- CPC, 10
- G06F3/1454
- H04W8/24
- H04L69/24
- G06F3/1423
- G09G2340/02
- G09G2350/00
- G09G2370/04
- G09G2370/10
- G09G2370/16
- H04L67/131
- IPC, 4
- H04N21 436
- H04M1 00
- H04M11 00
- G06F13 00