Method and apparatus for displaying application data in wireless communication system

TECHNICAL FIELD

0001The present invention relates to a method and apparatus for displaying application data in a wireless communication system.

BACKGROUND ART

0002Recently, a wireless fidelity (Wi-Fi) display (WFD) standard has been defined based on a requirement to transmit audio/video (AV) data while satisfying high quality and low latency. A WFD network is a network system suggested by the Wi-Fi Alliance® that enables Wi-Fi devices to be connected to each other in a peer-to-peer fashion without participating in a home network, an office network, or a hot-spot network. WFD devices within a WFD network may be capable of discovering information regarding each other, e.g., capability information, establishing a WFD session, and rendering content received during the WFD session. Typically, a WFD network comprises a WFD source device and one or more WFD sink devices.

0003Between a WFD source and a WFD sink device, first sequence of message (M1 & M2) exchanges is related to device discovery and service discovery. After the device discovery and optional service discovery procedures are completed, the WFD source device and the WFD sink device are assigned an internet protocol (IP) address. Thereafter, a transmission Control Protocol (TCP) connection is established between the source device and the sink device after which real time streaming protocol (RTSP) and real time protocol (RTP) stacks on both devices become active.

0004Over RTSP, capability negotiation (M3 & M4) procedures are performed between the WFD source device and the WFD sink device. Then, WFD session control messages are exchanged between the WFD source device and the WFD sink device. Additionally, data session over RTP is established between WFD source device and WFD sink device in parallel.

0005For WFD devices such as mobile phone, size of display is very limited. Because of restricted size, users may not be able to view entire content on the WFD device. Also, the displayed content is small in size.

0006Current Wi-Fi display specification defined by Wi-Fi Alliance provides a solution for mirroring content on the mobile device to a larger display like TV or a projector. However, the content on the mobile device is as such mirrored to the bigger screen without any intelligence, thereby leading to poor user experience.

DISCLOSURE OF INVENTION

Technical Problem

0007Therefore, there is a need of method and system for providing enhanced mirroring experience of data on at least one sink device.

Solution to Problem

0008An embodiment of the present invention describes a method for displaying application data by a first device in a wireless communication system. The method comprises transmitting, to a second device information as to whether the first device can display data of an application run by the second device, if a connection to the second device is established, receiving, from the second device, the application data and display information for displaying the application data, if the first device can display the application data, and display the application data based on the display information.

0009Another embodiment of the present invention describes a method for transmitting information for displaying application data by a second device in a wireless communication system. The method comprises receiving, from a first device, information as to whether the first device can display data of an application run by the second device, if a connection to the first device is established, and transmitting the application data and display information for displaying the application data, to the first device, if the first device can display the application data.

BRIEF DESCRIPTION OF DRAWINGS

0010The aforementioned aspects and other features of the present invention will be explained in the following description, taken in conjunction with the accompanying drawings, wherein:

0011<figref idref="DRAWINGS">FIG. 1</figref> illustrates a system for providing enhanced mirroring experience of data on at least one sink device according to an embodiment.

0012<figref idref="DRAWINGS">FIG. 2</figref> illustrates a pictorial representation of enhanced mirroring of application session containing Vcard information on a sink device, according to one exemplary embodiment.

0013<figref idref="DRAWINGS">FIG. 3</figref> illustrates a pictorial representation of enhanced mirroring of application session containing application screen information on a sink device, according to another exemplary embodiment.

0014<figref idref="DRAWINGS">FIG. 4</figref> illustrates a flow diagram of a method of providing enhanced mirroring experience of data to a sink device, according to one embodiment.

0015<figref idref="DRAWINGS">FIG. 5</figref> illustrates a flow diagram of a method of providing enhanced mirroring experience of data to at least one sink device according to one embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

0016The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments. The present invention can be modified in various forms. Thus, the embodiments of the present invention are only provided to explain more clearly the present invention to the ordinarily skilled in the art of the present invention. In the accompanying drawings, like reference numerals are used to indicate like components.

0017The specification may refer to “an”, “one” or “some” embodiment(s) in several locations. This does not necessarily imply that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

0018As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms “includes”, “comprises”, “including” and/or “comprising” when used in this specification, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any and all combinations and arrangements of one or more of the associated listed items.

0019Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

0020<figref idref="DRAWINGS">FIG. 1</figref> illustrates a system <b>100</b> for providing enhanced data to at least one sink device in a Wi-Fi display environment, according to one embodiment. The system <b>100</b> comprises one source device <b>101</b> and one or more Wi-Fi sink devices <b>102</b> such as a sink device <b>1</b><b>102</b><i>a</i>, a sink device <b>2</b><b>102</b><i>b</i>, a sink device <b>3</b><b>102</b><i>c </i>. . . a sink device n <b>102</b><i>n. </i>

0021A Wi-Fi display (WFD) session is established between a WFD source device and a WFD sink device, where the WFD source device provides content and the WFD sink device consumes the content. In the present invention, the WFD source device mirrors data with the WFD sink device in an enhanced manner.

0022The system <b>100</b> comprises means for establishing a connection between a source device <b>101</b> and the at least one sink device, means for exchanging capability information between the source device <b>101</b> and the at least one sink device, means for initiating an enhanced display session between the source device <b>101</b> and the at least one sink device, means for providing session data with application description format (ADF) information to the at least one sink device for the enhanced display session, and means for displaying the data on the at least one sink device formatted according to the ADF information. In one embodiment, the means for establishing a connection, means for exchanging capability information, means for establishing an enhanced display session, and means for sending session data include but not limited to, one or more of a multiplexer, processor, encoder, and transceiver at the source device <b>101</b>. Further, the means for displaying the data includes display unit.

0023Extended Data Transferring

0024In exemplary embodiment, whenever a user mirrors display screen displayed on the WFD source device to the WFD sink device, the user has an option to select an application such as phonebook to view contact details of friends or applications list to view all the applications installed on the mobile phone (i.e., the WFD source device). Once the phonebook is mirrored with the WFD sink device, the user is able to see not only the contact details of the selected contact but also the additional stored information of the selected contact.

0025<figref idref="DRAWINGS">FIG. 2</figref> illustrates a pictorial representation of enhanced mirroring of application session containing Vcard information on a sink device, according to one exemplary embodiment. In <figref idref="DRAWINGS">FIG. 2</figref>, a smartphone <b>201</b> is a WFD source device and a television <b>202</b> is a WFD sink device. At first, user minors a video from the smartphone <b>201</b> to the TV <b>202</b>. During an ongoing WFD session, the user selects “Contacts” application as a contact application on the smartphone <b>201</b>. Then, the ongoing WFD session is paused and the smartphone <b>201</b> opens the contact application and new application session starts between the smartphone <b>201</b> and the TV <b>202</b> which contains relevant application information i.e., the application information is sent to the TV <b>202</b>. Consequently, application related detailed information is shown on the display screen of the TV <b>202</b>.

0026<figref idref="DRAWINGS">FIG. 3</figref> illustrates a pictorial representation of enhanced mirroring of application session containing application screen information on a sink device, according to another exemplary embodiment. In <figref idref="DRAWINGS">FIG. 3</figref>, a smart phone <b>301</b> is a WFD source device and a television <b>302</b> is a WFD sink device. User minors a video from the smartphone <b>301</b> to the TV <b>302</b>. During an ongoing session, the user selects “Apps” icon related to an application list on the smartphone <b>301</b>. A notification (e.g. “Do you want see apps on TV?”) is popped on the display screen of the smartphone <b>301</b> to confirm whether the applications list is to be displayed on the larger screen of the TV <b>302</b> with which a WFD session is ongoing. If the user confirms, then, an application session is initiated between the smartphone <b>301</b> (WFD source device) and the TV <b>302</b> (WFD sink device) containing the application information which may in affect also contain the sink screen information. Thus, the user can select a desired application from the applications list displayed on the TV. When the user selects the desired application, a notification is sent to the mobile phone using the back channel.

0027<figref idref="DRAWINGS">FIG. 4</figref> illustrates a flow diagram of a method of providing enhanced data to a sink device, according to one embodiment.

0028In one embodiment, a WFD source device <b>400</b> has been configured with an enhanced display application. The enhanced display application can either be installed in the WFD source device <b>400</b> or can be embedded in the WFD source device <b>400</b> at the stage of manufacturing of the WFD source device <b>400</b>.

0029At steps <b>401</b> to <b>409</b>, the WFD source device <b>400</b> and a WFD sink device <b>420</b> may exchange capability information with each other. Specifically, at step <b>401</b>, the WFD source device <b>400</b> and the WFD sink device <b>420</b> may exchange messages M1 and M2 for device discovery (DD) and service discovery (SD) operations, with each other.

0030If a connection between the WFD source device <b>400</b> and the WFD sink device <b>420</b> is established through the operation at step <b>401</b>, the WFD source device <b>400</b> may send an M3 request (Display_Enhanced_Apps_Support) message for requesting support of the running of an enhanced display application, to the WFD sink device <b>420</b> at step <b>403</b>. In response to the M3 request, the WFD sink device <b>420</b> may send an M3 response (Support) message including information indicating whether the running of the enhanced display application is supported, to the WFD source device <b>400</b> at step <b>405</b>.

0031If the WFD sink device <b>420</b> supports the running of the enhanced display application, the WFD source device <b>400</b> may send an M4 request (SET WFD Display_Enhanced_Apps_Support) message to the WFD sink device <b>420</b> at step <b>407</b>. In response, the WFD sink device <b>420</b> may send a response message to the M4 request message to the WFD source device <b>400</b> at step <b>409</b>.

0032During the exchange of capability information, the WFD source device <b>400</b> understands that the WFD sink device <b>420</b> has support for the enhanced display applications. In one embodiment, capability of the WFD sink device <b>420</b> to run enhanced display applications is indicated using “Display_Enhanced_Apps_Support” parameter exchanged between the WFD source device <b>400</b> and the WFD sink device <b>420</b> during the exchange of capability information. For example, a value of ‘1’ indicates that the feature of running an enhanced display application is supported by the WFD sink device <b>420</b> while value ‘0’ indicates that the feature is not supported by the WFD sink device <b>420</b>. Another parameter “Max_No Of_Display_Enhanced_Apps” is used to indicate maximum number of display enhanced apps that a sink device supports running parallelly.

0033In one embodiment, as shown at step <b>411</b>, there is a regular display session ongoing between the WFD source device <b>400</b> and the WFD sink device <b>420</b>. At the WFD source device <b>400</b>, the user starts an enhanced display application at step <b>413</b>. Then, at step <b>415</b>, a RTSP PAUSE (M5, M9 request/response (pause)) message is sent from the WFD source device <b>400</b> to the WFD sink device <b>420</b> and the regular display session is paused. Further, at step <b>417</b>, an application session is started between the WFD source device <b>400</b> and the WFD sink device <b>420</b> which also includes application description format (ADF) information. At step <b>419</b>, a sink formatted according to the ADF information may be displayed.

0034In case WFD source device <b>400</b> sends a request with new ADF information to the

0035WFD sink device <b>420</b>, the ongoing application session on the WFD session is updated accordingly at step <b>421</b>. Once the ongoing application session between the WFD source device <b>400</b> and the WFD sink device <b>420</b> terminates at step <b>423</b>, the earlier WFD session which was paused, is resumed between the WFD source device <b>400</b> and the WFD sink device <b>420</b> at step <b>425</b>.

0036In one embodiment, the present invention also requires a standard format for description of an enhanced display application. Any application which is enhanced for display use cases as described above is called as enhanced display application. The application description is provided in the form of an XML file. An exemplary application description is given in Table 1.

0000<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="left" /><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE 1</entry></row><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry></entry></row><row><entry><xsd:schemaxmlns:xsd=“http://www.w3.org/2001/XMLSchema”</entry></row><row><entry>targetNamespace=“http://www.wi-fi.org/xmlschema/</entry></row><row><entry>WfaDisplayServiceSchema”</entry></row><row><entry>xmlns:tns=“http://www.wi-fi.org/xmlschema/WfaDisplayServiceSchema”</entry></row><row><entry> elementFormDefault=“qualified”></entry></row><row><entry> </entry></row><row><entry><xsd:simpleTypename=“appType”></entry></row><row><entry><xsd:restrictionbase=“xsd:string”></entry></row><row><entry><xsd:enumerationvalue=“enhancedData”/></entry></row><row><entry><xsd:enumerationvalue=“excludedData”/></entry></row><row><entry></xsd:restriction></entry></row><row><entry> </xsd:simpleType></entry></row><row><entry> </entry></row><row><entry><xsd:simpleTypename=“name”></entry></row><row><entry><xsd:restrictionbase=“xsd:string”/></entry></row><row><entry> </xsd:simpleType></entry></row><row><entry> </entry></row><row><entry><xsd:complexTypename=“applicationInfo”></entry></row><row><entry><xsd:sequence></entry></row><row><entry><xsd:elementname=“name”type=“xsd:string”></xsd:element></entry></row><row><entry><xsd:elementname=“type”type=“tns:appType”></xsd:element></entry></row><row><entry></xsd:sequence></entry></row><row><entry> </xsd:complexType></entry></row><row><entry></xsd:schema></entry></row><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0037The schema for individual applications “AppData” is defined as per the application schema. The App schema includes but not limited to, rules on type of data, layout, UI, data etc.

0038The schema for sink configuration “SinkData” is defined as per the sink schema defined by the sink devices. Sink schema includes but not limited to, rules on sink layout, size, orientation etc.

0039In one embodiment, the XML file contains information about one or more applications. This is required if the WFD sink device supports running multiple enhanced display applications at the same time. An example XML file which has information related to two applications is shown in Table 2.

0000<tables id="TABLE-US-00002" num="00002"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="offset" colwidth="42pt" align="left" /><colspec colname="1" colwidth="175pt" align="left" /><thead><row><entry /><entry namest="offset" nameend="1" rowsep="1">TABLE 2</entry></row><row><entry /><entry namest="offset" nameend="1" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry><xml></entry></row><row><entry /><entry>  <application1></entry></row><row><entry /><entry>    <name>VCard </name></entry></row><row><entry /><entry>    <type> Enhanced Data </type></entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="42pt" align="left" /><colspec colname="1" colwidth="84pt" align="left" /><colspec colname="2" colwidth="91pt" align="left" /><tbody valign="top"><row><entry /><entry>    <appdata></entry><entry></appdata></entry></row><row><entry /><entry>    <sinkdata></entry><entry></sinkdata></entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="offset" colwidth="42pt" align="left" /><colspec colname="1" colwidth="175pt" align="left" /><tbody valign="top"><row><entry /><entry>  </application1></entry></row><row><entry /><entry>  <application2></entry></row><row><entry /><entry>    <name>Apps Screen </name></entry></row><row><entry /><entry>    <type> Excluded Data </type></entry></row><row><entry /><entry>  </application2></entry></row><row><entry /><entry></xml></entry></row><row><entry /><entry namest="offset" nameend="1" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0040The system and method of the present invention is configured to provide support to following types of applications:

0041a) Excluded Data Application—Applications of this type wants to display data on larger screen which is excluded from displaying on the mobile screen because of size limitations.

0042b) Enhanced Data Application—Applications of this type wants to display data on larger screen which is enhanced data from the data that is displayed on the mobile device.

0043c) Original/primary Data Application—Applications of this type wants to display same primary data (which displays on the source device) on a larger screen of the sink device.

0044In an embodiment of the present invention where the WFD source device has split screen functionality inbuilt and the hardware supports running two applications in the foreground, such source device has an option to still display two applications in the foreground but one on the source device itself and one on the sink device which is associated with the source device. Because of which the screen usage of each application increases and provides good user experience.

0045Virtual Screen Mirroring

0046In one embodiment of Virtual screen mirroring, the WFD source device has multiple screens in which one of the screens is the main screen and other screens are virtual screens. Instead of mirroring the main screen, the WFD source device mirrors one of the virtual screens with the WFD sink device. This enables the user to maintain the privacy of the source device.

0047The source device maintains two or more screen buffers depending on the use case.

0048One screen buffer is dedicated to the main screen. For all the other virtual screens, the WFD source device has either a single screen buffer or individual screen buffer for each of the sink screens.

0049Whenever a WFD session is started between the WFD source device and the WFD sink device, the WFD source device can choose to start the session with the screen buffer of the main screen or screen buffer of the other virtual screens. The information about which buffer is being used by the source device may be informed to the sink device using a new parameter “Screen_Buffer_Used” exchanged during capability negotiation phase. The source device is able to dynamically change the buffer it uses during an ongoing WFD session. By informing the sink device about the buffer change, the sink device will be able to clear the data from its buffer and start receiving data from the newly selected buffer. This will avoid the delay in the WFD session update.

0050<figref idref="DRAWINGS">FIG. 5</figref> illustrates a flow diagram of method for providing enhanced data to at least one sink device according to one embodiment of the present invention. A connection is established between a source device and the at least one sink device at step <b>501</b>. Capability information are exchanged between the source device and the at least one sink device at step <b>502</b>. The capability information includes a request and response messages in respect of the sink device providing support to the enhanced display application. An enhanced display session is established between the source device and the at least one sink device at step <b>503</b>. Session data with application description format (ADF) information is sent to the at least one sink device for the enhanced display session at step <b>504</b>. The data is displayed on the at least one sink device, the data having been formatted according to the ADF information at step <b>505</b>.

0051Although the invention of the method and system has been described in connection with the embodiments of the present invention illustrated in the accompanying drawings, it is not limited thereto. It will be apparent to those skilled in the art that various substitutions, modifications and changes may be made thereto without departing from the scope and spirit of the invention.