System and method for display sharing

BACKGROUND OF THE INVENTION

0001Computer display sharing techniques are becoming popular. In general, display sharing enables the display of one computer to also be displayed (shared) on one or more other computers. Computers may be communicatively coupled via a communication network, and certain computers may “share” another computer to which they are communicatively coupled. Display sharing aids collaboration among various users, for example, because the users can see a common display on their respective computers. Various software applications are available for performing display sharing, e.g., the Netmeeting product available from Microsoft, and the ASAP Express and ASAP Pro products available from Convoq.

0002Remote desktop sharing enables users to call a remote computer to access its shared desktop and applications. In this sense, remote need not be physically distant, but may be a separate computer that is communicatively coupled to the given computer. Thus, for example, a user may operate their office computer from home or vice-versa. Similarly, multiple users in a single conference room may each have computers (e.g., laptops) and may share a display to aid in their collaboration during a meeting. Accordingly, such “remote” computer may be referred to herein as a “guest” computer, which shares the display of a “host” computer. The “host” computer refers to the computer on which an application is executing for generating the information to be displayed on the host and guest computers. In some instances, a computer may act as a “guest” for all or part of one display-sharing session, and it may act as a “host” for all or part of another display-sharing session.

0003In certain display sharing applications, a host computer may run applications, receive user input, etc., and one or more guest computers may display the display of the host computer. In certain display sharing applications, the guest computer(s) may communicate information to the host computer to interact with an application executing on the host computer, etc., just as a local user may interact directly with the host computer.

0004In display sharing, it is desirable to maintain the displays of the host and guest computers synchronized. That is, when a display of the host computer changes, the guest computer's display should change accordingly to keep the displays synchronized. Of course, some amount of delay generally exists in this synchronization, as the updated information is communicated over a communication network.

0005A known technique for enabling such synchronization is provided by the Microsoft Windows Graphics Device Interface (GDI), which copies drawing operations to a mirror driver as well as to the actual graphics hardware of the host computer (for rendering the graphics on the host computer's display). Thus, the mirror driver sees all of the same drawing operations that a real graphics device sees via GDI. The mirror driver uses the information from these operations to construct a list of regions that have changed on the display. Accordingly, the mirror driver can use this information to synchronize the display(s) of any guest(s) that may be sharing the host computer's display.

0006However, GDI only handles two-dimensional (2D) graphics operations. Three-dimensional (3D) drawing applications, such as OpenGL, communicate graphics operations (e.g., drawing commands) via a different communication path than 2D graphics operations. For efficiency concerns, 3D drawing applications generally communicate directly to the graphics hardware, and do not communicate their operations through GDI, and thus the operations are not copied to the mirror driver. Therefore, difficulty arises in maintaining synchronized displays between a host and guest when a 3D drawing application is being used.

0007Other instances may arise in the prior art wherein a graphics call (e.g., of a 2D or 3D graphics application) to the graphics interface of the operating system (OS) of a host computer does not trigger synchronization of a guest computer's display. For instance, the host computer's OS may have a graphics interface that does not trigger synchronization of a guest computer's display for 2D and/or 3D graphics calls. Therefore, difficulty arises in maintaining synchronized displays between a host and a guest.

BRIEF DESCRIPTION OF THE DRAWINGS

0008<figref idref="DRAWINGS">FIG. 1</figref> shows an exemplary system according to an embodiment of the present invention;

0009<figref idref="DRAWINGS">FIG. 2</figref> shows an exemplary operational flow diagram of an embodiment of the present invention;

0010<figref idref="DRAWINGS">FIG. 3</figref> shows another exemplary system according to an embodiment of the present invention;

0011<figref idref="DRAWINGS">FIG. 4</figref> shows an exemplary operational flow diagram of the embodiment of <figref idref="DRAWINGS">FIG. 3</figref>;

0012<figref idref="DRAWINGS">FIG. 5</figref> shows another exemplary operational flow of an embodiment, such as that of <figref idref="DRAWINGS">FIG. 3</figref>;

0013<figref idref="DRAWINGS">FIG. 6</figref> shows an exemplary implementation of a system according to the embodiment of <figref idref="DRAWINGS">FIG. 3</figref>;

0014<figref idref="DRAWINGS">FIG. 7</figref> shows an exemplary operational flow diagram of the embodiment of <figref idref="DRAWINGS">FIG. 6</figref>; and

0015<figref idref="DRAWINGS">FIG. 8</figref> shows another exemplary system in which embodiments of the present invention may be employed.

DETAILED DESCRIPTION

0016Embodiments of the present invention provide systems and methods for supporting display sharing when an operating system's graphics interface does not support display sharing. Systems and methods are provided for supporting display sharing when the operating system's graphics interface on a host computer does not trigger synchronization of a guest computer's display responsive to a graphics operation (two-dimensional (2D) and/or three-dimensional (3D) operation) being performed via such graphics interface on the host computer. Certain embodiments of the present invention provide systems and methods for performing display sharing when using 3D graphics operations, e.g., displays of a host computer and one or more guest computers are synchronized for 3D graphics operations.

0017Turning to <figref idref="DRAWINGS">FIG. 1</figref>, an exemplary system <b>100</b> according to an embodiment of the present invention is shown. System <b>100</b> includes a host computer <b>10</b> (or “host”), and guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>(or “guests”) that are communicatively coupled to the host computer <b>10</b> via a communication network <b>12</b>. Each of the host computer <b>10</b> and guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>may be any processor-based device, including without limitation a personal computer (PC), laptop, server, workstation, or personal data assistant (PDA). Further, the host computer <b>10</b> and guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>need not be the same type of processor-based device. For instance, host computer <b>10</b> may be a server or PC, while guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>may be workstations or laptops, as examples. As described further herein, in certain embodiments guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>are lower-ended computers that need not locally have the application being executed for generating the displayed data, the processing-power of the host computer <b>10</b>, and/or the graphics capabilities of the host computer <b>10</b>. While two guests are shown in this example as communicatively coupled to host <b>10</b> for display sharing, this is merely illustrative as any number of guests may be so coupled. Communication network <b>12</b> may be any suitable communication network, such as a local area network (LAN), the Internet or other Wide Area Network (WAN), an intranet, a wireless network, telephony network, any combination of the aforementioned networks, and/or any other communication network now known or later developed within the networking arts which enables two or more computers to communicate with each other.

0018In this exemplary system <b>100</b>, guests <b>11</b><i>a </i>and <b>11</b><i>b </i>“share” the display of host <b>10</b>. In this manner, an application <b>101</b> may be executing on host <b>10</b> to generate output to the host's display <b>106</b>, and guests <b>11</b><i>a </i>and <b>11</b><i>b </i>share such display such that their respective displays are synchronized with the display <b>106</b> of host <b>10</b>. In some instances, host <b>10</b> may be referred to as a “sender” and guests <b>11</b><i>a </i>and <b>11</b><i>b </i>may be referred to as “receivers”. However, because in certain embodiments host <b>10</b> is capable of not only sending display data to guests <b>11</b><i>a </i>and <b>11</b><i>b</i>, but is also capable of receiving input from such guests <b>11</b><i>a </i>and <b>11</b><i>b</i>, it is referred to broadly herein as a “host” because it hosts (executes) the graphics application <b>101</b> that generates the display data being shared. Similarly, because in certain embodiments guests <b>11</b><i>a </i>and <b>11</b><i>b </i>are capable of not only receiving display data from host <b>10</b>, but are also capable of sending data to be input to host <b>10</b>, they are referred to broadly herein as “guests.” In some instances, a computer may be a “host” at one time and a “guest” at other times. In other instances, a computer may not comprise a graphics application and/or sufficient graphics processing capability to act as a host, but may nevertheless be a guest.

0019Host <b>10</b> includes one or more processors <b>109</b>. Host <b>10</b> further comprises a graphics application <b>101</b> stored to a data storage device, such as random access memory (RAM), hard disk, floppy disk, optical disc, etc., that is executable by processor <b>109</b>. Host computer <b>10</b> also comprises operating system <b>103</b>, which may be any operating system (OS) now known or later developed, such as WINDOWS, UNIX, LINUX, HP-UX, and MacOS. Host computer <b>10</b> also comprises graphics hardware <b>105</b> which is operable to process graphics operations to generate display data for displaying on display <b>106</b>. Graphics hardware <b>105</b> may include such hardware as one or more of the following: NVidia 980XGL, NVidia FX2000, a peripheral component interconnect (PCI), accelerated graphics port (AGP) or PCI-Express graphics adapter, as examples.

0020OS <b>103</b> comprises a graphics interface <b>104</b> (e.g., an Application Program Interface (API), such as OpenGL). Graphics interface <b>104</b> is operable to receive graphics calls (e.g., from graphics application <b>101</b>) for graphics hardware <b>105</b> as is well-known. Host computer <b>10</b> further comprises mirror driver <b>107</b>, sender <b>108</b>, and display synchronization logic <b>102</b>.

0021In operation, graphics application <b>101</b> is executable by host <b>10</b> to generate graphics operations, which are directed to graphics interface <b>104</b> of OS <b>103</b>. In this system, graphics interface <b>104</b> does not natively support display sharing. That is, graphics interface <b>104</b> does not trigger synchronization of the displays of guests <b>11</b><i>a </i>and <b>11</b><i>b </i>responsive to a graphics operation being performed via such graphics interface on the host computer <b>10</b>. As discussed further below, in certain embodiments the graphics interface <b>104</b> is a 3D graphics interface. However, in certain embodiments the graphics operations may be any type of graphics operations (2D or 3D), and graphics interface <b>104</b> does not support display sharing for such graphics operations. For instance, OpenGL is a graphics interface that may, in some instances, be used for performing 3D graphics operations, and such graphics interface does not support display sharing, as discussed further herein.

0022In this exemplary embodiment, display synchronization logic <b>102</b> monitors the graphics operations directed from graphics application <b>101</b> to graphics interface <b>104</b>, and based upon such monitoring detects when updated display data should be sent to the guests <b>11</b><i>a </i>and <b>11</b><i>b </i>to reflect the graphics operations. For instance, display synchronization logic <b>102</b> may copy the detected graphics operations to a mirror driver <b>107</b>, and the mirror driver <b>107</b> uses the information from these operations to construct a list of regions that have changed on the host's display <b>107</b>. Sender <b>108</b> uses this information compiled by the mirror driver <b>107</b> to synchronize the displays of guests <b>11</b><i>a </i>and <b>11</b><i>b</i>. That is, sender <b>108</b> sends the appropriate display data to guests <b>11</b><i>a </i>and <b>11</b><i>b </i>to synchronize their displays with display <b>106</b> of host <b>10</b> in accordance with the information compiled by mirror driver <b>107</b>. Mirror driver <b>107</b> and sender <b>108</b> are known for use in existing systems for 2D display sharing. In general, the sender <b>108</b> queries the mirror driver <b>107</b> for region changes (i.e., changes in regions of a display/window), which ensures that the display sharing system does not get backed up with changes when no guest has handled pending changes. That is, the sender <b>108</b> recognizes the regions that have changed on the host's display <b>106</b> and attends to sending display data to guests <b>11</b><i>a </i>and <b>11</b><i>b </i>for updating the corresponding regions on their respective displays.

0023Display synchronization logic <b>102</b> directly interacts with mirror driver <b>107</b> in certain embodiments, while in other embodiments display synchronization logic <b>102</b> indirectly triggers synchronization by mirror driver <b>107</b> (e.g., via a graphics interface of the OS that supports display sharing). For instance, as described further below, in certain embodiments OS <b>103</b> comprises not only the graphics interface <b>104</b> that does not support display sharing, but it further comprises a graphics interface (e.g., <figref idref="DRAWINGS">FIG. 3</figref>) that does support display sharing. For instance, graphics interface <b>104</b> may be OpenGL, and OS <b>103</b> may further include Windows GDI, which as mentioned above supports display sharing. Further, in certain embodiments, display synchronization logic <b>102</b> utilizes the graphics interface of the OS that supports display sharing for triggering an update of display data to be sent to guests <b>11</b><i>a </i>and <b>11</b><i>b </i>in order to reflect graphics operations performed on host <b>10</b> via graphics interface <b>104</b> (as indicated by the dashed-line arrow from display synchronization logic <b>102</b> through OS <b>103</b> to mirror driver <b>107</b> in <figref idref="DRAWINGS">FIG. 1</figref>). Thus, display synchronization logic <b>102</b> provides support for display sharing for graphics operations that are performed on host <b>10</b> via graphics interface <b>104</b> which does not support display sharing.

0024Turning to <figref idref="DRAWINGS">FIG. 2</figref>, an exemplary operational flow diagram of an embodiment of the present invention is shown. In operational block <b>201</b>, display synchronization logic <b>102</b> detects a graphics call directed from an application <b>101</b> executing on host computer <b>10</b> to an OS's graphics interface <b>104</b>, wherein such graphics interface <b>104</b> does not trigger synchronization of a guest computer's display. In operational block <b>202</b>, responsive to the display synchronization logic <b>102</b> detecting a graphics call to graphics interface <b>104</b>, display synchronization logic <b>102</b> triggers synchronization of a display of at least one guest computer <b>11</b><i>a</i>, <b>11</b><i>b </i>that is communicatively coupled to host computer <b>10</b> in order to reflect the effect of the graphics call on their respective displays. Display synchronization logic <b>102</b> may trigger such synchronization by directing a graphics call (e.g., transparent fill) to a graphics interface of the OS that does support display sharing. Exemplary techniques for triggering such synchronization are described further herein.

0025Turning to <figref idref="DRAWINGS">FIG. 3</figref>, an exemplary system <b>300</b> according to one embodiment of the present invention is shown. System <b>300</b> comprises a host computer <b>10</b><i>a </i>(or “host”), and guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>(or “guests”) that are communicatively coupled to the host computer <b>10</b><i>a </i>via communication network <b>12</b>. Host computer <b>10</b><i>a </i>is one exemplary embodiment of host computer <b>10</b> described above with <figref idref="DRAWINGS">FIG. 1</figref>.

0026In this exemplary system <b>200</b>, a 3D graphics application <b>101</b><i>a </i>is executing on host <b>10</b><i>a </i>to generate output to the host's display <b>106</b>, and guests <b>11</b><i>a </i>and <b>11</b><i>b </i>share such display such that their respective displays (not specifically shown) are synchronized with the display <b>106</b> of host <b>10</b><i>a</i>. It should be recognized that in certain implementations a display <b>106</b> may not be connected to host <b>10</b><i>a </i>or the display <b>106</b> may, in some way, be disabled. In such instances, the display data (performed by the graphics hardware <b>105</b>, such as a graphics card) is still generated for host <b>10</b><i>a </i>even though it may not be actually presented on a display screen at host <b>10</b><i>a</i>. Those of ordinary skill in the art will recognize that embodiments of the present invention are applicable for display sharing, even though the display data generated at host <b>10</b><i>a </i>need not be output to an actual display screen. Thus, when referring herein to synchronizing the host's display with guest displays herein, such host and/or guests need not have a physical display for outputting the display data, but rather such synchronizing encompasses synchronizing the display data generated by graphics hardware <b>105</b> of host <b>10</b>A with the guests even though such display data may not be actually output on a physical screen at one or more of the host and guests.

00273D graphics application <b>101</b><i>a </i>is stored to a data storage device, such as random access memory (RAM), hard disk, floppy disk, optical disk, etc., that is executable by processor <b>109</b>. Examples of such 3D graphics application <b>101</b><i>a </i>include OpenGL and Direct3D. Host computer <b>11</b><i>a </i>also includes operating system <b>103</b><i>a</i>, which may be any operating system (OS) now known or later developed, such as WINDOWS, UNIX, LINUX, HP-UX, and MacOS. OS <b>103</b><i>a </i>is one example of OS <b>103</b> of <figref idref="DRAWINGS">FIG. 1</figref>. In this example, OS <b>103</b><i>a </i>includes a 3D graphics interface <b>104</b><i>a </i>(e.g., API, such as OpenGL), and a 2D graphics interface <b>301</b> (e.g., an API, such as WINDOWS GDI). 3D graphics interface <b>104</b><i>a </i>is one example of graphics interface <b>104</b> of <figref idref="DRAWINGS">FIG. 1</figref>. Such 3D graphics interface <b>104</b><i>a </i>is operable to receive 3D graphics calls (e.g., from 3D graphics application <b>101</b>_A) for graphics hardware <b>105</b> as is well-known, and 2D graphics interface <b>301</b> is operable to receive 2D graphics calls (e.g., from a 2D graphics application) for graphics hardware <b>105</b> as is well-known. Host computer <b>10</b><i>a </i>further comprises mirror driver <b>107</b>, sender <b>108</b>, and display synchronization logic <b>102</b><i>a. </i>

0028OpenGL (Open Graphics Library) is the computer industry's standard API for defining 3D graphic images. Prior to OpenGL, any company developing a graphical application typically had to rewrite the graphics part of it for each OS platform and had to be cognizant of the graphics hardware as well. With the standardization of OpenGL, portability has been improved as an application can create the same effects in any OS using any OpenGL-adhering graphics adapter.

0029As those of ordinary skill in the art will appreciate, OpenGL specifies a set of “commands” or immediately executed functions. Each command directs a drawing action or causes special effects. A list of these commands can be created for repetitive effects. OpenGL is independent of the windowing characteristics of each OS, but provides special “glue” routines for each OS that enable OpenGL to work in that system's windowing environment. OpenGL comes with a large number of built-in capabilities that can be requested through the API, including as examples hidden surface removal, alpha blending (transparency), antialiasing, texture mapping, pixel operations, viewing and modeling transformations, and atmospheric effects (fog, smoke, and haze). While OpenGL is provided as a specific exemplary 3D graphics interface that may be used in a system in which an embodiment of the present invention is employed, embodiments of the present invention may likewise be employed with any other 3D graphics interface now known or later developed and thus is not limited in application to OpenGL.

00303D graphics application <b>101</b><i>a </i>is executable by host <b>10</b><i>a </i>to generate 3D graphics operations. As is well known, 3D graphics operations differ from 2D graphics operations. In general, 3D computer graphics model a 3D object and then generate display data for a view of the modeled 3D object. Thus, 3D graphics operations generated by 3D graphics application <b>101</b><i>a </i>include operations for generating display data for a view of a modeled 3D object. For example, an automobile may be modeled as a 3D object and, depending on the view of the automobile, different display data may be generated. For instance, a view of the front of the automobile may be displayed, such that the headlights and other features on the front of the automobile are displayed. As a further example, a user may interact with 3D graphics application <b>101</b><i>a </i>to rotate the automobile to view it from the rear such that the taillights and other features of the rear of the automobile are displayed. It becomes desirable to synchronize the displays of host <b>10</b><i>a </i>and guests <b>11</b><i>a </i>and <b>11</b><i>b </i>such that if a user rotates the automobile in this manner, all of their respective displays update to reflect the new display data. As described further herein, while display sharing systems exist for some 2D graphics, embodiments are provided for display sharing systems for sharing 3D graphics, especially a portable display sharing system that is capable of sharing 3D graphics without requiring modifications to the host's architecture (e.g., OS, graphics hardware, etc.).

0031Display synchronization logic <b>102</b><i>a </i>monitors 3D graphics operations directed from 3D graphics application <b>101</b><i>a </i>to 3D graphics interface <b>104</b><i>a</i>, and based upon such monitoring, detects when updated display data should be sent to the guests <b>11</b><i>a </i>and <b>11</b><i>b </i>to reflect 3D graphics operations. In one embodiment, the 2D communication path (e.g., the 2D graphics interface <b>301</b>) is used by display synchronization logic <b>102</b><i>a </i>for triggering an update of display data to be sent to guests <b>11</b><i>a </i>and <b>11</b><i>b </i>in order to reflect the effects of 3D graphics operations performed on host <b>10</b><i>a</i>. For instance, upon display synchronization logic <b>102</b><i>a </i>detecting a 3D graphics operation directed from 3D graphics application <b>101</b><i>a </i>to 3D graphics interface <b>104</b><i>a</i>, display synchronization logic <b>102</b><i>a </i>may generate a 2D drawing command, such as a transparent fill, that it directs to 2D graphics interface <b>301</b>. 2D graphics interface <b>301</b> copies the 2D drawing command to mirror driver <b>107</b>, in its traditional manner. Thus, the mirror driver <b>107</b> is informed of an update in the display data, and accordingly the sender <b>108</b> uses this information compiled by the mirror driver <b>107</b> to synchronize the displays of guests <b>11</b><i>a</i>, <b>11</b><i>b. </i>

0032The 3D graphics interface <b>104</b><i>a </i>processes the 3D graphics call and generates an updated display for a region of display <b>106</b>. The 2D drawing command generated by display synchronization logic <b>102</b><i>a </i>may be a transparent fill to the corresponding region of display <b>106</b> that was affected by the 3D drawing operation. Mirror driver <b>107</b> then causes sender <b>108</b> to send to the guests <b>11</b><i>a </i>and <b>11</b><i>b </i>data reflecting the changed region of display <b>106</b>. For instance, 2D data representing the modified region of display <b>106</b> may be sent to guests <b>11</b><i>a </i>and <b>11</b><i>b</i>. Such 2D data may, for instance, be a “snapshot” of the modified region of display <b>106</b>. Thus, even though 3D graphics processing generated the resulting 2D display data for the host display <b>106</b>, in certain embodiments the resulting 2D display data may be sent to guests <b>11</b><i>a </i>and <b>11</b><i>b. </i>

0033Thus, this exemplary embodiment uses the 2D graphics interface <b>301</b> for notifying the mirror driver <b>107</b> when a 3D graphics operation has been performed on host <b>10</b><i>a </i>such that the updated display data resulting from the 3D graphics operation is sent to guests <b>11</b><i>a</i>, <b>11</b><i>b</i>. As described further herein, this exemplary embodiment may be implemented without requiring modification to the 3D graphics application <b>101</b><i>a</i>, the existing OS graphics interfaces <b>104</b><i>a </i>and <b>301</b>, the graphics hardware <b>105</b>, the mirror driver <b>107</b>, or sender <b>108</b>. Thus, display synchronization logic <b>102</b><i>a </i>can be implemented (e.g., as software) and used for enabling display synchronization for 3D graphics operations without requiring modifications to the architecture of host <b>10</b><i>a </i>(or to guests <b>11</b><i>a</i>, <b>11</b><i>b</i>).

0034Further, in certain embodiments, the guests <b>11</b><i>a </i>and <b>11</b><i>b </i>need not have 3D processing capability. Instead, the 2D data for a region of the host's display <b>106</b> that is presented as a result of the 3D processing by the host is sent to the guests. For example, when a 3D “car” object is rotated by application <b>11</b><i>a </i>to generate a display of the car's rear instead of its front, the host <b>10</b><i>a </i>performs this 3D processing to generate a resulting 2D display, which is presented on the host's display <b>106</b>. The resulting 2D display data for presenting the car's rear may be sent to the guests <b>11</b><i>a </i>and <b>11</b><i>b</i>. In this sense, the 3D data need not be sent to the guests <b>11</b><i>a </i>and <b>11</b><i>b</i>, but instead the resulting 2D display data for all or a portion of the host's display <b>106</b> that has changed as a result of the 3D processing may be sent to the guests <b>11</b><i>a </i>and <b>11</b><i>b</i>. The guests need not even be aware that the display data was generated via 3D image processing by host <b>10</b><i>a </i>in certain embodiments. That is, the guests <b>11</b><i>a </i>and <b>11</b><i>b </i>need not be aware of how the image was rendered on the host <b>10</b><i>a </i>(e.g., via 3D processing).

0035Turning to <figref idref="DRAWINGS">FIG. 4</figref>, an exemplary operational flow diagram of the embodiment of <figref idref="DRAWINGS">FIG. 3</figref> is shown. In operational block <b>401</b>, display synchronization logic <b>102</b><i>a </i>detects a 3D graphics call directed from an application <b>101</b><i>a </i>executing on host computer <b>11</b><i>a </i>to the host computer's graphics hardware <b>105</b> (via the OS's graphics interface <b>104</b><i>a</i>). In operational block <b>402</b>, responsive to the display synchronization logic <b>102</b><i>a </i>detecting a 3D graphics call, display synchronization logic <b>102</b><i>a </i>triggers synchronization of a display of at least one guest computer <b>11</b><i>a</i>, <b>11</b><i>b </i>that is communicatively coupled to host computer <b>10</b><i>a </i>in order to reflect the 3D graphics call on their respective displays. Exemplary techniques for triggering such synchronization are described further herein.

0036<figref idref="DRAWINGS">FIG. 5</figref> shows another exemplary operational flow of an embodiment, such as that of <figref idref="DRAWINGS">FIG. 3</figref>. In operational block <b>501</b>, display synchronization logic <b>102</b><i>a </i>determines that a 3D graphics operation is communicated along a 3D graphics communication path of host computer <b>10</b><i>a</i>. For instance, in the example of <figref idref="DRAWINGS">FIG. 3</figref>, such 3D graphics communication path is from 3D graphics application <b>11</b><i>a </i>through 3D graphics interface <b>104</b><i>a </i>to graphics hardware <b>105</b>. In operational block <b>502</b>, display synchronization logic <b>102</b><i>a </i>uses a communication path for 2D graphics operations on the host computer <b>10</b>_Ato trigger notification to a guest computer <b>11</b><i>a</i>, <b>11</b><i>b </i>of a display change resulting from the 3D graphics operation. For instance, in the example of <figref idref="DRAWINGS">FIG. 3</figref>, such 2D graphics communication path includes 2D graphics interface <b>301</b>, which copies a 2D graphics operation for a region of the display affected by the 3D graphics operation to mirror driver <b>107</b>, where mirror driver <b>107</b> causes sender <b>108</b> to send the modified region to the guests <b>11</b><i>a </i>and <b>11</b><i>b. </i>

0037<figref idref="DRAWINGS">FIG. 6</figref> shows an exemplary implementation of a system according to the embodiment of <figref idref="DRAWINGS">FIG. 3</figref>. The system of <figref idref="DRAWINGS">FIG. 6</figref> comprises a host computer <b>10</b><i>b </i>(or “host”), and guest computers <b>11</b><i>a </i>and <b>11</b><i>b </i>(or “guests”) that are communicatively coupled to the host computer <b>10</b><i>b </i>via communication network <b>12</b>. Host computer <b>10</b><i>b </i>is one exemplary embodiment of host computer <b>10</b><i>a </i>described above with <figref idref="DRAWINGS">FIG. 3</figref>.

0038In this exemplary system of <figref idref="DRAWINGS">FIG. 6</figref>, an OpenGL application <b>101</b><i>b </i>is executing on host <b>11</b><i>b </i>to generate output to the host's display, and guests <b>11</b><i>a </i>and <b>11</b><i>b </i>share such display such that their respective displays are synchronized with the display of host <b>10</b><i>b</i>. Host computer <b>10</b><i>b </i>also comprises operating system <b>103</b><i>b</i>, which is one example of OS <b>103</b><i>a </i>of <figref idref="DRAWINGS">FIG. 3</figref>. In this example, OS <b>103</b><i>b </i>comprises GDI graphics interface <b>301</b>_a, which is one example of 2D graphics interface <b>301</b> of <figref idref="DRAWINGS">FIG. 3</figref>. OS <b>103</b><i>b </i>further comprises OpenGL graphics interface <b>104</b><i>b</i>, which is one example of 3D graphics interface <b>104</b><i>a </i>of <figref idref="DRAWINGS">FIG. 3</figref>. Such Windows OS <b>103</b><i>b </i>and the GDI and OpenGL graphics interfaces <b>301</b><i>a </i>and <b>104</b><i>b </i>are well known in the art. As mentioned above, OpenGL (Open Graphics Library) is the computer industry's standard API for defining 3D graphic images.

0039Host computer <b>10</b><i>a </i>further comprises mirror driver <b>107</b>, sender <b>108</b>, and display synchronization logic <b>102</b><i>b</i>. In this example, display synchronization logic <b>102</b><i>b</i>, which is one exemplary implementation of display synchronization logic <b>102</b><i>a </i>of <figref idref="DRAWINGS">FIG. 3</figref>, comprises an intercepting OpenGL library <b>601</b> and a sender notification library <b>602</b>. In this exemplary implementation, sender <b>108</b> communicates with mirror driver <b>107</b> via shared memory and a messaging mechanism provided by the Windows OS <b>103</b><i>b</i>. Sender <b>108</b> communicates with the sender notification library (shown as “hprwsenderntfy.dll”) <b>602</b> via a Windows messaging mechanism and a shared data segment in the library. This shared data segment is also used as a means of communication between the sender <b>108</b> and the intercepting library <b>601</b> (which may be referred to as the “OpenGL32.dll” library, as discussed further herein). The intercepting library <b>601</b> is used for communication between the OpenGL application <b>101</b><i>b </i>and the mirror driver <b>107</b>, via a graphics operation described further below.

0040Intercepting library <b>601</b> may be placed in the same directory as the OpenGL application <b>101</b><i>b</i>. The application <b>101</b><i>b </i>loads this intercepting library <b>601</b> upon startup, which in turn loads the Windows OpenGL32.dll library, shown as OpenGL graphics interface <b>104</b><i>b</i>. The intercepting library <b>601</b> then intercepts calls <b>60</b> that application <b>101</b><i>b </i>makes to OpenGL graphics interface <b>104</b><i>b</i>. Whenever the sender <b>108</b> is connected to a receiver (of one or more guests), the sender <b>108</b> sets a state variable in the shared data segment of the sender notification library <b>602</b> (“hprwsenderntfy.dll”) that the intercepting library <b>601</b> can check to determine whether a guest <b>11</b><i>a</i>, <b>11</b><i>b </i>is connected to the host <b>10</b>_B. When a graphics call is made into the OpenGL graphics interface <b>104</b><i>b </i>that changes the display and a guest is connected to the host <b>10</b><i>b</i>, the intercepting library <b>601</b> performs a 2D graphics operation (operation <b>6</b>) to the window the 3D command was sent to. Preferably, the 2D graphics operation is transparent, such as a transparent fill operation. The mirror driver <b>107</b> then sees that the display changed via the 2D graphics operation (because the GDI graphics interface <b>301</b>A copies the 2D operation to the mirror driver <b>107</b>), and can in turn update its changed region list. Accordingly, sender <b>108</b> then sends the updated region (e.g., having the modified 2D display resulting from the 3D graphics operation) to guests <b>11</b><i>a </i>and <b>11</b><i>b</i>. Operation of the exemplary implementation of <figref idref="DRAWINGS">FIG. 6</figref> is described further below with <figref idref="DRAWINGS">FIG. 7</figref>.

0041<figref idref="DRAWINGS">FIG. 7</figref> shows an exemplary operational flow diagram of the system of <figref idref="DRAWINGS">FIG. 6</figref>. As mentioned above, an intercepting library (“OpenGL32.dll library”) <b>601</b> may be placed into the directory where the OpenGL application <b>101</b><i>b </i>resides. As shown in block <b>701</b> of <figref idref="DRAWINGS">FIG. 7</figref>, when the OpenGL application <b>101</b><i>b </i>starts, it loads this intercepting library <b>601</b> instead of the library provided by Windows. This intercepting library <b>601</b> then loads the OpenGL library (or “graphics interface”) <b>104</b><i>b </i>provided by Windows <b>103</b><i>b</i>, in block <b>702</b>. Intercepting library <b>601</b> acts as an intermediary between the OpenGL application <b>101</b><i>b </i>and the Windows OpenGL library <b>104</b><i>b</i>, intercepting all calls into OpenGL graphics interface <b>104</b><i>b</i>, in block <b>703</b>. Thus, as OpenGL application <b>101</b><i>b </i>makes calls <b>60</b> (<figref idref="DRAWINGS">FIG. 6</figref>) directed to OpenGL graphics interface <b>104</b><i>b</i>, intercepting library <b>601</b> intercepts those calls.

0042In operational block <b>704</b>, the intercepting library <b>601</b> determines whether an intercepted call is to trigger display synchronization. That is, the intercepting library <b>601</b> determines whether an intercepted call is a call that generates a display change such that updated display data should be sent to guests <b>11</b><i>a</i>, <b>11</b><i>b </i>to maintain their displays synchronized with that of host <b>10</b><i>b</i>. If the call is not one that triggers display synchronization (i.e., does not result in a change in the display), operation advances to block <b>705</b> whereat the intercepting library <b>601</b> forwards the call to the Windows OpenGL graphics interface <b>104</b><i>b</i>, and then operation returns to block <b>703</b> to continue intercepting OpenGL calls. That is, in block <b>705</b> the intercepting library forwards <b>61</b> (<figref idref="DRAWINGS">FIG. 6</figref>) the intercepted call to the Windows OpenGL graphics interface <b>104</b><i>b</i>, which in turn sends <b>62</b> (<figref idref="DRAWINGS">FIG. 6</figref>) the proper command(s) to graphics device <b>105</b>.

0043When determined in block <b>704</b> that the intercepted OpenGL call is one that triggers display synchronization, such as glSwapBuffers( ), glFlush( ), or glFinish( ) as examples, the intercepting library <b>601</b> forwards <b>61</b> (<figref idref="DRAWINGS">FIG. 6</figref>) the intercepted call to the Windows OpenGL graphics interface <b>104</b><i>b </i>as usual in block <b>706</b>, and then proceeds to operational block <b>707</b>. In block <b>707</b>, the intercepting library <b>601</b> checks <b>63</b> (<figref idref="DRAWINGS">FIG. 6</figref>) with the sender notification library <b>602</b> (hprwsenderntfy.dll) to determine whether a guest is communicatively coupled to host <b>10</b><i>b</i>. If no guest is coupled to host <b>10</b><i>b </i>(i.e., display sharing is not being performed), operation returns to block <b>703</b> to continue intercepting OpenGL calls. If determined in block <b>707</b> that a guest is coupled to host <b>10</b><i>b </i>operation advances to block <b>708</b>.

0044In block <b>708</b> the intercepting library <b>601</b> determines whether the intercepted call was a glFlush( ) or glFinish( ) call directed to a front buffer or was a glSwapBuffers( ) call. OpenGL distinguishes off-screen and on-screen drawing with the use of front and back buffer terminology. Hence, this exemplary embodiment checks if a glFlush( ) or glFinish( ) was done on the front or back buffer. If such a command is requested for the front buffer, then a change is made on the display (and operation advances to block <b>709</b>), otherwise (if on the back buffer) the call is ignored as it did not change anything on the display. glSwapBuffers( ) indicates that off-screen graphics are being transferred on screen, which results in a change in the display and so operation advances to block <b>709</b>. If determined in block <b>708</b> that the intercepted call is neither a glFlush( ) or glFinish( ) call directed to a front buffer nor a glSwapBuffers( ) call, operation returns to block <b>703</b> to continue intercepting OpenGL calls. If determined in block <b>708</b> that the intercepted call was a glFlush( ) or glFinish( ) call directed to a front buffer or was a glSwapBuffers( ) call, operation advances to block <b>709</b>.

0045In block <b>709</b>, the intercepting library <b>601</b> performs <b>64</b> (<figref idref="DRAWINGS">FIG. 6</figref>) a transparent fill to the window that the intercepted 3D graphics command was sent to using GDI <b>301</b><i>a</i>. Accordingly, GDI <b>301</b><i>a </i>communicates <b>65</b> (<figref idref="DRAWINGS">FIG. 6</figref>) instructions to the graphics hardware <b>105</b> for the transparent fill. In block <b>710</b>, the intercepting library <b>601</b> calls glFinish( ) in the OpenGL graphics interface <b>104</b><i>b</i>. This helps reduce tearing that sometimes occurs when grabbing the frame buffer during a glSwapBuffers( ) call.

0046In block <b>711</b>, mirror driver <b>107</b> receives <b>66</b> (<figref idref="DRAWINGS">FIG. 6</figref>) notification of the 2D drawing fill via GDI <b>301</b><i>a </i>and updates its changed region list accordingly. In block <b>712</b>, when the sender <b>108</b> queries <b>67</b> (<figref idref="DRAWINGS">FIG. 6</figref>) the mirror driver <b>107</b> for the list of regions on the display that changed, the 3D window (the region of the display that was changed as a result of the 3D operation) is now part of that list. Operation returns to block <b>703</b> whereat the intercepting library continues intercepting OpenGL calls.

0047An exemplary process of a guest computer <b>11</b><i>a</i>, <b>11</b><i>b </i>connecting and the sender <b>108</b> setting the state in the sender notification library <b>602</b> is as follows. A guest <b>11</b><i>a</i>, <b>11</b><i>b </i>establishes a connection with the sender <b>108</b> (as indicated by line <b>69</b> of <figref idref="DRAWINGS">FIG. 6</figref>). The sender <b>108</b> then sets <b>68</b> (<figref idref="DRAWINGS">FIG. 6</figref>) a state variable in the shared data segment of the sender notification library <b>602</b> that indicates a guest is connected to host <b>10</b><i>b </i>for display sharing.

0048<figref idref="DRAWINGS">FIG. 8</figref> shows another exemplary system <b>800</b> in which embodiments of the present invention may be employed. System <b>800</b> illustrates an exemplary display sharing scenario in which a host <b>10</b><i>c </i>shares its display with a guest <b>11</b>. As described above, host <b>10</b><i>c </i>includes display synchronization logic <b>102</b> that captures <b>801</b> display data for updating the display of guest <b>11</b>. In certain embodiments, the captured display data may be compressed <b>802</b>, then communicated <b>803</b> over communication network <b>12</b> to guest <b>11</b> where the display data is decompressed <b>804</b> and displayed <b>805</b>. Further, in this exemplary embodiment, guest <b>11</b> may capture inputs, such as keyboard and mouse events <b>806</b>, which are communicated <b>807</b> over communication network <b>12</b> to host <b>10</b><i>c </i>where the inputs are applied to the application <b>101</b> executing locally on host <b>10</b><i>c. </i>

0049The exemplary embodiments described above provide techniques for synchronizing a guest's display with a host's display when the host's OS provides a graphics interface that does not support display sharing. The exemplary techniques provided herein enable high-efficiency because they enable changes that are made to a display to be tracked as they are made, rather than polling the full screen for detecting display changes. Further, only those regions that have changed need be sent for updating a guest's display, rather than repeatedly sending the entire content of the host's display, which minimizes network traffic and CPU utilization. Additionally, the exemplary techniques described above do not require changes to the graphics hardware, but are capable of working with all graphics hardware. The exemplary techniques described above do not require architectural changes to the host or guests. Rather, the solution may be added (e.g., as software) to the host's existing architecture without requiring any changes to the applications executing on the host, the host's OS, or the host's graphics hardware.

0050Some exemplary embodiments provide techniques for synchronizing a guest's display with a host's display when a 3D drawing operation has resulted in a change on a display, and the host only has 2D display sharing capability.