Interacting with software applications displayed in a web page

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

INTERACTING WITH SOFTWARE APPLICATIONS DISPLAYED IN A WEB PAGE

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to co-pending U.S. patent application number 09/086,898, filed May 29, 1998; co-pending U.S. patent application number 08/855,977, filed May 14,1997; co-pending U.S. patent application number 09/247,220, filed February 10, 1999; co-pending U.S. patent application number 10/068,461, filed February 6, 2002; and co-pending U.S. patent application number 10/288,821, filed November 6, 2002, the entirety of which are incorporated herein by reference.

TECHNICAL FIELD

[0002] The present invention relates to the display of information in a communications network and more specifically to displaying the output of executing application programs in a web page.

BACKGROUND

[0003] Businesses in today’s fast-paced global marketplace strive to improve their productivity and profitability by providing their employees with access to business-critical applications and data at fixed locations within the workplace as well as at mobile locations. The popularity of web-based computing, combined with the need to expedite information access for mobile users, has spurred adoption of enterprise portals. Enterprise portals are company web sites that aggregate, personalize and serve applications, data and content to users, while offering management tools for organizing and using information more efficiently. In some companies, portals have replaced traditional desktop software with browser-based access to a virtual workplace that is easy to use, convenient and ubiquitous. Companies that implement portals also benefit from a fast return on investment due to increased worker productivity and greater efficiency in their information technology infrastructure.

[0004] True desktop software replacement requires that a portal offer a full complement of information resources. Business applications are, arguably, the most vital information resource that workers need to access. Business applications also typically represent a major investment,

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-2and often a competitive advantage, which must not be lost when moving to web-based systems. From the standpoint of productivity, application access via the portal is needed so that users are not forced to switch back and forth between the browser and the desktop to do their work. With a split browser/desktop system, it is more difficult to locate and coordinate material from various sources. Users are also typically tied to the desktop device because it provides key applications that may not be accessible via the browser.

[0005] These business drivers provide compelling motivation to include existing and upcoming applications in portal implementations. However, few applications have been developed specifically for web-based delivery and those that have often provide reduced functionality as compared to their equivalent desktop applications. Although it is possible to use existing applications in a portal by re-engineering them for web publication using HTML, scripting, Java™ and other proprietary means, this approach is time-consuming and expensive and may delay portal implementation. Likewise, such implementations may experience reduced functionality and/or may not be feasible because the “download and run” model of application execution is too resource-intensive.

[0006] Accordingly, methods and systems are desired that enable efficient deployment of legacy applications in enterprise portals without undertaking expensive development efforts that may be marginally effective and which dilute the return on investment of the portal.

SUMMARY OF THE INVENTION

[0007] The present invention overcomes these shortcomings by applying web-enablement technology to legacy applications so that these applications remain in their original form, with the same user interface and full functionality to which their users are already accustomed, while allowing such viewers to interact with the applications via application-output windows displayed within a web page.

[0008] Tn one embodiment, the invention provides a method of displaying application-output data within one or more application-output windows positioned within a web browser window. The application-output data can be generated by a web enabled application program and/or a non-web enabled application program that is unmodified (e.g., so called “legacy applications”). The application programs that provide the source of the application-output data can reside on different application servers and the application-output data from these different application

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-3servers is displayed within one or more application-output windows in the same web browser window. In one aspect, the application-output windows are child windows of the web browser window. In one embodiment, the application-output windows are relocatable beyond the boundaries of the web browser window. In another aspect, window attribute information associated with the application-output windows is received via a first virtual channel and application-output data (e.g., graphical data) is received via a second virtual channel. In yet another aspect, the window attribute information of the application-output windows displayed within the web browser window is modifiable independently of the web browser that formed the web browser window. The invention displays the application-output data in the applicationoutput window in accordance with the window attribute information.

[0009] In one embodiment, the invention provides a client agent that monitors and responds to events associated with the application-output windows, such as detecting a resize event input by a viewer of the web browser window and in response resizing an affected application-output window. The client agent performs these functions independently of the web browser. In one embodiment, the client agent is executed/instantiated in response to an application object (e.g., an ActiveX control) embedded in the web page displayed in the web browser window. In this embodiment, the client agent uses the window attribute information of the application-output windows received via the second virtual channel to modify at least one property of the application object, where the modified property triggers a corresponding change in the associated application-output windows. In a further aspect, the client agent establishes the first and second virtual channels independently of the web browser.

[0010] In this manner, the invention avoids expensive development and user training costs, and provides access to the full functionality of the original legacy application at mobile locations that have web access.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The invention is pointed out with particularity in the appended claims. The advantages of this invention described above, and further advantages, may be better understood by reference to the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1 schematically illustrates a client device, a web server, and a server farm connected via a data communications network, where a client agent on the client device and one or more

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-4server agents in the server farm operate in accordance with an embodiment of the invention;

FIG. 2 provides a high-level flow diagram illustrating steps performed by the client agent and web server in accordance with an embodiment of the invention;

FIG. 3 is an illustrative screen representation of the client desktop, where the applicationoutput windows displayed within the web browser window of the client desktop are formed in accordance with an embodiment of the invention;

FIG. 4 is a screen representation of the application-output windows of FIG. 3, illustrating that application-output windows can be moved and/or resized within the web browser window in accordance with an embodiment of the invention;

FIG. 5 is a screen representation of the application-output windows of FIG. 3, illustrating that the application-output windows can be displayed within and/or beyond the boundaries of the web browser window and can exhibit varying z-orders, in accordance with an embodiment of the invention; and

FIG. 6 schematically illustrates the processes of the client agent and the type of data that is transferred between the client agent and the server agents, in accordance with an embodiment of the invention.

DETAILED DESCRIPTION

[0012] Portals and other web-based implementations capable of displaying application-output data to remote users are, preferably, implemented using a server-based computing model. Server-based computing is analogous to enterprise portals, which improve user productivity through single-point access to information resources (e.g., application programs), in that serverbased computing enhances the overall efficiency of the portal via single-point application administration.

[0013] In server-based computing models, application processing, administration, support and deployment are typically based on one or more central servers, which may be geographically distant from a user’s display device. Remote users interact with particular applications hosted on these application servers by sending keystrokes, mouse movements, and other input/output actions to the application servers via data communication networks (e.g., LAN, MAN, WAN, Internet, Intranet, etc.) and receive screen/window updates, files and other data therefrom. Because much, if not all, of the application processing takes place on the server, the user’s

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-5display device operates essentially as a thin client and thus requires few resources (e.g., processing power, nonvolatile memory, volatile memory, etc.) to display the application-output data of what may be a compute-intensive application. Further, by reducing the overall quantity of data that travels across the network, significant improvements can be realized in application performance and security.

[0014] Coupling this server-based computing model with web-based implementations of application programs (e.g., portals) enables users to access business critical applications on virtually any device connected to the web, including home computers, laptop computers, computer workstations, wireless and handheld communication devices, and information appliances running on a wide array of platforms. Device and platform flexibility allows mobile workers to move seamlessly from one device to another and receive a consistent, personalized information set, which includes access to the full functionality of business-critical, legacy applications.

[0015] In brief overview, a user of a device connected to the web requests access to one or more application programs from a web server. After authenticating the user’s credentials, the web server accesses user-specific and application-specific parameters from a memory coupled to the web server. The web server subsequently communicates these parameters to one or more application execution servers hosting the requested application programs, and software processes operating on the application execution servers execute and initialize the requested application programs using the communicated parameters. In this manner, each instance of the application programs is personalized for a particular requesting user. The particular network addresses of the application execution servers hosting these personalized application programs are then forwarded to the user’s device, which establishes a communications link and client-server session therewith.

[0016] Commands, events, graphical data, and window attribute information associated with ₍the executing application programs are communicated between, the user device and the application execution servers during the client-server session to ensure that the applicationoutput data is displayed seamlessly on the desktop of the user device. Seamless display of the application-output data refers to the presentation of the data on the user desktop in a manner that is consistent with how locally-executing applications are presented and manipulated in the local desktop of the user device. In other words, a user views and interacts with the application-output data generated by the remote application programs as if the application programs were being executed locally.

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-6[0017] In one embodiment, the output of the application programs is displayed in one or more application-output windows positioned within a web page displayed by a web browser of the user’s device. In a further embodiment, the attributes of the application-output windows can be modified so that the application-output windows are moveable and resizeable within the boundaries of the web page. In another embodiment, the application-output windows initially appear within the boundaries of the web page and are subsequently moveable so that they are positioned outside the boundaries of the web page and thus give the appearance that the application-output windows correspond to locally-executing applications rather than to remotelyexecuting applications. In yet another embodiment, the application-output windows initially appear outside the boundaries of the web page and thus also appear to correspond to locallyexecuting applications. In one embodiment, the application output displayed in the applicationoutput windows and the attributes of the application-output windows themselves are communicated and manipulated by software processes on the user’s device and on the application execution servers, without involvement of the web server or web browser that initially provided access to the application programs.

[0018] In more detail and with reference to FIG. 1, a server-based computing architecture 100, capable of providing remote users with web-access to the full functionality of web and legacy applications (e.g., unmodified application programs that are not designed for web-based delivery), includes a client device 110 (e.g., any digital data processing device), a web server 112, one or more application execution servers 114 that are either standalone or clustered within a server farm 116 and winch are preferably protected by a firewall 118, and a data communications network 120 (e.g.,'Internet, Intranet, etc.) that provides the necessary connectivity to enable each of these elements to communicate with each other.

[0019] In operation and also with reference to FIG. 2, a user of the client device 110 directs a browser 122 executing on the client device 110 to submit a request for access to particular web page content 124 accessible via the web server 112 (step 210). In one embodiment, the user enters a universal resource locator (“URL”) address into the browser 122. The URL is associated with the web page content 124 hosted by the web server 112 and the browser 122 responds by transmitting the request for access to the appropriate URL address. The web server 112 receives the request for access, which typically includes user credential information (e.g., user ID, password, group/project membership identifier, etc.), and authenticates the user to the server

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-7farm 116 or to the individual servers 114 that provide at least some of the web page content 124 (step 212).

[0020] The web server 112 authenticates the user by accessing an authentication process that compares the credentials entered by the user with previously-assigned credentials. In. one embodiment, the authentication process and database of previously-assigned credentials are stored and maintained on the web server 112. In other embodiments, the previously-assigned credentials can be stored in the server farm 116, on individual application execution servers 114, and/or oil an administrative server (not shown) that is coupled to the web server 112 via the Internet or other data communication network.

[0021] In the scenario where the web page content 124 corresponds to an enterprise portal, which provides access to an application set 126 (e.g., the set of application programs that have been personalized for the user by a portal administrator), the web server 112 accesses one or more application objects 128 (e.g., COM-compliant Java objects, ActiveX objects, HTML tags, etc.) that call web server-side scripts to authenticate the user (step 212) and/or to obtain the application set 126 information associated with the portal and user from the server farm 116 (step 214). The application objects 128 also include properties that are associated with the user and/or the particular applications 130 in the application set 126 that are provided via the portal. The user properties include, for example, group/project information that identifies the particular applications 130 and data that the user needs to access in order to allow the user to collaborate with other members of the group/project. The application properties include, for example, the user’s preferences for each of the applications 130 in the application set 126.

[0022] The scripts called by the application objects 128 establish a network session between the web server 112 and the server farm 116 via, for example, a central administrative process (not shown), which monitors and controls each server 114 in the server farm 116. The administrative process selects one or more servers, which host the application programs 130 in the application set 126 specified by the application objects 128, based, for example, on a server and/or network performance basis. The desired application set 126 can be provided entirely by a single server 114 by selecting/allocating each application 130 in the application set 126 from a plurality of applications 130,132 hosted on the server 114. Alternatively, the application set 126’ can be provided by a plurality of servers 114 with each of the plurality of servers 114 hosting at least one of the application programs in the application set 126’. A more detailed description of

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-8server farms and their administration/operation can be found in International Patent Application No. PCT/US01/14314, which is incorporated herein by reference in its entirety.

[0023] The administrative process launches one or more server agents 134 on the selected/allocated servers 114 in response to the scripts called by the application objects 128. Server agents 134 are software processes that execute, initialize, and interact with each of the application programs 130 in the application set 126 in accordance with the properties specified by the application objects 128. In one embodiment, there is a server agent 134 for each application program 130 in the application set 126. In other embodiments, there is a single server agent 134 for the application set 130, to the extent that all of the application programs 130 are hosted on the same server 114. In yet another embodiment, there is a single server agent 134 for each server 114. The server agents 134 then provide the output of the application programs 130 in the application set 126 as well as any other information relating to the application set 126 to the web server 112, which subsequently formats the application set information into the web page content 124 (step 216). The web page content 124 can include application icons corresponding to one or more of the application programs 130 in the application set 126 as well as application-output data from one or more of the application programs 130. In one embodiment, the application-output data provided by the application programs 130 corresponds to graphical data that is formatted to fit into a window, which exhibits attributes (e.g., window position on the web page, size, style, z-order, etc.) as initially specified by the properties of the application objects 128.

[0024] In one illustrative embodiment and with reference to FIG. 3, the browser 122 receives and displays the web page content 124 within a browser window 320, which includes many possible graphical user interface (“GUI”) elements (e.g., menu 312, local window 314, etc.) that form the client desktop 310 displayed on a display device coupled to the client device 110 (step 218). In this particular embodiment, the web page content 124 is displayed within a web page 322 displayed by the browser 320 and includes one or more application icons 324 and/or one or more application-output windows 326, which are associated with the application set 126. In one embodiment, one or more of the application objects 128 also form part of the web page content 124 of the web page 322 and can therefore set the initial attributes (size, z-order, position) of the application-output windows 326. The initial orientation, size, position, and z-order of each of the application-output windows 326 displayed on the web page 322 can be modified, as described

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-9below, so that the application-output windows 326 exhibit different orientations, sizes, positions, and z-orders relative to the web page 322 and/or relative to the client desktop 310.

[0025] The application objects 128 can be any data constructs which indicate to the browser 122 displaying the web page content 124 that an application-output window 326 should be displayed at a particular location in the web page 322. The application objects 128 may include additional information, such as the height, width, border style, background color or pattern in the application-output window 326, along with indicia of which applications 130 may be displayed in the window 326, how often the output display should be updated, or any other additional information that is useful to enhance the display of the application output.

[0026] In one illustrative embodiment, the application objects 128 are window tags that are embedded in an HTML file, examples of such tags are delineated below.

ActiveX tag <object classid-‘clsid:238f6f83-b8b4-llcf-8771-00a024541ee3” data-Vica/direct.ica” CODEBASE=“/cab/wfica.cab” widthM-36 height=295>

<param name=“Start” value-^cAuto”>

<param name-‘Border” value^On’^ </object>

Netscape Plugin tag <embed src-http://www.citrix.com/ica/direct.ica” pluginspage=^:“http://www.citrix.com/plugin.html” height-295 width=436 Start=Auto Border=On>

<embed>

CA 02475366 2004-08-05

WO 03/067568

PCT/VS03/03647.

-10JAV A tag <applet code=JICA.class width=436 height=295>

<param name=Address <param name=InitialProgram <param name^Start <param name=Border value=“128.4.1.64”>

value=Microsoft Word 7.0>

value=Auto>

value=On>

</applet>

[0027] In each case above, the tag indicates that an application-output window 326 having a height of 295 pixels and a width of 436 pixels should be drawn to receive output data from the application program 130. Each tag also specifies that the application program 130 should automatically start execution and that the application-output window 326 in which the application output is displayed should be drawn with a border. The ActiveX and Netscape Plugin tags have the properties of the remote application 130 specified in the file “direct.ica” located in the directory “/ica.” The JAVA tag specifies the properties of the remote application 130 directly. In the example above, the address of the server 114 hosting the application program 130 is specified as well as the name of the application program 130 to be executed.

[0028] In one embodiment, the application program 130 executes substantially at the same time as the display of the web page 322., In another embodiment, the application program 130 executes when instructed to do so by the server agent 114, as part of providing web page content 124 to the web server 112. In yet another embodiment, the application program executes in response to a signal, such as a user-specified input (e.g., selecting an application icon 324 on the web page 322. Once execution of the application program 130 is commenced, the browser 122 instantiates a client agent 136 on the client device 110 (step 220). Alternatively, the client agent 136 is instantiated substantially at the same time as the display of the web page 322 or in response to user-specified inputs.

[0029] The client agent 136 comprises one or more software processes, which execute on the client device 110 and which are configured to interact with the server agent 134, browser 122, application-output window 326, and/or web server 112. In one embodiment, the client agent 136 is spawned as a child process of the browser 122. In other embodiments, the client agent 136 is a

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-11peer process of the browser 122 or a dynamically linked library associated with the browser 122. In one embodiment, a client agent 136 is instantiated for each application-output window 326 ' displayed in the web page 322. In another embodiment, a single client agent 136 is instantiated for one or more application-output windows 326 associated with a particular one of the application programs 130 in the application set 126. In yet another embodiment, a single client agent 136 is instantiated for each server agent 134, which contributed to the web page content 124. In yet another embodiment, a single client agent 136 is instantiated for the entire application set 126.

[0030] The browser 122 passes the properties of the application objects 128 relating to particular application programs 130 in the application set 126 to the client agent 136 associated with those same application programs 126. Additionally, the browser 122 may pass a handle for an application-output window 326 to the client agent 136 or the client agent 136 may query the browser 122 to retrieve the handle for the application-output window 326. Application properties, which are not specified by either the browser 122 or the application objects 128, may be set to default values. The client agent 136 may also have certain property defaults hard-coded, or the client agent 136 may access a file which contains property defaults.

[0031] The client agent 136 uses the name of the application program 130 and the address of the application execution server 114, which are both provided as part of the properties of the application objects 128, to establish a communications link and initiate a client-server session with the server agent 134 associated with the server 114 and application program 130 (step 222). The client agent 136 passes some or all of the properties of the application objects 128 to the server agent 134 along with any necessary default values. Alternatively, the server agent 134 may have already received some or all of the properties of the application objects 128 from the web server 112 prior to contributing to the web page content 124, which was subsequently displayed in the web page 322. If a particular property is not passed to the server agent 134, the server agent 134 may request it from the client agent 136 if it is a necessary property to which it has no default value (e.g., user ID) or the server agent 134 may provide its own default value for the property (e.g., execution priority).

[0032] The server agent 134 uses the properties received from the client agent 136 to authenticate the client agent 136 and to execute the desired application program 130 if it has not previously been started. Once the application program 130 is executing and the client agent 136 has been authenticated, the application program 130 communicates through the server agent 130

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

- 12directly with the client agent 136, without intervention of the browser 122 or web server 112. The client agent 136 receives output data from the application program 130 and displays the output data in the appropriate application-output window 326 in the web page 322. The client agent 136 also detects input events, such as mouse clicks and keyboard inputs, associated with the application-output window 130 and forwards any such input events to the application program 130 via the server agent 134. This type of client-server session is repeated for each application program 130 in the application set 126 that is selected by the user and thus enables the user to interact with all of the resources in the application set 126 (step 224).

[0033] The data exchanged between the client agent 136 and server agent 134 during the clientserver session includes not only input events and the graphical output data of the application program 130, but also window attribute information (e.g., window position, z-order, size, style, color, etc.). The window attribute information of the application-output windows 326 is initially specified by the application objects 128 embedded in the web page 322. For example, the application objects 128 can include an ActiveX control, which specifies and controls tire window attributes of the application-output windows 326 during the client-server session. In one embodiment, the application-output windows 326 exhibit the same dimensions as the corresponding ActiveX controls.

[0034] The client agent 136 communicates the initial window attributes of the local application-output windows 130 to the server agent 134 along with information relating to the client desktop 310 (e.g., size, resolution, etc.). The server agent 134 responds by conforming the size of its server desktop to that of the client desktop 310 and by conforming the window attributes of local server windows to those of the application-output windows 326 on the client desktop 310. The application-output windows 326 on the client desktop 310 and the server windows on the server desktop thus exhibit the same window attributes and display the same graphical output data that is generated by the application 130. Note that the server desktop can correspond to either an offscreen surface contained within the server’s video memory or to an onscreen surface displayed on a display device coupled to the server 114.

[0035] The user of the client device 110 can move, resize, and/or alter the z-order or other initial window attributes of the application-output windows 326 during the client-server session, by entering an input event that is detected by the client agent 136 and then communicated to the server agent 134. The server agent 134 conforms its desktop and/or windows to be consistent, with the input event and then transmits updated graphical output data and window attribute

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-13I information, corresponding to the input event, to the client agent 136 with instructions to update the application-output windows 326 so that they match the windows on the server 114.

[0036] For example, if the user of the client device 110 resizes one of the application-output windows 326 from that originally specified by the application objects 128 (such as by clicking with the mouse and dragging the border of the application-output window 326 to the desired location/size), the client agent 136 detects the input event generated by the mouse action and communicates it to the server agent 134, which effects the same resize event in the on or offscreen surfaces of the server 114. The server agent 134 then sends repaint and resize command messages to the client agent 136 along with updated graphical output data and window attribute information. In response, the client agent 136 modifies the appropriate application object 128 affected by the resize event (e.g., the ActiveX control discussed above) so that the corresponding application-output window 326 is resized and the updated graphical output data is painted within the borders of the application-output window 326.

[003η The invention thus enables the window attributes of the application-output window 326 to be modified so that the application-output window 326 can be moved, resized, etc., within the boundaries of the browser window 320. With reference to FIG. 4 and by way of nonlimiting example, application-output window B’ 410 can be resized using the methodology described above to form application-output window B” 420, which overlaps (thus exhibiting a different zorder from) application-output window F 430. Alternatively, the application-output window 326 can be moved or resized to extend beyond or be entirely outside of the browser window 320. By way of nonlimiting example and with reference to FIG. 5, application-output window J 510 lies within the boundaries of the browser window 320, while application-output window K 520 extends beyond the boundaries of the browser window 320 and application-output window L 530 is entirely outside the browser window 320. Note that the application-output windows can exhibit varying z-orders with respect to other elements in the client desktop 310. For example, local window 540 exhibits a z-order between that of the browser window 320 and applicationoutput window L 530. In this embodiment, the client agent 136 instructs the operating system of the client device 110 to draw the desired application-output window 326 in response to command messages received from the server agent 134, without having to first modify the properties of the application objects 128 embedded in the web page 322, which initially established the window attributes of the application-output window 326.

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-14[0038] In one embodiment, each input event affecting the application-output window 326 is transferred to and processed by the server agent 114, which then instructs the client agent 136 to effect corresponding changes’in the application-output window 326. In another embodiment, one or more input event types (e.g., click and drag mouse actions directed at moving the applicationoutput window 326 to another grid location on the web page 322) are processed entirely by the client agent 136 and not reported to the server agent 134, where the graphical output data displayed within the application-output window 326 remains unchanged.

[0039] In more detail and with reference to FIG. 6, the client agent 136 comprises a monitor process 610, a command process 620, a message receiving process 630, and a message transmission process 640. In one embodiment, each process 610, 620, 630, 640 is a separately functioning code segment that operates independently of the other processes. For example, the message receiving process 630 and the command process 620 can be implemented as separate threads, which communicate with each other via a named pipe or shared memory. Use of a common data set allows the message receiving process 630 and the message transmission process 640 to be synchronized.

[0040] The message receiving process 630 receives graphical data, window attribute information, and commands from the server agent 134 via the communications link that provides the connectivity between the client agent 136 and server agent 134 during the client-server session. The communications link preferably includes a first virtual channel 660 and a second virtual channel 670. Command, event, and window attribute information is passed between the client agent 136 and the server agent 134 via the first virtual channel 660, while graphical data corresponding to the graphical contents of the application-output windows 326 is passed via the second virtual channel 670. The message receiving process 630 informs the command process 620 of the commands, window attributes, and graphical data received from the server agent 134 and the command process 620 further processes this data.

[0041] In one embodiment, the command process 620 processes the commands received from the server agent 134 by instructing the client operating system 650 to form and/or modify affected application-output windows 326 in accordance with the window attributes specified by the server agent 134. The command process 620 also instructs the client operating system 650 to display the graphical data provided by the server agent 134 in the appropriate application-output windows 326. Tn one embodiment, the command process 620 implements changes to the application-output windows 326 in the client desktop 310 by issuing GDI commands. In other

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-15embodiments, the command process 620 issues commands directly to an associated graphics subsystem or via graphics API commands.

[0042] The command process 620 also instructs the monitor process 610 to periodically monitor the client desktop 310 in order to detect changes affecting the application-output windows 326. In one embodiment, the monitor process 610 instructs the client operating system 650 to return information relating to the client desktop 310 at predetermined polling intervals. In other embodiments, the monitor process 610 monitors the message queue maintained by the client operating system 650 in order to detect changes affecting the application-output windows. The monitor process 610 communicates some or all of the detected desktop changes to the command process 620 for further processing.

[0043] In one embodiment, the command process 620 instructs the message transmission process 640 to transmit all ofthe changes detected by the monitor process 610 to the server agent 134 via the first virtual channel. In another embodiment, the command process 620 instructs the message transmission process 640 to transmit a subset of the detected changes, such as changes which only affect the graphical data and/or window attributes of the application-output windows 326. The server agent 134 receives the detected changes along with any commands from the command process 620 and any input events made by the user of the client device 110 that triggered the detected changes. The server agent 134 then modifies its local desktop to accommodate the detected changes and transmits associated commands, window attributes, and graphical data back to the client’s message receiving process 630. In this manner, the present invention ensures that desktop elements, such as the application-output windows 326, that are common in the client and server desktops remain in lock step.

[0044] The command process 620 ofthe client agent 136 ensures that analogous/common elements in the client and server desktops remain in lock step by maintaining a common window list. The common window list includes the window attribute information for each window in the client desktop 310 and for each corresponding window in the server desktop. In embodiments, in which a plurality of client agents are executing on the client device 110, the command process 620 of a single client agent 136 has primary responsibility for maintaining the common window list. If the single client agent 136 terminates, while other client agents remain in operation, the remaining client agents will elect another primary client agent to maintain the common window list.

CA 02475366 2004-08-05

WO 03/067568 PCT/US03/03647

-16[0045] The present invention also enhances the performance of the client-server session by preferably using the techniques described in co-owned, International Patent Publication No. WO 01/92973, which is incorporated herein by reference, to reduce the amount of graphical data transmitted between the client and server agents. In one embodiment, the invention encodes the data into relatively small representations that repeat within the protocol stream transmitted between the client agent 136 and the server agent 134. In this manner, the invention minimizes the size of each discrete data element that must be transmitted and increases the repeatability of the data within the protocol stream so that compression algorithms that operate more efficiently on repetitive encoded data can realize a greater degree of compression efficiency. The invention uses a number of techniques to realize this enhanced compression, including the following: scanning a command queue for later-issued commands which supercede earlier-issued commands; disk-caching techniques that improve compression of data that has been previously encountered during a client-server session; transmission of relative coordinates using quantized regions to avoid sending both endpoints and/or the angle of an associated strip (i.e., series of consecutive pixels exhibiting a common angle); and manipulation of off-screen surfaces to enhance performance during the client-server session. The invention also reduces the frequency of overscroll problems encountered when there is a performance mismatch between a fast server and a relatively slow network or client device.

[0046] Having described certain embodiments of the invention, it will now become apparent to one of skill in the art that other embodiments incorporating the concepts of the invention may be used. Therefore, the invention should not be limited to certain embodiments, but rather should be limited only by the spirit and scope of the following claims.