Presentation file conversion system for interactive collaboration

RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 60/259,327 filed Dec. 29, 2000. Additionally, this application is related to the following copending applications filed on the same day and assigned to the same entity as the present application, which are incorporated herein by reference: U.S. Ser. No. ______ entitled Graphical User Interface For An Interactive Collaboration System; and U.S. Ser. No. ______ entiled Computer Based Interactive Collaboration System Architecture.

FIELD OF THE INVENTION

[0002] The present invention relates generally to computer based educational and collaboration services. More particularly, the invention relates to a method and apparatus for providing a computer based interactive, collaborative, educational and meeting system, coupled with direct consumer marketing, which allows both the presenter and participant a high level of real time interactivity without downloading or installing any software on either the presenter or participant computer.

BACKGROUND OF THE INVENTION

[0003] Networked educational and meeting services are generally known. However, they are limited by the constraints of the Internet and the vagaries of participant computers. More specifically, current services suffer from a lack of standardization in presentation formats and the requirement that participants have data presentation format specific software (e.g. MS Word, Word Perfect, Excel, etc.) resident on the participant computer. The master teaching or presenter computer dictates the presentation format, which may not be compatible with the presentation software resident on the participant computer, making the Internet learning/teaching experience a cumbersome and impractical alternative to traditional classroom attendance and participation.

[0004] The present invention solves these problems by providing improvements in several key areas but namely in presenter-participant interaction by supplying dynamic whiteboard capabilities, real-time full-duplex audio and video capabilities, web touring, session management, polling, file sharing, whisper capabilities, attendance, and hand raising features for participant hand-off capabilities. Along with underlying direct access technology by which presenter and participant can interact without any downloading or installation of software.

SUMMARY OF THE INVENTION

[0005] The present invention provides a computer-based system for facilitating collaborative interactions via the Internet or an intranet. In particular, the present invention provides a presenter/participant interactive computer based educational and meeting system, coupled with the ability for direct consumer marketing. Using multiple computers the system allows a multiplicity of individuals to mimic a live classroom or meeting setting by providing various parallel features such as real time audio and visual capabilities, hand raising features, whispering features, attendance tracking, participant polling, hand-off capabilities, an interactive whiteboard, and a variety of other information and content sharing capabilities, all without downloading any software.

[0006] Moreover, the present invention bridges the gap between text-only interactions and live interactive audio streaming. The present invention also includes the ability for the session presenter, as well as the participants, to speak and be heard. The live audio functionality allows the facilitator to talk to the participants as he/she guides them through presentations, training, product demos, or any other type of session. This allows a presenter to present sessions, which mimic or parallel “live” sessions. In addition, participants are able to speak in order to ask questions, make comments, or provide additional information.

[0007] In particular, the present invention provides a system and method for converting an application-specific presentation file stored in a first data store with corresponding metadata to a universal format for display on a web browser. The method includes the following steps: uploading the application specific file to the first data store, detecting the uploaded application specific file in the database, reading the metadata corresponding to the application specific file from the database, determining from the metadata whether the file extension corresponds to the specific application, loading the application specific file from the database, validating that the application specific file corresponds to the specific application by examining header information of the application specific file, converting the application specific file into a universal image file format, modifying the resolution of the universal format file, validating the resolution of the universal format file, storing the modified universal format file in a second data store for display on the web browser, and transmitting the modified universal format file to the web browser for display. Principally, the application specific files are Microsoft PowerPoint format files and the universal image file format is a JPEG format.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008]FIGS. 1 through 26 of the drawings depict a version of the current embodiment of the present invention for the purpose of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principals of the invention described herein.

[0009]FIG. 1 depicts a block diagram of the structural relationship between the presenter and participants in the present invention.

[0010]FIG. 2 shows a graphical user interface constituting the presenter window.

[0011]FIG. 3 shows a graphical user interface constituting the participant window.

[0012]FIGS. 4<i>a</i>-<i>b </i>show graphical user interfaces constituting the whiteboard menu screen for the presenter and participant, respectively.

[0013]FIGS. 5<i>a</i>-<i>d </i>show graphical user interfaces for the polling windows of the system.

[0014]FIG. 6 shows a graphical user interface constituting a presentation window for movies and other content.

[0015]FIG. 7 shows a graphical user interface constituting the attendance window.

[0016]FIG. 8 is a block diagram representative of the navigation of the system homepage.

[0017]FIG. 9 is a block diagram representative of the navigation through the Join Session portion of the system.

[0018]FIGS. 10<i>a</i>-<i>f </i>are block diagrams representative of the navigation through the Options portion of the system.

[0019]FIG. 11 is a block diagram representative of the navigation through the Registration portion of the system.

[0020]FIG. 12 is a block diagram of the system components, which facilitate the automated presentation conversion process.

[0021]FIG. 13 is a flow chart of the automated presentation conversion process in relation to the system components depicted in FIG. 9A.

[0022]FIG. 14 is a block diagram of the system architecture of the streaming audio collaboration process.

[0023]FIG. 15 is a block diagram further detailing the media streaming tunneling with respect to the overall system architecture.

[0024]FIG. 16 shows a block diagram detailing the streaming audio collaboration process of the system.

[0025]FIGS. 17<i>a</i>-<i>c </i>show the overall system layout detailing the various client side Java applet and server side servlet interactions.

[0026]FIG. 18<i>a </i>is a block diagram depicting portions of the conference applet architecture.

[0027]FIG. 18<i>b </i>is a block diagram depicting portions of the queue applet architecture.

[0028]FIG. 18<i>c </i>is a block diagram depicting portions of the whiteboard applet architecture.

[0029]FIG. 18<i>d </i>is a block diagram depicting portions of the breakout applet architecture.

[0030]FIG. 19 is a block diagram depicting portions of the main servlet architecture.

[0031]FIG. 20 shows a block diagram detailing multiple user connections in the system.

[0032]FIG. 21 shows a block diagram detailing categories of controls provided by the Java servlets, applets and scripts utilized by the system architecture.

[0033]FIG. 22 shows a block diagram detailing the system architecture of the white board component.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0034] Overview

[0035] The basic structural relationship between presenter computer <b>100</b> and participant computers <b>120</b> is depicted in FIG. 1. Presenter computer <b>100</b> and participant computers <b>120</b> are all linked together by web-based system server <b>140</b> via the Internet <b>130</b> for facilitating collaboration between a presenter and a plurality of participants. All of the presentation content is uploaded by the presenter to and maintained on server <b>140</b>. In order to control the collaboration process, all communications between presenter computer <b>100</b> and participant computers <b>120</b> are passed through and controlled by server <b>140</b>. There are no direct communications between presenter computer <b>100</b> and participant computers <b>120</b>. While only a single presenter computer <b>100</b> relative to multiple participant computers <b>120</b> is depicted in FIG. 1 to represent a single collaboration session, server <b>140</b> might be coupled to multiple presenter computers <b>100</b> since event server <b>140</b> can simultaneously process multiple collaboration sessions.

[0036] Server <b>140</b> is constructed of a variety of different applications including conversion engine <b>145</b> (developed in VC++), whiteboard application <b>150</b> (developed in Java), core engine <b>175</b> (developed in Java), audio/video media engine <b>170</b> (developed ATL in VC++), back-end application <b>185</b> (developed in JSP), and administrative application <b>190</b> (developed in JSP). Additionally, server <b>140</b> includes several different standard server technologies: web server <b>155</b> (which can be any commercially available web server application that provides web publishing functionality such as Java web server from Sun Microsystems or Apache-Tomcat servers), mail server <b>160</b> (which can be any commercially available mail server that provides SMTP mail functionality such as Internet Information Server from Microsoft), database <b>165</b> (which can be any specially configured commercial database product such as MS-SQL from Microsoft), and media server <b>195</b> (which can be any commercially available media server application that provides audio/video streaming functionality such as Media Streaming Server from Microsoft).

[0037] Core engine <b>175</b> controls communications and interactions between all of the other applications on server <b>140</b> as well as communication with presenter computer <b>100</b> and participant computers <b>120</b>.

[0038] The components of application server <b>140</b> comprise two layers. System application layer <b>142</b> includes system specific, specially programmed applications: whiteboard application <b>150</b>, media streaming application <b>170</b>, presentation conversion and publishing engine <b>145</b>, back-end application <b>185</b>, administration application <b>190</b> and core engine <b>175</b>. Standard server layer <b>144</b> includes commercially available third party server applications provide different type of services as needed: web server <b>155</b>, mail server <b>160</b>, database <b>165</b>, and media server <b>195</b>. The architecture of server <b>140</b> is described below in more detail in the System Architecture section.

[0039] The presenter is the person who initiates a session, or event. This is different from the perspective of those merely participating in the collaboration session. The presenter has access to many more functional controls than the participants.

[0040] The system allows a presenter to share numerous types of materials during a session with participants. Some of these materials include documents, presentations, spreadsheets, images, movies, and questionnaires. In addition to the different types of materials, the presenter also has several options on how to make the information available to participants. These options include making the material available for download, only for playback, available prior to the session, for interactive participation, available using special streaming technology.

[0041] The system also provides for participation by a specialist during a session. A specialist, while not the leader of the collaboration session, acts as a co-presenter when authorized. The system architecture treats specialist computer <b>180</b> physically like participant computers <b>120</b> as authorization is required for specialist computer <b>180</b> to exercise control over the session and logically like presenter computer <b>100</b> as specialist computer <b>180</b>, when authorized, has the same control (except web touring/get file, breakout sessions, poll results, attendance) over the session as presenter computer <b>100</b>.

[0042] Generally, the content can be classified as pre-session content, session content, movies, white board presentations (e.g., PowerPoint slide shows), or special files. Pre-session content is used to prepare participants for the session, promote the session, and encourage people to register and attend. The presenter loads the pre-session materials to server <b>140</b> when the session is set up and then can be downloaded before the start of the session by participants. Furthermore, the content is accessible prior to the session when reviewing session logistics and during registration. While the pre-session content can include any type of content, it is not recommended for movies.

[0043] The session content includes the same materials as pre-session content and often is used as reference material during the session. The materials are loaded by the presenter when setting up the session and then are available for download. The session content is accessible by the presenter and participants during the session. Furthermore the session content can include any type of content, but is not recommended for movies.

[0044] The presenter loads audio/visual content (e.g., movies and audio clips) to server <b>140</b> when the session is set up, and audio/visual content is accessible by the presenter and participants during the session. Audio/visual content is used for playing and streaming pre-recorded movies (video files) or audio clips and for streaming large files without any download. The audio/visual content may also include smaller files, which are delivered either via file download or through live streaming. Streamed materials cannot be downloaded or saved by participants.

[0045] Participants are able to use live audio streaming in a variety of ways to more easily accommodate the equipment at their disposal. The functionality of the present invention enables voice over internet protocol (VolP) to allow users to speak directly from one computer to another over the Internet. This allows voice communication even if the user has only one phone line. VolP does require, however, that the user have a sound card, microphone and speakers. For users without a microphone and speakers, the system also has enabled audio functionality via telephone. This allows participants to speak through a standard telephone. Audio streaming operates from pc to pc and telephone to pc.

[0046] Audio functionality makes user interactions more seamless and easier to use. Full voice capability is pushed out to the users without an application download, operating on 28.8 kbps connections or higher. Furthermore, the system offers this functionality in most cases without prompting the presenter or the participants to download any software from server <b>140</b> or any other source.

[0047] The system also has a dynamic whiteboard platform for information exchange. Whiteboard presentations are used by the presenter to drive presentations directly on participants' screens and allow for interactive presentations with annotations and where control can be given to participants. The participants cannot download these materials from server <b>140</b>. The presenter loads the presentations to server <b>140</b> when the session is set up and controls when participants can view it using whiteboard <b>400</b> (see FIG. 4). Additionally, the presenter can authorize specific participants to have access to whiteboard <b>400</b> to make annotations. One example of a white board presentation is a Microsoft PowerPoint slide show, which is the preferred presentation type of the present invention.

[0048] In the preferred embodiment, presentation conversion and publishing engine <b>145</b> utilizes MS PowerPoint format (PPT) files, which are converted into an image format file. Whiteboard application <b>150</b> then displays the image format file on whiteboard <b>400</b>. While presentation conversion and publishing engine <b>145</b> converts only PPT files, other types of files maybe displayed on whiteboard <b>400</b>. In particular, any presentation in a format that can be converted to a PPT file (e.g., MS Word, MS Excel) can be displayed on whiteboard <b>400</b> by converting the presentation into a PPT file before the presentation is processed by presentation conversion and publishing engine <b>145</b>.

[0049] Other content may include special files, images, web tours and interactive questionnaires, which are used by the presenter to display content directly on participants' screens. The types of files are useful as backup files for the presenter and can be used as necessary. The presenter loads the special files when setting up the session and controls when participants can view the files. The special files are pushed to participants when played.

[0050] The whiteboard platform provides a presenter with a strong set of tools to manage events. Key features of whiteboard application <b>150</b> include: presentation running (e.g., navigating backward and forward through whiteboard <b>400</b>), annotation tools, and the ability to hand-off controls to multiple participants (known as hand raising and authorization).

[0051] Presentation running allows the presenter to direct the image that each individual participant sees on his or her respective screens. For instance, a presenter can run a converted PowerPoint slide show on his or her whiteboard <b>400</b><i>a</i>, and as the presenter Flips from slide to slide, each participant is able to see the slides progress through his/her own whiteboard <b>400</b><i>b</i>. This puts the ability to guide the event in the hands of the presenter.

[0052] In addition to running presentations through whiteboard <b>400</b>, the presenter can also open a web browser and guide participants to various websites, i.e., a web tour. As the presenter directs his or her browsers and clicks through to new pages or sites, all of the participants view the same pages through their own browsers. This functionality can be applied for navigating Internet or intranet sites. A dedicated browser that is downloaded to participant computers <b>120</b> provides this web tour feature. The dedicated browser functions much in the same way as whiteboard <b>400</b> in that a hand raise button is provided on the participant view and a authorize buttons are provided on the presenter view in order to allow for co-share capability.

[0053] To provide additional support while using whiteboard <b>400</b>, the system features a built in set of annotation tools. The annotation tools enable the presenter to call attention to specific items on the whiteboard by using highlighters, pointers, drawing tools, and the ability to add text comments. The presenter can also undo specific annotations using a select button or erase the whole drawing including the slide by just pressing a clear button. By selecting an annotation tool, such as the highlighter, the presenter can highlight a specific area on his or her whiteboard <b>400</b><i>a</i>, and all of the participants will see the highlighting appear through their own whiteboards <b>400</b><i>b </i>at the same time. Freehand annotations can be made using a mouse or writing tablet.

[0054] The system not only gives a presenter the enhanced ability to guide an event, the presenter can also pass control of whiteboard <b>400</b> to individual participants as desired. For instance, if participants have questions, or additional information to share, the presenter can pass the controls to the participant. The participant controlling these features is then able to guide what all of the other participants see through their whiteboard <b>400</b><i>b </i>including the ability to run presentations and annotate. Participants can also be granted control to conduct web tours, if so desired by the presenter.

[0055] Participants can raise their hands (figuratively) directly from whiteboard <b>400</b><i>b </i>to request presenter controls. The presenter can see who has a raised hand and can authorize the participants directly from whiteboard <b>400</b><i>a. </i>

[0056] The ability to hand off control does have an additional requirement related to running applications. If the presenter wishes to give control to a participant for them to run an application, it is necessary that the participant have the application installed on their participant computer <b>120</b>. If the participant has the application installed, and the presenter grants him or her access, the participant can guide what is seen on whiteboard <b>400</b> and they can also add content, edit files and save updates. This functionality allows multiple participants in different locations to work together on the same files at the same time.

[0057] Graphical User Interface

[0058] Graphical user interfaces (GUI's) allow the presenter, participants and the session administrator to interact with the system and each other. The key windows of the system GUI's for the presenter and the participants are depicted in FIGS. <b>2</b>-<b>7</b>.

[0059] Referring now to FIG. 2, the system's primary graphical user interface (GUI) for the presenter, presenter window <b>200</b>, is shown. The presenter is the individual(s) who leads a meeting, instructs, or teaches a program for students or participants. Presenter window <b>200</b> is spatially divided into three console areas: control A console <b>200</b><i>a</i>, control B console <b>200</b><i>b</i>, and master communication console <b>200</b><i>c</i>. In general terms, control A console <b>200</b><i>a </i>contains controls for selecting and deselecting participants and files sent to those participants. Control B console <b>200</b><i>b </i>contains advertisement information and speech (Voice) controls. Master communication console <b>200</b><i>c </i>contains controls for the transmission and receipt of collaboration information between the presenter and participants.

[0060] On control A console <b>200</b><i>a</i>, audience box <b>202</b> lists the presenter and then the list of participants directly underneath. The presenter's name is shown on the top of the list with a line separating it from the user's name. Participants that wish to pose a query are shown to the presenter in hand-raised box <b>204</b>. Hand raiser box <b>204</b> contains the names of participant that have pressed hand raise button <b>305</b> (see FIG. 3).

[0061] Directly underneath the hand-raiser box <b>204</b> there is an authorized box <b>208</b>. Authorized box <b>208</b> informs the presenter who among the participants has been given authority (i.e., control) to draw on the white board and has use of audio. “Authorized: None” means that no participant has been authorized. The presenter may also grant whiteboard control directly from whiteboard <b>400</b> as depicted in and explained with reference to FIG. 4.

[0062] The presenter can also select a participant from audience box <b>202</b> to whom a personalized, private message can be sent. Whisper box <b>210</b> indicates to the presenter which participant will receive the personalized message.

[0063] A participant can be selected for whispering by clicking on a particular name within the audience list <b>202</b> and then clicking the “+” (whisper select) button <b>203</b>. Then, the presenter can use the “−” (whisper deselect) button <b>205</b> to remove, participants from whisper box <b>210</b>. Once a name is selected for whisper action, on master communication console <b>200</b><i>c </i>the presenter then enters the text in type here box <b>212</b> and presses send whisper button <b>214</b>. The presenter may leave the whisper name selected, until some text is entered and send whisper button <b>214</b> is pressed. No whispering takes place from the presenter until send whisper button <b>214</b> is pressed, but the presenter may receive whisper messages from other participants in the session. As discussed below in more detail, whisper messages are displayed in whisper box <b>232</b> of both the sender and recipient of the whisper message, and in message bar <b>242</b> of the recipient of the whisper message.

[0064] On control A console <b>200</b><i>a </i>below hand-raised box <b>204</b>, authorize and unauthorized buttons <b>216</b> and <b>218</b>, respectively, are provided. Authorize button <b>216</b> allows a presenter to select one of the hand-raised persons to authorize him or her for speaking and using whiteboard <b>400</b>. The name should be first selected from hand-raised box <b>204</b> before authorizing the participant. The name of the authorized participant appears in authorized box <b>208</b>.

[0065] If an authorized participant already exists and another participant becomes authorized, the previously authorized participant becomes unauthorized and authorized box <b>208</b> displays the name of the next selected participant. Unauthorized button <b>218</b> allows the presenter to unauthorize a currently authorized participant. This results in an “Authorized: None” message.

[0066] Below unauthorize button <b>218</b>, there is file selection combo box <b>220</b> and blank text box <b>222</b>. File selection combo box <b>220</b> provides a list of files provided by the presenter and available at the server. This list may contain audio/visual avi documents or other documents. Any file presented from this list can be shown to each participant as well as the presenter. To accomplish this, the presenter selects the file and clicks the send to group button <b>224</b>. The selected file is then pushed to the participant computers <b>120</b> which will display the file provided the corresponding application or viewer is already present on participant computer <b>120</b>.

[0067] If the presenter types in a URL in box <b>222</b> and presses send to group button <b>224</b>, a browser will open on participant computers <b>120</b> and the web page corresponding to the URL will be displayed. Provided the participants have downloaded the system's dedicated browser, the presenter can guide the participants along a web tour and authorize participants to do the same.

[0068] Below send to group button <b>224</b> there is breakout session button <b>226</b>. Clicking on this button will open up a dialog box to break the session into small groups of participants.

[0069] Turning to control B console <b>200</b><i>b</i>, the button at the right bottom of presenter window <b>200</b> shows a microphone. This is microphone selector <b>260</b>, which represents the audio streaming options and toggles between a “press to talk” and “press to stop” option. The button is in the on position (i.e., “press to stop”) as a default. When a presenter logs in, the button appears with the message: “Press to Stop” showing that the presenter is already on the air and can immediately start his speech or lecture. If the presenter wishes to stop broadcasting his or her voice, he or she simply clicks the button once to stop the broadcast and the caption will change to “Press to Talk.” This is basically a toggle button, which switches on/off by clicking on it. The same button appears on a participant's screen when that particular participant has been authorized.

[0070] Master communication console <b>200</b><i>c</i>, contains four text boxes: comments <b>228</b>, questions <b>230</b>, notes/whisper <b>232</b> and answers <b>234</b>. These text boxes display the incoming and outgoing comments, questions, answers and whisper messages respectively. When a user enters text in type here text box <b>212</b> and presses one of the buttons: question <b>236</b>, answer <b>238</b>, and comment <b>240</b> then text is sent to every user and displayed in the appropriate box. Pressing whisper <b>214</b> sends the text only to the designated whisper recipient in whisper box <b>210</b>. The text is also displayed in notes/whisper box <b>232</b> as a personal note for the sending user. If a user clicks any of these buttons (i.e., comment <b>240</b>, answer <b>238</b>, question <b>236</b> or whisper <b>214</b>) without having inserted any text, a reminder message is flashed on message bar <b>242</b> as a reminder to enter text.

[0071] In order to track the sender of a whisper message, message sent by different participants are marked in different colors with the color corresponding to the color associated with the sender in audience box <b>202</b>. This reminds the presenter and participants of a particular whispering person by identification with color. It should be noted that any user could whisper to any other user.

[0072] Log out button <b>248</b> is used to log out or exit from a session. The presenter and all participants should click this button when they are ready to leave the session. When log out button <b>248</b> is clicked, a window will appear asking if the user is sure they want to exit the session. If the user clicks “yes”, they will be removed from the session and their name will be removed from audience box <b>202</b>. If the user clicks “no”, they will rejoin the session. When a participant logs out, the presenter will receive a message in their notes/whisper box <b>232</b> that the participant has left. A message will also appear in message bar <b>242</b> when a participant logs out.

[0073] If the presenter or a participant accidentally logs out or closes their browser window, they can rejoin the session. To rejoin the session, simply, go to the Join Session screen (FIG. 9 for public sessions and FIG. 10<i>f </i>for private sessions) and login using the same User Name and ID that were used in the original login.

[0074] Help button <b>252</b> is located on main communication console <b>200</b><i>c </i>next to log out button <b>248</b>. Pressing help button <b>252</b> provides a user manual to the participants and presenter regarding how to use the system.

[0075]FIG. 3 depicts participant window <b>300</b>. Both presenter window <b>200</b> and participant window <b>300</b> have the same general layout. Essentially, participant window <b>300</b> (FIG. 3) provides the same view as the presenter window <b>200</b> (FIG. 2) but with less functionality. For example, participant <b>300</b> window does not include authorize button <b>216</b>, unauthorize button <b>218</b>, send to group button <b>224</b>, breakout session button <b>226</b>, answer button <b>238</b>, whiteboard button <b>244</b>, microphone selector <b>260</b> (unless authorized), poll button <b>246</b>, result button <b>256</b>, or attendance button <b>258</b>. However, participant window <b>200</b> does include some added functionality such as raise hand button <b>305</b>. Like buttons on participant window <b>300</b> provide the same functionality as those on presenter window <b>200</b>. Additional presenter buttons appear on participant window <b>300</b> to give the participant limited presenter like control, such as the ability to speak (microphone selector <b>260</b>), when authorized by the presenter.

[0076] Also on participant window <b>300</b> is audio message bar <b>310</b>, which indicates the audio streaming status, such as audio active, buffering and playing. This allows both the presenter and participants to keep abreast of the audio media player status and coordinate full duplex speech. When the presenter authorizes a participant to speak, audio message bar <b>310</b> also appears in the lower right-hand corner of presenter window <b>200</b> just below microphone selector <b>260</b>. Audio message bar <b>310</b> will first display the words “Audio Active” to indicate the system is ready to hear the authorized participant. Once the authorized participant speaks, audio message bar <b>310</b> will indicate “buffering” while the audio is buffered and then “playing” when the voice is output. Audio message bar <b>310</b> is always present in participant window <b>300</b> but only appears on presenter window <b>200</b> when a participant is authorized. Since FIG. 2 indicates that no one is authorized, audio message bar <b>310</b> does not appear.

[0077] Referring now to FIG. 4, shown is whiteboard presentation tool <b>400</b> of the present invention from the view of the presenter (FIG. 4<i>a</i>) and the view of an unauthorized participant (FIG. 4<i>b</i>). Whiteboard button <b>244</b> on the presenter menu (FIG. 2) is used to activate whiteboard <b>400</b> for display of presentation slides, and to draw on whiteboard <b>400</b> and send the drawing to the participants. If whiteboard <b>400</b> is not opened, the presenter simply clicks on whiteboard button <b>244</b>, which makes whiteboard <b>400</b> appear to every participant computer <b>120</b> in the session.

[0078] Content can be added to whiteboard <b>400</b> prior to the session. In addition, any type of static content can be used in whiteboard <b>400</b>, such as images, presentation slides, documents, and spreadsheets. Whiteboard <b>400</b> also allows users to create new content using blank slides. Content that is loaded into whiteboard <b>400</b> does not require any data conversion by the presenter. The presenter can load static content (as opposed to videos or other files that include motion) in any standard file type. Note that slides with animations can be loaded into whiteboard <b>400</b>, but the animations will not show during playback. Content may be used and displayed on the participant computers <b>120</b>, even if the participant does not have the corresponding content application resident on participant computer <b>120</b>. Server <b>140</b> provides an automated conversion process (driven by conversion engine <b>145</b>) to allow this functionality. The process for PowerPoint content is described below in the Automated PPT Conversion section. Although other formats can be used, the preferred embodiment of the present invention converts MS PowerPoint (PPT) format files for presentation on whiteboard <b>400</b>. Other file types are first converted into PPT format before entering the conversion process of the preferred embodiment.

[0079] As depicted in FIG. 4<i>a</i>, color selection tablet <b>405</b> on whiteboard <b>400</b><i>a </i>allows the presenter to draw text, objects, or other annotations in the color of his/her choice by allowing the presenter to select a color from color selection tablet <b>405</b> for the desired annotation tool. Whiteboard <b>400</b> includes a full array of annotation tools including: text button <b>410</b> to write text, line button <b>415</b> to draw lines and curves, oval button <b>420</b> to draw circles and ovals, rectangle button <b>425</b> is used to draw rectangles and squares, freehand button <b>465</b> to draw anything by hand like a pen on whiteboard <b>400</b>.

[0080] These buttons all activate well-known standard annotation tools and operate in a similar manner to those on many commercially available drawings programs. Generally, the presenter (or participant when authorized) selects the annotation tool by pressing the appropriate button. Next, the presenter clicks on the board area where they wish to start the annotation from and then drags it to its end point by the left button of the mouse pressed. The presenter can clear the drawing annotations by using select button <b>450</b> to select the annotations and then pressing clear button <b>470</b>.

[0081] Annotations can be added to any existing whiteboard <b>400</b> or they can be created on a new, blank whiteboard <b>400</b>. To open a blank screen, the presenter selects erase all button <b>475</b> before using the desired annotation tool. Erase all button <b>475</b> clears the entire screen of both the annotations and the slide content.

[0082] As a safety precaution, annotations on whiteboard <b>400</b> are not automatically sent to the participants. In order to send the drawings, the presenter presses send button <b>445</b>. The annotations made by the presenter will then appear on participant whiteboards <b>400</b><i>b. </i>

[0083] At the bottom of whiteboard <b>400</b>, topics list box <b>430</b> appears carrying the topic names of presentation slides. The presenter before the start of the session must supply these names while uploading the presentation(s). Previous button <b>435</b> and next button <b>440</b> are available to navigate through the presentation slides. If topics do not appear the first time, the presenter simply presses next button <b>440</b> to reinitiate the topic selection. If no topic is available, next and previous buttons <b>435</b> and <b>440</b> will have no effect.

[0084] The participant's view of whiteboard <b>400</b>, shown in FIG. 4<i>b</i>, is slightly different than the presenter's view, shown in FIG. 4<i>a</i>. The toolbar does not appear on the participants' view, unless the participant is authorized. When the presenter authorizes a participant to control whiteboard <b>400</b>, that participant's toolbar will be activated (and visible) on FIG. 4<i>b </i>in the same manner as seen from the presenter's view in FIG. 4<i>a</i>. When the presenter unauthorizes the participant, the toolbar will again automatically be removed and the whiteboard <b>400</b><i>b </i>will return to the view shown in FIG. 4<i>b. </i>

[0085] In order to be authorized, a participant must request authorization from the presenter. The participant pressing hand-raise button <b>480</b>, as shown on FIG. 4<i>b</i>, generates the authorization request. This will cause hand indicator <b>485</b> on both the presenter and participant's whiteboard <b>400</b> to change colors indicating an authorization request. The names of all participants that have raised their hands will appear on hand-raisers list box <b>490</b> on FIG. 4<i>a</i>. The presenter then selects a participant from hand-raisers list box and presses authorize button <b>492</b> to provide the selected participant control of whiteboard <b>400</b>. The presenter can unauthorized the selected participant by pressing unauthorized button <b>494</b>. Video conferencing button <b>496</b> on participant whiteboard <b>400</b><i>b </i>activates the video conferencing feature of the system, which is described in more detail in the Media Streaming section below.

[0086] Thus, the presenter can hand off the controls to an authorized participant so they can both share the driver's seat. The ability to share controls with the participants enables the session to be truly interactive. Once the presenter authorizes a participant, that participant can then navigate through the slides and annotate. The authorized participant's microphone is also activated, so the other participants can hear both her and the presenter's voices. Details are provided below in the Audio Streaming section.

[0087] When a participant is authorized to control whiteboard <b>400</b>, the presenter continues to also have access to the controls. Using full duplex audio streaming, both the presenter and the authorized participant can speak with each other at the same time, like with a telephone. The presenter also maintains the ability to unauthorize the participant, thereby removing their control of whiteboard <b>400</b>.

[0088] First, if a participant would like to ask a question or take control of whiteboard <b>400</b>, she must raise her hand using raise hand button <b>480</b>. When the presenter is ready to share the controls, the presenter selects the participant's name from hand-raised box <b>204</b> and clicks authorize <b>216</b>, or from hand-raisers list box <b>490</b> and clicks authorize button <b>492</b>. The participant will then receive the controls causing an “audio active” message to appear in audio message bar <b>310</b> and microphone indicator <b>260</b> to appear on participant window <b>300</b> just above audio message bar <b>310</b>. Additionally, message bar <b>310</b> indicating “audio active” will also appear on presenter window <b>200</b> as previously described.

[0089] When the participant is finished (or actually at anytime whether the authorized participant is finished or not), the presenter can click unauthorize button <b>218</b> or unauthorized button <b>494</b> to remove the controls. Only the presenter and one participant can share the controls at a given time, but once one participant is unauthorized, another can be given the controls.

[0090] Turning back to presenter window <b>200</b> (FIG. 2), poll button <b>246</b> on master communication console <b>200</b><i>c </i>allows the presenter to poll the participants. Pressing poll button <b>246</b> results in a small window <b>500</b> appearing with a text box (FIG. 5<i>a</i>) to type in a question and send it to the participants. Pressing poll button <b>505</b> on polling window <b>500</b> causes the polling question to be sent to all participants. When the presenter clicks on poll button <b>505</b>, a small polled window <b>510</b> appears on the participants' screens and the participants are given the option to answer by pressing any one of the buttons available in the window (i.e., “Yes” <b>515</b>, “No” <b>520</b>, and “Not Sure” <b>525</b>) (FIG. 5<i>b</i>). These labels can be changed. The presenter may then view the list of polled questions (FIG. 5<i>c</i>) and a graphical representation of the polling results for each question (FIG. 5<i>d</i>).

[0091] Additionally, using master communication console <b>200</b><i>c</i>, the presenter may view the poll results during a session by clicking poll-result button <b>256</b>. As shown in FIG. 5<i>c</i>, when the presenter clicks on poll result button <b>256</b>, a new window <b>530</b> appears displaying a list <b>535</b> all the questions asked during a particular session. The presenter can select any one of them, by highlighting the selection and clicking proceed button <b>540</b>. A graphical representation of the results will appear as shown in FIG. 5<i>d</i>. The participant may press refresh button <b>545</b> to refresh the question list displayed in drop down list <b>535</b>.

[0092] Continuing with FIG. 2, in the grouping of buttons with poll button <b>246</b> which appear on the right side of master communication console <b>200</b><i>c</i>, movie button <b>250</b> and content button <b>254</b> are present. By pressing movie button <b>250</b>, presentation window <b>600</b> appears as depicted in FIG. 6. Referring to FIG. 6, a user can select any of the links to watch a particular movie. FIG. 6 is representative of the presenter view, participant view and the authorized participant view.

[0093] Turning back to FIG. 2, content button <b>254</b> appears on the right side of main communication console <b>200</b><i>c </i>as well. Upon pressing content button <b>254</b>, presentation window <b>600</b> appears on the participant computer carrying hyperlinks to suggestive and informative material uploaded by the presenter for a particular session as depicted in FIG. 6. The content files may be in any standard file format.

[0094] Located near the top of control A console <b>200</b><i>a </i>is attendance button <b>258</b> that the presenter can use to see the session joining time of each user during a session. When attendance button <b>258</b> is clicked, a new attendance window <b>700</b> will appear as shown in FIG. 7. In attendance window <b>700</b> will be a list <b>710</b> of the participants' user names along with the time they joined the session. To return to presenter window <b>200</b>, the presenter simply closes attendance window <b>700</b>.

[0095] GUI Navigation

[0096] The navigation through all of the GUI's for registration, joining sessions and administrative purposes are depicted in FIGS. <b>8</b>-<b>11</b>. Among the many functions accessed via the GUI structure (FIGS. <b>8</b>-<b>11</b>), as shown in FIGS. 10<i>d </i>and <b>10</b><i>f</i>, the presenter and participants navigate the GUI's to reach presenter window <b>200</b> and participant window <b>300</b>, respectively. The functionality for controlling GUI navigation and allowing client administration is provided by back-end application <b>185</b> (see FIG. 1).

[0097]FIG. 8 depicts the structure of system homepage <b>800</b> accessible to anyone via the Internet. From the system homepage, a user has three options 1) join a session <b>810</b>, 2) access client administration <b>820</b>, or 3) register <b>830</b> as a user on the system.

[0098] Selecting join session option <b>810</b> provides participants and presenters with access to the publicly available sessions on the system. Only participants in public sessions access the system via join session option <b>810</b>. Join session option <b>810</b> leads the user to the menu structure depicted in FIG. 9. Users can choose from a listing of scheduled sessions <b>910</b> and view the session details <b>920</b>. Session login menu <b>930</b> provides users access to the selected session, participants via menu <b>940</b> and presenters via menu <b>950</b>. Upon accessing session login <b>930</b>, the system checks the users web browser to test for the presence of a current version of the Microsoft Media Encoder. The system either validates the presence of the encoder <b>960</b> or prompts <b>970</b> the user to obtain the current encoder. As discussed below in the audio streaming section, the encoder is necessary for audio streaming.

[0099] Selecting client administration option <b>820</b> provides the user access to client private sessions and client specific administration functions accessible via the menu structure depicted in FIGS. 10<i>a</i>-<i>f</i>. FIG. 10<i>a </i>provides an overview of all of the available client administrative options, while FIGS. 10<i>b</i>-<i>e </i>provide the detail of the menu structure underlying each menu option. FIG. 10<i>f </i>provides the detail of the menu structure for accessing client-scheduled sessions.

[0100] As depicted in FIG. 10<i>a</i>, upon selecting client administration option <b>820</b>, the user is prompted by menu <b>1000</b> to login to the system. Once logged in, the user selects access either to administrative options <b>1002</b> or scheduled sessions <b>1004</b>. The various administrative options include menus to maintain departments <b>1006</b>, manage users <b>1008</b>, maintain sessions <b>1010</b>, maintain specialists <b>1012</b>, maintain content <b>1014</b>, maintain advertisements <b>1016</b>, configure mailing lists <b>1018</b>, access send mail wizard <b>1020</b>, change passwords <b>1022</b>, view registrations <b>1024</b>, initiate sessions <b>1026</b>, maintain movies <b>1028</b>, maintain presentations <b>1030</b>, maintain files <b>1032</b> and log out <b>1034</b>. Each option is depicted in detail in FIGS. 10<i>b</i>-<i>e</i>, which are self-explanatory. These option menus are for use by the client's system administrator and presenters. Of note, a presenter accessing initiate sessions menu <b>1026</b>, after selecting from a listing of sessions, is directed to presenter window <b>200</b> for the selected session.

[0101] Selecting scheduled sessions <b>1004</b>, instead of options <b>1002</b>, leads the user (typically presenters and participants) to the menu structure depicted in FIG. 10<i>f </i>for accessing the client's private sessions. Participants select from a listing of sessions to either pre-register <b>1036</b> for an upcoming session or join <b>1038</b> a session that has started or is about to start. Profile information, such as the title, topic, date, time, fee and status, for each session are displayed on scheduled sessions menu <b>1004</b>. The registration process leads the participant through registration form <b>1040</b> followed by registration confirmation menu <b>1042</b>. Once the registration is confirmed, the participant may search other ongoing sessions <b>1044</b> for which the participant may pre-register <b>1046</b> (via registration form <b>1040</b>) or join <b>1048</b> (via session login menu <b>1050</b>).

[0102] To join a session, the participant accesses join session menu <b>1050</b> via join option <b>1038</b> on scheduled session menu <b>1004</b> or join option <b>1048</b> from ongoing session menu <b>1044</b>. Also, presenters access session login menu <b>1050</b> via join option <b>1038</b>. Upon access to join session menu <b>1050</b>, the system performs the same browser check that was performed with respect to join session menu <b>930</b> (see FIG. 9) and described above. After the user logs on as either a participant <b>1052</b> or presenter <b>1054</b>, the user is directed to participant window <b>300</b> (see FIG. 3) or presenter window <b>200</b> (see FIG. 2), respectively.

[0103] Selecting registration option <b>830</b>, provides the user with the client setup features of the system via the menu structure depicted in FIG. 11. From these menus, the user begins the client setup procedure by specifying the account type (e.g., corporate, university, clinical), user name, password and a password hint via client setup menu <b>1100</b>. The user is then directed to either company setup menu <b>1110</b>, university setup menu <b>1120</b>, or clinic setup menu <b>1130</b>, respectively depending upon the account type, where the user inputs critical contact information such as the client name, industry, contact name, telephone, address, and the like. Once the information is input, the user is directed to a corresponding setup confirmation menu <b>1140</b>, <b>1150</b> or <b>1160</b>, respectively depending upon the account type.

[0104] As explained above, the system may administer multiple clients and schedule multiple sessions for each client. The administration and accounting for multiple clients from the internal system administration perspective is handled by administration application <b>190</b> (see FIG. 1).

[0105] Advertisements

[0106] The system includes an automated advertisement placement capability to provide the opportunity for direct consumer marketing. As shown in FIGS. 2 and 3, advertisements <b>262</b> appear in the top of control B consoles <b>200</b><i>b </i>and <b>300</b><i>b</i>, respectively. The advertisements have active http links to designated URL's.

[0107] Control B consoles <b>200</b><i>b </i>and <b>300</b><i>b </i>provide space for two advertising links. Any image or animation can be inserted here along with a hyperlink to any desired web site. The advertising images are added from the backend management tools of the system when the session is setup. The advertisements are used to direct participants to any web-based content, or for specific e-commerce opportunities. If desired, the image can simply show a picture of the presenter.

[0108] The system allows the addition of advertisements to a company's database for use in future sessions. The ads can be any standard image type, logo, or photograph combined with a hyperlink to any live web site.

[0109] When the presenter or participants click on an advertisement during the session, that user will have a new browser window open on their desktop. The new browser will be directed to the URL specified by the presenter when the session was setup. The user can then navigate the new browser, as appropriate. To return to the session, the user must simply click the minimize button.

[0110] Using maintain advertisements option <b>1016</b> (FIG. 10<i>a</i>), a user may add, edit, or delete advertisements on the presenter's company profile as depicted in FIG. 10<i>c</i>. Manage advertisements screen <b>1017</b> appears showing the advertisements that are currently assigned to sessions.

[0111] Advertisements are added sessions in the company profile. To add advertisements, select ADD <b>1017</b><i>a </i>on manage advertisements screen <b>1017</b>. The following required fields are then entered via add advertisement screen <b>1019</b>:

[0112] 1. Session—Select the session to which you wish to add the advertisement

[0113] 2. First Advertisement

[0114] 3. First File

[0115] 4. First URL

[0116] 5. Second Advertisement

[0117] 6. Second File

[0118] 7. Second URL

[0119] To edit advertisements, the user goes to manage advertisements screen <b>1017</b>. The user then highlights the advertisements to edit and selects EDIT <b>1017</b><i>b</i>. Edit advertisements screen <b>1021</b> appears so that edits may be made to the required fields.

[0120] To delete advertisements, the user highlights the advertisement to delete, then selects DELETE <b>1017</b><i>c </i>on manage advertisements screen <b>1017</b>. A message box appears stating: “Are you sure you want to delete “XYZ” advertisement?” The user selects OK to delete the selected advertisement or CANCEL to return to the previous screen.

[0121] Automated PPT Conversion

[0122] The system also includes an automated application to convert and place Microsoft PowerPoint slides for the session to be displayed on whiteboard <b>400</b>. The platform is Microsoft Windows NT Server, 2000 Server and the application is written utilizing the Microsoft Visual C++ v6.0 Enterprise Edition programming language. The automated conversion process allows the presenter to display a presentation on whiteboard <b>400</b><i>b </i>on participant computers <b>120</b> without the need for the presentation application to be present on participant computers <b>120</b> or the download of any applications or plug-ins to participant computers <b>120</b>. The detailed description of the conversion process and structures described below focuses on PowerPoint format presentation files. However, one of ordinary skill could adapt the process to accommodate other presentation formats, such as Harvard Graphics or Freelance.

[0123] The interaction of PPT automate engine <b>1200</b> with the overall system as well as with the user is depicted generally in FIG. 12 and in more detail in FIG. 13. All of the structures depicted in FIG. 12 are contained within server <b>140</b>. These structures include PPT automate engine <b>1200</b> which is included within conversion engine <b>145</b>, presentation database <b>1205</b> within database <b>165</b>, web published directory <b>1210</b> within web server <b>155</b>, whiteboard application <b>150</b>, core engine <b>175</b> and session manager <b>1225</b>.

[0124] PPT automate engine <b>1200</b> facilitates the conversion of PowerPoint presentations for display on whiteboard <b>400</b>, as explained below in reference to FIG. 13. Additional detail is provided below with respect to FIG. 14. Engine <b>1200</b> interacts with presentation database <b>1205</b> and web published folder <b>1210</b> for retrieving uploaded presentations from users and storing converted presentations for display on whiteboard <b>400</b> by whiteboard application <b>150</b>.

[0125] Presentation database <b>1205</b> and web published folder <b>1210</b> are resident on the same storage device but could be easily distributed among multiple devices. Presentation database <b>1205</b> is segmented by client account so that only user's from different clients are segregated. PowerPoint files uploaded by users as well as corresponding metadata are stored in presentation database <b>1205</b>. The metadata includes data such as client information, session information and conversion status information (i.e., conversion status field <b>1230</b>).

[0126] Web published directory <b>1210</b> stores the converted presentations in JPEG format separate from presentation database <b>1205</b> due to the large size of the JPEG files. This allows more rapid access to presentations by whiteboard application <b>150</b>, which is necessary to provide seamless slide show presentations to participants. While the original PowerPoint format file remains in presentation database <b>1205</b> for an extended period, converted presentations are removed from web published directory at the end of a session due to the large file size.

[0127] Under the control of core engine <b>180</b> and in conjunction with session manager <b>1225</b>, whiteboard <b>400</b> is the presentation medium for the converted presentations stored in web published folder <b>1210</b>, which is a secure folder only accessible from whiteboard application <b>150</b>. Whiteboard application <b>150</b> accesses presentations from web published folder <b>1210</b> for presentation and metadata from presentation database <b>1205</b> for validation.

[0128] Turning to FIG. 13, the interaction processes between PPT automate engine <b>1200</b>, presentation database <b>1205</b>, web published directory <b>1210</b> and whiteboard application <b>150</b> is depicted in detail. To upload and convert presentation files, a user, typically the presenter or the client's system administrator, logs into the system and selects options feature to access options menu <b>1010</b> as shown in FIG. 10A. Assuming a session already exists, the user selects maintain presentations <b>1020</b> to access maintain presentation menu <b>1050</b> and then add <b>1055</b> to access add presentations menu <b>1060</b> as shown in FIG. 10<i>e</i>. The user then selects browse <b>1065</b> to choose the presentation and then save <b>1070</b> to upload the file to presentation database <b>1205</b>. The system then uploads <b>1300</b> the presentation to presentation database <b>1205</b> as shown in FIG. 13.

[0129] Independent from uploading <b>1300</b>, at the start of the PPT automate engine <b>1200</b> process, engine <b>1200</b> periodically checks (every few seconds) to detect newly uploaded files to presentation database <b>1205</b> and reads <b>1305</b> the metadata. Engine <b>1200</b> then determines <b>1310</b> if the file for which the metadata was read has a PPT PowerPoint file extension. If it is not a PPT extension, engine <b>1200</b> waits for a pre-determined time (programmable to any time set but preferably 5 to 15 seconds) <b>1315</b> before again reading <b>1305</b> metadata from presentation database <b>1205</b>. If it is a PPT extension, engine <b>1200</b> loads <b>1320</b> the PPT file from presentation database <b>1205</b>.

[0130] Format validator/dispatcher <b>1325</b> then validates that the file is in fact a PPT format file by examining the header information of the file and dispatches the file to the converter algorithm. Once validated and dispatched, engine <b>1200</b> using a converter algorithm then converts the slides in the PPT file into a series of JPEG format files and modifies the resolution (i.e., size) and format of the JPEG file <b>1330</b> for display on whiteboard <b>400</b>. Engine <b>1200</b> uses the PowerPoint COM Interfaces to convert the slides into a series of “jpg” (JPEG) images and modify the resolution. The JPEG files are modified from their standard resolution to 400×300 pixels. The PowerPoint application does not open the PPT file but merely performs the format conversion.

[0131] Engine <b>1200</b> then checks the converted and modified JPEG file to validate <b>1335</b> the conversion and modification process (i.e., correct resolution). If there is an error, engine <b>1200</b> returns to read step <b>1305</b>. If there is not an error, engine <b>1200</b> performs update/write step <b>1340</b> in which engine <b>1200</b> updates the metadata in presentation database <b>1205</b> to indicate a successful conversion and writes the converted file to an appropriate location in web published directory <b>1210</b> so whiteboard application <b>1215</b> of the particular session can gain rapid access. The PowerPoint application and the COM engine are then un-initialized, and the conversion status field in presentation database <b>1205</b> is marked to flag the conversion of the particular file. Engine <b>1200</b> then waits <b>1315</b> before re-initiating the process by reading <b>1305</b> the metadata from presentation database table <b>1205</b> again.

[0132] Turning to whiteboard application <b>150</b>, slide information (i.e., metadata) is loaded for a particular session from presentation database <b>1205</b>. Then, the JPEG format of the slides are loaded <b>1350</b> on demand from web published directory <b>1210</b>. The presenter can then navigate <b>1355</b> the slides using the buttons on whiteboard <b>400</b><i>a </i>to control the slide show seen by the participants on whiteboard <b>400</b><i>b. </i>

[0133] A color-coding scheme is used to mark the progress of the conversion (based upon the data in the conversion progress field) for the user to indicate that engine <b>1200</b> is: waiting for a new PowerPoint presentation to be uploaded; checking presentation database <b>1205</b> for a newly uploaded files; or converting the PowerPoint presentation into a series of JPEGs and placing them in web published directory <b>1210</b>.

[0134] The aspects of the system architecture, which support the whiteboard functionality are depicted in FIG. 22 and described in the system architecture section below.

[0135] Media Streaming

[0136] The concept of audio streaming is not new in IT. Still streaming data is an underdevelopment technology. Till now there are no standard defined by any of the standard defining organizations such as IEEE, IS0 9000 &amp; etc. There are various formats available for streaming media, offered by different companies. All the formats have been developed by independent parties which results in separate download and installation for each parties player or plug-in, such as RealTech™ G2. Additionally, Microsoft provides a streaming media platform built into the Windows operating system. These built-in “Windows Media Components” are predefined and made available in Windows 98 (2^nded.), Windows 2000 and higher versions. Although older versions of Windows do not have these components, upgrade patches to install the media components are readily available from Microsoft. Additionally, independent developers can embed the patches or link to the patches in their product for those users who lack up to date operating systems.

[0137] The preferred embodiment of the present invention utilizes Microsoft's Windows Media Encoder. As described with respect to FIG. 9, the system checks and updates, if necessary, the media encoder files of the remote computer's web browser.

[0138] As depicted in FIG. 14, in order to control the transmission and reception of the live audio stream, server <b>140</b> using media engine <b>1400</b>, which is part of media engine <b>170</b> (media including audio, video and the like), must administer the encoder at both broadcasting computer <b>1410</b> (possibly presenter computer <b>100</b>, specialist computer <b>180</b>, or an authorized participant computer <b>120</b><i>a</i>) and recipient computers <b>1420</b> (all computers <b>100</b>/<b>120</b>/<b>180</b> other than the broadcast computer <b>1410</b>) via the Internet <b>1430</b>. Server <b>140</b> retrieves pointers to the encoder agents from broadcasting computer <b>1410</b> and recipient computers <b>1420</b> that are running the encoder engines. Media engine <b>1400</b> (primarily constructed in C++ (ATL)) on server <b>140</b> acts as an administrator using Java Server Pages (JSP) sent by server <b>140</b>. Moreover, media engine <b>1400</b> utilizes DCOM (Distributed Component) to communicate (internal bridging is done with JSP) between server <b>140</b> and the remote computer (i.e., broadcasting and recipient computers <b>1410</b> and <b>1420</b>).

[0139] The agent locator can be global in scope and be available to media engine <b>1400</b> whenever the JSP page containing the locator is accessed. However, the encoder agent and the selected encoder engines have session scope. As a result, multiple encoder agents do not need to be created to handle multiple requests for encoder objects during a single session.

[0140] The system of the present invention also provides full duplex audio streaming components on server <b>140</b>. The components are primarily constructed in C++ (ATL). In order to control the flow of media streaming (i.e., direct the IP tunnel) to enable recipients to listen to the media stream, Java Server Pages (JSP) (in particular, listening.jsp as shown in FIG. 17<i>a</i>) are used by the system.

[0141]FIG. 15 depicts the audio and video streaming architecture in relationship to presenter computer <b>100</b>, participant computers <b>120</b> (authorized <b>120</b><i>a </i>and unauthorized <b>120</b><i>b</i>) and application server <b>140</b> (in particular, web server <b>155</b> and database <b>165</b>). When the presenter logs into web server <b>155</b>, login information including the presenters IP address and user name are provided to web server <b>155</b>. The login information allows the system to identify the presenter when speaking and provide a tunnel to the IP address of presenter computer <b>100</b>. On authorization, web server <b>155</b> recognizes the IP address of the authorized participant and pushes the control (see System Architecture section below) to the authorized participant computer <b>120</b><i>a </i>based on the IP address, which grants authorized participant computer <b>120</b><i>a </i>control over the IP tunnel.

[0142] Live media streaming is facilitated by the creation of an IP tunnel between presenter computer <b>100</b> and participant computers <b>120</b> through web server <b>155</b>. While web server <b>155</b> facilitates the IP tunnel, web server <b>155</b> does not process the live audio stream during presenter to participant audio/video communications.

[0143] In terms audio and video streaming there are three types of users—the presenter, authorized participant (or specialist) and unauthorized participants. The presenter has all the controls in default and can send and receive the media by default. Unauthorized participants can only receive the media stream and are prevented from transmitting a media stream.

[0144] Server <b>140</b> streams two basic types of media to users: on demand media files (i.e., clips) under the control of media server <b>195</b>, and live media under the control of media engine <b>170</b>. Both types of media streaming are discussed below.

[0145] On demand audio and video files are streamed to presenter computer <b>100</b> and participant computers <b>120</b> from media server <b>195</b>, while the clip information (i.e., metadata) is posted to and accessed from database <b>165</b> via web server <b>155</b> (see FIG. 1). Presenter computer <b>100</b> and participant computers <b>120</b> are connected with each other through core engine <b>175</b>. Thus, when presenter <b>100</b> requests an on demand audio/video clip from media server <b>195</b>, the request is processed by core engine <b>175</b>, which receives the request through web server <b>155</b>. Then, after required authentications using database <b>165</b>, core engine <b>175</b> sends the request to media server <b>195</b>, which streams the requested clip to presenter computer <b>100</b> and participant computers <b>120</b> where the resident media players render the streamed clip. Turning to live audio streaming, once the streaming media connection is established, the presenter and participants are free to collaborate audibly. The general process for the streaming audio collaboration is controlled by audio/video application <b>170</b> in conjunction with core engine <b>175</b> as depicted in FIG. 16. At broadcasting computer <b>1410</b>, voice input is received <b>1600</b> from a microphone (not shown) and is being encoded by encoder <b>1605</b>. Then, the audio stream is transmitted to media engine <b>1400</b> (contained within audio video application <b>170</b>), which pushes that stream to the user who sends the request (listening.jsp) for it using http/IP tunneling. The audio stream is then transmitted to recipient computer <b>1420</b> where the audio stream is optionally sampled <b>1615</b> for quality control of the audio signal, sent through a decompression algorithm <b>1620</b> performed by the codec, and then output <b>1625</b> to the listener on a speaker or other sound generation means (not shown). The streaming audio collaboration process depicted in FIG. 16 is described below in more detail.

[0146] More specifically, the system utilizes the following detailed processes for transmitting streaming live audio from broadcasting computer <b>1410</b> (i.e., the computer of a user that is speaking which may be presenter computer <b>100</b> or participant computers <b>120</b>):

[0147] 1. Server <b>140</b> under control of media engine <b>1400</b> activates the Microsoft Windows Media Encoder on broadcasting computer <b>1410</b>.

[0148] 2. The voice is captured from the sound card's microphone input (default audio device) of broadcasting computer <b>1410</b>.

[0149] 3. The voice/video is changed into data and vise versa by Marshing techniques.

[0150] 4. The data (voice) stream is converted into Advance Streaming Format (ASF).

[0151] 5. The data (voice) is then compressed to reduce its size of data (voice) with the help of Windows Media Audio Codec.

[0152] 6. The compressed stream is then transmitted from broadcasting computer <b>1410</b> on port <b>80</b>.

[0153] The system then utilizes the following process for receiving the streaming live audio at recipient computer <b>1420</b>:

[0154] 1. The Windows Media Player control is invoked by server <b>140</b> under control of media engine <b>1400</b> embedded in a Java Server Page (JSP) to recipient computer <b>1420</b> along with IP tunnel initiation.

[0155] 2. When the particular JSP is activated at recipient computer <b>1420</b>, an IP tunnel is automatically created with broadcasting computer <b>1410</b>, which is transmitting the audio stream on port <b>80</b>.

[0156] 3. When the IP tunnel is successfully created the embedded player in recipient computer <b>1420</b> starts rendering the audio.

[0157] 4. The buffer for the audio stream is first filled and then played.

[0158] The particular JSP is fully automated and automatically will create a new IP tunnel if the previous IP tunnel collapses or breaks-up due to any network issue in the Internet cloud between broadcasting computer <b>1410</b> and recipient computer <b>1420</b> (i.e., the computer transmitting the stream and the computer receiving the stream).

[0159] System Architecture

[0160] The system architecture is based upon the use of Java Applets, Java Servlets, and Java Server Pages (JSP) which provide the real time and highly functional interactive capabilities such as audio and video streaming allowing both the presenters and users the ability to introduce and react to visual and audio data instantaneously. A Java applet is a program executed from within another application. Applets and servlets are divided into classes, and within each class are data objects comprising fields (i.e., variables) and methods. Fields tell what an object is, and methods tell what an object does. Each class, which is the abstraction of an object, is developed to perform certain activities (i.e., one or more methods for carrying out a task). FIGS. 18<i>a</i>-<i>d </i>and <b>19</b>, which describe the main applets and servlets of the preferred embodiment, depict the key activities provided by the major classes and inner classes.

[0161] Unlike an application, applets cannot be executed directly from the operating system. With the growing popularity of OLE (object linking and embedding), applets are becoming more prevalent. A well-designed applet can be invoked from many different applications.

[0162] Web browsers, which are often equipped with Java virtual machines, can interpret applets locally from web servers. Because applets are small in file size, cross-platform compatible, and highly secure (can't be used to access users' hard drives if not signed), they are ideal for small Internet applications accessible from a browser and are very popular for development of thin client applications.

[0163] User Interface Architecture

[0164] Referring now to FIG. 17<i>a</i>, the overall system layout is shown detailing the relationship between server <b>140</b> side applications <b>1750</b> (comprising servlets <b>1752</b>, JSP's <b>1754</b> and conversion engine <b>145</b>), client <b>100</b>/<b>120</b> side applications <b>1760</b> (comprising applets <b>1762</b> and HTML pages <b>1764</b>), and client side browsers <b>1780</b>. Servlets <b>1752</b> of web server <b>155</b> control the push of applets <b>1762</b> to web browser <b>1780</b> of presenter computer <b>100</b> and participant computers <b>120</b>, as well as the access to database <b>165</b>. The client side applications <b>1769</b> facilitate the display of and user interaction with the graphical user interfaces depicted in FIGS. <b>2</b>-<b>7</b>.

[0165] Web browser applets <b>1762</b> pushed by web server <b>155</b> include four major applets: conference (ConfApp3) applet <b>1705</b>; queue (QueueApp) applet <b>1710</b>; whiteboard (White_Board) applet <b>1715</b>, and breakout applet <b>1720</b>. Conference applet <b>1705</b> is the main applet and its primary purpose is to provide conferencing functions. The primary purpose of queue applet <b>1710</b> is to provide threaded queue functions. Whiteboard applet <b>1715</b> is primarily responsible for drawing functions. Breakout applet <b>1720</b> is primarily responsible for breakout of a session into as many groups as desired.

[0166] As depicted in FIGS. 17<i>b </i>and <b>17</b><i>c</i>, applets <b>1762</b> are organized with respect to the client's web browser environment (see the graphical user interfaces depicted in FIGS. <b>2</b>-<b>7</b>). In particular, queue applet <b>1710</b> and breakout applet <b>1720</b> control the functions of control A console <b>200</b><i>a</i>, and conference applet <b>1705</b> and whiteboard applet <b>1715</b> control the functions of master communication console <b>200</b><i>c</i>. Each applet <b>1762</b> is responsible for certain functions on the graphical user interface. Queue applet <b>1710</b> controls the attendance, send, and authorization functions; breakout applet <b>1720</b> controls the breakout session function; conference applet controls the chat, polling, poll results, content, and audio/video clip and streaming functions; whiteboard applet <b>1715</b> controls the access to whiteboard <b>400</b> from main communication console <b>200</b><i>c </i>as well as the slide controls, authorization, annotation, and audio/video clip and streaming functions on whiteboard <b>400</b>. Questionnaire applet <b>1745</b> controls the dynamic questionnaire function for the session.

[0167] Web server <b>155</b> is constructed of several servlet applications <b>1762</b>. The major servlets include main <b>1725</b>, jointime <b>1730</b>, profile_test <b>1735</b> and intermed <b>1740</b>. The main servlet <b>1725</b> is primarily responsible for session initialization, user list refreshing, message writing and user disconnection activities carried out by web server <b>155</b>. These applets <b>1762</b>, servlets <b>1752</b>, as well as JSP's <b>1754</b> serve to facilitate the system functionality described in the User Interface, Advertisements, Automated PPT Conversion and Media Streaming Sections above. Jointime <b>1730</b>, profile_test <b>1735</b> and intermed <b>1740</b> servlets receive commands generated from various applets <b>1762</b>.

[0168] HTML pages <b>1764</b> provide the viewable portion of the graphical interface on web browser <b>1780</b> such as the presentation of ads <b>1762</b>. In comparison, applets <b>1762</b> provide control functions for the graphical user interface on web browser <b>1780</b>. JSP's <b>1754</b> provide many server operations to enable the graphical user interface to publish dynamic contents, for example, calculating details of questionnaire results, listing archived sessions, and many more supporting utilities.

[0169]FIG. 17<i>b </i>depicts the client side web browser environment for the graphical user interface on a presenter computer <b>100</b>, while FIG. 17<i>c </i>depicts the client side web browser environment for the graphical user interface on a participant computer <b>120</b>. When compared, participant computer <b>120</b> does not receive breakout applet <b>1720</b>, since participants do not have the ability to initiate break out sessions. Additionally, participant computers <b>120</b> only have conditional presentation slide control, i.e., only when authorized by the presenter. The same conditional control applies to microphone selector <b>260</b> on participant computers <b>120</b>.

[0170] Conference Applet <b>1705</b>

[0171] Referring now to FIG. 18<i>a</i>, shown is a block diagram detailing the major activities of conference applet <b>1705</b> broken down by class. Conference applet <b>1705</b> is comprised of the following principle classes: ConfApp3 class <b>1830</b> and ConfApp3$Run class <b>1836</b>. Other classes are provided for creating the logout dialog window, showing the dialog window and creating a canvas (20×20 pixels) for hand raising icon <b>485</b>. The principle classes and their respective activities are discussed below.

[0172] ConfApp3.Class <b>1830</b>

[0173] ConfApp3 class <b>1830</b> is the main applet class. It creates a separate thread (for each session) to communicate with the server. The class includes an initialize activity <b>1832</b>, which initializes the applet layout, retrieves references to queue <b>1710</b> and whiteboard <b>1715</b> applets and starts the thread run activity to contact main servlet <b>1725</b>. Check button activity <b>1834</b> handles the buttons and sets the ready flag on if a user message is ready to be sent.

[0174] Other activities (not shown in FIG. 18<i>a</i>) provided by ConfApp3 class <b>1832</b> include laying out the components (text boxes and buttons) on the screen, checking whether the user is a presenter or a participant, obtaining the reference of other applets (i.e., queue applet <b>1710</b> and whiteboard applet <b>1715</b>) in the page, obtaining a reference to the other applets in case reference could not be obtained during initialization, handling the button clicking events, mouse events, prefixing messages according to the button pressed (i.e., it sets the message prefix to “Ans:” or “Que:”, if the button pressed has the label “Answer” or “Question”), displaying an error message if a button is pressed but no text has been typed in the text box and the button requires some textual message, alerts, informing server <b>140</b> that the user has left so that the attendance be updated and other users in the session informed and assigning a different color to every new participant who whispers.

[0175] ConfApp3$Run Class <b>1836</b>

[0176] ConfApp3$Run class <b>1836</b> is an inner class, which executes in a separate thread and communicates with main servlet <b>1725</b>. It checks queue applet <b>1710</b>, whiteboard applet <b>1715</b>, and the instant applet for messages to send. If no messages are ready, then the applet sends only a message ID and retrieves messages from main servlet <b>1725</b>. It also passes the user list (i.e., the names in audience list box <b>202</b>) to queue applet <b>1710</b> and any drawing board related messages to whiteboard applet <b>1715</b> and displays other messages in conference applet <b>1705</b>. These functions are repeated every 100 milliseconds (in real time).

[0177] Other activities provided by ConfApp3$Run class <b>1836</b> (not shown on FIG. 18<i>a</i>) include displaying the user names in audience list box <b>202</b>, informing the users about any newcomers or departing users, parsing the whisper message string and displaying it on the message bar in the color associated with the whisperer, and displaying messages in appropriate text boxes or opening up whiteboard <b>400</b> depending on the message type.

[0178] Queue Applet <b>1710</b>

[0179] Referring now to FIG. 18<i>b</i>, shown is a block diagram detailing the major activities of queue applet <b>1710</b> broken down by class. The primary class of queue applet <b>1710</b> is Queueapp class <b>1840</b>, which has three key activities: initialize <b>1842</b>, check buttons and mouse events <b>1844</b>, and run thread <b>1846</b>. Additionally, the activities of this class maintain the users, hand-raisers, and whispering user lists. Apart from those lists, the activities in the class provide controls to authorize and unauthorize the participants as well as opening files and websites to the participants.

[0180] Initialize activity <b>1842</b> lays out the users and hand raisers' lists and check the user type (i.e., presenter, participant or specialist). If the user is a presenter applet <b>1710</b> presents other controls like authorize buttons, unauthorize buttons, file and web-site opening text boxes and buttons as well as break out session buttons. In case of a participant, queue applet <b>1710</b> displays the users and hand-raisers lists only. Run thread activity <b>1846</b> creates a new thread to check the break out session in case a presenter creates one. Check buttons activity <b>1844</b> monitors the button selections and sets the message variables accordingly.

[0181] Queue applet <b>1710</b> keeps the reference of breakout applet <b>1720</b> and conference applet <b>1705</b> keeps the reference of queue applet <b>1710</b>. This inter-applet communication is facilitated by variables whose values are shared by the applets.

[0182] An inner class of QueueApp class <b>1840</b> (not shown in FIG. 18<i>b</i>) provides the activities for creating the popup dialog box for polling, which is called from conference applet <b>1705</b> when the presenter presses poll button <b>246</b>, laying out the polling dialog box with buttons and a text box, responding to the buttons and depending on the button pressed makes the dialog box invisible.

[0183] Queue applet <b>1710</b> utilizes a number of key variables, which are monitored by conference applet <b>1705</b> thread to send messages to web server <b>155</b> which in response pushes applets to presenter computer <b>100</b> and participant computers <b>120</b>. By way of example, the presenter may authorize a participant to ask a question. A request is sent from presenter computer <b>100</b> via Internet <b>130</b> to server <b>140</b>, which processes the request and generates an applet, which is transmitted to presenter computer <b>100</b> and participant computers <b>120</b>.

[0184] Whiteboard Applet <b>1715</b>

[0185] Referring now to FIG. 18<i>c</i>, shown is a block diagram detailing the major activities of whiteboard applet <b>1715</b> broken down by class. Whiteboard applet <b>1715</b> includes the following classes:

[0186] White_Board.Class <b>1850</b>

[0187] White_Board class <b>1850</b> is the main class and includes several key activities: initialize <b>1856</b> to create the instance of whiteboard <b>400</b> and get the context of conference applet <b>1705</b> and queue applet <b>1710</b>, getslides <b>1858</b> to get the slides from server <b>140</b> according to the session presentation information, and paint <b>1860</b> to draw the heading information surrounded by a box on the top of whiteboard <b>400</b>. Important activities of White_Board class <b>1850</b> (not shown in FIG. 18<i>c</i>) include setting the size of the applet and opening a URL connection with server <b>140</b>.

[0188] MyCanvas Class <b>1852</b>

[0189] MyCanvas class <b>1852</b> provides several activities including mycanvas <b>1862</b> for laying out whiteboard <b>400</b>, drawall <b>1864</b> for drawing annotations, and createimage <b>1866</b> for creating and displaying images form the byte stream (i.e., image stream), actionperformed <b>1868</b> for handling all button events (i.e., selections by the user), and mousehandler <b>1878</b> for handling all mouse events such as tracking the mouse's start and end points and mouse movements when the presenter or authorized use draws on whiteboard <b>400</b>.

[0190] Additionally, inner classes of MyCanvas class <b>1852</b> (not shown in FIG. 18<i>c</i>) provide many activities such as closing of the text dialog, displaying alerts, displaying a text box, displaying hand icon <b>485</b> on presenter whiteboard <b>400</b><i>a </i>when participant presses raise-hand button <b>480</b>, displaying the rollover buttons and annotation buttons, calling the tooltip class to display the tool tips when the mouse moves over the annotation buttons, performing the navigation action of slides for the next and previous rollover buttons, adding the insets (borders) in the layout of whiteboard <b>400</b> to set its look and feel, and overriding the paint method for displaying the panels in light gray colors. ToolTip class is an external class used for displaying the tool tips on annotation (icon) buttons to make them more meaningful.

[0191] Point, Drawing, SessionArchive Classes <b>1854</b>

[0192] Point class is a simple utility class used to represent any point (represented by an x-position and y-position) on whiteboard <b>400</b> and return the points for annotations. Drawing class is used to display the annotations on whiteboard <b>400</b>. SessionArchive class is used to fetch the slide archives from server <b>140</b> and stream the archive string to server <b>140</b> to be stored in encoded format.

[0193] These classes provide a number of activities including: point <b>1870</b> to create an instance of an annotation, drawings <b>1872</b> to draw the annotations, toString <b>1874</b> to return variables for each annotation, and sessionarchive <b>1876</b> to send and receive archives of annotations with slides (complete presentation archiving) to server <b>140</b> for later use.

[0194] BreakOut Applet <b>1720</b>

[0195] Referring now to FIG. 18<i>d</i>, shown is a block diagram detailing the major activities of breakout applet <b>1720</b> broken down by class. This applet is comprised of the following primary classes: BreakOut class <b>1880</b> and BreakOut$BreakFrame$DialogWin class <b>1882</b>. BreakOut class <b>1880</b> is an entry point to the session breakout dialog window and one of its inner classes creates the session break out dialog window. The class provides initialize activity <b>1884</b> to create instances of the breakout dialog window.

[0196] BreakOut$BreakFrame$DialogWin class <b>1882</b> is the main class which actually controls the session breakout management. Initialize activity <b>1886</b> lays out the breakout management dialog window. An audience list activity initializes the audience list of the session and holds the names in a vector for future use in the session.

[0197] ActionPerformed activity <b>1888</b> handles the buttons and takes appropriate actions. If the “Create” button is selected, a new breakout session is created from the available audience list. If the “Switch User” button is selected, participants are switched from one breakout session to another and the list of sessions is displayed by calling fillchoices activity <b>1892</b>. In this case, if any session becomes empty (i.e., no participants) it is no longer listed. If the “OK” button is selected, Handletask activity <b>1890</b> is called to carry out the task (based on the task (button) selected first). If the “cancel” button is pressed, the initiated task is cancelled and the starting screen is displayed.

[0198] Handletask <b>1890</b> is called actionPerformed <b>1888</b> upon selection of the “OK” button, which carries out the task according to the task (button) selection and updates the breakOutString variable being monitored by queue applet <b>1710</b> and changes the layout of the dialog to the starting screen.

[0199] ItemStateChanged activity <b>1892</b> controls the lists (combo boxes) of breakout sessions and users in each list and calls activities to get the user lists and fillChoices <b>1892</b>. FillChoices activity <b>1894</b> simply fills the lists (combo boxes) with available sessions and names in the main session and calls getListOfUsers method.

[0200] Main Servlet <b>1725</b>

[0201] Referring now to FIG. 19, shown is a block diagram detailing the major activities of main servlet <b>1725</b> broken down by class. Main Servlet <b>1725</b> is comprised of the following primary classes: tSer class <b>1900</b>; tSer$SessionMessages class <b>1905</b> and tSer$Polling class <b>1910</b>. TSer class <b>1900</b> is the main servlet class which controls all the conferencing in text and drawings. The tSer$SessionMessages class <b>1910</b> objects control and hold the session messages. tSer$Polling class <b>1910</b> (via initialize activity <b>1960</b>) creates the polling object for the different sessions. Breakout sessions are tracked with a session number passed as a parameter. Each break out session number is negative with the session id encoded in that number.

[0202] Tser class <b>1900</b> allows main servlet <b>1725</b> to initialize sessions <b>1915</b>, refresh user lists <b>1930</b>, write files <b>1935</b> and delete names of disconnected users activity <b>1940</b>. Delete activity <b>1940</b> deletes the user name from the attendance list whose IP and session id is passed to it when the user's connection is lost or the user logs out of the session.

[0203] Upon initializing <b>1915</b>, main servlet <b>1725</b> connects with database <b>165</b> and gets the list of users in the audience table. It also creates a thread to remove the users with a lost connection from the audience table.

[0204] The run thread activity <b>1925</b> checks the connection time of all users every <b>100</b> seconds and deletes the user name from the audience list who has not connected for 5 minutes and refreshes the audience list by calling delete activity <b>1940</b>.

[0205] Once the connection is established, the audience list is refreshed by refresh list activity <b>1930</b> and write file activity <b>1935</b> is called to write the messages to the appropriate files, for the session id passed as a parameter, depending on the info type passed (question, answer or comments) for archiving the messages.

[0206] Service activity <b>1920</b> checks the audience list for illegal entries, records connection times of users, updates the polling table with the polling info passed to it for the particular session, and provides other service oriented functions. In more detail, service activity <b>1920</b> performs the following tasks in a stepwise manner:

[0207] 1. It finds the connecting users IP address.

[0208] 2. It refreshes the attendance list if the last refreshing has elapsed 10 seconds.

[0209] 3. It checks the user in the attendance list.

[0210] 4. If the user is not found in the list, it sends the message of “re-logging” or “wait” to the user.

[0211] 5. If the user is found in the attendance list, it performs the following tasks and checks:

[0212] a. Finds the presenter of the user's session and adds it to the send message string.

[0213] b. Finds the session number of the user.

[0214] c. Reads the incoming message and takes appropriate action.

[0215] d. If the message starts with “Slide” it call the activity to get slide information and returns the presentation slides info to the user.

[0216] e. Finds the session information and adds it to the send message string.

[0217] f. Finds the users in the session of the connecting user and adds it to the send message string.

[0218] g. Checks the whisper message for the connecting user and adds it to the send message if any.

[0219] h. Finds the client's message number from its message.

[0220] i. Checks the connecting user's session messages, if he/she is lagging behind by at least two messages then creates a whisper message for the presenter of the same session if available. Updates the connecting user's connection time for later reference.

[0221] j. If the connecting user's message starts with “Left”, it calls delete activity <b>1940</b> to delete his/her name from the attendance list. It also updates the session messages of this user's session by removing any invalid messages that identify that the user has raised the hand or that the user is authorized. It also sends the user a “Bye” message.

[0222] k. If the connecting user's message starts with “Poll”, it creates a “Polling” object for this session if not available. It updates polling info into the same object and the database.

[0223] l. If the connecting user's message starts with “Hum” (whisper message), it updates whisper messages for the user to whom this message is targeted. It also adds NoDataHeader to the send message string.

[0224] m. If the message starts with “Mes:” means the user has not sent any message but only message id (number). User's message id is compared with the message number of the session messages object for the same session. If the user is lagging behind then all the messages are retrieved for this user and message id is changed to the new number. Otherwise, NoDataHeader is attached to the send message string.

[0225] n. If the connecting user's message starts with “Break”, it calls the breakout session method to change the attendance table. It also attaches the NoDataHeader to the send message string.

[0226] o. If the message starts with “Auth”, then the name of the authorized user is found from the attendance list. This message is modified and IP of the authorized user is attached to it. The session messages object, for the connecting user's session is updated with this new message. If the message starts with “Pre:”, “Que:” or “Ans:”, write file activity <b>1935</b> is called for archiving of this message.

[0227] p. Finally, the messages are retrieved from the session messages object and attached to the send message string.

[0228] 6. The send message is sent to the connecting user.

[0229] Other activities of tSer class <b>1900</b> are provided to interrupt the thread and close the database connection when main servlet <b>1725</b> is unloaded and retrieve the presentation slides info from web published folder <b>1210</b>.

[0230] Some important variables used in tSer class <b>1900</b> include attendanceTime variable which holds the time of the last attendance refresh; whispers variable used to hold the whisper messages of the users; allPolls variable which holds the Polling information of the session; sessionsinfo variable which holds the session info for the each on going session; connectionTime variable which holds the last connection time of the users; sessMessages variable which holds the session messages objects of the on going sessions; and Attendees variable which holds the list of users in the attendance list.

[0231] The activities of tSer$SessionMessages class <b>1905</b> include retrieve messages activity <b>1645</b> which compares the lastMessage ID of the connecting user and retrieves all unsent (maximum 5) messages from the session messages object, add message activity <b>1950</b> which adds the new message from the user to the collection of messages of this session, get message count activity (not shown) which returns the last message number of the session in question, and refresh messages activity <b>1955</b> which is called when the user leaves the session so that any message related to him or her can be deleted.

[0232] The activities of tSer$Polling class <b>1910</b> include holding the poll message; holding the count of “yes”, “no”, and “unsure” responses; and holding the count of polled questions in the session.

[0233] Referring now to FIG. 20, shown is a block implementation diagram detailing multiple user connections in a load-sharing environment. Users <b>2000</b> (i.e., presenter computer <b>100</b>, specialist computer <b>180</b> and participant computers <b>120</b>) are connected to SSL/VPN (Secure Socket Layer/Virtual Private Network) <b>2010</b> through an Internet service provider (ISP) <b>2005</b>. Server Cluster <b>2015</b> is a collection of individual servers <b>2020</b>, can be clustered to share the load of number of sessions running at the same times (i.e., multiple server tiers <b>140</b>), which carry out the various functions of the system.

[0234] Much of the system architecture is built upon Java servlets, Java applets, JSP, HTML and Java script for controlling the system. FIG. 21 depicts the construction of various application controls of the system, which are divided into communication controls <b>2110</b>, session management control <b>2120</b>, and reporting and additional controls <b>2130</b>. The sub-components under each category correspond to the various functions provided to the user though the graphical user interfaces depicted in FIGS. <b>2</b>-<b>7</b> and facilitated by the system architecture as depicted in FIGS. <b>17</b>-<b>19</b>.

[0235] Whiteboard Architecture

[0236] Turning to the remaining figures. FIG. 22 is a block diagram detailing whiteboard application <b>150</b> of the system architecture. As shown, a request <b>2210</b> is made for a particular slide show (in the form of a slide stream) from a particular presentation via core engine <b>175</b> (whiteboard applet <b>1715</b> on the client side communicates with main servlet <b>1725</b> on the server side) to web published folder <b>1210</b> in web server <b>155</b>. Whiteboard application <b>150</b> then determines if the slide was found <b>2220</b>. If the slide stream is found, the requested slide stream is received and decoded <b>2220</b> by whiteboard application <b>150</b> into an image stream, which avoids caching by participant computers <b>120</b> and prevents participants from saving or accessing the presentation at the end of the session. This helps ensure the confidentiality and protect the presentation from unregulated dissemination. Whiteboard application <b>150</b> then pushes <b>2240</b> the slide image to participant computers <b>120</b> for display on whiteboard <b>400</b>.

[0237] In summary, the whiteboard presentation process is carried out by whiteboard applet <b>1715</b>, which sends the request to server <b>140</b> for a particular slide. Server <b>140</b> takes that request and searches for it in web published folder <b>1210</b>. Once found, server <b>140</b> converts the image into an image stream and sends that stream to the session presenter and all connected session participants. Then whiteboard application <b>150</b> (locally runs on each machine as whiteboard applet <b>1715</b>) converts the image stream back into an image and displays it on whiteboard <b>400</b>. The cycle then repeats for each slide as the presenter proceeds through the slide show presentation. To enhance performance, once a slide is decoded and loaded into virtual memory (but not cached), the next request for that same slide (e.g., the presenter back tracks in the presentation) reads the slide from memory not from web published folder <b>1210</b>.

[0238] The foregoing discussion discloses and describes merely exemplary methods and embodiments of the present invention. One skilled in the art will readily recognize from such discussion that various changes, modifications and variations may be made therein without departing from the spirit and scope of the invention. Accordingly, disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims and their legal equivalents.