Strategies for transmitting in-band control information

TECHNICAL FIELD

0001This subject matter pertains to strategies for transmitting control information, and, in a more particular implementation, to strategies for transmitting control information from a host to a client in a media presentation environment.

BACKGROUND

0002Computers are becoming an increasingly popular mechanism for presenting media content information, such as audio information and video information. For instance, a user can receive content information from a remote source using a personal computer in the user's home that is coupled to the remote source via a network. The user may receive the content information as a complete file or in piecemeal streaming fashion. Alternatively, if the computer includes a tuner mechanism, the user may receive content information from conventional broadcast sources (such as cable or satellite sources) by tuning to these sources. The user may forward such content information to one or more appropriate playback devices in the home, such as a television or stereo system. Microsoft Corporation's Media Center technology provides one exemplary suite of tools for receiving and presenting media content information in the above-described manner. Using other tools, the user may couple multiple playback devices in the home into a presentation network. The user can then transfer media information from one device to another within the home. Universal Plug and Play (UPnP) technology provides one suite of tools for conveniently setting up such a home network.

0003While these developments offer many interesting enhancements over the conventional presentation of media information, they also present a number of new challenges. For instance, in addition to transmitting media content information between computers, it may also be necessary to transmit control information. For instance, a host device may forward control information that instructs a receiving device how it should process the media content information. This can become problematic in a networked environment. For instance, networking and processing anomalies may result in cases where the receiving device receives media content information without its associated control information, or vice versa.

0004There is accordingly a need for more efficient techniques for transmitting control information in any environment, such as the above-described media presentation environment.

SUMMARY

0005According to one exemplary implementation, a method is described for streaming content information from a host module to a client module. The method comprises: (a) forming, at the host module, units of content information for inclusion in a stream to be transmitted to the client module; (b) forming, at the host module, at least one unit of control information for inclusion in the stream; (c) inserting, at the host module, said at least one unit of control information among the units of content information, wherein the position of said at least one unit of control information vis-à-vis the units of content information conveys instructions regarding the manner in which the client module is to operate upon receiving the control information; and (d) transmitting, by the host module, the stream containing the units of content information and said at least one unit of control information to the client module, wherein the host module dynamically varies the control information that it inserts in the stream based on actions taken by a user who controls the playback of the stream.

0006Additional exemplary implementations are described in the following.

BRIEF DESCRIPTION OF THE DRAWINGS

0007<figref idref="DRAWINGS">FIG. 1</figref> shows an exemplary system for implementing aspects of the features summarized above.

0008<figref idref="DRAWINGS">FIG. 2</figref> shows a more detailed block diagram of selected components of the system of <figref idref="DRAWINGS">FIG. 1</figref>.

0009<figref idref="DRAWINGS">FIG. 3</figref> shows an exemplary series of packets that can be transmitted using the system of <figref idref="DRAWINGS">FIG. 1</figref>, where the packets include control information dispersed among content information.

0010<figref idref="DRAWINGS">FIGS. 4-6</figref> show three different use scenarios in which the control information conveys different respective instructions pertaining to the content information.

0011<figref idref="DRAWINGS">FIG. 7</figref> shows an exemplary procedure in flowchart form for creating a stream including both content information and control information.

0012<figref idref="DRAWINGS">FIG. 8</figref> shows an exemplary procedure in flowchart form for interpreting received control information in the stream produced in <figref idref="DRAWINGS">FIG. 7</figref>.

0013<figref idref="DRAWINGS">FIG. 9</figref> shows an exemplary computing environment for implementing aspects of the system shown in <figref idref="DRAWINGS">FIG. 1</figref>.

0014The same numbers are used throughout the disclosure and figures to reference like components and features. Series <b>100</b> numbers refer to features originally found in <figref idref="DRAWINGS">FIG. 1</figref>, series <b>200</b> numbers refer to features originally found in <figref idref="DRAWINGS">FIG. 2</figref>, series <b>300</b> numbers refer to features originally found in <figref idref="DRAWINGS">FIG. 3</figref>, and so on.

DETAILED DESCRIPTION

0015The following describes exemplary mechanisms and procedures for transmitting control information from a host module to a client module. The mechanisms and procedures send the control information in-band, that is, in the same communication band as the media content information. More specifically, the mechanisms and procedures position the control information in a stream of content information to establish a nexus between the control information and the content information.

0016The control information may convey any information regarding the content information. For instance, where the control information is inserted at the end of a stream, it may convey end of stream information that identifies the end of the stream. Where the control information is inserted at the start of stream, it may convey information regarding a stream which follows the control information in time. For instance, in this scenario, the control information may indicate that the following stream should be processed using a prescribed decryption key, or using prescribed rendering resources at the client module. Where the control information is positioned between two streams of content information, it may convey information regarding the relationship of one stream to another. For instance, in this scenario, the control information may <b>1</b> indicate that there is a discontinuity between the two streams. There are additional applications of this control information transfer paradigm, to be described below.

0017As to terminology, the term “media content information” (or “content information” for brevity) can pertain to any kind of resources that can be consumed by a user in any format, such as audio resources (e.g., music, etc.), still picture resources (e.g., digital photographs, etc.), moving picture resources (e.g., television programs, movies, etc.), computer programs (e.g., games, etc.), markup language resources (e.g., hypertext markup language resources received via a wide area packet network), and so on. The information can be expressed in analog form or digital form or a combination of analog and digital forms. The information can also include (or can omit) interactive content (as in the case with computer games).

0018The term control information refers to any information which has any kind of bearing on the content information. For example, the control information may describe some property of the content information, or may describe the behavior of some processing to be performed on the control information. The term “control event” refers to any kind of event which invokes the creation of control information. A user may trigger such an event by invoking express input actions (e.g., using a remote control or other input mechanism). Or the system may automatically generate a control event when certain conditions are met. The term “control processing task” refers to an operation that is performed upon the occurrence of a control event.

0019This disclosure includes the following sections. Section A describes an exemplary system for implementing the features summarized above. Section B describes an exemplary flowchart which shows the operation of the system of Section A. And section C describes an exemplary computer environment for implementing the system of Section A.

0020A. Exemplary System

0021A.1. Overview of System

0022Generally, any of the functions described herein can be implemented using software, firmware (e.g., fixed logic circuitry), manual processing, or a combination of these implementations. The terms “module,” “functionality,” and “logic” as used herein generally represent software, firmware, or a combination of software and firmware. In the case of a software implementation, the terms “module,” “functionality,” or “logic” represent program code that performs specified tasks when executed on a processing device or devices (e.g., CPU or CPUs). The program code can be stored in one or more fixed and/or removable computer readable memory devices. The features of the techniques described below are platform-independent, meaning that they can be implemented on any commercial computing platform or networking strategy.

0023<figref idref="DRAWINGS">FIG. 1</figref> shows an exemplary system <b>100</b> for performing the operations described above. The system <b>100</b> includes a host module <b>102</b> coupled to a client module <b>104</b> via a communication mechanism <b>106</b>. Among other roles, the host module <b>102</b> serves to provide media content information and control information to the client module <b>104</b> for presentation thereat. The host module <b>102</b> may obtain the media content information from any source <b>108</b>. The client module <b>104</b> may present the content information—in accordance with the control information—on any presentation device <b>110</b>. A user <b>112</b> may interact with the client module <b>104</b> and/or the host module <b>102</b> via any kind of input mechanism, such as input module <b>114</b> (such as a remote control device or a keypad associated with the client module <b>104</b> or the presentation device <b>106</b>). <figref idref="DRAWINGS">FIG. 1</figref> additionally shows that one or more other client modules <b>116</b> may receive content information from the host module <b>102</b> and present such information on one or more other presentation devices <b>118</b>.

0024The infrastructure shown in <figref idref="DRAWINGS">FIG. 1</figref> can be applied to many different environments. In one case, the host module <b>102</b> represents a general purpose computer device within the home of the user <b>112</b> (or within some other local environment). The host module <b>102</b> can receive content information from any source <b>108</b>, such as a local source (e.g., as implemented by a hard drive of the computer device, a local video jukebox, a local video camera, a local microphone, and so forth), or from a remote source. One remote source can represent traditional television and radio broadcast sources, such as a traditional wired sources (e.g., cable) or traditional wireless sources (such as an earthbound antenna or a satellite). Another remote source can represent a server or like device coupled to the host module <b>104</b> via a network, such as a TCP/IP network (e.g., the Internet). Still other kinds of sources can provide content information to the host module <b>102</b>.

0025Any kind of business arrangement can govern the dissemination of content information to the host module <b>102</b>. The sources <b>108</b> can distribute the resource information on a fixed time schedule or in an on-demand fashion. The sources <b>108</b> can charge a fee to receive the content information, or can distribute this information free of charge.

0026Likewise, the content information itself can have many forms. The content information may represent live content information or pre-recorded content information. The content information can have an audio component and/or a visual (video) component and/or an interactive component. The content information can represent static information (as in the case of one or more photographs), or “moving” information (such as in video). The content information can be expressed in any format, such as MPEG-1, MPEG-2, or WMV for video information (among other formats), and MP3, WMA, or WAV (among other formats) for music information. The content information can be expressed in digital form, analog form, or a combination of analog and digital forms. Still other kinds of source formats can be received.

0027In one exemplary implementation, the client module <b>104</b> can represent another kind of computer device located in the user <b>112</b>'s home. For instance, the client module <b>104</b> may represent another general purpose computer device. Or the client module <b>104</b> can represent a special-purpose computer device, such as a game console or an extension console designed for the main purpose of receiving content information from the host module <b>102</b> (and thereby “extending” the number of output devices that the host module <b>102</b> can direct content information to within the home). Or the client module <b>104</b> can represent logic functionality integrated within the presentation device <b>110</b> itself.

0028In an exemplary home application, the user <b>112</b> may have situated several of the client modules <b>104</b> in different respective rooms of the home, which in turn are coupled to different media presentation devices located in these rooms. Each of these client modules can be configured to receive content information from the host module <b>102</b>. Where these client modules <b>104</b> are implemented as “extensions” to the host module <b>102</b>, they may be configured to run several instances of the functionality provided by the host module <b>102</b>. The system <b>100</b> can be further configured to allow concurrency in use among the separate components of the system <b>100</b>. For instance, a first user may use the host module <b>102</b> to perform a first task while a second user uses the client module <b>104</b> to perform a second task, without necessarily interfering with the first user.

0029The presentation device <b>110</b> can represent any type of device whereby a user can consume the content information. Possible types of presentation devices <b>110</b> include televisions, radios, stereo systems, computer monitors, and so forth.

0030The communication mechanism <b>106</b> can represent any conduit for transmitting information from the host module <b>102</b> to the client module <b>104</b>. In a local environment, the communication mechanism <b>106</b> can be implemented as a Local Area Network (LAN), such as an Ethernet network. Any protocol or combination of protocols can be used to forward information from the host module <b>102</b> to the client module <b>104</b>, such as the TCP/IP protocols. The communication mechanism <b>106</b> can be physically implemented using any combination of physical components, such as various wired communication paths (cooper wire, power line, fiber optic, etc.) and various wireless communication paths. Although not shown, the communication mechanism <b>106</b> can also incorporate various routers, name servers, gateways, and so forth.

0031The above-described environment is exemplary and non-limiting. In another application, the host module <b>102</b> can represent a server type of computer or a peer computer that is remote with respect to the client module <b>104</b>. For instance, the host module itself <b>102</b> can represent a server computer outside the home that provides content information to the client module <b>104</b> over the Internet or some other network. Still other applications are possible; the host module <b>102</b> is to be understood as any source of content information wherever situated and however implemented, and the client module <b>104</b> is to be understood as any recipient of content information wherever situated and however implemented. <figref idref="DRAWINGS">FIG. 9</figref>, to be discussed below in turn, provides further details regarding one implementation of the host module <b>102</b> or the client module <b>102</b> using an appropriately configured general purpose computer device (e.g., a personal computer, etc.) or special purpose computer device (e.g., a game console, an extension device, etc.)

0032With the above overview, attention will now be directed to the individual exemplary components of the host module <b>102</b> and the client module <b>104</b>.

0033To begin with the host module <b>102</b>, a reception module <b>120</b> receives information from the information source <b>108</b>. The reception module <b>120</b> can represent a tuner which tunes to a physical frequency associated with the information source <b>108</b>, or can represent multiple tuners that can simultaneously tune to multiple sources. Alternatively, or in addition, the reception module <b>120</b> can represent a network interface module (such any kind of modem) which receives content information from a digital network source.

0034A transfer processing module <b>122</b> performs a variety of processing tasks on the content information to format it for transmission over the communication mechanism <b>106</b> to the client module <b>104</b>, and then transmits such information. <figref idref="DRAWINGS">FIG. 2</figref> describes an exemplary composition of the transfer processing module <b>122</b>. Suffice it to say at this juncture in the discussion that the transfer processing module <b>122</b> can filter and/or analyze the content information to prepare it for transmission, adjust the bit rate of the information, assemble the information into packets, multiplex the packets into a transmission stream for output to the communication mechanism <b>106</b>, and so forth.

0035<figref idref="DRAWINGS">FIG. 1</figref> represents the transmission of information content as stream <b>124</b>. A stream refers to the transmission of content information in piecemeal fashion, such that the client module <b>104</b> can receive and render part of the content information without receiving the entire body of such information (as opposed to the transmission of a file containing the complete content information). The stream <b>124</b> can include a plurality of units <b>126</b> (e.g., packets) that primarily convey the content information, that is, in the case of an A/V resource, the actual audio and visual data. The stream <b>124</b> can also include one or more control units <b>128</b> (e.g., packets) dispersed in the stream of content information <b>126</b>. The position of the control packets <b>126</b> within the stream <b>124</b> conveys information regarding how the control information contained therein is to be applied to the content information. (The “position” may be reflected by the sequence numbers assigned to the packets within the stream <b>124</b>, rather than the physical ordering of packets in an actual transmitted stream; this is because, in a packet network, it can be expected that some packets will be received “out of order,” requiring the client module <b>104</b> to reassemble them in the proper order.) For instance, the exemplary control packet <b>128</b> may convey the fact that the content information stream that preceded it in time has come to an end. Or the exemplary control packet <b>128</b> may convey the fact that prescribed processing behavior should be applied to the content information <b>126</b> which follows it in time. Or the exemplary control packet <b>128</b> may convey that there is a discontinuity between the content information which precedes it and the content information <b>126</b> which follows it. Still other applications and interpretations of the control packet <b>128</b> are envisioned. Section A.2 (below) provides further details regarding the generation and processing of control information.

0036A player control module <b>130</b> controls the components in the host module <b>102</b>. For instance, the control module <b>130</b> can set up a communication session with the client module <b>104</b> and then control the state of the client module <b>104</b> across multiple potential player instances. This module <b>130</b> can also play a role in coalescing control processing tasks that meet certain criteria (as will be described below).

0037The player control module <b>130</b> can also forward control instructions to the client module <b>104</b> via a separate communication path <b>132</b>. Thus, the control information sent on path <b>132</b> supplements the control information sent in-band within the stream <b>124</b> (or, in another interpretation, it can be said that the control information sent in-band supplements the control information forwarded on communication path <b>132</b>). According to one exemplary design paradigm, the control information can be sent in-band in those circumstances where it is important to convey the timing (or positional alignment) of this information vis-à-vis the content information. Exemplary circumstances in which in-band transmission of control information is appropriate are described with reference to <figref idref="DRAWINGS">FIGS. 4-6</figref> (to be discussed below in turn). The communication path <b>132</b> can be used to communicate control event information which does not need to be conveyed in as precise a temporal/positional manner as the in-band control information. Generally, different applications can use different communication channels to communicate different control events depending on the unique requirements of these applications. The communication path <b>132</b> can employ a different communication mechanism and/or protocol than used by the communication mechanism <b>106</b>, or it can use the same mechanism and protocol.

0038Still additional control paths can be included. For instance, the host module <b>102</b> can use another control path (not shown) to control the display of graphical and metadata-related information on the client module <b>104</b>. For instance, this control channel can be used to coordinate the display of various menus, program bars, channel information, and so forth on the presentation device <b>110</b>. One exemplary and non-limiting protocol that can be used to accomplish this task is the Remote Desktop Protocol (RDP). This protocol allows the system <b>100</b> to essentially “project” the graphical functionality associated with the host module <b>102</b> onto the presentation device <b>110</b> via the client module <b>104</b>. However, this strategy is exemplary rather than limiting; other techniques can be used to forward graphical information to the presentation device <b>110</b>. Generally, metadata-related information and graphical information can originate from the host module <b>102</b>, and/or the client module <b>104</b>, and/or the presentation device <b>110</b>, and/or some other module; further, any module or combination of modules can be used to coordinate the display of this metadata-related information and graphical information.

0039Finally, the host module <b>102</b> can include a number of other modules <b>132</b> for performing other unspecified tasks (that are not relevant to the focus of this discussion).

0040Now addressing the components of the client module <b>104</b>, the client module includes a reception processing module <b>134</b> for receiving the stream <b>124</b> and performing processing on the stream. <figref idref="DRAWINGS">FIG. 2</figref> provides additional details regarding this component <b>134</b>. Suffice it to say at this juncture of the discussion that this component <b>134</b> can demultiplex the information within the stream <b>124</b> and extract the information contained therein. The packets can include ID information which identifies their composition—e.g., whether they contain audio information, video information, or control information. In the event that a packet includes control information, the reception processing module <b>134</b> performs one or more control operations based on the control information. Further, the reception processing module <b>134</b> forwards the content information down to appropriate renderers for presentation of this information. Further, the reception processing module <b>134</b>, as well as the client control module <b>136</b> (to be described below), can control the renderers to perform various operations. The reception processing module <b>134</b> can use the in-band control information (e.g., <b>128</b>) to control the renderers.

0041A client control processing module <b>136</b> also controls the components in the client module <b>104</b>. The client control processing module <b>136</b> also interacts with the player control module <b>130</b> to transmit and receive control information over path <b>132</b>. For instance, the client control module <b>136</b> can provide an interface used to transmit various asynchronous events to the player control module <b>102</b>, such as an end of stream event (indicating that the client module <b>104</b> has reached the end of the stream <b>124</b>), a pause event (indicating that the user <b>112</b> has paused the presentation of content information), a stop event (indicating that the user <b>112</b> has stopped the presentation of content information), various error events, and so forth.

0042A presentation processing module <b>138</b> can include various functionality for rendering the content information. For instance, it can include audio drivers, video drivers, etc. The presentation processing module <b>138</b> can be implemented as a separate module from the presentation device <b>110</b>, or can be integrated with the presentation device <b>110</b> itself.

0043Finally, the client module <b>104</b> can include a number of other modules <b>140</b> for performing other unspecified tasks (that are not relevant to the focus of this discussion).

0044<figref idref="DRAWINGS">FIG. 2</figref> shows further details regarding the transfer processing module <b>122</b>, the reception processing module <b>134</b> and the presentation processing module <b>138</b>.

0045To begin with, the transfer processing module <b>122</b> can include an information processing module <b>202</b>. The information processing module <b>202</b>, in turn, can include a suite of processing tools that can be flexibly configured to perform different processing operations on content information received from the information source <b>108</b>. That is, different collections and combinations of such tools can be selected based on the type of content information that is received, and based on what kind of output format is desired, and so forth. Exemplary such tools can include: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0046"> Buffer managers that provide personal video recorder (PVR) type functionality, such as the ability to record content information, pause the content information, jump to different locations within the content information, and so forth. </li><li id="ul0002-0002" num="0047"> Various encoders for encoding the content information into a desired format, or decoders for decoding the content information that is expressed in a given format. </li><li id="ul0002-0003" num="0048"> Various content filters and analyzers for modifying the content information to improve the quality of presentation of the content information at the client module <b>104</b>. Different applications can adopt a different collection of such filters and analyzers (or can entirely omit such filters and analyzers) depending on the characteristics and demands of the particular applications. </li><li id="ul0002-0004" num="0049"> Various rate filters that control the bit rate of the steam <b>124</b>. For instance, one such filter can lower the bit rate when the available network bandwidth drops because of congestion or interference. This filter can lower the bit rate by re-encoding the stream or dropping frames, etc. </li><li id="ul0002-0005" num="0050"> Various digital rights management (DRM) filters for encrypting the content for transmission over the control mechanism <b>106</b>, and for performing other rights management functions. </li></ul></li></ul>

0051A packet formation module <b>204</b> receives an output from the information processing module <b>202</b> and places this output into a form suitable for transmission over the communication mechanism <b>106</b>. The packet formation module <b>204</b> can use any protocol to process the content information. To provide one non-limiting illustration, <figref idref="DRAWINGS">FIG. 3</figref> shows the creation of a number of packets. A stream <b>302</b> can be formed by starting with a sequence of key frames and delta frames. A key frame represents a stand-alone representation of a video frame that can be used to reconstruct the video without reference to other frames. The delta frames can be used to predicitvely reconstruct a video frame based on other information in the sequence. A serialized stream is formed from the above-described data, containing video, audio, and other information. A packetized stream is then formed by breaking the serialized stream into packets. Each packet can include a header and a payload. Finally, packets can be further multiplexed into groups and sent over the communication mechanism <b>106</b> as a transmission stream. A number of standards can be used to implement the above concepts, including, but not limited to, MPEG-2, the Real-Time Transport Protocol (RTP), various proprietary standards, and so forth; however, the principles described herein are not wedded to any specific standards.

0052Referring to <figref idref="DRAWINGS">FIG. 2</figref>, the packet formation module <b>204</b> performs the above-described tasks by assembling the packets into groups by multiplexing them. A control information generation module <b>206</b> inserts control packets <b>128</b> into the stream <b>124</b> along with the content packets <b>126</b>. The packet formation module <b>204</b> uses a queue <b>208</b> to create and transmit the stream <b>124</b>. That is, the queue <b>208</b> stores packets in an order received and a worker thread (not shown) extracts packets from the queue <b>208</b> and combines them with other packets.

0053Advancing again to <figref idref="DRAWINGS">FIG. 3</figref>, this figure shows an exemplary series of packets (<b>304</b>, <b>306</b>, <b>308</b>, . . . <b>310</b>). Each packet (<b>304</b>, <b>306</b>, <b>308</b>, . . . <b>310</b>) includes a respective header (<b>312</b>, <b>314</b>, <b>316</b>, . . . <b>318</b>) and accompanying payload (<b>320</b>, <b>322</b>, <b>324</b>, . . . <b>326</b>). The payloads can include audio information or video information. Further, according to the present system <b>100</b>, the payloads can also include control information. For instance, payloads <b>320</b>, <b>322</b> and <b>324</b> include audio or video information (<b>328</b>, <b>330</b>, <b>332</b>), while payload <b>326</b> includes control information <b>334</b>. The headers (<b>312</b>, <b>314</b>, <b>316</b>, . . . <b>318</b>) include various identification data (<b>336</b>, <b>338</b>, <b>340</b>, . . . <b>342</b>), including an indication of what type of data their respective payloads (<b>320</b>, <b>322</b>, <b>324</b>, . . . <b>326</b>) contain. More specifically, headers <b>312</b>, <b>314</b> and <b>316</b> include identification data (<b>336</b>, <b>338</b>, <b>340</b>) that indicates that their payloads include audio information or visual information (<b>328</b>, <b>330</b>, <b>332</b>), while header <b>318</b> includes identification data <b>342</b> which indicates that its payload <b>326</b> contains control information <b>334</b>.

0054Returning to <figref idref="DRAWINGS">FIG. 2</figref>, the reception processing module <b>134</b> includes a receiver <b>210</b> configured to receive the stream <b>124</b>. An information extraction module <b>212</b> receives an <b>11</b> output of the receiver <b>210</b> and extracts various fields of information from the received stream <b>124</b>. For instance, the information extraction module <b>212</b> can demultiplex the stream <b>124</b> and determine on a packet-by-packet basis whether it contains audio information, video information or control information. This can be performed by investigating the identification data (<b>336</b>, <b>338</b>, <b>340</b>, . . . <b>342</b>) in the headers (<b>312</b>, <b>314</b>, <b>316</b>, . . . <b>318</b>) of the received packets (<b>304</b>, <b>306</b>, <b>308</b>, <b>310</b>). The reception processing module <b>134</b> performs appropriate processing on the content of the payloads (<b>320</b>, <b>322</b>, <b>324</b>, . . . <b>326</b>) based on their assessed content. Control information conveyed by control packets can include instructions which govern the behavior of the processing performed by the client module <b>104</b>.

0055The reception processing module <b>134</b> forwards content information that it has received to the presentation processing module <b>138</b>. The presentation processing module <b>138</b> renders the content information using various decoders and drivers (<b>214</b>, <b>216</b>), as well as other processing mechanisms.

0056Finally, <figref idref="DRAWINGS">FIG. 2</figref> indicates that the client module <b>104</b> includes a jitter buffer <b>218</b> (referred to as simply a buffer below). The buffer <b>218</b> stores a certain amount of content information that it receives from the stream <b>124</b> on a first-in-first-out (FIFO) basis. The presentation processing module <b>138</b> draws from this buffer <b>218</b> when rendering the content information. In the event of a glitch (e.g., a slight interruption) in transmission, the presentation processing module <b>138</b> can thereby pull previously received content information from the buffer <b>218</b> without the glitch negatively affecting the presentation of the content information. The presentation processing module <b>138</b> will, however, suffer performance degradation when it reaches the end of the content information stored in the buffer <b>218</b> without receiving more content information from the stream <b>124</b>, because presentation of additional content information is not possible. Generally, <figref idref="DRAWINGS">FIG. 2</figref> depicts the buffer <b>218</b> as a component of the reception processing module <b>134</b>; but this buffer <b>218</b> can be positioned elsewhere in the chain of modules that act on the received content information. Also, the client module <b>104</b> may include plural buffers.

0057There are various occasions when it is desirable to flush the buffer <b>218</b>. For instance, when the user jumps from one stream to another stream, the buffer <b>218</b> no longer stores useful content information that can be relied on in compensating for network glitches. The client module <b>204</b> thus flushes the buffer <b>218</b> in these circumstances and refills it with content information taken from the other stream. The client module <b>204</b> may flush the buffer <b>218</b> upon channel changes (where the user jumps from the stream of one program to the stream of another program), and upon seeks (where the user jumps from one portion of a program to another portion of the same programs (e.g., occurring earlier or later in time).

0058However, the client module <b>104</b> is not the only component in the system <b>100</b> that is affected by stream discontinuities. When the user <b>114</b> changes a channel, for instance, the client control processing module <b>136</b> can inform the host module <b>102</b> of this event, and the host module <b>102</b> can respond by taking appropriate measures. Namely, for example, the transfer processing module <b>122</b> on the host module side also may include information stored within queues as well as configuration settings; this information may need to be flushed upon a break in the stream <b>124</b>. Accordingly, the host module <b>102</b> can also coordinate flushing of relevant information stored in the transfer processing module <b>122</b>, as well as handling other configuration tasks. The transfer processing module <b>122</b> responds to discontinuities by cleanly demarcating such breaks in the stream by inserting various stream boundary information into the stream <b>124</b>. The reception processing module <b>134</b> can then skip packets in the received stream <b>124</b> until it receives the stream boundary information.

0059It is therefore apparent that the interaction between the client module <b>104</b> and the host module <b>102</b> can be relatively intricate, requiring the exchange of control information, the flushing and refilling of one or more buffers, the reconfiguration of various settings, and so forth. This intricacy can incur an appreciable latency when the user <b>112</b> invokes an operation which causes a break in the stream <b>124</b>. To address this issue, the system <b>100</b> can include a coalescing module which reduces the latency in situations in which the user <b>112</b> makes multiple control actions within a short period of time. For instance, the coalescing module can come into play by reducing the latency associated with the user repeatedly making a series of channel change commands or seek commands within a relatively short period of time.

0060A.2. The Generation and Interpretation of In-Band Control Information

0061<figref idref="DRAWINGS">FIGS. 4-6</figref> describe three general scenarios in which a control packet is inserted within a stream of content information to provide instructions to the client module <b>104</b> regarding the content information. The instructions may convey information regarding the content information, or may describe processing to be performed on the content information, and so forth. The position of the control packets within the stream of content information is significant because it determines how the client module <b>104</b> should apply the control instructions contained therein to the content information. (The “position” of the control packets among a group of packets can be represented by, for example, the sequence numbers assigned to the packets, rather than an actual order of receipt of such packets—because the packets may be received out of order in a typical packet network.)

0062Beginning with <figref idref="DRAWINGS">FIG. 4</figref>, this figure shows a scenario <b>400</b> in which the control information <b>402</b> is placed at the trailing end of a content information stream X <b>404</b>. (Note that, in <figref idref="DRAWINGS">FIGS. 4-6</figref>, the least recently transmitted information appears at the far right of the figures.) This scenario <b>400</b> is appropriate to the case where the control information <b>402</b> provides information that demarcates the end of the stream (EOS). Positioning EOS control information <b>402</b> in-band along with the content information is useful because it enables the reception processing module <b>134</b> to know exactly when to take any necessary steps to switch from playback mode to idle mode, and so forth.

0063<figref idref="DRAWINGS">FIG. 5</figref> shows another scenario <b>500</b> in which the control information <b>502</b> is positioned at the beginning of a stream Y <b>504</b>. This positioning case can be used to convey instructions regarding operations to be subsequently performed by the client module <b>104</b> on stream Y <b>504</b>. One example of this scenario <b>500</b> pertains to the change of decryption keys. In a key change, the host module <b>102</b> informs the client module <b>104</b> that it should use a different key to subsequently decrypt content information that it receives from the host module <b>102</b>. It can perform this task by sending the new key to the client module <b>104</b> in advance over the separate control channel <b>132</b>. Then the transfer processing module <b>122</b> can insert the key change control information <b>502</b> into the in-band content information stream <b>124</b>. The position of the control information <b>502</b> as well as control data contained therein informs the reception processing module <b>134</b> that it should apply the new key that it has received to interpret stream Y, which follows the control information <b>502</b>.

0064Other types of control information <b>502</b> placed at the beginning of a stream can be used to inform the reception processing module <b>134</b> what rendering resources provided by the client module <b>104</b> should be enabled and/or disabled in processing the ensuing stream Y <b>504</b>. For example, this can be used to disable the digital output of audio if an audio stream is not allowed to be copied in digital form (e.g., in the case of content protection).

0065<figref idref="DRAWINGS">FIG. 6</figref> shows another scenario <b>600</b> in which the control information <b>602</b> is positioned between stream X <b>604</b> and stream Y <b>606</b>. This positioning case can be used to convey information regarding the relationship between stream X <b>604</b> and stream Y <b>606</b>, such as a discontinuity between stream X <b>604</b> and stream Y <b>606</b>. There may be a discontinuity between these streams because of some data loss.

0066Or the discontinuity may be due to express commands issued by the user <b>112</b> via the client module <b>104</b>, or via the host module <b>102</b> (or via some other module), which are forwarded to the host module <b>102</b> (e.g., via the control path <b>132</b>). For instance, this kind of discontinuity may be triggered when the user <b>112</b> has switched from one data stream to another due to switching channels, or because the user <b>112</b> has switched from one part of a data stream to another part due to issuing a seek command. There can be still other causes of discontinuities.

0067In any of these cases, the host module <b>102</b> can mark the discontinuity by adding control information <b>602</b> to the stream <b>124</b>, where such control information <b>602</b> constitutes stream boundary control information. For example, consider the exemplary scenario represented by the following series of actions: <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0000"><ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0068"> The user <b>112</b> issues a channel change command. </li><li id="ul0004-0002" num="0069"> Channel change instructions are forwarded to the host module <b>102</b> (or are entered directly to the computer device associated with the host module <b>102</b>). </li><li id="ul0004-0003" num="0070"> The host module <b>102</b> issues an instruction to flush appropriate buffers in the system <b>100</b> to clear out the old content information from the system (associated with the old channel). </li><li id="ul0004-0004" num="0071"> The transfer processing module <b>122</b> of the host module <b>102</b> adds stream boundary control information <b>602</b> to the stream <b>124</b> to mark the discontinuity between the stream associated with the old channel and the stream associated with the new channel. </li><li id="ul0004-0005" num="0072"> The client module <b>104</b> receives this stream boundary control information <b>602</b> and reacts appropriately. For instance, the host module <b>102</b> can, in advance, alert the client module <b>104</b> that it will be sending stream boundary control information <b>602</b>, prompting the client module to set a flag. As a result this flag, the client module <b>602</b> can be configured to discard all received packets until it receives the promised stream boundary control information <b>602</b>. </li><li id="ul0004-0006" num="0073"> Upon receiving the stream boundary control information, the client module <b>104</b> proceeds to fill up its buffer with the new channel information and start presenting it when the buffer reaches a prescribed state of fullness. </li></ul></li></ul>

0074Generally, there are at least two advantages to embedding control information within the content information stream. First, the client module <b>104</b> can receive the control information synchronized with the associated content information, without the need of providing complex supplemental linking information that ties the control information to the content information. Second, sending control information in-band reduces the load imposed on the separate control channel <b>132</b>, if in fact a specific implementation chooses to use such separate control channel <b>132</b>.

0075The control information itself can be expressed in various formats. By way of illustration and not limitation, this information can be expressed using an extension packet within any protocols, such as RTP. The packets can include identification information which identifies the nature of the stream carried by the packet's payload. The identification information can identify that the payload contains audio-visual information, control information, or some other kind of information.

0076An exemplary format of a control packet is as follows: <tables id="TABLE-US-00001" num="1"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="OFFSET" colwidth="14PT" align="left" /><colspec colname="1" colwidth="203PT" align="left" /><thead><row><entry /><entry /></row><row><entry /><entry namest="OFFSET" nameend="1" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>// Control codes</entry></row><row><entry /><entry>enum CONTROL_CODES</entry></row><row><entry /><entry>{</entry></row><row><entry /><entry>STREAM_BOUNDARY = 0,</entry></row><row><entry /><entry>STREAM_EOS,</entry></row><row><entry /><entry>MACROVISION,</entry></row><row><entry /><entry>ROTATE_KEY</entry></row><row><entry /><entry>}</entry></row><row><entry /><entry>union MicrosoftExtensionPacketHeader</entry></row><row><entry /><entry>{</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="OFFSET" colwidth="28PT" align="left" /><colspec colname="1" colwidth="189PT" align="left" /><tbody valign="top"><row><entry /><entry>struct</entry></row><row><entry /><entry>{</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="OFFSET" colwidth="35PT" align="left" /><colspec colname="1" colwidth="182PT" align="left" /><tbody valign="top"><row><entry /><entry>// A control code indicating the type of control</entry></row><row><entry /><entry>packet</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="OFFSET" colwidth="42PT" align="left" /><colspec colname="1" colwidth="91PT" align="left" /><colspec colname="2" colwidth="84PT" align="left" /><tbody valign="top"><row><entry /><entry>DWORD controlCode</entry><entry>: 16;</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="OFFSET" colwidth="42PT" align="left" /><colspec colname="1" colwidth="175PT" align="left" /><tbody valign="top"><row><entry /><entry>// Length of the payload contained in this control</entry></row><row><entry /><entry>packet</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="OFFSET" colwidth="42PT" align="left" /><colspec colname="1" colwidth="91PT" align="left" /><colspec colname="2" colwidth="84PT" align="left" /><tbody valign="top"><row><entry /><entry>DWORD payloadLength</entry><entry>: 16;</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="OFFSET" colwidth="28PT" align="left" /><colspec colname="1" colwidth="189PT" align="left" /><tbody valign="top"><row><entry /><entry>} bitField;</entry></row><row><entry /><entry>DWORD dwExtPacketHdr;</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="OFFSET" colwidth="14PT" align="left" /><colspec colname="1" colwidth="203PT" align="left" /><tbody valign="top"><row><entry /><entry>};</entry></row><row><entry /><entry namest="OFFSET" nameend="1" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0077The Stream Boundary field indicates a discontinuity in the stream.

0078The End Of Stream (EOS) field indicates the end of the stream that is being streamed to the client module <b>104</b>.

0079The Macrovision field indicates that the payload of this field's packet contains vision control codes appropriate to a certain prescribed mode of operation.

0080The Rotate Key field indicates that all samples received after this field's packet are encrypted with a new key. The new key can be transmitted over the control channel before this packet is inserted.

0081B. Exemplary Method of Operation

0082<figref idref="DRAWINGS">FIGS. 7 and 8</figref> describe the operation of the system <b>100</b> of <figref idref="DRAWINGS">FIG. 1</figref> in flow chart form. To facilitate discussion, certain operations are described as constituting distinct steps performed in a certain order. Such implementations are exemplary and non-limiting. Certain steps described herein can be grouped together and performed in a single operation, and certain steps can be performed in an order that differs from the order employed in the examples set forth in this disclosure.

0083<figref idref="DRAWINGS">FIG. 7</figref> describes a procedure for creating the stream <b>124</b> containing both content information <b>126</b> and control information <b>128</b>.

0084In step <b>702</b>, the host module <b>102</b> forms the content information <b>126</b>. This may involve collected the content information from the sources <b>108</b> and optionally performing processing on it to place it in desired form for transmission.

0085In step <b>704</b>, the host module <b>102</b> forms the control information <b>128</b>. The host module <b>102</b> forms the control information when prompted to do so by various circumstances, such as to signal the end of a stream, to signal the use of a new key, to signal the presence of a discontinuity, and so forth. In one subset of cases, the host module <b>102</b> forms the control information because it is prompted to do so by actions taken by the user <b>112</b> who is interacting with the client module <b>104</b> or the host module <b>102</b>.

0086In step <b>706</b>, the transfer processing module <b>122</b> of the host module <b>102</b> combines the content information <b>126</b> with the control information <b>128</b>. This can be performed by inserting the control information <b>128</b> into a multiplexed stream of packets containing content information (e.g., A/V information).

0087In step <b>708</b>, the transfer processing module <b>122</b> transfer the stream <b>124</b> to the client module <b>104</b>, where the stream <b>124</b> includes both content information <b>126</b> and control information <b>128</b>.

0088The procedure shown in <figref idref="DRAWINGS">FIG. 8</figref> is used to interpret and act on control information present in the received stream <b>124</b>.

0089In step <b>802</b>, the reception processing module <b>134</b> receives the stream <b>124</b> containing in-band control information <b>128</b>.

0090In step <b>804</b>, the reception processing module <b>134</b> determines whether a received packet from the stream <b>124</b> includes control information. This can be performed by examining the header information of the packet to determine if it contains information that indicates that it contains control information.

0091In step <b>806</b>, the client module <b>104</b> performs processing of the received content information <b>126</b> based on the control information <b>128</b>. Such processing can take the form of any of operations described above with reference to <figref idref="DRAWINGS">FIGS. 4-6</figref>. For example, the control information <b>128</b> may inform the client module <b>104</b> that it should use certain rendering resources or decryption keys to process the received content information. In the case of a stream discontinuity, the client module <b>104</b> may be discarding packets until it receives stream boundary control information, upon which time it commences refilling its buffers (e.g., buffer <b>218</b>).

0092C. Exemplary Computer Environment

0093In one exemplary implementation, both the host module <b>102</b> and the client module <b>104</b> can be implemented by two computer devices that are appropriately configured to act in a host and client capacity, respectively. In this case, <figref idref="DRAWINGS">FIG. 9</figref> provides information regarding an exemplary computer environment <b>900</b> that can be used to implement either the host module <b>102</b> or the client module <b>104</b>.

0094The computing environment <b>900</b> includes a general purpose type computer <b>902</b> and a display device <b>904</b>. However, the computing environment <b>900</b> can include other kinds of computing equipment. For example, although not shown, the computer environment <b>900</b> can include hand-held or laptop devices, set top boxes, game consoles, extension-type computers, mainframe computers, logic functionality embedded in rendering devices, and so forth. Further, <figref idref="DRAWINGS">FIG. 9</figref> shows elements of the computer environment <b>900</b> grouped together to facilitate discussion. However, the computing environment <b>900</b> can employ a distributed processing configuration. In a distributed computing environment, computing resources can be physically dispersed throughout the environment.

0095Exemplary computer <b>902</b> includes one or more processors or processing units <b>906</b>, a system memory <b>908</b>, and a bus <b>910</b>. The bus <b>910</b> connects various system components together. For instance, the bus <b>910</b> connects the processor <b>906</b> to the system memory <b>908</b>. The bus <b>910</b> can be implemented using any kind of bus structure or combination of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures.

0096Computer <b>902</b> can also include a variety of computer readable media, including a variety of types of volatile and non-volatile media, each of which can be removable or non-removable. For example, system memory <b>908</b> includes computer readable media in the form of volatile memory, such as random access memory (RAM) <b>912</b>, and non-volatile memory, such as read only memory (ROM) <b>914</b>. ROM <b>914</b> includes an input/output system (BIOS) <b>916</b> that contains the basic routines that help to transfer information between elements within computer <b>902</b>, such as during start-up. RAM <b>912</b> typically contains data and/or program modules in a form that can be quickly accessed by processing unit <b>906</b>.

0097Other kinds of computer storage media include a hard disk drive <b>918</b> for reading from and writing to a non-removable, non-volatile magnetic media, a magnetic disk drive <b>920</b> for reading from and writing to a removable, non-volatile magnetic disk <b>922</b> (e.g., a “floppy disk”), and an optical disk drive <b>924</b> for reading from and/or writing to a removable, non-volatile optical disk <b>926</b> such as a CD-ROM, DVD-ROM, or other optical media. The hard disk drive <b>918</b>, magnetic disk drive <b>920</b>, and optical disk drive <b>924</b> are each connected to the system bus <b>910</b> by one or more data media interfaces <b>928</b>. Alternatively, the hard disk drive <b>918</b>, magnetic disk drive <b>920</b>, and optical disk drive <b>924</b> can be connected to the system bus <b>910</b> by a SCSI interface (not shown), or other coupling mechanism. Although not shown, the computer <b>902</b> can include other types of computer readable media, such as magnetic cassettes or other magnetic storage devices, flash memory cards, CD-ROM, digital versatile disks (DVD) or other optical storage, electrically erasable programmable read-only memory (EEPROM), etc.

0098Generally, the above-identified computer readable media provide non-volatile storage of computer readable instructions, data structures, program modules, and other data for use by computer <b>902</b>. For instance, the readable media can store the operating system <b>930</b>, application modules <b>932</b>, other program modules <b>934</b>, and program data <b>936</b>.

0099The computer environment <b>900</b> can include a variety of input devices. For instance, the computer environment <b>900</b> includes the keyboard <b>938</b> and a pointing device <b>11940</b> (e.g., a “mouse”) for entering commands and information into computer <b>902</b>. The computer environment <b>900</b> can include other input devices (not illustrated), such as a microphone, joystick, game pad, satellite dish, serial port, scanner, card reading devices, digital or video camera, etc. Input/output interfaces <b>942</b> couple the input devices to the processing unit <b>906</b>. More generally, input devices can be coupled to the computer <b>902</b> through any kind of interface and bus structures, such as a parallel port, serial port, game port, universal serial bus (USB) port, etc.

0100The computer environment <b>900</b> also includes the display device <b>904</b>. A video adapter <b>944</b> couples the display device <b>904</b> to the bus <b>910</b>. In addition to the display device <b>904</b>, the computer environment <b>900</b> can include other output peripheral devices, such as speakers (not shown), a printer (not shown), etc. Any of these units can constitute the target entities (<b>120</b>, <b>122</b>, . . . <b>124</b>) shown in <figref idref="DRAWINGS">FIG. 1</figref>.

0101Computer <b>902</b> operates in a networked environment using logical connections to one or more remote computers, such as a remote computing device <b>946</b>. The remote computing device <b>946</b> can comprise any kind of computer equipment, including a general purpose personal computer, portable computer, a server, a game console, a network extension device, and so forth. Remote computing device <b>946</b> can include all of the features discussed above with respect to computer <b>902</b>, or some subset thereof.

0102Any type of network <b>948</b> can be used to couple the computer <b>902</b> with remote computing device <b>946</b>, such as a WAN, a LAN, etc. The computer <b>902</b> couples to the network <b>948</b> via network interface <b>950</b>, which can utilize broadband connectivity, modem connectivity, DSL connectivity, or other connection strategy. Although not illustrated, the computing environment <b>900</b> can provide wireless communication functionality for connecting computer <b>902</b> with remote computing device <b>946</b> (e.g., via modulated radio signals, modulated infrared signals, etc.).

0103In one implementation, the computer <b>902</b> and computer <b>946</b> can correspond to the host module <b>102</b> and client module <b>104</b>, respectively. In another implementation, the computer <b>902</b> and computer <b>946</b> can correspond to the host module <b>102</b> and source <b>108</b>, respectively (where the source <b>108</b> can constitute a server computer). Still other applications are possible.

0104In closing, a number of examples were presented in this disclosure in the alternative (e.g., case A or case B). In addition, this disclosure encompasses those cases which combine alternatives in a single implementation (e.g., case A and case B), even though this disclosure may not have expressly mention these conjunctive cases in every instance.

0105More generally, although the invention has been described in language specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed invention.