Adaptive bitrate streaming of media stored in matroska container files using hypertext transfer protocol

ES 2 807 884 T3

DESCRIPTION

Adaptive bit rate streaming of media stored in Matroska container files using hypertext transfer protocol

Field of the invention

The present invention relates generally to adaptive streaming and more specifically to adaptive bit rate streaming of encoded media contained within Matroska container files using hypertext transfer protocol.

Background

The term "streaming media" describes the playback of media on a playback device, in which the media is stored on a server and continuously sent to the playback device over a network during playback. Typically, the playback device stores a sufficient amount of media in a buffer at any given time during playback to avoid playback interruption due to the playback device completing playback of all buffered media before receiving the next portion of media. Adaptive Bitrate Streaming or Adaptive Streaming involves detecting current streaming conditions (for example, network bandwidth and user CPU capacity) in real time and adjusting the quality of the broadcast media. continually accordingly. Typically, the source media is encoded at multiple bit rates and the playback device or client switches between streaming of different encodings depending on available resources.

Adaptive streaming solutions typically use either Hypertext Transfer Protocol (HTTP), published by the Internet Engineering Task Force and the World Wide Web Consortium as RFC 2616, or Real Time Streaming Protocol (RTSP). ), published by the Internet Engineering Task Force as RFC 2326, to stream media between a server and a playback device. HTTP is a stateless protocol that enables a playback device to request a range of bytes within a file. HTTP is described as stateless, because the server is not required to record information regarding the state of the playback device requesting information or the byte ranges requested by the playback device to respond to requests received from the playback device. RTSP is a network control protocol used to control streaming media servers. Playback devices issue control commands, such as play and pause, to the server that streams the media to control the playback of media files. When using RTSP, the media server records the status of each client device and determines the media to stream based on the instructions received from the client devices and the status of the client.

In adaptive streaming systems, the source media is typically stored on a media server as a top-level index file pointing to a number of alternate streams containing the actual video and audio data. Each stream is typically stored in one or more cabinet files. Different adaptive streaming solutions typically use different index and media containers. The Synchronized Multimedia Integration Language (SMIL) developed by the World Wide Web Consortium is used to create indexes on various adaptive streaming solutions including IIS Smooth Streaming developed by Microsoft Corporation of Redmond, Washington, and Flash Dynamic Streaming developed by Adobe Systems Incorporated of San Jose, California. HTTP Adaptive Bitrate Streaming developed by Apple Computer Incorporated of Cupertino, California implements index files using an extended M3U playlist file (.M3U8), which is a text file containing a list of URIs that typically identifies a container file of media. The most commonly used media container formats are the MP4 container format specified in MPEG-4 Part 14 (i.e. ISO / IEC 14496-14) and the MPEG Transport Stream Container (TS) specified in MPEG-2 Part. 1 (i.e. ISO / IEC 13818-1 standard). The MP4 container format is used in IIS Smooth Streaming and Flash Dynamic Streaming. The TS container is used in HTTP Adaptive Bitrate Streaming.

The Matroska container is a media container developed as a draft open standard by the non-profit organization Matroska of Aussonne, France. The Matroska container is based on Extensible Binary Meta Language (EBML), which is a binary derivative of Extensible Markup Language (XML). Many consumer electronics (CE) devices support Matroska container decoding. The DivX Plus file format developed by DivX, LLC of San Diego, California uses an extension of the Matroska container format (that is, it is based on the Matroska container format, but includes items that are not specified within the Matroska format).

A method for switching video playback in response to changing network conditions is described in US 2009/328124 A1, in which playback of the video file is started by streaming the high bit rate version, and after an indication of restricted network conditions, a transition point is selected and the video file playback is switched to the low bitrate player after the point is found

ES 2 807 884 T3 transition.

US 2009/150557 A1 describes a download agent that dynamically selects between archived media with different media quality.

Document XP055009366 (IIS Smooth Streaming Technical Overview, Alex Zambelli, March 2009) discloses an HTTP-based adaptive streaming technology.

Summary of the invention

The invention relates to a playback device as described in claim 1. Additional features of the playback device are described in the dependent claims. The embodiments, except those related to the claims, refer to examples useful for understanding the invention but which do not represent embodiments of the present invention claimed. These examples are provided for illustrative purposes only.

Systems and methods for adaptive bit rate streaming of media stored in Matroska container files using Hypertext Transfer Protocol (HTTP) are disclosed in accordance with embodiments of the invention. In one embodiment, a processor configured, through a client application, to request portions of files from a remote server. In addition, the client application further configures the processor to retrieve top-level index data that identifies a plurality of EBML cabinet files and describes at least a maximum bit rate of the alternate streams contained within the EBML cabinet files, parsing the data. top-level index to obtain information that identifies the plurality of EBML cabinet files, request a portion of at least one of the EBML container files that contains the at least one element that specifies the encoding parameters of the stream contained within the EBML container file, retrieve an index that references each element that contains portions of video encoded within al minus one of the EBML cabinet files, use the index to request portions of a first EBML container file that includes elements containing portions of encoded video, receive and buffer the requested elements, decode the encoded video contained within the buffered elements using the parameters of encoding, measure current streaming conditions, and select another of the EBML container files from which to retrieve items containing portions of encoded video to decode, where the selection is based on the measured streaming conditions and the description of the alternate stream bit rate contained within of the top-level data.

A further embodiment includes retrieving top-level index data identifying each of the plurality of EBML container files and describing at least the maximum bit rate of each of the alternate streams contained within the EBML container files, analyzing the data from top-level index for information identifying the plurality of EBML cabinet files, request a portion of at least one of the EBML container files that contains the at least one element that specifies the encoding parameters of the stream contained within the EBML container file, retrieve an index that references each element that contains portions of video encoded within al minus one of the EBML cabinet files, use the index to request portions of a first EBML container file that includes elements containing portions of encoded video, receive and buffer the requested elements, decode the encoded video contained within the buffered elements using encoding parameters , measure current streaming conditions, and select another of the EBML container files from which to retrieve items containing portions of encoded video to decode, where the selection is based on the measured streaming conditions and the description of the alternate stream bit rate contained within of the top-level index data.

Another embodiment of the invention includes a processor configured through a source encoding application to consume at least one multimedia file containing source video. Furthermore, the source encoding application further configures the processor to select a portion of the source video, transcode the selected portion of the source video into a plurality of alternate portions of encoded video, wherein each alternate portion is encoded using a set different encoding parameters and starts with an intra frame starting a closed group of snapshots (GOP), writing each of the alternate portions of encoded video to an element of a different EBML container file, where each element is located within an EBML container file that also includes another element indicating the encoding parameters used to encode the alternate portion encoded video, and adding an entry to at least one index that identifies the location of the item that contains one of the alternate portions of encoded video within each of the EBML container files.

Still another embodiment includes repeatedly selecting a portion of the source video using the source encoder, transcoding the selected portion of the source video into a plurality of alternate portions of encoded video using the source encoder, wherein each alternate portion is encoded using a different set of encoding parameters and starts with an intra frame starting a closed group of snapshots (GOP),

ES 2 807 884 T3 writing each of the alternate portions of encoded video to an element of a different EBML container file using the source encoder, where each element is located within an EBML container file that also includes another element containing a set of encoding parameters corresponding to the encoding parameters used to encode the video portion, and adding an entry to at least one index that identifies the location of the item that contains one of the alternate portions of encoded video within each of the EBML container files.

Brief description of the drawings

Figure 1 is a network diagram of an adaptive bit rate streaming system in accordance with one embodiment of the invention.

Figure 2 conceptually illustrates a top-level index file and Matroska container files generated by source media encoding in accordance with embodiments of the invention.

Figure 3 conceptually illustrates a specialized Matroska container file incorporating a modified Token element in accordance with one embodiment of the invention.

Figures 4a-4c conceptually illustrate the insertion of different types of media into the Clusters element of a Matroska container file subject to various constraints that facilitate adaptive bitrate streaming in accordance with embodiments of the invention.

Figure 4d conceptually illustrates the multiplexing of different types of media in the Clusters element of a Matroska container file subjected to various constraints that facilitate adaptive bitrate streaming in accordance with one embodiment of the invention.

Figure 4e conceptually illustrates the inclusion of a non-standard playback track in the Clusters element of a Matroska container file subject to various constraints that facilitate adaptive bit rate streaming in accordance with one embodiment of the invention.

Figure 5 conceptually illustrates a modified Hints element of a specialized Matroska container file, in which the Hints element includes information that enables retrieval of Cluster elements using HTTP byte interval requests in accordance with one embodiment of the invention. .

Figure 5a conceptually illustrates a modified Hints element of a specialized Matroska container file in accordance with one embodiment of the invention, in which the Hints element is similar to the Hints element shown in Figure 5 except that they are removed attributes that are not used during adaptive bitrate streaming.

Figure 6 conceptually illustrates the indexing of Grouping elements within a specialized Matroska container file using modified CuePoint elements within the container file in accordance with embodiments of the invention.

Figure 7 is a flow chart illustrating a process for encoding source media for adaptive bit rate streaming in accordance with one embodiment of the invention.

Figure 8 conceptually illustrates communication between a playback device and an HTTP server associated with the start of streaming encoded media contained within Matroska container files indexed by a top-level index file in accordance with one embodiment of the invention. .

Figures 9a and 9b conceptually illustrate communication between a playback device and an HTTP server associated with switching between streams in response to streaming conditions experienced by the playback device and dependent on the index information available to the device. of playback prior to the decision to switch streams in accordance with embodiments of the invention.

Detailed disclosure of the invention

Turning now to the drawings, systems and methods for adaptive bit rate streaming of media stored in Matroska container files using Hypertext Transfer Protocol (HTTP) are illustrated in accordance with embodiments of the invention. In a number of the embodiments, source media is encoded as a number of alternate streams. Each stream is stored in a Matroska container file (MKV). In many embodiments, the Matroska container file is a specialized Matroska container file in which the manner in which the media in each stream is encoded and stored within the container is restricted to improve streaming performance. In various embodiments, the Matroska container file further specializes in that additional index items (that is, items that are not specified as part of the Matroska container format) can be included within the file to facilitate retrieval of desired media during streaming. adaptive bitrate. In various embodiments, each stream (ie, audio, video, or subtitle) is stored within a separate Matroska container file. In other embodiments, an encoded video stream is multiplexed with one or more encoded audio, and / or subtitle streams in each Matroska container file. A top-level index file is also generated that contains an index to the streams contained within each of the container files to enable adaptive bitrate streaming of the encoded media. In many embodiments, the top-level index file is a Synchronized Multimedia Integration Language (SMIL) file that contains URIs for each of the Matroska cabinet files. In other embodiments, any of a number of file formats can be used in generating the top-level index file.

ES 2 807 884 T3

The performance of an adaptive bit rate streaming system in accordance with embodiments of the invention can be significantly improved by encoding each portion of the source video at each bit rate such that the video portion is encoded in each stream as a single (or at least one) closed group of snapshots (GOP) starting with a Decoder Instant Regeneration frame (| Dr). The GOP for each flow can then be stored as a Grouping item within the Matroska container file for the flow. In this way, the playback device can switch between streams at the completion of the playback of a pool and, regardless of the stream from which a Pool is obtained, the first frame in the pool will be an IDR frame and can be decoded without reference to no encrypted media other than the encrypted media contained within the Grouping element. In many embodiments, the sections of the source video that are encoded as GOP are all the same length. In a number of the embodiments each two second sequence of the source video is encoded as a GOP.

Media retrieval using HTTP during adaptive streaming can be enhanced by adding additional index information to the Matroska container files used to contain each of the encoded streams. In a number of the embodiments, the index is a reduced index in that the index only points to the IDRs at the start of each cluster. In many embodiments, the Matroska container file index includes additional non-standard attributes (that is, attributes that are not part of the Matroska container file format specification) that specify the size of each of the pools such that a storage device Replay can retrieve a grouping item from the Matroska container file over HTTP using a byte range request.

Adaptive streaming of source media encoded in the manner described above may be coordinated by a playback device in accordance with embodiments of the invention. The playback device obtains information regarding each of the available streams from the top-level index file and selects one or more streams to be used in the playback of the media. The playback device can then obtain header information from the Matroska container files containing the one or more bit streams or streams, and the headers provide information regarding decoding of the streams. The playback device may also request index information that indexes the encoded media stored within the relevant Matroska container files. Index information can be stored within Matroska cabinet files or separately from Matroska cabinet files in the top-level index or in separate index files. The index information enables the playback device to request byte ranges that correspond to Pooling items within the Matroska container file that contains specific portions of HTTP-encoded media from the server. As the playback device receives the Pool items from the HTTP server, the playback device can evaluate the current streaming conditions to determine whether to increase or decrease the bit rate of the streamed media. In the event that the playback device determines that a change in bit rate is necessary, the playback device may obtain header information and index information for the container file (s) containing the desired stream (s) (assuming that the playback device has not yet obtained this information). The index information can then be used to identify the byte range of the Pool item that contains the next portion of the source media encoded at the desired bit rate and the identified Pool item can be retrieved from the server via HTTP. . The next portion of the source media that is requested is typically identified based on the Pool items already requested by the playback device and the Pool items buffered by the playback device. The next requested source media portion of the alternate stream is requested to minimize the likelihood that the playback device buffer will be underutilized (that is, run out of media to playback) prior to receipt of the item from the Pool that contains the next portion of source media by the playback device. In this way, the playback device can achieve adaptive bit rate streaming by retrieving sequential Grouping elements from the various streams as appropriate for the streaming conditions using the top-level index and index information describing the elements. Clustering within each of the Matroska container files.

In a number of the embodiments, the variation in the bit rate between different streams can be achieved by modifying the encoding parameters for each stream including, but not limited to, the bit rate, frame rate, and resolution. When different streams include different resolutions, the display aspect ratio of each stream is the same and the sample aspect ratios are altered to ensure smooth transitions from one resolution to another. The following is further discussed source video encoding for use in adaptive bit rate streaming and playback of encoded source video using HTTP requests to achieve adaptive bit rate streaming according to embodiments. of the invention.

ARCHITECTURE OF THE ADAPTIVE CONTINUOUS DIFFUSION SYSTEM

An adaptive streaming system in accordance with one embodiment of the invention is illustrated in Figure 1. Adaptive streaming system 10 includes a source encoder 12 configured to encode source media as a number of alternate streams. In the illustrated embodiment, the source encoder is a server. In other embodiments, the source encoder can be any processing device that includes a

ES 2 807 884 T3 processor and sufficient resources to perform source media transcoding (including but not limited to video, audio and / or subtitles). As discussed further below, source encoding server 12 generates a top-level index to a plurality of container files containing the streams, at least a plurality of which are alternate streams. Alternate streams are streams that encode the same media content in different ways. In many cases, alternate streams encode media content (such as but not limited to video) at different bit rates. In a number of the embodiments, the alternate streams are encoded at different resolutions and / or at different frame rates. The top-level index file and cabinet files are uploaded to an HTTP 14 server. Various playback devices may then use HTTP or other appropriate stateless protocol to request portions of the top-level index file and container files over a network 16 such as the internet.

In many embodiments, the top-level index file is a SMIL file, and the media is stored in Matroska cabinet files. As discussed further below, the media can be stored within the Matroska container file in a way that facilitates adaptive bitrate streaming of the media. In many embodiments, Matroska cabinet files are specialized Matroska cabinet files that include enhancements (that is, elements that are not part of the Matroska file format specification) that make it easier to retrieve specific portions of media over HTTP during broadcast. Adaptive Bitrate Streaming Media.

In the illustrated embodiment, playback devices include personal computers 18 and mobile phones 20. In other embodiments, playback devices can include consumer electronic devices such as DVD players, Blu-ray players, televisions, living room set-top boxes, game consoles. video games, tablets, and other devices that are capable of connecting to a server via HTTP and playing encrypted media. Although a specific architecture is shown in Figure 1, any of several architectures that enable playback devices to request portions of the top-level index file and cabinet files may be used in accordance with embodiments of the invention.

FILE STRUCTURE

Files generated by a source encoder and / or stored on an HTTP server for streaming to playback devices are illustrated in Figure 2 in accordance with embodiments of the invention. The files used in adaptive bit rate streaming of the source media include a top-level index 30 and a plurality of container files 32 each containing at least one stream. The top-level index file describes the contents of each of the cabinet files. As discussed further below, the top-level index file can take a number of forms including a SMIL file, and the cabinet files can take a number of forms including a specialized Matroska cabinet file.

In many embodiments, each Matroska cabinet file contains a single stream. For example, the stream could be one of a number of alternate video streams, an audio stream, one of a number of alternate audio streams, a subtitle stream, one of a number of alternate subtitle streams, a stream of non-standard playback or one of a number of alternative non-standard playback streams. In various embodiments, the Matroska container file includes multiple multiplexed streams. For example, the Matroska container could include a video stream, and one or more audio streams, one or more subtitle streams, and / or one or more non-standard playback streams. As discussed further below, in many embodiments the Matroska cabinet files are specialized files. The encoding of the media and the manner in which the media is stored within Grouping elements within the Matroska container file may be subject to restrictions designed to improve the performance of an adaptive bitrate streaming system. Additionally, the Matroska container file can include index items that make it easy to locate and download Pool items from the various Matroska container files during adaptive media streaming. Top-level index files and Matroska container files that can be used in adaptive bit rate streaming systems in accordance with embodiments of the invention are discussed below.

TOP LEVEL INDEX FILES

Playback devices according to many embodiments of the invention use a top-level index file to identify container files that contain the streams available to the playback device for use in adaptive bit rate streaming. In many embodiments, the top-level index files may include references to cabinet files that each include an alternate stream of encoded media. The playback device may use the information in the top-level index file to retrieve encoded media from each of the container files in accordance with the streaming conditions experienced by the playback device.

In various embodiments, the top-level index file provides information by enabling the playback device to retrieve information regarding the encoding of the media in each of the cabinet files and an index to media encoded within each of the cabinet files. . In a number of

In ES 2 807 884 T3 embodiments, each container file includes information regarding the encoded media contained within the container file and an index to the encoded media within the container file and the top-level index file indicates the portions of each container file that contains this information. Therefore, a playback device can retrieve the top-level index file and use the top-level index file to request the portions of one or more of the cabinet files that include information regarding the encoded media contained within the container file and an index to the encoded media within the container file. Various higher level index files that may be used in adaptive bit rate streaming systems in accordance with embodiments of the invention are further discussed below.

TOP LEVEL INDEX SMIL FILES

In a number of embodiments, the top-level index file used in adaptive bitrate media streaming is an SMIL file, which is an XML file that includes a list of URIs that describe each of the streams and the cabinet files that contain the streams. The URI may include information such as the system-bitrate of the stream contained within the stream and information regarding the location of specific pieces of data within the container file.

The basic structure of an SMIL file involves providing an XML declaration and a SMIL element. The SMIL element defines the streams available for use in adaptive bit rate streaming and includes a HEAD element, which is usually left empty, and a BODY element which usually only contains one pAr (parallel) element. The PAR element describes streams that can be played simultaneously (that is, they include media that can be presented at the same time).

The SMIL specification defines a number of child elements to the PAR element that can be used to specify the streams available for use in adaptive bitrate streaming. The VIDEO, AUDIO and TEXTSTREAM items can be used to define a specific video, audio or subtitle stream. The VIDEO, AUDIO and TEXTSTREAM items can be collectively referred to as media objects. The basic attributes of a media object are the SRC attribute, which specifies the full path or a URI to a container file containing the relevant stream, and the XML: LANG attribute, which includes a 3-letter language code. Additional information regarding a media object can be specified using the PARAM element. The PARAM element is a standard way within the SMIL format to provide a general name value pair. In a number of embodiments of the invention, specific PARAM elements are defined that are used during adaptive bit rate streaming.

In many embodiments, a PARAM header-request element is defined that specifies the size of the header section of the container file that contains the stream. The value of the PARAM header-request element typically specifies the number of bytes between the start of the file and the start of the encoded media within the file. In many embodiments, the header contains information regarding how the media is encoded and a playback device retrieves the header prior to playback of the encoded media to be able to configure the decoder for playback of the encoded media. An example of a PARAM header-request element is as follows:

<param name = 'header-request value = 1026 valuetype = data />

In a number of embodiments, a PARAM mime element is defined that specifies the MIME type of the stream. The PARAM mime element that identifies the stream as being an H.264 stream (that is, a stream encoded according to the MPEG-4 Advanced Video Codec standard) is as follows:

<param name = 'mime value = V_MPEG4 / ISO / AVC valuetype = data />

The MIME type of the stream can be specified using a PARAM mime element as appropriate to the encoding of a specific stream (eg AAC audio or UTF-8 text stream).

When the media object is a VIDEO element, additional attributes are defined within the SMIL file format specification that includes the systemBitrate attribute, which specifies the bitrate of the stream in the container file identified by the VIDEO element, and width and height attributes, which specify the dimensions of the encoded video in pixels. Additional attributes can also be defined using the PARAM element. In various embodiments, a PARAM vbv element is defined that specifies the VBV buffer size of the video stream in bytes. The Video Buffer Verifier (VBV) is a buffer model of

ES 2 807 884 T3 theoretical MPEG video used to ensure that an encoded video stream can be properly cached and played back on the set-top box device. An example of a PARAM vbv element that specifies a VBV size of 1000 bytes is as follows:

<param name = vbv value = 1000 valuetype = data />

An example of a VIDEO element that includes the attributes discussed above is as follows:

<video src = http: //cnd.com/video1_620kbps.mkv systemBitrate = 620 width = 480 height = 270>

<param name = vbv value = 1000 valuetype-'data /> </video>

Adaptive bit rate streaming systems in accordance with embodiments of the invention can support non-standard playback streams, which can be used to provide smooth visual search through source content encoded for adaptive bit rate streaming. A non-standard playback stream can be encoded which appears to be an accelerated visual search through the source media when played back, when in reality the non-standard playback stream is simply a separate track that encodes the source media at a lower rate. frames. In many system embodiments, a VIDEO element that refers to a non-standard playback track is indicated by the VIDEO element's systemProfile attribute. In other embodiments, any of a number of techniques can be used to signal within the top-level index file that a specific stream is a non-standard playback stream. An example of a non-standard playback stream VIDEO element according to one embodiment of the invention is as follows:

<video src = http: //cnd.com/video_test2_600kbps.mkv systemProfile = DivXPlusTrickTrack width = 480 height = 240>

<param name = 'vbv value = 1000 valuetype = data />

<param name = header-request value = 1000 valuetype = data />

</video>

In a number of embodiments of the invention, a PARAM reservedBandwidth element may be defined for an AUDIO element. The PARAM element reservedBandwidth specifies the bit rate of the audio stream in Kbps. An example of a specified AUDIO element according to one embodiment of the invention is as follows:

<audio src = http: //end.com/audio_test1_277kbps.mkv xml: lang = gem <param name = reservedBandwidth value = 128 valuetype = data /> />

In various embodiments, the PARAM element reservedBandwidth is also defined for a TEXTSTREAM element. An example of a TEXTSTREAM element that includes a PARAM reservedBandwidth element according to one embodiment of the invention is as follows:

&lt;textstream src = http: //cnd.com/text_stream_ger.mkv xml: lang = gem &lt;param

ES 2 807 884 T3 name = reservedBandwidth value = 32 valuetype = 'data />

/>

In other embodiments, any of a number of mechanisms may be used to specify information regarding VIDEO, AUDIO, and SUBTITLE items as appropriate for specific applications.

A SWITCH element is a mechanism defined within the SMIL file format specification that can be used to define adaptive or alternate flows. An example of how a SWITCH element can be used to specify alternative video streams at different bit rates is as follows:

<switch>

<video src = http: //cnd.com/video_test1_300kbps.mkv/>

<video src = http: //cnd.com/video_test2_900kbps.mkv/>

<video src = http: //cnd.com/video_test3_1200kbps.mkv/>

</switch>

The SWITCH element specifies the URLs of three alternate video streams. The file names indicate the different bit rates of each of the streams. As discussed further below, the SMIL file format specification provides mechanisms that can be used in accordance with embodiments of the invention to specify within the top-level index SMIL file additional information regarding a stream and the container file in the stream. that is contained.

In many embodiments of the invention, the EXCL (exclusive) element is used to define alternative tracks that do not adapt during playback under streaming conditions. For example, the EXCL element can be used to define alternate audio tracks or alternate subtitle tracks. An example of how an EXCL element can be used to specify alternate English and French audio streams is as follows:

<excl>

<audio src = http: //cnd.com/english-audio.mkv xml: lang = eng />

<audio src = http: //cnd.com/french-audio.mkv xml: lang = fre />

</excl>

An example of a top-level index SMIL file defining the attributes and parameters of two alternative video levels, an audio stream and a subtitle stream in accordance with one embodiment of the invention is as follows:

<smil xmlns = http: //www.w3.org/ns/SMIL version = 3.0 baseProfile = Language>

<head>

</head>

<body>

<par>

<switch>

<video src = http: //cnd.com/video_test1_300kbps.mkv systemBitrate = 300 vbv = 600 width = 320 height = 240>

<param name = vbv value = 600 valuetype = data />

<param name = header-request value = 1000 valuetype = data />

</video>

&lt;video

ES 2 807 884 T3 src = http: //cnd.com/video_test2_600kbps.mkv systemBitrate = 600 vbv = 900 width = 640 height = 480>

<param name = vbv value = 1000 valuetype = data />

<param name = header-request value = 1000 valuetype = data /> </video>

</switch>

<audio src = http: //cnd.com/audio.mkv xml: lang = eng>

<param name = header-request value = 1000 valuetype = data /> <param name = reservedBandwidth value = 128 valuetype = data /> </audio>

<textstream src = http: //cnd.com/subtitles.mkv xml: lang: =: eng>

<param name = header-request value = 1000 valuetype = data />

<param name = reservedBandwidth value = 32data /> </textstream>

</par>

</body>

</smil>

The top-level index SMIL file can be generated when the source media is encoded for playback via adaptive bitrate streaming. Alternatively, the top-level index file SMIL can be generated when a playback device requests the start of playback of the encoded media. When the playback device receives the top-level index SMIL file, the playback device can analyze the SMIL file to identify available streams. The playback device can then select the streams to use to play the content and can use the SMIL file to identify the portions of the container file to download to obtain information regarding the encoding of a specific stream and / or to obtain an index. to the encoded media within the container file.

Although top-level index SMIL files are described above, any of a number of top-level index file formats can be used to create top-level index files as appropriate for a specific application in accordance with one embodiment of the invention. The use of higher level index files to enable playback of encoded media using adaptive bit rate streaming in accordance with embodiments of the invention is further discussed below.

MEDIA STORAGE IN MATROSKA FILES FOR ADAPTIVE BIT RATE CONTINUOUS BROADCAST

A Matroska container file used to store encoded video in accordance with one embodiment of the invention is illustrated in Figure 3. Container file 32 is an Extensible Binary Markup Language (EBML) file that is an extension of the Matroska container file format. The specialized Matroska container file 32 includes a standard EBML element 34, and a standard Segment element 36 that includes a standard Search Heading element 40, a standard Segment Information element 42, and a standard Tracks element 44. These elements Standard describe the media contained within the Matroska container file. Segment element 36 also includes a standard Clusters element 46. As described below, the manner in which encoded media is inserted into individual Cluster elements 48 within Cluster element 46 is restricted to improve media playback in an adaptive streaming system. In many embodiments, the restrictions placed on encoded video are consistent with the Matroska container file format specification and involve encoding the video so

ES 2 807 884 T3 that each cluster includes at least one closed GOP beginning with an IDR frame. In addition to the standard elements above, the Segment 36 element also includes a modified version of the standard Clues 52 element. As discussed further below, the Hints element includes specialized CuePoint elements (that is, non-standard CuePoint elements) that facilitate retrieval of media contained within specific Grouping elements over HTTP.

The restrictions imposed upon media encoding and formatting of the encoded media within the Bundles element of a Matroska container file for adaptive bitrate streaming and additional index information inserted within the container file according to embodiments of the invention is discussed further below.

CODING MEANS FOR INSERTION IN GROUPING ELEMENTS

An adaptive bit rate streaming system provides a playback device with the option to select between different encoded media streams during playback in accordance with the streaming conditions experienced by the playback device. In many embodiments, switching between streams is facilitated by separately precoded discrete portions of the source media according to the encoding parameters of each stream and then including each separately encoded portion in its own Clustering element within the Container file for the flow. Additionally, the media contained within each pool is encoded such that the media is capable of reproducing without reference a media contained in any other pool within the stream. In this way, each stream includes a Grouping element that corresponds to the same discrete portion of the source media and, at any time, the playback device can select the Grouping element of the stream that is most appropriate for the broadcast conditions. through the playback device and can start playback of the media contained within the Grouping item. Accordingly, the playback device can select groupings of different streams as the streaming conditions experienced by the playback device change over time. In various embodiments, the Cluster elements are further limited such that each Cluster element contains a portion of encoded media from the source media that has the same duration. In a number of the embodiments, each Pool item includes two seconds of encoded media. The following discusses the specific restrictions applied to media encoded within each Grouping item that depend on the type of media (ie, video, audio, or subtitles).

Illustrated in Figure 4a is a Clusters element of a Matroska container file containing a video stream in accordance with one embodiment of the invention. Cluster element 46 includes a plurality of Cluster elements 48 each containing a discrete portion of encoded video. In the illustrated embodiment, each Cluster element 48 includes two seconds of encoded video. In other embodiments, the Grouping elements include encoded video that is greater or less than two seconds long. The smaller the Pool items (that is, the shorter the duration of the encoded media within each Pool item), the greater the overhead associated with requesting each Pool item. Therefore, there is a trade-off between the sensitivity of the playback device to changes in streaming conditions and the effective data rate of the adaptive streaming system for a given set of streaming conditions (i.e., the portion of the available bandwidth actually used to stream encrypted media). In various embodiments, the video sequences encoded in the element array have different lengths. Each Grouping element 48 includes a Timecode element 60 indicating the start time of the encoded video within the Grouping element and a plurality of BlockGroup elements. As noted above, the encoded video stored within the pool is restricted so that the encoded video can be played without reference to the encoded video contained within any of the other Pool elements in the container file. In many embodiments, encoding the video contained within the Grouping element as a GOP in which the first frame is an IDR frame applies the restriction. In the illustrated embodiment, the first element of BlockGroup 62 contains an IDR frame. Therefore, the first element of BlockGroup 62 does not include an element of ReferenceBlock. The first element of BlockGroup 62 includes a Block 64 element, which specifies the Timecode attribute of the encoded frame within the Block 64 element relative to the Timecode of the Grouping 48 element. Subsequent elements of BlockGroup 66 are not restricted in the types of frames they can contain (other than that they cannot reference frames that are not contained within the Grouping element). Therefore, subsequent elements of BlockGroup 66 may include elements of ReferenceBlock 68 that refer to another element or elements of BlockGroup used in decoding the frame contained within the BlockGroup or they may contain IDR frames and are similar to the first element of BlockGroup 62 . As noted above, the way encoded video is inserted into the Grouping elements of the Matroska file conforms to the Matroska file format specification.

The insertion of encoded audio and subtitle information within a Clusters 46 element of a Matroska container file in accordance with embodiments of the invention is illustrated in Figures 4b and 4c. In the illustrated embodiments, the encoded media is inserted into the Cluster elements 48 subject to the same restrictions applied to encoded video discussed above with respect to Figure 4a. Besides, the

ES 2 807 884 T3 duration of audio and subtitle information encoded within each Grouping element corresponds to the duration of the encoded video in the corresponding Grouping element of the Matroska container file containing the encoded video. In other embodiments, the Grouping elements within the container files that contain the audio and / or subtitles streams need not correspond to the start time and duration of the Grouping elements in the container files containing the alternate video streams.

MULTIPLEXING FLOWS IN A SINGLE MKV CONTAINER FILE

The Clusters elements shown in Figures 4a - 4c assume that a single stream is contained within each Matroska container file. In various embodiments, media from multiple streams are multiplexed into a single Matroska container file. In this way, a single container file can contain a video stream multiplexed with one or more corresponding audio streams, and / or one or more corresponding subtitle streams. Storing streams in this way can result in duplication of audio and subtitle streams across multiple alternate video streams. However, the seek time to retrieve encoded media from a video stream and associated audio, and / or subtitle stream may be shortened due to adjacent storage of the data on the server. Illustrated in Figure 4d is the Clusters element 46 of a Matroska container file containing multiplexed video, audio, and subtitle data in accordance with one embodiment of the invention. In the illustrated embodiment, each Grouping element 48 includes additional BlockGroup elements for each of the multiplexed streams. The first Grouping element includes a first 62v BlockGroup element for encoded video that includes a 64v Block element that contains an encoded video frame and indicates the Timecode attribute of the frame relative to the Grouping element's start time ( i.e. Timecode attribute 60). A second element of BlockGroup 62a includes a Block element 64a that includes an encoded audio stream and indicates the timecode of the encoded audio relative to the start time of the Grouping element, and a third element of BlockGroup 62s that includes a Block element 64s containing an encoded subtitle and indicating the time code of the encoded subtitle in relation to the start time of the Grouping element. Although not shown in the illustrated embodiment, each Grouping element 48 would likely include additional BlockGroup elements containing additional encoded video, audio, or subtitles. Despite the multiplexing of encoded video, audio, and / or subtitle streams, the same restrictions apply with respect to encoded media.

INCORPORATION OF NON-STANDARD PLAYBACK TRACKS IN MKV CONTAINER FILES FOR USE IN ADAPTIVE BITRATE STREAMING SYSTEMS

Incorporation of non-standard playback tracks within Matroska container files is proposed by DivX, LLC in U.S. Patent Application No. 12 / 260,404 entitled Application Enhancement Tracks, filed October 29, 2008, the disclosure of which is incorporated hereby by reference in its entirety. Non-standard playback tracks similar to non-standard playback tracks described in US Patent Application No. 12 / 260,404 can be used to provide a non-standard playback stream in a bit rate streaming system. adaptive according to one embodiment of the invention for providing smooth visual searching through source content encoded for adaptive bit rate streaming. A separate non-standard playback track can be encoded that appears to be an accelerated visual search through the source media when playing, when in reality the non-standard playback track is simply a separate track that encodes the source media at a rate less than frames. In various embodiments, the non-standard playback track is created by generating a non-standard playback track in the manner described in US Patent Application No. 12 / 260,404 and inserting the non-standard playback track into a Matroska container file. subject to the restrictions mentioned above regarding inserting a video stream into a Matroska container file. In many embodiments, the non-standard playback track is also subjected to the additional restriction that each frame in the GOP of each Grouping item in the non-standard playback track is encoded as an IDR frame. As with the other video streams, each Pool item contains a GOP that corresponds to the same two seconds of source media as the corresponding Pool items in the other streams. There are simply fewer frames in the GOPs of the non-standard playback track, and each frame has a longer duration. In this way, transitions to and from a non-standard playback stream can be treated in the same way that transitions between any of the other encoded streams are treated within an adaptive bit rate streaming system in accordance with embodiments of the invention. . Reproduction of the frames contained within the non-standard playback track to achieve accelerated visual search usually involves the playback device manipulating the time codes assigned to the encoded video frames before providing the frames to the decoder of the playback device to achieve a desired increase in accelerated search rate (eg x2, x4, x6, etc.).

A Bundles item is shown in Figure 4e containing encoded media from a non-standard playback track. In the illustrated embodiment, the encoded non-standard playback track is inserted within Grouping elements 48 subject to the same restrictions applied to encoded video discussed above with respect to Figure 4a. However, each Block element contains an IDR. In other embodiments, the Grouping elements within the container files that contain the non-standard playback tracks need not correspond to the start time and duration of the Grouping elements.

ES 2 807 884 T3 in the container files containing the alternate video streams.

In many embodiments, source content can be encoded to provide a single non-standard playback track or multiple non-standard playback tracks for use by the adaptive bit rate streaming system. When a single non-standard playback track is provided, the non-standard playback track is typically encoded at a low bit rate. When multiple alternative non-standard playback tracks are provided, adaptive rate streaming can also be performed with respect to the non-standard playback tracks. In various embodiments, multiple non-standard playback tracks are provided to support different accelerated visual search rates through the encoded media.

INCORPORATION OF INDEXING INFORMATION WITHIN MKV CONTAINER FILES

The specification for the Matroska container file format provides an additional Hints element that is used to index Block elements within the container file. Illustrated in Figure 5 is a modified Token element 52 that may be incorporated into a Matroska container file in accordance with one embodiment of the invention to facilitate grouping requests by a playback device using HTTP. The modified Hint element 52 includes a plurality of CuePoint elements 70 each including a CueTime attribute 72. Each CuePoint element includes a CueTrackPositions element 74 that contains the attributes of CueTrack 76 and CueClusterPosition 78. In many embodiments, the CuePoint element is primarily configured to identify a specific Grouping element as opposed to a specific Block element within a Grouping element. However, in several applications the ability to search for specific BlockGroup elements within a Grouping element is required and additional index information is included in the Hits element.

In Figure 6 the use of a modified Indicia element to index encoded media within a Clusters element of a Matroska file is illustrated in accordance with one embodiment of the invention. A CuePoint element is generated to correspond to each Grouping element within the Matroska container file. The CueTime attribute 72 of the CuePoint 70 element corresponds to the Timecode 60 attribute of the corresponding Grouping 48 element. Additionally, the CuePoint element contains a CueTrackPositions 74 element that has a CueClusterPosition 78 attribute that points to the start of the corresponding Grouping 48 element. The CueTrackPositions 74 element can also include a CueBlockNumber attribute, which is commonly used to indicate the Block element containing the first IDR frame within the Grouping element 48.

As can be easily seen the modified Hints element 52 forms an index to each of the Grouping elements 48 within the Matroska container file. Additionally, CueTrackPosition elements provide information that can be used by a playback device to request the byte range of a specific Pool element 48 via HTTP or other suitable protocol from a remote server. The Hints element of a conventional Matroska file does not directly provide a playback device with information regarding the number of bytes to request from the start of the Grouping element to get all the encoded video contained within the Grouping element. The size of a Grouping element can be inferred in a Clues element using the CueClusterPosition attribute of the CueTrackPositions element which indexes the first byte of the next Grouping element. Alternatively, additional CueTrackPosition elements could be added to the modified Tracks elements according to embodiments of the invention that index the last byte of the Grouping element (in addition to the CueTrackPositions elements that index the first byte of the Grouping element), and / or a non-standard CueClusterSize attribute specifying the size of the Grouping element pointed to by the CueClusterPosition attribute is included in each CueTrackPosition element to help with the retrieval of specific Grouping elements within a Matroska container file via requests byte range of HTTP or a similar protocol.

Modifying the Hints element in the manner described above significantly simplifies the retrieval of Grouping elements from a Matroska container file over HTTP or a similar protocol during adaptive bitrate streaming. Also, indexing only the first frame in each Pool reduces the size of the index significantly. Since the index is typically downloaded before playback, the small size of the Hits element (that is, index) means that playback can start more quickly. Using the CueClusterPosition elements, a replay device can request a stream specific Grouping element most suitable for the streaming conditions experienced by the replay device by simply referencing the relevant Matroska container file index using the Timecode attribute for the desired Grouping item.

In some embodiments, a number of the attributes within the Token element are not used during adaptive bit rate streaming. Therefore, the Hints element can be further modified by removing unused attributes to reduce the overall size of the index for each Matroska container file. Illustrated in Figure 5a is a modified Token element that can be used in a Matroska container file that includes a single encoded stream in accordance with one embodiment of the invention. The Clues element 52 'shown in Figure 5a is similar to the Clues element 52 shown in Figure 5 with the exception that the CuePoint elements 70' do not include a CueTime attribute (see 72 in Figure 5) and / or the elements of

ES 2 807 884 T3

CueTrackPositions 74 'do not include a CueTrack attribute (76 in Figure 5). When the portions of media encoded in each Grouping element in the Matroska container file have the same duration, the CueTime attribute is not required. When the Matroska container file includes a single encoded stream, the CueTrack attribute is not required. In other embodiments, the Hints element and / or other elements of the Matroska container file can be modified to remove elements and / or attributes that are not necessary for adaptive bitrate streaming of the encoded stream contained within the Matroska container file, given the way the stream is encoded and inserted into the Matroska container file.

Although various modifications to the Hits element are described above to include information regarding the size of each of the Grouping elements within a Matroska container file and to remove unnecessary attributes, many embodiments of the invention utilize a conventional Matroska container. In various embodiments, the playback device simply determines the size of Grouping items on the fly using information obtained from a conventional Token item, and / or relies on a separate index file containing information regarding size and / or or location of the Grouping elements within the MKV container file. In various embodiments, the additional index information is stored in the top-level index file. In a number of embodiments, the additional index information is stored in separate files that are identified in the top-level index file. When index information used to retrieve Grouping items from a Matroska container file is stored separately from the container file, the Matroska container file is still typically restricted to encode media for inclusion in the Grouping elements in the manner described above. In addition, whenever index information is located, the index information will typically index each item in the Grouping and will include (but not be limited to) information regarding at least the starting location and, in many cases, the size of each. Grouping item.

CODING OF SOURCE MEDIA FOR CONTINUOUS BROADCAST OF ADAPTIVE BIT RATE

Illustrated in Figure 7 is a process for encoding source media as a top-level index file and a plurality of Matroska container files for use in an adaptive bit rate streaming system in accordance with one embodiment of the invention. The encoding process 100 begins by selecting (102) a first portion of the source media and encoding (104) the source media using the encoding parameters for each stream. When the media portion is video, then the source video portion is encoded as a single GOP starting with an IDR frame. In many embodiments, encoding parameters used to create the alternative GOPs vary based on bit rate, frame rate, encoding parameters, and resolution. In this way, the media portion is encoded as a set of interchangeable alternatives and a playback device can select the alternative most appropriate to the streaming conditions experienced by the playback device. When different resolutions are supported, the encoding of the streams is restricted so that each stream has the same display aspect ratio. A constant display aspect ratio can be achieved across streams of different resolution by varying the sample aspect ratio with the resolution of the stream. In many cases, reducing resolution can result in higher quality video compared to higher resolution video encoded at the same bit rate. In many embodiments, the source media is encoded itself and the encoding process (104) involves transcoding or transclassing the source media encoded in accordance with the encoding parameters of each of the alternate streams supported by the broadcast system. adaptive bit rate streaming.

Once the source media has been encoded as a set of alternate encoded media portions, each of the alternate encoded media portions is inserted (106) into a Grouping element within the Matroska container file that corresponds to the stream to which the encrypted media portion belongs. In many embodiments, the encoding process also builds indexes for each Matroska container file as media is inserted into Grouping elements within the container. Therefore, process 100 can also include creating a CuePoint element that points to the Grouping element inserted within the Matroska container file. The CuePoint item can be kept in a buffer until the source media is fully encoded. Although the above process describes encoding each of the alternate portions of sequentially encoded media in a single pass through the source media, many embodiments of the invention involve performing a separate pass through the source media to encode each of alternative flows.

Referring back to Figure 7, the process continues to select (102) and encode (104) portions of the source media and then insert (106) the encoded media portions into the Matroska container file that corresponds to the appropriate stream until that all source media is encoded for adaptive bit rate streaming (108). At which point, the process can insert an index (110) into the Matroska container for each stream and create (112) a top-level index file that indexes each of the encoded streams contained within the Matroska container files. As noted above, indexes can be created as encoded media is inserted into the Matroska cabinet files so that a CuePoint item indexes each Grouping item within the Matroska cabinet file. After the completion of the encoding, each of the CuePoint elements can be included in an element of Hints and

ES 2 807 884 T3 the Clues element can be inserted into the Matroska container file after the Clusters element.

Following the encoding of the source media to create Matroska cabinet files that contain each of the streams generated during the encoding process, which may include the generation of non-standard playback streams, and a top-level index file that indexes each of the streams within the Matroska container files, the top-level index file and Matroska cabinet files can be uploaded to an HTTP server for adaptive bitrate streaming to playback devices. Adaptive bit rate streaming of encoded media in accordance with embodiments of the invention using HTTP requests is further discussed below.

CONTINUOUS DIFFUSION OF ADAPTIVE BITRATE FROM MKV CONTAINER FILES USING HTTP

When source media is encoded so that there are alternate streams contained in separate Matroska container files for at least one video, audio, and subtitle content, adaptive streaming of the media contained within the Matroska container files can be achieved using HTTP requests or a similar stateless data transfer protocol. In many embodiments, a playback device requests the server-resident top-level index file and uses the index information to identify the streams that are available to the playback device. The playback device can then retrieve the indexes for one or more of the Matroska files and can use the indexes to request media from one or more of the streams contained within the Matroska cabinet files using HTTP requests or using a similar stateless protocol. . As noted above, many embodiments of the invention implement the indexes for each of the Matroska cabinet files using a modified Indexes element. In a number of embodiments, however, the encoded media for each stream is contained within a standard Matroska cabinet file, and separate index file (s) may also be provided for each of the cabinet files. Based on the streaming conditions experienced by the playback device, the playback device can select alternate stream media encoded at different bit rates. When media from each of the streams is inserted into the Matroska container file in the manner described above, transitions between streams may occur upon completion of media playback within a Grouping item . Therefore, the size of the Grouping elements (that is, the duration of the media encoded within the Grouping elements) is usually chosen so that the playback device is capable of responding fast enough to broadcast conditions in continuously changing and user instructions involving the use of a non-standard playback track. The smaller the Pool items (that is, the shorter the duration of the encoded media within each Pool item), the greater the overhead associated with requesting each Pool item. Therefore, there is a trade-off between the sensitivity of the playback device to changes in streaming conditions and the effective data rate of the adaptive streaming system for a given set of streaming conditions (i.e., the portion of the available bandwidth actually used to stream encrypted media). In many embodiments, the size of the Pool items is chosen such that each Pool item contains two seconds of encoded media. In other embodiments, the duration of the encoded media may be greater or less than two seconds and / or the duration of the encoded media may vary from Pool item to Pool item.

Figure 8 illustrates communication between a playback device or client and an HTTP server during playback of media encoded in separate streams contained within Matroska container files indexed by a top-level index file in accordance with one embodiment of the invention. invention. In the illustrated embodiment, the playback device 200 begins playback by requesting the top-level index file from the server 202 using an HTTP request or similar protocol to retrieve data. Server 202 provides the bytes that correspond to the request. Playback device 200 then parses the top-level index file to identify the URIs of each of the Matroska container files that contain the encoded media streams derived from a specific piece of source media. The playback device can then request the byte ranges that correspond to headers from one or more of the Matroska cabinet files via HTTP or a similar protocol, whereby the byte ranges are determined using the information contained in the URI. for the relevant Matroska cabinet files (see description above). The server returns the following information in response to a request for the byte range that contains the headers of a Matroska container file:

ELEM (EBML)

ELEM (SEEKHEAD)

ELEM (SEGMENTINFO) ELEM (TRACKS)

The EBML element is routinely processed by the playback device to ensure that the file version is supported. The SeekHead element is parsed to find the location of the Matroska index elements and the Segmentlnfo element contains two key elements used in replay: TimecodeScale and

ES 2 807 884 T3

Duration. The TimecodeScale specifies the timecode scale for all code times within the Segment of the Matroska container file, and the Duration specifies the duration of the Segment based on the TimecodeScale. The Tracks element contains the information used by the playback device to decode the encoded media contained within the Bundles element of the Matroska file. As indicated above, adaptive bit rate streaming systems according to embodiments of the invention can support different encoded streams using different encoding parameters including but not limited to frame rate and resolution. Therefore, the playback device can use the information contained within the Matroska container file headers to configure the decoder each time a transition is made between encoded streams.

In many embodiments, the playback device does not retrieve the headings for all Matroska cabinet files indexed in the top-level index file. Instead, the playback device determines the stream or streams that will be used to initially start playback and requests the headings of the corresponding Matroska cabinet files. Depending on the structure of the URIs contained within the top-level index file, the playback device can use either URI information or Matroska cabinet file header information to request byte ranges from the containing server at least a portion of the index from relevant Matroska cabinet files. Byte ranges can correspond to the entire index. The server provides the relevant byte ranges containing the index information to the playback device, and the playback device may use the index information to request the byte ranges of Pool items containing encoded media using this information. When the Grouping elements are received, the playback device can extract encoded media from the Block elements within the Grouping element, and can decode and play the media within the block elements according to their associated Timecode attributes.

In the illustrated embodiment, the playback device 200 requests sufficient index information from the HTTP server prior to the start of playback that the playback device can stream all of each of the selected streams using the index information. In other embodiments, the playback device continually retrieves index information as the media is played. In various embodiments, all the index information for the lower bit rate stream is requested before playback so that the index information for the lower bit rate stream is available to the playback device in the event that streaming conditions deteriorate rapidly during playback.

SWITCHING BETWEEN FLOWS

The communications illustrated in Figure 8 assume that the playback device continues to request media from the same streams (ie Matroska container files) throughout the playback of the media. In reality, the streaming conditions experienced by the playback device are likely to change during the playback of the streaming media and the playback device may request alternate streaming media (i.e. different Matroska container files) for provide the best snapshot quality for the streaming conditions experienced by the playback device. Furthermore, the playback device can switch streams to perform a non-standard playback function using a non-standard playback track stream.

In Figure 9a communication between a playback device and a server is illustrated when a playback device switches to a new stream in accordance with embodiments of the invention. The communications illustrated in Figure 9a assume that the playback device has not previously requested the index information for the new stream and that the download of Grouping items from the old stream continues while information is obtained regarding the Matroska container file containing the stream. new flow. When the playback device 200 detects a change in streaming conditions, determines that a higher bit rate stream can be used under the present streaming conditions, or receives a non-standard playback instruction from a user, the device Replayer can use the top-level index file to identify the URI for an alternate stream more appropriate for at least one of the video streams, audio or subtitles from which the playback device is currently requesting encrypted media. The playback device may save the information regarding the current stream (s) and may request the header byte ranges for the Matroska container file (s) containing the new stream (s) using the parameters of the corresponding URIs. Caching information in this way can be beneficial when the playback device tries to tailor the stream's bitrate downward. When the playback device experiences a reduction in available bandwidth, the playback device will ideally switch quickly to a lower bitrate stream. Due to the reduced bandwidth experienced by the playback device, the playback device is unlikely to have additional bandwidth to request header and index information. Ideally, the replay device uses all available bandwidth to download already requested Higher Rate Pooling items and uses locally cached index information to initiate requesting Pooling items from file or Matroska container files that contain stream or streams lower bit rate.

Byte ranges for index information for the Matroska cabinet file or files that contain the stream

ES 2 807 884 T3 or new streams can be requested from the HTTP server 202 in a manner similar to that described above with respect to Figure 8. At each point, the replay device can stop downloading Pool items from previous streams and can begin requesting byte ranges of the appropriate Pool items from the Matroska cabinet file (s) that contain the new stream (s) of the stream. HTTP server, using the index information of the Matroska cabinet file (s) to identify the Pool item (s) containing the encoded media following the media encoded in the last Pool item retrieved by the playback device. As noted above, the smooth transition from one stream to another is facilitated by encoding each of the alternate streams so that corresponding Grouping elements begin with the same Timecode element and an IDR frame.

When the playback device caches the header and the entire index for each stream that has been used in media playback, the process of switching back to a previously used stream can be simplified. The playback device already has the header and index information for the Matroska file that contains the previously used stream and the playback device can simply use this information to initiate the request for Grouping items from the Matroska container file of the previously used stream to via HTTP. Figure 9b illustrates communication between a playback device and an HTTP server when switching back to a stream or streams for which the playback device has cached header and index information in accordance with one embodiment. of the invention. The process illustrated in Figure 9b is ideally performed when down-tuning bitrate, because a reduction in available resources can be exacerbated by a need to download index information in addition to media. The probability of playback interruption is reduced by increasing the speed with which the playback device can switch between streams and by reducing the amount of overhead data downloaded to achieve switching.