Provision of video data

0001This invention relates to the provision of video data over a communications medium. In particular it relates to the playback of recorded video data over a data network such as the “Internet”.

0002One technique for delivering video data over a network is to split the data into equal-sized “chunks”. Each chunk can be uniquely identified by a URI (Uniform Resource Identifier) and a client wishing to receive such a video sequence will initially download a manifest file that describes the sequence of URIs that it needs to request in order to play the video stream.

0003In some systems a separate set of metadata (a media presentation description—MPD) is associated with each of several presentation modes, such as required for different bitrates. A separate MPD may be provided for use when the user has selected a “trick mode” such as “fast forward”, as is described by Irij Sodagar “The MPEG-DASH standard for multimedia streaming over the Internet” (IEEE Multimedia, New York, USA, vol 18, No 4—1 Apr. 2011 pages 62-67 ISSN 1070-986X). Thus the metadata will depend on the playback speed (normal/fast) that the user has selected.

0004Various ways have been devised to generate “bookmark” metadata to identify the locations of highlights within a longer video sequence. It is known′, for example from International Patent Specification WO2004/025508 to provide an automated procedure to identify ‘highlights’ within a video sequence in order to provide the viewer with navigation aids or to simplify and reduce costs in video production and editing. The prior art typically identifies video clips automatically through analysis of changes in video or audio levels within a video sequence. More specifically, the boundaries of the ‘highlight’ video segment are typically identified using various super-histograms, frame signatures, cut detection methods, closed caption information, audio information, and so on, by analysing the visual, audio and transcript portions of the video signal. For example, based on the volume of audience cheers (U.S. Pat. No. 7,831,112) recorded in response to the performance of the athletes, or analysis of the area of the image depicting the score to detect changes (U.S. Pat. No. 7,983,442). It is also known, for example from European Patent Application EP0142178, to provide for a viewer to manually bookmark a ‘highlight’ for later access. The bookmarks can be augmented with metadata identifying the nature of the event (score, dismissal, accident etc.), the individual competitors involved, etc., to allow a user, or an automated system programmed with a user's preferences, to compile a sequence tailored to the user's interests.

0005As described in our International patent application GB2014/000036, each chunk of video data (which typically may represent 2 seconds of content) is given a Narrative Interest Value (NIV), to indicate the likely interest to the viewer of the content. This value can be used by the download apparatus to select which chunks to display, thereby allowing a highlights programme to be generated according to the individual viewer's requirements.

0006The NIV is derived from an event in the video sequence that is deemed to be of significant interest e.g. a goal in a football match. The actual event is recorded by an Event Marker which identifies the chunk of video in which the event occurs. This chunk of data is given a NIV of 1. Successive chunks of data either side of the Event Marker (EM) have decreasing narrative interest and are marked appropriately 2, 3, 4 etc. A chunk with a NIV of 1 has the most interest, and interest decreases as the numerical value of the NIV increases.

0007As described in our International Patent application GB2014/000037, metadata may be used to skew the duration of a video clip to be asymmetric with respect to the bookmark. Different events in a video stream will have different distributions of narrative interest in relation to the Event Marker. For example, for a goal in football the most interesting content for the viewer will be that which precedes the event, whereas for the award of a penalty or free kick it is what happens after the event that is more interesting. To take this into account, a Pre/Post Event Ratio (PPER) is defined that skews the distribution of NIV values around the event marker. For example a PPER of 1:1 indicates that chunks either side of the event marker carry equal significance, whereas a PPER of 3:1 is appropriate if the content before the event marker is more interesting than that following the event marker.

0008A video sequence may contain a number of event markers. Each chunk in the video sequence may consequently have several NIV values, each attributed to a particular event marker. A Combined NIV (CNIV) can be calculated for each chunk by considering all the individual NIVs associated with that chunk. In the preferred method the CNIV for each chunk is defined to be the minimum NIV value associated with that chunk.

0009Viewers watching a programme, particularly of a sporting event, often wish to see only the “highlights” rather than its full duration. Traditionally, highlights programmes have been manually edited and compiled by the broadcaster, often with added commentary or analysis. Such an arrangement is prescriptive, and can be labour-intensive and time-consuming to produce.

0010It is desirable to provide a “catch-up” facility to enable a viewer starting to view a match already in progress to watch the highlights of that part of the match that has already taken place before watching the remainder of the match “live” (in real time). The relative importance of events in the match, and therefore whether they merit space in a catch up sequence, will change dynamically as the match progresses, so a catch-up sequence to cover the first twenty minutes of a game may include very little of the content of a sequence of the same length covering the first ten minutes. The desired duration of the “catch-up” sequence may also vary: for example a longer catch-up sequence may be appropriate if the viewer joins during an interlude when no live action is taking place, such as a half-time break.

0011Most video playback devices have a facility known as “accelerated play” “fast forward” or “trick play” which allows a viewer of stored or recorded content to move at faster (or slower) than the real-time speed of the programme, in order to step rapidly forward or back to different parts of the sequence, or to interrupt (pause) the sequence. In many systems some content continues to be visible during fast modes, which aids identification of the content required. A viewer can use this to “fast forward” through a recording of the programme, reverting to normal speed to watch sections that catch his interest. However, it can be difficult to identify the sections that are interesting. In particular, the user may have to backtrack, having overshot. This can be time-consuming.

0012The invention provides a method for playing video data retrieved from a database as a plurality of video segments, each segment having metadata associated therewith indicative of the relative narrative interest of the respective segment of the data, wherein the video data is capable of being played at a selectable plurality of speeds, and wherein the speed at which each segment is played is selected by a playback apparatus, in which the playback apparatus detects the metadata associated with each segment, performs an analysis of the metadata to determine the relative narrative interest of the segment, and selects a playback speed for each segment in accordance with the relative narrative interest derived from the metadata associated with that segment.

0013The invention therefore uses the metadata to control the playback speed. Thus the playback speed (normal/fast etc) of each segment will depend on the metadata associated with that segment.

0014In the preferred arrangement, the speed of each segment is controlled in accordance with a Narrative Interest Value for each segment, derived from the proximity of the segment to an event marker.

0015The Event Markers may be created/determined in a number of ways. For use in a catch-up facility they would need to be created as the events take place, in near-real time. In one approach, the Event Markers may be created by a studio/production house as part of the editing of the video stream. Another approach is through the aggregation of “bookmarks” supplied by other viewers (<figref idref="DRAWINGS">FIG. 2</figref>) as described in our earlier International patent application GB2014/000037.

0016The invention also provides a video playback device having a store for storing video content and associated metadata, a playback element for playing the stored video content, a “trick play” system for controlling the playback element to allow the stored video content to be played at a selectable plurality of speeds, and wherein the trick play system has a speed control input controllable in response to metadata retrieved from the store and associated with the video content.

0017A catch-up sequence, or other video highlights sequence can therefore be compiled by governing the video sequence playback speed according to the associated metadata. The metadata can be compiled according to the likely interest of the associated content to particular users, so that on playback the content of most interest to the user requesting it would be played at normal speed (or even in slow motion) while less interesting content would be played back faster. This provides potentially visually appealing content to the user.

0018In a “catch-up” facility, provided to allow a viewer who has missed the start of a game to view most of the remainder in “real time”, but desires to first see highlights of the part he has missed, requires the sequence to be generated almost in real time. Satisfying this requirement requires compiling a sequence “on the fly” tailored to the content that has already passed, and may need modification even as the sequence is viewed, should a further significant event take place whilst the catch up sequence is being viewed, either by extending the duration of the catch up sequence, or omitting part of it to accommodate the extra event.

0019The playback device may be embodied in a consumer product such as a “set top box” associated with the user's viewing apparatus. In accordance with this aspect, it may provide a video playback device incorporated in a client device and having a receiving system for receiving video data and associated metadata, a processor for adapting a playback speed of the video data in accordance with the associated metadata, and a video playback system for playing the video data at the adapted speed.

0020However, in the preferred arrangement, the invention may be embodied in a server device for control, through a data connection such as an Internet connection, by an end user (“client”) device associated with the user's viewing apparatus. In this aspect a video playback device is incorporated in a server platform, having an addressable storage facility for storing a plurality of sets of video data arranged to be played at different respective speeds, and a store for address data associated with the video data, and a store for metadata associated with the video data, a manifest compiling system for compiling a set of addresses of video data selected from the stored sets of video data in accordance with the metadata associated with the video data

0021In the preferred arrangements, only the video content that is actually to be played is transmitted over the data connection between the data provider and the user. This reduces the amount of data required to be downloaded, making it suitable for devices on mobile networks and other low bandwidth networks, and reducing the download time, so avoiding buffering problems that could be caused if the playback speed were to exceed the speed the network is capable of supporting. This may be achieved either by performing the selection process at the head end (service provider) or by having the client device performing the selection process and compiling a manifest request specifying each part of the content in the required final format (resolution, quality and associated bit rate). Several mechanisms can be used to achieve this, and will be described briefly here and in more detail later.

0022In a first arrangement, the live video stream may be encoded at a defined bit rate for subsequent retransmission at a number of predefined playback speeds, using an Adaptive Bit Rate technology (ABR). An MPD (Media Presentation Description) file would then be created which defines from which stream each section of the video should be selected for presentation to the viewer.

0023Use of the MPD file (similar to previously described manifest file) would enable personalised streams to be created. It would also enable different lengths of content to be created suited to the required catch-up time required.

0024In a second arrangement, a video file is created which unifies the variable speed videos into a ‘single’ stream. This stream could also be encoded with ABR, enabling delivery suited to available bandwidth.

0025In a third arrangement, the video is encoded at normal (×1) speed using ABR, and when a catch-up is required the video is streamed faster than real-time for playback on the client device, which would control the playback speed of the video. To provide some level of bandwidth saving to be performed, ABR could be used to reduce the bitrate per frame, for video sections that are to be played back at high speed e.g. ×16, for example by only transmitting “I-frames” or reducing error checking.

0026A fourth embodiment uses the MPD file to define byte ranges within video segments which would be requested by the client and downloaded for playback.

0027In the preferred embodiment the metadata comprises a “narrative interest value” (NIV) assigned to each segment. The NIV for each segment can be generated by the content provider or by the playback device itself, and can be derived from the chronological distance (number of segments) between one or more bookmarks or “event markers” and the segment in question. In the embodiment described, an NIV derived from one or more bookmarks is referred to as a “combined NIV” (CNIV).

0028In an embodiment, a smoothing function may be applied to the NIV values, or to the resulting playback speeds, to prevent frequent changes in playback rates.

0029The range of NIVs appropriate to each playback speed may be selected such that the overall duration of the sequence, or the time that it is completed, meets a predetermined condition.

0030In one embodiment, event markers applied by multiple viewers are aggregated. A threshold can be applied to limit the number of aggregated event markers by only considering regions of the video sequence where there have been more than a minimum number of user-supplied event markers for a given event type within a given timeframe.

0031In use, the viewer can therefore download and play back content in a special mode in which each “chunk” of content is played back at a rate which is related to the CNIV associated with that chunk.

0032The playback system may ascribe different weightings of NIV values to different events, for example to take into account a viewer's preferences—for example to pay more attention to one team's scoring successes than those of their opponents. This can be done by ascribing an NIV other than “1” to an event of minor significance, according to the associated metadata, or by adjusting the rate at which NIVs vary with proximity to their associated event marker.

0033An embodiment of the invention will now be described, by way of example, and with reference to the drawings, in which:

0034<figref idref="DRAWINGS">FIG. 1</figref> depicts a media server, a content server, and an associated client device arranged to operate according to the invention to control a catch up sequence of a video transmission

0035<figref idref="DRAWINGS">FIG. 2</figref> is a flow diagram illustrating the processes performed in the creation of bookmark timestamp data.

0036<figref idref="DRAWINGS">FIG. 3</figref> is a diagram illustrating the capture of bookmark data.

0037<figref idref="DRAWINGS">FIG. 4</figref> is a further diagram illustrating the capture of bookmark data

0038<figref idref="DRAWINGS">FIG. 5</figref> illustrates a first method of operation of the invention

0039<figref idref="DRAWINGS">FIG. 6A</figref> illustrates a second method of operation of the invention

0040<figref idref="DRAWINGS">FIG. 6B</figref> illustrates a third method of operation of the invention

0041<figref idref="DRAWINGS">FIG. 7</figref> illustrates a fourth method of operation of the invention

0042A number of mechanisms can be used to enable the creation of ‘bookmarks’ for events in sports TV programmes, and to enable service providers to utilise these bookmarks in the identification, delivery and provision of video programmes. One such system, fully described in our co-pending applications GB2014/000036 and GB2014/000037, will be summarised here with reference to <figref idref="DRAWINGS">FIGS. 1, 2, 3 and 4</figref>. It will be appreciated that the present invention does not depend on the way the event markers were created. This brief description is included for completeness, but the reader is referred to those applications for further details.

0043<figref idref="DRAWINGS">FIG. 1</figref> illustrates a video playback device <b>41</b> for use by a viewer to download video data from a server <b>420</b> and play the data on an output device <b>40</b> such as a conventional television set.

0044Users can identify key events (such as goals) within a video sequence, either during live transmission or whilst watching a recording, and mark them using ‘temporal bookmarks’. These ‘temporal bookmarks’ can then be used by that viewer to navigate easily back to those events in the video sequence. They may also be transmitted to a database <b>43</b> to allow their use by other viewers to identify the parts of the video content which are considered, by the viewers, to be of particular interest. Bookmarked events are stored as time-codes along with associated metadata <b>44</b> provided by the viewers to classify the events. The metadata enable individual bookmarked sections to be classified so as to allow individual sections to be prioritised for viewing depending on the interests of the viewer.

0045As shown in <figref idref="DRAWINGS">FIGS. 1 and 2</figref>, the viewer uses a video device <b>40</b> such as a television set to view broadcast or recorded video content such as coverage of a sports event. For ease of use the viewer may also be using a tablet device <b>41</b> or other terminal onto which has been loaded a software application (“App”). A number of such applications are available which can synchronise such a terminal to the output of a video device <b>40</b> linked to a server <b>42</b> so that the server can monitor the content being watched by the viewer. The server <b>42</b> also has data related to the content being watched. For example for a football match, it would have data on the identities of the teams and individual players. The App provides a user interface that enables the viewer to bookmark the video stream and add metadata to that bookmark.

0046The application loaded on the tablet device <b>41</b> allows a user to bookmark events in the television content and add metadata relating to that event, to be stored on the server <b>42</b>. This process, illustrated in <figref idref="DRAWINGS">FIG. 2</figref>, enables the user to locate the event in order to share a video clip of it with other users using a social network, or to review the clip again after the event, or participate in a vote or competition which requires the nomination and submission of a clip.

0047As shown in <figref idref="DRAWINGS">FIG. 2</figref>, the viewer ‘bookmarks’ the event by first selecting an action (goal, free kick, etc.). This creates an “event marker” The event marker is then saved to the server together with metadata items, for example identifying the sportsperson involved). This bookmarking process is identified collectively as step <b>60</b> in <figref idref="DRAWINGS">FIG. 2</figref>. Each event is identified using a single point in time within a video sequence, referred to herein as an Event Marker (EM). The EM would typically identify the exact moment of the event of interest, for example when a ball crosses the goal mouth, or the moment a bad tackle was committed. Each EM is specified by a time-code, frame or video chunk identifier within a video sequence.

0048The viewer may also provide additional metadata associated with the bookmark (step <b>63</b>). This would include further information such as the type of event—(goal, pass, shot on goal, save, corner kick, red card, etc.), the main primary person associated with the event—(Footballer, Referee, Manager, linesman, etc.), and comments or descriptions the viewer may wish to add to the event to support later retrieval or to add context to the video clip.

0049The event markers and additional metadata are recorded in associated XML files <b>43</b>, <b>44</b>, (steps <b>62</b>, <b>64</b>) typically stored in association with the server device <b>42</b> so as to be accessible to both the viewer's Local Area Network and the video service provider.

0050There may be a mismatch between when the event actually occurred and when the user pressed the ‘save bookmark’ button. This mismatch can be reduced using either or both of the following methods.

0051The server <b>42</b> can modify the bookmark time-code by using a user-specific Reaction Time Offset value (RTO), (step <b>61</b>) which attempts to minimise differences between when the event actually occurred and when the viewer indicated it occurred. The RTO for each user can be calculated by using reference video content in which event markers have previously been identified. The average difference between the time-code for a reference event marker and the time code of the equivalent event marker as identified by an individual viewer can be used to calculate that users RTO. In <figref idref="DRAWINGS">FIG. 2</figref> this process <b>61</b> is depicted as being performed by the server platform <b>42</b>, but it could be done by the client device <b>41</b>.

0052The bookmark time-code can be further improved through comparison with bookmarks (<b>600</b>, <b>601</b>, <b>602</b>) created by other viewers. By identifying a significant number of bookmarks saved by viewers relating to the same event (as identified by metadata applied by the users, or simply by chronological proximity to each other), and calculating a single representative value from them, a more accurate time code for an event marker can be calculated (step <b>65</b>). This time-code adjustment process would take place on the server's centralised bookmark store <b>42</b> and returned to the event marker store <b>43</b>.

0053An example of the process performed in our co-pending applications can be understood with reference to <figref idref="DRAWINGS">FIG. 3</figref>. The curve <b>90</b> represents the cumulative total, over time, of bookmarks <b>60</b>, <b>600</b>, <b>601</b>, <b>602</b> etc. transmitted by different users in relation to a single event, and identified as relating to the same event by comparison of the metadata <b>63</b> associated with them. The horizontal time axis is graduated in terms of individual “chunks” <b>1</b>, <b>2</b>, <b>3</b>, . . . <b>22</b> of video data, typically each of about two seconds in duration, into which the stream is divided.

0054The details of the shape of the distribution of bookmarks over time will vary somewhat depending on the type of event being flagged. Empirical systems may be used to estimate, using the shape of the distribution <b>90</b> and the metadata <b>64</b> supplied by the users bookmarking the data, the optimum time to place the event marker <b>93</b>. For example the event marker can be defined as a predetermined percentile of the bookmark times—for example the first quartile (25^thpercentile) <b>92</b>, or the median (50^thpercentile) <b>95</b>, depending on the type of event being bookmarked. In the example depicted in <figref idref="DRAWINGS">FIG. 3</figref> the event marker <b>93</b> is placed at the median point, which is chunk <b>12</b>. For some types of event, different criteria may be found to be optimum, for example the steepest point of the cumulative curve, representing the statistical mode of the distribution, may be used

0055<figref idref="DRAWINGS">FIG. 4</figref> illustrates how event markers may be generated in the inventions of our co-pending applications from peaks in the number of bookmark events generated by individual viewers. The location of each peak identifies an event EM<b>1</b>, EM<b>2</b> etc. A weighting a, b, c may be applied to the NIVs associated with each event marker according to the number of bookmark events associated with it.

0056As well as the viewer-participation process depicted in <figref idref="DRAWINGS">FIG. 2</figref>, metadata can also be created automatically to simplify the bookmark creation process. As shown in <figref idref="DRAWINGS">FIG. 1</figref>, for example, using other data sources <b>444</b> (for example the real-time data feed provided by OPTA Sports Ltd) it would be possible for the system to provide data on which player had possession of the ball when the ‘save bookmark’ button was selected by the viewer.

0057The refined or aggregated bookmarks are added to a bookmark list <b>43</b> stored on the media server <b>42</b> available to the application stored on individual client devices <b>41</b>.

0058Each bookmark (or Event Marker) (e.g “<b>93</b>”, <figref idref="DRAWINGS">FIG. 3</figref>) only defines a specific moment in time. By dissecting large video files into multiple smaller files (called chunks), each being for example two seconds in length, a video “clip” can be defined by specifying a number of chunks that occur before and after a given Event Marker.

0059As described in the co-pending applications referenced above, a number of “chunks” preceding and following an event marker are each allocated a respective “narrative interest value” (NIV). The event marker itself is allocated a NIV of 1, and the other chunks are allocated NIVs proportional to their distance from the event marker. The NIVs may increase at different rates before and after the event marker, in proportions determined by a “pre/post event ratio” (PPER) defined by metadata associated with the event marker. So, for example, with a PPER of 1:1, chunks will be allocated NIVs in ascending order at the same rate both before and after the event marker. For a PPER of 2:1 two chunks will be allocated before the EM for each chunk afterwards. Similarly for a PPER of 3:1—three chunks are downloaded preceding the EM for each one following it.

0060In the co-pending applications, each chunk in a sequence has a unique NIV. This is done because in those earlier inventions the NIVs determine the order in which the video chunks are to be downloaded. In the present invention, the catch up sequence is to be viewed immediately, so the order of download will typically be the chronological order of the events depicted, and the NIV is used for a different purpose. It is therefore possible for a plurality of chunks associated with a single event marker to all be allocated the same NIV. However, in this embodiment a unique NIV is allocated to each chunk of a given sequence. This allows the same NIV allocation process to be used as for the earlier inventions. The same NIVs can then be used both for download and for playback.

0061The present invention allows this bookmark data to be used to download a video package at a variable rate, as will now be described. Returning to discussion of <figref idref="DRAWINGS">FIG. 1</figref>, this Figure depicts a client device <b>41</b> having an output to a television set <b>40</b> or other monitor device, and a connection over a communications network to a media server <b>42</b> and a content server <b>420</b>. The media server <b>42</b> has a number of associated data stores <b>43</b>, <b>44</b>, <b>45</b>, <b>46</b> which store event markers and other metadata associated with the video data, and manifest files for specifying which parts of the video data are to be delivered to individual clients, and in what order. Some manifest files <b>46</b> may be pre-prepared so that they can be downloaded on request, but for the catch-up service which is the subject of the present invention the manifest is created dynamically, in response to specific requests <b>69</b> from clients, as each manifest needs to cover events recorded over a period which ends at the instant the request is made.

0062The client server <b>41</b> has a store for downloaded manifest files <b>46</b>, and may also have a store of personalised priorities <b>47</b> for modifying the manifest file in order to prepare a request <b>48</b> for delivery of video content. In other embodiments personalisation is done using an agent <b>470</b> in the media server.

0063The server content platform <b>420</b> has a store of video content <b>49</b> from which is retrieved the content <b>490</b> specified in the request <b>48</b> for delivery to the client device <b>41</b>.

0064The steps in the delivery process are as follows:

0065The client <b>41</b> requests a “catch up” manifest file <b>46</b> from the server <b>42</b> (step <b>69</b>), This file may constitute several parts, each containing a sequence of “chunks”. For example, separate parts may be defined by interruptions or scheduled breaks in play, so that the catch up sequence skips those breaks, rather than running through them at fast forward speed, (Even at 16× speed, the standard fifteen-minute half-time break in Association Football (“Soccer”) would take nearly 60 seconds to play)

0066The manifest file is compiled from the EM and NIV data (step <b>70</b>—<figref idref="DRAWINGS">FIG. 2</figref>) For a catch-up facility the length of the sequence video is defined by the server, taking into account the number of interesting events that have been flagged so far, the personal profile of the viewer, which may affect which of those events are expected to be of most interest to that particular viewer, and any fixed points in the schedule of the live play, for example the end of a scheduled break (e.g. half-time) at which time it would be desirable to return to the live broadcast. For example, the length of the catch-up video may be set to be a standard duration, but capable of being extended to the end of such a natural break if it would otherwise end during that break.

0067For reach such request <b>69</b> from a client the media server <b>42</b> uses the data <b>44</b>, <b>45</b> associated with each event marker (steps <b>701</b>, <b>702</b>) to determine a priority list (step <b>703</b>). This list may be tailored towards the user's preferences—for example in a football game the user may give priority to viewing goals scored for the team he supports rather than those scored for the opposition, and may prioritise goals over other incidents, such as events surrounding a caution or dismissal of a player, and generates a manifest file <b>46</b> specifying the segments required and the speed at which each segment is to be presented.

0068Processes by which the manifest file <b>46</b> may be compiled will be described later, with reference to <figref idref="DRAWINGS">FIGS. 6A and 6B</figref>

0069The server <b>42</b> delivers the manifest files <b>46</b> to the client <b>41</b> (step <b>72</b>). The Client <b>41</b> reads the manifest file <b>46</b> (step <b>73</b>), and interprets it to generate a video chunk list <b>48</b>. This is a list of file addresses from which the required video chunks are to be downloaded. The client <b>41</b> transmits this chunk list <b>48</b> to the content server <b>420</b> (step <b>76</b>).

0070The media server <b>42</b> and content server <b>420</b> may in practice be associated in a single platform or, as shown, with the generation of the manifest file <b>46</b>, and the locations of the video chunks <b>49</b> to be called for by that manifest file, located in different parts <b>42</b>, <b>420</b> of a distributed system.

0071The server <b>420</b> retrieves the video chunks <b>49</b> and transmits them in the order specified in the request (step <b>77</b>)

0072The client <b>41</b> receives the video chunks <b>49</b> and a copy is made <b>490</b> which is processed (buffered, decompressed, etc) for display.

0073The manifest file could include the NIV data by including a special field into an MPEG DASH (Dynamic Adaptive Streaming using HTTP) MPD file. A DASH client would retrieve and play the video content using these steps: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0074">1. The client downloads and reads the MPD file to get important information, such as the content locations, segment encodings, resolution, minimum and maximum bandwidths, accessibility features like closed captioning and content restrictions (such as DRM).</li><li id="ul0002-0002" num="0075">2. The client selects an appropriate segment encoding and begins streaming the content through a series of HTTP requests. The server creates and encodes each segment on demand for each request, all from the same source.</li><li id="ul0002-0003" num="0076">3. The client buffers data as it is downloaded, while also keeping track of fluctuations in the connection bandwidth. If necessary, the client automatically changes to a different segment encoding (from those listed by the MPD file) that's more compatible with the current bitrate. This ensures the client maintains a sufficient buffer throughout the video without downloading more data than is needed.</li></ul></li></ul>

0077<figref idref="DRAWINGS">FIG. 5</figref> illustrates how the NIV data stored in the server may be used to vary the playback speed, in accordance with the invention. It depicts a sequence <b>20</b> of fourteen segments <b>20</b><i>a</i>-<b>20</b><i>n</i>. Two event markers EM<b>1</b>, EM<b>2</b> are associated, respectively, with segments <b>20</b><i>g</i>, <b>20</b><i>j</i>. These have PPERs of 1:2 and 2:1 respectively, resulting in different sequences of NIVs, as indicated at references <b>211</b>, <b>212</b>. NIVs are allocated with ascending values for the chunks following the event marker in chronological order, and NIVs are allocated with ascending values for the chunks preceding the event marker, in reverse chronological order, in proportions determined by the respective PPER for that sequence. A combined NIV is generated for each segment by taking the minimum value of the NIVs <b>211</b>, <b>212</b> for that segment. The sequence of combined NIVs is depicted at reference <b>210</b>, together with the marker (EM<b>1</b>, EM<b>2</b>) from which each MV is derived. (EM<b>12</b> where both EM values are the same)

0078A maximum NIV is set, in this example at 11, and any segment more remote than the segment with maximum NIV is assigned a null NIV, as shown at reference <b>200</b>.

0079The CNIV is used to determine the playback speed. In the example shown in <figref idref="DRAWINGS">FIG. 5</figref>, playback speed is selected as follows.

0000<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="42pt" align="left" /><colspec colname="1" colwidth="56pt" align="left" /><colspec colname="2" colwidth="119pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row><row><entry /><entry>CNIV Range</entry><entry>Playback speed</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>1, 2</entry><entry>x1</entry></row><row><entry /><entry>3, 4, 5</entry><entry>x2</entry></row><row><entry /><entry>6, 7, 8, 9, 10, 11</entry><entry>x4</entry></row><row><entry /><entry>null value</entry><entry>X8</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0080This gives playback speeds as shown at <b>220</b>, and the actual duration of each chunk is illustrated at <b>230</b>. In this example, the number of NIVs allocated to each speed increase with the speed so that changes of speed occur at approximately equal intervals in real time.

0081When using CNIVs derived from user-generated bookmarks, it is possible to modify the CNIVs by their corresponding event popularity, by using the relative number of user-generated events that contributed to an event marker as a weighting factor to modify the CNIV.

0082To avoid too many changes of speed, which can be distracting, a smoothing function may be applied to the playback speeds. For example, the speed control unit may be arranged so that if a first chunk has a CNIV that is associated with a first speed is both preceded and followed by chunks having CNIVs associated with “normal” speed, all three chunks are played at the normal speed, as is illustrated at <b>222</b>.

0083A number of processes will now be described by which a variable-speed catch up sequence may be delivered to the viewer. In these embodiments, the NIV values are stored in the media server <b>42</b> as metadata, (for example adding a field into the MPEG DASH (Dynamic Adaptive Streaming Http) file as will be described later, with reference to <figref idref="DRAWINGS">FIG. 7</figref>), and used to control playback speed. In some embodiments, the client device <b>41</b> includes a speed control system <b>50</b> which can be controlled by NIV data associated with the video chunks <b>49</b> streamed from the content server <b>420</b>.

0084To provide some level of bandwidth-saving to be performed, ABR could be used so that low bitrate video chunks would be provided for sections that are to be played back at high speed e.g. ×16. Similarly, when playback speed is e.g. ×1 then higher bitrates could be selected. It would not in general be possible to preserve the rate of bits/frame—I-frames require more bits than B- and P-frames, so if only B- and P-frames are omitted, leaving a higher proportion of I-frames, the average number of bits/frame will increase. However, the number of bits transmitted per second will be less than if′ all the frames were to be transmitted at a speed sufficient for the speeded up playback to keep up.

0085In the preferred embodiment, also depicted in <figref idref="DRAWINGS">FIG. 1</figref>, the variable speeds are encoded by the server <b>42</b>. This allows the client terminal <b>41</b> to operate conventionally, and also reduces the amount of data that needs to be streamed from the server <b>42</b> to the client <b>41</b>. The video encoding unit <b>499</b> encodes the live video stream <b>409</b> at a number of predefined playback speeds. This may be done using any Adaptive Bit Rate technology such as HLS (Http Live Streaming) and MPEG DASH. Different chunks <b>49</b> are generated <b>499</b> for storage in the content server <b>420</b>, with the required speeds already incorporated in the chunks before they are downloaded.

0086In one embodiment, as depicted in <figref idref="DRAWINGS">FIG. 5</figref>, a single stream <b>20</b> is stored, and the server <b>420</b> (or the client device <b>41</b>) modifies the individual chunks <b>20</b><i>a</i>, <b>20</b><i>b </i>etc to allow their display at the required speed. This can be done by omitting individual frames from each chunk to reduce the playing length to that required. The standard MPEG compression system requires some frames (“B” and “P” frames) to be defined with reference to each other and to independently-defined “I”-frames, the frames should be omitted such that a frame is omitted only if all other frames whose definition depends on it are also omitted—in practice this means that B- and P-frames will generally be omitted before I-frames.

0087To maintain a reasonable download speed, error-checking and other overheads may be reduced during segments with higher speed playbacks, as quality degradation is less noticeable at speed. Data relating to the soundtrack may also be omitted to reduce the required bit rate.

0088In other embodiments, depicted in <figref idref="DRAWINGS">FIG. 6A</figref> and <figref idref="DRAWINGS">FIG. 6B</figref>, a number of streams <b>81</b>, <b>82</b>, <b>84</b>, <b>88</b>: (<b>801</b>, <b>802</b>, <b>804</b>, <b>808</b>) are stored. Each stream is encoded at a defined bit rate, and each chunk of each stream is stored in a server <b>420</b> with an individual address. Part of the resulting store of video data is represented schematically at <figref idref="DRAWINGS">FIG. 6A</figref>. Each stream <b>81</b>, <b>82</b>, <b>84</b>, <b>88</b> is stored as a series of chunks <b>810</b>, <b>811</b> etc. (In <figref idref="DRAWINGS">FIGS. 6A and 6B</figref>, the last digit of the reference numeral for each stream has been used to designate the playback speed of the chunks in that stream—e.g the normal (×1) speed stream is <b>81</b> in <figref idref="DRAWINGS">FIG. 6<i>a</i></figref>, <b>801</b> in <figref idref="DRAWINGS">FIG. 6<i>b</i></figref>, the respective 4× speed streams are <b>84</b>, <b>804</b> etc. Chunks whose start times coincide in real time have corresponding final digits, e.g <b>814</b>, <b>824</b>, <b>844</b> in <figref idref="DRAWINGS">FIG. 6A</figref>: or, <b>8014</b>, <b>8024</b>, <b>8044</b> in <figref idref="DRAWINGS">FIG. 6B</figref>)).

0089As shown in <figref idref="DRAWINGS">FIG. 6A</figref>, the chunks for different playback speeds may each be of the same duration in playback time (so that a <b>2</b><i>x </i>chunk <b>820</b> covers the same real-time period as two 1× chunks <b>810</b>, <b>811</b>). Alternatively, as shown in <figref idref="DRAWINGS">FIG. 6B</figref>, each may cover the same real-time period (so that a <b>2</b><i>x </i>chunk <b>8020</b> takes half the time to play back that a 1× chunk <b>8010</b> would require, but both cover the same real-time period).

0090On request from a user for a catch up stream, one or more MPD files are created by the media server <b>42</b>, which uses a user profile, and metadata <b>44</b> to determine which stream is to be provided at what time, depending on the preferences of the user, i.e. which sections of the video should be presented to the viewer. For example, with reference to <figref idref="DRAWINGS">FIG. 6A</figref>, the manifest may specify chunks <b>820</b>, <b>812</b>, <b>813</b>, <b>824</b>, <b>826</b>, and <b>848</b>, (shown shaded) in order, and assemble them into a video stream <b>80</b> of “chunks” each of equal playback duration

0091Similarly, in the arrangement of <figref idref="DRAWINGS">FIG. 6B</figref> in which the chunks each represent the same duration in real time but, in consequence, have different playback durations, a set of chunks e.g. <b>8040</b>, <b>8021</b>, <b>8012</b> etc can be assembled to form a stream <b>800</b> in which the playback speed varies.

0092Use of the MPD file would enable a personalised manifest to be generated, from which a personalised stream can be generated. It would also enable different lengths of content to be created suited to the required catch-up time required. The manifest <b>46</b> is generated by the server using inputs from the NIV data file, and delivered to the client <b>41</b> to allow the client to request the specified chunks using their respective addresses (step <b>76</b>) for delivery to the video player <b>41</b> and viewing on the output <b>40</b>. As the chunks <b>820</b>, <b>824</b>, <b>826</b>, <b>848</b> have been encoded at the fast forward speeds, no special playback function is required at the user terminal.

0093The chunks representing accelerated speeds may be generated from the initial content, each being coded separately using standard MPEG coding techniques. Alternatively, they may be derived from the normal-speed content by omitting as many frames as necessary to achieve the speed required. The frames would be deleted according to an order of precedence, in which I-frames would be deleted only if the B-frames and P-frames on which they depend have also been deleted.

0094In an alternative arrangement, a combined media/content server <b>42</b>/<b>420</b> can itself generate a bespoke video file from the manifest <b>46</b>, by instructing the content server <b>420</b> (step <b>700</b>) to extract the required video chunks <b>49</b> and unify the variable speed videos into a ‘single’ stream for downloading to the user. This stream could be encoded with ABR enabling delivery suited to available bandwidth. Instead of a manifest, with a list of addresses of individual chunks, the user is then given a single address (step <b>72</b>) from which to retrieve the video file (step <b>76</b>, <b>77</b>) generated for him.

0095In the variant of <figref idref="DRAWINGS">FIG. 7</figref>, each chunk <b>810</b>, <b>811</b>, <b>812</b>, etc, is stored in the server as an unencoded stream <b>88</b> of I-frames <b>880</b>, etc. Using the DASH system described above, a client can download specific byte ranges of data from a chunk or from a segment comprising several chunks. The client can calculate the specific byte ranges required to allow it to effectively play back at a selected-frame rate. In the example shown, the client selects to download only those byte ranges <b>890</b>, <b>894</b>, etc relating to every fourth frame <b>880</b>, <b>884</b>, resulting in a 4× speed playback. As illustrated, each frame <b>880</b>, <b>881</b>, etc comprises 1 Mbyte, so selecting every fourth frame <b>880</b>, <b>884</b>, <b>888</b> etc, requires a http request of the form: <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0000"><ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0096">GET /some/video/chunk.mpeg HTTP/1.1</li><li id="ul0004-0002" num="0097">Host: hostname.provider.com</li><li id="ul0004-0003" num="0098">Range: bytes=0-999999, 4000000-4999999, 8000000-8999999 . . .</li></ul></li></ul>