An integrated interactive video and internet system

The present invention relates to a system and method for presenting video programming and related Internet information.

The Internet and other on-line services offer a variety of different services to users, including news feeds, electronic databases (either searchable by the user directly on the on-line system, or downloadable to the user's own computer), private message services, electronic newsletters, real time games for play by several users at the same time, and job placement services, to name a few. Although many on-line communications occur merely through text, it is expected that audio/visual programs will proliferate and text will become less and less dominant in the on-line environment. However, the Internet may remain user unfriendly due to its very massiveness, organization, and randomness. Simply stated, there is no order or direction in the Internet so that specific pieces of information are hard to find, and harder yet, is the ability to put that piece of information into a meaningful context.

Television, on the other hand, has been criticized for being a passive medium. While interactive television systems have increased the level of user interaction, and thus, provided greater learning and entertainment opportunities, vast information resources such as databases are inaccessible from such a medium.

<patcit id="pcit0001" dnum="WO9727546A"><text>WO97 27546A</text></patcit> describes the embedding of uniform resource locators (URLs) in data packets which are broadcast to computers. However, at its earliest priority date, <patcit id="pcit0002" dnum="WO9727546A"><text>WO97 27546A</text></patcit> does not contemplate television programming, for example, as the broadcast medium.

<patcit id="pcit0003" dnum="EP0879536A"><text>EP-A-0879536</text></patcit> describes a system for presenting video programming and related Internet information, the system comprising: <ul id="ul0001" list-style="none" compact="compact"><li>means for receiving programming containing a video signal, an audio signal and one or more embedded uniform resource locators, wherein the embedded uniform resource locators specify one or more Internet addresses of information segments which relate specifically to the content of the video and audio signals of the programming;</li><li>a controller connected to the receiving means comprising decoding means, and display means connected to the controller and to the receiving means for presenting the video and audio signals and for presenting the related Internet information;</li><li>said decoding means being arranged to decode the uniform resource locators to determine the specified Internet addresses; and</li><li>the controller further comprising means connected to the decoding means and arranged to retrieve one or more Internet information segments residing at the determined Internet addresses.</li></ul>

According to the present invention there is provided a system as defined above, the system being adapted such that: said decoding means automatically decodes said uniform resource locators; after decoding of the specified internet addresses by the decoding means, the one or more Internet information segments are automatically retrieved by the means for retrieving; and said display means is arranged to present the video and audio signals concurrently with the retrieved Internet information segments.

A system of the invention combines broadcast television programming and/or video programming which appears on a VHS or Beta tape, CD-ROM, DVD or other medium, or video programming at a video server (hereinafter "video programming") with the Internet. In this manner a powerful educational and entertainment medium is created.

A system of the invention allows consumers to receive more information in a more efficient manner than either television or the Internet alone. For example, consumers cannot only see a news report on television, but they can also read pertinent information about the report, as well as explore related information about the story. The television program becomes the introduction to a particular subject, rather than the entire subject itself. The act of viewing a program has now become a more engaging, enriching experience.

The system may also be used to create a more intimate relationship between the viewer and the program. The user might be solving problems or performing virtual experiments on the Internet site that a teacher is discussing in an educational television program. Similarly, the consumer might be solving problems that the fictional characters in a television program must solve. In both cases, the consumer is an active participant in the process, rather than a passive observer.

Instead of an undirected and unfocused exploration of Internet sites, by synching specific Internet pages to the video signal, the system puts the Internet in context. The television program producers now can decide what additional information to offer their audience. This material can now be seen in the context of the television program.

An additional advantage is that consumers do not have to search through the literally hundreds of millions of pages on the Internet to find appropriate material. The material has already been filtered by the program producers and delivered to the consumer automatically.

Another advantage of the system is that it changes the nature of advertising. Since additional information can be given to consumers automatically, advertising can now be more substantive, allowing customers to make more informed choices. Now, the act of purchasing a product seen on television can be streamlined - the consumer can be given the choice of buying the product instantly using the two-way capabilities of the system.

In addition, users can take advantage of the two-way capabilities of the Internet to respond to polls, to send e-mail or to link to additional sites. For example, a viewer watching a television news program, through a system of the invention, can receive a stream of Web pages which provide additional, specific information relating to the news content - whether background on the Presidential primaries or the latest change in interest rates.

The video programming and corresponding Internet pages can be viewed on personal computers equipped with a television card, but the open software-based approach enables anyone with a television set and a JAVA enabled PC to experience a system of the invention.

By marrying the appeal of television with the two-way data transfer capabilities of the Internet, the system creates a powerful new medium: Television producers and Internet site creators can enhance their content to extend their brand identity and differentiate their program offerings to the millions of people who are spending more time navigating through the resources of the World Wide Web rather than watching television; advertisers can speak more directly to consumers by directly sending Web pages to the consumer instead of only displaying Web addresses in their commercials; and consumers can gain a new level of interest and interactivity over a video-based medium. In addition to providing significant and immediate benefits to broadcasters and advertisers, the system will also present educational programmers with a way to more effectively use Internet resources in the classroom.

Recently, several media companies have joined to create a system for linking the Internet and television on the personal computer, called "Intercast". In this system, content will be provided simultaneously with the TV video signal. This system, however, requires that stripped down Web pages be sent in the vertical blanking interval (VBI) of the video signal, using up to three scan lines limiting effective bandwidth to approximately 28.8 kbps. This approach, however, requires specialized hardware to both insert the Web pages into the VBI and extract these codes at each PC since it takes up to three scan lines of the VBI. Thus, the complexity and cost of the PC is increased. Because the Web pages are transmitted with the video signal, the Intercast system is not a true "two-way" system, but merely a one-way "piggyback" system. In addition, the Intercast is an analog video product, and thus, cannot handle digital video data.

A system of the invention, on the other hand, is a much more flexible, but less complex, system. The present invention supports either analog or digital television broadcasts without broadcasters or end-users having to alter their existing systems, thus enabling broadcasters to reach a wide audience within a short time.

In an embodiment, the actual Web pages are not forced into the very limited bandwidth of the vertical blanking interval (VBI). Instead, merely eight fields of line 21 of the VBI are used to deliver the relevant Internet Web page addresses to the PC. These addresses are called "uniform resource locators" (URLs). The system then directs the particular Web browser to retrieve the identified Web pages from the Internet. Upon receipt of the particular Web page(s), the system syncs the Web page(s) to the video signal, and at the appropriate times, presents the Web pages on one portion of the computer screen with the television video signal, shown in a window on another portion of the screen, and thus, provides the synergistic Internet and television experience. One of the advantages of a system of the invention is that no specialized chip set need be produced and implemented into the standard PC. Thus, complexity is kept to a minimum.

Embodiments of the present invention will hereinafter be described, by way of example, with reference to the accompanying drawings, in which:- <ul id="ul0002" list-style="none" compact="compact"><li><figref idref="f0001">Figure 1</figref> shows diagrammatically a system of the invention for receiving programming at a subscriber location,</li><li><figref idref="f0001">Figure 2</figref> shows diagrammatically an alternative embodiment of a system of the invention able to integrate Internet information with video content,</li><li><figref idref="f0002">Figure 3</figref> shows a flow diagram of basic software design for a system of the invention, and</li><li><figref idref="f0003">Figure 4</figref> shows another system able to directly transmit URLs to a user, but this system is not an embodiment of the invention.</li></ul>

<figref idref="f0001">Figure 1</figref> shows an embodiment of a system of the invention which is a computer based system arranged to receive a video program together with embedded uniform resource locators (URLs), which direct the user's computer 16 to address locations, or Web sites, on the Internet 20 to retrieve related Web pages. These Web pages correspond to the video presentation. The particular video programming can be delivered in analog, digital or digitally compressed formats (e.g., MPEG2) via any transmission means, including satellite, cable, wire, or television broadcast.

The video programming is preferably created at a centralized location, i.e., content creation 4 as shown in <figref idref="f0001">Figure 1</figref>, for distribution to subscribers in their homes, for example. Program creation is accomplished according to any conventional means known in the art. After a video program is created, uniform resource locators are embedded, in one preferred embodiment, into the vertical blanking interval of the video programming by the URL encoder 8, shown in <figref idref="f0001">Figure 1</figref>. In this embodiment, the URLs are encoded onto eight fields of line 21 of the VBI. Line 21 is the line associated with close captioning, among other things. However, the URLs could also be embedded in other fields of the VBI, in the horizontal portion of the video, as part of the audio channel, or in any subcarrier to the video. Preferably, the URLs have associated time stamps which indicate to the subscriber stations when, during the video program, to display the particular Web pages addressed by the URLs.

The particular information in line 21 is not part of the visual part of the program, and thus, is not perceptible to the human eye, thereby making it ideal to send data information to the users. While the bandwidth capacity of line 21 is limited, because the system of the present invention transmits only the uniform resource locators (URLs), and not full Web pages, there is more than enough capacity. Furthermore, no additional hardware is necessary at the PC 16 to implement the elements of the present invention. Thus, the present invention has the additional advantages of being very efficient and takes advantage of conventional hardware.

Once the video program is created, it can be transmitted to user sites over any transmission means, including broadcast, cable, satellite, or Internet, and may reside on video servers. Furthermore, the video program, with embedded URLs, can be encoded on a VHS or Beta tape, DVD or other medium.

Preferably, each receiver station comprises any Intel x86 machine (preferably a 486 processor, pentium processor, etc.), an Apple Computer, UNIX or any other type of standard computer workstation. The local PC 16 is preferably connected to either a cable and/or broadcast television connection or to a local VCR or other video source. At each subscriber site, the local personal computer 16 preferably receives the cable transmission by cable connection on the back of the personal computer 16. The video/audio program can then be processed for display on the computer screen using any conventional PC card capable of displaying NTSC signals on a computer monitor, such as a WinTV card. In addition to the cable connection, however, in the present invention there is also an Internet 20 connection created concurrently with the cable connection.

The Internet 20 connection can be via high-speed line, RF, conventional modem or by way of two-way cable carrying the video programming. The local PC 16 has Internet access via any of the current ASCII software mechanisms. In a preferred embodiment, at each subscriber home, an associated local URL decoder 12 receives the cable video television program, as shown in <figref idref="f0001">Figure 1</figref>. The local URL decoder 12 extracts the URLs, preferably embedded in the vertical blanking interval, with the use of any conventional VBI decoder device. The URL decoder 12 may be either a stand-alone unit or a card which is implemented into the personal computer 16.

In another preferred embodiment shown in <figref idref="f0001">Figure 2</figref>, the uniform resource locators (URLs) are encoded into the video in the same manner as described above. Again, the URLs are preferably encoded onto eight fields of line 21 of the VBI. However, the URL decoder 24 is located at the server site, as opposed to the subscriber location. When the decoder 24 receives the video program signal, it strips out the URL codes on line 21 of the VBI and delivers these codes independently to an Internet server 28. The URL code is then subsequently delivered over the Internet 20 to the user PC 16. Simultaneously, the video is broadcast over conventional broadcast or cable transmission means 36 to the user's personal computer 16.

The system shown in <figref idref="f0003">Figure 4</figref> is not an embodiment of the present invention, but is described and illustrated to aid in the explanation of the invention. This system does not depend on, or even use, the VBI, but will run on online service over the Internet 20. This service will be in form of an Internet Web site 62 that provides a user-interface to a database 78 and to one or more associated data servers 90. The service will provide member-accounts to TV broadcasters 66 who sign up to use the system of the invention in conjunction with their broadcasts. Each member broadcaster will enter the service at their computer 70 through Web browser software 74 using their member account by entering various identification and password information. Once within their account, the member will be provided with a graphical user interface for pre-scheduling URLs for transmission to users 118 over a direct Internet connection 94 at particular times of day. The same user interface, or a variation on it, can be used by broadcasters for live transmission 82 of URLs to users at the same time as a broadcast 86.

For example, one example of this interface might be a scheduling calendar (daily, weekly, monthly, yearly) in which the broadcaster 66 may allocate time periods which coincide with their broadcasts 86, and during which they will send out URLs to their users to link to Web pages. For each time period (for example, a particular hour long period during the day) determined by the broadcaster 66 to be a broadcast period (a period during which they want to transmit URLs that correspond to a television show being broadcast from their TV broadcast facility 110 to the external TV 114 of the user 118 at that time), the broadcaster 66 may then enter a series of URLs into an associated file ("Link File") for transmission over the Internet 20 at that time. This Link File might have a user interface such as a spreadsheet, table, or list, or it could be simply a tab-delimited or paragraph-delimited text-file. As an example, each of the records in the Link File consists of a data structure which could contain information such as: (&lt;timecode&gt;,&lt;URL&gt;,&lt;label or title&gt;,&lt;additional information&gt;,&lt;additional information&gt;,...)

The above data structure is just one example. The records in the Link File preferably specify the time, Internet address (i.e. URL), label (such as an associated name), and some optional additional information, for each Web page the broadcaster 66 desires to launch during a show.

When a broadcaster 66 modifies their calendar and/or the Link File associated with any given time period(s) in their calendar, this information is saved onto the database 78 that is attached to the site 62. Each broadcaster 66 may maintain multiple calendars in the database 78 if they broadcast in different time zones, for example.

The database 78 provides the Link File records for upcoming time periods to a server 90, which may be one server or a distributed network of server programs on multiple computers across the network, to be utilized for scaling to large national or global audiences. The server 90 provides the Link File records, including the URLs, to the user's personal computer 16, which is connected via a network. Examples of possible networks include the public Internet 94, a direct private network, or even a wireless network.

One feature of the above system is that one or more broadcasters 66 may utilize the same schedule in the database 78 for their own broadcasts 86 or during the same broadcast. For example, a network broadcaster may develop a master schedule and various affiliate broadcasters may subscribe to that schedule or copy it (in the database) and add or delete specific URLs in the schedule for their local audiences or unique programming. This scheme enables affiliates to insert URLs for local advertisers or local subjects into a sequence of more general URLs provided by their network broadcaster 66. In other words, the affiliate can add links that ride on the network feed and then redistribute it to their local audiences.

The above system can also enable personalization in the form of unique series of URLs specific to each user's unique profile, which is directly sent over the Internet 20 to each user's specific client software 106. This can be achieved from the broadcaster 66 to each individual user 118, or to particular collections of users. To accomplish personalization, the service may send a different stream of URLs to each user's client software program 106. The stream of URLs sent would depend on a user profile stored in the database 78 or the client software program 106, a user profile which is built on demand or over time for each user 118 based on criteria such as the location of the user, choices the user makes while using a client software program 106, or choices the broadcaster 66 makes during a broadcast 86, or automatic choices made by an algorithm (such as a filter) residing on the service 62. Personalization enables each user to receive URLs which are uniquely relevant to their interests, demographics, history, or behaviour in the system.

System Operation

Once the URLs have reached the personal computer 16, system operation is similar for all of the embodiments.

In a preferred embodiment, a JAVA enabled browser 98 as well as specialized software 106 for performing part of the method of the present invention are installed on the computer 16. The JAVA enabled browser 98 allows the computer 16 to retrieve the Web pages 102 and is preferred software, since it is platform independent, and thus, enables efficient and flexible transfer of programs, images, etc., over the Internet 20. Specialized interface software 106 (hereinafter, "client software") acts as an interface between the video programming and the Internet functions. The client software 106 retrieves URLs from the video program (embodiment of <figref idref="f0001">Figure 1</figref>) or directly from the Internet connection (embodiment of <figref idref="f0001">Figure 2</figref>) interprets these URLs and directs the JAVA enabled browser 98 to retrieve the particular relevant Web pages 102, and synchronizes the retrieved Web pages to the video content for display on the user's computer 16, as shown in <figref idref="f0002">Figures 3</figref> and <figref idref="f0003">4</figref> and explained in more detail below.

In a preferred method, the URLs are encoded and embedded into the video signal by inserting them into the vertical blanking interval (VBI), as mentioned above.

One method of the present invention has the capability to detect identical URLs sent directly after one another which causes the browser not to fetch URLs in these particular cases. As shown in <figref idref="f0002">Figure 3</figref>, once the URL code is received at the computer, the client software 106 first interprets the URL and determines in step 42 whether the particular URL has been received previously. If it has already been received, the next received URL is interpreted for determination of prior receipt. If the particular URL has not been detected before, the software checks for misspelling in step 46 and any other errors, and if errors exist, corrects these particular errors. Once again, it is determined whether the URL has been previously detected. If it has, the next URL is accessed in step 38. If the URL has not been detected, the specific URL is added to the URL list in step 54. The specific URL is then sent to the Web browser, preferably a JAVA enabled browser 98. Upon receipt of the URL, the browser 98, in step 58, will access the Web site address 122 (<figref idref="f0003">Figure 4</figref>) indicated by the URL and retrieve the cited Web page(s) 102 via the Internet.

Viewers can view the integrated presentation in the following manner. As mentioned above, the video signal is processed and displayed on a video window on the PC screen using a WinTV card, for example. The corresponding audio is forwarded to the audio card and sent to the PC speakers.

The actual retrieved Web pages 102, referenced by the URL, are optionally time stamped to be displayed on the computer screen when predetermined related video content is displayed in the video window, thus, enlightening and enhancing the video presentation by providing in-depth information related to the video content thereto. Another section on the screen is also preferably used to represent an operational control panel. This control panel provides a list of the URLs that have been broadcast and correspondingly received by the computer 16. This control panel is updated to add a URL code each time a new URL code is received by the PC 16. This list gives the subscriber the flexibility to go back and retrieve the particularly informative or interesting Web pages that have already been displayed earlier in the program, or alternatively, to print them out for future reference. Furthermore, the list could include URLs referring to Web pages not displayed with the broadcast program, but that provide further information on a certain topic of interest to the viewer.

The present invention can best be understood with reference to an example. A viewer can begin watching a musical video featuring a new band, for example. As the video is received by the PC 16, URLs are either being extracted from the vertical blanking interval or are being received directly via the Internet 20, and are being interpreted by the client software 106. Upon direction and command, the JAVA enabled browser 98 retrieves particular Web pages 102 from Internet 20 Web sites identified in the URLs. These Web pages 102 will then be displayed on the video screen at particular times. Thus, for example, while the viewer is watching the music video, biographical information on the band can also be displayed adjacently to the video window. Web pages 102 could also include an upcoming concert schedule, or even audio clips of the band's music may be downloaded from the Internet 20. As another example, a user could be watching a program relating to financial news. While the narrator is shown discussing high tech stocks, Web pages corresponding to detailed financial performance information on high tech stocks, environment and characteristics can be displayed with the video on the computer screen. When the program narrator switches to a discussion on the weekly performance of the Dow Jones, Web pages presenting related financial performance information can be simultaneously displayed. Thus, it is evident that the present invention profoundly enriches the viewing and learning experience.

It is understood that there can exist alternative embodiments for use with the present invention. For example, the user can view the interactive program using a television set 114 or other display monitor in conjunction with the display screen of the personal computer 16. In this embodiment, the relevant Web pages are shown on the personal computer 16 while the video program is displayed on the television monitor 114. In this alternative embodiment, a cable set top box receives the television program from the multichannel cable. The personal computer 16 also receives the video program from the multi-channel cable and extracts the URLs, embedded in the vertical blanking interval of the video signal or directly transmitted 94 over the Internet 20. The client software 106 extracts the URLs and retrieves the particular Web pages as described above. The Web pages are then synchronized with the particular video frames and presented to the user. It is understood that a hyperlink may exist on the Web site that will allow the user to automatically load the client software and call up the specific television channel referenced in the Web site. For example, someone browsing the Internet 20 may come upon a major television network's Web site. They scroll to an interesting story then click on an hyperlink to turn on the software which tunes the TV window to the network to enhance the information residing at the Web site.

Furthermore, instead of receiving the video program from a transmission means, the video program can be addressed directly from the user site if the video program, with embedded URLs, is stored on a VHS, Beta, DVD or other medium. In this embodiment, the user PC 16 and/or television 114 are connected to a VCR, DVD player or other appropriate device.