Communication system and method

ES 2 620 002 T3

DESCRIPTION

Communication system and method

Field of the invention

The present invention relates to bidirectional video communications over a packet-based communication network.

Background

Packet-based communication systems allow the user of a device, such as a personal computer, to communicate over a computer network such as the internet. Packet-based communication systems include video communication systems over internet protocol (IP), which allow users to exchange live video streams over the internet. These systems are beneficial to the user as they are often significantly lower in cost than fixed line or mobile networks. This can be particularly the case for long distance communication. To use a video over IP system, the user must install and run client software on their device. The client software provides the video IP connections as well as other functions such as registration and authentication. In addition to video communication, the client can also provide additional features such as audio, instant messaging ("IM"), SMS messaging, and voicemail.

One type of packet-based communication system uses a peer-to-peer ("P2P") topology built on proprietary protocols. To enable access to a peer-to-peer system, the user must run the P2P client software provided by a P2P software provider on their computer, and register with the P2P system. When the user registers with the P2P system, the client software is provided with a digital certificate from a server. Once the client software has been provided with the certificate, communication can subsequently be established and routed between users of the P2P system without the additional use of a server. In particular, users can establish their own communication routes through the P2P system based on the exchange of one or more digital certificates (or user identity certificates, "UIC"), which allow access to the P2P system. The exchange of digital certificates between users provides proof of the identities of the users and that they are properly authorized and authenticated in the P2P system. Therefore, the presentation of digital certificates provides confidence in the identity of the user. It is therefore a characteristic of peer-to-peer communication that communication is not routed using a server, but directly from end user to end user. Additional details on such a P2P system are disclosed in WO 2005/009019.

Users can therefore make face-to-face video calls, exchanging video images taken from a webcam mounted on each of their respective computer terminals.

However, there is still a problem with such video calls in that the degree of interaction between users is limited. When they meet in person, information is not communicated solely by voice, facial expression, and gestures: some information is also added in the context of a shared experience, situation, or environment. When communicating remotely using a video call, this context can be lost, making communications more complicated or confusing.

Summary

In accordance with one aspect of the present invention, a video data transmission method is provided, the method comprising: establishing a bidirectional video channel between a first terminal and a second terminal in a packet-based communication network; start a live face-to-face video call over the established channel by generating first video data from a video camera of the first terminal, transmitting the first video data to the second terminal for display on a screen of the second terminal, receiving second data video generated from a video camera of the second terminal, and displaying the second video data on a screen of the first terminal; generating third video data in the first terminal from a source other than said video camera of the first terminal; receiving a user selection at the first terminal; and in response to the user selection, transmitting the third video data to the second user through said established channel of the packet-based communication network for display on the second terminal screen.

By adding video content to the call from sources such as a TV stream, camcorder, video disc player or the like, the user can better share their ideas with another user and thus facilitate communication between the two.

In a preferred embodiment, said source may comprise a television receiver of the first terminal and a signal processing apparatus of the first terminal, and the generation of the third video data may

ES 2 620 002 T3 comprising receiving a broadcast television signal using said television receiver and operating said signal processing apparatus to process the received television signal for transmission over the packet-based communication network.

For example, the first terminal can be a television set, or the first terminal can be a living room decoder for connection to a television set.

In embodiments, said sources may comprise an auxiliary input of the first terminal for connection to an external video playback device, and a signal processing apparatus of the first terminal; and generating the third video data may comprise receiving an auxiliary video signal through the auxiliary input and operating said signal processing apparatus to process the auxiliary video signal for transmission over the packet-based communication network.

In accordance with another aspect of the invention, a first terminal is provided comprising: a first video camera for supplying first video data; an alternative video source that operates to supply third parties video data; a memory that stores a communication client application; processing apparatus, coupled to said memory, video signal source and first video camera, operating to execute the communication client application; and a user input device communicable with the processing apparatus; where the client application is scheduled to run, allowing a user of the first terminal to establish a two-way video channel with a second terminal over a packet-based communication network, thereby starting an expensive video call Live face-to-face through the established channel by transmitting the first video data from the first video camera to the second terminal to be displayed on a screen of the second terminal, receiving second video data generated from a second video camera of the second terminal, and displaying the second video data on a screen of the first terminal; wherein the client application is further programmed to receive a user selection from the user input device, and in response to the user selection transmit the third video data from the third video source to the second user through said established channel of the packet-based communication network for display on the second terminal screen.

According to another aspect of the present invention, a communication system is provided comprising a first terminal and a second terminal connected to a packet-based communication network, where: the first terminal comprises a memory that stores a first communication client application, the first processing apparatus arranged to execute the first communication client application, a first video camera coupled to the first processing apparatus, an alternative video source coupled to the first processing apparatus, and a user input device communicable with the first processing apparatus; the second terminal comprises a second memory that stores a second communication client application, the second processing apparatus arranged to execute the second communication client application, and a second video camera coupled to the second processing apparatus; where said first and second client applications are programmed so that when they are executed they allow a bidirectional video channel to be established between the first terminal and the second terminal through the packet-based communication network, and thus start a live face-to-face video call through the established channel generating first video data from the first video camera of the first terminal, transmitting the first video data to the second terminal, displaying the first video data on a screen of the second terminal, generating the second video data from the second video camera of the second terminal, transmitting the second video data to the first terminal, and displaying the second video data on a screen of the first terminal; and the first client application is further programmed to generate third video data at the first terminal from said alternate source, receive a user selection from said user input device, and in response to the user selection transmit the third data from video to the second terminal via said established channel of the packet-based communication network for display on the second terminal screen.

In accordance with another aspect of the present invention, a communication client application product is provided.

Brief description of the drawings

Figure 1 is a block diagram showing the functional blocks of a TV with an embedded communication client;

Figure 2 shows a remote control of a remote control unit for use with the TV of Figure 1; Figure 3 shows an illustrative packet-based communication system; and Figure 4 shows a series of schematic screen shots as viewed by a user on a two-way video call.

ES 2 620 002 T3

Detailed description of the preferred embodiments

In embodiments, a user of a terminal of a packet-based video communication system can insert alternate video into a face-to-face video call through a two-way video channel established with the respective terminal or terminals of one or more other users of the packet-based communication system. The terminal of any of the users could be a personal computer, mobile phone, personal digital assistant (PDA) or the like, but in a preferred embodiment at least one of the terminals is a television set or a living room decoder that has a memory which stores a communication client for the packet-based video communication system and a central processing unit (CPU) arranged to run the client, thus allowing two-way packet-based video communications to be accessed using a television through a packet-based communication system such as a P2P system implemented by a packet-based communication network such as the internet.

One problem with packet-based communication networks is that the accessibility of packet-based communication for users is limited. In particular, such communications are commonly accessed using a personal computer. This can be a disadvantage for some users in that they must be technically competent enough to download and install the packet-based communication client software on their personal computer, which provides a barrier to adoption of the packet-based communication service. packages. Even when the communication client is installed and running on a personal computer, use of the packet-based communication system may be limited because personal computers are often not located in a place where the user is familiar or comfortable with. the communication. For example, a personal computer is often located in an office that for many users is not the most natural or comfortable environment for making phone calls.

Although packet-based communication systems can also be accessed by certain mobile devices, these generally do not have processing resources or display screens available to offer a full range of features, such as video calling.

It would therefore be desirable to make packet-based communications more accessible to users. To achieve this, a system has been developed that enables a user to access such networks from a television ("TV"). This is accomplished by embedding the communication client into the TV itself, or into a separate living room set-top box ("STB") television receiver connected to the TV. The client application is in the form of software stored in memory and arranged for execution in a central processing unit (CPU), the memory and CPU being integrated together with a television receiver (and screen in the case of a TV). in a single domestic device, and therefore marketed together as a single product, in a single housing preferably with remote control. The TV or STB product is preferably a specialized TV receiver system, in that its default mode of operation is like a television or television receiver.

This arrangement is advantageous since the TV can be provided with all the embedded hardware and software required to access the integrated packet-based communication system. Alternatively, it can be embedded in a STB that is directly connected to an existing TV using known interfaces (such as SCART or HDMI, for example). This eliminates the need for the user to download and install software on their personal computer, and provides an easier method by which non-technical users can access the packet-based communication system at home. Furthermore, the TV is typically located in a living room of a home, which enables the packet-based communication system to be accessed from the home location that is most familiar to many users to communicate with friends and family.

Incorporating a packet-based communication client into a TV also has the advantage that a large screen is present, which can be used for video calling.

Additionally, significant processing power can be provided in TV, particularly since the power requirements for a large consumer electronics device powered by grid electricity are less stringent than, for example, mobile devices. This enables a full range of features to be included in the embedded communication client, such as high quality voice and video encoding.

Additionally, in accordance with a particularly preferred embodiment of the present invention, incorporating a video over IP client on the TV provides a unique opportunity: during a video call, the client can be programmed so that a user can transmit provided video data from a data source connected to the TV, as an alternative or in addition to your local live video provided from the webcam. By default a webcam is established for face-to-face communications, but additionally:

- the user can transmit their received TV stream to the remote party, or

- the user can alternatively or additionally transmit input data from an auxiliary audio-video connection (eg provided from a camcorder or DVD player) to the remote party.

Therefore a user can advantageously choose to transmit alternate or additional video streams during a

ES 2 620 002 T3 video call established between two or more users.

Reference is now made to Figure 1, which illustrates the hardware and software functional blocks embedded in a TV 100. The TV 100 comprises a screen 102 for displaying images to the user, which is controlled by the video controller hardware 104 arranged to converting the video signals into the form required to be displayed correctly on the screen 102. The video controller hardware 104 is provided with digital video data from two frame buffers 106 and 108. The frame buffers 106 and 108 are storage devices that buffer video data to be displayed at the time. Username. The frame buffer 3 ("FB3") 108 receives conventional TV video signals, as is known for broadcast TV viewing. Frame buffer 1 ("FB1") 106 stores video data related to the packet-based communication client, as currently described. An audio amplifier 110 receives TV audio signals and amplifies these to output them through at least one speaker 112.

The TV audio and video input signals originate themselves from broadcast television signals by any suitable means such as satellite repeater stations, wireless or cable terrestrial repeater stations; and are received by a television receiver unit of the TV 100 (not shown). Note that broadcasting is distinct from point-to-point communication, which includes being other than multicasting (ie, point-to-multiple points). In broadcasting, the signals are transmitted indiscriminately, that is, regardless of whether the user has chosen to receive the signal (although a decryption key or the like may still be required so that only authorized users can access the broadcast); whereas in point-to-point communication signals, the signals must be requested by the user or users who receive them. Or put another way, to receive a broadcast a user simply "tunes in" without needing to send any signal to the broadcaster, while to establish a point-to-point connection then the signals must be exchanged between the user and the broadcaster.

The TV receiver unit can comprise for example an antenna, satellite dish or cable entry; sampling circuitry; a filter, a low noise amplifier; a mixer and / or an analog-to-digital converter.

After being received by the receiver unit, the signals are then processed by a signal processing apparatus (also not shown) before being fed into the frame buffers and the amplifier of Figure 1. The signal processing may comprise for example a digital filter, demultiplexer, decoder, decryption block and / or error checking block; that can be implemented in hardware-on-chip in the form of one or more peripherals-on-chip, hardware off-chip in the form of one or more off-chip units that are accessed by one or more I / O peripherals, or in software stored in a memory and executed in a central processing unit (CPU) of the television 100, or in any combination of these.

In the case of traditional analog television broadcasts, signals from a plurality of different concurrent programs (or different TV channels) are frequency division multiplexed through airwaves by transmitting on different frequencies. The receiving TV then requires a tuning circuit to demultiplex the broadcasts into an output signal separate from the required program. In the case of digital television broadcasts, signals from different concurrent programs are each divided into packets and interleaved to time division multiplex the signals from different programs in a transport stream for broadcast. The receiving TV then requires a packet filter to demultiplex the packets and separate them from the required program signal.

In a preferred embodiment, the TV 100 receives and processes a TV stream, including the possibility of a live TV stream. The fact that the TV signal is in the form of a stream means that the packets in that stream have a certain sequential order and real-time requirement related to their information content. Although a stream can be stored for later consumption, and / or its order or timing requirements need not necessarily be maintained during pre-consumption processing, when the stream is finally issued to the user for consumption then the order and real-time requirements must be respected. (at least on a practical level to a degree that is imperceptible or tolerable by the user). Also, the fact that the stream is “live” means that it is a broadcast currently in progress (although not necessarily that the broadcast is being filmed live at its source).

Note: The transport packets of the TV stream are different from the packets of the packet-based communications implemented by the client, which would typically be IP packets; and the television network is separate from the packet-based communication network 120 accessed using the client. The TV network broadcasts in one direction from a central source, via relay stations, to a plurality of users; while the packet-based communication network 120 allows two-way communications to be established between end users. Furthermore, the TV network is hierarchical, while the packet-based communication network 120 is preferably non-hierarchical, especially in the case of a P2P network.

ES 2 620 002 T3

The packet-based communication client in the TV 100 is based on three main elements. Preferably, these three elements are software elements that are stored in memory and executed in a CPU embedded in the TV 100. The three elements are: a client engine 114; a speech engine 116; and a TV user interface (UI) 118. An electronic program guide (EPG) may also be implemented in software, which provides television program planning.

Client engine 114 and voice engine 116 establish and perform point-to-point video communications (including the possibility of point-to-multiple points), based on bidirectional packets over a packet-based communication network such as the internet; for example, establishing a peer-to-peer connection (P2P) through a peer-to-peer network implemented over the internet 120.

The client engine 114 is responsible for establishing connections to the packet-based communication system, and therefore establishing a video channel with a terminal of another user through the packet-based communication system. This is done through a connection from the TV 100 to the internet 120. The TV 100 is connected to the internet 120 via a network interface 122 such as a modem, and the connection between the TV 100 and the network interface 122 may be via a wired connection or a wireless connection. Client engine 114 performs call setup, authentication, encryption, and connection management, as well as other functions related to the packet-based communication system such as firewall traversal, presence status update, and contact list management.

The voice engine 116 is responsible for encoding voice signals input to the TV 100 as VoIP packets for transmission over the internet 120 and decoding VoIP packets received from the internet 120 for presentation as audio information to the user of the TV. 100. The user interface ("UI") of the TV 118 is responsible for presenting visual information to the user of the TV 100 in the form of a graphical user interface displayed on the screen of the TV 102.

The client engine 114 is connected to the UI of the TV 118 to control what the UI displays to the user. Client engine 114 is also closely integrated with speech engine 116 for efficient transmission and reception of voice packets over the internet.

The speech engine 116 is connected to the TV UI 118 as the user's voice signals are passed through the TV UI 118 to the speech engine 116, and vice versa.

The TV UI 118 is connected to a frame buffer 1 ("FB1") 106, so that the graphical user interface data is buffered and finally displayed to the user on the screen 102. The UI of the TV 118 is also connected to the amplifier 110, which enables sound (such as voice signals or notifications) to be produced from the speakers of the TV 112. The UI of the TV 118 is also connected to an infrared ("IR") receiver 124 and a Bluetooth transceiver 126 that are used to communicate with a remote control unit, as will be discussed below.

In addition, the TV 100 may comprise an encoder / decoder block 130 connected to receive the video and audio inputs from the TV. The encoder / decoder block also has as input a connection to connect to a webcam, and one or more other auxiliary inputs to connect to another auxiliary device such as a camcorder, video disc player, personal video recorder (PVR), etc. Alternatively, the webcam could be integrated into the same device as the TV 100 or the STB. Encoder / decoder block 130 is coupled to TV UI 118 and client engine 114, and is configured to perform transcoding of video and audio signals by decoding a signal from its incoming codec and re-encoding according to another codec to output to a different type of device from which it was input (or alternatively a direct transcoder could be used). Therefore, the encoder / decoder block 130 will decode the TV signal of the codec used to broadcast to the TV screen and re-encode it in a codec suitable for transmission of video over IP through the internet 120, by means of the client engine. 114. The encoder / decoder block 130 can also decode the webcam signal or auxiliary signal and re-encode it for broadcast to the TV screen 102 via the TV UI 118 or re-encode it for video over IP transmission via internet 120 via client engine 114. Advantageously, the encoder / decoder 130 therefore allows TV signals received by a TV receiver of the TV set 100 to be transmitted to another user via the internet 120 using the packet-based video over IP communication system, as either instead of or in conjunction with the webcam video signals used for face-to-face video calls. Similarly, the encoder / decoder 130 advantageously allows video to be output from auxiliary inputs such as a camcorder, PVR, DVD player or other video disc player to the TV screen 102 and / or transmitted to another. user over the internet 120 using the packet-based video over IP communication system, either in place of or in conjunction with the webcam video signals used for face-to-face video calls.

The encoder / decoder block 130 is connected to the TV, webcam, and auxiliary inputs via a selector 132 configured to allow the user to select between any of the TV, webcam, or auxiliary input for supply to the client engine 114 and therefore for the transmission to the other user through the video system

ES 2 620 002 T3 over IP. Selector 132 can also be configured to allow the user to select a combination of TV, webcam, and / or auxiliary input for transmission to the other user through the video over IP system. Selector 132 may also be configured to allow the user to select from the TV, webcam, or auxiliary input, or a combination thereof, for display on screen 102 by sending appropriate signals to the TV UI 118.

Encoder / decoder block 130 may be implemented in one or more specialized hardware units, or in software stored in memory and executed on a CPU of the TV 100.

The TV UI 118 may also be connected to one or more additional frame buffers ("FB2") 107, where the video signals to be transmitted through the video over IP system are also broadcast for display locally on a smaller "preview window" on screen 102. This will be described later with reference to Figure 4.

Known TVs are not designed to accommodate any form of two-way communications. Therefore, a system is required to enable user interaction with the TV to make and receive calls and messages that is intuitive for users. To achieve this, the TV remote control is preferably enhanced to provide functionality that enables the user to make and receive calls, as well as send and receive messages using the TV.

Reference is now made to Figure 2, which illustrates an example of the physical layout of a remote control unit 200 for use with the TV 100. The remote control unit 200 resembles conventional remote controls for TVs. However, the remote control unit 200 includes an integrated microphone 302, and optionally an integrated speaker 304. This enables the remote control unit 200 to be held on the user's head in a manner similar to a conventional telephone. Integrated with the unit is an IR transmitter 204 and Bluetooth transceiver 206. The IR transmitter 204 sends IR signals to the IR receiver 124 on the TV 100, which allows the user to change channels, turn the television on or off, or select alternative auxiliary inputs such as the input from a DVD player or other system. video disc. This is done in a similar way to conventional remote control units used with televisions. The Bluetooth transceiver 206 that can communicate with the Bluetooth transceiver 126 located on the TV 100. The Bluetooth transceiver 206 is paired with the Bluetooth transceiver 126 located on the TV 100 so that a data connection can be easily formed between them . The data connection enables data transmission from remote control unit 200 to TV 100, and optionally, data reception from TV 100 to remote control unit 200. This allows data to be communicated with the voice and client engine 114 and 116 for use in two-way packet-based communications with other users over the internet 120.

The remote control unit 200 further comprises a keypad 306, which is used for conventional TV control purposes, and also for entering information for the embedded packet-based communication client. Numeric keypad 306 comprises number keys that can also be used to enter alphabetic characters. A standby button 308 is used to place the TV 100 in standby mode. Specialized function keys 310 are used to control the operation of the packet-based communication client, and a directional keypad 312 is used to navigate the user interface of the TV.

To describe the operation of the TV 100 and the enhanced remote control unit 200 with the packet-based communication system, reference is now made to Figure 3, which illustrates the use of the TV 100 in a portion of an example system. 400.

Note that although the illustrative embodiment shown in Figure 3 is described with reference to a P2P communication system, other types of communication system could also be used, such as non-P2P, video over IP, or IM systems. The system 400 shown in Figure 3 shows a first user 402 of the communication system operating a TV 404 (similar to the TV 100 of Figure 1) shown connected to a network 406. Note that communication system 400 uses a network such as the internet. The TV 404 is connected to the network 406 via a network interface 408 such as a modem, and the connection between the user terminal 104 and the network interface 108 can be via a wired (wired) connection or a wireless connection. The TV 404 shown in Figure 3 is a separate unit, but it should be appreciated that a separate TV and STB can also be used.

The TV 404 is running an embedded communication client 410. Note that in alternate embodiments, the embedded communication client may be running on a room decoder. The embedded communication client 410 comprises software stored in memory and executed on a local processor on the TV 404.

The TV 404 is arranged to receive information from, and broadcast information, to the user 402. A remote control unit 412 acts as the user-operated input device 402 for control of the TV 404. The remote control 412 comprises a speaker. and microphone to enable the user to listen and speak on a voice or video call. Remote control unit 412 communicates wirelessly with TV 404, as described above.

ES 2 620 002 T3

The TV 404 may also receive broadcast television signals, and display these as video (television programs) to the user on the TV screen. Broadcast television signals can be delivered by terrestrial, satellite, or cable broadcast, and be in the form of analog signals or digital data. User 402 can control the display of television signals (eg, which channels to view) using remote control unit 412. The TV 404 may also receive one or more audio-video inputs via the auxiliary inputs from alternative sources such as from a DVD player or other video disc system, which again can be selected by the user 402 using the control unit. remote 412.

The embedded communication client is arranged to establish and manage calls made through the packet-based communication system using the network 406. The embedded communication client 410 is also arranged to present information to the user 402 on the TV screen 404 in form of a user interface. The user interface comprises a list of contacts associated with the user 402. Each contact in the contact list has a presence status chosen by the contact associated with it, and each of these contacts has authorized user 402 of customer 410 to view their contact details and presence status. The presence status informs other authorized users (eg, 414) of the packet-based communication network 400 about the user-defined availability of user 402.

The contact list for users of packet-based communication systems is stored on a contact server (not shown in Figure 3). When the client 410 first logs into the communication system, the contact server is contacted, and the contact list is downloaded to the client 410. This allows the user to log into the communication system from any terminal. and still access the same contact list.

The contact server is also used to store a mood message (a short user-defined text-based status that is shared with all users in the contact list); and a snapshot selected to represent the user, which can be referred to as an "avatar". This information can be downloaded to client 410, and allows this information to be uniform for the user when logging in from different terminals. The client 410 also communicates periodically with the contact server to obtain any changes to the information about the contacts in the contact list, including the avatar, or to update the stored contact list with any new contacts that have been added.

Also connected to network 406 is a second user 414. In the illustrative example shown in Figure 3, user 404 is operating a user terminal 416 in the form of a personal computer. Note that in alternative embodiments, other types of user terminal may also be connected to the packet-based communication system. In addition to personal computers (“PCs”) (including, for example, Windows ™ PCs, Mac OS ™, and Linux ™), a personal digital assistant (“PDA”), a mobile phone, or a device may also be connected. games. In a preferred embodiment of the invention, user terminal 416 comprises a display such as a screen and an input device such as a keyboard, mouse, joystick, and / or touch screen. User device 416 is connected to network 406 via a network interface 418 such as a modem.

Note that in alternative embodiments, user terminal 416 can connect to communication network 406 via additional intermediate networks not shown in Figure 1. For example, if user terminal 416 is a mobile device, then it can connect to network communication 406 via a mobile network (eg a GSM or UMTS network).

User terminal 416 is running a communication client 420, provided by the software vendor. Communication client 420 is a software program running on a local processor in user terminal 416 that comprises similar elements for embedded communication client 410. Communication client 420 enables user terminal 416 to connect to the computer system. packet-based communication. User terminal 416 is also connected to a handset 422, which comprises a speaker and microphone to enable the user to listen and speak on a voice or video call. The microphone and loudspeaker do not necessarily have to be in the form of a traditional telephone handset, but can be in the form of a headband headset or an over-ear headset with an integrated microphone, such as a separate loudspeaker and microphone independently connected to the user terminal. 416, or integrated into the user terminal 416.

To be listed as a contact, a user must have been authorized. This preferably implies that one user 402 sends a request to the client 420 of another user 414, and that the other user 414 selects an option to accept the request (or vice versa), thus indicating that the requesting user is in fact recognized as a friend or associate who should want to communicate with the other user.

Therefore, assuming then that the first user 402 is an authorized contact of the second user, the first user will be listed in the customer contact list 420 presented to the second user 414. The second user 414 can then initiate a call to the first user. user 402 through the communication system by selecting the contact and clicking a "call" button using a pointing device such as a

ES 2 620 002 T3 mouse. Call setup is performed using proprietary protocols, and the route through the network 406 between the calling user and the called user is determined by the peer-to-peer system without the use of servers.

After authentication through presentation of digital certificates (to prove that users are genuine subscribers of the communication system - described in more detail in WO 2005/009019), the call can be established.

Referring back to Figures 1 and 2, when the incoming call is received at the client engine 114 the TV UI 118 is notified of the incoming call. This puts the UI of the TV 118 in an incoming call state, so that remote control key presses are interpreted accordingly. The UI of the TV 118 may output graphics to the FB1 106 to display a notification of the incoming call on the screen of the TV 102 so that the user 402 is aware of the incoming call. Alternatively or additionally, the TV UI 118 may generate a ringing sound and other audible signal to notify the user of the incoming call. The notification can only be active for a predetermined time, after which time the notification will fade and the status of the incoming call will be deactivated.

User 402 can select to answer the incoming call by pressing a key on remote control unit 412 or by executing a physical movement such as sliding a movable microphone unit. In response to user 402 selecting to answer the incoming call, remote control unit 200 transmits a command to TV 100 using IR transmitter 204. This command indicates that the user has selected to accept the call (either by pressing a button or taking an action). The remote control unit 200 then activates the Bluetooth transceiver 206. The Bluetooth transceiver 206 is not activated until it is necessary to save battery power consumption. The Bluetooth transceiver 206 establishes a connection with the Bluetooth transceiver 126 on the TV 100. Microphone circuitry 208 is activated to receive audio inputs.

When the UI of the TV 118 receives the command transmitted by the IR transmitter 204 at the IR receiver 124, it interprets this as a call accept command, since the UI of the TV 118 is in the incoming call state. . Note that in alternate embodiments, the IR sent from remote command 200 may be omitted, and establishing the Bluetooth connection between TV 100 and remote 200 may be interpreted as a call acceptance command instead.

The TV UI 118 issues a "answer call" command to the client engine 114. In response to the "answer call" message, the client engine 114 establishes the call connection with the communication client 420 of the calling user 414 .

When the user 402 speaks to the microphone 302 (shown in Figure 2), the audio signal is converted to digital data by the microphone circuitry and the Bluetooth transceiver 206 to transmit the audio signal to the Bluetooth transceiver 126 of the TV. 100. Audio streaming can use a conventional Bluetooth protocol to transmit audio information.

After receiving the audio information at the Bluetooth transceiver 126, the TV UI 118 passes the audio information to the speech engine 116. The speech engine 116 encodes the audio information as packets and passes these to the client engine 114 Client engine 114 also receives video signals from encoder / decoder block 130 and packages them. The client engine 114 transmits the packets to the network 406 via the network interface 408, where they are routed to the communication client 420 running on the user terminal 416 of the second user 414. The client 420 decodes the packets to produce a an audio signal that can be heard by the user 414 using the handset 422 and a video signal that can be displayed on the screen of his terminal 416.

Conversely, when second user 414 speaks on handset 422, client 420 running on user terminal 416 encodes the audio signals into packets and transmits them over network 406 to TV 404. Client 420 also receives and packages video signals. Packets are received by client engine 114 and passed to speech engine 116 and client engine 114. Speech engine 116 decodes packets to produce audio information. The audio information is passed to the TV UI 118. The client engine 114 decodes packets to produce video signals for output to the screen 102 via the FB3 frame buffer (108).

In one embodiment, the audio information is passed from the TV UI 118 to the amplifier 110, such that the voice from the second user 414 is heard from the TV 112 speakers. If the TV 100 is currently being used for watch a TV program, then the audio of the TV program can be turned off. Alternatively, the audio from the call can be mixed with the audio from the TV program, which can be reduced in volume. In an alternative embodiment, if the remote control unit 200 comprises the optional speaker circuitry 210, as described above with reference to Figure 2, the audio information from the TV UI 118 can be passed to the Bluetooth transceiver 126 and transmitted to remote control unit 200 and converted to audible signals by speaker circuitry. The user can then listen to the voice of the second user 414 from the speaker (304 in Figure 2) on the remote control unit 200.

ES 2 620 002 T3

Remote control unit 200 can also be used to initiate a call to another party. For example, the first user 402 may use the remote control unit 200 to initiate a call to the second user 414. The UI of the TV 118 has a "set call" status that can be entered by the user. A call functionality option can be entered by the user into the selected call setup state using the remote control, for example using a dedicated button on the remote control or by navigating an on-screen option using the directional keypad (312 in Figure 2 ). Alternatively, the call-in state can be entered by performing a physical action on the remote control, such as by swiping a movable microphone. In response to this, a command is sent to the TV 100 causing it to enter the call-in state. Typically, the call entry status displays a UI to the user comprising the contact list and a contact name / phone number entry field.

The first user 402 can select the second user 414 from the contact list and initiate the call using the remote control unit. As described above for an incoming call, it sends an IR command to TV 100, activates Bluetooth transceiver 206, and activates microphone circuitry 208. TV UI 118 sends a message to client engine 114 to initiate call to second user 414. This is done in a manner similar to that described above for a call initiated from the second user 414 to the first user 402. The call then continues in the same manner as described above.

Video over IP packets for calls between users (such as 402 and 414) are passed through network 406 only, and the public switched telephone network ("PSTN") (424) is not involved. Additionally, in the preferred embodiment of a P2P system, actual voice and video calls between users of the communication system can be made without using central servers. This has the advantages that the network scales quickly and maintains high quality, and the call can be made at no cost to users.

However, in addition, calls can also be made from the embedded communication client 410 using the packet-based communication system to landline or mobile phones (eg 426), routing the call to the PSTN network 424. So Similarly, calls from landline or mobile phones 426 can be made to the packet-based communication system via PSTN 424.

In addition to making video calls, user 402 of client 410 can communicate with users listed in the contact list in various other ways. For example, an instant message (also known as a chat message) can be sent to a contact. As with voice or video calls, remote control unit 200 can be used for instant messaging. Text data can be entered using the numeric keys, which are also used to enter alphabetic characters. Text data is transmitted to TV 100 using IR transmitter 204, as this is more power efficient than Bluetooth and does not require a high data rate.

The TV UI 118 has a "chat input state" in which remote control unit key presses (received at IR receiver 124) are interpreted as alphanumeric characters that are passed to client engine 114 The chat input state can be entered when a user responds to an incoming chat message, or when user 402 selects a "chat" option displayed on the UI. The chat message data is encoded and sent from the client engine 114 over the network 406 to the communication client 420 of, for example, the second user 414. The message is displayed to the second user 414 at the user terminal. 416. Second user 414 may respond by entering his or her own chat message, which is sent by client 420 and received by client engine 114. The client engine passes the message to the TV UI 118, which displays the message to the user 402 on the screen 102.

The above-described system therefore provides the ability to make and receive IM calls and messages on a TV.

Referring to Figures 4a-4d, the screen 102 of one user 402 can be used to display the incoming video image of another user 414 on a first larger portion of the screen 502; and to display the outgoing video image transmitted to the other user 414 in a second smaller portion 504 which may be referred to as the preview image. The image in the preview window 504 is a smaller version of what the second user 414 will see on their respective terminal 416.

In Figures 4a-4d, the main (ie first) screen portion 502 displays the second user 414 face-to-face video image as received through the video over IP system 400 from the second user's webcam. As discussed above, the first user 402 can select whether to stream face-to-face video back from his own webcam to the second user 414, or whether to stream video back from a live TV stream to an auxiliary input such as from a camcorder, DVD player, PVR, etc. The first user can select to stream the auxiliary TV or video instead of or in addition to the face-to-face webcam video. By default a webcam is established for face-to-face communications, but additionally the user 402 can transmit his received TV stream to the remote party 414 or the user can alternatively or additionally transmit data input from the auxiliary connection to the remote party 414 .

ES 2 620 002 T3

Referring back to Figure 1, selector 132 is coupled to TV UI 118, so that selection of TV, webcam, or auxiliary or potential, any combination of these can be selected by the user via the control unit. remote control 200, preferably via IR interface 124, 204. For example, the selection could be chosen from an on-screen menu using the arrow keys and the "select" button 312, or the different options or combinations could be toggled through repetitive pressing of a specialized button included in the remote control 200 to this end. The TV UI 118 is arranged to output an appropriate selection signal to selector 132.

To illustrate some examples of possible selections, the screen 102 as viewed by the first user 102 is shown schematically in Figures 4a-4d, with the received (incoming) video shown on the main portion 502 and the transmitted (outgoing) video at preview window 504. In Figure 4a, the first user receives face-to-face video from the webcam from the second user 412 and has selected to transmit face-to-face video outgoing from his own webcam to the second user 412. In Figure 4b, the first user 402 has selected instead transmitting the outgoing video to the second user 412 from a TV broadcast as received at the TV receiver of the first user's TV 404. This could be for example from a live TV broadcast. In Figure 4c, the first user 402 has selected instead to transmit the outgoing video to the second user 412 from an auxiliary input of the first user's TV 404. This could for example be a playback of a previously recorded home video on a camcorder. , or from a DVD player or other video disc player, or from a PVR. In Figure 4d, the first user has selected three sources for transmission to the second user 414 (shown at this point as seen on the first user's own TV 404 in three separate preview windows 504, 506 and 508). The first and second screen views of the user can also be supplemented with a message informing them of the selected video source.

A user can therefore advantageously share another video with another remote user as part of his face-to-face video call, and thus communicate the supplementary information via the video call.

In embodiments, encoder decoder block 130 may comprise a first encoder for encoding the video signal from the webcam for transmission through the packet-based communication system, and one or more second encoders for encoding the video signal from the receiver. the TV or the auxiliary input of the TV for transmission through the packet-based communication system. If the user selects the option to switch video streams, the selector block stops broadcasting the webcam input to the first encoder and instead outputs the selected video stream to encoder 1. In this case, a stream is provided encoded to client engine 114 to be broadcast to all call participants. If the user selects an option to add alternative video streams, the selected block continues to output the webcam data to the first encoder and begins to stream the additional video stream (from for example the TV input or AUX input) to the second. encoder. In this case, two encoded video streams are provided to the client engine 114 to be transmitted to the call participants.

As mentioned, the transmitted video streams can also be displayed on the broadcast user's TV screen 102 on the preview screen 502. Therefore the data encoded by the first encoder can also be provided to the frame buffer FB2. under the control of TV UI 118. The data encoded by the second encoder may also be provided to the frame buffer FB2 under the control of the TV UI 118.

Note that the other video is sent as part of the same video call, that is, same session, since the face-to-face video is either switched instead of the face-to-face video or sent together with the face-to-face video. When the client engine 114 establishes a video call, it establishes a two-way video channel or connection to one or more other clients 420 of the same packet-based communication system 400. It is through this same channel or connection, between the same clients and using the same packet-based communication system, that both the face-to-face video from the webcams and the other video such as that generated from a TV stream are sent. live or auxiliary input. The same call can be identified as such in the system by a unique call or session identifier. The same call or session can be made under the same authorization.

Note that the video data transmitted from the webcam is live, or in real time, in the sense that it captures and transmits the user's current actions in progress (or at least an approximation of these, taking into account that there may be delays in the user's terminal or the network and that the video may be choppy). This is in contrast to video received via the auxiliary input, which has been received from a video playback device where it has been stored, for example from a pVr, DVD player or other video disc player, or storage in a camcorder. The webcam video is also distinguished from TV signals in that its origin is a camera located in the user terminal, while the TV signal is received from a broadcast and then forwarded from the user terminal.

Although this invention has been particularly shown and described with reference to preferred embodiments, it will be understood by those skilled in the art that various changes in shape and detail can be made without departing from the

ES 2 620 002 T3 scope of the Invention as defined by the appended claims.