Communication system for videoconferencing

~z^~ COMMUNICATION SYSTEM FOR VIDEOCONFERENCING 18,009 FIELD AND BACKGROUND OF THE INVENTION The present invention relates in general to telecommunication equipment and in particular to a new and useful system for conducting a videoconference between a plurality of participants who are physically present at different locations.

A comprehensive review of the prior art in lQ this field is given in the German publication Kommunikations-Endgerate Grundlagen, Verfahren, 3austeine, Gerate, Systeme (Communication Terminals-Fundamentals, Methods, Building Blocks, Systems) by Friedrich Ohmann, Springer publishing house, Berlin, Heidelberg, New York, Tokyo 1983, chapter 7.5 "Bildkonferenz" (Videoconference) pages 414-423.

According to that source, what is required of such conferences is that all subscribers (interlocutors participants) see and hear each other during the whole time-simultaneously, and in the correct direction, and be able to show one another written documents and objects.

Not considering the minimum technical eauipment necessary at every location for carrying out the conference, the costs of terminals and their interconnections increase in an unjustifiable proportion with the number of interconnected locations, if no particular measures are taken which all, however, have their advantages and disadvantages.

To reduce these costs, particular such relating to taking, transmit~ing, and reproducing pictures, parallel processing and ~ ~ 3~9 Q'~ switchill~, and mixed forms developed therefrom, may be provided. IY;th tlle first named measure, the number of needed terminals and channels can be reduced, but always several conference participants are required to be present togcther at the location. ~Yith the other measure, a single Picture must always be transmitted to all of the interlocutors, thus no full interconnection is required, however, a continual - voice-controlled or manual switching is necessary, preferably by one of the participants.

A solution, considered trivial, and requiring to equip every one of the participants with complete terminals, such that picture and sound from one participant is *ransmitted individuall~ to all other locations by conference and there reproduced to everyone with a particular loudspeaker and screen, must certainly be rejected as unjustifiably expensive. That is why prior art videoconferencing systems have been limited to arrangements where the interlocutors are concentrated at two locations between which a point-to-point communication fo~ picture and sound in the two directions i5 established ~studio conference, committee room conference). Even in systems designated "working place coIlEerence'', the main communication, for example through a public network, is established between two localities, with the individual wor~ing places being in the same building or at the same location, each equipped with - 25 a minimum of terminals and being connected together within e~ch of the localities through concentrators.

Prior art videoconferencing therefore remained res~ricted to conferences between two regional groups of participants.

SU~IARY OF THE INVENTION The present invention is directed to a system for permitting a kind of working place confcrence between participan~s at -- 2 ~23 l~ --3- different, mutually independent locations, and therefore, goes partly back to the already mentioned solution, which is trivial in itself, of interconnecting the individual locations by communication means in a manner such that all video and sound signals of each of the locations can pass to all the other locations. It must be ensured, as before, that all the participants can see and hear each other continually and simultaneously and can show written documents, objects, etc. However, the requirement men 1~ tioned above that the participants look at one another in the correct direction applies only to several participants present at the same location and is therefore substantially irrelevant for individual participants at locations of their own.

The excessive costs which might have been expected with such a solution do not materlalize if the features of the present invention are provided.

Accordingly, an object of the present invention is to provide a communication system holding videoconferences between a plurality of participants who are present at different participant locations, which comprises a participant terminal at each location including a camera for producing a video source signal corresponding to the scene at the participant location, a monitor or other rn/` ,~ .

~23~t90~t -3a video sink means for receiving a video signal to generate a picture, a microphone and loudspeaker for generating and receiving audio source and sink signals, and control means which can be manipulated to initiate communication with other participants and to control the images displayed. A central station is connected to all of the participant terminals and includes means for receiving and transmitting the source and sink signals from and to all the participant terminals as well as storage units for storing a large size picture and at least one small coarse grained ~23~Q'~ C~ ~ picture for each'part;cipant location, the central station having a programmed processor for rece;ving control signals from the participant terminals to control the display of large and coarse pictures at the participant terminals.

The control station also includcs multiplexers for providing the video signals to the various video monitors at the participant terminals. The programmed processor includes a progr~m read and lYrite control for assembling the large and coarse pictures. Connecting lines are connected between each participant terminal and the central station with the control lines having sufficient bandwidth for video, audio and control si~nal channels.

The inventive solution substantially requires only a first and f-inal expenditure for the equipment of the central station lS for tl~e videoconference. It is important that every participat ing location needs, of course, a camera, a single monitor, and the usual microphone-loudspeaker means~ i.e. a minimum terminal equipment, and also the broad-band go and return channels anyway needed for any videoconference. Another considerable advantage of the invention is that nobody is needed for the master control of the picture. The control is effected from every location individually, anytime as wanted and desired, without affecting the other locations, within several predetermined patterns. This is a new performance feature for videoconferencing.

All the pictures under individual control, with the information displayable in size and arrangement on the local monitor, are produced by the central station of the videoconference and correspondingly transmitted as a full size individual or mixed picture. The central station thus operates as an intermediary between sources and sinks and, therefore, the relatively expensive equipment for this mediation and processing for all the connectable locations of individual video signals to image mixtures is needed only in -- 4 -~LZ~ 9{)~ this station and only to an extent actually required b~ the purpose. The same applies analogously to the far less expensive processing and mediation in the audio sec-tion.

Particularly advantageous embodiments of the invention include respective features which provide various advantages which will now be discussed.

If picture storage units and multiplexers are employed for video signals in digital form, conventional networks can be used, as before, equipped for transmitting analog signals.

As far as the connecting lines are concerned it should be possible to make sure that future requirements imposed on quality will also easily be met, or at least not be difficult to meet.

Such requirements regard the transmission bandwidth of TV standard, e.g. 5 MHz,or future digital HDTV (high definition television) instead of picturephone standard with presently lM~lz; and audio channels of hi fi quality, e.g. 10 ~Hz; instead of telephone quality bandwidth, usually 3 ~Hz.

Since even in local networks advance systems with a broadband si~nal transmission through optical fibers, i.e., multimode gradient fibers, have already been provided, signal mode systems must also be considered in both the local and long distance regions.

obvious limits are set on the number of participants in any conference, not only videoconferences.

This limit is from two to not more than about twelve participants.

Inventive s~t io~s central s~*~o~ can be equipped for any number of picture storàge units and multiplexers but at least of a number needed for a videoconference.

With a sufficient number, the equipment may then be interconnected in the central station ~r~ ~7 ) -~2~49()'~ to groups for every one of the conferences, with a set of corresponding control devices for each of these groups.

The problem of how to make the video signals coming from the individual sources available in the central station, as both a complete and at least one coarsened picture, i.e., reduced or small picture, can be solved, as to the hardware, with picture storage units, for example, having a storage capacity exceeding that of a large picture. That is, the large pictures and small pic-tures are stored already in the way in which they are later to be available.

However, since in a mixed picture comprising a large picture and a plurality of small ones, the large picture must be somewhat smaller than the screen, i.e., particularly the marginal zones must be suppressed vertically and/or horizontally, at one or both sides, this can be done better in software terms while reading it out` from a picture storage, than in hardware terms.

Picture storage units having a capacity of only one complete picture are also possible in which the complete picture, a large picture, and small pictures are read out under a corresponding program control.

At every location, also the participant's own image can be displayed as a complete picture or a mixture on the individial monitors, since with the control freedom, there is no difference between the transmittinq and the receiving location.

The size of a large picture and a coarsened picture, i.e., a small one, should of course be only as small as needed, and as large as possible.

Keeping the size of a complete picture constant the size of the large picture can freely be chosen at every location, as a function of the number and size of the small pictures in the mixtures~ and conversely, within the pattern of variants prede-termined by the central station.

-- 6 jrc:~ ~ ,9O 4 For examplc, if at a location no display of split screen pictures of all the articipants is proviclcd, i.c. ~ more or less narrol~ selection from the panel is prefcrred, or if only a few locations are interconnected for a videoconference, the small p;ctures may remain relatively large.

Every location may be occupied by as few as a single participant, wh;ch may even be the rule. I~Titll a plurality of ~articipants at a location, they will be seated adjacent each other. Then a control instruction may be delivered to the central station to suppress in the uicture received from this location only the horizontal margins, and to make available small pictures having a certain minimum width, f~r ex~mpl~e, corresponding to only half the width of the pictu~.

If no picture control instruction is received from a location, a picture is delivered to this location corresponding to a standard program. This may take into account control instructions from othcr locations, majority decisions, etc. This also means that with a consent, a control ~eyboard as a terminal equipment may be omitted at a location withcut any functional disadvantage.

To produce small pictures, thus reduce the number of picture elements in the picture, a low pass filtering may be provided.

Portraits, however, furnish soft contours, so that with digital storage units, for example, only the writing rate is to be varied relative to a complete storage. Written documents and objects etc. are usually anyway displayed as large or full pictures and if in such an instance a document is displayed in small size, with not quite distinct characters, even such a quality is satisfactory if it is capable of stirring interest in seeing a large,-more distinct pi~ture.

-- 7 1 ~ 3 4~ ~ ~ The individually sclectable distribution in a split screcn picture shotlld bc based on a struct-lre predctcrmilled by a standard program of the central unit. For example, ~ith five participants, one at each location, the narticipant at the location might appear in a large pic-ture, the other four in a vertical row of four small pictures at the right side of the - screen.

First the raster of the mixed picturc is transmittcd and each of the participants may now individually make changes in the raster.

lO The audio channel may be switched in parallel with the switching of the video signals, and in any case this is done also in the central s~ation. Quite generally, conferences are kept to a speaking discipline, so that usually a single participant will be heard at any one time. Because of the large freedom with 15 the picture control, it may happen with a parallel switching of audio and video channels, that only a large picture of a participant will just appear at one location while another participant throws in an important comment which then will not be heard at that location. It is there-fore advisable to transmit ~o the ~udio signals of all the othcr sources as a summation signal to every sin~. In terminals with a two-channel sound reproduct - ion, it is advantageous in addition to reproduce the speech signals helonging to the small pictures at the right-hand or left-hand side of a monitor, correspondingly to the right or 25 left audio channel, and the speech signals be]on~ing to a large picture and to participants who are not in the picture at the same time, from the middle.

A sound control with individually controllable sound mixers, i.e. contro]lable interme~iate amplificrs in the central statioo 30 for every audio signal coming from a source, and suitable sound control equipment at the individual locations,make it possible to simulate the so-called "cocktail party effect' -- 8 1~3'~9(~4 That is, should one of the participants ~ish to concentrate acoustically on a ccrtain other participant whose audio signal is too drowned by the summation signal, he or she ma~r increase through the audio control the level of the respective audio signal in the central station, and even decrease at the same $ime the other levels.

- For this purpose, also to permit an individual picture control, the participants may be aided with a useful expedient consisting in distin~uishing marks which are permanently or temporarily 10 mixed into the pictures and comprise figure or letter combinations, or pictorial characters, etc., and permit a quick and clear identification of a picture with the source and the corresponding button of the keyboard. These measures hol~ever, do not contribute to the sought directional accuracy 15 in looking at and rece;ving the sound from the other pqrtici~ants.

Other embodiments of the invention are particularly advantageous from the viewpoin~ of communication technology. They may readily be employed in radial networks, one of the most usual structures o public communication systems. To successfully conduct a 2~ video conference, tlle central station therefore is of course to be located in the proximity of most of the participating locations. Locations which are far or very far away therefrom may be supplied at least through a common return channel, to save channel capacity and cost for long transmission, such as 25 through intercontinental connections. In such an instance, of course~ only individual control wishes of a single one of such remote locations can be taken into account. Changes in the control, however, may be agreed in the acoustical way.

The various eatures of novelty which characterize ~he inventio - 30 are pointed out with particularity in the claims annexed to and forming a part of this disclosure. For a better unders~anding of g ~Z3~9()4 - 1 o the invention~ its operating advantages and spccific objects atta;ned by its uses, rc~erence is made to the acco~.panying dra~ings and descriptive matter in which a preferred embodiment of thc invcntion is illustrated.

BRIEF DESCRIPTION OF THE DRAIY~INGS . ~ Details of the invention are diagrammatically shown in the drawings ;n ~.-hich:

Fig. 1 is an overall diagram of a videoconference communication system;

10 Fig 2 is a simplified block diagram ol the equipment for mixing, s~itching and transmitting the video signal in the central station;

Fig. 3 is a simplified block diagram of the equipment for mixing, switching, and transmitting the audio sign~ in the 15 central station;

Fig. 4 illustrates the size proportion between a complete picture and three variants thereof;

Figs. 5 to 7 show three variants of small pictures;

~igs. 8 to 11 show variants of mixed pictures~ composed of 20 large and small pictures; and Fig.12 shows a control desk unit with elements for selecting and controlling the sound and pictures.

-10 ~Z34gO'~ DESCRIPTION OF THE PREFERRED EMBODIMENT Fig. 1 illustrates the concept of an inventive system. Considering its performance, the system comes closest to those which are known under the designation of "working place conference". Such systems can conduct a videoconference between plural participants who are each at their own work place at spaced locations from each other.

At each of the selected locations SO, shown ln are five such locations SO-A to SO-E, equipment is needed (see location SO-A where details are indicated) having a terminal unit AE for the following pieces of equip ment EE:

sinks forvideo and audio signals BSs, TSs (namely a monitor and a loudspeaker):

sources of vi.deo and audio signals, BSq, TSq (namely a camera and a microphone):

selector and, if provided, signaling devices WE, BE, TE for establishing communication or entering control instructions.

The equipment may comprise to a very large extent conventional, commercially available means.

Two or more channel stereophonic devices should be provided for the audio signals (picking up, transmission, reproduction).

~234~0'~ -lla From such a location SO, one or several participants can hold a videoconference with participants at other, very remote and mutually independent locations SO, through connecting lines AL and a videoconference central station BZ.

Connecting lines AL must be provided for a go channel HK and a return channel RK. The go channel as well as the return channel require transmission capacities for a video channel BK, and audio channel TK, and a signal channel SK (not shown in Fig. 1).

rn/ ~ C~()4 Audio channel T~ may also be dimensioned for a twochannel or multichannel stereophony. The signalling channel SK is needed initially for building up the communication and later, i.e.~ after the conference circuitry is established, occasionally for control instruc'ions. ~ - The equipment outside the videoconference control station BZ thus does not comprise more than that of a TV-standard picturetelephone station. This already is a substantial advantage of the invention since it makes any such picturetelephone station suitable for videoconferencing.

The videoconference central station BZ includes all equipment necessary for distributing and mixing the incoming source signals to outgoing sink signals, but only to the extent sO ~ ~s~-y ~ of satifying the traf~ic needs of the videoconference.

This e~ emmt is assigned to the participating locations always for the duration ofa video conference and it substantially comprises large picture storages GSp and small picture storages KSp as well as multiplexers MUX for the video signals BSs~ For audio signals, sound mixers TM are provided.

The needed large picture storages GSp, small picture storages KSp, multiplexers MUX, and sound mixers TM, provided in a number corresponding to the participating locations SO, are interconnected to respective groups and associated with the video control BC and an audio control TC.

The structure and function of a videoconference central station BZ as far as video signals BS are concerned, may be learned from Fig. 2 where the control equipment in the central station which is necessary for one location, is shown.

The video signals BSq coming frcm a camera are supplied to an analog-to-digital converter AD wherefrom they pass, through an intermediate storage jrc: ~ ~;23~909~ FlFo, if provided, to a large picture storage GSp~ Through a detecting and selecting synchronization separator, for example, the synchronizing information is retrieved from tne incoming video signal BSq, for a write control SC. Parallel to large picture storage GSp a coarsened picture i.e. one whose number of picture elements is two-dimensionally reduced, is produced, for example by low-pass filtering, and supplied to a small picture storage KSp.

In a video control processor BCP, a standard program is stored according which mixed pictures M:B are combined of the individual large pictures GB and/or small pietures KB, to the size of a full or complete picture VB (Fig. 8). By means of control signals which are received from the individual partieipating loeations and include individual video eontrolling instructions, the video signals BS for a complete picture VB, for large pictures GB and small pietures KB are directed, in aeeordanee with the transmission cloek fre~ueney, through an addressing program and a read eontrol LC and, if provided, also through write eontrol SC, to multiplexers ~IUX and therefrom, through a digitalto-analog converter DA, as an individual monitor video signal BSs to the respective connecting line.

12349()4 -13a As to audio signals, the structure and function of the videoconference central station may be learned from Fig. 3. The audio signal TSg coming from a microphone is amplified through a preamplifier W and passes throuqh level ad~justors PS to summation amplifier SVL for the left-hand channel and an identical amplifier ~VR for the riqht-hand channel. The participants receive output signals TSsL and TSsR for the left-hand and right-hand audio sink (such as a loudspeaker).

The amplification and damping of the individual level adjustors PS may be set by means of an audio control proqram. Particularly advantageous is blocking of the participant's own sound, to eliminate acoustic feedback.

rn/ ., -l't 34~Q~ For thc audio control, wllich also may be effected according to a standard program applying to all the locations, i.e~ with or ~ithout individual overriding, the videocon-ference control stat;on B~ includes the sound mixer T~ and audio control TC already mentioned in connection with Fig. 1. A cocktail party effec~ can be simulated in a simple way by setting the respective level ~J~tc~s PS, i.e. the audio signal TSq of the participant ~ whose voice is to be emphasized is correspondingly amplified and those of the other participants are damped.

10 Fig. 4 shows a monitor screen in true proportions according to the conventional TV standard. A complete picture VB has a width of 4/5 d and a height of 3/5 d, ~ith d being the diagonal of the complete picture. Picture sizes which show a face while omitting horizontal or vertical J or horizontal and 15 vertical 7 margins, may be considered a large picture GB.

One of the variants of a large picture, GB ~u), for example, is reduced only in its width, thus is a square of the dimensions 3/5 d and 3/5 d with d still being the diagonal of the complete picture VB. Another large picture ~ v) having the dimensions 2~ 4/5 d and 2/5 d is useful i~ a plurality of particioants is shown which are all at the same location, since in such an instance only the horizontal margins are omitted. Still another large picture 6~ ~w) having the dimensions 3/5 d and 2/5 d has all the margins omitted.

- 25 Figs. 5,6 and 7 show three different sizes of small pictures ~B.

All of these have the same height of 1/5 d, which permits ~he use of the marginal zones of all the large picture sizes GB (u), GB ~v) and GB (~), shown in ~ig. 4. The dimensions 1/5 d, 4/15 d clnd 2~ d are provided as useful widtns of small pictures 30 KB~x), KB~y), and KB~z), respectively. I~Thich of the small picture sizes to produce in central station BZ depends pa~ticularly on the provided maximum number of pictures of a split-screen -14 9 0 ~ -15 combination MB to be shown. If a greater number of large and small picture sizes is programmed which are not all absolutely necessary, the number of possible combinations is correspondingly enlarged too, of course.

Fig.s. 8, 9 and 10 show a selection of splitscreen combinations MB of larqe pictures 5B(u), GB(v), and GB(w) and small pictures KB(x), KB(y), KB~z).

Fig. 11 shows the raster for 12 small pictures Kb(x) and, in broken lines, for 9 small pictures KB(y).

In each of the small picture areas, for example in one of the upper corners, a mark KM for the respective participant may be reproduced (not shown), which is delivered by a special signal generator GKM in the central station (see Fig. 1).

Fig. 12 is a diagrammatical perspective view of a controller unit at a participant location, for carryinq actuatin~ elements needed by a participant to produce individual control instructions.

The dialing, audio control, or video control instructions are entered throuqh a conventional keyboard WE, for example, and through switches BE and potentiometers TE. The instructions are transmitted through the terminal unit (see Fig. 1 at AE) to the central station BZ. The participants receive check-back information Gn the status of the selector WE, the audio 349~4 -16 control TE, the video control BE, the terminal unit AE, the central station BZ, etc. through an optical indicator ZA, such as signal lamps, alphanumeric display, etc.

Accordingly, the invention comprises a communication system for conducting a videoconference between a plurality of participants who are located at a plurality of participant locations, comprising:

a participant terminal (SO) at each participant location, each having video signal source means for generating a video source signal (BSq) of the same at the participant location, video signal sink means for receiving a video sink siqnal (BSs) to produce a complete picture (VB), and selector means including a video control selector (BE) for generating control signals for controlling images from other locations used to make up the complete picture;

a central station (BZ) for receiving video source signals and control signals from each terminal and for outputting video sink signals to each terminal, said central station including video storage means (~Sp, KSp) for storing video source signals from each terminal as both a large picture (~,B) and a coarse picture (KB), multiplexer means (MUX) connected to said rn~ lZ34904 -16a video storage means and controllable to selectively pro vide video sink signals corresponding to large and coarse pictures from each participant terminal to each other participant terminal, video control means (BC) connected to said video storage means and said multiplexer means and responsive to control signals from said participant terminals to generate video sink signals corresponding to a complete picture for each terminal selectively composed of large pictures and small pictures from other participant terminals, and programmable write and read means (SC,LCj connected to said video storage means and to said video control means for generating the large and small pictures from the video source signals; and connecting lines (AL) connected between each participant terminal and said central station for carrying said video source signals, said video sink signals and said control signals, each connecting line including go and return direction video and control. siq nal channels (HK,RK) of sufficient bandwidth to carrysaid video source, video sink and control signals between 1;~34~0~ -17 said p~rticip~nt tcrminals and said central station.

ile a specific embodiment of the invention has been shown and described in detail to illustrate the application of the principles of the invcntion, it will be understood that the invention may be embodied otherwise ~ hout departing from such principlcs.

-17