Method for generating an overall room noise for passing to a real endpoint, use of said method and teleconferencing system

The invention relates to a method for generating a total spatial sound to be transmitted to a real endpoint, wherein an audio signal is assigned to the real endpoint. Furthermore, the invention relates to the use of the method and a teleconferencing system.

As a rule, several participants participate in a teleconference and communicate with each other. For this purpose, audio signals, in this case speech signals of the participants, must be processed and transmitted between the participants. In particular, one participant in each case must receive the audio signals of the other participants in the form of a total signal.

In advanced teleconferencing systems, a virtual (conference) room is simulated, which is virtually entered by all participants for purposes of the teleconference, or appears to communicate in it.

Such a teleconferencing system is for example from the <patcit><text>US 2010/0316232 A1</text></patcit> known. The teleconferencing system has teleconferencing end points each with an audio recording and playback unit, for example in the form of a headset with a headphone and a microphone. The teleconferencing endpoints allow participants to participate in a teleconference.

The document specifically describes that each of the participants is assigned a virtual position in a virtual space, in particular on a virtual conference table.

In order to provide the subscribers of the teleconferencing system with a realistic or lifelike hearing impression, room sounds are generated from audio signals recorded by respective other teleconferencing end points via their audio recording units. For this purpose, a receiving teleconferencing endpoint or listening subscriber is given the impression that an audio signal of a respective other teleconferencing endpoint or subscriber originates from its assigned virtual position within the virtual space and is received at its own position on the virtual conference table. would be heard.

For each (receiving) teleconferencing endpoint or subscriber, a total room sound from all room sounds of all other teleconferencing endpoints or subscribers is then mixed and presented to the associated subscriber via the associated audio playback unit, so that each subscriber ultimately receives an auditory impression as if he were actually would sit at the (virtual) conference table and communicate with the other participants.

The disadvantage here, however, is that the number of room sounds to be generated increases approximately quadratically with the number of participants. Thus, demands on available computing power and the like increase disproportionately with the number of participants. As a result, the number of participants in such a teleconference is usually limited and / or it requires large server farms to cope with the resulting computational effort.

From the <patcit><text>WO 199053673 A1</text></patcit> is another teleconferencing system known with similar disadvantages.

Another way to enable teleconferencing for larger subscriber numbers is in the <patcit><text>US Pat. No. 8,627,213 B1</text></patcit> proposed. It is envisaged to worsen the quality of the generated room sounds with increasing number of participants and thus reduce the amount of computation required per generated surround sound. This, however, suffers the ease of use of such a teleconferencing system.

The problem outlined above is further exacerbated by the fact that the spatial sounds ultimately represent multi-channel, in particular stereophonic, signals. Thus, at least two individual signals must be calculated for each surround sound to be generated. This further increases the computational effort actually required.

It is therefore an object of the present invention to provide a method and a teleconferencing device with which the computational outlay for the generation of overall room sounds, in particular for a large number of subscribers, can be reduced.

The object is achieved by a method for generating a total space sound to be transmitted to a real end point, wherein an audio signal is assigned to the real end point, with the method steps: <ul list-style="bullet"><li>a. Set a default recipient location within a virtual space</li><li>b. Assigning the real endpoint with associated audio signal to a defined position within the virtual space,</li><li>c. for at least one real endpoint generating a surround sound based on the associated audio signal, the associated defined position and the default receiver position,</li><li>d. Generating at least one total spatial sound for at least one real endpoint from at least a part of previously generated spatial sounds withException of the spatial sound of the at least one real endpoint and</li><li>e. Transmitting the at least one overall sound to the at least one real endpoint.</li></ul>

Thus, audio signals from a real endpoint, such as a first endpoint of a teleconferencing system, to another real endpoint, such as a second endpoint of the teleconferencing system, can be communicated by generating and transmitting a total surround sound for each participant based on the audio generated by other participants.

The audio signals can be recorded, for example via microphones at the real endpoints and used according to the method. The audio signals may preferably be voice signals, in particular monophonic voice signals.

According to the invention, a respectively "listening" participant or a real endpoint assigned to it for receiving audio signals of other participants or endpoints is positioned virtually in a virtual space at a position identical to all participants or real endpoints, the standard receiver position. The standard receiver position may, for example, correspond to a head page of a virtual conference table arranged in virtual space. This definition can already take place in the construction of a system applying the method according to the invention. It can also be done at the beginning of each use of the method.

In addition, a defined, preferably different from the standard receiver position, position within the virtual space is assigned to each real endpoint. For example, the defined positions may correspond to virtual seating positions on side areas of the virtual conference table.

Based on the audio signal of at least one real endpoint as well as its defined position and the standard receiver position in the virtual space, a surround sound is now generated. In particular, the surround sound may be generated as if the audio signal were being radiated from the defined position and received at the standard receiver position. For example, a direction and a distance from the position of the defined position and the standard receiver position to each other can be detected and a surround sound can be generated from the audio signal according to the direction and distance.

Thus, the impression is created as if audio signals of their respective associated real endpoints were each emitted from the respective defined position.

To generate the surround sound, it can be simulated that a receiver pair with at least two receivers spaced apart from one another is arranged at the standard receiver position. Then, for example, additionally a standard receiver direction, thus a "gaze" or "hearing direction", can be taken into account, with which the relative orientation of the receiver pair to the defined position can be described. Thus, further spatial sound influencing properties, for example predefinable emission characteristics or reception characteristics, can be taken into account.

For at least one real endpoint, spatial sounds of one or more other real endpoints are then combined to form a total sound sound. This overall sound is then transmitted to the at least one real endpoint.

Thus, the at least one real endpoint receives surround sounds based on the respective audio signals of other real endpoints. In particular, a real endpoint "hears" the overall sound associated with it. Its overall sound provides a surround sound impression, after which the other real endpoints would send their audio signals from the respective defined positions to its apparently own receiver position within the virtual space. Since the at least one real endpoint is only transmitted in each case the total space sound assigned to it, there is no direct possibility for this or its associated subscriber to recognize that the apparently own receiver position corresponds in each case to the standard receiver position and identical for all real endpoints is.

According to the invention, it suffices thereby to generate at most one surround sound and one overall sound for each real endpoint.

Thus, the number of room sounds required no longer increases approximately quadratically, but only approximately linearly with the number of real endpoints. The inventive method thus proves to be particularly efficient.

Also, an additional real endpoint with associated audio signal may simply be added by assigning it a defined position and creating surround sound for it, preferably before full-scale sounds are generated.

The method can be used or requires in particular for large numbers of participants Given number of participants only low computing power and transmission capacity. In other words, compared to the prior art, a substantial increase in efficiency is achieved without there being noticeable quality losses with regard to the spatial sound impression.

A further increase in efficiency can be achieved if audio signals are preselected overall and / or packet by packet. For example, only the audio signals of the real endpoints that exceed a predefinable level or performance level can be taken into account for the generation of surround sounds and / or overall room sounds. This can take place overall, that is to say for a complete audio signal, or else packet by packet, that is to say for segments of the audio signal. Other audio signals can thus be hidden. Thus, for example, unnecessary background noise can be eliminated. Also, the amount of signals or signal data to be processed can be reduced.

A surround sound or effect can be generated particularly easily if a surround sound is generated by means of an interaural transit time difference (ILD) and / or an interaural level difference (IPD). ILD and IPD, for example, make it possible to convert a monophonic audio signal into a (stereophonic) surround sound and to memorize the surround sound in an apparent direction. For this purpose, the audio signal can be fed into different channels of the room sound with a time delay to produce an ILD. To produce an IPD, the audio signal can be fed with different volume levels or different gains and / or attenuations into different channels of the surround sound.

The realism of generated spatial sounds or total room sounds can be further increased if an audio signal and / or a surround sound and / or a total room sound are filtered by means of a head-related transfer function.

Also, an echo and / or a reverberation effect, preferably as a function of at least one defined position, can be added to an audio signal, a surround sound and / or an overall sound.

It is particularly advantageous if the echo and / or the reverberation effect, for example, for imitation of different wall or ceiling materials of the virtual space to be set.

For example, the number of simulated reflections of audio signals on walls and ceilings can be adjusted. For example, it can be set whether only direct connections between a defined position and the standard receiver position for spatial sound generation are taken into account and / or indirect connections, in particular two, three or more virtual reflections, are taken into account. It is also possible to simulate materials in virtual space arranged virtual objects or space limitations, for example, in terms of their reflection, absorption and / or sound conduction properties. Thus, further, realism-enhancing effects can be implemented in the generated total sounds. This has a particularly advantageous effect,

The scope of the invention also includes a use of the method according to the invention in a multimedia system for at least two users, in particular a teleconferencing system or an Internet-based multi-player system. Multi-user multimedia systems may be designed, for example, as web-based audiovisual systems. In these, it is often provided that an audio signal is assigned to a real endpoint, for example a computer unit equipped with an audio playback unit, in particular loudspeakers. The audio signal may originate from a sound recording of an audio recording unit of the computer unit or in the computer unit and / or in a central unit of the multimedia system, for. As a central Internet server, be generated as the real endpoint associated audio signal.

Then, the inventive method can be used to generate based on the audio signals overall sounds with a high degree of realism and to transmit this efficiently to other real endpoints of the multimedia system.

The scope of the invention further includes a teleconferencing system having at least three teleconferencing endpoints, each of the teleconferencing endpoints having an audio playback unit and an audio recording unit, wherein the teleconferencing system is configured, a first and a second audio signal, such as a first and a second subscriber to record via the audio recording units of a first and a second teleconferencing endpoint, wherein the teleconferencing system is set to, a total room sound according to the inventive method from the first and the second audio signal to generate to transmit to a third teleconferencing endpoint, such as a third party, and output via the audio playback unit, each teleconferencing endpoint corresponding to a real endpoint in the sense of the method.

Such a teleconferencing system is suitable for applying the method according to the invention and thus drastically reducing the computation and / or transmission capacities required for a teleconference, but at the same time satisfying high quality requirements with regard to the overall room sounds formed.

It is particularly preferred if the teleconferencing system has a moderation unit which is set up to activate, deactivate, amplify and / or attenuate at least one audio signal, surround sound and / or overall sound. Thus, a guided via the teleconferencing teleconference can be moderated by means of one or more moderation units. For example, subscribers of the teleconference can thus be switched on or hidden and / or subsets of subscribers can be formed within a teleconference, so that one subgroup can be temporarily separated acoustically from another subgroup, for example.

In particular, it can be provided that the teleconferencing system is designed as a video conferencing system, preferably as a web conferencing system, wherein preferably each teleconferencing endpoint has an image display unit. The web conferencing system can be designed in particular as a browser-based web conferencing system. A web conference system makes it possible, for example, to be able to participate in a web conference produced by means of the web conferencing system via a browser which is executively installed on the respective computer unit. In particular, it can be provided that the teleconferencing system has a computer program component that is downloadable and / or executable by means of a browser.

It is particularly preferred if the teleconferencing system is set up to display the virtual space by means of the image display unit of the third teleconference endpoint, wherein in each case a first and second virtual object assigned to the first and the second participant are displayed at the defined positions within the virtual space corresponding to the first are assigned to the first and second teleconferencing endpoint and a third virtual object associated with the third participant is displayed at the standard receiver position.

Thus, for a subscriber or user of a teleconferencing endpoint or receiver of the associated overall sound, the created acoustic impression can be enriched by a visual impression matching thereto. Thus, an impression of a virtual reality can be further increased.

Further features and advantages of the invention will become apparent from the following detailed description of variants of the method according to the invention and of embodiments of the subjects of the invention, with reference to the figures of the drawing, which shows details essential to the invention, and from the claims. The features shown there are not necessarily to scale and presented in such a way that the features of the invention can be made clearly visible. The various features may be implemented individually for themselves or for a plurality of combinations in variants of the invention.

In the schematic drawing variants and embodiments of the invention are shown and are explained in more detail in the following description.

Show it:

<figref>1</figref> a teleconference with three subscribers according to the prior art;

<figref>2</figref> a teleconference in which the method according to the invention is used;

<figref>3</figref> an inventive teleconferencing system;

<figref>4a</figref> and <figref>4b</figref> Teleconference endpoints of the teleconferencing system of <figref>3</figref> two different participants.

<figref>1</figref> schematically illustrates the operation of a teleconferencing system in the prior art.

By way of example, three participants T take part in a teleconference_A, T_B, T_C part. The participants T_A, T_B, T_C each use teleconferencing endpoints A ', B', C '.

The teleconferencing endpoints A ', B', C 'are in a virtual space V₁ each positions P_{A '}, P_{B '}, P_{C '} assigned. To make a teleconference, first take theTeleconferencing endpoints A ', B', C 'each associated with them audio signals A_A, A_B, A_C, For example, monophonic speech signals, from the teleconference endpoints A ', B', C 'using participants T_A, T_B, T_Con microphones, for example. Subsequently, suitable room sounds R₁ to R₆ and transmitted to the respective other (receiving) teleconferencing end points A ', B', C '. There the room sounds R₁ to R₆ the participants T_A, T_B, T_C, for example via stereo headphones, presented. The room sounds R₁ to R₆ be formed, the participants T_A, T_B, T_C to give a surround sound impression, as if the audio signals A_A, A_B, A_C apparently in virtual space V₁ from the positions P assigned to the respective other teleconferencing end points A ', B', C '_{A '}, P_{B '}, P_{C '} to their own position P_{A '}, P_{B '} or P_{C '} would arrive.

Again <figref>1</figref> It is not difficult to deduce that in the prior art, only three subscribers T_A, T_B, T_C the total of 6 room sounds R₁ to R₆ are generated, each of the room sounds R₁ to R₆ each comprises at least one left and one right channel.

Thus, already with only three participants T_A, T_B, T_C Teleconference endpoints A ', B', C 'a total of 12 surround channels or individual signals are calculated and transmitted. The computational effort increases approximately quadratically with the number of teleconferencing endpoints.

This high calculation and transmission costs can be drastically reduced by using the method according to the invention. This should now be based on the<figref>2</figref> be explained in more detail using the example of a teleconference to be carried out.

In the <figref>2</figref> schematically a teleconference is shown, at which the three participants T_A, T_B, T_C over 3 teleconferencing endpoints A, B, C participate. The teleconferencing endpoints A, B, C represent real endpoints E_A, E_B, E_C in the sense of the method according to the invention. In turn, the teleconferencing end points A, B, C are each the audio signals A_A, A_B, A_C assigned.

First, a standard receiver position P_S within a virtual space V₂ established.

Subsequently, the telephoto endpoints A, B, C defined positions P_A, P_B, P_C in virtual space V₂ assigned.

In the <figref>2</figref> the case is shown in which the teleconferencing end point A the audio signals A_B and A_C in the form of a total room sound G_A from the other two teleconferencing B, C should receive. In the case illustrated here, therefore, the total space sound G is_A in such a way and to transmit to the teleconferencing end point A, that a Raumklangeindruck arises as if the audio signals A_B and A_C from the defined positions P_B, P_C to the standard receiver position P_S sent and received there. Analogous to the procedure illustrated for this case, total space sounds G_B and G_C for the teleconferencing endpoints B, C formed by the defined positions P_A, P_C (for teleconferencing endpoint B) or P_A, P_B (for teleconferencing endpoint C) in each case in connection with the standard receiver position P_S be based on.

For this purpose, in a further method step, at least for the teleconferencing end points B and C, room sounds R_B, R_C on the basis of the respective assigned audio signal A._B or A_C, the assigned defined positions P_B, P_C and the standard receiver position P_S generated.

If the total room sounds G_B and / or G_C are generated to increase efficiency analogous to a surround sound R_A generated and the room sounds R_C (for total room sound G_B) and R_B (for total room sound G_C) reused, that is reused without recalculation. If already created, the surround sound R_A, reused.

The example of the room sound R_C let the generation of the spatial sounds R_A, R_B or R_C be explained in more detail.

First of all, the audio signal A becomes_C pre-selected in packages. For this purpose, the audio signal A_C disassembled into individual packages. Within each individual package it is checked whether a given minimum sound level is reached. If this is not the case, this individual package is replaced by a zero signal of equal duration, otherwise the individual package is processed unchanged. Thus, unnecessary background noise can be minimized and at the same time the required later processing or computational effort can be further reduced.

In the <figref>2</figref> It can now be seen that the standard receiver position P_S within the virtual space V₂ two more positions P_L, or P_R assigned or previously defined. These are spaced from each other by a distance d.

Thus, within the virtual space different distances d_L and d_R of positions P_L and P_R to the teleconference end point C associated position P_C,

A virtually from the position P_C propagating audio signal would thus correspond to a suitably selected speed of sound with an interaural transit time difference ILD, in other words proportional to the difference of the distances d_L and d_R, at positions P_L or P_R arrive.

Also, due to propagation at different levels of sound, the audio signal would be at positions P_L or P_R Thus, this would also result in an interaural level difference IPD.

Therefore, the surround sound R_C formed in this process variant by the audio signal A_C offset in time by the resulting interaural transit time difference ILD and with different sound levels corresponding to the resulting interaural level difference IPD in a left and a right channel of the spatial sound R to be generated_C fed or calculated for the two channels.

In addition, both channels of the room sound R_C by means of a position P_L, P_R and P_C appropriately selected Head-Related Transfer Function filtered.

In this variant of the method, the surround sound R is used to improve a spatial sound impression_C added or added further effects.

In particular, echo and reverberation effects become the surround sound R_C according to a suitably chosen embodiment of the virtual space V₂ added to even apparent non-direct acoustic connections between the positions P_L, P_R and P_C to account for surround sound generation. In particular, there are limitations in advance<figref>21</figref>. <figref>22</figref>. <figref>23</figref>. <figref>24</figref> of the virtual space V₂For example, virtual walls, ceilings and floors, as well as virtual objects, such as a virtual conference table <figref>25</figref>, predefined in dependence on a desired embodiment and within the virtual space V₂ positioned. In addition, beforehand the limits<figref>21</figref>. <figref>22</figref>. <figref>23</figref>. <figref>24</figref> and sound properties associated with the virtual objects, in particular acoustic reflection and absorption properties, according to which the echo and reverberation effects are subsequently generated or adjusted.

In a further method step, the total space sound G_A for the teleconferencing endpoint A from the generated surround sounds R_B, R_Cthat is, from all room sounds R_A, R_B, R_C with the exception of the "own" room sound R_A, generated by channelwise addition of the two spatial sounds.

In a last method step, the total space sound G_A transmitted to the teleconferencing endpoint A and stereo headphones of the teleconferencing endpoint A the subscriber T._A presents.

Since it is thus only the generation of three room sounds R_A, R_B, R_C requires, is saved in accordance with the method in this way, compared to the prior art significantly computational capacity. In addition, the computational need increases only approximately linearly with the number of teleconferencing endpoints.

Furthermore, the <figref>2</figref> to deduce the idea of ​​the inventive method that the apparent radiation of the audio signals A_A, A_B, A_C or the room sounds R_A, R_B, R_C is spatially separated from the apparent reception of the total space sounds G_A, G_B, G_C, which according to the invention at the standard recipient position P_S is located.

In the <figref>3</figref> is now an inventive teleconferencing system <figref>100</figref> shown.

Of the <figref>3</figref> it can be seen that in turn the participants T_A, T_B, T_C Teleconferencing endpoints A, B, C are assigned. The teleconferencing end points A, B, C correspond to real endpoints E_A, E_B, E_C of the inventive method. Other participants or teleconferencing endpoints can the teleconferencing system<figref>100</figref> to be added at will; the number of 3 participants or teleconferencing endpoints is chosen here only as an example.

The teleconferencing endpoints A, B, C each have an audio playback unit <figref>101</figref> and an audio recording unit <figref>102</figref> on. The audio playback unit<figref>101</figref> and the audio recording unit <figref>102</figref> are in this embodiment as a headset, that is designed as headphones and microphones. The audio playback units<figref>101</figref> or the audio recording unit <figref>102</figref> are each with a computer unit <figref>103</figref> the respective teleconferencing endpoint A, B or C connected. About the audio recording units<figref>102</figref> become audio signals A_A, A_B, A_C, in particular speech signals of the participants T_A, T_B, T_C, added.

Furthermore, the teleconferencing endpoints A, B, C are by means of their computer units <figref>103</figref> with a central unit designed as a computing server <figref>108</figref> in terms of data, in particular via a communications network, in this case the Internet.

The central unit <figref>108</figref> receives from the teleconferencing end points A, B, C the audio signals A_A, A_B, A_C, generates the total space sounds G from these_A, G_B, G_C and transmits them to the respective teleconferencing end points A, B, C for playback or presentation via the respective audio playback units <figref>101</figref>, The processing of the audio signals A_A. A_B, A_C takes place by means of a central computer program component <figref>109</figref> on that in the central unit <figref>108</figref> executable installed.

The teleconferencing system <figref>100</figref> is especially designed as a web conferencing system. These are on the computer units<figref>103</figref> the teleconferencing endpoints B and C respectively include instances of an endpoint computer program component <figref>104</figref> executable installed.

On the teleconference endpoint A is an instance of an endpoint computer program component <figref>104 '</figref> executable installed. The endpoint computer program component<figref>104 '</figref> corresponds to the endpoint computer program component <figref>104</figref> with the difference that they additionally have a moderation unit <figref>106</figref> having.

In particular, these instances may be an endpoint computer program component <figref>104 '</figref> by means of one on the computer units <figref>103</figref> executable installed browser. The endpoint computer program components<figref>104 '</figref> are also by the computer units <figref>103</figref> downloadable on the central unit <figref>108</figref> installed, allowing use of the teleconferencing system <figref>100</figref> is easily enabled by the endpoint computer program components <figref>104 '</figref> from the central unit <figref>108</figref> by means of a browser on computer units <figref>103</figref> downloaded and executed there.

The moderation unit <figref>106</figref> is designed to provide settings with which a participant each one of the audio signals A_A, A_B, A_C amplify and generally be able to activate or deactivate. For this, the respectively selected settings are changed by the moderation unit<figref>106</figref> by means of the respective computer units <figref>103</figref> to the central unit <figref>108</figref> which transmits the audio signals A received from it_A, A_B, A_C appropriately reinforced, attenuated and / or selected before using the method according to the invention.

Thus, for example, temporary subgroups of subscribers of the teleconference can be formed via these setting options, between which at least temporarily no audio signals A are formed_A, A_B, A_C be replaced. In this embodiment, therefore, the subscriber has T_A Access to these settings of the moderation unit installed on its assigned teleconference point A. <figref>106</figref>, Thus participant T represents_A a presenter of the teleconference.

Furthermore, the <figref>3</figref> Image display units <figref>105</figref> to be taken with the respective computer units <figref>103</figref> the teleconferencing endpoints A, B, C are connected.

By way of example, the show <figref>4a</figref> and the <figref>4b</figref> Screen issues, as they participants T_A (<figref>4a</figref>) or T_B (<figref>4b</figref>) are presented via their teleconferencing endpoints A and B, respectively. A virtual space V can be seen in each case₃in which virtual objects A ', B', C ', in this case avatars, at the defined positions P_A, P_B, P_C or the standard receiver position P_S corresponding graphic positions within the virtual space V₃ are arranged. In each case, this is the respective subscriber T_A or T_B associated virtual object A 'and B' at the standard receiver position P_S corresponding graphic position. The other virtual objects B ', C' or A ', C' are on the other hand at the positions P_B and P_C or P_A and P_C, Thus, a visual impression corresponding to the auditory auditory impression by the screen output of the image display units<figref>105</figref> generated for the respective teleconferencing end points A, B, C.

QUOTES INCLUED IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

<ul list-style="bullet"><li>US 2010/0316232 A1 <b>[0004]</b></li><li>WO 199053673 A1 <b>[0009]</b></li><li>US 8627213 B1 <b>[0010]</b></li></ul>