Multi-point connection device, signal analysis and device, method, and program
Abstract
This record has no abstract on file.
Term
Projected expiry 26 June 2028.
- Priority
- Filed
- Granted
- Today
- Projected expiry
41 claims: 33 independent, 8 dependent
- 1複数の構成要素を含む第1の信号と前記第1の信号に含まれる複数の構成要素間の関係を表す第1の分析情報とを受ける第1の信号受信部と、 複数の構成要素を含む第2の信号と前記第2の信号に含まれる複数の構成要素間の関係を表す第2の分析情報とを受ける第2の信号受信部と、 前記第1の信号と前記第2の信号とを混合する信号混合部と、 前記第1の分析情報と前記第2の分析情報とを混合する分析情報混合部とを含むことを特徴とする多地点接続装置。
- 2前記分析情報混合部は、前記第1の分析情報と前記第2の分析情報とをそれぞれの周波数成分を表す第1の分析パラメータと第2の分析パラメータとに変換し、前記第1の分析パラメータと前記第2の分析パラメータとを周波数成分毎に混合する分析パラメータ混合部を含むことを特徴とする請求項1に記載の多地点接続装置。
- 3前記信号混合部は混合情報を生成し、 前記分析情報混合部は前記混合情報に基づいて前記第1の分析情報と前記第2の分析情報とを混合することを特徴とする請求項1に記載の多地点接続装置。
- 4前記分析情報混合部は、前記第1の分析情報と前記第2の分析情報とをそれぞれの周波数成分を表す第1の分析パラメータと第2の分析パラメータとに変換し、前記混合情報に基づいて前記第1の分析パラメータと前記第2の分析パラメータとを周波数成分毎に混合する分析パラメータ混合部を含むことを特徴とする請求項3に記載の多地点接続装置。
- 5前記分析情報混合部は、前記第1の分析パラメータと前記第2の分析パラメータとを選択する選択部を含むことを特徴とする請求項4に記載の多地点接続装置。
- 6前記混合情報は前記第1の信号と前記第2の信号との重み付けであることを特徴とする請求項3または請求項4に記載の多地点接続装置。
- 7前記第1の信号受信部は、前記前記第1の信号の周波数成分毎の特性を表す第1のオブジェクト情報を受け、 前記第2の信号受信部は、前記前記第2の信号の周波数成分毎の特性を表す第2のオブジェクト情報を受け、 前記第1のオブジェクト情報と前記第2のオブジェクト情報とを混合するオブジェクト情報混合部をさらに含むことを特徴とする請求項1に記載の多地点接続装置。
- 8前記オブジェクト情報混合部は、前記第1のオブジェクト情報と前記第2のオブジェクト情報とを重要度に応じて選択することを特徴とする請求項7に記載の多地点接続装置。
- 9複数の構成要素を含む入力信号と前記複数の構成要素間の関係を表す分析情報と前記入力信号の周波数成分毎の特性を表すオブジェクト情報とを受ける信号受信部と、 前記構成要素の出力を制御する構成要素レンダリング情報を受け、前記分析情報と前記オブジェクト情報と前記構成要素レンダリング情報とに基づいて前記構成要素が制御された出力信号を生成する出力信号生成部とを含むことを特徴とする信号制御装置。
- 10前記出力信号生成部は、 前記分析情報に基づいて前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成する構成要素情報変換部と、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて前記レンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 9 に記載の信号制御装置。
- 11前記出力信号生成部は、 前記分析情報と前記入力信号とに基づいて前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成する構成要素情報変換部と、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて前記レンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 9 に記載の信号制御装置。
- 12前記出力信号生成部は、さらに特定の構成要素を制御する信号制御情報を受け、前記分析情報と前記オブジェクト情報と前記構成要素レンダリング情報と前記信号制御情報とに基づいて前記構成要素が制御された出力信号を生成することを特徴とする請求項 9 に記載の信号制御装置。
- 13前記出力信号生成部は、 前記オブジェクト情報に基づいて前記信号が周波数成分に分解されたオブジェクト信号を生成し、前記分析情報に基づいて前記オブジェクト信号の構成要素を分解し、前記信号制御情報に基づいて前記構成要素を修正し、前記修正構成要素から修正信号を生成し、前記修正構成要素と前記修正信号との関係を表したパラメータを生成する信号制御部と、 前記パラメータに基づいて前記構成要素レンダリング情報を前記修正信号と前記出力信号との関係を表すレンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記修正信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 12 に記載の信号制御装置。
- 14前記出力信号生成部は、 前記分析情報と前記信号制御情報とに基づいて前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成する構成要素情報変換部と、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて前記レンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 12 に記載の信号制御装置。
- 15前記出力信号生成部は、 前記分析情報と前記信号制御情報と前記入力信号とに基づいて前記オブジェクト信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成する構成要素情報変換部と、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて前記レンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 12 に記載の信号制御装置。
- 16複数の構成要素を含む入力信号と前記複数の構成要素間の関係を表す分析情報と前記入力信号の周波数成分毎の特性を表すオブジェクト情報とを受ける信号受信部と、 前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報と特定の構成要素を制御する信号制御情報とを受け、前記分析情報と前記オブジェクト情報と前記オブジェクトレンダリング情報と前記信号制御情報とに基づいて前記構成要素が制御された出力信号を生成する出力信号生成部とを含むことを特徴とする信号制御装置。
- 17前記出力信号生成部は、 前記オブジェクト情報に基づいて前記信号が周波数成分に分解されたオブジェクト信号を生成し、前記分析情報に基づいて前記オブジェクト信号の構成要素を分解し、前記信号制御情報に基づいて前記構成要素を修正し、前記修正構成要素から修正信号を生成し、前記修正構成要素と前記修正信号との関係を表したパラメータを生成する信号制御部と、 前記パラメータと前記オブジェクトレンダリング情報とに基づいて前記修正信号と前記出力信号との関係を表すレンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記修正信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 16 に記載の信号制御装置。
- 18前記出力信号生成部は、 前記分析情報と前記信号制御情報とに基づいて前記オブジェクトレンダリング情報を修正するオブジェクトレンダリング情報修正部と、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて前記レンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 16 に記載の信号制御装置。
- 19前記出力信号生成部は、 前記分析情報と前記信号制御情報と前記入力信号とに基づいて前記オブジェクトレンダリング情報を修正するオブジェクトレンダリング情報修正部と、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて前記レンダリング情報を生成するレンダリング情報生成部と、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成するレンダリング部とを含むことを特徴とする請求項 16 に記載の信号制御装置。
- 20複数の構成要素を含む第1の信号と、前記第1の信号に含まれる複数の構成要素間の関係を表す第1の分析情報とを受信し、 複数の構成要素を含む第2の信号と、前記第2の信号に含まれる複数の構成要素間の関係を表す第2の分析情報とを受信し、 前記第1の信号と前記第2の信号とを混合し、 前記第1の分析情報と前記第2の分析情報とを混合することを特徴とする多地点接続方法。
- 21前記第1の分析情報と前記第2の分析情報とをそれぞれの周波数成分を表す第1の分析パラメータと第2の分析パラメータとに変換し、前記第1の分析パラメータと前記第2の分析パラメータとを周波数成分毎に混合することを特徴とする請求項 20 に記載の多地点接続方法。
- 22前記第1の信号と前記第2の信号とを混合して混合情報を生成し、 前記混合情報に基づいて、前記第1の分析情報と前記第2の分析情報とを混合することを特徴とする請求項 20 に記載の多地点接続方法。
- 23前記第1の分析情報と前記第2の分析情報とをそれぞれの周波数成分を表す第1の分析パラメータと第2の分析パラメータとに変換し、前記混合情報に基づいて前記第1の分析パラメータと前記第2の分析パラメータとを周波数成分毎に混合することを特徴とする請求項 22 に記載の多地点接続方法。
- 24前記第1の分析パラメータと前記第2の分析パラメータとを選択することを特徴とする請求項 23 に記載の多地点接続方法。
- 25前記混合情報は前記第1の信号と前記第2の信号との重み付けであることを特徴とする請求項 22 または請求項 23 に記載の多地点接続方法。
- 26前記前記第1の信号の周波数成分毎の特性を表す第1のオブジェクト情報を受信し、 前記前記第2の信号の周波数成分毎の特性を表す第2のオブジェクト情報を受信し、 前記第1のオブジェクト情報と前記第2のオブジェクト情報とを更に混合することを特徴とする請求項 20 に記載の多地点接続方法。
- 27前記第1のオブジェクト情報と前記第2のオブジェクト情報とを重要度に応じて選択することを特徴とする請求項 26 に記載の多地点接続方法。
- 28複数の構成要素を含む入力信号と、前記複数の構成要素間の関係を表す分析情報と、前記入力信号の周波数成分毎の特性を表すオブジェクト情報とを受信し、 前記構成要素の出力を制御する構成要素レンダリング情報を受信し、前記分析情報と前記オブジェクト情報と前記構成要素レンダリング情報とに基づいて、前記構成要素が制御された出力信号を生成することを特徴とする信号制御方法。
- 29前記分析情報に基づいて前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成し、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて、前記レンダリング情報を生成し、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成することを特徴とする請求項 28 に記載の信号制御方法。
- 30前記分析情報と前記入力信号とに基づいて、前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成し、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて、前記レンダリング情報を生成し、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成することを特徴とする請求項 28 に記載の信号制御方法。
- 31特定の構成要素を制御する信号制御情報を受信し、前記分析情報と前記オブジェクト情報と前記構成要素レンダリング情報と前記信号制御情報とに基づいて、前記構成要素が制御された出力信号を生成することを特徴とする請求項 28 に記載の信号制御方法。
- 32前記オブジェクト情報に基づいて、前記信号が周波数成分に分解されたオブジェクト信号を生成し、前記分析情報に基づいて前記オブジェクト信号の構成要素を分解し、前記信号制御情報に基づいて前記構成要素を修正し、前記修正構成要素から修正信号を生成し、前記修正構成要素と前記修正信号との関係を表したパラメータを生成し、 前記パラメータに基づいて前記構成要素レンダリング情報を前記修正信号と前記出力信号との関係を表すレンダリング情報を生成し、 前記レンダリング情報に基づいて、前記修正信号から前記出力信号を生成することを特徴とする請求項 31 に記載の信号制御方法。
- 33前記分析情報と前記信号制御情報とに基づいて、前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成し、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて、前記レンダリング情報を生成し、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成することを特徴とする請求項 31 に記載の信号制御方法。
- 34前記分析情報と前記信号制御情報と前記入力信号とに基づいて、前記オブジェクト信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報を生成し、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて、前記レンダリング情報を生成し、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成することを特徴とする請求項 31 に記載の信号制御方法。
- 35複数の構成要素を含む入力信号と、前記複数の構成要素間の関係を表す分析情報と、前記入力信号の周波数成分毎の特性を表すオブジェクト情報とを受信し、 前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報と特定の構成要素を制御する信号制御情報とを受信し、前記分析情報と前記オブジェクト情報と前記オブジェクトレンダリング情報と前記信号制御情報とに基づいて、前記構成要素が制御された出力信号を生成することを特徴とする信号制御方法。
- 36前記オブジェクト情報に基づいて前記信号が周波数成分に分解されたオブジェクト信号を生成し、前記分析情報に基づいて前記オブジェクト信号の構成要素を分解し、前記信号制御情報に基づいて前記構成要素を修正し、前記修正構成要素から修正信号を生成し、前記修正構成要素と前記修正信号との関係を表したパラメータを生成し、 前記パラメータと前記オブジェクトレンダリング情報とに基づいて、前記修正信号と前記出力信号との関係を表すレンダリング情報を生成し、 前記レンダリング情報に基づいて、前記修正信号から前記出力信号を生成することを特徴とする請求項 35 に記載の信号制御方法。
- 37前記分析情報と前記信号制御情報とに基づいて、前記オブジェクトレンダリング情報を修正し、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて、前記レンダリング情報を生成し、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成することを特徴とする請求項 35 に記載の信号制御方法。
- 38前記分析情報と前記信号制御情報と前記入力信号とに基づいて、前記オブジェクトレンダリング情報を修正し、 前記オブジェクト情報と前記オブジェクトレンダリング情報とに基づいて、前記レンダリング情報を生成し、 前記レンダリング情報に基づいて、前記入力信号から前記出力信号を生成することを特徴とする請求項 35 に記載の信号制御方法。
- 39複数の構成要素を含む第1の信号と前記第1の信号に含まれる複数の構成要素間の関係を表す第1の分析情報とを受信する処理と、 複数の構成要素を含む第2の信号と前記第2の信号に含まれる複数の構成要素間の関係を表す第2の分析情報とを受信する処理と、 前記第1の信号と前記第2の信号とを混合する処理と、 前記第1の分析情報と前記第2の分析情報とを混合する処理とを情報処理装置に実行させるプログラム。
- 40複数の構成要素を含む入力信号と前記複数の構成要素間の関係を表す分析情報と前記入力信号の周波数成分毎の特性を表すオブジェクト情報とを受信する処理と、 前記構成要素の出力を制御する構成要素レンダリング情報を受け、前記分析情報と前記オブジェクト情報と前記構成要素レンダリング情報とに基づいて前記構成要素が制御された出力信号を生成する処理とを情報処理装置に実行させるプログラム。
- 41複数の構成要素を含む入力信号と前記複数の構成要素間の関係を表す分析情報と前記入力信号の周波数成分毎の特性を表すオブジェクト情報とを受信する処理と、 前記入力信号と前記出力信号との関係を周波数成分毎に表したオブジェクトレンダリング情報と特定の構成要素を制御する信号制御情報とを受け、前記分析情報と前記オブジェクト情報と前記オブジェクトレンダリング情報と前記信号制御情報とに基づいて前記構成要素が制御された出力信号を生成する処理とを情報処理装置に実行させるプログラム。
Independent claims41
435 paragraphs, as filed
The present invention relates to a multipoint connection device, a signal analysis and device, and a method and a program thereof.
A multipoint connection system that connects a plurality of points to each other and allows participants in remote locations to participate in a conference is widely used as, for example, a remote conference system.
An example of a multipoint connection system is disclosed in Patent Document 1 (hereinafter, prior art 1). As shown in FIG. 74, the multipoint connection system includes terminals 9000, 9001 and 9002 distributed at each point, and a multipoint control unit (MCU) 9005 that controls data exchange between terminals. It has. The multipoint connection device 9005 mixes the signals output from each terminal and distributes them to all terminals. At the time of mixing, only the signal output from the distribution destination terminal is excluded. For example, the signal distributed to the terminal 9000 is a mixture of the signals output from the terminals 9001 and 9002.
FIG. 75 shows a configuration example of the multipoint connection device 9005. Although FIG. 75 shows an example of connecting three points, an arbitrary number of points can be connected. In FIG. 75, the transmission signals received from the terminals installed at the first to third points are output to the decoding units 9020, 9030, 9040 via the input terminals, and are decoded by the decoding units 9020, 9030, 9040. .. The decoded signal is further output to the mixing units 9021, 9031, and 9041. The mixing unit 9021 mixes the decoded signals from the second and third points, generates a mixed signal to be sent to the first point, and outputs the mixed signal to the coding unit 9022. The mixing unit 9031 mixes the decoded signals from the first and third points, generates a mixed signal to be sent to the second point, and outputs the mixed signal to the coding unit 9032. The mixing unit 9041 mixes the decoded signals from the first and second points, generates a mixed signal to be sent to the third point, and outputs the mixed signal to the coding unit 9042. The coding units 9022, 9032, and 9042 encode the mixed signal and output it to each terminal via the output terminal. The mixing units 9021, 9031, and 9041 can not only mix a plurality of signals but also apply various predetermined media processing (image processing, audio processing, data processing, etc.).
FIG. 74 shows a first configuration example of terminals 9000, 9001 and 9002. Since these terminals can have the same configuration, only the terminal 9000 is shown as a configuration example. Hereinafter, the terminal 9000 will be described as an example. The terminal 9000 includes a transmission unit 9006 including a noise suppression unit 9010 and an encoding unit 9011, and a receiving unit 9007 including a decoding unit 9012. An input signal is input to the noise suppression unit 9010 of the transmission unit 9006 via the input terminal. For example, in a general mobile phone, a signal (microphone signal) captured by a microphone becomes an input signal. The microphone signal is composed of a desired voice and background noise (hereinafter referred to as noise), and the noise suppression unit 9010 suppresses only the noise, keeps the desired voice as it is as much as possible, and informs the coding unit 9011 as a noise suppression voice. Output. The coding unit 9011 encodes the noise suppressing voice output from the noise suppressing unit 9010 based on a coding method such as CELP. The encoded information is output via the output terminal, modulated / amplified, and then output to the transmission line. That is, the transmission unit 9006 performs processing such as voice coding after the noise suppression processing, and outputs the signal to the transmission line. The receiving unit 9007 demodulates the signal received from the transmission line, digitizes it, and then outputs it to the decoding unit 9012. The decoding unit 9012 decodes the input signal and outputs it as an output signal. The output signal is input to the speaker and reproduced as an audible signal.
The noise suppression unit 9010 is generally known as a noise suppressor (noise suppression system), and suppresses noise superimposed on a desired audio signal. Generally, the power spectrum of the noise component is estimated using the input signal converted into the frequency domain, and the estimated power spectrum is subtracted from the input signal to suppress the noise mixed in the desired audio signal. .. By continuously estimating the power spectrum of the noise component, it can also be applied to the suppression of unsteady noise. An example of the noise suppressor technique is disclosed in Patent Document 2 (hereinafter referred to as the prior art 2).
Further, a technique for reducing the amount of calculation for noise suppression is disclosed in Non-Patent Document 1 (hereinafter, prior art 3).
The basic operation of each of these methods is the same. That is, the input signal is converted into the frequency domain by linear conversion, the amplitude component is extracted, and the suppression coefficient is calculated for each frequency component. The product of the suppression coefficient, the amplitude at each frequency component, and the phase of each frequency component are combined and inversely converted to obtain a noise-suppressed output. At this time, the suppression coefficient is a value between zero and 1, and if it is zero, the output is zero with complete suppression, and if it is 1, the input is output as it is without suppression.
As a second configuration example of the terminals 9000, 9001 and 9002, there is a case where the noise suppression unit 9010 does not exist from the first configuration example shown in FIG. 74. This configuration corresponds not only when the terminal does not include the noise suppression unit 9010, but also when the user turns off the function or when the suppression degree of the noise suppression unit 9010 is insufficient. In such a terminal, noise or the like mixed in a desired audio signal is not sufficiently suppressed and is transmitted to another terminal as it is. In this case, noise is mixed in the mixed signal heard by the participants of the conference, and the sound quality is lowered. For this reason, there is a problem that an important phrase is misunderstood or fatigue increases with long-term use. Even if the terminal of the first configuration example having the noise suppression unit 9010 is used, if the suppression of the noise suppression unit 9010 is insufficient or the function of the noise suppression unit 9010 is disabled, the same problem occurs. There is.
Here, in general, there is a trade-off relationship between the residual noise that remains unsuppressed and the distortion of the output noise-suppressed voice. Reducing the residual noise increases the distortion, and reducing the distortion increases the residual noise. In the prior art 1, the noise suppression unit in the transmission unit adjusts the balance between residual noise and distortion, that is, controls desired voice and noise.<patcit num="1"><text>Japanese Unexamined Patent Publication No. 2000-83229</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 2002-204175</text></patcit><nplcit num="1"><text>May 2006, Proceedings of ICASSP, (PROCEEDINGS OF ICASSP, VOL.I, PP.473-476, MAY, 2006), pp. 473-476</text></nplcit>
<p num="0011"> However, the prior art 1 has a problem that the receiving unit cannot control each component (for example, desired voice and noise) corresponding to each sound source of the input signal at each point. For example, the optimum state of the balance between residual noise and distortion differs depending on the environment on the receiving unit side, but it cannot be adjusted on the receiving unit side. Moreover, it cannot be controlled for each signal from each point.</p><p num="0012"> Therefore, the present invention has been invented in view of the above problems, and an object of the present invention is to provide a technique capable of controlling each component corresponding to each sound source of an input signal at each point in a receiving unit. There is.</p>
<p num="0013"> The present invention, which solves the above problems, receives a first signal including a plurality of components and a first signal receiving information representing a relationship between the plurality of components included in the first signal. A second signal receiving unit that receives a unit, a second signal including a plurality of components, and a second analytical information representing a relationship between the plurality of components included in the second signal, and the first signal receiving unit. A multipoint connection device including a signal mixing unit that mixes the signal of the above and the second signal, and an analysis information mixing unit that mixes the first analysis information and the second analysis information. Is.</p><p num="0014"> The present invention that solves the above problems includes a signal receiving unit that receives an input signal including a plurality of components, a signal analysis unit that generates analysis information representing a relationship between the plurality of components from the input signal, and the input. The signal analyzer is characterized by including an object information extraction unit that generates object information representing the characteristics of each frequency component of the signal.</p><p num="0015"> The present invention that solves the above problems is a signal receiving unit that receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. And an output signal generation unit that receives component rendering information that controls the output of the component and generates an output signal in which the component is controlled based on the analysis information, the object information, and the component rendering information. It is a signal control device characterized by including.</p><p num="0016"> The present invention that solves the above problems is a signal receiving unit that receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. The analysis information, the object information, and the object rendering information are received, and the object rendering information representing the relationship between the input signal and the output signal for each frequency component and the signal control information for controlling a specific component are received. The signal control device is characterized by including an output signal generation unit for generating a controlled output signal based on the signal control information.</p><p num="0017"> The present invention that solves the above problems receives a first signal including a plurality of components and a first analytical information representing a relationship between the plurality of components included in the first signal, and receives a plurality of first signals. The second signal including the components and the second analysis information representing the relationship between the plurality of components included in the second signal are received, and the first signal and the second signal are combined. It is a multipoint connection method characterized by mixing and mixing the first analysis information and the second analysis information.</p><p num="0018"> The present invention that solves the above problems generates analytical information representing the relationship between the plurality of components from an input signal including a plurality of components, and generates object information representing the characteristics of each frequency component of the input signal. This is a signal analysis method characterized by the above.</p><p num="0019"> The present invention that solves the above problems receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. , The component rendering information that controls the output of the component is received, and the component generates a controlled output signal based on the analysis information, the object information, and the component rendering information. This is a signal control method.</p><p num="0020"> The present invention that solves the above problems receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. The object rendering information representing the relationship between the input signal and the output signal for each frequency component and the signal control information for controlling a specific component are received, and the analysis information, the object information, and the object rendering information The signal control method is characterized in that the component generates a controlled output signal based on the signal control information.</p><p num="0021"> The present invention that solves the above problems includes a process of receiving a first signal including a plurality of components and a first analysis information representing a relationship between the plurality of components included in the first signal, and a plurality of processes. The process of receiving the second signal including the components of the above and the second analysis information representing the relationship between the plurality of components included in the second signal, the first signal, and the second signal. This is a program that causes an information processing apparatus to execute a process of mixing the first analysis information and a process of mixing the second analysis information.</p><p num="0022"> The present invention that solves the above problems includes a process of receiving an input signal including a plurality of components, a process of generating analytical information representing a relationship between the plurality of components from the input signal, and a frequency of the input signal. This is a program that causes an information processing device to execute a process of generating object information representing the characteristics of each component.</p><p num="0023"> The present invention that solves the above problems is a process of receiving an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. , Receives component rendering information that controls the output of the component, and processes processing to generate an output signal in which the component is controlled based on the analysis information, the object information, and the component rendering information. It is a program to be executed by the device.</p><p num="0024"> The present invention that solves the above problems is a process of receiving an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. The analysis information, the object information, the object rendering information, and the object rendering information are received, and the object rendering information representing the relationship between the input signal and the output signal for each frequency component and the signal control information for controlling a specific component are received. This is a program that causes an information processing apparatus to execute a process of generating an output signal whose components are controlled based on the signal control information.</p>
<p num="0025"> In the present invention, the transmitting unit analyzes the signal, calculates the analysis information, mixes the analysis information with the multipoint connection device, and controls the input signal in the receiving unit based on the mixed analysis information. ..</p><p num="0026"> Therefore, in the receiving unit, it is possible to control each component corresponding to each sound source of the input signal at each point. For example, the receiving unit can make adjustments such as adding noise to the sound from the main venue but removing noise from the sound from the sub-venue.</p><p num="0027"> Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.</p>
<figref num="1">Block diagram showing the first embodiment of the present invention</figref><figref num="2">Configuration example of coding unit 100</figref><figref num="3">Configuration example of decoding unit 150</figref><figref num="4">Configuration example of signal analysis unit 101</figref><figref num="5">Configuration example of signal control unit 151</figref><figref num="6">Configuration example of multipoint connection device (MCU) 2105</figref><figref num="7">Configuration example of analysis information mixing unit 2114</figref><figref num="8">Second configuration example of the multipoint connection device (MCU) 2105</figref><figref num="9">Configuration example of analysis information mixing unit 2117</figref><figref num="10">Configuration example of analysis information calculation unit 121</figref><figref num="11">Configuration example of background sound information generation unit 202</figref><figref num="12">Configuration example of signal processing unit 172</figref><figref num="13">Configuration example of suppression coefficient decoding unit 260</figref><figref num="14">Configuration example of analysis parameter mixing unit 2151</figref><figref num="15">Configuration example of analysis parameter mixing unit 2153</figref><figref num="16">Second configuration example of background sound information generation unit 202</figref><figref num="17">Configuration example of suppression coefficient reconstruction unit 250</figref><figref num="18">Second configuration example of background sound information generation unit 202</figref><figref num="19">Second configuration example of analysis parameter mixing unit 2151</figref><figref num="20">Second configuration example of analysis parameter mixing unit 2153</figref><figref num="21">Second configuration example of analysis information calculation unit 121</figref><figref num="22">Second configuration example of signal processing unit 172</figref><figref num="23">Configuration example of suppression coefficient calculation unit 252</figref><figref num="24">Third configuration example of signal processing unit 172</figref><figref num="25">Third configuration example of analysis parameter mixing unit 2151</figref><figref num="26">Third configuration example of analysis parameter mixing unit 2153</figref><figref num="27">Block diagram showing a third embodiment of the present invention</figref><figref num="28">Configuration example of signal control unit 350</figref><figref num="29">Configuration example of signal processing unit 360</figref><figref num="30">Configuration example of suppression coefficient reconstruction unit 450</figref><figref num="31">Configuration example of suppression coefficient correction unit 460</figref><figref num="32">Second configuration example of suppression coefficient correction unit 460</figref><figref num="33">Third configuration example of suppression coefficient correction unit 460</figref><figref num="34">Second configuration example of suppression coefficient reconstruction unit 450</figref><figref num="35">Third configuration example of suppression coefficient reconstruction unit 450</figref><figref num="36">Second configuration example of signal processing unit 360</figref><figref num="37">Configuration example of suppression coefficient calculation unit 452</figref><figref num="38">Third configuration example of signal processing unit 360</figref><figref num="39">Second configuration example of suppression coefficient calculation unit 452</figref><figref num="40">Block diagram showing a fifth embodiment of the present invention</figref><figref num="41">Configuration example of multipoint connection device (MCU) 2105</figref><figref num="42">Configuration example of output signal generator 2550</figref><figref num="43">Second configuration example of output signal generator 2550</figref><figref num="44">Configuration example of component information conversion unit 2565</figref><figref num="45">Third configuration example of output signal generator 2550</figref><figref num="46">Configuration example of component information conversion unit 2566</figref><figref num="47">Block diagram showing a seventh embodiment of the present invention</figref><figref num="48">Configuration example of output signal generator 2700</figref><figref num="49">Second configuration example of output signal generator 2700</figref><figref num="50">Configuration example of object rendering information correction unit 2770</figref><figref num="51">Third configuration example of output signal generator 2700</figref><figref num="52">Configuration example of object rendering information correction unit 2780</figref><figref num="53">Block diagram showing a ninth embodiment of the present invention</figref><figref num="54">Configuration example of output signal generator 2900</figref><figref num="55">Configuration example of component information conversion unit 2910</figref><figref num="56">Second configuration example of output signal generator 2900</figref><figref num="57">Configuration example of component information conversion unit 3001</figref><figref num="58">Block diagram showing the eleventh embodiment of the present invention</figref><figref num="59">Configuration example of signal analysis unit 900</figref><figref num="60">Second configuration example of signal analysis unit 900</figref><figref num="61">Configuration example of analysis information calculation unit 911</figref><figref num="62">Second configuration example of analysis information calculation unit 912</figref><figref num="63">Block diagram showing the thirteenth embodiment of the present invention</figref><figref num="64">Configuration example of coding unit 1100</figref><figref num="65">Configuration example of signal analysis unit 1101</figref><figref num="66">Configuration example of decoding unit 1150</figref><figref num="67">Configuration example of signal control unit 1151</figref><figref num="68">Second configuration example of signal analysis unit 101</figref><figref num="69">Second configuration example of signal control unit 151</figref><figref num="70">Third configuration example of signal analysis unit 101</figref><figref num="71">Third configuration example of signal control unit 151</figref><figref num="72">Block diagram showing the fourteenth embodiment of the present invention</figref><figref num="73">Block diagram showing the fifteenth embodiment of the present invention</figref><figref num="74">Block diagram showing a conventional multipoint conference system</figref><figref num="75">Configuration example of multipoint connection device 9005</figref>
Code description
10, 2506, 90 Transmitter 15, 35, 2507 Receiver 100, 1100, 2113, 2123, 2133, 2513, 9011, 9022, 9032, 9042 Coding section 101, 900, 1101 Signal analysis department 102, 2115, 2125, 2135, 2511, 2514 Multiplexing section 110, 120, 171, 920 Converter 111 Quantization Department 121, 911, 912, 121 Analysis Information Calculation Department 150, 1150, 2111, 2121, 2131, 2511, 2521, 2531, 9012, 9020, 9030, 9040 Decryptor 151, 2560, 2760 Signal control unit 152, 2110, 2120, 2130, 2510, 2520, 2530, 2551 Separator 160 Inverse quantization unit 161, 173 Inverse converter 172, 360 Signal processing unit 200 Background sound estimation unit 201, 252, 452 Suppression coefficient calculation unit 201 Suppression coefficient calculation unit 202 Background sound information generator 203, 207 Signal to background sound ratio calculation unit No. 204 background sound ratio coding unit 205 Background sound coding section 250, 450 Suppression coefficient reconstruction part 251, 451, 470 Multiplier 253 subtractor 260 Suppression coefficient decoding unit 261 Signal to background sound ratio decoding unit 262 Suppression coefficient converter 263 Background sound decoding unit 264 Suppression coefficient generator 460 Suppression coefficient correction part 461 Signal to background sound ratio correction section 464 Background sound correction part 471 Comparison section 472 Designated suppression coefficient control unit 473 switch 651, 653 Component parameter generator 910 Quantization strain calculator 1020 Background sound estimation unit 1200 Signal Separation Analysis Department 1201 Separation filter coding unit 1202 Separation filter decoding unit 1203 filter 1210 Sound Environment Analysis Department 1211 Sound environment information coding unit 1212 Sound environment information decoding unit 1213 Sound Environment Information Processing Department 2021 Suppression coefficient coding unit 2100, 2101, 2102, 2300, 2301, 2302, 2500, 2501, 2502, 3000, 3001, 3002, 3300, 3301, 3302, 3401, 3402, 3403, 3404, 3405, 9001, 9002, 9006, 9007 terminals 2105, 2505, 3410, 3411, 9005 Multipoint connection device (MCU) 2112, 2122, 2132, 2116, 2126, 2136, 2512, 9021, 9031, 9041 Mixing part 2114, 2117, 2124, 2127, 2134, 2137, 2516 Analytical information mixing section 2150, 2160 Analytical information decoding unit 2151, 2153 Analytical parameter mixing section 2152 Analytical information coding unit 2201, 2231 Background sound mixing section 2202, 2232, 2200, 2230 Selection section 2203, 2233 Suppression coefficient mixing part 2204, 2214 Suppression coefficient converter 2205 Suppression coefficient inverse converter 2510 Object information extraction unit 2515 Object information mixing section 2550, 2700, 2900 Output signal generator 2561, 2564 Rendering information generator 2563 Rendering section 2565, 2566 Component information conversion unit 2611 Object rendering information generator 2770, 2780 Object Rendering Information Correction Department 2810 Object rendering information change part 2910, 3001 Component information conversion unit 3000, 3002 Component parameter generator 3500, 3501, 3502, 3503 Computer 9010 Noise suppressor
The first embodiment of the present invention will be described with reference to FIG. The present embodiment is characterized in that the receiving terminal controls each component corresponding to each sound source of the input signal at each point based on the analysis information.
As shown in FIG. 1, the multipoint connection system according to the present embodiment is a multipoint connection device (MCU:) that controls data exchange between terminals 2100, 2101 and 2102 and terminals distributed at each point. It is equipped with Multipoint Control Unit) 2105. The multipoint connection device 2105 mixes the signals output from each terminal and distributes them to all terminals. At the time of mixing, only the signal output from the distribution destination terminal is excluded. For example, the signal distributed to the terminal 2100 is a mixture of the signals output from the terminals 2101 and 2102. Although FIG. 1 shows an example of connecting three points, an arbitrary number of points can be connected.
A configuration example of terminals 2100, 2101, and 2102 will be described with reference to FIGS. 1 to 5. Since these terminals can have the same configuration, only the terminal 2100 is shown as a configuration example. Hereinafter, the terminal 2100 will be described as an example.
Referring to FIG. 1, the terminal 2100 is composed of a transmission unit 10 including a coding unit 100, a signal analysis unit 101, and a multiplexing unit 102, and a receiving unit 15 including a decoding unit 150, a signal control unit 151, and a separation unit 152. Will be done. The input signal is input to the coding unit 100 and the signal analysis unit 101 in the transmitting unit 10. The coding unit 100 encodes the input signal and outputs the coded signal to the multiplexing unit 102. The signal analysis unit 101 calculates the analysis information of the components corresponding to each sound source included in the input signal, and outputs the analysis information to the multiplexing unit 102. The multiplexing unit 102 multiplexes the coded signal output from the coding unit 100 and the analysis information output from the signal analysis unit 101, and outputs it as a transmission signal to the transmission line. The transmission signal is input to the separation unit 152 in the reception unit 15. The separation unit 152 separates the transmission signal into a coded signal and analysis information, and outputs the coded signal to the decoding unit 150 and the analysis information to the signal control unit 151. The decoding unit 150 decodes the coded signal to generate a decoded signal, and outputs the decoded signal to the signal control unit 151. Here, the decoded signal is composed of a general plurality of sound sources. Based on the analysis information output from the separation unit 152, the signal control unit 151 operates the decoding signal output from the decoding unit 150 for each component corresponding to each sound source, and outputs it as an output signal. The signal control unit 151 may operate in units of component groups composed of a plurality of components instead of the components corresponding to each sound source.
A configuration example of the coding unit 100 will be described in detail with reference to FIG. The coding unit 100 receives the input signal and outputs the coded signal. The coding unit 100 includes a conversion unit 110 and a quantization unit 111. First, the input signal is input to the conversion unit 110. Next, the conversion unit 110 decomposes the input signal into frequency components to generate the first conversion signal. The conversion unit 110 outputs the first conversion signal to the quantization unit 111. Then, the quantization unit 111 quantizes the first conversion signal and outputs it as a coded signal.
The conversion unit 110 puts together a plurality of input signal samples to form one block, and applies frequency conversion to this block. Known examples of frequency transform include Fourier transform, cosine transform, and KL (Carunenlebe) transform. The technology and its properties related to the specific calculation of these conversions are described in Non-Patent Document 2 (1990, "Digital Coding of Waveforms", Prentice Hall (DIGITAL CODING OF WAVEFORMS, PRINCIPLES AND APPLICATIONS). It is disclosed in TO SPEECH AND VIDEO, PRENTICE-HALL, 1990.)).
The conversion unit 110 can also apply the above-mentioned conversion to the result of weighting one block of the input signal sample with the window function. As such window functions, window functions such as Humming, Hanning (Han), Kaiser, and Blackman are known. It is also possible to use more complicated window functions. Techniques related to these window functions are described in Non-Patent Document 3 (1975, "Digital Signal Processing", Prentice Hall (DIGITAL SIGNAL PROCESSING, PRENTICE-HALL, 1975.)) and Non-Patent Document 4 (1993.). It is disclosed in "MULTIRATE SYSTEMS AND FILTER BANKS", Prentice Hall (MULTIRATE SYSTEMS AND FILTER BANKS, PRENTICE-HALL, 1993.).
When the conversion unit 110 constitutes one block from a plurality of input signal samples, overlap (overlap) may be allowed in each block. For example, when applying an overlap of 30% of the block length, the last 30% of the signal samples belonging to one block are duplicated in multiple blocks as the first 30% of the signal samples belonging to the next block. .. Techniques related to blocking and conversion with overlap are disclosed in Non-Patent Document 2.
Further, the conversion unit 110 may be configured by a band division filter bank. .. The band division filter bank is composed of a plurality of band pass filters. The band division filter bank divides the received input signal into a plurality of frequency bands and outputs the received input signal to the quantization unit 111. Band division The frequency bands of the filter bank may be evenly spaced or unequally spaced. By dividing the band at unequal intervals, it is possible to divide the low frequency band into a narrow band to lower the time resolution, and to divide the high frequency band into a wide band to increase the time resolution. Typical examples of non-equidistant division include octave division in which the band is sequentially halved toward the low frequency range and critical band division corresponding to human auditory characteristics. A technique related to a band division filter bank and its design method is disclosed in Non-Patent Document 4.
The quantization unit 111 removes the redundancy of the input signal and outputs a coded signal. As a method of removing redundancy, control is performed so that the correlation of the input signal is minimized. Furthermore, a signal component that is not perceived by hearing may be removed by utilizing an auditory characteristic such as a masking effect. Quantization methods such as linear quantization and non-linear quantization are known as quantization methods. The quantized signal can be further stripped of redundancy by using Huffman coding or the like.
A configuration example of the decoding unit 150 will be described in detail with reference to FIG. The decoding unit 150 receives the main signal and outputs the decoding signal. The decoding unit 150 includes an inverse quantization unit 160 and an inverse conversion unit 161. The dequantization unit 160 dequantizes the received main signal of each frequency and generates a first conversion signal composed of a plurality of frequency components. Then, the inverse quantization unit 160 outputs the first conversion signal to the inverse conversion unit 161. The inverse conversion unit 161 reverse-converts the first conversion signal to generate a decoding signal. Then, the inverse conversion unit 161 outputs a decoding signal.
As the inverse conversion applied by the inverse conversion unit 161, it is desirable that the inverse conversion corresponding to the conversion applied by the conversion unit 110 is selected. For example, when the conversion unit 110 collectively constitutes one block of a plurality of input signal samples and applies frequency conversion to this block, the inverse conversion unit 161 performs the corresponding inverse conversion for the same number of samples. Apply. Further, when the conversion unit 110 constitutes one block from a plurality of input signal samples, if overlap is allowed in each block, the inverse conversion unit 161 corresponds to this after the inverse conversion. Apply the same overlap to the signal. Further, when the conversion unit 110 is composed of the band division filter bank, the inverse conversion unit 161 is composed of the band synthesis filter bank. The technology related to the band synthesis filter bank and its design method is disclosed in Non-Patent Document 4.
In the description of the coding unit 100 and the decoding unit 150 in FIGS. 2 and 3, the conversion coding including the conversion unit inside has been assumed, but pulse code modulation (PCM) and adaptive differential pulse code modulation (ADPCM) have been described. , Further, analytical synthesis coding typified by CELP or the like may be applied. The technology related to PCM / ADPCM is disclosed in Non-Patent Document 2. The technology related to CELP is described in Non-Patent Document 5 (March 1985, IE International Conference on Acoustic Speech and Signal Processing, 25.1.1, (IEEE INTERNATIONAL). CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 25.1.1, MAR, 1985, pp.937-940) pp. 937-940).
Further, the coding unit 100 outputs the input signal as it is to the multiplexing unit 102 without performing the coding process, and the decoding unit 150 inputs the main signal as it is to the signal control unit 151 without performing the decoding process. May be good. With this configuration, it is possible to eliminate the distortion of the signal due to the coding / decoding processing. Further, the distortion-free compression / decompression processing may be performed by the coding unit 100 and the decoding unit 150. With this configuration, the signal control unit 151 can receive the decoded signal without causing distortion in the input signal.
A configuration example of the signal analysis unit 101 will be described in detail with reference to FIG. The signal analysis unit 101 receives the input signal and outputs the analysis information. The signal analysis unit 101 includes a conversion unit 120 and an analysis information calculation unit 121. The conversion unit 120 decomposes the received input signal into frequency components and generates a second conversion signal. The conversion unit 120 outputs the second conversion signal to the analysis information calculation unit 121. The analysis information calculation unit 121 decomposes the second conversion signal into components corresponding to the sound source, and generates analysis information representing the relationship between the plurality of components. Then, the analysis information calculation unit 121 outputs the analysis information. Further, the analysis information calculation unit 121 may decompose the second conversion signal into a component group composed of a plurality of components and calculate the analysis information. The signal analysis unit 101 may encode the analysis information when the analysis information has redundancy. As a result, the redundancy of the analysis information can be minimized. Regarding the conversion method in the conversion unit 120, the conversion method in the conversion unit 110 may be used.
A configuration example of the signal control unit 151 will be described in detail with reference to FIG. The signal control unit 151 receives the decoding signal and the analysis information, and outputs an output signal. The signal control unit 151 includes a conversion unit 171, a signal processing unit 172, and an inverse conversion unit 173. The conversion unit 171 decomposes the received decoding signal into frequency components to generate a second conversion signal. The signal control unit 151 outputs the second conversion signal to the signal processing unit 172. The signal processing unit 172 decomposes the second converted signal into components corresponding to the sound source using the analysis information, changes the relationship between the plurality of components, and generates a modified decoding signal. Then, the signal processing unit 172 outputs the modified decoding signal to the inverse conversion unit 173. Further, the signal processing unit 172 may be decomposed into a component group composed of a plurality of components and the relationship between the plurality of components may be changed. When the analysis information is encoded in the analysis information calculation unit 121, the signal processing unit 172 performs the decoding process and then performs the above process. The inverse conversion unit 173 reverse-converts the modified / decoded signal to generate an output signal. Then, the inverse conversion unit 173 outputs an output signal. As for the reverse conversion method in the reverse conversion unit 173, the reverse conversion method in the reverse conversion unit 161 can be used.
Next, a first configuration example and a second configuration example of the multipoint connection device 2105 will be described.
First, the first configuration example will be described with reference to FIGS. 6 and 7.
FIG. 6 shows a first configuration example of the multipoint connection device 2105. Although FIG. 6 shows an example of connecting three points, an arbitrary number of points can be connected. The multipoint connection device 2105 includes separation units 2110, 2120, 2130, decoding units 2111, 2121, 2131, mixing units 2112, 2122, 2132, coding units 2113, 2123, 2133, and analysis information mixing unit 2114, It is composed of 2124 and 2134 and multiplexing units 2115, 2125 and 2135.
Referring to FIG. 6, the transmission signals output from the terminals installed at the first to third points are input to the separation units 2110, 2120, and 2130 via the input terminals. Separation units 2110, 2120, and 2130 separate transmission signals into coded signals and analysis information, respectively, output the coded signals to decoding units 2111, 2121, and 2131, and output analysis information to analysis information mixing units 2114, 2124, and 2134. Output to. The decoding units 2111, 2121, and 2131 decode the encoded signal to generate a decoded signal, and output the encoded signal to the mixing units 2112, 2122, and 2132.
The analysis information mixing unit 2114 mixes the analysis information from the second and third points to generate the mixed analysis information, and outputs the mixed analysis information to the multiplexing unit 2115. The analysis information mixing unit 2124 mixes the analysis information from the first and third points to generate the mixed analysis information, and outputs the mixed analysis information to the multiplexing unit 2125. The analysis information mixing unit 2134 mixes the analysis information from the first and second points to generate the mixed analysis information, and outputs the mixed analysis information to the multiplexing unit 2135.
The mixing unit 2112 mixes the decoded signals from the second and third points to generate a mixed signal, and outputs the mixed signal to the coding unit 2113. The mixing unit 2122 mixes the decoded signals from the first and third points to generate a mixed signal, and outputs the mixed signal to the coding unit 2123. The mixing unit 2132 mixes the decoded signals from the first and second points to generate a mixed signal, and outputs the mixed signal to the coding unit 2133. The coding units 2113, 2123, and 2133 encode the mixed signal and output the mixed coded signal to the multiplexing units 2115, 2125, and 2135, respectively.
The multiplexing units 2115, 2125, and 2135 multiplex and transmit the mixed coding signal output from the coding units 2113, 2123, and 2133 and the mixed analysis information output from the analysis information mixing units 2114, 2124, and 2134, respectively. It is output as a signal to the transmission line at each point.
The mixed analysis information and the mixed coded signal described above are the same as the analysis information and the coded signal described in the terminals 2100, 2101, and 2102 of FIG. When it is output as a transmission signal from the multiplexing units 2115, 2125, and 2135 and separated by the separation unit 152 of the terminals 2100, 2101, and 2102, the mixed analysis information and the mixed signal coded signal are the analysis information and the coding, respectively. It is treated as a signal. Here, in order to clarify that the analysis information and signals at multiple points are mixed, they are described as mixed analysis information and mixed coded signals. The same applies to the following description.
Further, since the detailed operations of the decoding units 2111, 2121, and 2131 are the same as those of the decoding unit 150 and the detailed operations of the coding units 2113, 2123, and 2133 are the same as those of the coding unit 100, the description thereof will be omitted.
FIG. 7 is a configuration example of the analysis information mixing units 2114, 2124, and 2134 in FIG. Since these terminals can have the same configuration, the analysis information mixing unit 2114 will be described below as an example.
The analysis information mixing unit 2114 is composed of analysis information decoding units 2150 and 2160, analysis parameter mixing unit 2151, and analysis information coding unit 2152. The analysis information decoding unit 2150 inputs the analysis information output from the separation unit 2120, and the analysis information decoding unit 2160 inputs the analysis information output from the separation unit 2130. The analysis information decoding units 2150 and 2160 decode the input analysis information and convert them into analysis parameters, and output the analysis parameters to the analysis parameter mixing unit 2151. The analysis parameter mixing unit 2151 mixes each analysis parameter output from the analysis information decoding units 2150 and 2160 for each frequency component, and outputs the mixed analysis parameter to the analysis information coding unit 2152. As a mixing method, all the input analysis parameters may be mixed, or the analysis parameters may be selected according to the importance and only the selected analysis parameters may be mixed. As another mixing method, a plurality of input analysis parameters may be multiplexed into one analysis parameter group. The analysis information coding unit 2152 encodes the mixed analysis parameters and outputs them as mixed analysis information.
Next, a second configuration example will be described with reference to FIGS. 8 and 9.
FIG. 8 shows a second configuration example of the multipoint connection device 2105. Although FIG. 8 shows an example of connecting three points, an arbitrary number of points can be connected. Compared with the first configuration example shown in FIG. 6, the configurations of the mixing units 2116, 2126, 2136 and the analysis information mixing units 2117, 2127, 2137 are different. Specifically, the mixing units 2116, 2126, and 2136 are different in that the mixing information generated when the mixing signal is generated is output to the analysis information mixing units 2117, 2127, and 2137. Correspondingly, the analysis information mixing units 2117, 2127, and 2137 mix a plurality of analysis information using the mixed information, and output the mixed analysis information to the multiplexing units 2115, 2125, and 2135.
Here, the mixed information may be used as a weighting coefficient when the mixed signal is created by weighting and adding the decoded signals of each terminal. For example, in a normal conversation, speakers of all terminals rarely speak at the same time, and speakers of some terminals often speak. In such a case, in the mixing units 2116, 2126, and 2136, the weighting coefficient for the decoded signal from the uttering terminal may be larger than the weighting coefficient for the decoded signal from the other terminal. More effectively, if the weighting coefficient for the decoded signal from the terminal being spoken is set to 1 and the other is set to 0, the amount of weighted addition processing in the mixing unit can be reduced. It can also be used to select the analysis parameter mixing process in the analysis information mixing unit, which will be described later.
FIG. 9 is a configuration example of the analysis information mixing units 2117, 2127, and 2137 of FIG. Since these terminals can have the same configuration, the analysis information mixing unit 2117 will be described below as an example.
Referring to FIG. 9, the analysis information mixing unit 2117 includes analysis information decoding units 2150 and 2160, analysis parameter mixing unit 2153, and analysis information coding unit 2152. Compared with the analysis information mixing unit 2114 in FIG. 7, the analysis parameter mixing unit 2151 is replaced with the analysis parameter mixing unit 2153, and the mixing information is input to the analysis parameter mixing unit 2153. Hereinafter, the analysis parameter mixing unit 2153 will be described.
The analysis parameter mixing unit 2153 mixes the analysis parameters output from the analysis information decoding units 2150 and 2160 for each frequency component using the input mixing information, and outputs the analysis information coding unit 2152. As a mixing method, all the input analysis parameters may be mixed, or the analysis parameters may be selected according to the importance and only the selected analysis parameters may be mixed. As another mixing method, a plurality of input analysis parameters may be multiplexed into one analysis parameter group.
As described above, according to the first embodiment of the present invention, the receiving unit corresponds to each sound source of the input signal at each point based on the analysis information in which the analysis information at each point is mixed. It can be controlled for each component.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information. Further, since the plurality of input signals are mixed in the multipoint connection device and the analysis information of the plurality of input signals is mixed, the transmission amount can be reduced. Further, since the mixed analysis information corresponding to the mixed input signals is generated in the multipoint connection device, it is not necessary to generate the mixed analysis information in the receiving unit, and the calculation amount related to the calculation of the analysis information in the receiving unit is further increased. It can be reduced.
A second embodiment of the present invention will be described. In the present embodiment, as a sound source, an input signal in which desired sound (hereinafter, target sound) and noise (hereinafter, background sound) are mixed is targeted, and the target sound and background sound are controlled based on analysis information. It is characterized by.
The configuration of this embodiment is shown in FIG. With reference to FIG. 1, since the terminals 2100, 2101, and 2102 can have the same configuration, only the terminal 2100 is shown as a configuration example. Hereinafter, the terminal will be described by taking the terminal 2100 as an example. In this embodiment, the configuration of the signal analysis unit 101 and the signal control unit 151 in the terminal 2100 and the configuration of the multipoint connection device 2105 are different from those of the first embodiment. Hereinafter, the details of the signal analysis unit 101, the signal control unit 151, and the multipoint connection device 2105 in the terminal 2100 will be described for each embodiment.
In the first embodiment of the present embodiment, the analytical information will be described as a suppression coefficient (described as a coded suppression coefficient. Hereinafter, it is assumed that the analytical information in all the embodiments is also encoded in the same manner. ). The signal analysis unit 101 calculates the suppression coefficient as the analysis information and outputs the analysis information. Correspondingly, the multipoint connection device 2105 mixes and outputs the analysis information output from each terminal, and the signal control unit 151 uses the analysis information output from the multipoint connection device 2105 to obtain a decoding signal. To control.
First, the signal analysis unit 101 and the signal control unit 151 in the terminal 2100 will be described.
Referring to FIG. 4, the signal analysis unit 101 includes a conversion unit 120 and an analysis information calculation unit 121. Since the configuration of the analyzer information calculation unit 121 is different from that of the first embodiment, the analyzer information calculation unit 121 will be described below.
A configuration example of the analysis information calculation unit 121 will be described in detail with reference to FIG. The analysis information calculation unit 121 receives the second conversion signal and outputs the suppression coefficient as the analysis information. The analysis information calculation unit 121 includes a background sound estimation unit 200 and a background sound information generation unit 202. The background sound estimation unit 200 receives the second conversion signal, estimates the background sound, and generates the background sound information. The background sound estimation unit 200 outputs background sound information to the background sound information generation unit 202. The background sound information includes the absolute amplitude value and energy of the background sound, the amplitude ratio and energy ratio of the background sound and the input signal, and the average value thereof. The background sound information generation unit 202 receives the second conversion signal and the background sound information. The background sound information generation unit 202 calculates the suppression coefficient based on the second conversion signal and the background sound information. Then, the background sound information generation unit 202 outputs the suppression coefficient or the encoded suppression coefficient as analysis information.
A configuration example of the background sound information generation unit 202 will be described in detail with reference to FIG. The background sound information generation unit 202 receives the second conversion signal and the background sound information, and outputs the suppression coefficient as the analysis information. The background sound information generation unit 202 includes a suppression coefficient calculation unit 201 and a suppression coefficient coding unit 2021. The suppression coefficient calculation unit 201 calculates an appropriate suppression coefficient for suppressing the background sound by using the second conversion signal and the background sound information. Then, the suppression coefficient calculation unit 201 outputs the suppression coefficient to the suppression coefficient coding unit 2021. As a technique related to the calculation method of the suppression coefficient, Non-Patent Document 6 (December 1984 , IEEE Transactions on Acoustics Speech and Signal Processing, Vol. 32, No. 6, (IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL.32, NO. 6, PP. 1109-1121, Dec. 1984) Minimum Mean Squared Error Short-term Spectral Amplitude Based Method (MMSE STSA), Non-Patent Document 7 (April 1985, IEEE Transactions on Acoustics) Speech and Signal Processing, Vol. 33, No. 2, (IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL.33, NO. 2, PP. 443-445, Apr. 1985) pp. 443-445), Method Based on Minimum Mean Squared Error Logarithmic Spectral Amplitude (MMSE LSA), Non-Patent Document 8 (July 2005, Euraship Journal on Applied Signal Processing, Vol. 2005, No. No. 7, (EURASIP JOURNAL ON ADVANCES IN SIGNAL PROCESSING, VOLUME 2005, Issue 7, JUL, 2005, pp.1110-1126.) Pages 1110 to 1126) Etc. are disclosed.
The suppression coefficient coding unit 2021 receives the suppression coefficient and encodes it. The suppression coefficient coding unit 2021 outputs the encoded suppression coefficient as analysis information. The suppression coefficient coding unit 2021 performs quantization such as linear quantization and nonlinear quantization, and outputs a suppression coefficient encoded by Huffman coding or the like. As a result, the redundancy of the suppression coefficient can be removed. Further, when it is not necessary to reduce the amount of information, the suppression coefficient coding unit 2021 may output the suppression coefficient as analysis information without performing these coding processes.
Next, a configuration example of the signal processing unit 172 will be described in detail with reference to FIG. The signal processing unit 172 receives the second conversion signal and the analysis information, and outputs a correction / decoding signal. The signal processing unit 172 is composed of a suppression coefficient reconstruction unit 250 and a multiplier 251. The second conversion signal is input to the multiplier 251 and the analysis information is input to the suppression coefficient reconstruction unit 250. The suppression coefficient reconstruction unit 250 reconstructs the suppression coefficient using the input analysis information, and outputs the suppression coefficient to the multiplier 251. The multiplier 251 multiplies the second conversion signal by the suppression coefficient to generate a modified decoding signal. The multiplier 251 outputs the modified decoding signal to the inverse conversion unit 173.
A configuration example of the suppression coefficient reconstruction unit 250 will be described in detail with reference to FIG. The suppression coefficient reconstruction unit 250 receives the suppression coefficient encoded as the analysis information and outputs the suppression coefficient. The suppression coefficient reconstruction unit 250 includes a suppression coefficient decoding unit 260. The suppression coefficient decoding unit 260 decodes the received suppression coefficient. When the suppression coefficient is not encoded, the suppression coefficient decoding unit 260 does not perform the decoding operation and outputs the suppression coefficient.
Next, the first configuration example and the second configuration example of the multipoint connection device 2105 will be described.
A first configuration example is shown in FIG. The configuration of the analysis information mixing units 2114, 2124, and 2134 is different from that of the first embodiment. Hereinafter, the analysis information mixing units 2114, 2124, and 2134 in this embodiment will be described with reference to FIGS. 7 and 14. Since these can have the same configuration, the analysis information mixing unit 2114 will be described as an example.
Referring to FIG. 7, the analysis information mixing unit 2114 is composed of analysis information decoding units 2150 and 2160, analysis parameter mixing unit 2151, and analysis information coding unit 2152. The analysis information decoding unit 2150 inputs the analysis information output from the separation unit 2120, and the analysis information decoding unit 2160 inputs the analysis information output from the separation unit 2130. The analysis information decoding units 2150 and 2160 decode the analysis information, calculate the suppression coefficient, and output the suppression coefficient to the analysis parameter mixing unit 2151, respectively. The analysis parameter mixing unit 2151 mixes the suppression coefficients output from the analysis information decoding units 2150 and 2160 for each frequency component, and outputs the analysis information coding unit 2152. The analysis information coding unit 2152 encodes the mixed suppression coefficients to generate analysis information, and outputs the analysis information.
Referring to FIG. 14, the analysis parameter mixing unit 2151 is composed of a selection unit 2202 and a suppression coefficient mixing unit 2203.
The selection unit 2202 selects a predetermined suppression coefficient from the suppression coefficients output from the analysis information decoding units 2150 and 2160, and outputs the suppression coefficient to the suppression coefficient mixing unit 2203. Examples of the selection method include a method of selecting only the suppression coefficient output from the terminal in which the energy of the decoding signal is equal to or higher than the threshold value. Here, the information representing the energy of the input signal may be multiplexed in advance in the signal analysis unit 101 with the analysis information and used instead of the energy of the decoded signal. As another method, all the suppression coefficients may be output to the suppression coefficient mixing unit 2203 without making a selection.
The suppression coefficient mixing unit 2203 mixes the suppression coefficients output from the selection unit 2202. As a mixing method, for example, mixing can be performed according to the energy ratio. When the number of suppression coefficients output from the selection unit 2202 is L', the suppression coefficient g after mixing can be calculated by the following formula.
[Number 1]<img id="000002" he="52" wi="63" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" /> Where E<sub>i</sub>Is the suppression coefficient g<sub>i</sub>Represents the energy of the decoded signal corresponding to. Here, the information representing the energy of the input signal may be multiplexed in advance in the signal analysis unit 101 with the analysis information and used instead of the energy of the decoded signal. Alternatively, all suppression coefficients may be multiplexed and output.
Since the analysis information coding unit 2152 has the same configuration as the suppression coefficient coding unit 2021 in FIG. 11, description thereof will be omitted.
Next, a second configuration example will be described. Figure 8 shows a second configuration example. Compared with the first configuration example shown in FIG. 6, the configurations of the mixing units 2116, 2126, 2136 and the analysis information mixing units 2117, 2127, 2137 are different. Specifically, the mixing units 2116, 2126, and 2136 are different in that the mixing information generated when the mixing signal is generated is output to the analysis information mixing units 2117, 2127, and 2137. Therefore, the analysis information mixing units 2117, 2127, and 2137 will be described below with reference to FIGS. 9 and 15.
FIG. 9 shows a configuration example of the analysis information mixing units 2117, 2127, and 2137. Since these can have the same configuration, the analysis information mixing unit 2117 will be described below as an example.
Referring to FIG. 9, the analysis information mixing unit 2117 is composed of the analysis information decoding units 2150 and 2160, the analysis parameter mixing unit 2153, and the analysis information coding unit 2152. Compared with the analysis information mixing unit 2114 in FIG. 7, the analysis parameter mixing unit 2151 is replaced with the analysis parameter mixing unit 2153, and the mixing information is input to the analysis parameter mixing unit 2153. Therefore, the analysis parameter mixing unit 2153 will be described below.
FIG. 15 shows a configuration example of the analysis parameter mixing unit 2153. The analysis parameter mixing unit 2153 is composed of a selection unit 2232 and a suppression coefficient mixing unit 2233.
The selection unit 2232 uses the mixing information output from the mixing unit 2116 to select a predetermined suppression coefficient from the suppression coefficients output from the analysis information decoding units 2150 and 2160, and outputs the suppression coefficient to the suppression coefficient mixing unit 2233. To do. As a selection method, for example, when the mixed information is a weighting coefficient for the decoded signal of each terminal, a method of selecting the suppression coefficient of the terminal whose weighting coefficient is equal to or larger than the threshold value can be mentioned. Alternatively, all suppression coefficients may be output to the suppression coefficient mixing section 2203 without making a selection.
The suppression coefficient mixing unit 2233 uses the mixing information output from the mixing unit 2116 to mix the suppression coefficients output from the selection unit 2232. As a mixing method, for example, mixing can be performed according to the energy ratio. When the number of suppression coefficients input to the selection unit is L', and the suppression coefficient after mixing is g, the suppression coefficient g after mixing can be calculated by the following equation.
[Number 2]<img id="000003" he="59" wi="75" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" /> Where E<sub>i</sub>Is the suppression coefficient g<sub>i</sub>Represents the energy of the decoded signal corresponding to. Information representing the energy of the input signal may be multiplexed in advance in the analysis information in the signal analysis unit 101 and used instead of the energy of the decoded signal. m<sub>i</sub>Is the suppression factor g<sub>i</sub>Represents the mixing information output from the mixing unit corresponding to. As another mixing method, all suppression coefficients may be multiplexed and output.
The second embodiment in the present embodiment is a case where the analysis information is a signal-to-background sound ratio. In the second embodiment, referring to FIG. 1, the signal analysis unit 101 outputs the signal-to-background sound ratio, which is the ratio of the target sound and the background sound, as analysis information. Correspondingly, the multipoint connection device 2105 mixes the signal to background sound ratio output from each terminal, and the signal control unit 151 uses the signal to background sound ratio output from the multipoint connection device 2105. To control the decoding signal. The configuration of the signal analysis unit 101, the signal control unit 151, and the multipoint connection device 2105 in the terminal 2100 is different from that of the first embodiment.
First, the terminal 2100 will be described.
First, the signal analysis unit 101 will be described. The signal analysis unit 101 is represented by FIG. 4, as in the first embodiment. Comparing this embodiment with the first embodiment, the configuration of the background sound information generation unit 202 included in the analysis information calculation unit 121 shown in FIG. 10 is different.
The background sound generation unit 202 of this embodiment will be described in detail with reference to FIG. The background sound information generation unit 202 receives the second conversion signal and the background sound information, and outputs the signal-to-background sound ratio encoded as the analysis information. The background sound information generation unit 202 includes a suppression coefficient calculation unit 201, a signal-to-background sound ratio calculation unit 203, and a signal-to-background sound ratio coding unit 204. The suppression coefficient calculation unit 201 calculates an appropriate suppression coefficient for suppressing the background sound by using the second conversion signal and the background sound information. Then, the suppression coefficient calculation unit 201 outputs the suppression coefficient to the signal-to-background sound ratio calculation unit 203. As the calculation method of the suppression coefficient, the calculation method of the suppression coefficient calculation unit 201 of the first embodiment shown in FIG. 11 can be used. The signal-to-background sound ratio calculation unit 203 calculates the signal-to-background sound ratio R using the input suppression coefficient G. If the input signal is X, the target sound is S, and the background sound is N, the following relationship holds.
[Number 3]<img id="000004" he="12" wi="39" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />
[Number 4]<img id="000005" he="12" wi="39" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />
[Number 5]<img id="000006" he="24" wi="32" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" /> R by this definition is known as the signal-to-noise ratio (pre-SNR) when the background sound is noise.
Substituting the equations [Equation 3] and [Equation 4] into [Equation 5]
[Number 6]<img id="000007" he="28" wi="82" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />To get. The signal-to-background sound ratio calculation unit 203 outputs the calculated signal-to-background sound ratio R to the signal-to-background sound ratio coding unit 204. The signal-to-background sound ratio coding unit 204 encodes the input signal-to-background sound ratio R. The signal-to-background sound ratio coding unit 204 outputs the encoded signal-to-background sound ratio R as analysis information. As for the details of the coding process, the same coding process as the coding process in the suppression coefficient coding unit 2021 can be used. As a result, the redundancy of the signal-to-background sound ratio R can be removed. Further, when it is not necessary to reduce the amount of information, the signal-to-background sound ratio coding unit 204 outputs the signal-to-background sound ratio as analysis information without performing the coding process of the signal-to-background sound ratio R. You may.
Next, the signal control unit 151 of this embodiment will be described in detail. The signal control unit 151 is represented by FIG. 5, as in the first embodiment. The configuration of the suppression coefficient reconstruction unit 250 included in the signal processing unit 172 shown in FIG. 12 is different between this embodiment and the first embodiment.
A configuration example of the suppression coefficient reconstruction unit 250 will be described in detail with reference to FIG. The suppression coefficient reconstruction unit 250 receives the encoded signal to background sound ratio R as an analysis signal, and outputs the suppression coefficient G. The suppression coefficient reconstruction unit 250 includes a signal-to-background sound ratio decoding unit 261 and a suppression coefficient conversion unit 262. The signal-to-background sound ratio decoding unit 261 decodes the received encoded signal-to-background sound ratio R and outputs the signal-to-background sound ratio R to the suppression coefficient conversion unit 262. When the signal-to-background sound ratio R is not encoded, the signal-to-background sound ratio decoding unit 261 does not perform the decoding operation and outputs the signal-to-background sound ratio. The suppression coefficient conversion unit 262 converts the signal-to-background sound ratio R into the suppression coefficient G. Then, the suppression coefficient conversion unit 262 outputs the suppression coefficient G. The conversion from R to G is based on [Equation 6]. Solving [Equation 6] for G
[Number 7]<img id="000008" he="28" wi="43" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />To get. The background sound is suppressed by multiplying the decoded signal by G with the multiplier 251.
Further, with reference to FIG. 18, another configuration example of the background sound information generation unit 202 will be described in detail. Comparing the background sound information generation unit 202 shown in FIG. 16, the difference is that the background sound information generation unit 202 of this configuration example does not include the suppression coefficient calculation unit 201. In the configuration of the background sound information generation unit 202 shown in FIG. 18, [Equation 8] is used instead of [Equation 5] as the definition of the signal-to-background sound ratio R. R by this definition is known as the posterior signal-to-noise ratio (posterior signal-to-noise ratio) when the background sound is noisy.
[Number 8]<img id="000009" he="28" wi="30" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" /> That is, in this configuration example, when the background sound is noise, the post-SNR is used as the analysis information instead of the pre-SNR. R of [Equation 8] does not require the suppression coefficient G and is calculated from the input signal and the background sound. As a result, the signal-to-background sound ratio calculation unit 207 can calculate the signal-to-background sound ratio based on the second conversion signal and the background sound information. Then, the signal-to-background sound ratio calculation unit 207 outputs the signal-to-background sound ratio to the signal-to-background sound ratio coding unit 204. Since the operation of the signal-to-background sound ratio coding unit 204 is the same as the operation of the signal-to-background sound ratio coding unit 204 shown in FIG. 16, the description thereof will be omitted.
On the other hand, assuming that [Equation 3] and [Equation 4] are substituted into [Equation 8] and S and N are uncorrelated,
[Number 9]<img id="000010" he="28" wi="41" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />To get. The signal-to-background sound ratio calculation unit 203 may calculate the signal-to-background sound ratio R using [Equation 9].
In this configuration example, the suppression coefficient reconstruction unit 250 shown in FIG. 12 on the receiving side is represented by FIG. 17 in the same manner as in the above configuration example. The signal-to-background sound ratio decoding unit 261 decodes the received encoded signal-to-background sound ratio R and outputs the signal-to-background sound ratio R to the suppression coefficient conversion unit 262. The suppression coefficient conversion unit 262 converts the signal-to-background sound ratio R into the suppression coefficient G, and outputs the suppression coefficient G. The conversion from R to G is based on [Equation 10]. That is, when [Equation 9] is solved for G,
[Number 10]<img id="000011" he="28" wi="43" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />To get. The background sound is suppressed by multiplying the decoded signal by G with the multiplier 251.
Next, the first configuration example and the second configuration example of the multipoint connection device 2105 will be described.
A first configuration example is shown in FIG. The configurations of the analysis information mixing units 2114, 2124, and 2134 are different from those of the first embodiment. FIG. 7 shows a configuration example of the analysis information mixing units 2114, 2124, and 2134. Since these can have the same configuration, they will be described below with reference to FIGS. 7 and 19 by taking the analysis information mixing unit 2114 as an example.
Referring to FIG. 7, the analysis information mixing unit 2114 is composed of analysis information decoding units 2150 and 2160, analysis parameter mixing unit 2151, and analysis information coding unit 2152. The analysis information decoding unit 2150 inputs the analysis information output from the separation unit 2120, and the analysis information decoding unit 2160 inputs the analysis information output from the separation unit 2130. The analysis information decoding units 2150 and 2160 decode the analysis information, calculate the signal-to-background sound ratio, and output the signal-to-background sound ratio to the analysis parameter mixing unit 2151. The analysis parameter mixing unit 2151 mixes the signal-to-background sound ratio output from the analysis information decoding units 2150 and 2160 for each frequency component, and outputs the signal to the analysis information coding unit 2152. The analysis information coding unit 2152 is the same as the signal-to-background sound ratio coding unit 204 in FIG. 18, and has already been described.
FIG. 19 shows a configuration example of the analysis parameter mixing unit 2151. The analysis parameter mixing unit 2151 is composed of a suppression coefficient conversion unit 2204, 2214, a selection unit 2202, a suppression coefficient mixing unit 2203, and a suppression coefficient inverse conversion unit 2205. The suppression coefficient conversion unit 2204 inputs the signal to background sound ratio output from the analysis information decoding unit 2150, and the suppression coefficient conversion unit 2214 inputs the signal to background sound ratio output from the analysis information decoding unit 2160. .. The suppression coefficient conversion units 2204 and 2214 calculate the suppression coefficient from the signal-to-background sound ratio using [Equation 7] or [Equation 10]. The calculated suppression coefficient is output to the selection unit 2202. Since the selection unit 2202 and the suppression coefficient mixing unit 2203 operate in the same manner as the selection unit 2202 and the suppression coefficient mixing unit 2203 shown in FIG. 14 in the first embodiment, detailed description thereof will be omitted. The selection unit 2202 selects a predetermined suppression coefficient from the input suppression coefficients, and outputs the selected suppression coefficient to the suppression coefficient mixing unit 2203. The suppression coefficient mixing unit 2203 mixes the suppression coefficients output from the selection unit 2202 and outputs them to the suppression coefficient inverse conversion unit 2205. The suppression coefficient inverse conversion unit 2205 converts the suppression coefficient into a signal-to-background sound ratio using [Equation 6] or [Equation 9], and outputs a mixed signal-to-background sound ratio.
Next, a second configuration example will be described. Figure 8 shows a second configuration example. Compared with the first configuration example shown in FIG. 6, the configurations of the mixing units 2116, 2126, 2136 and the analysis information mixing units 2117, 2127, 2137 are different. Specifically, the mixing units 2116, 2126, and 2136 are different in that the mixing information generated when the mixing signal is generated is output to the analysis information mixing units 2117, 2127, and 2137. Therefore, the analysis information mixing units 2117, 2127, and 2137 will be described below with reference to FIGS. 9 and 20.
FIG. 9 shows a configuration example of the analysis information mixing units 2117, 2127, and 2137. Since these can have the same configuration, the analysis information mixing unit 2117 will be described below as an example.
Referring to FIG. 9, the analysis information mixing unit 2117 is composed of the analysis information decoding units 2150 and 2160, the analysis parameter mixing unit 2153, and the analysis information coding unit 2152. Compared with the analysis information mixing unit 2114 in FIG. 7, the analysis parameter mixing unit 2151 is replaced with the analysis parameter mixing unit 2153, and the mixing information is input to the analysis parameter mixing unit 2153. Therefore, the analysis parameter mixing unit 2153 will be described below.
Figure 20 shows a configuration example of the analysis parameter mixing section 2153. The analysis parameter mixing unit 2153 is composed of a suppression coefficient conversion unit 2204, 2214, a selection unit 2232, a suppression coefficient mixing unit 2233, and a suppression coefficient inverse conversion unit 2205. The analysis parameter mixing unit 2153 is different from the first configuration example shown in FIG. 19 in that the selection unit 2002 is replaced with the selection unit 2232 and the suppression coefficient mixing unit 2203 is replaced with the suppression coefficient mixing unit 2233.
Here, the selection unit 2232 and the suppression coefficient mixing unit 2233 are the same as the second configuration example of the multipoint connection device 2105 in the first embodiment of the present embodiment. Since the description has been made with reference to FIG. 15, the description will be omitted.
The third embodiment is the case where the analysis information is a background sound. Referring to FIG. 1, the signal analysis unit 101 calculates the background sound itself as analysis information by the signal analysis unit 101. Correspondingly, the multipoint connection device 2105 mixes the background sound output from each terminal, and the signal control unit 151 controls the decoding signal using the background sound output from the multipoint connection device 2105. .. The configuration of the signal analysis unit 101, the signal control unit 151, and the multipoint connection device 2105 in the terminal 2100 is different from that of the first embodiment.
First, the terminal 2100 will be described.
First, the signal analysis unit 101 will be described. The signal analysis unit 101 is represented by FIG. 4, as in the first embodiment. The configuration of the analysis information calculation unit 121 of this embodiment is different from the configuration of the analysis information calculation unit 121 of the first embodiment shown in FIG.
A configuration example of the analysis information calculation unit 121 of this embodiment will be described in detail with reference to FIG. 21. Compared with the configuration example of the analysis information calculation unit 121 of the first embodiment shown in FIG. 10, the background sound information generation unit 202 is composed of the background sound coding unit 205. The analysis information calculation unit 121 of this configuration example receives the second conversion signal and outputs the background sound encoded as the analysis information. The analysis information calculation unit 121 of this configuration example is composed of a background sound estimation unit 200 and a background sound coding unit 205. The background sound estimation unit 200 of this configuration example outputs the background sound itself to the background sound information generation unit 202. The background sound coding unit 205 encodes the input background sound and outputs it. As a result, the redundancy of the background sound can be removed. Further, when it is not necessary to reduce the amount of information, the background sound coding unit 205 may output the background sound as analysis information without performing the background sound coding process.
As the coding process, the same coding process as that of the suppression coefficient coding unit 2021 can be used.
Next, the signal control unit 151 will be described. The signal control unit 151 is represented by FIG. 5, as in the first embodiment. The configuration of the signal processing unit 172 is different from the configuration of the signal processing unit 172 of the first embodiment shown in FIG.
A configuration example of the signal processing unit 172 of this embodiment will be described in detail with reference to FIG. 22. Compared with the configuration example of the signal processing unit 172 of the first embodiment shown in FIG. 12, the suppression coefficient reconstruction unit 250 is composed of the suppression coefficient calculation unit 252. The signal processing unit 172 receives the second conversion signal and the background sound encoded as the analysis information, and outputs the correction / decoding signal. The signal processing unit 172 is composed of a suppression coefficient calculation unit 252 and a multiplier 251. The second conversion signal is input to the suppression coefficient calculation unit 252 and the multiplier 251, and the encoded background sound is input to the suppression coefficient calculation unit 252 as analysis information. The suppression coefficient calculation unit 252 calculates the suppression coefficient based on the background sound and the second conversion signal. Then, the suppression coefficient calculation unit 252 outputs the suppression coefficient to the multiplier 251. The multiplier 251 multiplies the second conversion signal by the suppression coefficient, and outputs the modified decoding signal to the inverse conversion unit 173.
Further, with reference to FIG. 23, the configuration of the suppression coefficient calculation unit 252 will be described in detail. The suppression coefficient calculation unit 252 is composed of a background sound decoding unit 263 and a suppression coefficient generation unit 264. The background sound decoding unit 263 receives the background sound encoded as the analysis information. Then, the background sound decoding unit 263 decodes the encoded background sound and outputs the background sound to the suppression coefficient generation unit 264. When the background sound is not encoded, the background sound decoding unit 263 does not perform the decoding operation and outputs the background sound. The suppression coefficient generator 264 receives the background sound and the second conversion signal. Then, the suppression coefficient generation unit 264 calculates an appropriate suppression coefficient for suppressing the background sound based on the background sound and the second conversion signal. The calculation of the suppression coefficient may be performed by the same calculation method as that of the suppression coefficient calculation unit 201 shown in FIG. The suppression coefficient generation unit 264 outputs the suppression coefficient. As a technique related to the calculation method of the suppression coefficient, there is a technique disclosed in the above-mentioned Non-Patent Document 6, Non-Patent Document 7, or Non-Patent Document 8.
Further, another configuration example of the signal processing unit 172 will be described in detail with reference to FIG. 24. The signal processing unit 172 receives the second converted signal and the encoded background sound, and outputs the signal from which the background sound has been removed as a modified decoding signal. The signal processing unit 172 of this configuration example is composed of a background sound decoding unit 263 and a subtractor 253. The second conversion signal is input to the subtractor 253, and the background sound encoded as analysis information is input to the background sound decoding unit 263. The background sound decoding unit 263 decodes the encoded background sound and outputs the background sound to the subtractor 253. When the analysis information is an unencoded background sound, the background sound decoding unit 263 is unnecessary. The subtractor 253 subtracts the background sound from the second conversion signal. Then, the subtractor 253 outputs the signal from which the background sound has been removed as a correction decoding signal. If the background sound is noise, this subtraction is known as spectral subtraction. Techniques related to spectrum subtraction are described in Non-Patent Document 9 (April 1979, IEEE Transactions on Acoustics Speech and Signal Processing, Vol. 27, No. 2, (IEEE TRANSACTIONS ON ACOUSTICS, It is disclosed in SPEECH, AND SIGNAL PROCESSING, VOL.27, NO. 2, PP. 113-120, April 1979) pages 113-120).
The subtractor 253 can also include additional functions in addition to the subtraction. For example, as an additional function, when the subtraction result becomes negative, it is corrected to zero or a minute positive value, a limiter function that sets the minimum value of the subtraction result to a positive value, or background sound information. There is a function to correct by multiplying a coefficient or adding a constant and then subtracting.
Next, the first configuration example and the second configuration example of the multipoint connection device 2105 will be described.
A first configuration example is shown in FIG. The configurations of the analysis information mixing units 2114, 2124, and 2134 are different from those of the first embodiment. FIG. 7 shows a configuration example of the analysis information mixing units 2114, 2124, and 2134. Since these can have the same configuration, the analysis information mixing unit 2114 will be described below with reference to FIGS. 7 and 25 as an example.
Referring to FIG. 7, the analysis information mixing unit 2114 is composed of analysis information decoding units 2150 and 2160, analysis parameter mixing unit 2151, and analysis information coding unit 2152. The analysis information decoding unit 2150 inputs the analysis information output from the separation unit 2120, and the analysis information decoding unit 2160 inputs the analysis information output from the separation unit 2130. The analysis information decoding units 2150 and 2160 decode the input analysis information, calculate the background sound, and output the background sound to the analysis parameter mixing unit 2151. The analysis parameter mixing unit 2151 mixes the background sounds output from the analysis information decoding units 2150 and 2160 for each frequency component, and outputs the background sound to the analysis information coding unit 2152. The analysis information coding unit 2152 is the same as the background sound coding unit 205 in FIG. 21, and has already been described.
FIG. 25 shows a configuration example of the analysis parameter mixing unit 2151. The analysis parameter mixing unit 2151 is composed of a selection unit 2200 and a background sound mixing unit 2201.
The selection unit 2200 selects a predetermined background sound from the background sounds output from the analysis information decoding units 2150 and 2160, and outputs the background sound to the background sound mixing unit 2201. As a selection method, for example, only those whose background sound is equal to or higher than the threshold value are selected. It is also possible to select only the background sound that deteriorates the audible sound quality. All background sounds may be output to the background sound mixing unit 2201 without making a selection.
The background sound mixing unit 2201 mixes the background sounds output from the selection unit 2200 and outputs the mixed background sounds. As the mixing method, for example, the sum of all the input background sounds can be used. Further, a compensation coefficient for compensating for the correlation may be calculated in consideration of the correlation of the background sound, and the background sound may be mixed using the compensation coefficient. Alternatively, all background sounds may be multiplexed and output.
Next, a second configuration example is shown in FIG. The configurations of the analysis information mixing units 2117, 2127, and 2137 are different from the first configuration example. FIG. 9 shows a configuration example of the analysis information mixing units 2117, 2127, and 2137. Since these can have the same configuration, the analysis information mixing unit 2117 will be described below as an example.
Next, a second configuration example will be described. Figure 8 shows a second configuration example. Compared with the first configuration example shown in FIG. 6, the configurations of the mixing units 2116, 2126, 2136 and the analysis information mixing units 2117, 2127, 2137 are different. Specifically, the mixing units 2116, 2126, and 2136 are different in that the mixing information generated when the mixing signal is generated is output to the analysis information mixing units 2117, 2127, and 2137. Therefore, the analysis information mixing units 2117, 2127, and 2137 will be described below.
FIG. 9 shows a configuration example of the analysis information mixing units 2117, 2127, and 2137. Since these can have the same configuration, the analysis information mixing unit 2117 will be described below as an example.
Referring to FIG. 9, the analysis information mixing unit 2117 is composed of the analysis information decoding units 2150 and 2160, the analysis parameter mixing unit 2153, and the analysis information coding unit 2152. Compared with the analysis information mixing unit 2114 in FIG. 7, the analysis parameter mixing unit 2151 is replaced with the analysis parameter mixing unit 2153, and the mixing information is input to the analysis parameter mixing unit 2153. Therefore, the analysis parameter mixing unit 2153 will be described below.
FIG. 26 shows a configuration example of the analysis parameter mixing unit 2153. The analysis parameter mixing unit 2153 is composed of a selection unit 2230 and a background sound mixing unit 2231. Compared with the first configuration example in FIG. 25, the selection unit 2000 is replaced with the selection unit 2230, and the background sound mixing unit 2201 is replaced with the background sound mixing unit 2231.
The selection unit 2230 uses the mixing information output from the mixing unit 2116 to select a predetermined background sound from the background sounds output from the analysis information decoding units 2150 and 2160, and outputs the background sound to the background sound mixing unit 2231. To do. As a selection method, for example, when the mixed information is a weighting coefficient for the decoded signal of each terminal, the background sound of the terminal whose background sound is weighted using the weighting coefficient is equal to or larger than the threshold value is selected. Alternatively, the background sound of the terminal whose weighting coefficient is equal to or greater than the threshold value can be selected. Further, only the background sound that deteriorates the audible sound quality may be selected. All background sounds may be output to the background sound mixing unit 2231 without making a selection.
The background sound mixing unit 2231 mixes the background sound supplied from the selection unit 2230 by using the mixing information output from the mixing unit 2116. As a method of mixing, for example, when the mixing information is a weighting coefficient for the decoded signal of each terminal, it can be expressed as a weighted addition of background sounds using the weighting coefficient. Further, a compensation coefficient for compensating for the correlation may be calculated in consideration of the correlation of the weighted background sound, and the weighted background sound may be corrected using the compensation coefficient before mixing. As another method, all the background sounds of the terminals constituting the mixed signal, which is the output signal of the mixing unit 2116, may be multiplexed and output.
This concludes the description of the third embodiment.
Further, in the present embodiment, when the input signal is composed of a plurality of channels, the transmission unit 10 may independently calculate the analysis information of the first to third embodiments described above for each channel. Further, the transmission unit 10 may calculate the sum of all channels of the input signal and calculate the analysis information common to all channels from the sum signal. Alternatively, the transmission unit 10 may divide the input signal into a plurality of groups, calculate the sum of the input signals of each group, and calculate the analysis information common to the groups from the sum signal. Correspondingly, the receiving unit 15 controls the decoded signal by using the analysis information corresponding to each channel.
Further, the analysis information described in the first to third embodiments may be calculated as common analysis information in a plurality of frequency bands. For example, the transmission unit 10 may divide the frequency band at equal intervals and calculate the analysis information for each of the divided frequency bands. Further, the transmission unit 10 may finely divide the low frequency band and roughly divide the high frequency band according to the human auditory characteristics, and calculate the analysis information in the divided units. As a result, the amount of analysis information can be reduced.
As described above, according to the second embodiment of the present invention, the receiving unit is composed of the target sound and the background sound of each point based on the analysis information in which the analysis information of each point is mixed. The input signal to be input can be controlled independently for each target sound and background sound at each point. For example, the amount of background sound at each point can be made the same, or the amount of background sound at the main venue can be adjusted, and the amount can be adjusted to suit one's taste at each point.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
Further, since the plurality of input signals are mixed in the multipoint connection device and the analysis information of the plurality of input signals is mixed, the transmission amount can be reduced. Further, since the mixed analysis information corresponding to the mixed input signals is generated in the multipoint connection device, it is not necessary to generate the mixed analysis information in the receiving unit, and the calculation amount related to the calculation of the analysis information in the receiving unit is further increased. It can be reduced.
Next, a third embodiment of the present invention will be described. The present embodiment is characterized in that, based on the analysis information and the signal control information, the receiving terminal controls each component corresponding to each sound source of the input signal at each point.
FIG. 27 is a block diagram showing a third embodiment of the present invention. Compared with FIG. 1 showing the first embodiment, terminals 2100, 2101 and 2102 are replaced by terminals 2300, 2301 and 2302, receiving unit 15 is replaced by receiving unit 35, and signal control unit 151 is replaced by signal control unit 350. The point is different. That is, when the receiving terminal controls each component corresponding to each sound source of the input signal at each point, the configuration of the signal control unit 350 that uses not only the analysis information but also the signal control information is different. Therefore, the signal control unit 350, which is a feature of this embodiment, will be described below.
A configuration example of the signal control unit 350 will be described in detail with reference to FIG. 28. The signal control unit 350 includes a conversion unit 171, a signal processing unit 360, and an inverse conversion unit 173. Compared with the first embodiment, the signal processing unit 172 included in the signal control unit 151 is replaced by the signal processing unit 360 in the present embodiment. The signal control unit 350 receives the analysis information and the signal control information, and outputs an output signal. The signal control unit 350 operates the decoded signal received from the decoding unit 150 for each component corresponding to each sound source, based on the signal control information and the analysis information. Further, the signal control unit 350 can operate the component group composed of a plurality of components as a unit instead of the component corresponding to each sound source. The signal processing unit 360 receives the second conversion signal from the conversion unit 171 and the signal control information. The signal processing unit 360 controls the components of the frequency component of the second conversion signal based on the analysis information and the signal control information, and generates a modified decoding signal. The signal processing unit 360 outputs the modified / decoded signal to the inverse conversion unit 173.
Further, specifically, the signal processing unit 360 derives analysis parameters for each frequency based on the analysis information. Then, the signal processing unit 360 decomposes the second converted signal into the components corresponding to the sound source based on the analysis parameters. Further, the signal processing unit 360 creates a modified decoding signal in which the relationship between the plurality of components is changed according to the parameter for each frequency based on the signal control information. The signal processing unit 360 outputs the modified / decoded signal to the inverse conversion unit 173. Further, the signal processing unit 360 may be decomposed into a component group composed of a plurality of components based on the analysis parameters.
The signal control information may be input from the outside by the user. For example, the signal control information input from the outside includes personal information such as user preferences registered in advance in the receiving unit, and the operating state of the receiving unit (including external environment information such as when the speaker is turned off). , The type and type of receiver, the usage status and remaining amount of power supply and battery, and the type and status of antenna (shape such as folded, orientation, etc.). Further, the signal control information may be automatically acquired in another format. The signal control information may be automatically acquired via a sensor installed inside or in the vicinity of the receiving unit. For example, the signal control information automatically acquired includes external noise amount, brightness, time zone, geographical position, temperature, synchronization information with video, barcode information through a camera, and the like.
As described above, according to the third embodiment of the present invention, the receiving unit corresponds to each sound source of the input signal at each point based on the analysis information in which the analysis information at each point is mixed. It can be controlled for each component. It is also possible to independently control only a specific sound source based on the signal control information.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
A fourth embodiment of the present invention will be described. The present embodiment is characterized in that an input signal in which a target sound and a background sound are mixed as a sound source is targeted, and the target sound and the background sound are controlled based on the analysis information and the signal control information.
The present embodiment will be described in detail with reference to FIG. 27. Comparing the present embodiment and the second embodiment, the signal control unit 151 included in the receiving unit 15 shown in FIG. 1 is composed of the signal control unit 350 included in the receiving unit 35 shown in FIG. 27. ing. Further, in the present embodiment, signal control information is input to the signal control unit 350. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted. Further, the configuration of the signal control unit 350 will be described with reference to FIG. 28. The signal control unit 350 includes a conversion unit 171, a signal processing unit 360, and an inverse conversion unit 173. Compared with the second embodiment, the signal processing unit 172 included in the signal control unit 151 shown in FIG. 5 is composed of the signal processing unit 360 in the present embodiment.
Subsequently, the first embodiment will be described. The first embodiment uses the suppression coefficient as analytical information.
A configuration example of the signal processing unit 360 will be described in detail with reference to FIG. 29. As compared with the second embodiment, in the signal processing unit 360, the suppression coefficient reconstruction unit 250 included in the signal processing unit 172 shown in FIG. 12 is replaced with the suppression coefficient reconstruction unit 450. The suppression coefficient reconstruction unit 450 receives signal control information from the outside. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted.
The signal processing unit 360 receives the second conversion signal, analysis information, and signal control information, and outputs a correction / decoding signal. The signal processing unit 360 includes a suppression coefficient reconstruction unit 450 and a multiplier 451. The second conversion signal is input to the multiplier 451 and the analysis information and the signal control information are input to the suppression coefficient reconstruction unit 450. The suppression coefficient reconstruction unit 450 generates a modified suppression coefficient using the input analysis information and signal control information. The modified suppression coefficient is a modification of the suppression coefficient received as analysis information using signal control information. The suppression coefficient reconstruction unit 450 outputs the modified suppression coefficient to the multiplier 451. The multiplier 451 multiplies the second conversion signal by the modified suppression factor to generate a modified decoding signal. The multiplier 451 outputs the modified decoding signal to the inverse conversion unit 173.
With reference to FIG. 30, the configuration of the suppression coefficient reconstruction unit 450 of the first embodiment will be described in detail. The suppression coefficient reconstruction unit 450 includes a suppression coefficient correction unit 460. The suppression coefficient reconstruction unit 250 shown in FIG. 14 of the second embodiment does not include the suppression coefficient correction unit 460. The suppression coefficient correction unit 460 corrects the suppression coefficient using signal control information input from the outside. This signal control information is the same as that already used in the third embodiment, and the description thereof will be omitted.
The suppression coefficient reconstruction unit 450 receives the suppression coefficient encoded as the analysis information and the signal control information, and outputs the modified suppression coefficient. The suppression coefficient reconstruction unit 450 includes a suppression coefficient decoding unit 260 and a suppression coefficient correction unit 460. The suppression coefficient decoding unit 260 decodes the received suppression coefficient. When the suppression coefficient is not encoded, the suppression coefficient decoding unit 260 does not perform the decoding operation and outputs the suppression coefficient to the suppression coefficient correction unit 460. The suppression coefficient correction unit 460 corrects the input suppression coefficient by using the signal control information input from the outside. The suppression coefficient correction unit 460 outputs the correction suppression coefficient.
A first configuration example of the suppression coefficient correction unit 460 will be described in detail with reference to FIG. 31. The suppression coefficient correction unit 460 receives the suppression coefficient and signal control information, and outputs the correction suppression coefficient. The suppression coefficient correction unit 460 of this configuration example is composed of a multiplier 470. The multiplier 470 calculates the product of the suppression coefficient and the signal control information and outputs the modified suppression coefficient. In this configuration example, the signal control information is input with a magnification with respect to the suppression coefficient. With such a configuration, the suppression coefficient can be controlled by simple signal control information.
A second configuration example of the suppression coefficient correction unit 460 will be described in detail with reference to FIG. 32. The suppression coefficient correction unit 460 receives the suppression coefficient and signal control information, and outputs the correction suppression coefficient. The suppression coefficient correction unit 460 of this configuration example is composed of a comparison unit 471. The comparison unit 471 compares the suppression coefficient and the signal control information, and outputs a signal according to the comparison result. For example, the comparison unit 471 outputs the larger value of the suppression coefficient and the signal control information when performing the maximum comparison. Further, the comparison unit 471 may perform a minimum comparison. In these cases, the maximum value or the minimum value of the suppression coefficient is input as the signal control information. With such a configuration, the range of the output signal can be defined in advance, and it is possible to prevent an unexpected signal from being output and impairing the sound quality.
A third configuration example of the suppression coefficient correction unit 460 will be described in detail with reference to FIG. 33. The third configuration example of the suppression coefficient correction unit 460 is a combination of the first configuration example and the second configuration example described above. The suppression coefficient correction unit 460 receives the suppression coefficient and signal control information, and outputs the correction suppression coefficient. The suppression coefficient correction unit 460 of this configuration example is composed of a multiplier 470, a comparison unit 471, a designated suppression coefficient control unit 472, and a switch 473. The designated suppression coefficient control unit 472 outputs signal control information to the multiplier 470, the comparison unit 471, or the switch 473. Here, the signal control information includes at least the magnification of the suppression coefficient used in the multiplier 470 and the maximum value or the minimum value of the suppression coefficient used in the comparison unit 471. Further, the signal control information may include control information for selection in the switch 473. When the designated suppression coefficient control unit 472 receives the magnification of the suppression coefficient as signal control information, the designated suppression coefficient control unit 472 outputs the magnification of the suppression coefficient to the multiplier 470. The multiplier 470 calculates the product of the suppression coefficient and the multiplication factor of the suppression coefficient, and outputs the modified suppression coefficient to the switch 473. When the designated suppression coefficient control unit 472 receives the maximum value or the minimum value of the suppression coefficient as signal control information, the designated suppression coefficient control unit 472 outputs the maximum value or the minimum value of the suppression coefficient to the comparison unit 471. The comparison unit 471 compares the suppression coefficient with the maximum value or the minimum value of the suppression coefficient, and outputs a signal corresponding to the comparison result to the switch 473 as a correction suppression coefficient. When the designated suppression coefficient control unit 472 receives the control information for selection, the designated suppression coefficient control unit 472 outputs the control information to the switch 47. When the control information is input from the designated suppression coefficient control unit 472, the switch 473 selects and outputs either the output of the multiplier 470 or the output of the comparison unit 471 according to the signal control information.
Next, the second embodiment will be described. In the second embodiment, the signal-to-background sound ratio, which is the composition ratio of the target sound and the background sound, is used as the analysis information. The signal processing unit 360 of the second embodiment is the same as the signal processing unit of the first embodiment shown in FIG. 29, but the configuration of the suppression coefficient reconstruction unit 450 is different.
A configuration example of the suppression coefficient reconstruction unit 450 of the second embodiment will be described in detail with reference to FIG. 34. Compared with the suppression coefficient reconstruction unit 250 of the second embodiment shown in FIG. 17, the suppression coefficient reconstruction unit 450 of this configuration example further includes a signal-to-background sound ratio correction unit 461.
The suppression coefficient reconstruction unit 450 receives the encoded signal-to-background sound ratio and the signal control information, and outputs the modified suppression coefficient. The suppression coefficient reconstruction unit 450 includes a signal-to-background sound ratio decoding unit 261, a signal-to-background sound ratio correction unit 461, and a suppression coefficient conversion unit 262. The signal-to-background sound ratio decoding unit 261 decodes the received encoded signal-to-background sound ratio and outputs the signal-to-background sound ratio to the signal-to-background sound ratio correction unit 461. When the signal-to-background sound ratio is not encoded, the signal-to-background sound ratio decoding unit 261 does not perform the decoding operation and outputs the signal-to-background sound ratio. The signal-to-background sound ratio correction unit 461 corrects the input signal-to-background sound ratio by using the signal control information received from the outside, and generates the corrected signal-to-background sound ratio. Regarding the correction of the signal-to-background sound ratio, the same correction method as that of the suppression coefficient correction unit 460 in the first embodiment may be applied. That is, the signal-to-background sound ratio may be modified by inputting the magnification of the signal-to-background sound ratio as the signal control information. Further, the signal-to-background sound ratio may be modified by inputting the maximum value or the minimum value of the signal-to-background sound ratio as the signal control information. Further, as signal control information, control information for selecting a signal-to-background sound ratio modified by the magnification of the signal-to-background sound ratio and a signal-to-background sound ratio modified by the maximum value or the minimum value of the signal-to-background sound ratio is provided. It may be modified by inputting. The signal-to-background sound ratio correction unit 461 outputs the correction signal-to-background sound ratio to the suppression coefficient conversion unit 262. The suppression coefficient conversion unit 262 converts the correction signal to background sound ratio into a suppression coefficient, and outputs the correction suppression coefficient. As a method of converting the signal-to-background sound ratio into the suppression coefficient, the same conversion method as that of the suppression coefficient conversion unit 262 shown in FIG. 11 may be used. In the second embodiment, the signal-to-background sound ratio is corrected by the signal control information, and then converted into the corrected signal-to-background sound ratio suppression coefficient. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted.
Further, a third embodiment will be described. Compared with the second embodiment described above, the third embodiment has a configuration in which the signal-to-background sound ratio is converted into a suppression coefficient, and then the suppression coefficient is corrected by signal control information.
The suppression coefficient reconstruction unit 450 of the third embodiment will be described in detail with reference to FIG. 35. Compared with the suppression coefficient reconstruction unit 250 of the second embodiment shown in FIG. 17, the suppression coefficient reconstruction unit 450 of this embodiment further includes the suppression coefficient correction unit 460.
The suppression coefficient reconstruction unit 450 receives the encoded signal-to-background sound ratio and the signal control information, and outputs the modified suppression coefficient. The suppression coefficient reconstruction unit 450 includes a signal-to-background sound ratio decoding unit 261, a suppression coefficient conversion unit 262, and a suppression coefficient correction unit 460. The signal-to-background sound ratio decoding unit 261 receives and decodes the encoded signal-to-background sound ratio. The signal-to-background sound ratio decoding unit 261 outputs the signal-to-background sound ratio to the suppression coefficient conversion unit 262. The suppression coefficient conversion unit 262 converts the decoded signal-to-background sound ratio into a suppression coefficient. The suppression coefficient conversion unit 262 outputs the suppression coefficient to the suppression coefficient correction unit 460. The suppression coefficient correction unit 460 corrects the suppression coefficient input from the background sound information conversion unit 262 by using the signal control information received from the outside. The suppression coefficient correction unit 460 outputs the correction suppression coefficient. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted. The configuration of the suppression coefficient correction unit 460 is the same as that of the suppression coefficient correction unit 460 of the first embodiment shown in FIG. 30, and the description thereof will be omitted.
Subsequently, a fourth embodiment will be described. The fourth embodiment is a configuration example in which the background sound itself is used as the analysis information. A first configuration example of the signal processing unit 360 of the fourth embodiment will be described in detail with reference to FIG. 36. The suppression coefficient calculation unit 252 of the signal processing unit 172 of the second embodiment shown in FIG. 22 is replaced by the suppression coefficient calculation unit 452 in the signal processing unit 360 of this embodiment. The suppression coefficient calculation unit 452 receives signal control information from the outside. The signal processing unit 360 receives the second conversion signal, the encoded background sound, and the signal control information, and outputs the correction / decoding signal. The signal processing unit 360 includes a suppression coefficient calculation unit 452 and a multiplier 251. The second conversion signal is input to the suppression coefficient calculation unit 452 and the multiplier 251, and the encoded background sound is input to the suppression coefficient calculation unit 452 as analysis information. The suppression coefficient calculation unit 452 calculates the correction suppression coefficient based on the encoded background sound, the second conversion signal, and the signal control information. Then, the suppression coefficient calculation unit 452 outputs the modified suppression coefficient to the multiplier 251. The multiplier 251 multiplies the second conversion signal by the suppression coefficient, and outputs the modified decoding signal to the inverse conversion unit 173. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted.
A configuration example of the suppression coefficient calculation unit 452 will be described in detail with reference to FIG. 37. The suppression coefficient calculation unit 452 receives the second conversion signal, the encoded background sound, and the signal control information, and outputs the suppression coefficient. The suppression coefficient calculation unit 452 is composed of a background sound decoding unit 263, a background sound correction unit 464, and a suppression coefficient generation unit 264.
The background sound decoding unit 263 receives the encoded background sound and decodes it. The background sound decoding unit 263 outputs the decoded background sound to the background sound correction unit 464. When the background sound is not encoded, the background sound decoding unit 263 does not perform the decoding operation and outputs the background sound to the background sound correction unit 464. The background sound correction unit 464 corrects the background sound by using the signal control information input from the outside. Regarding the correction of the background sound, the same correction method as that of the suppression coefficient correction unit 460 in the first embodiment may be applied. That is, the background sound may be corrected by inputting the magnification of the background sound as the signal control information. Further, the background sound may be modified by inputting the maximum value or the minimum value of the background sound as the signal control information. Further, it may be corrected by inputting control information for selecting the background sound corrected by the magnification of the background sound and the background sound corrected by the maximum value or the minimum value of the background sound as the signal control information. The background sound correction unit 464 suppresses the corrected background sound to the suppression coefficient generation unit 264. Output to. The suppression coefficient generator 264 calculates an appropriate suppression coefficient for suppressing the background sound using the second conversion signal and the modified background sound. The calculation of the suppression coefficient may be performed by the same calculation method as that of the suppression coefficient calculation unit 201 shown in FIG. The suppression coefficient generation unit 264 outputs the suppression coefficient. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted.
A second configuration example of the signal processing unit 360 will be described in detail with reference to FIG. 38. The signal processing unit 360 of this configuration example is composed of a background sound decoding unit 263, a background sound correction unit 464, and a subtractor 253. The signal processing unit 360 receives the second conversion signal, the encoded background sound, and the signal control information, and outputs a signal in which the background sound is controlled.
The second conversion signal is input to the subtractor 253. Further, the encoded background sound is input to the background sound decoding unit 263 as analysis information. The background sound decoding unit 263 decodes the input encoded background sound. Then, the background sound decoding unit 263 outputs the decoded background sound to the background sound correction unit 464. When the background sound is not encoded, the background sound decoding unit 263 does not perform the decoding operation and outputs the background sound. The background sound correction unit 464 corrects the background sound information using the signal control information and generates the corrected background sound. The background sound correction unit 464 outputs the corrected background sound to the subtractor 253. The subtractor 253 subtracts the corrected background sound from the second conversion signal and outputs the subtraction result.
In the fifth embodiment, contrary to the fourth embodiment, the background sound is decoded, the suppression coefficient is generated, and then the suppression coefficient is corrected by the signal control information. At this time, as shown in FIG. 39, the suppression coefficient calculation unit 452 is composed of a background sound decoding unit 263, a suppression coefficient generation unit 264, and a suppression coefficient correction unit 461. The background sound decoding unit 263 receives the analysis information as the encoded background sound, decodes the analysis information, and obtains the decoding analysis information. The suppression coefficient generation unit 264 generates a suppression coefficient using the decoding signal and the decoding analysis information (decoded background sound) output from the background sound decoding unit 263. The suppression coefficient correction unit 461 corrects the suppression coefficient received from the suppression coefficient generation unit 264 using signal control information received from the outside, and outputs this. The signal control information has already been described with respect to the third embodiment, and the description thereof will be omitted. Next, a fifth embodiment will be described. Compared with the fourth embodiment, this embodiment has a configuration in which the suppression coefficient is generated from the decoded background sound and then the suppression coefficient is corrected by the signal control information.
The suppression coefficient calculation unit 452 will be described in detail with reference to FIG. 39. The suppression coefficient calculation unit 452 receives the second conversion signal, the encoded background sound, and the signal control information, and outputs the modified suppression coefficient. The suppression coefficient calculation unit 452 is composed of a background sound decoding unit 263, a suppression coefficient generation unit 264, and a suppression coefficient correction unit 460.
The background sound decoding unit 263 receives the encoded background sound and decodes it. Then, the background sound decoding unit 263 outputs the decoded background sound to the suppression coefficient generation unit 264. The suppression coefficient generation unit 264 generates a suppression coefficient from the second conversion signal and the decoded background sound. For the calculation of the suppression coefficient, the same calculation method as that of the suppression coefficient calculation unit 201 shown in FIG. 27 may be used. Then, the suppression coefficient generation unit 264 outputs the suppression coefficient to the suppression coefficient correction unit 460. The suppression coefficient correction unit 460 corrects the suppression coefficient using the received signal control information and generates a correction suppression count. Regarding the correction of the suppression coefficient, the same correction method as that of the suppression coefficient correction unit 460 shown in FIG. 33 may be applied. That is, it may be corrected by inputting the magnification of the suppression coefficient as the signal control information. Further, it may be corrected by inputting the maximum value or the minimum value of the suppression coefficient as the signal control information. Further, it may be modified by inputting the control information for selecting the magnification of the suppression coefficient and the maximum value or the minimum value of the suppression coefficient as the signal control information. The suppression coefficient correction unit 460 outputs the correction suppression coefficient. The signal control information is the same as that used in the third embodiment, and the description thereof will be omitted.
As described above, according to the fourth embodiment of the present invention, the receiving unit is composed of the target sound and the background sound of each point based on the analysis information in which the analysis information of each point is mixed. The input signal to be input can be controlled independently for each target sound and background sound at each point. It is also possible to independently control only a specific sound source based on the signal control information.
Further, since the transmitting unit calculates the analysis information such as the suppression coefficient or the signal-to-background sound ratio, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
A fifth embodiment of the present invention will be described. The present embodiment is characterized in that the receiving terminal controls each component corresponding to each sound source of the input signal at each point based on the analysis information, the object information, the signal control information, and the component rendering information. ..
Referring to FIG. 40, the multipoint connection system according to the present embodiment is a multipoint connection device (MCU) that controls data exchange between conference terminals 2500, 2501 and 2502 and conference terminals distributed at each point. : Multipoint Control Unit) 2505 is provided. The multipoint connection device 2505 mixes the transmission signals output from each terminal and distributes the same mixed signal to each terminal. Although FIG. 40 shows an example of connecting three points, an arbitrary number of points can be connected.
Similarly, with reference to FIG. 40, a configuration example of terminals 2500, 2501, and 2502 will be described. Since these terminals can have the same configuration, only the terminal 2500 is shown as a configuration example. Hereinafter, the terminal 2500 will be described as an example.
The terminal 2500 is transmitted from a transmission unit 2506 including an encoding unit 100, a signal analysis unit 101, an object information extraction unit 2510, and a multiplexing unit 2511, and a receiving unit 2507 including a decoding unit 150, an output signal generation unit 2550, and a separation unit 2551. It is composed.
The input signal is input to the coding unit 100, the signal analysis unit 101, and the object information extraction unit 2510 in the transmission unit 2506. The coding unit 100 encodes the input signal and outputs the coded signal to the multiplexing unit 2511. The signal analysis unit 101 calculates the analysis information of the components corresponding to each sound source included in the input signal, and outputs the analysis information to the multiplexing unit 2511. The object information extraction unit 2510 catches the input signal as one object signal, performs frequency conversion of the object signal, extracts object parameters representing the characteristics of the object signal for each frequency component, and collects these object parameters into object information. Is output to the multiplexing unit 2511. The multiplexing unit 2511 multiplexes the coded signal output from the coding unit 100, the analysis information output from the signal analysis unit 101, and the object information output from the object information extraction unit 2510, and converts it into a transmission line as a transmission signal. Output. The detailed operations of the coding unit 100, the signal analysis unit 101, and the decoding unit 150 have already been described in the first embodiment.
The separation unit 2551 in the reception unit 2507 separates the transmission signal input from the transmission line into a coded signal, analysis information, and object information, the coded signal is sent to the decoding unit 150, and the analysis information and object information is generated as an output signal. Output to unit 2550. The decoding unit 150 decodes the coded signal to generate a decoded signal, and outputs the decoded signal to the output signal generation unit 2550. Here, the decoded signal is composed of a general plurality of sound sources. The output signal generation unit 2550 decodes output from the decoding unit 150 based on the analysis information and object information output from the separation unit 2551 and the signal control information and component rendering information input via the input terminal. The signal is operated for each component corresponding to each sound source, and the output signal is output. The output signal generation unit 2550 can be operated in units of component groups composed of a plurality of components instead of the components corresponding to each sound source.
Here, the component rendering information is information representing the relationship between the component included in the decoded signal and the output signal of the receiving unit for each frequency component. For example, in the acoustic scene reproduced by the output signal, information representing the sound image localization of each component mixed in the decoded signal may be used.
FIG. 41 shows a configuration example of the multipoint connection device 2505. Although FIG. 41 shows an example of connecting three points, an arbitrary number of points can be connected. In FIG. 41, the transmission signals received from the terminals installed at the first to third points are input to the separation units 2510, 2520, and 2530 via the input terminals.
Separation units 2510, 2520, and 2530 separate transmission signals into coded signals, analysis information, and object information, respectively, with the coded signals in the decoding units 2511, 2521, and 2531, and the analysis information in the analysis information mixing unit 2516. The information is output to the object information mixing unit 2515. The decoding units 2511, 2521, and 2531 decode the encoded signal to generate a decoded signal, and output the decoded signal to the mixing unit 2512. The mixing unit 2512 mixes all the decoded signals from each point to generate a mixed signal, and outputs the mixed signal to the coding unit 2513. The coding unit 2513 encodes the mixed signal and outputs the encoded mixed signal to the multiplexing unit 2514. The analysis information mixing unit 2516 mixes all the analysis information from each point and outputs the mixed analysis information to the multiplexing unit 2514. The object information mixing unit 2515 mixes all the object information from each point and outputs the mixed object information to the multiplexing unit 2514.
Here, the object information is composed of object parameters representing each frequency component of the decoded signal, and the object parameters may be mixed by mixing all the input object parameters or selecting the object parameters according to the importance. , Only selected object parameters may be mixed. As another mixing method, a plurality of input object parameters may be multiplexed into one object parameter group. The object information output from the object information mixing unit 2515 represents the relationship between the mixed signal generated by the mixing unit 2512 and each decoding signal (hereinafter referred to as an object signal) that is an input thereof for each frequency component. Become.
The multiplexing unit 2514 includes the encoded mixed signal output from the coding unit 2513, the mixed analysis information output from the analysis information mixing unit 2516, and the mixed object information output from the object information mixing unit 2515. , Are multiplexed and output as a transmission signal to the transmission line at each point. The detailed operations of the decoding units 2511, 2521, and 2531 are the same as those of the decoding unit 150 in the first embodiment, and the detailed operations of the coding unit 2513 are the same as those of the coding unit 100 in the first embodiment. is there. The detailed operation of the analysis information mixing unit 2516 is the same as that of the analysis information mixing unit 2114 shown in FIG. 6 in the first embodiment.
Hereinafter, a first configuration example and a second configuration example of the output signal generation unit 2550 will be described.
A first configuration example is shown in FIG. The output signal generation unit 2550 is composed of a signal control unit 2560, a rendering information generation unit 2561, and a rendering unit 2563.
The signal control unit 2560 inputs the decoding signal, the object information, and the analysis information. Object information and analysis information are decoded to generate object parameters and analysis parameters, respectively. Next, the object parameters are used to decompose the decoded signal into object signals (signals before mixing), and the analysis parameters are used to decompose the object signals into components. Further, after generating a modified component in which the component is modified based on the signal control information, a modified decoded signal (a mixed signal, that is, a signal obtained by modifying the decoded signal based on the signal control information) is output from the modified component. It is reconstructed and the modified decoding signal is output to the rendering unit 2563. Further, the signal control unit 2560 generates a correction parameter representing the relationship between the correction / decoding signal and the correction component for each frequency component, and outputs the correction parameter to the rendering information generation unit 2561.
As another operation example of the signal control unit 2560, the decoded signal may be converted into the modified decoded signal by using the object parameter, the analysis parameter, and the signal control information without generating the modified component. In that case, the correction parameter used when converting the decoding signal into the correction decoding signal is output to the rendering information generation unit 2561.
A specific example of the operation of the signal control unit 2560 will be described below.
X the frequency component of the decoded signal in a certain frequency band f<sub>k</sub>(f), k = 1,2, ..., P (P is the number of decoded signal channels), and the frequency component of the object signal is Z.<sub>ih</sub>(f), i = 1,2, ..., L, h = 1,2, ..., K (L is the number of objects, that is, the number of terminals, K is the number of object signal channels), i Y the frequency component of the component of the second object signal<sub>ij</sub>(f), j = 1,2, ..., M<sub>i</sub>(M<sub>i</sub>Is the number of components of the i-th object signal), the frequency component of the component modified based on the signal control information is Y'<sub>ij</sub>(f), Z'correct object signal<sub>ij</sub>(f), where the modified decoding signal is X'(f), the conversion function F specified by the object parameters<sub>500</sub>And the transformation function F specified by the analysis parameters<sub>501</sub>And the conversion function F specified by the signal control information<sub>502</sub>The following relationship is established using.
[Number 11]<img id="000012" he="16" wi="141" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />
[Number 12]<img id="000013" he="16" wi="149" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />
[Number 13]<img id="000014" he="16" wi="90" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />
[Number 14]<img id="000015" he="16" wi="89" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Where the conversion function F<sub>503</sub>Is a function that converts a modified component into a modified decoded signal, and the modified parameter is the conversion function F.<sub>503</sub>It is a parameter that represents the inverse function of.
As mentioned above as another example of operation, the function F<sub>500</sub>, F<sub>501</sub>, F<sub>502</sub>, F<sub>503</sub>Integrate
[Number 15]<img id="000016" he="16" wi="81" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />May be. At this time, the conversion function F<sub>504</sub>Is defined by object parameters, analysis parameters, signal control information, and correction parameters.
As a specific example of the above conversion, the object parameter C (f) of the frequency band f is set.
[Number 16]<img id="000017" he="57" wi="159" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />The analysis parameter B (f) is expressed as M, in which all matrix elements are 0.<sub>i</sub>Row-K column zero matrix<img id="000018" he="13" wi="17" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Using,
[Number 17]<img id="000019" he="28" wi="159" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />The signal control information A (f) is expressed as
[Number 18]<img id="000020" he="40" wi="117" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />When expressed as, [number 11] to [number 15] are
[Number 19]<img id="000021" he="71" wi="128" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Can be expressed as. That is, the matrix that converts the decoded signal into the modified decoded signal can be calculated as D (f) × A (f) × B (f) × C (f). Here, D (f) is an arbitrary P-by-M matrix, and if the correction parameter is E (f),
[Number 20]<img id="000022" he="14" wi="57" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. For example, if the inverse matrix of B (f) × C (f) is used as D (f), the correction parameter is E (f) = B (f) × C (f). As is clear from [Equation 19], using the inverse matrix of B (f) × C (f) as D (f) is appropriate as an operation for converting a modified component into a modified decoded signal.
The rendering information generation unit 2561 uses the correction parameters output from the signal control unit 2560 to convert the component rendering information input via the input terminal into rendering information, and outputs the rendering information to the rendering unit 2563.
As a specific example of converting component rendering information into rendering information, component rendering information U (f) and rendering information W (f) are used.
[Number 21]<img id="000023" he="29" wi="151" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Can be expressed as W (f) = U (f) × E (f). Here, Q is the number of channels of the output signal.
The rendering information is information that expresses the relationship between the modified decoding signal and the output signal of the output signal generation unit 2550 for each frequency component, and can be expressed by using the energy difference between the signals, the time difference, the correlation, and the like.
The rendering unit 2563 uses the rendering information output from the rendering information generation unit 2561 to convert the modified decoding signal output from the signal control unit 2560 to generate an output signal, which is used as the output signal of the output signal generation unit 2550. Output.
In the above, the configuration in which the correction-decoding signal decomposed into frequency components is output to the rendering unit 2563 in the signal control unit 2560 has been described. However, in the output of the signal control unit 2560, the correction-decoding signal is inversely converted and the time is When the signal is output to the rendering unit 2563, the rendering unit 2563 decomposes the time signal into frequency components before processing. The output of the rendering unit 2563 outputs a signal obtained by inversely converting the signal decomposed into frequency components as an output signal.
V the frequency component of the output signal<sub>k</sub>(f), k = 1,2, ..., Q (Q is the number of output signal channels)
[Number 22]<img id="000024" he="57" wi="59" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Then, the operation of the rendering unit is V (f) = W (f) × X'(f).
A specific example of the method of converting the modified decoding signal in the rendering unit 2563 into an output signal and the rendering information is described in Non-Patent Document 10 (2007, ISO / IC 23003-1: 2007 Part 1 EM PEG Surround, (ISO / IEC). 23003-1: 2007 Part 1 MPEG Surround)).
When the conversion method disclosed in Non-Patent Document 10 is used as the rendering unit 2563, the data stream format disclosed in Non-Patent Document 10 may be used as the rendering information input to the rendering unit 2563. Note that the rendering information may be input to the conversion processing unit disclosed in Non-Patent Document 10 without being converted into the data stream format.
Next, a second configuration example will be described. Referring to FIG. 43, the output signal generation unit 2550 is composed of a rendering information generation unit 2564, a component information conversion unit 2565, and a rendering unit 2563.
The component information conversion unit 2565 inputs analysis information, signal control information, and component rendering information. Decode the analysis information and generate the analysis parameters. Further, using the analysis parameters and the signal control information, the component rendering information is converted into the object rendering information, and the object rendering information is output to the rendering information generation unit 2564.
Here, the object rendering information is information expressing the relationship between the object signal and the output signal of the output signal generation unit 2550 for each frequency component. Specifically, if the object rendering information is T (f),
[Number 23]<img id="000025" he="14" wi="80" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. Here, B (f), A (f), and U (f) are as defined in [Equation 17], [Equation 18], and [Equation 21].
The rendering information generation unit 2564 inputs the object information and the object rendering information. The object information is decoded and the object parameter is generated, the rendering information is generated from the object parameter and the object rendering information, and the rendering information is output to the rendering unit 2563. In particular,
[Number 24]<img id="000026" he="14" wi="58" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. Here, C (f), W (f), and T (f) are as defined in [Equation 16], [Equation 21], and [Equation 23].
The rendering information is information that expresses the relationship between the decoded signal and the output signal of the output signal generation unit 2550 for each frequency component, and can be expressed by using the energy difference between the signals, the time difference, the correlation, and the like. An example of rendering information is disclosed in Non-Patent Document 10.
The rendering unit 2563 has already been described with reference to FIG. 42 in the first configuration example. In this case, the rendering operation is V (f) = W (f) × X (f).
As described above, according to the fifth embodiment of the present invention, the receiving unit corresponds to each sound source of the input signal at each point based on the analysis information in which the analysis information at each point is mixed. It can be controlled for each component. It is also possible to independently control only a specific sound source based on the signal control information. It is also possible to control the localization of individual sound sources based on the component rendering information.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
Further, in the present embodiment, the object signals corresponding to the respective input signals are mixed, and control is performed on the receiving side based on the mixed object signals. Therefore, the same signal can be output to each of the plurality of receiving units. As a result, in the multipoint connection device, the plurality of input signals can all be mixed, and the coding may be performed on one signal, so that it is not necessary to perform the coding a plurality of times. Therefore, it is possible to reduce the amount of calculation related to coding in the multipoint connection device.
A sixth embodiment of the present invention will be described. In this embodiment, an input signal in which a target sound and a background sound are mixed as a sound source is targeted, and the target sound and the background sound are controlled based on analysis information, object information, signal control information, and component rendering information. It is a feature.
Referring to FIG. 40, the multipoint connection system according to the present embodiment is a multipoint connection device (MCU) that controls data exchange between conference terminals 2500, 2501 and 2502 and conference terminals distributed at each point. : Multipoint Control Unit) 2505 is provided. The multipoint connection device 2505 mixes the signals output from each terminal to generate a mixed signal, and distributes the same mixed signal to each terminal. Although FIG. 40 shows an example of connecting three points, an arbitrary number of points can be connected. Since the terminals 2500, 2501 and 2502 can have the same configuration, only the terminal 2500 shows a configuration example. Therefore, the terminal will be described below by taking the terminal 2500 as an example.
First, FIG. 41 shows a configuration example of the multipoint connection device 2505. The present embodiment has the same configuration as the fifth embodiment, except that the operation of the analysis information mixing unit 2516 is different. The analysis information mixing unit 2516 in the present embodiment has the same operation as the analysis information mixing unit 2114 shown in FIG. 7 described in the second embodiment. Detailed description will be omitted.
Next, a configuration example of the terminal 2500 is shown in FIG. The present embodiment has the same configuration as the fifth embodiment, except that the operations of the signal analysis unit 101 and the output signal generation unit 2550 are different. Therefore, the signal analysis unit 101 and the output signal generation unit 2550 will be described in detail below.
The first embodiment in the present embodiment is the case where the analytical information is the suppression coefficient. With reference to FIG. 40, the signal analysis unit 101 calculates the suppression coefficient as analysis information. Correspondingly, the multipoint connection device 2505 mixes the suppression coefficients as described in the second embodiment, and the output signal generator 2550 combines the signal control information, the component rendering information, and the object information. Based on this, the decoding signal is controlled using the suppression coefficient. Since the configuration of the signal analysis unit 101 that calculates the suppression coefficient as analysis information is described in detail in the first embodiment of the second embodiment, the description thereof will be omitted. Hereinafter, the output signal generation unit 2550 will be described in detail.
The configuration of the output signal generation unit 2550 of FIG. 40, which controls the target sound and the background sound using the suppression coefficient, is shown in FIG. 43 as in the second configuration example of the output signal generation unit 2550 in the fifth embodiment. However, there are differences in the configuration of the component information conversion unit 2565. Therefore, the component information conversion unit 2565 will be described below.
FIG. 44 shows a configuration example of the component information conversion unit 2565. The component information conversion unit 2565 is composed of a component parameter generation unit 651 and an object rendering information generation unit 2611. The component parameter generation unit 651 calculates the suppression coefficient from the analysis information, calculates the component parameter using the calculated suppression coefficient and the signal control information, and outputs it to the object rendering information generation unit 2611.
As a specific example of the above conversion, the suppression coefficient of the object signal i in the frequency band f is g.<sub>ih</sub>(f), i = 1,2, ..., L, h = 1,2, ..., K (L is the number of objects, K is the number of object signal channels), and the number of components of the object signal i M<sub>i</sub>If = 2, the analysis parameter B (f) of [Equation 17] is
[Number 25]<img id="000027" he="30" wi="155" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />The signal control information A (f) of [Equation 18] is the signal control information for controlling the target sound of the object signal i.<sub>main</sub><sup>i</sup>(f), A signal control information for controlling the background sound<sub>sub</sub><sup>i</sup>If (f),
[Number 26]<img id="000028" he="48" wi="129" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Can be expressed as. At this time, the component parameter H (f) is
[Number 27]<img id="000029" he="14" wi="71" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Can be expressed as. In the multipoint connection device 2505, when the suppression coefficients are mixed, a common suppression coefficient may be used for the mixed object signals.
The object rendering information generation unit 2611 outputs object rendering information representing the relationship between the object signal and the output signal based on the component parameters and the component rendering information. As a specific example of the above conversion, the object rendering information T (f) can be T (f) = U (f) × H (f) by using U (f) of [Equation 21].
As another configuration example of the component information conversion unit 2566, the component parameter generation unit 651 and the object rendering information generation unit 2611 in FIG. 44 can be integrated. In this case, the analysis information is decoded to calculate the suppression coefficient, the object rendering information is calculated from the calculated suppression coefficient, the signal control information, and the component rendering information, and the object rendering information is output to the rendering information generation unit 2564. .. That is, the object rendering information T (f) can be T (f) = U (f) × A (f) × B (f).
The second embodiment in the present embodiment is a case where the analysis information is a signal-to-background sound ratio. Referring to FIG. 40, the signal analysis unit 101 outputs the signal-to-background sound ratio as analysis information. Correspondingly, the multipoint connection device 2505 mixes the signal-to-background tone ratio, and the output signal generator 2550 decodes it based on the signal-to-background tone ratio, object information, signal control information, and component rendering information. Control the signal. Since the signal analysis unit 101 when the signal-to-background sound ratio is used as the analysis information is described in the second embodiment of the second embodiment, the output signal generation unit 2550 will be described in detail below. ..
The configuration of the output signal generation unit 2550 in FIG. 40, which controls the target sound and the background sound using the signal-to-background sound ratio, is shown in FIGS. 43 and 44 as in the first embodiment. Comparing this embodiment with the first embodiment, the operation of the component parameter generation unit 651 in FIG. 44 is different. Therefore, the component parameter generation unit 651 will be described below.
The component parameter generation unit 651 decodes the analysis information, calculates the signal-to-background sound ratio, calculates the component parameter from the signal-to-background sound ratio based on the signal control information, and outputs it to the object rendering information generation unit 2611. .. For example, as described in the second embodiment, after converting the signal-to-background tone ratio into a suppression coefficient, as described in the first embodiment, [Equation 25], [Equation 26], [Equation]. 27] can be used to calculate the component parameters based on the suppression coefficient and signal control information. Further, as another method, as described in the fourth embodiment, the signal-to-background sound ratio is operated based on the signal control information, and the operated signal-to-background sound ratio is converted into a suppression coefficient. Element parameters may be calculated. In this case, the converted suppression coefficient of the object signal i is g'<sub>ih</sub>If (f), the analysis parameter B'(f) is
[Number 28]<img id="000030" he="27" wi="158" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />And the component parameter H (f) becomes B'(f).
As another configuration example of the component information conversion unit 2565 of FIG. 43, the component parameter generation unit 651 and the object rendering information generation unit 2611 in FIG. 44 can be integrated. In this case, the analysis information is decoded to calculate the signal-to-background sound ratio, the object rendering information is calculated from the calculated signal-to-background sound ratio, the signal control information, and the component rendering information, and the object rendering information is rendered information. Output to the generator 2564. As a specific example, for example, as described in the second embodiment, after the signal-to-background sound ratio is converted into the suppression coefficient, the suppression coefficient, the signal control information, and the configuration are described as described in the first embodiment. The object rendering information is calculated from the element rendering information, and the object rendering information is output to the rendering information generator 2564. That is, the object rendering information T (f) includes the component rendering information defined in [Equation 21], the analysis parameters that can be calculated from the suppression coefficient defined in [Equation 25], and the signal control information defined in [Equation 26]. Using, T (f) = It can be U (f) × A (f) × B (f). Further, as another method, as described in the fourth embodiment, the signal-to-background sound ratio is operated based on the signal control information, the operated signal-to-background sound ratio is converted into the suppression coefficient, and then the operation is performed. Object rendering information may be calculated from the converted suppression coefficient and component rendering information. In this case, the converted suppression coefficient of the object signal i is g'<sub>ih</sub>Assuming (f), the object rendering information T (f) is calculated using B'(f) of [Equation 28].
[Number 29]<img id="000031" he="14" wi="71" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. Here, U (f) is as defined in [Equation 21].
The third embodiment in the present embodiment is a case where the analysis information is a background sound. With reference to FIG. 40, the signal analysis unit 101 calculates the background sound as analysis information. Correspondingly, the multipoint connection device 2505 mixes the background sound, and the output signal generation unit 2550 controls the decoded signal based on the background sound, the object information, the signal control information, and the component rendering information. Since the signal analysis unit 101 when the background sound is used as the analysis information is described in the third embodiment in the second embodiment, the description thereof will be omitted. Therefore, the operation of the output signal generation unit 2550 will be described in detail below.
FIG. 45 shows a configuration example of the output signal generator 2550. The output signal generation unit 2550 shown in FIG. 45 is different from the output signal generation unit 2550 of the first embodiment shown in FIG. 43 in that the component information conversion unit 2565 is replaced with the component information conversion unit 2566. different. Hereinafter, the component information conversion unit 2566 will be described.
Referring to FIG. 45, the component information conversion unit 2566 inputs the decoded signal, the analysis information, the signal control information, and the component rendering information. Based on these, object rendering information representing the relationship between the object signal included in the decoding signal and the output signal for each frequency component is generated, and is output to the rendering information generation unit 2564.
FIG. 46 shows a configuration example of the component information conversion unit 2566. The component information conversion unit 2566 is composed of a conversion unit 171, a component parameter generation unit 653, and an object rendering information generation unit 2611. The conversion unit 171 takes the decoding signal as an input, decomposes the decoding signal into each frequency component to generate a second conversion signal, and outputs the second conversion signal to the component parameter generation unit 653.
The component parameter generation unit 653 inputs the second conversion signal, analysis information, and signal control information. The analysis information is decoded to calculate the background sound, the component parameters are calculated from the calculated background sound and the second conversion signal based on the signal control information, and the components are output to the object rendering information generation unit 2611.
Hereinafter, a specific example of the calculation method of the component parameters will be shown. In the first method, the suppression coefficient is calculated from the background sound and the second conversion signal as described in the third embodiment in the second embodiment. Furthermore, using [Equation 25], [Equation 26], and [Equation 27], the component parameters are calculated based on the suppression coefficient and the signal control information. In the second method, the suppression coefficient is calculated from the background sound, the signal control information, and the second conversion signal by the method described in the fourth embodiment and the fifth embodiment of the fourth embodiment. For the suppression coefficient calculated by the above method, the analysis parameter B'(f) is calculated using [Equation 28], and the component parameter H (f) is defined as B'(f).
The operation of the object rendering information generation unit 2611 is as described in the first embodiment of the present embodiment.
As another configuration example of the component information conversion unit 2566, the component parameter generation unit 653 and the object rendering information generation unit 2611 in FIG. 46 can be integrated. In this case, the object rendering information is calculated from the second conversion signal decomposed into each frequency component, the background sound obtained by decoding the analysis information, the signal control information and the component rendering information, and the object rendering information is rendered as the rendering information. Output to the generator 2564.
Hereinafter, a specific example of the calculation method of the object rendering information will be shown. In the first method, as described in the third embodiment in the second embodiment, the suppression coefficient is calculated from the background sound by using the decoding signal. Further, as described in the first embodiment, the object rendering information is calculated from the suppression coefficient, the signal control information, and the component parameters. That is, the object rendering information T (f) includes the component rendering information defined in [Equation 21], the analysis parameters that can be calculated from the suppression coefficient defined in [Equation 25], and the signal control information defined in [Equation 26]. Can be used to make T (f) = U (f) × A (f) × B (f). In the second method, the suppression coefficient is calculated from the background sound, the signal control information, and the second conversion signal by the method described in the fourth embodiment and the fifth embodiment of the fourth embodiment. Using the suppression coefficient calculated by the above method, B'(f) of [Equation 28] and U (f) of [Equation 21] as described in the second embodiment of the present embodiment. Therefore, the object rendering information may be calculated using [Equation 29].
As described above, according to the sixth embodiment of the present invention, the receiving unit is composed of the target sound and the background sound of each point based on the analysis information in which the analysis information of each point is mixed. The input signal to be input can be controlled independently for each target sound and background sound at each point. It is also possible to independently control a specific target sound and background sound based on the signal control information. In addition, the localization of individual target sounds and background sounds can be controlled by using the component rendering information.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
A seventh embodiment of the present invention will be described. The present embodiment is characterized in that the receiving terminal controls each component corresponding to each sound source of the input signal at each point based on the analysis information, the object information, the signal control information, and the object rendering information.
FIG. 47 shows the configuration in the seventh embodiment of the present invention. Compared with FIG. 40 showing the fifth embodiment, the output signal generation unit 2550 of the reception unit 2507 is replaced with the output signal generation unit 2700. The difference is that the object rendering information is entered without. Therefore, the first configuration example and the second configuration example of the output signal generation unit 2700 will be described below.
Referring to FIG. 48, the first configuration example of the output signal generation unit 2700 is composed of a signal control unit 2760, a rendering information generation unit 2561, and a rendering unit 2563.
The signal control unit 2760 inputs the decoded signal, object information, analysis information, and signal control information. Decode object information and analysis information to generate object parameters and analysis parameters. Next, the object parameters are used to decompose the decoded signal into object signals (signals before mixing), and the analysis parameters are used to decompose the object signals into components. After that, the components are modified based on the signal control information to generate a modified component, and the modified decoded signal (the mixed signal, that is, the decoded signal modified based on the signal control information) is generated from the modified component. Is reconstructed and the modified decoding signal is output to the rendering unit 2563. Further, the signal control unit 2760 generates a modification object parameter that expresses the relationship between the modification decoding signal and the modification object signal (a signal obtained by modifying the signal before mixing based on the signal control information) for each frequency component, and generates a modification object. Output the parameters to the rendering information generator 2561.
As another operation example of the signal control unit 2760, the decoded signal may be converted into the modified decoded signal by using the object parameter, the analysis parameter, and the signal control information without generating the modified component. Also in this case, when converting the decoded signal into the modified decoded signal, a modified object parameter representing the relationship between the modified decoded signal and the modified object signal is generated for each frequency component, and the modified object parameter is output to the rendering information generator 2561. To do.
The calculation method of the modified object parameter is shown. As shown in the fifth embodiment, the relationship of [Equation 11] to [Equation 15] is established. At this time, if [Equation 16] to [Equation 19] are used as specific examples, the matrix for converting the decoding signal into the modified decoding signal is D (f) × A (f) × B (f) × C (f). Can be represented. Here, D (f) is a matrix of P rows and M columns, and if the inverse matrix of B (f) × C (f) is used, the modified object parameter can be represented by C (f), that is, the object parameter.
The rendering information generation unit 2561 inputs the modified object parameter output from the signal control unit 2760 and the object rendering information input via the input terminal. The object rendering information is converted into rendering information by using the modified object parameter, and the rendering information is output to the rendering unit 2563. The specific operation is as explained using [Equation 24].
The rendering unit 2563 generates an output signal from the modified decoding signal output from the signal control unit 2760 based on the rendering information output from the rendering information generation unit 2561, and outputs the output signal. Since the description has been made with reference to FIG. 42 in the fifth embodiment, detailed description thereof will be omitted.
Next, a second configuration example of the output signal generation unit 2700 will be described. Referring to FIG. 49, the output signal generation unit 2700 is composed of a rendering information generation unit 2564, an object rendering information correction unit 2770, and a rendering unit 2563.
The object rendering information correction unit 2770 inputs analysis information, signal control information, and object rendering information. The analysis information is decoded to generate analysis parameters, the object rendering information is modified using the analysis parameters and signal control information, and the modified object rendering information is output to the rendering information generation unit 2564. Specifically, if the modified object rendering information is T'(f),
[Number 30]<img id="000032" he="14" wi="121" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. Here, B (f) and A (f) are as defined in [Equation 17] and [Equation 18].
The rendering information generator 2564 inputs object information and modified object rendering. Object information and modification Rendering information is generated based on the object rendering information and output to the rendering unit 2563. Since the description has been given with reference to FIG. 43 in the fifth embodiment, detailed description will be omitted.
The rendering unit 2563 inputs the decoding signal and the rendering information. An output signal is generated from the decoded signal based on the rendering information, and the output signal is output. Since the description has been given with reference to FIG. 43 in the fifth embodiment, detailed description will be omitted.
As described above, according to the seventh embodiment of the present invention, the receiving unit corresponds to each sound source of the input signal at each point based on the analysis information in which the analysis information at each point is mixed. It can be controlled for each component. It is also possible to independently control only a specific sound source based on the signal control information. It is also possible to control the localization of individual object signals by using the object rendering information.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
Eighth embodiment of the present invention will be described. The present embodiment is characterized in that an input signal in which a target sound and a background sound are mixed as a sound source is targeted, and the target sound and the background sound are controlled based on analysis information, object information, signal control information, and object rendering information. And.
The configuration of the multipoint connection system in the present embodiment is shown in FIG. 47 as in the seventh embodiment. It is equipped with conference terminals 2500, 2501 and 2502 distributed at each point, and a multipoint control unit (MCU) 2505 that controls data exchange between conference terminals. Since the terminals 2500, 2501 and 2502 can have the same configuration, only the terminal 2500 is shown as a configuration example. Therefore, the terminal will be described below by taking the terminal 2500 as an example.
Since the multipoint connection device 2505 has been described with reference to FIG. 41 in the sixth embodiment, the description thereof will be omitted.
Next, the terminal 2500 will be described. FIG. 47 shows the configuration of the terminal 2500 in this embodiment. The configuration is the same as that of the seventh embodiment, but the operations of the signal analysis unit 101 and the output signal generation unit 2550 are different. Therefore, the signal analysis unit 101 and the output signal generation unit 2700 will be described in detail below.
The first embodiment in the present embodiment is the case where the analytical information is the suppression coefficient. With reference to FIG. 47, the signal analysis unit 101 outputs the suppression coefficient as analysis information. Correspondingly, the multipoint connection device 2505 mixes the suppression coefficients, and the output signal generation unit 2700 controls the decoding signal using the signal control information, the object rendering information, the object information, and the suppression coefficient. The operation of the signal analysis unit 101 when the suppression coefficient is used as the analysis information is as described in the first embodiment in the second embodiment. Therefore, the output signal generation unit 2700 will be described in detail below.
FIG. 49 shows the configuration of the output signal generator 2700. It is the same as the second configuration example of the output signal generation unit 2700 in the seventh embodiment, but there are differences in the operation of the object rendering information correction unit 2770. Therefore, the object rendering information correction unit 2770 will be described below.
FIG. 50 shows a configuration example of the object rendering information correction unit 2770. The object rendering information correction unit 2770 is composed of a component parameter generation unit 651 and an object rendering information change unit 2810.
The component parameter generation unit 651 inputs analysis information and signal control information, and outputs component parameters. Since the detailed operation has been described in the first embodiment of the sixth embodiment, the description thereof will be omitted.
The object rendering information change unit 2810 inputs analysis information and component parameters. The analysis information is decoded to generate the suppression factor, and the object rendering information is modified based on the suppression coefficient and the component parameters.
Specifically, if the object rendering information is T (f) and the modified object rendering information is T'(f),
[Number 31]<img id="000033" he="14" wi="100" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. Here, B (f) and H (f) are as defined in [Equation 25] and [Equation 27].
As another configuration example of the object rendering information correction unit 2770 in FIG. 49, the component parameter generation unit 651 and the object rendering information change unit 2810 in FIG. 50 can be integrated. In this case, the analysis information is decoded to calculate the suppression coefficient, the object rendering information is corrected from the calculated suppression coefficient and the signal control information, and the modified object rendering information is output to the rendering information generation unit 2564.
Specifically, the modified object rendering information T'(f) is
[Number 32]<img id="000034" he="16" wi="120" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Can be. Here, A (f) and B (f) are as defined in [Equation 26] and [Equation 25].
The second embodiment in the present embodiment is a case where the analysis information is a signal-to-background sound ratio. In the second embodiment, referring to FIG. 47, the signal analysis unit 101 outputs the signal-to-background sound ratio as analysis information. Correspondingly, the multipoint connection device 2505 mixes the signal-to-background sound ratio, and the output signal generator 2700 determines the decoding signal based on the signal-to-background sound ratio, the object information, the signal control information, and the object rendering information. To control. The operation of the signal analysis unit 101 when the signal-to-background sound ratio is used as the analysis information is as described in the second embodiment.
A configuration example of the output signal generator 2700 is shown in FIGS. 49 and 50. Comparing this embodiment with the first embodiment, the configurations of the component parameter generation unit 651 and the object rendering information change unit 2810 in FIG. 50 are different.
Since the component parameter generation unit 651 is described in the second embodiment of the sixth embodiment, the description thereof will be omitted.
The object rendering information changing unit 2810 inputs analysis information, component parameters, and object rendering information. The analysis information is decoded to generate a signal-to-background tone ratio, the object rendering information is modified from the signal-to-background tone ratio and the component parameters, and the modified object rendering information is output.
An example of a method of calculating the modified object rendering information will be described. As described in the second embodiment of the second embodiment, the suppression coefficient is calculated from the signal-to-background sound ratio. Further, as described in the first embodiment of the present embodiment, [Equation 31] is applied to calculate the modified object rendering information based on the object rendering information, the suppression coefficient, and the component parameters.
As another configuration example of the object rendering information correction unit 2770, the component parameter generation unit 651 and the object rendering information change unit 2810 in FIG. 50 can be integrated. In this case, the analysis information is decoded to calculate the signal-to-background tone ratio, the modified object rendering information is calculated from the calculated signal-to-background tone ratio, signal control information, and object rendering information, and the modified object rendering information is rendered information. Output to the generator 2564.
An example of a method of calculating the modified object rendering information in that case will be described. As described in the second embodiment of the second embodiment, the suppression coefficient is calculated from the signal-to-background sound ratio. Further, as described in the first embodiment of the present embodiment, [Equation 32] is applied to calculate the modified object rendering information based on the suppression coefficient, the signal control information, and the object rendering information.
The third embodiment in the present embodiment is a case where the analysis information is a background sound. In the third embodiment, referring to FIG. 47, the signal analysis unit 101 outputs a background sound as analysis information. Correspondingly, the multipoint connection device 2505 mixes the signal-to-background sound ratio, and the output signal generator 2700 controls the decoded signal based on the background sound, the object information, the signal control information, and the object rendering information. .. The operation of the signal analysis unit 101 when the background sound is used as the analysis information is as described in the third embodiment in the second embodiment.
FIG. 51 shows a configuration example of the output signal generation unit 2700. Compared with the first embodiment in FIG. 49, the object rendering information correction unit 2770 is replaced with the object rendering information correction unit 2780. Therefore, the object rendering information correction unit 2780 will be described below.
The object rendering information correction unit 2780 inputs analysis information, a decoding signal, signal control information, and object rendering information. The object rendering information is modified by using the analysis information, the decoding signal, and the signal control information, and the modified object rendering information is output to the rendering information generation unit 2564.
FIG. 51 shows a configuration example of the object rendering information correction unit 2780. The object rendering information correction unit 2780 is composed of a conversion unit 171, a component parameter generation unit 653, and an object rendering information change unit 2810. The conversion unit 171 decomposes the decoded signal into each frequency component to generate a second conversion signal, and outputs the second conversion signal to the component parameter generation unit 653.
The component parameter generation unit 653 inputs the second conversion signal, signal control information, and analysis information, and outputs the component parameters to the object rendering information change unit 2810. Since the detailed operation has already been described in the third embodiment of the sixth embodiment with reference to FIG. 46, the description thereof will be omitted.
The object rendering information changing unit 2810 calculates the background sound from the analysis information, corrects the object rendering information from the background sound and the component parameters, and outputs the modified object rendering information to the rendering information generation unit 2564.
An example of a method of calculating the modified object rendering information will be described. As described in the third embodiment of the second embodiment, the suppression coefficient is calculated from the background sound. Further, as described in the first embodiment of the present embodiment, [Equation 31] is applied to calculate the modified object rendering information from the suppression coefficient, the component parameters, and the object rendering information.
As another configuration example of the object rendering information correction unit 2780, the component parameter generation unit 653 and the object rendering information change unit 2810 in FIG. 52 can be integrated. In this case, the modified object rendering information is calculated from the second conversion signal, the background sound obtained by decoding the analysis information, the signal control information, and the object rendering information, and the modified object rendering information is output to the rendering information generator 2564. To do.
An example of a method of calculating the modified object rendering information at this time will be described. As described in the third embodiment of the second embodiment, the suppression coefficient is calculated from the background sound. Further, as described in the first embodiment of the present embodiment, [Equation 32] is applied to calculate the modified object rendering information from the suppression coefficient, the signal control information, and the object rendering information.
As described above, according to the eighth embodiment of the present invention, the receiving unit is composed of the target sound and the background sound of each point based on the analysis information in which the analysis information of each point is mixed. The input signal to be input can be controlled independently for each target sound and background sound at each point. It is also possible to independently control a specific target sound and background sound based on the signal control information. It is also possible to control the localization of individual object signals by using the object rendering information.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
A ninth embodiment of the present invention will be described. The present embodiment is characterized in that the receiving terminal controls each component corresponding to each sound source of the input signal at each point based on the analysis information, the object information, and the component rendering information.
Referring to FIG. 53, in the present embodiment, the signal control information input to the output signal generation unit 2900 of the terminal 2500 is mixed with the component rendering information as compared with FIG. 40 showing the fifth embodiment. The difference is that the output signal generation unit 2550 of the reception unit 2507 is replaced with the output signal generation unit 2900 correspondingly. Therefore, the output signal generation unit 2900 will be described below.
Referring to FIG. 54, the output signal generation unit 2900 in the present embodiment is composed of a rendering information generation unit 2564, a component information conversion unit 2910, and a rendering unit 2563.
The component information conversion unit 2910 inputs the analysis information and the component rendering information. The analysis information is decoded and the analysis parameters are generated, the component rendering information is converted into the object rendering information by using the analysis parameters, and the object rendering information is output to the rendering information generation unit 2564. Specifically, it can be expressed by the formula T (f) = U (f) × B (f). Here, T (f) is defined in [Equation 29], U (f) is defined in [Equation 21], and B (f) is defined in [Equation 17].
The rendering information generation unit 2564 generates rendering information by inputting object information and object rendering information, and outputs the rendering information to the rendering unit 2563. Since the detailed operation has been described with reference to FIG. 43 in the fifth embodiment, the description thereof will be omitted.
The rendering unit 2563 generates an output signal by inputting the decoding signal and the rendering information, and outputs the output signal. Since the detailed operation has been described with reference to FIG. 43 in the fifth embodiment, the description thereof will be omitted.
As described above, according to the ninth embodiment of the present invention, the receiving unit corresponds to each sound source of the input signal at each point based on the analysis information in which the analysis information at each point is mixed. It can be controlled for each component. Further, it is also possible to independently control only a specific sound source and control the localization of each sound source based on the component rendering information.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
A tenth embodiment of the present invention will be described. The present embodiment targets an input signal in which a target sound and a background sound are mixed as a sound source, and the present embodiment controls the target sound and the background sound based on analysis information, object information, and component rendering information. It is characterized by that.
Referring to FIG. 53, in the present embodiment, the signal control information input to the output signal generation unit 2900 of the terminal 2500 is integrated into the component rendering information as compared with the sixth embodiment of FIG. 40. The difference is that the output signal generation unit 2550 of the reception unit 2507 is replaced with the output signal generation unit 2900 correspondingly. Therefore, the output signal generation unit 2900 will be described below.
The first embodiment in the present embodiment is the case where the analytical information is the suppression coefficient. With reference to FIG. 53, the signal analysis unit 101 outputs the suppression coefficient as analysis information. Correspondingly, the multipoint connection device 2505 mixes the suppression coefficients, and the output signal generation unit 2900 controls the decoding signal using the component rendering information including the signal control information, the object information, and the suppression coefficient. ..
FIG. 54 shows a configuration example of the output signal generation unit 2900. Compared with the sixth embodiment shown in FIG. 43, the difference is that the component information conversion unit 2565 is replaced with the component information conversion unit 2910. Therefore, the component information conversion unit 2910 will be described below.
FIG. 55 shows a configuration example of the component information conversion unit 2910. The component information conversion unit 2910 is composed of a component parameter generation unit 3000 and an object rendering information generation unit 2611. Compared with the component information conversion unit 2565 of the sixth embodiment shown in FIG. 44, the component parameter generation unit 651 is replaced with the component parameter generation unit 3000.
The component parameter generation unit 3000 inputs the analysis information. The analysis information is decoded, the suppression coefficient is calculated, the component parameters are generated, and the component rendering information generation unit 2611 is output.
An example of the calculation method of the component parameters will be described. It differs from the first embodiment in the sixth embodiment in that signal control information is not used. That is, the signal control information A for controlling the target sound of the object signal i in [Equation 26].<sub>main</sub><sup>i</sup>(f) and signal control information A for controlling the background sound<sub>sub</sub><sup>i</sup>(f) is A<sub>main</sub><sup>i</sup>(f) = 1, A<sub>sub</sub><sup>i</sup>Corresponds to the case of (f) = 1, component parameter H<sub></sub>(f) is
[Number 33]<img id="000035" he="14" wi="50" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Will be. The object rendering information generation unit 2611 receives component parameters and component rendering information as inputs, and generates object rendering information. Since the detailed operation is described in detail in the first embodiment of the sixth embodiment, the description thereof will be omitted. In the multipoint connection device 2505, when the suppression coefficients are mixed, the suppression coefficient common to all objects may be used.
As another configuration example of the component information conversion unit 2910 in FIG. 54, the component parameter generation unit 3000 and the object rendering information generation unit 2611 in FIG. 55 can be integrated. In this case, the analysis information is decoded to calculate the suppression coefficient, the object rendering information is calculated from the suppression coefficient and the component rendering information, and the object rendering information is output to the rendering information generation unit 2564. That is, the object rendering information T (f) is
[Number 34]<img id="000036" he="14" wi="122" file="JP5282906B2_D0001.tif" img-format="tif" img-content="drawing" />Can be.
The second embodiment in the present embodiment is a case where the analysis information is a signal-to-background sound ratio. In the second embodiment, referring to FIG. 53, the signal analysis unit 101 outputs the signal-to-background sound ratio as analysis information. Correspondingly, the multipoint connection device 2505 mixes the signal-to-background sound ratio, and the output signal generator 2900 uses the signal-to-background sound ratio and the object information to generate the decoded signal based on the component rendering information. Control.
The configuration of the output signal generation unit 2900 is shown in FIGS. 54 and 55, as in the first embodiment. Comparing this embodiment with the first embodiment, the configuration of the component parameter generation unit 3000 in FIG. 55 is different. Therefore, the component parameter generation unit 3000 will be described below.
The component parameter generation unit 3000 inputs the analysis information. The analysis information is decoded to calculate the signal-to-background sound ratio, the component parameters are calculated from the calculated signal-to-background sound ratio, and the component parameters are output to the object rendering information generation unit 2611.
An example of the calculation method of the component parameters will be described. As described in the third embodiment of the second embodiment, the suppression coefficient is calculated from the signal-to-background sound ratio. Further, as described in the first embodiment of the present embodiment, [Equation 33] can be applied to calculate the component parameters.
As another configuration example of the component information conversion unit 2910 in FIG. 54, the component parameter generation unit 3000 and the object rendering information generation unit 2611 in FIG. 55 can be integrated. In this case, the analysis information is decoded to calculate the signal-to-background sound ratio, the object rendering information is calculated from the calculated signal-to-background sound ratio and the component rendering information, and the object rendering information is sent to the rendering information generator 2564. Output.
An example of the calculation method of the object rendering information at this time will be described. As described in the third embodiment of the second embodiment, the suppression coefficient is calculated from the signal-to-background sound ratio. Further, as described in the first embodiment of the present embodiment, [Equation 34] is applied to calculate the object rendering information from the suppression coefficient and the component parameters.
The third embodiment in the present embodiment is a case where the analysis information is a background sound. In the third embodiment, referring to FIG. 53, the signal analysis unit 101 outputs a background sound as analysis information. Correspondingly, the multipoint connection device 2505 mixes the background sound, and the output signal generation unit 2900 controls the decoding signal based on the background sound, the object information, and the component rendering information.
The configuration of this embodiment shown in FIG. 56 is different from the configuration of the first embodiment shown in FIG. 54 in that the component information conversion unit 2910 is replaced with the component information conversion unit 3001. Therefore, the component information conversion unit 3001 will be described below.
FIG. 57 shows a configuration example of the component information conversion unit 3001. The component information conversion unit 3001 is composed of a conversion unit 171, a component parameter generation unit 3002, and an object rendering information generation unit 2611.
The conversion unit 171 receives the decoding signal as an input, generates a second conversion signal obtained by decomposing the decoding signal into each frequency component, and outputs the second conversion signal to the component parameter generation unit 3002.
The component parameter generation unit 3002 inputs the analysis information and the second conversion signal. The analysis information is decoded and the background sound is calculated. Further, the component parameters are calculated from the background sound and the second conversion signal, and the component parameters are output to the object rendering information generation unit 2611.
An example of the calculation method of the component parameters will be described. As described in the third embodiment of the second embodiment, the suppression coefficient is calculated from the background sound and the decoded signal. Further, as described in the first embodiment of the present embodiment, the component parameters are calculated using [Equation 33].
The object rendering information generation unit 2611 inputs component parameters and component rendering information, and outputs object rendering information. Since the detailed operation has been described in the third embodiment of the sixth embodiment, the description thereof will be omitted.
As another configuration example of the component information conversion unit 3001 in FIG. 56, the component parameter generation unit 3002 and the object rendering information generation unit 2611 in FIG. 57 can be integrated. In this case, the object rendering information is calculated from the second conversion signal, the background sound calculated by decoding the analysis information, and the component rendering information, and the object rendering information is output to the rendering information generation unit 2564.
An example of a method of calculating object rendering information will be described. As described in the third embodiment of the second embodiment, the suppression coefficient is calculated from the background sound and the decoded signal. Further, as described in the first embodiment of the present embodiment, rendering information is calculated from the suppression coefficient and the component parameters using [Equation 34].
As described above, according to the tenth embodiment of the present invention, the receiving unit is composed of the target sound and the background sound of each point based on the analysis information in which the analysis information of each point is mixed. The input signal to be input can be controlled independently for each target sound and background sound at each point. Further, it is possible to independently control a specific target sound and background sound based on the component rendering information, and control the localization of individual sound sources.
Further, since the transmitting unit calculates the analysis information, the receiving unit can reduce the amount of calculation related to the calculation of the analysis information.
The eleventh embodiment of the present invention will be described. The present embodiment is characterized by reducing the quantization distortion generated when the receiving unit performs decoding by performing an analysis in consideration of the effect of the quantization distortion generated in the coding unit by the transmitting unit. And.
Referring to FIG. 58, the present embodiment is different from the first embodiment in that the transmitting unit 10 is replaced with the transmitting unit 90. Comparing the transmission unit 10 and the transmission unit 90, the signal analysis unit 101 is replaced by the signal analysis unit 900, and the signal analysis unit 900 is further input with a signal from the coding unit 100 in addition to the input signal. different.
A first configuration example of the signal analysis unit 900 will be described in detail with reference to FIG. 59.
The signal analysis unit 900 generates analysis information from the input signal and the coded signal from the coding unit 100. Since the coded signal is a signal to which the quantization distortion is added, it is possible to generate the analysis information in consideration of the amount of the quantization distortion.
The signal analysis unit 900 receives the input signal and the coded signal from the coding unit 100, and outputs the analysis information. The signal analysis unit 900 includes a conversion unit 120, a decoding unit 150, a quantization distortion calculation unit 910, an analysis information calculation unit 911, and a conversion unit 920.
The input signal is input to the conversion unit 120. Further, the coded signal from the coding unit 100 is input to the decoding unit 150.
The decoding unit 150 decodes the coded signal input from the coding unit 100. The decoding unit 150 outputs the decoding signal to the conversion unit 920. The conversion unit 920 decomposes the decoded signal into frequency components. The conversion unit 920 outputs the frequency component-decomposed decoded signal to the quantization distortion calculation unit 910.
The conversion unit 120 decomposes the input signal into frequency components. The conversion unit 120 outputs the frequency component-decomposed input signal to the quantization distortion unit 910 and the analysis information calculation unit 911. The quantization distortion calculation unit 910 compares the frequency component-decomposed decoded signal with the frequency component-decomposed input signal, and calculates the quantization distortion amount for each frequency component. For example, the difference between the magnitude of each frequency component of the frequency component-decomposed decoded signal and the magnitude of each frequency component of the frequency component-decomposed input signal may be used as the quantization distortion at that frequency. The quantization distortion calculation unit 910 outputs the quantization distortion amount of each frequency to the analysis information calculation unit 911.
The analysis information calculation unit 911 receives the input signal whose frequency components are decomposed from the conversion unit 120, and receives the quantization distortion amount of each frequency from the quantization distortion calculation unit 910. The analysis information calculation unit 911 decomposes the input signal obtained by decomposing the frequency component into each component corresponding to the sound source. Then, the analysis information calculation unit 911 generates analysis information representing the relationship between the plurality of components. The analysis information calculation unit 911 outputs the analysis information. Further, the analysis information calculation unit 911 may decompose the input signal whose frequency component has been decomposed into a component group composed of a plurality of components.
The analysis information calculation unit 911 considers the amount of quantization distortion and calculates the analysis information so that the quantization distortion is reduced at the time of decoding in the receiving unit. For example, the analysis information calculation unit 911 calculates analysis information from the magnitude of each frequency component of the input signal decomposed into frequency components and the magnitude of the quantization distortion at that frequency so that the quantization distortion is auditory masked. You may. Here, the analysis information calculation unit 911 may utilize in auditory masking that a small component becomes difficult to hear at a peripheral frequency of a frequency having a large magnitude of the frequency component. The masking characteristic is the size of the component that becomes difficult to hear at the peripheral frequency from the size of each frequency component. The analysis information calculation unit 911 calculates the masking characteristics at all frequencies. The analysis information calculation unit 911 corrects the analysis information at each frequency in consideration of the influence of the quantization distortion. When the magnitude of the quantization distortion is smaller than the masking characteristic, the quantization distortion is hard to hear. In this case, since the influence of the quantization distortion is small, the analysis information calculation unit 911 does not correct the analysis information. If the magnitude of the quantization distortion is larger than the masking characteristic, masking is not performed. In this case, the analysis information calculation unit 911 corrects the analysis information so as to reduce the quantization distortion. For example, when the suppression coefficient is used as the analysis information, the suppression coefficient may be small so that the quantization distortion is also suppressed at the same time as the background sound.
As described above, when the analysis information calculation unit 911 corrects the analysis information, the quantization distortion is auditory masked when the decoding is performed in the reception unit, and the distortion and noise are reduced.
So far, we have described the correction of analytical information so as to reduce the quantization distortion in consideration of auditory masking. However, the analysis information may be corrected so as to reduce the quantization distortion at all frequencies without considering the auditory masking.
A second configuration example of the signal analysis unit 900 will be described in detail with reference to FIG. 60.
The signal analysis unit 900 receives the input signal and the coded signal from the coding unit 100, and outputs the analysis information. The signal analysis unit 900 includes a conversion unit 120, a decoding unit 150, a quantization distortion calculation unit 910, an analysis information calculation unit 912, and a conversion unit 920.
The input signal is input to the conversion unit 120. Further, the coded signal from the coding unit 100 is input to the decoding unit 150.
The decoding unit 150 decodes the coded signal input from the coding unit 100. The decoding unit 150 outputs the decoding signal to the conversion unit 920. The conversion unit 920 decomposes the decoded signal into frequency components. The conversion unit 920 outputs the frequency component-decomposed decoded signal to the quantization distortion calculation unit 910 and the analysis information calculation unit 912.
The conversion unit 120 decomposes the input signal into frequency components. The conversion unit 120 outputs the frequency component-decomposed input signal to the quantization distortion calculation unit 910. The quantization distortion calculation unit 910 compares the frequency component-decomposed decoded signal with the frequency component-decomposed input signal, and calculates the quantization distortion amount for each frequency component. For example, the difference between the magnitude of each frequency component of the frequency component-decomposed decoded signal and the magnitude of each frequency component of the frequency component-decomposed input signal may be used as the quantization distortion at that frequency. The quantization strain calculation unit 910 outputs the quantization strain amount of each frequency to the analysis information calculation unit 912.
The analysis information calculation unit 912 receives the frequency component-decomposed decoded signal from the conversion unit 920, and receives the quantization distortion amount of each frequency from the quantization distortion calculation unit 910. The analysis information calculation unit 912 decomposes the input signal into each component corresponding to the sound source for the decoded signal whose frequency component is decomposed. Then, the analysis information calculation unit 912 generates analysis information representing the relationship between the plurality of components. The analysis information calculation unit 912 outputs the analysis information corrected so as to reduce the quantization distortion. The calculation of the analytical information such that the quantization distortion is reduced is the same as that of the first configuration example, and thus the description thereof will be omitted.
As described above, the first configuration example and the second configuration example of the signal analysis unit 900 generate analysis information so as to reduce the effect of the coding distortion generated in the coding unit 100. It has the effect of being able to reduce the quantization distortion that occurs when decoding is performed in the receiving unit 15.
Subsequently, the twelfth embodiment of the present invention will be described. A twelfth embodiment of the present invention controls an input signal composed of a target sound and a background sound as a sound source. The configuration of the eleventh embodiment of the present invention is shown in FIGS. 58 and 59 in the same manner as the configuration of the ninth embodiment of the present invention. In this embodiment, the configuration of the analysis information calculation unit 911 in FIG. 59 is different.
A configuration example of the analysis information calculation unit 911 according to the twelfth embodiment of the present invention will be described in detail with reference to FIG. 61. Comparing the analytical information calculation unit 121 shown in FIG. 10 and the analytical information calculation unit 911 shown in FIG. 61, the difference is that the quantization strain amount of each frequency is input from the quantization distortion calculation unit 910. .. Further, the background sound estimation unit 200 included in the analysis information calculation unit 121 is composed of the background sound estimation unit 1020 included in the analysis information calculation unit 911. The description of the part that overlaps with the description of FIGS. 10 and 43 will be omitted.
The analysis information calculation unit 911 receives the input signal decomposed into frequency components and the amount of quantization distortion of each frequency, and outputs the analysis information. The analysis information calculation unit 911 includes a background sound information generation unit 202 and a background sound estimation unit 1020.
The background sound estimation unit 1020 receives the input signal decomposed into frequency components and the amount of quantization distortion of each frequency. The background sound estimation unit 1020 estimates the background sound in consideration of the amount of quantization distortion. For example, the background sound estimation unit 1020 can perform the same processing as the background sound estimation unit 200 included in the analysis information calculation unit 121, using the estimated background sound plus the quantization distortion as the estimated background sound. The background sound estimation unit 1020 outputs background sound information in consideration of quantization distortion to the background sound information generation unit 202. The background sound information generation unit 202 generates analysis information based on the background sound information. Then, the background sound information generation unit 202 outputs the analysis information in consideration of the quantization distortion.
The receiving unit 15 controls the decoded signal based on the analysis information in consideration of the quantization distortion. With this configuration, it is possible to perform high-quality control in consideration of quantization distortion in the control of the decoded signal. Further, it has an effect that the quantization distortion generated when the receiving unit 15 performs decoding can be reduced.
Further, in the description of the twelfth embodiment of the present invention, the background sound information generation unit 202 may output the suppression coefficient, the signal-to-background sound ratio, or the background sound itself as analysis information.
When the signal-to-background sound ratio is encoded and output as the analysis information, the analysis information calculation unit 911 in FIG. 59 calculates and encodes the signal-to-background sound ratio. In order to encode the signal-to-background sound ratio, the configuration shown in FIG. 16 or the configuration shown in FIG. 18 may be used as the background sound information generation unit 202 in the analysis information calculation unit 911. In this case, the signal control unit 151 of the reception unit 15 in FIG. 58 has a configuration corresponding to the control of the decoded signal by the signal-to-background sound ratio.
When the background sound itself is encoded and output as the analysis information, the analysis information calculation unit 911 in FIG. 59 encodes and outputs the estimated background sound estimated by the background sound estimation unit 1020.
A configuration example of the analysis information calculation unit 911 that outputs the background sound itself as analysis information will be described with reference to FIG. 62. The analysis information calculation unit 911 of this configuration example receives the input signal decomposed in frequency components and the amount of quantization distortion of each frequency, and outputs the encoded background sound. The analysis information calculation unit 911 is composed of a background sound coding unit 205 and a background sound estimation unit 1020. The operation of the background sound estimation unit 1020 is omitted because it overlaps with the description of FIG. 61. The operation of the background sound coding unit 205 will be omitted because it overlaps with the description of FIG. 21.
In this case, the signal control unit 151 of the reception unit 15 in FIG. 58 has a configuration corresponding to the control of the decoded signal by the background sound.
As described above, the twelfth embodiment of the present invention controls the decoded signal based on the suppression coefficient, the signal-to-background sound ratio, or the background sound in consideration of the quantization distortion. With this configuration, it is possible to perform high-quality control in consideration of quantization distortion in the control of the decoded signal. Further, it has an effect that the quantization distortion and the coding distortion generated when the receiving unit 15 performs decoding can be reduced.
Next, the thirteenth embodiment of the present invention will be described. The eleventh embodiment of the present invention reduces the amount of calculation on the transmitting side and the amount of calculation related to the control of each component corresponding to each sound source on the receiving side based on the analysis information.
A thirteenth embodiment of the present invention will be described with reference to FIG. 63. In the first embodiment of the present invention shown in FIG. 1 and the thirteenth embodiment of the present invention shown in FIG. 63, the transmitting unit 10 is composed of the transmitting unit 13, and the receiving unit 15 is It differs in that it is composed of a receiving unit 18. With this configuration, the thirteenth embodiment of the present invention can share the conversion unit in the transmitting unit and the conversion unit in the receiving unit. As a result, the amount of calculation of the transmitting unit 13 and the receiving unit 18 can be reduced.
In the transmission unit 10 shown in FIG. 1 and the transmission unit 13 shown in FIG. 63, the coding unit 100 is composed of the coding unit 1100, and the signal analysis unit 101 is composed of the signal analysis unit 1101. It differs in that. In this embodiment, the coding unit 1100 outputs the input signal whose frequency components have been decomposed to the signal analysis unit 1101.
A configuration example of the coding unit 1100 will be described in detail with reference to FIG. 64. The coding unit 100 shown in FIG. 2 and the coding unit 1100 shown in FIG. 64 are different in that the first conversion signal, which is the output of the conversion unit 110, is output to the signal analysis unit 1101. Since the operations of the conversion unit 110 and the quantization unit 111 overlap with FIG. 2, the description thereof will be omitted. Here, the calculation amount of the coding unit 1100 is almost the same as the calculation amount of the coding unit 100 because the output signal is different from that of the coding unit 100 shown in FIG.
A configuration example of the signal analysis unit 1101 will be described in detail with reference to FIG. 65. The signal analysis unit 101 shown in FIG. 4 and the signal analysis unit 1101 shown in FIG. 65 are points in which the conversion unit 120 included in the signal analysis unit 101 is deleted. The operation of the analysis information calculation unit 121 is omitted because it overlaps with the description of FIG.
The signal analysis unit 1101 receives the first conversion signal from the coding unit 1100. The received first conversion signal is input to the analysis information calculation unit 121. Here, comparing the conversion unit 110 in the coding unit 1100 shown in FIG. 64 with the conversion unit 120 in the signal analysis unit 101 shown in FIG. 4, the input signals supplied to the conversion unit are the same. If the operation of the conversion unit is the same, the first conversion signal and the second conversion signal, which are the outputs of each, are the same. Therefore, when the operations of the conversion unit 110 and the conversion unit 120 are the same, the signal analysis unit 1101 deletes the conversion unit 120 and uses the first conversion signal output by the signal analysis unit 1101 as the second conversion signal. Can be done. With this configuration, the calculation amount of the signal analysis unit 1101 is reduced as compared with the signal analysis unit 101 by the amount corresponding to the calculation amount of the conversion unit 120.
Regarding the receiving unit, the receiving unit 15 shown in FIG. 1, the receiving unit 18 shown in FIG. 63, and the decoding unit 150 are composed of the decoding unit 1150, and the signal control unit 151 is composed of the signal control unit 1151. It differs in that it is.
A configuration example of the decoding unit 1150 will be described with reference to FIG. 66. The decoding unit 150 and the decoding unit 1150 shown in FIG. 3 are different from each other in that the inverse conversion unit 161 is deleted in the decoding unit 1150. The operation of the inverse quantization unit 160 is omitted because it overlaps with the description of FIG. The decoding unit 150 shown in FIG. 3 reverse-converts the first conversion signal output by the inverse quantization unit 160 into a time domain signal by the inverse conversion unit 161 and outputs the decoding signal to the conversion unit 171 shown in FIG. doing. In FIG. 5, the conversion unit 171 receives the decoding signal and performs a process of converting it into a second conversion signal. Here, as described above, when the operations of the conversion unit 110 and the conversion unit 120 are the same, the first conversion signal can be used as the second conversion signal. As a result, in the embodiment of the present embodiment, the decoding unit 1150 outputs the first conversion signal output by the inverse quantization unit 160 to the signal processing unit 172 included in the signal control unit 1151. Therefore, in the present embodiment, the inverse conversion unit 161 is deleted.
FIG. 67 is a diagram showing a configuration example of the signal control unit 1151. The difference between the signal control unit 151 and the signal control unit 1151 is that in the signal control unit 151 shown in FIG. 5, the input signal supplied as a time domain signal is converted into a frequency component by the conversion unit 171 and then the signal processing unit 172. In the signal control unit 1151, the conversion unit 171 is removed, and the frequency component from the decoding unit 1150 is directly supplied to the signal processing unit 172. Focusing on the frequency component supplied to the signal control unit 1151 from the decoding unit 1150, the difference between the first embodiment shown in FIG. 1 and the thirteenth embodiment shown in FIG. 63 is the inverse quantum. The difference is whether or not the frequency component output by the conversion unit 160 has passed through the inverse conversion unit 161 and the conversion unit 171. In either case, the same signal as the frequency component output by the inverse quantization unit 160 is supplied. It can be seen that the signal processing unit 172 in the signal control unit 1151 outputs the same result. Here, when the calculation amount of the receiving unit is considered, it can be seen that the calculation amount of the decoding unit 1150 is reduced as compared with the decoding unit 150 by the amount corresponding to the calculation amount of the inverse conversion unit 161. Further, it can be seen that the calculation amount of the signal control unit 1151 is reduced as compared with the signal control unit 151 by the amount corresponding to the calculation amount of the conversion unit 171.
A configuration example of the signal control unit 1151 will be described in detail with reference to FIG. 67. The signal control unit 151 shown in FIG. 5 and the signal control unit 1151 shown in FIG. 67 are different from each other in that the conversion unit 171 is deleted in the signal control unit 1151. The operations of the signal processing unit 172 and the inverse conversion unit 173 are omitted because they overlap with the description of FIG.
The signal control unit 151 of FIG. 5 converts the decoded signal input as the time domain signal into a second conversion signal by the conversion unit 171 and outputs it to the signal processing unit 172. As described above, when the operations of the conversion unit 110 and the conversion unit 120 are the same, the first conversion signal can be used as the second conversion signal. As a result, the signal processing unit 172 included in the signal control unit 1151 can receive the first conversion signal output by the inverse quantization unit 160. Therefore, in this embodiment, the conversion unit 171 is removed.
Here, focusing on the signal input from the decoding unit 1150 to the signal control unit 1151, the first embodiment shown in FIG. 1 and the thirteenth embodiment shown in FIG. 63 are inversely quantized. There is a difference in whether or not the signal output by unit 160 has passed through the inverse conversion unit 161 and the conversion unit 171. When the first conversion signal can be used as the second conversion signal, the frequency component and signal control of the signal output by the inverse quantization unit 160 in both the first embodiment and the thirteenth embodiment. The frequency components of the signal input to the processing unit 172 are the same. Therefore, the signal processing unit 172 in the signal control unit 1151 outputs the same result as the signal processing unit 172 shown in FIG. Further, the calculation amount of the decoding unit 1150 is reduced as compared with the decoding unit 150 by the amount corresponding to the calculation amount of the inverse conversion unit 161 shown in FIG. Further, the calculation amount of the signal control unit 1151 is reduced as compared with the signal control unit 151 by the amount corresponding to the calculation amount of the conversion unit 171 shown in FIG.
As described above, the thirteenth embodiment of the present invention, in addition to the effect of the first embodiment of the present invention, corresponds to the respective calculation amounts of the conversion unit 120, the inverse conversion unit 161 and the conversion unit 160. However, it has the effect that the amount of calculation is reduced as compared with the first embodiment. Further, the configuration of the calculation amount reduction of the thirteenth embodiment can be applied from the second embodiment to the twelfth embodiment of the present invention. As a result, each embodiment has the same effect of reducing the amount of calculation as the thirteenth embodiment of the present invention.
As described above, so far, in the first to thirteenth embodiments of the present invention, input signals composed of a plurality of sound sources have been analyzed, analysis information has been calculated, and the receiving side has been based on the analysis information. The method of controlling the decoding signal has been described. Here, further details will be described with reference to specific examples. The input signal varies depending on the usage method, and includes, for example, voice and musical instrument sound. In addition, when the purpose is to monitor by sound, there are operation sounds generated by each machine, voices of operators, footsteps, and the like.
When the input signal has a plurality of components, the present invention analyzes the input signal and encodes the analysis result as analysis information. If there are multiple components, a configuration similar to that shown in Figure 1 applies. The configuration of the signal analysis unit 101 and the signal control unit 151, the information output by the signal analysis unit 101 to the multiplexing unit 102, and the information sent from the separation unit 152 to the signal control unit 151 will be described in detail.
A second configuration example of the signal analysis unit 101 will be described in detail with reference to FIG. 68. The second configuration example of the signal analysis unit 101 is a configuration applied when there are a plurality of components. The signal analysis unit 101 includes a sound environment analysis unit 1210 and a sound environment information coding unit 1211. The sound environment analysis unit 1210 analyzes the information of a plurality of components contained in the input signal. The sound environment analysis unit 1210 outputs the component analysis information to the sound environment information coding unit 1211. The sound environment information coding unit 1211 encodes the component analysis information input from the sound environment analysis unit 1210. Then, the sound environment information coding unit 1211 outputs the coded component analysis information to the multiplexing unit 102 shown in FIG. Here, the multiplexing unit 102 shown in FIG. 1 performs multiplexing corresponding to the component analysis information input from the sound environment information coding unit 1211.
The sound environment analysis unit 1210 will be described in more detail.
Various methods can be used as a method for analyzing the information of a plurality of sound sources in the sound environment analysis unit 1210. For example, as a method for analyzing information of multiple sound sources, Non-Patent Document 11 (2005, "Speech Enhancement", Springer, (Speech Enhancement, Springer, 2005, pp. The signal separation method described in 371-402), pp. 371 to 402) may be used. In addition, as a method of analyzing information of multiple sound sources, sound scene analysis and Computational Auditory Scene Signal separation techniques such as Analysis), single input signal separation, single channel signal separation, etc. may be used. By these signal separation methods, the sound environment analysis unit 1210 separates the input signal into a plurality of components. Further, the sound environment analysis unit 1210 converts each of the separated components into component analysis information to be output and outputs the information. This component analysis information can be output in various formats. For example, the component analysis information includes a suppression coefficient for suppressing the background sound, the ratio of each component in each frequency component, and the magnitude of each frequency component of the signal of each component itself. The proportions of the components include, for example, the amplitude ratio with the input signal, the energy ratio with the input signal, and the average value thereof. The magnitude of each frequency component of the signal includes, for example, the absolute amplitude value, the energy, and the average value thereof. Further, depending on the method of signal separation, an analysis result itself to be output or a signal that can be easily converted into an analysis result to be output can be obtained in the middle of signal separation. In that case, it is also possible to perform the process of obtaining the analysis result to be output from the middle of the signal separation without performing the signal separation to the end.
A configuration example of the signal control unit 151 will be described in detail with reference to FIG. 69. The configuration example of the signal control unit 151 is a configuration applied when there are a plurality of components. The signal control unit 151 includes a sound environment information decoding unit 1212 and a sound environment information processing unit 1213. The signal control unit 151 receives the decoding signal from the decoding unit 150 and the signal in which the analysis information is encoded from the separation unit 152. The sound environment information decoding unit 1212 decodes the signal encoding the analysis information received from the separation unit 152. The sound environment information decoding unit 1212 outputs the decoded analysis information and outputs it to the sound environment information processing unit 1213. The analysis information corresponds to the analysis information output by the sound environment analysis unit 1210 included in the signal analysis unit 101 shown in FIG. 68. The sound environment information processing unit 1213 controls the decoded signal based on the analysis information input from the sound environment information decoding unit 1212. The method of this control depends on the purpose of the control. For example, as in the second embodiment, control for suppressing the background sound may be performed.
As described above, when there are a plurality of components included in the input signal, the present invention can be applied and the effect of the first embodiment of the present invention can be obtained.
The first embodiment of the present invention has been described above by exemplifying a configuration applied when there are a plurality of components included in an input signal. Similarly, the signal analysis unit, the signal control unit, or the output signal generation unit may be changed from the second embodiment to the thirteenth embodiment. Further, as in the configuration of the fifth embodiment to the tenth embodiment, control may be performed to localize the output of each component to an output signal composed of a plurality of channels.
Further, when the number of channels of the input signal is multiple, the method of analysis in the signal analysis unit 101 of the present invention includes directional control, beamforming, blind source separation, and independent components. A technique called Independent Component Analysis may be used. In particular, when the number of channels of the input signal is larger than the number of target sounds, directional control and beam forming are performed without using the above-mentioned background sound information estimation method or the analysis method in the thirteenth embodiment. , The analysis may be performed using only Blind Source Separation or Independent Component Analysis. For example, techniques related to directivity control and beamforming are described in Non-Patent Document 12 (2001, "Microphone Arrays", Springer, (Microphone Arrays, Springer,). 2001)) and Non-Patent Document 13 (2005, "Speech Enhancement", Springer, (Speech Enhancement, Springer, 2005, pp. 229-246), pp. 229-246). In addition, techniques related to the method of blind signal separation and independent component analysis are described in Non-Patent Document 14 (2005, "Speech Enhancement", Springer, (Speech Enhancement, Springer, 2005, pp. 271-369), 271. It is disclosed on page 369).
When the method of analysis described above is used, the configuration shown in FIG. 1 is applied to the first embodiment of the present invention. Further, the configuration of the signal analysis unit 101, the configuration of the signal control unit 151, the information output by the signal analysis unit 101 to the multiplexing unit 102, and the information sent from the separation unit 152 to the signal control unit 151 will be described in detail. The input signal is a multi-channel signal. The basic operation is the same as the operation of the first embodiment, and since it overlaps with FIG. 1, the description thereof will be omitted.
A third configuration example of the signal analysis unit 101 will be described in detail with reference to FIG. 70. The third configuration example of the signal analysis unit 101 corresponds to the case where the number of channels of the input signal is a plurality. The signal analysis unit 101 of this configuration example uses an independent component analysis method as a method of analyzing the input signal. The signal analysis unit 101 of this configuration example outputs the filter coefficient for signal separation of the components corresponding to each sound source included in the input signal as analysis information.
The signal analysis unit 101 includes a signal separation analysis unit 1200 and a separation filter coding unit. The signal separation analysis unit 1200 calculates the separation filter coefficient by the method of independent component analysis. The separation filter coefficient is a filter coefficient used for signal separation of components corresponding to each sound source included in the input signal. Then, the signal separation analysis unit 1200 outputs the separation filter coefficient to the separation filter coding unit 1201. The separation filter coding unit 1201 encodes the separation filter coefficient input from the signal separation analysis unit 1200. The separation filter coding unit 1201 outputs the coded separation filter coefficient as analysis information.
A third configuration example of the signal control unit 151 will be described in detail with reference to FIG. 71. The third configuration example of the signal control unit 151 corresponds to the case where the number of channels of the input signal is a plurality.
The signal control unit 151 includes a separation filter decoding unit 1202 and a filter 1203. The separation filter decoding unit 1202 receives the separation filter coefficient encoded as analysis information from the separation unit 152. Then, the separation filter decoding unit 1202 decodes the coded separation filter coefficient and outputs the separation filter coefficient to the filter 1203. The filter 1203 receives the decoding signals of a plurality of channels from the decoding unit 150, and receives the separation filter coefficient from the separation filter decoding unit 1202. Then, the filter 1203 performs a filter process based on the separation filter coefficient on the decoded signals of a plurality of channels. The filter 1203 outputs a signal in which the signals of the components corresponding to each sound source are separated.
As described above, when the number of channels of the input signal is a plurality, the present invention analyzes the input signal at the transmission unit. With this configuration, even when the number of channels of the input signal is multiple, the input signal composed of a plurality of sound sources is controlled by the receiving unit for each component corresponding to each sound source based on the signal analysis information in the transmitting unit. can do. Further, since the transmitting unit analyzes the signal, the receiving unit can reduce the amount of calculation related to the signal analysis.
Further, in the configuration examples shown in FIGS. 70 and 71, the filter coefficient of the separation filter was used as the analysis information of the input signal, but the analysis information used in the first to thirteenth embodiments. May be used. For that purpose, the signal separation analysis unit 1200 shown in FIG. 70 may be configured to calculate the separation filter and perform signal separation using the separation filter. As a result, the separation filter coding unit 1201 is replaced with the sound environment information coding unit 1211 shown in FIG. 68.
Further, as the method of analyzing the input signal in the signal analysis unit 101, not only the independent component analysis but also the method disclosed in Non-Patent Documents 12 to 15 may be used. Further, these analysis methods may be used in combination with the analysis methods in the first to thirteenth embodiments of the present invention. Further, depending on the analysis method, an analysis result to be output or a signal that can be easily converted into an analysis result to be output can be obtained in the middle of the analysis. In that case, the analysis process may be changed so that the analysis result is output without performing the analysis to the end.
A fourteenth embodiment of the present invention will be described. FIG. 72 shows the configuration of the multipoint connection system according to the present embodiment. In this embodiment, the multipoint connection device is connected in multiple stages. As the terminals 3401, 3402, 3403, 3404, 3405 and the multipoint connection device 3410, 3411 in FIG. 72, for example, the terminal 2500 and the multipoint connection device 2105 described in the first embodiment can be used. Since one set of input / output of the multipoint connection device and the input / output of the terminal are the same, it is clear that there is no inconvenience in connecting the multipoint connection device in multiple stages. By the multi-stage connection of the multi-point connection device, in addition to the effect of the first embodiment of the present invention, the processing amount load of the multi-point connection device can be distributed, and a large-scale remote conference system can be constructed. It is also possible to apply it to the second to thirteenth embodiments of the present invention.
A fifteenth embodiment of the present invention will be described. FIG. 73 is a block diagram of a signal processing device based on the fifteenth embodiment of the present invention. A fifteenth embodiment of the present invention comprises computers (central processing units; processors; data processing units) 3500, 3501, 3502, and 3503 that operate under program control. The computers 3500, 3501, and 3502 process the receiving unit and the transmitting unit described in the first to fourteenth embodiments, receive an input signal, output a transmission signal, and receive a transmission signal to generate an output signal. It works based on the program to do. On the other hand, the computer 3503 is a program for processing the multipoint connection device described in the first to fourteenth embodiments, mixing transmission signals from each terminal, and distributing the mixing result to each terminal. It works based on. In the present embodiment, an example in which three computers are connected to the computer 3503 has been described, but any number of terminals can be used.
Although the present invention has been described above with reference to preferred embodiments and examples, the present invention is not necessarily limited to the above embodiments and examples, and is variously modified and implemented within the scope of the technical idea thereof. Can be done.
As described above, the first invention is a first signal that receives a first signal including a plurality of components and a first analytical information representing a relationship between the plurality of components included in the first signal. A second signal receiving unit that receives a receiving unit, a second signal including a plurality of components, and a second analytical information representing a relationship between the plurality of components included in the second signal, and the first signal receiving unit. A multipoint connection including a signal mixing unit that mixes the signal 1 and the second signal, and an analysis information mixing unit that mixes the first analysis information and the second analysis information. It is a device.
Further, in the second invention, in the first invention, the analysis information mixing unit uses the first analysis information and the second analysis information as a first analysis parameter representing each frequency component and a first. It is characterized by including an analysis parameter mixing unit that is converted into 2 analysis parameters and mixes the first analysis parameter and the second analysis parameter for each frequency component.
Further, in the third invention, in the first invention, the signal mixing unit generates mixing information, and the analysis information mixing unit performs the first analysis information and the second analysis based on the mixing information. It is characterized by mixing with information.
Further, in the fourth invention, in the third invention, the analysis information mixing unit uses the first analysis information and the second analysis information as the first analysis parameter and the first analysis parameter representing the respective frequency components. It is characterized by including an analysis parameter mixing unit that converts the analysis parameters into 2 and mixes the first analysis parameter and the second analysis parameter for each frequency component based on the mixing information.
Further, the fifth invention is characterized in that, in the fourth invention, the analysis information mixing unit includes a selection unit for selecting the first analysis parameter and the second analysis parameter.
Further, the sixth invention is characterized in that, in the third or fourth invention, the mixing information is a weighting of the first signal and the second signal.
Further, in the seventh invention, in the first invention, the first signal receiving unit receives the first object information representing the characteristics of each frequency component of the first signal, and the second signal receiving unit receives the first object information representing the characteristics of each frequency component of the first signal. The signal receiving unit further includes an object information mixing unit that receives second object information representing the characteristics of each frequency component of the second signal and mixes the first object information and the second object information. It is characterized by including.
Further, the eighth invention is characterized in that, in the seventh invention, the object information mixing unit selects the first object information and the second object information according to the importance.
Further, the ninth invention includes a signal receiving unit that receives an input signal including a plurality of components, a signal analysis unit that generates analysis information indicating a relationship between the plurality of components from the input signal, and the input signal. It is a signal analyzer characterized by including an object information extraction unit that generates object information representing the characteristics of each frequency component of the above.
Further, the tenth invention is a signal receiving unit that receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. An output signal generation unit that receives component rendering information that controls the output of the component and generates an output signal in which the component is controlled based on the analysis information, the object information, and the component rendering information. It is a signal control device characterized by including.
Further, in the eleventh invention, in the tenth invention, the output signal generation unit generates object rendering information representing the relationship between the input signal and the output signal for each frequency component based on the analysis information. A component information conversion unit, a rendering information generation unit that generates the rendering information based on the object information and the object rendering information, and a rendering that generates the output signal from the input signal based on the rendering information. It is characterized by including a part.
Further, in the twelfth invention, in the tenth invention, the output signal generation unit expresses the relationship between the input signal and the output signal for each frequency component based on the analysis information and the input signal. A component information conversion unit that generates object rendering information, a rendering information generation unit that generates the rendering information based on the object information and the object rendering information, and an output from the input signal based on the rendering information. It is characterized by including a rendering unit that generates a signal.
Further, in the thirteenth invention, in the tenth invention, the output signal generation unit further receives signal control information for controlling a specific component, and receives the analysis information, the object information, and the component rendering information. It is characterized in that the component generates a controlled output signal based on the signal control information.
Further, in the fourteenth invention, in the thirteenth invention, the output signal generation unit generates an object signal in which the signal is decomposed into frequency components based on the object information, and the output signal generation unit generates the object signal based on the analysis information. A component of the object signal is decomposed, the component is modified based on the signal control information, a modification signal is generated from the modification component, and a parameter representing the relationship between the modification component and the modification signal is obtained. The signal control unit to be generated, the rendering information generation unit that generates rendering information representing the relationship between the correction signal and the output signal, and the correction based on the rendering information. It is characterized by including a rendering unit that generates the output signal from the signal.
Further, in the fifteenth invention, in the thirteenth invention, the output signal generation unit expresses the relationship between the input signal and the output signal for each frequency component based on the analysis information and the signal control information. The component information conversion unit that generates the object rendering information, the rendering information generation unit that generates the rendering information based on the object information and the object rendering information, and the input signal based on the rendering information. It is characterized by including a rendering unit that generates an output signal.
Further, in the sixteenth invention, in the thirteenth invention, the output signal generation unit determines the relationship between the object signal and the output signal based on the analysis information, the signal control information, and the input signal. A component information conversion unit that generates object rendering information represented for each component, a rendering information generation unit that generates the rendering information based on the object information and the object rendering information, and the above-mentioned based on the rendering information. It is characterized by including a rendering unit that generates the output signal from the input signal.
Further, the seventeenth invention is a signal receiving unit that receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. The analysis information, the object information, the object rendering information, and the object rendering information are received, and the object rendering information representing the relationship between the input signal and the output signal for each frequency component and the signal control information for controlling a specific component are received. The signal control device is characterized by including an output signal generation unit for generating a controlled output signal based on the signal control information.
Further, in the eighteenth invention, in the seventeenth invention, the output signal generation unit generates an object signal in which the signal is decomposed into frequency components based on the object information, and the output signal generation unit generates the object signal based on the analysis information. A component of the object signal is decomposed, the component is modified based on the signal control information, a modification signal is generated from the modification component, and a parameter representing the relationship between the modification component and the modification signal is obtained. A signal control unit to be generated, a rendering information generation unit that generates rendering information representing the relationship between the correction signal and the output signal based on the parameters and the object rendering information, and the modification based on the rendering information. It is characterized by including a rendering unit that generates the output signal from the signal.
Further, in the nineteenth invention, in the seventeenth invention, the output signal generation unit has an object rendering information correction unit that corrects the object rendering information based on the analysis information and the signal control information, and the object. It is characterized by including a rendering information generation unit that generates the rendering information based on the information and the object rendering information, and a rendering unit that generates the output signal from the input signal based on the rendering information.
Further, in the twentieth invention, in the seventeenth invention, the output signal generation unit modifies the object rendering information based on the analysis information, the signal control information, and the input signal. A rendering information generation unit that generates the rendering information based on the object information and the object rendering information, and a rendering unit that generates the output signal from the input signal based on the rendering information. It is a feature.
Further, the twenty-first invention receives a first signal including a plurality of components and a first analytical information representing a relationship between the plurality of components included in the first signal, and receives a plurality of configurations. The second signal including the elements and the second analysis information representing the relationship between the plurality of components included in the second signal are received, and the first signal and the second signal are mixed. The multipoint connection method is characterized in that the first analysis information and the second analysis information are mixed.
Further, in the twenty-second invention, in the twenty-first invention, the first analysis information and the second analysis information are converted into a first analysis parameter and a second analysis parameter representing each frequency component. The first analysis parameter and the second analysis parameter are mixed for each frequency component.
Further, in the twenty-first invention, in the twenty-first invention, the first signal and the second signal are mixed to generate mixed information, and the first analytical information is based on the mixed information. And the second analysis information are mixed.
Further, in the twenty-fourth invention, in the twenty-third invention, the first analysis information and the second analysis information are converted into a first analysis parameter and a second analysis parameter representing each frequency component. Then, the first analysis parameter and the second analysis parameter are mixed for each frequency component based on the mixing information.
The 25th invention is characterized in that, in the 24th invention, the first analysis parameter and the second analysis parameter are selected.
The 26th invention is characterized in that, in the 23rd or 24th invention, the mixed information is a weighting of the first signal and the second signal.
Further, in the twenty-seventh invention, in the twenty-first invention, the first object information representing the characteristics of each frequency component of the first signal is received, and the characteristics of each frequency component of the second signal are received. The second object information representing the above is received, and the first object information and the second object information are further mixed.
The 28th invention is characterized in that, in the 27th invention, the first object information and the second object information are selected according to their importance.
Further, the 29th invention is to generate analysis information representing the relationship between the plurality of components from an input signal including a plurality of components, and generate object information representing the characteristics of each frequency component of the input signal. It is a signal analysis method characterized by.
Further, the thirtieth invention receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. It is characterized in that it receives component rendering information that controls the output of the component, and generates a controlled output signal by the component based on the analysis information, the object information, and the component rendering information. It is a signal control method.
Further, in the thirty-first invention, in the thirtieth invention, object rendering information representing the relationship between the input signal and the output signal for each frequency component is generated based on the analysis information, and the object information and the above-mentioned object information. The rendering information is generated based on the object rendering information, and the output signal is generated from the input signal based on the rendering information.
Further, in the thirty-second invention, in the thirtieth invention, object rendering information representing the relationship between the input signal and the output signal for each frequency component is generated based on the analysis information and the input signal. The rendering information is generated based on the object information and the object rendering information, and the output signal is generated from the input signal based on the rendering information.
Further, in the thirty-third invention, the thirty-third invention receives signal control information for controlling a specific component, and is based on the analysis information, the object information, the component rendering information, and the signal control information. The component is characterized in that it produces a controlled output signal.
Further, in the thirty-fourth invention, in the thirty-third invention, an object signal in which the signal is decomposed into frequency components is generated based on the object information, and a component of the object signal is generated based on the analysis information. It is disassembled, the component is modified based on the signal control information, a modification signal is generated from the modification component, a parameter showing the relationship between the modification component and the modification signal is generated, and the parameter is converted into the parameter. Based on the component rendering information, rendering information representing the relationship between the correction signal and the output signal is generated, and the output signal is generated from the correction signal based on the rendering information.
Further, the 35th invention generates object rendering information representing the relationship between the input signal and the output signal for each frequency component based on the analysis information and the signal control information in the 33rd invention. Then, the rendering information is generated based on the object information and the object rendering information, and the output signal is generated from the input signal based on the rendering information.
The 36th invention is an object in which the relationship between the object signal and the output signal is expressed for each frequency component based on the analysis information, the signal control information, and the input signal in the 33rd invention. It is characterized in that rendering information is generated, the rendering information is generated based on the object information and the object rendering information, and the output signal is generated from the input signal based on the rendering information.
Further, the 37th invention receives an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. The object rendering information representing the relationship between the input signal and the output signal for each frequency component and the signal control information for controlling a specific component are received, and the analysis information, the object information, the object rendering information, and the object rendering information are received. The signal control method is characterized in that the component generates a controlled output signal based on the signal control information.
Further, in the 38th invention, in the 37th invention, an object signal in which the signal is decomposed into frequency components is generated based on the object information, and a component of the object signal is decomposed based on the analysis information. Then, the component is modified based on the signal control information, a modification signal is generated from the modification component, a parameter showing the relationship between the modification component and the modification signal is generated, and the parameter and the modification signal are generated. Based on the object rendering information, rendering information representing the relationship between the correction signal and the output signal is generated. It is characterized in that the output signal is generated from the correction signal based on the rendering information.
Further, in the thirty-seventh invention, the thirty-ninth invention modifies the object rendering information based on the analysis information and the signal control information, and the object rendering information is based on the object information and the object rendering information. It is characterized in that rendering information is generated and the output signal is generated from the input signal based on the rendering information.
Further, in the thirty-fourth invention, in the thirty-fourth invention, the object rendering information is modified based on the analysis information, the signal control information, and the input signal, and the object information and the object rendering information are obtained. Based on the above, the rendering information is generated, and the output signal is generated from the input signal based on the rendering information.
Further, the 41st invention comprises a process of receiving a first signal including a plurality of components and a first analysis information representing a relationship between the plurality of components included in the first signal, and a plurality of processes. The process of receiving the second signal including the components and the second analysis information representing the relationship between the plurality of components included in the second signal, and the first signal and the second signal. Is a program that causes an information processing apparatus to execute a process of mixing the first analysis information and a process of mixing the second analysis information.
Further, the 42nd invention is the process of receiving an input signal including a plurality of components and the process of generating analysis information representing the relationship between the plurality of components from the input signal in the 30th invention. , A program that causes an information processing apparatus to execute a process of generating object information representing the characteristics of each frequency component of the input signal.
Further, the 43rd invention comprises a process of receiving an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. An information processing device that receives component rendering information that controls the output of the component, and generates an output signal in which the component is controlled based on the analysis information, the object information, and the component rendering information. It is a program to be executed by.
Further, the 44th invention comprises a process of receiving an input signal including a plurality of components, analysis information representing the relationship between the plurality of components, and object information representing the characteristics of each frequency component of the input signal. Upon receiving object rendering information representing the relationship between the input signal and the output signal for each frequency component and signal control information for controlling a specific component, the analysis information, the object information, the object rendering information, and the signal are received. This is a program that causes an information processing apparatus to execute a process of generating an output signal whose components are controlled based on the control information.
This application claims priority on the basis of Japanese Application Japanese Patent Application No. 2007-168547 filed on June 27, 2007, and incorporates all of its disclosures herein.
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| JP2000022603A | Cites | Japan | Examiner |
| JP2000069179A | Cites | Japan | Search report |
| JP2000069179A | Cites | Japan | Examiner |
| JPH0678062A | Cites | Japan | Search report |
| JPH0678062A | Cites | Japan | Examiner |
| JP2000022603A | Cites | Japan | – |
| JP06078062A | Cites | Japan | – |
| JP2000069179A | Cites | Japan | – |
11 members in 6 offices
Priority claims11
| Document | Office | Kind | Date |
|---|---|---|---|
| 2007168547 | Japan | A | |
| 2007168547 | Japan | A | |
| 2007168547 | Japan | – | |
| 2008061622 | Japan | W | |
| 2008061622 | Japan | W | |
| 2009520630 | Japan | A | |
| 20072007168547 | – | – | – |
| 2008061622 | – | – | – |
| JP20070168547 | – | – | – |
| JP20090520630 | – | – | – |
| WO2008JP61622 | – | – | – |
Members11
| Document | Office | Kind | |
|---|---|---|---|
| WO2009001887A1 | World Intellectual Property Organization (WIPO) | A1 | |
| KR20100022989A | Republic of Korea | A | |
| EP2164238A1 | European Patent Office (EPO) | A1 | |
| CN101715643A | China | A | |
| US2010198990A1 | United States of America | A1 | |
| JPWO2009001887A1 | Japan | A1 | |
| EP2164238A4 | European Patent Office (EPO) | A4 | |
| CN101715643B | China | B | |
| EP2164238B1 | European Patent Office (EPO) | B1 | |
| JP5282906B2This record | Japan | B2 | |
| US9118805B2 | United States of America | B2 |
8 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 5282906
- Publication, DOCDB
- 5282906
- Publication, EPODOC
- JP5282906B
- Application
- 2009520630
- Application, DOCDB
- 2009520630
- Application, EPODOC
- JP20090520630
Titles2
- Japanese
- 多地点接続装置、信号分析及び装置と、その方法及びプログラム
- English
- Multipoint connection device, signal analysis and device, and their methods and programs
Classification
- CPC, 5
- H04M3/56
- H04N7/152
- G10L19/008
- G10L21/02
- G10L19/00
- IPC, 3
- H04M3 56
- G10L21 0208
- G10L19 008