Compression of signals using a perceptual model
16 claims: 16 independent, 0 dependent
- 1A method for coding a signal including the steps of dividing said signal into N spectrum bands, and coding selected energy bands, and CHARACTERISED BY selecting n out of N bands that have the highest energy relative to a given noise measure for that band to form said selected energy bands. Procédé pour coder un signal comprenant les étapes de division du signal en N bandes spectrales, et de codage de bandes d'énergie sélectionnées, et caractérisé par :la sélection de n parmi N bandes ayant l'énergie la plus élevée par rapport à une mesure de bruit donnée pour cette bande afin de former les bandes d'énergie sélectionnées. Verfahren zur Codierung eines Signals mit den Schritten: Unterteilung des Signals in N Spektralbänder und Codierung gewählter Energiebänder, gekennzeichnet durch Auswählen von n aus N Bändern, die die höchste Energie mit Bezug auf ein gegebenes Rauschmaß für dieses Band besitzen, um die gewählten Energiebänder zu bilden.
- 2A method as claimed in claim 1 wherein said given noise measure of a band relates to the perception of noise in that band. Procédé selon la revendication 1, dans lequel la mesure de bruit donnée d'une bande est relative à la perception du bruit dans cette bande. Verfahren nach Anspruch 1, bei dem das gegebene Rauschmaß eines Bandes sich auf die Wahrnehmbarkeit von Rauschen in diesem Band bezieht.
- 3A method as claimed in claim 1 wherein said given noise measure of a band is related to the energy level of just noticeable noise in that band. Procédé selon la revendication 1, dans lequel la mesure de bruit donnée d'une bande est relative au niveau d'énergie du bruit à peine perceptible dans cette bande. Verfahren nach Anspruch 1, bei dem das gegebene Rauschmaß eines Bandes sich auf den Energiepegel eines gerade bemerkbaren Rauschens in diesem Band bezieht.
- 4A method as claimed in claim 1 wherein said signal is an analog signal, said step of dividing results in analog band signals, and said coding is digital coding. Procédé selon la revendication 1, dans lequel le signal est un signal analogique, l'étape de division conduit à des signaux en bande analogique, et le codage est un codage numérique. Verfahren nach Anspruch 1, bei dem das Signal ein Analogsignal ist und der Unterteilungsschritt zu Analogbandsignalen führt und bei dem die Codierung eine digitale Codierung ist.
- 5A method as claimed in claim 4 wherein said digital coding concatenates the digital code of each of the analog band signals to form a stream of digital codes. Procédé selon la revendication 4, dans lequel le codage numérique concatène le code numérique de chacun des signaux en bande analogique pour former un train de codes numériques. Verfahren nach Anspruch 4, bei dem die digitale Codierung den Digitalcode jedes der Analogbandsignale zur Bildung eines Stroms von Digitalcodierungen miteinander verknüpft.
- 6A method as claimed in claim 1 wherein said signal is an analog signal, and said step of coding includes down-shifting of the selected n bands to form a baseband signal. Procédé selon la revendication 1, dans lequel le signal est un signal analogique, et l'étape de codage comporte un abaissement des n bandes sélectionnées pour former un signal en bande de base. Verfahren nach Anspruch 1, bei dem das Signal ein Analogsignal ist und der Codierschritt ein Nach-Unten-Verschieben der gewählten n Bänder zur Bildung eines Grundbandsignals umfaßt.
- 7A method as claimed in claim 6 wherein said step of dividing results in analog band signals of substantially equal bandwidth, W, and said step of coding includes down-shifting of the selected n bands to form a baseband signal of bandwidth nW. Procédé selon la revendication 6, dans lequel l'étape de division conduit à des signaux en bandes analogique ayant des largeurs de bande sensiblement égales W, et l'étape de codage comprend un abaissement de fréquence des n bandes sélectionnées pour former un signal en bande de base de largeur de bande nW. Verfahren nach Anspruch 6, bei dem der Unterteilungsschritt zu Analogbandsignalen im wesentlichen gleicher Bandbreite W führt und der Codierschritt ein Nach-Unten-Schieben der gewählten n Bänder zur Bildung eines Grundbandsignals der Bandbreite nW umfaßt.
- 8A method as claimed in claim 6 said step of coding encodes said baseband signal. Procédé selon la revendication 6, dans lequel l'étape de codage code le signal en bande de base. Verfahren nach Anspruch 6, bei dem der Codierschritt das Grundbandsignal codiert
- 9A method as claimed in claim 6 said step of coding encodes said baseband signal to digital form. Procédé selon la revendication 6, dans lequel l'étape de codage code le signal en bande de base sous forme numérique. Verfahren nach Anspruch 6, bei dem der Codierschritt das Grundbandsignal in Digitalform codiert.
- 10A method for coding an analog signal including the step of dividing the signal into signals of distinc N frequency bands to form band signals, and CHARACTERISED BY selecting n out of said N band signals characterized by a signal energy level relative to a threshold that is not lower than the signal energy level relative to a threshold of non-selected band signals, where n is a number such that the sum of bandwidths of the selected band signals does not exceed a preselected bandwidth; anddown-shifting said n selected band signals to occupy a unique band within a baseband having said preselected bandwidth. Procédé pour coder un signal analogique comprenant l'étape de division du signal en signaux dans N bandes de fréquence distinctes afin de former des signaux dans des bandes, et caractérisé par la sélection de n parmi les N signaux dans des bandes caractérisés par un niveau d'énergie de signal par rapport à un seuil qui n'est pas inférieur au niveau d'énergie de signal par rapport à un seuil de signaux dans des bandes non sélectionnées, où n est un nombre tel que la somme des largeurs de bande des signaux dans les bandes sélectionnées, ne dépasse pas une largeur de bande présélectionnée ; etl'abaissement de fréquence des n signaux dans des bandes sélectionnées afin qu'ils occupent une bande unique à l'intérieur d'une bande de base ayant la largeur de bande présélectionnée. Verfahren zum Codieren eines Analogsignals mit dem Schritt:Unterteilen des Signals in Signale bestimmter N Frequenzbänder zur Bildung von Bandsignalen, gekennzeichnet durch Auswählen von n der N Bandsignale, die durch einen Signalenergiepegel mit Bezug auf einen Schwellenwert gekennzeichnet sind, der nicht kleiner ist als der Signalenergiepegel mit Bezug auf einen Schwellenwert nicht gewählter Bandsignale, wobei n eine so gewählte Zahl ist, daß die Summe der Bandbreiten der gewählten Bandsignale eine vorbestimmte Bandbreite nicht übersteigt, und Nach-Unten-Schieben der n gewählten Bandsignale zur Belegung eines besonderen Bandes innerhalb eines Grundbandes mit der vorgewählten Bandbreite.
- 11A method as claimed in claim 10 wherein the threshold for each band signal is a given threshold associated with said each band. Procédé selon la revendication 10, dans lequel le seuil de chaque signal dans une bande est un seuil donné associé à chacune de ces bandes. Verfahren nach Anspruch 10, bei dem der Schwellenwert für jedes Bandsignal ein gegebener, jedem Band zugeordneter Schwellenwert ist.
- 12A method as claimed in claim 10 wherein the threshold for each band signal is a given "just noticeable noise" energy threshold for said band signal. Procédé selon la revendication 10, dans lequel le seuil pour chaque signal dans une bande est un seuil d'énergie de "bruit à peine perceptible" donné pour ce signal dans une bande. Verfahren nach Anspruch 10, bei dem der Schwellenwert für jedes Bandsignal ein gegebener Energieschwellenwert für gerade wahrnehmbares Rauschen für das Bandsignal ist.
- 13A method as claimed in claim 12 wherein said step of selecting, is considering the selection of a band signal, subtracts the given just noticeable noise energy of band signal from the signal energy level of the band signal. Procédé selon la revendication 12, dans lequel l'étape de sélection, considèrant la sélection d'un signal dans une bande, soustrait l'énergie du bruit à peine perceptible donnée du signal dans la bande du niveau d'énergie de signal du signal dans la bande. Verfahren nach Anspruch 10, bei dem der Auswählschritt die Auswahl eines Bandsignalls betrachtet und die gegebene, gerade wahrnehmbare Rauschenergie des Bandsignals von dem Signalenergiepegel des Bandsignals subtrahiert.
- 14A method as claimed in claim 1 wherein said signal is an analog signal representing an image, and said step of dividing divides said signal into two-dimensional spectrum bands. Procédé selon la revendication 1, dans lequel le signal est un signal analogique représentant une image, et l'étape de division divise le signal en des bandes spectrales bidimensionnelles. Verfahren nach Anspruch 1, bei dem das Signal ein Analogsignal ist, das ein Bild darstellt, und bei dem der Unterteilungsschritt das Signal in zweidimensionale Spektralbänder unterteilt.
- 15A method as claimed in claim 1 wherein said signal is an analog signal representing a three dimensional image, and said step of dividing divides said signal into three-dimensional spectrum bands. Procédé selon la revendication 1, dans lequel le signal est un signal analogique représentant une image tridimensionnelle, et l'étape de division divise le signal en des bandes spectrales tridimensionnelles. Verfahren nach Anspruch 1, bei dem das Signal ein Analogsignal ist, das ein dreidimensionales Bild darstellt, und bei dem der Unterteilungsschritt das Signal in dreidimensionale Spektralbänder unterteilt.
- 16A method as claimed in claim 1 wherein said signal is an analog signal representing a sequence of images and thereby defining a three dimensional surface, and said step of dividing divides said signal into three-dimensional spectrum bands. Procédé selon la revendication 1, dans lequel le signal est un signal analogique représentant une séquence d'images et définissant ainsi une surface tridimensionnelle, et l'étape de division divise le signal en des bandes spectrales tridimensionnelles. Verfahren nach Anspruch 1, bei dem das Signal ein Analogsignal ist, das eine Folge von Bildern darstellt und dadurch eine dreidimensionale Oberfläche definiert, und bei dem der Unterteilungsschritt das Signal in dreidimensionale Spektralbänder unterteilt.
Independent claims16
43 paragraphs, as filed
This invention relates to methods for coding a signal.
The processing of signals for transmission often includes sampling of the input signal, quantizing the samples and generating a set of codes that represent the quantized samples. Most signals of interest (e.g., such as in speech or video signals) are highly correlated, which means that the signal can be thought of comprising a predictable component and an unpredictable component. Coding compression is achieved by encoding essentially only the unpredictable component. Moreover, since these signals are often destined to be received and perceived by humans, concepts that relate to the human perception of the information received have been employed to further compress the coding of such signals and, consequently, the rate of the transmitted signals.
In connection with both speech and video signals, the prior art coding approaches that most closely relate to this invention are transform coding and linear predictive coding.
In a communications system utilizing transform coding, the signal is divided into segments. The segments are sampled and the samples of a segment are transformed into a set of frequency domain transform coefficients. The coefficient signals are then quantized and applied to the transmission channel. In systems that account for noise perception characteristics, the quantization mode applied to the coefficients is made to depend on the signal characteristics and on the sensitivity of the recipient to the resulting quantization noise, achieving thereby coding efficiency. Superimposed on those considerations is the limited bandwidth that is available. Bit allocation is one approach for handling the available bandwidth. In this approach, bits are allocated to the encoding of the transform coefficients in a manner that attempts to achieve a constant bandwidth. Examples of transform coding are found, among others, in U.S. Patent 4,949,383, U.S. Patent 4,184,049, an article by J.D. Johnston titled "Transform Coding of Audio Signals Using Perceptual Noise Criteria", IEEE Journal on Selected Areas in Communications, Vol. 6, No. 2., February 1988, etc.
Linear predictive coding in the speech environment dates back to the mid 1960's. The article by B.S. Atal and M.R. Schroeder titled "Predictive Coding of Speech Signals", Proceedings of the 1967 Conference on Communications and Processing, Cambridge, MA, pp 360-361, is an early example of that. Later, it has been recognized that predictive coding may be improved by taking account of the not unlimited ability to perceive noise. For example, the article by M.R. Schroeder, B.S. Atal and J.L. Hall titled "Optimizing Digital Speech Coders by Exploiting Masking Properties of the Human Ear", Journal of the Acoustical Society of America, December 1979, pp 1647-1652, describes the benefits that may accrue from considering the perceptual characteristics of the human ear.
In Linear Predictive Coding (LPC) that accounts for the perception of noise, a signal segment is predicted from historical information, and an error signal is derived by subtracting the predicted signal from the actual signal. The error signal is typically transformed and weighted by a noise-perception frequency-sensitive function, to result in a modified transform. The modified transform is encoded and transmitted to the receiver.
In the area of video signals the situation is not dissimilar. For example, sub-band coding was applied to image signals by J. Woods and S.D. O'Neil in "Sub-Band Coding of Images", IEEE ASSP, Vol 34, No. 5, October 1986, pp 1278-1288. The arrangement proposed by Woods et al. divides the image into two-dimensional frequency bands and the signal of each band is compressed via DPCM. Two-dimensional frequency bands, in effect, measure the signal variability in the two dimensions that form the image. Vector quantization of video is described, for example, in "Sub-Band Coding of Images Using Vector Quantization" by P.H. Westerink et al., Proc. of Seventh Benelux Information Theory Symposium, pp. 143-150, 1986; and in U.S. Patent 4,811,112 issued to C.W. Rutledge on March 7, 1989. The "human visual system" (HVS) characteristics were incorporated by K.N. Ngan, et al. in an article titled "Cosine Transform Coding Incorporating Human Visual System Model", SPIE Vol 707, Visual Communications and Image Processing (1986) pp. 165-171. The system described by Ngan et al. basically executes a two-dimensional cosine transform on the source information and weights the derived coefficients in accordance with an HVS function. The weighted coefficients are then quantized, coded and sent to a buffer prior to being applied to the transmission medium. To insure a desired global bit rate, a buffer fullness indication is fed back to the quantizer to control the number of bits that are generated by the quantizer. More recently, is a co-pending application Serial No. 07/350435, filed May 4, 1989, publication number US-A-5 309 526, published 3 May 1994; J.D. Johnston and R.J. Safranek disclosed a sub-band analysis method where the quantization schema for each pixel is adapted so that the amount of quantizing noise that is produced is near, but below, the limit of perceptibility. By allowing the quantization noise to rise while still keeping it below perceptibility, greater compression of the signal is achieved.
The above-described coding approaches operate with sampled and quantized signals. To achieve a more compressed code, prior art approaches typically transform the signal to the frequency domain and thereafter operate in that domain. Given a fixed bandwidth, they allocate the available bits between the different frequency components to do as good a job as possible on all of the frequency components or on a prespecified number of them. In other words, the decision that is made is how well to encode the frequency coefficients; not whether to encode them in the first instance. The result is an encoding schema that is more complex than necessary and, when the total bit rate is constrained, is perceptually suboptimal.
EP-A-0 193 143 discloses a coding scheme in which the signal is first divided into bands. Next, the absolute maximum of the entire frequency spectrum is encoded, then the maxima of each band is encoded, relative to the first maximum. Thereafter, each band is divided into sub-bands. In encoding the amplitude values of the sub-bands, the amplitude is quantized into levels, down from the maximum of the band to which the sub-bands belong. Finally, a two bit code is transmitted to designate the range in which the amplitude of the sub-band resides. Those amplitudes that are in the lowest amplitude range get the 00 code, corresponding to a setting to zero, and are therefore effectively discarded.
WO-A-89/10661 discloses a coding scheme which selects for coding and transmission four bands from a plurality of bands, the four bands selected having the highest absolute amount of energy.
According to one aspect of this invention there is provided a method as claimed in claim 1.
According to another aspect of this invention there is provided a method as claimed in claim 10.
The underlying principle of this invention recognizes that good performance is achieved through a simple decision of whether or not to encode certain frequency bands; not how well to encode all or a prescribed number of the frequency bands. Recognizing that an analog signal has a baseband frequency spectrum, and that corresponding to the signal's frequency spectrum there is a "just noticeable difference" (jnd) noise spectrum, it is clear that since signal frequency bands that dip below the jnd spectrum cannot be perceived anyway there is no need to transmit those frequency bands. Furthermore, when the available bandwidth is limited, it makes sense to concentrate on transmitting only those of the signal's frequency bands that most exceed the jnd spectrum.
The jnd spectrum is applied to a decision circuit that selects a number of frequency bands of the signal to be encoded. Each of the selected bands is encoded and transmitted to the receiver.
In one embodiment, for example, the analog input signal is separated into bands through a transform circuit which, in effect, is a bank of bandpass filters. The jnd level within each band is evaluated and a quotient signal is developed for each band which corresponds to the input signal in the band, divided by the jnd signal in the band. A selection circuit identifies the n bands having the highest quotient, and when an analog signal transmission is desired, an inverse transform develops n analog signals which are modulated to form a baseband having a contiguous spectrum. The baseband signal is then directly modulated onto a carrier. When digital transmission is preferred, the outputs of the selection are formatted and applied to the transmission medium.
<u>Brief Description of the Drawing</u>
<ul id="ul0001" list-style="none" compact="compact"><li>FIG. 1 illustrates the spectrum of a signal and the spectrum of noise that might be barely perceived in the presence of the signal;</li><li>FIG. 2 presents the architecture of a coder/transmitter</li><li>FIG. 3 details the structure of selector 120 of FIG. 2;</li><li>FIG. 4 presents the schematic diagram of a switch used in selector 120;</li><li>FIG. 5 presents the structure of a decoder/receiver adapted to the signals developed by the coder/ transmitter of FIG. 2; and</li><li>FIG. 6 presents one embodiment for developing the jnd values in an video signal environment;</li></ul>
<u>Detailed Description</u>
A band-limited time varying signal can be represented by a finite frequency spectrum. Typically, the spectrum is very jagged within the band because information-laden signals don't generally contain the entire spectrum of frequencies. When bands of signals are considered a form of averaging occurs and the spectrum of such a signal is less jagged, so it may have a contour like the one depicted by curve 10 in FIG. 1 (also denoted by the letter S). Curve 10 may represent, for example, a speech signal. A noise signal (one that sounds like noise when it is an audio signal, or looks like a salt and pepper mixture when it is a video signal) typically contains all of the possible frequency components, and has a frequency spectrum that varies very slowly with frequency. A flat spectrum contour is typically referred to as "white noise".
Researchers have established that there exists a threshold below which a noise signal cannot be perceived by most people. This threshold varies with frequency. Because of masking properties, this threshold also varies with the spectrum of the information signal that exists in the presence of the noise signal. This masking phenomenon is well known even to lay people. Noise-like artifacts in an image of a blank wall are easily detected, whereas the same artifacts in an image of a jungle are not detected.
The exact relationship of the detectable noise to the information signal is not important to the principles of this invention, so only an illustrative relationship is depicted in FIG. 1. Curve 20 in FIG. 1, also denoted by jnd, illustrates the variation in the noise threshold as a function of frequency (the jnd spectrum), for the speech signal represented by curve 10. D in FIG. 1 denotes the energy difference between the speech signal and the perceived noise threshold. Actually, the ordinate of the FIG. 1 graph has a log scale, so the depicted difference signal D is really a log of the quotient S/jnd.
As indicated above, jnd spectrum is employed in the prior art merely as a means for modifying, or weighting the frequency coefficients of S, prior to coding of the modified spectrum. Through bit allocation and/or through quantization mode control (control of the number of bits used to quantize the signal) artisans have attempted to do the best job possible in encoding the modified frequency coefficients.
In contradistinction, the jnd spectrum enters into the decision of whether or not to encode the frequency bands; not how to encode them. Additionally, the encoding of this invention is adapted to achieve a constant perceived quality encoding process.
The spectrum of the signal to be encoded is divided into N frequency bands, and a jnd spectrum is computed and applied to a decision means that selects n out of the N bands. Both N and n are parameters that are under designer control. The resulting compression ratio is N/n, which means that for a given N, a smaller value of n yields a greater level of compression. Of course, a greater level of compression also results in lower fidelity at a receiver that is connected to the transmission medium. It can be shown that for a given ratio n/N, the fraction of the retained energy is greater for larger values of N (a finer and hence better selection is made of the peaks in the signal's spectrum). On the other hand, the value of N may be limited by the amount of hardware that the designer is willing to specify.
Whereas the principles of this invention can be implemented with analog circuits (or at least analog circuits and some switching, or sampling), in recognition of the fact that digital implementations predominate today's designs, the following discussion presents a digital embodiment.
In FIG. 2, the input signal is assumed to be a train of samples. This input signal is applied to analysis filter bank 100 via a serial to parallel register 105. Filter bank 100 receives a set of input signal samples with the arrival of every N samples at register 105 and develops therefrom N frequency coefficients. Filter bank 100 may be implemented in a number of ways, such as with a cosine transform circuit, or with generalized quadrature mirror filters (GQMF). See, for example, J. Makhoul, "A Fast Cosine Transform in One and Two Dimensions," IEEE Trans. Acoustics, Speech and Signal Processing, C. ASSP-28, No. 1, Feb. 1980, pp 27-34; and R.V. Cox "The Design of Uniformly and Nonuniformly Spaced Pseudo Quadrature Mirror Filters", IEEE Trans. ASSP, Vol. ASSP-34, No. 5, October 1986, pp. 1090-1096.
The output of filter bank 100 is applied to perceptual model block 110. The function of block 110 is to develop the jnd band signals of FIG. 1. The manner in which those signals are generated is strictly a function of the perceptual model selected and, in fact, while FIG. 2 shows block 110 to be responsive to the output of filter bank 100, it should be appreciated that some perceptual models may call for a connection between the input to bank 100 and the input to block 110, in addition to, or instead of, the connection of the output of bank 100 to block 110.
The concept of a perceptual sensitivity to noise has been studied by a number of researchers. See, for example, R. P. Hellmans, "Asymmetry of masking between noise and tone," Percept. and P Psychophys., Vol 11, pp. 241-246, 1972. Another article that is very informative is "Transform Coding of Audio Signals Using Perceptual Noise Criteria," by J. D. Johnston, IEEE Journal on Selected Areas in Communications, Vol 6, No. 2, Feb 1988, pp 314-323. Building on this information, in a copending application titled Perceptual Coding of Audio Signals, Serial No. 07/423,088, filed October 18, 1989 publication numbers US-A-5 040 217, published 13 Aug. 1991, one actual coder design is disclosed which includes a means for developing the threshold values; i.e., a perceptual motel 110. The teachings of this application are hereby identified by reference. For sake of completeness, however, a FORTRAN program is included herein and summarized below, which develops the outputs of perceptual coder 110 as taught by the incorporated application. A useful reference for understanding the FORTRAN program is FX/FORTRAN Programmer's Handbook, Alliant Computer Systems Corp., July 1988. Tables 1 and 2 present a list of constants used in connection with the illustrative program of Listing 1, such as the absolute thresholds used and the band definitions.
The program comprises primarily the "strt" routine and the "calcthri" routine. The "strt" routine is called initially to calculate various parameters. The "calcthri" routine is called thereafter, with every new block of sampled data, and it develops the output set labeled "ratio". This output set is actually representative of the ratio <maths id="math0001" num=""><math display="inline"><mrow><mfrac><mrow><mtext>S</mtext></mrow><mrow><mtext>jnd</mtext></mrow></mfrac></mrow></math><img file="EP0508581B1_D0001.tif" /></maths> for the different defined frequency bands and it eliminates the need for the separate dividers that are shown in FIG. 2. Noting that the listing's final step in developing the "ratio" is a division (as expected), and that divisions are time consuming, a better realization employs hardware dividers as shown in FIG. 2.
As indicated above, the FORTRAN program included herein computes the <maths id="math0002" num=""><math display="inline"><mrow><mfrac><mrow><mtext>S</mtext></mrow><mrow><mtext>jnd</mtext></mrow></mfrac></mrow></math><img file="EP0508581B1_D0002.tif" /></maths> measure of sensitivity to the perception of noise. Other measures can also be employed, such as<maths id="math0003" num=""><math display="block"><mrow><mfrac><mrow><msup><mrow><mtext>(S - jnd)</mtext></mrow><mrow><mtext>2</mtext></mrow></msup></mrow><mrow><mtext>jnd</mtext></mrow></mfrac><mtext> , </mtext><mfrac><mrow><mfenced open="|" close="|"><mrow><mtext>S - jnd</mtext></mrow></mfenced></mrow><mrow><mtext>jnd</mtext></mrow></mfrac><mtext>,</mtext></mrow></math><img file="EP0508581B1_D0003.tif" /></maths> or the logarithm of any of the above.
FIG. 2 illustrates an embodiment where, as suggested above, the ratio <maths id="math0004" num=""><math display="inline"><mrow><mfrac><mrow><mtext>S</mtext></mrow><mrow><mtext>jnd</mtext></mrow></mfrac></mrow></math><img file="EP0508581B1_D0004.tif" /></maths> in implemented with hardware dividers. Thus, in FIG. 2 the signal of coefficient i where i= 1,2...N, at the output of filter bank 100 (i.e., signal S<sub>i</sub>) is divided in circuit 111-i by the jnd output of block 110 that corresponds to coefficient i (i.e., signal jnd<sub>i</sub>), to develop thereby a quotient signal q<sub>i</sub>. Circuits 111-i are identical read-only lookup tables that develop the quotient signaL With an 8 bit S<sub>i</sub> and an 8 bit jnd<sub>i</sub>, each lookup table is merely a 2<sup>16</sup> memory. (If time permits, a single memory can be shared.) The output of each of the divider circuits and its associated S<sub>i</sub> signal is applied to selector circuit 120. The identity, i, of each signal is also applied to circuit 120. Circuit 120 thus receives N triplets of signals, and its function is to select the n signals S<sub>i</sub> (and their identities) that are associated with the n largest quotient signals q<sub>i</sub>. The identity information is typically referred to as "side information". As will become apparent hereinafter, this information must be sent to the receiver.
In a digital environment, the signals chosen by selector 120 and their identities are formatted (and perhaps further encoded) in formatter 125 and transmitted to a remote receiver. This is depicted in FIG. 2 by the line marked "DIGITAL". The formatting may be done in a number of ways. One way is to send the information in the form of packets. That is, each signal identity is placed in a header field, and the corresponding coefficient signal is placed in the immediately following data field.
There are situations, however, where it is desirable to employ an analog transmission medium. In such a case, it is useful to convert the coefficients chosen by selector or 120 to a narrow band analog signal with a bandwidth that reflects the compression achieved by the system.
The latter is achieved in the FIG. 2 circuit with inverse transform circuit 130, a parallel to serial register pair 140 and 141, a low pass filter 150, and a formatter 126. Most advantageously, inverse transform circuit 130 is of the type used to realize the transform circuit of bank 100, except that it needs to respond only to sets of n inputs rather than to sets of N inputs. It develops n signals which correspond to time samples. Those time samples are shifted out serially through register 140 and filtered to excise the high frequency components with low pass filter 150. Of course, the clock of register 140 is <maths id="math0005" num=""><math display="inline"><mrow><mfrac><mrow><mtext>N</mtext></mrow><mrow><mtext>n</mtext></mrow></mfrac></mrow></math><img file="EP0508581B1_D0005.tif" /></maths> times slower than the clock of register 105. The developed baseband analog signal is applied to formatter 126 where it is modulated onto a carrier in preparation for transmission. Concurrently, the side information is shifted out to formatter 126 through register 141 where it is encoded, via pulse amplitude modulation, for example, to develop a second analog signal that is modulated onto a carrier in preparation for transmission. The analog transmission is depicted in FIG. 2 by the line marked "ANALOG". It is expected, of course, that either one of the two modes (analog or digital transmission) will be employed. Both are shown in FIG. 2 merely for illustrative purposes.
FIG. 3 depicts one implementation for selector circuit 120. It is based on the Batcher network. See, for example, U.S. Patent 3,428,946 issued February 18, 1969. In that patent, Batcher explicitly teaches how to sort arbitrarily arranged sets of four inputs and eight inputs. He also taught how to merge two bitonic sequences into a single sorted sequence. Applying these teachings directly, FIG. 3 includes four 4-input sort networks 127 and two bitonic merge networks 128. They are arranged to form a bitonic of sequence of length 16 where the top eight inputs are descending in the sort key (the q<sub>i</sub> values) and the bottom eight inputs are ascending in the sort key. These networks are structured according to the teachings of Batcher, using switches 125 as described below. The bitonic sequence developed by networks 128 is applied to a modified bitonic merge network 129. Unlike bitonic merge network 128 which contains a log<sub>2</sub>M number of stages, where M is the number of inputs of the network, the number of stages in merge network 129 is log<sub>2</sub>(M/ n). In FIG. 3, where n is 4 and M= N= 16, the number of stages is log<sub>2</sub> (16/ 4) or 2. Also, because many of the outputs are not utilized, each of the switches 125 within merge network 129 can be simplified, compared to the canonic design of switches 125 within networks 127 and 128.
The structure of switch 125 is presented in FIG. 4. Each switching block 125 receives two input coefficient signals S<sub>m</sub> and S<sub>n</sub>, the corresponding two quotient signals q<sub>m</sub> and q<sub>n</sub> and the signal identity values m and n. The function of each switching block 125 is to ascertain which quotient signal is larger and to route that quotient signal, its corresponding coefficient signal, and the identity value, to a selected one of two outputs of three respective switch elements. This is accomplished with a subtractor 121 that is responsive to the quotient signals and with three "double pole - double throw" switch elements 122, 123 and 124 that are responsive to the output of subtractor 121. Switch 122 routes the quotient signals, switch 123 routes the coefficient signals and switch 124 routes the identity values. It may be noted that at the last stage of selector 120, the quotient signals need not be routed because no further decisions need to be made. Also, when log<sub>2</sub> N/n stages are employed in network 129, the n outputs of the network are not ordered; they are only guaranteed to be the ones that correspond to the signals with the largest quotient signals.
Needless to say, the timing of the selection process of circuit 120 must be controlled. It is not useful, for example, to allow circuit 120 to effect a different selection too frequently. Each selection provides a new set of side information that must be sent to the receiver, and that overhead should be kept to a minimum. Accordingly, selector circuit 120 includes a register 126 at the output of network 129 that is used to capture and store each selection. The question is what schema to apply to the clock of register 126. The simplest approach is to use a constant clock. This provides a constant bandwidth to the side information. The clock can be a N<sup>th</sup> sub-multiple of the sampling clock of the input signal, or perhaps a KN<sup>th</sup> sub-multiple of the sampling clock of the input signal, where K is an integer. A third alternative uses a small value of K (perhaps even K= 1) and at each appearance of the clock a decision is made as to whether or not a sufficient benefit results from changing the selection. This can be done with conventional circuitry (not shown in FIG. 3) that measures the ratio of the selected quotient signals to the unselected quotient signals. By comparing the ratio of the selections at the output of selector 120 to the ratio of the selections at the input of selector 120 a decision whether or not to allow a change in the selection can be made based on the observed difference in the ratios.
FIG. 5 depicts a receiver in conformance with the digital transmission approach of the FIG. 2 transmitter. Since it receives sets of n packets that are not necessarily sorted by frequency, the input signals are applied to a serial to parallel converter 200 which, while keeping each packet in serial form, applies the n packets, in parallel, to sorter 210. Sorter 210 sorts its input signals based on the header field, which identifies the frequency coefficients. The sorted outputs are applied to expander network 220 which routes the signals to appropriate outputs of the expander network. Sorter 210 may be a batcher network, as described above, except that the sorting is keyed to the identifier, i, and not on the quotient signals q<sub>i</sub>. Expander network 220 may be a shuffle exchange network as described, for example, is U.S. Patent 4,516,238 issued to A. Huang and S.C. Knauer on May 7, 1985.
It may be noted in passing that the selection process carried out by selector circuit 120 is tantamount to a selection of frequency bands and a down-shifting to baseband of the selected frequency bands. The operation of sorter 210 is a sorting of the selected bands, and the operation of expander 220 is tantamount to an up-shifting of the sorted frequency bands to their appropriate place.
The output signals of expander 220 are applied to inverse transform circuit 230. Circuit 230 is akin to circuit 130, except that it is responsive to N inputs (although only n of them are non-zero) rather than to n inputs. The N outputs of inverse transform circuit 230 are serially shifted out via parallel to serial register 240 and filtered through low pass filter 250 to yield the final reconstructed signal.
For sake of simplicity the above description concentrates on "one dimensional" signals, such as speech. It should be clearly understood, however, that the principles of this invention apply quite well to higher dimensional signals. With video signals, for example (which can be thought to be two dimensional), the only change that needs be made is in filter bank 100, in inverse transform circuits 130 and 230, and in the perceptual model circuit 110. The modifications that relate to the two-dimensional aspects s of video signals are described in the aforementioned 07/ 350435 application. Both the perceptual model and the two-dimensional transform processes are described.
Again, for sake of completeness, the following describes in summary form one embodiment for perceptual model 100 in the video environment. In FIG. 6, N inputs are received from filter bank 100. One of them represents the band where the two dimensional "dc value" of the frame is found. All bands other than dc value band are applied to lookup tables 301. Each of the lookup tables develops a weighted measure of the power in the band. That is, each of the lookup tables develops a value of k<sub>i</sub>S<sup>2<sub2>i</sub2></sup> where k<sub>i</sub> may be different for different values of i. The computed weighted power measures represent an estimate of the visual "texture" of the image at that frequency band. Adder 302 sums the "texture" outputs of the N-1 tables 301 to produce an overall texture estimate. That estimate is applied to lookup table 303 which transforms the power domain output of adder 302 to an amplitude domain masking threshold. The mapping function within table 303 is a log-like function, which reduces the dynamic range at the output of table 303. A brightness correction is introduced in lookup table 304, which multiplies the input from table 303 by the "dc value" band. Lastly, lookup table 305 multiplies the masking threshold developed by table 304 by a set of constants, where each constant is related to the noise sensitivity of the human visual system in each of the frequency bands. The N outputs thus developed form the set of outputs that are applied by perceptual model 100 to dividers 111 in FIG. 2.
Extension to three dimensional cases, such as a time succession of video signal frames or a three-dimensional topographical map, is also straight forward in accordance with the well known principles by which the extension is made from one dimension to two dimensions.<img file="EP0508581B1_D0006.tif" /><img file="EP0508581B1_D0007.tif" /><img file="EP0508581B1_D0008.tif" /><img file="EP0508581B1_D0009.tif" /><img file="EP0508581B1_D0010.tif" /><img file="EP0508581B1_D0011.tif" /><img file="EP0508581B1_D0012.tif" /><img file="EP0508581B1_D0013.tif" /><img file="EP0508581B1_D0014.tif" /><img file="EP0508581B1_D0015.tif" /><img file="EP0508581B1_D0016.tif" /><img file="EP0508581B1_D0017.tif" /><img file="EP0508581B1_D0018.tif" /><img file="EP0508581B1_D0019.tif" /><img file="EP0508581B1_D0020.tif" /><img file="EP0508581B1_D0021.tif" /><img file="EP0508581B1_D0022.tif" /><tables id="tabl0001" num="0001"><img file="EP0508581B1_D0023.tif" /></tables><tables id="tabl0002" num="0002"><img file="EP0508581B1_D0024.tif" /></tables><tables id="tabl0003" num="0003"><img file="EP0508581B1_D0025.tif" /></tables><tables id="tabl0004" num="0004"><img file="EP0508581B1_D0026.tif" /></tables>
29 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| WO8910661A1 | Cites | World Intellectual Property Organization (WIPO) | Examiner |
| EP0176243A | Cites | European Patent Office (EPO) | – |
| EP0193143A | Cites | European Patent Office (EPO) | – |
| WO8910661A | Cites | World Intellectual Property Organization (WIPO) | – |
13 members in 7 offices
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 667851 | United States of America | – | |
| 66785191 | United States of America | A | |
| 66785191 | United States of America | A | |
| 667851 | – | – | – |
| US19910667851 | – | – | – |
Members13
| Document | Office | Kind | |
|---|---|---|---|
| CA2061366A1 | Canada | A1 | |
| EP0508581A2 | European Patent Office (EPO) | A2 | |
| KR920019190A | Republic of Korea | A | |
| EP0508581A3 | European Patent Office (EPO) | A3 | |
| JPH05122166A | Japan | A | |
| TW240354B | Taiwan Province of China | B | |
| JPH0779312B2 | Japan | B2 | |
| US5559900A | United States of America | A | |
| EP0508581B1This record | European Patent Office (EPO) | B1 | |
| CA2061366C | Canada | C | |
| DE69219943D1 | Germany | D1 | |
| DE69219943T2 | Germany | T2 | |
| KR100273071B1 | Republic of Korea | B1 |
26 legal events, as 3 offices reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | Office | |
|---|---|---|---|
| Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25 | PG25 | EP | |
| Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25 | PG25 | EP | |
| Patent expired after termination of 20 yearsExpiredPE20 | PE20 | GB | |
| Expiry of rightR071 | R071 | DE | |
| Expiry of rightR071 | R071 | DE | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| European patent in force as of 2002-01-01IF02 | IF02 | GB | |
| No opposition filedOpposition26N | 26N | EP | |
| No opposition filed within time limitOppositionORIGINAL CODE: 0009261PLBE | PLBE | EP | |
| Information on the status of an ep patent application or granted ep patentGrantedSTATUS: NO OPPOSITION FILED WITHIN TIME LIMITSTAA | STAA | EP | |
| Fr: translation filedET | ET | EP | |
| Corresponds to:REF | REF | EP | |
| Designated contracting statesAK | AK | EP | |
| (expected) grantORIGINAL CODE: 0009210GRAA | GRAA | EP | |
| Despatch of communication of intention to grant a patentORIGINAL CODE: EPIDOS IGRAGRAH | GRAH | EP | |
| Despatch of communication of intention to grant a patentORIGINAL CODE: EPIDOS IGRAGRAH | GRAH | EP | |
| Despatch of communication of intention to grantORIGINAL CODE: EPIDOS AGRAGRAG | GRAG | EP | |
| First examination report despatched17Q | 17Q | EP | |
| Party data changed (applicant data changed or rights of an application transferred)RAP3 | RAP3 | EP | |
| Request for examination filed17P | 17P | EP | |
| Designated contracting statesAK | AK | EP | |
| Search report despatchedORIGINAL CODE: 0009013PUAL | PUAL | EP | |
| Designated contracting statesAK | AK | EP | |
| Public reference made under article 153(3) epc to a published international application that has entered the european phaseORIGINAL CODE: 0009012PUAI | PUAI | EP |
Numbers
- Publication
- 0508581
- Publication, DOCDB
- 0508581
- Publication, EPODOC
- EP0508581
- Application
- 92301778
- Application, DOCDB
- 92301778
- Application, EPODOC
- EP19920301778
Titles3
- German
- Signalkompression unter Verwendung eines Empfindungsmodells
- English
- Compression of signals using a perceptual model
- French
- Compression de signaux utilisant un modèle perceptif
Classification
- CPC, 6
- H04B1/667
- H04N7/12
- H04N19/63
- H04N19/115
- H04N19/61
- H04N19/146
- IPC, 5
- H03M1 48
- G06T9 00
- H04B1 66
- H04B14 04
- H04N7 26
Designated states3
- Contracting states, 3
- Germany
- France
- United Kingdom
