Signal encoding method and signal decoding method

TECHNICAL FIELD [0001] One or more exemplary embodiments relate to the encoding and decoding of an audio or speech signal, in particular a method and apparatus for encoding and decoding a spectral coefficient in the frequency domain.

BACKGROUND OF THE INVENTION [0002] Quantization elements based on various schemes have been proposed for efficiently coding spectral coefficients in the frequency domain. For example, a quantization element based on TCQ (trellis coded quantization), uniform scalar quantization (USQ), pulse factor coding (FPC) was used factorial pulse coding), algebraic vector quantization (AVQ) and pyramid vector quantization (PVQ) and the like. Therefore, a lossless encoder optimized for each quantization element has also been implemented. Examples of such quantization elements can be found in WO2009 / 055493 A1

DISCLOSURE

TECHNICAL PROBLEMS [0003] One or more exemplary embodiments include a method and apparatus for adaptively encoding or decoding a spectral coefficient for different bit rates or different sizes of subbands in the frequency domain.

[0004] One or more exemplary embodiments include a durable, computer readable storage medium storing a program for performing a signal coding method or a signal decoding method.

[0005] One or more exemplary embodiments include a multimedia device employing a signal coding method or signal decoding method.

TECHNICAL SOLUTION [0006] According to an embodiment of the invention, a spectrum coding method according to claim 1 has been proposed. [0007] According to another embodiment of the invention, a spectrum decoding method according to claim 4 has been proposed.

BENEFIT EFFECTS [0008] According to one or more of the above exemplary embodiments, the spectral coefficient is adaptively encoded and decoded for different bit rates or different sizes of subbands.

DESCRIPTION OF THE DRAWINGS [0009]

FIG. 1A and 1B are block diagrams of audio coding devices and audio decoding devices according to an exemplary embodiment, respectively.

FIG. 2A and 2B are block diagrams of audio coding devices and audio decoding devices according to another exemplary embodiment, respectively.

FIG. 3A and 3B are block diagrams of audio coding devices and audio decoding devices according to another exemplary embodiment, respectively.

FIG. 4A and 4B are block diagrams of audio coding devices and audio decoding devices according to another exemplary embodiment, respectively.

FIG. 5 is a block diagram of an audio frequency encoding device according to an exemplary embodiment.

FIG. 6 is a block diagram of an audio decoding device in the frequency domain according to an exemplary embodiment.

FIG. 7 is a block diagram of a spectrum encoding device according to an exemplary embodiment.

FIG. 8 shows an example of sub-band division.

FIG. 9 is a block diagram of a spectrum quantization and coding apparatus according to an exemplary embodiment.

FIG. 10 is a diagram of an important spectral component (ISC) collection operation.

FIG. 11 depicts an example of TCQ used in the exemplary embodiment.

FIG. 12 is a block diagram of an audio decoding device in the frequency domain according to an exemplary embodiment.

FIG. 13 is a block diagram of a spectrum decoding device according to an exemplary embodiment.

FIG. 14 is a block diagram of an apparatus for decoding and dequantizing a spectrum according to an exemplary embodiment.

FIG. 15 is a block diagram of a multimedia device according to an exemplary embodiment.

FIG. 16 is a block diagram of a multimedia device according to another exemplary embodiment.

FIG. 17 is a block diagram of a multimedia device according to yet another exemplary embodiment.

METHOD OF IMPLEMENTING THE INVENTION [0010] Since the inventive idea may have a variety of modified embodiments, preferred embodiments are illustrated in the drawings and described in the detailed description of this inventive idea. However, this does not limit the inventive idea within specific embodiments and it should be understood that the inventive idea includes all modifications, equivalents and replacements within the technical scope of the inventive idea. Furthermore, detailed descriptions related to well-known functions or configurations will be excluded so as not to obscure the subject matter of this inventive idea unnecessarily.

[0011] It should be understood that although the first and second terms are used herein to describe various elements, these elements should not be limited by these terms. The terms are only used to distinguish one subassembly from other subassemblies.

[0012] In the following description, technical terms are used only to explain a specific exemplary embodiment, without limiting the inventive idea. The terms used in the inventive idea have been chosen as generic terms that are now widely used, taking into account the functions of the disclosed idea, but may be changed according to the intentions of those of ordinary skill in the field of technology, traditional practice or the introduction of new technology. In addition, if there is a concept that has been freely chosen by the applicant in a particular case, then in this case the meaning of this concept will be described in detail in the relevant part of the description of the invention. Thus, concepts should be defined on the basis of the entire content of this document instead of the simple name of each of these concepts.

[0013] Singular expressions may include plural expressions, unless otherwise specified. The meaning of "includes", "contains" or "has" defines a property, area, fixed number, stage, process, element and / or subassembly, but does not exclude other properties, areas, fixed numbers, stages, processes, elements and / or subassemblies .

[0014] In the following, exemplary embodiments will be described in detail with reference to the accompanying drawings. Similar numbers refer to similar elements throughout the description of the drawings, and there is no repeated description of the same element.

[0015] FIG. 1A and 1B are block diagrams of audio coding devices and audio decoding devices, according to an exemplary embodiment, respectively.

[0016] The audio coding device 110 shown in FIG. 1A may include preprocessor 112, frequency encoder 114 and parameter encoder 116. These components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0017] In FIG. 1A, the preprocessor 112 may perform filtering, downsampling or the like for an input signal, but is not limited to this. The input signal may include a speech signal, a music signal or a mixed speech and music signal. For the convenience of the explanation below, the input signal is called the audio signal.

[0018] The frequency domain encoder 114 may perform time-frequency conversion on the audio signal provided by the preprocessor 112, select an encoding tool according to the number of channels, coding band and bit rate of the audio signal, and encode the audio signal using the selected encoding tool. Time-frequency transformation can use modified discrete cosine transform (MDCT) modified discrete cosine transform), but is not limited to Fast Fourier transform (MLT) or Fast Fourier transform (FFT). If the number of data bits is sufficient, then the overall transformation coding scheme may be applied to the entire bands, and if the number of data bits is not sufficient, a bandwidth extension scheme may be applied to the partial bands. If the audio signal is a stereo or multi-channel channel, if the number of data bits is sufficient, the coding is carried out for each channel, and if the number of data bits is not sufficient, then a down-mixing scheme can be used. The encoded spectral coefficient is generated by the frequency domain encoder 114.

[0019] The parameter encoder 116 may extract the parameter from the encoded spectral coefficient provided from the frequency domain encoder 114 and encode the extracted parameter. The parameter can be extracted, for example, for each sub-band, which is a unit of grouping of spectral coefficients and can have an even or uneven length by reflecting the critical band. If each sub-band has an uneven length, then the sub-band existing in the low frequency band may have a relatively short length compared to the existing sub-band in the high frequency band. The number and length of subbands contained in one frame vary depending on the codec algorithms and may affect coding performance. The parameter may include, but is not limited to, scaling factor, power, average energy, or Norm value. The spectral coefficients and parameters obtained as a result of the encoding form a bit stream, and this bit stream can be recorded on the storage medium or can be sent in the form of, for example, packets over the channel.

[0020] The audio decoding device 130 shown in FIG. 1B may include a parameter decoder 132, frequency domain decoder 134 and postprocessor 136. The frequency domain decoder 134 may include a frame error concealing algorithm or a packet loss concealing algorithm. The components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0021] In FIG. 1B, the parameter decoder 132 can decode parameters from the received bit stream and check if an error has occurred, such as erasure or loss in frame units based on the decoded parameters. Various well known methods can be used to check for errors, and information on whether the current frame is a good frame or whether an erase or loss frame is provided to frequency domain decoder 134. For the convenience of explanation, below, the erase or loss frame is called the erroneous frame.

[0022] If the current frame is a good frame, the frequency domain decoder 134 can generate synthesized spectral coefficients by performing decoding using the general transformation decoding process. If the current frame is an erroneous frame, the frequency domain decoder 134 may generate synthesized spectral coefficients by repeating the spectral coefficients of the previous good frame (PGF) (ang. previous good frame) on an erroneous frame or by scaling PGF spectral coefficients using regression analysis, which will then be repeated on the erroneous frame, using the frame error concealing algorithm or the packet loss concealing algorithm. Frequency domain decoder 134 can generate a time domain signal by performing a frequency-time transformation on synthesized spectral coefficients.

[0023] The postprocessor 136 may perform filtering, upsampling or the like to improve audio quality with respect to the time domain signal provided from the frequency domain decoder 134, but is not limited thereto. Postprocessor 136 provides the reconstructed audio signal as an output signal.

[0024] FIG. 2A and 2B are block diagrams of audio coding devices and audio decoding devices according to another exemplary embodiment, respectively, which have a switching structure.

[0025] The audio coding apparatus 210 shown in FIG. 2A may include preprocessor unit 212, mode determining element 213, frequency domain encoder 214, time domain encoder 215 and parameter encoder 216. These components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0026] In FIG. 2A, because preprocessor 212 is substantially the same as preprocessor 112 in FIG. 1A, its description will not be repeated.

[0027] The mode determining element 213 may determine the coding mode, referring to the characteristics of the input signal. Mode setting element 213 may determine according to the characteristics of the input signal whether the coding mode suitable for the current frame is a speech mode or music mode, and may also determine whether the coding mode efficient for the current frame is a time domain mode or a frequency domain mode. The input signal characteristics can be seen using the short-term frame characteristics or the long-term characteristics of multiple frames, but are not limited to them. For example, if the input signal corresponds to a speech signal, then the coding mode may be specified as a speech mode or time domain mode, and if the input signal corresponds to a non-speech signal, i.e. a music signal or a mixed signal, then the coding mode may be specified as a music mode or frequency domain mode. Mode setting element 213 may provide output from preprocessor 212 to frequency domain encoder 214 when the input signal characteristics correspond to music mode or frequency domain mode and may provide preprocessor 212 output signal to time domain encoder 215 when the input signal characteristics correspond to the speech mode or mode time domains.

[0028] Because the frequency domain encoder 214 is substantially the same as the frequency domain encoder 114 of FIG. 1A, its description will not be repeated.

[0029] The time domain encoder 215 may perform code excited linear prediction (CELP) coding for the audio signal provided by preprocessor 212. More specifically, algebraic CELP may be used for CELP encoding, but CELP encoding is not limited to that. The encoded spectral coefficient is generated by the time domain encoder 215.

[0030] The parameter encoder 216 may extract the parameter from the encoded spectral coefficient provided from frequency domain encoder 214 or time domain encoder 215 and encode this extracted parameter. Because the parameter encoder 216 is substantially the same as the parameter encoder 116 of FIG. 1A, its description will not be repeated. The spectral coefficients and parameters obtained as a result of coding may form a bit stream together with information about the coding mode, and this bit stream may be transmitted as packets over the channel or may be recorded on a storage medium.

[0031] The audio decoding device 230 shown in FIG. 2B may include parameter decoder 232, mode determining element 233, frequency domain decoder 234, time domain decoder 235 and postprocessor 236. Each of the frequency domain decoder 234 and the time domain decoder 235 may include a frame error concealment algorithm or an algorithm for concealing packet loss in any corresponding field. The components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0032] In FIG. 2B, the parameter decoder 232 can decode parameters from the bit stream transmitted as packets and check if an error has occurred in the frame units based on the decoded parameters. Various well-known methods can be used to check for errors, and information about whether the current frame is a good frame or whether an erroneous frame is provided to frequency domain decoder 234 or time domain decoder 235.

[0033] The mode setting element 233 may check the coding mode information contained in the bit stream and provide the current frame to frequency domain decoder 234 or to time domain decoder 235.

[0034] Frequency domain decoder 234 may operate if the coding mode is music mode or frequency domain mode and generate synthesized spectral coefficients by performing decoding in the general transformation decoding process if the current frame is a good frame. If the current frame is an erroneous frame and the coding mode of the previous frame is music mode or frequency domain mode, frequency domain decoder 234 can generate synthesized spectral coefficients by repeating spectral coefficients from the previous good frame (PGF). previous good frame) in an erroneous frame or by scaling PGF spectral coefficients using regression analysis, and then repeating in an erroneous frame, using the frame error concealing algorithm or the packet loss concealing algorithm. Frequency domain decoder 234 may generate a time domain signal by performing frequency-time conversion on synthesized spectral coefficients.

[0035] Time domain decoder 235 may operate if the coding mode is a speech mode or a time domain mode and generate a time domain signal by performing decoding in the general CELP decoding process if the current frame is a regular frame. If the current frame is an erroneous frame and the coding mode of the previous frame is speech mode or time domain mode, time domain decoder 235 may perform a frame error concealment algorithm or a time domain hide packet loss algorithm.

[0036] The postprocessor 236 may perform, but is not limited to, filtering, upsampling or the like for the time domain signal provided from frequency domain decoder 234 or time domain decoder 235. The postprocessor 236 provides the reconstructed audio signal as an output signal.

[0037] FIG. 3A and 3B are block diagrams of audio coding devices and audio decoding devices according to another exemplary embodiment, respectively.

[0038] The audio coding apparatus 310 shown in FIG. 3A may include a preprocessor 312, a linear prediction (LP) analyzer 313, a mode determining element 314, a frequency domain excitation encoder 315, a time domain excitation encoder 316 and a parameter 317 encoder. These components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0039] In FIG. 3A, because preprocessor 312 is substantially the same as preprocessor 112 in FIG. 1A, its description will not be repeated.

[0040] The LP 313 analyzer can extract LP coefficients by performing an LP analysis for the input signal and generate an excitation signal from the extracted LP coefficients. The excitation signal may be provided to one of the frequency domain excitation encoder units 315 and the time domain excitation encoder 316 according to the coding mode.

[0041] Because the mode setting element 314 is substantially the same as the mode setting element 213 of FIG. 2A, its description will not be repeated.

[0042] The frequency domain excitation encoder 315 can operate if the coding mode is the music mode or frequency domain mode, and because the frequency domain excitation encoder 315 is substantially the same as the frequency domain encoder 114 of FIG. 1A except that the input signal is an excitation signal, its description is not repeated.

[0043] The time domain excitation encoder 316 can operate if the coding mode is a speech mode or time domain mode, and because the time domain coder unit 316 is substantially the same as the time domain encoder 215 of FIG. 2A, his description is not repeated.

[0044] The parameter encoder 317 may extract the parameter from the encoded spectral coefficient provided from the frequency domain excitation encoder 315 or the time domain excitation encoder 316 and encode this extracted parameter. Because the parameter encoder 317 is substantially the same as the parameter encoder 116 of FIG. 1A, his description is not repeated. The spectral coefficients and parameters obtained as a result of coding may form a bit stream together with information about the coding mode, and this bit stream may be transmitted as packets over the channel or may be recorded on a storage medium.

[0045] The audio decoding device 330 shown in FIG. 3B may include a parameter decoder 332, a mode determining element 333, a frequency domain excitation decoder 334, a time domain excitation decoder 335, a LP synthesizer 336 and a postprocessor 337. Each of the frequency domain excitation decoder 334 and the time domain excitation decoder 335 may include a frame error concealment algorithm or an algorithm for concealing packet loss in any relevant field. The components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0046] In FIG. 3B, parameter decoder 332 can decode parameters from a bit stream transmitted as packets and check if an error has occurred in the frame units based on the decoded parameters. Various well-known methods can be used to check for errors, and information about whether the current frame is a good frame or an erroneous frame is provided to the frequency domain excitation decoder 334 or the time domain excitation decoder 335.

[0047] The mode setting element 333 may check the coding mode information contained in the bit stream and provide the current frame to the frequency domain excitation decoder 334 or to the time domain excitation decoder 335.

[0048] The frequency domain excitation decoder 334 may operate if the coding mode is music mode or frequency domain mode and generate synthesized spectral coefficients by performing decoding in the general transformation decoding process if the current frame is a good frame. If the current frame is an erroneous frame and the coding mode of the previous frame is music or frequency domain mode, the frequency domain excitation decoder 334 can generate synthesized spectral coefficients by repeating the spectral coefficients from the previous good frame (PGF). previous good frame) in an erroneous frame or by scaling PGF spectral coefficients using regression analysis, and then repeating in an erroneous frame, using the frame error concealing algorithm or the packet loss concealing algorithm. The frequency domain excitation decoder 334 can generate an excitation signal which is a time domain signal by performing a time-frequency transformation on synthesized spectral coefficients.

[0049] The time domain excitation decoder 335 can operate if the coding mode is a speech mode or a time domain mode and generate an excitation signal which is a time domain signal by performing decoding in the general CELP decoding process if the current frame is a good frame. If the current frame is an erroneous frame and the coding mode of the previous frame is speech mode or time domain mode, then the time domain excitation decoder 335 may perform a frame error hiding algorithm or a time domain hiding packet loss algorithm.

[0050] The LP synthesizer 336 may generate a time domain signal by performing LP synthesis for an excitation signal provided from a frequency domain excitation decoder 334 or a time domain excitation decoder 335.

[0051] The postprocessor 337 may perform, but is not limited to, filtering, upsampling or the like for a time domain signal provided from the LP 336 synthesizer. The 337 postprocessor provides the reconstructed audio signal as an output signal.

[0052] FIG. 4A and 4B are block diagrams of audio coding devices and audio decoding devices according to another exemplary embodiment, respectively, which have a switching structure.

[0053] The audio coding apparatus 410 shown in FIG. 4A may include a preprocessor 412, mode setting element 413, frequency domain encoder 414, LP analyzer 415, frequency domain excitation encoder 416, time domain excitation encoder 417 and parameter encoder 418. These components can be integrated in at least one module and can be implemented as at least one processor (not shown). Because the audio coding device 410 shown in FIG. 4A is obtained by connecting the audio encoding device 210 of FIG. 2A and audio coding devices 310 of FIG. 3A, the description of the operations of the common parts has not been repeated and the operation of the mode determination unit 413 will now be described.

[0054] The mode setting element 413 may determine the encoding mode of the input signal, referring to the characteristics and bit rate of the input signal. Mode setting element 413 may determine the coding mode as CELP or other mode based on whether the current frame is a speech mode or a music mode according to the characteristics of the input signal and based on whether the coding mode efficient for the current frame is a time domain mode or a mode frequency domain. Mode setting element 413 may set the coding mode as CELP if the input signal characteristics correspond to the speech mode, specify the coding mode as frequency domain mode, if the input signal characteristics correspond to the music mode and high bit rate, and specify the coding mode as audio mode if the signal characteristics input corresponds to music mode and low bit rate. Mode setting element 413 may provide input to frequency domain encoder 414 if the coding mode is frequency domain mode, provide input to frequency domain exciter encoder 416 via the LP 415 analyzer, if the coding mode is audio mode and provide input to encoder 417 time domain excitation via the LP 415 analyzer if the coding mode is CELP.

[0055] The frequency domain encoder 414 may correspond to the frequency domain encoder 114 in the audio encoding apparatus 110 of FIG. 1A or the frequency domain encoder 214 in the audio encoding apparatus 210 of FIG. 2A and frequency domain excitation encoder 416 or time domain excitation encoder 417 may correspond to frequency domain excitation encoder 315 or time domain excitation encoder 316 in the audio encoding apparatus 310 of FIG. 3A.

[0056] The audio decoding device 430 shown in FIG. 4B may include parameter decoder 432, mode setting element 433, frequency domain decoder 434, frequency domain excitation decoder 435, time domain excitation decoder 436, LP synthesizer 437, and postprocessor 438. Each of the frequency domain decoder 434 and the time domain excitation decoder 435, and the time domain excitation decoder 436 may include a frame error concealment algorithm or a packet loss concealment algorithm in each respective domain. These components can be integrated in at least one module and can be implemented as at least one processor (not shown). Because the audio decoding device 430 shown in FIG. 4B is obtained by connecting the audio decoding device 230 of FIG. 2B and audio decoding devices 330 of FIG. 3B, the description of the operations of the common parts has not been repeated and now the operation of the mode setting element 433 will be described.

[0057] Mode setting element 333 may check the coding mode information contained in the bit stream and provide the current frame to frequency domain decoder 434, frequency domain excitation decoder 435 or to time domain excitation decoder 436.

[0058] The frequency domain decoder 434 may correspond to the frequency domain decoder 134 in the audio decoding apparatus 130 of FIG. 1B or the frequency domain decoder 234 in the audio encoding apparatus 230 of FIG. 2B and the frequency domain excitation decoder 435 or time domain excitation decoder 436 may correspond to the frequency domain excitation decoder 334 or time domain excitation decoder 335 in the audio decoding apparatus 330 in FIG. 3B.

[0059] FIG. 5 is a block diagram of an audio frequency encoding device according to an exemplary embodiment.

[0060] The frequency domain audio coding apparatus 510 shown in FIG. 5 may include a transient detector 511, transformation element 512, signal classifier 513, energy encoder 514, spectrum normalization element 515, bit allocation element 516, spectrum encoder 517 and multiplexer 518. These components may be integrated in at least one module and may be implemented in the form of at least one processor (not shown). The frequency domain audio coding apparatus 510 may perform all the frequency domain audio encoder 214 functions and the partial parameter encoder functions 216 shown in FIG. 2. The frequency domain encoding device 510 may be replaced with the encoder configuration disclosed in ITU-T G.719, except for signal classifier 513, and the transform element 512 may use a transform window having a 50% overlap time. In addition, the frequency domain encoding device 510 may be replaced with the encoder configuration disclosed in ITU-T G.719, except for the transient detector 511 and the signal classifier 513. In each case, although not shown, the noise estimation unit may further be included on the end page of the spectrum encoder 517, as in the ITU-T G.719 standard, to estimate the noise level for a spectral coefficient for which the bit is not was allocated in the bit allocation process and insert the estimated noise level into the bit stream.

[0061] Referring to FIG. 5, the transient detector 511 can detect the duration showing the transient characteristics by analyzing the input signal and generate transient signaling information for each frame in response to the detection result. Various well-known methods can be used to detect the duration of the transient. According to an exemplary embodiment, the transient detector 511 may first of all determine if the current frame is a transient frame, and then verify the current frame that has been determined as the transient frame. Transient signaling information may be inserted in the bit stream using multiplexer 518 and may be provided to transform element 512.

[0062] The transform element 512 may determine the size of the window to be used for the transformation according to the result of detecting the duration of the transient and perform time-frequency transformation based on the determined window size. For example, a short window may be applied to the subband from which the transition duration has been detected, and a long window may be applied to the subband from which the transition duration has not been detected. As another example, a short window may be applied to the frame containing the duration of the transition.

[0063] The signal classifier 513 may analyze the spectrum provided from the transform element 512 in frame units to determine if each frame corresponds to a harmonic frame. Various well-known methods can be used to determine the harmonic frame. According to an exemplary embodiment, the signal classifier 513 may divide the spectrum provided from the conversion element 512 into multiple sub-bands and obtain a peak energy value and an average energy value for each sub-band. Then, the signal classifier 513 may obtain the number of sub-bands in which the peak energy value is greater than the average energy value using a predetermined ratio or above for each frame and set, as a harmonic frame, the frame in which the obtained number of sub-bands is greater than or equal to a predetermined value. A predetermined ratio and a predetermined value can be determined in advance experimentally or by means of simulation. Information about harmonic signaling can be inserted into the bit stream by the 518 multiplexer.

[0064] The energy encoder 514 can obtain energy in each unit of the subband and quantize and losslessly encode energy. According to an embodiment, the Norm value corresponding to the mean spectral energy in each unit of the subband can be used as energy and a scale factor or power can also be used, but energy is not limited to this. The Norm value for each sub-band may be provided to the spectrum normalization element 515 and the bit allocation element 516 and may be inserted into the bit stream by multiplexer 518.

[0065] The spectrum normalization element 515 can normalize the spectrum using the Norm value obtained in each sub-band unit.

[0066] The bit allocation element 516 may allocate bits in integer or fractional units, using the Norm value obtained in each subband band. In addition, the bit allocation element 516 may calculate the masking threshold using the Norm value obtained in each subband unit and estimate the perceptually required number of bits, i.e. the allowable number of bits, using the masking threshold. The bit allocation element 516 may limit that the allocated number of bits will not exceed the allowable number of bits for each subband. The bit allocation element 516 may sequentially allocate bits from a higher sub-band Norm and weigh the Norm value of each sub-band according to the perceptual validity of each sub-band to adjust the allocated number of bits so that a larger number of bits is allocated to perceptionally significant sub -pasma. The quantized Norm value supplied from the energy encoder 514 to the bit allocation element 516 can be used to allocate bits after prior adjustment to account for psychoacoustic weighing and the masking effect, as in the ITUT G.719 standard.

[0067] The spectrum encoder 517 can quantize the normalized spectrum using the allocated number of bits of each subband and losslessly encode the quantization result. For example, TCQ, USQ, FPC, AVQ and PVQ or a combination thereof can be used to encode the spectrum and for each quantization element an optimized lossless encoder. In addition, lattice coding can also be used for spectrum coding, but spectrum coding is not limited to this. In addition, different spectrum coding methods can also be used either depending on the environment in which the corresponding codec is used or depending on the needs of the user. Spectrum information encoded by the spectrum encoder 517 may be inserted into the bit stream by multiplexer 518.

[0068] FIG. 6 is a block diagram of an audio frequency encoding device according to an exemplary embodiment.

[0069] The frequency domain audio coding apparatus 600 shown in FIG. 6 may include a preprocessor 610, frequency domain encoder 630, time domain encoder 650 and multiplexer 670. Frequency domain encoder 630 may include transient detector 631, transform element 633 and spectrum encoder 635. These components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0070] Referring to FIG. 6, the preprocessor 610 may perform filtering, downsampling or the like of an input signal, but is not limited to this. The preprocessor 610 can determine the coding mode according to the signal characteristics. The preprocessor 610 can determine according to the signal characteristics whether the coding mode suitable for the current frame is a speech mode or a music mode, and can also determine whether the coding mode efficient for the current frame is a time domain mode or a frequency domain mode. Signal characteristics can be seen using the short-term characteristics of a frame or the long-term characteristics of multiple frames, but there is no limit to this. For example, if the input signal corresponds to a speech signal, then the coding mode may be specified as a speech mode or time domain mode, and if the input signal corresponds to a non-speech signal, i.e. a music or mixed signal, the coding mode may be specified as music mode or frequency domain mode. The preprocessor 610 can provide input to the frequency domain encoder 630 if the signal characteristics correspond to the music mode or frequency domain mode and can provide the input to the time domain encoder 660 if the signal characteristics correspond to the speech mode or time domain mode.

[0071] The frequency domain encoder 630 may process the audio signal provided from the preprocessor 610 based on a transform coding scheme. More specifically, the transient detector 631 may detect the transient component of the audio signal and determine if the current frame matches the transient frame. The transform element 633 may determine the length or shape of the transformation window based on the type of frame, i.e., the transient information provided from the transient detector 631 and may convert the audio signal to the frequency domain based on the determined transformation window. As an example of a transformation tool, you can use modified discrete cosine transform (MDCT), fast Fourier transform (FFT), or modulated lap transform (MLT). Generally, a short transform window can be applied to the frame containing the transition component. The spectrum encoder 635 can perform encoding on an audio spectrum transformed into the frequency domain. The spectrum encoder 635 will be described below in more detail with reference to FIG. 7 and 9.

[0072] The time domain encoder 650 may perform coded linear excitation coding (CELP) on the audio signal provided by the preprocessor 610. In particular, algebraic CELP may be used for CELP encoding, but CELP encoding is not limited to this.

[0073] Multiplexer 670 may multiplex spectral or signal components and variable indices generated as a result of coding in frequency domain encoder 630 or time domain encoder 650 to generate a bit stream. The bit stream can be saved on a storage medium or can be sent as packets over the channel.

[0074] FIG. 7 is a block diagram of a spectrum encoding device according to an exemplary embodiment. The spectrum encoding device shown in FIG. 7 may correspond to the spectrum encoder 635 of FIG. 6, may be included in another frequency domain coding device, or may be implemented independently.

[0075] The spectrum encoding device shown in FIG. 7 may include an energy estimation unit 710, an energy quantization and coding unit 720, a bit allocation element 730, a spectrum normalization element 740, a spectrum quantization and coding unit 750, and a noise filling element 760.

[0076] Referring to FIG. 7, the energy estimation unit 710 may split the original spectral coefficients into a plurality of sub-bands and estimate energy, e.g., the Norm value for each sub-band. Each sub-band may have the same length in the frame. If each sub-band has a non-uniform length, the number of spectral coefficients contained in the sub-band may be increased from low frequency to high frequency.

[0077] The energy quantization and coding unit 720 can quantize and code the estimated Norm value for each subband. The Norm value can be quantized using various tools, such as vector quantization (VQ), scalar quantization (SQ), trellis coded quantization (TCQ), lattice vector quantization (LVQ) and the like. The energy quantization and coding unit 720 can additionally perform lossless coding to further increase coding efficiency.

[0078] The bit allocation element 730 may allocate the bits required for encoding taking into account the allowable bits of the frame, based on the quantized Norm value for each subband.

[0079] The spectrum normalization element 740 may normalize the spectrum using the Norm value obtained for each subband.

[0080] The spectrum quantization and coding unit 750 may quantize and code a normalized spectrum based on the allocated bits for each sub-band.

[0081] The noise filling element 760 may add noise to the quantized component to zero due to the limit of allowable bits in the spectrum quantization and coding unit 750.

[0082] FIG. 8 shows an example of sub-band division.

[0083] Referring to FIG. 8, if the input signal uses a sampling frequency of 48 kHz and has a frame length of 20 ms, then the number of samples to be processed for each frame is 960. This means that if the input signal is converted using MDCT with 50% overlap, received there are 960 spectral coefficients. The overlap factor can be set variable according to the coding scheme. In the frequency domain, it is possible to theoretically process a band up to 24 kHz and a band up to 20 kHz can be represented taking into account the audible range. In the low band from 0 to 3.2 kHz, the sub-band contains 8 spectral coefficients. In the band from 3.2 to 6.4 kHz, the sub-band contains 16 spectral coefficients. In the band from 6.4 to 13.6 kHz, the sub-band contains 24 spectral coefficients. In the band from 13.6 to 20 kHz, the sub-band contains 32 spectral coefficients. In the case of a predetermined set of bands in an encoding device, coding can be performed based on Norm values, and for a high band above a predetermined band, coding based on variable schemes such as bandwidth extension can be used.

[0084] FIG. 9 is a block diagram of an apparatus 900 for quantizing and coding a spectrum according to an exemplary embodiment. The spectrum quantization and coding apparatus 900 of FIG. 9 may correspond to the spectrum quantization and coding unit 750 of FIG. 7, may be included in another frequency domain coding device, or may be implemented independently.

[0085] The spectrum quantization and coding device 900 of FIG. 9 may include a coding selection element 910, a zero encoder 930, a factor 950 encoder, a quantized component reconstruction element 970, and a reverse scaling element 990. The coefficient encoder 950 may include a scaling element 951, a spectral component (ISC) selection element 952, position information encoder 953, ISC collection element 954, size information encoder 955 and character information encoder 956.

[0086] Referring to FIG. 9, the coding method selection element 910 may choose a coding method based on the assigned bit for each band. The normalized spectrum may be provided to the zero encoder 930 or coefficient encoder 950, based on the coding method that was chosen for each band.

[0087] Zero encoder 930 can encode all samples to 0 for the band in which the assigned bit is 0.

[0088] The coefficient encoder 950 can perform coding using a quantization unit that is selected for a band in which the allocated bit is not equal to 0.

More specifically, the coefficient encoder 950 may select a significant spectral component in band units for the normalized spectrum and encode information about a selected significant spectral component for each band based on number, position, size and sign. The magnitude of the relevant spectral component can be coded using a scheme that differs from the number, position and character encoding scheme. For example, the magnitude of the spectral significant component may be quantized and arithmetically coded using a method selected from USQ and TCQ, and the number, position and sign of the spectral significant component may be coded using arithmetic coding. If it is determined that a particular band contains relevant information, USQ may be used, otherwise TCQ may be used. According to an exemplary embodiment, one of the TCQ and USQ may be selected based on the signal characteristics. Here, the signal characteristics may include the length of each band or the number of bits allocated to each band. For example, if the average number of bits allocated to each sample contained in the band is equal to or greater than the threshold value (e.g. 0.75), then the corresponding band can be determined to contain very important information, and thus USQ can be used. Also, in a low band where the band length is short, USQ can be used as the case may be.

[0089] Scaling element 951 may perform scaling on the normalized spectrum based on the number of bits allocated to the band to control the bit rate. The scaling element 951 can perform scaling, taking into account the average bit allocation for each spectral coefficient, namely for each sample included in the band. For example, as the average bit allocation increases, more scaling can be performed.

[0090] ISC selection element 952 may select an ISC based on a scaled spectrum for bit rate control based on a predetermined reference. ISC selection element 953 can analyze the degree of scaling based on scaled spectrum and obtain the actual non-zero position. Here, the ISC may correspond to the actual non-zero spectral coefficient before scaling. ISC selection element 953 may select a spectral coefficient (i.e., a non-zero position) to be encoded, taking into account the distribution and variance of spectral coefficients, based on the allocation of bits for each band. TCQ can be used to select the ISC.

[0091] Location information encoder 953 may encode ISC position information selected by ISC selection element 952, namely position information of a non-zero spectral coefficient. Location information may include the number and location of selected ISC components. Arithmetic coding can be used to encode location information.

[0092] The ISC collection element 954 may accumulate selected ISC components to form a new buffer. Zero band and unselected spectrum can be excluded from collecting ISC components.

[0093] The size information encoder 955 may perform the encoding based on the size information of the newly created ISC. In this case, quantization may be performed using a method selected from TCQ and USQ, and arithmetic coding may additionally be performed. To increase the efficiency of arithmetic coding, information about non-zero position and number of ISC components can be used for arithmetic coding.

[0094] The character information encoder 956 may perform the encoding of the character information of the selected ISC. Arithmetic coding can be used to encode character information.

[0095] The quantized component reconstruction element 970 may recover the quantized component based on the location, size and ISC sign information. Here, 0 can be assigned to a zero position, namely, a spectral coefficient encoded at 0.

[0096] The inverse scaling element 990 can perform inverse scaling on the reconstructed quantized component to output a quantized spectral coefficient at the same level as the normalized spectrum. The scaling element 951 and the inverse scaling element 990 can use the same scaling factor.

[0097] FIG. 10 is a diagram illustrating an ISC collection operation. First, the zero band is excluded, namely the band to be quantized to 0. Then, a new buffer can be created using an ISC selected from the spectrum components that exist in the non-zero band. USQ or TCQ may be performed for newly created ISC in bandwidth units, and corresponding lossless coding may be performed.

[0098] FIG. 11 shows an example of TCQ used in the exemplary embodiment, and corresponds to an 8-state, 4-layer, 2-layer lattice structure. Detailed descriptions regarding TCQ are disclosed in US Patent No. 7,605,727. [0099] FIG. 12 is a block diagram of an audio decoding device in the frequency domain according to an exemplary embodiment.

[0100] The frequency 1200 audio decoding device 1200 shown in FIG. 12 may include a frame error detector 1210, frequency domain decoder 1230, time domain decoder 1250 and postprocessor 1270. Frequency domain decoder 1230 may include a spectrum decoder 1231, memory update unit 1233, inverse transformation element 1235 and overlay and add (12) overlap and add). The components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0101] Referring to FIG. 12, the frame error detector 1210 may detect whether the frame error is based on the received bit stream.

[0102] Frequency domain decoder 1230 may operate if the coding mode is music mode or frequency domain mode and generate a time domain signal by means of the general transform decoding process if a frame error occurs and by means of a frame error concealment algorithm or packet loss concealment algorithm the frame does not occur. More specifically, spectrum 1231 can synthesize spectral coefficients by performing spectral decoding based on the decoded parameter. The spectrum decoder 1033 will be described below in more detail with reference to FIG. 13 and 14.

[0103] Memory update unit 1233 may update, for the next frame, the synthesized spectral coefficients, information obtained using the decoded parameter, the number of erroneous frames that still existed until now, information about the signal characteristics or type of frame of each frame, and the like with respect to to the current frame, which is a good frame. The signal characteristics may include a transient characteristic or a fixed characteristic, and the type of frame may include a transient frame, a fixed frame or a harmonic frame.

[0104] The inverse transform element 1235 may generate a time domain signal by performing a time-frequency inverse transformation on the synthesized spectral coefficients.

[0105] OLA unit 1237 may perform OLA processing using the time domain signal of the previous frame, generate the final time domain signal of the current frame as a result of OLA processing, and provide the final time domain signal to the postprocessor 1270.

[0106] Time domain decoder 1250 may operate if the coding mode is speech mode or time domain mode and generate a time domain signal by performing a general CELP decoding process if there is no erroneous frame and performing a frame error concealing algorithm or packet loss concealing algorithm in the case of an error frame occurs.

[0107] Postprocessor 1270 may perform, but is not limited to, filtering, upsampling or the like for a time domain signal provided from frequency domain decoder 1230 or time domain decoder 1250. The 1270 postprocessor provides the reconstructed audio signal as an output signal.

[0108] FIG. 13 is a block diagram of a spectrum decoding device according to an exemplary embodiment.

[0109] The spectrum decoding device 1300 shown in FIG. 13 may include an energy decoding and dequantization unit 1310, a bit allocation element 1330, a spectrum decoding and dequantization unit 1350, a noise filling element 1370 and a spectrum shaping unit 1390. The noise filling element 1370 may be at the rear of the spectrum shaping unit 1390. The components can be integrated in at least one module and can be implemented as at least one processor (not shown).

[0110] Referring to FIG. 13, the energy decoding and dequantization unit 1310 can perform lossless decoding on a parameter on which lossless coding is performed in an encoding process, e.g., energy, such as a Norm value, and dequantize a decoded Norm value. In the coding process, the Norm value can be quantized using one of various methods, for example vector quantization (VQ), scalar quantization (SQ), lattice coded quantization (TCQ), lattice vector quantization (LVQ) and the like, and in the decoding process, Norm value can be quantized using the appropriate method.

[0111] The bit allocation element 1330 may allocate the required bits in sub-band units based on a quantized Norm value or a dequantized Norm value. In this case, the number of bits allocated in sub-band units may be the same as the number of bits allocated in the encoding process.

[0112] The spectrum decoding and dequantization unit 1350 can generate normalized spectral coefficients by performing lossless decoding on the encoded spectral coefficients based on the number of bits allocated in the sub-band units and dequantizing the decoded spectral coefficients.

[0113] The noise filling element 1370 may supplement noise in a portion requiring noise filling in sub-band units from normalized spectral coefficients.

[0114] The spectrum shaping unit 1390 can shape normalized spectral coefficients using a dequantized Norm value. Ultimately decoded spectral coefficients can be obtained in the process of shaping the spectrum.

[0115] FIG. 14 is a block diagram of an apparatus 1400 for decoding and dequantizing a spectrum according to an exemplary embodiment. Apparatus 1400 for decoding and dequantizing the spectrum of FIG. 14 may correspond to the spectrum decoding and dequantization unit 1350 of FIG. 13, may be included in another frequency decoding device, or may be implemented independently.

[0116] The apparatus 1400 for decoding and dequantizing the spectrum of FIG. 14 may include a decoding method selection element 1410, zero decoder 1430, factor decoder 1450, quantized component reconstruction element 1470 and inverse scaling element 1490. The factor decoder 1450 may include position information decoder 1451, size information decoder 1453 and sign information decoder 1455.

[0117] Referring to FIG. 14, the decoding method selection element 1410 may choose a decoding method based on the allocated bit for each band. The normalized spectrum may be provided to the zero decoder 1430 or to the coefficient decoder 1450, based on the decoding method that has been selected y for each band.

[0118] Zero decoder 1430 can decode all samples into 0 for the band in which the assigned bit is 0.

[0119] The coefficient decoder 1450 can perform decoding using a quantizer that has been selected for a band in which the allocated bit is not equal to 0. The coefficient decoder 1450 can obtain information about a significant spectral component in band units for the encoded spectrum and decode information about the information obtained with a significant spectral component based on number, position, size and sign. The magnitude of the relevant spectral component may be decoded by a scheme that differs from the number, position and sign decoding scheme. For example, the magnitude of the spectral significant component may be decoded arithmetically and quantized using a method selected from USQ and TCQ, and arithmetic decoding may be performed for the number, position and sign of the spectral significant component. The dequantizer selection can be made using the same result as the coefficient encoder 950 of FIG. 9. The coefficient decoder 1450 can dequantize the band in which the allocated bit is not equal to 0, using a method selected from USQ and TCQ.

[0120] Location information decoder 1451 may decode an indicator associated with the location information contained in the bit stream to restore the number and position of ISC components. Arithmetic decoding can be used to decode location information. The size information decoder 1453 may perform arithmetic decoding on the indicator associated with the size information contained in the bit stream, and dequantize the decoded indicator using a method selected from USQ and TCQ. Nonzero location and number of ISC information can be used to increase arithmetic decoding performance. The character information decoder 1455 may decode an indicator associated with the character information contained in the bit stream to restore the ISC character. Arithmetic decoding can be used to decode character information. According to an exemplary embodiment, the number of pulses necessary for a non-zero band can be estimated and can be used to decode size information or sign information.

[0121] The quantized component reconstruction element 1470 may recover the quantized component based on the restored position, size and ISC character information. Here, 0 can be allocated to the zero position, namely the non-quantized part, which is the spectral coefficient decoded to 0.

[0122] The inverse scaling element 1490 may perform inverse scaling on the recovered quantized component to output a quantized spectral coefficient at the same level as the normalized spectrum.

[0123] FIG. 15 is a block diagram of a multimedia device comprising a coding module according to an exemplary embodiment.

[0124] Referring to FIG. 15, multimedia device 1500 may include a communication unit 1510 and an encoding module 1530. In addition, multimedia device 1500 may further include a memory unit 1550 for storing the audio bit stream obtained by encoding according to the use of the audio bit stream. In addition, the multimedia device 1500 may further include a 1570 microphone. This means that the 1550 memory unit and 1570 microphone can be optionally attached. The multimedia device 1500 may further comprise any decoding module (not shown), e.g. a decoding module for performing the general decoding function or a decoding module according to an exemplary embodiment. The coding module 1530 can be implemented using at least one processor (not shown) by connecting to other components (not shown) contained in the multimedia device 1500 as an assembly.

[0125] The communication unit 1510 may receive at least one of the audio signal or encoded bit stream provided externally, or may transmit at least one of the reconstructed audio signal or encoded bit stream obtained by coding in the coding module 1530.

[0126] The communication unit 1510 is configured to send and receive data to and from an external multimedia device or server via a wireless network, such as wireless Internet, wireless intranet, wireless telephone network, wireless local area network (LAN), Wi-Fi, direct Wi -Fi (Direct WiFi) (WFD), third generation (3G), fourth generation (4G), Bluetooth, infrared (ang. Infrared Data Association (IrDA), Radio Frequency Identification (RFID), Ultra WideBand (UWB), Zigbee or Near Field Communication (Near Field Communication) ( NFC) or a wired network, such as a wired telephone network or wired Internet.

[0127] According to an exemplary embodiment, the coding module 1530 may select ISCs in band units for the normalized spectrum and encode information about a selected significant spectral component for each band based on number, position, size and sign. The magnitude of the relevant spectral component can be coded using a scheme that differs from the number, position and character encoding scheme. For example, the magnitude of the spectral significant component may be quantized and arithmetically coded using a method selected from USQ and TCQ, and the number, position and sign of the spectral significant component may be coded using arithmetic coding. According to an exemplary embodiment, the coding module 1530 can perform scaling on the normalized spectrum based on the allocation of bits for each band and select an ISC based on the scaled spectrum.

[0128] The memory unit 1550 may store the encoded bit stream generated by the coding module 1530. In addition, the 1550 memory unit can store various programs required to operate the 1500 multimedia device.

[0129] The microphone 1570 may provide an audio signal from a user or from outside to the coding module 1530.

[0130] FIG. 16 is a block diagram of a multimedia device comprising a decoding module according to an exemplary embodiment. [0131] Referring to FIG. 16, multimedia device 1600 may include a communication unit 1610 and a decoding module 1630. In addition, according to the use of the reconstructed audio signal obtained by decoding, the multimedia device 1600 may further include a memory unit 1650 for storing the reconstructed audio signal. In addition, the multimedia device 1600 may further include a 1670 speaker. That is, a 1650 memory unit and a 1670 speaker may be optionally included. The multimedia device 1600 may further comprise a coding module (not shown), e.g., a coding module for performing a general coding function or a coding module according to an exemplary embodiment. The decoding module 1630 can be implemented by means of at least one processor (not shown) by connecting to other components (not shown) contained in the multimedia device 1600 as one unit.

[0132] The communication unit 1610 may receive at least one of the audio signal or encoded bit stream provided externally, or may transmit at least one reconstructed audio signal obtained by decoding at the decoding module 1630 or the audio bit stream obtained by encoding. The communication unit 1610 may be implemented substantially and similar to the communication unit 1510 of FIG. 15.

[0133] According to an exemplary embodiment, the decoding module 1630 may receive the bit stream provided by the communication unit 1610 and obtain information about the significant spectral component in bandwidth units for the encoded spectrum and decode information from the obtained information about the significant spectral component based on the number, position, size and mark. The magnitude of the relevant spectral component may be decoded by a scheme that differs from the number, position and sign decoding scheme. For example, the magnitude of the spectral significant component may be decoded arithmetically and quantized using a method selected from USQ and TCQ, and arithmetic decoding may be performed for the number, position and sign of the spectral significant component.

[0134] Memory unit 1650 may store the reconstructed audio signal generated by the decoding module 1630. In addition, the memory unit 1650 can store various programs required to operate the 1600 multimedia device.

[0135] The loudspeaker 1670 may output the reconstructed audio signal generated by the decoding module 1630 externally.

[0136] FIG. 17 is a block diagram of a multimedia device comprising an encoding module and a decoding module according to an exemplary embodiment.

[0137] Referring to FIG. 17, multimedia device 1700 may include a communication unit 1710, an encoding module 1720 and a decoding module 1730. In addition, media device 1700 may further include a memory unit 1740 for storing the audio bit stream obtained by encoding or reconstructed audio signal obtained by decoding according to using the audio bit stream or reconstructed audio signal. In addition, the multimedia device 1700 may further include a microphone 1750 and / or a speaker 1760. The coding module 1720 and the decoding module 1730 may be implemented by means of at least one processor (not shown) by connecting to other components (not shown) contained in the device 1700 as one team.

[0138] Since the components of the multimedia device 1700 shown in FIG. 17 correspond to the components of the multimedia device 1500 shown in FIG.

or subassemblies of the multimedia device 1600 shown in FIG. 16, their detailed description has been omitted.

[0139] Each of the multimedia devices 1500, 1600 and 1700 shown in FIG. 15 and 17 may include a dedicated voice communication terminal, such as a telephone or cellular telephone, a broadcast or music dedicated device, such as a television or MP3 player, or a hybrid terminal device that is a dedicated voice communication terminal and a broadcast or dedicated music device, but are not limited to them. In addition, each of the 1500, 1600 and 1700 multimedia devices can be used as a client, server or transducer located between the client and server.

[0140] If the multimedia device 1500, 1600 or 1700 is for example a mobile phone, although not shown, the multimedia device 1500, 1600 or 1700 may further include a user input unit such as a keyboard, a display unit for displaying information processed by using the user interface or mobile phone and a processor to control the functions of the mobile phone. In addition, the cellular telephone may additionally comprise a camera unit with image acquisition function and at least one subassembly to perform the function required for the cellular telephone.

[0141] If the multimedia device 1500, 1600 or 1700 is for example a television, although not shown, the multimedia device 1500, 1600 or 1700 may further include a user input unit such as a keyboard, a display unit for displaying received broadcast information and processor to control all TV functions. In addition, the television may further include at least one subassembly to perform the functions of the television.

[0142] The above-described exemplary embodiments may be saved as computer executable programs and may be implemented in general purpose digital computers that execute these programs using a durable, computer readable storage medium. In addition, data structures, program instructions or data files that can be used in embodiments can be saved in various ways on a durable, computer-readable storage medium. A durable, computer-readable storage medium is any data storage device that can store data that can then be read by a computer system. Examples of durable, computer-readable storage media include magnetic storage media such as hard drives, floppy disks and magnetic tapes, optical storage media such as CD-ROMs and DVDs, magneto-optical media such as optical disks, and hardware devices such as ROM, RAM and flash memory, specially configured to store and execute program instructions. In addition, a durable, computer-readable storage medium may be a transmission medium for transmitting a signal denoting program instructions, data structures or the like. Examples of program instructions may include not only mechanical language codes created by the compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

[0143] Although the exemplary embodiments have been thoroughly illustrated and described, it will be understood by those of ordinary skill in the art that various changes in form and detail can be made without departing from the scope of the inventive idea as defined by the attached claims. It should be understood that the exemplary embodiments described herein should be considered only in a descriptive sense, and not as a limitation. Descriptions of features or aspects in each exemplary embodiment should generally be considered available to other similar features or aspects in other exemplary embodiments.