Apparatus and method for watermark data insertion and apparatus and method for watermark data detection

BACKGROUND OF THE INVENTION

1. FIELD OF THE INVENTION:

0001The present invention relates to a watermark dots insertion method for inserting watermark data into a digitalized signal representing images, sounds, and/or the like to indicate their copyrights, and a watermark data detection method for detecting inserted watermark data.

2. DESCRIPTION OF THE RELATED ART:

0002Techniques for inserting watermark data into a digitalized signal representing images, sounds, and/or the like to indicate their copyrights having been vigorously studied and developed. Such watermark data is required to have minimum interference with the images and/or sounds of interest, be hardly visible (in the case of image data) and hardly audible (in the cane of sound data), and yet be easily detected by a person knowing the detection method while preventing unauthorized removal of the watermark data by third parties. By using such watermark data, for example, infringement of a copyright (by illegal copying of images and/or sounds) by unauthorized third parties can be established or prevented.

0003One conventional method for inserting watermark data into an original signal representing images, sounds. and/or the like utilizes a spread spectrum technology. In principle; this method involves: multiplying the original signal representing images, sounds, and/or the like by random numbers to obtain a uniform spectrum; adding watermark data to the multiplied signal; and again multiplying the resultant signal with the same random numbers to restore the original signal. At the time of restoring the original signal, the added watermark data is also multiplied by the random numbers. As a result, the watermark data has a uniformly spread spectrum, i.e. becomes white noise, in the restored original signal.

0004Figure <b>7</b> is a block diagram illustrating the structure of a conventional watermark data insertion apparatus utilizing a spread spectrum technology. As shown in Figure <b>7</b>, an original signal <b>a</b>_<b>i</b> (into which watermark data <b>x</b>_<b>i</b> is to be inserted) is input to a multiplier <b>52</b>, where it is multiplied by predetermined pseudo random numbers to become <b>a</b>_<b>i</b><b>×c</b>_<b>i</b>. Since the pseudo random numbers <b>c</b>_<b>i</b> are a random sequence of +1 and -1, the frequency spectrum of <b>a</b>_<b>i</b><b>×c</b>_<b>i</b> becomes uniform.

0005Next, the watermark data <b>x</b>_<b>i</b> is input to an adder <b>53</b> to be added to a DC component of the above multiplication product <b>a</b>_<b>i</b>×<b>c</b>_<b>i</b>. Although not shown, the adder <b>53</b> includes a cascade connection of a Fourier transformer, an adder, and an inverse Fourier transformer for extracting a DC component of an input signal.

0006The signal <b>a</b>_<b>i</b>×<b>c</b>_<b>i</b> + <b>x</b>_<b>i</b> (i.e., the randomized original signal plus the watermark data) is input to a multiplier <b>54</b>, where it is again multiplied by the same pseudo random numbers to give <b>a</b>_<b>i</b>×<b>c</b>_<b>i</b>^<b>2</b><b> + x</b>_<b>i</b>×<b>c</b>_<b>i</b>. Since the pseudo random numbers are +1 or -1, <b>c</b>_<b>i</b>^<b>2</b> equals 1. Therefore, a signal having a value of <b>a</b>_<b>i</b> + <b>x</b>_<b>i</b>×<b>c</b>_<b>i</b><b> (= a</b>_<b>i</b><b>×c</b>_<b>i</b>^<b>2</b><b> + x</b>_<b>i</b><b>×c</b>_<b>i</b>) is obtained, i.e., a signal with embedded watermark data.

0007Figure <b>8</b> is a block diagram illustrating the structure of a conventional watermark data detection apparatus utilizing a spread spectrum technology for detecting embedded watermark data. As shown in Figure <b>8</b>, a signal with embedded watermark data <b>a</b>_<b>i</b> + <b>x</b>_<b>i</b>×<b>c</b>_<b>i</b> is input to a multiplier <b>62</b>, where it ii multiplied by predetermined pseudo random numbers to become <b>a</b>_<b>i</b>×<b>c</b>_<b>i</b> + <b>x</b>_<b>i</b>×<b>c</b>_<b>i</b>^<b>2</b> = <b>a</b>_<b>i</b><b>×c</b>_<b>i</b> + <b>x</b>_<b>i</b>. Since the pseudo random numbers <b>ci</b> are a uniform random sequence of +1 and -1, the DC component of <b>a</b>_<b>i</b><b>×c</b>_<b>i</b> becomes zero, so that only the watermark data <b>x</b>_<b>i</b> is extracted by an extractor <b>63</b>. The extractor <b>63</b> includes a Fourier transformer for extracting a DC component of an input signal.

0008The above-described technique is reported in "Watermark data signature method for images utilizing PN series", Electronic Information Communication Society, "Encryption and Information Security Symposium" proceedings, SCIS '97-26B, 1997.

0009However, the above-described conventional technique has a disadvantage in that the watermark data may become noticeable where the original signal has a small amplitude because the watermark data is inserted into the original signal in a uniform manner, i.e., regardless of the characteristics of the original signal. There is also a disadvantage in that, if the amplitude of the watermark data is decreased in order to prevent the watermark data from becoming noticeable, the watermark data may be masked by the original signal and therefore difficult to detect. Furthermore, unauthorized third parties can make the watermark data impossible to detect by intentionally adding noise or the like to the original signal.

SUMMARY OF THE INVENTION

0010A watermark data insertion method for inserting watermark data into an input original signal according to the present invention includes: a frequency transform step for applying a frequency transform to the original signal to form an intermediate signal; a watermark data insertion step for inserting the watermark data into a first set of frequency components of the intermediate signal; and a frequency inverse transform step for applying an inverse frequency transform to the intermediate signal, the watermark data having been inserted into the intermediate signal, thereby obtaining a signal containing the watermark data embedded therein. wherein the watermark data insertion step includes: determining, based on pseudo random numbers, the first set of frequency components of the intermediate signal into which the watermark data in inserted; and using a second set of frequency components of the intermediate signal as a reference signal, the watermark data not being inserted into the second set of frequency components.

0011In one embodiment of the invention, the frequency transform applied in the frequency transform step is a wavelet transform, and the inverse frequency transform applied in the frequency inverse transform step is an inverse wavelet transform.

0012In another embodiment of the invention, the watermark data insertion step includes: a weighting step for increasing or decreasing a signal amplitude of the watermark data in accordance with an amplitude of the first set of frequency components of the intermediate signal into which the watermark data is inserted.

0013In still another embodiment of the invention, the watermark data insertion step includes determining the first set of frequency components of the intermediate signal into which the watermark data is inserted by using pseudo random numbers having frequency-dependent weights.

0014In still another embodiment of the invention, the watermark data insertion step includes: a weighting step for increasing or decreasing a signal amplitude of the watermark data in accordance with a frequency of the first sat of frequency components of the intermediate signal into which the watermark data is inserted.

0015In still another embodiment of the invention, the watermark data insertion step includes converting binary data representing the watermark data into predetermined positive and negative values, and thereafter inserting the converted binary data into the first set of frequency components of the intermediate signal.

0016Alternatively, the watermark data insertion method for inserting watermark data into an input original signal according to the present invention includes: determining a position at which the watermark data is to be inserted in accordance with pseudo random numbers which have been subjected to an inverse transform; subjecting the watermark data to an inverse transform; increasing or decreasing a signal amplitude of the inverse-transformed watermark data in accordance with an amplitude of the original signal; and inserting the inverse-transformed watermark data with the increased or decreased signal amplitude into the original signal.

0017A watermark data detection method for detecting watermark data within a signal containing embedded watermark data according to the present invention includes: an inverse-transformed pseudo random number generation stop for applying an inverse frequency transform to pseudo random numbers used for determining an insertion position of the watermark data and complements of the pseudo random numbers, respectively; a multiplication step for multiplying the signal containing embedded watermark data by the values generated in the inverse-transformed pseudo random number generation step to obtain signal components to which the watermark data has been added and signal components to which the watermark data has not been added; and an averaging stop for deriving a difference between an average of the signal components to which the watermark data has been added and an average of the signal components to which the watermark data has not been added as obtained in the multiplication step, whereby the difference is detected as the watermark data.

0018In one embodiment of the invention, the averaging step includes deriving the average of the signal components to which the watermark data has been added by applying a weight to an average of high frequency components among the signal components to which the watermark data has been added, the weight being smaller than weights applied to averages of frequency components other than the high frequency components.

0019Alternatively, the watermark data detection method for detecting watermark data within a signal containing embedded watermark data according to the present invention includes: a frequency transform stop for applying a frequency transform to the signal containing embedded watermark data to form an intermediate signal; and a watermark data extraction step for extracting the watermark data from a predetermined frequency component of the intermediate signal.

0020In one embodiment of the invention, the watermark data extraction stop includes determining the predetermined frequency component of the intermediate signal from which the watermark data is extracted based on pseudo random numbers.

0021A watermark data insertion apparatus for inserting watermark data into an input original signal according to the present invention includes: a frequency transform section for applying a frequency transform to the original signal to form an intermediate signal; a watermark data insertion section for inserting the watermark data into a first set of frequency components of the intermediate signal; and a frequency inverse transform section for applying an inverse frequency transform to the intermediate signal, the watermark data having been inserted into the intermediate signal, thereby obtaining a signal containing the watermark data embedded therein, wherein the watermark data insertion section determines the first set of frequency components of the intermediate signal into which the watermark data is inserted based on pseudo random numbers, and uses a second set of frequency components of the intermediate signal as a reference signal, the watermark data not being inserted into the second set of frequency components.

0022A watermark date detection apparatus for detecting watermark date within a signal containing embedded watermark data includes: an inverse-transformed pseudo random number generation section for applying an inverse frequency transform to pseudo random numbers used for determining an insertion position of the watermark data and complements of the Pseudo random numbers, respectively; a multiplication section for multiplying the signal containing embedded watermark data by the values generated in the inverse-transformed pseudo random number generation section to obtain signal components to which the watermark data has been added and signal components to which the watermark data has not been added; and an averaging section for deriving a difference between an average of the signal components to which the watermark data has been added and an average of the signal components to which the watermark date has not been added as obtained by the multiplication section, whereby the difference is detected as the watermark data.

0023In accordance with the above-described method: and apparatuses of the present invention, the insertion position and the signal amplitude of watermark data are varied in accordance with the frequency components and amplitude of an original signal, thereby making it difficult to detect the watermark data. This makes possible the insertion of watermark data that has a large amplitude, thereby preventing the destruction of watermark data by an intentional addition of noise or the like. Since the insertion position of the watermark data is determined by utilizing pseudo random numbers, those who do not know the insertion position cannot read the watermark data, thereby preventing unauthorized tampering with the watermark data. Furthermore, the random numbers used for determining the insertion position of the watermark data are visually weighted so as to render the perception of the watermark data even more difficult.

0024Thus, the invention described heroin makes possible the advantage of providing a watermark data insertion method and a watermark data detection method for inserting watermark data in such a manner that the watermark data cannot be easily detected and yet imposes minimum interference with an original signal, such that the watermark data is hardly visible and/or audible and difficult to remove by unauthorized means and yet easily detectable through an authorized or intended procedure.

0025This and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

0026<ul id="ul0001" list-style="none"><li>Figure <b>1</b> is a block diagram illustrating the structure of a watermark data insertion apparatus according to one embodiment of the watermark data insertion method of the present invention.</li><li>Figure <b>2</b> is a diagram showing the data arrangement of transformed coefficients of an image which has been broken down into two-dimensional frequency components by wavelet transform.</li><li>Figure <b>3</b> is a diagram showing the position at which watermark date is embedded in an original signal coordinate system corresponding to the insertion of watermark data into a subband i in Figure <b>2</b>.</li><li>Figure <b>4</b> in a block diagram illustrating the structure of a watermark data insertion apparatus as another embodiment of the watermark data insertion method according to the present invention.</li><li>Figure <b>5</b> is a block diagram illustrating the structure of a watermark data detection apparatus according to one embodiment of the watermark data detection method of the present invention.</li><li>Figure <b>6</b> is a block diagram illustrating the structure of a watermark data detection apparatus as another embodiment of the watermark data detection method according to the present invention.</li><li>Figure <b>7</b> is a block diagram illustrating the structure of a conventional watermark data insertion apparatus utilizing a spread spectrum technology.</li><li>Figure <b>8</b> is a block diagram illustrating the structure of a conventional watermark data detection apparatus utilizing a spread spectrum technology.</li></ul>

DESCRIPTION OF THE PREFERRED EMBODIMENTS

0027Hereinafter, an example of the watermark data insertion method according to the present invention will be described with reference to the accompanying figures. Figure <b>1</b> is a block diagram illustrating the structure Of a watermark data insertion apparatus according to one embodiment of the watermark data insertion method of the present invention. The watermark data insertion apparatus includes a transform section <b>1</b>, a watermark data insertion section <b>2</b>, and an inverse transform section <b>3</b>.

0028The transform section <b>1</b>, which is mainly composed of a wavelet transform section <b>11</b> in the illustrated embodiment, breaks down an input image signal into two-dimensional frequency components. The wavelet transform section <b>11</b> includes a pair of filters, i.e., a high-pass filter and a low-pass filter (not shown), for separating an image signal into a high frequency component and a low frequency component, respectively. By repetitively passing the resultant high and low components through the same pair of filters, the wavelet transform section <b>11</b> breaks down the image signal into a large number of two-dimensional frequency components. A simplest wavelet transform section can be implemented by using a low-pass filter having multiplication coefficients (1/2, 1/2) and a high-pass filter having multiplication coefficients (1/2, -1/2) and discarding every other sample signal of the low frequency components and the high frequency components which passed through the respective filters.

0029Figure <b>2</b> is a diagram showing the data arrangement of transformed coefficients of an image which has been broken down into two-dimensional frequency components by wavelet transform. By a first-level wavelet transform, transformed coefficients in regions (subbands) <b>LL</b>, <b>LH</b>, <b>HL</b>, and <b>HH</b>, in each of which the number of transformed coefficients is reduced to 1/2 along both the vertical and horizontal dimensions, are provided. For example, the subband <b>LL</b> consists of transformed coefficients which can be obtained by discarding every other pixel along both the vertical and horizontal dimensions. Therefore, restoring an image using the transformed coefficients from this subband results in an image having a 1/2 resolution of that of the original image. The subband <b>LL</b> consists of transformed coefficients which are derived from image components existing in the lower half of the frequency spectrum along both the vertical and horizontal dimensions. The subband <b>HH</b> consists of transformed coefficients which are derived from image components existing in the higher half of the frequency spectrum along both the vertical and horizontal dimensions. The subband <b>HL</b> consists of transformed coefficients which are derived from image components existing in the lower half along the vertical dimension, and in the higher half along the horizontal dimension, of the frequency spectrum. Similarly, the subband <b>LH</b> consists of transformed coefficients which are derived from image components existing in the higher half along the vertical dimension, and in the lower half along the horizontal dimension, of the frequency spectrum.

0030By further breaking down an image represented by the subband <b>LL</b> into higher and lower frequency components, subbands are obtained, each of which can be represented by a smaller number of transformed coefficients. Hereinafter, the transformed coefficients (frequency components) of a subband i will be denoted as <b>B</b>_<b>i</b> (intermediate signal), where i is a natural number. It will be appreciated that the wavelet transform is similarly applicable to audio data as well.

0031The watermark data insertion section <b>2</b> shown in Figure <b>1</b> includes a pseudo random number generation section <b>21</b>, a multiplication section <b>22</b>, a weighting/addition section <b>23</b>, and a subtraction section <b>24</b>.

0032The pseudo random number generation section <b>21</b> generates pseudo random numbers <b>PN</b> and outputs a signal <b>PN</b>_<b>ij</b>, which is weighted by the pseudo random numbers <b>PN</b>, by employing an internal audio or visual filter (not shown). The multiplication section <b>22</b> multiplies the output <b>B</b>_<b>i</b> of the transform section <b>1</b> by the output <b>PN</b>_<b>ij</b> of the pseudo random number generation section <b>21</b> so as to output a multiplication product <b>b</b>_<b>i</b> = <b>B</b>_<b>i</b> × <b>PN</b>_<b>ij</b>.

0033The weighting/addition section <b>23</b> receives watermark data <b>WM</b>_<b>j</b> (representing the actual watermark), which is converted into watermark data <b>W</b>_<b>j</b> weighted by weighting coefficients corresponding to the respective subbands. The weighting/addition section <b>23</b> adds the watermark data <b>W</b>_<b>j</b> and the output signal <b>b</b>_<b>i</b> of the multiplication section <b>22</b> so as to output a signal <b>x</b>_<b>i</b> = <b>b</b>_<b>i</b> + <b>W</b>_<b>j</b>. The subtraction section <b>24</b> subtracts the output signal <b>b</b>_<b>i</b> of the multiplication section <b>22</b> from the output <b>B</b>_<b>i</b> of the transform section <b>1</b> so as to output the subtraction result <b>r</b>_<b>i</b>. The inverse transform section <b>3</b> includes an inverse wavelet transform section <b>31</b> for receiving the, signals <b>x</b>_<b>i</b> and <b>r</b>_<b>i</b> and transforming these signals into an image signal with embedded watermark data, which is output from the inverse transform section <b>3</b>.

0034The operation of the watermark data insertion apparntus having the above-described structure will be described in more detail. A digitalized image signal is transformed into signals <b>B</b>_<b>i</b> of each subband <b>i</b> (where <b>i</b> is a natural number) by the wavelet transform section <b>11</b>. The sensitivity of human eyes or ears becomes relatively high for the case where the signals <b>B</b>_<b>i</b> of a subband represent the components of a middle frequency band, and relative low for the case where the signals <b>B</b>_<b>i</b> of a subband represent the components of a low or high frequency band. The pseudo random number generation section <b>21</b> and the weighting/addition section <b>23</b> subject each subband to a weighting in accordance with such characteristics of the human visual or auditory sense.

0035The pseudo random number generation section generates pseudo random numbers <b>PN</b> consisting of 0 or 1. It will be appreciated that by employing these pseudo random numbers <b>PN</b> as they are, it would result in the multiplication section <b>22</b> selecting all the subbands in an impartial or uniform manner.

0036Rather, in order to prevent the unauthorized detection of the insertion position of the watermark data <b>WM</b>_<b>ij</b>, the uniform pseudo random numbers <b>PN</b> are converted into pseudo random numbers having the opposite characteristics of those of the human visual or auditory sense (frequency sensitivity), by being passed through a filter for cutting off the components of the middle frequency band, for example. The pseudo random number generation section <b>21</b> converts the pseudo random number <b>PN</b> into pseudo random numbers <b>PN</b>_<b>ij</b> which are weighted by means of a frequency weighting filter (not shown). The use of such pseudo random numbers <b>PN</b>_<b>ij</b> advantageously changes the random pattern from a white noise-like pattern into a pink noise-like pattern, which has less middle frequency components, hence making it difficult to detect.

0037The multiplication section <b>22</b> multiplies the signals <b>B</b>_<b>i</b> of each subband by the predetermined pseudo random numbers <b>PN</b>_<b>ij</b> to output a multiplication product signal <b>b</b>_<b>i</b> (i.e., <b>b</b>_<b>i</b> = <b>B</b>_<b>i</b> × <b>PN</b>_<b>ij</b>)<b>.</b> Thus, at least one of the transformed coefficients of each subband that is multiplied by bit 1 of the pseudo random numbers <b>PN</b>_<b>ij</b> is extracted so as to be output as the signal <b>b</b>_<b>i</b>.

0038Upon receiving the watermark data <b>WM</b>_<b>j</b> (of binary values of 0 or 1), the weighting/addition section <b>23</b> converts each value of the watermark data <b>WM</b>_<b>j</b> into either a positive predetermined value (when the value of the watermark data <b>WM</b>_<b>j</b> is 1) or a negative predetermined value (when the value of the watermark data <b>WM</b>_<b>j</b> is 0). Furthermore, the weighting/addition section <b>23</b> multiplies the transformed watermark data by a weight <b>K</b>_<b>ij</b> corresponding to the subband from which the signals <b>b</b>_<b>i</b> have been extracted so as to output the multiplication product as the watermark data <b>W</b>_<b>j</b><b>.</b> One of respectively different values previously assigned to the respective subbands is used as the weight <b>K</b>_<b>ij</b> for each subband i.

0039Specifically, the value of the weight <b>K</b>_<b>ij</b> is decreased in the middle frequency band, where humans have high frequency sensitivity. That is, the values of the weight <b>K</b>_<b>ij</b> are selected to realize the opposite characteristics of those of the human visual or auditory sense. As a result, even if the watermark data <b>W</b>_<b>j</b> is inserted in a subband containing middle frequency band components, the watermark data <b>W</b>_<b>j</b> is hardly visible (in the case of an image signal) and hardly audible (in the case of an audio signal).

0040By using the pseudo random numbers <b>PN</b>_<b>ij</b> thus-weighted by the use of the frequency weighting filter and the weight <b>K</b>_<b>ij</b> having values corresponding to the respective subbands, the watermark data <b>W</b>_<b>j</b> will nonetheless be inserted in all subbands as a result of processing a series of image signals representing a plurality of pictures. Therefore, even if tampering is attempted by passing an image signal containing embedded watermark data through a low-pass filter or removing a portion of the signal, the watermark data will remain in the transformed coefficients in the middle and low frequency bands and therefore be detectable despite such tampering.

0041The weighting/addition section <b>23</b> adds the watermark data <b>W</b>_<b>j</b> to the signal <b>b</b>_<b>i</b> and outputs a signal <b>x</b>_<b>i</b> (= <b>b</b>_<b>i</b> + <b>W</b>_<b>j</b>) containing embedded watermark data. The signal amplitude of the watermark data <b>W</b>_<b>j</b> is selected to be smaller than that of the signal <b>b</b>_<b>i</b> by increasing or decreasing the signal amplitude of the watermark data <b>W</b>_<b>j</b> in such a manner that the watermark data is masked by the signal <b>b</b>_<b>i</b> to prevent unauthorized detection.

0042The subtraction section <b>24</b> calculates a signal <b>r</b>_<b>i</b> (= <b>B</b>_<b>i</b> - <b>b</b>_<b>i</b>) into which the watermark data was not inserted, and supplies the signal <b>r</b>_<b>i</b> to the inverse transform section <b>3</b>. The inverse transform section <b>3</b> synthesizes the signals <b>x</b>_<b>i</b> and <b>r</b>_<b>i</b> and performs a transform process at an inverse wavelet transform section <b>31</b> for restoring the original image signal. As a result of thus restoring the image, an image signal with embedded watermark data is obtained.

0043The inverse wavelet transform can be achieved by, interpolating 0 between sample signals of each subband, passing the interpolated sample signals through the same high-pass filter and low-pass filter that were used for the wavelet transform, and repetitively deriving a difference between each pair of a low frequency component and a high frequency component output from the high-pass filter and low-pass filter.

0044The above-described process is repeated for the same watermark data, whereby image signals representing images containing the same watermark data embedded therein are sequentially output. Alternatively, the addition of the same watermark data <b>W</b>_<b>j</b> to the signal <b>b</b>_<b>i</b> is performed multiple times so that a plurality of instances of watermark data are inserted in the signal <b>x</b>_<b>i</b>. In either case, a plurality of instances of the same watermark data will always be inserted into the image signal.

0045Figure <b>3</b> is a diagram showing the position at which watermark data is embedded in an original signal coordinate system as restored by an inverse transform based on the simplest wavelet transform, assuming that the watermark data was embedded at a position <b>X</b>_<b>i</b> among the transformed coefficients <b>B</b>_<b>i</b> of a subband i (that lies in a region of high frequency components along both the horizontal direction and the vertical direction of the subband <b>LL</b> in Figure <b>2</b>). The signal amplitude of the watermark data is reduced to 1/16 due to the inverse transform, and its polarity (i.e., positive or negative) is also changed, so that the inverse-transformed pseudo random numbers do not take binary values but may take any real number between -1 and +1.

0046Figure <b>4</b> is a block diagram illustrating the structure of a watermark data insertion apparatus as another embodiment of the watermark data insertion method according to the present invention. As shown in Figure <b>4</b>, a multiplication section <b>72</b> directly multiplies an input image signal <b>A</b>_<b>i</b> by inverse-transformed pseudo random numbers which are generated by an inverse-transformed pseudo random number generation section <b>71</b> so as to form a sample <b>a</b>_<b>i</b> (i.e., a predetermined frequency component of an intermediate signal), which is output so that watermark data will be subsequently added thereto. Watermark data <b>WM</b>_<b>j</b> is transformed into watermark data <b>W</b>_<b>j</b> in a base band region by an inverse transform section <b>73</b>, and the transformed watermark data <b>W</b>_<b>j</b> is input to a weighting/addition section <b>74</b>. The weighting/addition section <b>74</b> adds the transformed watermark data <b>W</b>_<b>j</b> and the sample <b>a</b>_<b>i</b> so as to output a signal <b>a</b>_<b>i</b> + <b>W</b>_<b>j</b> containing embedded watermark data. A subtraction section <b>75</b> calculates a sample <b>A</b>_<b>i</b> - <b>a</b>_<b>i</b> in which the watermark data is not embedded. An addition section <b>76</b> adds the output of the weighting/addition section <b>74</b> and the output of the substraction section <b>75</b> so as to output an image signal containing embedded watermark data <b>A</b>_<b>i</b> + <b>W</b>_<b>j</b>.

0047In accordance with the above apparatus, an inverse transform of pseudo random numbers (indicating positions at which watermark data is to be inserted) based on an inverse wavelet transform, etc., is previously calculated. Thus, the inverse-transformed pseudo random numbers can be used to multiply an input image signal directly, i.e., without subjecting the image signal to a transform process. Moreover, it becomes unnecessary to perform an inverse transform after the addition of watermark data. As a result, the cost associated with the insertion of watermark data can be reduced.

0048Next, one embodiment of the watermark data detection method according to the present invention will be described. Figure <b>5</b> is a block diagram illustrating the structure of a watermark data detection apparatus according to one embodiment of the watermark data detection method of the present invention. The watermark data detection apparatus includes a watermark data detection section <b>4</b>, which in turn includes an inverse-transformed pseudo random number generation section <b>41</b>, a multiplication section <b>42</b>, and an averaging section <b>43</b>.

0049The inverse-transformed pseudo random number generation section <b>41</b> includes a pseudo random number generation section, which is identical with that in the watermark data insertion apparatus shown in Figure <b>1</b>, for generating pseudo random numbers <b>PN</b>_<b>ij</b> and (1 - <b>PN</b>_<b>ij</b>) and inverse transforming the pseudo random numbers <b>PN</b>_<b>ij</b> and (1 - <b>PN</b>_<b>ij</b>) by inverse wavelet transform to generate and output inverse-transformed pseudo random numbers <b>PN</b>_<b>ij</b>' and (1 - <b>PN</b>_<b>ij</b>)'. In Figure <b>5</b>, the inverse-transformed pseudo random numbers <b>PN</b>_<b>ij</b>' and (1 - <b>PN</b>_<b>ij</b>)' are shown as <b>N</b>_<b>i</b> and <b>M</b>_<b>i,</b> respectively; <b>N</b>_<b>i</b> is an inverse wavelet transform of the pseudo random numbers <b>PN</b>_<b>ij</b> used for the determination of watermark data insertion position; and <b>M</b>_<b>i</b> is an inverse wavelet transform of the complement of the pseudo random numbers <b>PN</b>_<b>ij</b><b>.</b>

0050Upon receiving an image signal <b>P</b> containing embedded watermark dots, the multiplication section <b>42</b> multiplies the image signal <b>P</b> by the inverse-transformed pseudo random numbers <b>N</b>_<b>i</b> and <b>M</b>_<b>i</b> to output a signal <b>x</b>_<b>i</b>' (= <b>P × PN</b>_<b>ij</b>') and a signal <b>r</b>_<b>i</b><b>'</b> (= <b>P</b> × (1 - <b>PN</b>_<b>ij</b>)').

0051By using these inverse-transformed pseudo random numbers <b>N</b>_<b>i</b> and <b>M</b>_<b>i</b>, it becomes unnecessary to subject image data containing watermark data to a wavelet transform in order to achieve detection of the watermark date. As a result, the time and cost required for detection are reduced.

0052The resultant signal <b>x</b>_<b>i</b><b>'</b> is a signal containing embedded watermark data. The resultant signal <b>r</b>_<b>i</b>ʼ is a signal not containing the watermark data embedded therein.

0053The signals <b>x</b>_<b>i</b><b>'</b> and <b>r</b>_<b>i</b>' are input to an averaging section <b>43</b>, where a cumulative sum Σ<b>xi'</b> and a cumulative sum Σ<b>ri'</b> are calculated for the signals <b>x</b>_<b>i</b>' and <b>r</b>_<b>i</b>', respectively. A difference therebetween (Σ<b>xi'</b> - Σ<b>ri'</b>) is calculated.

0054The value of the above difference equals <b>x</b>_<b>i</b>'- <b>r</b>_<b>i</b>ʼ = (<b>b</b>_<b>i</b><b>'</b> + <b>WM</b>_<b>j</b>') - <b>r</b>_<b>i</b>ʼ. Since the signals <b>b</b>_<b>i</b> are generated by nondiscriminatory selection based on random numbers, an average of a multitude of <b>b</b>_<b>i</b><b>'</b> and an average of a multitude of <b>r</b>_<b>i</b>' will satisfy <b>b</b>_<b>i</b><b>'</b> = <b>r</b>_<b>i</b><b>'</b> (because the population of <b>b</b>_<b>i</b><b>'</b> and the population of <b>r</b>_<b>i</b>' are identical). Thus, the above subtraction leaves <b>WM</b>_<b>j</b>' (= (<b>b</b>_<b>i</b><b>'</b> + <b>WM</b>_<b>j</b>') - <b>r</b>_<b>i</b>'), i.e., the watermark data. Based on the signs (i.e., positive or negative) of this value, the value of the watermark data <b>WM</b>_<b>j</b> can be obtained.

0055When calculating an average of the signals <b>x</b>_<b>i</b>' containing embedded watermark data, the average of higher frequency components receives a weight which is smaller than the weights for other average values. As a result, even if an unauthorized third party attempts to destroy or render undetectable the watermark data, or if the size or the position of the signal containing embedded watermark data is changed, or if noise is admixed in the high frequency components, such influences can be effectively eliminated.

0056Figure <b>6</b> is a block diagram illustrating the structure of a watermark data detection apparatus as another embodiment of the watermark data detection method according to the present invention. As shown in Figure <b>6</b>, a transform section <b>81</b> receives an image signal <b>P</b> containing embedded watermark data and performs a wavelet transform to transform the image signal <b>P</b> into transformed coefficients of each subband. This process is in the opposite direction with respect to the process performed by the inverse transform section <b>3</b> in Figure <b>1</b>. The signal (i.e., intermediate signal) which results from this process is equal to <b>x</b>_<b>i</b> + <b>r</b>_<b>i</b> in the embodiment described with reference to Figure <b>1</b>. A multiplication section <b>83</b> multiplies the signal <b>x</b>_<b>i</b><b> + r</b>_<b>i</b> by pseudo random numbers generated by a pseudo random number generation section <b>82</b> (which are the same pseudo random numbers as those used in the embodiment described with reference to Figure <b>1</b>) so as to extract a signal <b>x</b>_<b>i</b> containing embedded watermark data. The signal <b>x</b>_<b>i</b> consists of the original signal <b>b</b>_<b>i</b> with the watermark data <b>W</b>_<b>i</b> added thereto. The averaging section <b>85</b> derives an average of a plurality of signals <b>x</b>_<b>i</b> containing embedded watermark data to give a sum (<b>B</b> + <b>W</b>) of an average <b>B</b> of the original signals <b>b</b>_<b>i</b> and an average <b>W</b> of the watermark data.

0057Only the signal <b>r</b>_<b>i</b> (= <b>B</b>_<b>i</b> - <b>b</b>_<b>i</b>) in which the watermark data was not embedded is output from a subtraction section <b>84</b>. An averaging section <b>86</b> derives on average of a plurality of signals <b>r</b>_<b>i</b>, thereby giving an average <b>B</b> of the original signals. A subtraction section <b>87</b> subtracts the average <b>B</b> from the average containing the watermark data (<b>B</b> + <b>W</b>), thereby obtaining the watermark data <b>W</b>.

0058Since the input signal is subjected to wavelet transform in the apparatus shown in Figure <b>6,</b> the same pseudo random numbers as those used in the above-described watermark data insertion apparatuses can be used.

0059Although a wavelet transform section is exemplified as a transform section in the above-illustrated embodiments, the some effects will be provided by employing a frequency transform means such as Hadamard transform, discrete cosine transform, Fourier transform, Karhunen-Loéve transform, or the like. Pseudo random numbers can be generated by using linear feedback shift registers. Alternatively, previously calculated pseudo random numbers can be stored in a memory or the like for use in the process of the present invention.

0060Various sections illustrated in the embodiments can be implemented as computer programs or dedicated digital circuits.

0061As described above, the watermark data insertion method and watermark data detection method according to the present invention make it possible to insert watermark data in a manner such that the watermark data is very difficult to detect by unauthorized third parties while being easily detectible by an authorized person, e.g., one who inserted the watermark data. Thus, the present invention makes possible the establishment of infringement of a copyright of copyrightable works and/or prevention of illegal copying of such works in the field of digital contents containing images and/or sounds.

0062In accordance with one embodiment of the present invention, a wavelet transform encompasses the pixel information of the entire single frame regardless of the subband that is being used. Therefore, watermark data can be easily embedded in the entire image through one watermark data insertion process. The method of the present invention also prevents block distortion, which is typically observed in DCT transform.

0063In accordance with one embodiment of the present invention, watermark data is inserted with its amplitude being adjusted in accordance with the luminance values of pixels, whereby the watermark data can be better concealed within the image.

0064In accordance with one embodiment of the present invention, watermark data can be inserted in positions in the pixel frequency space where the watermark data will not be easily noticeable.

0065In accordance with one embodiment of the present invention, the DC component of the watermarked data is not changed after insertion, thereby improving the signal to noise ratio in the averaging process during detection.

0066In accordance with one embodiment of the present invention, it is unnecessary to detect watermark data by referring to a reference image, thereby reducing the burden on the detection system.

0067In accordance with one embodiment of the present invention, watermark data can be detected even after processing a signal with a high-cut filter.

0068Various other modifications will be apparent to and can be readily made by those skilled in the art without departing from the scope and spirit of this invention. Accordingly, it is not intended that the scope of the claims appended hereto be limited to the description as set forth herein, but rather that the claims be broadly construed.