Real-time signature embedding in video

Real-time signature embedding in video

FIELD OF THE INVENTION The present invention generally relates to the field of signal verification, and more specifically, to embedding signatures in audiovisual signals for image and video verification.

Background technique

The success of digital images and videos has led to the widespread application of this technology in many areas of daily life. Technologies for editing, changing or modifying digital images or video sequences are commercially available, allowing the content of the images or videos to be modified without leaving a trace. For various applications, such as evidence-based image records from security cameras in law enforcement, medical documents, loss assessment for insurance purposes, etc., it is necessary to ensure that the image or video has not been modified and is consistent with the original The image or video is the same. This led to the development of a signal verification system. The system shown in Figure 1 is an example in which a signature is generated at 1.20 for an audiovisual signal such as an image or video acquired at 1.10. At 1.30, the signature is embedded in the signal in the form of a watermark, for example. After that, at 1.40, the signal is processed or tampered with, played, recorded, or extracted at 1.50, and finally checked at 1.60, in order to either ensure that the authenticity of the signal is proven, or reveal that the signal has been modified.

Embedding data in a video signal is known in US-B-6211919, where an analog video signal is converted into a digital signal, the data is embedded in the digital signal, and then converted back to an analog signal. Cross-frame error correction is required to compensate for transmission loss. The technical nature of the solution disclosed there is complex and requires a large buffer memory for storing the entire frame or several frames of the video signal. These memories are expensive, so it is necessary to minimize the amount of memory required.

In addition, especially for the above-mentioned signature verification application, it is important that each video frame has the ability to verify itself, because for example, in the above-mentioned security camera application, not all frames of a sequence are stored, but only For example, every fiftyth frame is retained. Similarly, for medical images, only a subset of the images may be retained. It is generally unknown which frame was recorded and which frame was discarded. Therefore, all the information needed to verify a certain frame of a video sequence must be available in the frame itself and can be derived from the frame itself. If, as in the above-mentioned documents, a frame depends on the preceding or succeeding frame, it is impossible to enable the frame to be verified.

Signature calculation and embedding must be performed as soon as possible after the video signal is generated to prevent the video from being tampered with before the verification information is stored in the video. Therefore, it is beneficial if the signature calculation and embedding are placed close to the image capturing device, for example, in a security camera, and the generated video stream is calculated and embedded in real time. Today's solutions like those in the above-mentioned documents are technically complex and expensive.

Finally, according to the prior art, in order to embed the signature bits calculated at 1.20 for an audiovisual signal such as a digital image in the form of a watermark in the audiovisual signal itself at 1.30, all frames of the audiovisual signal must be buffered in one In a large and expensive memory, at the same time, the signature bits of the frame of the audiovisual signal are calculated to form a watermark with the signature bits as the payload, and finally the watermark is embedded in the frame of the audiovisual signal. This makes this solution expensive due to the expensive memory required.

Therefore, the problem to be solved by the present invention is defined as how to provide low-cost real-time generation of audiovisual signals for self-verified frames.

Summary of the invention

According to the attached independent claims, a signature is embedded in a video signal or digital image by completely avoiding the need to buffer the entire frame of the audiovisual signal in a large memory during the calculation of the signature bits and the embedding of the watermark. Therefore, the cost of the required memory is significantly reduced in an audiovisual signal, and the present invention overcomes the defects of the prior art indicated above.

According to an embodiment of the present invention, a method, a device and a computer-readable medium for verifying audiovisual signals are disclosed. According to these embodiments, the first part of a frame based on the audiovisual signal constitutes a signature. Then, the formed signature is embedded in the first part of the frame to be verified in the audiovisual signal or at least one second part of the frame, so that each part is a part of the audiovisual signal. The pattern of horizontal lines has fewer lines than the total number of lines of the entire audiovisual signal.

Therefore, a real-time low-cost solution is proposed, which requires only a few rows of memory for audiovisual signals, and does not require memory for storing all frames of audiovisual signals. All the information needed to verify the frame is put into the frame itself, so that each frame is self-verifying.

These and other aspects of the present invention will become apparent and explained with reference to the examples described below.

BRIEF DESCRIPTION OF THE DRAWINGS In the following detailed description, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, Figure 1 shows a prior art verification system; Figure 2 shows an embodiment of the present invention; Figure 3 shows another embodiment of the present invention; Figure 4 shows another embodiment of the present invention; Figure 5 shows another embodiment according to the present invention The device of the embodiment; and FIG. 6 shows a computer-readable medium according to another embodiment of the present party.

detailed description

Although the video signal represents a two-dimensional image, it is transmitted and processed by scanning the image line by line as a one-dimensional signal. Analog or digital video is classified into interlaced video and non-interlaced video. The latter is also called progressive scan video. For example, video signals according to the NTSC, PAL, and SECAM standards are interlaced, most PC monitors are non-interlaced, and HDTV (high-definition television) signals can be interlaced in a higher resolution mode, or can be Non-interlaced in the lower resolution mode.

An interlaced audiovisual signal, such as video, is defined as: each frame of the signal consists of two fields, and thus, each field is a specific part of the frame, which contains every interval of the frame. Horizontal line. When processing interval-scanned video by, for example, transmitting or displaying video, the field containing all odd-numbered lines including the top scan line is processed first and called the top field; the field containing even-numbered lines is called the bottom field and is called the bottom field. Continuous processing to generate a frame or complete image. Therefore, for interlaced signal lines 1, 3, 5... (that is, all lines of the first field) are processed first, and then lines 2, 4, 6... (that is, all lines of the second field) are processed. Each field can be subdivided into segments of consecutive lines of the frame, so-called slices, for example, 3 line segments: [1, 3, 5], [7, 9, 11] , [2, 4, 6] or [8, 10, 12]. A special case of segments of consecutive lines in an interlaced signal is when the first segment contains all the odd or even lines of a frame and the other segment contains the remaining even or odd lines of the frame.

Non-interlaced video displays each line of a frame in sequence, thus, a frame is defined as a complete image in an image sequence that constitutes a video. Therefore, for non-interlaced signal lines 1, 2, 3... (that is, all lines of the frame) are processed. Such a frame can be subdivided into segments of successive rows, for example, segments of 3 rows: [1, 2, 3] or [4, 5, 6].

Interlaced and non-interlaced video involves capturing, transmitting, and displaying video sequences.

A portion of a frame is defined as a single share of the frame that is a part of the frame (e.g., a segment or field as defined above).

A region (region) of a frame of an audiovisual signal such as a digital image in a video stream is defined as a spatial region within the frame, such as top, middle, and bottom.

Figure 2 shows an embodiment of the present invention, in which an audiovisual signal captured in step 2.10 is interlaced. The top field composed of, for example, n lines of one frame of the interlaced audiovisual signal is allocated to a first part, and is loaded and stored in a memory circuit in step 2.20. In step 2.30, a signature of the first field is calculated, whereby the signature contains information for verifying all regions of the frame, because the first field contains all image content, although only alternate lines of image content. Subsequently, the next field consisting of, for example, m lines of the same frame in the audiovisual signal is allocated to a second field in step 2.40. The second field is stored in the same memory circuit instead of the first field in the memory circuit. One game. The memory circuit therefore requires a maximum capacity of m or n rows, preferably m or n row memories. Therefore, the memory requirements are limited to half of the requirements of the prior art discussed above. The signature bits of the signature also need to be saved in step 2.50, which embeds the signature in the second field of the audiovisual signal in the memory circuit. However, the amount of storage required for the signature bits is negligible compared to the requirement for storing pixels in audiovisual signals such as video. The signature bits can be stored in the nth row of the memory, for example, because in practice, depending on the frame size, the number of rows contained in the second field is often 1 less than that in the first field, that is, m=(n-1).

Figure 3 shows another embodiment in which the audiovisual signal is non-interlaced and is captured by progressive scanning in step 3.10. In step 3.20, a segment of the audiovisual signal consisting of N horizontal lines is loaded and held in a memory circuit with sufficient capacity to accommodate the N lines, for example, in N line memories. Then, in step 3.30, a signature is calculated for the fragment. If you want to embed the current signature into the current fragment itself, go directly to step 3.50. If the signature is to be embedded in the next consecutive segment, the next segment will now be loaded into the N line memories to replace the current segment. If the current segment is already the last segment of the frame, the signature can only be embedded in the current segment itself. If a common signature is to be embedded for all fragments, then in optional step 3.50, the signature of the current fragment is added to a common signature with the signatures of each fragment previously calculated separately. If only the current signature is embedded in the segment currently located in N line memories, it is not combined with the previously calculated signature. In step 3.60, embed the signature in the segment currently located in N line memories. Subsequently, the audiovisual signal is either further processed, for example by storage or transmission-if the signatures of all areas of the image have been calculated, that is, if the signatures of all segments have been calculated; or by returning to step 3.20 and loading the next segment In the memory-or, if the new fragment has been loaded into N line memories, the signature is directly calculated in step 3.30, and so on. The storage of the calculated signature bits is similar to that described in the previous embodiment. This embodiment only requires one segment to be held in the memory, so N line memories are required. When a certain segment is in the memory, it is possible to calculate the signature bits for the segment, and it is better to embed the signature in the segment in the form of a watermark. The watermark can carry a payload consisting of the signature bits of the fragment and the signature bits of the previous fragment. Therefore, the i-th segment can be embedded with the signature bits from segment 1 to i. The first segment can only be embedded with the signature bits of the first segment, and the last segment can be embedded with any or all of the signature bits in the entire frame of the audiovisual signal. Therefore, the signature bits of the first segment can be embedded in any segment, preferably in all segments, and the signature bits of the last segment can only be embedded in the last segment itself. Therefore, self-verification of the image is maintained.

Fig. 4 shows another embodiment of the present invention. In step 4.10, an audiovisual signal is captured. As described above, the audiovisual signal is captured by scanning a line having a certain position within the frame constituting one image in the image/frame sequence. In the current embodiment, there is no distinction between signals that are line-scanned and non-interlaced. In step 4.20, a DC value is calculated for the current line of the audiovisual signal, and in step 4.20, a signature bit is formed according to the DC value of the current line. The calculated signature bits are either directly embedded in the current line itself in step 4.50, and then continue to calculate for the next line, until the signature is calculated and embedded in all lines; or in step 4. In 40, the currently calculated signature bits are stored in the memory for later embedding together with the signature bits of the subsequent row(s) or the signature bits of the current row, or even the signature bits calculated in the previous rows. In subsequent rows, they are both embedded in the current row and saved in the memory for subsequent use. In this way, a signature calculation scheme for inexpensive real-time operations is shown, which requires only one line of memory instead of storing the entire field as described in the first embodiment. The calculation of the signature bits representing a given image area only needs to be based on the image area itself and other nearby image areas, which means one or several lines of the audiovisual signal processed by the present invention. In addition, the signature is based on several image characteristics, such as DC values, edges, moments, or histograms, which only require the calculation of the characteristics instead of pixels and storage in memory. Therefore, the storage requirement for the calculated signature is generally much lower than that of a field memory. As in the above-mentioned embodiment, several line memories are sufficient. Under certain circumstances, depending on the characteristics used, even less memory is required. For example, calculating the DC value is performed by averaging-that is, summing the values of the pixels of the audiovisual signal. In this case, it is not necessary to store the pixel value itself, so the storage requirements are further minimized compared with the previous embodiment. Similarly, for the calculation of the watermark, once the payload is known, that is, the complete signature is available, the formation of the watermark can generally be completed with a few lines of memory. This is because, in order to obtain robustness and The best balance between concealment, watermarking to correct the image, involves considering the complexity of the image in the local area around the currently calculated watermarked pixel, that is, the amount of image activity such as edges and textures And other features. This only requires several line memories to store nearby image pixels, and uses the same line memories as when forming the signature above, without additional memory circuits.

Fig. 5 shows an embodiment of the present invention in a system 100 for verifying audiovisual signals. The audiovisual signal is generated in 110. The audiovisual signal is preferably captured in 110 by an image capturing device camera such as a surveillance camera or a CCD array, and/or a suitable device for capturing audio signals such as a microphone. However, the audiovisual signal may also originate from a transmission signal, such as a video signal, or from a storage device, such as a hard disk drive or similar computer-readable medium. The audiovisual signal is further processed in the device 101 according to the embodiment of the invention. The audiovisual signal captured in 110 is sent to the device 101. The N-line segments of the audiovisual signal are stored and maintained in the memory 120. The memory 120 is composed of N line memories and includes an additional memory for storing signature bits. The number of lines N is much lower than the number of lines of the entire audiovisual signal. An example is that three line memories in the device 120 are used for 480 horizontal lines of one audiovisual signal captured in 110. According to the discussion in the previous paragraph, the additional memory required for the signature bits is much lower than that required for each row. The device 130 communicates with the memory circuit and calculates a signature for each row in the memory 120. The signature is formed according to each row in the memory 120. When the signature is formed, it is embedded in each line of content still held in the memory 120. The generated signature is saved in the memory 120 for later use, such as embedded in a subsequent segment of the audiovisual signal. The calculated signature is preferably embedded in the device 140 as a watermark, preferably a robust watermark. Robust watermark is a watermark embedded in audiovisual signals, which is not affected by permitted image operations such as lossy compression. Subsequently, the line of the audiovisual signal with the embedded signature is output outside the device 101 for further processing at 150. Subsequently, the next N lines of the same frame of the audiovisual signal are loaded into the memory 120, a signature is formed for the content of the new line, and the signature-preferably together with the signature bits previously calculated and stored in the memory-is embedded Place in each row. The combined signature is also calculated by the device 120. Repeat the above process until the signature is calculated and embedded for all rows of a frame. Then delete the stored content in 120, and process the new frame generated in 110 in 101.

The device 101 is preferably implemented in the form of a module in the system 100, and preferably includes a microprocessor or similar electronic device, such as a programmable array or similar electronic circuit.

FIG. 6 shows another embodiment of the present invention. The embodiment includes a computer-readable medium 220 in the system 200 for verifying an audiovisual signal, thereby generating an audiovisual signal at 230. The audiovisual signal is preferably captured in 230 by an image capturing device camera such as a monitoring camera or a CCD array camera and/or a suitable device for capturing audio signals such as a microphone. However, the audiovisual signal may also originate from a transmission signal, such as a video signal, or from a storage device, such as a hard disk drive or similar computer-readable medium. The first program module 240 instructs the computer 210 to form a signature for a segment of N lines of a frame of the audiovisual signal. In the second program module 250, the signature generated by the first program module, preferably as a watermark, more preferably a robust watermark, is embedded in the segment of the frame of the audiovisual signal . The steps performed by the program modules 240 and 250 are repeated for the segments of each row of the frame until the signature is calculated and embedded for the entire frame. The audiovisual signal with the embedded signature is then further processed at 270, such as verifying the audiovisual signal.

In some applications of the present invention, for example, in security image recording, only one frame among a plurality of frames, such as one frame in every 50 frames, is stored. Therefore, it is important that each frame can verify itself without referring to previous and subsequent frames. According to the invention, the signature is embedded in the frame itself. Therefore, the above method meets this requirement because it treats each video frame as a separate still image. This also means that the method is equally applicable to still images and videos.

For security reasons, signature calculation and embedding are placed as close as possible to the image capture device. This can prevent the video signal from being tampered with before the signature is calculated. Therefore, the signature calculation and subsequent embedding, which is preferably carried out in a watermarking manner, is preferably carried out in real time in a video stream generated in an image capture device such as a camera. According to the present invention, only a part of a complete frame of the video stream is stored in the memory. Therefore, the method and device according to the present invention are well suited for real-time embedding of signatures. Those skilled in the art will therefore clearly use a signature generation suitable for real-time applications. However, the present invention is not limited to a specific type of signature calculation.

In order to judge the authenticity of an image, a process similar to signature formation is used, that is, a signature is calculated again based on a first part of a frame of the audiovisual signal. In order to verify the content of the part, the original signature embedded in a part of the frame is extracted and compared with the newly calculated signature for this part. Thus, the part that has been embedded with the original signature is compared with the original signature for which it was calculated. The part does not have to be the same part. For example, the signatures of rows 1, 3, and 5 of a frame may be embedded in rows 13, 15, and 17. When the two signatures are different from each other, tampering can be detected. If tampering is detected, if, for example, it is desired to determine where the frame tampering has occurred in the content, an analysis of the modification is performed based on the information derived from the embedded signature.

The applications and uses of the above-mentioned signal verification according to the present invention are various, including exemplary fields such as: security cameras or surveillance cameras for law enforcement, imaging that can be used as evidence, or fingerprints, such as remote Medical systems, medical scanners and healthcare systems for patient document records, such as insurance document record applications for auto insurance, property insurance, and health insurance.

The present invention has been described above with reference to specific embodiments. However, within the scope of the appended claims, other embodiments than the above-mentioned preferred embodiments are also possible, for example, different field modes from the above, implementation of the above methods by hardware or software, and combinations of features in the embodiments-such as A segment is formed in the field of the interlaced audiovisual signal content, or several lines of memory are used to embed a signature in the interlaced content, and so on.

In addition, the term "comprising" does not exclude other elements or steps, and the term "a" does not exclude that multiple and one processor or other unit can perform the functions of several units or circuits recited in the claims.