Digital watermarking for workflow by tracking content or content identifiers with respect to time

RELATED APPLICATION DATA

0001This application is a continuation-in-part of U.S. patent application Ser. No. 10/278,742 (published as U.S. 2003-0138127 A1), filed Oct. 21, 2002. This patent application also claims the benefit of U.S. Provisional Patent Application No. 60/554,748, filed Mar. 19, 2004. This application is also related to U.S. patent application Ser. No. 09/858,336 (published as U.S. 2002-0124024 A1), filed May 15, 2001. Each of these patent documents is herein incorporated by reference.

TECHNICAL FIELD

0002The present invention relates generally to digital watermarking. In a specific example, the present invention embeds digital watermarks in media content to provide a chain of custody.

BACKGROUND AND SUMMARY

0003Movie studios have a problem in that they have not established a successful mechanism to provide a chain of custody for their video or other media. In other words, their video is not linked to a current person, usually an employee, whom is in possession of the video. This problem is described even more fully in a paper by Simon Byers, et al. titled “Analysis of Security Vulnerabilities in the Movie Production and Distribution Process,” Sep. 13, 2003, (http://lorrie.cranor.org/pubs/drm03-tr.pdf), incorporated herein by reference.

0004Digital watermarks can be used to link video to a specific person. I disclose herein several processes to make this linking efficient in terms of a conventional video workflow.

0005Digital watermarking is a process for modifying physical or electronic media to embed a hidden machine-readable code into the media. The media may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media signals such as images, audio signals, and video signals. However, it may also be applied to other types of media objects, including documents (e.g., through line, word or character shifting), software, multi-dimensional graphics models, and surface textures of objects.

0006Digital watermarking systems typically have two primary components: an encoder that embeds the watermark in a host media signal, and a decoder that detects and reads the embedded watermark from a signal suspected of containing a watermark (a suspect signal). The encoder embeds a watermark by subtly altering the host media signal. The reading component analyzes a suspect signal to detect whether a watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.

0007Several particular watermarking techniques have been developed. The reader is presumed to be familiar with the literature in this field. Particular techniques for embedding and detecting imperceptible watermarks in media signals are detailed in the assignee's U.S. Pat. Nos. 6,122,403 and 6,614,914, which are hereby incorporated by reference.

0008One aspect of the invention provides workflow efficiency by matching a time based log of personal computer (PC) users with a watermark embedder ID (e.g., a number or plural-bit identifier) and date/time stamp. The embedder ID is uniquely associated with the PC, so that a user can be matched to a computer via the embedder ID and a log of user activity on the PC. This system works automatically, without an operator needing to keep track of IDs and video that is processed on the system.

0009Another aspect of the invention is a system responsive to a rights command. The rights command requires a video player to embed a forensic ID in content, where a watermark embedder—and not the rights command—controls which data is embedded. More specifically, a rights language, such as MPEG-21 REL (ISO/EEC 21000-5 draft, incorporated herein by reference) is used to mandate watermark embedding, but is not used to specify a particular identifier or data to be embedded.

0010Still another aspect is an encrypted video file with certain video segments duplicated therein. The duplicated segments can be encrypted according to different keys. The segments are used to generate different unique watermarks when the content is rendered by applying device specific decryption keys. Consider a typical movie trail. A movie is passed from one executive to another, perhaps even to editors and directors, each giving their approval prior to release of the movie. The video is played on a rendering device (e.g., video player). To establish an audit trail the player preferably generates a unique ID that is associated with the player or executive in the video, based upon selective decryption of the duplicated video segments as dictated by device-specific keys. An embedded identifier is then dependent upon which player is rendering the video (e.g., which movie studio executive is currently viewing the video as the video is passed along to each other after approval of a master version for release). The embedding happens during viewing (a.k.a. rendering) in the player.

0011(Related information for embedding is found, e.g., in EP 1134977 entitled “Method and system for providing copies of scrambled content with unique watermarks, and system for descrambling scrambled content” (incorporated herein by reference) and J. Benaloh, et al's, “Efficient Fingerprinting to Protect Digital Content,” a copy of which is found at http://research.microsoft.com/crypto/FingerMark.ppt, which is incorporated herein by reference.)

0012Yet another aspect of the present invention is identifying an intended recipient of a DVD disk by embedding a recipient identifier in a DVD disk master file during transport of the disk to the recipient.

0013The foregoing and other features, aspects and advantages of the present invention will be more readily apparent from the following detailed description, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idref="DRAWINGS">FIG. 1</figref> is flowchart illustrating a conventional workflow during video creation.

<figref idref="DRAWINGS">FIG. 2</figref> is illustrates an architecture for automatic logging.

<figref idref="DRAWINGS">FIG. 3</figref> is a flowchart illustrating a rights command controlled embedding.

DETAILED DESCRIPTION

Workflow Overview

0017<figref idref="DRAWINGS">FIG. 1</figref> illustrates a workflow during video creation. Camera segments (e.g., images or frames) are recorded to film in step <b>101</b>. The segments are edited by using, e.g., software editing tools (e.g., Avid editing tools for Macs and Windows PCs) in step <b>102</b>. A finished version is written to a digital master file (e.g., stored on DigiBeta) via a channel (e.g., SDI) in step <b>103</b>. The master may include a final version (or semi-final like a rough cut version) to which sound and effects have been added. (Of course, other editing software, such as is provided by Apple and Adobe, and other master storage formats can be suitably interchanged with these aspects of the invention.). Steps <b>101</b>-<b>103</b> are typically repeated as additional camera segments are captured. And, of course, these steps can be substituted or augmented with digital video as well.

0018In the process of shooting film on a day-to-day basis, reviews are often performed on previous work. These reviews are provided in video segments and are referred to as “dailies.” (Dailies are rough cuts or segments that may or may not include sound tracks or effects. Dailies can be viewed by directors and crew to determine what else needs to be accomplished for a completion of a particular scene or to track a movie timeline.). Thus, these dailies are produced for review or approval in step <b>102</b><i>a</i>. The daily segments can be provided via Windows Media Video (WMV) and/or QuickTime (QT), although they may be recorded to, e.g., DigiBeta and converted later.

0019A Master can be copied to a WMV/QT file for executive review, where executives (or directors, editors, assistants, advertisers, promoters, etc., herein collectively referred to as “executives”) may pass the same file along to other executives to gain approval or exposure as shown in step <b>104</b>. The executives will play this content in a player <b>108</b>, such as Windows Media Player, QuickTime player, or other suitable player. In step <b>105</b> and <b>106</b>, after approval by required movie executives, the DigitBeta Master is converted to film for theatrical release and to a DVD disk image file (e.g., MPEG2 disk image file), respectively. (As will be appreciated, the disk image file is the file that is used to direct the burning of DVDs.). In step <b>107</b> and <b>108</b>, the disk image is distributed, e.g., using a content distribution service like WamNet, to a duplication house for DVD disk creation and release.

0020In addition, a DigiBeta Master or editor version may be converted to MPEG-2 (8-10 Mbps) and this file is converted to WMV and/or QuickTime using, e.g., Telestream Flip Factory™.

0021Instead of film as in step <b>105</b>, a final movie may be digitally stored as a master copy for transmission in digital cinema applications.

0022Automatic Logging

0023The first improvement in such a workflow involves automatic logging, and can occur, e.g., at step <b>102</b><i>a </i>and/or <b>104</b> in <figref idref="DRAWINGS">FIG. 1</figref>.

0024One preferred logging system is illustrated in <figref idref="DRAWINGS">FIG. 2</figref>. A personal computer (PC) or video editing terminal includes a PC controller <b>200</b> (e.g., software executing on the personal computer or video editing terminal). PC controller <b>200</b> control a video router <b>220</b> as video copies are made or transmitted. A watermark embedder <b>230</b> embeds an embedder ID via digital watermarking and, preferably, embeds a time stamp into video (or audio) copies <b>240</b>. The embedded ID may include, e.g., a number, text, code or plural-bit identifier that is preferably uniquely associated with the watermark embedder, PC, PC controller <b>200</b> and/or a database or data record. Ideally, the time stamp includes or indexes enough data to resolve the time, month, day and year. Many methods can be used, such as including minutes from a fixed date (e.g., how many minutes from Jan. 1, 2004). This example requires 27 bits for a 50 year range. The watermarking is preferably automatically (and unavoidably) activated as copies are generated or routed by video router. (In fact, the watermark embedder can be viewed as a continuous process, which embeds all content from a video router.) Usually, the video communications channel is a Serial Digital Interface (SDI), but other channels can be suitably interchanged herewith.

0025The PC controller <b>200</b> helps manage or track user activity. For example, an active user <b>201</b> logs on to the PC. The PC controller <b>200</b> communicates which user accesses the PC to a database or record <b>210</b>. Thus, the data record includes a record of which users accessed the PC.

0026When content <b>240</b> is found (especially if in an illegitimate or unexpected location), the watermark is read with digital watermark (DWM) detector <b>250</b>. The embedder ID and timestamp are obtained from the watermark by detector <b>250</b>. The embedder ID is used to locate a corresponding record in database or log <b>210</b>. (Alternatively, the embedder ID identifies a particular PC, which includes the data record itself.). The corresponding record indicates that the embedder ID is associated with the PC or PC controller <b>200</b>. A particular user is determined from the timestamp (e.g., a user is identified as using the PC during a time represented by the watermark date/time stamp). The log <b>210</b> may also include times representing when a user logged-on and log off of the PC, or which current users were logged on at time intervals, like every 15 minutes. Thus, the timestamp and recorded user access can be used to determine who is responsible for copying or transmitting the video in which the embedder ID was found.

0027User identity can be authenticated to avoid spoofing the PC with a username and password. For example, the PC may use biometrics, such as a fingerprint, eye scan or face recognition; thus, further increasing authentication of the user. A digital image of the user's face or eye (e.g., iris or retina scan) can be saved in the log <b>210</b> to verify the user. Secure IDs can be used, such that the PC is secured by what you know (e.g., secure ID), what you are (e.g., employee level <b>10</b>, which may or may not be authorized to copy video), and what you have (e.g., perhaps a user is required to present a feature for biometric sampling, or even present a watermarked object for verification as discussed, e.g., in assignee's U.S. patent application Ser. No. 09/790,322, published as U.S. 2001-0037313 A1).

0028In an alternative implementation, the controller PC <b>200</b> itself includes one or more computers and includes or cooperates with one or more digital watermark embedders.

0029As an additional alternative, PC controller <b>200</b> does not “control” the video router. Instead, the PC controller <b>200</b> monitors and records user access on the PC. A watermark embedder automatically embeds the Embedder ID in any content routed by the router as discussed above.

0030As mentioned above, an embedder ID is preferably uniquely associated with a PC, PC controller or particular data record. Thus, the embedder ID can always be traced to the PC or PC controller via the data repository <b>210</b>. In a multiple computer—but only one watermark embedder—scenario, unique association between a PC controller and embedder ID is preserved by connecting the watermark embedder <b>230</b> to the PC controllers and changing the embedder ID based upon which PC controller is active. (Of course, this interconnection is avoided by providing a separate watermark embedder for each PC controller, and ensuring the watermark embedder is embedding a video stream controlled by the PC controller.).

0031Automatic Embedding Based Upon Rights Command

0032A second improvement to the <figref idref="DRAWINGS">FIG. 1</figref> work flow provides automatic watermark embedding during video rendering, as shown in <figref idref="DRAWINGS">FIG. 3</figref>. The term “render” includes its familiar meaning of playing, providing, displaying or performing, etc. Specifically, a rights command <b>300</b>, which states that watermark embedding is required, is used to force a video player <b>108</b> to embed data with embedder <b>320</b>.

0033A novel twist is that embedder <b>320</b>, and not rights command <b>300</b>, controls what is embedded in content. The rights command <b>300</b> may be part of a rights language, such as MPEG-21 REL, mentioned above. The embedder <b>320</b> may be a software plug-in that must be installed on a computer to play the video, such as a DirectX Media Object (DMO), as used with Windows Media Player. The embedder embeds a unique forensic ID related to the user, user's computer, video player or user's license <b>310</b>. The embedding occurs during (or just prior to) rendering of the video. User license <b>310</b> is optional (as shown by the dotted box) as the other forensic ID options could be used.

0034The forensic ID is used for tracking purposes.

0035Thus, a database is preferably maintained to link unique forensic IDs to users or players. (The association can be through predetermined identifiers or through physical addresses or computer serial numbers, etc.). This database is preferably protected for privacy.

0036With reference to <figref idref="DRAWINGS">FIGS. 1 and 3</figref>, content is optimally encrypted. Player <b>108</b> then must search for rights commands <b>300</b> for decryption permissions (or instructions) and preferably enables embedder <b>320</b> prior to decrypting the content for rendering. A trusted player <b>108</b> could work with non-encrypted content, as it is “trusted” (or programmed) to look for rights commands <b>300</b> and enable embedder <b>320</b>. In either case, the player <b>108</b> preferably does not play content if the embedder <b>320</b> is not available or working.

0037If illegitimate video is found, the forensic ID is read from the watermark with a watermark detector. If the detector cannot read the watermark in the illegitimate video sample, the original can be used to separate the DWM from the video. (For example, the illegitimate sample and/or the original is converted to similar formats, aspect ratio, size, etc., and then the difference between the sample and original which will typically yield at least some of the watermark signal.).

0038Automatic Embedding Using Encrypted and DWM Segments

0039A third improvement to the <figref idref="DRAWINGS">FIG. 1</figref> workflow is related to the second improvement, above, but uses a different player side approach.

0040Protection is provided in player <b>108</b>. An encrypted file with certain segments duplicated with different watermark payloads is used. When the encrypted file is rendered, the rendered content contains a digital watermark with a forensic ID in its payload, which may include the active user ID, the player ID, or user's license ID. The digital watermark payload is created from decrypting different watermarked segments. The player is usually located, e.g., on a PC of a movie studio executive. The executive passes video along to another executive after she approves the version.

0041In other words, this approach identifies the rendering device in a peer-to-peer (P2P) environment, where the P2P environment consists of movie executive's PC and email or ftp.

0042A forensic ID, e.g. player or user license ID, cooperates with a database that records a corresponding executive's ID or rendering device ID. This database is preferably protected for privacy.

0043DWM in DVD Disk Image During Transit

0044A fourth improvement embeds one or more digital watermarks in a DVD disk image during transport of the disk image to a duplication house in block <b>107</b> (<figref idref="DRAWINGS">FIG. 1</figref>). At least one watermark includes a payload that identifies the duplication house, e.g. a duplication ID. This system includes an embedder sitting on a server (or router) that is in the path of video distribution. The server may embed the video before distributing it based upon the request of the duplication house <b>107</b>. The embedder may alternatively reside at the duplication house <b>107</b> receiving server and embed video with a duplication ID upon receipt, or reside in a predictable location between the studio and distribution house <b>107</b>.

0045If content is found in an unauthorized or unexpected location, such as on a public P2P network, a watermark detector can read the forensic ID, e.g. duplication ID, and link it to the duplication house <b>107</b>. (The duplication ID can be used to query a data record to identify a corresponding duplication house <b>107</b>. Protecting this data record for privacy is not a large issue since consumers are not involved, and the watermark detector is usually held by the content owner in secrecy.)

0046Some combinations of the above disclosure are provided below. Of course, these are not the only combinations that are possible for the above disclosure, and are provided by way of example only. Other combinations are provided in the claims and still further combinations can be achieved based on the above disclosure.

0047A. An apparatus comprising:

0048a media player; and

0049a digital watermark embedder,

0050wherein said media player searches content or data associated with the content for a rights command, wherein the right command requires said digital watermark embedder to embed an identifier into content rendered by said media player and, absent such embedding, the media player will not render the content, and

0051wherein the identifier is generated by said digital watermark embedder. <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0052">A1. The apparatus of A wherein the identifier is related to a user license.</li><li id="ul0002-0002" num="0053">A2. The apparatus of A wherein the identifier comprises a media player identifier.</li><li id="ul0002-0003" num="0054">A3. The apparatus of A wherein the identifier is related to a user via a restricted database.</li><li id="ul0002-0004" num="0055">A4. The apparatus of A wherein said content comprises video.</li><li id="ul0002-0005" num="0056">A5. The apparatus of A3 wherein the identifier is related to a user license.</li><li id="ul0002-0006" num="0057">A6. The apparatus of A3 wherein the identifier comprises a player identifier.</li><li id="ul0002-0007" num="0058">A7. The apparatus of A3 wherein the identifier is determined solely by the watermark embedder.</li></ul></li></ul>

0059B. A method comprising:

0060tracking which users access a computer system according to time;

0061communicating the users and related times for storage in a data record;

0062causing, without user input, unmarked video to be steganographically marked upon a copy or transmission of the video, wherein the steganographic marking embeds an identifier that is uniquely associated with the computer system or data record.

0063B1. The method of B wherein upon receipt of a timestamp, said method further comprises indexing the data record with at least a portion of the time stamp to identify a user.

0064C. A method to forensically identify content leaked from a video workflow comprising:

0065upon rendering of content by a movie executive, steganographically embedding a device specific identifier in the content, wherein the device specific identifier is determined based on decryption keys to decrypt duplicate encrypted segments in the content; and

0066rendering the content.

0000Concluding Remarks

0067Having described and illustrated the principles of the technology with reference to specific implementations, it will be recognized that the technology can be implemented in many other, different, forms.

0068To provide a comprehensive disclosure without unduly lengthening the specification, applicants incorporate by reference the U.S. patent documents referenced above.

0069The methods, processes, and systems described above may be implemented in hardware, software or a combination of hardware and software. For example, the auxiliary data encoding processes may be implemented in a programmable computer or a special purpose digital circuit. Similarly, auxiliary data decoding may be implemented in software, firmware, hardware, or combinations of software, firmware and hardware. The methods and processes described above may be implemented in programs executed from a system's memory (a computer readable medium, such as an electronic, optical or magnetic storage device).

0070While <figref idref="DRAWINGS">FIG. 1</figref> illustrates a particular conventional workflow, my inventive techniques can be applied to other chain of custody and forensic tracking workflows as well.

0071The particular combinations of elements and features in the above-detailed embodiments are exemplary only; the interchanging and substitution of these teachings with other teachings in this and the incorporated-by-reference patents/applications are also contemplated.