Continuous Color Grading Technique
Claim Score by NHIP
Abstract
The continuity of at least one image characteristic, such as color, is maintained through image creation, processing and distribution by correlating that characteristic to a frame of reference. During a first processing operation such as image capture, an initial image decision list is generated indicative of at least a one change in at least one characteristic of an image adjusted during that operation. At the completion of that operation, the image is transformed to emulate a film print that serves as the reference for subsequent operations. In this way, the initial image decision list can be modified to further adjust the at least one characteristic of the image using the emulated film print during a second image processing operation.

Term
Projected expiry 28 December 2028.
- Priority
- Filed
- Published
- Today
- Projected expiry
18 claims: 2 independent, 16 dependent
- 1Broadest claimClaim Score 72, broad(NHIP)A method comprising the steps of:(a) generating an initial image decision list indicative of at least a one change in at least one characteristic of an image adjusted during a first image processing operation;(b) transforming the image to emulate a film print;and (c) modifying the initial image decision list to further adjust of the at least one characteristic of the image in accordance with the emulated film print during a second image processing operation.
- 14Apparatus comprising:an image decision list data entry device for entering an initial image decision list indicative of at least a one change in at least one characteristic of an image adjusted during a first image processing operation;and first processing means for transforming the image to emulate a film print;and second processing means for modifying the initial image decision list to further adjust of the at least one characteristic of the image in accordance with the emulated film print during a second image processing operation.
Independent claims2
61 paragraphs in 6 sections, as filed
CROSS-REFERENCE TO RELATED APPLICATIONS
0001This application claims priority under 35 <b>119</b>(e) to U.S. Provisional Patent Application Ser. No. 60/773,807, filed Feb. 15, 2006, the teachings of which are incorporated herein.
TECHNICAL FIELD
0002This invention relates to a technique for maintaining the consistency of at least one image parameter, such as color, during different phases of producing an audio-visual program, such as a movies or television show.
BACKGROUND ART
0003Providers of audio-visual programs, such as movies, and television shows generally follow a well known workflow to create, process, and distribute such audio-visual programs to viewers. The process of creating content generally entails the capture of visual images and audio (although in some instances, the audio gets added and/or modified after image capture). Traditionally, motion picture film cameras have been used to capture visual images. While film cameras often remain the device of choice, digital video cameras, such as the Thomson Grass Valley Viper film stream camera have increased in popularity. The use of digital video cameras afford the advantage of eliminating the need to digitize the images captured by the film camera(s) for subsequent digital processing, e.g., adding visual effects.
0004In the course of content creation, the director and/or director of photography typically want to review all or at least a portion of the captured images, usually on a daily basis. The terms “digital dailies” or “rushes” generally refer to the captured images viewed on such a daily basis. In the case of film, the captured images usually undergo a transfer to a telecine device such as the Thomson Grass Valley Spirit telecine device. The telecine device scans the film to create a digital output file(s) or a digital video signal for recording and for display on a television set or digital projector. In some instances, such digital output file(s) first undergo processing prior to display. In the course of scanning the film, a colorist can apply color correction, giving rise to a color decision list (CDL) that typically exists as metadata associated with the digital output file(s) of the telecine. Such metadata will indicate the color correction decisions made by the colorist. By its very nature, a digital video camera, such as the Thomson Grass Valley Viper digital video camera, produces digital output file(s) representative of the captured images. Thus, no need exists for any transfer to a telecine device. Like the digital output file(s) from the telecine, the digital output file(s) from the digital video camera(s) usually undergo color correction in the course of creating digital dailies (i.e., digital files representative of captured images).
0005In the past, the CDL generated in the course of creating digital dailies has played little if any role in later processing operations such as Digital Intermediate generation. Presently, each successive processing operation in the workflow associated with the creation, processing and production of an audio-visual program produces different CDLs that bear little if any relationship with the CDL created during any of the preceding processing operations. One reason for the lack of a relationship between the CDLs associated with successive processing operations has been the inability to provide a consistent frame of reference (base exposure and color and final intended display device emulation) for such CDL for each processing operation.
0006Thus a need exists for an image processing technique that enables image processing information, such as, but not limited to, the CDL associated with a given processing operation to be carried forward to a subsequent processing operation.
BRIEF SUMMARY OF THE INVENTION
0007Briefly, in accordance with an illustrative embodiment of the present principles, there is provided a method for processing image information that enables at least one image decision list associated with a given image processing operation to be carried through to at least one subsequent processing operation. The method of the present principles commences by generating initial image decision list indicative of a change in at least one image characteristic adjusted during a first image processing operation. For example, the initial image decision list could comprise a Color Decision List (CDL) indicative of a color adjustment made during an image processing operation. This operation can include displaying the image while emulating the final intended display medium. During a second image processing operation, the initial image decision list undergoes modification to further adjust the at least one image characteristic in accordance with the emulating the final intended display medium used in the first step. In the case of film, a rescan of the film must yield the identical relationship of density (and color) to code value of the original scan by using a standardized calibration procedure.
0008The method of the present principles advantageously enables an image decision list, such as a CDL, made during a particular image processing operation to be carried forward during a subsequent processing operation. The transformation of the image following the initial processing operation while emulating the final intended display medium, and same emulation (and standardized calibration procedure if a rescan is required) as the basis for making subsequent image adjustments affords a mechanism for normalizing the image decision list made during each processing operation. In this way, the images decision list will track image adjustments during subsequent image processing operations.
BRIEF DESCRIPTION OF THE DRAWINGS
0009<figref idref="DRAWINGS">FIG. 1</figref> depicts a flow chart illustrating the elements of a workflow for producing an audio-visual program in accordance with an illustrative embodiment of the present principles;
0010<figref idref="DRAWINGS">FIG. 2</figref> depicts a block schematic diagram of the elements associated with a digital cinematography workflow in accordance with an illustrative embodiment of the new principles;
0011<figref idref="DRAWINGS">FIG. 3</figref> depicts a block schematic diagram of the elements associated with a digital cinematography workflow for producing digital dailies from images captured by a digital video camera in accordance with an illustrative embodiment of the new principles;
0012<figref idref="DRAWINGS">FIG. 4</figref> depicts a block schematic diagram of the elements associated with a workflow for producing dailies from images captured by a film camera in accordance with an illustrative embodiment of the new principles;
0013<figref idref="DRAWINGS">FIG. 5</figref> depicts a block schematic diagram of the elements associated with a workflow for producing digital dailies and for achieving quality control in accordance with an illustrative embodiment of the new principles;
0014<figref idref="DRAWINGS">FIG. 6</figref> depicts a block schematic diagram of the elements associated with a workflow for on-set dailies viewing in accordance with an illustrative embodiment of the new principles;
0015<figref idref="DRAWINGS">FIG. 7</figref> depicts a block schematic diagram of the elements associated with a workflow for post-preview conformation and timing in accordance with an illustrative embodiment of the new principles;
0016<figref idref="DRAWINGS">FIG. 8</figref> depicts a block schematic diagram of the elements associated with a workflow for post visual effects scanning in accordance with an illustrative embodiment of the new principles; and
0017<figref idref="DRAWINGS">FIG. 9</figref> depicts a block schematic diagram of the elements associated with a workflow for post-Digital Intermediate generation and finishing in accordance with an illustrative embodiment of the new principles.
DETAILED DESCRIPTION
0018<figref idref="DRAWINGS">FIG. 1</figref> depicts the elements in a workflow <b>10</b> associated with the production of an audio-visual work in accordance with a preferred embodiment of the present principles. Illustratively in <figref idref="DRAWINGS">FIG. 1</figref>, the audio-visual program takes the form of a pre-recorded digital versatile disc (DVD), some times referred to as a digital video disk, containing at least one feature presentation such as a movie or television show. Alternatively, or in addition to the pre-recorded DVD, the audio-visual program could take the form of a movie in either digital or analog form for theatrical presentation.
0019The workflow <b>10</b> commences with content creation (e.g., image capture) by making use of one or more digital image capture devices (e.g., Thomson Grass Valley Viper film stream cameras) during digital production in step <b>12</b>. Image capture also can occur during by making use of one or more film cameras during film production in step <b>14</b>. (Indeed both digital production and film production could occur in some circumstances) The capture of audio information can occur simultaneously during either of steps <b>12</b> or <b>14</b>. Alternatively, or in addition to audio capture during image capture, audio capture and/or processing can occur during subsequent processing operations.
0020During digital production in step <b>12</b>, preliminary adjustments to the captured images, can occur such as, but not limited to, color adjustment, by passing the digital video output signal/file(s) of the digital image capture device(s) through a color corrector (not shown) in a manner describer in greater detail hereinafter. In the case of color adjustment, the transfer of the digital output signal/file(s) through the color corrector will yield a color decision list (CDL), taking the form of metadata reflective of the color adjustments that were made.
0021In the case of film production during step <b>14</b>, the film typically undergoes transfer to a telecine device (not shown), such as the Thomson Grass Valley Spirit film scanner which has been calibrated to a reference film, allowing repeatability in successive scans, to yield a digital output signal/file(s) as described in greater detail hereinafter. Just as the digital output signal/file(s) generated during digital production in step <b>12</b> can undergo color correction, the digital output signal/file(s) generated by the telecine device during step <b>14</b> can also undergo transfer through a color corrector, thereby giving rise to a CDL indicative of the color adjustments that were made.
0022Following steps <b>12</b> or <b>14</b>, the individuals responsible for content creation, such as the Director and/or the Director of Photography, often want to view all or a portion of the captured images, usually on a daily basis, during step <b>16</b>. The portions of the captured images that undergo daily viewing bear the designation “dailies” or “rushes”. In the case of images captured by the digital image capture devices during digital production in step <b>12</b>, the viewing of dailies during step <b>16</b> can occur by simply transferring the digital output file(s) following image (e.g., color) correction, if any to a digital display device, e.g., a television monitor or digital projector. In the case of film production, the viewing of dailies during step <b>16</b> occurs by sending the digital output signal/file(s) from the film scanner following image (e.g., color) correction if any, to the display device.
0023During dailies viewing in step <b>16</b>, the Director and/or Director of Photography might wish to make further make image adjustments, such as color correction, in addition to those previously made during steps <b>12</b> and <b>14</b>. Typically, color color adjustments can be made by the use of a color corrector (not shown). Previously, the CDL initially obtained during steps <b>12</b> and <b>14</b> had was not passed to step <b>16</b>, and for that matter, any of the succeeding processing operations. The lack of transference of the CDLs to successive image processing operations stemmed from the lack of a common frame of reference. For example, the digital video output signal/file(s)s generated at the completion of steps <b>12</b> and <b>14</b>, when displayed on one type digital output device will not necessarily have the same appearance (i.e., “look”) as the color-corrected dailies viewed during step <b>16</b> even if the initial CDLs undergoes an adjustment to correspond to the CDL associated with dailies viewing during step <b>16</b>.
0024As described in greater detail below, the technique of the present principles overcomes this disadvantage by providing a common frame of reference for the CDLs created during successive processing operations. The digital output signal/file(s)s generated during each of steps <b>12</b> and <b>14</b> undergoes a transformation via a Look-Up table (LUT) described hereinafter, to yield digital output signal/file(s) whose images emulate a “normal” film print. Thus, the CDL created during each of steps <b>12</b> and <b>14</b> becomes correlated to the emulated film print.
0025During step <b>16</b>, the color-corrected digital output signal/file(s) also undergo transformation to emulate a film print. Further, the device(s) utilized for display (e.g., the digital television set or digital projector) comprise calibrated devices. Thus, the digital video output file whose images emulate a film print will possess the same appearance regardless of the display device. Therefore, the CDL associated with the dailies viewing during step <b>16</b> could be obtained by a prescribed modification to initial CDL obtained during steps <b>12</b> or <b>14</b>. In other words, the CDL obtained during steps <b>12</b> or <b>14</b> can serve as the starting point for subsequent adjustments.
0026During step <b>16</b>, the digital video output signal/file(s) also undergo a transformation to emulate film print. The digital output signal/file(s)s generated following such transformation serves as the input signal/file(s) to the offline editorial operation performed during step <b>18</b>. During step <b>18</b>, the received input signal/file(s) typically undergo subsequent processing including, but not limited to, image editing, the addition of audio, and/or compression. As with the previous image processing operations, the digital video output signal/file(s) at the end of step <b>18</b> undergo a transformation to emulate a film print prior to receipt at step <b>20</b> during which preview screenings occur.
0027The preview screening operation of step <b>20</b> also receives as an input the visual effects created during step <b>22</b>. In practice, digital video output signal/file(s) generated during steps <b>12</b> and <b>14</b> undergo transfer to a special effects facility, along with the initial CDLs and a three-dimensional Look Up-Table. This LUT enables the transformation of the digital output signal/file(s) completed in the course of special effects creation in step <b>22</b> to emulate a film print. During step <b>20</b>, the special effects produced at the completion of step <b>22</b> and the digital output signal/file(s) produced following the offline editorial processing during step.<b>18</b> undergo conformation. Such conformation corresponds to the traditional process of segmentation and rearrangement of original camera negatives (OCNs) to yield a motion picture film. During preview screening in step <b>20</b>, one or more colorists will make final color adjustments using the CDLs generated during the previous processing operations steps as the starting points for final color grading. The use of LUTs to transform the digital output signal/file(s) at each processing operation to emulate a film print makes this possible. Following final color grading, a master CDL gets saved, and new files with corrections burned in undergo rendering and transformation via a LUT to yield an output file whose images emulate a film print for screening at the completion of step <b>20</b>.
0028Along with the special effects generated during step <b>22</b>, the transformed digital output signal/file(s) generated during step <b>20</b> serve as the input to the process of creating digital intermediates (e.g. digital audio-visual programs) during step <b>24</b>. During step <b>24</b>, the received digital signal/file(s) undergo further conformation and a master CDL created during step <b>20</b> gets loaded into a Digital Intermediate color correction system (not shown) which can take the same form as the color correction system used for conformation during step <b>20</b>. To the extent that the captured images were originated during film production in step <b>14</b>, the original negative typically will undergo calibrated scanning at a higher resolution, 2K or 4K. This higher resolution digital video output file gets conformed and loaded into the Digital Intermediate color correction system for final color grading and subsequent rendering during which a final color grade gets applied. As before, the master CDL created during such color grading gets saved. The digital video output signal/file(s) produced during step <b>24</b> undergoes transformation via an LUT to emulate a film print that can then undergo distribution to one or more digital cinemas by various means, such as via Internet, Satellite, or physical delivery on a storage medium such as a disk.
0029The digital video output file generated at the completion of step <b>24</b> also serves as the final Digital Intermediate (digital master) for creating home video digital versatile (DVD) disks during step <b>26</b>. Further color grading can occur during step <b>26</b> for consumer-type displays. As before, the digital output generated after final grading undergoes a transformation to emulate a film print. Dow conversion to HD or SD occurs as well with the final color grade burned in.
0030<figref idref="DRAWINGS">FIG. 2</figref> depicts a block schematic diagram showing elements of a workflow for digital production during step <b>12</b> of <figref idref="DRAWINGS">FIG. 1</figref>. As shown in <figref idref="DRAWINGS">FIG. 2</figref>, at least one digital image capture device <b>200</b> serves to capture images in digital form (e.g., a stream of digital video). In the illustrative embodiment, the digital image capture device comprises one or more digital television camera, such as the “Viper” Film Stream digital camera from Thomson Grass Valley. A storage device <b>202</b> stores the digital video output signal/file(s) generated by the image capture device <b>200</b>, along with time code knowledge, synchronizing signal/file(s)s and metadata. The storage device <b>202</b> typically comprises one or more magnetic disk drives arranged in an array. For example, the storage device <b>202</b> can comprise the “Venom” disk storage pack from Thomson Grass Valley. In practice, the time code knowledge, synchronizing signal/file(s)s and metadata pass to the storage device <b>202</b> out of band from the digital video signal/file(s) generated by the digital image capture device <b>200</b>.
0031A production system <b>206</b> receives the time code knowledge, synchronizing signal/file(s)s and metadata from the digital image capture device <b>200</b> for embedding in the audio recorded by a digital sound recorder <b>208</b>. The production system <b>206</b> also collects
0032Color Decision List (CDL) and/or Automatic Link Enablement data (ALE) generated hereinafter as discussed below for subsequent export for use by other systems (not shown). In practice, the digital sound recorder <b>208</b> records audio associated the visual information captured by the digital image capture device <b>200</b>. Audio from the digital sound recorder <b>208</b> undergoes storage in the storage device <b>202</b>, along with the digital output signal/file(s) representing captured images, along with, the time code knowledge, synchronizing signal/file(s)s and metadata.
0033In addition to embedding the time code knowledge, synchronizing signal/file(s)s and other metadata, the production system <b>206</b> can also embed external information entered by an operator through a keyboard <b>210</b> such as Color Decision List (CDL) and/or Avid Log Exchange (ALE) data. Typically, the keyboard <b>210</b> connects to the production system <b>206</b> through a Universal Serial Bus (USB) connection. Other well known protocols could serve to facilitate coupling the keyboard <b>210</b> to the production system <b>206</b>.
0034A processing system <b>212</b> processes digital output signal/file(s) from the storage device <b>202</b> using the accompanying time code knowledge, synchronizing signal/file(s)s and metadata. Additionally, the processing system <b>212</b> can make use of CDL data received from a CDL entry device <b>214</b> to enable color adjustments. In practice, an operator, usually referred to as a “colorist” will make adjustments through the CDL entry device <b>214</b> to the color of the captured images. Such adjustments allow the Director and/or Director of Photography to see how corresponding color adjustments might impact the final version the audio-visual program produced from the captured video. Adjustments to other image parameters can be made in a similar manner, using an input device (not shown), similar to the CDL entry device <b>214</b>.
0035The processing system <b>212</b> typically comprises one or more programmed computers that include or have access to one or more Look-Up Tables (LUTs) that can transform the capture image information embodied in the digital output signal/file(s) originated by the digital image capture device(s) to emulate a film print as discussed previously. The LUTs can undergo dynamic change, especially in response to the color adjustments entered via the CDL entry device <b>214</b>. In this way, the processing system <b>212</b> can transform captured images embodied in the digital output signal/file(s) to emulate a film print as well as to change the overall appearance of captures images, such as by modifying their color and/or other image characteristics.
0036The digital video output signal/file(s) transformed by the processing system <b>212</b> typically undergo display on a display device <b>216</b>, such as a video monitor. To assure optimal picture quality, an automatic calibration system <b>218</b> will periodically calibrate the display device <b>216</b> with regard to various settings, such as color, linearity, focus and contrast for example.
0037<figref idref="DRAWINGS">FIG. 3</figref> a block schematic diagram showing the elements of a workflow for production of instant dailies in connection with dailies viewing during step <b>16</b> of <figref idref="DRAWINGS">FIG. 1</figref>. As seen in <figref idref="DRAWINGS">FIG. 3</figref>, at least one storage facility <b>300</b>, typically in the form of a Storage Area Network (SAN) receives digital video output signal/file(s) <b>301</b> from the system <b>212</b> of <figref idref="DRAWINGS">FIG. 2</figref> following their transformation to emulate a film print. Preferably, the SAN <b>300</b> makes use of Serial Advanced Technology Attachment (SATA) for connecting to other devices. An archive system <b>302</b> connected to the SAN <b>300</b> can create one or more deliverables comprising all or a portion of the digital output signal/file(s) stored in the SAN. In practice, such deliverables take the form of tapes or other transportable storage mediums, such as transportable hard disk drives, that store information for export to an external entity, such as a visual special effects (VFX) facility.
0038The SAN <b>300</b> of <figref idref="DRAWINGS">FIG. 3</figref> can also receive digital audio from an audio transfer station <b>304</b> supplied with audio from one or more sources <b>306</b>, only one of which is shown. In practice, the audio source(s) <b>306</b> could include digital sound recorders or other devices that provide audio in digital form. The audio transfer station <b>304</b> enables editing of the digital audio prior to storage in the SAN <b>300</b>. The audio transfer station <b>304</b> can include any of several different types of systems for performing post-production operations, including but not limited to audio correction and/or adjustment. In the illustrated embodiment, the audio transfer station <b>304</b> comprises part of the Thomson Grass Valley “Bones” post production framework, which includes one or more processors programmed with software that allows editing, correction and/or manipulation of digital audio.
0039The digital output signal/file(s) stored in the SAN <b>300</b>, which embody one or more audio-visual programs in digital form, undergo processing via a processing system <b>308</b>. Like the processing system <b>212</b> of <figref idref="DRAWINGS">FIG. 1</figref>, the processing system <b>308</b> of <figref idref="DRAWINGS">FIG. 3</figref> includes one or more LUTs for transforming digital output signal/file(s) so that the images embodied in those. files emulate the look of a motion picture film print. The LUT(s) also enable changing the appearance of one or more images characteristics, such as color, the latter occurring in accordance with Color Decision List (CDL) and/or Automatic Link Enablement (ALE) information <b>317</b> exported from the production system <b>206</b> of <figref idref="DRAWINGS">FIG. 2</figref>. In practice, the processing system <b>308</b> comprises the Thomson Grass Valley “Bones” post production framework. A single post production framework, such as the “Bones” post production framework, could perform the functions of the processing system <b>308</b> and the audio transfer station <b>304</b> of <figref idref="DRAWINGS">FIG. 3</figref>.
0040The processing system <b>308</b> generates digital output signal/file(s) that embody one or more audio-visual program(s) in digital form, along with associated time code knowledge, synchronizing information and metadata for distribution to several destinations. The audio-visual program(s) embodied in the digital output signal/file(s) generated by the processing system <b>308</b> typically have undergone a transformation to emulate a film-print. The processing system <b>308</b> can also provide the digital output signal/file(s) embodying the audio-visual program(s) to a recorder <b>310</b>. Also, the processing system <b>308</b> can output the digital output more files in any one of a variety of formats <b>312</b>, such as JPEG 2000, Avid® Media, Apple® FCP, DVD images and the like. Further, the processing system <b>308</b> can make use of one or more internal look-up tables to convert LOG-based video embodied in the digital output signal/file(s) to linear-based video and to apply film print emulation to yield video for display on a display system <b>314</b>, such as a video monitor or a scope. An auto calibration system <b>316</b>, similar to auto calibration system <b>218</b>, serves to calibrate the display system <b>314</b>.
0041<figref idref="DRAWINGS">FIG. 4</figref> depicts a block schematic diagram showing the elements of a workflow for producing digital dailies from filmed images created during film production in step <b>14</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The workflow of <figref idref="DRAWINGS">FIG. 4</figref> contains many elements in common with the workflow of <figref idref="DRAWINGS">FIG. 3</figref> and therefore, like reference numbers reference like elements in <figref idref="DRAWINGS">FIG. 3</figref>. To transform the film images created during film production in step <b>14</b> of <figref idref="DRAWINGS">FIG. 1</figref> into digital output signal/file(s), the workflow of <figref idref="DRAWINGS">FIG. 4</figref> makes use telecine <b>400</b>, such as the Thomson Grass Valley Spirit film scanner, for scanning Original Camera Negative (OCN) rolls <b>401</b>. The Thomson Grass Valley “Spirit” film scanner offers both 2K and 4K resolution, and thus can generate two separate files for a scanned film, one having 4K resolution and the other with 2K resolution. In practice, the scanner <b>400</b> operates at 30 frames per second (fps), which is typically faster than the 24 fps, the speed at which images are typically captured onto the OCN rolls <b>401</b>, thereby allowing the scanner to operate at greater than real time speed, as measured with respect to the film exposure rate.
0042The scanner <b>400</b> has a ten-bit High Definition Serial Data Interface (HD SDI) for generating digital output signal/file(s) in a 4:4:4 format for receipt at a receiving (e.g., ingest) station <b>402</b>, typically part of the Thomson Grass Valley “Bones” post processing framework. In practice, the ingest station <b>402</b> also receives a key code from the scanner <b>400</b> identifying the particular OCN roll <b>401</b> undergoing scanning. The receiving station <b>402</b> collects the digital output signal/file(s) from the scanner <b>400</b> for transfer to the SAN <b>300</b> to allow further processing by the processing system <b>308</b> as described previously with respect to <figref idref="DRAWINGS">FIG. 3</figref>. In addition to receiving digital output signal/file(s) originated by the scanner <b>400</b> and stored by the SAN <b>300</b>, the processing unit <b>308</b> also receives audio files stored by the SAN <b>300</b> following their receipt from the audio transfer station <b>304</b> which processed such files received from one of the audio sources <b>306</b>. Also, as described previously, the archive system <b>302</b> connected to the SAN <b>300</b> can create one or more deliverables comprising all or a portion of the digital output signal/file(s) stored in the SAN.
0043The processing system <b>308</b> generates more or more digital output signal/file(s) embodying one or more audio-visual programs for display on a picture monitor or scope <b>314</b>. As described previously with respect to <figref idref="DRAWINGS">FIG. 3</figref>, the processing system <b>308</b> can make use of one or more internal look-up tables to convert LOG-based video embodied in the digital output signal/file(s) to linear-based video and to apply film print emulation in connection with the digital output signal/file(s) supplied to the display device <b>314</b>.
0044<figref idref="DRAWINGS">FIG. 5</figref> depicts a block schematic diagram showing the elements of a workflow for dailies distribution in connection with the dailies viewing operation in step <b>16</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The dailies distribution workflow of <figref idref="DRAWINGS">FIG. 5</figref> utilizes many of the same elements as the workflows depicted in <figref idref="DRAWINGS">FIGS. 3 and 4</figref> and like reference numbers reference like elements. As discussed above with respect to <figref idref="DRAWINGS">FIGS. 3 and 4</figref>, the SAN <b>300</b> supplies the processing system <b>308</b> with digital output signal/file(s) processing, (the files including both video and audio, typically combined in one or more audio-visual programs). In the course of such processing by the processing system <b>308</b>, an operator can undertake quality control and verification as required. Also, as described previously, the SAN <b>300</b> provides digital output signal/file(s) to the archive system. <b>302</b> which can create one or more deliverables comprising all or a portion of the digital output signal/file(s) stored in the SAN.
0045The digital output signal/file(s) produced by the processing system <b>308</b> embody one or more audio-visual-programs in one or more formats <b>312</b>. Such files constitute digital dailies for display on the display device <b>314</b>. As discussed previously, the processing system <b>308</b> can make use of one or more internal look-up tables to convert LOG-based video embodied in the digital output signal/file(s) to linear-based video and to apply film print emulation in connection with the digital output signal/file(s) supplied to the display device <b>314</b>.
0046The audio-visual program(s) embodied in the files output by the processing system <b>308</b> typically undergo transmission via a network <b>500</b>. The transmission network <b>500</b> can comprise one or more public and/or private networks, which can include wired and/or wireless networks of various types. Selection of the network(s) will depend on bandwidth and time availability. Prior to transmission on the network <b>500</b>, the digital output signal/file(s) can undergo watermarking, typically via a watermarking station (not shown) that performs “Nextamp Watermarking” using software such as the “Portable Media Package (PMP).
0047<figref idref="DRAWINGS">FIG. 6</figref> depicts a block schematic diagram showing elements of a workflow for on-set dailies viewing during step <b>16</b> of <figref idref="DRAWINGS">FIG. 1</figref>. As illustrated in <figref idref="DRAWINGS">FIG. 6</figref>, a server <b>600</b> receives one or more of the digital output signal/file(s) containing one or more audio-visual programs embodying the digital dailies for viewing, such as those produced by the processing system, <b>308</b> of <figref idref="DRAWINGS">FIGS. 3-5</figref>. In practice, the digital output signal/file(s) received by the server <b>600</b> take the form of raw uncorrected source files encoded in the JPEG 2000 format. In order to commence play out, the serve <b>600</b> must receive a security code, typically entered via a Universal Serial Bus (USB) flash memory device <b>602</b>. Other data entry devices, such as a keyboard (not shown) could facilitate entry of the security code.
0048Upon receipt of the proper security code via the flash memory <b>602</b>, the server <b>600</b> will supply one or more digital output signal/file(s) whose video exists in a LOG format for display on a display device <b>604</b> to effect the display of digital dailies. To emulate a film print for display on the display device <b>604</b>, the server <b>600</b> can perform a LOG to LIN conversion of the video embodied in the digital output signal/file(s) via a LUT, prior to receipt of the signal/file(s) by the display device. To assure color consistency, a calibration system <b>606</b> can serve to calibrate the display device <b>604</b>.
0049The server <b>600</b> can makes use of CDL data <b>607</b> supplied from a server (not shown) at a post facility during real time viewing of digital dailies on the display device <b>604</b>. During such viewing, the Director and/or Director of Photography can make changes to the CDL. To that end, a CDL entry device <b>608</b> connects to the server <b>600</b> to enable adjustments to the CDL. Other image adjustments can be made in a similar manner through the use of corresponding image adjustment entry devices (not shown). Following changes made to the CDL via the CDL entry <b>608</b>, the server <b>600</b> will transmit those changes to the CDL server <b>607</b> at the post facility.
0050<figref idref="DRAWINGS">FIG. 7</figref> depicts a block schematic diagram showing the elements of a workflow for post-preview conformation and timing in connection with step <b>20</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The conformation performed during the workflow of <figref idref="DRAWINGS">FIG. 7</figref> corresponds to the traditional segmentation and rearrangement of original camera negatives (OCNs) to yield a motion picture film. The timing performed during the workflow of <figref idref="DRAWINGS">FIG. 7</figref>, corresponds to the traditional process of adjusting (e.g., grading) the color balance of each scene. Referring to <figref idref="DRAWINGS">FIG. 7</figref>, a first server <b>702</b> receives one or more digital output signal/file(s) from the archive system <b>302</b>, typically in a raw 4:4:4 log form. In practice, the server <b>702</b> comprises part of the Stone FS high speed fibre channel solution available from Autodesk Inc, San Raphael, Calif.
0051A conformation system <b>704</b> performs editing and finishing of the files supplied from the file server <b>702</b> in accordance with Edit Decision List (EDL) data, and time code information from one or more external sources. In practice, the conformation system <b>704</b> typically takes the form of one or more processors running image finishing software such as Autodesk FIRE® and/or Autodesk SMOKE® software. Such software (or its equivalent) enables the conformation system <b>704</b> to accomplish preview conformance (e.g., editing and assembling). Following editing and finishing, the conformation system returns <b>704</b> the signal/file(s) to the file server <b>702</b> for transfer to a second file server <b>706</b> and an archive system. <b>707</b>. Like the server <b>702</b>, the server <b>704</b> comprises part of the Stone FS high speed fibre channel solution available from Autodesk Inc, San Raphael, Calif.
0052A color grading system <b>708</b> performs color grading on the digital output signal/file(s) (i.e., audio-visual works) stored in the filer server <b>706</b> using CDL data associated with the digital dailies as a pre-grade. In practice, the color grading system <b>708</b> comprises one or more processors running color grading software such as Autodesk LUSTRE® software. The digital output signal/file(s) produced by the color grading system <b>708</b> undergo storage on a preview server <b>710</b>.
0053<figref idref="DRAWINGS">FIG. 8</figref> depicts a block schematic diagram showing the elements of a workflow for post-visual effects scanning in connection with visual effects production during step <b>22</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The post-visual effects workflow makes use many of the elements described previously with respect to <figref idref="DRAWINGS">FIG. 4</figref> and like reference numbers describe like elements. As described, the scanner <b>400</b> scans OCNs to yield both 2K and 4K resolution digital output signal/file(s) for storage in the SAN <b>300</b>. In addition to supplying the 2K and 4K resolution output signal/file(s), the scanner <b>400</b> also supplies a scan list identifying the film scanned to an asset management system <b>800</b>, typically comprising a database that resides on a server (not shown) for tracking various activities (typically referred to projects) in the course of post-visual effects scanning. Each project corresponds to a particular film scan, and to that end, the management system <b>800</b> will provide a link between such film scans and associated file names and key code relationships, as well as proxy information including links to other servers which can store information associated with that project. Additionally, for each project, the asset management system <b>800</b> can also store information regarding the identity of the entity responsible for providing visual special effects (VFX). Further, the asset management system <b>800</b> also has the capability of acquiring Edit Decision List(s) and/or ALE data from one or more external sources (not shown.)
0054The digital output signal/file(s) stored in the SAN <b>300</b> can undergo color grading via a color grading system <b>708</b> described with respect to <figref idref="DRAWINGS">FIG. 7</figref>. The color-graded digital output signal/file(s) then undergo transformation via a LUT <b>802</b> to emulate a film print. To that end, the LUT <b>802</b> typically will convert the video associated with the digital output signal/file(s) from LOG to LIN and apply print emulation for HD reference if needed. A remote preview system <b>804</b> serves to convert the digital output signal/file(s) transformed by the LUT <b>802</b>, which typically exist in a JPEG <b>200</b> format, to an IP format for network streaming, such as over the Internet.
0055<figref idref="DRAWINGS">FIG. 9</figref> depicts a block schematic diagram showing the elements of a workflow for post-Digital Intermediate creation and finishing to yield finished one or more audio-visual program(s) for home distribution and the like in connection with step <b>26</b> of <figref idref="DRAWINGS">FIG. 1</figref>. The workflow of <figref idref="DRAWINGS">FIG. 9</figref> shares many elements in common with the workflows of <figref idref="DRAWINGS">FIGS. 2-8</figref> and like reference numbers describe like elements. The workflow of <figref idref="DRAWINGS">FIG. 9</figref> makes use of a film scanner, such as scanner <b>400</b>, to scan OCNs to yield both 2K and 4K resolution digital output signal/file(s). The 2K files from the scanner typically undergo storage in a file server <b>900</b>, which like the file servers <b>702</b> and <b>706</b> of <figref idref="DRAWINGS">FIG. 7</figref> comprises part of the Stone FS high speed fibre channel solution available from Autodesk Inc, San Raphael, Calif. The 4K resolution output signal/file(s) from the scanner <b>400</b> undergo storage in a Fiber Channel Storage Area Network (SAN) <b>902</b> for subsequent transfer to the SAN <b>300</b> which has links to the file servers <b>702</b> and <b>706</b> as discussed previously with respect to <figref idref="DRAWINGS">FIG. 7</figref>.
0056As discussed previously, the scanner <b>400</b> generates a scan list which is received by an Asset Management System, such as Asset Management system <b>800</b> described previously with respect to <figref idref="DRAWINGS">FIG. 8</figref>. The scan list can include conformation data for receipt by the Asset management system and by a conformation station, such as conformation system <b>704</b> of <figref idref="DRAWINGS">FIG. 7</figref>. The conformation system <b>704</b> makes use of the conformation data to perform conformation (e.g., editing and finishing) of digital output signal/file(s) transferred from the SAN <b>300</b> to the file server <b>702</b>. The digital output signal/file(s) produced by the conformation system <b>704</b>, which embody one or more audio-visual programs, undergo storage in the file <b>702</b> for subsequent transfer back to the SAN <b>300</b>.
0057The now-conformed digital Output signal/file(s) undergo transfer from the SAN <b>300</b> to a file server, such as file server <b>706</b>, for subsequent transfer to a color grading system, such as color grading system <b>708</b> described previously with respect to <figref idref="DRAWINGS">FIG. 7</figref> for final color grading. After final color grading, the digital output signal/file(s), which now embody finished Digital Intermediate(s) (audio-visual program(s)) typically undergo conversion via a LUT <b>902</b> to transform the video embodied in the files into a HD 10-bit LOG DPX format suitable for using in mastering DVDs for consumer use. The transformed digital output signal/file(s) can include metadata <b>903</b> for dissolves. The now-conformed files also undergo a transfer to a remote pre-visualization server <b>904</b> to enable streaming to remote sites.
0058The SAN <b>300</b> of <figref idref="DRAWINGS">FIG. 9</figref> has a link to a quality control station <b>905</b> which typically comprises part of the Thomson Grass Valley “Bones” post production framework. Using the quality control station <b>905</b>, an operator can access stored digital output signal/file(s) embodying one or more audio-visual programs, from the SAN <b>300</b> for inspection and modification to improve the quality of at least one of the digital audio and video embedded within such files. To save time, the operator could begin a quality control review prior to completion of color grading.
0059The SAN <b>300</b> can deliver a digital output files, embodying one or more finished audio-visual program in digital form, for distribution to a Digital Cinema Distribution Master (DCDM) facility <b>906</b>. In turn, the DCDM <b>906</b> facility serves to distribute the digital output signal/file(s) embodying such audio-visual programs in digital form to one or more digital cinemas such as digital cinema <b>908</b>. The digital output signal/file(s) from the SAN <b>300</b> can also undergo transfer to a film printer <b>920</b>, such as manufacture by Arri Camera, which can print a corresponding celluloid film negative. The SAN <b>300</b> enjoys a link to an archive system <b>707</b> similar to the archive system <b>202</b>, for creating one or more deliverables for export to an external entity.
0060What has been described thus far with respect to <figref idref="DRAWINGS">FIG. 9</figref> has been the workflow associated with 4K resolution digital output files to the film scanned by the scanner <b>400</b>. As discussed above, the scanner <b>400</b> also yield 2K resolution digital output signal/file(s) stored in the server <b>900</b>. Such files can undergo processing in the same manner as described for the 4K digital output files.
0061The foregoing describes systems and method for processing at least one of audio-visual information in connection with digital cinematography.
Contents6
11 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US8077967B2 | Cited by | United States of America | Search report |
| US9832419B2 | Cited by | United States of America | Search report |
| US2016182815A1 | Cited by | United States of America | Pre-grant |
| US11212562B1 | Cited by | United States of America | Applicant |
| US12425535B2 | Cited by | United States of America | Applicant |
| US10713765B2 | Cited by | United States of America | Applicant |
| CN115706767A | Cited by | China | Search report |
| US10645356B1 | Cited by | United States of America | Search report |
| US2010329646A1 | Cited by | United States of America | Pre-grant |
| US2005134801A1 | Cites | United States of America | Pre-grant |
| US2005179775A1 | Cites | United States of America | Pre-grant |
| US2005280842A1 | Cites | United States of America | Pre-grant |
| US2007268411A1 | Cites | United States of America | Pre-grant |
| US3824336A | Cites | United States of America | Pre-grant |
| US4367284A | Cites | United States of America | Pre-grant |
| US4714962A | Cites | United States of America | Pre-grant |
| US4945406A | Cites | United States of America | Pre-grant |
| US5512396A | Cites | United States of America | Pre-grant |
| US5574659A | Cites | United States of America | Pre-grant |
| US5752122A | Cites | United States of America | Pre-grant |
| US5917914A | Cites | United States of America | Pre-grant |
| US6215901B1 | Cites | United States of America | Pre-grant |
| US6433824B1 | Cites | United States of America | Pre-grant |
| US6871003B1 | Cites | United States of America | Pre-grant |
| US6972828B2 | Cites | United States of America | Pre-grant |
| US7295281B2 | Cites | United States of America | Pre-grant |
| US7856414B2 | Cites | United States of America | Pre-grant |
| US7876955B2 | Cites | United States of America | Pre-grant |
7 members in 5 offices
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 77380706 | United States of America | P | |
| 2006048595 | United States of America | W |
Members7
| Document | Office | Kind | |
|---|---|---|---|
| WO2007097803A1 | World Intellectual Property Organization (WIPO) | A1 | |
| EP1984922A1 | European Patent Office (EPO) | A1 | |
| CN101375338A | China | A | |
| JP2009527189A | Japan | A | |
| US2010158366A1 | United States of America | A1 | |
| US8077967B2 | United States of America | B2 | |
| JP5094739B2 | Japan | B2 |
51 transactions on the USPTO file
Allowed after 1 non-final rejection and 1 final rejection.
- Non-final rejections
- 1
- Final rejections
- 1
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Expire PatentEXP. | EXP. | |
| Maintenance Fee Reminder MailedREM. | REM. | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Email NotificationEML_NTR | EML_NTR | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Reasons for AllowanceEX.R | EX.R | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Final ActionA.NE | A.NE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Final Rejection (PTOL - 326)Final rejectionMCTFR | MCTFR | |
| Final RejectionFinal rejectionCTFR | CTFR | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Non-Final ActionA... | A... | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Reference capture on IDSRCAP | RCAP | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| PG-Pub Issue NotificationPG-ISSUE | PG-ISSUE | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Reference capture on IDSRCAP | RCAP | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Filing ReceiptFLRCPT.O | FLRCPT.O | |
| Notice of DO/EO Acceptance MailedM903 | M903 | |
| Sent to Classification ContractorPGPC | PGPC | |
| Cleared by OIPE CSRL194 | L194 | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| New or Additional Drawing FiledC614 | C614 | |
| Request for Foreign Priority (Priority Papers May Be Included)RQPR | RQPR | |
| Reference capture on IDSRCAP | RCAP | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Preliminary AmendmentA.PE | A.PE | |
| 371 Completion Date371COMP | 371COMP | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Initial Exam Team nnIEXX | IEXX |
8 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Lapsed due to failure to pay maintenance feeLapsedFP | FP | |
| Lapse for failure to pay maintenance feesLapsedPATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYLAPS | LAPS | |
| Information on status: patent discontinuationPATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362STCH | STCH | |
| Fee payment procedureMAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Fee paymentFPAY | FPAY | |
| Information on status: patent grantGrantedPATENTED CASESTCF | STCF | |
| AssignmentAS | AS | |
| AssignmentAS | AS |
Numbers
- Publication
- 20100158366
- Application
- 12223886
Titles
- English
- Continuous Color Grading Technique
Patent term adjustment
- A delay
- +616 daysthe office missed an examination deadline
- B delay
- +123 dayspendency past three years
- Net adjustment
- 739 days
Classification
- CPC, 3
- G11B27/034
- G11B2220/2562
- H04N23/84
- IPC, 1
- G06T1 00