Printing a video summary

S & F Ref: 481110

AUSTRALIA

PATENTS ACT 1990

COMPLETE SPECIFICATION

FOR A STANDARD PATENT

ORIGINAL

<td> Name and Address of Applicant:</td><td> Canon Kabushlki Kaisha 30-2, Shimomaruko 3-chome Ohta-ku Tokyo 146 JAPAN</td>

<td> Actual Inventor(s):</td><td> Zhenya Yourlo</td>

<td> Address for Service:</td><td> Spruson &amp; Ferguson, Patent Attorneys Level 33 St Martins Tower, 31 Market Street Sydney, New South Wales, 2000, Australia</td>

<td> Invention Title:</td><td> Printing a Video Summary</td>

ASSOCIATED PROVISIONAL APPLICATION DETAILS

<td> [31J Application No(s) PP7436</td><td> [33] Country [32] Application Date All 30 November 1998</td>

The following statement is a full description of this invention, including the best method of performing it known to me/us:5815

- 1 PRINTING A VIDEO SUMMARY

FIELD OF THE INVENTION

The present invention relates to extraction of key frames from a video sequence, 5 and the processing and printing of these images for viewing on a lenticular film.

BACKGROUND OF THE INVENTION

Currently, distributors of digital video discs (DVD's) or other video storage media typically provide a preview of the contents of the medium by using a sequence of 10 key frames from the stored video material. A set of extracted frames is typically printed either horizontally or vertically on the cover, or as an insert inside the package, or possibly included in some other printed manner with the storage medium, to illustrate the contents of the video clip, or sequence of video clips stored on the storage medium. A potential user or customer of the stored information can thus quickly assess the contents 15 of the storage medium.

The aforementioned technique has a limitation in that the number of frames which may be printed on a DVD cover, or alternatively on an insert, is severely limited. Printing full frames as illustrated in Fig. 1 quickly consumes the available space on a DVD cover or insert, and the resulting limited number of frames which can be displayed 20 restricts the user's ability to obtain a representative understanding of the contents of the storage medium.

Another limitation of the aforementioned technique, is that it does not support the presentation of motion information. This further limits the user's ability to assess video content in the storage medium, as the motion has to be imagined based upon the 25 still images presented on the, say, DVD cover.

SUMMARY OF THE INVENTION

According to a first aspect of the invention, there is provided a method for printing an image summary, the method comprising the steps of:

(CFP1524AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDIA\MMEDIA19\481110.doc (a) selecting a plurality of images;

(b) periodically removing a predetermined number of substantially equal rows from each image, said row removal starting from a selected row position for each of said images;

(c) juxtapositioning the row-reduced images with interleaving of the rows so as to form a composite image; and (d) printing the composite image on the planar surface of a lenticular film, wherein the rows of said composite image are printed substantially parallel to the lenticles, and the width of each row is substantially a sub-multiple of a lenticle width.

¹⁰ According to a second aspect of the invention, there is provided a printed lenticular film manufactured in accordance with a method for printing an image summary, • ·

:.:: the method comprising the steps of:

...: (a) selecting a plurality of images;

• · · · • · · (b) periodically removing a predetermined number of substantially equal • · · *’* ’ 15 rows from each image, said row removal starting from a selected row position for each of .’’..J said images;

(θ) juxtapositioning the row-reduced images with interleaving of the rows • · · · • · · i .* so as to form a composite image; and • · ' (d) printing the composite image on the planar surface of a lenticular film, • · ·

.....: ²⁰ wherein the rows of said composite image are printed substantially parallel to the lenticles, and the width of each row is substantially a sub-multiple of a lenticle width.

According to a third aspect of the invention, there is provided a video storage medium having attached thereto, or to its packaging, a printed lenticular film manufactured in accordance with a method for printing an image summary, the method 25 comprising the steps of:

(a) selecting a plurality of images;

(b) periodically removing a predetermined number of substantially equal rows from each image, said row removal &lt;5 'Y^said images;

starting from a selected row position for each of

<img file="AU747359B2_D0001.tif" />

481110.doc (c) juxtapositioning the row-reduced images with interleaving of the rows so as to form a composite image; and (d) printing the composite image on the planar surface of a lenticular film, wherein the rows of said composite image are printed substantially parallel to the lenticles, and the width of each row is substantially a sub-multiple of a lenticle width.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the invention is described with reference to the drawings, in which:

Fig. 1 illustrates the prior art method of presenting image frames;

Fig. 2 is a block diagram of apparatus in accordance with a preferred embodiment of the invention;

Fig. 3 depicts the manner in which multiple image frames are superimposed;

Fig. 4 illustrates the operation of a lenticular film;

Fig. 5 depicts a mechanism used for feeding the lenticular film through a printer mechanism; and

<img file="AU747359B2_D0002.tif" />

481110.doc

- 4Fig. 6 depicts the embodiment being practiced using a conventional generalpurpose computer.

DETAILED DESCRIPTION

Fig. 1 illustrates a prior art method of presenting individual frames 502, 504, ... 506 in a linear contiguous fashion on strip 500 for printing, say, on a DVD cover. Clearly this strip can be either horizontal as shown, or vertical, or individual frames can be displayed in separate locations not necessarily adjacent to each other.

Fig. 2 depicts a preferred embodiment of equipment used to make lenticular film as used in the preferred embodiment of the display system. The video material in question is represented in a form accessible to and processible by a substantially conventional computer processing system 104 (described in more detail in relation to Fig. 6) and is made available to application software residing on that computer processing system 104. The video material is shown in Fig. 2 as residing on a storage medium 100, which is shown external to computer system 104 for ease of discussion, but which could, as is already known, be housed internal to the computer processing system 104.

The computer processing system 104 scans through the video material (which clearly can also contain associated audio material) by means of input line 106. The processing system 104 selects a set of image frames 158 which in some way are representative of a particular video segment. This frame selection, which is performed according to a set of rules, selects the frames 158 which are, for example, highlights of the video segment, or otherwise representative of the material in question. The frames 158 selected by the computer processing system 104 can be presented to the user of the system by means of a display 152, whereupon the user can approve or edit the material by means of a keyboard 154, or a track ball or mouse 156, thereby deleting unwanted frames, selecting addition frames for inclusion, and/or reorganisation in general of the selected frames.

The set of frames selected is then grouped into subsets 160 as schematically illustrated in Fig. 2, each subset containing, say, five or six frames. These images are then (CFP1524AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDIA\MMEDIA19\481110.doc

-5stored on the storage medium 100 by means of an output line 102. Each of the subsets 160 preferably includes the same number of frames. Corresponding frames of each set will be coincident upon each other in their final location upon the printed surface, as described in more detail in respect to Fig. 3 below.

After organising the frames 158 as described, and arranging them into subsets 160, each subset 160 is communicated by means of a line 110 to a printer 112. Fig. 2 illustrates how lengths of lenticular film 114 are fed in a direction 118 on a feed surface 116, to exit the printer 112 as shown as an output lenticular film 114 which passes through a slot 120 onto a feed-out surface 124. Thereafter, the lengths of lenticular film 114 follow a process direction 126 and are positioned on a surface 130 of a drying table 132 where the lenticular film 114 is dried by fans 134. Thereafter, the lenticular film 114 follows a process direction 136 whereupon white adhesive backing 140 is applied to a rear planar face 322 (see Fig. 4 below) of the film. Thereafter, the white adhesive backing 140 is trimmed to size on the lenticular film 114, which itself is then cut into strips 162 which are mounted on a DVD package 164.

Fig. 3 depicts how the image subsets 160 (shown in Fig. 3 to have 4 constituent image frames 200-206 for ease of representation), have their constituent images interleaved and superimposed so that they are coincident on a single composite image 208 having a nominal height 210 and a nominal width 212. Row 200a of the composite image 208 represents the very first row of the image 200, which has rows 200b to 200d removed, so that row 200e of composite image 208 is the next row of video information relating to the image 200 to be printed. Similarly considering image 202, rows 202b to 202d are removed, and the first row 202a of image 202 is printed in the composite image 208, whereafter the next three rows of image 202 having been removed, the next row from image 202 being printed in the composite image 208 is depicted as row 202e in Fig. 3.

Turning now to Fig. 4, there is shown a transverse cross-sectional view of a lenticular sheet 324 in the embodiment. It is seen that the sheet 324 has a sequence of lenses 300 (which are, say, semicircular), formed on its upper surface. A rear surface 322 of the sheet 324 is substantially planar. In Fig. 4, the row 200a in Fig. 3 is printed directly (CFP1S24AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDIA\MMEDIA19\481110.doc

- 6onto the planar surface 322 of the lenticular film 324 in a position 302, whereas the next row 200e associated with the image 200 in Fig. 3 is printed in position 304 on the rear planar surface 322 of the lenticular film 324. These two successive rows of information for the image 200 which are located at positions 302 and 304 respectively, are, by means of the lenses on the upper surface of lenticular film 324, focused at a point 314 defined by rays 310 and 312 respectively. Thus an observer whose eye is positioned at the location 314 will have corresponding rows 200a, 200e, ... (having reference to Fig. 3) in focus, while the rows associated with the other images will be out of focus. Similarly, the rows associated with the image 206, located in positions 306 and 308, will be focused at a location 320 defined by means of rays 316 and 318 respectively.

It is noted that Fig. 4 depicts the 4 rows associated with the four images 200-206 as being located in positions 302 to 308, thus completely filling the distance denoted as the pitch of the lenticular film between the markers 326 and 328 in Fig. 4. In fact, the distance consumed by the four aforementioned rows can be any submultiple of the distance between markers 326 and 328. Preferably, the four rows can substantially consume the whole distance between the aforementioned markers 326 and 328. Alternatively, the four rows can be a fraction of the distance between the markers 326 and 328. A minimum row width is typically a function of the printing resolution of the printer 112 (see Fig. 2). From another perspective, if P is the number of image frames to be printed in the image subset 160, and if L is defined to be the pitch of the lenticular film between the markers 326 and 328 in Fig. 4, and if M is a positive integer, then the printer prints at print resolution K given by:

K = ™

L

Fig. 5 illustrates an embodiment of a mechanism by which the film 114 is fed through the printer 112 with the precise orientation required to support printing onto the film 114 with required precision, accuracy and repeatability. The film 114 is positioned on the transparent feed surface 116 and is fed by split roller pairs 406 and 408 into, and (CFP1524AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDIA\MMEDIA19\481110.dOC

- 7 through, the printer mechanism 112. A coherent light source 418 with a beam 420 having a diameter substantially larger than the individual lenticle spacings (between markers 326 and 328, for example, in Fig. 3), is directed such that the beam 420 is normal to the surface of the film 114. Due to the lenticular action of the film or sheet 114, the beam 420 after transmission through the film 114 and the transparent feed surface 116, assumes an elongated shape 424. This shape 424 is substantially perpendicular to the individual lenticles 402 when projected onto a surface parallel to the surface 116. Sensors 426, 428, 430 and 432 are positioned as shown in Fig. 5 abutting the beam projection 424 below the surface 116. The sensors 426-432 are light sensitive devices that are matched to the spectral characteristics of the light source 418.

The film 114 is shown in Fig. 5 travelling through roller pairs 406 and 408 with the correct orientation, so that the difference signals for sensors 426 and 428 as detected by an amplifier 434 and output on a line 444, will balance the difference signal from sensors 430 and 432 as detected by an amplifier 436 and output on a line 446. In this case, a pair of driver amplifiers 410 and 412 continue to ensure that the roller pairs 406 and 408 respectively continue to feed at their current rates, thus maintaining the orientation of the film 114. If however, film 114 should undesirably rotate, say, slightly in a direction 438, the beam projection 424 will similarly rotate in a corresponding direction 440 about an axis 442, thus increasing the signal at the sensors 426 and 432, and decreasing the signals at the sensors 428 and 430. This results in a positive difference signal being detected by the amplifier 434, which when output on the line 444 results in the driver amplifier 410 reducing the drive speed of the roller pair 406. Similarly, the amplifier 436 detects a positive difference signal which when output on the line 446 results in an increased drive signal from the amplifier 412 increasing the feed rate of the roller pair 408. Reducing the feed rate of the roller pair 406 while increasing the feed rate of the roller pair 408 results in the lenticular film 114 being rotated in a direction opposite to that of arrow 438, thus reversing the undesirable rotational movement of film 114 in direction 438, and maintaining the required feed orientation.

(CFP1524AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDIA\MMEDIA19\481110.doc

-8The system implemented using a conventional general-purpose computer 600 is illustrated in Fig. 6. The various processes described are implemented as software executing on the computer 600. In particular, the various process steps are effected by instructions in the software that are carried out by the computer 600. The software can be stored in a computer readable medium, is loaded onto the computer 600 from the medium, and then executed by the computer 600. The use of the computer program product in the computer preferably extracts key frames from a video sequence, and processes and prints them for viewing on a lenticular film. The computer system 600 includes a computer module 602, a video input card 616, and input devices 618,620. In addition, the computer system 600 can have any of a number of other output devices including a video output card 610 and an output display 624. The computer system 600 is connected to one or more other computers using an appropriate communication channel such as a modem communications path, a computer network, or the like. The computer network can include a local area network (LAN), a wide area network (WAN), an Intranet, and/or Internet. Thus, for example, video sequences are input via the video input card 616, control commands are input via the keyboard 618, and desired frame subsets, for printing are output via the video output card 610. The computer 602 itself includes at least one central processing unit 604 (simply referred to as a processor hereinafter), a memory 606 which can include random access memory (RAM) and read-only memory (ROM), an input/output (IO) interface 608, a video input interface 622, and one or more storage devices generally represented by a block 612. The storage device(s) 612 can include one or more of the following: a floppy disk, a hard disk drive, a magneto-optical disk drive, CD-ROM, magnetic tape or any other of a number of non-volatile storage devices well known to those skilled in the art. Each of the components 604, 606, 608, 612 and 622, is typically connected to one or more of the other devices via a bus 604 that in turn can include data, address, and control buses. The video interface 622 is connected to the video input 616 and video output 610 cards, and provides video input from the video input card 616 to the computer 602 and from the computer 602 to the video output card 610.

(CFP1524AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDlA\MMEDIA19\481110.doc

-9The foregoing embodiment has been described with reference to image frames being representative of a particular video segment. The invention is however not limited thereto and the images from different sources or a collection of independent images, can be used without departing from the scope and spirit of the invention. For example, 5 selected images from an image library can be used as the image frames as hereinbefore described.

(CFP1524AU MMEDIA19) (481110AU) l:\ELEC\CISRA\MMEDIA\MMEDIA19\481110.doc